The Code Book: The Secret History of Codes and Code-breaking

Ruined trunks on withered forks,

Empty scarecrows, I and you!

Fill the cup, and fill the can:

Have a rouse before the mom:

Every moment dies a man,

Every moment one is born.

These are verses from a poem by Alfred Tennyson entitled ‘The Vision of Sin’. The keyword happens to be the first name of Tennyson’s wife, Emily Sellwood. I chose to use a section from this particular poem as an example for cryptanalysis because it inspired some curious correspondence between Babbage and the great poet. Being a keen statistician and compiler of mortality tables, Babbage was irritated by the lines ‘Every moment dies a man, Every moment one is born’, which are the last lines of the plaintext above. Consequently, he offered a correction to Tennyson’s ‘otherwise beautiful’ poem:

It must be manifest that if this were true, the population of the world would be at a standstill … I would suggest that in the next edition of your poem you have it read – ‘Every moment dies a man, Every moment 11/16 is born.’ … The actual figure is so long I cannot get it onto a line, but I believe the figure 11/16 will be sufficiently accurate for poetry.

I am, Sir, yours, etc.,

Charles Babbage.

Babbage’s successful cryptanalysis of the Vigenère cipher was probably achieved in 1854, soon after his spat with Thwaites, but his discovery went completely unrecognised because he never published it. The discovery came to light only in the twentieth century, when scholars examined Babbage’s extensive notes. In the meantime, his technique was independently discovered by Friedrich Wilhelm Kasiski, a retired officer in the Prussian army. Ever since 1863, when he published his cryptanalytic breakthrough in Die Geheimschriften und die Dechiffrir-k-unst (‘Secret Writing and the Art of Deciphering’), the technique has been known as the Kasiski Test, and Babbage’s contribution has been largely ignored.

And why did Babbage fail to publicise his cracking of such a vital cipher? He certainly had a habit of not finishing projects and not publishing his discoveries, which might suggest that this is just one more example of his lackadaisical attitude. However, there is an alternative explanation. His discovery occurred soon after the outbreak of the Crimean War, and one theory is that it gave the British a clear advantage over their Russian enemy. It is quite possible that British Intelligence demanded that Babbage keep his work secret, thus providing them with a nine-year head start over the rest of the world. If this was the case, then it would fit in with the long-standing tradition of hushing up codebreaking achievements in the interests of national security, a practice that has continued into the twentieth century.

From Agony Columns to Buried Treasure

Thanks to the breakthroughs by Charles Babbage and Friedrich Kasiski, the Vigenère cipher was no longer secure. Cryptographers could no longer guarantee secrecy, now that cryptanalysts had fought back to regain control in the communications war. Although cryptographers attempted to design new ciphers, nothing of great significance emerged during the latter half of the nineteenth century, and professional cryptography was in disarray. However, this same period witnessed an enormous growth of interest in ciphers among the general public.

The development of the telegraph, which had driven a commercial interest in cryptography, was also responsible for generating public interest in cryptography. The public became aware of the need to protect personal messages of a highly sensitive nature, and if necessary they would use encryption, even though this took more time to send, thus adding to the cost of the telegram. Morse operators could send plain English at speeds of up to 35 words per minute because they could memorise entire phrases and transmit them in a single burst, whereas the jumble of letters that make up a ciphertext was considerably slower to transmit, because the operator had to continually refer back to the sender’s written message to check the sequence of letters. The ciphers used by the general public would not have withstood attack by a professional cryptanalyst, but they were sufficient to guard against the casual snooper.

As people became comfortable with encipherment, they began to express their cryptographic skills in a variety of ways. For example, young lovers in Victorian England were often forbidden from publicly expressing their affection, and could not even communicate by letter in case their parents intercepted and read the contents. This resulted in lovers sending encrypted messages to each other via the personal columns of newspapers. These ‘agony columns’, as they became known, provoked the curiosity of cryptanalysts, who would scan the notes and try to decipher their titillating contents. Charles Babbage is known to have indulged in this activity, along with his friends Sir Charles Wheatstone and Baron Lyon Playfair, who together were responsible for developing the deft Playfair cipher (described in Appendix E). On one occasion, Wheatstone deciphered a note in The Times from an Oxford student, suggesting to his true love that they elope. A few days later, Wheatstone inserted his own message, encrypted in the same cipher, advising the couple against this rebellious and rash action. Shortly afterwards there appeared a third message, this time unencrypted and from the lady in question: ‘Dear Charlie, Write no more. Our cipher is discovered.’

In due course a wider variety of encrypted notes appeared in the newspapers. Cryptographers began to insert blocks of ciphertext merely to challenge their colleagues. On other occasions, encrypted notes were used to criticise public figures or organisations. The Times once unwittingly carried the following encrypted notice: ‘The Times is the Jeffreys of the press’. The newspaper was being likened to the notorious seventeenth-century Judge Jeffreys, implying that it was a ruthless, bullying publication which acted as a mouthpiece for the government.

Another example of the public’s familiarity with cryptography was the widespread use of pinprick encryption. The ancient Greek historian Aeneas the Tactician suggested conveying a secret message by pricking tiny holes under particular letters in an apparently innocuous page of text, just as there are dots under some letters in this paragraph. Those letters would spell out a secret message, easily read by the intended receiver. However, if an intermediary stared at the page, they would probably be oblivious to the barely perceptible pinpricks, and would probably be unaware of the secret message. Two thousand years later, British letter-writers used exactly the same method, not to achieve secrecy but to avoid paying excessive postage costs. Before the overhaul of the postage system in the mid-1800s, sending a letter cost about a shilling for every hundred miles, beyond the means of most people. However, newspapers could be posted free of charge, and this provided a loophole for thrifty Victorians. Instead of writing and sending letters, people began to use pinpricks to spell out a message on the front page of a newspaper. They could then send the newspaper through the post without having to pay a penny.

The public’s growing fascination with cryptographic techniques meant that codes and ciphers soon found their way into nineteenth-century literature. In Jules Verne’s Journey to the Centre of the Earth, the decipherment of a parchment filled with runic characters prompts the first step on the epic journey. The characters are part of a substitution cipher which generates a Latin script, which in turn makes sense only when the letters are reversed: ‘Descend the crater of the volcano of Sneffels when the shadow of Scartaris comes to caress it before the calends of July, audacious voyager, and you will reach the centre of the Earth.’ In 1885, Verne also used a cipher as a pivotal element in his novel Mathias Sandorff. In Britain, one of the finest writers of cryptographic fiction was Sir Arthur Conan Doyle. Not surprisingly, Sherlock Holmes was an expert in cryptography and, as he explained to Dr Watson, was ‘the author of a trifling monograph upon the subject in which I analyse one hundred and sixty separate ciphers’. The most famous of Holmes’s decipherments is told in The Adventure of the Dancing Men, which involves a cipher consisting of stickmen, each pose representing a distinct letter.

On the other side of the Atlantic, Edgar Allan Poe was also developing an interest in cryptanalysis. Writing for Philadelphia’s Alexander Weekly Messenger, he issued a challenge to readers, claiming that he could decipher any monoalphabetic substitution cipher. Hundreds of readers sent in their ciphertexts, and he successfully deciphered them all. Although this required nothing more than frequency analysis, Poe’s readers were astonished by his achievements. One adoring fan proclaimed him ‘the most profound and skilful cryptographer who ever lived’.

Figure 19 A section of the ciphertext from The Adventure of the Dancing Men, a Sherlock Holmes adventure by Sir Arthur Conan Doyle.

In 1843, keen to exploit the interest he had generated, Poe wrote a short story about ciphers, which is widely acknowledged by professional cryptographers to be the finest piece of fictional literature on the subject. The Gold Bug tells the story of William Legrand, who discovers an unusual beetle, the gold bug, and collects it using a scrap of paper lying nearby. That evening he sketches the gold bug upon the same piece of paper, and then holds his drawing up to the light of the fire to check its accuracy. However, his sketch is obliterated by an invisible ink, which has been developed by the heat of the flames. Legrand examines the characters that have emerged and becomes convinced that he has in his hands the encrypted directions for finding Captain Kidd’s treasure. The remainder of the story is a classic demonstration of frequency analysis, resulting in the decipherment of Captain Kidd’s clues and the discovery of his buried treasure.

Although The Gold Bug is pure fiction, there is a true nineteenth-century story containing many of the same elements. The case of the Beale ciphers involves Wild West escapades, a cowboy who amassed a vast fortune, a buried treasure worth $20 million and a mysterious set of encrypted papers describing its whereabouts. Much of what we know about this story, including the encrypted papers, is contained in a pamphlet published in 1885. Although only 23 pages long, the pamphlet has baffled generations of cryptanalysts and captivated hundreds of treasure hunters.

The story begins at the Washington Hotel in Lynchburg, Virginia, sixty-five years before the publication of the pamphlet. According to the pamphlet, the hotel and its owner, Robert Morriss, were held in high regard: ‘His kind disposition, strict probity, excellent management, and well ordered household, soon rendered him famous as a host, and his reputation extended even to other States. His was the house par excellence of the town, and no fashionable assemblages met at any other.’ In January 1820 a stranger by the name of Thomas J. Beale rode into Lynchburg and checked into the Washington Hotel. ‘In person, he was about six feet in height’, recalled Morriss, ‘with jet black eyes and hair of the same color, worn longer than was the style at the time. His form was symmetrical, and gave evidence of unusual strength and activity; but his distinguishing feature was a dark and swarthy complexion, as if much exposure to the sun and weather had thoroughly tanned and discolored him; this, however, did not detract from his appearance, and I thought him the handsomest man I had ever seen.’ Although Beale spent the rest of the winter with Morriss and was ‘extremely popular with every one, particularly the ladies,’ he never spoke about his background, his family or the purpose of his visit. Then, at the end of March, he left as suddenly as he had arrived.

Figure 20 The title page of The Beale Papers, the pamphlet that contains all that we know about the mystery of the Beale treasure.

The Beale Treasure – History of a Mystery by Peter Viemeister.

Two years later, in January 1822, Beale returned to the Washington Hotel, ‘darker and swarthier than ever’. Once again, he spent the rest of the winter in Lynchburg and disappeared in the spring, but not before he entrusted Morriss with a locked iron box, which he said contained ‘papers of value and importance’. Morriss placed the box in a safe, and thought nothing more about it and its contents until he received a letter from Beale, dated 9 May 1822 and sent from St Louis. After a few pleasantries and a paragraph about an intended trip to the plains ‘to hunt the buffalo and encounter the savage grizzlies’, Beale’s letter revealed the significance of the box:

It contains papers vitally affecting the fortunes of myself and many others engaged in business with me, and in the event of my death, its loss might be irreparable. You will, therefore, see the necessity of guarding it with vigilance and care to prevent so great a catastrophe. Should none of us ever return you will please preserve carefully the box for the period of ten years from the date of this letter, and if I, or no one with authority from me, during that time demands its restoration, you will open it, which can be done by removing the lock. You will find, in addition to the papers addressed to you, other papers which will be unintelligible without the aid of a key to assist you. Such a key I have left in the hand of a friend in this place, sealed and addressed to yourself, and endorsed not to be delivered until June 1832. By means of this you will understand fully all you will be required to do.

Morriss dutifully continued to guard the box, waiting for Beale to collect it, but the swarthy man of mystery never returned to Lynchburg. He disappeared without explanation, never to be seen again. Ten years later, Morriss could have followed the letter’s instructions and opened the box, but he seems to have been reluctant to break the lock. Beale’s letter had mentioned that a note would be sent to Morriss in June 1832, and this was supposed to explain how to decipher the contents of the box. However, the note never arrived, and perhaps Morriss felt that there was no point opening the box if he could not decipher what was inside it. Eventually, in 1845, Morriss’s curiosity got the better of him and he cracked open the lock. The box contained three sheets of enciphered characters, and a note written by Beale in plain English.

The intriguing note revealed the truth about Beale, the box, and the ciphers. It explained that in April 1817, almost three years before his first meeting with Morriss, Beale and 29 others had embarked on a journey across America. After travelling through the rich hunting grounds of the Western plains, they arrived in Santa Fé, and spent the winter in the ‘little Mexican town’. In March they headed north and began tracking an ‘immense herd of buffaloes’, picking off as many as possible along the way. Then, according to Beale, they struck lucky:

One day, while following them, the party encamped in a small ravine, some 250 or 300 miles north of Santa Fé, and, with their horses tethered, were preparing their evening meal, when one of the men discovered in a cleft of the rocks something that had the appearance of gold. Upon showing it to the others it was pronounced to be gold, and much excitement was the natural consequence.

The letter went on to explain that Beale and his men, with help from the local tribe, mined the site for the next eighteen months, by which time they had accumulated a large quantity of gold, as well as some silver which was found nearby. In due course they agreed that their new-found wealth should be moved to a secure place, and decided to take it back home to Virginia, where they would hide it in a secret location. In 1820, Beale travelled to Lynchburg with the gold and silver, found a suitable location, and buried it. It was on this occasion that he first lodged at the Washington Hotel and made the acquaintance of Morriss. When Beale left at the end of the winter, he rejoined his men who had continued to work the mine during his absence.

After another eighteen months Beale revisited Lynchburg with even more to add to his stash. This time there was an additional reason for his trip:

Before leaving my companions on the plains it was suggested that, in case of an accident to ourselves, the treasure so concealed would be lost to their relatives, without some provision against such a contingency. I was, therefore, instructed to select some perfectly reliable person, if such could be found, who should, in the event of this proving acceptable to the party, be confided in to carry out their wishes in regard to their respective shares.

Beale believed that Morriss was a man of integrity, which is why he trusted him with the box containing the three enciphered sheets, the so-called Beale ciphers. Each enciphered sheet contained an array of numbers (reprinted here as Figures 21, 22 and 23), and deciphering the numbers would reveal all the relevant details; the first sheet described the treasure’s location, the second outlined the contents of the treasure, and the third listed the relatives of the men who should receive a share of the treasure. When Morriss read all of this, it was some 23 years after he had last seen Thomas Beale. Working on the assumption that Beale and his men were dead, Morriss felt obliged to find the gold and share it among their relatives. However, without the promised key he was forced to decipher the ciphers from scratch, a task that troubled his mind for the next twenty years, and which ended in failure.

In 1862, at the age of eighty-four, Morriss knew that he was coming to the end of his life, and that he had to share the secret of the Beale ciphers, otherwise any hope of carrying out Beale’s wishes would die with him. Morriss confided in a friend, but unfortunately the identity of this person remains a mystery. All we know about Morriss’s friend is that it was he who wrote the pamphlet in 1885, so hereafter I will refer to him simply as the author. The author explained the reasons for his anonymity within the pamphlet:

I anticipate for these papers a large circulation, and, to avoid the multitude of letters with which I should be assailed from all sections of the Union, propounding all sorts of questions, and requiring answers which, if attended to, would absorb my entire time, and only change the character of my work, I have decided upon withdrawing my name from the publication, after assuring all interested that I have given all that I know of the matter, and that I cannot add one word to the statements herein contained.

To protect his identity, the author asked James B. Ward, a respected member of the local community and the county’s road surveyor, to act as his agent and publisher.