The Code Book: The Secret History of Codes and Code-breaking

This was an explicit reference to a masked prisoner at Pignerole, and a sufficiently serious crime, with dates that seem to fit the myth of the Man in the Iron Mask. Does this solve the mystery? Not surprisingly, those favouring more conspiratorial solutions have found flaws in Bulonde as a candidate. For example, there is the argument that if Louis XIV was actually attempting to secretly imprison his unacknowledged twin, then he would have left a series of false trails. Perhaps the encrypted letter was meant to be deciphered. Perhaps the nineteenth-century codebreaker Bazeries had fallen into a seventeenth-century trap.

The Black Chambers

Reinforcing the monoalphabetic cipher by applying it to syllables or adding homophones might have been sufficient during the 1600s, but by the 1700s cryptanalysis was becoming industrialised, with teams of government cryptanalysts working together to crack many of the most complex monoalphabetic ciphers. Each European power had its own so-called Black Chamber, a nerve centre for deciphering messages and gathering intelligence. The most celebrated, disciplined and efficient Black Chamber was the Geheime Kabinets-Kanzlei in Vienna.

It operated according to a rigorous timetable, because it was vital that its nefarious activities should not interrupt the smooth running of the postal service. Letters which were supposed to be delivered to embassies in Vienna were first routed via the Black Chamber, arriving at 7 a.m. Secretaries melted seals, and a team of stenographers worked in parallel to make copies of the letters. If necessary, a language specialist would take responsibility for duplicating unusual scripts. Within three hours the letters had been resealed in their envelopes and returned to the central post office, so that they could be delivered to their intended destination. Mail merely in transit through Austria would arrive at the Black Chamber at 10 a.m., and mail leaving Viennese embassies for destinations outside Austria would arrive at 4 p.m. All these letters would also be copied before being allowed to continue on their journey. Each day a hundred letters would filter through the Viennese Black Chamber.

The copies were passed to the cryptanalysts, who sat in little kiosks, ready to tease out the meanings of the messages. As well as supplying the emperors of Austria with invaluable intelligence, the Viennese Black Chamber sold the information it harvested to other powers in Europe. In 1774 an arrangement was made with Abbot Georgel, the secretary at the French Embassy, which gave him access to a twice-weekly package of information in exchange for 1,000 ducats. He then sent these letters, which contained the supposedly secret plans of various monarchs, straight to Louis XV in Paris.

The Black Chambers were effectively making all forms of monoalphabetic cipher insecure. Confronted with such professional cryptanalytic opposition, cryptographers were at last forced to adopt the more complex but more secure Vigenère cipher. Gradually, cipher secretaries began to switch to using polyalphabetic ciphers. In addition to more effective cryptanalysis, there was another pressure that was encouraging the move towards securer forms of encryption: the development of the telegraph, and the need to protect telegrams from interception and decipherment.

Although the telegraph, together with the ensuing telecommunications revolution, came in the nineteenth century, its origins can be traced all the way back to 1753. An anonymous letter in a Scottish magazine described how a message could be sent across large distances by connecting the sender and receiver with 26 cables, one for each letter of the alphabet. The sender could then spell out the message by sending pulses of electricity along each wire. For example, to spell out hello, the sender would begin by sending a signal down the h wire, then down the e wire, and so on. The receiver would somehow sense the electrical current emerging from each wire and read the message. However, this ‘expeditious method of conveying intelligence’, as the inventor called it, was never constructed, because there were several technical obstacles that had to be overcome.

For example, engineers needed a sufficiently sensitive system for detecting electrical signals. In England, Sir Charles Wheatstone and William Fothergill Cooke built detectors from magnetised needles, which would be deflected in the presence of an incoming electric current. By 1839, the Wheatstone-Cooke system was being used to send messages between railway stations in West Drayton and Paddington, a distance of 29 km. The reputation of the telegraph and its remarkable speed of communication soon spread, and nothing did more to popularise its power than the birth of Queen Victoria’s second son, Prince Alfred, at Windsor on 6 August 1844. News of the birth was telegraphed to London, and within the hour The Times was on the streets announcing the news. It credited the technology that had enabled this feat, mentioning that it was ‘indebted to the extraordinary power of the Electro-Magnetic Telegraph’. The following year, the telegraph gained further fame when it helped capture John Tawell, who had murdered his mistress in Slough, and who had attempted to escape by jumping on to a London-bound train. The local police telegraphed Tawell’s description to London, and he was arrested as soon as he arrived at Paddington.

Meanwhile, in America, Samuel Morse had just built his first telegraph line, a system spanning the 60 km between Baltimore and Washington. Morse used an electromagnet to enhance the signal, so that upon arriving at the receiver’s end it was strong enough to make a series of short and long marks, dots and dashes, on a piece of paper. He also developed the now familiar Morse code for translating each letter of the alphabet into a series of dots and dashes, as given in Table 6. To complete his system he designed a sounder, so that the receiver would hear each letter as a series of audible dots and dashes.

Back in Europe, Morse’s approach gradually overtook the Wheatstone-Cooke system in popularity, and in 1851 a European form of Morse Code, which included accented letters, was adopted throughout the Continent. As each year passed, Morse code and the telegraph had an increasing influence on the world, enabling the police to capture more criminals, helping newspapers to bring the very latest news, providing valuable information for businesses, and allowing distant companies to make instantaneous deals.

However, guarding these often sensitive communications was a major concern. The Morse code itself is not a form of cryptography, because there is no concealment of the message. The dots and dashes are merely a convenient way to represent letters for the telegraphic medium; Morse code is effectively nothing more than an alternative alphabet. The problem of security arose primarily because anyone wanting to send a message would have to deliver it to a Morse code operator, who would then have to read it in order to transmit it. The telegraph operators had access to every message, and hence there was a risk that one company might bribe an operator in order to gain access to a rival’s communications. This problem was outlined in an article on telegraphy published in 1853 in England’s Quarterly Review:

Means should also be taken to obviate one great objection, at present felt with respect to sending private communications by telegraph – the violation of all secrecy – for in any case half-a-dozen people must be cognisant of every word addressed by one person to another. The clerks of the English Telegraph Company are sworn to secrecy, but we often write things that it would be intolerable to see strangers read before our eyes. This is a grievous fault in the telegraph, and it must be remedied by some means or other.

Table 6 International Morse Code symbols.

The solution was to encipher a message before handing it to the telegraph operator. The operator would then turn the ciphertext into Morse code before transmitting it. As well as preventing the operators from seeing sensitive material, encryption also stymied the efforts of any spy who might be tapping the telegraph wire. The polyalphabetic Vigenère cipher was clearly the best way to ensure secrecy for important business communications. It was considered unbreakable, and became known as le chiffre indéchiffrable. Cryptographers had, for the time being at least, a clear lead over the cryptanalysts.

Mr Babbage Versus the Vigenère Cipher

The most intriguing figure in nineteenth-century cryptanalysis is Charles Babbage, the eccentric British genius best known for developing the blueprint for the modern computer. He was born in 1791, the son of Benjamin Babbage, a wealthy London banker. When Charles married without his father’s permission, he no longer had access to the Babbage fortune, but he still had enough money to be financially secure, and he pursued the life of a roving scholar, applying his mind to whatever problem tickled his fancy. His inventions include the speedometer and the cowcatcher, a device that could be fixed to the front of steam locomotives to clear cattle from railway tracks. In terms of scientific breakthroughs, he was the first to realise that the width of a tree ring depended on that year’s weather, and he deduced that it was possible to determine past climates by studying ancient trees. He was also intrigued by statistics, and as a diversion he drew up a set of mortality tables, a basic tool for today’s insurance industry.

Babbage did not restrict himself to tackling scientific and engineering problems. The cost of sending a letter used to depend on the distance the letter had to travel, but Babbage pointed out that the cost of the labour required to calculate the price for each letter was more than the cost of the postage. Instead, he proposed the system we still use today – a single price for all letters, regardless of where in the country the addressee lives. He was also interested in politics and social issues, and towards the end of his life he began a campaign to get rid of the organ-grinders and street musicians who roamed London. He complained that the music ‘not infrequently gives rise to a dance by little ragged urchins, and sometimes half-intoxicated men, who occasionally accompany the noise with their own discordant voices. Another class who are great supporters of street music consists of ladies of elastic virtue and cosmopolitan tendencies, to whom it affords a decent excuse for displaying their fascinations at their open windows.’ Unfortunately for Babbage, the musicians fought back by gathering in large groups around his house and playing as loud as possible.

The turning point in Babbage’s scientific career came in 1821, when he and the astronomer John Herschel were examining a set of mathematical tables, the sort used as the basis for astronomical, engineering and navigational calculations. The two men were disgusted by the number of errors in the tables, which in turn would generate flaws in important calculations. One set of tables, the Nautical Ephemeris for Finding Latitude and Longitude at Sea, contained over a thousand errors. Indeed, many shipwrecks and engineering disasters were blamed on faulty tables.

These mathematical tables were calculated by hand, and the mistakes were simply the result of human error. This caused Babbage to exclaim, ‘I wish to God these calculations had been executed by steam!’ This marked the beginning of an extraordinary endeavour to build a machine capable of faultlessly calculating the tables to a high degree of accuracy. In 1823 Babbage designed ‘Difference Engine No. 1’, a magnificent calculator consisting of 25,000 precision parts, to be built with government funding. Although Babbage was a brilliant innovator, he was not a great implementer. After ten years of toil, he abandoned ‘Difference Engine No. 1’, cooked up an entirely new design, and set to work building ‘Difference Engine No. 2’.

When Babbage abandoned his first machine, the government lost confidence in him and decided to cut its losses by withdrawing from the project – it had already spent £17,470, enough to build a pair of battleships. It was probably this withdrawal of support that later prompted Babbage to make the following complaint: ‘Propose to an Englishman any principle, or any instrument, however admirable, and you will observe that the whole effort of the English mind is directed to find a difficulty, a defect, or an impossibility in it. If you speak to him of a machine for peeling a potato, he will pronounce it impossible: if you peel a potato with it before his eyes, he will declare it useless, because it will not slice a pineapple.’

Figure 12 Charles Babbage.

Science and Society Picture Library, London.

Lack of government funding meant that Babbage never completed Difference Engine No. 2. The scientific tragedy was that Babbage’s machine would have been a stepping stone to the Analytical Engine. Rather than merely calculating a specific set of tables, the Analytical Engine would have been able to solve a variety of mathematical problems depending on the instructions that it was given. In fact, the Analytical Engine provided the template for modern computers. The design included a ‘store’ (memory) and a ‘mill’ (processor), which would allow it to make decisions and repeat instructions, which are equivalent to the ‘IF … THEN … ’ and ‘LOOP’ commands in modern programming.

A century later, during the course of the Second World War, the first electronic incarnations of Babbage’s machine would have a profound effect on cryptanalysis, but, in his own lifetime, Babbage made an equally important contribution to codebreaking: he succeeded in breaking the Vigenère cipher, and in so doing he made the greatest breakthrough in cryptanalysis since the Arab scholars of the ninth century broke the monoalphabetic cipher by inventing frequency analysis. Babbage’s work required no mechanical calculations or complex computations. Instead, he employed nothing more than sheer cunning.

Babbage had become interested in ciphers at a very young age. In later life, he recalled how his childhood hobby occasionally got him into trouble: ‘The bigger boys made ciphers, but if I got hold of a few words, I usually found out the key. The consequence of this ingenuity was occasionally painful: the owners of the detected ciphers sometimes thrashed me, though the fault lay in their own stupidity.’ These beatings did not discourage him, and he continued to be enchanted by cryptanalysis. He wrote in his autobiography that ‘deciphering is, in my opinion, one of the most fascinating of arts’.

He soon gained a reputation within London society as a cryptanalyst prepared to tackle any encrypted message, and strangers would approach him with all sorts of problems. For example, Babbage helped a desperate biographer attempting to decipher the shorthand notes of John Flamsteed, England’s first Astronomer Royal. He also came to the rescue of a historian, solving a cipher of Henrietta Maria, wife of Charles I. In 1854, he collaborated with a barrister and used cryptanalysis to reveal crucial evidence in a legal case. Over the years, he accumulated a thick file of encrypted messages, which he planned to use as the basis for an authoritative book on cryptanalysis, entitled The Philosophy of Decyphering. The book would contain two examples of every kind of cipher, one that would be broken as a demonstration and one that would be left as an exercise for the reader. Unfortunately, as with many other of his grand plans, the book was never completed.

While most cryptanalysts had given up all hope of ever breaking the Vigenère cipher, Babbage was inspired to attempt a decipherment by an exchange of letters with John Hall Brock Thwaites, a dentist from Bristol with a rather innocent view of ciphers. In 1854, Thwaites claimed to have invented a new cipher, which, in fact, was equivalent to the Vigenère cipher. He wrote to the Journal of the Society of Arts with the intention of patenting his idea, apparently unaware that he was several centuries too late. Babbage wrote to the Society, pointing out that ‘the cypher … is a very old one, and to be found in most books’. Thwaites was unapologetic and challenged Babbage to break his cipher. Whether or not it was breakable was irrelevant to whether or not it was new, but Babbage’s curiosity was sufficiently aroused for him to embark on a search for a weakness in the Vigenère cipher.

Cracking a difficult cipher is akin to climbing a sheer cliff face. The cryptanalyst is seeking any nook or cranny which could provide the slightest purchase. In a monoalphabetic cipher the cryptanalyst will latch on to the frequency of the letters, because the commonest letters, such as e, t and a, will stand out no matter how they have been disguised. In the polyalphabetic Vigenère cipher the frequencies are much more balanced, because the keyword is used to switch between cipher alphabets. Hence, at first sight, the rock face seems perfectly smooth.

Remember, the great strength of the Vigenère cipher is that the same letter will be enciphered in different ways. For example, if the keyword is KING, then every letter in the plaintext can potentially be enciphered in four different ways, because the keyword contains four letters. Each letter of the keyword defines a different cipher alphabet in the Vigenère square, as shown in Table 7. The e column of the square has been highlighted to show how it is enciphered differently, depending on which letter of the keyword is defining the encipherment:

If the K of KING is used to encipher e, then the resulting ciphertext letter is O.

If the I of KING is used to encipher e, then the resulting ciphertext letter is M.

If the N of KING is used to encipher e, then the resulting ciphertext letter is R.

If the G of KING is used to encipher e, then the resulting ciphertext letter is K.

Table 7 A Vigenère square used in combination with the keyword KING. The keyword defines four separate cipher alphabets, so that the letter e may be encrypted as O, M, R or K.

Similarly, whole words will be enciphered in different ways: the word the, for example, could be enciphered as DPR, BUK, GNO or ZRM, depending on its position relative to the keyword. Although this makes cryptanalysis difficult, it is not impossible. The important point to note is that if there are only four ways to encipher the word the, and the original message contains several instances of the word the, then it is highly likely that some of the four possible encipherments will be repeated in the ciphertext. This is demonstrated in the following example, in which the line The Sun and the Man in the Moon has been enciphered using the Vigenère cipher and the keyword KING.

Keyword K I N G K I N G K I N G K I N G K I N G K I N G

Plaintext t h e s u n a n d t h e m a n i n t h e m o o n

Ciphertext D P R Y E V N T N B U K W I A O X B U K W W B T

The word the is enciphered as DPR in the first instance, and then as BUK on the second and third occasions. The reason for the repetition of BUK is that the second the is displaced by eight letters with respect to the third the, and eight is a multiple of the length of the keyword, which is four letters long. In other words, the second the was enciphered according to its relationship to the key word (the is directly below ING), and by the time we reach the third the, the keyword has cycled round exactly twice, to repeat the relationship, and hence repeat the encipherment.

Babbage realised that this sort of repetition provided him with exactly the foothold he needed in order to conquer the Vigenère cipher. He was able to define a series of relatively simple steps which could be followed by any cryptanalyst to crack the hitherto chiffre indéchiffrable. To demonstrate his brilliant technique, let us imagine that we have intercepted the ciphertext shown in Figure 13. We know that it was enciphered using the Vigenère cipher, but we know nothing about the original message, and the keyword is a mystery.

The first stage in Babbage’s cryptanalysis is to look for sequences of letters that appear more than once in the ciphertext. There are two ways that such repetitions could arise. The most likely is that the same sequence of letters in the plaintext has been enciphered using the same part of the key. Alternatively, there is a slight possibility that two different sequences of letters in the plaintext have been enciphered using different parts of the key, coincidentally leading to the identical sequence in the ciphertext. If we restrict ourselves to long sequences, then we largely discount the second possibility, and, in this case, we shall consider repeated sequences only if they are of four letters or more. Table 8 is a log of such repetitions, along with the spacing between the repetition. For example, the sequence E-F-I-Q appears in the first line of the ciphertext and then in the fifth line, shifted forward by 95 letters.

Figure 13 The ciphertext, enciphered using the Vigenère cipher.

As well as being used to encipher the plaintext into ciphertext, the keyword is also used by the receiver to decipher the ciphertext back into plaintext. Hence, if we could identify the keyword, deciphering the text would be easy. At this stage we do not have enough information to work out the keyword, but Table 8 does provide some very good clues as to its length. Having listed which sequences repeat themselves and the spacing between these repetitions, the rest of the table is given over to identifying the factors of the spacing – the numbers that will divide into the spacing. For example, the sequence W-C-X-Y-M repeats itself after 20 letters, and the numbers 1, 2, 4, 5, 10 and 20 are factors, because they divide perfectly into 20 without leaving a remainder. These factors suggest six possibilities:

(1) The key is 1 letter long and is recycled 20 times between encryptions.

(2) The key is 2 letters long and is recycled 10 times between encryptions.

(3) The key is 4 letters long and is recycled 5 times between encryptions.

(4) The key is 5 letters long and is recycled 4 times between encryptions.

(5) The key is 10 letters long and is recycled 2 times between encryptions.

(6) The key is 20 letters long and is recycled 1 time between encryptions.

The first possibility can be excluded, because a key that is only 1 letter long gives rise to a monoalphabetic cipher – only one row of the Vigenère square would be used for the entire encryption, and the cipher alphabet would remain unchanged; it is unlikely that a cryptographer would do this. To indicate each of the other possibilities, a