Introduction
You live in a world where privacy depends entirely on secret codes. Every time you send a message, buy something online, or log into a bank account, you rely on complex mathematics to protect your data. However, you face a constant risk that your digital security will fail because every encryption method eventually falls to human ingenuity or advancing technology.
Simon Singh argues that the history of cryptography is an evolutionary arms race between codemakers and codebreakers. As you explore this dynamic, you discover how secret communication evolved from ancient hiding techniques to mechanical devices and modern digital algorithms. The book traces this struggle through historical conflicts, brilliant intellects, and mathematical breakthroughs.
- How did ancient rulers hide sensitive military orders from enemy eyes?
- What made supposedly unbreakable ciphers vulnerable to frequency analysis?
- Why did the mechanical complexity of the Enigma machine ultimately fail?
- How does modern public key cryptography secure the internet without exchanging shared secrets?
1. The Evolution of Secret Writing
You communicate securely by choosing between two distinct methods. Secret communication is divided into steganography, which hides the existence of a message, and cryptography, which hides its meaning.
Secret communication relies on either hiding the existence of a message through steganography or scrambling its meaning using transposition and substitution ciphers.
Steganography relies on concealment such as invisible inks or microdots, but fails entirely if the message carrier is searched. Cryptography scrambles the text so that even if intercepted, it remains unreadable without the protocol.
German agents in Latin America reduced a page of text to a microdot less than 1 millimeter in diameter and hid it under a full stop on an innocuous letter, which the FBI spotted in 1941.
Cryptographic ciphers operate through either transposition, which rearranges letter order, or substitution, which replaces letters with others. Transposition creates an anagram-like effect, such as the Spartan scytale where a leather strip was wound around a wooden staff. Substitution leaves letter positions intact while changing their identities.
In 404 B.C., Lysander of Sparta wrapped a battered messenger's belt around his scytale to discover an impending attack by Pharnabazus.
Steganography versus Cryptography
The security of a cryptographic system must rely on keeping the key secret, not on keeping the encryption algorithm secret. Formulated by Auguste Kerckhoffs in 1883, this principle states that even if an enemy knows the general system of encryption, they cannot read the message without knowing the specific key.
The Arab Cryptanalysts
Arab cryptanalysts invented frequency analysis, breaking substitution ciphers that had remained secure for centuries. Ninth-century scientist al-Kindi realized that different letters appear with characteristic frequencies in any given language. By counting symbol occurrences in a ciphertext, an analyst can match them to expected language letter frequencies.
To break a monoalphabetic substitution cipher, you follow these steps:
- Find a plaintext sample in the same language and count the occurrences of each letter, ranking them from most to least frequent.
- Classify the symbols in the ciphertext by their frequency of occurrence.
- Map the most frequent ciphertext symbols to the most frequent plaintext letters.
The Babington Plot
Thomas Phelippes intercepted and deciphered the messages of Mary Queen of Scots, forging a postscript to gather all conspirator names. Using frequency analysis and recognizing nulls, English spymaster Walsingham's codebreaker read Mary's correspondence with Anthony Babington. Phelippes then forged a postscript in Mary's style asking for the names of the six gentlemen planning the assassination, sealing Mary's conviction.
Gilbert Gifford smuggled letters into Chartley Hall inside a hollow beer barrel bung, while Walsingham intercepted, copied, and resealed every message during 1586, following 18 years of imprisonment for Mary before her trial.
2. Le Chiffre Indéchiffrable
For centuries, cryptographers sought a cipher that could resist frequency analysis entirely. The Vigenère cipher achieved this by using 26 distinct cipher alphabets organized in a square and selected via a repeating keyword.
Developed into its final form by Blaise de Vigenère, the cipher cycles through rows of a Vigenère square based on a keyword.
Because identical plaintext letters map to different ciphertext letters, frequency analysis fails. Encrypting 'divert troops to east ridge' using the keyword WHITE yields a secure ciphertext. You can construct and solve it using specific rules:
- Spell out the keyword above the message, repeating it so each letter aligns with a message letter.
- Identify the row in the Vigenère square corresponding to the key letter.
- Find the intersection of the plaintext letter column and the key row to determine the ciphertext letter.
The Black Chambers
European governments industrialized cryptanalysis in the eighteenth century by establishing dedicated intelligence centers called Black Chambers to systematically intercept, copy, and decode diplomatic mail. Each European power maintained a Black Chamber to intercept letters without disrupting postal schedules. Secretaries melted seals, stenographers duplicated messages, and language specialists handled unusual scripts before resealing and forwarding the mail within hours.
The Geheime Kabinets-Kanzlei in Vienna processed one hundred letters daily and even sold harvested intelligence to other European powers. Viennese embassies' mail arrived at the Black Chamber at 7 A.M., mail in transit arrived at 10 A.M., and mail leaving for outside destinations arrived at 4 P.M. In a 1774 arrangement, information was traded to the French Embassy for 1,000 ducats when Abbot Georgel paid for a twice-weekly intelligence package.
Cracking the Vigenère Cipher
Charles Babbage was a brilliant 19th-century polymath and inventor who laid the intellectual foundations for modern computers and made the greatest breakthrough in cryptanalysis since the ninth century. Born in 1791, Babbage pursued the life of a roving scholar, inventing cowcatchers and speedometers, establishing climate analysis via tree rings, and designing mechanical calculators. Frustrated by human errors in mathematical tables in 1821, he designed the Difference Engine and the programmable Analytical Engine, which featured a store and a mill. Babbage attempted to build Difference Engine No. 1 using £17,470 in government funding before abandoning it for Difference Engine No. 2, though neither was completed due to funding cuts and 25,000 precision parts.
Charles Babbage developed a systematic multi-step technique to break the Vigenère cipher by analyzing ciphertext repetitions, finding keyword lengths, and applying frequency analysis.
Challenged by Bristol dentist John Hall Brock Thwaites in 1854, Babbage cracked a Vigenère cipher based on the five-letter keyword EMILY, which decrypted into verses from Alfred Tennyson's poem 'The Vision of Sin'. Friedrich Wilhelm Kasiski published the technique in 1863 in 'Die Geheimschriften und die Dechiffrir-kunst', naming it the Kasiski Test. You can apply this method to any intercepted Vigenère message by following these steps:
- Look for sequences of letters that appear more than once in the ciphertext, restricting consideration to sequences of four letters or more.
- Log the repetitions along with the spacing between them.
- Identify the factors that divide evenly into each spacing to determine potential key lengths.
- Check all spacings for a common factor to deduce the exact length of the keyword.
- Split the ciphertext into parts based on the keyword length, so that each part corresponds to a single letter of the keyword.
- Establish a frequency distribution for the letters in each part of the ciphertext.
- Compare each ciphertext frequency distribution with the standard frequency distribution to deduce the shift and identify the corresponding keyword letter.
3. The Mechanization of Secrecy
The invention of the telegraph revolutionized long-distance communication in the nineteenth century, but it also created a major security flaw because network operators had to read every message before transmitting it. The telegraph allowed rapid message transmission across great distances using electrical pulses, but the requirement for human operators to handle and transmit the text exposed sensitive personal and business communications to bribery and espionage.
In 1844, news of the birth of Queen Victoria's second son, Prince Alfred, reached London within an hour, and in 1845 the telegraph helped police capture the murderer John Tawell by transmitting his description ahead of his train.
The rise of radio and telegraph communications forced nations to industrialize cryptanalysis and deploy complex cipher machines like the Enigma.
Guglielmo Marconi's invention of radio revolutionized long-range communication but introduced severe security vulnerabilities due to the ease of signal interception. Marconi successfully transmitted the letter S across the Atlantic from Poldhu in Cornwall to St. John's in Newfoundland, a distance of 3,500 km, proving radio was not limited by the Earth's curvature.
Telecommunications Vulnerabilities
Traffic analysis allows intelligence agencies to extract valuable tactical data from intercepted radio transmissions even when ciphers remain unbroken. During the First World War, French listening posts and direction-finding stations used operator fists and signal bearings to track the paths and objectives of German battalions despite cipher changes.
You can exploit radio patterns using specific investigative procedures:
- Operate listening posts to recognize individual radio operators' fists based on pauses, transmission speed, and dot-dash lengths.
- Establish direction-finding stations where antennas are adjusted until incoming signals are strongest.
- Combine directional information from multiple stations to pinpoint the exact source and movement of enemy transmissions.
British naval intelligence in Room 40 intercepted and decrypted the Zimmermann telegram, prompting the United States to abandon neutrality and enter World War I. Arthur Balfour presented the decrypted Zimmermann telegram to American Ambassador Walter Page on February 23, 1917, leading President Woodrow Wilson to ask Congress for a declaration of war against Germany.
Wartime Cryptanalysis
The complex German ADFGVX cipher used during the final offensive of World War I was successfully broken by French cryptanalyst Georges Painvin. Georges Painvin cracked an ADFGVX message on June 2, 1918, revealing a frantic munition request near Compiègne, which allowed the Allies to reinforce the front line and defeat the German offensive.
The Enigma Machine
Arthur Scherbius mechanized cryptography by creating the Enigma machine, an electrical version of Alberti's cipher disk featuring a keyboard, a rotating scrambler, and a lampboard. Enigma achieves extreme complexity by combining multiple interchangeable scramblers, a static reflector, and a plugboard to multiply the number of possible key settings, totaling approximately 10,000,000,000,000,000 possible keys.
You operate an Enigma machine through a structured procedure:
- Rotate the scramblers to the initial starting position dictated by the daily codebook.
- Type the plaintext letter by letter and note the illuminated ciphertext letter on the lampboard.
- Transmit the ciphertext via radio operator.
- Receiver sets up the machine according to the codebook and types in the ciphertext to read the plaintext.
4. Cracking the Enigma
A disaffected German insider provided Allied intelligence with the secret documents needed to replicate the military Enigma machine.
Hans-Thilo Schmidt, whose military career was cut short and whose business failed, sold secret Enigma instructions and operational manuals to French intelligence for money and revenge. On November 8, 1931, Schmidt met a French agent codenamed Rex at the Grand Hotel in Verviers, Belgium, selling photographs of two secret Enigma instruction documents for 10,000 marks, which was equivalent to $30,000 in modern money. Schmidt eventually provided monthly codebooks containing day keys for 38 months.
The Mathematical Approach
Marian Rejewski discovered that the number of links in letter chains derived from repeated message keys depends entirely on scrambler settings, independent of plugboard cables. By analyzing predictable relationships between message key positions, Rejewski tracked letter chains whose link counts reflected only the scrambler arrangement and orientations.
If a plugboard cable was swapped from connecting S and G to swapping T and K instead, some letters within the chains changed, but the exact number of links in each chain remained constant. This insight reduced the total number of possible day keys down to 105,456 possible scrambler settings, calculated as 6 arrangements multiplied by 17,576 orientations. The Polish team spent a year cataloguing these chain lengths and built automated electromechanical units called bombes to rapidly find the daily keys. Rejewski built an automated unit consisting of six adapted Enigma machines working in parallel to check all orientations, finding the day key in roughly two hours.
You break down the key discovery process through systematic steps:
- Set the Enigma replica scramblers according to the established scrambler part of the day key.
- Remove all cables from the plugboard so it has no effect.
- Type intercepted ciphertext into the machine to generate partial gibberish.
- Identify recognizable words or phrases to deduce which letter pairs were swapped by plugboard cables.
Alan Turing and the Bombe
Alan Turing bypassed the massive complexity of Enigma plugboards by inventing electromechanical machines called bombes that utilized loops in known message text. Overworked German Enigma operators frequently selected predictable message keys instead of truly random three-letter sequences, creating significant security vulnerabilities known as cillies. Operators chose three consecutive letters from the Enigma keyboard, such as QWE or BNM, or used initials like C.I.L. for message keys.
Turing noticed that rigid military protocols resulted in predictable daily messages, yielding known plaintext-ciphertext pairs called cribs. The prototype bombe Victory arrived at Bletchley Park on March 14, 1940, followed by an upgraded version named Agnus Dei on August 8, 1940.
The Strategic Impact of Ultra
Decrypted enemy traffic, codenamed Ultra, provided Allied commanders with critical advance knowledge that significantly shortened World War II. Intelligence gathered from intercepted messages gave the Allies decisive advantages in North Africa, the Mediterranean, and the D-Day landings. Historians estimated that without Ultra and the breaking of Enigma, the war would have been prolonged until 1948 instead of ending in 1945.
Sir Harry Hinsley noted that Allied estimates of enemy divisions prior to D-Day were accurate in all but two operational items thanks to accumulated Ultra intelligence, correctly tracking 58 total German divisions in the west. The German Navy used a more complex Enigma network with eight scramblers and a variable reflector with 26 possible orientations, forcing the Allies to rely on targeted mining missions and espionage to secure codebooks. Ian Fleming proposed Operation Ruthless, a plan to crash a captured German bomber in the English Channel to board a German ship and steal codebooks, though it was cancelled due to a lack of shipping.
5. The Language Barrier
Untranslated ancient scripts and indigenous military languages demonstrate that linguistic structures can secure messages when traditional mathematics is absent.
The Navajo Code Talkers
Philip Johnston recognized that the complex structure of the Navajo language, combined with its lack of exposure to German anthropologists and the absence of a written script, made it an ideal secure communication method for marine battalions in the Pacific. In 1942, twenty-nine Navajos began a marine communications course, and their rapid, flawless transmissions proved impenetrable to Navy Intelligence cryptanalysts who tried to crack them. You can apply these recruitment principles when you secure communications:
- Select tribes based on population size, literacy rates, and lack of previous exposure to foreign students or anthropologists.
- Employ pairs of Native American radio operators in each battalion to transmit and receive messages.
Because languages like Navajo lacked modern military jargon, trainees compiled a lexicon using natural world terms—such as birds for planes and fish for ships—alongside a phonetic spelling system using homophones for common letters to prevent frequency analysis. Commanding officers were called war chiefs, platoons were mud-clans, and the letter a was represented by ant, apple, or axe. This complete initial lexicon contained 274 words, and 234 common terms were added later. There was a 55 kg minimum weight requirement for recruits, and the program ultimately grew to 420 total Navajo code talkers.
Deciphering Linear B
Found in 1799, the Rosetta Stone featured the same decree inscribed in Greek, demotic, and hieroglyphics, providing a known plaintext via readable Greek text to unlock unknown ancient scripts. French soldiers discovered the black basalt slab at Fort Julien in Rosetta, which was later handed over to the British under the Treaty of Capitulation. This decree was dated 196 B.C., and the basalt slab measured 118 cm height, 77 cm width, 30 cm thickness, and three-quarters of a ton weight, containing 54 lines in the Greek text. Thomas Young and Jean-François Champollion used cartouches containing royal names like Ptolemy, Cleopatra, and Alexander to assign sound values to individual hieroglyphic symbols, disproving the dogma that the script was purely ideographic. Champollion unlocked traditional Egyptian writing on September 14, 1822, when he identified the sun semagram as ra in a cartouche, spelling out Rameses.
Alice Kober analyzed structural repetitions and inflections of Linear B word triplets, deducing that certain intermediate signs acted as bridging syllables carrying elements of both stem and grammatical ending. Alice Kober constructed a grid of ten signs—two vowels wide and five coefficients deep—establishing rigorous relationships between characters before her death in 1950 at age forty-three. You can follow her cryptanalytic steps when you analyze unknown inflective writing systems:
- Identify word triplets where stems are identical but endings vary across different cases.
- Deduce bridging syllables where a sign contributes a consonant to the stem and a vowel to the ending.
- Construct a grid mapping consonant and vowel relationships among unknown characters.
The script featured 90 distinct characters. Michael Ventris made a breakthrough by guessing place names like Amnisos, Knossos, and Tulissos, and later translating common inventory words like toso and tosa. The partnership between architect Michael Ventris and philologist John Chadwick successfully proved that Linear B was an archaic form of Greek, dating 500 years older than Homer.
6. Alice and Bob Go Public
The fundamental dilemma of traditional symmetric cryptography is that before two people can exchange a secret encrypted message, they must already securely share a secret key. If a sender wants to encrypt a message, they must use a key, which itself must be transmitted securely to the receiver. For two millennia, the necessity of key exchange was considered an indisputable axiom of cryptography, creating a catch-22 situation that threatened to deny the public access to digital privacy as computer networks expanded.
The Diffie-Hellman-Merkle scheme allows two parties to agree on a shared secret key over an insecure public channel without ever transmitting the key itself.
Alice and Bob want to exchange secret messages over the phone, but to do so they must meet in person once a week to exchange keys or use couriers. Whitfield Diffie, Martin Hellman, and Ralph Merkle developed and publicly demonstrated this key exchange scheme at the National Computer Conference in June 1976.
You can follow these steps to perform the key exchange:
- Agree on public values for Y and P
- Choose a secret number A (for Alice) or B (for Bob)
- Calculate alpha = Y^A (mod P) and beta = Y^B (mod P)
- Exchange alpha and beta over an insecure line
- Calculate the shared key by raising received values to your secret number
The Birth of Public Key Cryptography
A one-way function is a mathematical operation that is easy to perform in one direction but extremely difficult to reverse without special information. Multiplying two prime numbers together takes seconds, but factoring the resulting product back into its original primes is extremely time-consuming.
Prime Suspects and RSA
Ron Rivest, Adi Shamir, and Leonard Adleman created a practical asymmetric system where a user selects two prime numbers, p and q, and multiplies them to get N. Ron Rivest had a revelation while reading a mathematics textbook late at night after Passover in April 1977, leading to the creation of RSA.
Follow these steps for the RSA algorithm:
- Pick two prime numbers p and q
- Multiply them together to get N (N = p x q)
- Publish N as the public encryption key
- Keep p and q secret as the private decryption key
Martin Gardner issued a challenge in Scientific American in August 1977 to break an RSA cipher, which took 17 years and a distributed network of 600 volunteers to finally factor.
7. Encryption for the Masses
Phil Zimmermann created Pretty Good Privacy to provide ordinary citizens with strong encryption and protect individual privacy against mass surveillance. Moving from political activism against nuclear war to digital rights advocacy, he argued that electronic mail is inherently vulnerable to automated interception compared to traditional postal mail. This reality necessitates accessible encryption for the public to prevent driftnet fishing of the approximately 100 million e-mails sent globally each day.
Modern encryption software like PGP brought military-grade privacy to ordinary citizens, sparking an intense political debate between government surveillance and civil liberties.
Zimmermann received an email from Latvia during the shelling of the Moscow Parliament building, thanking him for spreading PGP from the Baltic to the Far East to help democratic people.
Pretty Good Privacy
PGP solves computing speed limitations of RSA by combining fast symmetric encryption for messages with asymmetric RSA encryption for keys. Because RSA encryption requires heavy computing power and is too slow for long messages, PGP encrypts the message using a fast symmetric IDEA cipher and then uses RSA solely to encrypt the much smaller IDEA key. It also incorporates mouse movements for generating unique prime pairs.
Follow these steps to encrypt a message:
- Write the email message.
- Select the PGP option from a menu on the computer screen.
- Type in the recipient's name to automatically find their public key.
- Let PGP automatically perform the encryption and digital signature generation.
To prevent PGP from being outlawed by anticipated anticrime legislation, Zimmermann released it as freeware on a Usenet bulletin board in 1991. This release triggered patent infringement claims from RSA Data Security, Inc., and criminal investigations for illegally exporting a weapon without a State Department license. In February 1993, two government investigators visited Zimmermann to question him regarding the illegal export of a weapon, and he was pursued by the FBI and subjected to a grand jury investigation for 3 years.
The Crypto Wars
The rise of strong cryptography ignited a global debate between law enforcement agencies wanting restrictions and civil libertarians and businesses demanding absolute privacy. Law enforcers argue that encryption protects criminals and terrorists, hampering legal wiretaps and intelligence gathering. Conversely, civil libertarians and businesses maintain that privacy is a fundamental human right and that strong encryption is indispensable for secure electronic commerce.
The FBI and American law enforcement agencies frequently cited the Four Horsemen of the Infocalypse—drug dealers, organized crime, terrorists, and pedophiles—as primary threats enabled by strong encryption. They contrasted a 1967 Supreme Court ruling allowing police wiretaps with court authorization against a mere 1,000 court-sanctioned wiretaps in America in 1994 compared with 250,000 federal cases.
8. A Quantum Leap into the Future
Modern public key cryptography relies on mathematical problems that are difficult for traditional computers to solve. Quantum computers operating on quantum mechanical principles threaten to break modern public key cryptography by performing vast parallel calculations.
Unlike classical bits, quantum bits or qubits can exist in a superposition of states. A quantum computer can test enormous numbers of possibilities simultaneously and factor large numbers exponentially faster than traditional supercomputers. With 250 qubits, you can represent 10 to the power of 75 combinations, a number that exceeds the total number of atoms in the universe.
Peter Shor developed a quantum computer program capable of factoring giant numbers to crack RSA. Lov Grover developed a program to search lists rapidly to crack DES.
Quantum Cryptography Protocols
While quantum computers threaten current ciphers, quantum theory simultaneously inspires quantum cryptography. This field provides perfect privacy that cannot be cracked even by the might of a quantum computer.
In 1984 at the Croton-Harmon railway station, Charles Bennett and Gilles Brassard had their eureka moment for quantum cryptography while waiting for a train. They transformed Wiesner's quantum money concept into a secure key distribution protocol for cryptography.
Quantum cryptography allows communicating parties to detect eavesdropping because any interception attempt alters the quantum state of photons and introduces errors. When an eavesdropper named Eve measures a photon with the wrong detector, she forces it to emerge in a modified state. This introduces measurable errors into the communication stream that Alice and Bob can spot during error-checking.
You can follow a precise protocol to establish a secure key:
- Alice sends Bob a series of photons, and Bob measures them.
- Alice tells Bob on which occasions he measured them in the correct way.
- Alice and Bob discard the measurements that Bob made incorrectly, and concentrate on those measurements he made correctly in order to create an identical pair of onetime pads.
- Alice and Bob check the integrity of their onetime pads by testing a few of the digits.
- If the verification procedure is satisfactory, they can use the onetime pad to encrypt a message; if the verification reveals errors, they know that the photons were being tapped by Eve, and they need to start all over again.
An initial sequence of 1,075 binary digits can be checked by picking 75 digits at random. The chance of Eve being on the line and not affecting Bob's measurement of the checked digits is one in a billion.
10 Key Ideas
- Secret communication relies on either hiding the existence of a message through steganography or scrambling its meaning using transposition and substitution ciphers.
- Arab cryptanalysts invented frequency analysis, breaking substitution ciphers that had remained secure for centuries.
- Developed into its final form by Blaise de Vigenère, the cipher cycles through rows of a Vigenère square based on a keyword.
- Charles Babbage developed a systematic multi-step technique to break the Vigenère cipher by analyzing ciphertext repetitions, finding keyword lengths, and applying frequency analysis.
- The rise of radio and telegraph communications forced nations to industrialize cryptanalysis and deploy complex cipher machines like the Enigma.
- Untranslated ancient scripts and indigenous military languages demonstrate that linguistic structures can secure messages when traditional mathematics is absent.
- The Diffie-Hellman-Merkle scheme allows two parties to agree on a shared secret key over an insecure public channel without ever transmitting the key itself.
- Modern encryption software like PGP brought military-grade privacy to ordinary citizens, sparking an intense political debate between government surveillance and civil liberties.
- Quantum computers operating on quantum mechanical principles threaten to break modern public key cryptography by performing vast parallel calculations.
- Quantum cryptography protocol ensures that any interception attempt alters the quantum state of photons, allowing communicating parties to detect eavesdropping instantly.