Science › Mathematics
Information theory (Shannon)
1948· Mathematics
ProvenTested many times; part of working physics.
Claude Shannon's 1948 mathematical theory of how much information a message carries and how reliably it can be sent.
See it on the Science map →How it works
Shannon measured information in bits: one bit answers a single yes/no question. A message's information is how surprising it is; predictable messages carry little. He proved each channel, like a phone line, has a maximum reliable data rate, and that clever coding can approach it with almost no errors despite noise. His formula for average information has the same mathematical form as the entropy formula of Boltzmann and Gibbs.
Why it matters
It underlies all digital communication and storage: compression, error correction, the internet, mobile phones. Physicists now treat information as a physical quantity.
How we know
Codes used in 4G, 5G and deep-space probes come within a fraction of a decibel of Shannon's limit. Landauer's 1961 principle, that erasing a bit releases heat, was measured in 2012.
Still a mystery
Whether information is more fundamental than matter and energy, John Wheeler's 'it from bit' idea, is an open and speculative question.
Connections (4)
▸Information theory (Shannon) has a quantum version in Quantum computingWell supported
The relationship
Shannon's bit is either 0 or 1. Its quantum version, the qubit, can be in a superposition of both and entangled with other qubits. Quantum information theory extends Shannon's ideas to qubits: how much they carry, how to correct their errors, and how to transmit them. Quantum computers aim to exploit this to solve certain problems far faster.
How the link was found
Richard Feynman proposed quantum computers in 1981. Peter Shor's 1994 factoring algorithm and 1995 error-correcting code showed real promise; Benjamin Schumacher coined 'qubit' around 1995.
Is it proven?
Quantum error correction and quantum communication have been demonstrated. Large, useful quantum computers are not yet built, so practical advantage is still largely unproven.
Open questions and mysteries
Whether large machines can be made reliable enough, and which practical problems will truly benefit.
▸Information theory (Shannon) gave the "bit" to Information is physicalProven
The relationship
Shannon gave a precise measure of information, the bit, and a formula for its uncertainty that matched thermodynamic entropy. With information quantified, physicists could ask what processing it costs. Landauer answered: erasing one bit must release a minimum amount of heat, tying the bit to energy.
How the link was found
Claude Shannon at Bell Labs in 1948; John Tukey coined the word "bit," which Shannon popularized. Rolf Landauer at IBM made the physical link in 1961.
Why it's proven
Landauer's limit was measured in 2012 by Bérut and colleagues. Shannon's theory underlies every working communication system.
Open questions and mysteries
Whether Shannon's information and physical entropy are truly identical or just share a formula is still discussed.
▸Information theory (Shannon) uses the same formula as Entropy: disorder always growsProven
The relationship
Shannon's measure of a message's average information has the same mathematical form as the entropy of Gibbs and Boltzmann: both add up probabilities weighted by their logarithms. Thermodynamic entropy can be read as the missing information about which exact microscopic arrangement a system is in. On this view heat and information are two faces of one quantity.
How the link was found
Shannon noticed the shared formula in 1948; Edwin Jaynes rebuilt statistical mechanics on information theory in 1957; Rolf Landauer linked erasing information to heat in 1961.
Why it's proven
The formulas are mathematically identical. Landauer's predicted heat from erasing one bit was measured in 2012 by Antoine Bérut and colleagues.
Open questions and mysteries
Whether thermodynamic entropy is fundamentally about information (an observer's knowledge) or about physical states alone is still debated.
▸Claude Shannon created Information theory (Shannon)Proven
The relationship
Shannon asked how to send messages reliably over noisy channels such as telephone lines. He separated meaning from information, measuring only the uncertainty a message removes, in bits. He then proved the limits on compression and on error-free transmission rates, turning communication engineering into a precise mathematical science.
How the link was found
Shannon, at Bell Labs, in his 1948 paper 'A Mathematical Theory of Communication', published in two parts in the Bell System Technical Journal.
Why it's proven
His theorems are mathematical proofs. Turbo codes (1993) and later codes came within a fraction of a decibel of his limit in real systems.
Open questions and mysteries
Information theory deliberately ignores meaning; how meaning, or semantic information, fits in is still unclear.