View source for Timeline of information theory

{{short description|None}}

A timeline of events related to [[File:Red pog.svg|12px]]&nbsp;[[information theory]], [[File:Orange pog.svg|12px]]&nbsp;[[quantum information theory]] and [[statistical physics]], [[File:Green pog.svg|12px]]&nbsp;[[data compression]], [[File:Purple pog.svg|12px]]&nbsp;[[error correcting code]]s and related subjects.

* 1872 [[File:Orange pog.svg|12px]] – [[Ludwig Boltzmann]] presents his [[H-theorem]], and with it the formula Σ''p''<sub>i</sub> log ''p''<sub>i</sub> for the entropy of a single gas particle
* 1878 [[File:Orange pog.svg|12px]] – [[J. Willard Gibbs]] defines the [[Gibbs entropy]]: the probabilities in the entropy formula are now taken as probabilities of the state of the ''whole'' system
* 1924 [[File:Red pog.svg|12px]] – [[Harry Nyquist]] discusses quantifying {{Tooltip|"intelligence"|"By the speed of transmission of intelligence is meant the number of characters, representing different letters, figures, etc., which can be transmitted in a given length of time assuming that the circuit transmits a given number of signal elements per unit time."}} and the speed at which it can be transmitted by a communication system
* 1927 [[File:Orange pog.svg|12px]] – [[John von Neumann]] defines the [[von Neumann entropy]], extending the Gibbs entropy to quantum mechanics
* 1928 [[File:Red pog.svg|12px]] – [[Ralph Hartley]] introduces [[Hartley information]] as the logarithm of the number of possible messages, with information being communicated when the receiver can distinguish one sequence of symbols from any other (regardless of any associated meaning)
* 1929 [[File:Orange pog.svg|12px]] – [[Leó Szilárd]] analyses [[Maxwell's demon]], showing how a [[Szilard engine]] can sometimes transform information into the extraction of useful work
* 1940 [[File:Red pog.svg|12px]] – [[Alan Turing]] introduces the [[deciban]] as a measure of information inferred about the German [[Enigma machine]] cypher settings by the [[Banburismus]] process
* 1944 [[File:Red pog.svg|12px]] – [[Claude Shannon]]'s theory of information is substantially complete
* 1947 [[File:Purple pog.svg|12px]] – [[Richard W. Hamming]] invents [[Hamming code]]s for error detection and correction (to protect patent rights, the result is not published until 1950)
* 1948 [[File:Red pog.svg|12px]] – [[Claude E. Shannon]] publishes ''[[A Mathematical Theory of Communication]]''
* 1949 [[File:Red pog.svg|12px]] – [[Claude E. Shannon]] publishes ''Communication in the Presence of Noise'' – [[Nyquist–Shannon sampling theorem]] and [[Shannon–Hartley law]]
* 1949 [[File:Red pog.svg|12px]] – [[Claude E. Shannon]]'s ''[[Communication Theory of Secrecy Systems]]'' is declassified
* 1949 [[File:Green pog.svg|12px]] –  [[Robert M. Fano]] publishes ''Transmission of Information''. M.I.T. Press, Cambridge, Massachusetts – [[Shannon–Fano coding]]
* 1949 [[File:Green pog.svg|12px]] – Leon G. Kraft discovers [[Kraft's inequality]], which shows the limits of [[prefix codes]]
* 1949 [[File:Purple pog.svg|12px]] – [[Marcel J. E. Golay]] introduces [[Golay code (disambiguation)|Golay code]]s for [[forward error correction]]
* 1951 [[File:Red pog.svg|12px]] – [[Solomon Kullback]] and [[Richard Leibler]] introduce the [[Kullback–Leibler divergence]]
* 1951 [[File:Green pog.svg|12px]] – [[David A. Huffman]] invents [[Huffman encoding]], a method of finding optimal [[prefix codes]] for [[lossless]] [[data compression]]
* 1953 [[File:Green pog.svg|12px]] – [[August Albert Sardinas]] and George W. Patterson devise the [[Sardinas–Patterson algorithm]], a procedure to decide whether a given [[variable-length code]] is uniquely decodable
* 1954 [[File:Purple pog.svg|12px]] – [[Irving S. Reed]] and [[David E. Muller]] propose [[Reed–Muller code]]s
* 1955 [[File:Purple pog.svg|12px]] – [[Peter Elias]] introduces [[convolutional code]]s
* 1957 [[File:Purple pog.svg|12px]] – [[Eugene Prange]] first discusses [[cyclic code]]s
* 1959 [[File:Purple pog.svg|12px]] – [[Alexis Hocquenghem]], and independently the next year [[Raj Chandra Bose]] and [[Dwijendra Kumar Ray-Chaudhuri]], discover [[BCH code]]s
* 1960 [[File:Purple pog.svg|12px]] – [[Irving S. Reed]] and [[Gustave Solomon]] propose [[Reed–Solomon code]]s
* 1962 [[File:Purple pog.svg|12px]] – [[Robert G. Gallager]] proposes [[low-density parity-check code]]s; they are unused for 30 years due to technical limitations
* 1965 [[File:Purple pog.svg|12px]] – [[Dave Forney]] discusses [[concatenated code]]s
* 1966 [[File:Green pog.svg|12px]] – [[Fumitada Itakura]] ([[Nagoya University]]) and Shuzo Saito ([[Nippon Telegraph and Telephone]]) develop [[linear predictive coding]] (LPC), a form of [[speech coding]]<ref>{{cite journal |last1=Gray |first1=Robert M. |title=A History of Realtime Digital Speech on Packet Networks: Part II of Linear Predictive Coding and the Internet Protocol |journal=Found. Trends Signal Process. |date=2010 |volume=3 |issue=4 |pages=203–303 |doi=10.1561/2000000036 |url=https://ee.stanford.edu/~gray/lpcip.pdf |issn=1932-8346|doi-access=free }}</ref>
* 1967 [[File:Purple pog.svg|12px]] – [[Andrew Viterbi]] reveals the [[Viterbi algorithm]], making decoding of convolutional codes practicable
* 1968 [[File:Purple pog.svg|12px]] – [[Elwyn Berlekamp]] invents the [[Berlekamp–Massey algorithm]]; its application to decoding BCH and Reed–Solomon codes is pointed out by [[James Lee Massey|James L. Massey]] the following year
* 1968 [[File:Red pog.svg|12px]] – [[Chris Wallace (computer scientist)|Chris Wallace]] and David M. Boulton publish the first of many papers on [[Minimum Message Length]] ([[Minimum Message Length|MML]]) statistical and inductive inference
* 1970 [[File:Purple pog.svg|12px]] – [[Valerii Denisovich Goppa]] introduces [[Binary Goppa code|Goppa codes]]
* 1972 [[File:Purple pog.svg|12px]] – [[Jørn Justesen]] proposes [[Justesen code]]s, an improvement of Reed–Solomon codes
* 1972 [[File:Green pog.svg|12px]] – [[N. Ahmed|Nasir Ahmed]] proposes the [[discrete cosine transform]] (DCT), which he develops with T. Natarajan and [[K. R. Rao]] in 1973;<ref name="Ahmed">{{cite web|url=https://www.scribd.com/doc/52879771/DCT-History|title=How I Came Up With the Discrete Cosine Transform|author=Nasir Ahmed|publisher = Digital Signal Processing, Vol. 1, Iss. 1, 1991, pp. 4-5|author-link=N. Ahmed}}</ref> the DCT later became the most widely used [[lossy compression]] algorithm, the basis for multimedia formats such as [[JPEG]], [[MPEG]] and [[MP3]]
* 1973 [[File:Red pog.svg|12px]] – [[David Slepian]] and [[Jack Keil Wolf|Jack Wolf]] discover and prove the [[Slepian–Wolf coding]] limits for distributed [[source coding]]<ref>{{cite journal |last1=Slepian |first1=David S. |author1link=David Slepian |last2=Wolf |first2=Jack K. |author2link=Jack Wolf |date=July 1973 |title=Noiseless coding of correlated information sources |publisher=[[IEEE]] |journal=[[IEEE Transactions on Information Theory]] |volume=19 |issue=4 |issn=0018-9448 |doi=10.1109/TIT.1973.1055037 |pages=471–480}}</ref>
* 1976 [[File:Purple pog.svg|12px]] – [[Gottfried Ungerboeck]] gives the first paper on [[trellis modulation]]; a more detailed exposition in 1982 leads to a raising of analogue modem [[Plain old telephone service|POTS]] speeds from 9.6&nbsp;kbit/s to 33.6&nbsp;kbit/s
* 1976 [[File:Green pog.svg|12px]] – Richard Pasco and [[Jorma J. Rissanen]] develop effective [[arithmetic coding]] techniques 
* 1977 [[File:Green pog.svg|12px]] – [[Abraham Lempel]] and [[Jacob Ziv]] develop Lempel–Ziv compression ([[LZ77]])
* 1982 [[File:Purple pog.svg|12px]] – [[Valerii Denisovich Goppa]] introduces [[algebraic geometry code]]s
* 1989 [[File:Green pog.svg|12px]] – [[Phil Katz]] publishes [[ZIP (file format)|the <code>.zip</code> format]] including [[DEFLATE]] (LZ77 + Huffman coding); later to become the most widely used archive container
* 1993 [[File:Purple pog.svg|12px]] – [[Claude Berrou]], [[Alain Glavieux]] and [[Punya Thitimajshima]] introduce [[Turbo code]]s
* 1994 [[File:Green pog.svg|12px]] – [[Michael Burrows (computer scientist)|Michael Burrows]] and [[David Wheeler (computer scientist)|David Wheeler]] publish the [[Burrows–Wheeler transform]], later to find use in [[bzip2]]
* 1995 [[File:Orange pog.svg|12px]] – [[Benjamin Schumacher]] coins the term [[qubit]] and proves the quantum noiseless coding theorem
* 2003 [[File:Purple pog.svg|12px]] – [[David J. C. MacKay]] shows the connection between information theory, inference and machine learning in his book.
* 2006 [[File:Green pog.svg|12px]] – [[Jarosław Duda (computer scientist)|Jarosław Duda]] introduces first [[Asymmetric numeral systems]] entropy coding: since 2014 popular replacement of [[Huffman coding|Huffman]] and [[arithmetic coding]] in compressors like [[Zstandard|Facebook Zstandard]], [[LZFSE|Apple LZFSE]], [[CRAM (file format)|CRAM]] or [[JPEG XL]] 
* 2008 [[File:Purple pog.svg|12px]] – [[Erdal Arıkan]] introduces [[Polar code (coding theory)|polar codes]], the first practical construction of codes that achieves capacity for a wide array of channels

==References==
{{Reflist|1}}
{{History of mathematics}}

{{DEFAULTSORT:Timeline Of Information Theory}}
[[Category:Science timelines|Information theory]]
[[Category:Information theory]]
[[Category:Thermodynamics]]