---
title: "History of information theory"
chunk: 2/2
source: "https://en.wikipedia.org/wiki/History_of_information_theory"
category: "reference"
tags: "science, encyclopedia"
date_saved: "2026-05-05T03:59:49.990118+00:00"
instance: "kb-cron"
---

This (Gibbs) entropy, from statistical mechanics, can be found to directly correspond to the Clausius's classical thermodynamic definition.
Shannon himself was apparently not particularly aware of the close similarity between his new measure and earlier work in thermodynamics, but John von Neumann was. It is said that, when Shannon was deciding what to call his new measure and fearing the term 'information' was already over-used, von Neumann told him firmly: "You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage."
(Connections between information-theoretic entropy and thermodynamic entropy, including the important contributions by Rolf Landauer in the 1960s, are explored further in the article Entropy in thermodynamics and information theory).

== Development since 1948 ==
The publication of Shannon's 1948 paper, "A Mathematical Theory of Communication", in the Bell System Technical Journal was the founding of information theory as we know it today. Many developments and applications of the theory have taken place since then, which have made many modern devices for data communication and storage such as CD-ROMs and mobile phones possible.
Notable later developments are listed in a timeline of information theory, including:

The 1951, invention of Huffman encoding, a method of finding optimal prefix codes for lossless data compression.
Irving S. Reed and David E. Muller proposing Reed–Muller codes in 1954.
The 1960 proposal of Reed–Solomon codes.
In 1966, Fumitada Itakura (Nagoya University) and Shuzo Saito (Nippon Telegraph and Telephone) develop linear predictive coding (LPC), a form of speech coding.
In 1968, Elwyn Berlekamp invents the Berlekamp–Massey algorithm; its application to decoding BCH and Reed–Solomon codes is pointed out by James L. Massey the following year.
In 1972, Nasir Ahmed proposes the discrete cosine transform (DCT). It later becomes the most widely used lossy compression algorithm, and the basis for digital media compression standards from 1988 onwards, including H.26x (since H.261) and MPEG video coding standards, JPEG image compression, MP3 audio compression, and Advanced Audio Coding (AAC).
In 1976, Gottfried Ungerboeck gives the first paper on trellis modulation; a more detailed exposition in 1982 leads to a raising of analogue modem POTS speeds from 9.6 kbit/s to 33.6 kbit/s
In 1977, Abraham Lempel and Jacob Ziv develop Lempel–Ziv compression (LZ77)
In the early 1980s, Renuka P. Jindal at Bell Labs improves the noise performance of metal–oxide–semiconductor (MOS) devices, resolving issues that limited their receiver sensitivity and data rates. This leads to the wide adoption of MOS technology in laser lightwave systems and wireless terminal applications, enabling Edholm's law.
In 1989, Phil Katz publishes the .zip format including DEFLATE (LZ77 + Huffman coding); later to become the most widely used archive container.
In 1995, Benjamin Schumacher coins the term qubit and proves the quantum noiseless coding theorem.

== See also ==
Timeline of information theory
Claude Shannon
Ralph Hartley
H-theorem

== References ==

== Further reading ==
Gleick, James (2011). The Information: A History, a Theory, a Flood (1st ed.). New York: Pantheon Books. ISBN 978-1-4000-9623-7. OCLC 607975727.