6.4 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Claude Shannon | 4/7 | https://en.wikipedia.org/wiki/Claude_Shannon | reference | science, encyclopedia | 2026-05-05T14:43:48.523076+00:00 | kb-cron |
=== Information theory === In 1948, the promised memorandum appeared as "A Mathematical Theory of Communication", an article in two parts in the July and October issues of the Bell System Technical Journal. This work focuses on the problem of how best to encode the message a sender wants to transmit. Shannon developed information entropy as a measure of the information content in a message, which is a measure of uncertainty reduced by the message. In so doing, he essentially invented the field of information theory. The book The Mathematical Theory of Communication reprints Shannon's 1948 article and Warren Weaver's popularization of it, which is accessible to the non-specialist. Weaver pointed out that the word "information" in communication theory is not related to what you do say, but to what you could say. That is, information is a measure of one's freedom of choice when one selects a message. Shannon's concepts were also popularized, subject to his own proofreading, in John Robinson Pierce's Symbols, Signals, and Noise. Information theory's fundamental contribution to natural language processing and computational linguistics was further established in 1951, in his article "Prediction and Entropy of Printed English", showing upper and lower bounds of entropy on the statistics of English – giving a statistical foundation to language analysis. In addition, he proved that treating space as the 27th letter of the alphabet actually lowers uncertainty in written language, providing a clear quantifiable link between cultural practice and probabilistic cognition. Another notable paper published in 1949 is "Communication Theory of Secrecy Systems", a declassified version of his wartime work on the mathematical theory of cryptography, in which he proved that all theoretically unbreakable cyphers must have the same requirements as the one-time pad. He is credited with the introduction of sampling theorem, which he had derived as early as 1940, and which is concerned with representing a continuous-time signal from a (uniform) discrete set of samples. This theory was essential in enabling telecommunications to move from analog to digital transmissions systems in the 1960s and later. He further wrote a paper in 1956 regarding coding for a noisy channel, which also became a classic paper in the field of information theory. However, also in 1956 he wrote a one-page editorial for the "IRE Transactions on Information Theory" entitled "The Bandwagon" which he began by observing: "Information theory has, in the last few years, become something of a scientific bandwagon" and which he concluded by warning: "Only by maintaining a thoroughly scientific attitude can we achieve real progress in communication theory and consolidate our present position." Claude Shannon's influence has been immense in the field, for example, in a 1973 collection of the key papers in the field of information theory, he was author or coauthor of 12 of the 49 papers cited, while no one else appeared more than three times. Even beyond his original paper in 1948, he is still regarded as the most important post-1948 contributor to the theory. In May 1951, Mervin Kelly received a request from the director of the CIA, general Walter Bedell Smith, regarding Shannon and the need for him, as Shannon was regarded as, based on "the best authority", the "most eminently qualified scientist in the particular field concerned". As a result of the request, Shannon became part of the CIA's Special Cryptologic Advisory Group or SCAG. In his time at Bell Labs, he also co-developed pulse-code modulation alongside Bernard M. Oliver, and John R. Pierce.
=== Artificial intelligence ===
==== Theseus, the mouse ==== In 1950, Shannon designed and built, with the help of his wife, Betty, a learning machine named Theseus. It consisted of a maze on a surface, through which a mechanical mouse could move. Below the surface were sensors (an electromechanical relay circuit) that followed the path of a mechanical mouse through the maze. The mouse was designed to search through the corridors until it found the target. Having travelled through the maze, the mouse could then be placed anywhere it had been before, and because of its prior experience it could go directly to the target. If placed in unfamiliar territory, it was programmed to search until it reached a known location and then it would proceed to the target, adding the new knowledge to its memory and learning new behavior. After much trial and error, this device would learn the shortest path through the maze, and direct the mechanical mouse through the maze. The pattern of the maze could be changed at will, by rearranging movable partitions. Shannon's mouse appears to have been the first artificial learning device of its kind. Mazin Gilbert stated that Theseus "inspired the whole field of AI. This random trial and error is the foundation of artificial intelligence."
==== Other artificial intelligence work ==== Shannon wrote multiple influential papers on artificial intelligence, such as his 1950 paper titled "Programming a Computer for Playing Chess", and his 1953 paper titled "Computers and Automata". Alongside John McCarthy, he co-edited a book titled Automata Studies, which was published in 1956. The categories in the articles within the volume were influenced by Shannon's own subject headings in his 1953 paper. Shannon shared McCarthy's goal of creating a science of intelligent machines, but also held a broader view of viable approaches in automata studies, such as neural nets, Turing machines, cybernetic mechanisms, and symbolic processing by computer. Shannon co-organized and participated in the Dartmouth workshop of 1956, alongside John McCarthy, Marvin Minsky and Nathaniel Rochester, and which is considered the founding event of the field of artificial intelligence.
=== Teaching at MIT === In 1956 Shannon joined the MIT faculty, holding an endowed chair. He worked in the Research Laboratory of Electronics (RLE). He continued to serve on the MIT faculty until 1978.
=== Later life === Shannon developed Alzheimer's disease and spent the last few years of his life in a nursing home; he died in 2001, survived by his wife, a son and daughter, and two granddaughters.
=== Hobbies and inventions ===