10 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Perceptrons (book) | 3/3 | https://en.wikipedia.org/wiki/Perceptrons_(book) | reference | science, encyclopedia | 2026-05-05T08:36:43.745538+00:00 | kb-cron |
=== Preliminary definitions === Let
R
{\textstyle R}
be a finite set. A predicate on
R
{\textstyle R}
is a boolean function that takes in a subset of
R
{\textstyle R}
and outputs either
0
{\textstyle 0}
or
1
{\textstyle 1}
. In particular, a perceptron unit is a predicate. A predicate
ψ
{\textstyle \psi }
has support
S
⊂
R
{\textstyle S\subset R}
, iff any
X
⊂
S
{\textstyle X\subset S}
, we have
ψ
(
X
)
=
ψ
(
X
∩
S
)
{\textstyle \psi (X)=\psi (X\cap S)}
. In words, it means that if we know how
ψ
{\textstyle \psi }
works on subsets of
S
{\textstyle S}
, then we know how it works on subsets of all of
R
{\textstyle R}
. A predicate can have many different supports. The support size of a predicate
ψ
{\textstyle \psi }
is the minimal number of elements necessary in its support. For example, the constant-0 and constant-1 functions both are supported on the empty set, thus they both have support size 0. A perceptron (the kind studied by Minsky and Papert) over
R
{\textstyle R}
is a function of form
θ
(
∑
i
a
i
ψ
i
)
{\displaystyle \theta \left(\sum _{i}a_{i}\psi _{i}\right)}
where
ψ
i
{\textstyle \psi _{i}}
are predicates, and
a
i
{\textstyle a_{i}}
are real numbers. If
Φ
{\textstyle \Phi }
is a set of predicates, then
L
(
Φ
)
{\textstyle L(\Phi )}
is the set of all perceptrons using just predicates in
Φ
{\textstyle \Phi }
. The order of a perceptron
θ
(
∑
i
a
i
ψ
i
)
{\textstyle \theta \left(\sum _{i}a_{i}\psi _{i}\right)}
is the maximal support size of its component predicates
{
ψ
i
}
i
{\textstyle \{\psi _{i}\}_{i}}
. The order of a boolean function on
R
{\textstyle R}
is the minimal order possible for a perceptron implementing the boolean function. A boolean function is conjunctively local iff its order does not increase to infinity as
|
R
|
{\displaystyle |R|}
increases to infinity. The mask of
A
⊂
R
{\textstyle A\subset R}
is the predicate
1
A
{\textstyle 1_{A}}
defined by
1
A
(
X
)
=
{
1
if
A
⊂
X
,
0
else.
{\displaystyle 1_{A}(X)={\begin{cases}1&{\text{ if }}A\subset X,\\0&{\text{ else.}}\end{cases}}}
=== Main theorems ===
Let
S
R
{\textstyle S_{R}}
be the permutation group on the elements of
R
{\textstyle R}
, and
G
{\textstyle G}
be a subgroup of
S
R
{\textstyle S_{R}}
. We say that a predicate
ψ
{\textstyle \psi }
is
G
{\textstyle G}
-invariant iff
ψ
∘
g
=
ψ
{\textstyle \psi \circ g=\psi }
for any
g
∈
G
{\textstyle g\in G}
. That is, any
X
⊂
R
{\textstyle X\subset R}
, we have
ψ
(
X
)
=
ψ
(
g
(
X
)
)
{\textstyle \psi (X)=\psi (g(X))}
. For example, the parity function is
S
R
{\textstyle S_{R}}
-invariant, since any permutation of the set preserves the size, and thus parity, of any of its subsets.
Proof: omitted. Proof sketch: By reducing the parity function to the connectness function, using circuit gadgets. It is in a similar style as the one showing that Sokoban is NP-hard.
== Reception and legacy == Perceptrons received a number of positive reviews in the years after publication. In 1969, Stanford professor Michael A. Arbib stated, "[t]his book has been widely hailed as an exciting new chapter in the theory of pattern recognition." Earlier that year, CMU professor Allen Newell composed a review of the book for Science, opening the piece by declaring "[t]his is a great book." On the other hand, H.D. Block expressed concern at the authors' narrow definition of perceptrons. He argued that they "study a severely limited class of machines from a viewpoint quite alien to Rosenblatt's", and thus the title of the book was "seriously misleading". Contemporary neural net researchers shared some of these objections: Bernard Widrow complained that the authors had defined perceptrons too narrowly, but also said that Minsky and Papert's proofs were "pretty much irrelevant", coming a full decade after Rosenblatt's perceptron. Perceptrons is often thought to have caused a decline in neural net research in the 1970s and early 1980s. During this period, neural net researchers continued smaller projects outside the mainstream, while symbolic AI research saw explosive growth. With the revival of connectionism in the late 80s, PDP researcher David Rumelhart and his colleagues returned to Perceptrons. In a 1986 report, they claimed to have overcome the problems presented by Minsky and Papert, and that "their pessimism about learning in multilayer machines was misplaced".
== Analysis of the controversy == It is most instructive to learn what Minsky and Papert themselves said in the 1970s as to what were the broader implications of their book. On his website Harvey Cohen, a researcher at the MIT AI Labs 1974+, quotes Minsky and Papert in the 1971 Report of Project MAC, directed at funding agencies, on "Gamba networks": "Virtually nothing is known about the computational capabilities of this latter kind of machine. We believe that it can do little more than can a low order perceptron." In the preceding page Minsky and Papert make clear that "Gamba networks" are networks with hidden layers. Minsky has compared the book to the fictional book Necronomicon in H. P. Lovecraft's tales, a book known to many, but read only by a few. The authors talk in the expanded edition about the criticism of the book that started in the 1980s, with a new wave of research symbolized by the PDP book. How Perceptrons was explored first by one group of scientists to drive research in AI in one direction, and then later by a new group in another direction, has been the subject of a sociological study of scientific development.
== Notes ==
== References == McCorduck, Pamela (2004), Machines Who Think (2nd ed.), Natick, Massachusetts: A. K. Peters, ISBN 1-5688-1205-1, pp. 104−107 Crevier, Daniel (1993). AI: The Tumultuous Search for Artificial Intelligence. New York, NY: BasicBooks. ISBN 0-465-02997-3., pp. 102−105 Russell, Stuart J.; Norvig, Peter (2003), Artificial Intelligence: A Modern Approach (2nd ed.), Upper Saddle River, New Jersey: Prentice Hall, ISBN 0-13-790395-2 p. 22 Marvin Minsky and Seymour Papert, 1972 (2nd edition with corrections, first edition 1969) Perceptrons: An Introduction to Computational Geometry, The MIT Press, Cambridge MA, ISBN 0-262-63022-2. Olazaran, Mikel (1996). "A Sociological Study of the Official History of the Perceptrons Controversy". Social Studies of Science. 26 (3): 611–659. doi:10.1177/030631296026003005. JSTOR 285702. S2CID 16786738. Olazaran, Mikel (1993-01-01), "A Sociological History of the Neural Network Controversy", in Yovits, Marshall C. (ed.), Advances in Computers Volume 37, vol. 37, Elsevier, pp. 335–425, doi:10.1016/S0065-2458(08)60408-8, ISBN 9780120121373, retrieved 2023-10-31