7.4 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Asymptotic equipartition property | 3/3 | https://en.wikipedia.org/wiki/Asymptotic_equipartition_property | reference | science, encyclopedia | 2026-05-05T14:39:53.761478+00:00 | kb-cron |
== Category theory == A category theoretic definition for the equipartition property is given by Gromov. Given a sequence of Cartesian powers
P
N
=
P
×
⋯
×
P
{\displaystyle P^{N}=P\times \cdots \times P}
of a measure space P, this sequence admits an asymptotically equivalent sequence HN of homogeneous measure spaces (i.e. all sets have the same measure; all morphisms are invariant under the group of automorphisms, and thus factor as a morphism to the terminal object). The above requires a definition of asymptotic equivalence. This is given in terms of a distance function, giving how much an injective correspondence differs from an isomorphism. An injective correspondence
π
:
P
→
Q
{\displaystyle \pi :P\to Q}
is a partially defined map that is a bijection; that is, it is a bijection between a subset
P
′
⊂
P
{\displaystyle P'\subset P}
and
Q
′
⊂
Q
{\displaystyle Q'\subset Q}
. Then define
|
P
−
Q
|
π
=
|
P
∖
P
′
|
+
|
Q
∖
Q
′
|
,
{\displaystyle |P-Q|_{\pi }=|P\setminus P'|+|Q\setminus Q'|,}
where |S| denotes the measure of a set S. In what follows, the measure of P and Q are taken to be 1, so that the measure spaces are probability spaces. This distance
|
P
−
Q
|
π
{\displaystyle |P-Q|_{\pi }}
is commonly known as the earth mover's distance or Wasserstein metric. Similarly, define
|
log
P
:
Q
|
π
=
sup
p
∈
P
′
|
log
p
−
log
π
(
p
)
|
log
min
(
|
set
(
P
′
)
|
,
|
set
(
Q
′
)
|
)
.
{\displaystyle |\log P:Q|_{\pi }={\frac {\sup _{p\in P'}|\log p-\log \pi (p)|}{\log \min \left(|\operatorname {set} (P')|,|\operatorname {set} (Q')|\right)}}.}
with
|
set
(
P
)
|
{\displaystyle |\operatorname {set} (P)|}
taken to be the counting measure on P. Thus, this definition requires that P be a finite measure space. Finally, let
dist
π
(
P
,
Q
)
=
|
P
−
Q
|
π
+
|
log
P
:
Q
|
π
.
{\displaystyle {\text{dist}}_{\pi }(P,Q)=|P-Q|_{\pi }+|\log P:Q|_{\pi }.}
A sequence of injective correspondences
π
N
:
P
N
→
Q
N
{\displaystyle \pi _{N}:P_{N}\to Q_{N}}
are then asymptotically equivalent when
dist
π
N
(
P
N
,
Q
N
)
→
0
as
N
→
∞
.
{\displaystyle {\text{dist}}_{\pi _{N}}(P_{N},Q_{N})\to 0\quad {\text{ as }}\quad N\to \infty .}
Given a homogenous space sequence HN that is asymptotically equivalent to PN, the entropy H(P) of P may be taken as
H
(
P
)
=
lim
N
→
∞
1
N
|
set
(
H
N
)
|
.
{\displaystyle H(P)=\lim _{N\to \infty }{\frac {1}{N}}|\operatorname {set} (H_{N})|.}
== See also == Cramér's theorem (large deviations) Noisy-channel coding theorem Shannon's source coding theorem
== Notes ==
== References ==
=== Journal articles === Claude E. Shannon. "A Mathematical Theory of Communication". Bell System Technical Journal, July/October 1948. Sergio Verdú and Te Sun Han. "The Role of the Asymptotic Equipartition Property in Noiseless Source Coding". IEEE Transactions on Information Theory, 43(3): 847–857, 1997. doi:10.1109/18.568696.
=== Textbooks === Cover, Thomas M.; Thomas, Joy A. (1991). Elements of Information Theory (first ed.). Hoboken, New Jersey: Wiley. ISBN 978-0-471-24195-9. MacKay, David J.C. (2003). Information Theory, Inference, and Learning Algorithms. Cambridge University Press. ISBN 0-521-64298-1.