---
title: "Asymptotic equipartition property"
chunk: 3/3
source: "https://en.wikipedia.org/wiki/Asymptotic_equipartition_property"
category: "reference"
tags: "science, encyclopedia"
date_saved: "2026-05-05T14:39:53.761478+00:00"
instance: "kb-cron"
---

== Category theory ==
A category theoretic definition for the equipartition property is given by Gromov. Given a sequence of Cartesian powers 
  
    
      
        
          P
          
            N
          
        
        =
        P
        ×
        ⋯
        ×
        P
      
    
    {\displaystyle P^{N}=P\times \cdots \times P}
  
 of a measure space P, this sequence admits an asymptotically equivalent sequence HN of homogeneous measure spaces (i.e. all sets have the same measure; all morphisms are invariant under the group of automorphisms, and thus factor as a morphism to the terminal object).
The above requires a definition of asymptotic equivalence. This is given in terms of a distance function, giving how much an injective correspondence differs from an isomorphism.  An injective correspondence 
  
    
      
        π
        :
        P
        →
        Q
      
    
    {\displaystyle \pi :P\to Q}
  
 is a partially defined map that is a bijection; that is, it is a bijection between a subset 
  
    
      
        
          P
          ′
        
        ⊂
        P
      
    
    {\displaystyle P'\subset P}
  
 and 
  
    
      
        
          Q
          ′
        
        ⊂
        Q
      
    
    {\displaystyle Q'\subset Q}
  
.  Then define

  
    
      
        
          |
        
        P
        −
        Q
        
          
            |
          
          
            π
          
        
        =
        
          |
        
        P
        ∖
        
          P
          ′
        
        
          |
        
        +
        
          |
        
        Q
        ∖
        
          Q
          ′
        
        
          |
        
        ,
      
    
    {\displaystyle |P-Q|_{\pi }=|P\setminus P'|+|Q\setminus Q'|,}
  

where |S| denotes the measure of a set S.  In what follows, the measure of P and Q are taken to be 1, so that the measure spaces are probability spaces. This distance 
  
    
      
        
          |
        
        P
        −
        Q
        
          
            |
          
          
            π
          
        
      
    
    {\displaystyle |P-Q|_{\pi }}
  
 is commonly known as the earth mover's distance or Wasserstein metric.
Similarly, define

  
    
      
        
          |
        
        log
        ⁡
        P
        :
        Q
        
          
            |
          
          
            π
          
        
        =
        
          
            
              
                sup
                
                  p
                  ∈
                  
                    P
                    ′
                  
                
              
              
                |
              
              log
              ⁡
              p
              −
              log
              ⁡
              π
              (
              p
              )
              
                |
              
            
            
              log
              ⁡
              min
              
                (
                
                  
                    |
                  
                  set
                  ⁡
                  (
                  
                    P
                    ′
                  
                  )
                  
                    |
                  
                  ,
                  
                    |
                  
                  set
                  ⁡
                  (
                  
                    Q
                    ′
                  
                  )
                  
                    |
                  
                
                )
              
            
          
        
        .
      
    
    {\displaystyle |\log P:Q|_{\pi }={\frac {\sup _{p\in P'}|\log p-\log \pi (p)|}{\log \min \left(|\operatorname {set} (P')|,|\operatorname {set} (Q')|\right)}}.}
  

with 
  
    
      
        
          |
        
        set
        ⁡
        (
        P
        )
        
          |
        
      
    
    {\displaystyle |\operatorname {set} (P)|}
  
 taken to be the counting measure on P.  Thus, this definition requires that P be a finite measure space.  Finally, let

  
    
      
        
          
            dist
          
          
            π
          
        
        (
        P
        ,
        Q
        )
        =
        
          |
        
        P
        −
        Q
        
          
            |
          
          
            π
          
        
        +
        
          |
        
        log
        ⁡
        P
        :
        Q
        
          
            |
          
          
            π
          
        
        .
      
    
    {\displaystyle {\text{dist}}_{\pi }(P,Q)=|P-Q|_{\pi }+|\log P:Q|_{\pi }.}
  

A sequence of injective correspondences 
  
    
      
        
          π
          
            N
          
        
        :
        
          P
          
            N
          
        
        →
        
          Q
          
            N
          
        
      
    
    {\displaystyle \pi _{N}:P_{N}\to Q_{N}}
  
 are then asymptotically equivalent when

  
    
      
        
          
            dist
          
          
            
              π
              
                N
              
            
          
        
        (
        
          P
          
            N
          
        
        ,
        
          Q
          
            N
          
        
        )
        →
        0
        
        
           as 
        
        
        N
        →
        ∞
        .
      
    
    {\displaystyle {\text{dist}}_{\pi _{N}}(P_{N},Q_{N})\to 0\quad {\text{ as }}\quad N\to \infty .}
  

Given a homogenous space sequence HN that is asymptotically equivalent to PN, the entropy H(P) of P may be taken as

  
    
      
        H
        (
        P
        )
        =
        
          lim
          
            N
            →
            ∞
          
        
        
          
            1
            N
          
        
        
          |
        
        set
        ⁡
        (
        
          H
          
            N
          
        
        )
        
          |
        
        .
      
    
    {\displaystyle H(P)=\lim _{N\to \infty }{\frac {1}{N}}|\operatorname {set} (H_{N})|.}
  

== See also ==
Cramér's theorem (large deviations)
Noisy-channel coding theorem
Shannon's source coding theorem

== Notes ==

== References ==

=== Journal articles ===
Claude E. Shannon. "A Mathematical Theory of Communication". Bell System Technical Journal, July/October 1948.
Sergio Verdú and Te Sun Han. "The Role of the Asymptotic Equipartition Property in Noiseless Source Coding".  IEEE Transactions on Information Theory, 43(3): 847–857, 1997. doi:10.1109/18.568696.

=== Textbooks ===
Cover, Thomas M.; Thomas, Joy A. (1991). Elements of Information Theory (first ed.). Hoboken, New Jersey: Wiley. ISBN 978-0-471-24195-9.
MacKay, David J.C. (2003). Information Theory, Inference, and Learning Algorithms. Cambridge University Press. ISBN 0-521-64298-1.