kb/data/en.wikipedia.org/wiki/Design_effect-7.md

---
title: "Design effect"
chunk: 8/12
source: "https://en.wikipedia.org/wiki/Design_effect"
category: "reference"
tags: "science, encyclopedia"
date_saved: "2026-05-05T09:49:56.844427+00:00"
instance: "kb-cron"
---

  
    
      
        
          Deff
        
        =
        
          
            
              n
              
                ∑
                
                  i
                  =
                  1
                
                
                  n
                
              
              
                w
                
                  i
                
                
                  2
                
              
            
            
              (
              
                ∑
                
                  i
                  =
                  1
                
                
                  n
                
              
              
                w
                
                  i
                
              
              
                )
                
                  2
                
              
            
          
        
        =
        
          
            
              
                
                  1
                  n
                
              
              
                ∑
                
                  i
                  =
                  1
                
                
                  n
                
              
              
                w
                
                  i
                
                
                  2
                
              
            
            
              
                (
                
                  
                    
                      1
                      n
                    
                  
                  
                    ∑
                    
                      i
                      =
                      1
                    
                    
                      n
                    
                  
                  
                    w
                    
                      i
                    
                  
                
                )
              
              
                2
              
            
          
        
        =
        
          
            
              
                w
                
                  2
                
              
              ¯
            
            
              
                
                  w
                  ¯
                
              
              
                2
              
            
          
        
      
    
    {\displaystyle {\text{Deff}}={\frac {n\sum _{i=1}^{n}w_{i}^{2}}{(\sum _{i=1}^{n}w_{i})^{2}}}={\frac {{\frac {1}{n}}\sum _{i=1}^{n}w_{i}^{2}}{\left({\frac {1}{n}}\sum _{i=1}^{n}w_{i}\right)^{2}}}={\frac {\overline {w^{2}}}{{\overline {w}}^{2}}}}
  

This version of the formula is valid when one stratum had several observations taken from it (i.e., each having the same weight), or when there are just many strata were each one had one observation taken from it, but several of them had the same probability of selection. While the interpretation is slightly different, the calculation of the two scenarios comes out to be the same.
When using Kish's design effect for unequal weights, you may use the following simplified formula for "Kish's Effective Sample Size"

  
    
      
        
          n
          
            eff
          
        
        =
        
          
            
              (
              
                ∑
                
                  i
                  =
                  1
                
                
                  n
                
              
              
                w
                
                  i
                
              
              
                )
                
                  2
                
              
            
            
              
                ∑
                
                  i
                  =
                  1
                
                
                  n
                
              
              
                w
                
                  i
                
                
                  2
                
              
            
          
        
      
    
    {\displaystyle n_{\text{eff}}={\frac {(\sum _{i=1}^{n}w_{i})^{2}}{\sum _{i=1}^{n}w_{i}^{2}}}}
  

===== Assumptions and proofs =====
The above formula, by Kish, gives the increase in the variance of the weighted mean based on "haphazard" weights. This can also be written as the following formula where y are observations selected using unequal selection probabilities (with no within-cluster correlation, and no relationship to the expectancy or variance of the outcome measurement), and y' are the observations we would have had if we got them from a simple random sample:

  
    
      
        
          
            Deff
          
          
            Kish
          
        
        =
        
          
            
              
                var
              
              
                (
                
                  
                    
                      
                        y
                        ¯
                      
                    
                  
                  
                    w
                  
                
                )
              
            
            
              
                var
              
              
                (
                
                  
                    
                      
                        y
                        ¯
                      
                    
                  
                  ′
                
                )
              
            
          
        
        =
        
          
            
              
                var
              
              
                (
                
                  
                    
                      
                        ∑
                        
                          i
                          =
                          1
                        
                        
                          n
                        
                      
                      
                        w
                        
                          i
                        
                      
                      
                        y
                        
                          i
                        
                      
                    
                    
                      
                        ∑
                        
                          i
                          =
                          1
                        
                        
                          n
                        
                      
                      
                        w
                        
                          i
                        
                      
                    
                  
                
                )
              
            
            
              
                var
              
              
                (
                
                  
                    
                      
                        ∑
                        
                          i
                          =
                          1
                        
                        
                          n
                        
                      
                      
                        y
                        
                          i
                        
                        ′
                      
                    
                    n
                  
                
                )
              
            
          
        
      
    
    {\displaystyle {\text{Deff}}_{\text{Kish}}={\frac {{\text{var}}\left({\bar {y}}_{w}\right)}{{\text{var}}\left({\bar {y}}'\right)}}={\frac {{\text{var}}\left({\frac {\sum \limits _{i=1}^{n}w_{i}y_{i}}{\sum \limits _{i=1}^{n}w_{i}}}\right)}{{\text{var}}\left({\frac {\sum \limits _{i=1}^{n}y_{i}'}{n}}\right)}}}
  

It can be shown that the ratio of variances formula can be reduced to Kish's formula by using a model based perspective. In it, Kish's formula will hold when all n observations (
  
    
      
        
          y
          
            1
          
        
        ,
        .
        .
        .
        ,
        
          y
          
            n
          
        
      
    
    {\displaystyle y_{1},...,y_{n}}
  
) are (at least approximately) uncorrelated (
  
    
      
        ∀
        (
        i
        ≠
        j
        )
        :
        
          cor
        
        (
        
          y
          
            i
          
        
        ,
        
          y
          
            j
          
        
        )
        =
        0
      
    
    {\displaystyle \forall (i\neq j):{\text{cor}}(y_{i},y_{j})=0}
  
), with the same variance (
  
    
      
        
          σ
          
            2
          
        
      
    
    {\displaystyle \sigma ^{2}}
  
) in the response variable of interest (y). It will also be required to assume the weights themselves are not a random variable but rather some known constants (e.g. the inverse of probability of selection, for some pre-determined and known sampling design).

The conditions on y are trivially held if the y observations are IID with the same expectation and variance. In such cases, 
  
    
      
        y
        =
        
          y
          ′
        
      
    
    {\displaystyle y=y'}
  
, and we can estimate 
  
    
      
        v
        a
        r
        
          (
          
            
              
                
                  y
                  ¯
                
              
            
            
              w
            
          
          )
        
      
    
    {\displaystyle var\left({\bar {y}}_{w}\right)}
  
 by using 
  
    
      
        
          
            
              
                var
              
              
                (
                
                  
                    
                      
                        y
                        ¯
                      
                    
                  
                  
                    w
                  
                
                )
              
            
            ¯
          
        
        =
        
          
            
              
                var
              
              
                (
                
                  
                    
                      y
                      ¯
                    
                  
                
                )
              
            
            ¯
          
        
        ×
        
          Deff
        
      
    
    {\displaystyle {\overline {{\text{var}}\left({\bar {y}}_{w}\right)}}={\overline {{\text{var}}\left({\bar {y}}\right)}}\times {\text{Deff}}}
  
. If the y's are not all with the same expectations then we cannot use the estimated variance for calculation, since that estimation assumes that all 
  
    
      
        
          y
          
            i
          
        
      
    
    {\displaystyle y_{i}}
  
s have the same expectation. Specifically, if there is a correlation between the weights and the outcome variable y, then it means that the expectation of y is not the same for all observations (but rather, dependent on the specific weight value for each observation). In such a case, while the design effect formula might still be correct (if the other conditions are met), it would require a different estimator for the variance of the weighted mean. For example, it might be better to use a weighted variance estimator.
If different 
  
    
      
        
          y
          
            i
          
        
      
    
    {\displaystyle y_{i}}
  
s values have different variances, then while the weighted variance could capture the correct population-level variance, Kish's formula for the design effect may no longer be true.
A similar issue happens if there is some correlation structure in the samples (such as when using cluster sampling).

===== Relation to the coefficient of variation =====
Notice that Kish's definition of the design effect is closely tied to the coefficient of variation (Kish also calls it relvariance or relvar for short) of the weights (when using the uncorrected (population level) sample standard deviation for estimation). This has several notations in the literature:

  
    
      
        
          Deff
        
        =
        1
        +
        L
        =
        1
        +
        
          
            
              C
              
                V
              
            
          
          
            2
          
        
        =
        1
        +
        
          relvar
        
        (
        w
        )
        =
        1
        +
        
          
            
              V
              (
              w
              )
            
            
              
                
                  
                    w
                    ¯
                  
                
              
              
                2
              
            
          
        
      
    
    {\displaystyle {\text{Deff}}=1+L=1+{C_{V}}^{2}=1+{\text{relvar}}(w)=1+{\frac {V(w)}{{\bar {w}}^{2}}}}
  
.
Where 
  
    
      
        V
        (
        w
        )
        =
        
          
            
              ∑
              (
              
                w
                
                  i
                
              
              −
              
                
                  
                    w
                    ¯
                  
                
              
              
                )
                
                  2
                
              
            
            n
          
        
      
    
    {\displaystyle V(w)={\frac {\sum (w_{i}-{\bar {w}})^{2}}{n}}}
  
 is the population variance of 
  
    
      
        w
      
    
    {\displaystyle w}
  
, and 
  
    
      
        
          
            
              w
              ¯
            
          
        
        =
        
          
            
              ∑
              
                w
                
                  i
                
              
            
            n
          
        
      
    
    {\displaystyle {\bar {w}}={\frac {\sum w_{i}}{n}}}
  
 is the mean. When the weights are normalized to sample size (so that their sum is equal to n and their mean is equal to 1), then 
  
    
      
        
          
            
              C
              
                V
              
            
          
          
            2
          
        
        =
        V
        (
        w
        )
      
    
    {\displaystyle {C_{V}}^{2}=V(w)}
  
 and the formula reduces to 
  
    
      
        
          Deff
        
        =
        1
        +
        V
        (
        w
        )
      
    
    {\displaystyle {\text{Deff}}=1+V(w)}
  
. While it is true we assume the weights are fixed, we can think of their variance as the variance of an empirical distribution defined by sampling (with equal probability) one weight from our set of weights (similar to how we would think about the correlation of x and y in a simple linear regression).