11 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Design effect | 2/12 | https://en.wikipedia.org/wiki/Design_effect | reference | science, encyclopedia | 2026-05-05T09:49:56.844427+00:00 | kb-cron |
The numerator represents the actual variance for an estimator of a parameter (
θ
^
w
{\displaystyle {\hat {\theta }}_{w}}
) under a given sampling design
p
{\displaystyle p}
; The denominator represents the variance assuming the same sample size, but if the sample were obtained using the estimator for simple random sampling without replacement (
θ
^
S
R
S
W
O
R
{\displaystyle {\hat {\theta }}_{SRSWOR}}
). So that:
Deff
p
(
θ
^
)
=
v
a
r
(
θ
^
w
)
v
a
r
(
θ
^
S
R
S
W
O
R
)
{\displaystyle {\text{Deff}}_{p}({\hat {\theta }})={\frac {var({\hat {\theta }}_{w})}{var({\hat {\theta }}_{SRSWOR})}}}
In other words,
Deff
{\displaystyle {\text{Deff}}}
measures the extent to which the variance has increased (or, in some cases, decreased) because the sample was drawn and adjusted to a specific sampling design (e.g., using weights or other measures) compared to if the sample was from a simple random sample (without replacement). Notice how the definition of
Deff
{\displaystyle {\text{Deff}}}
is based on parameters of the population that are often unknown, and that are hard to estimate directly. Specifically, the definition involves the variances of estimators under two different sampling designs, even though only a single sampling design is used in practice. For example, when estimating the population mean, the
Deff
{\displaystyle {\text{Deff}}}
(for some sampling design p) is:
Deff
p
=
v
a
r
p
(
y
¯
p
)
(
1
−
f
)
S
y
2
/
n
{\displaystyle {\text{Deff}}_{p}={\frac {var_{p}({\bar {y}}_{p})}{(1-f)S_{y}^{2}/n}}}
Where
n
{\displaystyle n}
is the sample size,
f
=
n
/
N
{\displaystyle f=n/N}
is the fraction of the sample from the population,
(
1
−
f
)
{\displaystyle (1-f)}
is the (squared) finite population correction (FPC),
S
y
2
{\displaystyle S_{y}^{2}}
is the unbiassed sample variance, and
v
a
r
p
(
y
¯
p
)
{\displaystyle var_{p}({\bar {y}}_{p})}
is some estimator of the variance of the mean under the sampling design. The issue with the above formula is that it is extremely rare to be able to directly estimate the variance of the estimated mean under two different sampling designs, since most studies rely on only a single sampling design. There are many ways of calculation
Deff
{\displaystyle {\text{Deff}}}
, depending on the parameter of interest (e.g. population total, population mean, quantiles, ratio of quantities etc.), the estimator used, and the sampling design (e.g. clustered sampling, stratified sampling, post-stratification, multi-stage sampling, etc.). The process of estimating
Deff
{\displaystyle {\text{Deff}}}
for specific designs will be described in the following section.
=== Deft === A related quantity to
Deff
{\displaystyle {\text{Deff}}}
, proposed by Kish in 1995, is the Design Effect Factor, abbreviated as
Deft
{\displaystyle {\text{Deft}}}
(or also
D
eft
{\displaystyle D_{\text{eft}}}
). It is defined as the square root of the variance ratios while also having the denominator use a simple random sample with replacement (SRSWR), instead of without replacement (SRSWOR):
Deft
=
var
(
θ
^
w
)
var
(
θ
^
S
R
S
W
R
)
{\displaystyle {\text{Deft}}={\sqrt {\frac {{\text{var}}({\hat {\theta }}_{w})}{{\text{var}}({\hat {\theta }}_{SRSWR})}}}}
In this later definition (proposed in 1995, vs 1965) Kish argued in favor of using
Deft
2
{\displaystyle {\text{Deft}}^{2}}
over
Deff
{\displaystyle {\text{Deff}}}
for several reasons. It was argued that SRS "without replacement" (with its positive effect on the variance) should be captured in the denominator part in the definition of the design effect, since it is part of the sampling design. Also, since often the use of the factor is in confidence intervals), it was claimed that using
Deft
{\displaystyle {\text{Deft}}}
will be simpler than writing
Deff
{\displaystyle {\sqrt {\text{Deff}}}}
. It is also said that for many cases when the population is very large,
Deft
{\displaystyle {\text{Deft}}}
is (almost) the square root of
Deff
{\displaystyle {\text{Deff}}}
(
Deft
≈
Deff
{\displaystyle {\text{Deft}}\approx {\sqrt {\text{Deff}}}}
), hence it is easier to use than exactly calculating the finite population correction (FPC). Even so, in various cases a researcher might approximate the
Deft
{\displaystyle {\text{Deft}}}
by calculating the variance in the numerator while assuming SRS with replacement (SRSWR) instead of SRS without replacement (SRSWOR), even if it is not precise. For example, consider a multistage design with primary sampling units (PSUs) selected systematically with probability proportional to some measure of size from a list sorted in a particular way (say, by number of households in each PSU). Also, let it be combined with an estimator that uses raking to match the totals for several demographic variables. In such a design, the joint selection probabilities for the PSUs, which are needed for a without replacement variance estimator, are 0 for some pairs of PSUs - implying that an exact design-based (i.e., repeated sampling) variance estimator does not exist. Another example is when a public use file issued by some government agency is used for analysis. In such a case the information on joint selection probabilities of first-stage units is almost never released. As a result, an analyst cannot estimate a with replacement variance for the numerator even if desired. The standard workaround is to compute a variance estimator as if the PSUs were selected with replacement. This is the default choice in software packages such as Stata, the R survey package, and the SAS survey procedures.
=== Effective sample size === The effective sample size, defined by Kish in 1965, is calculated by dividing the original sample size by the design effect. Namely:
n
eff
=
n
Deff
{\displaystyle n_{\text{eff}}={\frac {n}{\text{Deff}}}}