9.7 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Bayesian experimental design | 1/2 | https://en.wikipedia.org/wiki/Bayesian_experimental_design | reference | science, encyclopedia | 2026-05-05T09:48:59.414261+00:00 | kb-cron |
Bayesian experimental design provides a general probability-theoretical framework from which other theories on experimental design can be derived. It is based on Bayesian inference to interpret the observations/data acquired during the experiment. This allows accounting for both any prior knowledge on the parameters to be determined as well as uncertainties in observations. The theory of Bayesian experimental design is to a certain extent based on the theory for making optimal decisions under uncertainty. The aim when designing an experiment is to maximize the expected utility of the experiment outcome. The utility is most commonly defined in terms of a measure of the accuracy of the information provided by the experiment (e.g., the Shannon information or the negative of the variance) but may also involve factors such as the financial cost of performing the experiment. What will be the optimal experiment design depends on the particular utility criterion chosen.
== Relations to more specialized optimal design theory ==
=== Linear theory === If the model is linear, the prior probability density function (PDF) is homogeneous and observational errors are normally distributed, the theory simplifies to the classical optimal experimental design theory.
=== Approximate normality === In numerous publications on Bayesian experimental design, it is (often implicitly) assumed that all posterior probabilities will be approximately normal. This allows for the expected utility to be calculated using linear theory, averaging over the space of model parameters. Caution must however be taken when applying this method, since approximate normality of all possible posteriors is difficult to verify, even in cases of normal observational errors and uniform prior probability.
=== Posterior distribution === In many cases, the posterior distribution is not available in closed form and has to be approximated using numerical methods. The most common approach is to use Markov chain Monte Carlo methods to generate samples from the posterior, which can then be used to approximate the expected utility. Another approach is to use a variational Bayes approximation of the posterior, which can often be calculated in closed form. This approach has the advantage of being computationally more efficient than Monte Carlo methods, but the disadvantage that the approximation might not be very accurate. Some authors proposed approaches that use the posterior predictive distribution to assess the effect of new measurements on prediction uncertainty, while others suggest maximizing the mutual information between parameters, predictions and potential new experiments.
== Mathematical formulation ==
Given a vector
θ
{\displaystyle \theta }
of parameters to determine, a prior probability
p
(
θ
)
{\displaystyle p(\theta )}
over those parameters and a likelihood
p
(
y
∣
θ
,
ξ
)
{\displaystyle p(y\mid \theta ,\xi )}
for making observation
y
{\displaystyle y}
, given parameter values
θ
{\displaystyle \theta }
and an experiment design
ξ
{\displaystyle \xi }
, the posterior probability can be calculated using Bayes' theorem
p
(
θ
∣
y
,
ξ
)
=
p
(
y
∣
θ
,
ξ
)
p
(
θ
)
p
(
y
∣
ξ
)
,
{\displaystyle p(\theta \mid y,\xi )={\frac {p(y\mid \theta ,\xi )p(\theta )}{p(y\mid \xi )}}\,,}
where
p
(
y
∣
ξ
)
{\displaystyle p(y\mid \xi )}
is the marginal probability density in observation space
p
(
y
∣
ξ
)
=
∫
p
(
θ
)
p
(
y
∣
θ
,
ξ
)
d
θ
.
{\displaystyle p(y\mid \xi )=\int p(\theta )p(y\mid \theta ,\xi )\,d\theta \,.}
The expected utility of an experiment with design
ξ
{\displaystyle \xi }
can then be defined
U
(
ξ
)
=
∫
p
(
y
∣
ξ
)
U
(
y
,
ξ
)
d
y
,
{\displaystyle U(\xi )=\int p(y\mid \xi )U(y,\xi )\,dy,}
where
U
(
y
,
ξ
)
{\displaystyle U(y,\xi )}
is some real-valued functional of the posterior probability
p
(
θ
∣
y
,
ξ
)
{\displaystyle p(\theta \mid y,\xi )}
after making observation
y
{\displaystyle y}
using an experiment design
ξ
{\displaystyle \xi }
.
=== Gain in Shannon information as utility === Utility may be defined as the prior-posterior gain in Shannon information
U
(
y
,
ξ
)
=
∫
log
(
p
(
θ
∣
y
,
ξ
)
)
p
(
θ
|
y
,
ξ
)
d
θ
−
∫
log
(
p
(
θ
)
)
p
(
θ
)
d
θ
.
{\displaystyle U(y,\xi )=\int \log(p(\theta \mid y,\xi ))\,p(\theta |y,\xi )\,d\theta -\int \log(p(\theta ))\,p(\theta )\,d\theta \,.}
Another possibility is to define the utility as
U
(
y
,
ξ
)
=
D
K
L
(
p
(
θ
∣
y
,
ξ
)
‖
p
(
θ
)
)
,
{\displaystyle U(y,\xi )=D_{KL}(p(\theta \mid y,\xi )\|p(\theta ))\,,}
the Kullback–Leibler divergence of the prior from the posterior distribution. Lindley (1956) noted that the expected utility will then be coordinate-independent and can be written in two forms
U
(
ξ
)
=
∫
∫
log
(
p
(
θ
∣
y
,
ξ
)
)
p
(
θ
,
y
∣
ξ
)
d
θ
d
y
−
∫
log
(
p
(
θ
)
)
p
(
θ
)
d
θ
=
∫
∫
log
(
p
(
y
∣
θ
,
ξ
)
)
p
(
θ
,
y
∣
ξ
)
d
y
d
θ
−
∫
log
(
p
(
y
∣
ξ
)
)
p
(
y
∣
ξ
)
d
y
,
{\displaystyle {\begin{alignedat}{2}U(\xi )&=\int \int \log(p(\theta \mid y,\xi ))\,p(\theta ,y\mid \xi )\,d\theta \,dy-\int \log(p(\theta ))\,p(\theta )\,d\theta \\&=\int \int \log(p(y\mid \theta ,\xi ))\,p(\theta ,y\mid \xi )\,dy\,d\theta -\int \log(p(y\mid \xi ))\,p(y\mid \xi )\,dy,\end{alignedat}}\,}