1.20 (

) www In this exercise, we explore the behaviour of the Gaussian distribution in high-dimensional spaces. Consider a Gaussian distribution in D dimensions given by p(x) =

1

(2πσ2)D/2 exp



−

x2 2σ2



. (1.147)

We wish to find the density with respect to radius in polar coordinates in which the direction variables have been integrated out. To do this, show that the integral of the probability density over a thin shell of radius r and thickness , where 1, is given by p(r) where p(r) = SDrD−1

(2πσ2)D/2 exp



− r2 2σ2



(1.148)

where SD is the surface area of a unit sphere inD dimensions. Show that the function p(r) has a single stationary point located, for large D, at
r 

√

Dσ. By considering p(
r + ) where
r, show that for large D, p(
r + ) = p(
r) exp



− 3 2 2σ2



(1.149)

which shows that
r is a maximum of the radial probability density and also that p(r)

decays exponentially away from its maximum at
r with length scale σ. We have already seen that σ
r for large D, and so we see that most of the probability mass is concentrated in a thin shell at large radius. Finally, show that the probability density p(x) is larger at the origin than at the radius
r by a factor of exp(D/2).

We therefore see that most of the probability mass in a high-dimensional Gaussian distribution is located at a different radius from the region of high probability density.

This property of distributions in spaces of high dimensionality will have important consequences when we consider Bayesian inference of model parameters in later chapters.