Denote the pdf of the (mathscr{N}(mathbf{0}, boldsymbol{Sigma})) distribution by (varphi Sigma(cdot)), and let [ mathscr{D}left(boldsymbol{mu}_{0} boldsymbol{Sigma}_{0} mid
Question:
Denote the pdf of the \(\mathscr{N}(\mathbf{0}, \boldsymbol{\Sigma})\) distribution by \(\varphi \Sigma(\cdot)\), and let
\[ \mathscr{D}\left(\boldsymbol{\mu}_{0} \boldsymbol{\Sigma}_{0} \mid \boldsymbol{\mu}_{1}, \boldsymbol{\Sigma}_{1}\right)=\int_{\mathbb{R}^{d}} \varphi_{\Sigma_{0}}\left(\boldsymbol{x}-\boldsymbol{\mu}_{0}\right) \ln \frac{\varphi_{\Sigma_{0}}\left(\boldsymbol{x}-\boldsymbol{\mu}_{0}\right)}{\varphi_{\Sigma_{1}}\left(\boldsymbol{x}-\boldsymbol{\mu}_{1}\right)} \mathrm{d} \boldsymbol{x} \]
be the Kullback-Leibler divergence between the densities of the \(\mathscr{N}\left(\boldsymbol{\mu}_{0}, \boldsymbol{\Sigma}_{0}\right)\) and \(\mathscr{N}\left(\boldsymbol{\mu}_{1}, \boldsymbol{\Sigma}_{1}\right)\) distributions on \(\mathbb{R}^{d}\). Show that
\(2 \mathscr{D}\left(\boldsymbol{\mu}_{0} \boldsymbol{\Sigma}_{0} \mid \boldsymbol{\mu}_{1}, \boldsymbol{\Sigma}_{1}\right)=\operatorname{tr}\left(\boldsymbol{\Sigma}_{1}^{-1} \boldsymbol{\Sigma}_{0}\right)-\ln \left|\boldsymbol{\Sigma}_{1}^{-1} \boldsymbol{\Sigma}_{0}\right|+\left(\boldsymbol{\mu}_{1}-\boldsymbol{\mu}_{0}\right)^{\top} \boldsymbol{\Sigma}_{1}^{-1}\left(\boldsymbol{\mu}_{1}-\boldsymbol{\mu}_{0}\right)-d\)
Hence, deduce the formula in (B.22).
Step by Step Answer:
Data Science And Machine Learning Mathematical And Statistical Methods
ISBN: 9781118710852
1st Edition
Authors: Dirk P. Kroese, Thomas Taimre, Radislav Vaisman, Zdravko Botev