THE GAUSSIAN NATURE OF THE POSTERIOR AND THE PREDICTION

(present a pdf file with the theoretical answers and a Jupyter notebook with the Python functions.)

In this section, we shall consider the two following lemmas. Note that you are not required to prove them.

Lemma 1. If random variables x R^n and y R^p have the Gaussian probability distributions:

then the joint distribution of (x, y) and the marginal distribution of y are given as:

Lemma 2. If random variables x R^n and y R^m have joint the Gaussian probability distributions:

then the marginal and conditional distribution of x and y are given as follows:

Assuming that w = (w1,...,wn)^T , y = (y1,..., yn) and the matrix is the same as defined in lectures, answer the two following questions.

1. Relying on these lemmas, prove that the posterior distribution of w, i.e. f (w | y,,^2 ) is N (,), where:

2. Relying on these lemmas, prove that the prediction distribution of y for a new point x, i.e

, where:

and = y(x,), where and are defined by f (w | y,^2 ,) = N (,) and f = [1(x),...,n(x)]^T . Here ^2 are the values of ^2 and given by type II maximum likelihood.