4. Locally weighted linear regression and bias-variance tradeoff. (25 points) Consider a dataset with n data points (r;, Mi), 2; CRP, following the following linear model yi = B*ritti, i = 1, . ..,n, where ci ~ N(0, o? ) are independent (but not identically distributed) Gaussian noise with zero mean and variance of. (a) (5 points) Show that the ridge regression which introduces a squared 62 norm penalty on the parameter in the maximum likelihood estimate of / can be written as follows B(A) = arg min { (XB - y)"W(XB -y) + >llBl2} B for property defined diagonal matrix W, matrix X and vector y. (b) (5 points) Find the close-form solution for B(A) and its distribution conditioning on {ri}. (c) (5 points) Derive the bias as a function of A and some fixed test point . (d) (5 points) Derive the variance term as a function of A. (e) (5 points) Now assuming the data are one-dimensional, the training dataset consists of two samples x1 = 1.5 and 72 = 1, and the test sample a = 0.5. The true parameter ; = 1, B; = 0.5, the noise variance is given by of = 2, 0, = 1. Plot the MSE (Bias square plus variance) as a function of the regularization parameter A