4. Locally weighted linear regression and bias-variance tradeoff. (30 points) Consider a dataset with n data points (r;, y), I; ERP, following the following linear model yi = B*ritti, i = 1, . ..,n, where e; ~ M(0, o?) are independent (but not identically distributed) Gaussian noise with zero mean and variance of. (a) (5 points) Show that the ridge regression which introduces a squared 42 norm penalty on the parameter in the maximum likelihood estimate of / can be written as follows B B(X) = arg min {(XB - y)?W(XB - y) + AllB/}} for property defined diagonal matrix W, matrix X and vector y. (b) (5 points) Find the close-form solution for B(A) and its distribution conditioning on {ri}. (c) (5 points) Derive the bias as a function of A and some fixed test point I. (d) (5 points) Derive the variance term as a function of A. (e) (10 points) Now assuming the data are one-dimensional, the training dataset consists of two samples 21 = 1.5 and x2 = 1, and the test sample x = 0.5. The true parameter 8; = 1, Bf = 1, the noise variance is given by of = 2, o? = 1. Plot the MSE (Bias square plus variance) as a function of the regularization parameter 1