1 Bias-Variance Tradeoff Consider a dataset with n data points (X;, yi), X; C RPX], drawn from the following linear model: y = x 3* + E, where & is a Gaussian noise and the star sign is used to differentiate the true parameter from the estimators that will be introduced later. Consider the L2 regularized linear regression as follows: 3x = arg min- _(yi - X, B)2 + X13113 i=1 where A 2 0 is the regularization parameter. Let X E R"XP denote the matrix obtained by stacking x, in each row. Properties of an affine transformation of a gaussian random variable will be useful throughout this problem. a. Find the closed form solution for 3, and its distribution. b. Calculate the bias term Ex Bx] - x 3* as a function of A and some fired test point x. c. Calculate the variance term E (x x - ExBal)" as a function of A and some fired test point x. d. Use the results from parts (b) and (c) and the bias-variance theorem to analyze the impact of A in the squared error. Specifically, which term dominates when A is small or large