4. MLE and MAP, another interpretation of the penalty term in ridge regression Suppose we observe N data samples {(a:,-, y,) 1, where y, is generated by the following rule: yi : mill-'6 + 6i! Where 33,-, [3 6 Rd, 3;, E IR, and 6,; is an i.i.d random noise drawn from the Gaussian Distribution: 6,; N N(0,O'2) with a known constant 0'. We further denote Y = [y1,y2, ...,yN]T and X = [321,552, ..., 3N]T. Now, we are interested in estimating ,6 from the observed data. (a) (5 Points) What are the dimensions of Y and X? of Derive the likelihood function 03). (b) (5 Points) Show that the MLE estimator 8MLE of [3 is equivalent to the solution of the following linear regression problem: 1 ' YX 2 2 H3111 2 || 5H2 ( ) (c) (5 Points) Now we suppose 6 is not a deterministic parameter, but a random variable having a Gaussian prior distribution: 2 0' pm) ~ No, 51): where I is a d X d identity matrix and A > 0 is a known parameter. Show that the MAP estimation 8M AP 0f ,8 is equivalent to the solution of the following ridge regression problem: , 1 InlngllY-XlngrAllllg (3) (d) (5 Points) Refer to the closed form solutions of (2) and (3) in the lecture slides, what might be an issue of 3m] if d > N? How can Bmap possibly address it