Answered step by step
Verified Expert Solution
Question
1 Approved Answer
In the following, all the norms are the Euclidean ones. Here you will code a Matlab function that implements the Least Square (LS) algorithm seen
In the following, all the norms are the Euclidean ones. Here you will code a Matlab function that implements the Least Square (LS) algorithm seen in class and a generalization called Regularized Least Squares (RLS), or ridge regression. The aim of this problem is to familiarize with the implementation of these algorithms and also to refresh your linear algebra skills. The input to the algorithms is the training set S={(x1,y1),,(xm,ym)}, where xiRd and yiR, for i=1,,m. Define the design matrix XRmd with rows the inputs x1,,xm and the response vector y=[y1,,ym]. We want to fit a linear function f(x)=wx+b. As we did in class, define x~i=[1,xi], for i=1,,m, and the augmented design matrix X~=[1X], where 1 is the vector of all ones, and set w~=[b,w]. We saw in class that the LS problem can be written as: minw~yX~w~22. The RLS problem is: minw,bi=1m(wxi+byi)2+w22+b2, where is the regularization parameter and is there only to make the algorithm easily implementable. We will set to eps in matlab, that returns the smallest difference we can represent between floating point numbers. You can see that LS is nothing else than RLS with =0 and =0. Hence, we can just implement RLS. (a) Let C=X~X~+Q, where Q=[00I] 0 is the column vector of all zeroes, and I the identity matrix. Show that the solution to the RLS problem satisfies Cw~=X~y (b) From the above formula, solve for w~ and implement RLS with prototype \[ [\mathrm{w}, \mathrm{b}]=\text { train_rls }(\mathrm{X}, \mathrm{y}, \text { lambda, epsilon }) \] The code must work also in the case that matrix C is not invertible. Hint: The Matlab command for the pseudoinverse is pinv
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started