Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

In the following, all the norms are the Euclidean ones. Here you will code a Matlab function that implements the Least Square (LS) algorithm seen

image text in transcribed

In the following, all the norms are the Euclidean ones. Here you will code a Matlab function that implements the Least Square (LS) algorithm seen in class and a generalization called Regularized Least Squares (RLS), or ridge regression. The aim of this problem is to familiarize with the implementation of these algorithms and also to refresh your linear algebra skills. The input to the algorithms is the training set S={(x1,y1),,(xm,ym)}, where xiRd and yiR, for i=1,,m. Define the design matrix XRmd with rows the inputs x1,,xm and the response vector y=[y1,,ym]. We want to fit a linear function f(x)=wx+b. As we did in class, define x~i=[1,xi], for i=1,,m, and the augmented design matrix X~=[1X], where 1 is the vector of all ones, and set w~=[b,w]. We saw in class that the LS problem can be written as: minw~yX~w~22. The RLS problem is: minw,bi=1m(wxi+byi)2+w22+b2, where is the regularization parameter and is there only to make the algorithm easily implementable. We will set to eps in matlab, that returns the smallest difference we can represent between floating point numbers. You can see that LS is nothing else than RLS with =0 and =0. Hence, we can just implement RLS. (a) Let C=X~X~+Q, where Q=[00I] 0 is the column vector of all zeroes, and I the identity matrix. Show that the solution to the RLS problem satisfies Cw~=X~y (b) From the above formula, solve for w~ and implement RLS with prototype \[ [\mathrm{w}, \mathrm{b}]=\text { train_rls }(\mathrm{X}, \mathrm{y}, \text { lambda, epsilon }) \] The code must work also in the case that matrix C is not invertible. Hint: The Matlab command for the pseudoinverse is pinv

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Fundamentals Of Database Systems

Authors: Ramez Elmasri, Shamkant B. Navathe

7th Edition Global Edition

1292097612, 978-1292097619

More Books

Students also viewed these Databases questions