Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

As the picture shows: For leave-one-out cross validation (or equivalently c-fold cross validation with c = n) the cross validation error Eva for ridge regression

As the picture shows:

image text in transcribed
For leave-one-out cross validation (or equivalently c-fold cross validation with c = n) the cross validation error Eva for ridge regression actually has a closed- form solution Eval = yi - yi IM= n - [ H(v)]i. (1) where y; is the prediction of y; when the model is learned from all n data points (no data point is left out), y the regularization parameter and [H(y)] is element (i, i) of the matrix H(y) = X(X X + 1)-iXT. In this problem, we will let x, denote the entire ith row of X. (a) Let X_; denote the matrix X where row i is removed, and y_; is the column vector y with element i removed. Show that XIX_ = XX - xx], XI,y_ = Xy - Xiyi, and [H(Y)]a = x] (XX + 71) 'x: (b) Using the results from (a) and a special case of the matrix inversion lemma (A - VV)-1 = A-1 A vvTA-1 1 - VIA TV' show that 1 3_, = B+ 1- H(7)] u (XX+yl) 'x;(yi - yi). Here _; are the parameters learned from all data points except i, and 3 the parameters learned from all data. Hint: Start from the ridge regression expression for B_, as a function of X_i, y-; and . (c) Use your result from (b) to derive eq. (1), starting from (d) Describe (in a few sentences) what eq. (1) can be used for in practice

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Fundamentals of Heat and Mass Transfer

Authors: Incropera, Dewitt, Bergman, Lavine

6th Edition

978-0470055540, 471457280, 470881453, 470055545, 978-0470881453, 978-0471457282

More Books

Students also viewed these Mathematics questions

Question

Determine the normal reactions at A and B in Prob. 51.

Answered: 1 week ago

Question

3. Applying: Using a general concept to solve a particular problem.

Answered: 1 week ago