Consider a dataset with n data points (x, y), x, ERP, following the following linear model...
Fantastic news! We've Found the answer you've been seeking!
Question:
Transcribed Image Text:
Consider a dataset with n data points (x, y), x, ERP, following the following linear model +t, i=1. where ; N(0,02) are i.i.d. Gaussian noise with zero mean and variance o. (a) (5 points) Show that the ridge regression, which introduces a squared 2 norm penalty on the parameter in the maximum likelihood estimate of can be written as follows B(A) = arg min {XB-y|| + ||||} for property defined matrix X and vector y. (b) (5 points) Find the close-form solution for B(A) and its distribution conditioning on {x}. (c) (5 points) Derive the bias as a function of A and some fixed test point x. (d) (5 points) Derive the variance term as a function of A. (e) (5 points) Now assuming the data are one-dimensional, the training dataset consists of two samples x = 0.15 and x2 = 1.1, and the test sample x = 1. The true parameter 3 = -1, B = 1, the noise variance is given by 2 = 0.5. Plot the MSE (Bias square plus variance) as a function of the regularization parameter A. (f) (5 points) Now change the test sample to be a x = 1.5, and keep everything else to be the same as in the previous question. Plot the MSE (Bias square plus variance) as a function of the regularization parameter A, and comment on the difference from the previous result. Consider a dataset with n data points (x, y), x, ERP, following the following linear model +t, i=1. where ; N(0,02) are i.i.d. Gaussian noise with zero mean and variance o. (a) (5 points) Show that the ridge regression, which introduces a squared 2 norm penalty on the parameter in the maximum likelihood estimate of can be written as follows B(A) = arg min {XB-y|| + ||||} for property defined matrix X and vector y. (b) (5 points) Find the close-form solution for B(A) and its distribution conditioning on {x}. (c) (5 points) Derive the bias as a function of A and some fixed test point x. (d) (5 points) Derive the variance term as a function of A. (e) (5 points) Now assuming the data are one-dimensional, the training dataset consists of two samples x = 0.15 and x2 = 1.1, and the test sample x = 1. The true parameter 3 = -1, B = 1, the noise variance is given by 2 = 0.5. Plot the MSE (Bias square plus variance) as a function of the regularization parameter A. (f) (5 points) Now change the test sample to be a x = 1.5, and keep everything else to be the same as in the previous question. Plot the MSE (Bias square plus variance) as a function of the regularization parameter A, and comment on the difference from the previous result.
Expert Answer:
Answer rating: 100% (QA)
a To show that ridge regression introduces a squared L2 norm penalty on the parameter vector we star... View the full answer
Related Book For
Discrete Time Signal Processing
ISBN: 978-0137549207
2nd Edition
Authors: Alan V. Oppenheim, Rolan W. Schafer
Posted Date:
Students also viewed these mathematics questions
-
Merchandise is purchased on account, credit terms 2/10, n/30, $5,500. Merchandise is returned for credit before payment is made, $500. Payment is made within the discount period.
-
Q1. You have identified a market opportunity for home media players that would cater for older members of the population. Many older people have difficulty in understanding the operating principles...
-
answer all questions as instructed below. attend all questions. 4 Computer Vision (a) Explain why such a tiny number of 2D Gabor wavelets as shown in this sequence are so efficient at representing...
-
Scott incorporates his sole proprietorship as Superior Corporation and transfers its assets to Superior in exchange for all 100 shares of Superior stock and four $7,500 interest-bearing notes. The...
-
Assume that square ABCD (Figure 2) has sides of length 1 and that E, F; G, and 11 are the midpoints of the sides. If the indicated pattern is continued indefinitely, what will be the area of the...
-
How to calculate the question below?? Birthdaybook requires $125,000 to complete project. a)With a compensating balance requirement of 10%, how much will the firm need to borrow? (Round the final...
-
Graph the following numbers on the number line: 1. -10 2. 4 3. 0
-
The number of internal disk drives (in millions) made at a plant in Taiwan during the past 5 years follows: a) Forecast the number of disk drives to be made next year, using linear regression. b)...
-
I cant get my Trial balance to balance out?? Can you tell me why. I am using these journal entries. (1)Cash 20,000 Common Stock 20,000 (2)Cash 11,000 Preffered stock $100 par 10,000 Additiononal...
-
The equilibrium adsorption of methane on a given activated carbon was studied by Grant et al. (1962). They proposed a Langmuir-type adsorption isotherm with parameters \(q_{m}=48 \mathrm{~g}...
-
Adult Learning is a potential research subject. As you move through the dissertation process, you will undoubtedly receive feedback that: 1) you do not agree with, 2) contradicts other feedback, or...
-
Part 1 The Hovey and Beard Company manufactured a variety of wooden toys, including animals, pull toys, and the like. The toys were manufactured by a transformation process that began in the wood...
-
Donna Company began operations on June 1. The following transactions took place in June: a. Purchases of merchandise on account were \(\$ 750,000\). b. The cost of freight to receive the inventory...
-
What are Treasury bills, and how is a rate of return from them derived?
-
The method described in this chapter is a simulation method because the number of stages and the feed and withdrawal locations must all be specified. How do you determine the optimum feed stage?
-
Prove that for ideal systems (constant relative volatility) with no interaction between an added nonvolatile solute and the volatile components, there is no effect of adding the nonvolatile solute on...
-
On July 1, 2022, Pharoah Company purchased new equipment for $72,000. Its estimated useful life was 7 years with a $9,000 salvage value. On January 1, 2025, the company estimated that the equipment's...
-
(a) What is the focal length of a magnifying glass that gives an angular magnification of 8.0 when the image is at infinity? (b) How far must the object be from the lens?
-
Consider the system in Figure. (a) Find the impulse response h[n] of the overall system. (b) Find the frequency response of the overall system. (c) Specify a difference equation that relates the...
-
In real continuous-time signal x c (t) is bandlimited to frequencies below 5 kHz; i.e., X c (j) = 0 for || 2(5000). The signal x c (t) is sampled with a sampling rate of 10,000 samples per second...
-
The impulse response of a linear time-invariant system is lal." alt = "The impulse response of a linear time-invariant system is (a)" alt = "The impulse response of a linear time-invariant system is...
-
A purchase of merchandise for $300 with a trade discount of 10% would require a debit to Purchases of (a) $330. (c) $297. (b) $300. (d) $270.
-
A trade discount is a reduction from the list or catalog price offered to different classes of customers. True/False
-
Purchases Returns and Allowances is debited when merchandise is returned for credit. True/False
Study smarter with the SolutionInn App