Question: You are given a dataset D {(n. Yn)}=1, where Rd, d 1, and yn e {+1,-1}. For we Rd+1 and 2 e Rd+1, we
You are given a dataset D {(n. Yn)}=1, where Rd, d 1, and yn e {+1,-1}. For we Rd+1 and 2 e Rd+1, we wish to train a logistic regression modell h(x)=0 (b+ wx) = 0(wx), (1) 1+ R is the logistic function. Following the arguments on page 91 of LFD, where 0(2): = the in-sample error can be written as where Ein (w) == log| Pw(yla) = (a) Show that Ein(w) can be expressed as 1 N 1 [Pw (yn /2n)] + (ynn). [h(x) y = +1 1-h(x) y=-11 1 1 Ein (w): _1[3n = +1] log [h(2m)] + [3n = 1] log [1 h(2m)]), where [argument] evaluates to 1 if the argument is true and 0 if it is false. (b) Show that Ein (w) can also be expressed as Ein (w)= n=1 (2) (3) 3 log(1+exp(-yn w rn)). (c) Use (5) to show that VEin (w) = CN=1 - Ynn0 (-yn w rn), and argue that a "misclassi- fied" example contributes more to the gradient than a correctly classified one. (d) Show that VEin (w) can be expressed as (5) VEin (w) = Xp, for some expression p, where X is the data matrix you are familiar with from linear regression. What is p and how does it compare with the gradient of the in-sample error of linear regression? (6)
Step by Step Solution
3.33 Rating (138 Votes )
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
