Logistic regression estimating equations 2. (6 points) When 3;, is a binary 01 variable, it is awkward to model it as a linear function of 33,-, since it is bounded: any line we choose, as long as it has nonzero slope, will eventually leave the interval [0,1]. A more natural choice is to estimate some linear function and then pass it through a nonlineority. For various reasons, the most popular choice of nonlinearity is the logistic function ez 1 2 1-1-82 : 1-163\" f (Z) This is also sometimes called a sigmoid function because if the plot looks a bit like the letter S. A (simple) logistic regression is a regression of a binary response 3; on a single variable ac, where we estimate :1} = f (a, + bx), for f as dened above. Note that because 9 is always 0 or 1, it may seem nonsensical to predict it to be, e.g., 0.7. It does, however, make sense to predict that it has a 70% probability of being 1 and a 30% probability of being zero. We could try to use the squared error loss or absolute loss to estimate a simple logistic regression, but it is more natural to use the log-likelihood, i.e. the probability that $11,. . .,yn would come out exactly the same way they did if we were to collect ii new data points. If yl, . . . ,yn were independent, and each 3;, had a probability exactly 3},- of turning out to be 1, then the corresponding loglikelihood loss (or log loss for short] is LU} 7:!) : log(g}) ify = 1 log(1:r}] iii/:0 (b) (3 points) Give an expression for the empirical risk with respect to the logloss and take derivatives to obtain estimating equations for a and (3. Show that the estimating equations are exactly the same as the estimating equations for linear regression, except for the one difference that the form of t is t: + bx) instead of a + bx