I have a question about this problem:

Binary logistic regression can give poor results when the two classes are perfectly separated by a linear decision boundary. One way to address this problem is to use the Lasso applied to logistic regression.

Write the likelihood function for the logistic regression problem in terms of x; y; 0 and1. Assume for simplicity that we have n observations and only one variable (i.e. xi is areal number, for i = 1; : : : ; n).

Recall that the logistic regression coefficients 0,1are obtained by maximizing L(0, 1). Suppose that all of the xi corresponding to yi = 0 are negative, all other xi are positive. In this case, note that we can get L(0, 1) arbitrarily close to 1. Explain why this means that 0and 1are undefined.

Inspired by the Lasso, suggest a way to modify the negative log-likelihood function

log L(0, 1) so that0and1become defined even in the separable case above. Justifyyour answer.