In the context of a point estimate of a feature with domain {0, 1}

with no inputs, it is possible for an agent to make a stochastic prediction with a parameter p ∈ [0, 1] such that the agent predicts 1 with probability p and predicts 0 otherwise. For each of the following error measures, give the expected error on a training set with n0 occurrences of 0 and n1 occurrences of 1 (as a function of p).

What is the value of p that minimizes the error? Is this worse or better than the prediction of Figure 7.5 (page 277)?

(a) Mean absolute loss.

(b) Mean squared loss.

(c) Worst-case error.

Exercise 7.3

(a) Prove that for any two predictors A and B on the same dataset, if A dominates B (page 279), that is, A has a higher true-positive rate and lower falsepositive rate than B, then A has a lower cost (and so is better) than B for all cost functions that depend only on the number of false positives and false negatives (assuming the costs to be minimized are non-negative). [Hint:

Prove that the conditions of the statement imply the number of false negatives and the number of false positives is less (or, perhaps, equal), which in turn implies the conclusion.]

(b) Consider the predictors

(a) and

(c) in Figure 7.7 (page 280).

(i) Which of

(a) and

(c) has a better recall (true-positive rate)? (See Example 7.7.)

(ii) What is the precision of (a)? (Give both a ratio and the decimal value to 3 decimal points.)

(iii) What is the precision of (c)?

(iv) Which of

(a) and

(c) is better in terms of precision?

(v) Suppose false positives were 1000 times worse than false negatives, which of

(a) and

(c) would have a lower cost? (Consider the cost of a false negative to be 1 and a false positive to be 1000.)

(vi) Suppose false negatives were 1000 times worse than false positives, which of

(a) and

(c) would have a lower cost?