m argued that the cross-entropy is small if 01(2) :5 y for all training inputs. The argument relied on y being equal to either 0 or 1. This is usually true in classication problems, but for other problems (e.g., regression problems) y can sometimes take values intermediate between 0 and 1. Show that the cross- entropy is still minimized when 0(a) = y for all training inputs. When this is the case the cross-entropy has the value: C = 1 Elylny+ (1 y)1n(1 y)l. (64) n x The quantity -[y lny + (l - y) ln(l y)] is sometimes known as the binary entropy. For this problem, you can assume y as a single neuron (in the output layer} or a vector (i.e., the Whole output layer). - If you assume the former, you can do a rigorous proof by using calculus and minimizing the derivative of the function. But if you are not comfortable with calculus, you can pick at least three values fury (between I} and 1), and for each value of y, you should compute the Crossentropy value using that y and several varying values of a (e.g. 0.1, 0.2, 0.3,..., 0.9). - If you assume the latter, you can do a formal proof by calculus as well (although the derivative function will have several variables), but I recommend using Information Theory to prove the formula will minimize when y and a are equal