Loss functions. Consider the following two loss functions, including (1) mean-squared error $\mathrm{Loss}(T,O)=\frac{1}{2}(T-O{)}^2$, and (2) cross-entropy $\mathrm{Loss}(T,O)=-T\log O-(1-T)\log (1-O)$ for binary classification. Assume the activation function is sigmoid.

a. Show the derivation of the error $\delta$ for the output unit in backpropagation process and compare the two loss functions (e.g., potential problems they might produce).

b. Now, we wish to generalize the cross-entropy loss to the scenario of multiclass classification. The target output is a one-hot vector of length $C$ (i.e., the number of total classes), and the index of nonzero element (i.e., 1) represents the class label. The output is also a vector of the same length $O=[O_1,O_2,\dots ,O_C]$. Show the derivation of the categorical cross-entropy loss and the error $\delta$ of the output unit. (hint: there are two key steps, including (1) normalizing the output values by scaling between 0 and 1 , and (2) deriving the cross entropy loss following the definition for the binary case where the loss can be represented as $\mathrm{Loss}(T,O)=-{\sum }_{i=1}^2{T}_i\log \left(O_i\right)$.)