Suppose we have a deep MLP classifier and L hidden layers as follows. hi=(Wihi1+bi)i=1,,L where the input data vector h0=xRD01 and is the nonlinear activation function. Weights and the bias vector of the i-th layer are WiRDiDi1 and biRDi1 respectively. The readout function is the softmax. In other words, the probability of the input x belong to the class k is, P(xbelongtoclassk)=y[k]=jexp(hL[j])exp(hL[k]) where the input to the softmax hLRDL1 is often called logits. The dimension of the logits should be the same as the total number of classes K, i.e., DL=K. To train the classifier, we use cross-entropy loss as below, (y,y)=k=1Ky[k]logy[k], where yRK1 is the ground-truth probability vector of the input x. Typically, it is a onehot encoded vector, i.e., if input x of the training set belongs to the class c, then y[c]=1 and y[k]=0,k=c. N.B.: For simplicity, we only consider the case of a single sample in the above context, i.e., the batch dimesnion is ignored. 3.1 [10pts] Derive the gradients of loss w.r.t. all hidden activations hi. In particular, for any layer 1iL, you need to show how to derive hi. You need to write down the shapes of all tensors involved in the final expressions. 3.2 [15pts] Derive the gradients of loss w.r.t. all weights Wi and biases bi. In particular, for any layer 1iL, you need to show how to derive Wi and bi. You need to write down the shapes of all tensors involved in the final expressions. 3.3 [15pts] Derive the Kaiming initialization [3] in our context, i.e., applying the same variance analysis technique we learned from Xavier initialization [4] to this deep MLP with ReLU activations