Consider the following small convolutional neural network (sCNN), where (x1, x2, x3) is the input, followed by a convolution layer with 2 Figure 2:

A small convolutional neural net. a filter (w1, w2), a ReLU layer, and a fully connected layer with weight (v1, v2). The computational framework of the network is specified as follows:

a1 = x1w1 + x2w2

a2 = x2w1 + x3w2

o1 = max{0, a1}

o2 = max{0, a2}

y = o1v1 + o2v2

For an example (x, y) R 3 {1, +1}, the logistic loss of the CNN is L = ln(1 + exp(yy)), which is a function of the parameters of the network: w1, w2, v1, v2.

Write down L / v1 and L / v2 (show the intermediate steps that use chain rule). You can use the sigmoid function (z) = 1 / 1+ez to simplify your notation. (5 points)

Write down L / w1 and L / w2 (show the intermediate steps that use chain rule). The derivative of the ReLU function is H(a) = I[a > 0], which you can use directly in your answer.