1. A simple multi-layer perceptron (MLP) neural network is designed as shown in Figure 1. The network has two input feature variables X1, X2 and a single output node Y . A constant value 1 is also input to the hidden layer and the output layer respectively. The activation function at the hidden layer Z is f (a) = a2 + 3a. The activation function at the output layer Y is f (a) = a. Copy this diagram to your solution sheet and add the weights W to the edges of the network (distinguish different edges using superscript and subscript).

figure 1: A multi-layer perceptron neural network.

Describe the MLP network in question (a) using mathematical expressions in a forward passing procedure (i.e. write the mathematical expression of Y (X1, X2))

3. State the type of problem that the MLP network in question (a) can solve. Write down a typically used loss function for this type of problem.

4. Gradient descent method is used to train the MLP network. Given the loss function you provided in question (c), calculate the partial derivative of the total network error E with respect to the weight connecting the hidden node Z1 and the output node Y (figure 1).

5. In a convolutional neural network, the input to a convolutional layer is a 100 100 RGB colour image, and 64 filters with the size of 3 3 are applied in this convolutional layer. Calculate the number of parameters (or weights) to be learned in this layer.

6. For the task of object recognition from images, why is convolutional neural network more favourable than multi-layer perceptron neural network? Which technique overcomes the problem of vanishing gradient in neural network training, so that deeper networks can be trained?