Consider a $3-2-1$ neural network, in which the input layer is linear, the output layer has a nonlinearity

of the form:

$f(x)=\frac{e^{4x}}{1+e^{4x}}$

and the output of the $j$ th node in the hidden layer is given by a polynomial function of the form:

hat$(y{)}_j^{(1)}=w_{j1}{x}_1+w_{j2}{x}_2+w_{j3}{x}_3$

for $j=1$ and $2.$

$($i$)$ Draw this neural network showing all the neurons. Then, clearly list out all the parameters

of this neural network and the operations at each layer $($for $l=1$ and $2).$ Use a superscript $(l)$ to denote

the index of the layer, e$.$g$.,w_{ij}^{(l)}.$ How many trainable parameters does this neural network have?

$($iii$)$ Suppose that a sample $(1,1)$ is given with a label of $y=1.$ Find the backpropagation

update of the weight $w_{12}^{(2)}$ in the output layer using the quadratic cost function:

$J(w)=\frac{1}{2}|y-$hat$(y{)}^{(2)}{|}^2$

and a step$-$size of $.$ Your answer may only contain $w_{ji}\text{'}$s$,w_{kj}\text{'}$s$,q_j\text{'}$s and $.$

$($iv$)$ Suppose that a sample $(2,-1)$ is given with a label of $y=0.$ Find the backpropagation

update of the parameter $w_{13}^{(1)}$ using the same cost function and step$-$size as in part $($ii$).$ Your answer may

only contain $w_{ji}\text{'}$s$,$ and $.$