$($a$)$ Figure $1$ below plots the loss when batch gradient descent is used for training. Which optimizers plots the loss in figure $2$ and $3.($a$)$ Figure $1$ below plots the loss when batch gradient descent is used for training. Which

optimizers plots the loss in figure $2$ and $3.$

$($b$)$ In figure $2,$ assume that the learning rate used was $0\mathrm{.}5.$ Redraw the plot to show the

effect of increasing the learning rate to $2$ and decreasing the learning rate to $0\mathrm{.}01.[2]$

$($c$)$ You are given a simple neural network with a single hidden layer containing two neu$-$

rons, and an output layer containing one neuron. All neurons use the sigmoid activation

function. The weights and biases of the network are as follows:

Weights from input to hidden layer: $w_1=0\mathrm{.}5,w_2=-0\mathrm{.}6$

Biases in hidden layer: $b1=0\mathrm{.}1,b2=-0\mathrm{.}2$

Weights from hidden layer to output: $w3=0\mathrm{.}7,w4=-0\mathrm{.}8$

Bias in output layer: $b3=0\mathrm{.}3$

Given an input $x=0\mathrm{.}75,$ calculate the output of the network and mean squared loss

if the desired output is $1\mathrm{.}25.$ Use the sigmoid activation function. What will be the

effect in the loss if the loss function used is

$L(w,b)=\frac{1}{2}(d-$hat$(y){)}^2+||w^2||$