we have explored the intricate process of training neural networks, focusing on the Stochastic Gradient Descent $($SGD$)$ algorithm. We derived the equations and understood their roles in optimizing the weights of the network to minimize the loss function. The results are as follows.Putting it All Together

The stochastic gradient descent update rule for weight $j$ of neuron $k$ in

the $l-$th layer is:

$w_{jk}^{(l)}larr{w}_{jk}^{(l)}-{}_{ik}^{(l)}$hat$(y{)}_{ij}^{(l-1)},b_k^{(l)}larr{b}_k^{(l)}-{}_{ik}^{(l)}$

where

${}_{ik}^{(l)}=[hat(y{)}_{ik}^{(3)}-y_i]{}^{(l{)}^{\text{'}}}$ for output layer

${}_{ik}^{(l)}={\displaystyle \underset{m}{\overset{?}{}}}[{}_{im}^{(l+1)}{w}_{km}^{(l+1)}]{}^{(l{)}^{\text{'}}}$ for hidden layers

Substitute in relationships for activation functions and their

derivatives once they are chosen by user

Your task in this homework is to extend your understanding from SGD to Batch Gradient Descent $($BGD$).$

Specifically, you will derive the backpropagation equations for Batch Gradient Descent $($BGD$)$ when optimizing a

neural network for regression, using the MSE loss function. $($Hint: You just need to make minor modification to

the above results for SGD$).$Instructions:

Review SGD: Start by revisiting the equations and principles we covered for SGD$.$ Understand the role of each term and how the weights are updated iteratively.

Understand BGD: Understand how Batch Gradient Descent differs from SGD$.$

Derive Backpropagation for BGD: Using the knowledge from steps $1$ and $2,$ derive the backpropagation equations for BGD$.$