Stochastic Stochastic gradient descent $($SGD$)$ is an important optimization tool in machine learning, used

every$-$ where from logistic regression to training neural networks. In this problem, you will be asked

to first implement SGD for linear regression using the squared loss function. Then, you will analyze

how several parameters affect the learning process.

Linear regression learns a model of the form:

$f(x_1,x_2,$cdots,$x_d)=({\displaystyle \underset{i=1}{\overset{d}{}}}{w}_i{x}_i)+b$

$($a$)$ We can make our algebra and coding simpler by writing $f(x_1,x_2,$cdots,$x_d)=w^{T}x$ for vectors

$w$ and $x.$ But at first glance, this formulation seems to be missing the bias term $b$ from the

equation above. How should we define $x$ and $w$ such that the model includes the bias term?

Linear regression learns a model by minimizing the squared loss function $L,$ which is the sum across

all training data $\{(x_1,y_1),$cdots,$(x_N,y_N)\}$ of the squared difference between actual and predicted

output values:

$L(f)={\displaystyle \underset{i=1}{\overset{N}{}}}(y_i-w^T{x}_i{)}^2$

$($b$)$ SGD uses the gradient of the loss function to make incremental adjustments to the weight

vector $w.$ Derive the gradient of the squared loss function with respect to $w$ for linear

regression.gradient descent $($SGD$)$ is an important optimization tool in machine learning, used

every$-$ where from logistic regression to training neural networks. In this problem, you will be asked

to first implement SGD for linear regression using the squared loss function. Then, you will analyze

how several parameters affect the learning process.

Linear regression learns a model of the form:

$f(x_1,x_2,$cdots,$x_d)=({\displaystyle \underset{i=1}{\overset{d}{}}}{w}_i{x}_i)+b$

$($a$)$ We can make our algebra and coding simpler by writing $f(x_1,x_2,$cdots,$x_d)=w^{T}x$ for vectors

$w$ and $x.$ But at first glance, this formulation seems to be missing the bias term $b$ from the

equation above. How should we define $x$ and $w$ such that the model includes the bias term?