Python only

In mathematics, gradient descent is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point because this is the direction of the steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent. Gradient descent is commonly used in machine learning and neural networks to minimize a cost/loss function. For this assignment, you are going to use graduate descent to optimize a linear regression model on the scikit-learns diabetes data.

Description: In mathematics, gradient descent is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point because this is the direction of the steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent. Gradient descent is commonly used in machine learning and neural networks to minimize a cost/loss function. For this assignment, you are going to use graduate descent to optimize a linear regression model on the scikit-learns diabetes dataset.

Directions: For this assignment, you need to build a linear regression model from scratch in Python and optimize the linear regression model using the gradient descent algorithm. Below is a detailed instruction on what you may need to do.

Dataset Preparation: You need to load the dataset using sklearn>datasets>load_diabetes. More information about the function can be found at a certain internet location (piece it together because Chegg doesnt like external sites in their questions):

*It's an org site called sklearn. Under the page >stable/modules/generated/ sklearn>datasets>load_diabetes>html, (replace > with .)

*Combine these into one line address. After loading the dataset, randomly shuffle the dataset to split the dataset to train/dev/test sets. Use the 70% of data for the train set, 15% for the dev set, and 15% for the test set. Make sure that the labels and features are still matching after shuffling the data. You may want to use the random shuffle function provided by NumPy.

Linear Regression Development: to implement a single-variable linear regression model from scratch. Use a gradient descent algorithm to solve the optimal parameters for the linear regression model. The gradient descent algorithm must be implemented from scratch. No high-level libraries can be used. Using online reference and tutorials are allowed. The dataset contains 10 features; however, for a single-variable linear regression model, you may only use one feature. Thus, to build the linear regression model, you need to decide which feature to you. There are three approaches you may use: You may train 10 single-variable linear regression models, one model for each feature, and select the one with the best performance on the dev set. You may run a feature selection algorithm to select a feature. There are plenty of feature selection algorithms available. Feel free to search on the internet and use a method at your choice. You may use PCA for dimension reduction to reduce the number of features to 1. There are many python libraries that provide PCA algorithms you may use. In fact, scikit-learn also has a PCA function. Feel free to search on the internet and choose one at your choice.

Test the Model: Test the model using the test set.

Submission: You need to submit a written report for this assignment. For this report, you need to explain what you have done, report the best performance on the test set (in terms of MSE), and include your code as an appendix.

Assignment must be in Python only.