Dataset

You will work with the colon.csv file, which contains gene data from each patient. The dataset includes various gene expression measurements $($features$)$ and a label indicating the stage information.

Part $1$: Principal Component Analysis $($PCA$)$

$1\mathrm{.}1$ Implement PCA from Scratch:

a$.$ Write Python code to implement PCA from scratch. Include functions to compute the covariance matrix, eigenvalues, and eigenvectors.

b$.$ Apply your PCA implementation to reduce the dimensionality of the features in colon.csv$.$

c$.$ Choose an appropriate number of principal components to retain a significant amount of variance $($e$.$g$.,95\%).$

$1\mathrm{.}2$ PCA using scikit$-$learn:

a$.$ Use the PCA module from sklearn to perform dimensionality reduction on the dataset.

b$.$ Compare the results with your from$-$scratch implementation in terms of explained variance and the reduced feature set.

Part $2$: Kernel PCA $($KPCA$)$

$2\mathrm{.}1$ KPCA with RBF Kernel:

a$.$ Implement Kernel PCA with the Radial Basis Function $($RBF$)$ kernel from scratch.

b$.$ Apply your KPCA implementation to the dataset.

$2\mathrm{.}2$ KPCA with Polynomial Kernel:

a$.$ Implement Kernel PCA with a Polynomial kernel from scratch.

b$.$ Apply your KPCA implementation to the dataset.

$2\mathrm{.}3$ KPCA with Linear Kernel:

a$.$ Implement Kernel PCA with a Linear kernel from scratch.

b$.$ Apply your KPCA implementation to the dataset.

$2\mathrm{.}4$ Combining Kernels:

a$.$ Combine two different kernels $($e$.$g$.,$ RBF and Polynomial$)$ and apply the combined KPCA to the dataset.

b$.$ Evaluate the classification performance using accuracy metrics for the combined kernels.

Part $3$: Testing and Evaluation

Split your colon.csv into Train and Test datasets.

$3\mathrm{.}1$ Applying PCA and KPCA to the Test Dataset:

a$.$ Use the PCA and KPCA models $($RBF$,$ Polynomial, Linear, and combined kernels$)$ trained on the Train dataset to transform the Test dataset.

b$.$ Ensure the dimensionality reduction is consistent with what was performed on the training data.

$3\mathrm{.}2$ Covariance Matrix Analysis:

a$.$ Calculate the covariance matrix of the dataset.

b$.$ Identify the top $10$ features with the highest covariance values.

c$.$ Extract these top $10$ features and evaluate the classification performance using accuracy metrics.

$3\mathrm{.}3$ Classification Experiment:

a$.$ Choose a classifier $($minimum distance classifier: provided at the end of this assignment$)$ to classify the observations in the Test dataset.

b$.$ Evaluate the classification performance using accuracy metrics and compare the effectiveness of PCA and KPCA features.