Covariance, z$-$score, whitening, and PCA

PCA.pngThe image above shows a dataset of two dimensions:

$1.[2$ points$]$ Covariance

Please compute the covariance matrix C of the above data set. Here, we use C to denote a covariance matrix to prevent any confusion. Recall:

where $$ is the $2-$dimensional mean vector of the four instances, and d$,$ d$\text{'}$ represent the $1$st and $2$nd dimensions of the data, respectively. Please make sure you divide by N $($i$.$e$.,$ the number of data instances, $4)$ rather than N $1.$

$2.[2$ points$]$ Z$-$score normalization

Please compute the vectors

after applying Z$-$score normalization to the dataset.

Recall that Z$-$score normalization is calculated by:

where $$ is the $2-$dimensional mean vector of the four instances, and $\backslash$sigma d is the standard deviation of the same. Don't forget, Z$-$score normalization is calculated for all dimensions of the data independently.

Please make sure you divide by N $($i$.$e$.,$ the number of data instances, $4)$ rather than N $1$ in computing the standard deviation $\backslash$sigma d$.$ That is$,$ you should use the population standard deviation as shown in the slides, not the sample standard deviation.

$3.[2$ points$]$ Unbiased and uniformly scaled

What is the $2-$dimensional mean vector and what is the standard deviation of each dimension of the resulting

$?$

A dataset with four data instances. Each data instance is $2-$

dimensional.

The image above shows a dataset of two dimensions:

The mean vector $$ of the data set is:

And the covariance C is:

$4.[2$ points$]$ Whitening

Please applying whitening to the dataset. Recall, the formula of whitening is

What are the resulting vectors

$?$