Under the assumptions of Theorem 8.74, show that A B 2 is also minimized when B = A k among all matrices with rank

Under the assumptions of Theorem 8.74, show that ΙΙA − BΙΙ ² is also minimized when B = A_k among all matrices with rank B ≤ k.

Theorem 8.74

Theorem 8.74 says that, in the latter cases, the number of principal components, i.e.,  the number of significant singular values, will determine the approximate rank k of the  covariance matrix K and hence the data matrix A, also. Thus, the normalized data (approximately)  lies in a k-dimensional subspace. Further, the variance in any direction orthogonal  to principal directions is relatively small and hence relatively unimportant. As a  consequence, dimensional reduction by orthogonally projecting the data vectors onto the  k-dimensional subspace spanned by the principal directions (singular vectors) q₁, . . . , q_k, serves to eliminate significant redundancies.