Largest Singular Value :

Assume A e Mmxn (F) and A is not identically zero. Prove that (i) || A||2 = 01, where 01 is the largest singular value of A; (ii) if A is invertible, then ||A ? ||2 = onl; (iii) ||AH||2 = ||AT||2 = || AHA||2 = || A||2; (iv) if U E Mm (F) and V e Mn (F) are orthonormal, then ||U AV|| 2 = || A||2. 1 Singular values Let A be an mxn matrix. Before explaining what a singular value decom- position is, we first need to define the singular values of A. Consider the matrix AT A. This is a symmetric n x n matrix, so its eigenvalues are real. Lemma 1.1. If I is an eigenvalue of AT A, then 1 > 0. Proof. Let x be an eigenvector of AT A with eigenvalue 4. We compute that || Ax ||2 = (Ax) (Ax) = (Ax)? Ax = x+ A+ Ax = x+ (1x) = 4x+ x = = 1||2||2. Since || Ax||2 > 0, it follows from the above equation that 1 || 2 ||2 > 0. Since || 2||2 > 0 (as our convention is that eigenvectors are nonzero), we deduce that > 0. Let 11, ..., In denote the eigenvalues of AT A, with repetitions. Order these so that li > 12 > ... > In > 0. Let oi = di, so that 01 > 02 > > on>0. Definition 1.2. The numbers 01 > 02 > > on > 0 defined above are called the singular values of A. Proposition 1.3. The number of nonzero singular values of A equals the rank of A. Proof. The rank of any square matrix equals the number of nonzero eigen- values (with repetitions), so the number of nonzero singular values of A equals the rank of AT A. By a previous homework problem, AT A and A have the same kernel. It then follows from the "rank-nullity theorem that AT A and A have the same rank. Remark 1.4. In particular, if A is an mxn matrix with m 0. Proof. Let x be an eigenvector of AT A with eigenvalue 4. We compute that || Ax ||2 = (Ax) (Ax) = (Ax)? Ax = x+ A+ Ax = x+ (1x) = 4x+ x = = 1||2||2. Since || Ax||2 > 0, it follows from the above equation that 1 || 2 ||2 > 0. Since || 2||2 > 0 (as our convention is that eigenvectors are nonzero), we deduce that > 0. Let 11, ..., In denote the eigenvalues of AT A, with repetitions. Order these so that li > 12 > ... > In > 0. Let oi = di, so that 01 > 02 > > on>0. Definition 1.2. The numbers 01 > 02 > > on > 0 defined above are called the singular values of A. Proposition 1.3. The number of nonzero singular values of A equals the rank of A. Proof. The rank of any square matrix equals the number of nonzero eigen- values (with repetitions), so the number of nonzero singular values of A equals the rank of AT A. By a previous homework problem, AT A and A have the same kernel. It then follows from the "rank-nullity theorem that AT A and A have the same rank. Remark 1.4. In particular, if A is an mxn matrix with m <><>