THEOREM 4.5 (Best Low-rank Approximation). Let Y ERixn2, and consider the following optimization problem min ||X - Y||F, (4.2.10) subject to rank(X) On: 4.3 (Best Rank-r Approximation). We prove Theorem 4.5. First, consider the special case in which Y diag(01,...,On) with 01 > 02 > An arbitrary rank-r matrix X can be expressed as X = FG* with F ERMix? F* F = I and GERn2x. 1 Argue that for any fixed F, the solution to the optimization problem min ||FG* ||| GERn2 Xr (4.7.2) is given by = L*F, and the optimal cost is ||(1 - FF*)|12 (4.7.3) 2 Let P= I FF*, and write vi = ||Pei||2. Argue that Li-i Vi = ni - r and Vi (0,1). Conclude that ni n1 ||PE||* = ovi> o, (4.7.4) i=1 i=r+1 = Cn: with equality if and only if vi = V2 = ... = Vr = 0 and Vr+1 Conclude that Theorem 4.5 holds in the special case Y = ?. 3 Extend your argument to the situation in which the d are not distinct (i.e., Oi = 0;+1 for some i). 4 Extend your argument to any Y E Rnxn. Hint: use the fact that the Frobe- nius norm ||M||F is unchanged by orthogonal transformations of the rows and columns: ||M|F = ||RMSE for any orthogonal matrices R, S. THEOREM 4.5 (Best Low-rank Approximation). Let Y ERixn2, and consider the following optimization problem min ||X - Y||F, (4.2.10) subject to rank(X) On: 4.3 (Best Rank-r Approximation). We prove Theorem 4.5. First, consider the special case in which Y diag(01,...,On) with 01 > 02 > An arbitrary rank-r matrix X can be expressed as X = FG* with F ERMix? F* F = I and GERn2x. 1 Argue that for any fixed F, the solution to the optimization problem min ||FG* ||| GERn2 Xr (4.7.2) is given by = L*F, and the optimal cost is ||(1 - FF*)|12 (4.7.3) 2 Let P= I FF*, and write vi = ||Pei||2. Argue that Li-i Vi = ni - r and Vi (0,1). Conclude that ni n1 ||PE||* = ovi> o, (4.7.4) i=1 i=r+1 = Cn: with equality if and only if vi = V2 = ... = Vr = 0 and Vr+1 Conclude that Theorem 4.5 holds in the special case Y = ?. 3 Extend your argument to the situation in which the d are not distinct (i.e., Oi = 0;+1 for some i). 4 Extend your argument to any Y E Rnxn. Hint: use the fact that the Frobe- nius norm ||M||F is unchanged by orthogonal transformations of the rows and columns: ||M|F = ||RMSE for any orthogonal matrices R, S