$J={\displaystyle \underset{(a)}{\overset{?}{}}},$iinD$\frac{(Y_{ai}-[U{V}^T{]}_{ai}{)}^2}{2}+\frac{}{2}({\displaystyle \underset{a}{\overset{?}{}}},k{U}_{ak}^2+{\displaystyle \underset{i}{\overset{?}{}}},k{V}_{ik}^2)$

In order to break a big optimization problem into smaller pieces that we know how to

solve, we fix $U$ and find the best $V$ for that $U.$ But a subtle and important point is

that even if $V^{**}$ is best for $U,$ then the $U^{**}$ that is best for $V^{**}$ might not be the

original $U!$ It's like how we might be some lonely person's best friend, even though

they are not our best friend. In light of this, we repeat, like this: we fix $U$ and solve

for $V,$ then fix $V$ to be the result from the previous step and solve for $U,$ and repeat

this alternate process until we find the solution. This is an example of iterative

optimization, where we greedily take steps in a good direction, but as we do so the

context shifts so 'the good direction' evolves. Gradient descent, which we've already

seen, has the same structure.

Consider the case $k=1.$ The matrices $U$ and $V$ reduce to vectors $u$ and $v,$ such

that $u_a=U_{a1}$ and $v_i=V_{i1}.$

When $v$ is fixed, finding $u$ that minimizes $J$ becomes equivalent to finding $u$ that

minimizes $...$