Suppose that the pdf of $y$ has a $C$-component mixture form, $f(y\mid \pi )=$ $\sum _{j=1}^c{\pi }_j{f}_j(y)$, where $\pi =({\pi }_1,\dots ,{\pi }_C),{\pi }_j>0,\sum _{j=1}^c{\pi }_j=1$. The ${\pi }_j$ are unknown mixing proportions whereas the parameters of the densities $f_j(y)$ are presumed known.

(a) Given a random sample on $y_i,i=1,\dots ,N$, write the general log-likelihood function and obtain the first-order conditions for ${\hat{\pi }}_{\text{ML }}$. Verify that there is no explicit solution for ${\hat{\pi }}_{ML}$.

(b) Let ${\mathbf{z}}_i$ be a $C\times 1$ vector of latent categorical variables, $i=1,\dots ,N$, such that $z_{ji}=1$ if $y$ comes from the $j$ th component of the mixture and $z_{ji}=0$ otherwise. Write down the likelihood function in terms of the observed and latent variables as if the latent variable were observed.

(c) Devise an EM algorithm for estimating $\pi$. [If $z_{ji}$ were observable the MLE of ${\hat{\pi }}_j=N^{-1}\sum _i{z}_{ji}$. The E step requires calculation of $\mathrm{E}[z_{ji}\mid y_i]$; the $\mathrm{M}$ step requires replacing $z_{ji}$ by $\mathrm{E}[z_{ji}\mid y_i]$ and then solving for $\pi$.]