4. Mixture models. We have introduced the EM-algorithm for mixture models in class. In this problem, we will apply the Gibbs sampler to the same problem. Our goal is to nd the clusters in our data X = {221, - . - ,s:,,} with :13,- E R. We use the mixture of three Gaussians to model the population distribution: 3 Elf-'1?) = Zedwmh. (1) k=1 where (-; a, 0'2) is the one-dimensional Gaussian density function with KNOWN variance 02 and takes the form 1 1 W) = New (W. _ M). For each data point 3.3-, we introduce a latent variable Z, which indicates the component 3:,- belongs to. This corresponds to the hard assignment in the EM-algorithm. Z,- is a discrete ran- dom variable, and can take values from the set {1, 2, 3}. Now, we consider an approximation to model (1): 3 19035) = Z Impede; mg. 02). (2) k=1 where ]l{Z=;,} is the indicator function taking value 1 when Z = k and being 0 otherwise. Under the Bayesian framework, the parameters and latent variables of model (2) are m, for k = 1, 2, 3, and Z,- for 1' = l, ' - - ,n. We impose the following prior on the parameters: at ~N(0,72) for k 21,2,3, and the prior on Z,- are independent and uniform across the three components, namely (Zr = k)=1/3 for k = 1,2,3. Now we use the Gibbs sampler to sample from the posterior p(,u1,u2,u3,Z1, .. - , Z,,|X). (a) (5 points) Given all the parameters and latent variables, what is the likelihood of our data X. (b) (10 points) Now we consider the posterior distribution. What is the conditional dis- tribution of each m, given the other parameters and latent variables? Does it have a closed-form expression? Are the conditional distributions of #1, rig, and #3 independent given Z,- 1s? (c) (10 points) What is the conditional distribution of each Z,- given ahmnug and the other latent variables? Are they independent given the means (#1! pm, as) of the three components