1. Factorial clustering. Consider the following factorial model for clustering. There are K binary latent variables Zi E {0, 1}, and a real-valued observed vector Y E RM. The probability model is: K p( Z1, Z2 , . . . , 2 K | 7 ) = (1 - 12 ) 1-zi k=1 K p ( y | z1 , Z2 , . . . , Z K, M, 02) = N() zilli, 62 1), k=1 where each Mi is an M-dimensional vector and I is the identity matrix of dimension M x M. Let 0 = (#1, . .., MK, 71, . . ., TK, 62) denote the set of all of the parameters. Note that there are 2 possible configurations of the latent variables. Thus, for modest values of K, summing over all configurations of the latent variables is infeasible, and we need to consider approxi- mations. (a) Derive a mean-field approximation for inference in the factorial model. That is, consider a varia- tional distribution: K q( z1, Z2, . . ., ZK | X ) = II X5( 1 - X2 ) 1-zi, k=1 and derive a fixed-point algorithm that attempts to find values of the variational parameters {li} that minimize the KL divergence D(q(z | >) II p(z | y, 0)). (b) Derive a Gibbs sampler for inference in the factorial model. That is, show how to compute: p(zi | z1 , . . ., Zi -1, Zit1 , . . ., ZK, y, 0). (c) Derive the M step of an EM algorithm for this model. (d) Implement two versions of an approximate EM algorithm for this model, one using the mean-field algorithm for the E step, and the other using the Gibbs sampler for the E step. Hand in print-outs