2 Maximum Likelihood Estimation [40 pts] This problem explores maximum likelihood estimation (MLE), which is a technique for estimating an unknown parameter of a probability distribution based on observed samples. Suppose we observe the values of n i.i.d.1 random variables X1, . . . , Xn drawn from a single Bernoulli distribution with parameter . In other words, for each Xi, we know that: P(Xz-=1)=6 and P(X.,-=0)=16 Our goal is to estimate the value of 0 from these observed values of X1 through X\". For any hypothetical value 9, we can compute the probability of observing the outcome X1, . . . , Xn if the true parameter value 0 were equal to 9. This probability of the observed data is often called the data likelihood, and the function LG?) = P(X1, ...., Xn|9) that maps each El to the corresponding likelihood is called the likelihood function. A natural way to estimate the unknown parameter 9 is to choose the a that maximizes the likelihood function. Formally, @MLE = argmaxL() a Often it is more convenient to work with the log likelihood function Md) = logL(). Since the log function is increasing, we also have: MLE = argmaxl(9) 6 1. [8 Points] Write a formula for the log likelihood function, 15(3). Your function should depend on the random variables X1, . . . , X\2. [8 Points] Consider the following sequence of 10 samples: X = (0, 1,0, 1, 1, 0, 0, 1, 1, 1). Compute the maximum likelihood estimate for the 10 samples. Show all of your work (hint: recall that if 27* maximizes f (as), then f'(:r*) = 0). 1iid means Independent, Identically Distributed 3. [8 Points] Now we will consider a related distribution. Suppose we observe the values of m iid random variables Y1 ,....,Ym drawn from a single Binomial distribution B (n, 6). A Binomial distribution models the number of 1's from a sequence of n independent Bernoulli variables with parameter. In other words, )aku mn-k = _k!(n\"i k)! .oku arm-'6 n Pm=k)=(k 3. [8 Points] Now we will consider a related distribution. Suppose we observe the values of m iid random variables Y1 ,....,Ym drawn from a single Binomial distribution B (n, 6). A Binomial distribution models the number of 1's from a sequence of n independent Bernoulli variables with parameter. In other words, Pm- = k) = ('2') 6\"\"(l 0)\" = Mun:19)! -6'\"(1 0)"'\" Write a formula for the log likelihood function, 09). Your function Should depend on the random variables Y1, . . . , Ym and the hypothetical parameter (9. 4. [8 Points] Consider two Binomial random variables Y1 and Yg with the same parameters, n = 5 and 9. The Bernoulli variables for Y1 and Y2 resulted in (0, 1, 0, 1, 1) and (0, 0, 1, 1, 1), respectively. Therefore, Y1 = 3 and Yg = 3. Compute the maximum likelihood estimate for the 2 samples. Show your work. 5. [8 Points] How do your answers for parts 1 and 3 compare? What about parts 2 and 4? If you got the same or different answers, why was that the case