Bayesian Estimation and Linear Regression

Suppose that:

Y1,...,Yn are independent given the pair (0, 1)

Yi=0+1*Xi+ei, where each ei are i.i.d. N(0,1/) (which has variance 1/)

the Xi R are deterministic.

We will think of 0, 1 and as being random variables.

Suppose that we place an improper prior on (0, 1, ):

(0, 1, )=1/

Since this expression on the right hand side does not depend on the 's, we may take the conditional distribution (0, 1|) to be the uniform" improper prior: (0, 1|)=1.

Answer the following problems given these assumptions. As a reminder, we let be the design matrix, where the th row is the row vector (1,Xi), and let Y be the column vector ([[Y1]],...,[[Yn]]).

Suppose that: - Y1,,Yn are independent given the pair (0,1) - Yi=0+1Xi+i, where each i are i.i.d. N(0,1/) (which has variance 1/) - the XiR are deterministic. We will think of 0,1 and as being random variables. Suppose that we place an improper prior on (0,1,) : (0,1,)=1/ Since this expression on the right hand side does not depend on the 's, we may take the conditional distribution (0,1) to be the "uniform" improper prior: (0,1)=1 Answer the following problems given these assumptions. As a reminder, we let X be the design matrix, where the i th row is the row vector (1,Xi), and let Y be the column vector Y1Yn. (a) The Bayesian setup: The posterior distribution Observe that if 0,1 and are given, then each Yi is a gaussian: Yi(0,1,)N(0+1Xi,1/). Therefore, the likelihood function of the vector (Y1,,Yn) given (0,1,) is of the form (2/1)nexp(2i=1n(yi01Xi)2) It turns out that the distribution of (0,1) given and Y1,,Yn is a 2-dimensional Gaussian. In terms of X,Y and , what is its mean and covariance matrix? Hint: look ahead and see what part (b) is asking. What answer do you hope would come out, at least for one of these two things? (Type X for X, trans( X) for the transpose XT, and X(1) for the inverse X1 of a matrix X.) Suppose that: - Y1,,Yn are independent given the pair (0,1) - Yi=0+1Xi+i, where each i are i.i.d. N(0,1/) (which has variance 1/) - the XiR are deterministic. We will think of 0,1 and as being random variables. Suppose that we place an improper prior on (0,1,) : (0,1,)=1/ Since this expression on the right hand side does not depend on the 's, we may take the conditional distribution (0,1) to be the "uniform" improper prior: (0,1)=1 Answer the following problems given these assumptions. As a reminder, we let X be the design matrix, where the i th row is the row vector (1,Xi), and let Y be the column vector Y1Yn. (a) The Bayesian setup: The posterior distribution Observe that if 0,1 and are given, then each Yi is a gaussian: Yi(0,1,)N(0+1Xi,1/). Therefore, the likelihood function of the vector (Y1,,Yn) given (0,1,) is of the form (2/1)nexp(2i=1n(yi01Xi)2) It turns out that the distribution of (0,1) given and Y1,,Yn is a 2-dimensional Gaussian. In terms of X,Y and , what is its mean and covariance matrix? Hint: look ahead and see what part (b) is asking. What answer do you hope would come out, at least for one of these two things? (Type X for X, trans( X) for the transpose XT, and X(1) for the inverse X1 of a matrix X.)