In this question, we will compare AIC and BIC under multicollinearity, changing standard deviation of random errors and changing sample size. We will use the generate.X function defined on page 10 of the notes of Regression Part 2 to generate the design matrix X.

generate.X=function(n, mu, sigma.X, rho) {

a.list=rnorm(n=n, mean=0, sd=(sqrt(rho)*sigma.X))

Xcol1=rep(1, n)

Xcol2=rnorm(n=n, mean=mu, sd=sigma.X)

Xcol3=mu + a.list + rnorm(n=n, mean=0, sd=(sqrt(1-rho)*sigma.X))

Xcol4=mu + a.list + rnorm(n=n, mean=0, sd=(sqrt(1-rho)*sigma.X))

X=cbind(Xcol1, Xcol2, Xcol3, Xcol4)

return(X)

}

(a) (5 points) We know generate.X will return a matrix with first column standing for intercept, second for X1, third for X2 and last for X3. List out all the eligible subset model.

(b) (6 points) Now let {0.25, 0.5, 0.98}. Let X = 1, n = 50, = 10. For each , generate the design matrix with generate.X. Then use reps=1000 realizations of data to estimate (1) the probability that AIC choose the true model and (2) the probability that BIC choose the true model. In each realization, use = (1, 1, 0, 1) and use N(0,4) to generate the random errors. Create a 95% score CI for those two probabilities.