9.1 Comparing Bayesian and frequentist estimators for T. In Chapter 1 we learned that the frequentist procedure for evaluating a statistical procedure, namely looking at how it performs in the long-run, for a (range of) fixed but unknown parameter values can also be used to evaluate a Bayesian statistical procedure. This "what if the parameter has this value " type of analysis would be done before we obtained the data and is called a pre-posterior analysis. It evaluates the procedure by seeing how it performs over all possible random samples, given that parameter value. In Chapter 8 we found that the posterior mean used as a Bayesian estimator minimizes the posterior mean squared error.

Thus it has optimal post-data properties, in other words after making use of the actual data. We will see that Bayesian estimators have excellent pre-data

(frequentist) properties as well, often better than the corresponding frequentist estimators.

We will perfom a Monte Carlo study approximating the sampling distribution of two estimators of 7r. The frequentist estimator we will use is +irf = $, the sample proportion. The Bayesian estimator we will use is +~TB = s, which equals the posterior mean when we used a uniform prior for 7r. We will compare the sampling distributions (in terms of bias, variance, and mean squared error)

of the two estimators over a range of 7r values from 0 to 1. However, unlike the exact analysis we did in Section 9.3, here we will do a Monte Carlo study. For each of the parameter values, we will approximate the sampling distribution of the estimator by an empirical distribution based on 5000 samples drawn when that is the parameter value. The true characteristics of the sampling distribution (mean, variance, mean squared error) are approximated by the sample equivalent from the empirical distribution. You can use either Minitab or R for your analysis.

(a) For T = .1, .2,. . . , .9 i. Draw 5000 random samples from binomial (n = 10, T).
ii. Calculate the frequentist estimator i?f = $ for each of the 5000 samples.
iii. Calculate the Bayesian estimator f?~ = for each of the 5000 samples.
iv. Calculate the means of these estimators over the 5000 samples, and subtract T to give the biases of the two estimators. Note that this is a function of T.
v. Calculate the variances of these estimators over the 5000 samples.
Note that this is also a function of T.
vi. Calculate the mean squared error of these estimators over the 5000 samples. The first way is MS(f?) = (bius(f?))2 + Vur(;r).
The second way is to take the sample mean of the squared distance the estimator is away from the true value over all 5000 samples. Do it both ways, and see that they give the same result.

(b) Plot the biases of the two estimators versus 7r at those values and connect the adjacent points. (Put both estimators on the same graph.)
i. Does the frequentist estimator appear to be unbiased over the range ii. Does the Bayesian estimator appear to be unbiased over the range of

(c) Plot the mean squared errors of the two estimators versus T over the range of T values, connecting adjacent points. (Put both estimators on the same of T values?
the T values?
graph.)
i. Does your graph resemble Figure 9.2?
ii. Over what range of T values does the Bayesian estimator have smaller mean squared error than that of the frequentist estimator?