A sampling dispute goes to court. Sampling of Medicare and Medicaid claims by the federal and state agencies who administer those programs has become common practice to determine whether providers of those services are submitting valid claims. (See the Statistics in Action for this chapter.)

The reliability of inferences based on those samples depends on the methodology used to collect the sample of claims. Consider estimating the true proportion, p, of the population of claims that are invalid. (Invalid claims should not have been reimbursed by the agency.) Of course, to estimate a binomial parameter, p, within a given level of precision, we use the formula provided in Section 7.5 to determine the necessary sample size. In a recent actual case, the statistician determined a sample size large enough to ensure that the bound on the error of the estimate would not exceed .05, using a 95% confidence interval. He did so by assuming that the true error rate was p = .5, which, as discussed in Section 7.5, provides the maximum sample size needed to achieve the desired bound on the error.

a. Determine the sample size necessary to estimate p to within .05 of the true value using a 95% confidence interval.

b. After the sample was selected and the sampled claims were audited, it was determined that the estimated error rate was pn = .20 and a 95% confidence interval for p was (.15, .25). Was the desired bound on the error of the estimate met?

c. An economist hired by the Medicare provider noted that, since the desired bound on the error of .05 is equal to 25% of the estimated pn = .20 invalid claim rate, the “true” bound on the error was .25, not .05. He argued that a significantly larger sample would be necessary to meet the “relative error” (the bound on the error divided by the error rate) goal of .05, that the statistician’s use of the “absolute error” of .05 was inappropriate, and that more sampling was required. The statistician argued that the relative error is a moving target because it depends on the sample estimate of the invalid claim rate, which cannot be known prior to selecting the sample. He noted that if the estimated invalid claim rate turned out to be larger than .5, the relative error would then be lower than the absolute error bound. As a consequence, the case went to trial over the relative versus absolute error dispute. Give your opinion on the matter.

[Note: The court concluded that “absolute error was the fair and accurate measure of the margin of error.”

As a result, a specified absolute bound on the error continues to be the accepted method for determining the sample size necessary to provide a reliable estimate of Medicare and Medicaid providers’ claim submission error rates.]