"First, we need some data to mine. We'll use the annual closing price of the S&P 500 index for the ten years from 1983 to 1993...[Then] we're going to go find some data to mine in a big CD-ROM database of international data series published by the United Nations. There are all sorts of data series from all 140 member countries in here [over 10,000 unique sets]. If we were trying to do this S&P 500 fit for real, we might look at things like changes interest rates, economic growth, unemployment and the like, but we'll stay away from those. We found something even better: butter production in Bangladesh. Yes, there it is. A simple single dairy product that explains 75% of the variation in the S&P 500 over ten years. R2 is 0.75, not bad at all."

"Why stop here? Maybe we can do better. Let's go global on this and expand our selection of dairy products: we'll use put in cheese and include US production as well. This works remarkably well. We're up to 95% accuracy here? How much better can we do? How about 99% with our third variable: sheep population. This is an awesome fit. It seems too good to be true, but it is. It is utterly useless for anything outside the fitted period, a total crock before 1983 or after 1993. Just a chance association, which would inevitably show up if you look at enough data series, as we did."

Question 1

Briefly discuss the impact of the following to explain their results: the number of models tested, the sample size, and time correlation

Question 2

Briefly explain how out-of-sample back testing protects a researcher from making these kinds of mistakes