The question on the table is whether big data—that is, the accumulation and analysis of massive quantities of information—will change our world, or whether it’s just another overhyped technology with a too-good-to-be-true story line. Silver is arguably peerless at interpreting data in the domains of sports and politics. Early in his career, he created an analytical model for baseball stats known as PECOTA, which did an exceedingly good job of identifying the minor-league prospects most likely to perform well in the majors. More recently, his FiveThirtyEight.com blog famously analyzed polling and economic data to predict the results of the 2008 presidential election (calling 49 or 50 states correctly) and the 2012 election (going 50 for 50). He has since dabbled in predictions of Oscar winners, NCAA basketball champions, and the geographic distribution of support for gay marriage. For at least the past five years, his methods and models have been questioned, doubted, and ridiculed. But in response he has shown every doubter, with almost unerring consistency, that a super nerd with a big data set and a killer algorithm can be a winner.

So is big data going to change the world or not? “The revolutions we recognize in retrospect,” Silver says, “aren’t usually the ones we recognize in advance.” He’s right, as usual, but he’s not precisely addressing the question. We live in a complex world that barely makes sense. Often, we expect too much of computers and not enough of ourselves. “People blame the data,” he notes, “when they should be asking better questions.” 

Silver is quick to point out that the most familiar, and arguably most successful, applications of big data involve National Weather Service predictions and hurricane warnings, which rely on huge data sets and wizardly models and have become increasingly accurate and precise. But other familiar examples abound, too. The quants on Wall Street have been helping hedge funds interpret complex trading data for years. Watson, the IBM computer that won at Jeopardy! and is now being applied to medical treatment and financial planning, is a success with a certain kind of big data—“unstructured data” as IBM likes to call it, which describes information formatted as natural language rather than numerical figures. Palantir, a wilfully obscure company that crunches big data in the name of national security, is another. Above all are Amazon, Facebook, Google, and Twitter, which stand as the foremost practitioners at making informed conclusions from customer data. 

Questions 

1. Will big data analytics put marketing researchers out of business? Why?

2. Go to Nate Silver’s blog, www.fivethirtyeight.com and tell the class what topics he is currently discussing.

3. The case discusses several current nonbusiness uses of big data. Can you think of other nonbusiness problems that big data analytics might solve?

4. What factors have led to the big data era?