Statisticians have been in high demand for many years due to the rapid increase in corporate databases throughout the world. However, the rise of social media and mobile applications has created a flood of information content that overwhelms traditional data storage and analyses. In May 2013, independent research organization SINTEF stated that “a full 90 percent of all the data in the world has been generated over the last two years. The Internet companies are awash with data that can be grouped and utilized.”3 Companies like Yahoo, Google, Facebook, and eBay have had to invent new ways of capturing and processing the massive amount of information they receive on a daily basis. Much of this information lacks the form and structure of traditional databases, requiring new types of analysts and new methods of analysis. These analysts are referred to as data scientists because they work with a variety of different data elements in an exploratory fashion, much like an engineer or scientist. What once was megabytes and gigabytes of facts and figures is now terabytes, petabytes, and exabytes of data that include images, text, video, tweets, blogs, online and offline behaviors, global positioning coordinates, and so on.

One of the key tasks of a data scientist is to make sense of these disparate sources that cannot easily be linked together. All the raw data in the world, in and of itself, is not very useful until patterns and relationships can be identified. Data scientists must use cutting-edge tools, opensource programs, and custom programming to manipulate available information. Beyond these technical skills, they also must understand their company’s underlying business model in order to separate meaningful relationships from random patterns and coincidental correlations. Furthermore, data scientists need to be able to prepare clear and concise visuals that connect all these pieces into a coherent story that senior management can comprehend and act upon. Universities have not yet caught up to the requirements of this new role, so corporations are often training employees from within and drawing from other disciplines such as computer science, math, economics, biology, and other sciences. Introverts with limited social skills may be excellent database administrators or statisticians, but they may lack the communication skills required to interact with customers and various departments throughout an organization on their way to uncovering meaningful opportunities and solutions.

Questions

1. Where have you recently seen the use of statistics and data analysis where you might not have expected it? How was it being used?

2. Top companies are finding that many talented analysts and statisticians actually have backgrounds in other disciplines such as economics, mathematics, and computer sciences. How do you think these disciplines relate to and inform the approach to data analysis?