Big data for twenty-first-century economic statistics

"The measurement infrastructure for the production of economic statistics in the United States largely was established in the middle part of the 20th century. As has been noted by a number of commentators, the data landscape has changed in fundamental ways since this infrastructure was developed. Obtaining survey responses has become increasingly difficult, leading to increased data collection costs and raising concerns about the quality of the resulting data. At the same time, the economy has become more complex and users are demanding ever more timely and granular data. In this new environment, there is increasing interest in alternative sources of data that might allow the economic statistics agencies to better address users' demands for information. Recent years have seen a proliferation of natively digital data that have enormous potential for improving economic statistics. These include item-level transactional data on price and quantity from retail scanners or companies' internal systems, credit card records, bank account records, payroll records and insurance records compiled for private business purposes; data automatically recorded by sensors or mobile devices; and a growing variety of data that can be obtained from websites and social media platforms. Staggering volumes of digital information relevant to measuring and understanding the economy are generated each second by an increasing array of devices that monitor transactions and business processes as well as track the activities of workers and consumers. Incorporating these non-designed Big Data sources into the economic measurement infrastructure holds the promise of allowing the statistical agencies to produce more accurate, more timely and more disaggregated statistics, with lower burden for data providers and perhaps even at lower cost for the statistical agencies. The agencies already have begun to make use of novel data to augment traditional data sources. Modern data science methods for using Big Data have advanced sufficiently to make the more systematic incorporation of these data into official statistics feasible. Indeed, the availability of new sources of data offers the opportunity to redesign the underlying architecture of official statistics. Considering the threats to the current measurement model arising from falling survey response rates, increased survey costs and the growing difficulties of keeping pace with a rapidly changing economy, fundamental changes in the architecture of the statistical system will be necessary to maintain the quality and utility of official statistics. This volume presents cutting edge research on the deployment of big data to solve both existing and novel challenges in economic measurement. The papers in this volume show that it is practical to incorporate big data into the production of economic statistics in real time and at scale. They report on the application of machine learning methods to extract usable new information from large volumes of data. They also lay out the challenges-both technical and operational-to using Big Data effectively in the production of economic statistics and suggest means of overcoming those challenges. Despite these challenges and the significant agenda for research and development they imply, the papers in the volume point strongly toward more systematic and comprehensive incorporation of Big Data to improve official economic statistics in the coming years"--