Creating new administrative data to describe the scientific workforce: The STAR METRICS program

In common with many countries, the substantial United States investment in R&D is characterized by limited documentation of the nature and results of those investments (MacIlwain 2010, Marburger 2005). Despite the increased calls for reporting by key stakeholders, current data systems cannot meet the new requirements; indeed, the conclusion of the Science of Science Policy interagency group's Federal Research Roadmap (National Science and Technology Council 2008) was that the science policy data infrastructure was inadequate for decision-making. In response to this need, a new data system is being built (STAR METRICS) drawing from administrative records; this paper describes the initial results of that effort - focusing on documenting the scientific workforce supported by expenditures during the 2011 Federal fiscal year from awards made by the National Science Foundation. The contribution of the paper is threefold. First it describes in a non-technical fashion how these new data can contribute to our understanding of the initial results of science investments. Second, it shows how new computational technologies can be used to go beyond the traditional methods of manual reporting and administrative program coding to capture information at the most granular units of analysis possible. Finally, it discusses the lessons learned for the collection and analysis of data. The most important is leveraging existing data, not relying on surveys and manual reporting; the deficiencies of each have been well documented (Lane 2010).