Using State Administrative Data to Measure Program Performance

This paper uses administrative data from Missouri to examine the sensitivity of job training program impact estimates based on alternative nonexperimental methods. In addition to simple regression adjustment, we consider Mahalanobis distance matching and a variety of methods using propensity score matching. In each case, we consider estimates based on levels of post-program earnings as well as difference-in-difference estimates based on comparison of pre and post-program earnings. Specification tests suggest that the difference-in-difference estimator may provide a better measure of program impact. We find that propensity score matching is generally most effective, but the detailed implementation of the method is not of critical importance. Our analyses demonstrate that existing data available at the state level can be used to obtain useful estimates of program impact.