Assessing Proxies of Knowledge and Difficulty with Rubric-Based Instruments

The fields of psychometrics, economic education, and education have developed more statistically- valid methods of assessing knowledge and learning. These methods include item response theory, value-added learning models, and disaggregated learning. The focus of these methods are on multiple-choice or single response assessments. However, universities routinely assess knowledge through papers, thesis presentations, or other demonstrations of knowledge assessed with rubric rows (hereinafter “projects”). This paper presents a statistical approach to estimating a proxy for student ability and rubric row difficulty. Moreover, software has been developed so that practitioners can easily apply this method to their instruments. This approach can be used in researching education treatment effects, practitioners diagnosing and improving learning outcomes in their own classrooms, or estimating knowledge for administrative assessment. As an example, we have applied these new methods to projects in a large Labor Economics course