Reliability generalization: lapsus linguae

This study examines the proposed Reliability Generalization (RG) method for studying reliability. RG employs the application of meta-analytic techniques similar to those used in validity generalization studies to examine reliability coefficients. This study explains why RG does not provide a proper research method for the study of reliability, including describing how reliability is not a singular metric but a family of coefficients that are not interchangeable, along with other issues, such as sample and test administration. This research used Monte Carlo simulations designed to illustrate how the same instrument, administered repeatedly, can result in different reliability coefficients and to show that variation in reliability coefficients is due to sampling error; results illustrate that the reliability of a test will vary across test administrations based on the size and composition of the sample and how the sample was selected (randomly versus non-randomly).