Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning

Two nonparametric procedures for detecting differentialitem functioning (DIF)-the Mantel-Haenszel(MH) procedure and the simultaneous item bias (SIB)procedure-were compared with respect to their Type Ierror rates and power. Data were simulated to reflectconditions varying in sample size, ability distributiondifferences between the focal and reference groups, proportionof DIF items in the test, DIF effect sizes, and typeof item. 1,296 conditions were studied. The SIB and MHprocedures were equally powerful in detecting uniformDIF for equal ability distributions. The SIB procedurewas more powerful than the MH procedure in detectingDIF for unequal ability distributions. Both procedureshad sufficient power to detect DIF for a sample size of300 in each group. Ability distribution did not have asignificant effect on the SIB procedure but did affect theMH procedure. This is important because ability distributiondifferences between two groups often are found inpractice. The Type I error rates for the MH statistic werewell within the nominal limits, whereas they wereslightly higher than expected for the SIB statistic. Comparisonsbetween the detection rates of the two procedureswere made with respect to the various factors.Index terms: differential item functioning, Mantel-Haenszel statistic, power, simultaneous item bias statistic,SIBTEST, Type I error rates.