COMPARING DIF DETECTION FOR MULTIDIMENSIONAL POLYTOMOUS MODELS USING MULTI GROUP CONFIRMATORY FACTOR ANALYSIS AND THE DIFFERENTIAL FUNCTIONING OF ITEMS AND TESTS

This study evaluated the robustness of DIF detection for multidimensional polytomous items using two different estimation methods, MG-CFA and MGRM-DFIT. A simulation study across 960 study conditions was performed. The purpose of this study was to establish the Type-I error rate and Power of DIF detection for the MG-CFA and MGRM-DFIT estimation methods across the study conditions. The MGRM-DFIT method consistently controlled Type-I error rate under alpha across all study conditions. Though the MGRM-DFIT method demonstrated high power in detecting DIF for the combined items, it had lower power in detecting DIF for each item individually. The MGRM-DFIT method had higher power of DIF detection when impact (true distributional differences) is in the opposite direction of manipulated DIF. Overall, compared to the non-DIF items, NCDIF values are larger, and CDIF values are smaller for the 4 DIF items. Across the replications and the study conditions, CDIF was not as consistent as NCDIF. The MG-CFA method demonstrated slightly inflated Type-I error rate in a couple of study conditions (particularly in the presence of impact). However, the MG-CFA method demonstrated lower power across all study conditions. This could partly be explained by the low magnitude of DIF that was manipulated in the 'α/λ' parameter in this study. Parameter estimation for the MGRM, and the MGRM-DFIT method should be incorporated as part of commonly used software packages. In general, the MG-CFA method is recommended for DIF detection with multidimensional polytomous types of items, since it performs more consistently as a univariate test and as a multivariate test, and is easily available as part of several commonly used software packages.