Development of a differential item functioning (DIF) procedure using the hierarchical generalized liner model: a comparison study with logistic regression

Based on Kamatas item analysis model (2001), an extension for differential item functioning procedure was developed and the applicability was examined. The Kamatas item analysis model is a type of hierarchical generalized linear model for item analysis, in which items are considered as nested in examinees. Kamata showed that the item difficulty estimates and person ability estimates are mathematically equivalent to the Rasch model. Extending the Kamata model by adding matching variable and group membership variable and their interaction terms in the level-2 model, the approach to DIF, which is named the Hierarchical Generalized Linear Modeling DIF, was constructed. The main purpose of this study was to examine the equivalency of this procedure with a traditional DIF method, the logistic regression DIF method. An English test that consisted of 60 items with 4 sub-scales for freshmens diagnostic purpose was analyzed to answer the research questions in this study. For the extensive investigation of the two DIF methods, four different group membership variables were administrated; Male vs. Female, White vs. Asian, White vs. Black, Asian vs. Black. The comparisons were made with the coefficients for the group membership variable and interaction term. In addition to that, p-values of the regression coefficients were compared. The results of the empirical comparison show that the DIF outputs are highly similar to each other in both uniform DIF and non-uniform DIF. Beyond the similarity with the logistic regression DIF method, features of the HGLM DIF method and its strengths and weaknesses were discussed. Furthermore non-parametric examination of the DIF item by using the TestGraf program was considered in order to find the prominent difference in pattern of DIF identified by specific procedures