A Comparison of Person-Fit Indices to Detect Social Desirability Bias

Educational and Psychological Measurement, Volume 83, Issue 5, Page 907-928, October 2023.
Social desirability bias (SDB) has been a major concern in educational and psychological assessments when measuring latent variables because it has the potential to introduce measurement error and bias in assessments. Person-fit indices can detect bias in the form of misfitted response vectors. The objective of this study was to compare the performance of 14 person-fit indices to identify SDB in simulated responses. The area under the curve (AUC) of receiver operating characteristic (ROC) curve analysis was computed to evaluate the predictive power of these statistics. The findings showed that the agreement statistic [math] outperformed all other person-fit indices, while the disagreement statistic [math], dependability statistic [math], and the number of Guttman errors [math] also demonstrated high AUCs to detect SDB. Recommendations for practitioners to use these fit indices are provided.

Generalized Mantel–Haenszel Estimators for Simultaneous Differential Item Functioning Tests

Educational and Psychological Measurement, Volume 83, Issue 5, Page 1007-1032, October 2023.
The Mantel–Haenszel estimator is one of the most popular techniques for measuring differential item functioning (DIF). A generalization of this estimator is applied to the context of DIF to compare items by taking the covariance of odds ratio estimators between dependent items into account. Unlike the Item Response Theory, the method does not rely on the local item independence assumption which is likely to be violated when one item provides clues about the answer of another item. Furthermore, we use these (co)variance estimators to construct a hypothesis test to assess DIF for multiple items simultaneously. A simulation study is presented to assess the performance of several tests. Finally, the use of these DIF tests is illustrated via application to two real data sets.