Dominance Analysis for Latent Variable Models: A Comparison of Methods With Categorical Indicators and Misspecified Models

Educational and Psychological Measurement, Ahead of Print.
Dominance analysis (DA) is a very useful tool for ordering independent variables in a regression model based on their relative importance in explaining variance in the dependent variable. This approach, which was originally described by Budescu, has recently been extended to use with structural equation models examining relationships among latent variables. Research demonstrated that this approach yields accurate results for latent variable models involving normally distributed indicator variables and correctly specified models. The purpose of the current simulation study was to compare the use of this DA approach to a method based on observed regression DA and DA when the latent variable model is estimated using two-stage least squares for latent variable models with categorical indicators and/or model misspecification. Results indicated that the DA approach for latent variable models can provide accurate ordering of the variables and correct hypothesis selection when indicators are categorical and models are misspecified. A discussion of implications from this study is provided.

The Trade-Off Between Factor Score Determinacy and the Preservation of Inter-Factor Correlations

Educational and Psychological Measurement, Ahead of Print.
Regression factor score predictors have the maximum factor score determinacy, that is, the maximum correlation with the corresponding factor, but they do not have the same inter-correlations as the factors. As it might be useful to compute factor score predictors that have the same inter-correlations as the factors, correlation-preserving factor score predictors have been proposed. However, correlation-preserving factor score predictors have smaller correlations with the corresponding factors (factor score determinacy) than regression factor score predictors. Thus, higher factor score determinacy goes along with bias of the inter-correlations and unbiased inter-correlations go along with lower factor score determinacy. The aim of the present study was therefore to investigate the size and conditions of the trade-off between factor score determinacy and bias of inter-correlations by means of algebraic considerations and a simulation study. It turns out that under several conditions very small gains of factor score determinacy of the regression factor score predictor go along with a large bias of inter-correlations. Instead of using the regression factor score predictor by default, it is proposed to check whether substantial bias of inter-correlations can be avoided without substantial loss of factor score determinacy using a correlation-preserving factor score predictor. A syntax that allows to compute correlation-preserving factor score predictors from regression factor score predictors, and to compare factor score determinacy and inter-correlations of the factor score predictors is given in the Appendix.

Identifying Disengaged Responding in Multiple-Choice Items: Extending a Latent Class Item Response Model With Novel Process Data Indicators

Educational and Psychological Measurement, Ahead of Print.
Disengaged responding poses a severe threat to the validity of educational large-scale assessments, because item responses from unmotivated test-takers do not reflect their actual ability. Existing identification approaches rely primarily on item response times, which bears the risk of misclassifying fast engaged or slow disengaged responses. Process data with its rich pool of additional information on the test-taking process could thus be used to improve existing identification approaches. In this study, three process data variables—text reread, item revisit, and answer change—were introduced as potential indicators of response engagement for multiple-choice items in a reading comprehension test. An extended latent class item response model for disengaged responding was developed by including the three new indicators as additional predictors of response engagement. In a sample of 1,932 German university students, the extended model indicated a better model fit than the baseline model, which included item response time as only indicator of response engagement. In the extended model, both item response time and text reread were significant predictors of response engagement. However, graphical analyses revealed no systematic differences in the item and person parameter estimation or item response classification between the models. These results suggest only a marginal improvement of the identification of disengaged responding by the new indicators. Implications of these results for future research on disengaged responding with process data are discussed.

Do diagnostic criteria for ME matter to patient experience with services and interventions? Key results from an online RDS survey targeting fatigue patients in Norway

Journal of Health Psychology, Ahead of Print.
Public health and welfare systems request documentation on approaches to diagnose, treat, and manage myalgic encephalomyelitis and assess disability-benefit conditions. Our objective is to document ME patients’ experiences with services/interventions and assess differences between those meeting different diagnostic criteria, importantly the impact of post-exertional malaise. We surveyed 660 fatigue patients in Norway using respondent-driven sampling and applied validated DePaul University algorithms to estimate Canadian and Fukuda criteria proxies. Patients on average perceived most interventions as having low-to-negative health effects. Responses differed significantly between sub-groups for some key interventions. The PEM score was strongly associated with the experience of most interventions. Better designed and targeted interventions are needed to prevent harm to the patient group. The PEM score appears to be a strong determinant and adequate tool for assessing patient tolerance for certain interventions. There is no known treatment for ME, and “do-no-harm” should be a guiding principle in all practice.