Confirmatory factor analysis with sparse ordinal response data can make the familiar fit-index rules of thumb unreliable, according to a Monte Carlo simulation published in Behavior Research Methods. Liu Liu, Jin Liu, and David Okech calibrated cutoffs for the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR) across 324 simulated design cells and found that scenario-specific thresholds often diverged sharply from the conventional benchmarks of CFI and TLI at .95, RMSEA at .06, and SRMR at .08.
The simulation generated data from a three-factor population model with 15 five-category ordinal indicators, each factor measured primarily by five indicators, and one prespecified cross-loading of indicator u6 on Factor 1. Two loading regimes were examined, with all primary loadings set to .50 in the moderate-loading condition and .80 in the high-loading condition. Factor variances were fixed at 1 and all pairwise factor correlations at .40. For each generated dataset, the researchers fitted three analysis models: a Correct model that retained the cross-loading, a Local Misfit model that omitted it while keeping the three-factor structure, and a Severe Misfit model that combined the first two factors into a two-factor structure. Cutoffs were calibrated only from the Correct-model reference distributions, using lower-tail 5th-percentile quantiles for CFI and TLI and upper-tail 95th-percentile quantiles for RMSEA and SRMR at α = .05.
The design crossed three estimators, robust maximum likelihood (MLR), mean- and variance-adjusted maximum likelihood (MLMV), and weighted least squares mean- and variance-adjusted (WLSMV), with sample sizes of 250, 500, and 1,000, two loading regimes, original versus collapsed category handling, three sparsity patterns, and three sparse-item scopes of 1, 2, or 4 items per factor. This produced 324 full design cells for each fitted-model condition and 972 fitted model-by-design-cell combinations. The researchers requested 1,000 replications per cell, and the number of replications with available fit-index results ranged from 819 to 1,000 across fitted model-by-design-cell combinations.
The calibrated RMSEA cutoffs were consistently well below the conventional .06 benchmark. With original categories and moderate loadings, the MLR cutoff decreased from .0386 at N = 250 to .0248 at N = 500 and .0167 at N = 1,000, while the MLMV cutoff decreased from .0319 to .0223 and .0158. SRMR cutoffs were also smaller than the conventional .08 benchmark, typically falling between approximately .02 and .07 depending on sample size, loading strength, estimator, and category handling. The researchers attribute these shifts to the information structure of sparse ordinal CFA: in lower-information settings such as smaller samples, moderate loadings, and sparse original categories, fit-index distributions were more dispersed, whereas in higher-information settings such as larger samples and high loadings, incremental-index distributions concentrated closer to their upper boundary.
CFI and TLI cutoffs varied strongly with loading strength and sample size. Under high loadings, calibrated incremental-index cutoffs exceeded .95 even at N = 250 and increased as N increased; with original categories, the CFI cutoff rose from .9657 for MLR and .9652 for MLMV at N = 250 to .9858 and .9843, respectively, at N = 1,000, while WLSMV yielded still higher values, increasing from .9901 to .9980. Under moderate loadings at N = 250, robust ML cutoffs fell below .95 in the original-category condition, at .8877 for MLR and .9015 for MLMV. TLI followed the same general pattern and was typically slightly lower than CFI within the same scenario. These results show that the conventional .95 reference point occupied different positions relative to the Correct-model distributions across scenarios.
Detection depended strongly on the form of misspecification. Under moderate loadings, the CFI-RMSEA AND rule, which judged fit acceptable when both CFI and RMSEA met their calibrated thresholds, produced a mean false-rejection rate of .080 across estimator, category-handling, and sample-size scenarios. Mean detection was .391 for Local Misfit and .903 for Severe Misfit, with Local Misfit detection ranging from .092 to .740 and Severe Misfit detection ranging from .619 to .998. The researchers note that detection increased strongly with sample size, especially for the Local Misfit condition, and that the same calibrated rule can have very different sensitivity depending on whether the misspecification is localized or severe.
Single-index decisions showed distinct operating profiles. Under moderate loadings, CFI detection increased with sample size for both Local and Severe Misfit; under MLR with original categories, Local Misfit detection rose from .119 at N = 250 to .533 at N = 1,000, while Severe Misfit detection climbed from .644 to .997. WLSMV showed the lowest CFI detection in the original-category, N = 250, moderate-loading setting, with Local Misfit detection of .065 and Severe Misfit detection of .375, but collapsing categories raised these to .129 and .744. RMSEA showed stable false rejection near the nominal reference in most moderate-loading scenarios and increasing detection as N increased, while SRMR had higher false rejection than the other single-index decisions, with a mean of .169 and a range from .074 to .234, a pattern most pronounced for WLSMV.
Category collapsing had estimator-specific effects. For MLR and MLMV, paired collapsed-minus-original contrasts were generally small, especially at larger sample sizes and under high loadings, and the most consistent change was a modest reduction in false rejection; in overall averages across both loading regimes, false rejection for the CFI-RMSEA AND rule decreased from .144 to .122 for MLR and from .144 to .123 for MLMV, with detection remaining similar. For WLSMV, the clearest change was in detection: Severe Misfit detection increased from .867 under original categories to .925 under collapsed categories in the moderate-loading panel, and from .933 to .963 across both loading regimes, with the largest gains in lower-information settings where sparse bivariate information was most consequential.
To support applied use, the researchers fitted quantile regression models to the scenario-level cutoffs as an interpolation aid. Under the log10(N) specification, leave-one-scenario-out median absolute errors were approximately .053 for CFI, .065 for TLI, and .005 for RMSEA and SRMR, indicating that the prediction equations are best suited to rough interpolation when an exact scenario match is unavailable. The team also demonstrated the workflow on six household welfare perception items from the Sierra Leone baseline household survey, an analytic sample of N = 2,989, where the original five-category scale showed sparse marginal and bivariate response patterns and the collapsed three-category version removed these diagnostics and reduced RMSEA from .079 to .057.
The authors emphasize that the tabulated cutoffs are most informative for applications that closely resemble the simulated structure, a three-factor ordinal CFA with five-category indicators, the studied range of sample sizes and loading strengths, and similar sparse response patterns. Different factor structures, numbers of items, factor correlations, response scales, sparsity mechanisms, estimators, or misspecification types may produce different reference distributions and operating characteristics. For substantially different designs, they recommend a tailored scenario-specific simulation based on a substantively plausible and empirically adequate working model, and they advise researchers to report fit decisions together with the relevant decision rule and its operating-characteristic profile, including the model chi-square test alongside descriptive fit indices.
Subject of Research: Scenario-specific calibration of CFA fit-index cutoffs under sparse ordinal response conditions
Article Title: Calibrating fit-index cutoffs for sparse ordinal CFA: Scenario-specific thresholds and operating characteristics
Article References: Liu, L., Liu, J., & Okech, D. (2026). Calibrating fit-index cutoffs for sparse ordinal CFA: Scenario-specific thresholds and operating characteristics. Behavior Research Methods, 58(12), Article 341. https://doi.org/10.3758/s13428-026-03221-8
Image Credits: AI Generated
DOI: 10.3758/s13428-026-03221-8
Keywords: confirmatory factor analysis, fit indices, ordinal data, sparse categories, Monte Carlo simulation, cutoff calibration, CFI, RMSEA, SRMR, WLSMV, category collapsing, operating characteristics
Cite Scienmag News
Glenn Wilkins. (November 4, 2026). Fit-Index Cutoffs Shift Dramatically in Sparse Ordinal CFA, Simulation Shows. Scienmag. https://scienmag.com/fit-index-cutoffs-shift-dramatically-in-sparse-ordinal-cfa-simulation-shows/
Glenn Wilkins. "Fit-Index Cutoffs Shift Dramatically in Sparse Ordinal CFA, Simulation Shows." Scienmag, 4 November 2026, https://scienmag.com/fit-index-cutoffs-shift-dramatically-in-sparse-ordinal-cfa-simulation-shows/. Accessed 17 November 2026.
Glenn Wilkins. "Fit-Index Cutoffs Shift Dramatically in Sparse Ordinal CFA, Simulation Shows." Scienmag. November 4, 2026. https://scienmag.com/fit-index-cutoffs-shift-dramatically-in-sparse-ordinal-cfa-simulation-shows/








