Research report: Measuring change in mathematics enjoyment: longitudinal invariance, local dependence of negatively worded items, and complex-sample inference

Children's enjoyment of learning mathematics declined over a single school year in every one of nine education systems studied, according to a large-scale analysis of the TIMSS 2023 Longitudinal Study published in Frontiers in Psychology. The study, led by Georgios Sideridis of Boston Children's Hospital and Harvard Medical School and Mohammed Alghamdi of King Saud University, tracked more than 44,000 students from Grade 4 in 2023 to Grade 5 in 2024 and applied unusually strict statistical safeguards before interpreting any change. The result: latent mathematics enjoyment fell in all nine panels, with decrements ranging from 0.117 to 0.553 of a Grade 4 standard deviation, and every confidence interval excluded zero.

The nine participating education systems were Georgia, Italy, Jordan, Kosovo, Montenegro, North Macedonia, the Republic of Korea, Slovenia and Sweden. The researchers analyzed the public-use student databases of the TIMSS 2023 Longitudinal release, in which national centers reassessed the same sampled Grade 4 students one year later, following them even when they changed class or school. Of 44,325 linked records, 43,979 students had a positive sampling weight and at least one usable questionnaire response across the two occasions.

Item response rates ranged from 81.7 to 96.3 percent in 2023 and from 78.5 to 92.2 percent in 2024, and the pairwise sample sizes underlying the correlation inputs ranged from 3,242 to 5,390 students per system. Why all the statistical caution? Because an apparent drop in a questionnaire score is only interpretable as genuine developmental change if the instrument measures the same construct on both occasions.

If item thresholds, loadings, or response processes shift with age, an observed mean trajectory confounds real development with differential measurement. The team therefore tested longitudinal measurement invariance formally, using a sequence of ordinal confirmatory factor models: configural invariance, which establishes a common measurement pattern; threshold invariance, which aligns the category boundaries of the four-point agreement scale; metric invariance, which constrains how strongly each item relates to the latent enjoyment factor; and scalar invariance, which additionally aligns item locations and licenses comparison of latent means across occasions. A key technical innovation was the identification framework used for the ordered categorical items.

The researchers applied the approach developed by Wu and Estabrook in 2016, extended to repeated measures by Liu and colleagues in 2017, in which each level of invariance is defined independently of arbitrary baseline constraints. This matters because identification restrictions that are inconsequential in a single cross-sectional factor analysis can act as unintended restrictions when an entire invariance sequence is derived from one baseline parameterization, potentially distorting the test of whether the Grade 5 mean differs from the Grade 4 mean. The results were strikingly consistent.

All 36 longitudinal models, four invariance levels in each of nine systems, converged and passed admissibility checks. Scalar-model fit indices were excellent everywhere: comparative fit indices between 0.994 and 0.997, RMSEA between 0.023 and 0.053, and standardized root mean square residuals between 0.019 and 0.034. The largest absolute change in CFI across any successive invariance step was a negligible 0.00050.

In other words, constraining the seven positively worded items to function identically across the two school years cost essentially nothing in fit, so the seven-item core could legitimately be used to compare latent enjoyment across the Grade 4 to Grade 5 interval within each system. The study also delivered a pointed diagnostic about negatively worded items. The 2023 questionnaire included two reverse-worded statements, wishing one did not have to study mathematics and finding mathematics boring, which were dropped from the 2024 instrument.

When the researchers fitted a single common factor to all nine keyed 2023 items, the model performed markedly worse than a seven-item positive-only model. Freeing just one residual covariance between the two negatively worded items restored fit, and the standardized residual correlation between them ranged from 0.413 to 0.715 across systems. That is a substantial shared association persisting after the common enjoyment factor was accounted for, meaning the two items were not conditionally independent indicators of a single trait.

Because wording direction and their shared aversive content are confounded in this design, the authors could not determine whether reverse wording itself, the overlapping boredom and avoidance content, or both produced the dependence. Once scalar invariance was established, the team quantified change with a design-based variance procedure matched to TIMSS's complex sampling. TIMSS uses stratified multistage samples with students clustered within schools, and supplies jackknife zone and replicate indicators for every student.

The researchers re-estimated the full scalar latent variable model under every one of the 1,604 prescribed paired jackknife replicate weight vectors, so that the replicated statistic was the complete latent change estimator rather than a simple manifest mean. All 1,604 replicate models converged and were admissible, meaning no variance estimate depended on silently discarding failed runs. The resulting standard errors ranged from 0.021 to 0.054 and were not a simple function of sample size, reflecting variation in clustering, weight dispersion, missingness and measurement strength.

The headline developmental finding was a negative latent change in all nine systems. Georgia showed the largest decrement at minus 0.553 Grade 4 standard deviations, followed by Montenegro at minus 0.520, while the Republic of Korea showed the smallest at minus 0.117. Italy, Kosovo, North Macedonia, Slovenia and Sweden each fell between minus 0.25 and minus 0.37, and Jordan at minus 0.146.

The authors stress that each estimate is standardized on its own system's Grade 4 latent metric, and that no cross-country equality of change magnitudes was tested, since cross-national scalar invariance was not part of the analysis. The replicated negative direction is therefore a qualitative pattern, not a ranking of countries. Robustness checks reinforced the conclusion.

Latent change remained negative under all four response and weighting definitions tested, including complete-case restrictions requiring all seven items at both waves and an unweighted analysis, with the largest displacement from the primary estimate being just 0.048 standard deviations. Response-selection models, fitted with design-based jackknife variance and pooling five mathematics achievement plausible values, showed that students with higher baseline enjoyment were generally no more or less likely to respond at follow-up, with only Korea and North Macedonia showing small significant odds ratios. These diagnostics reduce, but do not eliminate, concern about nonresponse bias, and the authors note the analyses were not preregistered.

The broader lesson, the authors argue, is methodological as much as substantive. Developmental studies should define a content-matched longitudinal item core and test ordinal scalar invariance before interpreting change; mixed-key scales should be evaluated for local dependence rather than assumed independent after reverse scoring; and latent estimators should be carried through the supplied replicate-weight procedures when design-based uncertainty is required. The observed declines align with prior evidence of falling intrinsic motivation and task value across the school years, but the study followed a single cohort over one year without random assignment, so curriculum, instructional climate, assessment timing and post-pandemic conditions remain hypotheses for future research rather than established explanations.

For now, the message is clear: children's positive feelings about mathematics slipped measurably between fourth and fifth grade across nine diverse education systems, and the measurement machinery needed to detect that slip is now on display.

Subject of Research: Longitudinal measurement invariance and one-year decline in mathematics enjoyment among Grade 4 to 5 students in the TIMSS 2023 Longitudinal Study

Article Title: Measuring change in mathematics enjoyment: longitudinal invariance, local dependence of negatively worded items, and complex-sample inference

Article References: Sideridis, G., & Alghamdi, M. (2026). Measuring change in mathematics enjoyment: longitudinal invariance, local dependence of negatively worded items, and complex-sample inference. Frontiers in Psychology, 17, Article 1971121. https://doi.org/10.3389/fpsyg.2026.1971121

Image Credits: AI Generated

DOI: 10.3389/fpsyg.2026.1971121

Keywords: mathematics enjoyment, TIMSS 2023 Longitudinal Study, longitudinal measurement invariance, ordinal confirmatory factor analysis, negatively worded items, local dependence, jackknife repeated replication, complex sampling, Grade 4 to Grade 5, non-cognitive outcomes, academic motivation, psychometrics