Confidence Beats Enjoyment in Predicting Young Students’ Science Success, AI Study Finds

When it comes to predicting how well nine- and ten-year-olds will perform in science, how confident children feel about their abilities matters more than how much they enjoy the subject. That is the central finding of a new cross-national study that applied explainable machine learning to data from the 2023 Trends in International Mathematics and Science Study, known as TIMSS, comparing more than 41,000 fourth-grade students in the United Arab Emirates and Singapore. The research, published in the journal Large-scale Assessments in Education, offers one of the most detailed looks yet at how affective and motivational factors shape early science achievement across two very different education systems.

The study, conducted by Othman Abu Khurma of Emirates College for Advanced Education in Abu Dhabi, examined four motivational constructs drawn from the TIMSS student questionnaire: students’ confidence in science, digital self-efficacy, enjoyment of learning science, and the value students place on environmental preservation. Together, these variables were designed to capture complementary dimensions of expectancy-value theory, the influential psychological framework holding that students’ achievement-related behaviour is driven primarily by their expectations for success and the value they attach to learning tasks. Confidence in science represents an expectancy-related belief about one’s own competence, while enjoyment reflects intrinsic value, and environmental values tap perceived societal relevance.

The analytical sample comprised a weighted total of 34,842 fourth-graders from the UAE and 6,530 from Singapore, roughly 84 percent and 16 percent of the pooled dataset respectively, a distribution that mirrors the original TIMSS sampling design. The two countries were deliberately chosen as contrasting contexts: Singapore consistently ranks among the world’s top performers in international science assessments, while the UAE represents a rapidly reforming system investing heavily in digital learning platforms and curriculum modernisation. The comparison was not intended to rank the systems, the author stresses, but to test whether the same motivational constructs show similar predictive patterns in markedly different educational environments.

Methodologically, the study followed TIMSS technical guidelines by modelling each of the five plausible values for science achievement separately. Plausible values are draws from students’ estimated achievement distributions rather than simple observed test scores, and modelling all five reduces the risk that reported patterns are artefacts of a single estimate. Five supervised learning algorithms were then compared: linear regression, lasso regression, decision trees, random forests, and neural networks. Hyperparameters were tuned through cross-validation on the training data, and all analyses applied the student sampling weights provided by TIMSS to keep estimates representative of each national population.

The random forest emerged as the clear winner. It achieved a mean cross-validated R-squared of about 0.145 and a mean test R-squared of about 0.149, with a test root-mean-square error roughly 6.5 points lower than the linear models. The linear and lasso regressions explained only about 4 percent of out-of-sample variance, and the neural network added essentially nothing beyond the linear specification. The single decision tree told a cautionary tale: it fitted the training data closely, with a training R-squared of about 0.63, but produced negative out-of-sample R-squared values, a sign of severe overfitting in which the model reproduced noise rather than generalisable signal. Negative R-squared means the model predicted achievement less accurately than simply guessing the sample average.

That even the best model explained only about 15 percent of out-of-sample variance is itself a substantive finding. Roughly 85 percent of the variation in fourth-grade science achievement lies beyond what these four motivational variables can capture, underscoring the dominant role of factors the models could not include, from instructional quality to socioeconomic background. The author is explicit that this level of performance is insufficient for predicting individual students’ scores; the random forest serves a narrower descriptive purpose, revealing how the fitted model relates motivation to achievement and whether that relationship differs between countries. The consistency of performance across cross-validation, an independent test set, and all five plausible values suggests the model captured a reliable, if modest, signal.

Within that signal, science confidence dominated. Its average random-forest feature-importance value was 0.324, comfortably ahead of digital self-efficacy at 0.245, enjoyment of learning science at 0.226, and environmental values at 0.206. This ordering aligns with a substantial body of prior evidence, including meta-analyses, identifying confidence-related beliefs as among the strongest motivational correlates of science achievement. The zero-order correlation between science confidence and achievement was about 0.31 in the data, while digital self-efficacy correlated at roughly 0.22. Notably, Singaporean students scored significantly higher on achievement but reported slightly lower science confidence, whereas UAE students reported marginally higher enjoyment and digital self-efficacy, though these motivational differences carried small effect sizes.

To peer inside the random forest’s predictions, the study employed SHAP, or Shapley Additive exPlanations, a technique that decomposes a model’s output into additive contributions from each input variable. The SHAP interaction analysis revealed one striking cross-national difference: the contribution of enjoyment of learning science varied between the two countries, with the fitted model assigning larger positive contributions at high enjoyment levels in the UAE, while at moderate enjoyment levels Singaporean observations received somewhat larger contributions. The patterns for confidence, digital self-efficacy, and environmental values remained comparatively stable across the two samples. Given the model’s modest accuracy, the author treats these enjoyment differences as exploratory descriptions of model behaviour rather than evidence of established nonlinear effects or causal mechanisms, and the study explicitly does not identify the contextual factors, whether curriculum, pedagogy, or culture, that might underlie them.

The weakest predictor, environmental values, deserves careful interpretation rather than dismissal. TIMSS science achievement spans a broad range of content across life, physical, and Earth sciences, whereas pro-environmental attitudes are conceptually closer to environmental literacy constructs such as knowledge of ecosystems, climate change, and Earth resources. A child who champions school recycling programmes may express strong environmental values without that commitment registering on a general science test. The author suggests that environmental values may exert their strongest influence on domain-specific outcomes, and that future research should examine environmental knowledge subscales directly rather than broad composite scores.

The study also carries candid limitations that shape how its findings should be read. The analysed data file contained no socioeconomic, home-resource, parental-education, or school-level indicators, and TIMSS, being cross-sectional, records no measure of prior achievement, so the reported associations are unadjusted for potential common causes of both motivation and achievement. Part of the predictive signal attributed to motivational variables may therefore reflect background characteristics with which they share variance. Reciprocal relationships are also plausible, since high achievement can itself build confidence. Still, the work extends machine-learning research on large-scale assessments, which has concentrated largely on older students, into the early grades, a period when attitudes toward science form and often begin to decline. Its headline message is measured but clear: if educators want to understand which feelings matter most for young children’s science performance, belief in one’s own capability outranks pleasure in the subject, though motivation remains only one thread in a much larger tapestry of influences on how children learn science.

Subject of Research: Affective-motivational predictors of Grade 4 science achievement in TIMSS 2023 data from the UAE and Singapore using explainable machine learning

Article Title: Confidence over enjoyment? A cross-national explainable machine learning analysis of affective predictors of early science achievement

Article References: Confidence over enjoyment? A cross-national explainable machine learning analysis of affective predictors of early science achievement. (n.d.). https://doi.org/10.1186/s40536-026-00325-7

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00325-7

Keywords: TIMSS 2023, science achievement, machine learning, SHAP, self-efficacy, science confidence, motivation, primary education, United Arab Emirates, Singapore, random forest, expectancy-value theory