The study used the official Moroccan Grade 8 school-context file from the 2023 Trends in International Mathematics and Science Study, known as TIMSS. Schools, rather than students or teachers, were the unit of analysis because both the outcome and the explanatory variables came from principal questionnaires. The principal-rated outcome asked whether student absenteeism was not a problem, a minor problem, a moderate problem or a serious problem. For the primary statistical comparison, the researchers contrasted serious problems with the other three categories, while also examining the full four-level response as a sensitivity analysis. The common complete-case sample covered 232 of the 250 schools in the file and included 100 serious cases and 132 lower-severity cases.

Because TIMSS uses unequal school weights and a complex sampling design, population estimates incorporated the school weight, while uncertainty was calculated with 150 full jackknife repeated-replication estimates based on the survey’s 75 jackknife zones. The weighted models were binary logistic regressions. Their first specification contained five preselected contextual indicators: the share of pupils from economically disadvantaged homes, rural location, availability of high-speed internet, adequacy of mathematics and science resources, and the school’s emphasis on academic success. These variables represented social composition, geography, infrastructure, resources and academic organization, rather than a data-driven search across the full TIMSS questionnaire. The study was not preregistered, so the final predictor set was theory-guided rather than confirmatory in a preregistration sense.

None of the five contextual indicators remained statistically significant after false-discovery-rate correction within the contextual model. Resource adequacy had a nominal positive association with a serious principal rating, with an odds ratio of 1.81 per weighted standard deviation and a 95% confidence interval of 1.06 to 3.09, but its adjusted p-value was 0.145. Economic disadvantage had an odds ratio of 1.64, with a confidence interval of 0.98 to 2.72, while the point estimates for rural location and internet access were positive and that for academic emphasis was negative. None of these estimates provided a multiplicity-robust independent association in the available data. The authors emphasized that this does not show that contextual conditions are unimportant; the broad indicators may simply have been too indirect or imprecisely measured in a small design-weighted sample.

The analysis then added a two-item indicator combining principals’ reports of teacher lateness and teacher absenteeism. A one-standard-deviation increase in reported teacher-attendance problems was associated with 8.24 times the odds that student absenteeism would be rated serious, with a 95% confidence interval of 3.60 to 18.83 and an adjusted p-value below 0.001. Economic disadvantage also remained associated in this expanded model, with an odds ratio of 2.91 and a 95% confidence interval of 1.25 to 6.79. The association between economic disadvantage became stronger after teacher attendance was included, but the researchers said this might reflect suppression, changed adjustment or instability caused by unequal weights and should not be interpreted as evidence of a causal pathway mediated by teacher attendance.

For an assessment of discrimination, the researchers used regularized logistic regression with Elastic Net penalties and repeated nested cross-validation. The procedure comprised five outer folds repeated 10 times, with four-fold inner validation for choosing regularization strength and the balance between Ridge and L1 penalties. Folds were stratified by the binary outcome and grouped by jackknife zone, reducing the chance that design-linked schools would appear in both training and validation sets. All preprocessing was performed within training folds. Metrics included balanced accuracy, the area under the receiver operating characteristic curve, average precision, Brier score, log loss and calibration.

Broad context alone performed almost as poorly as random classification. Its mean balanced accuracy was 0.511 and its ROC area under the curve was 0.499, with average precision of 0.437. Sensitivity was 0.216 and specificity was 0.807, showing that the procedure identified relatively few of the schools whose principals called absenteeism serious. The mean Brier score was 0.258, log loss was 0.715, and the calibration slope was approximately zero. Once the teacher-attendance indicator was included, mean balanced accuracy rose to 0.728, ROC area rose to 0.774 and average precision rose to 0.695. Sensitivity increased from 0.216 to 0.688, while specificity changed only from 0.807 to 0.767. Probability error also fell, with a Brier score of 0.196 and log loss of 0.596.

The performance gain is striking but has a crucial measurement qualification. The principal who rated student absenteeism also rated teacher lateness and teacher absenteeism, using similar problem-severity categories. The added variable therefore shared both the respondent and the attendance domain with the outcome. Its strong association could reflect genuine shared organizational conditions, such as transport disruption, timetable instability, weak supervision or broader strains affecting pupils and staff. It could also partly reflect common-source measurement covariance: a principal with a generally severe assessment style might rate both staff and student attendance problems highly. Independent records would be required to separate those substantive and methodological explanations. Consequently, the results demonstrate incremental information within the questionnaire, not that teacher absenteeism causes student absenteeism.

Several further limitations constrain what the study can establish. It is cross-sectional, and the teacher and student attendance judgements were contemporaneous, so the validation results describe internal discrimination rather than prospective forecasting. The outcome was a principal’s perception of severity, not an administrative rate of recorded absence; it did not reveal the number of missed days, the identity of absent pupils, whether absences were excused or their causes. The raw sample of 232 schools also had a much lower Kish effective sample size of 60.5 because of unequal weighting. Complete-case analysis excluded 18 schools, including 11 with observed outcomes among which the weighted prevalence of serious reports was higher than in the retained sample, although this small, selectively observed group could not support a reliable estimate of bias. A four-category sensitivity analysis broadly supported the teacher-attendance result, but the lowest category contained only 13 schools and the academic-success variable violated the proportional-odds assumption.

Overall, the research identifies a sharp contrast between broad school context and information that is close to the outcome in both content and reporting source. The findings support further attendance research in Morocco using administrative student and teacher records, family information and reports from multiple informants. Such data could test whether the association between staff and student attendance persists when common-principal measurement effects are removed. Repeated or longitudinal observations would also be needed to establish temporal ordering, while larger samples and more specific measures of transport, health, caregiving, seasonal mobility and school attendance practices could clarify why economically disadvantaged schools appear more vulnerable. The current study does not validate the TIMSS item, establish causal effects or provide a forecasting tool, but it shows that how an absence problem is measured—and who reports it—can dramatically alter what appears knowable about school attendance.

Subject of Research: School-level associations with principal-reported student absenteeism in Moroccan Grade 8 schools using TIMSS 2023 data.

Article Title: Principal-reported school absenteeism in Morocco: contextual correlates and same-respondent attendance information in TIMSS 2023

Article References: Elouafi, A., Tammouch, I., Eddarouich, S., & laassar, I. (2026). Principal-reported school absenteeism in Morocco: contextual correlates and same-respondent attendance information in TIMSS 2023. Large-scale Assessments in Education, 14(1), Article 55. https://doi.org/10.1186/s40536-026-00327-5

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00327-5

Keywords: TIMSS 2023, Morocco, school absenteeism, teacher attendance, economic disadvantage, principal reports, large-scale assessment, survey weighting, out-of-sample validation, common-method bias, Principal-reported, school

The finding comes from a pre-registered umbrella review published in npj Science of Learning. Rather than analysing one experiment, the researchers brought together evidence from 10 meta-analyses reported in six documents, then combined their effect estimates and conducted a new meta-analysis. The studies represented a total sample range of 2,830 to 32,735, with a median sample size of 16,842. Searches covered PubMed, ProQuest PsycINFO, Scopus and Web of Science, while the review protocol was registered in PROSPERO as CRD420251207547.

CAT differs from a conventional computer-delivered test because the next question is selected in response to the test-taker’s answers. A software algorithm estimates ability from the items already completed, then chooses a subsequent item that is expected to provide useful information about that ability. In principle, this means fewer questions for some candidates, more efficient measurement for test developers and a shorter assessment experience. It also creates a distinctive psychological setting: the test appears to react continuously to the individual, which can make the process feel more interactive and tailored than a fixed test administered to everyone in the same order.

Across the reviewed evidence, CAT was associated with a small improvement in test-takers’ motivation, with an effect size of d = 0.08. It was also linked to more positive emotions, with d = 0.13. These are statistically discernible benefits, but the magnitudes are modest. In behavioural research, effect sizes are often easier to interpret through their size than through a simple pass-or-fail threshold: a value near 0.1 generally indicates a small difference between groups. The result therefore supports the idea that adaptive testing can make an assessment somewhat more engaging, without establishing a transformation in motivation.

That positive signal came with a parallel increase in anxiety, also measured at d = 0.08. The combination may seem contradictory, but it reflects a plausible feature of adaptive assessment: a more responsive and interactive system may make a test feel more relevant and motivating while simultaneously making it feel more consequential. The algorithm’s real-time adjustments can heighten the impression that the test is monitoring and evaluating each answer closely. CAT is therefore associated not with a single emotional profile, but with simultaneous gains in engagement and rises in pressure.

Most strikingly, the synthesis found lower test scores under CAT, with a median effect of d = −0.08. The negative sign indicates that scores tended to be lower in adaptive-testing conditions than in comparison conditions, while its small magnitude shows that the average difference was limited. Performance therefore moved in an unfavourable direction even as motivation and positive emotion improved. This combination complicates the idea that feeling more engaged during a test necessarily means knowing more, performing better or making fewer errors. It also suggests that CAT should be evaluated on several outcomes at once rather than through a single measure of test quality or user satisfaction.

The authors assessed the quality of the evidence using the Joanna Briggs Institute Critical Appraisal of Systematic Reviews and Research Synthesis tool. An umbrella review of this kind is valuable because the immediate question is often not whether one study works, but what the accumulated evidence says after accounting for a wider body of results. By synthesising meta-analyses rather than individual experiments, the approach can reveal recurring patterns across different populations, testing platforms and research designs. It also makes inconsistencies easier to see, such as the simultaneous presence of stronger motivation and greater anxiety, or the disconnect between emotional improvement and reduced scores.

Even so, the conclusions should be interpreted at the scale the effects allow. The small average effects do not mean that every learner is helped or harmed by CAT, nor do they show that adaptive algorithms inevitably reduce achievement. They summarise average differences across the evidence available at the time of the review, where results can vary with the subject being tested, the sophistication of the adaptive procedure, the purpose of the assessment and the characteristics of the test-takers. A technology that is valuable for precise measurement may still be uncomfortable for some users, while another implementation may produce different benefits. The umbrella review identifies a pattern across studies; it does not prescribe a single design for every educational or high-stakes setting.

The findings also arrive as CAT is being used increasingly for large-scale assessments worldwide. Its technical appeal is closely connected to the prevalence of digital infrastructure: tests can be delivered at scale, item selection can be automated and data can be collected during the assessment itself. Those capabilities may be especially valuable when many candidates need testing and a fixed-length paper test would be inefficient. But the evidence reviewed here adds an important constraint to the technology narrative. Personalisation at the level of the next question does not necessarily create a personalised emotional experience, and an apparently smoother interface can coexist with a less favourable performance result.

For schools, universities, testing organisations and policymakers, the practical lesson is not to abandon adaptive testing, nor to treat it as a self-evident solution to conventional exam design. The evidence supports testing it as a tool, with attention to implementation and to the experience of the people being assessed. Designs that provide reassurance, explain how item selection works, reduce abrupt changes in difficulty and monitor anxiety could be worth evaluating. Outcomes should include not only scores and measurement precision, but also motivation, emotion, accessibility and the burdens placed on test-takers.

The authors call for more high-quality meta-analyses covering CAT in broader contexts, a wider range of application settings and more diverse outcome variables. That recommendation matters because the current evidence already suggests there is no single, uniformly positive or negative effect. CAT can make testing more engaging and emotionally rewarding for some users, yet it can also make the same experience more stressful and coincide with lower performance. The most important question is therefore not whether adaptive testing beats a fixed test in every setting, but when its dynamic and personalised format improves the balance between efficient measurement, learning and psychological wellbeing.

Subject of Research: The benefits and risks of computerised adaptive testing in large-scale assessments

Article Title: Umbrella review of meta-analyses on the benefits and risks of computerised adaptive testing

Article References: Song, Y., Li, H., & Zheng, Q. (2026). Umbrella review of meta-analyses on the benefits and risks of computerised adaptive testing. npj Science of Learning. https://doi.org/10.1038/s41539-026-00460-4

Image Credits: AI Generated

DOI: 10.1038/s41539-026-00460-4

Keywords: computerised adaptive testing, adaptive assessment, digital testing, test motivation, test anxiety, student emotions, test scores, educational technology, meta-analysis, large-scale assessment, Umbrella, review

The study, published in SN Social Sciences by Musa Adekunle Ayanwale of the University of Pretoria and Maipato Charlotte Khoase of the National University of Lesotho, addresses a question that has received surprisingly little attention in the large literature on international assessments. Researchers have extensively examined the technical quality, comparability and policy use of cross-national testing programmes, but far less work has asked whether a country’s own national report clearly states its evidential boundaries, represents the sampled population faithfully, interprets results against national conditions, and actually reaches the stakeholders expected to act on them. To fill that gap, the authors developed a provisional Authenticity Evaluation Framework built around four reporting functions: methodological transparency, fidelity of representation, contextual interpretation, and policy accessibility and usefulness. The framework is explicitly presented as an analytical heuristic rather than a validated measurement model.

A central conceptual move in the paper is the separation of technical validity from what the authors call report authenticity. Validity, in the psychometric tradition associated with the Standards for Educational and Psychological Testing and Michael Kane’s argumentation framework, concerns whether accumulated evidence supports intended interpretations and uses of scores. Authenticity in assessment literature has traditionally referred to the realism or meaningfulness of assessment tasks, not to reports. The authors extend the term to documents, defining report authenticity as the fidelity, transparency, contextual adequacy and practical interpretability with which a report represents the educational reality within its stated scope. This distinction matters because a testing programme can generate sound scores while a national report communicates them with varying degrees of clarity about scope, uncertainty and implication. The two functions are complementary, and neither substitutes for the other.

The division of labour within the consortium underpins the study’s framing. Regional coordination structures, historically run with the UNESCO International Institute for Educational Planning, established common instruments, standardised administration, central data validation and cross-national reporting. National ministries, research coordinators and country teams were responsible for interpreting the evidence and producing national reports for domestic planning. Consequently, the authors direct claims about omitted context, accessibility or policy translation at Lesotho’s national reporting arrangements rather than at the regional coordination centre. The unit of analysis is the trio of Lesotho’s national reports, not the consortium’s international outputs, and the study does not extend to the consortium’s fifth cycle, now conducted under the name SEACMEQ.

Methodologically, the researchers adopted an interpretivist qualitative case study combining documentary analysis with semi-structured interviews involving four purposively selected senior stakeholders drawn from Lesotho’s Ministry of Education and Training and affiliated institutions. The informants, aged between 48 and 60, had direct experience in curriculum development, educational planning, assessment and quality assurance. Analysis followed inductive content analysis principles, supported by ATLAS.ti 24 for coding and theme development, with the four framework dimensions serving as sensitising concepts rather than predetermined codes. Trustworthiness was addressed through methodological triangulation, an audit trail, reflexive notes and repeated checking of themes against transcripts and reports. The authors are candid that four participants provide depth rather than representativeness, and that interview testimony cannot independently verify the implementation quality of every technical procedure described in the documents.

The documentary evidence confirmed a clear descriptive trend: mean literacy performance rose from 451 in SACMEQ II to 511 in SACMEQ IV, while numeracy climbed from 447 to 514, moving Lesotho from below the SACMEQ reference mean of 500 to slightly above it. Stakeholders interpreted these gains cautiously, distinguishing statistical improvement from the attainment of nationally desired standards. Interviewees accepted that the assessments were sufficiently aligned with the Grade 6 curriculum to support regional comparison, and they praised the data management chain, citing double data entry, national cleaning, central validation at the Coordination Centre and continuity in questionnaire design, with new modules added while core questions were retained. One informant noted that the consortium does not fundamentally change its questionnaires between cycles, which participants saw as supporting comparability, although the authors caution that perceived continuity does not by itself establish measurement invariance or full cross-cycle equivalence.

The limitations clustered around boundaries and context. Participants worried that the reports did not adequately represent learners with disabilities, with one informant observing that children with visual impairment in particular appeared to be excluded, in tension with the policy of inclusion. The authors are careful to note that a precisely defined sampling frame is not inherently a weakness; probability sampling permits defensible inference to a specified population even when that population is narrower than all children of the relevant age. The reporting problem arises when exclusions, non-response and the limits of generalisation are not made visible, or when conclusions drift beyond the eligible population. Participants also reported that reasons for non-response went unstated, that methodological limitations received only brief discussion, and that wider influences on achievement, including parental involvement, community support and household socio-economic conditions, received comparatively limited attention. A perceived divergence between SACMEQ results and Lesotho’s separate national assessment was treated strictly as a prompt for contextual inquiry, since the two programmes differ in grade, instruments, sampling and metric, with SACMEQ reporting Item Response Theory scale scores and the national programme reporting percentages.

On policy usefulness, the picture was again mixed. Stakeholders valued SACMEQ evidence for monitoring, benchmarking against the consortium’s member states, curriculum review, teacher development and resource allocation discussions, and the documentary record confirmed these intended functions. Yet dissemination emerged as the strongest constraint. One informant remarked that results typically stayed with government officials and a few participating academics, whereas some other countries make publication of the results a national agenda item. Participants also pointed to resistance to pedagogical change among some teachers and to systemic constraints, including shortages of qualified teachers, limited learning resources, inadequate facilities, financial pressure and weak parental participation. The authors stress that these conditions are not deficiencies of SACMEQ reporting; rather, they explain why credible evidence does not automatically translate into educational improvement, and they distinguish report credibility from implementation capacity.

The recommendations that follow from the evidence are practical and largely fall within national reporting and knowledge mobilisation. The authors suggest that Lesotho’s ministry and national research team make the target population, exclusions, non-response, uncertainty and limits of inference prominent in future reports; separate descriptive associations from causal explanations; explain how findings relate to national curriculum and system conditions; provide disaggregated analyses only where sample design and precision support them; and accompany technical reports with concise, audience-specific briefs for policymakers, curriculum specialists, district officers, school leaders and teachers. Participants additionally called for closer links between assessment evidence and in-service teacher training, and for reports to combine quantitative results with richer qualitative explanation. The authors emphasise that these strategies have not been evaluated prospectively, so their reach and effectiveness should be tested.

The study’s broader significance lies in its bounded, carefully qualified contribution to how the education world thinks about assessment reporting. It does not validate a new scale, estimate the psychometric quality of SACMEQ instruments, or generalise stakeholder perceptions to all report users, and it explicitly excludes SEACMEQ V and international reports from its corpus. Its theoretical claim is modest but pointed: validation of assessment-based inferences and evaluation of how national reports communicate those inferences are distinct tasks, and the second deserves systematic attention. The Authenticity Evaluation Framework offers a provisional vocabulary for that second task, pending expert content review, independent coding studies and multi-country testing. For Lesotho, the message is one of complementary responsibility: regional structures supply comparable, technically sound evidence, while national teams must supply the contextual interpretation, transparent boundaries and active dissemination that turn numbers into decisions. Technically defensible scores, the study concludes, are a foundation rather than a finished product.

Subject of Research: Authenticity and policy usefulness of Lesotho's national SACMEQ literacy and numeracy assessment reports

Article Title: Authenticity of Lesotho’s SACMEQ II-IV literacy and numeracy reports

Article References: Ayanwale, M. A., & Khoase, M. C. (2026). Authenticity of Lesotho’s SACMEQ II-IV literacy and numeracy reports. SN Social Sciences, 6(10), Article 543. https://doi.org/10.1007/s43545-026-01841-7

Image Credits: AI Generated

DOI: 10.1007/s43545-026-01841-7

Keywords: SACMEQ, Lesotho, report authenticity, large-scale assessment, literacy, numeracy, educational reporting, psychometric validity, qualitative case study, education policy, SEACMEQ, dissemination

The researcher, Jonas Mandalunes, exploited a rarely used survey item that asked students whether their parents limited screen time on school days and, crucially, whether that limit included the time they spent doing schoolwork on a screen. Students fell into three groups: an inclusive limit that treated an hour of homework like an hour of entertainment, an exempt limit that protected schoolwork and restricted only the rest, or no limit at all. Across the 28 systems, a median of 14.8 percent of students reported inclusive limits, 28.3 percent reported exempt limits, and 57.4 percent reported no limits. In other words, among families that set limits at all, roughly two-thirds carved schoolwork out of the daily budget.

The headline result is striking in its consistency. Students whose limits counted schoolwork scored 24.7 points lower in computer and information literacy than students whose limits exempted it, after adjusting for socioeconomic background, gender, home language, and immigrant background. That gap — about 0.30 within-system standard deviations, or roughly 29 percent of the width of a proficiency level — was statistically significant and negative in all 28 education systems, ranging from 48.1 points in Norway to 9.1 points in Oman. The pattern replicated almost perfectly in computational thinking, where the 19 systems that administered that assessment showed a pooled difference of 32.9 points, again negative everywhere.

Just as telling is what the study did not find. Students with no limit scored essentially the same as students with exempt limits in computer and information literacy — a pooled difference of just 0.6 points, small enough to satisfy a formal equivalence test with a margin of ten points. Having a rule, in itself, was not associated with lower competence. The entire penalty was concentrated among the minority of students, roughly one in seven, whose parents counted homework against the clock. This finding reframes the simpler comparison in the official ICILS report, which found students without limits scoring higher in most countries: the study shows mathematically that this binary gap is almost entirely a product of how many inclusive limits a system contains, since the share of inclusive limits among all limits ranged from about one-sixth in Denmark to more than half in Azerbaijan.

The study tested three competing explanations, and none fit alone. The first, an opportunity account, holds that digital skills grow through sustained, purposeful use, so a limit that swallows homework time leaves fewer chances to practice. The second, a displacement account, suggests that counting schoolwork might actually help by curbing the multitasking soup of homework and entertainment on the same device. The third, selection, argues that parents adopt inclusive limits reactively — because a child is struggling, shows weak self-control, or blends homework with play — so the rule marks pre-existing differences rather than causing them.

Several findings point toward selection. Students with inclusive limits came from lower socioeconomic backgrounds in 27 of 28 systems, had less computer experience in 27, expected a university degree less often in 24, and reported lower self-efficacy in 25. The gap shrank steadily as more characteristics were controlled: from 29.8 points without covariates to 24.7 with the basic adjustments, to 22.1 points when age and educational expectations — rough proxies for prior attainment — were added, and to 20.6 points when students were compared only with their own schoolmates. An exploratory meta-regression added a telling gradient: the gap was largest precisely where inclusive limits were rare, shrinking by 0.07 standard deviations for every 10-percentage-point increase in their prevalence. That is the pattern one would expect if an uncommon rule is adopted for specific, child-specific reasons, and harder to explain if the rule itself imposed a fixed cost.

Yet the opportunity account is not dismissed. Students with inclusive limits reported learning internet-related tasks — searching, evaluating sources, spotting scams, managing privacy — less often outside school, where parental rules apply, but not at school, where they do not. That setting-specific pattern is exactly what the opportunity account predicts. They also reported less recreational device use, which may mean fewer occasions for the informal, self-directed exploration through which some digital skills develop. But the simplest version of the story fails: these students reported just as much digital schoolwork at home as their peers, and slightly more long homework sessions. Meanwhile, students with no limits reported the most recreational use and the most multitasking, yet scored like the exempt group in literacy and lower in computational thinking — a reminder that raw hours of screen use track competence poorly.

The author also confronted the elephant in the room: unmeasured confounding. Prior achievement, self-regulation, and parental digital competence are not captured in ICILS, and any of them could shape both the rules families set and the skills students display. A sensitivity analysis following modern omitted-variable methods found that a confounder as strong as the socioeconomic index would shrink the gap only to about 21 points, and one three times as strong would leave roughly 13 points. But the analysis is candid about the limits of that reassurance: prior achievement likely predicts digital competence far better than socioeconomic status does, and a variable that strong would need only a weak link to rule type to account for the entire difference. The 25-point gap, the study stresses, should not be read as the causal effect of a rule.

The methodological care behind these estimates deserves note. The analysis used plausible values, sampling weights, and 75 jackknife replicate weights, pooled system-level estimates through random-effects meta-analysis, and verified the code by reproducing published national averages and the international report’s own limit-versus-no-limit comparison. Results proved robust to reweighting for item nonresponse, to using the parallel non-school-day item, and to adding or excluding individual systems. Students who skipped the rule item scored lower than those who answered it, but inverse-probability reweighting left the conclusions unchanged.

The practical upshot is twofold. For researchers, the message is that surveys of parental mediation must record what a limit covers, not merely whether one exists — studies linking screen rules to achievement, sleep, or wellbeing may otherwise average over rule types whose associations differ in sign and size. For families, the findings align with guidance from the American Academy of Pediatrics and Canadian movement guidelines, which already distinguish recreational from educational screen use, though the study is explicit that it cannot show that changing a rule would change a child’s competence. What it does show is that the one-in-seven students whose parents count homework as screen time differ from their peers in ways the data capture only partly — and that teachers may want to pay attention to exactly those students.

Subject of Research: The relationship between the scope of parental screen-time limits and adolescents' assessed digital competence in ICILS 2023

Article Title: When homework counts as screen time: parental limits and adolescents’ digital competence across 28 education systems in ICILS 2023

Article References: Mandalunes, J. (2026). When homework counts as screen time: parental limits and adolescents’ digital competence across 28 education systems in ICILS 2023. Large-scale Assessments in Education, 14(1), Article 54. https://doi.org/10.1186/s40536-026-00326-6

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00326-6

Keywords: screen time, parental mediation, digital competence, homework, ICILS 2023, computational thinking, computer and information literacy, adolescents, large-scale assessment, meta-analysis, educational technology, parenting

The study, published in the journal Large-scale Assessments in Education, drew on the German National Educational Panel Study, a long-running effort that follows children’s competence development across the life span. The researchers, led by Jana Welling of the Leibniz Institute for Educational Trajectories, analyzed reading and mathematics assessments administered in Grades 2 and 4, with 1,512 children participating in the second-grade assessments and 1,470 in the fourth-grade versions. Because the tests were administered on tablets or computers, every response came with a precise timestamp, allowing the team to reconstruct exactly how long each child spent on each item.

Rapid guessing matters because large-scale assessments are typically low-stakes: nothing happens to a child’s grades, placement, or future based on the outcome. Expectancy-value theory suggests that when a task carries no personal consequence, motivation to invest effort can collapse. The demands-capacity model adds a second dimension, proposing that effort depends on the match between what an item demands and the cognitive resources a test-taker is willing and able to deploy. When either willingness or capacity falls short, disengaged responding emerges—and in its most extreme form, responses arrive at implausibly high speed with accuracy near chance level.

Among adolescents and adults, this is no small problem. A 2022 meta-analysis found that roughly 7 percent of responses in low-stakes assessments are rapid guesses, and about 28 percent of examinees rapid-guess at least once. Worse, uncorrected proficiency estimates are biased downward by an average of 0.13 standard deviations. But children, the researchers reasoned, may be a different story. Young children tend to be overconfident in their abilities, which could boost their expectancy of success and keep them engaged. At the same time, their shorter attention spans could work in the opposite direction, making long tests feel more demanding and pushing them toward disengagement.

To detect rapid guessing, the team used response-time thresholds: any answer delivered faster than a cutoff was flagged as a guess and scored as missing. Because no single threshold method works for every test design, the researchers compared several approaches. For the reading tests and the fourth-grade mathematics test, they applied normative threshold methods, defining the cutoff as 10 or 20 percent of the average response time for each item. For the second-grade mathematics test—where items were read aloud via pre-recorded audio and children could not answer before the audio finished—they used visual inspection methods based on the shape of the response-time distribution and on how accuracy changes with speed. Thresholds were then validated by checking whether flagged responses showed chance-level accuracy and near-zero correlation with overall test scores.

The headline result: rapid guessing was scarce. In second-grade reading, 3.3 percent of responses were classified as rapid guesses; in second-grade mathematics, just 0.8 percent. By fourth grade, the rates fell even lower, to 0.5 percent in reading and 0.4 percent in mathematics. Contrary to the researchers’ hypothesis, disengagement did not increase between Grades 2 and 4—in the reading domain, it actually declined. The authors suggest that the fourth-grade reading test may simply have been too easy for most children, since easier items tend to elicit less rapid guessing, and that the real rise in disengagement documented in earlier studies typically emerges around puberty rather than in early elementary school.

Who rapid-guesses? Across all four assessments, only one characteristic reliably predicted the behavior: proficiency. Less proficient children were consistently more likely to fire off quick answers than their higher-performing peers, echoing patterns well established in older samples. A one-unit increase in estimated ability raised the probability of engaged responding by between 0.52 and 1.96 percentage points depending on the assessment. Notably, gender and parental socioeconomic status—both of which have been linked to test-taking effort in adolescent and adult studies—showed no significant effects. Descriptively, boys and children without a university-educated parent did guess somewhat more often, but these differences were too small and inconsistent to reach statistical significance, suggesting that the social gradients in test engagement familiar from older populations may simply not have taken hold yet in eight- and ten-year-olds.

The most consequential finding concerns what rapid guessing does to the numbers that policymakers actually use. The researchers compared estimates of ability differences between girls and boys, and between children of low and high parental socioeconomic status, computed once from the raw data and once from data in which rapid guesses had been scored as missing. The differences were essentially zero. Whether the team examined the modest gender gap in fourth-grade reading, the male advantage in mathematics, or the substantial socioeconomic gaps present in every domain, accounting for rapid guessing shifted the estimates by at most 0.01 standard deviations, well within the bootstrap confidence intervals. In other words, even the small amount of guessing observed did not distort subgroup comparisons in any meaningful way.

Why doesn’t a behavior that biases average scores elsewhere leave group comparisons untouched? The authors point to two mechanisms. First, when most test-takers are engaged, the handful of rapid guesses carries little statistical weight. Second, rapid guessing is strongly tied to low proficiency—and because less proficient respondents have a low probability of answering correctly even when trying, a random guess changes little. For a bias in group comparisons to emerge, the groups would need to differ substantially in their guessing rates, which appears to be rare in practice, particularly among children.

The study carries practical weight for the assessment community. It suggests that the alarming disengagement rates documented in adolescent samples cannot simply be projected downward onto younger children, and that large-scale assessments involving elementary schoolers may be more robust than critics fear. But the authors caution against complacency. Prevalence varied considerably between assessments even within the same children, pointing to the strong influence of test design, time limits, item presentation, and proctoring conditions—and a lack of rapid guessing does not guarantee full engagement, since children may express disengagement in other ways, such as omissions or slow, effortful guessing. The team also notes that no single threshold method fits every test, and that the rarity of the behavior itself makes its predictors statistically hard to pin down. Future work, they argue, should examine broader forms of disengagement, test model-based identification approaches in child samples, and experimentally vary assessment settings. For now, though, the message is clear: the youngest test-takers, by and large, are still trying.

Subject of Research: Rapid guessing behavior and test-taking engagement among elementary school children in large-scale educational assessments

Article Title: Prevalence, predictors, and consequences of rapid guessing among children in elementary school

Article References: Welling, J., Fischer, L., Thums, K., & Gnambs, T. (2026). Prevalence, predictors, and consequences of rapid guessing among children in elementary school. Large-scale Assessments in Education, 14(1), Article 51. https://doi.org/10.1186/s40536-026-00319-5

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00319-5

Keywords: rapid guessing, test-taking engagement, elementary school, large-scale assessment, response times, psychometrics, educational measurement, reading assessment, mathematics assessment, socioeconomic status, test validity, National Educational Panel Study

The study, published in the journal Large-scale Assessments in Education, was led by Guanyu Chen of the University of British Columbia together with colleagues at Iowa State University and the Hong Kong Examinations and Assessment Authority. Rather than treating the two systems as interchangeable examples of East Asian achievement, the researchers directly compared them within a single analytical framework, using Walberg’s theory of educational productivity to organize factors into four domains: student characteristics, student attitudes and behaviors, family background, and school context.

The technical backbone of the analysis was hierarchical linear modeling, or HLM, a statistical approach designed for data in which students are nested within schools. Because PIRLS samples intact classrooms rather than independent individuals, traditional regression would underestimate uncertainty and blur the distinction between student-level and school-level effects. The team used the EdSurvey R package, applying sampling and replicate weights and analyzing five plausible values per student to capture measurement uncertainty in the achievement scores. After listwise deletion of cases with missing data, the final models included 3,619 fourth-graders in Hong Kong and 5,734 in Singapore.

One of the most consistent findings concerned absenteeism. In both systems, students who reported missing school at least once every two months scored substantially lower in reading, with gaps of roughly 24 points in Hong Kong and nearly 30 points in Singapore in the fully adjusted models. This is notable because both jurisdictions tightly regulate attendance: Hong Kong legally obliges parents to ensure schooling during prescribed hours, while Singapore enforces mandatory attendance under its Compulsory Education Act. Despite these structures, around 11 percent of Hong Kong students and 18 percent of Singaporean students reported absences, and their underperformance suggests that digital learning infrastructure alone cannot compensate for lost in-person instruction.

Reading confidence emerged as the single strongest attitudinal predictor in both systems. Students with high confidence scored 47.52 points higher than low-confidence peers in Hong Kong and a remarkable 77.88 points higher in Singapore, effects that remained stable even after family and school factors were added. The researchers frame this through the lens of social-emotional learning, linking self-efficacy and self-management to academic engagement. Moderate amounts of reading outside school, between 30 minutes and two hours daily, also predicted higher scores, though the benefits appeared to plateau or diminish for students reading more than two hours a day, possibly because PIRLS is calibrated to the average global reader.

Not every behavioral factor behaved the same way in both places. Library use was consistently and positively associated with reading achievement in Singapore, where even occasional users outscored non-users by 11 to 16 points, but showed no significant relationship in Hong Kong. Classroom engagement mattered modestly in both systems, while self-reported enjoyment of reading was not significantly associated with achievement in either. These divergences hint that identical interventions may not transfer cleanly between superficially similar systems.

The starkest contrast appeared in the family and school domains. Home socioeconomic status was a strong predictor everywhere, but its estimated effect was more than twice as large in Singapore, where high-SES students outscored low-SES students by about 45 points, compared with roughly 21 points in Hong Kong. Intriguingly, only 5.52 percent of Singaporean students came from low-SES households versus 17.43 percent in Hong Kong, and the authors suggest that Hong Kong’s smaller SES coefficient may reflect greater equity in how educational resources are distributed, a pattern consistent with OECD findings on Hong Kong’s performance in PISA 2022.

Parental reading habits, parental satisfaction with the school, and a school’s emphasis on academic success all predicted higher reading scores in Singapore but were weak or nonsignificant in Hong Kong. The authors caution that these relationships may partly reflect broader differences in school resources and academic culture rather than independent causal effects. Still, the descriptive gaps are telling: 42.25 percent of Hong Kong students attended schools with a low emphasis on academic success, compared with just 15.72 percent in Singapore. The study points to Singapore’s School Excellence Model and initiatives like Read@School as examples of deliberately coordinated family-school alignment, while Hong Kong has pursued parent education through a 2017 task force, Parent-Teacher Associations, and a territory-wide Positive Parents Campaign.

Demographic patterns also diverged. The gender gap favoring girls disappeared in Hong Kong once attitudes and behaviors were accounted for, but remained significant in Singapore even after full adjustment, at nearly 8 points. Age mattered only in Singapore, where older students within the grade cohort scored up to 14 points higher, and birthplace effects were sensitive to which other factors were in the model. At the school level, resource shortages and teacher job satisfaction showed no significant associations with achievement in either system, though the authors note large standard errors that suggest more complex underlying mechanisms.

The authors are careful to stress that the study is cross-sectional and identifies associations, not causes, and they call for causal inference methods and qualitative research to unpack the mechanisms at work. The models explained about 59 percent of school-level variance in Singapore but only about 32 percent in Hong Kong, hinting that unmeasured factors, from peer dynamics to school leadership, may be especially important there. The overarching lesson is that even top-performing systems diverge in meaningful ways: both must support absent students and narrow gaps between privileged and underprivileged learners, but the levers that work in Singapore’s tightly coordinated ecosystem may not map directly onto Hong Kong’s more heterogeneous landscape. For any system hoping to emulate the world’s best readers, the message is that excellence is not a single recipe but a family of locally tuned strategies.

Subject of Research: Comparative analysis of student, family, and school predictors of fourth-grade reading achievement in Hong Kong and Singapore using PIRLS 2021 data

Article Title: Educational productivity in high-performing systems: a comparison of Hong Kong and Singapore on PIRLS 2021

Article References: Chen, G., Qi, Q., Jin, K.-Y., Wang, Z., & Xi, X. (2026). Educational productivity in high-performing systems: a comparison of Hong Kong and Singapore on PIRLS 2021. Large-scale Assessments in Education, 14(1), Article 50. https://doi.org/10.1186/s40536-026-00316-8

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00316-8

Keywords: PIRLS 2021, reading achievement, Hong Kong, Singapore, hierarchical linear modeling, educational productivity, literacy, socioeconomic status, absenteeism, family-school collaboration, social-emotional learning, large-scale assessment

The study exploited a rare opportunity. When Sweden administered PISA 2022, the national agency for education recorded students’ personal identification numbers, allowing the researchers to link the test scores of 6,072 fifteen-year-olds with official register data containing their Grade 9 national test results and final subject grades in Swedish, mathematics, and science. Because PISA samples by age rather than grade, nearly 97 percent of the participants were ninth-graders, making the comparison across the three assessment types unusually clean. Linkage was achieved for 99.5 percent of the sample, and the researchers applied survey weights and adjusted for the clustering of students within 262 schools to ensure that their statistical models reflected the true population structure.

The conceptual stakes are high because the three measures were built for different purposes. PISA, run by the OECD, deliberately avoids any national curriculum. It assesses what the OECD calls literacy: the capacity of students to analyse, reason, communicate, and apply knowledge in unfamiliar, real-world situations, whether or not those situations resemble classroom content. National tests and school grades, by contrast, are explicitly curriculum-based, evaluating mastery of subject syllabuses. The stakes also differ sharply. PISA is low-stakes for individual students, who receive no feedback and face no consequences, whereas Swedish grades determine admission to upper secondary and tertiary education, and national test results feed into grading decisions. Motivation research suggests these differences can affect how hard students try, potentially depressing PISA performance relative to what students know.

The first striking finding concerned the correlations among the PISA domains themselves. PISA reading, mathematics, and science scores were intercorrelated at values between 0.81 and 0.88, an extraordinarily high degree of overlap for tests that are supposedly distinct subject assessments. By comparison, the correlations between PISA scores and the school achievement measures ranged from 0.58 to 0.74, moderately strong but clearly not indicating interchangeability. Meanwhile, the school-based measures correlated with each other at 0.62 to 0.92, with the strongest link, 0.92, between mathematics grades and mathematics national test results. Notably, PISA reading correlated almost equally with grades and tests in Swedish, mathematics, and science, around 0.61 to 0.63, suggesting that reading literacy acts as a broad competence that underpins performance across the entire curriculum.

To disentangle what drives these patterns, the researchers turned to confirmatory factor analysis, specifically a bifactor model with a multitrait-multimethod structure. In this framework, the three subjects, Swedish or reading, mathematics, and science, serve as traits, while the three assessment types, PISA, national tests, and grades, serve as methods. A general achievement factor captures variance common to all nine measures, while orthogonal specific factors capture variance unique to particular subjects or particular assessment formats. The researchers built up their models step by step: a single general factor fit poorly, adding method factors improved things only marginally, but a model combining the general factor with both method and subject factors achieved acceptable fit across all standard indices, including CFI, TLI, RMSEA, and SRMR.

The factor loadings told a clear and, for PISA, sobering story. All nine measures loaded strongly on the general achievement factor, with loadings between 0.69 and 0.96, confirming that a broad academic ability underlies performance regardless of how it is measured. But the method factors revealed an asymmetry. The PISA method factor showed substantial loadings of 0.54 to 0.62, meaning a large share of PISA variance is specific to the PISA assessment format itself. The national test method factor, in contrast, loaded below 0.20, indicating that national tests share nearly all their variance with grades and add little method-specific noise. In other words, grades and national tests behave as close relatives, while PISA stands apart.

The subject factors delivered the study’s most consequential result. Grades and national tests showed strong loadings of roughly 0.50 on the Swedish and mathematics specific factors, demonstrating that these school-based measures genuinely differentiate between what a student knows in one subject versus another. The corresponding PISA loadings were tiny, between 0.12 and 0.16. PISA, in short, barely distinguishes reading from mathematics from science. Rather than mapping subject-specific curricular learning, it appears to capture a general, literacy-oriented competence shaped by its own assessment framework. This finding echoes earlier item-level studies, including work by Pokropek and colleagues across 33 countries, which found that a general factor accounted for about 84 percent of the common variance in PISA scores, with subject-specific factors too weak to interpret reliably.

Why does PISA blur the boundaries between subjects? The authors point to several plausible mechanisms. PISA’s literacy framework emphasizes applied, real-world contexts, and its items are notoriously text-heavy: research on Swedish PISA mathematics tasks found they demand far more reading than the national mathematics tests, so reading comprehension skill can drive performance even in the mathematics domain. PISA’s plausible-value imputation procedure may also inflate inter-domain correlations, because each student’s scores are conditioned on performance in other domains and on background variables. Additionally, PISA administers items from all domains in a single session, so fatigue, motivation, and other situational factors may influence performance across domains simultaneously. Low stakes may further dampen effort; as the OECD itself notes, PISA captures what students actually do under low-consequence conditions, not their maximum potential.

The policy implications are significant. Because PISA results often trigger national reforms, the authors warn against interpreting PISA scores as direct indicators of subject-level proficiency or using them to justify curriculum-specific interventions. Scholars have cautioned that over-reliance on international assessments can narrow curricula and misdirect priorities. At the same time, the researchers are careful not to declare PISA useless: it remains a valuable, curriculum-independent benchmark for cross-national comparison and trend monitoring, immune to national grade inflation. The honest conclusion is that PISA, national tests, and grades measure overlapping but non-interchangeable constructs. Whether PISA’s broader literacy focus or the schools’ subject-specific measures better capture socially relevant learning is, the authors stress, a normative question that statistics alone cannot settle. What statistics can settle, and this study does, is that treating these metrics as equivalent would be a measurement mistake with real consequences for education policy.

Subject of Research: Comparing the measurement structure of PISA scores, national tests, and teacher-assigned grades in reading, mathematics, and science

Article Title: Crossing metrics: are PISA, national tests, and grades measuring the same?

Article References: Borger, L., Strietholt, R., Johansson, S., & Greiff, S. (2026). Crossing metrics: are PISA, national tests, and grades measuring the same?. Large-scale Assessments in Education, 14(1), Article 47. https://doi.org/10.1186/s40536-026-00321-x

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00321-x

Keywords: PISA, educational assessment, psychometrics, school grades, national tests, large-scale assessment, bifactor model, multitrait-multimethod, test validity, education policy, Sweden, student achievement

The ten systems examined were Argentina, Chile, Chinese Taipei, Finland, Hungary, Italy, Japan, Korea, the Philippines, and the United States. The authors stress that this was a purposive comparative sample, chosen to span geographic regions, OECD membership status, and a wide range of average performance, from a weighted mean mathematics score of 352.57 in the Philippines to 531.14 in Chinese Taipei. The design supports comparisons across these ten systems but does not license claims about all countries or continents. Within the sample, 2,617 schools contributed students, with country sample sizes ranging from 4,838 in the United States to 11,975 in Argentina.

What distinguishes this study from much of the educational data-mining literature is its fidelity to PISA’s assessment architecture. Mathematics achievement in PISA is not a single number but a set of ten plausible values, multiple imputed estimates that represent uncertainty in each student’s latent proficiency. The researchers fitted and evaluated every model across all ten plausible values rather than relying on just the first. They also incorporated the final student weight, W_FSTUWT, in descriptive statistics, model fitting, and test-set evaluation, and used the 80 student replicate weights to estimate sampling variability. Total variance for each performance metric combined the mean replicate-weight sampling variance with the between-plausible-value variance, following the formula U-bar plus (1 + 1/M) times B, so that confidence intervals reflected both complex sampling uncertainty and achievement-scaling uncertainty. Few machine-learning studies of PISA data go to these lengths.

The predictor side of the analysis was equally disciplined. The authors constructed a transparent primary candidate pool of 33 predictors from PISA 2018 student questionnaire variables and OECD-derived indices, organized into six substantive domains: student demographics, family socioeconomic status and home resources, engagement and learning time, peer climate, belonging and parent support, and classroom climate. Rather than assuming all 33 variables mattered everywhere, they performed country-specific stability feature selection using only training schools. Three complementary methods were applied: mutual information, which captures general dependence; weighted elastic net regression, a regularized linear method that handles correlated predictors through combined L1 and L2 penalties; and weighted random forest ranking, which captures nonlinear and interaction-based structure. Feature rankings were repeated across ten plausible values and five internal school-level folds, yielding 50 selection runs per country. A predictor was deemed stable if selected in at least half of those runs. The resulting stable sets contained between 11 and 15 predictors, averaging 14.0 per country.

Four models were then compared under country-specific school-level holdout evaluation, with roughly 70 percent of students in training and 30 percent in test sets. Splitting by school rather than by student reduced leakage from students in the same school appearing on both sides. The contenders were weighted linear regression, weighted linear regression augmented with pairwise interactions, random forest, and CatBoost, a gradient-boosted tree ensemble designed for structured tabular data. Averaged across countries, CatBoost was the clear winner, achieving a mean weighted R-squared of 0.358 and a mean mean absolute error of 57.29 score points. Random forest followed with a mean weighted R-squared of 0.313 and an MAE of 59.30. The interaction-augmented linear model barely improved on the additive linear benchmark, with mean R-squared values of 0.290 versus 0.286. CatBoost posted the lowest weighted MAE in all ten systems, and its advantage over random forest was consistent but moderate: roughly 2.01 MAE points and 0.045 in R-squared on average. Its country-level R-squared ranged from 0.210 in Italy to 0.443 in Hungary.

The interpretive core of the study used SHAP, or Shapley additive explanations, a technique that decomposes each model prediction into additive contributions from individual features. The authors were careful to frame SHAP summaries as model-based explanations of predictive associations, not causal effects. Because CatBoost performed best, interpretation focused on its SHAP values, computed on held-out test schools and averaged across all ten plausible values to produce PV-robust stability summaries. The headline finding is that books at home was the most stable predictor of all, ranking among the top ten SHAP predictors in all ten countries on average and among the top five in 8.7 countries on average. Highest parental occupational status was nearly as stable, appearing in the top ten in 9.2 countries and the top five in 7.1. Socioeconomic status itself appeared in all ten country-specific models and ranked in the top ten in 8.3 countries on average, though its average top-five count was lower at 4.3.

Beyond the socioeconomic core, the picture became more heterogeneous. Grade placement entered the stable feature sets of seven countries and ranked in the top ten in all seven, but it was not selected in Chinese Taipei, Japan, or Korea. Mathematics learning time appeared in six countries and ranked in the top ten in all six, with an average top-five count of 5.5, and it ranked especially highly in Chinese Taipei, Japan, and the United States. Test effort was retained in only four systems but was consistently influential there, ranking in the top five in all four and even first in Finland, Italy, and Korea in the diagnostic visualization. Directed instruction and disciplinary climate also showed cross-national reach, ranking in the top ten in 7.0 and 5.8 countries on average respectively. The authors argue that this mix of stability and country-specificity is precisely why a single pooled importance ranking would be misleading.

Sensitivity analyses reinforced the robustness of these conclusions. Comparing SHAP summaries based on the first plausible value with summaries averaged across all ten showed only small differences, with maximum absolute gaps of 0.8 countries for top-ten counts, 1.2 countries for top-five counts, and 1.25 rank positions for mean rank, confirming that the main pattern was not an artifact of PV1MATH. A second sensitivity analysis removed grade placement from the stable feature sets where present and refitted the CatBoost models. Performance dropped modestly, with a mean R-squared change of -0.028 and a mean MAE increase of 1.17 score points, the largest decline occurring in Chile, where R-squared fell by 0.091 and MAE rose by 3.92 points. Crucially, the SHAP stability pattern remained broadly similar without grade: books at home, parental occupational status, learning time, socioeconomic status, test effort, directed instruction, and disciplinary climate all stayed among the most stable predictors.

The authors are explicit about the limits of their contribution. They do not claim to have discovered new determinants of mathematics achievement; domains such as socioeconomic resources, home literacy environments, learning time, and classroom climate are already well established. The value lies in showing how these familiar predictors behave under a rigorous, country-specific, survey-weighted, plausible-value-aware explainable machine-learning framework, and in separating three questions that are often conflated: which models predict best, which predictors are stable across systems, and which are context-specific. They also caution that the analysis is predictive rather than causal, that the ten-system sample is not statistically representative of global education, that the 33-predictor pool cannot exhaust all relevant influences such as school policies or teacher characteristics, and that cross-national questionnaire comparisons may be affected by response styles, translation, and measurement comparability.

The implications reach well beyond this dataset. The framework, with its train-only feature selection, school-level holdout evaluation, replicate-weight variance estimation, and all-plausible-value pooling, offers a reproducible template that could be extended to more PISA systems, to other assessments such as TIMSS and PIRLS, to school-level and system-level predictors, and to fairness analyses examining whether prediction errors or SHAP patterns differ across gender, socioeconomic, immigrant, or language groups. Repeated-cycle analyses could test whether cross-national stability patterns persist as PISA evolves. For a field increasingly drawn to black-box prediction, the study makes a compelling case that explainability and methodological rigor are not optional extras but the foundation of responsible machine learning in comparative education.

Subject of Research: Explainable machine learning applied to predicting and interpreting mathematics achievement in PISA 2018 across ten education systems.

Article Title: Explainable AI for predicting and interpreting mathematics achievement: a cross-national analysis of PISA 2018

Article References: Liu, L., & Dai, R. (2026). Explainable AI for predicting and interpreting mathematics achievement: a cross-national analysis of PISA 2018. Large-scale Assessments in Education, 14(1), Article 46. https://doi.org/10.1186/s40536-026-00320-y

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00320-y

Keywords: PISA 2018, mathematics achievement, explainable artificial intelligence, SHAP, CatBoost, machine learning, survey weights, plausible values, large-scale assessment, socioeconomic status, comparative education, feature selection

The study, published in the journal Large-scale Assessments in Education, was conducted by Markéta Žáková and Tomáš Lintner of Masaryk University in the Czech Republic. Drawing on PISA 2018 data from more than 600,000 fifteen-year-old students across 75 education systems, the pair set out to answer a deceptively simple question: does it matter which operational definition of academic resilience a researcher uses? The answer, they found, is a resounding yes, with consequences that ripple through prevalence estimates, country rankings, and conclusions about which factors help disadvantaged students beat the odds.

Academic resilience combines two ingredients: adversity, usually measured by socioeconomic status, and positive adaptation, usually measured by achievement. But researchers disagree about how to stitch those ingredients together. One common approach simply identifies students who land in the bottom slice of the socioeconomic distribution and the top slice of achievement, for example the poorest quarter who score in the top quarter. A second approach instead computes what each student’s achievement should be, statistically speaking, given their family background, and flags students who substantially outperform that expectation, an approach known as the residual method. A third treats resilience as a process rather than an outcome, predicting achievement as a continuous variable within the disadvantaged group and never labeling anyone resilient at all.

Žáková and Lintner compared these approaches systematically. They estimated a top-achiever definition, a residual definition computed within each country, and a residual definition benchmarked against an international standard, each at three different threshold levels, 20, 25, and 33 percent, spanning the range most commonly used in the published literature. All told, this produced nine binary operationalizations plus the continuous-outcome model, yielding nearly a thousand country-level analyses. Each estimate was pooled across ten plausible values for achievement and twenty multiply imputed datasets, with standard errors that fully accounted for PISA’s complex two-stage sampling design.

The headline finding is stark. The cross-country mean prevalence of academically resilient students ranged from 7.5 percent under the strictest definition to 35.3 percent under the most inclusive residual benchmark, a nearly fivefold difference computed from identical data. Part of this spread is arithmetic, since looser thresholds mechanically admit more students, but the deeper problem emerges when countries are ranked. Rankings were reasonably stable across thresholds within a given definition, yet they diverged sharply across definitions. The correlation between rankings produced by the top-achiever approach and the residual approaches fell as low as 0.51, and between the two residual variants, which differ only in whether the statistical expectation is local or global, it dropped to between 0.32 and 0.47. A country celebrated as an equity champion under one definition can rank unremarkably under another, and the discrepancy is itself informative about what its disadvantaged students actually do well.

The authors illustrate why with a thought experiment grounded in their results. A lower-performing system with a steep socioeconomic gradient may rank poorly on the top-achiever definition, because few of its disadvantaged students reach absolute excellence, yet rank highly on the within-country residual definition, because modest local expectations are easy to exceed. That same system may then sink again on the internationally benchmarked residual definition, since beating a weak local bar is not the same as meeting a global standard. The researchers argue that resilience rankings should therefore be reported under multiple definitions side by side, as complementary views of equity rather than competing estimates of a single quantity, a practice the OECD itself briefly adopted nearly a decade ago.

What about the protective factors that resilience research is meant to uncover? Here the news is more reassuring at the global level and more troubling at the level of individual countries. When the authors pooled effects meta-analytically across all 75 systems, three motivational constructs drawn from expectancy-value theory, students’ self-perceived reading competence, their enjoyment of reading, and their attitude toward learning, were consistently and positively associated with resilience under every operationalization and threshold, and girls consistently outperformed boys. Read that result alone, and the choice of definition seems inconsequential.

The country-by-country picture tells a different story. When each education system was analyzed separately, as is standard in PISA-based research, findings were consistent across all nine binary specifications in only 20 percent of systems for gender, 40 percent for attitude toward learning, 69 percent for reading enjoyment, and 72 percent for self-perceived competence. Just two of the 75 systems produced fully consistent results for all four predictors. Threshold level alone flipped conclusions in somewhere between 3 and 37 percent of countries depending on the factor. Much of this inconsistency reflects statistical power, since stricter thresholds shrink samples and smaller-effect predictors, notably gender and attitude toward learning, are the least stable. But the practical consequence does not depend on the cause: a researcher studying one country under one definition could legitimately conclude that a factor matters there when a defensible alternative would say it does not.

The most conceptually striking result came from a complementary interaction analysis that requires no thresholds at all. By modeling whether the socioeconomic gradient in achievement flattens at higher levels of each factor, the authors tested what the term protective factor actually implies. The findings complicate comfortable assumptions: the gradient was flatter, not steeper, among students with higher self-perceived reading competence and stronger attitudes toward learning, but it was significantly steeper, not flatter, among students who enjoyed reading more. In other words, reading enjoyment, though positively associated with achievement on average, was linked to wider rather than narrower socioeconomic gaps, a pattern that no threshold-based definition could ever detect. These moderation effects also varied in direction across countries, appearing in only a minority of systems individually.

The authors close with four practical recommendations: match the operationalization to the research question, report sensitivity analyses at a minimum of two thresholds, present cross-country rankings under multiple definitions, and document or pre-register operationalization decisions so findings can be interpreted in light of the choices that produced them. They caution that their analysis is cross-sectional and cannot establish causation, that their predictors were individual-level only, and that PISA’s socioeconomic index has itself attracted methodological criticism. Still, the broader message is hard to escape. Academic resilience, as measured in large-scale assessments, is not a fixed quantity waiting to be counted but a construct whose observed properties depend partly on how it is defined, and both researchers and policymakers ignore that dependence at their peril.

Subject of Research: How operationalization and threshold choices affect estimates of academic resilience and its protective factors across 75 education systems using PISA 2018 data.

Article Title: Academic resilience in 75 education systems: how operationalization and threshold choices shape findings on protective factors

Article References: Žáková, M., & Lintner, T. (2026). Academic resilience in 75 education systems: how operationalization and threshold choices shape findings on protective factors. Large-scale Assessments in Education, 14(1), Article 44. https://doi.org/10.1186/s40536-026-00318-6

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00318-6

Keywords: academic resilience, PISA 2018, socioeconomic status, educational equity, large-scale assessment, protective factors, operationalization, thresholds, student motivation, expectancy-value theory, educational measurement, cross-national comparison