Global School Poverty Measures Track National Data, But Only One Predicts Math Scores

A detailed validation study from Ireland has found that the socio-economic status measures used in two of the world’s largest education assessments, TIMSS and PISA, align closely with the national administrative indicators that the Irish government uses to channel extra funding to disadvantaged schools. The research, published in the journal Large-scale Assessments in Education, offers some of the strongest empirical evidence yet that international survey-based measures of school socio-economic context can be trusted, while also revealing a striking asymmetry: when it comes to predicting students’ mathematics achievement, the international indicators outperform the national ones.

The study was motivated by a practical question with high stakes. Policymakers across the world rely on measures of socio-economic status, or SES, for two main purposes. First, they use them to decide which schools should receive additional resources aimed at levelling the playing field for students from poor backgrounds. Second, researchers use them to understand how family and school circumstances shape achievement, and to monitor whether equity reforms are working. If the underlying measures are flawed, both the flow of funding and the scientific conclusions built on them are compromised.

The research team, led by Lorraine Gilleece of the Educational Research Centre in Dublin, together with Anastasios Karakolidis and Aidan Clerkin, compared four sets of indicators. Two came from international large-scale assessments: the 2019 Trends in International Mathematics and Science Study, or TIMSS, which tested Grade 8 students, and the 2022 Programme for International Student Assessment, or PISA, which tested 15-year-olds. From each assessment, the researchers extracted a school-average SES index, aggregated from individual student responses, together with principals’ own estimates of the percentage of their students coming from disadvantaged homes. These were set against two national indicators: the Pobal HP Deprivation Index, a continuous measure derived from census data on small residential areas, and a binary flag for whether a school participates in Ireland’s Delivering Equality of Opportunity in Schools programme, known as DEIS, which directs extra resources to schools with high concentrations of disadvantage.

Technically, the analyses were extensive. The team computed correlations, ran linear and logistic regressions, and fitted two-level multilevel models using survey weights and plausible values for achievement, in line with the statistical protocols of the International Association for the Evaluation of Educational Achievement and the OECD. The TIMSS sample covered 4,118 students in 149 schools, while the PISA 2022 sample included 5,569 students in 170 Irish schools. The HP index, built from census indicators such as educational attainment, social class composition, unemployment rates and demographic vitality, was matched to schools using student home addresses, meaning school-average values were based on almost the entire enrolled cohort rather than only the assessed students.

The convergent validity results were emphatic. Correlations between the international and national indicators were large by conventional statistical standards, with coefficients of roughly 0.7 among indicators derived from aggregated student-level data, such as school-mean SES and school-mean HP scores, and around 0.6 among inherently school-level indicators such as principals’ reports and DEIS status. When both TIMSS-derived indicators were entered together in a regression predicting the school-mean HP index, they explained 55 percent of its variance; the equivalent PISA model explained 50 percent. DEIS schools were also shown to differ markedly from non-DEIS schools on every international indicator, with large effect sizes throughout.

One of the most reassuring findings concerned school principals. Because principals’ estimates of the share of disadvantaged students rest on impressions that cannot be directly verified, they have long been treated with suspicion. Yet in this study, principals’ judgements aligned well with both the census-based HP index and the aggregated student-level SES indices. In the logistic regression predicting DEIS status, schools whose principals reported that more than half of their students came from disadvantaged homes had 65 times higher odds of DEIS categorisation than schools reporting up to a quarter, a pattern the authors read as reflecting the policy’s deliberate targeting of schools with the highest concentrations of disadvantage. Principals, the study concludes, appear to know their schools remarkably well.

The picture changed when the researchers turned to criterion-related validity, asking how well each family of indicators predicted mathematics achievement. The international indicators were consistently stronger. Together, school-mean SES and the principal-reported share of disadvantaged students explained 16 percent of the variance in TIMSS mathematics scores and 9 percent in PISA. The national indicators on their own explained less, at 11 percent for TIMSS and 6 percent for PISA. Crucially, in a combined model, only the ILSA-derived school-mean SES remained a statistically significant predictor; adding the national indicators to the international ones barely improved predictive power at all. Multilevel models told a consistent story, with the final predictor sets explaining 87 percent of between-school variance in TIMSS and 76 percent in PISA, despite relatively modest intraclass correlations of 21 percent and 11 percent respectively.

The authors are careful to note that the national indicators should not be dismissed as redundant simply because they add little explanatory power once international measures are controlled for. Loss of statistical significance in a combined model does not mean two variables measure the same construct, and the national indicators each capture unique variance consistent with their different conceptual origins. The PISA ESCS index reflects a gradient approach to SES, combining parental occupation, parental education and home possessions; the TIMSS Home Educational Resources scale draws on books in the home, study supports and parental education; the HP index is an area-level census composite; and DEIS status is an administrative designation shaped by policy history. Each is a different lens on the same underlying phenomenon.

Nevertheless, the findings carry a clear practical message for governments: wherever feasible, SES data should be collected at the student level rather than relying solely on school-level designations. Indicators aggregated from individual data tracked each other more tightly and predicted achievement more powerfully than inherently school-level labels. This matters because higher-level designations can blur over time, as Ireland’s own experience with successive waves of DEIS designation illustrates, complicating efforts to monitor outcomes. The lesson also echoes international research: in Sweden, register-based SES explained less variance in mathematics achievement than TIMSS measures and added almost nothing beyond them, while French work on TIMSS 2023 found the national Social Position Index correlated less strongly with performance than the international home resources indices.

For a field increasingly dependent on large-scale assessment data to guide equity policy, this study provides a methodological template. By linking TIMSS and PISA data to national administrative records, countries can audit their own indicators of disadvantage and decide, on evidence rather than habit, which measures deserve a place in funding formulae and research. Given that the OECD has urged Ireland to refine and validate the indicators underpinning its resource allocation model, and that similar debates over free school meals, Universal Credit eligibility and register-based measures are playing out in the United Kingdom, the Netherlands and Belgium, the Irish findings arrive at an opportune moment. Principals’ perceptions, census indices and international survey composites all tell substantially the same story about which schools serve disadvantaged communities. But if the goal is to understand and predict how socio-economic context shapes learning, the richest measures remain those built from the experiences of students themselves.

Beyond its immediate policy relevance, the study speaks to a long-standing methodological puzzle in comparative education research: the fact that no single operational definition of socio-economic status exists. Because SES is a latent construct, every measure — whether built from survey questionnaires, census small-area statistics, or administrative designations — is an imperfect proxy. This is precisely why validity evidence matters more than any one indicator’s apparent sophistication. A census-based index can be statistically elegant yet blunt at the level of individual schools, while a survey composite can capture family circumstances that area-level averages smooth over.

The choice of mathematics achievement as the criterion variable is also worth noting. Mathematics performance is often regarded as particularly sensitive to family resources, since out-of-school support — private tutoring, study materials, parental familiarity with the curriculum — tends to track household advantage more closely than in some other subjects. This may partly explain why the home-resources-based international measures showed such consistent predictive strength across both TIMSS and PISA cohorts in the study.

The multilevel modelling approach adopted by the researchers also deserves attention from an international audience. Because achievement data in assessments such as TIMSS and PISA are nested within schools, analysing them with single-level regression risks overstating the precision of school-level effects. By explicitly modelling between-school variance and reporting intraclass correlations alongside explained variance at each level, the study offers a transparent account of how much of the socio-economic gradient operates between rather than within schools. The relatively high proportion of between-school variance explained suggests that school-level SES indicators, whatever their construction, capture a substantial share of the structural differences among institutions.

Finally, the study’s timing is significant. As education systems grapple with rising data expectations, integrating international assessment data with national administrative records remains rare in practice, often blocked by technical or legal barriers rather than analytical ones. The Irish example demonstrates that such linkage is feasible and can be undertaken with existing public-use and administrative datasets, lowering the threshold for other countries to subject their own equity instruments to comparable empirical scrutiny.

Subject of Research: Comparing the validity of school socio-economic status measures from TIMSS and PISA against Irish national administrative indicators and their power to predict mathematics achievement

Article Title: The validity of school socio-economic status measures: how do indicators from TIMSS and PISA compare to national administrative data sources?

Article References: Gilleece, L., Karakolidis, A., & Clerkin, A. (2026). The validity of school socio-economic status measures: how do indicators from TIMSS and PISA compare to national administrative data sources?. Large-scale Assessments in Education, 14(1), Article 45. https://doi.org/10.1186/s40536-026-00305-x

Image Credits: AI Generated

DOI: 10.1186/s40536-026-00305-x

Keywords: socio-economic status, TIMSS, PISA, educational disadvantage, Ireland, DEIS, Pobal HP Deprivation Index, mathematics achievement, validity, large-scale assessments, education policy, administrative data