Enjoyment of reading shows measurement invariance across grades, IRT study finds

Children do not experience the same questionnaire when they report how much they love reading, according to a new study that examined whether the world’s most widely used measure of reading enjoyment actually means the same thing to a six-year-old as it does to a sixteen-year-old. The research, published in the journal Large-scale Assessments in Education, suggests that the answer is no for the youngest pupils, a finding with potentially significant consequences for how countries interpret international education data and how schools assess their own students’ reading motivation.

The study was conducted by Morten Rasmus Augestad-Puck of Konsulenthuset MR Puck in Norway and Simon Skov Fougt of Aarhus University in Denmark. Drawing on the “Students Like Reading” scale from the Progress in International Reading Literacy Study, known as PIRLS, the researchers surveyed an extraordinary sample: essentially every pupil in the public school system of a single Danish municipality, from the first year of school through the tenth and final year, covering ages six to sixteen. In total, 5,780 pupils were targeted, and 5,463 completed the survey, a completion rate of 95 percent. Because the researchers secured participation from virtually the entire population rather than a sample, no statistical weighting was needed, and the resulting dataset offers an unusually clean window into how the measurement of reading enjoyment behaves across an entire school career.

At the heart of the study lies a deceptively simple psychometric question. Reading enjoyment is what statisticians call a latent trait: it cannot be observed directly, only inferred from responses to questionnaire items. If the instrument is to be used to compare groups, whether grades, countries, or cohorts, then it must display what is known as measurement invariance. A pupil in grade 0 and a pupil in grade 9 with the same underlying level of reading enjoyment should have the same probability of responding to a given answer category on any given item. When this assumption breaks down, the phenomenon is called differential item functioning, or DIF: two pupils with identical levels of the measured trait respond differently to an item simply because they belong to different groups. Undetected DIF means some groups are systematically advantaged or disadvantaged on the scale, so any observed differences between groups may reflect flaws in the measuring instrument rather than genuine differences in the trait itself.

The theoretical motivation for the study is grounded in self-determination theory, which distinguishes intrinsic motivation, reading for its own sake, from extrinsic motivation, reading for utility or reward. PIRLS, administered to fourth graders, deliberately targets the joy-based, intrinsic side of reading motivation, while PISA, aimed at fifteen-year-olds, incorporates a more utilitarian framing. Decades of research link intrinsic reading enjoyment to better reading comprehension, richer vocabulary, and stronger academic outcomes, and the relationship appears reciprocal: children who read become better readers, and better readers enjoy reading more. Both PIRLS and PISA report substantial correlations between enjoyment of reading and reading ability. But because each of these international studies tests only its own target age group, no one had previously examined whether the enjoyment-of-reading scale functions equivalently across the whole span of elementary schooling, from children who have only just learned to decode text to teenagers on the cusp of adulthood.

The researchers adapted the PIRLS scale, retaining seven of its ten items after removing negatively worded questions that might demand more cognitive sophistication than young children possess. Items include statements such as “I learn a lot from reading” and statements about books helping the reader imagine other worlds. Pupils responded on a four-point Likert scale running from “Agree a lot” to “Disagree a lot,” and, in an accommodation for the youngest respondents, each response category was paired with an emoji intended to reinforce its emotional meaning. Teachers were instructed to read the items aloud where necessary, so that even pupils who could not yet read fluently could participate. Responses were reverse-coded and analyzed in R using the mirt package for item response theory modeling.

The technical analysis proceeded in carefully staged steps. The researchers first compared two item response theory models: the Rating Scale Model, which assumes that the thresholds between response categories are perceived identically regardless of the item, and the Partial Credit Model, which allows each item to have its own threshold structure. Information criteria including AIC, BIC, and log-likelihood all favored the Partial Credit Model, which became the baseline for the DIF analysis. Item fit statistics, the outlier-sensitive Outfit and the variance-weighted Infit, then revealed that most items fit the model imperfectly, a pattern that mirrors, though slightly exceeds, the fit statistics reported in official PIRLS analyses. Only the item “I learn a lot from reading” showed acceptable fit, hinting that some form of DIF was lurking in the scale.

The DIF investigation itself unfolded in two parts. First, the researchers examined the prior distributions of person ability used in the marginal maximum likelihood estimation, which employs an expectation-maximization algorithm in which individual ability estimates gravitate toward the mean of their group’s prior distribution. With grade 4, PIRLS’s own target population, serving as the reference group, the younger grades showed systematically higher prior means than the older grades. Then the researchers freed the item parameters for all items except a single anchor item, the best-fitting item, allowing each grade group its own estimates in a procedure known as a DIF split. Modern Wright maps, which visualize threshold parameters by group, revealed that grades 0 through 2 displayed a distinctly different pattern of item parameters and thresholds than grades 3 through 9.

The pattern that emerged was striking in its simplicity. The scale does not shift gradually year by year; instead, it splits into two clusters. Pupils in grades 0 through 2, roughly ages six to eight, perceive the items and response thresholds in a fundamentally different way than pupils in grades 3 through 9, ages nine to sixteen. Within each of these larger clusters, the measurement behaves reasonably consistently, but across the dividing line, it does not. The researchers offer several plausible explanations rooted in developmental psychology. Younger children may lack the self-awareness to articulate stable preferences about reading. The perceived benefit of reading may diminish with age, which would distort responses to the item about learning from reading. And children’s purposes for reading may mature, shifting from imagination and entertainment toward reflection and development, changing what statements like “books help me imagine other worlds” actually mean to the respondent.

The study also candidly examines its own accommodations as potential sources of DIF. The read-aloud procedure used for the youngest pupils could introduce misunderstandings: a child might mishear an item or forget its wording by the time the response options are read. The emoji-augmented response scale, likewise, might be interpreted differently from the verbal categories it accompanies, and pupils who respond based on the emoji alone would effectively be taking a different instrument than their peers. Even the design of the items themselves, calibrated for fourth graders with a certain level of cognitive development, may be misperceived by younger children who have not yet reached that developmental milestone.

The consequences for practice are concrete. The researchers compared group means and confidence intervals under three modeling scenarios: identical item parameters and priors across all grades, free priors with fixed items, and fully free parameters with one anchor. Ignoring DIF altogether compresses the differences between age groups, underestimating genuine developmental change. Fully adjusting for DIF widens the estimated gaps between groups without inflating standard errors, yielding more statistically significant and arguably more honest comparisons. The practical recommendation is twofold: researchers wanting a clean measure across the full age range should either apply DIF-split estimation or, more simply, restrict the scale to grades 3 through 9, where the instrument performs well without adjustment. For longitudinal work tracking individuals as they mature, a properly DIF-adjusted scale could isolate genuine developmental change from mere measurement artifacts.

The findings land at a moment of intense scrutiny of international large-scale assessments. PIRLS and PISA results shape national education policy debates on a regular cycle, and context questionnaires like the reading enjoyment scale feed directly into those debates. This study is a reminder, delivered with unusual statistical rigor and an unusually complete dataset, that the instruments behind the headlines carry developmental assumptions that deserve their own audit. A questionnaire that asks simply whether a child likes to read may seem universally comprehensible, but the way a six-year-old and a fourteen-year-old understand “agree a lot” when it sits beside a statement about reading may be two different acts of measurement entirely.