The research, conducted by Assistant Professor Shingo Nahatame of the Institute of Human Sciences at the University of Tsukuba and Associate Professor Tomoko Ogiso of the Faculty of Education at the University of Toyama, was published in the journal Reading & Writing. The study set out to address a central concern in English language education: how to evaluate the difficulty of the texts used in educational materials and language assessments. Traditionally, researchers have relied on objective measures of text difficulty grounded in linguistic features, including characteristics of words and sentences. Yet learners can differ substantially in how difficult they perceive the very same text to be, and understanding that gap matters for anyone who selects, writes, or assesses reading material.

To examine the relationship between objective and subjective text difficulty, the researchers designed a study with two complementary strands. First, they evaluated the objective difficulty of 30 English reading passages spanning a broad range of difficulty levels. Objective difficulty was assessed in two ways: using established readability measures based on linguistic characteristics, and using an evaluation method based on a large language model, reflecting the growing role of artificial intelligence tools in text analysis. This dual approach allowed the team to compare how well both traditional formula-based indices and modern computational methods captured the demands each passage placed on readers.

The second strand of the study turned to the readers themselves. Forty-one undergraduate and graduate students whose first language was Japanese read the same 30 passages and rated the difficulty of each one on a nine-point scale. These ratings provided a direct window into subjective difficulty—the felt experience of struggling with, or sailing through, a text. Because all participants read all passages, the design made it possible to compare, passage by passage and reader by reader, how objective estimates lined up with personal perceptions.

The analyses revealed a strong relationship between objective and subjective difficulty. Passages rated as more difficult by objective measures were also perceived as more difficult by readers. In other words, the traditional intuition behind readability formulas holds up: texts with longer sentences, rarer words, and more complex linguistic characteristics genuinely do feel harder to read, at least on average. The large language model–based evaluation method, likewise, produced difficulty estimates that corresponded with readers’ perceptions, suggesting that computational approaches can serve as viable complements to conventional readability indices.

But the study did not stop at ratings. The researchers also recorded participants’ eye movements as they read, adding a process-level measure of reading behavior to the product-level measures of text features and subjective judgments. Eye-tracking has long been used in reading research because the eyes provide a moment-by-moment record of cognitive effort: fixations lengthen when processing is demanding, and readers skip words that are easy to identify. In this study, objective and subjective difficulty measures were both associated with eye-movement patterns during reading. More difficult texts tended to be read more slowly, and readers skipped fewer words when reading them. The convergence is striking: three independent sources of evidence—linguistic formulas, learner judgments, and real-time gaze behavior—all pointed in the same direction.

The most novel finding, however, concerned the discrepancy between the two kinds of difficulty. The researchers found that the extent to which an individual reader’s rating differed from the group’s average rating for the same passage provided additional information for explaining eye-movement behavior beyond what could be accounted for by text-level difficulty measures alone. In practical terms, if a particular student found a passage harder than their peers did, that deviation was reflected in the way their eyes moved through the text, even after the general difficulty of the passage had been taken into account. Subjective difficulty, in other words, is not simply a noisy version of objective difficulty—it carries unique, reader-specific signal.

These findings indicate that objective measures of text difficulty and subjective perceptions of learners generally correspond closely but are not identical. Objective measures are useful for characterizing the general difficulty of a text, and they remain an efficient tool for grading passages, leveling curricula, and calibrating assessments across large populations. But subjective evaluations provide additional insight into how individual learners experience and process the same text. A readability score describes the passage; a learner’s rating describes the encounter between the passage and a particular mind. The study suggests that both perspectives are needed to build a complete picture of reading difficulty.

The implications reach into several corners of language education. For teachers and materials developers, the results support a two-step approach: objective measures can be used to shortlist texts at an appropriate general level, while learner feedback can then be used to fine-tune selections for specific students or classes. For researchers, the demonstration that rating deviations predict eye-movement behavior opens the door to more personalized models of second-language reading, in which individual differences in perception are treated as meaningful data rather than measurement error. The work also speaks to the growing interest in using large language models for text evaluation, showing that such tools can align with human perceptions while still leaving room for individual variability.

More broadly, the study is a reminder that difficulty is not a property of a text alone, nor of a reader alone, but of the interaction between them. The 30 passages, 41 readers, nine-point ratings, and recorded eye movements together paint a technically grounded picture of that interaction: strong average correspondence between formulas and feelings, reliable links between both and the microstructure of gaze, and a persistent, informative residue of individual difference. As English language education moves toward more personalized approaches to reading support, the message from Tsukuba and Toyama is clear—measure the text, but also listen to the reader, because the two are related, yet not the same.

Subject of Research: The relationship between objective and subjective text difficulty in second-language English reading and its link to eye-movement behavior

Article Title: Subjective and objective difficulty of English texts are related—but not the same

Article References: Subjective and objective difficulty of English texts are related—but not the same. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: text difficulty, readability, second-language reading, eye-tracking, English language education, large language models, reading assessment, subjective perception, University of Tsukuba, Reading & Writing, L2 learners, reading materials

The textbooks under scrutiny were written in 2020 and officially launched by Ethiopia’s Ministry of Education in 2022 for use across all primary grades. Since their introduction, parents and teachers have been divided. Some complained that the reading sections left children frustrated; others insisted the texts were appropriate or appropriately demanding. Wubalem set out to resolve this dispute empirically, asking whether the texts sit at the right readability level for their target age groups, which books are most affected, and what specific features of the texts drive the difficulty.

The methodological design was deliberately multi-pronged. Every reading passage in the four textbooks was assessed with Brown’s recalculated Flesch readability formula, an adaptation widely used in foreign language settings such as South Korea and Japan because it sets age-appropriate difficulty thresholds for EFL learners. A systematically drawn 35 percent sample of passages was then converted into cloze tests, in which every seventh content word was deleted, and administered to 280 learners, 70 from each grade. To separate genuine text difficulty from student weakness, the same students also completed standard cloze tests calibrated to A1/A2 levels of the Common European Framework of Reference. Finally, 64 teachers were identified through multi-stage sampling across the Amhara region, with 20 interviewed until thematic saturation was reached.

The Flesch-based results were striking. Grade 3 readers, ideally scored at a readability level of 98 for children aged roughly 8.5 to 10, produced scores ranging from 54.7 to 88, with several passages appropriate only for fifth, seventh, or even ninth graders. The grade 4 textbook fared comparatively well, averaging 84.6 against a benchmark of 95, and three of its passages met the minimum requirement. Grades 5 and 6 fared worst: grade 5 passages averaged 66.42 against an expected range of 75 to 90, and grade 6 texts averaged 67.4 against a benchmark of 80, with the most difficult passage suitable only for twelfth graders. Across the series, longer sentences and a surplus of polysyllabic words were the principal culprits.

Surface formulas, however, only reveal part of the picture, and critics have long argued that readability indexes built on sentence length and syllable counts ignore deeper cognitive dimensions of reading. A word like ‘television’ is polysyllabic yet concrete and familiar, while a monosyllabic word like ‘myth’ may defeat a young learner. To address this, Wubalem ran every passage through Coh-Metrix, a computational linguistics tool that quantifies lexical easability, syntactic density, syntactic cohesion, and semantic richness. The lexical analysis found that concreteness, imaginability, meaningfulness, and age-of-acquisition scores all exceeded expected cutoffs by roughly 3 to 5 points, meaning the passages were saturated with abstract words acquired late in a native speaker’s development. Grade 3 texts, for example, asked eight-year-olds to process terms such as ‘sustainability,’ ‘biodiversity,’ ‘photosynthesis,’ and ‘deforestation.’

Syntactic analysis told a similar story. Coh-Metrix norms suggest that young learners should encounter heavily modified noun, verb, and prepositional phrases in no more than about two sentences out of ten, with agentless passives and left-embedded clauses nearly absent. While grade 5 texts were only moderately dense, several passages in grades 3, 5, and 6 packed stretched modifiers, embedded clauses, and passive constructions into single sentences. One grade 6 passage on mobile phones contained a sentence with multiple embedded qualifiers and a final gerund clause requiring sophisticated parsing. From the perspective of cognitive load theory, such constructions overwhelm working memory precisely when grammatical processing has not yet been automatized, diverting mental resources away from meaning-making.

Interestingly, cohesion emerged as the texts’ one genuine strength. Measures of referential cohesion, stem overlap, argument overlap, and temporal cohesion were largely above the norms for each grade level, meaning the texts do supply explicit linguistic threads linking sentences. Yet the study identified what it calls a threshold effect: the extreme lexical and syntactic complexity exhausted learners’ cognitive resources before those cohesive scaffolds could be exploited. In other words, children could not benefit from well-connected discourse because they were already drowning in unfamiliar vocabulary and convoluted grammar. A related deficit appeared in semantic richness, with WordNet connectivity, latent semantic overlap, and intentionality falling below expected levels in most passages, forcing learners to make excessive inferences to construct coherent mental models of the content.

The cloze tests provided the human validation for these computational findings. Readability conventions classify texts as independent if students score above 60 percent on cloze measures, instructional between 40 and 60 percent, and frustrating below 40 percent. Learners in grades 3, 5, and 6 all scored below 35 percent on the textbook-derived tests, squarely in the frustration zone, while grade 4 students averaged 45 percent, reaching instructional level. Crucially, the same students scored more than 50 percent on standard cloze tests calibrated to their curriculum level, and independent-samples comparisons showed statistically significant gaps of 21 points in grade 3, 25 points in grade 5, and 26.5 points in grade 6. This pattern demonstrates that the failure lies in the materials, not in the children’s underlying comprehension abilities, and that the texts fall outside the learners’ zone of proximal development.

Teacher interviews reinforced the quantitative evidence. One grade 3 teacher described the texts as full of complex, abstract words that children could not process; another reported that repeated attempts to train inference and contextual word-guessing strategies failed because the passages were too complex to practice on. A third pointed to subordinate clauses, coordinating conjunctions, participle phrases, and abstract vocabulary blocking comprehension in grades 3 and 5. Educational administrators added an important caveat: learners’ difficulties were compounded by insufficient prior language input, pandemic-era schooling disruptions, and security problems in recent years, suggesting cautious interpretation of the absolute scores even as the text-based deficits remain clear.

The study’s implications are direct. Wubalem calls for aggressive simplification of overly complex passages, strategic replacement of the worst offenders, robust teacher training, and ongoing resources to help educators mediate texts that cannot be fixed quickly. Grade 4 demonstrates that appropriate leveling is achievable within the same national curriculum framework, making the failures of grades 3, 5, and 6 all the more conspicuous. The findings also speak to a broader global problem, since readability research remains scarce across much of Africa, Asia, and Latin America, where textbooks are often the only printed source of target-language input. As generative text-analysis tools like Coh-Metrix become more accessible, the study suggests education systems worldwide can no longer plead ignorance about whether their reading materials match the children assigned to read them.

Subject of Research: Readability assessment of reading passages in Ethiopian primary EFL textbooks for grades 3–6.

Article Title: An assessment on the accessibility of reading passages in Ethiopian primary EFL textbooks

Article References: Wubalem, A. (2026). An assessment on the accessibility of reading passages in Ethiopian primary EFL textbooks. Discover Education, 5(1), Article 899. https://doi.org/10.1007/s44217-026-02113-5

Image Credits: AI Generated

DOI: 10.1007/s44217-026-02113-5

Keywords: readability, EFL textbooks, Coh-Metrix, Flesch readability formula, cloze test, lexical density, syntactic complexity, semantic richness, reading comprehension, Ethiopia, primary education, text difficulty