The cloze task, introduced by Wilson Taylor in 1953, has become the workhorse of language prediction research. Researchers present participants with incomplete sentences and record the distribution of their completions. If most people supply the same word, the sentence is considered highly constraining; if responses scatter widely, it is not. Traditionally, this is quantified as cloze probability, the percentage of participants who produce the most frequent continuation. Some studies treat sentences below 37 percent as low-constraining and those above 67 percent as highly constraining. But cloze probability has a blind spot: a distribution of 60-10-10-10-10 and one of 60-40 yield the same score despite radically different uncertainty. To capture that hidden variability, the researchers also computed response entropy, a measure borrowed from information theory that sums the probabilities of every unique answer, weighted logarithmically. Entropy bottoms out at zero when everyone agrees and climbs toward its maximum when every response is unique. Previous work has suggested entropy may even predict reading behavior better than cloze probability alone.

The critical question Bautista and Martin posed was deceptively simple: does the population used to establish these norms actually matter? The validity of any cloze pre-test depends on how closely the norming sample matches the people who will later be tested in the main experiment. Studies of aging had already shown that cloze responses are stable across adulthood, which allowed researchers to norm materials with young students and use them with older adults. But for language, the intuition that proficiency shapes cloze answers had never been empirically tested. This gap has grown urgent as bilingualism research moves away from comparing native speakers against everyone else and toward more sophisticated designs, such as mirrored language groups or multiple language pairings, that control for confounds the old paradigms ignored.

To fill that gap, the team recruited 240 participants through the online platform Prolific, forming four groups of sixty. Two groups were dominant speakers: people born in France who reported French as their first language, and people born in Spain who reported Spanish as their first language. Two further groups were fluent but non-dominant speakers of French: Spanish-French bilinguals born in Spain and English-French bilinguals born in the United Kingdom. Proficiency was verified with LexTALE vocabulary tests, which confirmed that the two bilingual groups did not differ significantly in their French proficiency, a crucial control. Each participant completed a cloze task on 110 sentence contexts drawn from a shared pool of 330 sentences, each designed to bias toward a final noun phrase such as “a cow” after “On farms, to get milk, it is necessary to milk…”. Every sentence had a French and a Spanish translation equivalent, with matched target nouns in length and lexical frequency.

The first analysis asked whether language itself shapes sentence constraint. French-dominant and Spanish-dominant speakers completed the same sentences in their respective languages, and their average response entropy turned out to be strikingly similar: 1.05 in French versus 0.93 in Spanish, a difference that fell short of statistical significance. On the surface, translation seemed to preserve constraint. But when the researchers examined the item-by-item alignment between the two groups using Spearman correlations, benchmarked against bootstrapped estimates of chance-level and maximum possible alignment, the picture changed. The observed correlation of 0.55 was far above chance yet clearly below the internal consistency of each group, which sat at 0.67. In other words, individual sentences did not reliably elicit the same degree of constraint across the two languages, even though the languages are closely related Romance cousins sharing substantial vocabulary and culture.

One striking example reveals why. The sentence “The elderly woman bought potatoes and eggs because she wanted to prepare…” drove 90 percent of Spanish-dominant speakers to answer “tortilla”, the iconic Spanish omelette, but only 40 percent of French-dominant speakers gave the equivalent response. The constraint difference here reflects culturally specific knowledge rather than anything linguistic. The authors argue that such cultural iconicity can shape cloze answers beyond language per se, and they speculate that even French speakers from France, Quebec, and Senegal might diverge on identical sentences. The methodological implication is blunt: translating cloze materials from one language to another introduces response variability even under strict translation, so the most reliable practice remains pre-testing sentences with the target language community itself.

The second analysis tackled language dominance, comparing the three groups who completed sentences in French. Here the average differences were unmistakable. Response entropy averaged 1.05 for French-dominant speakers but jumped to 1.50 for Spanish-French bilinguals and 1.47 for English-French bilinguals, a highly significant difference. Sentences were simply less constraining for people reading in their non-dominant language, confirming empirically what researchers had long assumed but never demonstrated. Alignment between dominant and non-dominant groups was again only moderate, with correlations of 0.52 and 0.60 falling well short of the maximum-alignment benchmarks. Practically, this means that using cloze norms collected from dominant speakers to design experiments with non-dominant populations will inevitably introduce error in estimates of lexical constraint.

The most surprising result emerged when the two non-dominant groups were compared with each other. Spanish-French and English-French bilinguals, despite having entirely different dominant languages, showed nearly identical average entropy and an item-by-item correlation of 0.70, the highest in the study and the closest to what would be expected if the groups came from a single population. This suggests that what matters most for cloze responses is the speaker’s relationship to the language being tested, not the identity of their other language. Norms gathered from one non-dominant group may transfer to another non-dominant group of similar proficiency, potentially saving researchers an entire norming study. The authors caution, however, that both bilingual groups shared European cultural backgrounds, and the alignment might not extend to more distant pairings such as Mandarin Chinese-French bilinguals.

Both measures of constraint, the traditional cloze probability and the newer response entropy, led to the same conclusions and correlated above 0.96 within every group, suggesting neither needs to be abandoned. Yet the study also exposed limitations in the entropy formula, which can produce counterintuitive values, rating a 50-50 split as less uncertain than a 70-10-10-10 distribution. The authors propose that a normalized version might better capture predictive uncertainty. They also note that even within a single population, split-half consistency never exceeded 0.70, meaning cloze norms are inherently probabilistic estimates rather than fixed standards, and exact scores from a pre-test will never perfectly match those of the main experiment.

The findings reach beyond the laboratory into the age of large language models. Surprisal estimates from these models increasingly predict well-established neural markers of language prediction, such as N400 brain responses and eye movements during reading, but most of that evidence comes from English and other Germanic languages that dominate training data, and the models carry documented biases toward Western cultural norms. If human sentence expectations are shaped by language and dominance, models are unlikely to capture the learning trajectories and sociolinguistic profiles of non-English-dominant speakers. The authors recommend that researchers studying bilingual populations consider bilingual language models trained on comparable language pairs, and those studying monolingual non-English populations use monolingual models for that language. Systematic comparisons between model-derived surprisal and human cloze norms across diverse groups are now needed. The team has released its full database of cloze responses from all four populations to 330 translated sentences through the Open Science Framework, giving the field both the evidence and the resources to stop treating prediction as language-independent.

Subject of Research: How language and language dominance influence lexical sentence constraint measured by cloze tests

Article Title: Not that cloze: Sentence constraint is influenced by language and language dominance

Article References: Bautista, A., & Martin, C. D. (2026). Not that cloze: Sentence constraint is influenced by language and language dominance. Behavior Research Methods, 58(11), Article 317. https://doi.org/10.3758/s13428-026-03140-8

Image Credits: AI Generated

DOI: 10.3758/s13428-026-03140-8

Keywords: cloze test, sentence constraint, language prediction, bilingualism, language dominance, response entropy, psycholinguistics, cross-linguistic differences, large language models, surprisal, Behavior Research Methods, norming

The textbooks under scrutiny were written in 2020 and officially launched by Ethiopia’s Ministry of Education in 2022 for use across all primary grades. Since their introduction, parents and teachers have been divided. Some complained that the reading sections left children frustrated; others insisted the texts were appropriate or appropriately demanding. Wubalem set out to resolve this dispute empirically, asking whether the texts sit at the right readability level for their target age groups, which books are most affected, and what specific features of the texts drive the difficulty.

The methodological design was deliberately multi-pronged. Every reading passage in the four textbooks was assessed with Brown’s recalculated Flesch readability formula, an adaptation widely used in foreign language settings such as South Korea and Japan because it sets age-appropriate difficulty thresholds for EFL learners. A systematically drawn 35 percent sample of passages was then converted into cloze tests, in which every seventh content word was deleted, and administered to 280 learners, 70 from each grade. To separate genuine text difficulty from student weakness, the same students also completed standard cloze tests calibrated to A1/A2 levels of the Common European Framework of Reference. Finally, 64 teachers were identified through multi-stage sampling across the Amhara region, with 20 interviewed until thematic saturation was reached.

The Flesch-based results were striking. Grade 3 readers, ideally scored at a readability level of 98 for children aged roughly 8.5 to 10, produced scores ranging from 54.7 to 88, with several passages appropriate only for fifth, seventh, or even ninth graders. The grade 4 textbook fared comparatively well, averaging 84.6 against a benchmark of 95, and three of its passages met the minimum requirement. Grades 5 and 6 fared worst: grade 5 passages averaged 66.42 against an expected range of 75 to 90, and grade 6 texts averaged 67.4 against a benchmark of 80, with the most difficult passage suitable only for twelfth graders. Across the series, longer sentences and a surplus of polysyllabic words were the principal culprits.

Surface formulas, however, only reveal part of the picture, and critics have long argued that readability indexes built on sentence length and syllable counts ignore deeper cognitive dimensions of reading. A word like ‘television’ is polysyllabic yet concrete and familiar, while a monosyllabic word like ‘myth’ may defeat a young learner. To address this, Wubalem ran every passage through Coh-Metrix, a computational linguistics tool that quantifies lexical easability, syntactic density, syntactic cohesion, and semantic richness. The lexical analysis found that concreteness, imaginability, meaningfulness, and age-of-acquisition scores all exceeded expected cutoffs by roughly 3 to 5 points, meaning the passages were saturated with abstract words acquired late in a native speaker’s development. Grade 3 texts, for example, asked eight-year-olds to process terms such as ‘sustainability,’ ‘biodiversity,’ ‘photosynthesis,’ and ‘deforestation.’

Syntactic analysis told a similar story. Coh-Metrix norms suggest that young learners should encounter heavily modified noun, verb, and prepositional phrases in no more than about two sentences out of ten, with agentless passives and left-embedded clauses nearly absent. While grade 5 texts were only moderately dense, several passages in grades 3, 5, and 6 packed stretched modifiers, embedded clauses, and passive constructions into single sentences. One grade 6 passage on mobile phones contained a sentence with multiple embedded qualifiers and a final gerund clause requiring sophisticated parsing. From the perspective of cognitive load theory, such constructions overwhelm working memory precisely when grammatical processing has not yet been automatized, diverting mental resources away from meaning-making.

Interestingly, cohesion emerged as the texts’ one genuine strength. Measures of referential cohesion, stem overlap, argument overlap, and temporal cohesion were largely above the norms for each grade level, meaning the texts do supply explicit linguistic threads linking sentences. Yet the study identified what it calls a threshold effect: the extreme lexical and syntactic complexity exhausted learners’ cognitive resources before those cohesive scaffolds could be exploited. In other words, children could not benefit from well-connected discourse because they were already drowning in unfamiliar vocabulary and convoluted grammar. A related deficit appeared in semantic richness, with WordNet connectivity, latent semantic overlap, and intentionality falling below expected levels in most passages, forcing learners to make excessive inferences to construct coherent mental models of the content.

The cloze tests provided the human validation for these computational findings. Readability conventions classify texts as independent if students score above 60 percent on cloze measures, instructional between 40 and 60 percent, and frustrating below 40 percent. Learners in grades 3, 5, and 6 all scored below 35 percent on the textbook-derived tests, squarely in the frustration zone, while grade 4 students averaged 45 percent, reaching instructional level. Crucially, the same students scored more than 50 percent on standard cloze tests calibrated to their curriculum level, and independent-samples comparisons showed statistically significant gaps of 21 points in grade 3, 25 points in grade 5, and 26.5 points in grade 6. This pattern demonstrates that the failure lies in the materials, not in the children’s underlying comprehension abilities, and that the texts fall outside the learners’ zone of proximal development.

Teacher interviews reinforced the quantitative evidence. One grade 3 teacher described the texts as full of complex, abstract words that children could not process; another reported that repeated attempts to train inference and contextual word-guessing strategies failed because the passages were too complex to practice on. A third pointed to subordinate clauses, coordinating conjunctions, participle phrases, and abstract vocabulary blocking comprehension in grades 3 and 5. Educational administrators added an important caveat: learners’ difficulties were compounded by insufficient prior language input, pandemic-era schooling disruptions, and security problems in recent years, suggesting cautious interpretation of the absolute scores even as the text-based deficits remain clear.

The study’s implications are direct. Wubalem calls for aggressive simplification of overly complex passages, strategic replacement of the worst offenders, robust teacher training, and ongoing resources to help educators mediate texts that cannot be fixed quickly. Grade 4 demonstrates that appropriate leveling is achievable within the same national curriculum framework, making the failures of grades 3, 5, and 6 all the more conspicuous. The findings also speak to a broader global problem, since readability research remains scarce across much of Africa, Asia, and Latin America, where textbooks are often the only printed source of target-language input. As generative text-analysis tools like Coh-Metrix become more accessible, the study suggests education systems worldwide can no longer plead ignorance about whether their reading materials match the children assigned to read them.

Subject of Research: Readability assessment of reading passages in Ethiopian primary EFL textbooks for grades 3–6.

Article Title: An assessment on the accessibility of reading passages in Ethiopian primary EFL textbooks

Article References: Wubalem, A. (2026). An assessment on the accessibility of reading passages in Ethiopian primary EFL textbooks. Discover Education, 5(1), Article 899. https://doi.org/10.1007/s44217-026-02113-5

Image Credits: AI Generated

DOI: 10.1007/s44217-026-02113-5

Keywords: readability, EFL textbooks, Coh-Metrix, Flesch readability formula, cloze test, lexical density, syntactic complexity, semantic richness, reading comprehension, Ethiopia, primary education, text difficulty