ChatGPT Feedback Beat Teacher Comments in EFL Writing Trial, With Caveats

University students learning English as a foreign language made larger gains in overall writing quality when their drafts were critiqued by ChatGPT rather than by their own instructor, according to a four-week mixed-methods study published in Discover Education. The comparison, conducted with 46 upper-intermediate learners at a large public university, found a statistically robust baseline-adjusted advantage for the AI feedback condition on overall writing performance, but a far more complicated picture when researchers examined individual writing skills and asked students how they actually used the comments they received.

The study, led by Neda Naqdi Babaee, adopted a nonrandomized pretest-posttest design. Twenty-five learners received ChatGPT-generated feedback while 21 received teacher-written comments from the same instructor, who taught both groups within an identical instructional sequence. Across two complete feedback-revision cycles, students in each condition wrote argumentative and expository paragraphs of roughly 200 words and revised them after receiving comments. Writing was scored by two trained raters, blind to condition and testing phase, using adapted IELTS-based analytic descriptors covering task achievement, coherence and cohesion, lexical resource, and grammatical accuracy, with the four domain scores averaged into an overall composite.

The headline result was clear-cut. After adjusting posttest scores for corresponding pretest performance, the ChatGPT condition showed a robust advantage in overall writing, F(1, 43) = 16.86, p < .001, with a partial eta squared of 0.282. None of the baseline differences between groups was statistically significant, and tests confirmed that the relationship between pretest and posttest scores was comparable across conditions, supporting the validity of the adjustment approach.

The domain-level results demanded more caution. Unadjusted tests favored the ChatGPT condition for task achievement, coherence and cohesion, and lexical resource, but not for grammatical accuracy. Because five related outcomes were examined, the researchers applied a Holm adjustment across the four domain tests as a familywise sensitivity analysis. Only task achievement survived this correction, with an adjusted p value of .031. Lexical resource, at an adjusted p of .057, and coherence and cohesion, at .081, fell just above the threshold and were interpreted as suggestive rather than confirmatory. Robustness checks told a similar story: raw posttest comparisons, nonparametric tests, and Hedges g values ranging from 0.69 to 1.27 supported the ChatGPT advantage for several outcomes, but gain-score analyses reached the conventional significance boundary only for overall writing.

Perhaps the most practically important null finding concerned grammar. Both conditions directed attention to grammatical accuracy, and both groups improved descriptively over the four weeks, yet neither the adjusted models nor any of the sensitivity analyses detected a reliable difference between them. The authors suggest that short interventions may be insufficient to distinguish grammatical development, which typically requires repeated practice and sustained attention to recurring error patterns. The result, they stress, does not establish complementary roles for the two feedback sources; it indicates only that the implemented ChatGPT condition offered no clear grammatical advantage over teacher comments.

The qualitative strand of the study, based on open-ended post-intervention responses analyzed through reflexive thematic analysis, revealed two broad themes: actionable revision support, and trust, control, and feedback authority. Students in the ChatGPT condition repeatedly described the AI comments as immediate and elaborated. One learner explained that the feedback did not simply flag a weak paragraph but showed that an example was insufficient and suggested how to explain it more clearly, helping revise content rather than only grammar. Another said the most useful part was organization, because the system identified where connections between sentences were unclear, allowing new linking words or reordering of ideas — a shift, the student noted, from a previous habit of only checking grammar.

Vocabulary emerged as a particularly revealing area of learner experience. Students reported that ChatGPT offered alternatives for repeated or imprecise wording, expanding their options for academic expression, but they also emphasized the responsibility to judge appropriateness. One participant noted that some suggested words were too formal for their level and had to be checked against the intended sentence. This combination of availability and selection — a wider menu of choices paired with the learner’s own evaluative filtering — recurred throughout the accounts and is consistent with the favorable but statistically inconclusive lexical pattern in the quantitative results.

Trust worked differently across the two conditions. ChatGPT users valued speed and detail but did not treat the output as uniformly authoritative; several reported that suggestions sometimes altered their intended meaning or produced grammatical advice that made sense only in context, so they read carefully and rejected comments that distorted their ideas. Some linked immediacy to momentum, saying that instant access let them revise while their intended meaning was still fresh, and one student reported feeling more independent and confident. Teacher feedback recipients, by contrast, trusted comments because the instructor knew the course, the task, and the assessment expectations — but that authority could narrow revision. Several said they corrected only the parts the teacher marked and did not explore alternative versions, appreciating the prioritization while sometimes wanting more explanation or examples for content expansion and lexical refinement.

The authors are explicit that these findings concern implemented feedback conditions, not feedback source in isolation. The ChatGPT condition bundled AI generation with rapid access, extensive comments, and revision alternatives, while the teacher-written condition was more selective and contextually situated, so the estimated difference may reflect differences in amount, timing, specificity, and elaboration rather than the mere presence of AI. The study also cannot claim complementarity, because it included no combined ChatGPT-plus-teacher condition; a sequenced approach remains a pedagogical proposition for future testing rather than an empirical conclusion. Additional limitations include the nonrandomized allocation, the modest single-university sample, the four-week duration without a delayed posttest, and the fact that the qualitative strand captured perceptions rather than observed draft-to-draft uptake.

For educators navigating the rapid arrival of generative AI in language classrooms, the study offers a measured message. AI-generated feedback appears most useful when learners need elaborated options for task fulfillment, organization, or lexical revision, provided instruction requires them to evaluate suggestions against meaning and task criteria. The null grammatical result warns against assuming such tools can replace explicit grammar instruction or sustained corrective-feedback practice, and teacher feedback retains a distinct role in prioritizing problems and aligning revision with course goals. What the research most clearly demonstrates is not that ChatGPT is a superior teacher, but that the way feedback is delivered — how much, how fast, and how elaborated — may matter as much as who or what produces it.

Subject of Research: Comparing ChatGPT-generated and teacher-written feedback effects on EFL university students' writing development

Article Title: Comparing ChatGPT and teacher feedback for EFL writing development

Article References: Babaee, N. N. (2026). Comparing ChatGPT and teacher feedback for EFL writing development. Discover Education, 5(1), Article 1211. https://doi.org/10.1007/s44217-026-02329-5

Image Credits: AI Generated

DOI: 10.1007/s44217-026-02329-5

Keywords: ChatGPT, generative AI, EFL writing, written corrective feedback, second language writing, teacher feedback, quasi-experimental study, ANCOVA, learner perceptions, task achievement, lexical resource, grammatical accuracy