Multi-Agent Language Model Architecture Generates Rubric-Grounded Feedback for Language Learners

A computational framework combining error detection, rubric-aware retrieval, and a bounded two-agent refinement loop has demonstrated promising descriptive capabilities in providing structured pedagogical feedback on English-as-a-Foreign-Language student writing. Developing and delivering high-quality written guidance at classroom scale has long represented an operational challenge in educational institutions. While contemporary large language models can generate fluent textual commentary rapidly, unconstrained models frequently suffer from recognizable pedagogical failure modes, such as over-emphasizing surface grammatical anomalies, drifting away from established institutional grading criteria, hallucinating errors that are not present in learner drafts, or disclosing complete sentence rewrites that eliminate the revision process for the writer. To address these operational constraints, the proposed system, designated as RUMAR, integrates multi-dimensional error classification and domain-grounded retrieval with a deterministic controller that regulates interaction between a generative language model and an explicit rubric-based critic.

The system formulates the delivery of formative writing commentary as a structured output generation task governed by pedagogical theory. The input sequence comprises an individual learner essay, an external target evaluation rubric, and an offline curated knowledge base containing rubric descriptors alongside proficiency-matched exemplar essays. The underlying pedagogical framework targets four distinct linguistic dimensions: grammar, coherence, argumentation, and vocabulary. Rather than presenting monolithic commentary, the generated feedback within each targeted dimension is formally organized across the three hierarchical pedagogical layers identified by educational researchers Hattie and Timperley: feed-up, which clarifies target goals; feed-back, which evaluates current execution against standards; and feed-forward, which outlines actionable revision pathways. By distributing feedback across these three explicit layers, the architecture seeks to balance immediate error identification with forward-looking cognitive scaffolding.

To maintain rigorous computational boundaries, the system architecture explicitly partitions functional duties across four modular components designated M1 through M4, reserving the technical definition of an agent strictly for components backed by a large language model with dedicated role prompts. Modules M1, M2, and M4 function as deterministic computational stages, whereas Module M3 constitutes the primary generative core housing two dedicated language-model agents: a Generator and a Critic. Module M1 handles multi-dimensional error detection and dimension attribution. Module M2 performs contextual retrieval from the curated knowledge repository. Module M3 executes the iterative generation, evaluation, and targeted refinement cycle. Finally, Module M4 aggregates accepted per-dimension commentary into a consolidated final output. This separation prevents a single language model from simultaneously attempting to diagnose errors, retrieve supporting evidence, synthesize advice, and self-evaluate compliance without structural verification.

The first pipeline stage, Module M1, employs a fine-tuned DeBERTa-v3-large transformer model optimized simultaneously for span-level error detection and sentence-level argumentation probing. Span identification operates as a BIO-style sequence tagging task across twenty-nine distinct output labels based on grammatical error correction conventions. The model processes input tokens through overlapping windows of 512 tokens with a 64-token stride to accommodate longer student drafts, subsequently merging predictions at the individual token level. Concurrently, an integrated argumentation classification probe assesses sentence-level discourse functions, attributing claim, evidence, and warrant roles. Joint model optimization minimizes a composite loss weighting span cross-entropy at eighty-five percent and argumentation loss at fifteen percent using the annotated BEA-2019 corpus. Predicted error spans are mapped to the four pedagogical dimensions, yielding a structured inventory of detected anomalies alongside an essay-length-normalized severity metric that determines whether the writer exhibits low, mid, or high overall proficiency.

In the second stage, Module M2 grounds downstream generation in external instructional evidence by accessing an offline vector database. The underlying knowledge repository is populated with band-specific rubric descriptors, teacher-authored feedback templates, and training-partition exemplar excerpts extracted from the Automated Student Assessment Prize and English Language Learner Insight, Proficiency and Skills Evaluation corpora. Textual entries are mapped into 1024-dimensional normalized vector representations using an off-the-shelf BGE-large-EN-v1.5 embedding encoder. During live execution, Module M2 constructs four dimension-specific queries integrating the inferred student proficiency band, detected error summaries from Module M1, and representative essay excerpts. Fast inner-product similarity search executed via FAISS retrieves the top five context items per pedagogical dimension. To prevent data leakage, exact document identifiers from held-out evaluation splits were partitioned prior to indexing, ensuring test-set essays remained excluded from retrieval candidates.

Module M3 implements the core Generation-Evaluation-Refinement loop, iterating sequentially across grammar, coherence, argumentation, and vocabulary. For each dimension, a dedicated Prompt Builder synthesizes the detected errors from Module M1, the retrieved exemplar context from Module M2, a structured pedagogical template, and any targeted repair directives issued during preceding rejection rounds. The Generator agent, implemented either as an eight-billion-parameter open-source model fine-tuned via Low-Rank Adaptation on teacher feedback or as a commercial frontier model, generates a draft feedback candidate restricted to a ceiling of two hundred words. Generation parameters are constrained to a sampling temperature of 0.3 and top-p probability truncation of 0.9. Rather than relying on open-ended conversational debate among symmetrical models, the candidate commentary is immediately routed to a distinct Critic agent that operates deterministically at temperature zero to ensure scoring consistency across evaluation passes.

The Critic agent assesses the generated draft against five explicitly anchored evaluation criteria, assigning integer ratings from one to five for factual correctness relative to the essay, span-level specificity, actionable guidance, proficiency-band appropriateness, and the non-leakage of complete answer rewrites. A deterministic Refinement Controller evaluates the resulting score vector using a strict conjunctive decision rule: every individual criterion must meet or exceed a minimum threshold score of three. If the minimum component falls below three, the controller rejects the draft, selects an explicit repair instruction tailored to the lowest-scoring dimension, and triggers another generation iteration. To eliminate unbounded execution and unpredictable operational latency, the controller enforces a strict termination ceiling of at most two rejection rounds. If a candidate fails acceptance after two revisions, a deterministic fallback chooses the candidate possessing the highest minimum score, breaking ties by aggregate mean. This configuration guarantees an absolute algorithmic upper bound of twenty-four language-model calls per essay.

Accepted commentary from each dimension is transferred to Module M4 for consolidation into the final feedback artefact delivered to the learner. The aggregation module first executes an automated consistency check, utilizing sentence embedding cosine similarity with a 0.85 threshold to identify redundant remarks spanning overlapping essay segments, pruning duplicates to retain only the most specific instructional comment. Next, a length-control module condenses the consolidated commentary into a target window of 300 to 400 words, reflecting the approximate volume of a single handwritten instructional review sheet. This pruning utilizes extractive summarization weighted by pedagogical layers, deliberately prioritizing feed-forward recommendations over descriptive feedback. Finally, Module M4 annotates each selected text segment with its target linguistic dimension, pedagogical layer designation, and underlying evaluation scores, emitting a validated JSON data structure alongside the natural-language prose to support instructional interfaces and institutional learning analytics.

Systematic empirical evaluations conducted across the public ASAP-AES and ELLIPSE benchmark datasets revealed that the multi-agent framework achieved the highest observed point estimates across automatic evaluation metrics compared to six baseline systems. In blinded expert assessments, four English-as-a-Foreign-Language instructors reviewed three hundred essay-system feedback artefacts generated across three distinct proficiency bands, evaluating performance using anchored rubrics without knowledge of system origins. Descriptive ratings indicated that the multi-agent framework achieved substantial margins over baseline systems, notably recording an observed mean score for non-leakage that exceeded comparative baselines by 0.74 to 1.08 points, alongside observed advantages of 0.34 in factual correctness and 0.35 in span-level specificity. Within the evaluated profiles, argumentation remained the lowest-covered linguistic dimension, achieving an aggregate coverage metric of 77.4 percent.

Textual analysis of generated comments demonstrated marked shifts in pedagogical composition compared to baseline architectures. Independent applied linguistics coders categorized feedback sentences across Hattie and Timperley’s three instructional dimensions, finding that RUMAR maintained a balanced distribution with a three-layer standard deviation of 3.9, in contrast to a baseline standard deviation of 25.1 which skewed heavily toward backward-looking critique. Feature-level frequency annotations further confirmed that span-level references were present in 91.3 percent of the framework’s feedback items, explicit principle explanations appeared in 73.6 percent, and concrete actionable revision steps were included in 82.4 percent. Most noticeably, the occurrence of complete sentence rewrites dropped from 34.8 percent in the baseline model to 6.4 percent in the multi-agent system, representing an eighty-two percent relative reduction in undesirable answer disclosure directly attributable to the non-leakage evaluation gate.

The operational framework was further evaluated in a four-week classroom trial involving eighty undergraduate students staged at intermediate language proficiency levels within an academic English course at Xingtai Medical College. Participants were assigned across three automated conditions to complete weekly argumentative writing tasks based on IELTS Academic Task 2 prompts, producing an initial draft, reviewing assigned automated commentary, and submitting a revision within forty-eight hours. Evaluation of pre-test and post-test essays using the ELLIPSE grading standard showed that students receiving the multi-agent feedback exhibited the largest observed descriptive score gains, as well as the highest reported metrics for revision uptake, perceived instructional usefulness, and system trust among the automated conditions examined during the investigation.

Despite these encouraging descriptive metrics, the findings remain bounded by significant methodological and evidential limitations. The classroom investigation did not incorporate a human-teacher control group or a no-feedback control condition, preventing researchers from attributing performance gains uniquely to automated intervention or demonstrating superiority over traditional classroom instruction. Furthermore, the trial utilized a one-week delayed post-test window, which is methodologically insufficient to establish durable long-term learning transfer. Additional limitations stem from reproducibility constraints, as the absence of row-level data manifests and execution logs prevented formal verification of previously reported statistical confidence intervals or inferential tests. Consequently, the observed results substantiate the engineering feasibility and structural alignment of the multi-agent retrieval architecture within the evaluated conditions, rather than confirming broad classroom readiness or definitive pedagogical superiority.

Subject of Research: Multi-agent retrieval-augmented generation framework for English-as-a-Foreign-Language writing feedback

Article Title: A multi-agent retrieval-augmented generation framework for pedagogical feedback on EFL student writing

Article References: Chen, N. (2026). A multi-agent retrieval-augmented generation framework for pedagogical feedback on EFL student writing. Discover Artificial Intelligence, 6(1), Article 1804. https://doi.org/10.1007/s44163-026-02384-9

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02384-9

Keywords: Artificial Intelligence, Natural Language Processing, Retrieval-Augmented Generation, Multi-Agent Systems, Educational Technology, Automated Writing Evaluation, Pedagogical Feedback, Language Models, multi-agent, retrieval-augmented, generation, framework