New AI Prompt Teaches Machines to Grade Student Reports Like Critical Thinkers

Grading student coursework has always been one of the most labor-intensive responsibilities in higher education, and the arrival of large language models has promised a future in which machines could shoulder part of that burden. Yet a persistent problem has haunted efforts to automate the assessment of course project reports: artificial intelligence systems tend to judge writing on its surface qualities—grammar, structure, and fluency—while missing the deeper qualities that educators actually care about, such as logical reasoning, originality, and critical thinking. A new study published in Frontiers of Digital Education by researchers at Southern University of Science and Technology, Northwest Normal University, the University of Nottingham Ningbo China, and Wenzhou Medical University presents a promising solution. The team, led by Qingyang Sun and including corresponding authors Xiaoqing Zhang and Jiang Liu, has developed a carefully engineered prompting framework called PEG-Prompt that teaches general-purpose language models to evaluate student reports the way an experienced human grader would.

The course project report occupies a unique place in university assessment. Unlike a conventional essay, it documents a student’s journey through practical problem-solving, demanding evidence of technical competence, academic writing skill, and logical thought. When researchers first began applying large language models to automated essay scoring, the dominant paradigm treated these reports as ordinary pieces of prose. The result was scoring systems that could reward polished sentences while remaining blind to whether the underlying argument held together, whether the student demonstrated genuine command of the subject matter, or whether the citations were appropriate and meaningful. According to the study’s authors, existing LLM-based automated essay scoring methods are built almost exclusively around writing proficiency, which inevitably overlooks cognitive engagement and practical competencies that are central to project-based coursework.

The conceptual breakthrough behind PEG-Prompt lies in an unexpected place: a decades-old framework from philosophy and education theory. The researchers integrated the Paul-Elder critical thinking framework—a widely taught model that organizes critical thought around elements of reasoning and intellectual standards—directly into the design of the prompt given to the language model. Rather than asking an AI to simply rate a report, the framework instructs it to reason through the material using explicit critical thinking criteria. This integration serves a dual purpose, the authors explain: it enhances domain-specific knowledge transfer and strengthens the analytical capabilities of generative AI models when they confront discipline-specific student work. In effect, the prompt acts as a compact training manual, encoding the evaluative wisdom of human educators into text that the model can follow.

Technically, PEG-Prompt evaluates course project reports along six carefully chosen dimensions: structure, logic, coherence, originality, citation, and knowledge proficiency. Each dimension targets a different facet of student competence. Structure and coherence capture the organization and flow of the document, while logic probes whether arguments actually follow from one another. Originality measures the student’s independent contribution rather than the mere recycling of ideas. Citation assessment examines how sources are used and acknowledged, and knowledge proficiency gauges whether the student genuinely understands the technical content of the course. Together, the six dimensions allow the framework to assess practical competencies, analytical reasoning, and writing skills simultaneously, rather than collapsing everything into a single measure of linguistic polish. This multidimensional design reflects the authors’ conviction that course project writing is fundamentally a reflective process involving knowledge inquiry and cognition through critical thinking.

The framework does not rely on the prompt alone. To push performance further, the researchers combined PEG-Prompt with two complementary techniques drawn from established practice in natural language processing. First, they extracted key content from the reports themselves, giving the model a distilled representation of the most important material rather than forcing it to process the full document without guidance. Second, they incorporated few-shot scoring examples—representative instances of reports paired with the scores human evaluators assigned them. This approach, known as few-shot learning, allows a language model to calibrate its judgments against concrete precedents without any retraining of the model’s internal parameters. The combination transforms the prompt from a static instruction set into a guided evaluation protocol anchored in real grading behavior.

The experimental results reported in the study demonstrate that this multifaceted approach meaningfully improves the correlation between scores generated by large language models and scores assigned by human evaluators. Correlation with human judgment is the gold standard in automated essay scoring research, because a machine grader is only useful if it agrees with trained educators about what constitutes good work. By embedding critical thinking criteria, key content extraction, and few-shot examples into a single coherent pipeline, the researchers showed that a general-purpose LLM can move much closer to human-level assessment of complex, discipline-specific student reports—something previous prompt designs built purely around writing quality failed to achieve.

The broader context of this work is the rapidly growing field of education intelligence, where researchers harness artificial intelligence to personalize and improve learning at scale. Automated essay scoring itself has a long history stretching back decades, but the emergence of large language models has dramatically raised expectations for what machine graders can do. Earlier deep learning approaches required task-specific models trained on large labeled datasets for each new subject area. LLM-based approaches promise strong generalization and reasoning abilities across domains, but as this study makes clear, raw capability is not enough. Without carefully designed prompts that encode pedagogical values, these models default to shallow judgments. The PEG-Prompt research joins a growing body of work on prompt engineering—the craft of designing text instructions that reliably elicit desired behaviors from generative AI systems.

The practical implications for universities could be substantial. Once calibrated with human evaluators, the enhanced framework could allow students to receive detailed feedback and summaries of their course project results through generative AI systems, delivered quickly enough to inform revisions rather than arriving as a final verdict after the fact. In large courses where a single instructor may face hundreds of lengthy project reports, such a capability could free educators to focus their limited grading time on the students who need the most help, while giving every student at least a preliminary, structured critique. The six-dimension breakdown also offers richer feedback than a single letter grade, pointing students toward specific weaknesses—whether in argumentation, sourcing, or subject mastery—that they can address before resubmission.

At the same time, the researchers are careful to frame PEG-Prompt as an aid rather than a replacement for human judgment. The framework’s performance depends on calibration against human scores, and the study’s vision positions generative AI as a partner in assessment, one whose outputs become trustworthy only after alignment with expert graders. The work was supported by the Guangdong Provincial Teaching Quality and Teaching Reform Project, the Southern University of Science and Technology Teaching Reform Project, and the Medical and Health Science Program of Zhejiang Province. As institutions worldwide grapple with how generative AI should fit into education—both as a tool students use and as a tool that evaluates them—this study offers a concrete, technically grounded example of how the same technology that complicates academic integrity can also be harnessed, through thoughtful prompt design, to deepen rather than dilute the assessment of student learning.

Subject of Research: Automated evaluation of course project reports using large language models guided by the Paul-Elder critical thinking framework.

Article Title: Evaluating the Efficacy of a Multifaceted Prompt for Use with LLMs to Evaluate Course Project Reports

Article References: Sun, Q., Zhang, J., Sheng, P., Wang, Q., Wang, T., Li, H., Zhan, H., Zhang, X., & Liu, J. (2026). Evaluating the Efficacy of a Multifaceted Prompt for Use with LLMs to Evaluate Course Project Reports. Frontiers of Digital Education, 3(2), Article 12. https://doi.org/10.1007/s44366-026-0086-y

Image Credits: AI Generated

DOI: 10.1007/s44366-026-0086-y

Keywords: large language models, automated essay scoring, critical thinking, PEG-Prompt, prompt engineering, education intelligence, generative AI, course project reports, few-shot learning, assessment, higher education, Paul-Elder framework