The study, led by Bingjie Huang of Central China Normal University together with colleagues at Southeast University and Chengdu No. 7 High School, recruited 116 sophomore STEM students enrolled in a calculus-based introductory physics course and randomly assigned them to two groups. Ninety-five students completed all procedures: 50 in the experimental group and 45 in the control group. The experimental group received a structured training session built on Zimmerman’s self-regulated learning framework and cognitive load theory, while the control group received an equally long session on advanced problem-solving strategies with no AI guidance at all. Crucially, both groups then faced the same test: solving items from the Conceptual Survey of Electricity and Magnetism, a standardized 32-question assessment, and then revising their answers after conversing with the Chinese large language model Doubao.

The training itself was organized into three sequential phases. A ten-minute situational introduction contextualized generative AI, contrasting it with the homework-assisted apps that dominate Chinese students’ digital lives. A fifteen-minute strategy-teaching segment demonstrated how to prompt the AI in ways aligned with the three phases of self-regulated learning: forethought, performance, and self-reflection. It also introduced the model’s characteristic weaknesses, including reasoning errors, visual misinterpretation of graphs and figures, and outright hallucinations, along with practical remediation tactics. Finally, a twenty-minute hands-on session let students practice analyzing two mechanics problems with the AI and reflect on the obstacles they encountered. The design deliberately separated the training domain, mechanics, from the assessment domain, electromagnetism, so that any measured benefit would reflect transferable strategy rather than memorized solutions.

The quantitative results were striking. Students who received the AI training achieved significantly higher revision performance than the control group, with an effect size of 0.81, a large effect by conventional standards in educational research. The most dramatic behavioral difference appeared in the quality of revisions. Among trained students, 69.3 percent of revisions converted an incorrect answer into a correct one. In the control group, that figure was only 41.3 percent. Worse, 40 percent of the untrained students’ revisions actually changed correct answers into incorrect ones, a pattern the authors attribute to blind reliance: untrained students treated AI outputs as authoritative and capitulated reactively, even when the model’s reasoning was internally inconsistent.

Chat transcripts revealed exactly how the two groups diverged. The researchers coded six interaction strategies, ranging from the low-level image-only approach, in which students simply upload a photo of the problem, to higher-order behaviors such as asking for definitions, seeking insight into the structure of a problem, and generating new questions for self-assessment. Both groups leaned heavily on image-only prompting, a habit the authors link to the snap-and-solve culture fostered by apps like Zuoyebang and Xiaoyuan, which let students photograph homework and receive instant answers. But the trained group diversified markedly. The proportion of students using the obtaining-insight strategy more than doubled, from 20 percent to 42 percent, and 14 percent of trained students progressed to generating new questions, a behavior entirely absent among controls. Trained students also exchanged significantly more conversational turns with the AI, indicating deeper engagement rather than quick answer extraction.

The qualitative cases are perhaps the most vivid evidence of what the training accomplished. In one exchange, the AI misjudged an electromagnetic induction question, declaring that a coil in one figure formed an unclosed circuit. A trained student pushed back, noting that the coil was in fact closed, which prompted the model to recognize that a contracting closed coil changes its area and therefore its magnetic flux, leading it to the correct answer. Another student caught the AI misreading a graph in a mutual inductance problem and walked it to the right option by inspecting the answer patterns. By contrast, one control student, after the AI correctly graded an item, instructed it to simply show correct answers without analysis going forward, and then failed to catch a visual error the model made on the very next question.

Cognitive load measurements added a theoretically intriguing wrinkle. Overall cognitive load did not differ between the groups, but the trained students reported a marginally higher intrinsic load, the component tied to the inherent difficulty of the task itself. Rather than interpreting this as a cost, the authors propose that it may reflect productive struggle: the heightened working-memory demands that come from systematically decomposing problems and interrogating the AI strategically. They suggest refining cognitive load theory to distinguish task-imposed intrinsic load from productively engaged intrinsic load, arguing that when extra mental effort signals deep metacognitive processing rather than confusion, it may accompany, rather than undermine, better learning outcomes.

The study also probed how three learner characteristics, self-regulated learning skills, prior generative AI experience, and domain knowledge, shaped outcomes. Using robust linear models suited to the skewed distribution of pretest scores, the team found that self-regulated learning skills and prior knowledge both significantly predicted revision performance, while prior AI experience showed only a marginal effect. Notably, the training moderated the role of AI experience: in the control group, prior chatbot experience strongly predicted revision success, but in the trained group that relationship flattened almost to zero. In other words, structured instruction appeared to level the playing field, giving students without extensive AI backgrounds the strategic tools that experienced users had developed informally, if at all.

The authors are careful about the limits of their findings. The assessment was a one-shot revision task conducted on a single day, so the design cannot establish whether the gains reflect durable learning, conceptual change, or simply a temporary surge of attention. Because pretest and posttest used identical items, memory effects cannot be ruled out, and the observed advantage may partly stem from the control group’s lack of any AI guidance rather than from specific training components. The sample, drawn from a single top-tier Chinese university, also limits generalizability. The researchers call for longitudinal studies with delayed posttests, component-level comparisons to isolate the active ingredients of training, and richer measures such as interviews, eye tracking, or electroencephalography to capture students’ cognitive experiences more fully.

Even with those caveats, the implications are hard to ignore. As generative AI becomes embedded in education worldwide, the debate has often been framed as a binary between banning the tools and handing them out. This trial points toward a third path: teaching students explicitly how to collaborate with AI, including how to recognize its failure modes and correct them. The authors propose complementarity recognition, an awareness of the asymmetric capabilities of humans and machines and of when to trust each, as a construct worth measuring and cultivating. A 45-minute session cannot remake a student’s intellectual habits, but it demonstrably shifted behavior in the direction educators want: fewer blind capitulations to the machine, more probing questions, and a strikingly higher rate of turning wrong answers into right ones. The answer machine, it turns out, becomes a genuine partner only when someone teaches students how to work with it.

Subject of Research: The effect of structured generative AI training on students' AI-assisted scientific problem solving in STEM education

Article Title: From answer machine to collaborative partner: impact of structured generative AI training on scientific problem solving in STEM education

Article References: Huang, B., Xie, L., Liu, Y., Liu, X., Tang, H., Feng, X., Qiao, C., Guo, Q., Wu, C., & Bao, L. (2026). From answer machine to collaborative partner: impact of structured generative AI training on scientific problem solving in STEM education. International Journal of STEM Education, 13(1), Article 62. https://doi.org/10.1186/s40594-026-00652-9

Image Credits: AI Generated

DOI: 10.1186/s40594-026-00652-9

Keywords: generative AI, STEM education, self-regulated learning, cognitive load theory, human-AI collaboration, problem solving, metacognitive laziness, physics education, randomized controlled trial, AI literacy, prompting strategies, electromagnetism