Physics Students Turned AI Into a Feedback Machine When Rules Loosened

When universities first confronted generative artificial intelligence, the reflex was prohibition: ban the tools, deploy the detectors, punish the offenders. That era is quietly ending, and a new study from Australia offers one of the clearest pictures yet of what happens when the bans come off. In an undergraduate physics course at Southern Cross University, researchers tracked how students actually used AI tools across three assessments as institutional policy shifted from tightly restricted, purpose-specific use to fully open use. The results, published in the International Journal of STEM Education, suggest that the assessment itself, not the policy label, determines what students do with the technology.

The study, led by Zachery Quince of Southern Cross University’s Centre for Learning and Teaching together with Emily Faulconer of Monash University, examined a second-year physics unit serving engineering and education students. Before the term began, the university’s policy had been overhauled, collapsing a five-tier model of permitted AI use into three categories: no use, purpose-specific use, and open use. Because none of the unit’s existing assessments complied with the new rules, the course coordinator was forced to redesign all three tasks, creating a natural experiment in how policy-driven assessment change reshapes student behaviour.

The redesign was revealing in itself. The first assessment, an oral presentation worth 20 percent of the unit, operated under purpose-specific rules: students could use AI for background research, idea generation, slide drafting and script development, but the final recorded presentation had to be their authentic selves, not an AI-generated avatar. The second assessment, a laboratory workbook worth half the grade, and the third, a take-home quiz worth 30 percent, both ran under open-use rules that permitted unrestricted AI engagement, provided students submitted a mandatory declaration describing exactly which tools they used and why.

Those declarations became the study’s raw data. Across the three assessments, students reported more than 120 unique use cases, which the researchers coded into four functional categories: learning, formatting, doing the task, and feedback. The pattern that emerged was striking. In the first assessment, use was dominated by learning and formatting, together accounting for over 95 percent of reported uses, as students brainstormed ideas, checked grammar and structured their presentations. By the final take-home quiz, feedback had exploded to 66.7 percent of all reported uses, with students describing how they checked calculations, verified answers and clarified concepts such as the viability of formulas.

To interpret this shift, the researchers mapped each category onto the SAMR framework, a well-known model that classifies technology integration into four levels: substitution, augmentation, modification and redefinition. Learning and formatting uses sat mostly at the lower levels, where AI merely replaces or improves existing tools without changing the task. Doing the task, where AI generated step-by-step solutions to unfamiliar problems, reached modification and redefinition, the levels where technology genuinely transforms what students can do. Feedback proved the most interesting category of all: it spanned every SAMR level, from simple error-spotting at the bottom to an iterative, student-controlled dialogue at the top, in which students could question, test and immediately apply advice in real time, something no previous feedback mechanism offered at comparable speed or scale.

The performance data added a cautious footnote to the behavioural story. Students who declared AI use scored slightly higher than non-users on all three assessments, with the largest gap appearing in the scaffolded presentation, where users averaged 15.76 out of 20 against 14.80 for non-users, a moderate effect size. But none of the differences reached statistical significance, and the researchers are explicit that this trend should be read as a tentative observation rather than evidence that AI boosts grades. What the data does show is that declared AI use was not associated with poorer achievement in a course where assessments required students to explain reasoning and take final responsibility for their work.

That caveat matters because the wider literature is deeply divided. Some studies report that AI feedback improves performance; others find that heavy ChatGPT use correlates with worse exam results, procrastination and memory loss. Research on high school mathematics has shown that generative AI without guardrails can harm learning, while other work finds stronger outcomes when students use AI to construct knowledge rather than merely execute procedures. The Australian authors argue that much of this contradiction stems from studies conflating fundamentally different behaviours, such as substitution-level use that bypasses cognitive effort with modification-level use that supports it, and from unclear policy contexts that make results impossible to compare.

The study’s central insight is what the researchers call task contingency. Students did not use AI simply because it was available; they used it for the kinds of support each task made useful. A presentation task made planning, language and communication support relevant. A workbook and quiz made checking, clarification and verification relevant. This means a policy label such as purpose-specific or open use can define what is permitted but cannot determine what students will actually do. The assessment design creates the demand. Where a task rewards polished communication, students reach for AI to polish. Where it demands problem-solving, they reach for AI to test their reasoning.

The findings also complicate the academic integrity narrative that dominated early coverage of AI in education. Far from being passive cheaters, students in this cohort reported using AI for preparation, clarification, refinement and verification, not only for answer production. The authors acknowledge the limits of self-reported data, which may be incomplete or strategically framed, and note that the single-marker design and small cohort of 36 students restrict generalisability. Still, they argue that integrity responses built purely on restriction and suspicion are too blunt, and that the real design question is whether an assessment requires students to demonstrate judgement and reasoning that AI cannot simply declare on their behalf.

The practical recommendations are concrete. Where students use AI for feedback, the authors suggest requiring them to highlight the feedback received and explain how it did or did not shape their revisions, making evaluative judgement visible and building feedback literacy. Greater restriction remains appropriate where AI could perform the intended cognitive work without students demonstrating their own understanding. The broader lesson is that generative AI has not invalidated authentic assessment; it has made it urgent. When tools can instantly solve standard textbook problems, educators are pushed toward tasks demanding synthesis, justification and communication, the very skills that define professional practice in physics and engineering. In that sense, the technology that threatened to hollow out assessment may end up forcing higher education to design better ones.

Subject of Research: How changing institutional generative AI policies influence student AI use and academic performance in undergraduate physics assessment

Article Title: Student GenAI use under changing institutional policies: a mixed-methods case study in undergraduate physics

Article References: Quince, Z., & Faulconer, E. (2026). Student GenAI use under changing institutional policies: a mixed-methods case study in undergraduate physics. International Journal of STEM Education, 13(1), Article 61. https://doi.org/10.1186/s40594-026-00651-w

Image Credits: AI Generated

DOI: 10.1186/s40594-026-00651-w

Keywords: generative AI, higher education, physics education, assessment design, SAMR framework, academic integrity, institutional policy, feedback, evaluative judgement, STEM education, authentic assessment, constructive alignment