AI Feedback Helps Student Teachers Plan Better Lessons Than Human Feedback Alone

Student teachers in vocational business education who received feedback on their lesson plans from an analytical artificial intelligence platform produced plans that matched or, in several dimensions, outperformed those of peers who received only human feedback. That is the central result of a longitudinal intervention study published in Empirical Research in Vocational Education and Training, in which 103 master’s students planned a 90-minute lesson across a full semester while researchers tracked how their planning skills and technology-related competences developed.

The study, led by Julia Pargmann and Florian Berding with Karin Rebmann and Elisabeth Riebenbauer, set out to address a persistent bottleneck in teacher education. Providing detailed, individualized feedback on lesson plans is difficult for university instructors, who face time restrictions and large student groups. The researchers built a platform called EDDA, short for electronic didactic assistance for lesson planning, which uses a German BERT transformer model trained on 1,304 lesson plans and materials from three universities in Germany and Austria, plus 4,679 coded textbook tasks. Unlike generative AI systems that produce text from scratch, EDDA classifies an uploaded plan into ten main categories and 57 subcategories and returns pre-produced, expert-level feedback and recommendations in a graphical interface.

The researchers argue this analytical approach sidesteps several well-documented problems of generative AI in the classroom. Generative tools can lack reproducibility, rely on untransparent training data, reproduce bias, raise data protection concerns and consume substantial energy. Analytical systems, by contrast, sort work into an existing framework, making results easily reproducible and comparable in formative learning scenarios. Because students must incorporate the feedback into their plans themselves, the authors suggest this may also benefit their cognition.

The study followed a two-group design integrated into a mandatory master’s module. The experimental group of 55 students consulted the AI platform first and then requested human feedback to discuss the results. The control group of 48 students, recruited through an elective module at a different university, received human feedback only, mainly through written instructor comments for each main topic and on-demand oral feedback during working phases. Both groups received the same theoretical input from the same lecturer, completed the same preparatory assignments and worked under identical planning conditions, which included a 90-minute lesson, a focus on accounting or controlling, compulsory integration of sustainability and the use of a simulated company. Lesson plans were collected at the beginning and end of the semester and rated through content analysis using a coding manual with ten main and 57 subcategories covering action orientation, process orientation, problem orientation, motivation, digitalization, sustainability and basic planning dimensions.

The results were mixed but telling. The experimental group significantly outperformed the control group on several dimensions: social competences in sustainability, the inform, plan, decide and check phases of the complete action cycle, information streams, the motivation dimensions of competence and relatedness, lesson structure, learner activation and cross-linked learning. The AI group also scored higher on technology use and on the sustainability-related aspects of reflexivity and SDG 15, life on land. However, the control group performed better on the category covering conditions for technology use, and in all other categories there were no significant differences between the groups. Because the experimental group did not always receive higher ratings, the researchers rejected their first hypothesis that AI feedback would universally produce better plans.

A second consistent finding concerned starting points. Across nearly all dimensions, students who began the semester with high ratings showed less improvement than those with lower initial scores. The researchers accepted this hypothesis and interpret it as evidence that standardized feedback may offer limited room for development to already strong learners, pointing to the need for more differentiated and personalized education in higher education. They suggest AI platforms should allow a degree of personalization, for example by offering reflective questions on didactic decisions or letting students choose a feedback level that grows in complexity, similar to scaffolding.

The study also examined what the authors call meta-dimensions: motivation, self-reported AI literacy, AI attitudes, self-efficacy with digital technologies and digital competences, measured with established instruments including the Meta AI Literacy Scale, the DigKomp2.2.de framework and the SWIT scale. Here the picture was sobering. The researchers rejected their third hypothesis, as the meta dimensions did not all show positive growth and the experimental group did not significantly outperform the control group overall. Only two dimensions of the AI literacy construct, detecting AI and AI emotion regulation, showed a significant positive treatment effect. Both groups actually showed a significant decline in AI emotion regulation over the semester, but the intervention appeared to attenuate or even reverse that decline in the experimental group, possibly because students were continuously confronted with the technology and could not remove themselves from it.

The authors conclude that the mere use of an AI platform does not automatically enhance AI literacy, change attitudes or improve motivation. Students who used the platform judged their ability to detect AI better than those who did not, but there were no statistical differences in self-efficacy with digital technologies, competences in data processing and evaluation, or the ability to judge a product’s safety. Specific instruction remains necessary. The researchers also note that issues with EDDA’s recommendations could have limited motivating effects, since the platform tends to underestimate planning performance due to unbalanced training data, which could mask competence development and demotivate users.

Several limitations warrant caution. Students only planned lessons in theory and never realized them in practice, so critical reflection on planning decisions could not take place. The lesson plans were coded rather than qualitatively analyzed, meaning nuances and reasoning behind content-related aspects were not considered. The sample size of 103, while typical for intervention studies in educational science, carries a risk of inflated intervention effectiveness, which the authors tried to manage with Bayesian estimation cross-checked against maximum likelihood models. The two groups came from different institutions with different course formats, which may have influenced how students acquired and used feedback. The rapid rise of AI systems such as ChatGPT during the data collection period from October 2023 to February 2025 also poses a methodological limitation, as general AI-related skills developing in parallel were not controlled.

Despite these caveats, the study offers a concrete signal that analytical AI can supplement rather than replace human feedback in teacher education. The authors recommend that instructors using such tools provide more opportunities for improvement, for example through reflection questions or personalized profiles allowing more detailed or more critical feedback. Institutions considering platforms like EDDA should integrate them systematically across all stages of teacher education programs and consider using them within portfolios or reflective documentation for ongoing assessment. Future research, they suggest, should investigate to what degree human feedback can be supplemented or even substituted by AI feedback, and could extend the platform by coding lesson plans for goal formulation and the realization of learning goals, which was not part of this study.

Subject of Research: A longitudinal intervention study testing whether an analytical AI platform can support lesson planning in vocational business teacher education.

Article Title: How AI feedback supports lesson planning in vocational teacher education: a longitudinal intervention study using an analytical AI platform

Article References: Pargmann, J., Berding, F., Rebmann, K., & Riebenbauer, E. (2025). How AI feedback supports lesson planning in vocational teacher education: a longitudinal intervention study using an analytical AI platform. Empirical Research in Vocational Education and Training, 17(1), Article 26. https://doi.org/10.1186/s40461-025-00202-7

Image Credits: AI Generated

DOI: 10.1186/s40461-025-00202-7

Keywords: artificial intelligence, lesson planning, teacher education, vocational education, feedback, EDDA, AI literacy, business education, longitudinal study, sustainability, motivation, digital competences