Ghanaian study links supervised AI chatbots to stronger science learning

A four-week pilot intervention using the GPT-4o chatbot was associated with large improvements in science knowledge, conceptual understanding and several classroom-based indicators of holistic learning among 31 high-achieving Ghanaian students preparing for Senior High School. The study, conducted in a pre-SHS holiday class in Kumasi, assessed each participant before and after the program and found that the largest measured changes occurred in academic achievement and conceptual understanding. Because the research used a single-group pre-experiment without a comparison group, the results provide preliminary evidence of feasibility and potential educational value rather than proof that the chatbot alone caused the gains.

The intervention was designed for a setting in which personalised instructional support is limited and many Ghanaian Senior High Schools prohibit personal mobile phones. Students attended an intensive science programme from August to October 2024 and used ChatGPT’s May 2024 GPT-4o configuration on their personal devices, with researcher guidance and no plugins or customised external tools. All conversations were conducted in English to match the national science curriculum. The participating students, aged 13 to 16, had recently completed the Basic Education Certificate Examination in 2024 and came from more than ten Junior High Schools. They were selected through purposive convenience sampling from a holiday bridge programme, making the cohort accessible but limiting how widely the findings can be generalised.

Each lesson used a 65-minute chatbot cycle followed by 25 minutes of teacher-led consolidation. The first 10 minutes were devoted to learning objectives and inquiry planning, the next 30 to structured interactions with GPT-4o, and the final 25 to manual synthesis and refinement. Students received curriculum-aligned prompts for concept clarification, quizzes and problem-solving challenges involving metals, metalloids, non-metals and corrosion. They could adapt these prompts to their own conceptual difficulties, including asking why silicon is classified as a metalloid or requesting feedback on a draft explanation of lustre and sonority. The protocol also asked the chatbot to act as a Socratic tutor rather than provide direct answers.

Several safeguards were built into the program to move learners beyond passive information retrieval. Students completed training in prompt construction, were told not to copy and paste chatbot responses, and recorded their reflections in handwritten notebooks. A verification process compared initial drafts, final submissions and live note-taking to assess whether students used the chatbot for conceptual gap-filling. The lead instructor supervised sessions and audited chat histories for administrative compliance, while the substantive dialogues remained private. Students also received an orientation on responsible and ethical AI use, and no personally identifiable information was entered into the platform. This combination of prompting discipline, teacher oversight and written synthesis is central to interpreting the results: the study evaluated an AI-supported pedagogy, not unsupervised use of a chatbot.

Academic achievement improved from a mean of 11.74 before the intervention to 16.06 afterward on a 20-item test, while conceptual-understanding scores rose from 10.80 to 15.77 across four constructed-response questions. Paired one-tailed t-tests found significant differences for both outcomes at p < 0.001. The effect sizes were very large: Cohen’s d was 2.17 for academic achievement and 2.05 for conceptual understanding. Thematic analysis of the written responses showed that students moved from incomplete descriptions and misconceptions toward more mechanistic explanations. For example, post-test responses more often described rusting as a process involving both oxygen and water rather than attributing it to water alone, while explanations of element classes became more precise. These effects are striking, but repeated testing, maturation and the intensity of the holiday programme could have contributed to them.

Classroom observations and a post-intervention survey also indicated changes in critical thinking, problem-solving and social-emotional learning. Using the Wilcoxon signed-rank test, researchers reported significant improvements in critical thinking (Z = 2.93, p = 0.003; r = 0.52), problem-solving (Z = 2.55, p = 0.0107; r = 0.45) and social-emotional learning (Z = 3.26, p = 0.001; r = 0.58). The strongest observed effects included the application of concepts to analysis (r = 0.61), display of positive emotions (r = 0.63) and empathy and support for others (r = 0.68). These measures were researcher-developed and exploratory: the survey used single-item indicators of perceived competence, utility, intrinsic motivation, autonomy and relatedness, while two observers coded behavioural indicators in the first and final lessons.

Students’ perceptions of the chatbot were closely associated with their measured performance. Perceived confidence in understanding had a strong positive rank correlation with post-test conceptual understanding (Spearman’s rho = 0.821, p < 0.0001), while perceived academic improvement had a moderate correlation with post-test achievement (rho = 0.445, p = 0.012). Least Absolute Deviations regression, selected for robustness to non-normal data and outliers, linked a composite measure of chatbot-use frequency and interaction quality with higher median post-test achievement (beta = 0.21, t = 4.52, p < 0.0001) and conceptual understanding (beta = 0.19, t = 6.11, p < 0.0001). These analyses identify associations within a 31-student cohort; they do not establish that perceptions or engagement mediated the learning gains.

The findings are consistent with three educational theories, although the proposed mechanisms were not directly tested. Sociocultural Theory frames the chatbot as a digital More Knowledgeable Other that can provide explanations, feedback and guided prompts while helping learners work within their zone of proximal development. Cognitive Load Theory suggests that immediate clarification may reduce the effort spent finding or interpreting information, but cognitive load itself was not measured. Self-Determination Theory offers another interpretation: self-paced dialogue and immediate, non-judgemental feedback may support autonomy and competence. The researchers also caution that the observed changes in critical thinking and affective behaviour are particularly uncertain because the behavioural checklist was applied only at two instructional points, and full observer blinding was not feasible in a classroom-based intervention.

The study has several additional limitations. It included a small, purposively selected group of high-achieving students from one private holiday programme, lacked a randomised or non-AI comparison condition, and did not separate the chatbot’s contribution from instructor guidance, inquiry-based teaching, repeated assessment or heightened novelty. The four-week format also leaves unknown whether any gains persisted, and its supervised holiday-class setting differs from ordinary school conditions. Future work should compare AI-supported inquiry with teacher-facilitated inquiry without AI, use larger and more representative samples, track outcomes over time and directly examine cognitive load and student engagement processes. The results therefore support investigating supervised AI as one component of equitable science support, but they do not justify treating the chatbot as a demonstrated replacement for teachers or as grounds for broad policy change.

Subject of Research: Feasibility of supervised AI chatbot support for pre-SHS science learning in Ghana

Article Title: A Ghanaian pre-SHS feasibility study of AI chatbots as more knowledgeable others for supporting science learning in a resource-constrained context

Article References: Apeadido, S., Owusu-Fordjour, C., & Mahama, A. (2026). A Ghanaian pre-SHS feasibility study of AI chatbots as more knowledgeable others for supporting science learning in a resource-constrained context. Discover Artificial Intelligence, 6(1), Article 1054. https://doi.org/10.1007/s44163-026-02113-2

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02113-2

Keywords: AI chatbots, science education, Ghana, GPT-4o, conceptual understanding, critical thinking, pre-SHS learning, educational technology, pilot study, digital learning, Ghanaian, pre-SHS