Graduate Engineering Students Adapt Verification and Reasoning When Using ChatGPT

Graduate engineering students interacting with ChatGPT during discipline-specific problem solving adapt their cognitive engagement in nuanced ways rather than passively accepting artificial intelligence outputs, according to a mixed-methods investigation published in Higher Education. Conducted among twelve graduate students tackling three distinct mining engineering tasks, the study examined how emerging generative artificial intelligence tools influence learners’ cognitive engagement, evaluative judgment, and reliance. While generative artificial intelligence has become increasingly integrated into higher education settings, concerns have emerged regarding whether students develop an uncritical dependence on machine-generated solutions. By tracking students through multiple data collection modalities, the authors observed that learner interactions extended well beyond simple acceptance, exhibiting substantial qualitative variety in how engineering students plan, question, and critically evaluate automated responses during technical problem-solving exercises.

To capture the depth of student interactions, the researchers employed a comprehensive mixed-methods research design that focused specifically on graduate-level STEM education, an area where empirical evidence regarding artificial intelligence engagement has remained limited. A cohort of twelve graduate students in mining engineering participated in the study, where they were observed completing three discipline-specific engineering problem-solving tasks using ChatGPT. The research team gathered evidence from multiple complementary data streams, capturing participants’ real-time behaviors through screen recordings alongside written reflections completed during the exercises. In addition, the investigators administered structured surveys to measure student perceptions and conducted detailed follow-up interviews to clarify how participants approached their work. This rich collection of qualitative and quantitative evidence enabled the researchers to trace behavioral patterns and cognitive engagement across each distinct phase of the engineering assignments.

A qualitative analysis of the recorded sessions, reflections, and interview transcripts revealed five distinct interaction types that characterized how graduate students utilized the generative artificial intelligence tool. These identified categories comprised reliance with minimal verification, reliance with critical evaluation, collaborative engagement, skeptical or limited use, and learning-driven use. The investigators determined that these five behavioral patterns differed markedly across fundamental dimensions of student learning, specifically regarding learner agency, metacognitive engagement, task planning, verification habits, and evaluative judgment. In addition to qualitative distinctions in how students assessed machine responses, the observed interaction types were differentiated by concrete behavioral metrics recorded during the sessions, including the total duration of interaction time spent with the interface and the specific number of follow-up questions students posed while attempting their discipline-specific mining engineering tasks.

Rather than remaining static throughout the investigation, students’ interaction types shifted dynamically across the three consecutive mining engineering tasks. The researchers observed that later tasks in the sequence, which featured more open-ended problem structures, actively prompted participants to adopt more sophisticated cognitive approaches. Specifically, as the engineering problems grew less structured, students demonstrated higher levels of active verification, engaged in deeper personal reflection, and exercised greater independent reasoning instead of relying on immediate outputs generated by ChatGPT. This dynamic transition across problem contexts showed that the nature and framing of the discipline-specific prompt played a substantial role in guiding student agency. By adjusting their problem-solving tactics across tasks, the graduate students demonstrated an adaptive capacity to challenge, check, and refine machine output when confronted with complex engineering scenarios.

Alongside the qualitative behavioral findings, the investigation included quantitative comparisons of students’ survey responses collected before and after interacting with ChatGPT. Descriptively, the post-survey data showed higher average scores across several perceptual dimensions, with participants reporting increases in perceived credibility, believability, overall satisfaction, and perceived productivity following their use of the conversational tool. However, the researchers noted that none of these descriptive mean increases achieved formal statistical significance. When evaluating specific semantic differentials, the comparisons for Fair versus Unfair and Accurate versus Inaccurate demonstrated larger effect-size estimates, yielding Cohen’s d values of 0.72 and 0.70, respectively. Because the investigation evaluated a targeted cohort of twelve participants, the authors explicitly emphasized that these exploratory effect-size estimates require cautious interpretation and cannot be treated as definitive confirmatory conclusions.

Taken together, the empirical findings demonstrate that graduate engineering students do not engage with generative artificial intelligence as passive consumers, but instead demonstrate varied and adaptable behaviors that challenge common assumptions of uncritical dependence. Although students recognized practical utility in ChatGPT, their cognitive engagement varied considerably depending on task design, personal evaluative judgment, and the degree of verification applied to machine-generated outputs. In synthesizing these results, the authors highlighted the critical importance of developing instructional frameworks and deliberate prompting strategies within engineering education. By designing curriculum tasks that intentionally promote reflection, verification-oriented analysis, and responsible AI-assisted learning practices, educators can help graduate students navigate generative artificial intelligence technologies effectively while safeguarding rigorous evaluative standards and independent problem-solving skills across demanding technical disciplines.

Subject of Research: Graduate engineering students' cognitive engagement and perceptions of ChatGPT

Article Title: Cognitive engagement and students' perceptions of ChatGPT during discipline-specific problem solving in mining engineering graduate education

Article References: Alhaj-Bedar, R., Cunningham, N., Agioutantis, Z., Schafrik, S., Moradi, A., & Roghanchi, P. (2026). Cognitive engagement and students' perceptions of ChatGPT during discipline-specific problem solving in mining engineering graduate education. Higher Education. https://doi.org/10.1007/s10734-026-01806-y

Image Credits: AI Generated

DOI: 10.1007/s10734-026-01806-y

Keywords: ChatGPT, generative artificial intelligence, mining engineering education, graduate engineering students, cognitive engagement, evaluative judgment, metacognitive engagement, verification strategies, discipline-specific problem solving, AI-assisted learning, screen recordings, written reflections, mixed-methods study, interaction types, collaborative engagement, learning-driven use, prompting strategies, student perceptions