AI Model Spots Programming Blockages Before Students Ask for Help

When a novice programmer becomes stuck, the warning signs may appear long before a hand rises in the classroom. Typing slows, deletions increase, pauses stretch, and failed compilations begin to repeat. Yet those signals can also describe productive reflection, making it difficult for an instructor to know when intervention will help rather than interrupt. A study published in Discover Informatics presents a hybrid computational framework designed to distinguish these moments and identify programming blockages before they become obvious. The model analyzes fine-grained activity traces from students’ programming environments, combining observable workflow patterns with inferred cognitive states and longer-term changes across a coding session. In tests involving 70 first-year computer science students, the system detected emerging blockage an average of 2.8 minutes before it became visible to an instructor. Its authors argue that the main advantage is not higher classification accuracy than simpler algorithms, but a combination of early warning, uncertainty estimates, and explanations that instructors can use to decide how to respond.

The challenge has become more complicated as artificial-intelligence coding assistants have entered programming education. A student may now submit correct code after receiving suggestions from ChatGPT, GitHub Copilot, or a similar tool, while the process that produced that code remains hidden. A flawless final program does not necessarily show whether the learner understood the algorithm, struggled for half an hour, or accepted a generated solution without grasping its logic. The researchers therefore focused on the process rather than only the product. Programming environments record a continuous stream of events, including edits, compilations, executions, pauses, browser navigation, documentation searches, and interactions with course platforms. These events can reveal patterns that are invisible in the final source code. But the signals are inherently ambiguous: a pause can reflect careful planning or confusion, and frequent edits can indicate either systematic debugging or increasingly random attempts. The proposed system addresses that ambiguity by examining several dimensions of behavior at once.

The first layer is a Markov Chain, a probabilistic model that estimates how likely one observable action is to follow another. It can recognize workflow structures such as fluent editing followed by a validation compile, as well as less productive loops involving hesitant editing, repeated compilation, and long pauses. In mathematical terms, the model assigns probabilities to transitions between behavioral states, using smoothing so that rare or unseen transitions do not produce extreme conclusions. The second layer is a Hidden Markov Model, or HMM. Rather than treating cognitive condition as directly measurable, the HMM infers latent states from the observed sequence. The operational categories used in evaluation were Progressing, Hesitating, Blocked, and Confused. These labels are not diagnoses of a student’s mind; they are probabilistic summaries of behavior that can guide instructional decisions. A student classified as Hesitating might benefit from a targeted hint, while one classified as Confused may need a question that clarifies the strategy being attempted. A student identified as Blocked may require direct help with a persistent error.

The third layer is a recurrent neural network with attention. The study describes a bidirectional gated recurrent unit architecture that processes activity in both temporal directions and represents each time window using features such as typing speed, deletion ratio, pause duration, navigation density, compilation frequency, repeated errors, and code progress. Attention assigns greater weight to moments that are especially informative for the current prediction. This allows the system to connect a present difficulty with events that occurred several minutes earlier, overcoming the short memory of a basic Markov model. The final prediction combines the outputs of all three components using confidence-adaptive weights. If the transition probabilities are uncertain, the Markov contribution is reduced. If the inferred HMM state changes erratically, its influence falls. If attention is diffuse rather than concentrated on particular moments, the neural component contributes less. The result is intended to be not just a blockage score, but a record of which behavioral transitions, latent state patterns, and time points shaped the alert.

To evaluate the framework, the researchers analyzed 287,236 timestamped actions gathered from 70 first-year students enrolled in an introductory C++ course. The students had no prior programming experience and completed six exercises of increasing complexity in a standardized software environment. The analysis concentrated on 220 annotated sequences from two representative exercises. Events were converted into overlapping 30-second windows advancing in five-second steps, allowing the models to track changes during a session rather than relying only on totals such as the number of compilations. Two experienced programming instructors independently labeled a subset of the windows, reaching a Cohen’s kappa of 0.81, a measure of strong agreement. The dataset was divided using student-level five-fold cross-validation, so all sequences from a student remained in either the training or testing portion. This design reduces the risk that a model simply learns an individual student’s habits and then appears to generalize.

The results contain a notable twist. The hybrid model achieved a Macro-F1 score of approximately 90.7 percent across the four cognitive-state categories, but so did the simpler comparison models, including a Random Forest, a Markov Chain alone, an HMM alone, and a recurrent neural network with attention. A Friedman test found no statistically significant differences among the eight evaluated configurations, with a reported p-value of 0.83. The authors interpret this equivalence as evidence that the behavioral taxonomy itself is highly discriminating: once the observable categories are defined precisely, several machine-learning approaches can learn to recognize them. The hybrid architecture should therefore not be presented as a more accurate classifier. Its distinctive contribution lies elsewhere. The HMM supplies pedagogically meaningful state labels, the Markov layer exposes workflow transitions, and attention highlights relevant moments in the sequence. Together, these outputs can provide more context than a single risk label, even when the final classification accuracy is nearly identical.

Signals associated with impending blockage included progressive typing deceleration, a rising proportion of deleted characters, and lengthening pauses. In the study’s corpus, these patterns often appeared three to five minutes before a blockage was fully visible. A transition from neutral activity cycles to destructive cycles was another strong warning sign: when hesitation increased across consecutive observation windows and repetitive error attempts continued, blockage followed in 78 percent of the sequences examined. The model’s attention mechanism could emphasize earlier failed compilations or pauses, while the HMM summarized the broader trajectory from Progressing to Hesitating to Blocked. In a pilot deployment involving 12 instructors and 180 students across three institutions, 82 percent of alerts were judged accurate and actionable by instructors. The report also describes 18 percent more completed exercises, a 12 percent reduction in completion time, and final programming examination scores 6.3 percentage points higher than in control classrooms. These pilot outcomes are promising, but they should be interpreted alongside the study’s limitations and the authors’ description of the system as real-time-capable rather than fully validated in live classroom operation.

The research team emphasizes that behavioral tracking cannot reveal cognition with certainty. A student may pause because they are thinking deeply, because they are distracted, or because they have lost their strategy. The rare Confused category, representing 5.9 percent of windows, had the lowest F1 score at 79.0 percent and was frequently confused with Hesitating. Short sessions also produced more missed blockages because there was not enough time for precursor signals to accumulate. The dataset came from one institution, one introductory C++ course, and a relatively small group of students, so the thresholds may not transfer directly to other languages, teaching styles, or learners. The study also warns that attention weights show where the model focused, not necessarily what caused its decision. Any educational deployment would need strong privacy protections, informed consent, and safeguards preventing formative monitoring from becoming a grading mechanism. The authors propose testing the framework across institutions and programming languages, incorporating additional signals such as self-reports, and developing an instructor dashboard. For now, the work suggests that the most useful educational AI may not be the system that claims to know exactly why a student is struggling, but one that notices a changing pattern early, explains the evidence cautiously, and leaves the final judgment to a human teacher.

An important methodological distinction is between recognizing a labeled behavioral category and establishing that a learner is cognitively blocked. The study’s four-class taxonomy—progression, hesitation, blockage, and confusion—provides an operational language for analyzing traces, but its categories remain model-based interpretations of observable activity. This matters because the reported similarity in Macro-F1 across the tested approaches suggests that performance depends substantially on how the behavioral states are defined and represented, not only on architectural complexity. The absence of significant differences among models also cautions against treating a more elaborate system as automatically more accurate.

The hybrid design is therefore most valuable as a decision-support framework. Markov transition scores can describe local workflow changes, while the HMM offers a probabilistic account of how activity may correspond to a changing latent state. The recurrent component adds a way to connect events separated in time, and confidence-adaptive fusion can reduce the influence of a component when its evidence is unreliable. These signals could help an instructor distinguish a single unusual pause from a sustained deterioration across successive activity windows. Such distinctions are particularly relevant in programming, where debugging often involves temporary failure and repeated experimentation that should not be mistaken for learning collapse.

The reported pilot findings provide an initial indication that interpretable alerts can be linked to instructional outcomes, but they do not by themselves establish effectiveness across settings. The evaluation involved a limited number of students and instructors, and the source describes the deployment as a pilot. Future testing would need to examine whether alerts remain calibrated when students use different programming languages, development environments, or assistance tools, and whether interventions prompted by the system produce benefits beyond those attributable to increased instructor attention. It will also be important to assess how students perceive monitoring and whether uncertainty information is presented clearly enough to prevent probabilistic alerts from being treated as definitive judgments.

Subject of Research: Machine-learning detection of novice programming difficulties from fine-grained activity traces

Article Title: A multi-dimensional hybrid stochastic model for early and interpretable blockage detection in programming education

Article References: Abdelkader, G., Mohammed, E., Patrick, E., & Thierry, N. (2026). A multi-dimensional hybrid stochastic model for early and interpretable blockage detection in programming education. Discover Informatics, 1(1), Article 9. https://doi.org/10.1007/s44564-026-00007-0

Image Credits: AI Generated

DOI: 10.1007/s44564-026-00007-0

Keywords: programming education, learning analytics, educational data mining, student blockage detection, Hidden Markov models, Markov Chains, recurrent neural networks, explainable AI, multi-dimensional, hybrid, stochastic, model