AI Learns to Spot Teaching Behaviors Teachers Never Labeled

Every classroom is a storm of behavior. A teacher gestures toward a whiteboard, paces between desks, pauses to let a question hang in the air, then launches into an explanation. For decades, education researchers have tried to catalog these behaviors into meaningful indicator categories that can drive fair teaching evaluation. The problem is that nobody can label everything. Annotated datasets capture only the behaviors experts already know to look for, while countless other patterns unfold on camera, uncategorized and invisible to automated systems. A new study published in Applied Intelligence by Ting Cai, Qingyuan Tang, Yu Xiong, Lu Zhang of Chongqing University of Posts and Telecommunications and Attila Pandur of the University of Pécs tackles exactly this blind spot, and its solution may reshape how machines understand what happens at the front of a classroom.

The researchers built their approach around a machine learning paradigm known as Generalized Category Discovery, or GCD. The core idea is deceptively simple: given a small set of labeled examples and a much larger pile of unlabeled data, a model should both classify the known categories and discover brand-new ones hiding in the unlabeled pool. In the educational context, that means the system learns from a handful of expert-annotated teacher behavior indicator categories, then mines vast amounts of unlabeled classroom video to automatically identify and organize additional indicator categories nobody has defined yet. This matters because teaching evaluation has long depended on frameworks built by hand, and hand-built frameworks inevitably miss behaviors that matter, particularly in the complex and dynamic environments of real classrooms.

Existing GCD methods, however, stumble in two predictable ways when confronted with classroom footage. First, they struggle to capture spatiotemporal dynamics, the intertwined patterns of space and time that define a teacher’s actions: where the teacher moves, how gestures evolve, when a demonstration becomes an explanation. Second, they fall victim to semantic interference. Classrooms are visually chaotic environments filled with students, furniture, posters, and lighting changes that drown out the subtle signals of instructional behavior. A model trained naively may cluster videos together because they share a similar room rather than a similar teaching behavior, which limits the effectiveness of novel category discovery precisely where it is needed most.

The team’s answer is a framework called MGCL, short for multi-granularity contrastive learning. Contrastive learning, the engine behind many recent breakthroughs in computer vision, works by teaching a model to pull similar examples closer together in its internal representation space while pushing dissimilar examples apart. What makes MGCL distinctive is that it operates this push-and-pull mechanism at three different scales simultaneously, each addressing a different failure mode of earlier approaches. The framework is guided by known indicator category data, using that supervision as an anchor while it explores the unlabeled frontier.

The first scale is instance-level contrastive learning, abbreviated IL-CL. At this level, the model optimizes the characteristic representation of individual samples, learning what makes each video clip of teacher behavior distinctive in itself. This fine-grained optimization ensures that the raw features feeding into later stages are informative and robust, rather than noisy reflections of irrelevant background detail. It is the foundation layer: if individual samples are poorly represented, no amount of higher-level reasoning can rescue the categories built on top of them.

The second scale, neighborhood consistency contrastive learning or NC-CL, zooms out slightly. Rather than treating every sample as an island, it emphasizes the semantic consistency of neighboring samples, enhancing the discriminability of samples within local contexts. The intuition is that clips of genuinely similar teaching behavior should sit near one another in feature space, and enforcing agreement among neighbors suppresses spurious variation caused by classroom clutter. This local smoothing acts as a denoising mechanism, helping the model distinguish behavior-driven similarity from environment-driven coincidence, which is exactly where semantic interference does its damage.

The third and broadest scale is global category consolidation contrastive learning, GC-CL. Here the model leverages pseudo-labels, provisional assignments the system generates for unlabeled data, to optimize category partitioning at a holistic level and improve inter-class separation. Pseudo-labeling is a well-established strategy in semi-supervised learning, but MGCL uses it with a consolidation twist: the global objective repeatedly refines the boundaries between emerging categories so that classes become compact and well separated. Operating at the instance, neighborhood, and global levels respectively, the three components work in concert to collectively enhance feature discriminability and category stability, each level correcting weaknesses the others cannot address alone.

To test the framework, the researchers ran experiments on three datasets: a proprietary collection called TBU, developed by members of the same team for teacher behavior understanding, and two widely used public benchmarks for human action recognition, UCF101 and HMDB51. The TBU dataset is particularly significant because it contains authentic multi-mask classroom videos, and due to privacy and ethical restrictions associated with recordings involving human participants, its raw videos cannot be made publicly available. Instead, the dataset description, category definitions, data statistics, annotation information, and a controlled-access procedure are hosted on GitHub, and researchers may request access from the corresponding author after a compliance review. The source code, experimental configurations, and implementation details of MGCL are, by contrast, fully public in a separate repository, allowing other teams to reproduce and extend the method.

The results were striking. Across multiple evaluation metrics, MGCL achieved state-of-the-art performance, outperforming existing GCD approaches on both the education-specific TBU data and the general-purpose action recognition benchmarks. The consistency of the gains across such different domains suggests that the multi-granularity design is not a trick tuned to one dataset but a genuinely general principle for discovering categories in video where labels are scarce and the visual environment is messy. For the educational community, the study offers what the authors describe as a feasible and effective solution for discovering categories of instructional behavior indicators in complex educational environments, potentially automating a task that has historically consumed enormous amounts of expert time.

The implications reach well beyond the classroom. Generalized category discovery sits at the intersection of semi-supervised learning, clustering, and open-world recognition, and MGCL’s three-tier contrastive architecture draws on a rich lineage of research, from self-supervised video representation methods to parametric and prototype-based GCD models published at major computer vision venues. By demonstrating that instance, neighborhood, and global objectives can be layered productively, the work offers a template for any domain where experts can label only a fraction of the phenomena they care about: medical observation, wildlife monitoring, workplace safety analysis, and beyond. The study was supported by the National Natural Science Foundation of China and several Chongqing regional research programs, and it appears in Applied Intelligence as volume 56, article 473. As cameras multiply in classrooms around the world, frameworks like MGCL hint at a future in which the full vocabulary of teaching behavior, not just the parts we already know how to name, becomes visible to science.

Subject of Research: Machine learning for discovering categories of teacher instructional behavior indicators from classroom video

Article Title: Multi-granularity contrastive learning for discovering categories of teacher instructional behavior indicators

Article References: Cai, T., Tang, Q., Xiong, Y., Zhang, L., & Pandur, A. (2026). Multi-granularity contrastive learning for discovering categories of teacher instructional behavior indicators. Applied Intelligence, 56(15), Article 473. https://doi.org/10.1007/s10489-026-07420-w

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07420-w

Keywords: generalized category discovery, contrastive learning, teacher behavior, classroom video analysis, machine learning, teaching evaluation, pseudo-labels, video representation learning, educational data mining, clustering, Applied Intelligence, semi-supervised learning