The research addresses a practical bottleneck in display manufacturing: inspectors can label an entire image as defective or defect-free relatively quickly, but tracing every defect boundary with boxes or polygons is slow and costly. The proprietary dataset contained 18,804 images at 1,392 by 988 pixels, divided into 12,222 training images, 941 validation images and 5,641 test images. It included nine defect classes and 745 defect-free examples. Images originating from the same physical panel, manufacturing lot or contiguous production period were kept within a single split, reducing the risk that correlated views of a panel would leak information from training into testing.

Training images carried image-level class annotations rather than dense boundary coordinates. Detailed pixel-level polygon references were used for the separate validation and test subsets to tune activation thresholds and calculate spatial-overlap metrics. This tiered protocol allowed the core detector to learn from comparatively inexpensive labels while retaining ground-truth boundaries for evaluation. The authors estimated that their approach could reduce manual annotation time by approximately 71% relative to a baseline procedure in which operators traced multi-vertex polygons around every defect across the data repository.

At the center of the method is a division of labor among several processing routes. One branch screens full images, another estimates the likely center of a defect through class activation mapping, and a third examines local regions at their native pixel density. The activation-mapping models were PYLON-RN50E, built around a ResNet-50 encoder and pyramid attention, and DenseNet-201 with Guided Grad-CAM. Their heatmaps usually emphasize the most discriminative portion of an anomaly rather than its complete outline, so their role is to anchor a search rather than create a perfect defect mask. This distinction is important because overlap with pixel-level ground truth remained modest even when the resulting classification was strong: locating a useful defect core can support recognition without reproducing every boundary pixel.

The researchers also built a classical Canny-Harris region-of-interest detector as an offline reference. It compares a defect image with 138 defect-free backgrounds, extracts edges and corner responses, combines the differences and uses density-based spatial clustering to locate the densest candidate region. The approach captured structural cues that could complement deep models, but it required about 1.056 seconds to score the backgrounds and extract a region from each 1,392 by 988 image, with a larger image taking roughly three times as long. By contrast, the PYLON-RN50E activation-mapping branch processed an image in 17 milliseconds and allocated about 2,048 MiB of video memory at peak load. The authors therefore position Canny-Harris as a calibration or reference tool, while the deep branch is used for faster online inspection.

Threshold selection exposed a pronounced trade-off between sensitivity and false alarms. At their lowest tested activation threshold, DenseNet-201-GGC generated false detections on 58.40% of clean images and PYLON-RN50E on 84.15%, because weak background responses remained in the heatmaps. Raising the thresholds sharply improved performance on defect-free panels. At thresholds of 34 and 213, respectively, DenseNet-201-GGC achieved 98.85% specificity and a 1.15% false-positive rate, while PYLON-RN50E reached 99.15% specificity and a 0.85% false-positive rate. Pushing PYLON-RN50E to a threshold of 249 lowered the false-positive rate further to 0.40%, but its coverage of defect regions collapsed to 2.13%, showing why the threshold that produces the fewest clean-image alarms is not automatically the most useful. The selected settings balanced background suppression, defect coverage and the requirements of downstream classification.

To recover features that might disappear during full-image processing, the framework divides a panel into overlapping windows and classifies the resulting crops with EfficientNetV2-s, EfficientNetV2-s with convolutional block attention, or PYLON-RN50E. A 1,392 by 988 image produces 117 windows of 224 by 224 pixels when the stride is 100 pixels, although PYLON-RN50E uses 256 by 256 windows. The final sliding-window result is determined by the window with the highest defect confidence; if no window indicates a defect, the image is classified as defect-free. This mechanism proved important: a basic majority vote without the sliding-window models reached 91.19% accuracy, while including their three predictions raised the unoptimized vote to 95.81%. A search over 255 possible model subsets on the validation set then excluded the ROI-trained EfficientNetV2-s with attention, producing a locked optimized majority-voting ensemble that reached 96.33% on the independent test set.

The optimized system reported macro-averaged precision of 95.46%, recall of 95.98%, an F1 score of 95.72% and an area under the precision-recall curve of 0.96. The best individual classifier, EfficientNetV2-s trained on regions of interest, achieved 89.19% accuracy, making the ensemble’s advantage 7.14 percentage points. Across 10 training runs with different random seeds, the reported standard deviation was 0.35 for the ensemble. A Wilcoxon signed-rank comparison with the strongest individual baseline gave a p-value below 0.05, indicating a statistically significant difference under the study’s testing protocol. The authors attributed the improvement to two complementary effects: the sliding window recovered spatial information lost during downscaling, while architectural diversity allowed voting among DenseNet, EfficientNet and PYLON models to suppress isolated errors.

Running the branches concurrently required substantial hardware. The full parallel configuration used 21,154 MiB of GPU memory and completed inference in 425.3 milliseconds, compared with about 1.48 seconds for an equivalent sequential arrangement that used less memory. A more moderate configuration ran the sliding-window classifiers concurrently and the remaining branches sequentially, limiting peak allocation to 13,284 MiB. In a separate operational configuration that omitted the sliding-window classifiers, the reported per-image inference time was 41.4 milliseconds with 5,822 MiB of allocated memory. The experiments ran on a workstation with an AMD EPYC 9654P processor, 128 GiB of RAM and an Nvidia RTX 3090 GPU with 24,576 MiB of memory, so the results do not establish performance on lower-power embedded hardware.

Tests on two public datasets suggested that the architecture was not limited to the green-background display imagery used in the primary experiment. On the NEU hot-rolled steel dataset, the framework reported 95.88% accuracy and a 95.10% F1 score, compared with 86.50% and 85.12% for RT-DETR-L and 94.18% and 93.60% for Faster R-CNN. On the DAGM 2007 texture dataset, it reported 96.05% accuracy and a 95.42% F1 score; the corresponding Faster R-CNN results were 92.80% and 91.95%. The public benchmarks were deliberate stress tests outside the system’s manufacturing domain, and the qualitative examples still showed occasional false positives and class confusion. Static thresholds, changes in factory illumination, new material batches and diffuse sub-pixel scratches therefore remain practical limitations despite the strong aggregate scores.

The estimated 71% reduction in labeling hours depends on the assumed baseline workload and should not be read as a universal saving for every factory. The framework’s clearest demonstrated benefit is its combination of image-level training labels, a fast activation-map anchor, native-resolution window inspection and multi-model voting on the evaluated data. Its next step is to reduce memory use and latency, potentially through knowledge distillation and INT8 quantization, while testing on low-power edge devices and changing production environments. Even with those qualifications, the study presents a promising way to make automated optical inspection less dependent on exhaustive manual tracing while retaining the local detail needed to find small defects on display panels.

Subject of Research: Weakly supervised computer-vision framework for detecting defects in TFT-LCD manufacturing

Article Title: An annotation-efficient parallel processing framework for robust defect detection in TFT-LCD manufacturing

Article References: Ibrahim, M. I., Khanali, P., Sajid, S., Wanly, M., Weber, S., Drescher, B., Liu, C., & Mahmud, A. A. (2026). An annotation-efficient parallel processing framework for robust defect detection in TFT-LCD manufacturing. Results in Engineering, 32, Article 113476. https://doi.org/10.1016/j.rineng.2026.113476

Image Credits: AI Generated

DOI: 10.1016/j.rineng.2026.113476

Keywords: TFT-LCD manufacturing, industrial defect detection, weakly supervised learning, automated optical inspection, class activation mapping, sliding-window analysis, ensemble learning, computer vision, industrial quality control, annotation efficiency, annotation-efficient, parallel

To understand why this matters, it helps to unpack the core technique the authors build upon: knowledge distillation. First formalized by Geoffrey Hinton and Oriol Vinyals in 2015, distillation is the practice of transferring what a large, capable neural network knows into a smaller or differently structured network. In anomaly detection, the idea takes on a clever twist. A student network is trained to mimic a teacher network on images of normal products only. Because the student has never seen defects, it learns to reproduce only the features of normality. When a defective image is later presented, the teacher recognizes the full scene while the student, blind to the anomaly, produces a mismatched output. The magnitude of that mismatch becomes an anomaly map, highlighting precisely where the defect lies. The elegance of the method is that it requires no examples of failure at all, only a reliable supply of normal ones.

Where AnomalyMHKD departs from earlier work is in how the knowledge transfer is orchestrated. The framework integrates two complementary modes of distillation. The first, self-distillation, enhances intra-model feature alignment: different layers or heads within the same network teach one another, forcing the model to develop internally consistent representations of what a normal image looks like. This kind of internal harmonization has proven valuable in self-supervised representation learning, where the network generates its own training signal from the structure of the data rather than from human annotations. The second mode, cross-distillation, enables collaborative learning between heterogeneous architectures. Here the authors pair a Vision Transformer, or ViT, with a ResNet-style convolutional network, allowing each to distill its knowledge into the other.

The pairing of these two architectures is more than a technical flourish. Vision Transformers process an image by dividing it into patches and reasoning about the relationships between them through self-attention, giving them a global, holistic view of the scene. Convolutional networks like ResNet, by contrast, build understanding hierarchically from local patterns, excelling at capturing fine-grained textures such as scratches, dents or surface discolorations. When these two ways of seeing are forced into agreement through cross-distillation, the resulting feature representation inherits the strengths of both: the global context of the transformer and the local precision of the convolutional network. For patch segmentation-based anomaly detection, the task of localizing exactly which region of an image is defective, that hybrid representation is precisely what the doctor ordered.

The third ingredient is what the authors call a hybrid mode that toggles the teacher between online and offline learning. In offline distillation, a teacher is trained first and then frozen, its knowledge handed down in a one-way transfer. In online distillation, teacher and student learn simultaneously, adapting to one another throughout training. Each strategy has trade-offs: offline teachers are stable but can become stale, while online teachers stay current but can destabilize training. AnomalyMHKD dynamically switches between the two, and, crucially, employs a dynamic teacher-freezing approach that locks the teacher in place once its contributions plateau. According to the paper, this dynamic freezing is the primary driver of the approximately 30 percent reduction in training cost, since compute is no longer wasted updating a teacher that has already delivered most of what it knows.

The entire pipeline is self-supervised, meaning the model extracts its learning signal from the images themselves rather than from curated labels. This is the property that makes the approach attractive for real-world deployment. In manufacturing settings, collecting and annotating defective samples is expensive, slow and sometimes impossible, since rare defects may not appear until a process drifts months after deployment. A detector that learns normality from unlabeled images and flags deviations can be retrained cheaply whenever a product line changes, and the authors note that the method achieves its strong results with less dependence on labeled data than supervised alternatives demand.

The benchmarks on which the framework was evaluated are the de facto proving grounds for industrial anomaly detection. MVTech AD, introduced by researchers at the Technical University of Munich, contains fifteen categories of objects and textures, from screws and metal nuts to tiles and leather, each photographed with a variety of subtle defects. VisA is a more recent and arguably harder dataset spanning twelve categories of complex scenes, including printed circuit boards and capsules, where anomalies can be small, low-contrast or structurally ambiguous. Scoring 96.6 percent and 93.6 percent AUROC respectively places AnomalyMHKD among the leading published methods, and the authors report that it compares favorably with recent state-of-the-art systems such as EfficientAD, a detector engineered for millisecond-level inference that was presented at the IEEE/CVF Winter Conference on Applications of Computer Vision in 2024.

The study situates itself within a rapidly evolving lineage. Reverse distillation methods, such as the one proposed by Deng and Li at CVPR 2022, inverted the traditional teacher-student arrangement to improve one-class embeddings, and follow-up work like SK-RD4AD in 2025 added skip connections for robustness. FastFlow applied two-dimensional normalizing flows to model the distribution of normal features, while PatchCore pursued nearest-neighbor memory banks toward what its authors called total recall. On the distillation side, multi-mode online knowledge distillation for self-supervised visual representation learning appeared at CVPR 2023, and CrossKD introduced cross-head distillation for object detection in 2024. AnomalyMHKD synthesizes these threads, combining multi-mode distillation with heterogeneous cross-architecture transfer and dynamic teacher management, and directs the resulting machinery at the specific problem of unsupervised anomaly localization.

The practical implications extend well beyond factory floors. The same self-supervised machinery that spots a scratched circuit board can, in principle, be adapted to medical imaging, where labeled pathology is likewise scarce, or to surveillance, infrastructure inspection and any domain where the abnormal is rare by definition. The authors state that the datasets analyzed in the study are publicly available from their respective sources, which lowers the barrier for other groups to reproduce and extend the results. The work also received no specific funding, and the authors declare no conflict of interest, with all four contributors listed as having contributed equally to the manuscript.

There remain, of course, the usual caveats that accompany any new benchmark result. The reported figures come from the authors’ own experiments, and independent replication on additional datasets and deployment conditions will be needed to confirm the promised 30 percent training cost savings in production environments. The method’s reliance on patch segmentation means its localization quality, not just its image-level detection scores, will matter to adopters who need to know not only that a part is defective but where. Still, the conceptual contribution is clear and timely: by letting a transformer and a convolutional network teach each other, by letting a model’s layers align themselves, and by freezing teachers the moment they stop being useful, AnomalyMHKD demonstrates that the path to reliable anomaly detection may run not through bigger labeled datasets, but through smarter ways of transferring the knowledge machines already possess. As factories, hospitals and cities generate ever more visual data with ever fewer annotations, that lesson is likely to echo far beyond the inspection line.

Subject of Research: Self-supervised anomaly detection using multi-mode hybrid knowledge distillation between Vision Transformer and ResNet models

Article Title: AnomalyMHKD: A multi-mode hybrid knowledge distillation approach for self-supervised detection of anomalous objects

Article References: Bhuvana, J., Mirnalinee, T. T., Mohan, H., & M, O. (2026). AnomalyMHKD: A multi-mode hybrid knowledge distillation approach for self-supervised detection of anomalous objects. Multimedia Tools and Applications, 85(9), Article 745. https://doi.org/10.1007/s11042-026-21844-z

Image Credits: AI Generated

DOI: 10.1007/s11042-026-21844-z

Keywords: anomaly detection, knowledge distillation, self-supervised learning, Vision Transformer, ResNet, MVTech AD, VisA, industrial inspection, computer vision, EfficientAD, patch segmentation, teacher-student networks