Two AI Teachers, One Tiny Student: The Distillation Trick That Could Fit Full ADAS Into a 6 MB Model

A car that can read every stop sign on the road and simultaneously watch for the truck barreling toward it usually needs two separate AI systems, two inference passes, and roughly twice the computing hardware. A research team from the Department of Electronics and Communication Engineering at Amrita Vishwa Vidyapeetham in Chennai now argues that this duplication is unnecessary. In a study published in Frontiers in Future Transportation, M. Maddan, N. Sankkarshana, M. Gayathri, S. G. Rahul and T. M. Amirthalakshmi describe a unified Advanced Driver Assistance System (ADAS) architecture that compresses two specialist perception networks into a single lightweight student model using knowledge distillation, achieving near-perfect traffic sign recognition and solid collision-threat detection within a footprint of about 3 million parameters and 6.2 megabytes.

The core insight is that the two perception streams are complementary rather than redundant. Traffic Sign Recognition (TSR) supplies the static rules of the road: speed limits, stop commands, no-entry zones, pedestrian-crossing warnings, and directional information. Forward Collision Warning (FCW) supplies the dynamic state of the environment: vehicles, trucks, pedestrians, cyclists, and the color state of traffic lights. Neither stream alone gives a complete picture. A pedestrian detection is more alarming when a school-zone sign is active; a closing-speed calculation is more meaningful when the system knows the current speed limit. The team’s bimodal fusion distillation module is designed to capture exactly this conjunction of regulatory context and hazard detection.

Both specialist networks, which serve as teachers in the distillation framework, are built on YOLOv8n, the smallest variant of the YOLOv8 single-stage detector family. The architecture combines a C2f backbone with cross-stage partial connections for multi-scale feature extraction, a Feature Pyramid Network paired with a Path Aggregation Network (FPN-PAN) neck for fusing features across scales, and a decoupled, anchor-free detection head that separates bounding-box regression from classification. The anchor-free design is particularly suited to traffic signs, whose aspect ratios and scales vary widely. The FPN-PAN neck matters most for FCW, where the same vehicle might appear as a tiny distant blob at 100 meters or fill the frame at 10 meters.

The TSR teacher was trained on the German Traffic Sign Recognition Benchmark (GTSRB), comprising 39,209 images across 43 sign classes, split 80:20 into training and validation partitions. Because real-world sign frequency is heavily skewed, with dominant categories outnumbering rarer classes by roughly 34 to 1, the team applied color jitter, HSV enhancement, mosaic augmentation, motion blur simulation, random horizontal flipping, and perspective transformations to increase diversity. Training ran for 50 epochs at a batch size of 8, using Complete IoU loss for box regression, binary cross-entropy for classification, and Distribution Focal Loss for sub-pixel localization refinement. The result is striking: the TSR module achieved a mean Average Precision at IoU 0.5 (mAP@0.5) of approximately 0.994, with precision of 0.9935 and recall of 0.9943. The F1 score peaked at roughly 0.99 at a confidence threshold of 0.576, a deliberately low operating point that maximizes recall, since in an ADAS context a missed sign carries a higher penalty than a spurious detection.

The FCW teacher was trained on a Roboflow-hosted version of the Udacity self-driving car dataset, filtered from 15,000 images and 97,942 bounding-box labels down to 13,063 images across 11 classes, split 70:20:10 into training, validation, and testing. Class imbalance here is extreme: cars account for roughly 66 percent of all labels, while the rarest class, the yellow left-turn traffic light, has just 14 labels, a disparity exceeding 4,600 to 1. Training ran for 75 epochs at a batch size of 4, with the smaller batch providing more frequent gradient updates for underrepresented safety-critical classes. The FCW module achieved an mAP@0.5 above 0.81 for the primary car class (0.818) and 0.786 for trucks, with red traffic lights at 0.809. Performance drops for pedestrians (0.575) and cyclists (0.610), and the yellow light state reached only 0.444, reflecting both the scarcity of training data and the visual difficulty of these categories. Overall FCW precision was 0.7852 with recall of 0.6332, and the team acknowledges that further work is needed on underrepresented and visually complex object classes.

The distillation framework itself is where the architectural novelty lies. Rather than sharing a single backbone with multiple task-specific heads, as many multi-task perception systems do, the team trained two independently optimized teachers and then transferred their combined knowledge into a single student YOLOv8n network of approximately 3.02 million parameters, comparable in size to a single-task teacher. The student is trained to match the combined soft-probability logits of both teachers at the decoupled detection head, scaled by a softmax temperature of T = 3.0. The distillation loss weights the TSR teacher’s contribution at 0.4 and the FCW teacher’s contribution at 0.6, alongside the standard hard-label detection loss. Because the single student backbone and neck must simultaneously support accurate soft-label prediction for both teachers, it is implicitly driven to learn internal features general enough to handle both the rigid geometry of traffic signs and the variable appearance of dynamic road objects, without requiring a dedicated feature-matching term.

The efficiency gains are quantified in the team’s ablation study. Running the TSR and FCW models independently requires approximately 6.03 million parameters and 484.2 milliseconds of latency per frame, roughly double the cost of a single model. The distilled student achieves fused perception in 242.1 milliseconds at approximately 4.13 frames per second, with about 3.02 million parameters, effectively halving the computational cost while preserving the detection performance of both underlying modules. The fusion operation itself introduces negligible overhead because it performs only decision-level integration rather than additional neural network inference. Both trained models require 8.2 GFLOPs for inference and have checkpoint sizes of approximately 6.2 megabytes, making them deployable on resource-constrained automotive Electronic Control Units without a dedicated GPU.

The collision-warning logic builds on classical Time-to-Collision (TTC) thresholds established in earlier FCW research: TTC below 2.0 seconds triggers an immediate warning, while TTC below 1.0 seconds triggers critical danger and automated emergency braking. Because the system relies on a monocular camera without direct velocity measurement, the team uses a bounding-box area-growth heuristic as a practical proxy for TTC, with a 5-frame temporal smoothing buffer to suppress transient glitches. Under standard urban driving conditions at approximately 50 km/h, this proxy TTC estimation carried an error of about 18.5 percent, the False Warning Rate was 5.2 percent, and the Missed Warning Rate was 6.4 percent. The team notes that the current inference speed of roughly 4 FPS is appropriate for low-power, GPU-free embedded hardware but may not suffice for highly time-critical functions like automated emergency braking, which typically requires 15 to 30 FPS or higher; achieving higher speeds would require model quantization, optimized inference engines such as TensorRT or ONNX Runtime, or further pruning and compression, all identified as important directions for future work.

The team also validated reproducibility across independent training runs using different random seeds. The TSR module showed low variability, with precision of 0.9935 plus or minus 0.0018, recall of 0.9943 plus or minus 0.0021, and mAP@0.5 of 0.9944 plus or minus 0.0015. The FCW module showed higher variability, with precision of 0.7852 plus or minus 0.0112, recall of 0.6332 plus or minus 0.0145, and mAP@0.5 of 0.6995 plus or minus 0.0124, attributed to the greater complexity of dynamic object detection, variations in scale and appearance, and the pronounced class imbalance. Nevertheless, the observed standard deviations remain low, indicating stable and reproducible performance across different training runs. The team identifies larger-scale multi-seed statistical validation, including confidence intervals and formal significance testing, as an important avenue for future work.

The broader significance extends beyond the specific metrics. The team frames the work as contributing to UN Sustainable Development Goal 9 on Industry, Innovation and Infrastructure, by enabling computationally efficient AI-based intelligent transportation technologies, and to Goal 11 on Sustainable Cities and Communities, through improved road safety and intelligent mobility. The modular design also facilitates the integration of additional perception capabilities, such as lane detection, driver monitoring, or traffic signal prediction, without requiring substantial modifications to the overall framework. Planned extensions include lane detection and road-surface anomaly detection, depth estimation to improve TTC accuracy, a driver monitoring module for closed-loop interventions, expanded training sets for underrepresented classes, and closed-loop autonomous driving simulations to test the full system. The study demonstrates that accurate and reliable perception for embedded real-time ADAS applications can be built from lightweight, properly trained, and appropriately fused models, offering a practical template for next-generation intelligent transportation systems that must balance detection performance against the hardware constraints of cost-effective automotive platforms.

Subject of Research: A unified lightweight ADAS architecture integrating traffic sign recognition, forward collision warning, and bimodal distillation for autonomous perception

Article Title: A unified lightweight ADAS architecture integrating traffic sign recognition, forward collision warning, and bimodal distillation for autonomous perception

Article References: Maddan, M., Sankkarshana, N., Gayathri, M., Rahul, S. G., & Amirthalakshmi, T. M. (2026). A unified lightweight ADAS architecture integrating traffic sign recognition, forward collision warning, and bimodal distillation for autonomous perception. Frontiers in Future Transportation, 7, Article 1889785. https://doi.org/10.3389/ffutr.2026.1889785

Image Credits: AI Generated

DOI: 10.3389/ffutr.2026.1889785

Keywords: unified, lightweight, ADAS, architecture, integrating, traffic, sign, recognition, forward, collision, warning, bimodal