The core insight of the work is that the models that power modern chatbots, known as vision-language models, can do more than describe a scene. They can be taught to make bounded, auditable safety decisions. But the researchers found that off-the-shelf versions of these models fail at the task in a dangerous and instructive way. When general-purpose models were asked, without any task-specific training, to watch short video clips of an intersection and recommend an action, the larger seven-billion-parameter model intervened on nearly every clip it saw, including clips of completely normal traffic where the correct answer was to do nothing. Its apparent accuracy on hazardous events was an artifact of always sounding the alarm. The smaller three-billion-parameter model, meanwhile, produced outputs in the wrong format, so the signal controller could not execute them at all. For a safety layer, the decisive failure is not missing a hazard but crying wolf on every quiet street corner, eroding both traffic flow and trust.

To fix this, the team built a training dataset of 1,020 video clips generated inside the CARLA driving simulator, each roughly two to three seconds long and captured by two fixed virtual cameras whose views are stacked into a single composite image of the intersection. The clips cover nine combinations of weather and lighting, from clear noon to hard rain at night, and fall into three classes: jaywalking, red-light running, and normal traffic. Every clip was labeled automatically by a deterministic expert policy that maps each event and signal state to an ideal intervention. A jaywalker triggers a long all-red hold of eight to ten seconds, calibrated to the geometry of the crosswalks, which span about 14.5 to 15 meters; a red-light runner triggers a shorter hold of five seconds for cars and six for heavy vehicles, enough to clear the intersection before conflicting movements are released; normal traffic triggers no action at all.

One of the most technically interesting parts of the pipeline addresses a subtle data problem. The object detector that anchors the system, a YOLOv10-l model, performs well in daylight but nearly collapses at night and in heavy rain. In the first collection pass, the detector found around 40 jaywalking pedestrians per daytime condition but only six on a clear night and a single one under hard rain at night. Because the pedestrians were scripted actors whose positions the simulator knows exactly, the researchers projected ground-truth positions onto the camera images using pinhole camera geometry, recovering correct labels for 128 of the 348 jaywalking samples. Crucially, the projection changes only the labels, not the video itself, so the model still has to learn to recognize faint, barely visible hazards from raw pixels alone.

The fine-tuning itself is remarkably efficient. The researchers started with Qwen2.5-VL-3B-Instruct, a compact open model, and applied low-rank adaptation, a technique that freezes the original network and injects a small number of trainable update matrices into its attention and feed-forward layers. Only 37.15 million parameters, less than one percent of the model, were trained, and the vision encoder was left untouched. Training took about 40 minutes on a single workstation GPU, with loss decaying smoothly from 1.13 to 0.004 over three passes through the data. The model was trained to emit a strict JSON decision document: whether to intervene and why, how long to hold all-red, which legal signal combination to run, per-phase green times, safety notes, and a confidence level. Every field is range-checked by a separate rule-constrained controller, which rejects anything malformed, out of bounds, or inconsistent with traffic-engineering constraints. The AI never touches the signal directly.

The evaluation was deliberately harsh. Three of the nine weather conditions, wet sunset, clear night, and hard rain at night, were withheld entirely from training, and the fine-tuned model was tested on 313 clips from those unseen conditions. It achieved perfect schema validity, perfect adherence to controller bounds, and a perfect intervene-or-do-nothing decision on every single clip, with a mean hold-time error of just 0.15 seconds. Its false-positive rate on the 108 normal-traffic clips was zero, with a statistical upper bound of 3.4 per hundred. The zero-shot seven-billion-parameter baseline, by contrast, scored only 0.655 on decision accuracy, exactly the fraction of event clips in the test set, because it intervened on all of them, and its hold-time errors averaged over four seconds. Notably, a seven-billion-parameter model fine-tuned with the identical recipe performed no better than the three-billion one, and the smaller model was also faster, with a median decision time of 2.77 seconds versus 3.20 seconds for its larger zero-shot rival. Adaptation, not scale, is what makes these models usable inside a control loop.

The system also generalized beyond its training environment. At a second simulated intersection in a different virtual town, with different road geometry, surroundings, and camera positions, and with no retraining whatsoever, the fine-tuned model again made the correct decision on all 135 test clips. The only degradation was in hold timing: it recommended nine-second holds where the site-specific expert target was eight seconds, a one-second error that still fell within the mandated eight-to-ten-second band. Textual explanations produced at the new site referred to its actual crosswalks and approaches rather than echoing the training scene, suggesting the model carried across an imitated policy rather than a memory of one location’s appearance.

In closed-loop operation, where the full sensing, reasoning, and control stack ran live against scripted violation scenarios, the safety benefits were substantial but came with a measurable cost. Compared with blind fixed-time signal control, the ground-truth pedestrian exposure fraction, the share of total crosswalk-occupancy time overlapping a conflicting green, was 67.6 percent lower at low demand and 73.0 percent lower at medium demand, with every random seed favoring the AI layer. Against a stronger actuated pedestrian-clearance baseline, the reductions were 55.4 and 71.6 percent. An analysis showed the gains came not merely from serving less conflicting green but from shifting that green away from moments when pedestrians were actually present. The price, at medium demand, was a 105.9 percent increase in average stopped delay and a 13.8 percent throughput drop, though the researchers caution that the scripted scenarios inject hazardous events far more often than real intersections experience, inflating the intervention frequency and its delay burden by construction.

The study is equally candid about its limits, and the most consequential one lies upstream of the AI. At night, the object detector recovers only about 10.7 percent of ground-truth crosswalk occupancy from the same imagery, compared with 62.6 percent in daylight, and under clear-night conditions the closed-loop safety improvement shrank to a statistically insignificant 15.2 percent. A semantic layer cannot protect a pedestrian it cannot see. All results come from simulation, with safety-critical events injected by script rather than observed in natural traffic, so no claim of real-world transfer is made. Still, the authors argue the central conclusion stands: domain adaptation, not model scale, is what allows a vision-language model to operate reliably within a controller’s constraints, and the path to deployment now runs through weather-robust perception, sensor fusion, and shadow-mode validation on real traffic cameras. If that perception gap closes, the vision of intersections that watch, understand, and briefly freeze to protect the most vulnerable road users moves a significant step closer to reality.

Subject of Research: A lightweight vision-language AI safety layer for pedestrian protection at signalized intersections

Article Title: A Fine-Tuned Lightweight Vision-Language Safety Layer with Weather-Generalising Decisions at Signalized Intersections

Article References: Afshari, A., Lee, J., & Mashal, Y. (2026). A Fine-Tuned Lightweight Vision-Language Safety Layer with Weather-Generalising Decisions at Signalized Intersections. Machine Learning with Applications, 26, Article 101032. https://doi.org/10.1016/j.mlwa.2026.101032

Image Credits: AI Generated

DOI: 10.1016/j.mlwa.2026.101032

Keywords: vision-language models, traffic signal control, pedestrian safety, fine-tuning, LoRA, CARLA simulator, computer vision, intelligent transportation systems, weather generalization, all-red hold, machine learning, road safety

The work targets a long-standing pain point in multi-agent reinforcement learning (MARL), the dominant modern approach to decentralized traffic signal control. In a MARL setup, each intersection hosts an agent that learns its own policy — when to switch phases, how long to hold green, when to yield to cross traffic. The trouble is that the global reward, such as a network-wide reduction in travel time, is shared among all agents. Determining which intersection’s decisions actually produced the improvement is the credit assignment problem, and it is notoriously difficult when the effects of one signal ripple through neighboring blocks.

Conventional solutions tackle credit assignment with critic networks that attempt to decompose the global reward into per-agent contributions. These approaches carry two fundamental limitations, the authors argue. First, they are sample-hungry: agents must engage in extensive trial-and-error interactions with the traffic environment before the critics learn to allocate credit meaningfully, and every trial in a live or simulated network is expensive. Second, the critics must possess strong extrapolation and generalization capabilities — they must interpret policies they were never trained on and still distribute credit sensibly. When traffic patterns shift, as they constantly do, critic networks can produce misleading credit signals that push agents toward worse policies.

BootDiffLights sidesteps this entire machinery. The key insight is a reframing: rather than asking how to divide a global reward among agents, the framework adopts a semi-offline multi-agent reinforcement learning paradigm that transforms the problem from credit assignment into black-box optimization. In practical terms, the system no longer focuses on exploring optimal policies directly. Instead, it concentrates on generating optimal sample data — high-quality trajectories of traffic states and actions — and optimizes in that data space. The generative engine doing this work is a conditional diffusion model, the same class of model that has powered recent breakthroughs in image synthesis and, increasingly, decision-making.

Diffusion models work by learning to reverse a gradual noising process. Trained on data, they learn to transform pure noise into samples that resemble the training distribution, and conditioning signals can steer what they generate. In BootDiffLights, the conditional diffusion model serves as an implicit credit-assignment mechanism: it learns the relationship between traffic conditions, agent behaviors, and resulting rewards, and can then synthesize reward signals consistent with desired regional outcomes. Because the model captures the joint structure of the multi-intersection system, it implicitly encodes how each signal’s actions contribute to network-level performance — without an explicit critic network attempting the decomposition.

The semi-offline aspect matters as much as the diffusion engine. Fully offline reinforcement learning restricts agents to a fixed dataset, which is safe but often suboptimal; fully online learning requires costly and sometimes dangerous exploration. A semi-offline approach lets the system bootstrap from logged traffic data while selectively gathering new interaction data where it matters most. This is what enables BootDiffLights to improve sampling efficiency: the diffusion model can propose promising regions of the behavior space, and the learning system refines its data generation around those regions rather than wandering blindly through trial and error. The framework essentially shifts computational effort from interacting with the environment to generating better synthetic experience.

The authors evaluated the framework on both synthetic and real-world traffic datasets, running all experiments with five random seeds and reporting results as mean plus or minus standard deviation — a rigorous protocol that guards against lucky draws inflating performance claims. Across these benchmarks, BootDiffLights outperformed state-of-the-art baselines on regional traffic metrics, including average queue length and travel time. The comparison field includes well-established MARL techniques built on value decomposition and multi-agent actor-critic architectures, meaning the generative approach did not merely match conventional methods but surpassed them on the metrics that commuters actually feel.

The significance of this work extends beyond one benchmark suite. Traffic signal control has been dominated for decades by fixed-time plans and adaptive schemes such as SCOOT and SCATS, with reinforcement learning offering a data-driven alternative since the late 2010s. The field’s persistent obstacle has been scale: methods that work beautifully on a handful of intersections degrade as networks grow, because credit assignment becomes combinatorially harder. By replacing explicit credit decomposition with generative reward synthesis, BootDiffLights offers a path that may scale more gracefully. The diffusion model’s capacity to model complex joint distributions could prove decisive in dense urban cores where dozens of signals interact within a few blocks.

The approach also connects to a broader trend in machine learning: the migration of generative modeling into sequential decision-making. Diffusion-based policies and planners have recently shown impressive results in robotics and offline control, and BootDiffLights brings that momentum to a civic infrastructure problem with enormous practical stakes. Congestion costs cities billions annually in lost time and fuel, and emissions from idling vehicles degrade urban air quality. A control framework that learns efficiently from existing data, requires no explicit reward decomposition, and improves region-wide metrics could translate into real deployments — though the authors note that data availability constraints limit what can be shared publicly for now.

Challenges remain before such systems direct real intersections. Diffusion models carry their own computational burden at inference time, and traffic agencies will demand robustness guarantees that simulation studies alone cannot fully provide. Yet the conceptual shift is compelling: where previous frameworks asked a critic to guess who deserved credit, BootDiffLights asks a generative model to imagine the rewards worth pursuing — and then optimizes toward that imagination. For a problem as tangled as city-wide traffic coordination, that creative reframing may be exactly the kind of fresh thinking the field has been waiting for.

Subject of Research: Diffusion-based reward generation for decentralized multi-intersection traffic signal control in semi-offline multi-agent reinforcement learning

Article Title: BootDiffLights: Decentralized traffic signal control via diffusion-based reward generation

Article References: Hao, B., He, J., Zheng, J., Dong, W., Yang, Q., & Ma, X. (2026). BootDiffLights: Decentralized traffic signal control via diffusion-based reward generation. Applied Intelligence, 56(15), Article 437. https://doi.org/10.1007/s10489-026-07487-5

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07487-5

Keywords: diffusion model, traffic signal control, multi-agent reinforcement learning, credit assignment, offline reinforcement learning, semi-offline learning, black-box optimization, reward generation, queue length, travel time, smart infrastructure, Applied Intelligence