Efficient video violence detection through segmented temporal sampling and CNN–transformer modeling

Surveillance cameras watch over schools, streets, prisons, and public squares around the clock, yet the humans assigned to monitor those feeds remain the weakest link in the security chain. Operators juggling dozens of screens routinely miss incidents or report them late, a failure mode that has pushed researchers toward deep learning systems capable of flagging violence automatically. The obstacle is that violence is not a static object but an interaction: it depends on people, context, and the temporal evolution of actions, all distorted by lighting, perspective, crowd density, and capture distance. Frame loss, blurring, and insufficient sampling can erase the brief, decisive moments—blows, shoves, scuffles—that separate a violent situation from ordinary activity. Processing long frame sequences does not guarantee better results either, because many frames are redundant while excessive sampling may omit crucial brief movements.

A team of Peruvian researchers led by Fray L. Becerra-Suarez of Universidad Privada Norbert Wiener has now published a study in Frontiers in Computer Science that asks a deceptively simple question: how little temporal information does a model actually need to detect violence reliably? Their answer, arrived at through a rigorously controlled experimental protocol, is remarkably little. The framework they built combines a convolutional neural network for spatial perception with a Transformer encoder for temporal reasoning, and it achieves competitive accuracy while processing just four frames per video—no optical flow, no audio, no recurrent modules, and no dense frame sequences.

The technical design rests on segmented temporal sampling. Instead of feeding the model a continuous stream of frames, the system divides each video into approximately equal time segments and selects one frame from each. During training, the frame is chosen randomly within its segment; during validation and testing, selection is deterministic, using the midpoint of each interval. This distributed strategy reduces redundancy while preserving global temporal coverage across the entire video. The team evaluated three temporal budgets—4, 8, and 16 frames—sharing identical architectural components so that the only variable was the amount of temporal information provided to the model.

Each sampled frame is converted from BGR to RGB, resized to 224 by 224 pixels, and normalized with ImageNet statistics before entering the spatial extractor: EfficientNet-B0, a compact convolutional backbone pre-trained on ImageNet. The classifier head is removed, and each frame yields a 1280-dimensional spatial descriptor encoding appearance, posture, interaction, and scene context. These descriptors are then projected onto 256-dimensional tokens, supplemented with sinusoidal positional encoding so the Transformer can distinguish similar visual features observed at different temporal positions. A Transformer encoder with two layers and four self-attention heads compares all sampled frames, allowing each temporal position to incorporate information from every other segment without recurrent processing. An attention-pooling mechanism then learns the relative importance of each frame, producing a single 256-dimensional global vector that summarizes the most discriminative temporal evidence before a two-class linear layer outputs the violence probability.

The experimental protocol deserves attention because it addresses weaknesses common in the violence-detection literature, where heterogeneous partitions, undocumented seeds, and possible data leakage make reported scores difficult to trust. Here, dataset partitioning was performed at the video level with a 60/20/20 split, guaranteeing that no video appears in more than one partition. Training was repeated across three independent seeds—42, 123, and 2026—and configurations were compared using validation performance only, with the test set reserved exclusively for final evaluation. Bootstrap resampling with 2,000 iterations estimated metric uncertainty, and paired bootstrap comparisons assessed whether differences between models were statistically meaningful.

The headline finding is that more frames did not produce better performance. Under identical conditions, the 4-frame configuration achieved the most favorable multi-seed validation results, with an accuracy of 0.9808, an F1-score of 0.9809, and an MCC of 0.9617—outperforming both the 8-frame and 16-frame variants. The ablation study confirmed that the architecture depends on the combined effect of its components rather than any single mechanism. Removing the Transformer encoder dropped validation MCC to 0.9417, replacing attention pooling with temporal averaging lowered it to 0.9440, and keeping the EfficientNet-B0 backbone frozen reduced it to 0.9404, indicating that temporal interaction and progressive adaptation of the spatial extractor contribute substantially to the model’s discrimination.

On the Real Life Violence Situations test set—a collection of 2,000 videos, half depicting fights in streets, prisons, and schools and half showing generic human activities—the selected 4-frame model achieved an accuracy of 0.9508, an F1-score of 0.9517, an MCC of 0.9022, and a precision-recall AUC of 0.9915. Controlled comparisons against two temporal baselines built with the same backbone and sampling scheme—an EfficientNet-B0 with mean pooling and an EfficientNet-B0 with a GRU—showed statistically comparable performance in most runs, with the CNN–Transformer maintaining the highest mean PR-AUC in the controlled comparison. The team also applied Grad-CAM to the final convolutional stage and examined the attention-pooling weights, revealing that in true positives, activation concentrates on regions of physical interaction while temporal weights rise from 0.220 in the first frame to 0.286 in the fourth, showing the model learns to emphasize the segments where interaction becomes visually evident.

Perhaps the most operationally relevant contribution is the robustness analysis, an aspect many previous studies neglect by reporting metrics only under clean laboratory conditions. The researchers applied nine perturbations during inference without retraining or threshold recalibration: brightness changes, Gaussian noise, Gaussian blur, JPEG compression, frame dropping, early and late temporal selection, and a clean reference. Gaussian noise proved the most damaging condition, and here the CNN–Transformer showed the smallest degradation, with an MCC drop of −0.1853 compared to −0.2863 for mean pooling and −0.2941 for the GRU. The proposed architecture also remained practically stable against frame dropping and high brightness, and temporal shifts in frame selection had limited effect. The interpretation is that self-attention and learned temporal weighting help preserve useful information when some frame-level representations become unreliable—a property that matters enormously when surveillance footage is noisy or compressed.

The efficiency numbers make the framework genuinely deployable. The final model contains 5.39 million parameters, occupies roughly 20.57 MB, and achieves an average inference latency of 9.57 milliseconds per video, corresponding to a throughput of about 104 videos per second on the team’s test hardware. The mean-pooling baseline was slightly lighter at 4.34 million parameters and 9.16 milliseconds, but the Transformer’s additional temporal modeling did not produce a proportional increase in inference time. Independent benchmark evaluation extended the results beyond RLVS: trained directly on Hockey Fight Videos, the CNN–Transformer reached an accuracy of 0.9517, and on the more demanding AIRTLab dataset—whose non-violent subset deliberately contains hugging, clapping, high-fives, and exulting, movements that mimic violent interactions—it achieved an accuracy of 0.8857 while maintaining a high PR-AUC of 0.9684.

The study is equally candid about its limits. When models trained exclusively on RLVS were transferred to the external datasets without fine-tuning or threshold recalibration, discrimination collapsed: MCC fell to 0.3390 on Hockey Fight and 0.0505 on AIRTLab, with specificity dropping sharply as non-violent sequences from the new domains were misclassified as violent. This domain-shift experiment demonstrates that strong performance on data representative of the target domain does not guarantee discrimination under changes in visual and behavioral distribution. The authors also note that the reported latency covers only model inference, not the complete video-processing pipeline, and that the framework has not yet been validated on continuous live surveillance streams or resource-constrained edge devices. Future work, they suggest, should explore domain adaptation, multi-dataset training, multi-category classification of violent events, and validation in operational scenarios. Still, the core message stands: temporally distributed observations—just four well-chosen frames—can preserve the discriminative information needed for violence detection, offering a viable blueprint for surveillance systems where memory, latency, and computational capacity constrain practical implementation.

Subject of Research: Efficient video violence detection through segmented temporal sampling and CNN–transformer modeling

Article Title: Efficient video violence detection through segmented temporal sampling and CNN–transformer modeling

Article References: Becerra-Suarez, F. L., Menacho-Navarrete, K., Díaz, M., Recharte-Trucios, F. A., Landeo-Castillo, F. R., & Pinedo, L. (2026). Efficient video violence detection through segmented temporal sampling and CNN–transformer modeling. Frontiers in Computer Science, 8, Article 1934449. https://doi.org/10.3389/fcomp.2026.1934449

Image Credits: AI Generated

DOI: 10.3389/fcomp.2026.1934449

Keywords: Efficient, video, violence, detection, segmented, temporal, sampling, transformer, modeling, scientific research