AI Learns to Spot Abnormal Walking by Watching How Bodies Move

How a person walks can quietly reveal what is happening inside their nervous system, muscles, and joints. A slowing stride, a stiff knee, or a subtle tremor in the legs can be among the earliest visible signs of Parkinson’s disease, stroke, cerebral palsy, or vestibular disorders. For decades, clinicians have relied on subjective observation or expensive, laboratory-bound equipment to quantify these deviations. Now, a team of computer engineers at Kocaeli University in Turkey has shown that a carefully designed artificial intelligence system, watching nothing more than ordinary video footage, can distinguish abnormal gait from normal walking with remarkable reliability — and that the secret to success lies not in building ever more complex machines, but in choosing the right combination of what the computer sees.

The study, published in the open-access journal Discover Artificial Intelligence, tackles a problem that has long frustrated researchers in clinical gait analysis. Traditional sensor-based approaches, using pressure plates or wearable devices, are costly, laborious, and impractical for large-scale screening. Camera-based alternatives promise a simpler path, but they come with their own vulnerabilities. Systems that analyze raw video images — so-called appearance-based methods — can be thrown off by changes in lighting, cluttered backgrounds, loose clothing, or the angle of the camera. Methods that instead track the coordinates of body joints, known as skeleton-based approaches, are immune to many of these visual distractions but discard the rich visual cues embedded in the original footage. The Kocaeli team, led by Pınar Güner Şahan together with Erva Alacan and Suhap Şahin, set out to determine whether combining both views of the same walking person could produce something stronger than either alone.

Their testing ground was the Gait Abnormality Video Dataset, or GAVD, one of the few large, clinically annotated collections of real-world walking videos in existence. GAVD contains gait videos of more than 400 participants gathered from publicly available online platforms, with each clip labeled by expert clinicians as normal or abnormal, and abnormal cases further subdivided into disorders such as Parkinson’s disease, cerebral palsy, and stroke. Because the videos were recorded under uncontrolled conditions — varying lighting, indoor and outdoor settings, different camera angles and distances — the dataset closely mirrors the messy reality of clinical deployment. The researchers worked with 1,798 verified sequences, of which 1,509 were labeled abnormal and only 289 normal, an imbalance that itself poses a serious machine-learning challenge. A pixel-level analysis of the footage showed just how heterogeneous the data is: the walking subject occupied anywhere from roughly 15 percent of the frame height in distant shots to nearly the entire frame in close-ups, with a median of 70 percent.

The architecture the team built processes each video through two parallel branches. The first branch extracts appearance features from the raw RGB frames using SlowFast, a two-stream convolutional network originally developed for action recognition. SlowFast’s design is elegantly suited to gait: a slow pathway operating at a low frame rate captures detailed spatial information about the body in each frame, while a fast pathway running at a high frame rate tracks fine-grained temporal dynamics — the rapid, subtle movements that distinguish a pathological gait from a healthy one. The second branch operates on skeleton data derived from the same videos. Using the markerless pose estimator YOLOv8-Pose, the researchers extracted seventeen anatomical joint points per frame, then normalized the coordinates by centering the body in the image and scaling all positions relative to torso length, so that the model would not be distracted by where the person stood or how far they were from the camera. These joint sequences were fed into a Spatial Temporal Graph Convolutional Network, or ST-GCN, which treats the skeleton as a graph — joints as nodes, anatomical connections as edges — and learns how relationships between body parts evolve over time.

The two feature streams were then merged through a late fusion strategy: the 2,304-dimensional vector from the RGB branch and the 256-dimensional vector from the skeleton branch were simply concatenated into a single 2,560-dimensional representation, which was passed to a compact multi-layer perceptron classifier. That classifier compresses the fused vector to a 512-dimensional hidden layer, applies batch normalization and a ReLU activation, and produces final class probabilities through a softmax function. Crucially, the team froze both backbone networks after training them individually as unimodal classifiers, updating only the small fusion head during multimodal training. This frozen feature-extraction strategy, they would later discover, was not a compromise but a decisive advantage.

Evaluated with five-fold stratified cross-validation across the full dataset, the proposed SlowFast plus ST-GCN configuration achieved a recall of 0.947, an F1-score of 0.910, and an area under the receiver operating characteristic curve of 0.783, with an overall accuracy of 84.3 percent. Under the abnormal-only protocol of the original GAVD benchmark — a test set containing no normal sequences at all — the same model correctly detected 1,159 of 1,220 abnormal gait sequences, a sensitivity of 95.0 percent. In a clinical screening context, that sensitivity figure matters enormously: every false negative represents an abnormal gait that goes undetected, potentially delaying a diagnosis of a progressive neurological condition.

Perhaps the most scientifically interesting findings came from the team’s systematic ablation experiments, which isolated individual design decisions. When three fusion strategies were compared — direct concatenation, an attention-based weighting scheme, and a sigmoid gating mechanism — all three performed comparably, with no strategy dominating once fold-level variation was accounted for. The elaborate cross-modal interaction mechanisms, so popular in recent deep learning literature, provided no measurable benefit over simple concatenation on this dataset. The authors suggest a plausible reason: with only 289 normal sequences available, there may simply be too little data to reliably constrain the additional parameters that attention and gating heads introduce. Notably, an independent research group working on post-stroke assessment with a different modality pair reached the same conclusion, finding concatenation superior to multi-head attention in both accuracy and computational speed.

The training-strategy ablation delivered an equally counterintuitive message. Performance degraded progressively as more of the network was made trainable: the frozen regime, in which both backbones served purely as feature extractors and only the fusion head learned, attained the best and most stable results, while two-stage fine-tuning and full end-to-end training both fell behind. With a small and heavily imbalanced training set, increasing the number of trainable parameters appears to promote overfitting toward the majority abnormal class rather than improving generalization. It is a finding that resonates far beyond gait analysis, offering a caution to any applied deep learning project tempted to unfreeze everything in pursuit of higher numbers.

The study also confronted an uncomfortable truth about its own modality choices. When two representative multimodal methods from the literature were adapted to GAVD under a unified protocol, a silhouette-plus-skeleton baseline — pairing VGG-extracted silhouette features with ST-GCN — actually surpassed the proposed model on several balanced metrics, achieving the highest AUC of 0.814, balanced accuracy of 69.1 percent, and macro-F1 of 0.671. The proposed model, by contrast, excelled at identifying abnormal gait, posting the highest recall and F1-score. The authors attribute this asymmetry to the nature of the normal class in GAVD: normal gait is a residual category encompassing any walking not identified as pathological, making it visually far more heterogeneous than any single abnormal subclass. A shape abstraction like the silhouette may filter out this appearance variability, while a raw RGB stream risks capturing spurious correlations. Yet the silhouette approach demands an extra person-segmentation stage that can fail in crowded or cluttered scenes, whereas the proposed method uses the RGB frames it already has.

The researchers were candid about the limitations of their work. Analysis of the 61 abnormal sequences missed under the abnormal-only protocol revealed that most false negatives occurred in crowded scenes and low-resolution videos — conditions known to degrade pose estimation through occlusion and to blur the pathological cues that distinguish disordered walking. The study is also bounded by its reliance on a single dataset with a small normal class, its binary normal-versus-abnormal formulation rather than finer-grained disease classification, and its use of two-dimensional pose representations. Future directions include expanding the diversity of normal gait data and exploring richer, explainable cross-modal representations that could improve not only classification performance but also the clinical interpretability of predictions. Still, the broader takeaway stands: in real-world clinical video analysis, the skeleton carries most of the diagnostic signal, a strong appearance backbone adds genuine complementary value, and disciplined simplicity — frozen features, plain concatenation, a small classifier — can outperform architectural sophistication when data is scarce and the world is messy.

Subject of Research: Multimodal deep learning for detecting abnormal human gait from real-world clinical video

Article Title: Enhancing clinical gait abnormality analysis with multimodal deep learning

Article References: Güner Şahan, P., Alacan, E., & Şahin, S. (2026). Enhancing clinical gait abnormality analysis with multimodal deep learning. Discover Artificial Intelligence, 6(1), Article 1464. https://doi.org/10.1007/s44163-026-02522-3

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02522-3

Keywords: clinical gait analysis, gait abnormality detection, multimodal deep learning, SlowFast, ST-GCN, skeleton features, late fusion, computer vision, GAVD dataset, pose estimation, Parkinson's disease, medical AI