Motion-Only AI Learns to Recognize Actions While Blurring Out Faces and Skin Color

Every time a camera watches someone pour a cup of coffee or climb a staircase, it records far more than the action itself. Faces, skin tone, clothing, and the surrounding scene all ride along in the raw pixels, and modern action-recognition models are notorious for latching onto exactly these shortcuts. A network trained to spot exercise might quietly learn to recognize the gym, the outfit, even the person. Now a team at Delft University of Technology, working with the Dutch National Police’s Model-Driven Decisions Lab, has shown that you can strip nearly all of that away and still get a useful action recognizer — by feeding the network nothing but motion.

The new study, published in Frontiers in Imaging, tackles a deceptively simple question: how much action-recognition accuracy survives when appearance is removed from the input entirely? Instead of relying on learned privacy filters that try to scrub sensitive information out of video after the fact, the researchers turned to two classic, learning-free motion representations: dense optical flow, which captures pixel-by-pixel displacement between consecutive frames, and Motion History Images, which accumulate recent movement into a decaying temporal template. Because both are computed by deterministic algorithms, they inherently discard texture, color, and fine detail — no retraining, no privacy labels, no guarantee-breaking surprises when the camera moves to a new location.

The technical pipeline is deliberately asymmetric. The team sampled 128 optical flow fields at a modest 112-by-112 resolution, betting that short-horizon displacement benefits more from temporal density than spatial detail. Motion History Images, by contrast, already summarize motion over a window of roughly 25 frames, so only 32 snapshots were stored, but at a higher 224-by-224 resolution to preserve coarse body shape. These two streams feed a two-stream I3D network with late fusion, and the resulting video embedding is aligned to a CLIP-derived text space — a frozen text encoder provides semantic structure for 400 Kinetics action classes, augmented with large-language-model-generated descriptions, while a small residual adapter nudges the text features toward the motion domain without destroying the label geometry that makes zero-shot transfer possible.

The results reveal a clear privacy–utility trade-off. In zero-shot transfer, where models trained on Kinetics-400 are evaluated directly on unseen datasets, the motion-only I3D model reached roughly 31 percent top-1 accuracy on HMDB-51 and 52 percent on UCF-101. That trails CLIP-based methods like ViFi-CLIP and TC-CLIP, which benefit from massive appearance–language pretraining, but it beats older non-CLIP baselines such as R(2+1)D-18 trained on the same data — remarkable, given the model never sees a single RGB frame. With just 16 labeled examples per class, the same model climbed to 52 percent on HMDB-51 and nearly 84 percent on UCF-101, outperforming an RGB R(2+1)D baseline fine-tuned under an identical protocol.

The most striking numbers came from domain adaptation. On the TP-HMDB↔TP-UCF benchmark, a 12-class cross-dataset task with privacy annotations, the motion models trained with a DANN-style domain-adversarial objective exceeded every prior privacy-preserving method in recognition accuracy. The two-stream optical-flow-plus-MHI configuration hit 93.4 percent top-1 accuracy in one transfer direction and 83.9 percent in the other, surpassing dedicated learned-privacy approaches like VITA, SPAct, and GenPriv. Optical flow alone achieved the lowest sensitive-attribute F1 scores in both directions — and, crucially, it did so without any learned privacy transformation or privacy-label supervision.

Privacy was measured carefully, not just with raw scores. Because the benchmark suffers from severe attribute imbalance and strong correlation between action class and sensitive attributes, the researchers compared attribute predictability against two baselines: an always-majority predictor and an action-only predictor. Under both references, the optical-flow model showed the strongest reduction in attribute leakage. Its Δ Action F1 was actually negative in both transfer directions, meaning the motion representation exposed less attribute information than the action label alone would imply. RGB, by contrast, carried substantial extra sensitive information beyond what the action category already reveals.

On PA-HMDB51, a dataset with 515 human privacy annotations covering skin color, face visibility, gender, nudity, and relationship identifiability, optical flow proved the strongest motion representation for privacy preservation. Skin-color prediction approached chance level, and face and gender prediction remained well below the RGB baseline. The authors caution that Motion History Images retain coarse silhouettes — when a subject is close to the camera and moving, broad body shape or a rough gender distinction may still be inferable — so the choice between the two representations depends on the deployment: MHI when accuracy matters most, pure optical flow when minimizing leakage is paramount.

Efficiency is another selling point. The two-stream motion model requires roughly 4.9 times fewer parameters and 6.8 times less GPU memory than TC-CLIP, a state-of-the-art RGB transformer, with GPU inference about ten times faster. A lightweight X3D-XS variant cuts computation by a factor of 42 and runs at roughly 300 milliseconds per video on a single CPU thread — slow for real-time use, but feasible for multi-camera surveillance deployments where throughput and memory per stream determine whether inference is practical on shared hardware.

The framework is also sensor-agnostic. Because optical flow and MHI are driven by temporal change rather than semantic appearance, the same pipeline could be applied to depth cameras, infrared imaging, or even event cameras, which output asynchronous brightness-change events rather than frames. Accumulating events over short temporal windows would yield pseudo-frames suitable for MHI updates and approximate flow estimation, potentially reducing identity leakage even further than RGB-derived motion.

The authors are candid about limitations. Motion-only models cannot match CLIP-based methods that benefit from large-scale appearance–language pretraining, and the gap likely reflects pretraining scale rather than a fundamental weakness of motion as a signal. Text supervision can also mislead: actions like playing guitar and playing sitar share nearly identical motion patterns, so language alignment offers little help in distinguishing them. Still, the study demonstrates that learning-free motion representations offer a practical, computationally efficient privacy–utility trade-off for cross-domain action recognition — no learned privacy filter, no privacy-label supervision, just deterministic motion extraction that keeps the action while leaving the identity behind.

Subject of Research: Computer Science

Article Title: Motion representations for privacy-aware cross-domain action recognition

Article References: Benschop, P., van Gemert, J., Mense, J. P., & Dauwels, J. (2026). Motion representations for privacy-aware cross-domain action recognition. Frontiers in Imaging, 5, Article 1846329. https://doi.org/10.3389/fimag.2026.1846329

Image Credits: AI Generated

DOI: 10.3389/fimag.2026.1846329

Keywords: artificial intelligence, computer vision, privacy, action recognition, machine learning, optical flow, surveillance, deep learning, Motion, representations, privacy-aware, cross-domain