AI Video Generation Gets Human: A New Survey Maps the Field’s Biggest Hurdles

Video generation has quietly become one of the most competitive arenas in artificial intelligence, with text-to-video systems producing clips that are increasingly difficult to distinguish from real footage. But a comprehensive new survey argues that the field’s next great challenge is not visual spectacle at all. Instead, it is people: how machines depict human bodies in motion, preserve a person’s identity across frames, and respect the physical constraints of muscles, joints, and gravity. The review, published open access in Artificial Intelligence Review by Alaa Abdullah Albaghdadi and Ahmad R. Naghsh-Nilchi of the University of Isfahan, offers the first systematic treatment of what the authors call human-centric motion modeling within multimodal video diffusion models, and its conclusions are both encouraging and sobering for anyone hoping to build the next generation of video synthesis tools.

The survey’s starting point is architectural. Multimodal video diffusion models generate video through an iterative denoising process, gradually refining random noise into coherent moving imagery. What distinguishes the newest systems is the breadth of signals they can condition on: text prompts describe scenes and actions, reference images anchor visual style and character appearance, audio tracks can drive movement or synchronization, and pose sequences specify the exact positions of a body over time. The authors weave these disparate conditioning mechanisms into a unified architectural framework, examining how spatial and temporal representations interact inside modern diffusion models. This kind of unified view matters because, as the survey makes clear, the components do not behave independently. A model that handles pose superbly but fumbles temporal continuity will produce human figures that flicker, morph, or lose their identity mid-motion, and no amount of conditioning on a single modality can repair that.

Among the most striking technical findings is a fundamental trade-off between computational efficiency and generation quality. Diffusion models are notoriously expensive at inference time, requiring many denoising steps across hundreds of thousands of pixels. The survey documents specialized techniques for cutting this cost, including temporal block pruning, a strategy that removes redundant computational blocks along the temporal dimension of a model. Under specific baselines, the authors report, such pruning has achieved computational savings of up to 523 times with minimal degradation of quality. The authors are careful to attach caveats to this figure, noting that comparisons across different architectures and baselines are not always straightforward. Even with that caution, the magnitude of the savings suggests that the era of treating diffusion inference as a brute-force problem may be ending, opening the door to video generation that runs on far more modest hardware.

Yet efficiency is only half the story. The survey’s deepest technical analysis concerns what happens when the subject of a generated video is a human being, and here the news is less rosy. Three persistent gaps haunt the field: temporal consistency, multimodal alignment, and human-centric motion generation. Temporal consistency refers to the stability of content across frames, so that faces, clothing, and backgrounds do not drift or shift identity between the first and last frames of a clip. Multimodal alignment concerns whether the text, audio, pose, and visual signals actually reinforce one another rather than pulling the model in conflicting directions. In practice, the authors find, current approaches struggle with seamless integration across modalities, and the seams show up most visibly in videos of people, where viewers are exquisitely sensitive to any inconsistency.

One of the survey’s most intriguing conceptual contributions is its identification of what the authors call physics-perception asymmetry. When the physical constraints imposed on generated human motion are too rigid, videos fall into an uncanny valley: the figures move with a stiffness that reads as artificial, precisely because real human motion is slightly noisy, idiosyncratic, and imprecise. Too little constraint, and the results are physically implausible, with limbs bending in impossible ways. The lesson is that perceptual plausibility and physical exactness are not the same goal, and systems must be tuned to respect physiology without overconstraining the natural variability that makes human movement look alive. This finding has immediate practical implications for developers of pose-driven animation tools, avatars, and virtual presenters, suggesting that strict biomechanical enforcement can be actively counterproductive.

A related tension concerns identity preservation. A user generating a video of a specific person, whether a historical figure, a performer, or themselves, expects the face, body shape, and characteristic gestures to remain consistent throughout the clip. The survey finds that this expectation collides with the dynamics of the generation process itself: the more the model transforms a person across poses and actions to convey motion convincingly, the more it risks eroding the very features that make the person recognizable. Identity and motion, in other words, are in conflict inside current architectures, and resolving that conflict is one of the field’s most urgent open problems. For applications ranging from virtual try-on in e-commerce to accessible avatar communication, this single tension may determine whether the technology becomes trustworthy.

To organize these insights, the authors introduce a conceptual reference framework called MIME-Vid, which stands for Multi-modal Integration with Motion Enhancement for Video Generation. MIME-Vid is not a released model, and the survey is explicit that its empirical validation is deferred to follow-up work. Instead, it operationalizes three unifying principles that emerge from the analysis: reference flexibility, which governs how strictly a model must adhere to conditioning inputs; physics-perception asymmetry, the principle discussed above; and hierarchical disentanglement, the idea that content, motion, identity, and style should be represented in separable layers so that each can be controlled independently. The framework’s value at this stage is as a map: it tells researchers where the leverage points are in an otherwise sprawling landscape of architectures, training strategies, and conditioning tricks.

Equally important is the survey’s contribution to how the field measures itself. The authors argue that existing evaluation practices are inadequate for human-centric generation, where success is not captured fully by standard metrics of visual fidelity or prompt adherence. A video can score well on automated benchmarks while a subject’s face subtly changes shape or a walk cycle looks mechanically wrong to any human observer. The survey proposes novel evaluation paradigms aimed at judging physiological plausibility and identity consistency directly, and it charts a set of future research directions for advancing multimodal video generation more broadly. These include better mechanisms for fusing audio and visual streams, more principled treatments of temporal structure, and architectures that respect physical constraints without sacrificing expressiveness.

The stakes extend well beyond the laboratory. Human-centric video synthesis underlies applications in filmmaking, education, healthcare communication, sports analysis, and virtual collaboration, and the survey’s authors frame the entire enterprise in explicitly human-centered terms: the goal is efficient and controllable generation that serves people rather than simply impressing them. But the same technologies raise well-known concerns around deepfakes, consent, and identity misuse, and the identity-preservation problem the survey identifies cuts both ways: the harder it becomes to keep a generated person consistent, the harder it also becomes to weaponize their likeness convincingly, yet every technical improvement moves the field closer to that capability. The survey itself does not venture deeply into policy, but its technical findings will inform any serious debate about how this technology should be governed.

For now, the survey stands as the most complete map of a field in rapid flux, published under a Creative Commons license so that any researcher can consult it freely. Its unifying framework gives newcomers a way into a literature scattered across computer vision, graphics, and machine learning, while its candid accounting of failures, the uncanny valleys, the identity drift, the alignment seams, offers veterans a checklist of what still needs fixing. The reported five-hundred-fold efficiency gains suggest the computational barriers will fall sooner rather than later. Whether the human-perceptual barriers fall as fast is the open question that will decide whether AI-generated video becomes a trusted medium or remains a mesmerizing novelty.

Subject of Research: Multimodal video diffusion models for human-centered video generation

Article Title: Towards human-centered and efficient video synthesis: a survey of multimodal diffusion models

Article References: Albaghdadi, A. A., & Naghsh-Nilchi, A. R. (2026). Towards human-centered and efficient video synthesis: a survey of multimodal diffusion models. Artificial Intelligence Review. https://doi.org/10.1007/s10462-026-11699-z

Image Credits: AI Generated

DOI: 10.1007/s10462-026-11699-z

Keywords: diffusion models, video generation, multimodal synthesis, human-centric AI, temporal consistency, identity preservation, motion synthesis, computational efficiency, temporal block pruning, generative models, controllability, evaluation metrics