Robots That Learn by Touch: How Embodied Intelligence Is Rewriting Machine Manipulation

For more than seventy years, artificial intelligence has lived mostly in the abstract. Machines could beat grandmasters at chess and generate fluent prose, yet they struggled to pick up a wine glass without crushing it. A sweeping new review published in the journal Vicinagearth argues that this gap between digital brilliance and physical clumsiness is now closing, and that the key is a concept researchers call embodied intelligence: the idea that genuine understanding emerges only when a mind is wired directly into a body that senses and acts in the real world. The paper, led by Honghao Song and Zhe Sun of Northwestern Polytechnical University and colleagues, offers the first systematic map of how embodied intelligence is transforming robotic manipulation, the ability of machines to grasp, twist, fold, and assemble objects in unstructured environments.

The intellectual roots of the field trace back to Alan Turing, who in 1950 suggested that intelligence is not merely abstract computation but something demonstrated through dynamic interaction between a body and its environment. The review frames that insight as a theoretical foundation: a physical carrier is a necessary prerequisite for intelligence to operate in the physical world. In practice, the authors define embodied manipulation as a closed-loop process that takes embodied cognition as its engine and a physical robot as its carrier. Unlike traditional robotics, where perception and execution are decoupled modules stitched together, an embodied agent must simultaneously interpret human instructions, fuse low-level sensory data from multimodal sensors, and generate strategies adapted to its own mechanical constraints, correcting itself in real time through feedback.

Mathematically, the authors formalize this as a partially observable Markov decision process, a framework that acknowledges a sobering truth: a real robot almost never knows the true state of the world. Instead, it must maintain a probability distribution over possible states, updated with every observation and action. Where classical robot learning relied on tidy, low-dimensional state vectors, embodied manipulation confronts megapixel images, open-vocabulary language commands, tactile readings, and force signals all at once, forming a vast composite state space. Policies are therefore parameterized as deep neural networks trained to map this torrent of perception directly onto motor commands, maximizing expected cumulative reward over time.

What would an ideal embodied manipulator look like? The review identifies six hallmarks. It must achieve consistent multimodal perception, aligning vision, language, and haptics in a unified semantic space. It needs comprehensive multimodal understanding, the way large language models absorb web-scale knowledge. It must generalize across tasks and adapt zero-shot to unseen objects and scenes. It requires spatial intelligence, reasoning in three dimensions about geometry and the temporal consequences of its own actions. It needs basic physical commonsense, an intuitive grasp that objects fall, slide, and deform. And ultimately it must evolve, self-planning and self-correcting as environments shift.

Two competing technical philosophies currently dominate the field. The data-driven route treats manipulation as imitation at scale: collect enormous datasets of expert demonstrations, then train end-to-end vision-language-action models, or VLAs, that map what the robot sees and hears directly into what it does. Landmark systems such as RT-1, RT-2, PaLM-E, and the open-source OpenVLA exemplify this approach, and datasets like X-Embodiment, which aggregates a million-scale corpus spanning 22 robot morphologies and 527 skills, provide the fuel. Diffusion models, borrowed from image generation, have proven surprisingly effective here: the Diffusion Policy framework refines random noise into smooth, continuous action sequences, and successors like RDT-1B extend the idea to bimanual, cross-dataset learning.

The model-driven camp counters that data alone cannot carry robots through the open world. Researchers in this tradition build world models, internal simulations that let a robot imagine the consequences of its actions before committing to them. DayDreamer demonstrated that real robots can learn manipulation skills entirely through such imagined rollouts, updating their world model from live experience. Others inject the vast knowledge of multimodal large language models into the control loop: SayCan uses a language model to score which atomic skills actually serve a spoken instruction, while systems like ReKep generate spatial constraint functions from keypoints to solve manipulation trajectories without any task-specific training. Reinforcement learning adds a third pillar, fine-tuning imitation-trained policies through real-world trial and error, with frameworks like ConRFT balancing sample efficiency against safe execution.

Behind both paradigms lies a rapidly maturing infrastructure. Low-cost teleoperation rigs such as ALOHA and the handheld UMI gripper have slashed the price of collecting dexterous demonstration data, while Stanford’s HumanPlus tracks whole-body human motion to teach humanoids by shadowing. On the simulation side, GPU-accelerated engines like Isaac Gym and MuJoCo allow thousands of training environments to run in parallel, and newer frameworks such as Genesis and ManiSkill3 push toward photorealistic, four-dimensional worlds where policies can be trained before ever touching hardware. Generative techniques are now synthesizing data outright: RoboGen proposes its own skills and builds simulation scenes to practice them, while DemoGen mathematically replans existing trajectories to multiply datasets without a single new demonstration.

Yet the review is candid about how far robots remain from human dexterity. Vision-based policies depend on dense camera arrays that turn real workplaces into idealized laboratories, and a single modality can be catastrophically fooled; a robot wiping a table cannot feel whether it is pressing down or merely sliding the cloth. Contact-rich tasks like turning keys or opening valves expose the weakness of position-servo control that ignores force dynamics, and force sensors remain too expensive for mass deployment. Generalization is fragile, with minor lighting changes able to collapse performance, and full autonomy remains elusive. There is also a computational squeeze: the large models that perform best are precisely the ones too heavy for the edge processors a robot can physically carry, motivating compact architectures like SmolVLA, which achieves tenfold faster inference, and 1-bit compressed models like BitVLA.

The authors close with a plea that models and data should be treated as symbiotic rather than rival paradigms: data supplies the empirical knowledge that covers the world’s unpredictability, while models supply the interpretable framework that makes that knowledge usable. They also flag safety and ethics as unsolved frontiers, from concealed sim-to-real risks and adversarial attacks on embodied decision-makers to questions of job displacement and liability when autonomous machines cause harm. If the field’s trajectory holds, the milestone that matters may not be another benchmark score but the first robot that folds laundry in a stranger’s home as confidently as in its training lab, a quiet proof that intelligence, as Turing suspected, was always something a body does.

Subject of Research: Embodied intelligence for robotic manipulation, covering data-driven and model-driven approaches, their supporting infrastructure, and open challenges

Article Title: Embodied intelligence for robot manipulation: development and challenges

Article References: Song, H., Wang, L., Qiao, X., Chen, Y., Sun, D., & Sun, Z. (2025). Embodied intelligence for robot manipulation: development and challenges. Vicinagearth, 2(1), Article 8. https://doi.org/10.1007/s44336-025-00020-1

Image Credits: AI Generated

DOI: 10.1007/s44336-025-00020-1

Keywords: embodied intelligence, robot manipulation, vision-language-action models, world models, reinforcement learning, imitation learning, diffusion policy, multimodal perception, sim-to-real transfer, artificial general intelligence, humanoid robots, robot datasets