Robotics researchers have introduced InterReal, a unified physics-based imitation learning framework designed to empower humanoid robots with autonomous human-object interaction capabilities. While recent advances in deep reinforcement learning have enabled bipedal humanoids to master dynamic whole-body locomotion such as walking, jumping, and dancing, extending these autonomous behaviors to physical manipulation has remained an elusive target. InterReal bridges this critical divide by coupling a contact-preserving motion augmentation pipeline with an automatic meta-learning reward scheduler. In benchmark evaluations conducted within physics simulation environments, the system demonstrated unprecedented precision, reaching a task success rate of 96.41 percent on complex box-picking routines and 87.45 percent on box-pushing challenges. Furthermore, the team validated the framework through qualitative physical demonstrations using the Unitree G1 humanoid platform, showing that the learned interaction policies maintain operational stability and smooth motor execution even when confronted with real-time positional disturbances.
The development of closed-loop manipulation policies for humanoids has historically encountered steep theoretical and practical obstacles. Early foundations established within the computer graphics and physical animation domains, such as the InterMimic and CooHOI systems, demonstrated that simulated virtual characters could replicate rich human-object motions through deep reinforcement learning. However, these animation methodologies operated within idealized physical environments characterized by simplified friction mechanics, negligible mass parameters, and absent contact force dynamics, rendering them unsuited for real-world robotic hardware. Conversely, contemporary robotics has largely circumvented autonomous control by deploying teleoperation schemes. While upper-body and whole-body teleoperation frameworks, including CLONE, TWIST, and OpenWBT, can leverage large-scale motion capture repositories to mirror human movements promptly, they fundamentally constrain autonomous agency and exhibit notable deficiencies in physical stability, trajectory precision, and fine-grained contact modeling during hand-object engagements.
Deploying interactive whole-body controllers on physical hardware introduces severe operational vulnerabilities that do not manifest during non-interactive locomotion. Empirical investigations reveal that sensor noise and trajectory tracking discrepancies, especially relative spatial displacements between the robot end-effectors and target objects, regularly push learned policies outside their training distribution, precipitating catastrophic policy collapse. Although conventional reinforcement learning utilizes domain randomization and motion blending to bridge the simulation-to-reality reality gap, systematic techniques addressing interactive generalization have remained scarce. Compounding this difficulty is the notorious reward engineering bottleneck. Humanoid control architectures incorporate dozens of conflicting objective terms, ranging from joint posture fidelity and contact maintenance to torque penalties and center-of-mass preservation. Manually hand-tuning static weighting factors across heterogeneous reward objectives is fundamentally suboptimal, failing to account for the reality that functional control priorities shift dynamically across sequential phases of physical interaction.
To overcome these challenges, the authors structured InterReal around an end-to-end processing pipeline spanning five sequential components: motion preprocessing, motion augmentation, multi-motion algorithmic training, simulation verification in MuJoCo, and physical deployment on hardware. The workflow begins with reference motion capture sequences formatted in the standard SMPL kinematic structure. The researchers retarget these source trajectories to match the physical dimensions and structural kinematic parameters of the Unitree G1 robot while explicitly preserving hand-object contact geometry. Because raw captured motions and standard kinematic retargeting outputs regularly suffer from measurement noise, physical collisions, and kinematic boundary violations, the trajectories undergo rigorous physical verification. Operating within the IsaacGym simulation engine, the retargeted motions are filtered using dynamic physics constraints to ensure kinematic feasibility, producing verified anchor trajectories that serve as the baseline foundation for robust policy synthesis.
The second stage introduces a specialized motion augmentation scheme designed to inoculate policies against spatial perturbations during manipulation. Rather than attempting brute-force exploration, the method generates diverse training trajectories by applying synthetic horizontal offsets along the world-frame coordinate axes of the target object. To maintain physical plausibility, the spatial displacement applied to the object center of mass is matched across the robot wrist end-effectors, guaranteeing identical hand-object contact configurations. The framework transforms these perturbed wrist positions from the global coordinate system through the torso frame into the robot pelvis reference coordinate frame. Within this standardized pelvic reference system, the Ipopt nonlinear optimization solver calculates new arm joint angle trajectories via inverse kinematics based on initial arm configurations. By uniformly sampling horizontal offsets within predefined limits, the pipeline generates rich multi-motion trajectory sets that train the policy to handle continuous spatial discrepancies without breaking contact constraints.
At the core of the framework lies an inner-loop reinforcement learning process formulated as a Markov Decision Process defined by state spaces, degree-of-freedom actions, state transition probabilities, discounted returns, and parameterized reward functions. The state vector integrates humanoid proprioception features, object spatial features, interaction features, and motion phase indicators. Proprioceptive inputs comprise twenty-three joint positions and velocities, three-dimensional root angular velocities, gravity projection vectors, and previous action commands, mirroring the robot twenty-three controllable degrees of freedom. Object states capture three-dimensional position, linear velocity, and spatial orientation. Crucially, the interaction state incorporates link contact boolean flags alongside an interaction graph, which quantifies distance vectors between selected humanoid structural links and critical object surface points. The inner loop trains the interaction policy through Proximal Policy Optimization, maximizing cumulative discounted returns calculated across an extensive weighted sum of tracking rewards, balance terms, and actuation penalties.
To safeguard real-world viability, the inner loop adopts an asymmetric actor-critic architecture designed to eliminate reliance on privileged sensing. In laboratory simulations, the critic network evaluates full, perfect state observations, including root dynamics, high-fidelity object velocities, three-dimensional rotations, and complex interaction-graph matrices. However, physical real-world sensors cannot reliably provide low-latency velocity or rotation estimations for manipulated objects, and measuring full contact interaction graphs on hardware is notoriously fragile. The actor policy network is therefore restricted to an imperfect state observation space that entirely strips away the interaction graph, object velocities, and object rotation metrics. Instead, the actor relies solely on onboard humanoid proprioception and three-dimensional object center-of-mass positions extracted in real time from vision sensors, ensuring the resulting control policy remains robust against the sensory dropouts and noisy feedback that typify physical robotic environments.
The outer loop addresses the reward shaping bottleneck through an automated meta-learning mechanism optimized via Soft Actor-Critic. The system groups inner-loop Proximal Policy Optimization epochs into discrete latent subtasks spanning fixed epoch intervals. Throughout each subtask, the structural parameters of the meta-policy remain frozen, yet the meta-policy actively queries the instantaneous robot state and normalized motion phase at every simulation control step. This design enables the meta-policy to output continuously varying, phase-dependent reward weights that dynamically align with shifting operational demands, assigning higher priority to balance stabilization during initial reaching and reallocating weight toward contact preservation during lifting. The resulting adaptive weights scale initial baseline parameters through a bounded time-decaying clipping function and an internal scaling coefficient, ensuring exploratory stability across successive training cycles without triggering divergence in the low-level policy.
Guiding this meta-policy requires an overarching learning signal that circumvents the tedious heuristics of manual reward design. The investigators leveraged the core insight that all motion-imitation tracking rewards fundamentally share an identical objective: minimizing deviations from reference demonstration trajectories. Consequently, the outer-loop meta-reward is formulated directly as the reduction in aggregate tracking error evaluated over each training interval. This metric sums the squared differences between actual and reference values for robot joint positions, object spatial positions, and humanoid link coordinates, calculating the rate of error reduction over the training horizon. When an allocated reward distribution successfully accelerates tracking error reduction, the meta-learner receives a positive reward signal. By optimizing this objective with Soft Actor-Critic augmented by action entropy regularization, the outer loop systematically converges toward reward weighting schedules that maximize physical tracking fidelity.
The authors evaluated InterReal against two baseline paradigms adapted for rigorous comparison: ASAP*, which incorporates object tracking into a proprioceptive locomotion baseline but omits interaction graphs and relies on fixed reward weights, and InterMimic*, which retains interaction graph features while employing static heuristic reward weighting. Across fifty thousand training epochs on both the Box-picking and Omomo Box-pushing benchmarks, InterReal achieved superior tracking precision across nearly all kinematic criteria. In evaluations spanning twenty independent runs, InterReal recorded the lowest tracking discrepancies in mean per-joint position errors, upper link positions, key functional link alignments, and object trajectories. When subjected to rigorous simulation evaluation in MuJoCo comprising twenty independent batches of one hundred parallel environments, InterReal demonstrated marked superiority, achieving success rates of 96.41 percent on box picking and 87.45 percent on box pushing, substantially outperforming the fixed-weight baselines.
Ablation experiments underscored the critical necessity of both the interaction graph and the automatic reward learning mechanism. Disabling the meta-policy scaling factor entirely, which forces the system to rely on fixed heuristic reward weights, severely degraded tracking performance and confirmed that dynamic reward adaptation is vital for mastering multi-phase interactions. The complete framework was subsequently verified through physical deployments on the Unitree G1 humanoid. Equipped with a head-mounted three-dimensional LiDAR and a wide-angle RGB camera, the robot utilized a pretrained FoundationPose vision model to estimate the three-dimensional coordinates of a 1.5-kilogram plastic box measuring 38 centimeters on each side. The robot reliably adjusted its arm kinematics to varying initial object placements, executing stable, coordinated manipulation routines. While the authors acknowledged remaining challenges regarding visual latency and the need to generalize beyond box geometries, InterReal establishes a robust foundation for autonomous humanoid interaction in the physical world.
Subject of Research: Physics-based imitation learning framework for humanoid robot human-object interaction
Article Title: InterReal: a unified physics-based imitation framework for learning human-object interaction skills
Article References: Liang, D., Lin, Y., Liu, X., Shi, J., Liu, Y., & Bai, C. (2026). InterReal: a unified physics-based imitation framework for learning human-object interaction skills. Vicinagearth, 3(1), Article 18. https://doi.org/10.1007/s44336-026-00046-z
Image Credits: AI Generated
DOI: 10.1007/s44336-026-00046-z
Keywords: humanoid robots, reinforcement learning, physics-based imitation, motion retargeting, inverse kinematics, Unitree G1, MuJoCo, IsaacGym, Proximal Policy Optimization, Soft Actor-Critic, meta-learning, FoundationPose, human-object interaction, automatic reward shaping, asymmetric actor-critic, motion augmentation
Cite Scienmag News
Violet Maxwell. (November 22, 2026). InterReal Framework Enables Humanoid Robots to Master Real-World Object Manipulation Tasks. Scienmag. https://scienmag.com/interreal-framework-enables-humanoid-robots-to-master-real-world-object-manipulation-tasks/
Violet Maxwell. "InterReal Framework Enables Humanoid Robots to Master Real-World Object Manipulation Tasks." Scienmag, 22 November 2026, https://scienmag.com/interreal-framework-enables-humanoid-robots-to-master-real-world-object-manipulation-tasks/. Accessed 22 November 2026.
Violet Maxwell. "InterReal Framework Enables Humanoid Robots to Master Real-World Object Manipulation Tasks." Scienmag. November 22, 2026. https://scienmag.com/interreal-framework-enables-humanoid-robots-to-master-real-world-object-manipulation-tasks/








