DDOI: A Decomposed Approach to Discovering Object Interaction Skills

Robots are increasingly expected to manipulate objects in the messy, unpredictable world beyond the laboratory, yet most reinforcement learning systems still rely on hand-crafted rewards that spell out exactly what the robot should do. A new study tackles a harder and more fundamental question: can a robot discover useful object-manipulation behaviors entirely on its own, without any task-specific supervision at all? Researchers at Korea University have introduced DDOI, short for Decomposed Skill Discovery for Object Interaction, a framework that teaches robot agents to generate a rich repertoire of reusable behaviors for pushing and moving objects, and then recombines those behaviors to reach goals it has never seen during training.

The work, published as an open-access article in Neural Processing Letters by Shinsuk Park, Hyeyun Kim and Junyong Seo of the Department of Mechanical Engineering at Korea University, addresses a persistent weakness in unsupervised skill discovery. These methods, which typically build on mutual information objectives, allow an agent to learn a library of latent skills without external rewards. The idea is elegant: learn behaviors that are as diverse and as predictable as possible, so that a high-level controller can later select from this library to solve downstream tasks. In practice, however, existing mutual information-based approaches tend to discover skills that are static and locally confined. The robot wiggles, rolls, or wanders within a small region of its state space, producing broad coverage of its own body configuration but almost no meaningful change in the state of objects around it.

That limitation matters enormously for object interaction. A humanoid that has learned dozens of locomotion skills but cannot move a box across the room has learned, from the perspective of manipulation, very little. The core difficulty, the authors explain, is that inducing transitions in an object’s state is inherently harder than inducing transitions in the robot’s own state. Objects are passive; they move only when the robot makes deliberate, well-timed contact, and the resulting state changes may be large in the object’s coordinates while requiring only subtle adjustments of the robot’s joints. A skill discovery objective that treats the entire coupled system uniformly will naturally favor the easy diversity of the robot’s own motion over the difficult diversity of object motion.

DDOI’s central innovation is to decompose the latent skill variable into two parts: an object skill and a robot skill. Under the mutual information-based skill discovery framework, the object skill is trained through an object skill discriminator whose objective explicitly rewards far-reaching and hard-to-achieve transitions in the object’s state. The researchers provide two variants of this constraint. The first uses a Euclidean distance, encouraging the object to travel far from where it started. The second, a controllability-aware distance, goes further by weighting transitions according to how difficult they are to achieve, pushing the agent to discover behaviors that produce object-state changes which would otherwise rarely occur. In effect, the object skill learns to value not just movement, but consequential movement — the kind of dramatic state transitions that make manipulation genuinely useful.

The robot skill, by contrast, is trained through a separate robot skill discriminator that conditions on three things simultaneously: the robot’s own state, the object’s state, and the sampled object skill. This conditioning structure is the key coupling mechanism of the framework. Because the robot skill discriminator must be able to infer the intended object skill from what the robot is doing, the robot is driven to acquire behaviors that help realize the desired object-state transitions. If the object skill demands that a box slide to the left, the robot skill learns the pushing, pulling, and repositioning motions that make that slide happen. The decomposition thus splits a seemingly intractable problem into two mutually informative halves: one component dreams up ambitious object outcomes, and the other learns to physically deliver them.

To evaluate the framework, the team built planar single-object interaction benchmarks in three MuJoCo environments: Ant, Ant-Box, and Humanoid. In Ant, a quadrupedal agent interacts with a spherical object; in Ant-Box, the same agent must deal with a box-shaped object; and in Humanoid, a bipedal agent — notoriously prone to falling — must manipulate a sphere while remaining upright. The object is initialized at the origin, and the robot begins on a circle of fixed radius facing it, with the radius chosen per environment to avoid immediate collisions. The primary evaluation metric is object-state coverage: how much of the object’s reachable state space the learned skills actually explore. Downstream goal-reaching success rates serve as an additional metric of practical usefulness.

The training pipeline incorporates several pragmatic design choices that prove essential for stable learning. Because both the Ant and the Humanoid can collapse to the ground, the researchers add an explicit uprightness bonus, granting a discrete reward when the dot product between the torso’s local z-axis and the world z-axis exceeds 0.8. The final skill-learning reward is a weighted sum of this stability term and the intrinsic mutual information-based reward. For the Humanoid, an early-termination rule is also enforced: any episode in which the torso height drops below 0.8 meters is terminated, preventing the agent from earning skill-discovery credit for contacting the object during a fall or striking it while lying prone — behaviors that would score well on naive coverage metrics but represent exactly the kind of spurious solutions unsupervised learning is prone to.

Once the low-level skill repertoire is trained, it is frozen, and a goal-conditioned downstream policy is trained on top of it using Soft Actor-Critic. This high-level controller observes the environment state and a goal, selects a latent skill — itself a pair consisting of an object skill and a robot skill — and holds that skill for a fixed number of steps while the frozen skill policy executes actions. Goals are sampled as two-dimensional positions around the object’s initial location, and the downstream policy receives a sparse, discrete reward only when the object’s distance to the goal falls below a threshold. The Ant-Box variant adds a particularly demanding twist: a narrow passage sits between the object’s start and the goal region. Although the reward depends only on the object’s position, successfully transporting the box through the passage typically requires reorienting it mid-transit, making the task effectively orientation-sensitive despite its position-based reward. This is a clever probe of whether the discovered skills encode rich object manipulation rather than blind pushing.

The results show DDOI variants achieving more consistent object-state coverage and stronger downstream performance than the compared baselines across all three environments. The authors supplement these headline metrics with two effort-efficiency measures: effort per object state coverage and effort per object displacement, aggregated over 200 random-skill rollouts per method and task. Lower values indicate more efficient manipulation. In Ant, the controllability-aware DDOI variant achieves by far the lowest normalized effort, with particularly small standard deviations, indicating both low and stable control expenditure relative to the object motion achieved. In Ant-Box, the Euclidean variant yields the lowest normalized effort while the controllability-aware variant remains competitive and outperforms the non-DDOI baselines in effort per displacement. In Humanoid, the Euclidean variant again leads, with the controllability-aware version close behind, while prior-based baselines are substantially worse. Notably, the Ant-Box downstream results reveal that DDOI agents can reorient the box while steering it through the narrow passage — a manipulation behavior that was never explicitly rewarded.

The significance of this work extends beyond its benchmark numbers. Decomposing the skill variable along the object-robot divide offers a general recipe for making mutual information-based skill discovery relevant to manipulation, a domain where the interesting state changes happen in the world, not in the agent. The controllability-aware distance in particular points toward a broader principle: that intrinsic motivation for robots should be calibrated by how hard a transition is to achieve, not merely by how far it goes. A skill that nudges an object one meter with a lucky shove is less valuable than one that reliably produces large, difficult object-state changes on demand.

The research was supported by the Korea Institute for Advancement of Technology, funded by the Korean Ministry of Trade, Industry and Energy, under grant number P0028050. Hyeyun Kim and Junyong Seo contributed equally and share first authorship, with Shinsuk Park serving as corresponding author. The authors report no conflicts of interest, and the source code has been released publicly, allowing other laboratories to build on the framework immediately. As robots move from curated environments into homes, warehouses, and disaster zones, approaches like DDOI — which learn what to do with objects without ever being told — may prove essential to the next generation of autonomous manipulation.