To appreciate why the contribution matters, it helps to understand how modern optimal planners work. Algorithms such as Rapidly-exploring Random Tree star, or RRT*, introduced by Karaman and Frazzoli in 2011, guarantee asymptotic optimality, meaning that given enough samples the solution converges to the best possible path. The catch lies in the rate of that convergence. Early samples tend to produce long, winding, inefficient routes, and the planner must then sift through an astronomically large configuration space to find better alternatives. Informed sampling techniques addressed this by restricting new samples to a subset of the state space that could conceivably contain an improvement. The most famous example, Informed RRT*, draws samples from an ellipsoidal region whose size shrinks as the current best path cost decreases. Any point outside the ellipse, by the triangle inequality, cannot lie on a path shorter than the current solution, so sampling there is wasted effort.

The Toronto-led team, however, identified a subtle weakness in this widely adopted strategy. The informed subset is only as tight as the current solution path allows. When that path contains redundant detours, unnecessary loops, or tortuous segments, the ellipsoidal informed set remains bloated, and the planner continues to scatter samples across regions that are almost certainly useless. In other words, the quality of the guidance depends on the quality of the incumbent solution, and early solutions are typically poor. The researchers’ prior work proposed the greedy informed set as a remedy: instead of defining the sampling region by the total cost of the current path, it uses the maximum heuristic cost along that path, producing a substantially smaller region that focuses samples where improvements are genuinely plausible.

In the new article, the authors go well beyond the initial proposal by formally characterizing how the greedy informed set behaves within RRT-like planners. They analyze what happens to exploration when sampling is biased so aggressively, a crucial theoretical question because over-constraining the search risks excluding regions that a future, better solution might need to traverse. The analysis demonstrates that greedy sampling can be integrated without sacrificing the properties that make RRT attractive: the planner still finds an initial solution rapidly and still converges asymptotically to optimal paths. This combination of aggressive focus and retained completeness guarantees is the theoretical heart of the paper, and it distinguishes the approach from heuristic shortcuts that speed things up at the cost of optimality assurances.

Building on this foundation, the team presents Greedy RRT*, abbreviated G-RRT*, a bi-directional anytime variant of RRT* that exploits the greedy informed set in practice. The bi-directional design means the planner grows search trees simultaneously from the start and goal configurations, meeting in the middle, a strategy long known to accelerate the discovery of initial solutions. The anytime property means the planner can be stopped at any moment and return the best path found so far, then continue improving that path if given more time. By combining these established design principles with greedy informed sampling, G-RRT* concentrates its computational budget on the most promising pockets of the search space, tightening the sampling region aggressively as soon as any solution exists.

The empirical evaluation is notably broad. The researchers tested G-RRT* on abstract planning benchmarks, which isolate algorithmic behavior from the idiosyncrasies of any particular robot, and on manipulation tasks drawn from the MotionBenchMaker dataset, a tool developed by Chamzas and colleagues to generate standardized, reproducible motion planning benchmarks for robotic manipulation. They also tackled a dual-arm Barrett WAM problem, a genuinely high-dimensional scenario in which two articulated arms must coordinate their movements without colliding with each other or the environment. Across these domains, G-RRT* found initial solutions quickly and converged to optimal paths faster than state-of-the-art sampling-based planners, including informed variants that rely on the conventional ellipsoidal subset.

The practical implications extend across robotics. Manipulation planning for industrial arms, warehouse robots, surgical systems, and space robotics all involve configuration spaces with a dozen or more degrees of freedom, exactly the regime where sampling-based planners dominate and where convergence speed determines whether a robot can plan in real time or must pause awkwardly while computing. Faster convergence to high-quality paths translates directly into robots that respond more quickly, consume less energy executing shorter routes, and can replan on the fly when the environment changes. The authors note that implementations of G-RRT* are publicly available both within the Open Motion Planning Library, the field’s standard open-source planning framework maintained through OMPL, and in VAMP, a vector-accelerated motion planning platform, lowering the barrier for other research groups to adopt and extend the technique.

The study also situates itself within a rich lineage of sampling heuristics. Researchers have long experimented with biasing samples toward free space, toward narrow passages, or along potential fields, and variants such as RRT-Smart, Quick-RRT, and F-RRT* have each proposed tricks for accelerating convergence, often by shortcutting or smoothing the incumbent path. Batch-oriented planners such as Batch Informed Trees and bidirectional asymmetric search methods like Adaptively Informed Trees represent a parallel line of work that fuses sampling with graph-search ideas. The greedy informed set contributes a distinct and principled mechanism to this ecosystem: rather than post-processing paths or restructuring the search, it redefines the geometry of the sampling region itself, using a heuristic bound that remains valid even when the current path is far from optimal. Recent work on vectorized planning, exemplified by the VAMP system from Thomason, Kingston, and Kavraki, has shown that raw computational throughput can be pushed to remarkable heights, and greedy sampling composes naturally with such speedups because it reduces the number of wasted samples rather than the speed of each sample.

Simulations for the study were performed in an Ubuntu 20.04 Docker container on an Intel Core i7-10875H processor with 40 gigabytes of memory, with all planners implemented in C++, and path simplification was evaluated only in separate post-processing experiments so that it would not contaminate the convergence measurements. This methodological care matters in a field where benchmarking practices have historically varied widely, and the authors leveraged the Planner Developer Tools framework, designed for reproducible experiments and statistical analysis of motion planners. The paper’s theoretical proofs, which the authors note benefited from assistance with the formal arguments, establish the asymptotic guarantees that practitioners need before trusting a new sampling bias in safety-relevant applications.

What makes this work resonate beyond its immediate subfield is the way it reframes a familiar trade-off. Exploration versus exploitation is a tension that recurs throughout artificial intelligence, from reinforcement learning to combinatorial search, and motion planning offers an unusually concrete instance of it. The greedy informed set shows that, with the right heuristic characterization, a planner can lean hard toward exploitation, sampling only where improvement is provably possible, without forfeiting the exploratory completeness that guarantees eventual optimality. As robots proliferate in homes, hospitals, factories, and hazardous environments, the demand for planners that deliver near-optimal paths in milliseconds rather than seconds will only intensify. Greedy RRT* offers a mathematically grounded step toward that goal, and its availability in open-source planning libraries means the broader robotics community can put the greedy principle to work immediately.

Subject of Research: Greedy informed sampling for asymptotically optimal sampling-based robot motion planning in high-dimensional state spaces

Article Title: Greedy heuristics for sampling-based motion planning in high-dimensional state spaces

Article References: Kyaw, P. T., Le, A. V., Mohan, R. E., & Kelly, J. (2026). Greedy heuristics for sampling-based motion planning in high-dimensional state spaces. Autonomous Robots, 50(4), Article 44. https://doi.org/10.1007/s10514-026-10269-0

Image Credits: AI Generated

DOI: 10.1007/s10514-026-10269-0

Keywords: sampling-based motion planning, RRT*, greedy heuristics, informed sampling, robot motion planning, high-dimensional state spaces, asymptotic optimality, bidirectional search, robotic manipulation, MotionBenchMaker, OMPL, path optimization

The research, led by Yawen Liu and Wei Tang with colleagues Zhen Zhang and Xinrong Chen, addresses a persistent bottleneck in robotic manipulation. Conventional approaches to force sensing rely on physical instruments such as force and torque sensors or tactile arrays mounted directly on the gripper. Rigid-body models calculate grasping force through explicit mechanical equations, while soft-body models attempt to capture the nonlinear deformations of compliant structures. Although advances in materials science, including conductive hydrogels that endow soft robots with human-like sensory capabilities, have pushed the field forward, physical sensors bring inherent drawbacks: they add system complexity and cost, and their measurement range is limited, making it difficult to adapt to the diverse and dynamic demands of different grasping tasks.

Vision-based pressure estimation has emerged as an elegant alternative. By pointing cameras at the interaction between a gripper and an object, computer vision techniques can infer the magnitude and distribution of applied forces without any physical contact with the measured surface. This eliminates the need for structural modifications to the gripper and offers flexible, scalable perception. Yet most existing methods, including the current state-of-the-art system known as HiVPE, rely on single RGB images. That restriction means the network sees only surface appearance, missing explicit three-dimensional geometric deformation and high-level semantic context, both of which become crucial when dealing with severe occlusions or the complex hyperelastic behavior of soft materials.

The new framework breaks free of that single-modality constraint by weaving together four distinct streams of information. The first is the ordinary RGB image capturing the interaction between the gripper and a high-resolution pressure sensing array. The second is a binary segmentation mask that precisely delineates the gripper’s contact regions, generated by a pre-trained deep segmentation model that isolates the hand from the background. The third is a depth map produced from the RGB image by a monocular depth estimation network, encoding the geometric and spatial relationships of the contact. The fourth is perhaps the most surprising: a short textual prompt, either the word Straight or Roll, describing the gripper’s action, which is converted into a semantic embedding by the CLIP text encoder, the same contrastive language-image model that underpins many modern multimodal AI systems.

Architecturally, the system proceeds in three stages: feature extraction, feature fusion, and decoding. During extraction, a dual-branch hierarchical encoding scheme employs two parallel SE-ResNeXt50 networks. The first processes the raw RGB image to produce five feature maps at progressively abstract scales, spanning downsampling factors from 2 to 32 and channel dimensions from 64 to 2048, capturing everything from low-level textures to high-level semantics. The second encoder jointly processes the concatenated segmentation and depth inputs, yielding a parallel set of features rich in structural and geometric cues. In this way, one branch learns what the scene looks like while the other learns how it is shaped in space.

The fusion stage is where the framework’s distinctive machinery comes into play. Multi-head self-attention, with four attention heads of 64 dimensions each operating on 256-dimensional features, first refines the highest-resolution feature maps within each modality, strengthening intra-modal contextual dependencies through residual connections and feed-forward networks. A bidirectional cross-attention operator then lets the two streams talk to each other: the appearance features absorb structural context from the segmentation and depth branch, while the structural features incorporate appearance cues from the RGB branch. Finally, the 512-dimensional textual embedding is projected through a multilayer perceptron into a 2880-dimensional feature space, reshaped to align spatially with the visual features and concatenated to form a unified multimodal representation. A Feature Pyramid Network decoder then performs top-down aggregation, fusing high-level semantic cues with low-level spatial details before a segmentation head produces the final dense pressure prediction.

Training and evaluation were carried out on a publicly available benchmark containing roughly 650,000 synchronized RGB-pressure frames from two very different soft grippers, one tendon-actuated and one pneumatic. Ground-truth pressure annotations came from a high-resolution Sensel Morph pressure sensing array, spatially aligned with the camera images through calibration and homography transformation. The dataset covers four interaction types: contact, slide, close, and no-contact, the last serving as adversarial samples. The authors discretized continuous pressure values into nine logarithmically spaced categories, from zero background to a top bin of 64 kilopascals and above, and optimized the network with a pixel-wise weighted cross-entropy loss that penalizes underrepresented contact classes to counteract the imbalance between vast non-contact regions and sparse pressure-bearing areas. All results were averaged over six independent runs to verify statistical stability.

The results are striking for the tendon-actuated gripper, where the multimodal method surpassed the previous state of the art on every metric. It achieved a temporal accuracy of 96.57 percent, a contact area intersection-over-union of 77.20 percent, a volumetric IoU of 62.87 percent, and a mean absolute error of just 4.33 pascals, compared with 5.0 pascals for HiVPE and 5.3 for the earlier VPEC-Net. For the pneumatic gripper, the method attained the highest temporal accuracy and volumetric IoU, though it fell marginally short of HiVPE on contact IoU and mean absolute error, a shortfall the authors attribute to multimodal fusion occasionally introducing noise when soft surfaces deform so severely that depth and segmentation maps become blurred. Ablation experiments confirmed that each component earns its place: segmentation masks alone improved performance, adding attention strengthened contact-region predictions, depth maps reduced error further, and textual prompts delivered the final gains in spatial-pressure consistency.

The implications extend well beyond a leaderboard. Because the approach requires no embedded sensors, it could make force-aware grasping accessible to cheap, commodity robotic hardware, from laboratory manipulators to warehouse pickers handling fragile goods. The demonstration that a single descriptive word about the gripper’s action can measurably sharpen pressure predictions hints at a future where robots are instructed in natural language and perceive the physical consequences of their movements through vision alone. The authors are candid about the current limitations: the framework has so far been validated on static, offline benchmark data rather than deployed on physical robots in dynamic manipulation, where real-time sensor noise, unpredictable physical interactions, and inference latency constraints all come into play. They also note that soft grippers, with their complex nonlinear hyperelastic deformation, remain the hardest case, and they point toward more advanced depth estimation, adaptive fusion architectures that dynamically re-weight unreliable modalities, and explicit deformation modeling as the next frontiers. If those challenges are met, the line between seeing and touching may grow ever thinner, bringing robots one step closer to hands that truly understand what they hold.

Subject of Research: Vision-based multimodal pressure estimation for robotic gripper manipulators

Article Title: Vision-Based multimodal pressure estimation for robotic gripper manipulators

Article References: Liu, Y., Tang, W., Zhang, Z., & Chen, X. (2026). Vision-Based multimodal pressure estimation for robotic gripper manipulators. Results in Engineering, 32, Article 113295. https://doi.org/10.1016/j.rineng.2026.113295

Image Credits: AI Generated

DOI: 10.1016/j.rineng.2026.113295

Keywords: robotics, pressure estimation, multimodal AI, soft grippers, computer vision, deep learning, attention mechanism, CLIP, tactile sensing, depth estimation, feature pyramid network, robotic manipulation

The core insight behind Tiny-STM is that object permanence, the cognitive ability to understand that things continue to exist when hidden from view, does not require a learned memory mechanism of the kind used in large transformer-based systems. Instead, the researchers, Swarnajit Bhattacharya of National Yang Ming Chiao Tung University in Taiwan and Ian Chao of the University of Washington, paired a standard four-gate Long Short-Term Memory network with a deterministic four-slot coordinate bank. Each slot stores the most recently observed three-dimensional position of one task object. When a cube disappears behind a stack of other blocks, its stored coordinate simply holds steady until the camera sees it again, a scheme the authors describe as a zero-order-hold update rule.

The architecture itself is deliberately conventional. The LSTM takes a seven-dimensional input vector combining the vector from the gripper tip to the target object’s stored position, the current tip position, and a one-dimensional infrared range reading from a Sharp GP2Y0A41SK0F sensor mounted on the gripper. A hidden state of 128 dimensions feeds a linear output layer that predicts an incremental three-dimensional displacement of the end effector. With 70,019 parameters in total, the network is tiny by modern standards, and after INT8 quantization through the TensorFlow Lite Micro pipeline it occupies just 68.4 KB, a small fraction of the ESP32’s 520 KB of static RAM.

Training relied on remarkably little data. The authors manually guided a Waveshare 5-DOF RoArm-M3 robotic arm through exactly 30 pick-and-place demonstrations, recording joint angles, cube centroids from a side-mounted camera, and infrared range measurements at each control step. Backpropagation through time with mean-squared-error loss and the Adam optimizer at a learning rate of 0.001 produced the deployed checkpoint. This data efficiency stands in sharp contrast to imitation-learning platforms such as ALOHA, which achieve 80 to 90 percent success on fine-grained bimanual tasks but require roughly ten minutes of teleoperated demonstrations and substantially more powerful compute.

The division of labor between hardware components is a key part of the design. A host PC or single-board computer captures camera frames and performs deterministic color-thresholding segmentation, converting detected pixel centroids into three-dimensional workspace coordinates through a calibrated pinhole camera model. The camera was calibrated with 25 checkerboard images using Zhang’s method, yielding a mean reprojection error of 0.3 pixels, while camera-to-base extrinsic calibration over 12 known points achieved a root-mean-square alignment error of 1.18 millimeters. The ESP32 then handles everything on the control side: the recurrent forward pass, analytical inverse kinematics, and PWM servo command generation, all in a measured 2.80 milliseconds.

The inverse kinematics routine reduces the five-degree-of-freedom arm problem to a planar three-degree-of-freedom calculation, exploiting the fact that the task uses a top-down grasp with a vertically oriented gripper. Base rotation comes from an arctangent of the target coordinates, while the shoulder and elbow angles follow from two-link geometry via the law of cosines. The authors handle three degenerate cases explicitly: base singularity when the target lies on the rotation axis, arm singularity near full extension where the Jacobian becomes ill-conditioned, and unreachable targets outside the geometric annulus defined by the link lengths, which are simply rejected so the arm retains its previous pose.

Physical evaluation consisted of 20 trials in which the arm sorted four labeled cubes from randomized positions into an ordered arrangement, including a vertical stacking sequence in which the fourth block becomes completely hidden behind the first three before its final grasp. The single deployed INT8 checkpoint completed 17 of 20 trials, an observed success rate of 85.0 percent with a two-sided 95 percent exact binomial confidence interval of 62.1 to 96.8 percent. The authors are careful to note that this interval describes uncertainty in the 20 observed outcomes of one checkpoint, not variability across independently trained models, since the original training seed and stopping rule were not retained.

The three failures are instructive. Mechanical backlash in the servo gearboxes introduced joint-level positional uncertainty of roughly 1.5 to 2.5 millimeters, which can accumulate across joints. Fixed color thresholds proved vulnerable to strong lighting changes and shadows that shifted pixel values out of range, degrading the segmentation masks. And when an occluded object was physically moved while hidden, the stored coordinate became stale, causing the controller to reach toward a position where the object no longer was; the gripper tolerance is approximately 18 millimeters. The authors formalize this limitation in a corollary showing that if an occluded object is displaced, the memory error grows at least in proportion to the displacement.

On latency, the measured ESP32 computation of 2.80 milliseconds combines with the nominal 33.3-millisecond camera period at 30 frames per second, profiled segmentation and Wi-Fi communication times, and a datasheet-based actuator response allowance to yield an aggregated end-to-end latency estimate of 92.6 milliseconds, equivalent to roughly 10.8 hertz. The authors emphasize that this figure is an estimate assembled from components rather than a single timed end-to-end measurement. Peak SRAM usage on the microcontroller was 94.8 KB, or 18.23 percent of capacity, leaving substantial headroom. Offline comparisons suggest the FP32 model would occupy 273.5 KB with an estimated 11.20 milliseconds of inference, underscoring the value of INT8 quantization for embedded deployment.

The study is notable as much for its honesty about boundaries as for its results. The authors explicitly state that no ablation was performed to separate the contribution of the coordinate memory from that of the recurrent state, that no numerical comparison with RT-2, OpenVLA, TinyVLA, or Diffusion Policy is possible given different tasks and hardware, and that the color-thresholding front end was never validated against ground-truth masks. Future work, they write, should include matched ablation studies with multiple training seeds, documented reproducibility protocols, perception validation, tests of behavior when occluded objects are displaced, and scaling experiments beyond four tracked objects. Even so, Tiny-STM offers a compelling demonstration that a form of object permanence can live in 68 kilobytes, bringing occlusion-aware manipulation within reach of the cheapest embedded hardware.

Subject of Research: Lightweight recurrent neural network controllers with external coordinate memory for occlusion-aware robotic manipulation on microcontrollers

Article Title: Lightweight LSTM-based memory architecture for occlusion-aware robotic manipulation on microcontrollers

Article References: Bhattacharya, S., & Chao, I. (2026). Lightweight LSTM-based memory architecture for occlusion-aware robotic manipulation on microcontrollers. Discover Informatics, 1(1), Article 23. https://doi.org/10.1007/s44564-026-00023-0

Image Credits: AI Generated

DOI: 10.1007/s44564-026-00023-0

Keywords: TinyML, LSTM, robotic manipulation, object permanence, occlusion, ESP32, microcontroller, imitation learning, inverse kinematics, INT8 quantization, embedded robotics, recurrent neural networks

The framework, described in the journal Applied Intelligence, is called HALSI, short for Hierarchical Adaptive Latent Skill Inference. Its central insight is that the temporal structure of behavior matters as much as the behavior itself. Earlier skill-based hierarchical methods decompose complicated tasks into reusable behavior primitives, which eases the burden of long-horizon decision-making, but they typically fix the length of each skill in advance and compress behavior into oversimplified latent representations. That rigidity limits both temporal flexibility and the diversity of behaviors an agent can express. HALSI removes the fixed horizon entirely, learning skills of variable duration from data and adjusting them online as the task unfolds.

At the heart of the system sits a Transformer encoder that reads variable-length trajectories of state-action pairs and distills them into a compact latent embedding. The architecture is causal, meaning each token corresponds to a state-action tuple, and positional encodings preserve the sequential order so that self-attention can capture long-range dependencies across time. A special classification token at the output summarizes the overall temporal and behavioral pattern of the skill into a single vector. Crucially, during pretraining the system preserves the natural lengths of trajectory segments rather than chopping demonstrations into fixed windows, which allows the encoder to learn duration-agnostic representations that retain the genuine variability of different behaviors.

To turn those latent skills into actual motion, HALSI pairs the encoder with a conditional diffusion policy. Denoising diffusion probabilistic models, which have transformed image generation in recent years, work here by iteratively refining Gaussian noise into a coherent action sequence, conditioned on both a timestep embedding and the concatenation of the latent skill vector with the current state. This generative approach lets the decoder model complex, multimodal action distributions and produce temporally consistent behaviors that a simpler policy class would struggle to capture. Once pretrained on offline demonstration data, the encoder and decoder remain frozen during online reinforcement learning, serving as a stable library of skills that the higher levels of the hierarchy can draw upon.

The first major innovation addresses the fixed-horizon problem directly: an adaptive duration policy that predicts a discrete execution length for each skill it selects. Rather than committing to a predetermined number of steps, the high-level controller outputs both a latent direction and a duration, and the diffusion decoder generates a matching action sequence of exactly that length before a new skill is sampled. The researchers found that different tasks induce strikingly different patterns of skill durations. In a slippery pushing task, durations cluster tightly at short values, reflecting the need for frequent fine-grained corrections on low-friction surfaces. In a tool-use task involving a hook, the distribution becomes broad and multimodal, as the agent alternates between brief reactive skills and longer sustained motions for aligning, inserting, and pulling. A pick-and-place task showed a bimodal pattern corresponding to its two distinct sub-goals.

The second innovation is a multi-scale optimization scheme built on Proximal Policy Optimization. A high-level modulation policy refines latent skills with task-aware residuals, adding a learned direction to the encoded latent mean so that the skill embedding itself is adjusted rather than the raw actions. Because directly perturbing behavior early in training would destabilize learning, the team introduced a curriculum: a scheduling factor follows a logistic curve over training steps, starting near zero so the agent initially relies almost entirely on the pretrained skill encoder, then gradually rising to give the residual policy increasing influence. A low-level soft-blending controller complements this by mixing the offline-decoded skill action with an online-learned adaptive actor at every step, allowing moment-to-moment corrections while preserving the structure of the skill prior.

The ablation studies reveal how sensitive this balance is. When the blending coefficient was set low, the agent largely ignored its pretrained skills and fell back on the reactive actor alone, producing unstable and inefficient behavior in long-horizon tasks such as pyramid stacking, where structured skill priors are essential for guidance. When the coefficient was pushed very high, the agent over-trusted the offline decoder and lost the ability to respond to distribution shifts, a weakness most visible in the slippery pushing environment where precise reactive adjustments are frequently needed. The best overall performance emerged consistently at a coefficient of 0.8, a setting that retains the semantic consistency of the offline skills while leaving room for online refinement.

Evaluated on two demanding benchmark suites, the framework delivered substantial gains. On four long-horizon manipulation tasks from the Reskill benchmark, built on MuJoCo-based Fetch environments, the agent had to transfer skills learned from roughly 40,000 trajectories collected with deliberately suboptimal scripted controllers in simplified settings, then apply them in downstream environments featuring slippery surfaces, cluttered distractor objects, multi-stage stacking goals, and contact-rich tool use. On the D4RL Kitchen benchmark, where a Franka arm operates household appliances across episodes of roughly 280 steps with rewards granted only when subtasks are completed, the sparse-reward structure makes credit assignment notoriously difficult. Across these tests, HALSI surpassed state-of-the-art hierarchical baselines, achieving up to 30 percent higher returns and converging 25 percent faster.

Beyond the headline numbers, the work offers a broader lesson about what makes hierarchical learning succeed. The duration analysis showed that a fixed-duration scheme would inevitably fragment some behaviors while leaving others insufficiently abstract, whereas a learned duration model allocates temporal granularity according to context and task stage. The authors suggest that jointly modeling latent skills and their adaptive temporal scope is the key ingredient, rather than improving either component in isolation. The source code has been released publicly, and the datasets are available from the corresponding author on reasonable request, lowering the barrier for other groups to build on the approach.

The implications reach well beyond simulated kitchens and block-stacking. Long-horizon, sparse-reward problems pervade robotics, autonomous driving, and industrial automation, and any method that lets agents reuse skills flexibly across tasks with differing dynamics could accelerate progress in all of them. By combining the sequence-modeling power of Transformers, the expressive generative capacity of diffusion models, and a principled curriculum for handing control from imitation to online learning, HALSI points toward a future in which robots do not merely execute preprogrammed routines, but decide for themselves which skill fits the moment and how long to commit to it.

Subject of Research: Hierarchical reinforcement learning with adaptive latent skill inference for temporal abstraction in long-horizon robotic manipulation

Article Title: HALSI: Hierarchical adaptive latent skill inference for temporal abstraction in reinforcement learning

Article References: Si, X., Ping, Y., Wang, T., Chen, K., & Gao, Y. (2026). HALSI: Hierarchical adaptive latent skill inference for temporal abstraction in reinforcement learning. Applied Intelligence, 56(15), Article 444. https://doi.org/10.1007/s10489-026-07485-7

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07485-7

Keywords: reinforcement learning, hierarchical RL, skill abstraction, temporal abstraction, diffusion policy, transformer encoder, robotic manipulation, PPO, D4RL Kitchen, Fetch benchmark, machine learning, latent skills

The core contribution of the work is a neural architecture the authors call the Occlusion-Aware Pairwise Relational Network, or OAPRN. Rather than trying to understand an entire cluttered pile of objects in one monolithic computation, the network breaks the problem down into a sequence of pairwise judgments. For every pair of objects detected in the scene, the network answers a deceptively simple question: which one of the two is on top? This might sound like a crude simplification of the full three-dimensional reasoning problem, but the mathematics of stacking relationships is friendly to such decompositions. If object A occludes object B, and object B occludes object C, then a robot can infer that A must be dealt with before C, even if A and C never interact directly in the image. Stacking, in other words, is a transitive relation, and pairwise comparisons encode all the information needed to recover the full ordering.

This pairwise decomposition is not just an aesthetic choice; it carries real algorithmic consequences. When a relational network must process all objects and their interactions simultaneously, the size of its internal representation grows combinatorially with the number of objects, and training the network requires examples covering every conceivable clutter configuration. By contrast, a pairwise network sees a much narrower and more consistent learning problem: given two objects and their image evidence, predict the occlusion direction. Because each training example concerns just two objects, the diversity of scenes needed to teach the network the underlying visual cues is far smaller, and the learned features generalize more readily to piles with different numbers of items. Once individual pairwise predictions are made, the system assembles them into an overall relationship graph, a directed structure whose edges point from the covering object to the covered one, and from which the full stacking hierarchy of the scene can be read off.

The graph itself is where perception turns into action. With the relationship graph in hand, a robot asked to grasp a specific target object can trace the chains of occlusion leading to that object and identify exactly which items stand between the gripper and the goal. The grasping strategy then proceeds in a deliberately orderly fashion: objects that cover the target are removed first, beginning with those highest in the stack, and the target is grasped only once its immediate surroundings are clear. This sequential clearing avoids the classic failure mode of clutter manipulation, in which a greedy grasp of the target dislodges neighbors and buries it deeper. The authors describe how the robot can execute this plan quickly, because the computational cost of the pairwise network remains modest even as the pile grows, and because the graph reconstruction step operates on a small number of detected objects rather than on raw image data.

Efficiency is not merely a convenience in this setting; it is what allows the robot to cope with the fundamental non-determinism of manipulation. Every time a gripper removes an object, the remaining pile shifts, occlusions change, and previously hidden surfaces come into view. A system that needed several seconds to recompute the scene graph would be constantly planning against a world that no longer exists. Because OAPRN can rebuild the relationship graph rapidly from a fresh image, the robot can replan after every action, treating each removal as a new observation rather than a disruption to a fragile long-horizon plan. This closed-loop quality, in which perception, planning and action alternate at a pace matched to the physical dynamics of the clutter, is arguably the most practically important feature of the method, and it distinguishes the approach from heavier relational reasoning pipelines that produce accurate but stale scene interpretations.

The technical pipeline begins with standard object detection on the input image, for which the authors leverage modern detector tooling of the kind exemplified by the widely used YOLO family of models cited in their bibliography. Each detected object yields an image region and associated features, and these pairwise feature combinations are passed through the relational network to predict occlusion direction. The idea of detecting visual manipulation relationships in this way has a lineage in the robotics literature, notably in work on visual manipulation relationship networks and gated graph neural networks for robotic grasping, as well as graph-based reasoning for target-driven grasping in dense clutter. What the Shanghai Jiao Tong team adds is a focus on making this reasoning lightweight enough for real-time, iterative use, and on coupling it directly to a principled clearing strategy rather than to learned push-and-grasp policies that can be harder to interpret and verify.

The experimental section of the paper puts the method through its paces in cluttered scene trials designed to verify both the relationship detection and the resulting grasp behavior. The authors report that experiments were carried out to verify the proposed method, and the accompanying figures trace the pipeline from raw input images through pairwise relationship analysis, graph construction, and the executed sequence of removals culminating in the target grasp. A supplementary video accompanies the publication, offering a dynamic view of the system operating on physical clutter. The work was supported by the National Science and Technology Major Project under grant number 2024ZD1600204, and the authors note that no datasets were generated or analyzed during the study beyond those used in the reported experiments. Author contributions attribute the algorithm design to Yan Zha and Yuxuan Wang, the experimental work to Zha, Wang and Jianhua Wu, and the manuscript drafting to Zha, with all authors reviewing the final text.

Situating this work in the broader landscape of robotic grasping research clarifies why it matters. Grasp detection as a field has advanced enormously, from early deep learning approaches to grasp rectangles through modern six-degree-of-freedom grasp planners operating on point clouds, and systems such as AnyGrasp that perceive grasps robustly across space and time. Yet most of these methods answer the question of how to grasp an object, not whether it should be grasped yet. A parallel line of research, including work on target-oriented push-grasping synergies and hierarchical stacking relationship prediction, has tackled the order-of-operations problem, often at considerable computational expense or with limited adaptability to scene changes. Large vision-language-action models promise general manipulation competence, but their inference cost makes them poorly suited to the millisecond-level replanning loop that clutter demands. OAPRN occupies a pragmatic middle ground: a specialized, efficient relational reasoner that does one thing, determining what covers what, well enough to support fast and safe sequential manipulation.

The potential applications extend across both halves of the robotic world the authors name in their motivation. In industrial automation, random bin picking remains one of the highest-volume manipulation tasks in logistics and manufacturing, and any improvement in the speed or reliability with which a robot can extract a specific part from a tangled pile translates directly into throughput. In domestic service, where robots must fetch requested items from drawers, refrigerators, and countertops they have never seen arranged in advance, occlusion-aware reasoning is the difference between a helpful assistant and a hazard. The decomposition strategy also has an appealing quality for safety-critical settings: because the relationship graph is explicit and human-readable, an engineer can inspect exactly why the robot decided to remove one object before another, rather than trusting an opaque end-to-end policy. As robots move from structured cells into the genuinely messy environments where humans live and work, research of this kind, focused on making relational perception fast, transparent, and reactive, may prove as consequential as the grasp planners and foundation models that currently dominate the headlines.

Subject of Research: Occlusion-aware pairwise relational reasoning for efficient robotic grasping of target objects in cluttered scenes

Article Title: Efficient grasping of target objects in cluttered scenes

Article References: Zha, Y., Wang, Y., Wu, J., & Xiong, Z. (2026). Efficient grasping of target objects in cluttered scenes. International Journal of Intelligent Robotics and Applications. https://doi.org/10.1007/s41315-026-00595-y

Image Credits: AI Generated

DOI: 10.1007/s41315-026-00595-y

Keywords: robotic grasping, occlusion detection, relational networks, cluttered scenes, graph-based planning, robotic manipulation, object relationship detection, pairwise reasoning, computer vision, machine learning, industrial automation, Shanghai Jiao Tong University