The work, led by Oorappan G M of KGiSL Institute of Technology in Coimbatore and Lakshmanan S A of Amrita Vishwa Vidyapeetham in Chennai, introduces what the authors call an SSS–DDQN solution-space best-first search controller. The name is a mouthful, but the idea is elegant. At its heart lies a double deep Q-network, or DDQN, a reinforcement-learning architecture that learns to estimate the long-term value of corrective actions while deliberately reducing the overestimation bias that plagues standard Q-learning. Wrapped around that learned value function is SSS, a best-first search algorithm descended from the minimax tradition of adversarial game playing. Rather than diving depth-first through a tree of possible futures the way Alpha–Beta search does, SSS* maintains a priority queue of partial solutions and always refines the most promising cluster first. The authors argue that this shift from depth-first to best-first refinement is exactly what a robot needs when it cannot see its own state clearly and must choose corrective actions under sensor noise and dropout.

To understand why this matters, consider what the robot is actually up against. The researchers model a four-Mecanum-wheel robot as a planar rigid body whose posture is described by inertial-frame position, heading angle, and body-frame velocities. The dynamics include an inertia matrix, a nonlinear velocity-coupling term, and a damping matrix, all of which the controller must respect while distributing a generalized force to four independently driven wheels through the Mecanum force-mapping mechanism. The reference trajectory is a lemniscate, a figure-eight-like curve chosen because it continuously changes curvature and demands combined forward, sideways, and rotational motion, making it a rigorous test of omnidirectional tracking. On top of this, the simulation injects zero-mean sensor noise into position, heading, velocity, and yaw-rate measurements, and adds a 3 percent probability of position dropout, mimicking the temporary localization loss that real robots suffer from wheel slip, occlusion, or communication delay.

Because the true state is hidden, the framework begins with an extended Kalman filter, or EKF, that estimates the hidden robot state and maintains a belief mean and covariance. The EKF prediction step uses the nonlinear Mecanum robot model to forecast the next belief, while the correction step uses the innovation, the difference between the predicted and measured observation, and the Kalman gain to update the belief through a Joseph covariance update that preserves numerical stability. This belief state, rather than any raw measurement, is what the learning and search components operate on. The authors are careful to distinguish this physics-informed belief-consistent approach from latent-dynamics methods like MuZero or TD-MPC, which learn internal state representations through self-supervised consistency losses. Here, consistency is enforced by the Kalman filtering machinery itself, grounded in the nonlinear robot model and the measurement residual, rather than by an auxiliary loss function.

The DDQN component is trained with a 15-dimensional state vector that includes tracking errors, body-frame velocities, error rates, and reference trajectory information, and a discrete action space of 27 corrective actions spanning longitudinal force, lateral force, and yaw moment. The network architecture uses three fully connected hidden layers with 192, 192, and 128 neurons and ReLU activations, outputting 27 Q-values. Training employs prioritized experience replay with a priority exponent of 0.6, a discount factor of 0.985, and an exploration rate that decays from 1.0 to 0.05. The learned corrective action is added to a nominal model-based torque command, so the reinforcement-learning agent refines rather than replaces the physical controller. The reward function penalizes position error, heading error, velocity error, control effort, and actuator saturation, with a bonus for accurate tracking and a failure penalty for large deviations.

Where the proposed method diverges from its baselines is in how it uses the DDQN at search time. The SSS–DDQN controller ranks all 27 candidate actions using QMDP belief sampling, which averages DDQN Q-values over 15 belief samples drawn from the EKF covariance, and passes only the top-ranked actions to the SSS search. The search itself uses a three-ply MAX–MIN–MAX structure, where MAX nodes represent robot action selection and MIN nodes represent worst-case disturbance response. Each node in the OPEN priority queue carries a LIVE or SOLVED label, a bound value, a root action, accumulated cost, and a cluster identity. When a LIVE node reaches the search horizon, it is converted into a SOLVED node by blending the averaged DDQN value with a negative model-based tracking cost and adding a belief-uncertainty penalty proportional to the covariance trace. Beta-bound pruning prevents over-optimistic DDQN leaf estimates from dominating the final decision, delayed MIN-sibling refinement pushes less critical disturbance branches to lower priority, and a secondary lookahead checks for hidden finite-horizon disturbance effects before the final action is committed.

The authors compare this controller against four baselines under identical conditions: EKF-belief PER-DDQN, which uses filtered belief states directly; QMDP-PER-DDQN, which averages Q-values over belief samples; Alpha–Beta Minimax, which performs depth-first adversarial search; and AB–DDQN Hybrid, which combines Alpha–Beta search with DDQN leaf evaluation. The proposed SSS*–DDQN achieved the best overall ranking with a root-mean-square position error of 0.01906 meters, a mean absolute error of 0.016074 meters, and a cumulative reward of 1843.6. Compared with the AB–DDQN Hybrid baseline, it reduced the RMS position error by 12.93 percent and the mean absolute error by 17.80 percent. Statistical validation over multiple independent stochastic trials, each with different random seeds for noise, dropout, and disturbance, showed the lowest mean RMS error of 0.01847 plus or minus 0.00132 meters and the lowest mean absolute error of 0.01554 plus or minus 0.00128 meters, with narrow 95 percent confidence intervals. The EKF covariance trace converged to nearly identical values across all controllers, confirming that the performance differences arise from action-selection and search strategies rather than estimation uncertainty.

The search-efficiency numbers reveal how the controller achieves this. The proposed method refined 264,423 nodes, generated 264,423 additional nodes, and pruned 64.27 percent of them, far exceeding the 13.44 percent pruning ratio of Alpha–Beta Minimax and the 16.06 percent of AB–DDQN Hybrid. It also performed 36,383 secondary lookahead operations, a mechanism no baseline employs. An ablation study confirms that no single module alone drives the improvement: the EKF provides stable belief estimation, QMDP improves uncertainty-aware action evaluation, DDQN supplies learned value guidance, SSS* refines promising action clusters, beta-bound pruning reduces unnecessary expansion, and secondary lookahead improves final action robustness. The authors also verified with a fourth-order Runge–Kutta sensitivity check that the comparative conclusions are not artifacts of Euler integration, since all controllers were evaluated under identical numerical and sensing conditions.

Yet the study is candid about its costs. Single-step runtime profiling in a MATLAB–NVIDIA Isaac Sim co-simulation through a ROS 2 interface showed that the proposed controller required a mean runtime of 4,830.4 milliseconds per control step, with a 95th-percentile runtime of 6,587.9 milliseconds, against a 20-millisecond deadline for 50 Hz control. Component-level profiling revealed that SSS* OPEN-list refinement accounted for roughly 98.8 percent of total execution time, making it the dominant bottleneck rather than belief-state estimation. By contrast, EKF-belief PER-DDQN completed all steps within the deadline at a mean of 1.961 milliseconds. The authors therefore recommend adaptive search budgeting, uncertainty-triggered SSS* activation, reduced top-ranked action sets, fewer belief samples, parallel or GPU evaluation, and fallback to simpler controllers when computational resources are limited. The present full-search configuration is suitable for simulation-based validation, offline analysis, and supervisory decision-making, but not yet for deterministic real-time control on embedded hardware.

The broader significance lies in the demonstration that learned value functions and classical best-first search can be formally integrated at four levels: action ranking, node expansion, leaf-node evaluation, and final action selection. The DDQN does not replace the search process; it guides it by reducing the branching factor, evaluating terminal nodes, influencing pruning bounds, and supporting robust final action choice. This positions the work as a bridge between the AlphaGo lineage of value-guided tree search, which has transformed game playing, and the practical demands of mobile robot control under partial observability. The authors identify future work in hardware testing, real-time profiling, additional trajectory cases, and comparison with PPO, SAC, TD3, model-predictive control, and MCTS-based methods. If the computational bottleneck can be tamed, the approach could offer a template for robots that reason about their own uncertainty the way a chess engine reasons about its position, always refining the most promising line first.

Subject of Research: SSS–DDQN solution-space best-first search for robust trajectory tracking of Mecanum robots under partial observability

Article Title: SSS–DDQN solution-space best-first search for robust trajectory tracking of Mecanum robots under partial observability

Article References: G M, O., & S A, L. (2026). SSS–DDQN solution-space best-first search for robust trajectory tracking of Mecanum robots under partial observability. Frontiers in Robotics and AI, 13, Article 1896260. https://doi.org/10.3389/frobt.2026.1896260

Image Credits: AI Generated

DOI: 10.3389/frobt.2026.1896260

Keywords: DDQN, solution-space, best-first, search, robust, trajectory, tracking, Mecanum, robots, under, partial, observability

The central insight behind the research is a concept the authors call Critical Decision Points, or CDPs. These are specific states in a shared task where two or more competing human strategies would prescribe clearly different next actions. Imagine a kitchen robot collaborating with a person on a recipe: if two different dishes the person might be preparing require identical actions up to a certain moment, watching the person gives the robot no new information. But if the robot guides the interaction so that the person reaches a junction where making soup demands reaching for a pot while making a salad demands reaching for a cutting board, the very next action unambiguously discloses the goal. CDPs are precisely those junctions, and they maximally reveal intent because the divergent strategies force a distinguishing choice.

To turn this intuition into a computable method, the researchers formalized CDPs using a goal-conditioned policy divergence measure. In plain terms, the team evaluated, for any given state of the world, how differently an optimal policy would behave depending on which goal the human is pursuing. High divergence means the state is highly informative: small differences in behavior at that state separate the candidate goals cleanly. Low divergence means the state tells the robot almost nothing, because all plausible goals call for the same action. By quantifying this divergence, the robot can rank future states by how much information they would generate about the human’s hidden objective.

Measuring informativeness, however, is only half the problem. The robot also has to accomplish the actual task, whether that means assembling ingredients in a kitchen or chasing down an opponent in a game. The team therefore embedded the CDP framework inside a Receding-Horizon Planner, a control architecture rooted in model predictive control that repeatedly looks ahead over a short window of future actions, optimizes a cost function, executes the first action, and then replans. The cost function balances two competing objectives: task progress, meaning the robot keeps moving the joint activity toward completion or toward winning, and information gain, meaning the robot steers toward states that maximize policy divergence and therefore reveal the human’s goal. The receding-horizon structure keeps the planning computationally tractable, because the robot never has to search the full branching tree of future interactions; it only reasons a few steps ahead at a time while still accounting for how its own actions will shape what the human does next.

What makes the work especially notable is that the same unified framework operates in two dramatically different social regimes. In the collaborative setting, the robot and human share a fully observable cooking task, and both parties benefit from smooth coordination. Here, steering the human toward CDPs serves partnership quality: the earlier the robot knows whether its partner is preparing one dish or another, the earlier it can hand over the right utensil, clear the correct counter space, or divide labor sensibly. Faster and more accurate goal inference translates directly into more fluent teamwork, reduced awkward pauses, and fewer mistaken anticipations of what the human wants next.

In the competitive setting, the tables turn entirely. The two agents play a partially observable hide-and-seek game in which the human’s goal is hidden from the robot, and revealing it works to the robot’s advantage rather than the human’s. Even here, the CDP-driven planner proved effective: by maneuvering the human into states where the seeker’s and hider’s strategies prescribe different moves, the robot extracted decisive evidence about the opponent’s hidden objective. The authors frame this in the tradition of opponent modeling, in which an agent builds and refines a predictive model of a rival’s intentions from observed behavior. Provoking opponents into informative situations has precedent in the game-playing literature, but this study is among the first to unify adversarial information gathering and cooperative assistance under a single planning objective.

Evaluation was deliberately thorough. The researchers tested their method in both domains in simulation and then transferred the approach to real robots interacting with actual human participants, a transition that notoriously breaks many planning algorithms due to perception noise, motion constraints, and unpredictable human behavior. Across both simulated and physical trials, the CDP-based planner inferred human goals more accurately and, crucially, earlier than baseline strategies that either pursued the task greedily or gathered information without shaping the human. The earlier a robot can commit to a confident estimate of a partner’s goal, the more useful actions it can take on that partner’s behalf, which is why the timing improvement matters as much as the accuracy improvement.

The implications stretch well beyond kitchens and games. Assistive robots in homes, factories, and hospitals constantly face the problem of deciphering what a person is trying to do, often from ambiguous and fragmentary evidence. A wheelchair-mounted manipulator that can gently reposition a shared workspace so that a user’s next movement disambiguates their intent could provide safer and more responsive support. Warehouse cobots could coordinate with human pickers more fluidly. Even autonomous vehicles, which increasingly reason about the intentions of human drivers, could benefit from planners that recognize which traffic situations are most informative about another driver’s route. The study builds on a growing body of research showing that influence is a two-way street: humans adapt to robots, and robots that account for this adaptation, rather than treating human behavior as fixed background noise, plan fundamentally better.

At the same time, the work raises questions the authors themselves situate within a broader research conversation about robots that influence humans. Steering a person’s choices, even mildly and for legitimate informative purposes, touches on autonomy and transparency, and the competitive results in particular show that the same mathematics that helps a helpful kitchen assistant can empower an adversarial agent. The researchers report no competing interests and ground their contribution firmly in the engineering goal of faster mutual understanding. As robots move from controlled labs into shared human spaces, frameworks like CDPs suggest a future in which machine teammates do not merely watch and guess, but intelligently arrange the world so that human intentions become legible — a capability that promises smoother collaboration, sharper competition, and a clearer science of how people and machines can read each other’s minds in real time.

The study also connects to a longer intellectual lineage in machine learning and control. The idea of learning a reward function or objective from observed behavior traces back to inverse reinforcement learning, which showed that an observer can recover what an agent is optimizing by watching its actions. The present work inverts that logic: instead of passively accumulating observations, the robot actively engineers the observation conditions, choosing its own actions so that the human’s behavior becomes maximally diagnostic. This places the method in the emerging field of influence-aware planning, where an agent treats its partner’s future decisions as part of the environment it can shape rather than as an exogenous process it must simply predict.

Methodologically, the reliance on a receding-horizon architecture reflects decades of practice in model predictive control, a technique originally developed in industrial process control and now widespread in robotics because it handles constraints and replans gracefully under uncertainty. Adapting this mature machinery to a social objective — trading off task progress against epistemic value — is a notable engineering contribution, since classical cost functions rarely encode anything like information gain about a partner’s intent.

The dual-domain evaluation also strengthens the claim of generality. Many goal-recognition systems are validated in a single task, leaving open whether the underlying measure of informativeness is task-specific. By demonstrating that the same policy-divergence criterion improves inference in a fully observable cooperative setting and a partially observable adversarial one, the authors provide evidence that Critical Decision Points are a domain-independent structural property of sequential decision problems, not an artifact of a particular scenario. The consistency of results across simulation and physical hardware further suggests the approach is robust to the perception noise and behavioral variability that typically degrade deployed interactive planners.

Subject of Research: Robot planning strategies that influence human behavior to accelerate goal inference during collaboration and competition.

Article Title: Robots influencing humans to reveal their goals during collaboration and competition

Article References: Ghose, D., Gitelson, O., Lewkowicz, M., Brawer, J., Roncone, A., Vazquez, M., & Scassellati, B. (2026). Robots influencing humans to reveal their goals during collaboration and competition. Autonomous Robots, 50(3), Article 38. https://doi.org/10.1007/s10514-026-10267-2

Image Credits: AI Generated

DOI: 10.1007/s10514-026-10267-2

Keywords: human-robot interaction, goal inference, critical decision points, receding horizon planning, social robotics, opponent modeling, human-robot collaboration, game theory, artificial intelligence, planning, shared autonomy, Robots