Best-first search gives Mecanum robots a sharper eye under partial observability

Mecanum-wheeled mobile robots can slide sideways, spin in place, and glide forward without ever turning their bodies, which makes them the darlings of warehouse automation, hospital logistics, and indoor service robotics. But that omnidirectional freedom comes at a price. The longitudinal, lateral, and yaw motions of a four-Mecanum-wheel robot are nonlinearly coupled, the wheel-ground interaction varies with friction, actuators saturate, and the sensors that tell the robot where it is are noisy, occasionally dropping out entirely. In control-theory terms, the robot operates under partial observability: the true state is hidden, and the controller must act on a belief about where it is rather than a fact. A new study published in Frontiers in Robotics and AI proposes a controller that confronts this uncertainty head-on by fusing reinforcement learning with a classic game-tree search algorithm, and the results are striking enough to turn heads in the robot-learning community.

The work, led by Oorappan G M of KGiSL Institute of Technology in Coimbatore and Lakshmanan S A of Amrita Vishwa Vidyapeetham in Chennai, introduces what the authors call an SSS–DDQN solution-space best-first search controller. The name is a mouthful, but the idea is elegant. At its heart lies a double deep Q-network, or DDQN, a reinforcement-learning architecture that learns to estimate the long-term value of corrective actions while deliberately reducing the overestimation bias that plagues standard Q-learning. Wrapped around that learned value function is SSS, a best-first search algorithm descended from the minimax tradition of adversarial game playing. Rather than diving depth-first through a tree of possible futures the way Alpha–Beta search does, SSS* maintains a priority queue of partial solutions and always refines the most promising cluster first. The authors argue that this shift from depth-first to best-first refinement is exactly what a robot needs when it cannot see its own state clearly and must choose corrective actions under sensor noise and dropout.

To understand why this matters, consider what the robot is actually up against. The researchers model a four-Mecanum-wheel robot as a planar rigid body whose posture is described by inertial-frame position, heading angle, and body-frame velocities. The dynamics include an inertia matrix, a nonlinear velocity-coupling term, and a damping matrix, all of which the controller must respect while distributing a generalized force to four independently driven wheels through the Mecanum force-mapping mechanism. The reference trajectory is a lemniscate, a figure-eight-like curve chosen because it continuously changes curvature and demands combined forward, sideways, and rotational motion, making it a rigorous test of omnidirectional tracking. On top of this, the simulation injects zero-mean sensor noise into position, heading, velocity, and yaw-rate measurements, and adds a 3 percent probability of position dropout, mimicking the temporary localization loss that real robots suffer from wheel slip, occlusion, or communication delay.

Because the true state is hidden, the framework begins with an extended Kalman filter, or EKF, that estimates the hidden robot state and maintains a belief mean and covariance. The EKF prediction step uses the nonlinear Mecanum robot model to forecast the next belief, while the correction step uses the innovation, the difference between the predicted and measured observation, and the Kalman gain to update the belief through a Joseph covariance update that preserves numerical stability. This belief state, rather than any raw measurement, is what the learning and search components operate on. The authors are careful to distinguish this physics-informed belief-consistent approach from latent-dynamics methods like MuZero or TD-MPC, which learn internal state representations through self-supervised consistency losses. Here, consistency is enforced by the Kalman filtering machinery itself, grounded in the nonlinear robot model and the measurement residual, rather than by an auxiliary loss function.

The DDQN component is trained with a 15-dimensional state vector that includes tracking errors, body-frame velocities, error rates, and reference trajectory information, and a discrete action space of 27 corrective actions spanning longitudinal force, lateral force, and yaw moment. The network architecture uses three fully connected hidden layers with 192, 192, and 128 neurons and ReLU activations, outputting 27 Q-values. Training employs prioritized experience replay with a priority exponent of 0.6, a discount factor of 0.985, and an exploration rate that decays from 1.0 to 0.05. The learned corrective action is added to a nominal model-based torque command, so the reinforcement-learning agent refines rather than replaces the physical controller. The reward function penalizes position error, heading error, velocity error, control effort, and actuator saturation, with a bonus for accurate tracking and a failure penalty for large deviations.

Where the proposed method diverges from its baselines is in how it uses the DDQN at search time. The SSS–DDQN controller ranks all 27 candidate actions using QMDP belief sampling, which averages DDQN Q-values over 15 belief samples drawn from the EKF covariance, and passes only the top-ranked actions to the SSS search. The search itself uses a three-ply MAX–MIN–MAX structure, where MAX nodes represent robot action selection and MIN nodes represent worst-case disturbance response. Each node in the OPEN priority queue carries a LIVE or SOLVED label, a bound value, a root action, accumulated cost, and a cluster identity. When a LIVE node reaches the search horizon, it is converted into a SOLVED node by blending the averaged DDQN value with a negative model-based tracking cost and adding a belief-uncertainty penalty proportional to the covariance trace. Beta-bound pruning prevents over-optimistic DDQN leaf estimates from dominating the final decision, delayed MIN-sibling refinement pushes less critical disturbance branches to lower priority, and a secondary lookahead checks for hidden finite-horizon disturbance effects before the final action is committed.

The authors compare this controller against four baselines under identical conditions: EKF-belief PER-DDQN, which uses filtered belief states directly; QMDP-PER-DDQN, which averages Q-values over belief samples; Alpha–Beta Minimax, which performs depth-first adversarial search; and AB–DDQN Hybrid, which combines Alpha–Beta search with DDQN leaf evaluation. The proposed SSS*–DDQN achieved the best overall ranking with a root-mean-square position error of 0.01906 meters, a mean absolute error of 0.016074 meters, and a cumulative reward of 1843.6. Compared with the AB–DDQN Hybrid baseline, it reduced the RMS position error by 12.93 percent and the mean absolute error by 17.80 percent. Statistical validation over multiple independent stochastic trials, each with different random seeds for noise, dropout, and disturbance, showed the lowest mean RMS error of 0.01847 plus or minus 0.00132 meters and the lowest mean absolute error of 0.01554 plus or minus 0.00128 meters, with narrow 95 percent confidence intervals. The EKF covariance trace converged to nearly identical values across all controllers, confirming that the performance differences arise from action-selection and search strategies rather than estimation uncertainty.

The search-efficiency numbers reveal how the controller achieves this. The proposed method refined 264,423 nodes, generated 264,423 additional nodes, and pruned 64.27 percent of them, far exceeding the 13.44 percent pruning ratio of Alpha–Beta Minimax and the 16.06 percent of AB–DDQN Hybrid. It also performed 36,383 secondary lookahead operations, a mechanism no baseline employs. An ablation study confirms that no single module alone drives the improvement: the EKF provides stable belief estimation, QMDP improves uncertainty-aware action evaluation, DDQN supplies learned value guidance, SSS* refines promising action clusters, beta-bound pruning reduces unnecessary expansion, and secondary lookahead improves final action robustness. The authors also verified with a fourth-order Runge–Kutta sensitivity check that the comparative conclusions are not artifacts of Euler integration, since all controllers were evaluated under identical numerical and sensing conditions.

Yet the study is candid about its costs. Single-step runtime profiling in a MATLAB–NVIDIA Isaac Sim co-simulation through a ROS 2 interface showed that the proposed controller required a mean runtime of 4,830.4 milliseconds per control step, with a 95th-percentile runtime of 6,587.9 milliseconds, against a 20-millisecond deadline for 50 Hz control. Component-level profiling revealed that SSS* OPEN-list refinement accounted for roughly 98.8 percent of total execution time, making it the dominant bottleneck rather than belief-state estimation. By contrast, EKF-belief PER-DDQN completed all steps within the deadline at a mean of 1.961 milliseconds. The authors therefore recommend adaptive search budgeting, uncertainty-triggered SSS* activation, reduced top-ranked action sets, fewer belief samples, parallel or GPU evaluation, and fallback to simpler controllers when computational resources are limited. The present full-search configuration is suitable for simulation-based validation, offline analysis, and supervisory decision-making, but not yet for deterministic real-time control on embedded hardware.

The broader significance lies in the demonstration that learned value functions and classical best-first search can be formally integrated at four levels: action ranking, node expansion, leaf-node evaluation, and final action selection. The DDQN does not replace the search process; it guides it by reducing the branching factor, evaluating terminal nodes, influencing pruning bounds, and supporting robust final action choice. This positions the work as a bridge between the AlphaGo lineage of value-guided tree search, which has transformed game playing, and the practical demands of mobile robot control under partial observability. The authors identify future work in hardware testing, real-time profiling, additional trajectory cases, and comparison with PPO, SAC, TD3, model-predictive control, and MCTS-based methods. If the computational bottleneck can be tamed, the approach could offer a template for robots that reason about their own uncertainty the way a chess engine reasons about its position, always refining the most promising line first.

Subject of Research: SSS–DDQN solution-space best-first search for robust trajectory tracking of Mecanum robots under partial observability

Article Title: SSS–DDQN solution-space best-first search for robust trajectory tracking of Mecanum robots under partial observability

Article References: G M, O., & S A, L. (2026). SSS–DDQN solution-space best-first search for robust trajectory tracking of Mecanum robots under partial observability. Frontiers in Robotics and AI, 13, Article 1896260. https://doi.org/10.3389/frobt.2026.1896260

Image Credits: AI Generated

DOI: 10.3389/frobt.2026.1896260

Keywords: DDQN, solution-space, best-first, search, robust, trajectory, tracking, Mecanum, robots, under, partial, observability