Pareto-guided AI reduces simulated cobot welding errors and collisions

A reinforcement-learning system designed to balance several competing requirements at once produced accurate, efficient, and collision-avoiding trajectories for a collaborative robot performing automotive spot welding. In a controlled simulation benchmark, the Pareto-Guided Multi-Objective Deep Deterministic Policy Gradient method, or PMO-DDPG, achieved a mean trajectory error of 1.08 ± 0.29 millimeters, a collision rate of 1.2 ± 0.5%, and a success rate of 94.6 ± 1.8% across five independent runs. It also used 3.65 ± 0.28 kilojoules of energy per operation, completed cycles in 3.87 ± 0.25 seconds, and generated commands with a mean inference time of 21.5 ± 1.6 milliseconds. These were descriptive within-study results: the researchers did not perform a formal statistical significance test, and the experiments involved one robot, one welding cell, a predefined 20-point sequence, and a static workspace.

The method was developed for a six-degree-of-freedom collaborative manipulator carrying a resistance spot-welding gun. That setting is demanding because a successful trajectory must position the welding gun accurately while avoiding body structures and fixtures, moving smoothly, respecting the robot’s mechanical limits, maintaining the required force, and completing work quickly enough for an automotive production line. These aims can conflict. A more direct path may reduce cycle time but sacrifice clearance from nearby metal, while an especially precise alignment can add motion and delay the next weld. Conventional reinforcement-learning methods commonly combine such goals in one manually weighted scalar reward, allowing one objective to obscure another. PMO-DDPG instead retains separate value estimates for trajectory accuracy, motion smoothness, operational safety, and production efficiency.

At the core of the framework is one deterministic actor network that produces continuous Cartesian corrections and a force command. Four independent critics estimate long-term performance for accuracy, smoothness, safety, and productivity. Behind those four critics are six physically interpretable reward components: position and orientation tracking, motion smoothness based on acceleration and jerk, obstacle clearance and collision information, energy use, cycle time, and welding-force tracking. Each component is normalized to a common interval from minus one to zero, with values nearer zero indicating better performance. During actor updates, the component values are combined using phase-dependent weights and then mapped to the four critic objectives, allowing each critic to supply its own action gradient without erasing the distinction between the underlying physical criteria.

The priorities change as the robot moves through a welding cycle. During approach, the base weights emphasize safety and smoothness while the gun travels from a previous weld. Alignment places more emphasis on positioning because the tool is approaching the target in a confined area. Welding gives greatest weight to accuracy and force regulation, with less importance assigned to speed and energy. Retraction restores relatively greater attention to smooth, efficient withdrawal. These phase-specific weights are further adjusted using a performance deficit measured against reference values in an archive of non-dominated policies. A policy that trails the archive’s safety reference, for example, receives stronger pressure from the safety critic, while an objective already close to the reference contributes less pressure. This is intended to prevent a single reward scale from silently dominating the actor’s update.

PMO-DDPG also uses Pareto dominance to retain a set of useful trade-offs rather than collapsing the entire training process into one fixed compromise. At scheduled checkpoints, the current policy is evaluated using the four trajectory-level objectives and added to the archive only if no stored policy dominates it; policies that become dominated are removed. After training, an operator can select the non-dominated policy closest to a preferred ideal point, with different choices emphasizing precision, energy reduction, safety, or a balance among them. This selection occurs after learning, so it does not add archive-search work to the online control loop. During deployment, only the actor is needed to generate motion, while a separate constraint-checking layer enforces joint-position, velocity, acceleration, workspace, collision-clearance, and welding-force restrictions before a command can be executed.

The simulation used Webots and a body-in-white workstation containing fixtures, restricted regions, obstacles, narrow passages, and 20 predefined welding points. Domain randomization perturbed the initial robot configuration, target-localization uncertainty, and obstacle placement within specified margins. Each learning method was trained for 2,000 episodes with a 500-step limit and evaluated over five independent runs, using a batch size of 128, a discount factor of 0.99, and a 100,000-transition replay buffer for off-policy methods. The proposed policy was compared under the same environment and evaluation protocol with DDPG, TD3, PPO, SAC, DDPG-DG, Fuzzy-TD3, entropy-adjusted PPO, and recurrent SAC. Geometric and optimization-based planners, including model predictive control, interpolation, Bézier and spline methods, and A* search, were also executed in the same workspace, although they were not assigned reinforcement-learning training budgets.

Relative to the strongest conventional learning baseline, Soft Actor-Critic, PMO-DDPG reduced mean trajectory error by 66.4%, collision rate by 82.1%, energy consumption by 15.5%, cycle time by 22.0%, and inference latency by 22.7%, while raising the mean success rate by 7.5 percentage points. SAC’s corresponding values were 3.21 ± 0.41 millimeters of trajectory error, a 6.7 ± 1.2% collision rate, 4.32 ± 0.34 kilojoules of energy, a 4.96 ± 0.31-second cycle, an 87.1 ± 2.6% success rate, and 27.8 ± 2.1 milliseconds of inference time. Recurrent SAC achieved the same reported mean energy consumption as PMO-DDPG, at 3.65 kilojoules, but had a higher mean trajectory error of 2.11 ± 0.31 millimeters, compared with 1.08 ± 0.29 millimeters for PMO-DDPG. The results therefore support a favorable descriptive comparison under the common benchmark, not a claim of statistically demonstrated or universal superiority.

Training curves showed the proposed actor’s average episode reward rising from approximately minus 1,000 during early exploration to nearly 1,000 after convergence, while task success increased from below 10% to around 93%. The actor loss quickly fell and stabilized, critic loss declined from nearly 15 to about 1.5, and variability around the learning curves narrowed. A separate Pareto analysis showed that the proposed policies offered more favorable accuracy–energy trade-offs than the DDPG, TD3, PPO, and SAC fronts within the experiment. Representative trajectories were more direct and collision-free than those produced by several geometric planners, and the learned path had the lowest and most stable curvature profile in the reported examples. The authors also note that a mean inference time of 21.5 milliseconds should not be interpreted as proof of meeting a hard 10-millisecond deadline implied by the simulator’s 100-hertz control setting.

The study then moved from simulation to 100 repeated trials in a real automotive spot-welding production cell. A vision system detected each target and estimated the offset between the robot’s tool center point and the desired welding position; that correction was inserted into the state used by the actor before execution. Planned and actual robot motions were recorded for comparison. A representative pre-weld measurement recorded Cartesian errors of 0.19 millimeters in X, minus 0.17 millimeters in Y, and 0.12 millimeters in Z, together with an angular error of 0.22 degrees. Across the repeated physical trials, errors in all three Cartesian directions remained below approximately 0.4 millimeters and tended to stabilize near 0.2 millimeters, while angular deviation remained around 0.2 degrees. These trials tested one body-in-white door frame and did not include confidence intervals or replication at additional manufacturing sites.

The physical results show that the learned policy operated repeatably in a controlled industrial setting when combined with vision-guided target correction, but the experiment did not include a pre-versus-post or concurrent comparator demonstrating that PMO-DDPG improved the industrial trials. Important limitations also remain. The experiments did not evaluate moving human workers, dynamic obstacles, multi-robot coordination, changing production schedules, or long-term electrode wear. Process-level outcomes such as weld current, nugget formation, thermal deformation, and weld quality were not incorporated into the multi-objective reward. The work used only five independent simulation runs per learning method and did not conduct inferential testing. Its promise is therefore a practical way to preserve several manufacturing objectives as separate signals while allowing their influence to shift between approach, alignment, welding, and retraction. Establishing broader reliability would require larger multi-site studies, formal statistical testing, public benchmarks, dynamic-obstacle experiments, and feedback based on the resulting weld itself.

Subject of Research: Multi-objective reinforcement learning for collaborative-robot trajectory planning in automotive spot welding

Article Title: Multi-objective deep deterministic policy gradient for cobot trajectory planning in automotive spot welding

Article References: Omarov, B. (2026). Multi-objective deep deterministic policy gradient for cobot trajectory planning in automotive spot welding. Frontiers in Robotics and AI, 13, Article 1939068. https://doi.org/10.3389/frobt.2026.1939068

Image Credits: AI Generated

DOI: 10.3389/frobt.2026.1939068

Keywords: collaborative robots, automotive manufacturing, spot welding, trajectory planning, reinforcement learning, multi-objective optimization, Pareto analysis, industrial safety, machine vision, deep deterministic policy gradient, Multi-objective, deep