Smarter Baselines Help Drones Stay on Course When Wind, Noise and Delay Strike

Autonomous drones promise to transform everything from package delivery and agricultural surveying to search-and-rescue and infrastructure inspection, but the real world is an unforgiving place for a learning algorithm. A quadrotor navigating a cluttered environment rarely faces just one problem at a time. Gusts of wind push it off course, its motors respond with noisy imperfection, control commands arrive late, and onboard sensors degrade in rain, dust or glare. When all of these disturbances strike simultaneously, the reinforcement learning algorithms that train such drones begin to struggle in a subtle but fundamental way: the yardstick they use to judge whether an action was good or bad becomes unreliable. A new study published in Applied Intelligence by Xin Wang and Jiankang Zhao of Shanghai Jiao Tong University tackles precisely this problem, and its proposed fix, called the Gradient-Decoupled Privileged Baseline, or GDPB, pushes drone navigation success rates above 96 percent in simulated compound-disturbance trials while outperforming several of the field’s most widely used algorithms.

To understand why compound disturbances are so corrosive to learning, it helps to recall how policy-gradient reinforcement learning works. In methods descended from the classic REINFORCE algorithm, a neural network policy is nudged in the direction of actions that produced higher-than-expected returns, where the return is the cumulative reward collected over a trajectory. The crucial phrase is higher than expected. Because the raw return varies wildly from episode to episode, even for identical behavior, practitioners subtract a baseline from the return to compute an advantage, a quantity that estimates how much better an action fared than average. Subtracting a baseline does not change the expected direction of the gradient, but it can dramatically reduce the variance of the estimate, and variance reduction is often the difference between a policy that converges and one that thrashes. The trouble, as Wang and Zhao point out, is that most standard methods use a single batch-mean return as the baseline for every state in the batch.

Under compound disturbances, that shortcut breaks down. Training samples within a batch differ enormously in their states and environmental conditions: one drone may be fighting a strong crosswind with a lagging controller while another glides through calm air with pristine sensors. A single scalar average of returns is simply an inadequate reference for the expected return at each individual state. The result is advantage estimates polluted by environmental noise, gradients that point in misleading directions, and training curves that are unstable across repeated runs. The problem is not that the learning signal is absent, but that it is measured against the wrong yardstick, and the mismatch grows precisely in the conditions where robust navigation matters most.

GDPB’s answer is to make the baseline state-dependent and, crucially, to give it information the policy itself will never see at deployment. The method trains a separate baseline branch that predicts Monte Carlo returns from two sources of input: the onboard observation features that the policy uses, and privileged disturbance parameters supplied by the simulator, such as the true wind field, actuation noise characteristics, control delay, and sensor degradation levels. Because the simulator knows exactly what adversity each drone is facing, the baseline branch can learn a far more accurate map from situation to expected return than any function of onboard observations alone. This idea of exploiting privileged information during training while keeping the deployed system lean belongs to a lineage of asymmetric actor-critic methods and teacher-student approaches that have proven successful in quadrupedal locomotion and agile flight, but GDPB applies it specifically to the variance-reduction machinery of policy gradients.

The second key ingredient is a gradient-decoupling trick that keeps the two branches from interfering with each other. When computing advantages for the policy update, a stop-gradient operation is applied to the baseline prediction, which means the policy loss cannot directly update the baseline branch through backpropagation. Instead, the baseline branch is trained separately through return regression, fitting its predictions to the actual Monte Carlo returns observed during training. This separation matters because entangled gradients can create a feedback loop in which the baseline shifts to flatter the policy rather than to accurately predict returns, degrading the very variance reduction it was meant to provide. By decoupling the gradients, GDPB preserves the theoretical property that a state-dependent baseline leaves the expected policy gradient unchanged while still delivering the practical benefit of lower-variance estimates. The technique echoes a broader lesson from recent work on REINFORCE-style optimization, including its prominent role in training large language models, that careful handling of baselines and gradient flow is central to stable policy learning.

Perhaps the most elegant feature of the design is what happens at deployment: the baseline branch and all privileged inputs are simply removed. The trained policy relies only on onboard observations, exactly as a real drone would, so GDPB adds no computational burden to the deployed system whatsoever. The privileged information and the extra network capacity are pure training-time scaffolding, discarded once their job of shaping a clean learning signal is done. This stands in contrast to approaches that attempt to estimate disturbance parameters online or carry recurrent memory of past disturbances, both of which impose runtime costs and can introduce their own failure modes. For weight- and power-constrained aerial platforms, a method that improves robustness without adding a single floating-point operation at flight time is a meaningful engineering advantage.

The empirical evaluation was conducted on a compound-disturbance navigation task in Isaac Sim, NVIDIA’s physics-accurate robotics simulator, where wind fields, actuation noise, control delay and sensor degradation were applied simultaneously. The results are striking. GDPB achieved a composite success rate of 96.47 percent, outperforming four established baselines: Proximal Policy Optimization, the workhorse of robotic reinforcement learning; Group Relative Policy Optimization, the method that has attracted attention for its role in reasoning models; Soft Actor-Critic, a maximum-entropy off-policy algorithm; and Twin Delayed Deep Deterministic Policy Gradient, a staple of continuous control. Just as important as the headline number is the consistency: GDPB exhibited a smaller standard deviation across repeated runs, indicating that its advantage is not a lucky seed but a reproducible property of the training process.

The gains were not uniform across conditions, and that pattern is itself informative. Wang and Zhao report that GDPB’s advantage over PPO became more pronounced under moderate-to-strong disturbances, exactly the regime where the batch-mean baseline is most miscalibrated and where state-dependent, disturbance-aware baselines should shine. In calm conditions, when all samples in a batch face similar environments, a simple average is a reasonable reference and the sophisticated baseline has less to correct. As the disturbance distribution widens, the spread of returns within a batch grows, the single-yardstick assumption fails harder, and GDPB’s per-state reference pulls further ahead. This dose-response relationship between disturbance intensity and method advantage lends credibility to the authors’ mechanistic explanation rather than suggesting an incidental benchmark win.

The work sits within a rapidly maturing research landscape. Deep reinforcement learning has already produced champion-level drone racing pilots and high-speed flight through unstructured wild environments, and robustness techniques such as domain randomization, dynamics randomization, adaptive adversaries and rapid motor adaptation have each chipped away at the sim-to-real gap. Yet most of these efforts address disturbances one at a time or treat robustness as a property to be baked into the policy through exposure to varied conditions. GDPB reframes the problem as one of statistical estimation: under compound disturbances, the learning signal itself must be denoised before a policy can extract reliable lessons from experience. That perspective connects drone navigation to a deep current in reinforcement learning theory, where variance reduction techniques for gradient estimates have been studied for two decades, and suggests that ideas from the estimation side of the field may be underexploited in robotics.

For the field at large, the study offers both a practical tool and a conceptual nudge. Practically, GDPB provides a drop-in improvement for policy-gradient training of aerial vehicles destined for messy, disturbance-rich environments, with the reassuring property that deployment-time behavior is unchanged. Conceptually, it demonstrates that privileged simulation information, so often used to shape policies directly, can be just as valuable when routed through the humble baseline that calibrates the learning signal. As drones move from curated test arenas into gusty cities, dusty fields and degraded sensor conditions, methods that make training stable under exactly those compound stresses may prove to be the quiet enabler of the next generation of autonomous flight. The authors note that the data supporting their findings are available from the corresponding author upon reasonable request, and the full details appear in Applied Intelligence, volume 56, article 482.

Subject of Research: Deep reinforcement learning for robust UAV navigation under compound disturbances

Article Title: GDPB: a gradient-decoupled privileged baseline for UAV navigation under compound disturbances

Article References: Wang, X., & Zhao, J. (2026). GDPB: a gradient-decoupled privileged baseline for UAV navigation under compound disturbances. Applied Intelligence, 56(15), Article 482. https://doi.org/10.1007/s10489-026-07533-2

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07533-2

Keywords: UAV navigation, deep reinforcement learning, compound disturbances, policy gradient, privileged information, variance reduction, PPO, GRPO, Isaac Sim, robust control, sim-to-real transfer, drone autonomy