The starting point is the so-called W-shaped development process, proposed by the European Union Aviation Safety Agency together with the company Daedalean in their Concepts of Design Assurance for Neural Networks reports. The classic V-model, which underpins standards such as DO-178C in avionics, ISO 26262 in automotive engineering, and the ECSS framework in aerospace, arranges development as two arms: requirements are refined into code on one side, and hierarchical testing verifies them on the other. The W-shaped process extends this with a second V dedicated to what EASA calls learning assurance—verifying that a learned model generalizes to unseen operational data and behaves robustly within its Operational Design Domain, the envelope of conditions in which it is meant to operate. For supervised learning, where datasets with labels exist before training begins, the process works. But reinforcement learning is different, and that difference matters.

In reinforcement learning, an agent learns by interacting with its environment: it observes a state, takes an action, and receives a reward that reflects how good the action was for achieving its goal. There are no ground-truth labels to check outputs against, and—crucially—there is no dataset before training. Training data is generated by the very interactions that produce learning. The researchers, led by Ángel-Grover Pérez-Muñoz of Universidad Politécnica de Madrid and colleagues, found that this single characteristic fractures the W-shaped process at its foundation. The process assumes data management precedes learning; in reinforcement learning the two are inseparable. EASA itself has deferred guidance on reinforcement learning to 2028, which makes an empirical test of the framework both timely and rare.

To probe the limits, the team designed a use case that is as much a safety exercise as an AI challenge. A defender drone, controlled entirely by a neural-network reinforcement learning agent, must follow a predefined sequence of waypoints through a simulated urban landscape built in Microsoft’s AirSim simulator, while an adversary drone tries to ram it. Three attacker strategies escalate in difficulty: a deterministic pursuer that simply chases the defender’s last known position; a predictive pursuer that estimates the defender’s position one second ahead with random scaling to mimic sensor error; and a still harder variant that randomizes its prediction horizon between one and two seconds. The defender must reach each waypoint within three meters, maintain at least 1.25 meters of separation from the attacker, and repeat this under waypoint altitudes it never saw during training.

The technical machinery underneath is careful and deliberate. The agent’s state is a twelve-dimensional vector built entirely from relative quantities—distance to the attacker, distance to the next waypoint, relative velocity, and the defender’s own velocity—so that the learned policy is invariant to where the drone happens to be in the world. Actions are continuous velocity commands on three axes, bounded at seven meters per second. The reward function stacks a cubic distance penalty, large bonuses for reaching waypoints and completing the course, and steep penalties for collisions, proximity violations, and wasted time. Training used the Soft Actor-Critic algorithm with four-layer actor and critic networks of 512 neurons each; notably, the team found SAC outperformed Proximal Policy Optimization because it prioritized minimizing penalties over chasing rewards—an intuitive fit for a system whose primary duty is not crashing.

The results carry a cautionary lesson about overtraining. An agent trained for 400,000 steps against the simple deterministic attacker achieved success rates between roughly 75 and 81 percent against all three attackers, including strategies it had never encountered, evaluated over 1,100 episodes across eleven random seeds. Its sibling trained for a full million steps on the same attacker fared dramatically worse, dropping as low as 27 percent against the predictive pursuers—a statistically significant collapse, confirmed with bootstrap tests of ten million resamples. Extended training, it appears, caused the agent to overspecialize on the deterministic chase pattern. Agents trained directly against the harder attackers did no better, with one achieving near-zero success. The lesson for safety engineers is stark: more compute and more training do not guarantee more general, more trustworthy behavior, and the certification process must be able to catch this.

What elevates the study beyond a simulation exercise is what happened next. The actor network—the part of the agent that turns observations into actions—was automatically translated from Python into C code compliant with MISRA-C, the conservative subset of C used in safety-critical software, which forbids dynamic memory allocation and thereby eliminates memory leaks, heap fragmentation, and unpredictable timing. The code was deployed on a Zynq UltraScale+ MPSoC board under the XtratuM hypervisor, which partitions the hardware in both space and time so that a fault in the AI component cannot propagate to other subsystems. Because the deployed agent must never learn in operation, only the frozen policy flies; the critic networks stay behind.

The embedded system passed its examinations. Running at a required control period of 200 milliseconds with a 100-millisecond deadline, static worst-case execution time analysis using the OTAWA tool bounded a single inference at 38 milliseconds, while 1,500 measured inferences averaged 23.62 milliseconds with a maximum of 71.18 milliseconds—comfortably inside the deadline even with interference from other tasks. Numerical comparison between the Python original and the C implementation, necessarily reduced from 64-bit to 32-bit floating point, showed errors on the order of one hundred-thousandth of a meter per second, negligible against an action range of fourteen meters per second. Success rates on the hardware—79.1, 81.4, and 79.6 percent against the three attackers—fell within the confidence intervals of the original model, demonstrating that the certified translation did not silently degrade the learned behavior.

The broader significance lies in the mapping the researchers drew between reinforcement learning’s needs and the W-shaped process’s stages. They found that the entire development-assurance half of the process—implementation, integration, and verification on real hardware—is agnostic to the learning technique and applied cleanly. The learning-assurance half required reinvention: instead of predefined datasets, the team defined scenarios derived from the Operational Design Domain to govern data generation; data quality requirements such as representativeness and completeness, which normally apply to whole datasets, were reinterpreted as properties of those scenarios; and a new step was inserted to define the agent-environment interface—the states, actions, and reward function—before training could begin.

This is, by the authors’ account, the first application of EASA’s W-shaped process to a reinforcement learning system, and it arrives as EUROCAE and SAE finalize the ARP6983/ED-324 standard for aeronautical AI. The work suggests a realistic path forward: certification frameworks built for supervised learning can stretch to accommodate learning agents that discover their own strategies, provided the emphasis shifts from curating data to curating the scenarios that generate it. The researchers caution that their results live in simulation, and closing the sim-to-real gap—sensor noise, communication delays, environmental perturbations—remains future work, as does confronting learning-based attackers. But for a field waiting on regulators until 2028, the demonstration that a reinforcement learning drone can be trained, certified, and embedded within real-time safety constraints is a milestone worth more than its 71-millisecond inference time might suggest.

Subject of Research: Applying the EASA W-shaped development process to reinforcement learning for safe drone navigation and collision avoidance

Article Title: Application of the W-shaped process for a Reinforcement Learning use case on drone navigation

Article References: Pérez-Muñoz, Á.-G., García-Quijano, H., López-García, G., Alonso, A., & Pérez, M. S. (2026). Application of the W-shaped process for a Reinforcement Learning use case on drone navigation. Machine Learning with Applications, 26, Article 101009. https://doi.org/10.1016/j.mlwa.2026.101009

Image Credits: AI Generated

DOI: 10.1016/j.mlwa.2026.101009

Keywords: reinforcement learning, drone navigation, aviation safety, EASA, W-shaped process, machine learning certification, neural networks, embedded systems, collision avoidance, Soft Actor-Critic, operational design domain, safety-critical systems

The core challenge the researchers set out to solve is deceptively simple to state: given a start point and a goal in a cluttered 3D environment, generate a trajectory that is short, smooth, dynamically feasible, and energy-efficient, all at the same time. These objectives frequently conflict with one another. The shortest path may hug obstacles so tightly that a real aircraft could not safely follow it. The smoothest path may waste energy in wide, sweeping arcs. Traditional optimization methods can balance these goals but often struggle in unknown or changing environments, where they must be re-run from scratch whenever conditions shift.

Deep reinforcement learning has emerged in recent years as an attractive end-to-end alternative. In this paradigm, an artificial intelligence agent learns to fly by trial and error, receiving rewards for progress toward the goal and penalties for collisions, erratic motion, or excessive energy use. Over many training episodes, the agent internalizes a policy that maps what it observes directly to the actions it should take. The appeal is obvious: once trained, such a policy can react to new situations in milliseconds without recomputing a full trajectory. The drawback, as the new paper emphasizes, is that learning from scratch is painfully slow. Random exploration in a vast three-dimensional action space means the agent spends most of its early training bumping into obstacles or wandering aimlessly, a problem the authors describe as blind exploration combined with low sample efficiency.

To inject common sense into this process, the team turned to the artificial potential field, a concept that has guided robot navigation since the 1980s. In an artificial potential field, the goal exerts an attractive force that pulls the vehicle toward it, while obstacles exert repulsive forces that push it away. The drone is imagined as a ball rolling downhill on a landscape sculpted by these forces, and at every instant the field suggests a sensible direction of travel. The elegance of the approach is that the guidance is state-dependent: as the drone’s position and surroundings change, the forces change with them, always pointing toward safer, more productive regions of space.

Rather than replacing the learning algorithm with this classical method, the researchers fused the two. The state-dependent forces generated by the artificial potential field are integrated directly with the policy of a Soft Actor–Critic agent, a state-of-the-art deep reinforcement learning algorithm prized for its stability and its ability to balance exploration against exploitation. Soft Actor–Critic maximizes both the expected reward and the entropy, or randomness, of the agent’s behavior, which prevents it from collapsing prematurely into a mediocre strategy. By blending the potential field’s heuristic push into the action-selection process, the hybrid system no longer explores blindly. From the very first training episode, the agent is nudged in directions that the physics suggests are promising, while retaining the freedom to deviate when the heuristic is wrong, as it can be in local minima where attractive and repulsive forces cancel out.

The second innovation addresses a different weakness: memory. Standard actor–critic networks treat each moment in isolation, deciding what to do based only on the current observation. A flying vehicle, however, is a dynamical system whose future depends on its recent past. A sudden change in heading that was perfectly safe at low speed may be catastrophic at high speed, and the network cannot know the difference if it has no access to the sequence of states that led to the present moment. To capture these temporal dependencies, the authors embedded a long short-term memory, or LSTM, structure into both the actor and the critic networks. LSTMs are recurrent neural networks equipped with gating mechanisms that allow them to retain information over many time steps and to forget what is no longer relevant, giving the agent an effective working memory of its own flight history.

The practical consequence of this architectural choice is smoother, more dynamically feasible trajectories. Because the policy can perceive trends, accelerations, and oscillations in the state sequence rather than single snapshots, it learns to produce control commands that flow naturally from one to the next, avoiding the jerky, high-frequency corrections that plague memoryless policies and that translate directly into wasted energy and mechanical stress on real airframes. The combination of potential field guidance and recurrent memory gives the method its name: APF–LSTM–SAC.

The team validated the approach in comprehensive tests within complex simulated three-dimensional environments, comparing it against pure deep reinforcement learning baselines. The results were striking. The hybrid method achieved improvements in convergence speed of at least 15.14 percent, meaning the agent reached competent flight policies substantially faster than its unguided counterparts, and it reduced energy consumption by at least 10.36 percent, a figure that matters enormously for battery-powered aircraft whose mission endurance is measured in minutes. Faster training also carries a practical dividend: fewer simulated flight hours are needed before a policy is deployable, which lowers the computational cost of developing autonomous capabilities for new vehicle types or new environments.

The significance of the work extends beyond the specific percentages. It exemplifies a growing trend in robotics toward physics-informed machine learning, in which decades of classical control theory and heuristic reasoning are used to scaffold, rather than be replaced by, modern data-driven methods. Pure learning systems must rediscover from scratch lessons that engineers already know, such as the fact that obstacles should be avoided and goals approached. By encoding those lessons as inductive biases inside the learning pipeline, researchers can preserve the adaptability of reinforcement learning while dramatically shrinking the search space the algorithm must explore. The potential field component supplies a sensible prior; the LSTM supplies temporal awareness; and the Soft Actor–Critic framework supplies robust, entropy-regularized optimization that can gracefully reconcile the two.

The authors note that the datasets generated and analyzed during the study are available from the corresponding author on reasonable request, and the work was supported by the National Natural Science Foundation of China, the CAAC Key Laboratory of General Aviation Operation, and the Fundamental Research Funds for the Central Universities. As drones take on ever more ambitious roles, from delivering medical supplies to surveying disaster zones, the ability to plan safe, efficient three-dimensional trajectories autonomously will only grow in importance. Hybrid approaches like APF–LSTM–SAC suggest that the fastest route to capable autonomous flight may not be to make learning algorithms bigger, but to make them wiser, by letting the accumulated physics of navigation light the way through the darkness of blind exploration.

Subject of Research: UAV three-dimensional path planning using artificial potential field guidance and an LSTM-enhanced Soft Actor–Critic deep reinforcement learning algorithm

Article Title: UAV 3D Path Planning Based on Artificial Potential Field Guidance and LSTM-Enhanced Soft Actor–Critic

Article References: Liu, J., Zhou, H., Lei, X., Tu, X., Hua, H., & Wang, X. (2026). UAV 3D Path Planning Based on Artificial Potential Field Guidance and LSTM-Enhanced Soft Actor–Critic. International Journal of Aeronautical and Space Sciences. https://doi.org/10.1007/s42405-026-01292-7

Image Credits: AI Generated

DOI: 10.1007/s42405-026-01292-7

Keywords: UAV path planning, deep reinforcement learning, artificial potential field, Soft Actor–Critic, long short-term memory, drone navigation, trajectory optimization, energy efficiency, 3D path planning, autonomous flight, machine learning, robotics