Certifying drone-brain AI: aviation’s W-shaped safety process put to the reinforcement learning test

Safety-critical software has long been built on a simple assumption: the behavior of a program follows directly from its code. Regulators could inspect control variables, branches, and loops, and certify that a system would do what it was designed to do. Machine learning breaks that assumption, because behavior is not written but learned. Now a team of Spanish researchers has taken one of the most consequential questions in AI safety—can the aviation industry’s emerging certification framework cope with reinforcement learning, the technique behind some of the most capable autonomous systems—and answered it by building a drone that must survive a determined attacker in a simulated city.

The starting point is the so-called W-shaped development process, proposed by the European Union Aviation Safety Agency together with the company Daedalean in their Concepts of Design Assurance for Neural Networks reports. The classic V-model, which underpins standards such as DO-178C in avionics, ISO 26262 in automotive engineering, and the ECSS framework in aerospace, arranges development as two arms: requirements are refined into code on one side, and hierarchical testing verifies them on the other. The W-shaped process extends this with a second V dedicated to what EASA calls learning assurance—verifying that a learned model generalizes to unseen operational data and behaves robustly within its Operational Design Domain, the envelope of conditions in which it is meant to operate. For supervised learning, where datasets with labels exist before training begins, the process works. But reinforcement learning is different, and that difference matters.

In reinforcement learning, an agent learns by interacting with its environment: it observes a state, takes an action, and receives a reward that reflects how good the action was for achieving its goal. There are no ground-truth labels to check outputs against, and—crucially—there is no dataset before training. Training data is generated by the very interactions that produce learning. The researchers, led by Ángel-Grover Pérez-Muñoz of Universidad Politécnica de Madrid and colleagues, found that this single characteristic fractures the W-shaped process at its foundation. The process assumes data management precedes learning; in reinforcement learning the two are inseparable. EASA itself has deferred guidance on reinforcement learning to 2028, which makes an empirical test of the framework both timely and rare.

To probe the limits, the team designed a use case that is as much a safety exercise as an AI challenge. A defender drone, controlled entirely by a neural-network reinforcement learning agent, must follow a predefined sequence of waypoints through a simulated urban landscape built in Microsoft’s AirSim simulator, while an adversary drone tries to ram it. Three attacker strategies escalate in difficulty: a deterministic pursuer that simply chases the defender’s last known position; a predictive pursuer that estimates the defender’s position one second ahead with random scaling to mimic sensor error; and a still harder variant that randomizes its prediction horizon between one and two seconds. The defender must reach each waypoint within three meters, maintain at least 1.25 meters of separation from the attacker, and repeat this under waypoint altitudes it never saw during training.

The technical machinery underneath is careful and deliberate. The agent’s state is a twelve-dimensional vector built entirely from relative quantities—distance to the attacker, distance to the next waypoint, relative velocity, and the defender’s own velocity—so that the learned policy is invariant to where the drone happens to be in the world. Actions are continuous velocity commands on three axes, bounded at seven meters per second. The reward function stacks a cubic distance penalty, large bonuses for reaching waypoints and completing the course, and steep penalties for collisions, proximity violations, and wasted time. Training used the Soft Actor-Critic algorithm with four-layer actor and critic networks of 512 neurons each; notably, the team found SAC outperformed Proximal Policy Optimization because it prioritized minimizing penalties over chasing rewards—an intuitive fit for a system whose primary duty is not crashing.

The results carry a cautionary lesson about overtraining. An agent trained for 400,000 steps against the simple deterministic attacker achieved success rates between roughly 75 and 81 percent against all three attackers, including strategies it had never encountered, evaluated over 1,100 episodes across eleven random seeds. Its sibling trained for a full million steps on the same attacker fared dramatically worse, dropping as low as 27 percent against the predictive pursuers—a statistically significant collapse, confirmed with bootstrap tests of ten million resamples. Extended training, it appears, caused the agent to overspecialize on the deterministic chase pattern. Agents trained directly against the harder attackers did no better, with one achieving near-zero success. The lesson for safety engineers is stark: more compute and more training do not guarantee more general, more trustworthy behavior, and the certification process must be able to catch this.

What elevates the study beyond a simulation exercise is what happened next. The actor network—the part of the agent that turns observations into actions—was automatically translated from Python into C code compliant with MISRA-C, the conservative subset of C used in safety-critical software, which forbids dynamic memory allocation and thereby eliminates memory leaks, heap fragmentation, and unpredictable timing. The code was deployed on a Zynq UltraScale+ MPSoC board under the XtratuM hypervisor, which partitions the hardware in both space and time so that a fault in the AI component cannot propagate to other subsystems. Because the deployed agent must never learn in operation, only the frozen policy flies; the critic networks stay behind.

The embedded system passed its examinations. Running at a required control period of 200 milliseconds with a 100-millisecond deadline, static worst-case execution time analysis using the OTAWA tool bounded a single inference at 38 milliseconds, while 1,500 measured inferences averaged 23.62 milliseconds with a maximum of 71.18 milliseconds—comfortably inside the deadline even with interference from other tasks. Numerical comparison between the Python original and the C implementation, necessarily reduced from 64-bit to 32-bit floating point, showed errors on the order of one hundred-thousandth of a meter per second, negligible against an action range of fourteen meters per second. Success rates on the hardware—79.1, 81.4, and 79.6 percent against the three attackers—fell within the confidence intervals of the original model, demonstrating that the certified translation did not silently degrade the learned behavior.

The broader significance lies in the mapping the researchers drew between reinforcement learning’s needs and the W-shaped process’s stages. They found that the entire development-assurance half of the process—implementation, integration, and verification on real hardware—is agnostic to the learning technique and applied cleanly. The learning-assurance half required reinvention: instead of predefined datasets, the team defined scenarios derived from the Operational Design Domain to govern data generation; data quality requirements such as representativeness and completeness, which normally apply to whole datasets, were reinterpreted as properties of those scenarios; and a new step was inserted to define the agent-environment interface—the states, actions, and reward function—before training could begin.

This is, by the authors’ account, the first application of EASA’s W-shaped process to a reinforcement learning system, and it arrives as EUROCAE and SAE finalize the ARP6983/ED-324 standard for aeronautical AI. The work suggests a realistic path forward: certification frameworks built for supervised learning can stretch to accommodate learning agents that discover their own strategies, provided the emphasis shifts from curating data to curating the scenarios that generate it. The researchers caution that their results live in simulation, and closing the sim-to-real gap—sensor noise, communication delays, environmental perturbations—remains future work, as does confronting learning-based attackers. But for a field waiting on regulators until 2028, the demonstration that a reinforcement learning drone can be trained, certified, and embedded within real-time safety constraints is a milestone worth more than its 71-millisecond inference time might suggest.

Subject of Research: Applying the EASA W-shaped development process to reinforcement learning for safe drone navigation and collision avoidance

Article Title: Application of the W-shaped process for a Reinforcement Learning use case on drone navigation

Article References: Pérez-Muñoz, Á.-G., García-Quijano, H., López-García, G., Alonso, A., & Pérez, M. S. (2026). Application of the W-shaped process for a Reinforcement Learning use case on drone navigation. Machine Learning with Applications, 26, Article 101009. https://doi.org/10.1016/j.mlwa.2026.101009

Image Credits: AI Generated

DOI: 10.1016/j.mlwa.2026.101009

Keywords: reinforcement learning, drone navigation, aviation safety, EASA, W-shaped process, machine learning certification, neural networks, embedded systems, collision avoidance, Soft Actor-Critic, operational design domain, safety-critical systems