Challenge accepted: High-speed AI drone overtakes world-champion drone racers

Remember when IBM’s Deep Blue won against Gary Kasparov at chess in 1996, or Google’s AlphaGo crushed the top champion Lee Sedol at Go, a much more complex game, in 2016? These competitions where machines prevailed over human champions are key milestones in the history of artificial intelligence. Now a group of researchers from the University of Zurich and Intel has set a new milestone with the first autonomous system capable of beating human champions at a physical sport: drone racing.

The AI system, called Swift, won multiple races against three world-class champions in first-person view (FPV) drone racing, where pilots fly quadcopters at speeds exceeding 100 km/h, controlling them remotely while wearing a headset linked to an onboard camera.

Learning by interacting with the physical world

“Physical sports are more challenging for AI because they are less predictable than board or video games. We don’t have a perfect knowledge of the drone and environment models, so the AI needs to learn them by interacting with the physical world,” says Davide Scaramuzza, head of the Robotics and Perception Group at the University of Zurich – and newly minted drone racing team captain.

Until very recently, autonomous drones took twice as long as those piloted by humans to fly through a racetrack, unless they relied on an external position-tracking system to precisely control their trajectories. Swift, however, reacts in real time to the data collected by an onboard camera, like the one used by human racers. Its integrated inertial measurement unit measures acceleration and speed while an artificial neural network uses data from the camera to localize the drone in space and detect the gates along the racetrack. This information is fed to a control unit, also based on a deep neural network that chooses the best action to finish the circuit as fast as possible.

Training in an optimised simulation environment

Swift was trained in a simulated environment where it taught itself to fly by trial and error, using a type of machine learning called reinforcement learning. The use of simulation helped avoid destroying multiple drones in the early stages of learning when the system often crashes. “To make sure that the consequences of actions in the simulator were as close as possible to the ones in the real world, we designed a method to optimize the simulator with real data,” says Elia Kaufmann, first author of the paper. In this phase, the drone flew autonomously thanks to very precise positions provided by an external position-tracking system, while also recording data from its camera. This way it learned to autocorrect errors it made interpreting data from the onboard sensors.

Human pilots still adapt better to changing conditions

After a month of simulated flight time, which corresponds to less than an hour on a desktop PC, Swift was ready to challenge its human competitors: the 2019 Drone Racing League champion Alex Vanover, the 2019 MultiGP Drone Racing champion Thomas Bitmatta, and three-times Swiss champion Marvin Schaepper. The races took place between 5 and 13 June 2022, on a purpose-built track in a hangar of the Dübendorf Airport, near Zurich. The track covered an area of 25 by 25 meters, with seven square gates that had to be passed in the right order to complete a lap, including challenging maneuvers including a Split-S, an acrobatic feature that involves half-rolling the drone and executing a descending half-loop at full speed.

Overall, Swift achieved the fastest lap, with a half-second lead over the best lap by a human pilot. On the other hand, human pilots proved more adaptable than the autonomous drone, which failed when the conditions were different from what it was trained for, e.g., if there was too much light in the room.

Pushing the envelope in autonomous flight is important way beyond drone racing, Scaramuzza notes. “Drones have a limited battery capacity; they need most of their energy just to stay airborne. Thus, by flying faster we increase their utility.” In applications such as forest monitoring or space exploration, for example, flying fast is important to cover large spaces in a limited time. In the film industry, fast autonomous drones could be used for shooting action scenes. And the ability to fly at high speeds could make a huge difference for rescue drones sent inside a building on fire.

Video available https://youtu.be/fBiataDpGIo

Literature:

Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, Davide Scaramuzza: Champion-Level Drone Racing using Deep Reinforcement Learning. Nature. 30 August 2023. DOI: 10.1038/s41586-023-06419-4

Contact:

Prof. Davide Scaramuzza
Robotics and Perception Group
Department of Informatics
University of Zurich
Phone: +41 44 635 24 09

Video available https://youtu.be/fBiataDpGIo