AI Learns to Predict a Fleeing Spacecraft’s Moves in Orbital Chase Games

In the increasingly crowded corridors of Earth orbit, spacecraft are no longer simply passing neighbors. As orbital resources grow scarce and strategic interests in space intensify, scenarios in which one spacecraft must chase down another have moved from science fiction into the realm of serious engineering research. Among the most demanding of these scenarios is the multiple-to-one orbital pursuit-evasion game, in which several pursuing spacecraft must work together to capture a single, non-cooperative target that is actively trying to escape. A new study published in Space: Science & Technology by a team led by Bai Chengchao of the School of Astronautics at Harbin Institute of Technology now offers a fresh answer to one of the central puzzles of this problem: how can pursuers anticipate what an evader will do next, and use that foresight to coordinate their own moves more effectively?

The challenge is formidable. Multiple pursuit spacecraft must cooperatively capture an evasive target while operating under impulsive maneuver constraints and strict fuel budgets, producing an optimal control problem marked by strong nonlinearity and complicated constraints. The researchers note that the tools traditionally brought to bear on such problems each fall short in some way. Classical differential game methods suffer from high computational complexity and poor real-time performance, making them difficult to deploy on hardware with limited processing power. Numerical optimization approaches, meanwhile, are notoriously sensitive to initial guesses and struggle to handle constraints that persist over long durations. Deep reinforcement learning has shown genuine promise in some orbital scenarios, but existing algorithms lack the ability to proactively predict the actions of an opponent confronting them with an unknown evasion strategy, which limits how well the pursuit can be planned. The key question, therefore, became how to predict the evading spacecraft’s actions in multi-to-one scenarios and then optimize the cooperative strategies of the pursuers accordingly.

The team’s answer is a pursuit strategy construction method that fuses an evader action prediction network with the Deep Deterministic Policy Gradient algorithm, a well-established deep reinforcement learning technique for continuous control. The mathematical foundation of the work rests on the Clohessy-Wiltshire equations, the standard linearized description of relative spacecraft motion, which the researchers use to build a model of the multi-to-one impulsive orbital pursuit-evasion game in the Local Vertical-Local Horizontal reference frame. In this modeled world, both sides execute impulsive velocity-increment maneuvers at fixed time intervals, and each side faces constraints on both the size of each individual maneuver and the cumulative total velocity increment it may expend. Multiple pursuers cooperatively close in on the evader through finite sequences of these impulses, and the mission is declared a success the moment the distance between any pursuer and the evader falls below a preset capture threshold. Communication among the pursuers is itself constrained: information can be shared only within a certain proximity, adding a further layer of realism to the problem.

To train the pursuit policy, the researchers adopted a centralized training with decentralized execution framework, an architecture that has become popular in multi-agent reinforcement learning. During training, a single policy network is updated using global state information, but at execution time each pursuer relies only on its own local observations to generate impulsive maneuver commands. Crucially, all pursuers share the same network parameters, meaning that a fleet of chasing spacecraft can behave in a coordinated manner without needing a central commander in the loop during the actual engagement. The observation available to each pursuer is rich: it comprises the relative state with respect to the evader, the states of the two nearest fellow pursuers within communication range, and the predicted future state of the evader at the next time step, produced by the prediction network, for a total of 29 dimensions. The policy network maps this observation to a three-dimensional continuous impulsive velocity-increment command.

The reward structure that shapes the learning process is deliberately simple but effective. It combines a continuous reward for approaching the evader, a large constant reward for a successful capture, and an additional penalty for fuel consumption, so that the pursuing agents learn to be aggressive yet economical. Alongside the policy network, the team trained a separate evader action prediction network through supervised learning. Its input consists of the relative states of the three pursuers closest to the evader, and its output is a prediction of the evader’s velocity increment, effectively giving each pursuer a glimpse of the opponent’s likely next move. Importantly, this prediction network is trained using the global state information and the evader’s true action data collected during the pursuit policy training itself, so no additional data acquisition is required. All of the networks involved, including the policy network, the Q-value network, and the prediction network, are four-layer fully connected architectures in which the first three layers use the ReLU activation function and the output layer employs tanh.

The simulation campaign that validated the approach produced striking results. Over 200,000 training episodes, the reward trends of the pursuer and evader policies ran in generally opposite directions and stabilized after roughly 125,000 episodes, a signature that both sides had reached an approximate game equilibrium. When the researchers compared training curves for the pursuit policy with and without evader action prediction, the incorporation of prediction significantly improved both the convergence speed and the reward values achieved, while the loss curve of the prediction network itself converged after about 1,000 epochs. In test engagements, trajectories produced under three different prediction approaches, the learned prediction network, conventional Kalman filtering, and simply assuming the evader repeats its previous action, all achieved successful capture, but their accuracy differed markedly.

The prediction network proved decisively superior in tracking the evader’s future position. Its prediction error remained essentially below 0.3 kilometers throughout the pursuit-evasion process, with an average of just 0.137 kilometers, and it gradually decreased in the later stage of the engagement as the pursuers closed in. By contrast, the errors of the comparative methods approached 0.6 kilometers in the later stages, more than four times worse. The statistical picture was equally compelling. With three pursuers deployed, the capture success rate of the prediction-augmented method reached 0.95, an improvement of approximately 0.3 over the conventional DDPG approach. The researchers highlight that this advantage effectively compensates for the disadvantage of insufficient pursuer numbers, a point of real practical significance in a domain where every additional spacecraft carries substantial cost.

The study is also candid about the limits of the method. When the evader adopted a non-maneuvering strategy, the capture success rate climbed to 0.99, but performance degraded when the pursuers confronted unseen random or periodic evasion strategies, exposing a generalization bottleneck that is common to learning-based policies. The geometry of the opening position mattered as well: the initial distribution of the pursuers significantly affected the success rate, with an optimal initial distance of approximately 38 kilometers and an optimal enclosure area of approximately 2,300 square kilometers. The authors acknowledge that the approach currently relies on prior action data of the evader, and they outline future work on prediction methods that do not require such data, as well as extension to the even harder multi-to-many pursuit-evasion game problem.

For the wider space community, the significance of this work lies in its demonstration that foresight and learning can be combined to produce autonomous, fuel-aware, and cooperative pursuit behavior under realistic constraints. As activities in orbit multiply, from Mega-constellations to debris removal and on-orbit servicing, the ability of spacecraft to make fast, intelligent decisions in adversarial or uncertain encounters is poised to become a core capability. By showing that a supervised prediction of an opponent’s maneuver can be woven directly into a reinforcement learning pursuit policy, and by quantifying exactly how much that foresight is worth in terms of convergence speed and capture success, the Harbin Institute of Technology team has provided both a working method and a meaningful engineering reference for the autonomous decision-making systems that future space missions will demand.

The reliance on the Clohessy-Wiltshire equations places this work within a long tradition of orbital relative-motion analysis. These linearized equations assume a circular reference orbit and short engagement distances, which keeps the dynamics tractable enough for learning-based methods while remaining a reasonable approximation for proximity operations. The choice of impulsive maneuvers, rather than continuous thrust, mirrors the pulsed thruster firings common on real spacecraft, where each burn is short compared with the orbital period and fuel is measured in velocity increment.

The decentralized execution aspect of the training framework carries practical weight for on-orbit deployment. Because each pursuer generates its maneuver commands from local observations alone, the approach tolerates the intermittent communication links and latency that plague multi-spacecraft operations, where a central coordinator could become a single point of failure. The communication-range constraint built into the model reflects this operational reality.

The comparison against Kalman filtering is also instructive. Kalman filters estimate a target’s state from noisy measurements but do not anticipate deliberate maneuvering, so their predictions degrade precisely when an evader fires its thrusters. A learned predictor trained on maneuver data can instead capture behavioral patterns that a purely dynamical filter cannot represent. The reported average prediction error of 0.137 kilometers, against roughly 0.6 kilometers for the alternatives late in the engagement, quantifies this advantage in terms familiar to guidance engineers.

The acknowledged dependence on prior evader action data remains the principal caveat, since real adversaries may behave in ways no training set anticipated.

Subject of Research: A predictive learning-based pursuit strategy for multiple-to-one orbital pursuit-evasion games among spacecraft.

Article Title: A predictive learning-based pursuit strategy for the multiple-to-one orbital pursuit-evasion game

Article References: A predictive learning-based pursuit strategy for the multiple-to-one orbital pursuit-evasion game. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: orbital pursuit-evasion, spacecraft, deep reinforcement learning, DDPG, action prediction, Clohessy-Wiltshire equations, impulsive maneuvers, CTDE, autonomous decision-making, space security, Harbin Institute of Technology, Space Science and Technology