The authors deliberately narrow the term adversarial. Rather than treating any risky environment as adversarial, they focus on scenarios involving competing teams of agents that must physically move through an environment to achieve their objectives. This framing captures everything from natural swarms evolving against predators to military and civilian security applications such as patrolling, target tracking, and interception. The review’s central insight is that many multiagent coordination problems can be understood as versions of task allocation, but adversarial scenarios stress two aspects of allocation that benign settings rarely do: the level of coupling between tasks, and the dynamic, strategic nature of the assignments themselves.

The coupling issue is fundamental. A common way of deploying a multiagent system is the divide-and-conquer approach, which allows spatially distributed tasks to be executed efficiently. In coverage control for mobile sensing networks, for example, each agent handles its own patch, and the success of one agent does not affect the success of its teammates. But when the overall group must work as a cohesive team to compete against another team, the task may not decompose into independent subtasks. Instead, agents may have to take specialized roles that augment each other, where the failure of one member can significantly affect the overall success of the team. The review distinguishes these two regimes with the terms coordination, for loosely coupled tasks whose contributions to team performance are linear or sublinear, and cooperation, for tightly coupled tasks where outcomes depend on joint action.

The second stress point is strategic. When the adversary is modeled as another decision-making entity, the problem is often formulated as a non-cooperative dynamic game. In such a formulation, the adversary can take advantage of static or open-loop assignment strategies, so the assignment must consider the evolution of system states as well as possible adversarial actions. This is one step beyond an assignment policy based on static analysis performed repeatedly over time. The authors categorize dynamic assignments as either passive, meaning re-assignments based on instantaneous optimality as the situation changes, or strategic, meaning assignments determined by anticipating the adversary’s decisions and how the engagement will unfold over time.

The review first examines the workhorse problems of patrolling, routing, and tracking. Patrolling involves systematically visiting designated areas to uphold security and monitor changes, and prevailing approaches often rely on pre-determined cyclic or partition strategies, sometimes extended to non-deterministic behaviors. Persistent surveillance has also been modeled as a vehicle routing problem. Yet successful patrolling teams demand adaptive decision-making, dynamically adjusting strategies based on unexpected events or threats. A specific research category centers on adversarial patrolling, where explicit attacker models and adversary detection are taken into account, allowing the development and analysis of defense mechanisms and countermeasure strategies that significantly enhance resilience against a wide array of potential attack scenarios.

Routing problems, where the objective is to determine time- or length-optimized paths to deliver information, goods, or services, undergo distinct formulations when posed adversarially. Researchers have considered the presence of hazardous zones in orienteering-style problems, and perimeter defense has emerged as an important task where the goal is to protect a region from targets attempting to enter or escape. Dynamic vehicle routing formulations have been proposed for a single agent capturing linearly translating targets, with the objective of intercepting the maximum number of targets en route before they reach the perimeter. The same concept has been extended to the multiagent case using adaptive partition strategies and network flows formulations for heterogeneous teams. In target tracking, which underlies surveillance, search and rescue, environmental monitoring, and military operations, a fundamental adversarial approach involves reconfiguring team displacement to restore impacted capabilities, and robust frameworks have been designed to withstand sensing and communication attacks or failures.

The heart of the review is differential games, where direct engagement between a red team and a blue team is modeled mathematically. A central example is the reach-avoid game, a variant of pursuit-evasion games in which a group of evaders, the attackers, seeks to reach a target area while a group of pursuers, the defenders, tries to minimize the number of breaches by intercepting them. The dominant solution approach is based on agent-agent assignments: the team-versus-team problem is approximated as a combination of one-versus-one problems solvable with traditional differential game techniques, and the pairwise outcomes are encoded into a bipartite graph. Classic algorithms such as the Hungarian algorithm, which solves maximum matching or minimum linear sum assignment problems in polynomial time, then determine which pursuer should go after which evader. Many variants of multiagent reach-avoid games use this decomposition, with changes in game geometry producing different low-level control strategies while the high-level team strategies all reduce coordination to pairwise assignments.

Progress on deeper teaming has come from breaking this pairwise mold. Researchers have considered how a coalition of pursuers can team up against a single evader, generalizing the assignment approach. In perimeter defense games, where defending agents are constrained to move along the perimeter of the target, a two-versus-one pincer movement was proposed, and this result was further used to construct bounded regions within which n-versus-m local games were defined. While most multiagent reach-avoid games assume individually greedy attacker strategies, this n-versus-m formulation showed how a subteam of attackers can concentrate force on a single point so that, when there are more attackers than defenders, a calculated number survive and score. The authors categorize this as force concentration, which has tighter coupling than force distribution because the success of a subteam relies on the cooperation of its members rather than on the sum of individual performances. A distinct type of teaming appears in the turret-runner-penetrator differential game, where a turret-like defender neutralizes attackers by turning and pointing at them. Under certain initial conditions, it is optimal for the attackers to split into two roles: the runner acts as a decoy to waste as much time as possible before the turret pursues the penetrator, who seeks to breach the perimeter. Such a sacrificial role is absent from the homogeneous task assignment seen in most reach-avoid games, hinting at more sophisticated teaming beyond divide and conquer.

At a higher abstraction level, the review considers Colonel Blotto games, in which two colonels each allocate a fixed budget of resources across a set of battlefields, and a battlefield is won by whoever commits more resources to it. One type of teaming that arises here is coalition formation: in three-player coalition Blotto games, two players each fight a common opponent while remaining selfish, yet each has an incentive to leverage the other’s actions to implicitly cooperate and beat the stronger player together. The dynamic attacker-defender Blotto game bridges motion control and resource allocation, played over multiple stages with sequential actions and a fundamental asymmetry: the attacker wins by achieving numerical advantage at any single node, whereas the defender must maintain advantage at all nodes simultaneously. Strategies that control platoons of defender resources require coordination to ensure no gap in defended nodes, a behavior the authors label formation maintenance.

Perhaps the most striking source of evidence comes from human team sports. Humans work together under limited information, mixed motives, and nonstationarity, and while human strategies may not always be optimal in the language of traditional game theory due to cognitive biases, bounded rationality, or the compromises of coalition-building, they work remarkably well. Recent advances in motion tracking technology now enable the collection of large quantities of player trajectory data, and machine learning techniques are being used to learn player-specific action templates and to quantify high-level concepts such as space-creating and space-exploiting behavior in professional soccer. The authors view team sports as representing the full diversity of behavior and complexity that robotic systems should attain in adversarial environments. The review concludes by identifying the frontiers where the challenge includes scaling game-theoretic strategies to large groups, enhancing teaming through specialized roles, and incorporating adversarial decision making into decentralized multiagent systems—steps toward robots that, like a well-drilled team, can anticipate an opponent and adapt together in real time.

Subject of Research: Multiagent systems and teaming behavior in adversarial scenarios

Article Title: Teaming behavior in adversarial scenarios

Article References: Shishika, D., Dorothy, M., & Macharet, D. G. (2023). Teaming behavior in adversarial scenarios. Frontiers in Control Engineering, 4, Article 1279454. https://doi.org/10.3389/fcteg.2023.1279454

Image Credits: AI Generated

DOI: 10.3389/fcteg.2023.1279454

Keywords: multiagent systems, adversarial scenarios, dynamic games, task assignment, pursuit evasion, team sports analytics, Teaming, behavior, adversarial, scenarios, scientific research

The scenario the researchers studied is a three-body engagement involving the target, the pursuer, and the defender. Once the defender is deployed, two coupled pursuit–evasion relationships emerge simultaneously: the pursuer chases the target, while the defender chases the pursuer in an attempt to spoil the interception. Because the defender bends the pursuer’s trajectory away from the target, the two friendly vehicles must act in a coordinated fashion rather than as independent agents. The difficulty is compounded by the fact that the pursuer may employ any of several established interception laws, including optimal control guidance, proportional navigation, or differential game guidance. A defense scheme tuned to a single assumed attack strategy can be defeated the moment the adversary switches tactics. At the same time, neither the target nor the defender has access to complete state information; both must rely on noisy local measurements of range and line-of-sight angles, which makes the problem one of partial observability as well as multi-strategy adversarial behavior.

Existing guidance approaches fall short in this setting for distinct reasons. Unilateral optimal control methods assume a known, fixed adversary model and cannot adapt when the pursuer changes strategy mid-engagement. Differential game formulations deliver elegant theoretical solutions but typically presume perfect information on both sides, an assumption that collapses in the presence of sensor noise and hidden intentions. Conventional reinforcement learning, meanwhile, has shown promise in adversarial settings, but standard algorithms struggle to converge when observations are incomplete and the opponent’s policy varies across encounters. The Sun Yat-sen team identified this triple challenge—unknown pursuit strategies, information deficiency, and high-maneuverability confrontation—as the key technological bottleneck standing in the way of practical, cooperative active defense for spacecraft.

To overcome it, the researchers reframed the three-body engagement as a multi-agent partially observable Markov decision process, or Multi-POMDP. In this formalism, each agent receives only a noisy local observation rather than the true global state, and the opponent’s policy is treated as uncertain and potentially drawn from a set of diverse strategies. The solution architecture is a reinforcement learning guidance framework built on an adaptive dueling double deep Q-network, abbreviated AD3QN. The central idea is that a single neural network learns, from experience, how to issue coordinated maneuver commands to both the target and the defender, so that the pair adapts on the fly to whatever interception strategy the pursuer happens to be executing. The framework separates perception from decision-making: first the raw, incomplete observation history is transformed into a compact situational representation, and then that representation drives the selection of acceleration commands.

The perception pipeline is a fusion of two well-established neural components. Current and historical incomplete observations are stacked along the time dimension into a two-dimensional tensor, which a convolutional neural network processes to extract spatial features—the critical geometric signatures of an unfolding engagement, such as the configuration of lines of sight among the three vehicles. A gated recurrent unit then models the temporal structure of those features, using its internal update and reset gates to produce a history tensor that encapsulates the trajectory characteristics of the encounter. This history encoding is what allows the policy to infer which pursuit strategy the adversary appears to be following, something no single-frame observation can reveal. To make training on partial information effective, the team restructured the experience replay buffer so that it stores the stacked observation tensor, the action taken, the history tensor, and the resulting reward, enabling the network to exploit temporal correlations during learning rather than treating each decision as an isolated snapshot.

The decision-making core of AD3QN builds on the dueling double deep Q-network formulation, in which the state value function and the action advantage function are estimated separately. This decomposition reduces the variance of value estimates, a property that proves crucial in multi-strategy environments where the consequences of a maneuver differ sharply depending on the adversary’s current policy. Equally important is the reward design. Rather than rewarding success only at the final moment of the engagement, the researchers constructed a continuous reward function based on potential-field differences computed from the zero-effort miss distance—the miss distance that would result if both vehicles stopped maneuvering. Before the defender–pursuer encounter, the reward landscape encourages the defender to shrink its zero-effort miss distance relative to the pursuer, driving it toward interception. After that encounter, the shaping switches, guiding the target to enlarge its zero-effort miss distance from the pursuer and thereby accomplish evasion. The authors provide a theoretical proof that this reward formulation does not alter the optimal policy, which means the shaping improves training stability without sacrificing optimality.

The numerical validation compared AD3QN against a demanding field of baselines, including Deep Deterministic Policy Gradient, Twin Delayed Deep Deterministic Policy Gradient, Proximal Policy Optimization, and Deep Recurrent Q-Learning. Under the combined stresses of multi-strategy adversaries and incomplete information, these mainstream algorithms all struggled to achieve stable policy optimization, whereas AD3QN converged reliably thanks to its fusion architecture and its explicit handling of observation history. Computational efficiency is a decisive factor for flight implementation, and here the method posted a striking result: a decision frequency of 85 Hz in a simulated single-chip microprocessor environment, roughly 30 percent faster than the DRQN comparison and comfortably within the real-time requirements of spacecraft actuation systems. In representative sample engagements, the defender successfully deflected the pursuer’s trajectory by approximately 20 meters relative to the target—a deviation large enough to convert a lethal interception into a clean miss.

Statistical robustness was assessed through Monte Carlo analysis. Across 1,000 randomized simulations, the proposed method achieved an evasion success rate of 99.7 percent, dramatically outperforming the optimal switching cooperative guidance law at 51.8 percent and various reinforcement learning baselines, which fell below 0.2 percent. The robustness tests are perhaps the most compelling evidence of practical value: even when observation noise was inflated to 40 times the typical level—with range noise of 400 meters and line-of-sight angle noise of 40 milliradians—the method still maintained an evasion success rate of 73.5 percent. Parameter sensitivity studies added further engineering insight. When the pursuer’s maneuvering capability increased, the target’s achievable miss distance fell by roughly 30 percent, but boosting the defender’s agility effectively compensated, improving evasion performance. This trade-off quantifies a design principle for future defensive architectures: investment in the defender’s maneuverability can offset a faster adversary.

The broader significance of the study lies in demonstrating that cooperative, adaptive active defense is achievable under realistic sensing conditions rather than idealized perfect information. By combining a Multi-POMDP problem formulation, a CNN–GRU fusion network for perception under partial observability, dueling double Q-learning for stable value estimation in adversarial settings, and a theoretically sound potential-field reward, the Sun Yat-sen team has assembled a guidance framework that is simultaneously adaptive, robust, and computationally light enough for onboard implementation. For operators of high-value satellites in congested orbital regions, the work suggests a path toward survivability that does not depend on predicting the attacker’s playbook in advance. As orbital confrontation scenarios grow more complex, methods of this kind—able to learn coordinated counter-interception from noisy observations and to withstand sensor degradation far beyond nominal levels—are likely to become a cornerstone of spacecraft autonomy and space security engineering.

Subject of Research: Active defense guidance for spacecraft in three-body pursuit–evasion engagements with incomplete information and multi-strategy adversaries

Article Title: Active defense guidance for spacecraft in multi-strategy engagement with incomplete information

Article References: Active defense guidance for spacecraft in multi-strategy engagement with incomplete information. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: spacecraft guidance, active defense, pursuit-evasion, reinforcement learning, deep Q-network, partial observability, Multi-POMDP, space security, zero-effort miss distance, convolutional neural network, gated recurrent unit, spacecraft survivability