The core of the framework is a re-optimization subproblem the authors call the adaptive green vehicle routing problem, or AG-VRP. The green vehicle routing problem, originally developed in operations research to model fleets of alternative-fuel vehicles with limited range and mandatory refueling stops, assumes that every vehicle starts and ends at a single depot. That assumption collapses in a replanning scenario: when an agent is lost or a task set changes, the surviving agents are scattered across the mission area with varying amounts of fuel remaining, and they must continue from wherever they happen to be. The AG-VRP therefore allows arbitrary start locations for each agent, heterogeneous initial fuel states, and a single shared destination, while still requiring that every task be visited exactly once and that refueling depots be used as needed to keep each vehicle within its maximum travel range.

To guarantee that this subproblem can be solved to proven optimality when time permits, the researchers formulated it as a mixed-integer linear program. The formulation tracks binary decision variables indicating which arcs each agent traverses, continuous fuel variables that decrease with distance traveled and reset to maximum capacity at depots, and a sequence variable that eliminates subtours, the closed loops disconnected from the start and end nodes that plague vehicle routing formulations. Virtual depots, generated by copying each physical refueling location, allow depots to be visited multiple times by different agents. This mathematical model serves a dual role: it defines the theoretical lower bound against which faster methods are measured, and it provides optimal reference solutions in small problem instances where a commercial solver can exhaust the search space.

Optimality, however, is a luxury that mid-mission replanning cannot afford. The authors set a fixed computation budget of ten seconds per replanning epoch, reflecting the operational reality that a drone hovering with dwindling fuel cannot wait minutes for a solver. Within that budget, exact mixed-integer programming quickly becomes intractable as the number of tasks grows. The team therefore built a hierarchical solution suite. The first layer is a modified sequential greedy algorithm, a fast heuristic that assigns tasks to agents one at a time using a weighted cost that combines the distance to a candidate task with the distance from that task to the destination. The weighting factor discourages agents from picking up tasks near the end node early in their routes, which would otherwise produce inefficient gradual-approach behavior. The greedy algorithm also embeds explicit fuel logic: an agent only accepts a task if its remaining range can cover the trip to the task plus the worst-case distance onward to a depot or the destination, and it falls back to progressively less attractive tasks when the best candidate is fuel-infeasible.

The second and centerpiece layer is a reinforcement learning approach designed to overcome the myopia of greedy assignment. The authors cast the AG-VRP as a Markov decision process in which each decision step selects one agent and one node for it to visit next. The state captures every agent’s current location, remaining fuel, and partial trajectory, along with the set of unserved tasks. The reward is simply the negative of the total distance traveled by all agents, awarded only at the end of an episode, so maximizing expected return is mathematically equivalent to minimizing total travel distance. Training uses the REINFORCE gradient estimator with a rollout baseline, a technique in which a frozen copy of the policy serves as a comparison standard and is updated only when a paired statistical test confirms that the current policy has genuinely improved.

The neural architecture borrows from the attention models that have transformed combinatorial optimization in recent years. An encoder embeds all nodes of the problem, distinguishing depot nodes from task and start nodes and from the end node, and applies multi-head self-attention layers to capture the relationships among them. On the decoding side, the framework introduces two specialized modules: a vehicle selection decoder that chooses which agent should move next, and a node selection decoder that chooses where that agent should go. Crucially, each agent maintains a hidden state in a long short-term memory network that summarizes its entire partial trajectory, including refueling decisions, so the policy can reason about history rather than just the current snapshot. Remaining fuel is concatenated with the trajectory embedding before a linear layer produces the final vehicle representation, allowing the network to weigh range constraints directly when scoring candidate actions.

In deployment, the system does not gamble on a single method. At each replanning epoch, the mission snapshot, comprising agent positions, fuel levels, and remaining tasks, is converted into an AG-VRP instance, and both the greedy algorithm and the trained reinforcement learning policy run independently. The framework, dubbed RL plus MSGA, then selects whichever solution yields the lower total travel distance. Because the two methods have comparable runtimes, this selection strategy improves solution quality at essentially no additional computational cost. The case study illustrates the framework in action: three agents tasked with thirty tasks and supported by two refueling depots lose one of their number after ten tasks are completed, and the two survivors must absorb the remaining workload. The replanned routes produced by the exact solver, the learned policy, and the greedy heuristic are visually similar, but the reinforcement learning method achieves a smaller optimality gap than the greedy baseline.

The numerical experiments scale this evaluation dramatically, covering one thousand problem instances across six task counts from ten to one hundred and two fleet sizes, for a total of twelve thousand problem geometries. The results reveal a striking divide. As problem size grows, the proportion of instances for which the exact solver can even find a feasible solution within ten seconds falls sharply, whereas the proposed heuristics deliver feasible solutions in under one second with a one hundred percent success rate across all scenarios. On instances where the exact solver proves optimality, the reinforcement learning policy consistently lands closer to the optimum than the greedy method, and the combined strategy achieves the smallest gaps overall. Most remarkably, for the largest instances, where the solver struggles to find any high-quality feasible route, the learned policy sometimes matches or beats the solver’s output, producing negative optimality gaps.

Beyond raw distance, the study examined mission-oriented metrics that matter to operators. The reinforcement learning policy consistently leaves the largest fuel reserve at mission completion, indicating a more conservative routing style with a bigger safety margin, while the combined strategy minimizes the number of refueling operations without sacrificing route quality. Workload balance, measured by the standard deviation of tasks assigned to each agent, remained comparable across all methods, showing that the quality gains of the learned policy do not come at the price of overburdening individual vehicles. Even for one-hundred-task instances where optimal solutions could still be found, the combined approach kept optimality gaps below ten percent, a level of performance the authors attribute to the complementary strengths of learned global structure and fast greedy feasibility.

The implications extend beyond military surveillance and reconnaissance, the scenarios that motivate much of this line of research. The same mathematics applies to planetary surface exploration, where in-situ resource utilization stations serve as refueling depots for rovers, and to green logistics fleets of electric or alternative-fuel delivery vehicles that must recharge mid-route. The authors note that future work could incorporate kinematic path constraints such as minimum turning radius, which would replace straight-line distances with Dubins paths and make travel cost dependent on vehicle heading, as well as time windows, agent-specific tasks, and inter-agent collaboration. For now, the study demonstrates that a carefully trained attention-based policy, paired with a fuel-aware greedy fallback, can keep a fuel-constrained drone team flying coherently through the kind of mid-mission shocks that would ground a statically planned fleet.

Subject of Research: Real-time replanning of fuel-constrained multi-UAV missions using reinforcement learning and greedy heuristics

Article Title: Adaptive Planning for Multiple UAVs with In-Flight Refueling

Article References: Gwon, B., Choi, E., Lee, J., & Ahn, J. (2026). Adaptive Planning for Multiple UAVs with In-Flight Refueling. International Journal of Aeronautical and Space Sciences. https://doi.org/10.1007/s42405-026-01249-w

Image Credits: AI Generated

DOI: 10.1007/s42405-026-01249-w

Keywords: UAV, reinforcement learning, vehicle routing problem, in-flight refueling, rolling-horizon replanning, attention model, mixed-integer linear programming, greedy algorithm, LSTM, dynamic environments, fuel constraints, KAIST