The core difficulty lies in the mismatch between how standard reinforcement learning agents learn and how a moving charging vehicle actually behaves. In the classic SARSA algorithm, an agent learns by trial and error, updating the value of state-action pairs as it experiences rewards and penalties. That works reasonably well when the world is static, because the consequences of a given action in a given cell of a grid map stay consistent across episodes. But a mobile charging vehicle follows irregular motion patterns that are difficult to predict, which means the reward landscape itself shifts beneath the learner. The researchers identify three intertwined failure modes in this setting: exploration is inefficient because the agent wastes effort sampling regions of the environment that no longer matter, the vehicle’s motion is irregular enough to defeat simple models, and prediction errors about where the charger will be grow large enough to poison the learned policy.

ASDE, which stands for adaptive SARSA for dynamic environments, attacks all three problems at once. The first ingredient is an explicit motion model of the mobile charging vehicle woven directly into the learning framework, so that the drone’s planner has a structured expectation of how the charger moves rather than treating its position as an arbitrary, unknowable quantity. The second is a restructured reward function that reshapes the feedback the agent receives, steering it toward trajectories that are not merely successful but also efficient and smooth. Together these changes give the learning process a much better-shaped objective, reducing the amount of random wandering the agent must do before it discovers useful behavior.

The third ingredient is perhaps the most conceptually interesting: a time-varying epsilon-greedy strategy. In textbook reinforcement learning, epsilon-greedy exploration means the agent takes a random action with a fixed probability epsilon and otherwise exploits its current knowledge. A fixed epsilon is a blunt instrument. Too high, and the agent never settles into the good policy it has found; too low, and it stops exploring before it has found one. ASDE instead adapts the exploration rate continuously according to environmental feedback, exploring more aggressively when the situation is uncertain or the charger’s motion has invalidated prior assumptions, and exploiting more heavily when the environment appears stable and the learned value estimates are trustworthy. This dynamic balance is what allows the algorithm to remain responsive in a setting where the target of the entire mission is itself in motion.

The fourth component extends the algorithm’s memory of its own trajectory. Standard SARSA is a one-step temporal difference method: each update looks only one step into the past. ASDE employs a hybrid temporal difference lambda mechanism with multi-step backtracking, which propagates credit backward across a stretch of recent states and actions rather than a single transition. Crucially, the effective backtracking horizon is not fixed. It adjusts dynamically based on the mobile charging vehicle’s instantaneous motion, stretching out when the charger’s behavior is predictable and contracting when it changes abruptly. The researchers report that this adaptive backtracking enhances both predictive accuracy and responsiveness, allowing the drone’s value estimates to track a moving target without the lag that plagues fixed-horizon methods.

To find out whether these design choices actually matter, the team ran simulation experiments across grid environments ranging from a compact 20 by 20 layout to a more demanding 50 by 50 space, comparing ASDE against benchmark algorithms including conventional SARSA. The results, as summarized in the paper, are consistent across scales. ASDE improved task success rates by between 4.4 and 10.4 percent relative to the benchmarks, a meaningful margin in a domain where a failed rendezvous can mean a drone stranded far from its base. The learned paths were also substantially shorter, with reductions of 21.2 to 30.7 percent in path length, which translates directly into energy saved and mission time recovered.

One of the most practically significant findings concerns path quality rather than raw performance. ASDE reduced the number of inflection points along planned trajectories by 33.0 to 46.0 percent compared with the benchmark algorithms. Inflection points are the sharp corners where a path changes direction abruptly, and for a flying vehicle each one costs energy and imposes maneuvering loads. A smoother path is easier for a flight controller to track accurately, gentler on the airframe, and more predictable for any surrounding traffic. That a learning algorithm can produce paths that are simultaneously shorter, more reliable, and smoother suggests the restructured reward function and adaptive exploration are doing real work in shaping the geometry of the solutions, not just the success statistics.

The convergence result deserves particular attention because learning speed is often the hidden bottleneck in deploying reinforcement learning on real hardware. An agent that needs millions of training episodes to converge is impractical to train in simulation for every new environment a drone might face. By reporting that ASDE reaches convergence approximately 45 percent faster than conventional SARSA, the authors are making a claim about deployability as much as about benchmark performance. Faster convergence means the planner can adapt more quickly when the mobile charging vehicle’s behavior shifts, and it lowers the computational cost of retraining, both of which matter for operations where conditions change from mission to mission.

The broader context makes the work timely. Drones are being pressed into service for search and rescue, infrastructure inspection, agricultural monitoring, and delivery, and in many of these roles the endurance limit of batteries is the binding constraint on how the technology can be used. Charging infrastructure that moves with the mission, rather than waiting at a fixed pad, is one of the more elegant proposed answers, and the concept has close cousins in research on electric vehicles and mobile robotic refueling. What has been missing is a planning layer robust enough to handle the uncertainty of a charger that is itself navigating traffic, terrain, and its own constraints. The Shenyang team’s contribution is a concrete, quantitatively evaluated step toward that layer, grounded in one of the workhorse algorithms of reinforcement learning rather than requiring an entirely new theoretical apparatus.

There are, of course, the usual caveats that separate simulation from the sky. The reported gains come from grid-world experiments of up to 50 by 50 cells, and real flight adds wind, sensing noise, communication delays, and safety constraints that no grid abstraction fully captures. The authors state that the datasets generated and analyzed in the study are available from the corresponding author on reasonable request, and the work was published in the International Journal of Aeronautical and Space Sciences on 14 July 2026 under the auspices of the Korean Society for Aeronautical and Space Sciences. Even with those caveats, the pattern of results is striking enough to suggest that the adaptive machinery at the heart of ASDE, the feedback-driven exploration schedule and the motion-aware temporal difference backtracking, could generalize beyond charging rendezvous to any drone task that involves intercepting a moving target. For a field where the difference between a 60 percent and a 70 percent success rate can determine whether an autonomous mission is viable at all, an algorithm that delivers higher success, shorter paths, smoother trajectories, and faster learning in a single package is the kind of incremental engineering advance that quietly enables the next generation of flying robots.

Subject of Research: Reinforcement learning-based path planning for UAV dynamic charging with mobile charging vehicles

Article Title: ASDE Algorithm-Based UAV Dynamic Charging Path Planning Method

Article References: Shan, D., Zhang, M., He, J., Zhang, T., & Li, Y. (2026). ASDE Algorithm-Based UAV Dynamic Charging Path Planning Method. International Journal of Aeronautical and Space Sciences. https://doi.org/10.1007/s42405-026-01253-0

Image Credits: AI Generated

DOI: 10.1007/s42405-026-01253-0

Keywords: UAV, reinforcement learning, SARSA, ASDE algorithm, mobile charging vehicle, path planning, dynamic environments, temporal difference learning, epsilon-greedy exploration, drone battery charging, autonomous flight, Shenyang Jianzhu University

The core of the framework is a re-optimization subproblem the authors call the adaptive green vehicle routing problem, or AG-VRP. The green vehicle routing problem, originally developed in operations research to model fleets of alternative-fuel vehicles with limited range and mandatory refueling stops, assumes that every vehicle starts and ends at a single depot. That assumption collapses in a replanning scenario: when an agent is lost or a task set changes, the surviving agents are scattered across the mission area with varying amounts of fuel remaining, and they must continue from wherever they happen to be. The AG-VRP therefore allows arbitrary start locations for each agent, heterogeneous initial fuel states, and a single shared destination, while still requiring that every task be visited exactly once and that refueling depots be used as needed to keep each vehicle within its maximum travel range.

To guarantee that this subproblem can be solved to proven optimality when time permits, the researchers formulated it as a mixed-integer linear program. The formulation tracks binary decision variables indicating which arcs each agent traverses, continuous fuel variables that decrease with distance traveled and reset to maximum capacity at depots, and a sequence variable that eliminates subtours, the closed loops disconnected from the start and end nodes that plague vehicle routing formulations. Virtual depots, generated by copying each physical refueling location, allow depots to be visited multiple times by different agents. This mathematical model serves a dual role: it defines the theoretical lower bound against which faster methods are measured, and it provides optimal reference solutions in small problem instances where a commercial solver can exhaust the search space.

Optimality, however, is a luxury that mid-mission replanning cannot afford. The authors set a fixed computation budget of ten seconds per replanning epoch, reflecting the operational reality that a drone hovering with dwindling fuel cannot wait minutes for a solver. Within that budget, exact mixed-integer programming quickly becomes intractable as the number of tasks grows. The team therefore built a hierarchical solution suite. The first layer is a modified sequential greedy algorithm, a fast heuristic that assigns tasks to agents one at a time using a weighted cost that combines the distance to a candidate task with the distance from that task to the destination. The weighting factor discourages agents from picking up tasks near the end node early in their routes, which would otherwise produce inefficient gradual-approach behavior. The greedy algorithm also embeds explicit fuel logic: an agent only accepts a task if its remaining range can cover the trip to the task plus the worst-case distance onward to a depot or the destination, and it falls back to progressively less attractive tasks when the best candidate is fuel-infeasible.

The second and centerpiece layer is a reinforcement learning approach designed to overcome the myopia of greedy assignment. The authors cast the AG-VRP as a Markov decision process in which each decision step selects one agent and one node for it to visit next. The state captures every agent’s current location, remaining fuel, and partial trajectory, along with the set of unserved tasks. The reward is simply the negative of the total distance traveled by all agents, awarded only at the end of an episode, so maximizing expected return is mathematically equivalent to minimizing total travel distance. Training uses the REINFORCE gradient estimator with a rollout baseline, a technique in which a frozen copy of the policy serves as a comparison standard and is updated only when a paired statistical test confirms that the current policy has genuinely improved.

The neural architecture borrows from the attention models that have transformed combinatorial optimization in recent years. An encoder embeds all nodes of the problem, distinguishing depot nodes from task and start nodes and from the end node, and applies multi-head self-attention layers to capture the relationships among them. On the decoding side, the framework introduces two specialized modules: a vehicle selection decoder that chooses which agent should move next, and a node selection decoder that chooses where that agent should go. Crucially, each agent maintains a hidden state in a long short-term memory network that summarizes its entire partial trajectory, including refueling decisions, so the policy can reason about history rather than just the current snapshot. Remaining fuel is concatenated with the trajectory embedding before a linear layer produces the final vehicle representation, allowing the network to weigh range constraints directly when scoring candidate actions.

In deployment, the system does not gamble on a single method. At each replanning epoch, the mission snapshot, comprising agent positions, fuel levels, and remaining tasks, is converted into an AG-VRP instance, and both the greedy algorithm and the trained reinforcement learning policy run independently. The framework, dubbed RL plus MSGA, then selects whichever solution yields the lower total travel distance. Because the two methods have comparable runtimes, this selection strategy improves solution quality at essentially no additional computational cost. The case study illustrates the framework in action: three agents tasked with thirty tasks and supported by two refueling depots lose one of their number after ten tasks are completed, and the two survivors must absorb the remaining workload. The replanned routes produced by the exact solver, the learned policy, and the greedy heuristic are visually similar, but the reinforcement learning method achieves a smaller optimality gap than the greedy baseline.

The numerical experiments scale this evaluation dramatically, covering one thousand problem instances across six task counts from ten to one hundred and two fleet sizes, for a total of twelve thousand problem geometries. The results reveal a striking divide. As problem size grows, the proportion of instances for which the exact solver can even find a feasible solution within ten seconds falls sharply, whereas the proposed heuristics deliver feasible solutions in under one second with a one hundred percent success rate across all scenarios. On instances where the exact solver proves optimality, the reinforcement learning policy consistently lands closer to the optimum than the greedy method, and the combined strategy achieves the smallest gaps overall. Most remarkably, for the largest instances, where the solver struggles to find any high-quality feasible route, the learned policy sometimes matches or beats the solver’s output, producing negative optimality gaps.

Beyond raw distance, the study examined mission-oriented metrics that matter to operators. The reinforcement learning policy consistently leaves the largest fuel reserve at mission completion, indicating a more conservative routing style with a bigger safety margin, while the combined strategy minimizes the number of refueling operations without sacrificing route quality. Workload balance, measured by the standard deviation of tasks assigned to each agent, remained comparable across all methods, showing that the quality gains of the learned policy do not come at the price of overburdening individual vehicles. Even for one-hundred-task instances where optimal solutions could still be found, the combined approach kept optimality gaps below ten percent, a level of performance the authors attribute to the complementary strengths of learned global structure and fast greedy feasibility.

The implications extend beyond military surveillance and reconnaissance, the scenarios that motivate much of this line of research. The same mathematics applies to planetary surface exploration, where in-situ resource utilization stations serve as refueling depots for rovers, and to green logistics fleets of electric or alternative-fuel delivery vehicles that must recharge mid-route. The authors note that future work could incorporate kinematic path constraints such as minimum turning radius, which would replace straight-line distances with Dubins paths and make travel cost dependent on vehicle heading, as well as time windows, agent-specific tasks, and inter-agent collaboration. For now, the study demonstrates that a carefully trained attention-based policy, paired with a fuel-aware greedy fallback, can keep a fuel-constrained drone team flying coherently through the kind of mid-mission shocks that would ground a statically planned fleet.

Subject of Research: Real-time replanning of fuel-constrained multi-UAV missions using reinforcement learning and greedy heuristics

Article Title: Adaptive Planning for Multiple UAVs with In-Flight Refueling

Article References: Gwon, B., Choi, E., Lee, J., & Ahn, J. (2026). Adaptive Planning for Multiple UAVs with In-Flight Refueling. International Journal of Aeronautical and Space Sciences. https://doi.org/10.1007/s42405-026-01249-w

Image Credits: AI Generated

DOI: 10.1007/s42405-026-01249-w

Keywords: UAV, reinforcement learning, vehicle routing problem, in-flight refueling, rolling-horizon replanning, attention model, mixed-integer linear programming, greedy algorithm, LSTM, dynamic environments, fuel constraints, KAIST

The research, led by Jilin Zhang of the University of Jinan with colleagues from Shandong Normal University, the University of Jinan and Lunan Technician College, targets a weak spot in modern visual Simultaneous Localization and Mapping, or SLAM. Classical SLAM systems assume the world they observe is static. When people walk through the frame, feature points attached to them move for reasons that have nothing to do with camera motion, and the system’s estimate of its own trajectory drifts. Recent dynamic SLAM methods, such as Dyna-SLAM and SaD-SLAM, attack this by detecting humans and other obviously dynamic objects and discarding their pixels. But the authors point out a stubborn category of uncertainty: movable objects like books and cups. Most of the time a book sits still on a desk, so treating it as static background is reasonable—until someone picks it up and carries it across the room. A SLAM system that blindly trusts those features inherits the object’s motion as phantom camera motion.

The team’s answer is a dynamic SLAM system built around what they call human contact experience. Rather than hard-coding which objects are dynamic and which are static, the system learns from observation how often humans come into contact with particular categories of objects. Objects that are frequently held, such as cups and books, receive a prior state that makes the system suspicious of their apparent motion; objects that people rarely touch keep their static status. When a human and a movable object are in contact, the object’s features are treated as unreliable and excluded from pose estimation. In effect, the robot accumulates a form of common-sense knowledge about indoor life and uses it to decide which of the things it sees can be trusted as reference points.

Technically, the system weaves together three modules. The first is an adaptive frame selection strategy driven by semantic segmentation results. Instead of feeding every RGB-D frame into the computationally expensive segmentation pipeline, the system adaptively chooses which frames to process, reducing computational resource consumption while improving the quality of the prior information available to later stages. This matters for real robots, which must localize in real time on hardware far less powerful than a laboratory workstation. The second module refines the geometric analysis: by combining optical flow with epipolar constraints, the system determines the motion states of both humans and movable objects. Optical flow captures how pixels shift between consecutive frames, while the epipolar constraint describes how a static point in a rigid scene should move given the camera’s own motion. A point that violates the epipolar geometry is almost certainly moving independently of the camera—an elegant, geometry-based way to flag dynamic content without relying on semantics alone.

The third and conceptually novel piece is the contact experience module itself. Drawing on information from multiple consecutive frames, the module records the contact frequency between humans and movable objects and uses that history to update the prior state of objects in the indoor environment. An object seen repeatedly in human hands shifts its prior toward dynamic; an object that has never been touched retains a static prior. Because this knowledge is updated continuously, the system adapts to a particular environment over time rather than relying on a fixed, hand-tuned list of dynamic classes. The authors describe this as using human contact experience with movable objects to predict their true states—a statistical prior grounded in the everyday physics of how people interact with their belongings.

Everything rests on accurate camera trajectories, so the researchers evaluated their system on the TUM RGB-D benchmark, the standard dataset for testing RGB-D SLAM under dynamic conditions. The benchmark includes sequences in which people walk, sit and interact with objects while the camera moves through the scene—precisely the conditions that break static-world assumptions. The proposed method was compared against ORB-SLAM2, the widely used open-source baseline for monocular, stereo and RGB-D cameras, and against two representative dynamic-environment systems, Dyna-SLAM and SaD-SLAM.

The reported results show the new system operating stably in dynamic environments and, crucially, handling state changes of indoor movable objects more effectively than its predecessors. Where Dyna-SLAM and SaD-SLAM can mask out walking people, they have no principled mechanism for the cup that was static in frame one and moving in frame ten. By combining adaptive frame selection, flow-and-epipolar geometry and contact-frequency priors, the new method covers both ends of the problem: it ignores pixels belonging to independently moving entities and reclassifies movable objects the moment their behavior changes. The authors note that the adaptive frame selection also keeps the computational cost in check, which matters for indoor companion robots that must run continuously.

The implications reach beyond a cleaner trajectory estimate. Indoor companion robots are expected to interact naturally with humans, and that requires knowing not just where the robot is, but what in the room is trustworthy as a landmark. A robot that understands that a person carrying a mug makes the mug’s features unreliable, but that the mug becomes a valid landmark again once set down, gains a more realistic model of its environment. The contact experience framework is also a small but suggestive step toward robots that learn everyday physics from observation—the kind of implicit knowledge humans use constantly without noticing.

The work was supported in part by the National Natural Science Foundation of China, the Taishan Scholar Foundation of Shandong Province and the Outstanding Youth Foundation of Shandong Province. As robots move from factory floors into homes, offices and hospitals, the ability to localize reliably amid human activity will stop being a research curiosity and become a baseline requirement. This study suggests that some of the best clues for separating a stable world from a shifting one may come from simply paying attention to what people are holding.

Subject of Research: A dynamic RGB-D SLAM method that uses human contact experience to determine the motion states of movable objects for robot localization in dynamic indoor environments.

Article Title: A RGB-D SLAM method based on contact experience in dynamic environment

Article References: Zhang, J., Huang, K., Geng, H., Song, C., & Zhang, M. (2026). A RGB-D SLAM method based on contact experience in dynamic environment. Autonomous Robots, 50(4), Article 40. https://doi.org/10.1007/s10514-026-10268-1

Image Credits: AI Generated

DOI: 10.1007/s10514-026-10268-1

Keywords: SLAM, RGB-D camera, dynamic environments, robotics, semantic segmentation, optical flow, epipolar constraints, contact experience, movable objects, ORB-SLAM2, Dyna-SLAM, TUM RGB-D dataset