The research team, Yuexiang Hu and Qin Li, identified two persistent weaknesses in existing lane-changing trajectory planning methods: low prediction accuracy and insufficient multi-objective collaborative optimization. Many current approaches either cascade a prediction module with a decision-making network in a modular assembly, where the two components are trained independently and gradient flow is truncated at the interface, or they rely on manually designed state spaces and reward functions that transform abstract safety constraints into rigid rules. In such architectures, the predictor may generate trajectories that are statistically accurate yet strategically suboptimal for lane-changing decisions, because the prediction module optimizes only for coordinate regression accuracy while remaining unaware of decision-critical factors such as collision risk or comfort requirements. The new framework eliminates this modular boundary by constructing a unified training objective where prediction and control are co-trained under shared gradient flows, so prediction errors directly influence policy gradients and the policy network can adapt its state representation based on prediction reliability.

At the heart of the trajectory prediction module lies a carefully engineered combination of three neural network components: a temporal convolutional network (TCN), a spatial multi-head attention mechanism, and a Transformer encoder–decoder. The TCN employs three stacked dilated causal convolution layers with a two-dimensional kernel size of (7,1), extracting multi-scale temporal dependencies from historical vehicle states through exponentially increasing receptive fields. Layer normalization and ReLU activation follow each TCN layer to stabilize training dynamics. The spatial attention mechanism operates on a dynamic spatiototemporal graph where nodes represent vehicle states, temporal edges connect consecutive historical states across a 10-second observation window, and spatial edges encode instantaneous interactions between vehicles within a perception range extending 50 meters ahead, 30 meters behind, and across the full lateral coverage of adjacent lanes. Eight attention heads with a model dimension of 32 extract different interaction patterns in parallel, with attention weighting based on relative motion states rather than fixed positional encoding—enabling the network to adaptively model inter-vehicle topological interactions through the dynamic spatiototemporal graph.

The Transformer encoder–decoder consists of six encoder layers and six decoder layers, processing spatiototemporal features of variable-length historical encoding and multi-step future decoding within a 5-second prediction range. Dropout at 0.11 prevents overfitting, and the network trains using the Adam optimizer with a learning rate of 0.0001, with the output layer employing linear activation to directly regress future trajectory coordinates. To quantify potential collision risk during lane changes, the researchers built a driving risk field model comprising three components with specific parameters: static obstacle risk following Gaussian decay with an influence radius of 40 meters and a decay coefficient of 10 meters; dynamic obstacle risk with directional correction using an approach factor of 0.5 and a recede factor of −0.3; and road boundary risk following a power function with a boundary coefficient of 2.0. Lane line risk uses a Gaussian-like function with a deviation penalty of 0.5 m⁻² and a peak risk of 1.0, with field strength set to 1.0 for solid lines and 0.5 for dashed lines. Virtual mass characterizing the risk field is 1,500 kilograms, reflecting a typical passenger vehicle.

With the prediction module and risk field in place, the researchers turned to the deep deterministic policy gradient (DDPG) algorithm—a deep reinforcement learning method prized for efficiently processing continuous action spaces and enabling end-to-end optimization. DDPG employs four neural networks: an online policy network (Actor) that outputs deterministic actions, an online value network (Critic) that estimates the Q value of state–action pairs, and their corresponding target networks used to calculate stable target Q values and reduce sample correlation. The Actor network receives a 13-dimensional state vector—comprising self-vehicle state, nearest surrounding vehicle state, predicted trajectories, and driving risk field values—and outputs three continuous control actions (throttle, steering, and brake) through a Tanh-activated output layer. The Critic network evaluates the Q value of the state–action pair using a linear output layer, providing gradient guidance for policy optimization. Target networks soft-update at each training step with a factor of 0.004 to stabilize temporal difference error estimation, while Ornstein–Uhlenbeck noise with parameters (0.14, 0.00, 0.24) superimposes on the Actor output for continuous action–space exploration.

One of the study’s most consequential improvements concerns the experience replay pool. Traditional DDPG uses uniformly random sampling, treating all experiences equally—a design that dilutes key safety experiences such as emergency obstacle avoidance and close-range car following beneath a mass of conventional lane-changing data, slowing convergence and reducing decision-making safety. The improved mechanism introduces a priority screening based on the temporal difference error: newly collected experiences deposit into a collection pool, and those with temporal difference errors exceeding a threshold of 0.10 filter into a high-quality pool for prioritized batch sampling. This ensures that high-value safety experiences obtain higher sampling probability, effectively solving the problem of important experiences being submerged. The reward function guiding the agent comprises four weighted components: efficiency reward (weight 0.3) encouraging forward travel distance, comfort reward (weight 0.2) penalizing sudden acceleration and steering angle changes, safety reward (weight 0.4) directly adopting the negative risk field intensity so higher risk yields greater penalty, and driving state reward (weight 0.1) providing positive rewards for normal driving and negative rewards for lane departure or collision. A sensitivity analysis confirmed that the nominal configuration achieves a balanced trade-off among safety, efficiency, and comfort, maintaining a collision rate of 3.27 percent and lane departure rate of 2.36 percent while ensuring reasonable velocity.

The experimental data came from the NGSIM US-101 vehicle trajectory public dataset, with trajectories divided into a complete observation period of 15 seconds—the first 10 seconds serving as historical input features and the last 5 seconds as output labels to predict. Savitzky–Golay filtering removed outliers, and after preprocessing 14,341 pieces of data were obtained, split 70 percent for training and 30 percent for verification. When compared against Structural-LSTM, Social-GAN, and constant turn rate and acceleration (CTRA) models, the proposed prediction module achieved root mean square errors (RMSE) of 1.59 meters for going straight, 1.19 meters for turning left, and 1.27 meters for turning right—substantially lower than the 2.08, 1.51, and 1.98 meters of Structural-LSTM, the 2.47, 1.62, and 1.98 meters of Social-GAN, and the 2.86, 1.98, and 2.86 meters of CTRA under corresponding conditions. Repeated 10 times with different random seeds, these results held as mean ± standard deviation, with paired t-tests between the proposed method and each baseline yielding p < 0.01, indicating statistically significant improvements.

Ablation experiments confirmed the contribution of each module. Removing the TCN module led to the largest increase in prediction RMSE by 0.62 meters, indicating its critical role in temporal feature extraction. Without the attention mechanism, obstacle avoidance success rate dropped by 2.71 percent. The risk field contributed most significantly to safety—its removal caused a 13.25 percent drop in obstacle avoidance success rate—while the improved replay mechanism accelerated convergence, with its absence resulting in a 15.69 percent decrease in obstacle avoidance success rate. The simultaneous removal of the risk field and replay mechanism caused the most severe performance degradation by 20.47 percent and 19.03 percent, demonstrating their joint importance. In simulation experiments on the CAR Learning to Act platform with 200 obstacles and targets randomly arranged, the proposed method achieved obstacle avoidance success rates of 98.92 ± 0.35 percent and target advancement success rates of 98.26 ± 0.42 percent, outperforming LSTM-DDPG (94.37 ± 0.52 and 92.21 ± 0.58), TRPO (90.26 ± 0.68 and 89.87 ± 0.62), and TD3 (89.24 ± 0.71 and 88.64 ± 0.65), with paired t-tests again yielding p < 0.01.

Beyond raw success rates, the method demonstrated superior trajectory quality in overtaking and obstacle avoidance scenarios. In lane-changing overtaking, all algorithms completed the left lane change in approximately 10 seconds, but the proposed method completed the right reversal operation in 45 seconds compared with approximately 53 seconds for LSTM-DDPG, while TRPO and TD3 stayed in the original lane entirely. The proposed method maintained stable vehicle speed throughout and quickly increased speed after overtaking, indicating higher overtaking efficiency. In obstacle avoidance, the maximum lateral displacement of the proposed method was −3.18 meters, and it quickly returned to the positive direction after 40 seconds with stable movement trajectory—compared with −3.76, −3.47, and −3.62 meters for LSTM-DDPG, TRPO, and TD3, which showed slower back-to-alignment speed and poorer trajectory control accuracy. The proposed method maintained vehicle speed at 57.87 km/h during obstacle avoidance with the smallest fluctuation, and its steering angle control curve was smoother with fewer peaks, indicating better comfort and stability.

Perhaps most strikingly, the method achieved an inference time of 19.8 ± 1.5 milliseconds per step, corresponding to a frame rate of 50.5 ± 3.8 Hz—meeting the real-time requirement for autonomous driving applications—while consuming the lowest GPU memory at 95.2 ± 6.4 MB. Comparison methods including SAC, PPO, Graph-based, Social-LSTM, LSTM-DDPG, TRPO, and TD3 exhibited inference times ranging from 35.4 to 52.4 milliseconds and memory consumption from 108.6 to 156.8 MB. Robustness testing under challenging conditions further validated the approach: in dense traffic with 8 to 12 surrounding vehicles exhibiting aggressive behaviors including frequent lane cut-ins, reduced headways of 1.5 seconds, and elevated acceleration up to 6 m/s², the proposed method achieved obstacle avoidance success rates of 95.34 ± 0.48 percent and target advancement success rates of 94.87 ± 0.52 percent. In rainy weather with precipitation intensity of 15 mm/h, road friction coefficient reduced from 0.9 to 0.4, perception range limited from 50 to 35 meters, and Gaussian noise with standard deviation of 0.3 meters added to position and velocity measurements, the method attained obstacle avoidance and target advancement success rates of 93.18 ± 0.55 percent and 92.65 ± 0.61 percent—exceeding all baselines under both scenarios. The researchers acknowledge three principal limitations: experiments were confined to the CAR Learning to Act simulation platform, leaving sim-to-real transfer challenges including domain discrepancy between simulated and real-world sensor inputs, unmodeled actuator latency and steering dynamics, and divergence between idealized traffic behaviors and intricate human driving maneuvers; the model was trained and evaluated on a single dataset with fixed simulation settings, leaving generalization performance on unseen road geometries, diverse traffic rules, and rare edge cases unassessed; and reward weights and risk field parameters were manually calibrated under nominal conditions, warranting future exploration via online meta-learning or domain randomization. Future work will focus on constructing a real-vehicle test platform, incorporating vehicle–road cooperative perception data to mitigate the sim-to-real gap, and introducing domain randomization and adversarial training during simulation to improve policy robustness prior to physical deployment.

Subject of Research: Multi-objective optimization of lane-changing trajectories for autonomous vehicles based on deep reinforcement learning

Article Title: Multi-objective optimization of lane-changing trajectories for autonomous vehicles based on deep reinforcement learning

Article References: Hu, Y., & Li, Q. (2026). Multi-objective optimization of lane-changing trajectories for autonomous vehicles based on deep reinforcement learning. Frontiers in Mechanical Engineering, 12, 1900959. https://doi.org/10.3389/fmech.2026.1900959

Image Credits: AI Generated

DOI: 10.3389/fmech.2026.1900959

Keywords: autonomous vehicles, deep reinforcement learning, DDPG, lane-changing trajectory planning, Transformer, driving risk field, trajectory prediction, Frontiers in Mechanical Engineering, Multi-objective, optimization, lane-changing, trajectories

A research team at the Shanghai Institute of Technology, led by Liqiong Chen together with Liubing Xiao, Xiaotian Chen, and Huaiying Sun, has now introduced a scheduling framework designed specifically for this chaos. The framework, called SPARCD, is described in the journal Cluster Computing and tackles what the authors identify as a stubborn weakness in the mainstream approach to vehicular task offloading. Most current systems rely on deep reinforcement learning to decide where and when to run each task, and the dominant algorithm in this space is Deep Deterministic Policy Gradient, better known as DDPG. DDPG excels at learning continuous control policies, but in the vehicular setting it suffers from high task interruption rates and slow convergence, because it makes offloading decisions largely blind to the future motion of the vehicles whose tasks it is scheduling.

The core insight behind SPARCD is deceptively simple: if the scheduler knows where a vehicle is going to be, it can avoid making decisions that will be invalidated by that movement. The framework therefore integrates three coordinated mechanisms into a DDPG-based learning pipeline. The first is a lightweight position prediction mechanism that incorporates vehicle motion state awareness and forecasts future locations, giving the policy a window into the near-term kinematic future of each vehicle. The second is an action masking mechanism, a technique borrowed from recent policy gradient research that restricts the agent’s action space to physically or logically valid choices at each decision point. The third is a state filling technique that enriches the observation vector so the neural network sees a complete and informative picture of the environment rather than a sparse or incomplete one.

To understand why these additions matter, it helps to consider what happens inside a conventional DDPG agent managing a VEC system. At each time step, the agent observes the state of the network, including queue lengths at edge servers, channel conditions between vehicles and access points, and the computational demands of pending tasks. It then selects an offloading action, perhaps assigning a task to a local processor, to a roadside edge server, or to a neighboring vehicle capable of cooperative computation. The problem is that the agent evaluates these choices using only current information. A link that looks strong now may dissolve in half a second as the transmitting vehicle rounds a corner or accelerates away from the access point. When that happens, the in-flight task is interrupted, its partial results are wasted, and the energy already spent on transmission and computation is lost.

SPARCD attacks this failure mode from multiple angles simultaneously. The position prediction component lets the framework estimate whether a vehicle will remain within effective communication range of a candidate edge server for the expected duration of the task. If the forecast says otherwise, the scheduler can steer the task toward a more durable option before committing. The action masking mechanism complements this by filtering out invalid or unsafe actions before they reach the policy network’s output layer, which both prevents the agent from learning from pointless experiences and accelerates convergence by shrinking the effective exploration space. The multi-objective formulation at the heart of the framework jointly optimizes three quantities that normally pull against each other: task latency, energy consumption, and reliability. Reducing latency often demands more transmit power, which raises energy use; maximizing reliability may require conservative offloading choices that increase delay. SPARCD balances these competing demands within a single scheduling model.

The authors validated the framework through simulations of typical urban scenarios, with vehicle densities of 30 vehicles per kilometer and task generation rates ranging from 0.5 to 1 task per second. The headline numbers are striking. Compared with a traditional DDPG algorithm operating in the same environment, SPARCD reduced the task interruption rate by 42.7 percent, cutting it from 18.2 percent to 10.4 percent. Average task latency dropped by 31.5 percent, falling from 3.2 seconds to 2.2 seconds. Perhaps most importantly from an engineering standpoint, these gains in speed and dependability did not come at a ruinous energy price: system energy consumption increased by no more than 7.9 percent, a trade-off the authors characterize as a significant overall performance improvement rather than a robbing of one metric to pay another.

The most demanding test came in high-dynamic conditions, where vehicles moved at 25 meters per second, roughly 90 kilometers per hour, a speed at which wireless connections to fixed infrastructure are fleeting and prediction windows shrink dramatically. Even here, SPARCD maintained a task completion rate of 82.3 percent. For comparison, a Q-learning based method, a simpler tabular reinforcement learning approach often used as a baseline, achieved only about 58.7 percent in the same scenario, meaning SPARCD outperformed it by more than 40 percentage points. The authors point to this result as evidence of the framework’s robustness and generalization ability in complex dynamic environments, suggesting that the learned policy does not merely memorize one traffic pattern but adapts across a range of mobility regimes.

The significance of this work extends beyond one benchmark. Vehicular edge computing sits at the intersection of several rapidly maturing technologies, including cellular vehicle-to-everything communication, roadside multi-access edge servers, and increasingly autonomous vehicles that cannot tolerate cloud round-trip latencies for safety-critical computation. Applications such as cooperative perception, where vehicles fuse sensor data to see around corners, and real-time navigation AI depend on offloading that is both fast and dependable. An interruption rate of 18 percent, which the baseline DDPG exhibited, would translate into a substantial fraction of these workloads failing or restarting, wasting bandwidth and battery resources in the process. Cutting that figure to roughly 10 percent while simultaneously lowering latency represents exactly the kind of systems-level improvement that makes edge-assisted driving applications practically viable.

Technically, the study also contributes to a broader conversation in the reinforcement learning community about how to make deep policies reliable in non-stationary environments. Action masking, highlighted in recent research on invalid action masking in policy gradient algorithms, has proven effective in games and robotics by preventing agents from selecting actions that are meaningless in a given state. Applying it to vehicular offloading, combined with mobility prediction, illustrates a general design principle: domain-specific structure can be injected into the learning pipeline to compensate for the brittleness of generic algorithms. Rather than waiting for the policy network to discover, through thousands of costly failed episodes, that offloading to a server the vehicle is about to drive away from is a bad idea, the framework encodes that knowledge architecturally.

The work arrives amid a wave of related research, including digital twin based offloading schemes, multi-agent deep reinforcement learning systems with trajectory prediction, and hybrid frameworks that combine transformer-style prediction models with reinforcement learning policies. Within that landscape, SPARCD distinguishes itself by treating mobility awareness, action validity, and state completeness as first-class citizens of the scheduling architecture rather than afterthoughts, and by quantifying the latency, energy, and reliability trade-offs under realistic urban traffic parameters. The authors, affiliated with the Department of Intelligent Technology at the Shanghai Institute of Technology and the Shanghai Engineering Research Center of Smart Energy, report no competing interests. As connected and autonomous vehicles continue to multiply on roads worldwide, frameworks like SPARCD point toward an urban computing fabric in which the movement of traffic is not an obstacle to computation but an input the scheduler can anticipate and exploit.

Subject of Research: Multi-objective task offloading scheduling in vehicular edge computing using deep reinforcement learning with mobility prediction

Article Title: SPARCD: a multi-objective cooperative scheduling framework for vehicular edge computing

Article References: Chen, L., Xiao, L., Chen, X., & Sun, H. (2026). SPARCD: a multi-objective cooperative scheduling framework for vehicular edge computing. Cluster Computing, 29(15), Article 844. https://doi.org/10.1007/s10586-026-06655-z

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06655-z

Keywords: vehicular edge computing, task offloading, DDPG, deep reinforcement learning, action masking, position prediction, multi-objective optimization, task interruption, latency reduction, energy consumption, Internet of Vehicles, edge servers

The research, published in Frontiers in Control Engineering by Ahmad Al Ali and Zheng H. Zhu of York University’s Mechanical Engineering Department, tackles a challenge that has limited space robotics for decades. Space robotic systems come in two flavors. Free-flying robots use thrusters to actively hold the base spacecraft steady while the arm works, which makes control straightforward but burns precious fuel. Free-floating robots let the base drift naturally in response to the arm’s motions, conserving momentum and stretching the mission’s operational life. The trade-off is mathematical chaos: the position and orientation of the arm’s end effector depend not just on the current joint angles but on the entire history of joint and base velocities, a nonholonomic coupling that renders classical fixed-base planning techniques ineffective.

Earlier attempts to tame this coupling relied on inverse kinematics respecting angular momentum constraints, but dynamic singularities, unpredictable from kinematics alone, could derail the arm mid-task. Optimization approaches such as particle swarm optimization and sliding mode control improved matters but demanded heavy computation and required complete re-optimization whenever initial conditions changed. Reinforcement learning offered a compelling alternative. In this framework, an agent learns a control policy through trial and error, guided by a reward function, within a Markov Decision Process. Crucially, once training ends, the entire learned policy is compressed into the weights and biases of a neural network, lightweight enough to run on modest space-qualified hardware, a decisive advantage where computing power is scarce.

Prior reinforcement learning studies on space manipulators mostly targeted simpler systems, often three degrees of freedom, and frequently ignored practical constraints. The authors’ survey of the field found that most work neglected joint velocity limits, avoided quaternion-based attitude representation, and omitted obstacle avoidance entirely. Their new work addresses all three gaps simultaneously, applying the Deep Deterministic Policy Gradient algorithm, which combines an actor network that outputs continuous joint torques with a critic network that estimates the expected discounted return of state-action pairs. The architecture uses two hidden layers of 400 and 300 neurons, a replay buffer of 50,000 transitions, and a discount factor of 0.99, trained in a MATLAB and Simulink simulation of a six-link manipulator on a 1,700-kilogram base spacecraft.

The heart of the method is a carefully engineered reward function. A guidance term rewards the agent for shrinking both the positional distance and the orientation difference between the end effector and the target. Once the end effector enters a predefined capture zone, large positive rewards kick in to hold it there while fine alignment proceeds. Large negative penalties punish self-collision between links, contact with the target, and encroachment on an external obstacle placed directly in the approach path. A velocity penalty, proportional to the squared sum of joint angular velocities, discourages jerky motion and produces smoother trajectories.

One of the study’s most elegant technical choices is the use of quaternions instead of Euler angles to represent orientation. Euler angle differences discard information about rotation sequence, inflating the search space during learning and sometimes causing divergence. Quaternions, four-component representations of rotation, avoid the singularities that plague Euler angles and preserve full rotational information. The orientation error is computed from the relative quaternion between the end effector and the target, giving the learning agent a cleaner, more informative signal and accelerating convergence.

The simulation results demonstrate the approach across three progressively harder scenarios. In the first, without velocity constraints, the agent guided the end effector into the capture range within roughly 6 seconds of the 10-second episode, achieving a minimum distance error of 0.058 meters and an orientation error of 4.85 degrees, while threading the arm past the obstacle without a single collision. Training converged within about 5,000 episodes, with the guidance term pulling the end effector toward the target and the penalty terms actively preventing collisions throughout.

The second scenario added the velocity penalty, and the effect was striking. The squared sum of joint velocities along the path barely exceeded 80, compared with peaks of 160 in the unconstrained case, and the trajectory converged more smoothly during the first 3 seconds. The final pose errors were 0.088 meters and 6.61 degrees, a modest cost in precision for a dramatically smoother path, exactly the kind of motion a real gripper needs before capturing a tumbling satellite.

The third scenario confronted the elephant in every real mission: imperfect sensors. Vision-based estimation of a target’s pose is inherently noisy, so the researchers injected white noise with a 10-to-1 signal-to-noise ratio into the observed target pose and obstacle position during training. The agent still succeeded, reaching a distance error of 0.048 meters and an orientation error of 7.79 degrees, with somewhat more oscillatory joint torques reflecting the added uncertainty. The most revealing finding came from the cross-test: the noise-trained agent performed flawlessly in a noise-free environment, while the agent trained on clean data failed when noise was introduced. Training under realistic conditions, it turns out, is not optional but essential.

The implications reach well beyond one simulated arm. As orbital servicing, debris removal, and in-space assembly grow into commercial industries, robots that can plan their own collision-free paths without thruster support will be indispensable. Because the trained policy is just a set of neural network weights, deployment requires no onboard supercomputer, and the modular reward function can be retuned for different missions, adding torque or energy penalties as needed. The authors describe their work as a theoretical foundation for reinforcement learning in complex free-floating operations, and the results suggest that the next generation of orbital robots may learn to move the way astronauts do: gracefully, cautiously, and with an instinct for danger.

Subject of Research: Reinforcement learning-based path planning for a free-floating space robotic manipulator with collision avoidance and observation noise

Article Title: Reinforcement learning for path planning of free-floating space robotic manipulator with collision avoidance and observation noise

Article References: Al Ali, A., & Zhu, Z. H. (2024). Reinforcement learning for path planning of free-floating space robotic manipulator with collision avoidance and observation noise. Frontiers in Control Engineering, 5, Article 1394668. https://doi.org/10.3389/fcteg.2024.1394668

Image Credits: AI Generated

DOI: 10.3389/fcteg.2024.1394668

Keywords: space robotics, reinforcement learning, DDPG, free-floating manipulator, path planning, collision avoidance, quaternions, observation noise, on-orbit servicing, deep learning, spacecraft dynamics, Reinforcement

Large-scale group decision making, or LSGDM, is the formal discipline behind many high-stakes choices that involve twenty or more experts with different backgrounds, risk tolerances, and information. In emergencies, the stakes are extreme: hesitation or misjudgment can cost lives, yet forcing rapid agreement risks steamrolling minority opinions that may contain crucial insights. The traditional remedy is the consensus reaching process, an iterative loop in which experts whose views stray too far from the group are asked to adjust them until a preset consensus threshold is met. The trouble, the authors argue, is that the feedback parameters governing how strongly experts are pushed to compromise have historically been fixed by hand, based on experience. Too large a step and expert preferences get bulldozed; too small a step and the discussion drags on for rounds that an unfolding disaster cannot spare.

The core innovation is to model the entire consensus process as a Markov decision process and let a Deep Deterministic Policy Gradient, or DDPG, agent learn the feedback strategy. In this formulation, the state at each round captures the group’s evolving condition: the vector of opinion deviations between each expert cluster and the group center, the current group consensus level, the historical effect of previous feedback adjustments, and the trend in how quickly opinions are changing. Because these variables contain the essential statistics of the consensus environment, the next state depends only on the current state and the action taken, satisfying the Markov property that makes reinforcement learning well-posed. The action is deceptively simple: a single continuous value between zero and one, the feedback parameter that determines how far identified experts should move their opinions toward the group position.

What makes the problem genuinely hard is the reward. The authors designed a bi-objective, piecewise reward function that pays the agent for improving group consensus, penalizes it for each additional discussion round, and rewards it for preserving what they call the opinion retention degree, a measure of how much of each expert’s original judgment survives the adjustment. The result is an agent that learns to walk a narrow line: converge quickly, but not by erasing the diversity of expert judgment that makes large-group decisions robust in the first place. Sensitivity analysis over reward weight configurations confirmed the trade-off is real and tunable. A consensus-heavy weighting cut the rounds to agreement to 5.2 but dropped preference preservation to 0.6498, while a preference-oriented configuration raised preservation to 0.7235 at the cost of more rounds. The default weights of 0.5, 0.3, and 0.2 struck the best balance.

Before the learning agent ever sees a decision problem, the raw expert input must be made computable. Experts express preferences using hesitant fuzzy linguistic term sets, which allow a decision maker to offer several linguistic evaluations at once, for instance rating an alternative as both good and very good when uncertain. Conventional score functions for these sets can misjudge cases where a multi-element set and a single-element set yield identical scores. The team introduced a hesitation degree into the score function, quantifying how much uncertainty an expert’s linguistic expression carries, so that more hesitant judgments are weighted accordingly. In the case study, this refinement changed the final ranking of the middle alternatives, evidence that the hesitation information embedded in expert language genuinely matters to the outcome rather than being mathematical decoration.

Clustering twenty or more experts into manageable subgroups poses its own trap. Grouping by preference similarity alone ignores the trust relationships that determine who actually influences whom; grouping by trust alone can lump together mutually trusting experts who hold wildly divergent views. The method fuses both signals into a comprehensive relationship matrix, blending preference similarity and normalized trust values with a balance coefficient, then applies K-Means clustering with the number of clusters chosen by the silhouette coefficient. Sensitivity tests across coefficient values from 0.0 to 1.0 showed that equal weighting produced the best clustering, the fewest consensus rounds, and the highest final consensus level, while relying on either signal alone degraded performance significantly. Expert and cluster weights are then computed from network topology, in-degree and out-degree in the trust graph, and credibility, using a Hamacher t-norm to fuse scale and credibility weights.

The real-world test replayed the emergency decision scenario sparked by the 2023 rainstorm, in which 33 people died in Beijing and more than one million people across the region were evacuated. Twenty simulated experts evaluated four emergency alternatives, spanning headcounts and medical treatment, safe transfer of trapped residents, infrastructure repair, and water rescue with secondary disaster management, across four attributes including personnel search and rescue, resource allocation, public opinion management, and timeliness. The initial group consensus level came out at 0.8599, below the emergency threshold of 0.9. The DDPG agent then generated a sequence of feedback parameters, starting conservatively at 0.2195 and escalating round by round to 0.5946, reaching the threshold in six iterations with a final consensus level of 0.9043.

Notably, the attribute weights were not set by the experts themselves. The team crawled more than 50,000 Weibo posts about the rainstorm and applied the TF-IDF algorithm to extract keyword frequencies, matching terms like search and rescue, relief supplies, and rumor control to the decision attributes. Personnel search and rescue emerged as the heaviest weighted attribute at 0.3051, followed by public opinion management at 0.2542. This grounds the final ranking of alternatives in the aggregate concern of the affected public, an unusual touch of democratic signal in a mathematical procedure. The comprehensive scores ranked the fourth alternative, focused on water rescue and secondary disaster control, as optimal, and the refined score function preserved that top ranking while reshuffling the middle options compared with the original function.

Head-to-head comparisons against three benchmark paradigms, a fixed-parameter feedback method, a random small-step adjustment method without reinforcement learning, and a traditional linearly increasing consensus approach, showed the DDPG method reaching consensus in 6.2 rounds for 50 experts versus 8.4, 12.4, and 10.5 respectively, with the gap widening as groups grew to 200 experts. The random method did preserve slightly more of the experts’ original preferences, but needed roughly twice as many rounds, a cost the authors argue is unacceptable when disaster conditions are changing by the hour. Simulated experiments with 20 to 200 experts, and with uniform, normal, and multimodal preference distributions, all reached the 0.9 threshold within eight rounds, with preference preservation rates between 0.65 and 0.73, suggesting the approach generalizes beyond the showcase case.

The authors are candid about the model’s central assumption: that experts adjust their opinions gradually according to the feedback intensity the agent prescribes. Real experts accept, reject, negotiate, or partially adopt suggestions, and the team flags modeling those behaviors as the next frontier of human-machine collaborative decision making. Even so, the study marks a meaningful shift in how consensus might be engineered, from static rules tuned by intuition to a learned policy that reads the live state of a debating group and modulates its pressure in real time. If such agents mature, the quiet work of reconciling expert disagreement in the next flood, wildfire, or pandemic could begin not in a tense conference room, but inside a neural network that has already rehearsed thousands of arguments.

Subject of Research: Reinforcement learning-based adaptive feedback consensus for large-scale group emergency decision making

Article Title: Machine-assisted adaptive feedback consensus method for large-scale group decision making based on reinforcement learning

Article References: Yu, H., Xu, X., & Zhang, W. (2026). Machine-assisted adaptive feedback consensus method for large-scale group decision making based on reinforcement learning. Frontiers in Big Data, 9, Article 1921037. https://doi.org/10.3389/fdata.2026.1921037

Image Credits: AI Generated

DOI: 10.3389/fdata.2026.1921037

Keywords: reinforcement learning, DDPG, large-scale group decision making, consensus reaching, hesitant fuzzy linguistic term sets, emergency decision making, Markov decision process, social network analysis, K-Means clustering, TF-IDF, human-machine collaboration, disaster response

The study, authored by Li He of Ningbo University of Finance and Economics and published on 15 October 2026, addresses a well-known weakness in existing approaches. Rule-based driving models, such as the widely used Intelligent Driver Model, are transparent and easy to implement, but they tend to react late and leave thin safety margins when traffic becomes dense and nonlinear. Deep reinforcement learning systems adapt better, yet most of them optimize either discrete choices like whether to change lanes or continuous quantities like acceleration, rarely both at once. Optimizing the two levels independently can produce conflicting objectives, delayed responses, and unstable vehicle control, because a smooth acceleration policy is useless if it is executing a bad strategic decision, and a clever lane-change plan is dangerous if the steering and braking that follow are jerky or mistimed.

The proposed solution is a hierarchical reinforcement learning architecture that assigns each modality to the algorithm best suited for it. At the top of the hierarchy sits a Double Deep Q Network, or DDQN, which handles discrete behavioral intentions: following, lane changing, and avoidance. DDQN was chosen over the standard Deep Q Network because it separates action selection from value evaluation, correcting the systematic overestimation of Q-values that can push an agent toward overconfident, unsafe maneuvers. At the bottom of the hierarchy, a Deep Deterministic Policy Gradient, or DDPG, controller translates those high-level intentions into continuous acceleration commands. Because longitudinal control is naturally a continuous optimization problem, DDPG’s actor-critic structure is a natural fit: the actor network maps the current state directly to an acceleration, while the critic network evaluates state-action pairs and guides policy improvement without ever needing to discretize the action space.

The two layers communicate through an explicit constraint mechanism, formalized in the study as a policy function in which the continuous action at each time step is conditioned on the discrete decision handed down from above. In other words, the DDPG controller does not simply imitate the DDQN layer; it executes that layer’s intentions under the physical and safety constraints of the moment. A long short-term memory network encodes a window of five historical time steps, each containing the ego vehicle’s velocity and acceleration, relative distances, and surrounding vehicle information, giving both layers a short-term memory of traffic dynamics rather than forcing them to act on instantaneous snapshots. This temporal context is what allows the agent to recognize patterns such as a neighbor vehicle drifting toward a cut-in before the cut-in happens.

One of the more technically interesting contributions involves how the DDPG controller itself was tuned. Deep reinforcement learning is notorious for training instability, where results swing wildly depending on hyperparameters like the learning rate, discount factor, and exploration noise. Rather than hand-tuning these values, the study employs a Multi-Strategy Particle Swarm Optimization, or MSPSO, procedure in which candidate hyperparameter configurations are treated as particles moving through a search space. Each particle’s fitness is scored by the cumulative reward and control stability achieved in preliminary DDPG training runs. The swarm blends global and local search strategies, applies inertia weights and learning factors to shift from broad exploration early on to fine convergence later, and injects random perturbations when population diversity collapses, guarding against premature convergence into local optima. Crucially, the optimization runs offline: the winning hyperparameters are then fixed and handed to DDPG for its actual training, avoiding an unstable co-optimization loop between the two algorithms.

The reward design reflects a philosophy familiar to anyone who has ridden in an autonomous vehicle prototype: safety trumps everything. The reward function weights three objectives at 0.5 for safety, 0.3 for comfort, and 0.2 for efficiency. Safety rewards maintain a following distance above a dynamic threshold computed from a minimum standstill gap plus the current speed multiplied by an expected safe headway, and it imposes steep penalties as collision risk rises. Comfort rewards penalize excessive acceleration changes and control jitter, while efficiency rewards keeping pace with the desired speed so the agent does not become timid. This prioritized weighting prevents any single objective from dominating the learning process and ensures the vehicle never trades away collision avoidance for a smoother or faster ride.

Training and evaluation relied on the nuScenes public dataset, roughly 1,000 real driving scenarios of about 20 seconds each, collected by a full sensor suite of six surround-view cameras, one lidar, and five millimeter-wave radars across conditions including night, rain, congestion, and dense multi-vehicle interaction. Because nuScenes is an open-loop dataset, the author could not simply drop a learning agent into it. Instead, the recorded trajectories, relative distances, velocities, and interaction patterns were used to reconstruct realistic traffic states, and a closed-loop evaluation environment was built around them by combining vehicle motion constraints and traffic interaction rules. The agent perceives a state, issues a decision or acceleration command, the environment updates the vehicle state through a transition model, and the loop continues. All experiments were repeated five times with different random seeds, and reported results are means with standard deviations.

The car-following results are striking. The proposed model held an average inter-vehicle distance of 20.1 meters, about 8.1 percent greater than the comparison models, with spacing fluctuations contained within plus or minus 1.2 meters, compared with fluctuations of roughly 2.0 meters for a TD3 benchmark and even wider swings for the rule-based IDM, which at times let the gap fall to around 14.2 meters. The collision rate dropped to 1.3 percent, more than 55 percent lower than the TD3 benchmark, and the average headway settled at a stable 1.95 seconds, almost entirely above the 1.5-second safety threshold. Comfort improved in parallel: average jerk, the rate of change of acceleration that passengers feel as lurching, fell to 0.72 meters per second cubed, roughly 23 percent below TD3, and the comfort satisfaction rate reached 93.8 percent, with a safety constraint satisfaction rate of 96.5 percent. The model also converged in about 150 training rounds, roughly 28.6 percent fewer than TD3, achieving the highest average reward of 96.8.

The hierarchical decision layer showed comparable gains. Tested against standard DQN and Proximal Policy Optimization baselines, the DDQN decision module sustained accuracy between 0.88 and 0.96, peaking near 0.96 in a mid-session window and staying above roughly 0.90 during a late-session period when both baselines degraded, an improvement of about 5 to 8 percent overall. Its conflict rate stayed between 0.05 and 0.10, and at one point measured about 0.08 where DQN reached 0.22, a reduction of more than 60 percent. The agent also changed lanes about 23.7 percent less often than DQN on average, with a redundant decision share of only 10.4 percent, and it responded faster: decision latency averaged around 140 milliseconds, more than 25 percent quicker than DQN, with the narrowest latency distribution across free-flow, dense, cut-in, and ramp-merge scenarios. An ablation study confirmed that neither module alone explains the gains: the full DDQN-DDPG framework lifted decision accuracy to 93.5 percent, cut the collision rate to 7.6 percent, and raised the average reward to 97.2, outperforming both single-module variants on every metric.

The study is candid about its limits. The dual-module hierarchy adds computational overhead that could strain resource-constrained onboard platforms, although the author notes the extra cost falls mainly on offline training while online decisions remain fast. The evaluation environment, however realistic its initialization from nuScenes, still substitutes a transition model for the raw unpredictability of the open road, and real-vehicle closed-loop validation remains future work. The framework also does not yet account for heterogeneous road users, uncertain road conditions, or driver-specific preferences. Even so, the central lesson travels well beyond one paper: when a driving task spans two fundamentally different kinds of choice, deciding what to do and deciding how precisely to do it, forcing a single network to learn both may be the bottleneck. Splitting the problem across cooperative specialists, each trained with the other’s constraints in view, may be how autonomous vehicles finally learn to drive not just safely, but smoothly enough that their passengers stop noticing the robot at the wheel.

Subject of Research: Hierarchical reinforcement learning integrating DDQN and DDPG for intelligent driving decision-making in complex traffic scenarios

Article Title: Multi-modal intelligent driving decision-making in complex traffic scenarios integrating DDQN and DDPG

Article References: He, L. (2026). Multi-modal intelligent driving decision-making in complex traffic scenarios integrating DDQN and DDPG. Discover Artificial Intelligence, 6(1), Article 1503. https://doi.org/10.1007/s44163-026-02454-y

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02454-y

Keywords: autonomous driving, hierarchical reinforcement learning, DDQN, DDPG, car following, decision-making, particle swarm optimization, nuScenes, traffic safety, lane changing, deep learning, intelligent vehicles

The research, led by Zexin Mu and colleagues at the State Grid Heilongjiang Electric Power Company Limited Electric Power Research Institute, together with a collaborator at Tianjin Pingyun Electric Power Technology, tackles a stubborn problem in modern grid operations. As China pushes toward its dual carbon goals and builds power systems dominated by renewable energy, the volatility and intermittency of wind and solar output have become the grid’s greatest vulnerability. Under extreme weather, the spatial and temporal distribution of renewable generation is tightly constrained by meteorological conditions, and the risk of a sudden supply-demand imbalance grows sharply. Traditional defense strategies, which span pre-disaster prevention, during-disaster emergency response, and post-disaster recovery, have helped, but they lean on precise physical models and deterministic optimization solvers that struggle when system states change rapidly and control variables multiply.

The core difficulty, the authors explain, is twofold. First, physics-based methods suffer from accumulated modeling errors when extreme weather makes it nearly impossible to obtain accurate parameters in real time. Second, with large-scale renewable integration, the dimensionality of the control problem explodes, imposing computational burdens that make it hard to generate feasible pre-control schemes within the limited decision-making windows available to operators. Deep reinforcement learning offers an alternative: by letting an agent learn through trial-and-error interaction with a simulated environment, it can discover effective policies without requiring perfect models. But existing DRL approaches have their own flaws, including blind early exploration, a tendency to generate physically impossible actions, and a narrow focus on single resource types rather than the coordinated use of multiple flexible assets.

To close those gaps, the team built their method on an improved version of the deep deterministic policy gradient, or DDPG, algorithm, fusing physical knowledge with data-driven learning. The centerpiece is a reward function embedded with the physical rules of active regulation flexibility. Rather than treating equipment constraints as an afterthought—simply clipping actions or adding generic penalties—the researchers introduced equipment operating boundary constraint penalties and regulation margin incentives directly into the policy update process. The agent learns to proactively perceive and maintain the system’s flexibility margin throughout training. Crucially, the constraint-penalty pairs are constructed dynamically according to the severity of the extreme weather event, so the penalty intensity adapts as environmental conditions shift, unlike the static action clipping used in methods such as TD3.

The framework also incorporates two additional mechanisms borrowed and refined from recent reinforcement learning research. A dynamic entropy regularization term is added to the action-value function, adaptively balancing exploration against exploitation. The entropy term rewards policies whose action distributions remain relatively uniform, encouraging the agent to try low-probability actions during early training and avoid premature convergence to suboptimal strategies. Meanwhile, a twin-delayed network structure employs two independent Critic networks that estimate Q-values separately, with the smaller of the two estimates taken as the target, while the Actor network’s update frequency is delayed. This combination mitigates the notorious Q-value overestimation problem that plagues standard DDPG, improving the robustness of value estimation and the stability of policy training. The authors note that, unlike the original TD3 algorithm, their approach integrates the twin-delayed structure uniformly into the knowledge-data fusion framework, forming a synergistic optimization with dynamic constraint construction and physical rule embedding.

On the modeling side, the researchers constructed mechanism models for wind power and photovoltaic output, including a piecewise wind power curve governed by cut-in, rated, and cut-out wind speeds, and a photovoltaic model that accounts for real-time irradiance and temperature effects. An hourly time-series energy storage model tracks the state of charge of storage units, with charging and discharge efficiencies set at 0.95 and initial state of charge fixed at 50 percent of rated capacity each day. These models, combined with 24-hour time-series meteorological data, allow the team to quantify the supply-demand imbalance risk under extreme weather scenarios before the AI ever begins generating control schemes.

The experiments drew on publicly available datasets, including the CNPS-Met meteorological dataset built from ERA5 reanalysis data and observations from 2,598 national ground stations, spanning 31 years of daily gridded data with 11 categories of high-impact weather events. Load data came from the China Southern Power Grid load forecasting competition dataset within the EnergyTS collection, while energy storage and unit parameters referenced clean energy scenario data from the National Basic Science Data Center. Extreme weather events were selected using meteorological thresholds: typhoon scenarios required sustained winds above 17.2 meters per second—Beaufort scale 8 or above—lasting more than six hours, while ice disaster scenarios required daily average temperatures below minus 5 degrees Celsius accompanied by precipitation. The data were split into training, validation, and test sets at a ratio of 7:1.5:1.5, and all experiments ran on an Intel Xeon Gold 6248R CPU with an NVIDIA RTX 4090 GPU and 128 GB of memory.

The results are striking. Under extreme weather scenarios, the proposed method achieved a total pre-control cost of 120.6 CNY, a success rate of 94.7 percent, and a violation rate of 0.0 percent, outperforming five benchmark schemes including preventive control of successive failures, a multi-level operation method, a risk-averse deep learning approach, mixed-integer linear programming, and standard DDPG. The total cost was 13.1 percent lower than the second-best method, and the success rate exceeded MILP by 3.1 percentage points and standard DDPG by 15.4 points. On computational efficiency, the method’s average decision time of 675.9 milliseconds was slower than the risk-averse deep learning method’s 198.3 milliseconds but vastly faster than MILP’s 47.8 seconds, comfortably meeting the real-time requirements of day-ahead scheduling. Ablation experiments confirmed that each improved module contributes synergistically: removing the twin-delayed network raised total cost by 28.8 percent and cut the success rate by 11.8 percentage points, while removing boundary clipping or the penalty term allowed violation rates to climb to 6.2 and 4.8 percent respectively.

Visualization experiments under a compound extreme weather event—strong wind accompanied by low temperature lasting eight hours—illustrated the difference in physical realism. The proposed method kept the energy storage state of charge above 0.33 during the extreme period, never violating the lower bound of 0.2, whereas standard DDPG let it drop to 0.12 between 12:00 and 14:00. Storage power under the proposed scheme stayed smoothly within plus or minus 10 MW, well below the 15 MW rating, while standard DDPG pushed discharge to 18 MW beyond the rated boundary. The proposed method also raised conventional unit output smoothly from 24 MW to roughly 30 MW to provide orderly power support, while standard DDPG exhibited erratic fluctuations. In scenario identification tests across typhoon, ice disaster, extreme heat, and compound events, the method achieved 94.0 percent overall accuracy compared with 74.5 percent for standard DDPG, and generalization experiments on modified IEEE 33-bus and 123-bus systems showed success rates of 93.2 and 91.8 percent with zero violations. Together, these findings suggest that embedding physical knowledge deep inside reinforcement learning could give grid operators a powerful new tool for staying ahead of the weather’s worst moods.

Subject of Research: Deep reinforcement learning for pre-control of power imbalance risk under extreme weather

Article Title: Pre-control scheme generation for extreme-weather power imbalance risk using multi-resource frequency regulation and deep reinforcement learning

Article References: Mu, Z., Hu, Y., Chen, H., Zheng, J., & Wang, Z. (2026). Pre-control scheme generation for extreme-weather power imbalance risk using multi-resource frequency regulation and deep reinforcement learning. Discover Artificial Intelligence, 6(1), Article 1379. https://doi.org/10.1007/s44163-026-02335-4

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02335-4

Keywords: deep reinforcement learning, power grid resilience, extreme weather, DDPG, energy storage, frequency regulation, renewable energy, power imbalance risk, entropy regularization, twin-delayed networks, pre-control schemes, smart grid

The team’s starting point was a full six-degree-of-freedom missile model, the standard simulation framework for capturing how an air-to-ground weapon translates and rotates in three dimensions under the combined influence of thrust, gravity, drag, and aerodynamic moments. Within this virtual environment, the researchers kept the guidance architecture conventional where it worked well: yaw was handled through waypoint navigation, and pitch followed Proportional Navigation Guidance, the time-honored law that steers an interceptor by closing the line-of-sight rotation rate toward zero. The roll channel, however, was the battleground. A baseline proportional-integral-derivative controller, the workhorse of flight control for decades, provided the reference performance against which every artificial intelligence approach would be measured.

Roll control matters more than casual observers might assume. For an axisymmetric missile, roll couples into pitch and yaw dynamics, meaning that an uncontrolled roll can smear the seeker’s view of the target, distort the effectiveness of canard or fin deflections, and induce structural loading that a slender airframe was never designed to absorb. Prior research has documented how elastic deformation, wrap-around fins, and spin-deformation coupling all conspire to make the roll channel notoriously difficult, particularly for small air-to-surface missiles where actuator authority is limited and disturbances arrive fast. The Turkish-Indonesian team set out to see whether learned controllers could integrate sensor alignment with structural integrity constraints better than a tuned PID loop, especially when the missile is forced into aggressive terminal maneuvers.

The researchers trained and compared four AI strategies. Linear regression served as the simplest baseline, learning a direct mapping from flight states to roll control inputs. A neural network built on long short-term memory units brought the ability to retain information over time, a natural fit for a dynamical system whose history matters. Deep Deterministic Policy Gradient, a reinforcement learning algorithm that continuously refines a control policy through interaction with the environment, added the capacity to discover control strategies that no human engineer would hand-code. Finally, the team built an ensemble combining all three, letting the strengths of regression, recurrent memory, and reinforcement learning compensate for one another’s weaknesses. Crucially, all models were trained on data generated by the PID controller itself, meaning the AI systems learned from the accumulated experience of the classical controller before surpassing it.

Testing took place under two families of scenarios: straight flight toward the target, and aggressive yaw maneuvers involving 90-degree and 180-degree turns during the terminal phase. Performance was scored on two fronts. The first was roll angle deviation, a direct measure of how well each controller kept the missile’s roll attitude near its commanded value. The second was terminal miss distance, the ultimate metric in guided munitions, capturing how close the weapon came to its intended impact point. A controller that holds roll beautifully but misses the target by meters is useless; one that hits within centimeters while keeping the airframe stable is the goal.

The ensemble method emerged as the clear winner. In straight flight it delivered near-minimal roll error, essentially matching the best achievable stabilization. During the punishing 90-degree and 180-degree yaw maneuvers, where rapid changes in lateral acceleration ripple through the roll dynamics, the ensemble maintained superior stability while the simpler models struggled. Most strikingly, it achieved the lowest miss distance recorded in the study, 0.14 meters, in the most demanding scenarios. For context, a miss distance measured in tens of centimeters rather than meters represents the difference between a weapon that reliably defeats a target and one that may require multiple engagements.

Deep Deterministic Policy Gradient on its own also proved robust, posting consistently low miss distances and handling the nonlinearities and disturbances that plague the terminal phase with notable composure. This is consistent with a broader trend in aerospace research, where reinforcement learning controllers have shown an appetite for exactly the kinds of high-dimensional, nonlinear problems that frustrate classical gain-scheduled autopilots. The reinforcement learning agent, in effect, learns through trial and error within the simulation which control actions keep the roll channel quiet under conditions no single linear model can describe, and it can do so without an engineer manually re-tuning gains for every flight regime.

The weaker performers were equally instructive. Linear regression and the LSTM network exhibited noticeably higher roll errors and miss distances, indicating limited adaptability to rapid changes in flight conditions. The lesson is not that machine learning fails, but that the choice of architecture matters enormously. A static regression model can interpolate within the data it has seen but has little capacity to respond gracefully when the missile’s dynamics depart from its training distribution, as they inevitably do during a violent turn. Even a recurrent network with memory, trained purely on PID-generated trajectories, appears to inherit some of the classical controller’s limitations. The ensemble’s success suggests that diversity of learning paradigms, rather than any single clever algorithm, is what buys robustness at the edge of the flight envelope.

The implications extend beyond munitions. The same roll-stabilization problem appears in reusable launch vehicles, hypersonic gliders, and small autonomous aircraft, and the study’s authors sit squarely within a research community, spanning institutions from Marmara University to Indonesia’s National Research and Innovation Agency, that has been systematically probing where AI control can replace or augment classical loops. Their work was supported by Turkey’s TÜBİTAK 2209-A university research program and Indonesia’s Rumah Program, and it follows earlier contributions from the same group on multi-phase missile guidance and reinforcement learning for landing guidance. The pattern across this literature is consistent: learned and hybrid controllers tend to shine precisely where dynamics are fastest and most nonlinear, which is to say where classical control is weakest.

Caveats remain, and the authors are candid about them. Everything reported here is simulation. A six-degree-of-freedom model, however sophisticated, cannot fully capture sensor noise, actuator saturation and lag, structural flexing, or the computational constraints of real flight hardware. The researchers explicitly state that real-world hardware validation remains essential to confirm the simulation outcomes and to support any operational deployment, a caution that echoes throughout the field of AI-augmented flight control. Verification and certification of learned controllers is an unsolved regulatory problem, and no air force will fly a neural network it cannot formally verify. Still, the trajectory of the evidence is hard to ignore. When an ensemble of regression, memory networks, and reinforcement learning can hold a simulated missile steady through a 180-degree terminal turn and guide it to within 14 centimeters of its aim point, the classical autopilot has been put on notice. The next step is proving it in the air.

Subject of Research: AI-based roll stabilization control for air-to-ground missiles during dynamic terminal flight

Article Title: AI-Augmented Roll Stabilization for Missiles Under Dynamic Terminal Flight Conditions

Article References: Abdurrauf, A., Nugroho, L., & Ülkü, E. E. (2026). AI-Augmented Roll Stabilization for Missiles Under Dynamic Terminal Flight Conditions. International Journal of Aeronautical and Space Sciences. https://doi.org/10.1007/s42405-026-01233-4

Image Credits: AI Generated

DOI: 10.1007/s42405-026-01233-4

Keywords: missile guidance, roll stabilization, artificial intelligence, reinforcement learning, DDPG, LSTM neural networks, PID control, six-degree-of-freedom simulation, terminal flight, miss distance, flight control systems, ensemble learning

The core of the system is a two-layer architecture that separates the geometry of pathfinding from the judgment of how cautious to be. The lower layer generates candidate trajectories using the A* algorithm, a classic graph-search method dating back to a landmark 1968 paper by Hart, Nilsson and Raphael, which finds minimum-cost paths by exploring the most promising branches of a grid first. What makes the Inha team’s implementation distinctive is the structure of its cost function. Every candidate path is scored by a weighted combination of two terms: a distance term that favors shorter routes and shorter required times of arrival, and a risk term that penalizes exposure to hazards. Those hazards come in two flavors, static and dynamic. Static risk is the urban landscape itself, the buildings that a low-altitude air taxi must thread between. Dynamic risk is the weather, specifically wind and gust severity, which can change from one minute to the next and turn a safe corridor into a dangerous one.

The upper layer, and the true innovation of the study, is the AI pilot itself: a reinforcement-learning agent trained with the deep deterministic policy gradient algorithm, or DDPG, a technique introduced by Lillicrap and colleagues in 2016 that excels at continuous control problems. The agent’s single job is to choose a continuous weighting factor, denoted lambda, which controls the relative influence of the distance and risk terms inside the A* cost function. In other words, the AI does not draw the path directly. Instead, it decides how much the path planner should care about speed versus safety, given the current operational context. When the battery is healthy and the air is calm, the agent can push lambda toward efficiency, letting the vehicle fly direct, fast routes. When the state of charge falls or gusts intensify, the agent shifts the weighting toward risk avoidance, and the planner responds by carving out longer but safer detours around buildings and turbulent zones.

One design decision stands out as particularly consequential. Battery state-of-charge and wind information are fed to the policy as continuous observations, not as discrete emergency flags. This choice matters because it changes the character of the system’s behavior at the boundary between normal and abnormal flight. A flag-based system, in which an emergency is declared when some threshold is crossed, tends to produce abrupt, discontinuous responses: the trajectory is one thing before the threshold and something quite different after it. The learned policy, by contrast, produces a lambda that responds smoothly and monotonically to gradually degrading conditions. As the battery drains cell by cell or the wind stiffens knot by knot, the safety weighting tightens proportionally, and the planned path bends incrementally away from hazard. Yet the researchers report that the same policy still reacts decisively when a genuine emergency arrives without warning, snapping to a strongly safety-dominant configuration when the situation demands it.

Validation came through two complementary evaluation strategies. The first was a Monte Carlo benchmark, in which the AI-assisted system was pitted against rule-based baselines and against a planner with a fixed lambda value across many randomized scenarios. Monte Carlo testing is the standard way to expose brittleness in autonomous systems, because it samples the space of possible conditions rather than testing a handful of hand-picked cases. The second was a controlled context-sweep case study, in which a single operational variable, such as battery level or wind severity, was systematically varied while the system’s responses were recorded. This sweep allowed the team to verify the smooth, monotonic behavior of the weighting factor directly, confirming that the agent’s decisions track the operational context in an interpretable way rather than jumping erratically between modes.

Performance was judged on three metrics that map cleanly onto what a passenger would actually feel and what an operator would actually care about. Path length measures efficiency, the extra distance flown to reach the destination. Risk exposure quantifies how much of the flight is spent near buildings or in severe wind and gust conditions. Trajectory smoothness captures how gentle the resulting path is, a property with direct consequences for passenger comfort, vehicle dynamics and control authority. Smoothness is not a cosmetic concern; certification frameworks for piloted aircraft, including the long-standing military flying-qualities specification MIL-F-8785C, treat handling qualities as central to airworthiness, and a replanning system that produced jagged, erratic paths would be unusable in practice regardless of how safe it was on paper.

The study situates itself within a decade of growing interest in flight-deck decision support for advanced air mobility. NASA researchers, including Karr, Ballin, Barrows and colleagues, have developed autonomous operations planners and flight path management automation specifically for high-density urban environments, and flight evaluations of such systems have already been conducted. The NASA urban air mobility maturity level scale, described by Goodrich and Theodore in 2021, provides the community’s roadmap for how these operations might scale from today’s limited demonstrations to high-density fleets. What the Inha work adds to this lineage is the learned, context-sensitive middle layer: rather than encoding the safety-efficiency trade-off as a fixed engineering choice made in advance, the system lets a trained policy make that choice in flight, informed by the actual state of the vehicle and the atmosphere.

The choice of DDPG as the learning algorithm is also worth appreciating on its own terms. Reinforcement learning agents come in many forms, but many of the most successful ones, including the policy-gradient methods behind celebrated game-playing systems, operate over discrete action spaces. Trajectory weighting does not fit that mold. The appropriate balance between speed and safety is not one of several options but a dial with infinitely many settings, and DDPG was designed precisely for such continuous action spaces, using an actor network to propose actions and a critic network to evaluate them. Related work has already shown the algorithm’s usefulness in safety-critical planning, including an improved DDPG model for emergency fire-escape path planning published in 2024, and the Inha team’s application extends that idea into the three-dimensional, hazard-rich environment of urban airspace.

The implications for the emerging air taxi industry are considerable. Certification authorities and operators alike have struggled with the question of how much autonomy to trust in the cockpit, and a system like this one offers a middle path: the AI does not fly the vehicle, it advises the trajectory planner, adjusting a single, physically meaningful parameter that a human pilot or ground operator could inspect and, in principle, override. The researchers describe their AI pilot as a decision-support tool for both gradual context changes and emergency responses during urban air mobility operations, a framing that keeps the human in the loop while offloading the relentless, second-by-second arithmetic of risk and distance. The work was supported by the National Research Foundation of Korea, and the authors report no competing interests.

Challenges remain before such a system carries passengers. The study validates the concept in simulation, and the gap between Monte Carlo benchmarks and flight test in a real city, with real air traffic, real sensor noise and real regulatory scrutiny, is the widest gap in aviation. The authors note that the data supporting the findings are available upon reasonable request, which will help other groups scrutinize and extend the approach. Still, the conceptual contribution is clean and potentially durable: treat the safety-efficiency trade-off not as a design constant but as a learned function of the operational context, and let a continuous, smoothly responding policy turn that function into concrete flight paths. As fleets of electric vertical-takeoff vehicles move closer to routine operation over the world’s cities, the flights that matter most will not be the ones that go according to plan. They will be the ones that do not, and the machines that can quietly, smoothly find a better way home may prove to be the technology that finally makes urban flight trustworthy.

Subject of Research: AI-based emergency trajectory replanning for urban air mobility vehicles

Article Title: AI-Based Pilot Assistance System for Emergency Trajectory Replanning in Urban Air Mobility

Article References: Yu, J., & Choi, K. (2026). AI-Based Pilot Assistance System for Emergency Trajectory Replanning in Urban Air Mobility. International Journal of Aeronautical and Space Sciences. https://doi.org/10.1007/s42405-026-01258-9

Image Credits: AI Generated

DOI: 10.1007/s42405-026-01258-9

Keywords: urban air mobility, AI pilot, reinforcement learning, DDPG, A* algorithm, trajectory replanning, flight path management, safety-efficiency trade-off, wind hazards, battery state of charge, path planning, decision support

The research, led by Dhanish Ladwani, Siddhi Jaiswal, Gurupreet Dhande, Arnav Kalambe and Akhil Gupta of Symbiosis Institute of Technology in Nagpur, India, addresses one of the most stubborn engineering bottlenecks on the road to 6G. Their proposed framework, dubbed DT-RIS-AI, fuses three previously separate strands of wireless research: physics-aware terahertz channel modelling, reconfigurable intelligent surfaces (RIS), and reinforcement learning running on a continuously synchronized digital replica of the network. The authors’ central insight is architectural as much as algorithmic — rather than treating these components in isolation, they wire them into a single closed feedback loop in which a virtual copy of the network teaches an AI agent how to configure physical hardware before conditions in the real world turn hostile.

Reconfigurable intelligent surfaces are the linchpin of the scheme. An RIS is essentially a flat panel studded with dozens to hundreds of nearly passive, sub-wavelength reflecting elements, each of which can apply a programmable phase shift to an incoming electromagnetic wave. When the phase shifts are aligned correctly, the reflections combine coherently at the receiver, and the received power scales quadratically with the number of elements — an enormous advantage at terahertz frequencies, where direct links are frequently buried in noise. Left unconfigured, the same reflections add non-coherently, yielding only a linear, and largely useless, power gain. The catch is that finding the optimal phase configuration for hundreds of elements, while simultaneously allocating transmit power across multiple interfering users, is a high-dimensional, non-convex optimization problem that classical techniques cannot solve quickly enough for channels that change in milliseconds.

The terahertz channel model underpinning the framework is deliberately physics-consistent. The team’s path-loss formulation combines the standard frequency-scaled free-space term with an exponential molecular absorption factor, using a reference absorption coefficient of approximately 0.0033 per metre at 1 terahertz — a value representative of standard atmospheric conditions at 296 kelvin, 1 atmosphere, and 50 percent relative humidity. Water vapor dominates absorption in this band, and because the coefficient grows with the square of frequency, the attenuation is strongly frequency-selective. Residual scattering is captured with a conservative Rayleigh fading model, while the dominant line-of-sight and RIS-reflected components are handled explicitly through deterministic path-loss terms. The authors note that the framework is agnostic to the specific fading statistics, so a Rician or sparse-multipath model could be swapped into the digital twin’s channel module without touching the control logic.

What distinguishes DT-RIS-AI from prior digital-twin proposals is the way synchronization is coupled to the learning loop. The virtual model ingests channel state information and user-distribution data every 10 milliseconds, matching the coherence time of slowly varying terahertz IoT deployments, and maintains a mismatch tolerance of 5 percent in estimated signal-to-interference-plus-noise ratio. If observed divergence exceeds that threshold, an event-triggered re-synchronization cycle fires immediately rather than waiting for the next scheduled update. This mismatch-triggered mechanism bounds how much stale or erroneous virtual state can leak into the learned control policy — a level of synchronization-aware coupling the authors identify as the specific mechanism-level contribution of the work. A short-horizon linear extrapolation model projects channel gain, user distance, and SINR trends one interval ahead, giving the AI agent a predictive, rather than purely reactive, view of the network.

Inside this virtual environment, a deep deterministic policy gradient (DDPG) agent learns to jointly set RIS phase shifts and transmit power. The reward function is explicitly multi-objective, weighting aggregate throughput against total power consumption and average latency, subject to minimum per-user SINR constraints that guarantee quality of service. The actor and critic networks are fully connected feedforward models with hidden layers of 256 and 128 neurons, trained with the Adam optimizer, soft target updates, and a replay buffer of one hundred thousand transitions. The authors selected DDPG for its sample efficiency and deterministic policy structure, which suits continuous control of phase vectors and power levels, though they acknowledge that newer algorithms such as TD3, SAC, and PPO represent natural extensions awaiting comparative evaluation. Training converges when the moving average of episodic reward changes by less than 1 percent over fifty consecutive episodes.

The simulation results are striking. Across sweeps of user density, transmit power, and RIS size, the DT-RIS-AI framework improved per-user signal-to-noise ratio by more than 10 decibels over a no-RIS baseline and by 8 to 10 decibels over a passive RIS configuration, with additional gains of 3 to 5 decibels over a digital-twin-assisted greedy beamforming scheme that the team included as an intermediate benchmark. Throughput improved by 48 to 63 percent and energy efficiency by 35 to 42 percent compared with conventional and passive RIS-assisted approaches. At larger surface sizes around 256 elements, the framework delivered 25 to 30 percent higher throughput than the greedy method and 60 to 70 percent more than passive RIS, while coverage probability climbed 10 to 12 percent above the greedy approach. Outage probability, meanwhile, fell by up to two orders of magnitude relative to the greedy scheme at moderate SINR thresholds, and the framework reached near-stable throughput within 40 to 60 time steps — a convergence speed the authors attribute to the AI agent’s efficient exploration of the configuration space.

The researchers are candid about scope. The evaluation is simulation-based rather than validated on hardware or ray-traced testbeds, the baselines exclude alternative optimization paradigms such as successive convex approximation or semidefinite relaxation, and the reported ranges reflect parameter sweeps rather than statistical variation across repeated randomized trials. Practical hardware impairments — discrete phase quantization, phase noise, and mutual coupling between closely spaced elements — would erode the idealized quadratic RIS gain, and the authors note that at their low-power, extended-range operating point the absolute per-user SNR still sits below the conventional threshold for reliable low-order modulation, meaning link closure would demand more transmit power, shorter distances, or larger surfaces. They also flag the computational and privacy costs of continuously mirroring a physical network inside a virtual one, an increasingly pressing concern as digital twins migrate from manufacturing floors into telecommunications infrastructure.

Even with those caveats, the work sketches a persuasive blueprint for what self-optimizing 6G networks might look like. By shifting RIS control from reactive channel estimation to proactive, prediction-driven configuration, the DT-RIS-AI architecture reduces the crippling signaling overhead that has plagued terahertz deployments, where feedback rates normally scale with the inverse of channel coherence time. The authors point toward future extensions including multi-RIS distributed multi-agent optimization, hybrid active-passive surface architectures, realistic mobility and blockage modeling, and joint communication-and-sensing capabilities. If those promises materialize, the invisible wallpaper of intelligent surfaces lining our buildings may one day quietly reconfigure itself hundreds of times per second — choreographed not by engineers, but by an AI agent rehearsing endlessly inside its own digital twin.

Subject of Research: AI-driven digital twin control of reconfigurable intelligent surfaces in terahertz IoT mesh networks for 6G

Article Title: Artificial intelligence driven digital twin for dynamic reconfigurable intelligent surface control in terahertz internet of things mesh networks

Article References: Ladwani, D., Jaiswal, S., Dhande, G., Kalambe, A., & Gupta, A. (2026). Artificial intelligence driven digital twin for dynamic reconfigurable intelligent surface control in terahertz internet of things mesh networks. Discover Artificial Intelligence, 6(1), Article 1212. https://doi.org/10.1007/s44163-026-02255-3

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02255-3

Keywords: terahertz communication, reconfigurable intelligent surface, digital twin, deep reinforcement learning, 6G networks, Internet of Things, mesh networks, DDPG, energy efficiency, molecular absorption, beamforming, SINR

The challenge is formidable. Multiple pursuit spacecraft must cooperatively capture an evasive target while operating under impulsive maneuver constraints and strict fuel budgets, producing an optimal control problem marked by strong nonlinearity and complicated constraints. The researchers note that the tools traditionally brought to bear on such problems each fall short in some way. Classical differential game methods suffer from high computational complexity and poor real-time performance, making them difficult to deploy on hardware with limited processing power. Numerical optimization approaches, meanwhile, are notoriously sensitive to initial guesses and struggle to handle constraints that persist over long durations. Deep reinforcement learning has shown genuine promise in some orbital scenarios, but existing algorithms lack the ability to proactively predict the actions of an opponent confronting them with an unknown evasion strategy, which limits how well the pursuit can be planned. The key question, therefore, became how to predict the evading spacecraft’s actions in multi-to-one scenarios and then optimize the cooperative strategies of the pursuers accordingly.

The team’s answer is a pursuit strategy construction method that fuses an evader action prediction network with the Deep Deterministic Policy Gradient algorithm, a well-established deep reinforcement learning technique for continuous control. The mathematical foundation of the work rests on the Clohessy-Wiltshire equations, the standard linearized description of relative spacecraft motion, which the researchers use to build a model of the multi-to-one impulsive orbital pursuit-evasion game in the Local Vertical-Local Horizontal reference frame. In this modeled world, both sides execute impulsive velocity-increment maneuvers at fixed time intervals, and each side faces constraints on both the size of each individual maneuver and the cumulative total velocity increment it may expend. Multiple pursuers cooperatively close in on the evader through finite sequences of these impulses, and the mission is declared a success the moment the distance between any pursuer and the evader falls below a preset capture threshold. Communication among the pursuers is itself constrained: information can be shared only within a certain proximity, adding a further layer of realism to the problem.

To train the pursuit policy, the researchers adopted a centralized training with decentralized execution framework, an architecture that has become popular in multi-agent reinforcement learning. During training, a single policy network is updated using global state information, but at execution time each pursuer relies only on its own local observations to generate impulsive maneuver commands. Crucially, all pursuers share the same network parameters, meaning that a fleet of chasing spacecraft can behave in a coordinated manner without needing a central commander in the loop during the actual engagement. The observation available to each pursuer is rich: it comprises the relative state with respect to the evader, the states of the two nearest fellow pursuers within communication range, and the predicted future state of the evader at the next time step, produced by the prediction network, for a total of 29 dimensions. The policy network maps this observation to a three-dimensional continuous impulsive velocity-increment command.

The reward structure that shapes the learning process is deliberately simple but effective. It combines a continuous reward for approaching the evader, a large constant reward for a successful capture, and an additional penalty for fuel consumption, so that the pursuing agents learn to be aggressive yet economical. Alongside the policy network, the team trained a separate evader action prediction network through supervised learning. Its input consists of the relative states of the three pursuers closest to the evader, and its output is a prediction of the evader’s velocity increment, effectively giving each pursuer a glimpse of the opponent’s likely next move. Importantly, this prediction network is trained using the global state information and the evader’s true action data collected during the pursuit policy training itself, so no additional data acquisition is required. All of the networks involved, including the policy network, the Q-value network, and the prediction network, are four-layer fully connected architectures in which the first three layers use the ReLU activation function and the output layer employs tanh.

The simulation campaign that validated the approach produced striking results. Over 200,000 training episodes, the reward trends of the pursuer and evader policies ran in generally opposite directions and stabilized after roughly 125,000 episodes, a signature that both sides had reached an approximate game equilibrium. When the researchers compared training curves for the pursuit policy with and without evader action prediction, the incorporation of prediction significantly improved both the convergence speed and the reward values achieved, while the loss curve of the prediction network itself converged after about 1,000 epochs. In test engagements, trajectories produced under three different prediction approaches, the learned prediction network, conventional Kalman filtering, and simply assuming the evader repeats its previous action, all achieved successful capture, but their accuracy differed markedly.

The prediction network proved decisively superior in tracking the evader’s future position. Its prediction error remained essentially below 0.3 kilometers throughout the pursuit-evasion process, with an average of just 0.137 kilometers, and it gradually decreased in the later stage of the engagement as the pursuers closed in. By contrast, the errors of the comparative methods approached 0.6 kilometers in the later stages, more than four times worse. The statistical picture was equally compelling. With three pursuers deployed, the capture success rate of the prediction-augmented method reached 0.95, an improvement of approximately 0.3 over the conventional DDPG approach. The researchers highlight that this advantage effectively compensates for the disadvantage of insufficient pursuer numbers, a point of real practical significance in a domain where every additional spacecraft carries substantial cost.

The study is also candid about the limits of the method. When the evader adopted a non-maneuvering strategy, the capture success rate climbed to 0.99, but performance degraded when the pursuers confronted unseen random or periodic evasion strategies, exposing a generalization bottleneck that is common to learning-based policies. The geometry of the opening position mattered as well: the initial distribution of the pursuers significantly affected the success rate, with an optimal initial distance of approximately 38 kilometers and an optimal enclosure area of approximately 2,300 square kilometers. The authors acknowledge that the approach currently relies on prior action data of the evader, and they outline future work on prediction methods that do not require such data, as well as extension to the even harder multi-to-many pursuit-evasion game problem.

For the wider space community, the significance of this work lies in its demonstration that foresight and learning can be combined to produce autonomous, fuel-aware, and cooperative pursuit behavior under realistic constraints. As activities in orbit multiply, from Mega-constellations to debris removal and on-orbit servicing, the ability of spacecraft to make fast, intelligent decisions in adversarial or uncertain encounters is poised to become a core capability. By showing that a supervised prediction of an opponent’s maneuver can be woven directly into a reinforcement learning pursuit policy, and by quantifying exactly how much that foresight is worth in terms of convergence speed and capture success, the Harbin Institute of Technology team has provided both a working method and a meaningful engineering reference for the autonomous decision-making systems that future space missions will demand.

The reliance on the Clohessy-Wiltshire equations places this work within a long tradition of orbital relative-motion analysis. These linearized equations assume a circular reference orbit and short engagement distances, which keeps the dynamics tractable enough for learning-based methods while remaining a reasonable approximation for proximity operations. The choice of impulsive maneuvers, rather than continuous thrust, mirrors the pulsed thruster firings common on real spacecraft, where each burn is short compared with the orbital period and fuel is measured in velocity increment.

The decentralized execution aspect of the training framework carries practical weight for on-orbit deployment. Because each pursuer generates its maneuver commands from local observations alone, the approach tolerates the intermittent communication links and latency that plague multi-spacecraft operations, where a central coordinator could become a single point of failure. The communication-range constraint built into the model reflects this operational reality.

The comparison against Kalman filtering is also instructive. Kalman filters estimate a target’s state from noisy measurements but do not anticipate deliberate maneuvering, so their predictions degrade precisely when an evader fires its thrusters. A learned predictor trained on maneuver data can instead capture behavioral patterns that a purely dynamical filter cannot represent. The reported average prediction error of 0.137 kilometers, against roughly 0.6 kilometers for the alternatives late in the engagement, quantifies this advantage in terms familiar to guidance engineers.

The acknowledged dependence on prior evader action data remains the principal caveat, since real adversaries may behave in ways no training set anticipated.

Subject of Research: A predictive learning-based pursuit strategy for multiple-to-one orbital pursuit-evasion games among spacecraft.

Article Title: A predictive learning-based pursuit strategy for the multiple-to-one orbital pursuit-evasion game

Article References: A predictive learning-based pursuit strategy for the multiple-to-one orbital pursuit-evasion game. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: orbital pursuit-evasion, spacecraft, deep reinforcement learning, DDPG, action prediction, Clohessy-Wiltshire equations, impulsive maneuvers, CTDE, autonomous decision-making, space security, Harbin Institute of Technology, Space Science and Technology