Users with long packets, typically located far away and hidden behind buildings, are served through a cascaded link in which the signal travels from a ground base station to an RIS array mounted on the UAV, which reflects and steers the wave toward the user. Users with short packets, positioned near the UAV with a clear line of sight, are served directly by the drone, which acts as a compact semantic transmitter after receiving highly compressed information from the base station. The two modes share the spectrum through frequency division, while users within each mode are separated in time, so the model excludes cross-user interference by design.

The key conceptual shift is that the system does not optimize bit throughput. Instead, it targets semantic quality of experience, a measure that combines how much of the task-relevant meaning arrives with how efficiently it is delivered. Semantic communication, in this framework, extracts only the core features a downstream task needs, quantified through a semantic entropy measure: the expected minimum number of semantic symbols sufficient to predict the task outcome.

The authors build a semantic rate model from a deep-learning encoder trained offline, and couple it to a semantic accuracy curve derived from the DeepSC benchmark system, interpolated with cubic splines so accuracy can be evaluated at any signal-to-noise ratio and any symbol count. Semantic accuracy rises steeply with SNR and then saturates, and at a given SNR more semantic symbols yield better reconstruction. On top of this dual-mode link model, the researchers formulate a joint optimization problem of daunting complexity.

Four families of variables must be tuned simultaneously: the number of semantic symbols allocated to each user class, the phase shifts of the N-element RIS array, the three-dimensional trajectory of the UAV, and the transmit power for both user groups. The objective is a weighted average of per-user satisfaction, expressed through sigmoid functions that quantify how closely the achieved semantic rate and semantic accuracy approach the values each user's task demands. The problem is non-convex and mixed-integer, with constraints covering minimum power thresholds, symbol-count limits, a bounded three-dimensional flight box between 50 and 100 meters altitude, and the unit-modulus phase constraints of passive RIS elements.

Conventional convex optimization cannot handle this cocktail of discrete choices and continuous controls, particularly when the UAV's own motion reshapes the wireless channels it is trying to optimize. The authors' answer is an Action Space Decoupled Hybrid Deep Reinforcement Learning framework, or ASD-HDRL. Rather than forcing one algorithm to handle everything, the framework partitions the action space by physical character.

A Dueling Double Deep Q-Network (D3QN) agent makes the discrete decisions, selecting how many semantic symbols to grant long-packet and short-packet users. A Deep Deterministic Policy Gradient (DDPG) agent handles the continuous variables: where the drone flies, how the RIS phases are set, and how power is distributed. The decoupling is more than a convenience.

If D3QN alone tackled the problem, the continuous variables would need to be discretized, causing dimensionality explosion and quantization error. Pure DDPG, conversely, has no native mechanism for discrete choices. Standard combinations of the two algorithms typically share experience samples and update rules, which the authors argue creates conflicts between the jump-like nature of discrete decisions and the smooth evolution of continuous controls.

ASD-HDRL instead gives each agent its own experience replay buffer, its own learning rate, and differentiated target update schemes: periodic hard updates for the D3QN target network, and soft tau-based updates for DDPG to prevent abrupt swings in flight and phase commands. A shared warm-up phase and synchronized update interval keep the two training rhythms aligned, while exploration factors and noise variances are annealed independently on linear schedules. The reward function blends the instantaneous system-level semantic QoE with a potential-based shaping term, defined as the weighted increment of overall QoE between successive time slots, a formulation the authors note is policy-invariant and accelerates convergence.

A penalty term restrains total transmit power above the system limit. The state space captures the map boundary, altitude constraints, user cluster centers, mode activation flags and user counts, giving both agents a normalized picture of the environment at every time slot. Simulation results, run with a base station at a fixed position, four long-packet users, four short-packet users, an 8-element RIS and an 8-antenna base station at 3 GHz, show the framework outperforming a broad set of benchmarks.

Against schemes that freeze one variable at a time, such as fixed RIS phases, random power allocation, a stationary UAV or constant semantic symbol counts, ASD-HDRL converges faster and reaches a higher steady-state QoE, and its cumulative distribution of per-episode QoE shifts markedly rightward, indicating more high-quality service samples and less fluctuation. Against single-mode alternatives, a convex-optimization-assisted DDPG variant, and the widely used proximal policy optimization algorithm, the hybrid framework again leads in both convergence speed and saturated QoE. The convex-optimization baseline oscillated severely early in training because it could not adapt semantic symbol allocation to changing channels, while PPO converged with slight fluctuations to a lower final level.

Two behavioral findings stand out. The optimized UAV trajectory converges toward the geometric center of the ground user clusters and hovers there, rather than simply minimizing distance to any single link, because the joint optimizer balances the channel quality of the RIS-reflected long-packet links against the direct short-packet links. Meanwhile, semantic symbol allocation settles into a stable differentiated pattern: long-packet users, suffering heavy cascaded link attenuation and low SNR, receive 16 semantic symbols, while short-packet users on high-quality direct links need only 8.

The authors report that the proposed algorithm improves the median system-level semantic QoE by more than 77 percent compared with the baseline schemes in their simulations. The study carries clear caveats. It is a simulation study, not a field trial, and its models assume accurate channel state information, ideal RIS phase control and quasi-static fading within each time slot; the authors acknowledge that phase misalignment or outdated channel knowledge would appear mainly as an equivalent SNR penalty, with bounded accuracy deviation thanks to the saturation of the accuracy curve.

The architecture also restricts the short-packet mode to highly compressed semantic payloads due to airborne storage limits, and all users within a class share the same symbol level to keep the decision space tractable. The authors point to dynamic-scenario online learning, deeper joint optimization across semantic coding and the physical layer, and distributed multi-UAV, multi-RIS resource allocation as the next steps.

Subject of Research: Deep reinforcement learning-based resource optimization for RIS-aided UAV semantic communication networks

Article Title: Resource optimization for RIS-UAV integrated air-ground cooperative semantic communication driven by deep reinforcement learning

Article References: Dai, Y., Yao, Y., Huang, S., & Yue, X. (2026). Resource optimization for RIS-UAV integrated air-ground cooperative semantic communication driven by deep reinforcement learning. Frontiers in Physics, 14, Article 1988637. https://doi.org/10.3389/fphy.2026.1988637

Image Credits: AI Generated

DOI: 10.3389/fphy.2026.1988637

Keywords: 6G, semantic communication, reconfigurable intelligent surface, UAV, deep reinforcement learning, D3QN, DDPG, resource allocation, quality of experience, wireless networks, low-altitude communications, simulation study