AI Learns to Juggle the Internet of Things: Survey Maps Deep Reinforcement Learning’s Rise in Edge and Fog Computing

Every time a self-driving car decides whether to process sensor data on board or send it to a roadside server, an invisible scheduling decision is being made in microseconds. Multiply that choice by billions of connected devices, from smart factories to hospital monitors, and you get one of the hardest computational puzzles of the modern era. A comprehensive survey published in Cluster Computing by Zouheyr Bouafia, Mansour Mededjel, Belalem Ghalem and Riad Mokadem now offers the most detailed map yet of how deep reinforcement learning, a family of algorithms that learn by trial and reward, is being deployed to solve it. The work systematically compares task scheduling and offloading algorithms across edge and fog computing environments, two paradigms that push computing power away from distant cloud data centers and closer to the devices that generate the data in the first place.

The reason this problem is so difficult lies in its mathematical structure. Task scheduling in edge and fog environments is formally classified as NP-hard, meaning that the time required to find the perfect solution grows explosively with the size of the problem. A fog network might contain dozens of heterogeneous nodes, each with different processing speeds, memory capacities, energy budgets and network conditions, while the tasks arriving at those nodes vary wildly in size, deadline sensitivity and dependencies on other tasks. Classical heuristics and metaheuristics, such as ant colony optimization or grey wolf optimization, have long been used to find good-enough solutions quickly. But these approaches typically rely on static assumptions about the environment, and edge and fog networks are anything but static. Devices join and leave, wireless links fluctuate, and workloads surge unpredictably, which is precisely where learning-based methods promise an advantage.

Reinforcement learning reframes scheduling as a sequential decision game. An agent observes the state of the system, such as queue lengths, resource availability and task characteristics, selects an action like assigning a task to a particular node, and receives a numerical reward reflecting outcomes such as reduced latency or lower energy consumption. Over many iterations, the agent learns a policy that maximizes cumulative reward. Deep reinforcement learning extends this idea by using deep neural networks to approximate the value of states or actions, allowing the agent to cope with enormous, continuous state spaces that would overwhelm tabular methods. The survey traces how this family of techniques, from foundational Q-learning through deep Q-networks, double deep Q-networks, actor-critic methods such as A2C and SAC, and policy gradient approaches like deep deterministic policy gradient, has been adapted to the specific demands of edge and fog infrastructures.

What sets the new survey apart is the breadth of its comparison framework. Rather than simply listing algorithms, the authors evaluate them across a systematic set of dimensions: the case study context, the type of tasks being scheduled, how the optimization problem is formally formulated, the specific reinforcement learning technique employed, the simulation or evaluation tools used, the performance metrics reported, and the learning and execution architecture, whether centralized, distributed, or federated. They also catalog advanced features such as security awareness, fault tolerance, energy optimization and privacy preservation, alongside the limitations each algorithm admits. This multidimensional lens reveals patterns that a simple performance comparison would miss, for instance that the choice between value-based and actor-critic methods often hinges on whether the action space is discrete, as in choosing one of several servers, or continuous, as in tuning transmission power levels.

Among the most consequential trends the survey documents is the shift toward multi-agent and federated learning architectures. In a centralized scheme, a single agent observes everything and decides everything, which becomes a bottleneck and a single point of failure at scale. Multi-agent deep reinforcement learning distributes the intelligence, letting individual nodes or device groups learn cooperatively or independently. Federated reinforcement learning goes further, allowing agents to share learned knowledge without exchanging raw data, a property that matters enormously in privacy-sensitive settings such as healthcare and smart cities. The reviewed literature includes federated deep Q-networks paired with clustering techniques, federated deep recurrent Q-learning for satellite-assisted industrial IoT, and multi-agent approaches for vehicular fog networks, where cars themselves act as fleeting computing nodes whose positions change by the second.

Vehicular computing emerges as one of the most demanding and active application domains in the surveyed corpus. Vehicles move at high speed, producing tasks with hard latency deadlines for safety-critical functions like collision avoidance, while connectivity to fog nodes is intermittent. Studies cited in the survey apply asynchronous deep reinforcement learning to secure offloading in vehicular edge networks, priority-aware offloading based on deep Q-learning, and distributed twin delayed deep deterministic policy gradient, known as TD3, for multi-objective optimization across mobile edge and vehicular-fog systems. Other domains represented include industrial IoT, where decentralized multi-agent systems coordinate partial task offloading across factories, augmented reality offloading in mobile edge networks, smart agriculture, power grid IoT, and unmanned aerial vehicles that serve as airborne edge servers with their own energy constraints.

The survey also shines a light on the unglamorous but critical question of how these algorithms are actually tested. The authors catalog the dominant evaluation platforms, including iFogSim, EdgeCloudSim, CloudSim, EdgeSimPy, SimPy, OMNeT++, the FogBus framework, the COSCO container orchestration system and the AirFogSim simulator for UAV-integrated vehicular fog. On the machine learning side, implementations typically rely on PyTorch, TensorFlow and Keras, with OpenAI Gym and Stable-Baselines3 providing standardized training environments and reliable algorithm implementations. This tooling inventory matters because reproducibility remains a weak point in the field: results obtained in one simulator often fail to transfer cleanly to another, and the survey’s accounting of which studies used which tools gives researchers a common baseline for fair comparison. Performance metrics cluster around makespan, response time, energy consumption, load balance, cost, throughput and quality-of-service violation rates, with recent work increasingly optimizing several of these simultaneously through weighted or Pareto-based multi-objective formulations.

Despite the evident progress, the survey is candid about the field’s open problems. Many proposed algorithms are trained and evaluated under simplified assumptions that ignore security threats, node failures and the full heterogeneity of real hardware. The gap between simulation and deployment remains wide, and the computational cost of training deep agents can itself conflict with the resource constraints of edge devices. Scalability of centralized training looms large, and the survey highlights emerging directions meant to close these gaps: transformer-enhanced distributed deep reinforcement learning, meta-reinforcement learning that lets agents adapt rapidly to unseen environments, hybrid schemes combining fuzzy logic or graph attention networks with learned policies, and privacy-preserving hierarchical frameworks for low-altitude vehicular fog. The authors also point toward better handling of dependent tasks, where directed acyclic graphs of interlinked computations must be placed jointly rather than one at a time.

For the broader technology landscape, the stakes of this research line are hard to overstate. The number of IoT devices continues to climb, and the latency-sensitive applications they enable, from autonomous driving to remote surgery to real-time industrial control, simply cannot tolerate the round trip to a distant cloud. Edge and fog computing supply the physical infrastructure, but intelligence about where and when to run each computation is what turns that infrastructure into a functioning system. By consolidating and comparing the deep reinforcement learning literature across environments, task types, algorithms, tools and metrics, the survey provides both a status report and a research agenda. It suggests that the next generation of schedulers will be distributed, privacy-aware, energy-conscious and capable of learning on the fly, qualities that will determine whether the promised era of ubiquitous, responsive computing actually arrives on schedule.

Subject of Research: Deep reinforcement learning-based task scheduling and offloading in edge and fog computing

Article Title: A comparative analysis of deep reinforcement learning based task scheduling and offloading in edge and fog computing: a comprehensive survey

Article References: Bouafia, Z., Mededjel, M., Ghalem, B., & Mokadem, R. (2026). A comparative analysis of deep reinforcement learning based task scheduling and offloading in edge and fog computing: a comprehensive survey. Cluster Computing, 29(14), Article 795. https://doi.org/10.1007/s10586-026-06601-z

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06601-z

Keywords: edge computing, fog computing, deep reinforcement learning, task scheduling, task offloading, Internet of Things, multi-agent reinforcement learning, federated learning, vehicular edge computing, NP-hard optimization, resource allocation, Cluster Computing