In an experimental evaluation of network-connected embedded device security, a newly developed testing framework integrating coverage-guided fuzzing with deep reinforcement learning achieved approximately 73.5 percent code coverage across emulated firmware images, substantially exceeding the 35.0 percent coverage recorded by standard American Fuzzy Lop tooling under identical operational conditions. The study, conducted across forty-eight MIPS32 firmware images representing twelve commercial device models, demonstrated that adaptive mutation scheduling directed by an actor-critic model doubled binary exploration while discovering twenty unique execution paths and twenty-three abnormal execution crashes. By replacing static, predefined mutation heuristics with an intelligent decision architecture, the researchers resolved a major testing bottleneck in embedded software analysis, where complex dependencies, compressed file structures, and heterogeneous communication interfaces frequently impede automated vulnerability discovery.
The security of embedded Internet of Things devices has become a pressing technical concern because modern connected equipment relies on low-level firmware to govern core hardware operations, telemetry acquisition, and distributed network communications. While coverage-guided fuzzing provides automated dynamic testing, conventional engines rely on static or randomized mutation algorithms that continuously apply unguided bit permutations. Such static scheduling frequently generates redundant test cases that repeatedly traverse shallow input validation functions without exercising deeper control logic, causing rapid test saturation and preventing the discovery of complex, path-dependent firmware vulnerabilities.
To establish a rigorous evaluation pipeline, the researchers created an end-to-end data preprocessing workflow that extracts and normalizes operational software components from monolithic firmware binaries. Forty-eight complete firmware images spanning twelve device models were acquired from the public D-Link Firmware Center repository, providing diverse software revisions targeting the MIPS32 architecture. The Binwalk extraction utility unpacked each binary archive, reconstructing standard Unix filesystem hierarchies across primary directories including administrative configurations, shared libraries, and web services. Non-executable assets such as static icons, localization files, and documentation were systematically pruned to prevent unnecessary processing overhead during continuous fuzzing runs. From two hundred twenty-six total executable programs recovered across the corpus, sixty-one network-facing binaries were isolated for targeted vulnerability assessment based on external interface exposure. These designated targets included eighteen HTTP daemons, nine Telnet services, eleven Universal Plug and Play handlers, and twenty-three Common Gateway Interface input parsers representing primary attack vectors.
Dynamic execution and runtime monitoring of the selected firmware targets were orchestrated inside the FirmAE full-system emulation platform, which recreates MIPS32 execution environments without requiring physical hardware deployments. Because embedded binaries depend closely on virtualized network interfaces and simulated operating system services, FirmAE provided the runtime framework necessary to execute mutated inputs and record fine-grained software faults. Input samples were mapped into byte sequences within an arbitrary binary space, allowing American Fuzzy Lop to mutate network payloads directly before injection into the emulator. The initial seed corpus consisted of five hundred structured input samples selected to represent primary IoT interface formats: two hundred twenty HTTP requests comprising forty-four percent of the corpus, one hundred ten CGI inputs representing twenty-two percent, seventy UPnP messages at fourteen percent, fifty Telnet commands at ten percent, and fifty malformed network packets comprising ten percent to foster diverse initial exploration behaviors.
The central intelligence of the fuzzing framework is provided by an actor-critic deep reinforcement learning agent trained through Proximal Policy Optimization to direct mutation operator selection. The agent observes a fifteen-dimensional runtime state vector derived from four telemetry categories: measured code coverage, newly discovered execution paths, crash feedback signals, and execution speed metrics collected after each testing cycle. The actor network features two fully connected hidden layers of one hundred twenty-eight and sixty-four neurons with ReLU activations, culminating in a five-neuron Softmax output layer that distributes selection probabilities across five mutation operators: Bitflip, Byteflip, Arithmetic, Interest, and Havoc. Totaling ten thousand seven hundred fifty-seven trainable parameters, the actor network chooses input transformations to balance fine-grained byte edits with aggressive data restructuring. Concurrently, the critic network uses a matching two-layer structure with ten thousand three hundred sixty-nine parameters terminating in a linear value estimator, creating a unified model of twenty-one thousand one hundred twenty-six parameters.
The reinforcement learning policy was optimized across four hundred training episodes utilizing the AdamW algorithm with a learning rate of 1.0 multiplied by 10 to the negative fourth power and weight decay set at 0.01. Training stability was enforced using a clipped objective function with a clipping boundary of 0.2 and a temporal discount factor of 0.99. To prevent early policy collapse onto a single mutation action, an entropy regularization coefficient was decayed linearly from 0.45 to 0.05 across the four hundred episodes, sustaining exploratory diversity before consolidating into stable exploitation. The agent computed scalar rewards using a combined function weighting incremental code coverage at 0.7 and binary crash discovery at 0.3, with all rewards normalized to the interval between negative one and positive one. Model parameters were initialized with a fixed random seed of forty-two, ensuring strict consistency across ten independent twelve-hour testing sessions totaling one hundred twenty evaluation hours.
The intelligent framework achieved approximately 73.5 percent code coverage on the MIPS32 test suite, whereas standard American Fuzzy Lop attained only 35.0 percent coverage under identical runtime configurations. In addition to doubling coverage, the adaptive system discovered twenty unique execution paths compared to only seven paths mapped by the baseline tool, proving that dynamic mutation choices successfully guided test inputs through complex branching constraints. Furthermore, the reinforcement learning pipeline detected twenty-three unique abnormal execution crashes across the testing episodes, whereas American Fuzzy Lop identified only six unique crashes. The framework also surpassed other intelligent fuzzing baselines documented in the study, exceeding the 65.2 percent coverage achieved by SIoTFuzzer and the 61.4 percent coverage reported for DeepFuzzer, confirming that closed-loop policy adaptation generates more effective test inputs than static mutation schedules.
Throughput analysis demonstrated that the integrated fuzzing system maintained efficient execution rates despite the computational demands of continuous policy inference. Over four hundred training episodes, the framework recorded an average processing speed of approximately twelve hundred executions per second, with operational throughput ranging between eight hundred fifty and fifteen hundred executions per second according to emulation overhead. Coverage discovery velocity remained robust, averaging approximately 2.0 new coverage units per episode, with peak progress exceeding 3.0 units following strategic mutation transitions. Statistical evaluations revealed a positive correlation of r equals 0.45 between policy entropy and episode rewards, verifying that maintaining diverse mutation selection improved cumulative learning outcomes. Two-dimensional principal component analysis demonstrated that the first two principal components accounted for 28.1 percent and 12.2 percent of state-space variance, showing that high-reward trajectories consistently aligned with expanded code coverage and new execution path discoveries, confirming that the agent systematically identified advantageous exploration states.
Post-test diagnostic analysis of the twenty-three discovered crashes verified that the adaptive mutation framework exposed diverse security flaws across the emulated firmware binaries. Ablation experiments demonstrated the structural necessity of each framework component. Completely disabling the reinforcement learning feedback loop severely degraded code coverage and crash discovery metrics. These comparative ablation runs confirm that the integrated combination of actor-critic policy learning, multi-factor reward formulation, and entropy-guided exploration is indispensable for effectively discovering hidden vulnerabilities within complex embedded firmware binaries.
The authors highlighted several methodological limitations that provide essential context for interpreting the experimental findings. Consequently, the researchers noted that the twenty-three recorded crashes represent emulation-based vulnerability candidates that require downstream manual verification and physical device validation before they can be confirmed as exploitable flaws. Future research will explore expanding the evaluation corpus to diverse hardware architectures, incorporating hierarchical and multi-agent reinforcement learning algorithms, and investigating transformer-based sequence generation models to further improve automated vulnerability detection in connected embedded systems.
Subject of Research: Reinforcement learning-guided firmware vulnerability detection in embedded Internet of Things devices
Article Title: An integrated fuzzing and PPO-based deep reinforcement learning framework for firmware vulnerability detection in embedded IoT devices
Article References: Liu, Q., Yang, G., Li, T., Yin, Y., Wang, H., & Yin, C. (2026). An integrated fuzzing and PPO-based deep reinforcement learning framework for firmware vulnerability detection in embedded IoT devices. Journal of King Saud University Computer and Information Sciences, 38(9), Article 975. https://doi.org/10.1007/s44443-026-01296-z
Image Credits: AI Generated
DOI: 10.1007/s44443-026-01296-z
Keywords: firmware security, embedded IoT devices, coverage-guided fuzzing, Proximal Policy Optimization, deep reinforcement learning, actor-critic network, AFL++, FirmAE emulation, mutation scheduling, code coverage, unique crashes, execution path discovery, MIPS32 architecture, D-Link firmware, software vulnerability detection, AdamW optimizer, cybersecurity testing
Cite Scienmag News
Denise Maddox. (November 21, 2026). Reinforcement Learning Guided Fuzzing Doubles Firmware Code Coverage in Embedded Device Testing. Scienmag. https://scienmag.com/reinforcement-learning-guided-fuzzing-doubles-firmware-code-coverage-in-embedded-device-testing/
Denise Maddox. "Reinforcement Learning Guided Fuzzing Doubles Firmware Code Coverage in Embedded Device Testing." Scienmag, 21 November 2026, https://scienmag.com/reinforcement-learning-guided-fuzzing-doubles-firmware-code-coverage-in-embedded-device-testing/. Accessed 21 November 2026.
Denise Maddox. "Reinforcement Learning Guided Fuzzing Doubles Firmware Code Coverage in Embedded Device Testing." Scienmag. November 21, 2026. https://scienmag.com/reinforcement-learning-guided-fuzzing-doubles-firmware-code-coverage-in-embedded-device-testing/








