Reinforcement Learning Tames Slow Hard Drives by Cutting Costly Rewrite Operations

Hybrid storage systems that pair fast solid-state drives with inexpensive shingled magnetic recording hard drives promise the best of both worlds: the speed of flash memory and the capacity of spinning disks. Yet a stubborn problem has kept these combinations from reaching their full potential. When data arrives out of order, the overlapping track layout of shingled magnetic recording forces the drive to perform read-modify-write operations, rewriting far more data than the application actually requested. A research team at Capital Normal University in Beijing has now developed an adaptive reinforcement learning framework, called RLFC, that dramatically reduces these costly operations, cutting the worst-case tail latency of the system by more than 20 percent while extending the life of the solid-state drive.

The findings, published in Cluster Computing, address one of the most persistent bottlenecks in modern data center storage. Shingled magnetic recording packs tracks so closely together that writing to one track overwrites its neighbors, which means any non-sequential write triggers a cascade of internal rewrites. In hybrid systems, a solid-state drive typically acts as a persistent cache in front of the shingled drive, absorbing writes and flushing them to disk in batches. The trouble intensifies during the persistent cache cleaning process, when accumulated data must be migrated to the hard drive. Frequent read-modify-write cycles during this migration inflate write latency and push up the tail latency, the slowest 0.1 percent of requests that often dominate user-perceived performance in large-scale services.

The core insight behind RLFC is that many of the writes destined for the shingled drive are redundant before they ever reach it. Applications frequently update the same logical block address multiple times in quick succession, and each duplicate write that lands on the hard drive triggers unnecessary read-modify-write work. The researchers trained a reinforcement learning agent, built on a Double Deep Q-Network, to perceive the state of the storage system and learn to identify these duplicate writes targeting the same logical block address. When the agent detects such redundancy, it selectively redirects the write to the solid-state drive instead, where the latest version can be absorbed cheaply and only the final value eventually migrates to the shingled disk.

Double Deep Q-Networks were chosen deliberately over the standard Deep Q-Network architecture. Conventional deep Q-learning is known to overestimate action values, a bias that can lead the agent to make overly optimistic decisions about which writes to redirect. The double Q-learning formulation decouples action selection from action evaluation, using two separate networks to reduce this overestimation and produce more stable learning in the complex, high-dimensional state space that a live storage system presents. Under dynamic workloads, where request patterns shift constantly, this stability matters: an agent that misjudges which requests carry high latency risk can end up worsening the very problem it is meant to solve.

Redirecting writes, however, creates a secondary hazard. Every additional write absorbed by the solid-state drive consumes its limited program-erase cycles, and an aggressive policy that funnels too much traffic to the flash device would trade short-term latency gains for long-term hardware wear. To prevent this, the RLFC framework incorporates a Ghost Cache filtering mechanism that dynamically regulates the solid-state drive’s write traffic. The filter screens incoming write requests and blocks those deemed unnecessary, ensuring that the reinforcement learning agent’s redirection decisions do not translate into excessive wear. The result is a dual objective: reduce read-modify-write operations on the shingled drive while simultaneously protecting the endurance of the solid-state cache.

The experimental evaluation drew on publicly available storage trace datasets from Microsoft Research Cambridge, the Storage Networking Industry Association, and Alibaba Cloud, giving the tests a grounding in realistic enterprise and cloud workloads rather than synthetic patterns. Across these traces, RLFC outperformed a comparable Deep Q-Network-based scheme on every key metric the team measured. The 99.9th percentile tail latency, the figure that captures the most extreme delays users experience, dropped by 20.73 percent. Average request latency fell by 21.36 percent, meaning the improvement was felt across the whole distribution of requests, not just at the tail.

Perhaps just as significant for anyone budgeting storage hardware, the total write volume sent to the solid-state drive decreased by 12.57 percent. Because flash endurance is directly determined by how many bytes are written to the device over its lifetime, this reduction translates into a notable extension of the drive’s usable life. The researchers report that the RLFC strategy effectively controls SSD wear and extends device lifetime, addressing the reliability concern that has often been raised against learning-based cache management schemes that optimize aggressively for latency without accounting for hardware degradation.

The work builds on a growing body of research applying machine learning to storage system management. Earlier efforts have used reinforcement learning for adaptive data placement in hybrid storage, for cache cleaning to mitigate long-tail latency in device-managed shingled drives, and for predictive data management in SSD-SMR hybrids. The Beijing team itself had previously developed Augur, a predictive and adaptive data management strategy for the same class of systems. What distinguishes RLFC is its explicit focus on flow control during the persistent cache cleaning process, the phase where read-modify-write amplification is most damaging, combined with an endurance-aware filtering layer that keeps the learning agent’s decisions sustainable.

The broader significance extends beyond the laboratory. Shingled magnetic recording has become an increasingly attractive technology for cloud providers because it packs roughly 15 to 20 percent more capacity onto each platter than conventional magnetic recording, lowering the cost per terabyte of archival and object storage. But that density advantage has historically come with unpredictable latency penalties under random write workloads. If reinforcement learning agents can reliably orchestrate the division of labor between flash caches and shingled drives, data center operators could adopt high-density recording technology more broadly without sacrificing the responsiveness that interactive services demand.

The research was supported in part by the Natural Science Foundation of China, and the team has made its evaluation traces publicly available through the Microsoft Research Cambridge, SNIA, and Alibaba Cloud repositories, allowing other groups to reproduce and extend the results. As artificial intelligence increasingly manages the physical infrastructure of computing, from storage flow control to network scheduling, studies like this one suggest that the same learning techniques powering recommendation engines and language models may also quietly determine how fast the world’s data gets written, and how long the devices holding it survive.

Subject of Research: Reinforcement learning-based flow control for SSD-SMR hybrid storage systems to reduce latency and improve SSD endurance

Article Title: RLFC: An adaptive RL-driven framework for efficient flow control in SSD-SMR hybrid storage with improved latency and endurance

Article References: Zhou, G., Wang, G., Shi, Z., Zhang, Q., Shao, Z., & Guan, Y. (2026). RLFC: An adaptive RL-driven framework for efficient flow control in SSD-SMR hybrid storage with improved latency and endurance. Cluster Computing, 29(15), Article 846. https://doi.org/10.1007/s10586-026-06639-z

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06639-z

Keywords: shingled magnetic recording, SSD-SMR hybrid storage, reinforcement learning, Double Deep Q-Network, tail latency, SSD endurance, persistent cache, read-modify-write, storage systems, machine learning, data centers, flash memory