Every major streaming platform relies on bitrate ladders, discrete sets of resolution and bitrate combinations that allow players to switch quality as network conditions fluctuate. These ladders have traditionally been designed around a single assumption: that the decoded video is what the viewer sees. But that assumption is crumbling. Modern client devices increasingly run Video Super-Resolution (VSR), deep learning models that reconstruct high-resolution imagery from lower-resolution decoded streams. When a viewer’s device can upscale 540p content to near-4K quality on the fly, the entire logic of ladder design breaks down, because the decoded resolution no longer determines the displayed quality.

The complication deepens with the Versatile Video Coding (VVC) standard, the latest generation of video compression that achieves up to 40 percent bitrate reductions over its predecessor HEVC. Those savings come at a price: VVC decoding is significantly more computationally demanding, particularly at high resolutions. Meanwhile, the AI models that perform client-side enhancement add their own inference costs. The interaction between these two trends, heavier codecs and smarter clients, has been largely invisible to existing ladder design approaches, whether fixed industry guidelines, per-title encoding, or machine-learned prediction methods.

E2-Stream attacks this problem with brute-force empiricism. Rather than relying on heuristics or predictions, the framework exhaustively encodes video at three resolutions (540p, 1080p, and 2160p) across twelve bitrates, decodes each stream on real client hardware, applies five different enhancement methods ranging from classical interpolation (Bicubic, Lanczos) to lightweight neural networks (FSRCNN, ESPCN) and a temporally-aware model (EDVR), and measures the energy consumed at every stage. Energy profiling uses Intel RAPL counters for CPU measurements and GPU vendor tooling, cross-validated with CodeCarbon, with each experiment repeated at least five times on thermally stabilized devices.

The framework then synthesizes ladders using a scalarized utility function that balances perceptual quality, measured with VMAF at the display resolution after enhancement, against client-side decoding and inference energy. A tunable regularization parameter allows system designers to explicitly express energy budgets: setting it to zero reduces the optimization to classical quality-only selection, while higher values increasingly favor energy-efficient representations. A Just Noticeable Difference (JND)-aware pruning strategy further removes ladder entries whose quality differences fall below the threshold of human perception, cutting storage requirements by up to 58.98 percent without compromising visual quality.

Experiments on the Inter-4K dataset, a diverse 4K corpus evaluated at 60 frames per second, produced results that expose the device-dependence of enhancement-aware streaming. On a CPU-only Linux laptop with software VVC decoding, decoding and VSR inference increased total client-side energy by up to 85 percent, with the FSRCNN model producing particularly dramatic energy spikes at quality-dominant settings. On a GPU-equipped Windows laptop with an NVIDIA GeForce RTX 5070, the same configurations reduced total client-side energy by 25 to 60 percent thanks to hardware acceleration, while maintaining bitrate reductions of up to 40 percent.

The cross-over effects are perhaps the most visually compelling finding. Applying post-decode VSR significantly elevates the quality of low-resolution encodes: 1080p streams enhanced with VSR frequently match or surpass the VMAF scores of native 2160p encodes, and 540p plus VSR approaches native 1080p at constrained bitrates. These cross-overs reshape the rate-distortion frontier, shifting high-quality operating points toward lower bitrates and enabling ladders that favor lower base resolutions while still delivering high display-resolution quality. Across all models, enhancement-aware ladders preserved sub-JND perceptual quality while achieving 12 to 28 percent bitrate savings.

The choice of enhancement model proved highly device-dependent. On the CPU-only client, Bicubic interpolation consistently achieved negative energy deltas, down to nearly 27 percent savings, because its negligible computational cost enables simultaneous bitrate and energy gains. Learned models, by contrast, delivered stronger quality uplift but incurred substantial inference overhead on CPU-bound hardware. On the GPU-enabled client, the situation reversed: hardware acceleration allowed FSRCNN to achieve energy reductions of up to 57 percent while maintaining strong bitrate savings, and the temporally-aware EDVR model, which leverages multi-frame alignment for improved temporal consistency, became viable only on GPU-accelerated platforms.

Real-time feasibility emerged as another critical constraint. On the CPU-only device, several VSR-based configurations failed to satisfy segment playback deadlines, particularly at higher resolutions, indicating that SISR inference dominates both energy and latency on software-based systems. On the GPU-enabled client, most configurations including FSRCNN and EDVR satisfied real-time constraints across the evaluated bitrate range, demonstrating that energy efficiency and real-time feasibility align closely on GPU-capable hardware but diverge on CPU-only systems.

The broader implication is that optimal bitrate ladders are inherently device-dependent, and streaming systems can no longer derive them solely from codec-centric rate-distortion behavior. Modern platforms must incorporate hardware capabilities, enhancement models, and device-level energy constraints when constructing their catalogs. E2-Stream provides a principled, reproducible foundation for this cross-layer optimization, and the researchers suggest future extensions could integrate device-side telemetry such as thermal and DVFS states, evaluate emerging mobile-optimized VSR architectures, and incorporate Common Media Client Data signaling to enable fully dynamic, device-aware ladder selection during playback. As streaming demand continues to grow, such enhancement-aware and energy-conscious design may prove essential for sustainable next-generation video delivery.

Subject of Research: Science News

Article Title: Energy-aware bitrate ladder for VVC streaming with client-side super-resolution

Article References: Premkumar, A., Menon, V. V., & Herglotz, C. (2026). Energy-aware bitrate ladder for VVC streaming with client-side super-resolution. Frontiers in Signal Processing, 6, Article 1802483. https://doi.org/10.3389/frsip.2026.1802483

Image Credits: AI Generated

DOI: 10.3389/frsip.2026.1802483

Keywords: video streaming, VVC, super-resolution, energy efficiency, bitrate ladder, AI enhancement, VSR, Energy-aware, bitrate, ladder, streaming, client-side