Every major streaming platform relies on bitrate ladders, discrete sets of resolution and bitrate combinations that allow players to switch quality as network conditions fluctuate. These ladders have traditionally been designed around a single assumption: that the decoded video is what the viewer sees. But that assumption is crumbling. Modern client devices increasingly run Video Super-Resolution (VSR), deep learning models that reconstruct high-resolution imagery from lower-resolution decoded streams. When a viewer’s device can upscale 540p content to near-4K quality on the fly, the entire logic of ladder design breaks down, because the decoded resolution no longer determines the displayed quality.

The complication deepens with the Versatile Video Coding (VVC) standard, the latest generation of video compression that achieves up to 40 percent bitrate reductions over its predecessor HEVC. Those savings come at a price: VVC decoding is significantly more computationally demanding, particularly at high resolutions. Meanwhile, the AI models that perform client-side enhancement add their own inference costs. The interaction between these two trends, heavier codecs and smarter clients, has been largely invisible to existing ladder design approaches, whether fixed industry guidelines, per-title encoding, or machine-learned prediction methods.

E2-Stream attacks this problem with brute-force empiricism. Rather than relying on heuristics or predictions, the framework exhaustively encodes video at three resolutions (540p, 1080p, and 2160p) across twelve bitrates, decodes each stream on real client hardware, applies five different enhancement methods ranging from classical interpolation (Bicubic, Lanczos) to lightweight neural networks (FSRCNN, ESPCN) and a temporally-aware model (EDVR), and measures the energy consumed at every stage. Energy profiling uses Intel RAPL counters for CPU measurements and GPU vendor tooling, cross-validated with CodeCarbon, with each experiment repeated at least five times on thermally stabilized devices.

The framework then synthesizes ladders using a scalarized utility function that balances perceptual quality, measured with VMAF at the display resolution after enhancement, against client-side decoding and inference energy. A tunable regularization parameter allows system designers to explicitly express energy budgets: setting it to zero reduces the optimization to classical quality-only selection, while higher values increasingly favor energy-efficient representations. A Just Noticeable Difference (JND)-aware pruning strategy further removes ladder entries whose quality differences fall below the threshold of human perception, cutting storage requirements by up to 58.98 percent without compromising visual quality.

Experiments on the Inter-4K dataset, a diverse 4K corpus evaluated at 60 frames per second, produced results that expose the device-dependence of enhancement-aware streaming. On a CPU-only Linux laptop with software VVC decoding, decoding and VSR inference increased total client-side energy by up to 85 percent, with the FSRCNN model producing particularly dramatic energy spikes at quality-dominant settings. On a GPU-equipped Windows laptop with an NVIDIA GeForce RTX 5070, the same configurations reduced total client-side energy by 25 to 60 percent thanks to hardware acceleration, while maintaining bitrate reductions of up to 40 percent.

The cross-over effects are perhaps the most visually compelling finding. Applying post-decode VSR significantly elevates the quality of low-resolution encodes: 1080p streams enhanced with VSR frequently match or surpass the VMAF scores of native 2160p encodes, and 540p plus VSR approaches native 1080p at constrained bitrates. These cross-overs reshape the rate-distortion frontier, shifting high-quality operating points toward lower bitrates and enabling ladders that favor lower base resolutions while still delivering high display-resolution quality. Across all models, enhancement-aware ladders preserved sub-JND perceptual quality while achieving 12 to 28 percent bitrate savings.

The choice of enhancement model proved highly device-dependent. On the CPU-only client, Bicubic interpolation consistently achieved negative energy deltas, down to nearly 27 percent savings, because its negligible computational cost enables simultaneous bitrate and energy gains. Learned models, by contrast, delivered stronger quality uplift but incurred substantial inference overhead on CPU-bound hardware. On the GPU-enabled client, the situation reversed: hardware acceleration allowed FSRCNN to achieve energy reductions of up to 57 percent while maintaining strong bitrate savings, and the temporally-aware EDVR model, which leverages multi-frame alignment for improved temporal consistency, became viable only on GPU-accelerated platforms.

Real-time feasibility emerged as another critical constraint. On the CPU-only device, several VSR-based configurations failed to satisfy segment playback deadlines, particularly at higher resolutions, indicating that SISR inference dominates both energy and latency on software-based systems. On the GPU-enabled client, most configurations including FSRCNN and EDVR satisfied real-time constraints across the evaluated bitrate range, demonstrating that energy efficiency and real-time feasibility align closely on GPU-capable hardware but diverge on CPU-only systems.

The broader implication is that optimal bitrate ladders are inherently device-dependent, and streaming systems can no longer derive them solely from codec-centric rate-distortion behavior. Modern platforms must incorporate hardware capabilities, enhancement models, and device-level energy constraints when constructing their catalogs. E2-Stream provides a principled, reproducible foundation for this cross-layer optimization, and the researchers suggest future extensions could integrate device-side telemetry such as thermal and DVFS states, evaluate emerging mobile-optimized VSR architectures, and incorporate Common Media Client Data signaling to enable fully dynamic, device-aware ladder selection during playback. As streaming demand continues to grow, such enhancement-aware and energy-conscious design may prove essential for sustainable next-generation video delivery.

Subject of Research: Science News

Article Title: Energy-aware bitrate ladder for VVC streaming with client-side super-resolution

Article References: Premkumar, A., Menon, V. V., & Herglotz, C. (2026). Energy-aware bitrate ladder for VVC streaming with client-side super-resolution. Frontiers in Signal Processing, 6, Article 1802483. https://doi.org/10.3389/frsip.2026.1802483

Image Credits: AI Generated

DOI: 10.3389/frsip.2026.1802483

Keywords: video streaming, VVC, super-resolution, energy efficiency, bitrate ladder, AI enhancement, VSR, Energy-aware, bitrate, ladder, streaming, client-side

The conceptual foundation of the new framework departs sharply from the dominant view of video compression that has prevailed for decades. Classical communication theory, rooted in the Shannon-Weaver model articulated by Claude Shannon in 1948, distinguishes three levels of communication. Level A concerns the technical problem of transmitting data accurately, Level B addresses whether transmitted symbols convey the intended meaning, and Level C concerns the effectiveness problem: whether the received information produces the desired outcome. Video standards such as HEVC have concentrated almost exclusively on Level A, maximizing signal fidelity under constrained bandwidth by minimizing distortion between the original and the reconstructed signal. GVC instead places Level C at the heart of its design, asking not whether every pixel survives the journey but whether the reconstructed video meets perceptual expectations or supports the task at hand.

The driving principle behind GVC is elegantly simple to state: trade computation for compression rate. Instead of transmitting detailed visual data, the framework encodes video into extremely compact representations and delegates content reconstruction to the receiver, where powerful generative priors synthesize high-quality video from minimal transmitted information. The researchers offer a vivid metaphor to explain the shift. Traditional compression is like photographing a painting and sending the photograph; GVC is like describing the painting’s composition and style, then relying on an AI painter at the far end to recreate it. Modern generative video models are so expressive that they can synthesize convincing footage from sparse latent representations, or in the limit even from pure noise guided by learned priors. That capability transforms the encoder’s job from preserving every pixel to selecting and transmitting only the most task-relevant information.

The system is built from two primary components working in tandem. On the sending side, a neural encoder, a pre-trained neural network, ingests an input video sequence, which might be surveillance footage, a video call stream, or a live broadcast, and compresses it into a set of compact representations called compressed tokens. These tokens blend discrete and continuous elements: compressed keyframes, high-level descriptors of video segments, and low-level continuous features that together capture the essential semantics and motion dynamics of the scene while drastically reducing dimensionality. The tokens are further encoded into a bitstream using techniques such as residual coding to squeeze out remaining redundancy. On the receiving side, a pre-trained diffusion-based generative video model performs what is essentially a conditional video generation task. Some tokens serve as direct inputs to the denoising process while others act as conditioning signals, and the model synthesizes frames that are visually faithful to the original input.

What gets transmitted depends, critically, on the purpose of the reconstruction. If the goal is human perception, the encoder sends features that help the generative decoder produce perceptually similar content. If the goal is machine understanding, for instance segmentation or recognition by a downstream algorithm, the encoder focuses on semantically meaningful representations instead. This task-oriented orientation is where GVC aligns itself with the AI Flow framework, proposed by TeleAI at the end of 2024, which envisions communication networks distributing intelligence for ubiquitous AI-powered services. The theoretical underpinning draws on the concept of Information Capacity, a measure of how efficiently generative models compress data, as well as earlier work on task-oriented feature compression for multimodal understanding via device-edge co-inference. The GVC concept itself was first introduced publicly by TeleAI at the World Artificial Intelligence Conference in mid-2025, where a prototype for maritime communications demonstrated ultra-low bitrate video transmission over bandwidth-limited satellite links.

Extreme compression, however, introduces a new bottleneck: the computational cost of high-quality generative reconstruction. Diffusion-based decoders are computationally intensive, and hardware, power, and latency constraints impose an upper bound on how much computation can realistically be traded for compression, especially in real-time applications like video conferencing or edge-device streaming. The team’s answer is a second, complementary principle: trading compression rate for practicality. By sacrificing a small fraction of the compression ratio, the system can send richer latent representations that reduce reliance on massive generative models, unlocking the use of smaller and faster decoders. The researchers further apply model compression techniques to shrink key components such as 3D variational autoencoders, and they employ distillation and sampling acceleration methods for the diffusion-based decoder to lower inference time. The result is a flexible balance across the compression-computation-quality triangle that adapts to whatever resources the deployment environment offers.

The empirical results are striking. Benchmarked on the standard MCL-JCV dataset using a 14-billion-parameter video generative model, GVC maintained competitively high perceptual quality, measured with the Learned Perceptual Image Patch Similarity metric, at an average bitrate of just 0.008 bits per pixel, equivalent to roughly a 0.02% compression rate. Conventional video coding schemes exhibit a substantial performance gap at this bitrate; on certain challenging sequences, traditional methods require approximately six times more bandwidth to match the perceptual quality achieved by GVC. In one benchmark example, the framework achieved visually compelling reconstruction at 0.005 bits per pixel. In real-world surveillance scenarios, the compression can go further still: the team reports bitrates below 0.002 bits per pixel, crossing the 0.01% compression threshold, while retaining sufficient visual quality for the task.

Importantly, the extreme compression does not come at the expense of semantic integrity. To test downstream utility, the researchers applied the reconstructed videos to video object segmentation on the DAVIS2017 benchmark, evaluating performance with the Jaccard index, contour accuracy, their average, and contour recall. The compressed-and-regenerated videos achieved highly competitive segmentation results, indicating that even at astonishingly low bitrates the framework preserves the semantic information machines need to understand a scene. This validates the core promise of effectiveness-level communication: the reconstructed video is not merely visually plausible but genuinely useful for the tasks the transmission was intended to serve.

Deployment readiness was demonstrated on real hardware. After miniaturization, distillation, and quantization of the generative decoder, the system can reconstruct a group of 29 frames in a single pass with inference latency of around two seconds on consumer-grade GPUs, a response time comparable to what users routinely experience with large language models. Although the miniaturized model incurs some loss in visual quality and bandwidth efficiency relative to its full-scale counterpart, it still maintains competitively high perceptual quality, with a demonstrated LPIPS score of 0.273 on a sample sequence. That combination of speed, quality, and modest hardware requirements makes GVC a plausible candidate for the environments that need it most: emergency rescue operations, remote surveillance, narrowband mobile networks, in-vehicle and wearable devices, and maritime satellite links where bandwidth is scarce and expensive.

The authors frame GVC not merely as another codec but as a task-oriented communication paradigm tailored for the era of generative intelligence. By transmitting only what is necessary for perception and decision making, and letting generative priors at the receiver do the heavy lifting of reconstruction, the framework opens the door to communication systems that are more efficient, adaptive, and intelligent than the fidelity-obsessed pipelines of the past. Whether video transmission at one hundredth of one percent of its original size becomes a routine capability will depend on further advances in generative model efficiency and edge computing, but this work offers a credible, empirically validated path toward that frontier, and a glimpse of a future in which the networks we build carry descriptions rather than copies, and understanding rather than pixels.

The work appears as a brief communication in Vicinagearth, an open-access journal, published on 23 March 2026 as volume 3, article number 7, with a correction issued on 3 June 2026. Its placement at the intersection of coding and information theory, computer vision, and multimedia systems reflects the increasingly hybrid nature of compression research, where ideas from generative modeling are being grafted onto classical transmission problems.

Historically, the Shannon-Weaver model dates to the 1940s, and the authors note that video communication technology has spent decades optimizing its Level A, the technical problem of accurate signal delivery. The Information Capacity metric, proposed as a way to evaluate how effectively generative models compress data, laid methodological groundwork for this line of research, and in early 2025 the same group extended the approach to task-oriented communications for multimodal understanding via device-edge co-inference. The maritime prototype unveiled at the World Artificial Intelligence Conference demonstrated ultra-low bitrate transmission over bandwidth-limited satellite connections, a setting where every saved bit carries direct operational value.

Within the AI Flow framework, the researchers position GVC as opening new possibilities for video communication in bandwidth- and resource-constrained environments such as emergency rescue, remote surveillance, and mobile edge computing, describing it as a viable path toward an effective, efficient, scalable, and practical video communication paradigm.

Subject of Research: Extreme low-bitrate video compression using generative AI models to reconstruct video from minimal transmitted information

Article Title: Generative video compression: towards 0.01% compression rate for video transmission

Article References: Generative video compression: towards 0.01% compression rate for video transmission. (n.d.). https://doi.org/10.1007/s44336-026-00035-2

Image Credits: AI Generated

DOI: 10.1007/s44336-026-00035-2

Keywords: generative video compression, video compression, task-oriented communication, AI Flow, diffusion models, bitrate, Shannon-Weaver model, edge computing, video transmission, LPIPS, surveillance, neural encoder