The core problem the researchers identify is a dual bottleneck. Modern large language models have grown from the roughly 12 to 60 million parameters of ResNet in 2016 to hundreds of billions or even trillions of parameters in systems like Llama-4, released in 2025. That hundredfold expansion in less than a decade means inference can demand tens to hundreds of gigabytes of memory, far beyond the 4 to 32 gigabytes typical of consumer devices. Compression techniques such as quantization and pruning help, but they trade away model capability. At the same time, communication networks strain under the load: split-inference schemes that ship high-dimensional activation features from devices to servers can generate tens to hundreds of megabytes per inference step, while multi-agent systems that synchronize reasoning traces amplify overhead further. Congestion, jitter, and wireless instability compound the challenge.

AI Flow responds with three interlocking pillars. The first is a device-edge-cloud architecture that treats the network itself as a computational hierarchy. End devices handle lightweight tasks and real-time interaction; edge servers at base stations and roadside units provide nearby, low-latency processing; and cloud clusters supply the scalable horsepower for training and compute-intensive inference. By orchestrating workloads across these tiers, the framework balances resource scalability against latency, offloading latency-critical inference to the edge while reserving the cloud for heavy operations.

Within that hierarchy, the team introduces two collaboration techniques designed to cut communication costs. The first, task-oriented feature compression, targets vision-language model inference. Rather than transmitting raw images, the device merges visual features produced by a CLIP-style encoder using density peaks clustering based on K nearest neighbors, then encodes the merged features with a hyperprior-based entropy model whose parameters are modeled on a Laplacian distribution. A router network selects the best entropy model for each feature. In experiments on the LLaVA-OneVision-7B model using an NVIDIA Jetson AGX Orin device and an RTX 4090 edge server, the method reduced transmitted data by 25 to 45 percent compared with WebP and 35 to 60 percent compared with JPEG at equal accuracy on the RealWorldQA benchmark, and cut inference latency to roughly a third of server-only inference on the MME benchmark.

The second technique, hierarchical collaborative decoding, accelerates large language model generation through speculative decoding spread across network tiers. A lightweight model on the device drafts tokens locally, while a larger edge model validates and corrects them using a soft acceptance strategy, shifting the big model’s role from full generation to error correction. The researchers extend this into a parallel pipeline in which the device keeps generating without blocking while the edge server refines tokens at intervals. On the MATH-500 benchmark, a two-tier configuration pairing a 1.5-billion-parameter device model with a 7-billion-parameter edge model achieved about 40 tokens per second, a 1.25-fold speedup over edge-only decoding at the same accuracy, with a three-tier setup adding a 14-billion-parameter cloud model for further gains.

The second pillar of AI Flow is the concept of familial models: families of different-sized models whose hidden features are aligned so that intermediate results from a small model can be directly reused by a larger one without any middleware. Two enabling techniques make this possible. Early exit allows inference to terminate at intermediate layers while preserving acceptable accuracy, with lightweight branch modules refining features before prediction. Weight decomposition splits the linear layers of transformer blocks into pairs of low-rank matrices whose combined parameter count is smaller than the original, with the hidden dimension tuned to hit nearly any target size. Initialization via singular value decomposition on whitened data keeps distortion low, and the team shows that compression loss is quantitatively determined by the squared singular values of discarded components, enabling per-layer compression decisions.

The researchers demonstrate two implementation strategies. Hierarchical principal component decomposition trains a series of low-rank components that progressively fit the residuals of earlier ones, producing TeleChat-based models from 2.38 billion to 6.30 billion parameters that, despite limited training tokens, perform comparably to established models such as LLaMA2-7B and ChatGLM2-6B on benchmarks including MMLU, CMMLU, C-Eval, GSM8K, MATH, and BBH. The second strategy, early exiting with scalable branches, inserts decomposed transformer blocks between exit points and a shared language model head. Applied to LLaVA-1.5-7B, it retained 98.2 percent of the backbone’s average performance on six visual question answering benchmarks using only 3.17 billion parameters, while a baseline without the branch design needed at least 4.63 billion parameters to reach 90 percent.

The third pillar is perhaps the most provocative: connectivity- and interaction-based intelligence emergence. Here, the network becomes a medium through which heterogeneous models, including large language models, vision-language models, and diffusion models, collaborate to achieve capabilities exceeding any single model. A device-server collaboration scheme lets specialized on-device models generate preliminary responses in parallel, which a central server model aggregates into a unified answer that is then returned to devices for revision. Evaluations on MT-Bench, AlpacaEval 2.0, and Arena-Hard showed consistent gains, with weaker models benefiting most, and performance on Arena-Hard rose nearly linearly with the number of participating agents, suggesting practical scalability.

Diffusion models receive their own collaboration paradigms. A serial scheme for multi-person motion generation chains an interleaved interaction synthesis module with a relative coordination refinement module, achieving state-of-the-art results on the InterHuman benchmark, including a 25.3 percent improvement in Top-1 R-Precision and a 50.6 percent reduction in Fréchet inception distance compared with prior methods. A parallel scheme for monocular depth estimation splits processing into near-field and far-field decoder branches fused around a sliding anchor, topping benchmarks on both indoor NYU-V2 and outdoor KITTI data. A networked scheme, OmniVDiff, unifies RGB, depth, segmentation, and edge modalities within a single video diffusion transformer, outperforming baselines on depth-conditioned video generation.

The authors ground the framework in application scenarios that include embodied AI, where drones and ground robots share aligned intermediate features to avoid redundant computation; wearable devices, where smart glasses offload heavy recognition tasks to edge and cloud tiers while keeping latency-sensitive processing local; and smart cities, where the low-altitude economy of delivery drones and aerial mobility systems demands ultra-low-latency coordination across thousands of heterogeneous devices. Future directions include federated learning adapted to large models, distributed edge inference resilient to device churn, and adaptive network orchestration for volatile wireless conditions. The team also articulates guiding principles, including a Law of Information Capacity that defines efficiency as the ratio of text compression gain to inference cost, and a Law of Multi-model Collaboration showing that ensembles of diverse models follow power-law scaling with a better loss floor than single-series collaboration. Together, the researchers argue, these ideas chart a path toward AI that is not confined to data centers but flows through the networks that already surround us.

Subject of Research: A multidisciplinary framework integrating AI and communication technologies for ubiquitous, low-latency intelligence across device-edge-cloud networks

Article Title: AI Flow: perspectives, scenarios, and approaches

Article References: An, H., Hu, W., Huang, S., Huang, S., Li, R., Liang, Y., Shao, J., Song, Y., Wang, Z., Yuan, C., Zhang, C., Zhang, H., Zhuang, W., & Li, X. (2026). AI Flow: perspectives, scenarios, and approaches. Vicinagearth, 3(1), Article 1. https://doi.org/10.1007/s44336-025-00031-y

Image Credits: AI Generated

DOI: 10.1007/s44336-025-00031-y

Keywords: AI Flow, edge AI, device-edge-cloud collaboration, familial models, large language models, speculative decoding, intelligence emergence, task-oriented feature compression, vision-language models, diffusion models, ubiquitous intelligence, 6G networks