The review, authored by Xinyu Li, Zhaofa Zhou, Zhili Zhang, Zhenjun Chang, Zheng Lian, Hui Duan and colleagues at Rocket Force Engineering University in Xi’an, China, positions TFNAS as a distinct and emerging research paradigm within automated machine learning. Neural architecture search has long served as what the authors call a performance engine for AutoML, driving intelligent advancements across domains from computer vision to speech processing. Yet its development has been significantly constrained by prohibitively high computational costs, a bottleneck that has kept sophisticated architecture discovery largely out of reach for all but the best-funded laboratories.

The core insight behind TFNAS is deceptively simple. Rather than training each candidate network to convergence and measuring its accuracy, the method scores architectures at initialization or after only minimal computation, using proxy metrics that correlate with final performance. Metrics in this family include analyses of network expressivity, progress indicators measured without gradient updates, and zero-cost probes that estimate how well a freshly initialized model will learn. Because no candidate network is trained inside the search loop, thousands of architectures can be screened in the time it would take to train just one.

Importantly, the review offers a careful definition of what training-free actually means. In its strict sense, the term refers to the absence of candidate network training during the search process itself. In a broader interpretation, it also permits the offline pre-training of lightweight performance predictors, so long as those predictors do not constitute training at the search stage. This distinction matters, the authors argue, because it clarifies the boundary of the field and prevents confusion with related approaches such as one-shot and supernet-based methods, which still rely heavily on weight sharing and partial training.

To bring order to a rapidly proliferating literature, the review proposes a two-dimensional taxonomy organized around the source of information used by a proxy metric and the methodology by which that metric is constructed. The information-source dimension captures whether a metric draws on architectural properties, weight statistics, activation patterns, gradient signals or learned predictors. The construction dimension distinguishes metrics that are hand-designed from theoretical principles, those derived empirically from large collections of trained architectures, and hybrid approaches that combine both. The authors use this framework to trace the developmental trajectory of state-of-the-art techniques, showing how the field has moved from simple initialization statistics toward increasingly sophisticated, multi-signal evaluation schemes.

One of the review’s most consequential claims concerns the co-evolution of evaluation metrics and search strategies. The authors contend that the two components should not be designed in isolation. Instead, they posit that joint optimization characterized by dynamic feedback and mutual reinforcement is key to maximizing efficacy. In practice, this means a search algorithm can be tuned to exploit the strengths and tolerate the weaknesses of its scoring function, while the scoring function can be refined in light of where the search actually explores. This evolutionary mechanism, the review suggests, explains why some TFNAS pipelines dramatically outperform others even when built from superficially similar parts.

The promise is substantial. If architecture evaluation that once required days of GPU time can be compressed into seconds, automated machine learning becomes accessible to researchers without massive compute budgets, and the design cycle for deployment-critical models on edge devices, smartphones and embedded systems shrinks dramatically. The technique also reduces the energy footprint of architecture discovery, an increasingly pressing concern as the computational appetite of AI research continues to grow. By lowering the barrier to entry, TFNAS could democratize a layer of AI engineering that has been effectively reserved for industrial giants.

Yet the review is candid about the obstacles. The authors identify four core contradictions that persist as the technology moves toward practical application. These include the persistent gap between what cheap proxy metrics measure and what real trained performance requires, the risk that a metric optimized for one architecture family or dataset fails to generalize elsewhere, tensions between evaluation speed and evaluation fidelity, and the difficulty of integrating training-free scores into search strategies that were originally conceived around trained validation accuracy. While the article’s abstract names these contradictions without exhaustively detailing each, the authors make clear that resolving them is the central task standing between current research and reliable, widely applicable deployment.

The review also outlines future directions for the field. These include developing more robust and interpretable proxy metrics, designing search strategies purpose-built for training-free signals, and establishing standardized benchmarks that would allow fair comparison across the fragmented landscape of published methods. The emphasis on interpretability echoes a broader concern in machine learning: practitioners are more likely to trust a zero-cost score if they understand why it predicts performance, rather than treating it as an opaque oracle. The authors frame their taxonomy and analysis as a foundation intended to guide that maturation process.

Published open access on 11 September 2026 after being received in March and accepted in early September, the article arrives at a moment when the AI community is scrutinizing the cost of progress. By systematically mapping what training-free architecture search has achieved, where it falls short, and how its evaluation and search components can be co-designed, the review offers both a reference work and a research agenda. If its diagnosis is right, the next generation of neural networks may be discovered not by brute-force training marathons, but by fast, principled measurement, a shift that could reshape who gets to build AI and how quickly the technology evolves.

Subject of Research: Training-free neural architecture search methods for reducing the computational cost of automated machine learning

Article Title: Systematic review of training-free neural architecture search: research progress, core challenges, and future directions

Article References: Li, X., Zhou, Z., Zhang, Z., Chang, Z., Lian, Z., & Duan, H. (2026). Systematic review of training-free neural architecture search: research progress, core challenges, and future directions. Artificial Intelligence Review. https://doi.org/10.1007/s10462-026-11701-8

Image Credits: AI Generated

DOI: 10.1007/s10462-026-11701-8

Keywords: neural architecture search, training-free evaluation, automated machine learning, proxy metrics, NAS, deep learning, AutoML, computational cost, search strategy, interpretability, model design, practical deployment