The scale of the problem is difficult to overstate. Drawing on scenario-based 2025 estimates, the review reports that inference for a single model, GPT-4o, was modeled to consume between 391.5 and 463.3 gigawatt-hours of electricity annually, alongside 1,335 to 1,580 megaliters of freshwater and 138 to 163 kilotonnes of carbon-dioxide-equivalent emissions. Across 369 generative AI models, global training plus inference has been estimated at 25 to 41 terawatt-hours, concentrated largely in United States and Chinese data centers. Meanwhile, global data-center electricity consumption reached approximately 415 terawatt-hours in 2024, with AI identified as a key driver of projected growth through 2030. Against these figures, the brain’s 20-watt operating budget looks almost absurdly frugal, and the authors are careful to note they use it as contextual evidence that competent cognition can persist under tight constraints, not as a directly transferable efficiency benchmark.

To untangle the mess, the team conducted a systematized narrative review that began with 6,284 identified records, screened 898 at the full-text stage, and selected an initial set of 150 focal studies for detailed synthesis. From this evidence base they constructed a cross-layer framework organizing AI’s resource pressures into seven interacting regimes: diminishing returns from scaling, the cost of broad empirical experimentation, hardware energy floors, data-movement penalties, centralized infrastructure demands, benchmark-driven evaluation incentives, and static inference allocation. The central insight is that these regimes feed one another. Scaling inflates experimentation and memory traffic, infrastructure concentration shapes who can do research at all, and benchmark incentives push labs toward ever more compute-intensive optimization loops.

The first regime confronts the celebrated scaling laws that have governed transformer development. Performance improves predictably as parameters, data, and compute grow, but the returns follow shallow power laws. Using the fitted exponents reported by Kaplan and colleagues, the authors calculate that a mere 10 percent reduction in fitted test loss would require roughly 3.03 times more training tokens, 4.00 times more non-embedding parameters, or 8.23 times more minimum training compute. Human learning offers a striking contrast: people generalize new concepts from a handful of examples by reusing and recombining structured prior knowledge, including parts, rules, and skills, rather than accumulating vast task-specific experience. Translating that principle into AI suggests data selection and pruning, modular adapters, meta-learning, and compositional representations as promising directions, though the review cautions that most reported gains are measured in data volume or convergence time rather than actual energy saved.

The second regime examines the hidden cost of experimentation itself. Hyperparameter searches, architecture comparisons, ablation studies, and failed runs can dwarf the compute of a final training run, yet they are chronically underreported. The authors’ own analysis of 374 AI systems published between 1952 and 2022 found that training compute doubled every 20.6 months before 2010, but accelerated to every 7.3 months for regular-scale systems from 2010 to 2022 and every 6.6 months for higher-scale systems, and these figures exclude search and tuning overhead entirely. Human decision-making research suggests an alternative: rather than broad trial and error, people prune their search space using subgoals, means-ends analysis, and backward planning from desired outcomes. Multi-fidelity evaluation, successive halving, and early termination of unpromising configurations offer AI analogues, with energy-aware successive halving already demonstrating that wasteful experimental runs can be cut short.

Hardware tells a sobering story in the third regime. Although Moore’s law and Dennard scaling once delivered decades of performance gains at roughly constant power, that coupling has broken down. The review’s analysis of 62 machine-learning accelerators from 2017 to 2025 found that TDP-normalized peak tensor throughput increased about 20.0 percent per year, yet reported thermal design power rose 13.1 percent annually. Between 2023 and 2025, median efficiency improved 49 percent while median TDP climbed 94 percent. In other words, chips are getting more efficient per operation but also more power-hungry per device, and quantum tunnelling limits and voltage-scaling constraints impose physical floors that cannot be engineered away. The brain, by contrast, works within a fixed metabolic budget through sparse, discrete spiking, in which only a small fraction of neurons fires at any moment, and efficient coding that strips redundant information from sensory streams. Near-term AI translations include low-precision computation, quantization, and sparse activation, though the authors stress that savings depend heavily on hardware support and workload.

Perhaps the most technically striking regime concerns data movement. Modern AI hardware repeatedly shuttles weights and activations between memory levels, and accessing off-chip DRAM can cost hundreds of times more energy than a single arithmetic operation. The review cites normalized figures in which DRAM access costs 200 times the energy of one multiply-accumulate operation, compared with 6 times for an on-chip buffer and just 1 times for a local register file. Between 2012 and 2022, transistor-switching energy per bit fell roughly tenfold, yet reported end-to-end energy per instruction remains 2.3 times ten-to-the-fourth to 9.7 times ten-to-the-sixth times higher than transistor-level energy per bit once fetching, buffering, and control overheads are counted. Biological neural circuits sidestep this problem entirely by storing memories as synaptic changes within the very assemblies that process them, eliminating any von Neumann-style separation of storage and computation. Near-memory and in-memory computing architectures pursue the same locality principle, and three-dimensional analog in-memory accelerators have been reported to outperform conventional GPUs on memory-bound mixture-of-experts workloads, though much of the evidence still comes from simulation rather than production systems.

The remaining regimes highlight institutional and deployment-level inefficiencies. Centralized infrastructure concentrates frontier training in a handful of well-resourced industry labs, raising barriers to academic research and independent evaluation. Benchmark-driven optimization rewards marginal score gains while ignoring resource costs, a pattern the authors associate with so-called Red AI, and a sampled set of ACL papers confirmed that authors tend to prioritize accuracy over efficiency. Human cognition, by contrast, employs satisficing, accepting the first option that clears a sufficiency threshold rather than exhaustively maximizing a proxy. On the deployment side, standard dense models process every input through the full network regardless of difficulty, an approach that evidence suggests wastes computation on easy cases and can even hurt accuracy through overthinking. Adaptive techniques such as early exit, conditional routing, and speculative decoding can match computation to demand, mirroring how selective attention in the brain gates processing toward task-relevant signals.

The review’s most important caution is that efficiency claims rarely survive contact with the full system boundary. Retrieval-augmented generation can let smaller models rival larger ones, but corpus embedding, index construction, vector search, and reranking add their own energy costs. Mixture-of-experts routing reduces activated computation, yet expert imbalance and inter-device communication can erode the arithmetic savings. Multi-agent systems may improve task performance while adding repeated inference and coordination overhead, and controlled experiments indicate that sufficiently capable single agents can outgrow the benefits of collaboration entirely. The authors conclude that data curation, energy-aware experimentation, hardware-supported low-precision computation, and demand-adaptive inference represent the strongest near-term opportunities, while near- and in-memory computing hold longer-term promise. Their overarching prescription is cross-layer co-design evaluated with direct end-to-end energy measurements at matched capability, reliability, latency, and service quality, because a watt saved in one layer is all too easily spent, invisibly, in another.

Subject of Research: Energy constraints in large-scale artificial intelligence and human-cognition-inspired efficiency strategies for large language models

Article Title: Energy constraints in large scale artificial intelligence and insights from human cognition for efficient large language models

Article References: Energy constraints in large scale artificial intelligence and insights from human cognition for efficient large language models. (n.d.). https://doi.org/10.1007/s44163-026-02369-8

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02369-8

Keywords: large language models, energy efficiency, Green AI, human cognition, scaling laws, data movement, in-memory computing, adaptive inference, data centers, resource rationality, sustainability, machine learning