New Runtime Tool Brings Energy Awareness to Supercomputer Applications Across Chip Architectures

Supercomputers consume enormous amounts of electricity, and as they grow toward exascale performance, their energy appetite has become one of the field’s most pressing problems. A research team from the University of Bologna, IT4Innovations National Supercomputing Center in Czechia, and CINECA in Italy now reports a software framework that could make it far easier to measure and manage that appetite. The tool, called CNTD_MERIC, was described in the journal Frontiers in High Performance Computing and validated on production hardware at the IT4Innovations supercomputing center.

The core challenge the researchers tackled is fragmentation. Modern high-performance computing spans multiple processor architectures, each with its own proprietary interface for reading power and energy data. Intel and AMD chips expose energy counters through an interface called RAPL, NVIDIA GPUs use NVML, and Fujitsu’s ARM-based A64FX processor, which powers the Fugaku supercomputer, provides its own set of APIs. A power-management tool written for one architecture often cannot simply be ported to another, leaving operators without a unified view of energy consumption across a heterogeneous machine.

CNTD_MERIC addresses this by merging two established software libraries into a single runtime system. The first, COUNTDOWN, is an energy-aware runtime for applications built on the Message Passing Interface, or MPI, the standard way parallel programs exchange data and synchronize work across compute nodes. COUNTDOWN intercepts MPI communication calls and selectively reduces processor performance during phases when the application is waiting on communication rather than computing, exploiting what engineers call communication slack. The second, MERIC, is a platform-agnostic energy measurement library that identifies distinct phases within an application and adjusts hardware configurations to minimize energy use for each one.

The integration required careful engineering. The team built a C-compatible wrapper layer, named MERICext, that bridges MERIC’s C++ template-based monitoring framework with COUNTDOWN’s C-based runtime. This wrapper exposes five core functions: one to initialize the energy monitoring domains requested by the user, one to collect timestamped energy readings, one to compute energy consumed between two timestamps, one to free the memory holding those timestamps, and one to finalize the domains and release resources. The wrapper is compiled into a static library and linked into COUNTDOWN’s shared library during the build process, so any MPI application linked with the runtime can be transparently profiled without modifying a single line of application code.

Once initialized, the integrated system enters a time-based sampling loop. At regular intervals, typically one second, a high-resolution POSIX timer triggers a sampling function that reads raw counter values from all enabled energy domains into timestamp structures. The system then computes the difference between successive readings to determine energy consumption in Joules for each interval, propagating the results into COUNTDOWN’s internal cumulative energy tracking. This double-buffering scheme allows the tool to build a detailed time-series profile of energy consumption throughout an application’s execution, all while the application runs normally.

To verify that this added abstraction layer does not slow anything down, the researchers ran seven NAS Parallel Benchmarks on Intel Xeon Max 9468 nodes, part of the Sapphire Rapids family launched in 2023. Each benchmark was executed six times to ensure statistical reliability. The results showed that execution-time overhead remained consistently below 0.5 percent, ranging between minus 0.25 and plus 0.45 percent, while energy and power measurements fluctuated within a narrow band of minus 5 to plus 3 percent, reflecting normal run-to-run variability rather than systematic bias. When the power-saving mode was activated, the framework achieved energy and power reductions of approximately 13 to 21 percent across the benchmarks, confirming that the integration preserved COUNTDOWN’s full optimization capability.

The team then turned to the ARM side of the comparison, deploying the framework on Fujitsu A64FX nodes at IT4Innovations. The A64FX, built on TSMC’s 7-nanometer process and featuring 48 cores with ARMv8.2-A scalable vector extensions and tightly integrated HBM2 memory, was the first ARM-based HPC processor deployed at large scale, powering Fugaku, which held the number one spot on the Top500 list from June 2020 to June 2022. Running six NAS benchmarks that could execute on both architectures, the A64FX achieved significantly lower execution times, reduced energy-to-solution, and dramatically lower average power consumption, operating between 110 and 132 watts compared with 400 to 590 watts on the Intel nodes.

A particularly striking finding emerged from the communication statistics that COUNTDOWN collects. On the Intel Xeon system, benchmarks such as SP, CG, and IS spent more than 90 percent of their runtime in MPI communication phases, even within a single node. On the A64FX, more than 90 percent of runtime was devoted to actual application computation across all benchmarks. The researchers attribute this shift to architectural differences: the A64FX’s tightly integrated HBM2 memory and wide vector units provide high memory bandwidth and efficient computation, while the Intel node’s combination of DDR5 and HBM2e introduces higher communication costs for memory-bound workloads.

The evaluation extended beyond synthetic benchmarks to real-world workloads. The team tested a convolutional neural network, ResNet50, pretrained on the ImageNet dataset with approximately 25.5 million parameters, running distributed inference with PyTorch and an MPI backend. They also ran the STREAM memory bandwidth benchmark and the Quantum ESPRESSO electronic-structure suite in multi-node configurations scaling up to eight A64FX nodes. Execution time dropped substantially as process counts increased, with a larger Wide ResNet model improving from 723.6 seconds on two nodes to 252.3 seconds on eight nodes. Energy behavior proved more nuanced: for Wide ResNet and the BT benchmark, energy consumption followed a non-monotonic trend, first rising, then reaching a minimum at intermediate process counts, before increasing again at the highest counts due to growing communication and synchronization costs.

The researchers acknowledge that their current validation covers only CPU-based systems, specifically Intel x86 and ARM A64FX architectures, and that portability to GPUs and other accelerators remains unexplored. They plan to extend the framework toward co-tuning strategies that coordinate COUNTDOWN’s optimization of communication phases with MERIC’s tuning of computation, memory, and I/O-bound phases, and to explore integration with system-wide frameworks such as PowerStack. For now, CNTD_MERIC stands as a demonstration that unified, architecture-independent energy monitoring and power management for MPI applications is achievable with negligible performance cost, offering supercomputer operators a practical tool for the energy-conscious era of computing.

Subject of Research: A unified runtime framework for energy-aware monitoring and power management of MPI applications on x86 and ARM HPC systems

Article Title: Toward energy-efficiency: CNTD_MERIC approach for energy-aware MPI applications

Article References: Ad Dooja, K., Yasal, O., Vysocky, O., Riha, L., Cesarini, D., & Bartolini, A. (2026). Toward energy-efficiency: CNTD_MERIC approach for energy-aware MPI applications. Frontiers in High Performance Computing, 4, Article 1664774. https://doi.org/10.3389/fhpcp.2026.1664774

Image Credits: AI Generated

DOI: 10.3389/fhpcp.2026.1664774

Keywords: high-performance computing, energy efficiency, MPI, CNTD_MERIC, COUNTDOWN, MERIC, A64FX, Sapphire Rapids, power management, DVFS, ARM processors, supercomputing