Reinforcement Learning Optimizes Cooling Systems in Danish Data Centers

A deep reinforcement learning framework developed using high-fidelity digital twins successfully optimizes data center cooling operations while adapting dynamically to varying weather conditions, fluctuating electricity prices, and changing cooling demands. In experimental evaluations focusing on sustainable digital infrastructure, the computational control framework achieved energy-efficient cooling operation while simultaneously maintaining required thermal stability and storage balance across facilities. The findings demonstrate that combining physics-based digital twins with advanced reinforcement learning algorithms provides an intelligent and scalable approach to managing data center cooling systems, which currently represent a persistent operational challenge and account for a significant fraction of total operational energy consumption in modern digital infrastructure installations.

The framework utilizes high-fidelity digital twins developed in Modelica. These digital twins are deployed through Functional Mock-up Units.

The experimental investigation examined two representative Danish data center case studies featuring distinct thermal energy storage configurations. The first Danish data center facility investigated in the research employed Aquifer Thermal Energy Storage, utilizing an aquifer thermal storage system to manage cooling loads. The second Danish data center case study utilized Cold Storage technology to support facility cooling requirements.

To control the facility cooling systems, the researchers trained both on-policy and off-policy deep reinforcement learning controllers. The investigated algorithms included Proximal Policy Optimization, representing on-policy reinforcement learning, and Soft Actor-Critic, representing an off-policy reinforcement learning approach. Both the Proximal Policy Optimization and Soft Actor-Critic controllers were trained to minimize overall cooling costs while strictly maintaining thermal stability within the facility and preserving storage balance in the respective thermal reserves. In addition, reward redistribution mechanisms are introduced to improve long-term operational performance.

Experimental results demonstrate that the proposed framework successfully adapts to varying weather conditions, electricity prices, and cooling demands while achieving energy-efficient operation. The study highlights the potential of combining physics-based digital twins with deep reinforcement learning for intelligent and scalable data center cooling management.

Subject of Research: Data center cooling optimization using deep reinforcement learning and digital twins

Article Title: Cooling optimization of data centers utilizing on and off-policy deep reinforcement learning algorithms – two Danish data centers case studies

Article References: Ali, K., Filoniko, K., & Dominković, D. F. (2026). Cooling optimization of data centers utilizing on and off-policy deep reinforcement learning algorithms – two Danish data centers case studies. Applied Intelligence, 56(16), Article 543. https://doi.org/10.1007/s10489-026-07439-z

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07439-z

Keywords: data center cooling, deep reinforcement learning, digital twins, Modelica, Functional Mock-up Units, Aquifer Thermal Energy Storage, Cold Storage, Proximal Policy Optimization, Soft Actor-Critic, Cooling, optimization, data