How does NVIDIA's innovative "hot water cooling" technology significantly save electricity and water?

NVIDIA's next-generation AI servers operate by heating the coolant to 45 degrees Celsius, thereby bypassing the most power-hungry chiller and using dry coolers to dissipate heat outdoors.
(Previous Summary: Dell Teams Up with NVIDIA to Launch 'All-Liquid Cooling' AI Servers! First to Feature Vera Rubin Architecture, Single Rack with 144 GPUs Breaks Computing Power Records)
(Background: NVIDIA's 800V Power Supply Revolution Not Delayed! Confirmed with Delta Electronics and ABB Partners: Mass Production on Schedule in Q3)

Table of Contents

Toggle

• From 20kW to 140kW: Air Cooling Hits the Wall for Server Racks
• 45°C Hot Water Cooling: Why Does It Actually Save More Power?
• 25x Energy Efficiency, 300x Water Savings

On a data center's bill, GPUs aren't the only greedy component; the cooling system has long consumed up to 40% of total electricity, making it one of the biggest burdens on profit margins.

However, NVIDIA's next-generation Rubin server provides a counterintuitive answer: to save power, first heat the water. The new 'hot water cooling' technology raises the operating temperature of the coolant to 45 degrees Celsius, abandoning the conventional wisdom that 'colder is better.'

From 20kW to 140kW: Air Cooling Hits the Wall for Server Racks

Three years ago, a single data center rack consumed about 20 kW, which could be handled by regular air conditioning and fans. After GPU generations, that number has more than sextupled. Rack density in hyperscale facilities now commonly exceeds 135 kW, and the total thermal power of a single NVIDIA GB200 NVL72 rack approaches 130 to 140 kW.

There is a physical limit to how much heat fans can dissipate. Once rack density crosses a certain threshold, air cooling is no longer an efficiency issue but a feasibility issue. This is why liquid cooling has become the default architecture for AI factories. With chips stacked up to the scale of NVL72, the only way to remove heat in real time is to have liquid directly contact the chip surfaces.

45°C Hot Water Cooling: Why Does It Actually Save More Power?

The core of a liquid cooling architecture is to directly flow coolant over the heatsink fins on the chip surface, carrying away heat and then sending it out of the rack, eliminating the inefficient intermediary of fans blowing air.

The Rubin generation's coolant enters the chip at 45°C and exits at about 55°C after absorbing heat, in a fully closed loop with no fans inside the system, and performance is not compromised.

The single most power-hungry component in a data center is the mechanical chiller. Simply put, it is the 'heart of the air conditioner' that uses a compressor to forcibly cool water. Industry estimates suggest that for every 1°C increase in chiller outlet temperature, about 4% of cooling energy can be saved. By raising the water temperature all the way to 45°C, they essentially skip the compressor stage and switch to a dry cooler.

In simple terms, it is a passive device that uses only fans to blow the heat from the hot water into the outdoor air, without evaporating water or using a compressor.

25x Energy Efficiency, 300x Water Savings

The results are reflected in the numbers pointed out on NVIDIA's official blog: The GB200 NVL72 offers 25x better energy efficiency and 300x better water efficiency compared to traditional air-cooled architectures; the next-generation GB300 NVL72's energy efficiency advantage expands to 30x.

Converted into the PUE metric that measures data center power efficiency, it effectively cuts a large chunk of inefficient energy consumption.

The disparity on the water side is even more straightforward. Traditional cooling mainly uses evaporative cooling towers. Simply put, it is a cooling method that relies on water evaporation to remove heat, which is effective but evaporates millions of gallons of water each year. Such systems consume about 2.6 million gallons of water per MW per year.

Switching to closed-loop liquid cooling with dry coolers can reduce water usage to nearly zero in suitable climate zones, cutting it by up to 100%. For a 50 MW hyperscale data center, after converting to liquid cooling infrastructure, the savings on cooling-related electricity and water costs alone can exceed $4 million per year.