As Nvidia works to install some of its newest chips in Microsoft data centers, an employee at the GPU giant observed in early fall that Microsoft’s cooling approach at one facility seemed “wasteful.”
Nvidia has been deploying its GB200 Blackwell architecture at Microsoft and other tech giants as demand for compute to train and run AI models surges.
Blackwell, announced in March 2024, is roughly twice as powerful as its predecessor, Hopper, Nvidia CEO Jensen Huang said at launch. GB200 is part of an earlier wave of Blackwell deployments, with the GB300 generation now available.
In early fall, an internal email sent by a staffer on the Nvidia Infrastructure Specialists (NVIS) team described one Blackwell installation of server racks for OpenAI, which Microsoft supports as its cloud partner and largest investor.
The email described the setup of two GB200 NVL72 racks, each of which houses 72 Nvidia GPUs. The setup uses liquid cooling technology, given the heat generated by multiple GPUs operating closely in tandem.
The staffer wrote that Microsoft’s “cooling system and data center cooling approach for their GB200 deployment seems wasteful due to the size and lack of facility water use, but does provide a lot of flexibility and fault tolerance,” according to the memo.
While liquid cooling is used for the servers, data centers also use a second, building-level system to expel heat from the facility, according to Shaolei Ren, an associate professor of electrical and computer engineering at the University of California.
The Nvidia employee may have been referring to a building-level system that uses air-cooling instead of water, explained Ren, who studies how data centers use water and other resources.
“This type of cooling system tends to be using more energy,” he said, “but it doesn’t use water.”
A Microsoft spokesperson described a cooling setup consistent with Ren’s two-phase explanation.
“Microsoft’s liquid cooling heat exchanger unit is a closed-loop system that we deploy in existing air-cooled data centers to enhance cooling capacity on first and third-party platforms,” the Microsoft spokesperson told Business Insider in a statement.
“These systems ensure we maximize our existing global data center footprint for scale while promoting efficient heat dissipation and optimizing power delivery to meet the demands of AI and hyperscale systems,” the spokesperson added.
“A trade-off” between resources
As AI infrastructure expands, energy and water use in data-center cooling have become flashpoints globally, prompting pushback in some regions where new facilities are being built.
Ren noted that because data centers can use air cooling, water cooling, or a hybrid system at the building level, “there’s a trade-off” between resources.
Air cooling requires more energy, but can “address some of the public concerns with water consumption — because water is something people can really see,” he said.
“These companies are profit-driven,” he added, “they weigh in the water cost, the energy cost, and also the publicity cost.”
Microsoft, for its part, said it intends to be “carbon negative, water positive, and zero waste” by 2030.
“We’ve also announced a zero water cooling design for our next-generation data centers and breakthroughs in on-chip cooling,” the spokesperson said.
Inside the Blackwell installation
The internal email from the Nvidia staffer described some logistical hiccups that occurred during the Blackwell installation in early fall, which can be typical in the early deployment of new data center hardware.
“Onsite support for this activity was a necessity,” the staffer wrote. “Many hours were spent creating the validation process documentation as well as vetting the steps worked and made sense to those less familiar with how cluster and system validation is usually performed.”
Additionally, the handover processes between Nvidia and Microsoft “required a lot more solidification than what was performed before arrival.”
Still, the memo suggested Blackwell’s production hardware quality had improved compared to early samples.
The email said GB200 NVL72 production hardware “has good quality” compared to the qualified samples sent to customers for early testing. Both racks had a 100% pass rate on certain compute performance tests.
An Nvidia spokesperson told Business Insider that its Blackwell systems “deliver exceptional performance, reliability, and energy efficiency for a wide variety of computing applications.”
“Our customers, including Microsoft, have successfully deployed hundreds of thousands of Blackwell GB200 and GB300 NVL72 systems to meet the world’s growing need for artificial intelligence,” the spokesperson said.
