Amazon Web Services unveils custom cooling for Nvidia AI chips

Investing.com -- Amazon Web Services (NASDAQ:AMZN) introduced new in-house cooling hardware on Wednesday designed specifically for Nvidia’s (NASDAQ:NVDA) latest graphics processing units that power generative artificial intelligence systems.

AWS developed its custom-built infrastructure solution—the In-Row Heat Exchanger (IRHX)—instead of adopting widespread industry-standard liquid cooling solutions. This approach allows AWS to accommodate Nvidia’s high-density GPU racks without major data center renovations.

Nvidia’s Blackwell GPUs, created for large-scale AI training and inference, consume substantial energy and produce significant heat, requiring advanced cooling systems. Traditional air cooling methods that worked for earlier chip generations cannot handle the thermal demands of racks like Nvidia’s GB200 NVL72, which packs 72 GPUs into a single rack.

"They would take up too much data center floor space or increase water usage substantially," said AWS Vice President of Compute and Machine Learning Services Dave Brown in a presentation video. "And while some of these solutions could work for lower volumes at other providers, they simply wouldn’t be enough liquid-cooling capacity to support our scale."

The IRHX cooling system combines liquid and air-based components for efficient heat management. Developed in partnership with Nvidia, it circulates coolant to GPU chips through cold plates and removes heat via fan-coil arrays, while maintaining the air-cooled mechanical layout of AWS’s standard racks.

This cooling innovation coincides with AWS launching new computing instances: EC2 P6e and P6e Ultra. These instances give customers access to Nvidia’s HGX B200 and GB200 NVL72 stack respectively, with the latter being one of AWS’s most powerful server configurations to date, designed for advanced model development and deployment.

AWS also highlighted its Nitro infrastructure platform, which provides networking and system monitoring for these new instances. The Nitro system enables firmware updates without service interruption and supports up to 28.8 terabytes per second of networking bandwidth per Ultra server, essential for AI workloads that scale across tens of thousands of GPUs.

This article was generated with the support of AI and reviewed by an editor. For more information see our T&C.