AI is rewriting how power flows through the datacenter

Rising rack densities are driving changes from grid connection to chip-level delivery

Power semiconductors are soon set to become as vital as GPUs and CPUs in datacenters, handling the rapidly increasing loads forecast for AI infrastructure.

The AI craze has seen datacenter operators stuffing servers with as many accelerators (typically GPUs) as will fit inside, then filling the racks in their data halls with these high-performance boxes.

With some GPUs now using 700 W of energy each, the power required by an entire rack or datacenter of kit is growing rapidly, putting a strain on electrical distribution infrastructure.

Last year, Nvidia introduced an integrated rack-scale system for AI processing, the DGX GB200 NVL72, which needs 120 kW per unit, while cloud giant Google disclosed this year that it is planning for datacenter racks supporting 1 MW of IT hardware loads.

To cap it all, some of the key infrastructure components such as transformers are reported to be holding things up through long lead times of 28 weeks or more.

This is where power semiconductors come into the picture. They enable solid-state transformers (SSTs), which are much smaller and more efficient than their conventional counterparts – perhaps just 500 kg as opposed to multiple tons.

"What's now being introduced is a solid-state transformer that has a couple of advantages on the supply chain side, but also on this performance side, and that is a pretty new application," said Dr Peter Wawer, division president for Green Industrial Power at chipmaker Infineon.

SSTs are a great deal lighter and smaller, Wawer told us, because they operate at very high frequency, a couple of orders of magnitude higher than the AC power supply frequency fed into them. Because the solid-state transformer switches at such high frequency, it requires silicon carbide components.

"Silicon carbide is the only material that has the capability to switch at high frequency and has high blocking voltages. And for blocking voltage, we mean up to two or three kilovolts (kV), which this semiconductor device will need to block," Wawer said. "But then, of course, you have input voltages of more than 30 kV, and therefore you have to stack more than 10 of the devices in a kind of cascaded structure."

Infineon is a prominent supplier of power semiconductors, and so has an interest in pushing the development of solid-state components that are set to be in demand as the datacenter build boom continues.

The firm expects that SSTs will replace a portion of the datacenter transformer market and could be worth up to $1 billion by 2030, for example.

Circuit breakers are nother application where power semiconductors may replace legacy kit. These electromechanical devices open up in a few milliseconds in the event of a short circuit, but a silicon carbide solid-state version can open much faster, in microseconds – handy if you are dealing with high voltages being supplied to racks full of very expensive GPUs.

Power semiconductors, based on silicon carbide or gallium nitride, are also set to play a part in the power distribution around the datacenter itself.

Nvidia has signposted that it sees 800 VDC power supplies as the way forward, as the power demands of AI infrastructure get ever higher, saying this move will be required to support 1 MW IT racks and beyond. It claims this reduces the number of voltage conversions needed, which can introduce inefficiencies and increase the complexity of the electrical system, among other things.

It is also a way of minimizing losses, according to Infineon, because the power loss is proportional to the square of the current. The trick therefore, when you want to go to a higher power and keep losses under control, is to increase the voltage to a level that means you keep the current constant.

"Today, you have a combined rack that has the IT payload as well as the power delivery and the backup power. Now, as the GPUs, TPUs, and ASICs increase in power, that means that you need to bring in more power into the racks themselves," said Adam White, division president for Power & Sensor Systems at Infineon.

"So we now move to this potential 800 volt DC supply, but you've also got the plus-minus 400-volt topologies that some vendors [such as Google] may end up using. We call this then the hybrid microgrid."

"Why 800 volts DC? Effectively, you can bring that directly into the motherboards or into the compute trays, using fewer conversion steps," he added.

However, Infineon believes that before we can get to that step, there will have to be an intermediate stage, where a "power sidecar" – effectively a second rack full of power delivery and backup circuitry – will sit alongside the rack of IT kit it serves. Whether this serves just one rack or several will depend on the IT load.

This arrangement is expected to see racks hitting 600 kW of power, up from the maximum of about 125 kW today, to support configurations using Nvidia's Rubin Ultra, expected to arrive in late 2027.

Google also mentions a "sidecar" in its plans for 1 MW kit, as a dedicated AC-to-DC power rack that feeds power to the other racks.

As power demand rises, the power supply unit (PSU) within the rack will need to handle more power, with 8 kW units available now. There will also be a transition to a three-phase design to move from a 12 kW unit to 16 kW or above, which is expected to happen next year. This transition will see greater use of components made with silicon carbide and gallium nitride (GaN), according to Infineon.

Finally, the increasing density of the AI accelerators themselves and the amount of power all those transistors guzzle is starting to have an impact at the chip level.

The voltage regulator modules (VRMs) are currently discrete packages that sit alongside the accelerator chip package (GPU, ASIC or TPU) on the system board, but as the current to the chip rises, parasitic losses also increase, wasting energy.

"Some of the industry has already recognized that and started to move to what we call BVM, or back-side vertical power module," White said. This moves the VRMs to the other side of the system board, below the accelerator chip package, and simplifies the board layout as well as reducing overall power delivery network losses.

Ultimately, the industry would like to get to the point where the VRM moves inside the chip package and is embedded in the substrate directly beneath the chip, but this is a development set for beyond 2027, according to Infineon. ®