Powering the AI Factory

Introduction: The AI Factory

AI data centers are fundamentally different from the cloud data centers of the previous decade. Data centers of the past delivered data; modern AI data centers deliver intelligence. This fundamental shift has led to entirely new vocabulary: these new data centers have been dubbed AI factories to emphasize their industrial scale.

AI factories present unprecedented challenges for the power system. On the demand side, AI factories increase the demand for electricity by an order of magnitude relative to previous generations of data center: from the range of ~20 megawatts (MW) to ~1-gigawatt (GW). To supply this enormous power demand, renewable, nuclear and natural gas power plants have seen major investment inflows.

But it is not enough to simply match gigawatt-scale power demand with gigawatt-scale energy sources. The demand and supply have to be connected to allow this huge amount of energy to be delivered, stored, and used to power the chips that produce the intelligence that enables the unfolding AI revolution. Current data centers, designed to handle the power requirements of the past, use legacy technologies and system designs that must be reimagined and reinvented to meet the 50x increase in power requirements of the coming AI factories.

In Part 1 of this blog, we will explain the basics of AI factory design and highlight the challenges of delivering 50x more power. In Part 2, we will look at how QS solid-state lithium-metal battery technology can address these challenges and boost the intelligence density of AI factories.

The AI Factory Floor: The Token Forge

AI factories are extremely complex works of engineering: pipes for liquid cooling, wiring for power delivery, cables for networking, and so on. Fundamentally, the building block of an AI factory is the rack. Different hyperscalers use different accelerators; for example, compute racks may consist of graphics processing units (GPUs), tensor processing units (TPUs), or other custom designs (XPUs).

These compute racks are what transform electricity into intelligence; the quantity of input and output is measured as tokens. Tokens are the basic input and output of AI models and are how AI companies generate revenue. To a first approximation, the more tokens you sell, the more money you make.

To power compute racks, a rack of power electronics sits in the adjacent rack. These power racks feed the compute racks with the high-quality, stable electrical power at the voltages required for the chips to operate. Because AI workloads are highly variable and unpredictable, a buffer is required to decouple the volatile power demands of the chips from the steady flow of power provided by the grid or local power plant.

In data centers of the past, this buffer could be provided by capacitors or uninterruptable power supply (UPS) systems based on lead-acid or lithium-ion batteries. For example, if a compute rack draws a peak of 140 kilowatts (kW), there is enough battery storage to provide up to 140 kW of power for a short period of time, typically 45-90 seconds.[1]

Intelligence Density and its Implications

The diagram below shows a real-world example of what a 140-kW compute rack and power rack look like today. As you can see, in this 140 kW system, lithium-ion batteries already occupy a meaningful fraction of the available space in the power rack. As future AI factories move toward much higher-power racks, this will become a more and more significant issue.

For example, current plans envision racks that draw 1 MW before the end of the decade. If these AI factories were designed using current lithium-ion batteries, an 8x increase in power per rack would require an 8x increase in the amount of space needed for these lithium-ion batteries. Because of the space taken up by these batteries, every compute rack would then require multiple power racks.

This is a problem: increasing the number of power racks per compute rack means that fewer compute racks can be installed in a given area. Because the compute racks hold the chips and sophisticated low-latency networking that actually generate the intelligence of an AI factory, reducing the amount of compute per square foot reduces the intelligence density – and raises the cost – of the AI factory.

This reflects a limitation of conventional lithium-ion batteries: high-power lithium-ion cells, the kind required to feed the power-hungry AI workloads, are fundamentally less energy-dense. What’s more, because of the safety risks of lithium-ion batteries, they must be loosely packed together, further reducing intelligence density of the overall system. Conventional lithium-ion batteries present a serious safety risk to data centers: in at least one case, an explosion in a lithium-ion battery was blamed for a data center fire.[2] Fires in data centers endanger human lives, knock out essential services, and put billions of dollars’ worth of capital equipment at risk.

It’s important to take a step back and look at the system-level implications: reduced intelligence density increases the engineering overhead of the whole AI factory. This is because every cable, pipe, and wire connecting the compute racks must now be longer, which increases the cost and reduces the efficiency of the system. The building itself must also be larger to achieve the same amount of intelligence output, or produce fewer tokens in the same space, reducing the revenue the AI factory can generate. Consequently, the safety and performance limitations of current lithium-ion batteries do not just affect the batteries themselves: the reduction in intelligence density negatively impacts every other system in the AI factory.

This is a significant problem. An AI factory at the 1 GW scale is estimated to cost tens of billions of dollars to construct, and from this vantage point, if reduced intelligence density leads to even a 1% reduction in capital efficiency, it would mean hundreds of millions of dollars in unnecessary expenditure. However, the reverse is also true: if intelligence density can be improved, the savings could be substantial, and the value of such a technology is potentially enormous.

Conclusion

Power delivery for 1 MW racks in the AI factories of the future is seriously constrained by the limitations of conventional lithium-ion batteries, and this constraint is set to become more and more limiting over time. There is an obvious and compelling opportunity to improve on conventional lithium-ion technology in the AI data center.

In part 2, we will look at how QS solid-state lithium-metal battery technology can help overcome the energy, power, and safety limitations of lithium-ion batteries and boost the intelligence density of AI factories.

[1] In a typical system design, these batteries are rated around 10C-15C discharge.

[2] For example, the data center fire at the National Information Resources Service in Daejeon, South Korea knocked out hundreds of government services for days, with dozens of services rendered unrecoverable. The cause of the fire was a lithium-ion battery that went into thermal runaway.