In my last blog, that detailed the difference of selling AI “Training vs. Inference” use cases, I made a passing remark about the power limitations and requirements of running AI workloads- I’d like to dig into power a little bit more today…
Mark Zuckerberg stated in this very enlightening interview/podcast by Dwarkesh Patel, that the GPU supply chain (while problematic last Winter &Spring) is no longer the limitation in deploying AI workloads- it’s power.
For example some GPUs consume ~700W, which means (as reported by Electronic Specifier) running at 61% utilization the energy consumption translates to “roughly 3,740 kilowatt-hours (kWh)” annually, or the “equivalent to the average American household”. Further, according to Paul Churnock, Microsoft’s Principal Electrical Engineer of Datacenter Technical Governance and Strategy, the installation of millions of these high-powered GPUs will consume more energy than all households in Phoenix, Arizona by the end of 2024.
Indeed, Goldman Sachs predicts that data center power consumption will be 160 times what it is today by 2030 to accommodate the compute power required by evolving AI applications.
So, for the owner/operator of a standard datacenter, even if you have the means to acquire these GPUs, would you have the power to use them?
What will we see in this space as datacenters become more power hungry? Nuclear.
That’s right. A recent article in the Washington Post discusses how tech firms are seeking to pioneer everything from small-scale nuclear reactors to driving breakthroughs in nuclear fusion to meet the crushing power demands of artificial intelligence.
Further, if you’re not reading/following Bill Kleyman, I’d recommend it. He recently posted to LinkedIn about the GIANT fans Tesla is installing to cool their datacenters. In this recent article on DataCenter Knowledge, he says, “The amount of energy that generative AI consumes can be pretty staggering. A single Google search can power a 100-watt light bulb for about 11 seconds. GPT-like instances can be anywhere from 600 to 800 times more powerful than a single Google search.”
It was in this same article that I first learned the term “SMR” (small modular reactors) which are mini, pre-assembled nuclear plants perfect for powering the datacenters of our new reality.
If you have the time, I also recommend researching SMR technology to see how it may apply to your customer base. A couple helpful sites can be found here, here, and here…
We’ll talk more about how composable infrastructure software and solutions integrate into a more sustainable data center in future blogposts. For now, if you missed it, check out my previous blog entry on how resellers and integrators should capitalize on inference infrastructure needs now and follow us on LinkedIn for our latest thoughts on the evolving world of AI-driven IT.
