Racks of AI Chips Are Too Damn Heavy (theverge.com) 48
The weight of AI server racks has reached a point where legacy data centers cannot accommodate them even with significant retrofitting efforts, The Verge reports. Chris Brown, chief technical officer at Uptime Institute, said most retrofitting attempts would require "bulldozing the building and starting over from scratch."
AI racks are projected to reach 5,000 pounds compared to the 400 to 600 pounds that racks weighed three decades ago. The dramatic increase stems from hundreds to 1,000 GPUs packed densely into each rack alongside memory chips and liquid cooling systems that can add substantial weight. AI workloads now consume up to 350 kilowatts per rack, 35 times the 10 kilowatts that traditional computer chip workloads averaged a decade ago. Legacy data centers with raised floors typically max out at around 1,250 pounds per square foot for static loads.
Chris McLean, president of Critical Facility Group, said that rack heights have grown from 6 feet to 9 feet over nearly two decades, creating problems with doorframes and freight elevators in older buildings.
AI racks are projected to reach 5,000 pounds compared to the 400 to 600 pounds that racks weighed three decades ago. The dramatic increase stems from hundreds to 1,000 GPUs packed densely into each rack alongside memory chips and liquid cooling systems that can add substantial weight. AI workloads now consume up to 350 kilowatts per rack, 35 times the 10 kilowatts that traditional computer chip workloads averaged a decade ago. Legacy data centers with raised floors typically max out at around 1,250 pounds per square foot for static loads.
Chris McLean, president of Critical Facility Group, said that rack heights have grown from 6 feet to 9 feet over nearly two decades, creating problems with doorframes and freight elevators in older buildings.
Yes, New Buildings (Score:5, Informative)
New buildings for AI datacenters are clearly the plan. The weight per rack and the rack height are minor considerations compared to the electricity and cooling requirements. This is obviously the approach Tesla is taking, having custom-built the Cortex datacenter, and now with construction well underway for Cortex 2.
With 350kW/rack, a large datacenter is looking at hundreds of MW. No wonder Tesla had to build a new substation for Cortex 2. I haven't looked at what the other companies are doing, but it can't be that different.
Re: (Score:3, Insightful)
No wonder Tesla had to build a new substation for Cortex 2
They should be building entire power plants on-site. If they can't build a power plant at a given site, then build the whole thing, power plant and battery storage included, somewhere else.
An alternative to on-site power is a power plant that's close by (within tens of miles) then build dedicated power-transmission lines from the power plant to the data center.
Both solutions avoid the "we are going to bust the electrical grid with our power demands" problem.
Re: (Score:1)
Elon Musk, is that you?
Re: (Score:2)
Re: (Score:3)
I saw a video complaining they did just that. The problem was that the power plant was a bunch of mobile gas-powered generators and it played havoc with people living nearby, noise and air pollution.
If they'd declared it up front, it'd have been less of an issue, because they'd have been forced to build a little further from the city and probably given clean air mandates to adhere to. But Musk is cheap and figured out he could bypass that by pretending it was just a data center, and then bringing in mobile
To save the environment we have to ruin it? (Score:1)
To save the environment we have to ruin it? At what point during building, construction, and production does the cost of it all eclipse the savings of ICE over EV? For a small savings in carbon (if any when including the significant environmental toll of the battery) is it worth building entire datacenters, no wait, entire power plants just to power AI to power the thinking behind this eco saver car?
Re: (Score:2)
To save the environment we have to ruin it? At what point during building, construction, and production does the cost of it all eclipse the savings of ICE over EV? For a small savings in carbon (if any when including the significant environmental toll of the battery) is it worth building entire datacenters, no wait, entire power plants just to power AI to power the thinking behind this eco saver car?
What does AI have to do with "this eco saver car" - you are crossing topics. This topic has to do with powering AI datacenters, not electric cars.
Re: (Score:3)
No wonder Tesla had to build a new substation for Cortex 2.
Err datacentres have literally always needed their own dedicated substations. The bar required for building substations is actually ludicrously low. Heck some commercial buildings without datacentres need their own substations. You pulling more than 1-2MW then having a dedicated substation is almost a necessity, unless you're feeding that power into a single load at the originally supplied voltage.
That's not to diminish your astonishment. No doubt Cortex 2 will draw a shitton of power, but the word substati
And just think... (Score:3, Funny)
If they are able to achieve their goals with the AI, then in another seven or eight years they'll finally have figured out how to optimize personalized mattress sales.
ANGH (Score:2)
Absolutely not gonna' happen in reality since the people with the actual money are now realizing they're not going to get a return on what they thought was an investment.
Re: (Score:2)
Question: How does my mattress company reach 100% saturation.
AI Answer: Kill all humans. Target population: 0%. 100% Saturation achieved.
... (some screams later) ...
Police Transcript: Perp: "The AI told me to do it! It's a government service so this is entrapment. You can't arrest me!"
Police Transcript: Agent 1: "Dang, our Police Rulebook Chatbot says he's right. Let him go."
just ask (Score:1)
I bet if they just ask ChatGPT it will have the answer.
Re: (Score:1)
If you run computers underground they don't make heat? What if they are below sea level? Datacenters underground in Death Valley FTW!
Re: Are there engineers working there? (Score:5, Insightful)
Re: (Score:2)
What are you thinking? You can't just magic the heat away by building underground
The dramatic increase stems from hundreds to 1,000 GPUs packed densely into each rack alongside memory chips and liquid cooling systems..
Worked in data centers buried inside 2,000 feet of solid granite for a while (NORAD). Is there some new challenge maintaining liquid cooled systems underground?
Re: (Score:1, Funny)
Random internet commenter incorrectly assumes that they've discovered a silver bullet in their five-minute analysis that the experts have simply missed.
Must be a day ending in Y.
Re: (Score:2)
Underground? You'll have to pump air in there or your staff will die.
And you'll need to send the waste heat somewhere, maybe into the walls but the square-cube law indicates that this is not scalable. A really big data center will need to pump waste heat out through a medium, liquid or gas. Water is the obvious choice. So your underground data center ought to have a hyperboloid cooling tower sitting above it. Seems kind of pointless to have dug that big hole after all that.
Re: (Score:2)
Underground? You'll have to pump air in there or your staff will die.
And you'll need to send the waste heat somewhere, maybe into the walls but the square-cube law indicates that this is not scalable. A really big data center will need to pump waste heat out through a medium, liquid or gas. Water is the obvious choice. So your underground data center ought to have a hyperboloid cooling tower sitting above it. Seems kind of pointless to have dug that big hole after all that.
As data center requirements become more and more extreme, I imagine human limitations being supplemented or eliminated altogether (robotics).
In other words, that data center ain’t built for you, meatsack. Get your oxygen tank and mask on, and go swap that failed hard drive already.
Re: (Score:3)
I've never seen a more Dunning-Kruger post in my entire history on Slashdot, than what you just wrote there. You've in 4 sentences called engineers incompetent while demonstrating you don't know how cooling works, and proposed a solution that violates the basic physics behind conservation of energy.
Additionally you claim engineers forgot about weight when they literally didn't, and this article points out that weight very much has always been a fundamental consideration.
Please go home and sober up before po
Re: (Score:2)
How does putting it underground mean less heat? It gets hotter the lower you go, and on top of that the problem is heat generated by the servers themselves - where's it supposed to go in a room that's that well insulated?
If you'd had suggested building them atop mountains, you might have been on to something, but those places are typically hard to reach and thus construct upon, and you'd also still need to power them and, of course, connect them to the Internet.
Sounds like we need new data centers (Score:2)
Re: (Score:2)
One could use heat exchangers such that the cooling water and the sea water never intermix.
Re: (Score:2)
What would you build the heat exchangers out of? Materials that don't corrode (like plastic) tend to not be very good conductors of heat.
Re: (Score:2)
We build a fair number of metal things that spend their lives in sea water. Most of them have heat exchangers too. [wikipedia.org]
Re: (Score:2)
Microsoft did that. It apparently worked okay. They've ended the experiments and haven't decided to deploy it in production.
Minor problems, from what I'm reading. (Score:5, Insightful)
AI racks are projected to reach 5,000 pounds ...
... spread over probably 24" x 48", or 8 square feet, for a total of 625 pounds per square foot.
Legacy data centers with raised floors typically max out at around 1,250 pounds per square foot for static loads.
Ignoring that the numbers above are probably less than half the static load limit, the static load limit is as low as it is because A. the raised floor has weight, and B. the raised floor likely has a much lower weight limit than the concrete slab under it.
Solution: Remove the raised floor.
... rack heights have grown from 6 feet to 9 feet over nearly two decades ...
Which means you can now run overhead cable trays above the height of your tallest employees. No need for the raised floors. Also, by ripping out the raised floors, you can have that extra height, so no need to rebuild the building.
... creating problems with doorframes and freight elevators in older buildings ...
Because people can't tip the racks up to get them through the doors, or lie them flat corner-to-corner in a freight elevator? This seems like a problem with movers not being creative enough when moving things, rather than a building problem.
This whole article reads like people looking for an excuse to spend more money and build monuments to themselves, rather than an actual problem. What am I missing?
The AI business is a grift (Score:3)
Consider it an investment in UFO science.
Re: (Score:2)
Re: (Score:1)
... spread over probably 24" x 48", or 8 square feet, for a total of 625 pounds per square foot.
Let's stick to units everyone can understand. How many hogsheads of nacho cheese per American football field? *
* 0.00434629, assuming the density of nacho cheese is 1.057
This got me thinking (Score:3)
All those GPU chips, all those "AI factories"
I wonder what the invoice price for a 53' trailer of these GPU chips would be? Almost worth....a Netflix movie idea.
Missed Opportunity (Score:4, Funny)
Chris McLean, president of Critical Facility Group, said that rack heights have grown from 6 feet to 9 feet over nearly two decades, creating problems with doorframes and freight elevators in older buildings.
Missed a perfect opportunity for a better headline: "The Rack is Too...Damn...High" [ref [youtube.com]]
Damn... (Score:5, Funny)
... and I thought the VAX6000 was a heavy power hog.
Re: (Score:2)
Too bad that heat dispensation in space is major issue although it is cold as hell.
Height Requirements (Score:3)
Chris McLean, president of Critical Facility Group, said that rack heights have grown from 6 feet to 9 feet over nearly two decades.
Do those come with a library rolling ladder mounted to the cable chase, or am I to assume I get a 6-month subscription to Shaquille O’Neal to maintain those servers waaaaay the hell up there?
Gonna have NBA height requirements and a vertigo test in the data center soon.
”OK, here’s the new hard drive we need to swa..dude, why the hell are you wearing protective fall gear?”
Too heavy to fail? (Score:2)
NExt investment boom will be in data center building?
Re: (Score:2)
Not paying any attention, are you? Datacenter buildouts are 90% of the current activity. Excluding that, the US GDP rose... I forget if it was 0.1% or 0.01%.
Oh, and there's two datacenters in Nvidia's home county, completed.. just sitting there because of lack of electricity.
Density is expensive (Score:2)
Probably More Cooling than Weight (Score:2)