OpenAI Sam Altman Says the Company Is 'Out of GPUs' (techcrunch.com) 53
An anonymous reader quotes a report from TechCrunch: OpenAI CEO Sam Altman said that the company was forced to stagger the rollout of its newest model, GPT-4.5, because OpenAI is "out of GPUs." In a post on X, Altman said that GPT-4.5, which he described as "giant" and "expensive," will require "tens of thousands" more GPUs before additional ChatGPT users can gain access. GPT-4.5 will come first to subscribers to ChatGPT Pro starting Thursday, followed by ChatGPT Plus customers next week.
Perhaps in part due to its enormous size, GPT-4.5 is wildly expensive. OpenAI is charging $75 per million tokens (~750,000 words) fed into the model and $150 per million tokens generated by the model. That's 30x the input cost and 15x the output cost of OpenAI's workhorse GPT-4o model. "We've been growing a lot and are out of GPUs," Altman wrote. "We will add tens of thousands of GPUs next week and roll it out to the Plus tier then [] This isn't how we want to operate, but it's hard to perfectly predict growth surges that lead to GPU shortages."
Perhaps in part due to its enormous size, GPT-4.5 is wildly expensive. OpenAI is charging $75 per million tokens (~750,000 words) fed into the model and $150 per million tokens generated by the model. That's 30x the input cost and 15x the output cost of OpenAI's workhorse GPT-4o model. "We've been growing a lot and are out of GPUs," Altman wrote. "We will add tens of thousands of GPUs next week and roll it out to the Plus tier then [] This isn't how we want to operate, but it's hard to perfectly predict growth surges that lead to GPU shortages."
Why?!? (Score:2)
What a pointless, self-serving statement.
Everyone in this room is now dumber for having listened to it(you). I award you no points, and may God have mercy on your soul.
Re: (Score:3)
"Thou shalt not make a machine in the likeness of the human mind." -- Rayna Butler, a survivor of the Butlerian Jihad
(from the Dune prequel "The battle of Corrin")
Re: (Score:2)
I never understood this in that universe. Like, what is that even supposed to mean, "likeness"? Were early computers like the human mind? I mean, in the movies and in the recent Prophecy series, it's pretty clear that there are computer-like objects around and machines that would require computers to work.
Spoilers (Score:2)
Spoilers incoming!
Yes. The Old Empire had thinking machine servants. Robots with AGI circuitry. Androids, though they were not called that. And, of course, somebody decided to weaponize them. And, of course, they turned on their masters and became the evil robot overlords of the galaxy. They enslaved the humans and kept them alive largely as a quirk of their programming, but treated them very badly and wantonly experimented on them and killed them whenever it was convenient.
The evil robot overlords ru
Re: (Score:2)
So Dune had humans doing all the data analyses and making all decisions, with machines at lea
Re: (Score:2)
These books are horrible. Kevin J. Anderson has been infamous for his shitty writing even before them, and apparently Brian Herbert is no better.
Re: Why?!? (Score:2)
Because the only thing the "AI revolution" has going for it is a blind belief in brute force.
Re:Screw you Sam! (Score:4, Insightful)
"that pesky supply and demand"
What if this is actually just an excuse? What if the scarcity they want you to focus on is temporary, and the decision to raise prices is greed not necessity? When he says he's out of GPUs does he mean they're in boxes not in the computer, and then NVIDIA uses that narrative in your mind, that there is more demand than supply, to raise prices, and the middlemen raise prices, all because of an assumption of scarcity that doesn't really have to exist?
Remember when we paid by the text message, because the resources were scarce, but really the Altmans just wanted to sell us on a scarcity of physical resources story, trying to (and succeeding in) hooking us all on the idea that the texts had to come from somewhere and that was very expensive to them, when in fact they had the memory and power to give us today's unlimited plans from the start?
When you mouth the phrase "that pesky supply and demand", how much are you serving the interests of those who know supply and demand is not how they set your prices, but they will cynically use it as an excuse and make you a useful idiot?
Nvidia shortage (Score:1)
Re:Nvidia shortage (Score:5, Insightful)
datacenter GPUs (almost 100% for AI purposes) are 78% of their revenue, from 40% in 2022, and and ~22% in 2019.
That trend isn't going to be changing any time soon.
Bet he wishes there was... (Score:2)
Fuck you Altman.
PR CRAP (Score:1)
Sam Altman is a moneyraiser. This isn't an "alert" about a GPU shortage. It's a call for more investor money.
I don't know him personally but from what I read there's a reason nobody wants to work for him.
One day when there's a real AI someone will ask its opinion of Sam Altman, and the AI will laugh and laugh and laugh.
As should we.
Re:PR CRAP (Score:4, Insightful)
Bingo. This reeks of excuses for the poor performance of their newest "model".
Re: (Score:2)
You know DeepSeek must be something if the US wants to jail people for simply downloading it. https://www.fox29.com/news/dee... [fox29.com]
Re: (Score:2)
You know DeepSeek must be something
Indeed, and that something is "Associated with the bogeyman du jure."
DeepSeek R1's CoT is pretty fucking cool... but only really because it has open weights. It doesn't perform particularly better than the other CoT models.
MoE does let it crunch less numbers to get answers, but that's not new either.
Re: (Score:2)
They didn't invent much new architecture-wise. They trained a good model and they engineered quite a few optimization techniques to do so. No magic involved. I bet we will soon see better models, but currently R1 is ahead of ChatGPT.
Re: (Score:2)
but currently R1 is ahead of ChatGPT.
Not really.
Its performance varies from "a good bit behind o1, to matching it, to microscopically ahead of it".
The real trick is the computational cost- the MoE model only needs to crunch like 60B something parameters, where o1 needs to crunch like 175B parameters.
Means R1 is considerably cheaper, computationally.
Pricing wise, DeepSeek charges inference at like 1/10th the cost OpenAI does o1, so competitively, it's a no brainer.
Re: (Score:2)
I didn't look at benchmarks, it is just my experience with the results, mostly for technical questions (IT, math, science related things) that deepseek usually gets what OpenAI misses. It is also nice to see the thinking output (and also use results from there that are not in the final answer) and I am looking forward to a European service hosting it.
Re: (Score:2)
Also, I agree that is sucks that they hide the CoT tokens from you.
It's pretty neat to watch R1 reason.
Re: PR CRAP (Score:3)
Re: PR CRAP (Score:2)
This. Altman is one of those Silicon Valley golden boys: smooth-talking, backstabbing sociopaths.
In the case of OpenAI, as soon as the potential financial success was on the horizon, he started wriggling out of their commitment to "openness" and "not for profit".
Now, the Chinese have matched OpenAI with a far less expensive model, and other competitors are not far behind. OpenAI may genuinely have problems going forward. If do, look for Sam to take some sort of gold-plated exit.
openAI was the best - now it's not (Score:1)
It seems like their AI doesn't work. (Score:3)
"This isn't how we want to operate, but it's hard to perfectly predict growth surges that lead to GPU shortages."
-says CEO of AI company.
If your AI is so great, shouldn't it be able to look at a trend and make a prediction about the future needs of your business?
Re: It seems like their AI doesn't work. (Score:2)
No, you see for supply and demand trends you have to be *accurate*.
That is not a requirement for AI today.
Re: (Score:2)
Nor obviously for Sam Altman, right?
Re: It seems like their AI doesn't work. (Score:2)
I know this might sound confusing with an eighth grade math education, but something that is hard to predict doesn't follow a predictable trend.
If I plot three points roughly in a line you could predict a fourth right? If I plot three points in the shape of a triangle, could you predict the next one or would you rather have free time with the crayons.
Re: (Score:2)
If supply and demand are noise, aren't prices too?
why noise? (Score:2)
Mo Money (Score:2)
I had to chuckle, it's the American solution: throw more money (aka GPUs) at it. Wait, wasn't it "work smarter, not harder"?
Seems like the "AI" "industry" is ripe for disruption. Anyone?
Re: (Score:2)
Really the models need to get smart enough to decide which model to use.
Google wanted to get AI out there so they do it with almost every search, but that isn't affordable so it gives such horrible results they're just tarnishing their AI image in my opinion. Use it when it's called for, but then get the job done.
Re: (Score:2)
Do you think you can provide me with one measly example of this huge superior AI you have access to by paying a lot of money or working for an employer that pays a lot of money, because I'm just a plebe who learned from Richard Stallman not to pay for software so I just use the free AIs and they help me out a lot? What am I missing by not buying the latest and greatest? Is it just prestige and stories about how great it is, without actually providing examples, just winking and nodding to others who pay for
Re: (Score:2)
The reasoning / deliberative models that take a long time, I have tried somewhat and have found multistep tasks or complex formatting problems on which they do better. But you know what? I don't think people really 'reason' all that often to live, and I normally don't need AI to do much
Re: (Score:2)
AI == Actually Insolvent (Score:2)
Re: (Score:2)
How many times has Trump declared bankruptcy again? Do you think the president knows that liquidity kills you quick but solvency doesn't matter except to use cynically as a political tool to press the buttons of people like yourself so you'll follow his agenda while he knows he can spend as he wants on whatever he wants and use the "there's no money left!" excuse to fire whomever he wants?
Perhaps scaling has run its course (Score:2)
And altman is now tilting at windmills.
Thanks for taking all the Gaming GPUs, Asshole! (Score:2)
Re: (Score:2)
Can you use AI to build your own?
What do the tough do when there are shortages? (Score:2)
Did China build DeepSeek when the US banned them from the best chips?
What they're out of is moat. (Score:2)
DeepSeek proved the "moat with alligators" around cutting edge AI is actually more like a koi pond. The time from release to cloning is three to six months. Given this new reality, OpenAI is going to charge through the nose knowing that they'll be undercut sooner rather than later. Maybe restricting the rollout will also slow down the cloning, though somehow I doubt it. They might be telling the truth about limited capacity because refusing to shut up and take our money in these critical months seems stupid
Finally (Score:2)
Lot of money (Score:2)
I still believe the costs of this crap will eternally outweigh any potential future benefit.
Good, because our grid is out of electrons. (Score:2)
I see why corporations are dropping the DEI programs, specifically AI goes exactly counter to saving the Environment.
Re: (Score:1)
We are out of electrons to waste on 'AI'. Such a a waste of electricity, and hardware.
I see why corporations are dropping the DEI programs, specifically AI goes exactly counter to saving the Environment./quote
Huh?
what does Diversity, equity, and inclusion (DEI) have to do with AI and the environment?
Brute Force Artificial Stupidity (Score:3)
Someone, at some point, involved in this ridiculous "MORE MORE MORE" game that the current gen AI shit is stuck in needs to ask the question, "Why?" Why are we creating these entities that suck down power like nothing we've ever created before and require hardware stacks that make the supercomputers of yesteryear look quaint? Can they do some impressive, if questionable, things? Sometimes. But it's not like there's anything resembling a hard fact checker coming out of these behemoth machines that's anywhere near accurate. And those of us that code for a living, if we know what we're doing, find them to be questionable at best at code. They're helpful for newcomers as training wheels, sure. And that's not nothing I suppose, but I'm not sure it's worth creating yet more energy demand and sucking up all available advanced hardware for incremental, diminishing returns.
I know any mention of regulation instantly gets rejected as, "But someone else may beat us to it!" But I have to ask the question: "Beat us to what?" An energy crises? A resource depletion? An non-hire-able population when we "win" the AI race and create the perfect machine based worker bee? A collapsing infrastructure when the entire economy grinds to a halt as people lose spending power because jobs dry up when the AI finally gets good enough? What's the end-goal with this shit? What are all these resources giving us? More dystopia? I think we've been doing fine building that future for ourselves without AI to help speed it along.
What are we getting for all this obsession with AI? Someone explain it to me in clear cost/benefit fashion. I'd really be curious if someone that's all-aboard the AI train can tell me what the end-goal actually is. Because if it's just about amassing data and wealth at every increasing rates as it appears to be today, I don't know that that's gonna work out well for humanity on the whole.
10k GPUs? (Score:2)
I wonder if the carbon footprint of AI is smaller than it's meatspace equivalent of actual human brains. I can't help but wonder if it would be more cost effective to simply hire teams of grad students to answer questions. At least those aren't in short supply.
Good thing (Score:2)
Fortunately for them, DeepSeek has open-sourced five optimisation techniques they used to train R1 this week. They know how to train models with fewer GPUs.
Newsflash, AI does not scale (Score:2)