AI Developers Stymied by Server Shortage at AWS, Microsoft, Google (theinformation.com) 24
Startups and other companies trying to capitalize on the artificial intelligence boom sparked by OpenAI are running into a problem: They can't find enough specialized computers to make their own AI software. The Information: A spike in demand for server chips that can train and run machine-learning software has caused a shortage, prompting major cloud-server providers including Amazon Web Services, Microsoft, Google and Oracle to limit their availability for customers, according to interviews with the cloud companies and their customers. Some customers have reported monthslong wait times to rent the hardware. "All the startups who are trying to get into this space...maybe they can get one [server] but there's no way they're going to get five," said Johnny Dallas, founder and CEO of Zeet, which sells software that makes it easier for engineers to run apps across multiple clouds.
The server chip shortage is a frustrating hangup for software developers trying to build AI tools hinging on recent advancements in machine-learning models. These programmers, at small and big companies alike, are developing large-language models to make personalized writing coaches or search engines that respond to questions with written answers rather than links, similar to OpenAI's ChatGPT. Many others are licensing and augmenting software from OpenAI and its rivals to create specialized customer service chatbots and research tools for corporate employees. For instance, OpenAI software is helping Morgan Stanley bankers find the best locations to auction a work of art, based on the bank's myriad internal reports on art markets.
The server chip shortage is a frustrating hangup for software developers trying to build AI tools hinging on recent advancements in machine-learning models. These programmers, at small and big companies alike, are developing large-language models to make personalized writing coaches or search engines that respond to questions with written answers rather than links, similar to OpenAI's ChatGPT. Many others are licensing and augmenting software from OpenAI and its rivals to create specialized customer service chatbots and research tools for corporate employees. For instance, OpenAI software is helping Morgan Stanley bankers find the best locations to auction a work of art, based on the bank's myriad internal reports on art markets.
There is probably money to be made here... (Score:3)
Some scalper is probably buying up this specialized AI hardware and selling it on eBay at a 200% markup.
Of course, I have no REAL way of knowing that because Slashdot posted yet another article behind a freaking paywall. Again. Why don't the editors bother reading these things before posting them?!?
Re:Why don't the editors bother reading... (Score:2)
They do not work for an entity that cares, and apparently there is no human above them to engage.
Slashdot NOT sucking could make far more profit. If the owners were aware of that they'd shitcan current staff.
Re: (Score:3)
Rich scalpers... The A100 GPU (what's in high demand right now) is $10,000 or more.
Or maybe they could use human intelligence (Score:3)
Re: (Score:1)
The people being laid off weren't producing much of use.
iTulip (Score:4, Insightful)
There's an AI craze... right as the demand for mining fell of a cliff.
Re: (Score:3)
renting? (Score:2)
I know it's all the craze to spin up servers in AWS and the like, but if a company is going to d a lot of this work, then go buy a server and then you can use it as much as you with with only a little charge for electricity.
Re:renting? (Score:5, Insightful)
...then go buy a server and then you can use it as much as you [want] with only a little charge for electricity.
I think there's a hidden, but predicted, problem on the horizon. I think cloud providers are colluding to raise prices on locked-in customers in the near future. A shortage, whether real or manufactured, is a common excuse for doing so. If they can all profess a common resource shortage, then there is nowhere for committed customers to turn. They are all equally screwed.
I agree: buy your own servers. You'll be better off.
Re: (Score:2)
To have enough hardware to run something like ChatGPT, you need most of a dozen $10,000 A100 GPUs to run it. To train an equivalent model in a reasonable amount of time would take 100s or 1000s of A100 units. For training the model especially, I'd rather rent it.
Re: (Score:2)
Then you have little choice but to invest in your own hardware, which you probably should have done to begin with if you're an artificial intelligence startup.
Paywalled (Score:2)
Article is paywalled and more of the text is available in the summary than the actual article. My question is is the demand for training models, or for processing requests in an existing model?
Patience, nerds... (Score:2)
need to take out WOPR befor it can start WW3 (Score:2)
need to take out WOPR befor it can start WW3
Re: (Score:2)
Cyberdyne was making robots. Very very few companies are making robots (especially for manufacturing), which would actually be the disruptive tech.
Well yeah... (Score:5, Interesting)
Of course - Google, Microsoft and Amazon can make far more money using that compute power than renting it to someone else who wants to compete in that space.
Re: (Score:2)
Maybe, maybe not. During a boom, providing infrastructure may be more profitable than providing the product itself. Imagine 100 companies all trying to make AI products, all needing AI compute infrastructure. If 90 of those companies fail, the infrastructure companies still profited. And the 10 successful companies will probably take over that infrastructure. It's like selling arms during a war: you profit no matter who wins (or even if nobody wins).
You will own nothing and be happy (Score:2)
Doesn't sound so good when businesses buy into that scornworthy drivel.
Ownership of objects is control. Freedom is unusable without property,
An interesting problem to have. (Score:3)
I can recall a very different problem in computing, where people would find it difficult to justify large investments in new computers because there was "always" another faster computer coming soon for less money that did more. An analogy comes out of space exploration, a problem seen in the real world and extrapolated out in science fiction. If planning a satellite launch then it is in the benefit of the people operating the satellites to delay the launch as long as possible since costs of the launch keeps coming down. Taken to extremes in science fiction is someone landing on some distant planet only to find people that left later, on a more advanced ship, got there before them. Nobody wants to make an investment in something that takes them on the "slow path" if waiting for something later gets them where they are going sooner.
If people believe that something better comes along so quickly that they'd be losing money or wasting time then we get into the problem of there being limited funds in development of new products because old products aren't selling well, the companies need income from old products to fund development of new products. There's classic economic lessons from history showing how an announcement of a newer faster computer can sink a company because that announcement killed demand for everything they have in stock, and if they can't sell anything then they face bankruptcy.
If there's a demand for computers today to the point supply can't keep up then it seems to me that there's not the same kind of expectation on newer computers being significantly less expensive and more powerful. That may be the kind of income and competition needed to drive development of newer computers that cost less and are more powerful. The difference today from what happened before is we are hitting physical limits of the technology we have. To go faster means only getting bigger, and therefore more expensive. Some new technology that isn't limited in ways the silicon technology we have today could change things. If or when that happens then we can expect to fall into the same problems of people holding off new purchases as long as possible so as to get the best return on their investment.
Re: (Score:2)
However, AI probably has a lot of room for improvement. There's been an all-out race to see how far we can get just by going bigger, and the results have been astonishing. But we don't really know that we need models this big to get most of these results. Also, some fundamentally different computing paradigm (such as analog c
A use for Bitcoining hardware now unwanted? (Score:4, Interesting)
This is extraordinary news; the whole point of trusting the cloud is the assumption that there would always be excess capacity. Now even that rationale is being removed. Certainly convenient for the tech firms; they'll be able to raise price a lot, of course. How long till the cloud is regulated as a utility?