Google Must Double AI Serving Capacity Every 6 Months To Meet Demand 57
Google's AI infrastructure chief told employees the company must double its AI serving capacity every six months in order to meet demand. In a presentation earlier this month, Amin Vahdat, a vice president at Google Cloud, gave a presentation titled "AI Infrastructure." It included a slide on "AI compute demand" that said: "Now we must double every 6 months.... the next 1000x in 4-5 years." CNBC reports: The presentation was delivered a week after Alphabet reported better-than-expected third-quarter results and raised its capital expenditures forecast for the second time this year, to a range of $91 billion to $93 billion, followed by a "significant increase" in 2026. Hyperscaler peers Microsoft, Amazon and Meta also boosted their capex guidance, and the four companies now expect to collectively spend more than $380 billion this year.
Google's "job is of course to build this infrastructure but it's not to outspend the competition, necessarily," Vahdat said. "We're going to spend a lot," he said, adding that the real goal is to provide infrastructure that is far "more reliable, more performant and more scalable than what's available anywhere else." In addition to infrastructure build-outs, Vahdat said Google bolsters capacity with more efficient models and through its custom silicon. Last week, Google announced the public launch of its seventh generation Tensor Processing Unit called Ironwood, which the company says is nearly 30 times more power efficient than its first Cloud TPU from 2018.
Vahdat said the company has a big advantage with DeepMind, which has research on what AI models can look like in future years. Google needs to "be able to deliver 1,000 times more capability, compute, storage networking for essentially the same cost and increasingly, the same power, the same energy level," Vahdat said. "It won't be easy but through collaboration and co-design, we're going to get there."
Google's "job is of course to build this infrastructure but it's not to outspend the competition, necessarily," Vahdat said. "We're going to spend a lot," he said, adding that the real goal is to provide infrastructure that is far "more reliable, more performant and more scalable than what's available anywhere else." In addition to infrastructure build-outs, Vahdat said Google bolsters capacity with more efficient models and through its custom silicon. Last week, Google announced the public launch of its seventh generation Tensor Processing Unit called Ironwood, which the company says is nearly 30 times more power efficient than its first Cloud TPU from 2018.
Vahdat said the company has a big advantage with DeepMind, which has research on what AI models can look like in future years. Google needs to "be able to deliver 1,000 times more capability, compute, storage networking for essentially the same cost and increasingly, the same power, the same energy level," Vahdat said. "It won't be easy but through collaboration and co-design, we're going to get there."
Re: (Score:2)
My only demand for AI is "please stop" (Score:2)
And my AI strategy is still "No."
Re: (Score:3)
And my AI strategy is still "No."
Computers used to be the size of a room and now they fit in a watch. It’s okay to be scared of the new magic rock that thinks, but holding onto your slide rule won't stop it.
Re:My only demand for AI is "please stop" (Score:5, Insightful)
I am tired of asking questions to AI and getting useless results.
Re: (Score:2)
At first, I figured it was pure cope. Now I'm wondering if it has progressed to honest delusion.
Re: (Score:2)
But, just because some 20-something is using LLM-AI (and paying for it) doesn't mean that I'm gonna use the thing.
The reason so many are using this crap and paying for it, is because of the ease of use... you don't have to read a whole webpage to find the directions on how to replace a turn signal bulb (is that even something you replace anymore?)... the LLM-AI will dumb it down and hold your hand as it walks you through the whole complicated process.
But, I'm not gonna use it unless I come across some usage
Re: (Score:2)
The older generations will probably always have limited adoption. The younger generations will probably continue to expand adoption. For better or worse (worse, probably)
Re: (Score:2)
Right, but should we devote so many resources to the implementation of LLM-AI into everyone's daily lives?
Cramming it into factories to replace human workers?
Helping kids cheat on essays and tests?
Working as fake friends for kids?
I don't need my car to steal control from me to avoid a pothole or a big chunk of snow... I can do that on my own, thank you very much.
My kids can talk to and hang out with other kids... my kids wouldn't have $800 iPhones or $500 game consoles... they can go out and bike and stuff.
Re: (Score:3)
People thinking of this stuff as a "magic rock that thinks" is exactly the problem. And the main people conceiving of it that way are the ones trying to jam it down everyone's throat.
Re: (Score:3)
Oh buzz off (Score:2)
I've been doing AI since the 90s. This current bubble is just a weird blend of Theranos and Enron and it's a shame because it's going to kill off the actually useful parts of AI when the scammy parts go away.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
And my AI strategy is still "No."
Computers used to be the size of a room and now they fit in a watch. It’s okay to be scared of the new magic rock that thinks, but holding onto your slide rule won't stop it.
My slide rule doesn’t require a nuclear reactor re-deployment abusing the unaccountable power of girl math to fuel shitcoin mining and help drive a human-employed planet into premature poverty and death ushering in the next Depression, because stock price.
Greta will try and label me evil because my slide rule is not made out of..oh wait, it IS made out of sustainable bamboo. Meanwhile teachers are quitting their profession because college bound students armed with a bubble-wrapped high school diploma
Re: (Score:2)
So, LLM-AI is capable of thinking individually and freely? Wow! How'd that not make the news here? It doesn't just consult a database full of words with percentages and reference numbers attached to the words?
They really do need to work on efficiency (Score:1)
Because at this rate, by my math, the number of AI cores Google requires will exceed the number of atoms in the visible universe within about 120 years.
Re: (Score:2)
Re: (Score:2)
I keep hearing about this overpopulation- I had no idea it had gotten that bad.
Just use a shittier model (Score:2)
Re: (Score:2)
Well, the performance of larger models is better, so there's a demand for larger models.
That's kind of orthogonal about whether or not there should be a demand.
Are they making a profit yet??? (Score:5, Insightful)
It's doesn't sound like a successful business venture if you're having to increase operation expenses at this rate and not be raking in the revenue.
Re:Are they making a profit yet??? (Score:5, Insightful)
It's doesn't sound like a successful business venture if you're having to increase operation expenses at this rate and not be raking in the revenue.
Yes, Google is profitable now. Tremendously so. But they're at risk of losing revenue and ceasing to be profitable as people cease using Google search and switch to asking questions of their AIs. So to retain their position as the place people go first for information, they have to stay ahead of the AI race. Well, they could also just sit back and wait to see if their competitors are overwhelmed by the query volume, but that risks losing traffic and then having to win it back. It's much better to keep it. And Google is better-positioned to win this race than its competitors both because of its existing infrastructure and expertise and because it already has the eyeballs.
In addition, you seem to be assuming that doubling serving capacity means doubling cost. Clearly Google is not planning to increase their annual operating expenses by 1000X. As the summary actually says in the third paragraph, Google is also going to have to improve efficiency to achieve the growth rate, with better models and better hardware. This is what the AI chief is challenging the employees to do; he's not challenging them to write bigger OPEX checks, that's his job.
Re: (Score:2)
Yes, Google is profitable now.
I'm going to assume you're misunderstanding my question. I'm only referring to the AI business.
The AI business is not succeeding if it needs to be amalgamated on a balance sheet with other ventures to hide that it's bleeding money.
Re: (Score:2)
The AI business is not succeeding if it needs to be amalgamated on a balance sheet with other ventures to hide that it's bleeding money.
That's not necessarily true in the slightest.
If it is, they won't be able to hide that- they're a publicly traded company. And lying about it is literally a federal crime.
More likely, they're talking about the 650 million MAU that Gemini has.
They claim that the AI search results only take 10x the power as a regular search result, which means the model that produces those is absolutely microscopically tiny.
That would make sense if its mostly just fed a context of search results and snippets and asked t
Re: (Score:2)
That's not necessarily true in the slightest.
If it is, they won't be able to hide that- they're a publicly traded company. And lying about it is literally a federal crime.
It's not lying about it. There's no requirement a company line-item every single product on earnings. Apple has been doing it for years in their reporting, that's why none of the rumors sites can give hard numbers on how badly the iPhone Mini flopped (or the iPhone Air is flopping now) -- Apple has them reported mixed with the more successful models. Their "services" category in earnings includes AppleCare, iCloud, Apple Music subscriptions, App Store fees, and even the Google Search licensing payment. Here [apple.com]
Re: (Score:2)
It's not lying about it.
They would have to in order to hide it.
There's no requirement a company line-item every single product on earnings.
That is correct. That is, however, not support for your position.
You can shuffle it around wherever you like- the loss is going to show up somewhere. If P suddenly drops for "Search", then you know it's there.
Re: (Score:2)
The question becomes what are LMMs good at? Open public uses seem quite limited because of misinformation corrupting the data sources. Search is one effective use.
They are being used in closed environments like for GPs where the data input is clean and note-taking, transcribing, form-filing, summarising are all time savers.
I can't see trillions of dollars in LLMs. The spend is insane. Google might be the only sane player on the field.
Re: (Score:3)
Yes, Google is profitable now.
I'm going to assume you're misunderstanding my question. I'm only referring to the AI business. The AI business is not succeeding if it needs to be amalgamated on a balance sheet with other ventures to hide that it's bleeding money.
The AI business is also the search business.
Re: (Score:2)
You're still deflecting from the actual question. Is the AI business, when measured alone regarding its costs and revenue, profitable? Whether it's considered a "Search Business" or not is inconsequential. Saying we have to look at it combined with other Google search entities is literally the issue I'm warning about.
Google can grow their "search business" revenue without having to steam-shoveling resources
Re: (Score:3)
Google can grow their "search business" revenue without having to steam-shoveling resources into an AI furnace at an exponential rate.
No, they can't. They're already seeing searches decline as people move to asking LLMs instead. The LLMs actually end up using Google, but that doesn't generate any ad revenue for Google (that's something Google may have to figure out how to put a stop to). The solution so far is the addition of "AI results" to search output... but those AI results are produced by a far inferior model because Google search gets hundreds of thousands of queries per second, and it's currently infeasible to service those que
Re: (Score:2)
No, they can't. They're already seeing searches decline as people move to asking LLMs instead.
Google Search has been declining for a very long time. It's not LLM's driving people away from Google Search, it's the experience itself. People have been lamenting long before AI about how much less useful Google Search has become over the years. First it was because of companies using SEO to gamify the whole thing and get their commercial interests listed over actual helpful info. Then it was Google pushing more into advertising so you had to search inside the search to find the relevant results that were
Re: (Score:2)
Google Search has been declining for a very long time.
This is a common perception among techies, but is not true. Google search usage and profitability has been rising steadily. It's actually still rising, though LLM usage is slowing it. I'm not sure whether the numbers I saw were public, so I won't cite them. If you'd like to verify, I suggesting looking at the annual reports.
Re: (Score:2)
Ah, but who is the consumer of those resources. Perhaps Google is selling compute to other companies. That could be quite a profitable approach.
Re: (Score:2)
Re: (Score:2)
Nothing to See Here (Score:2)
Re: (Score:1)
Yeah! Remember COVID massive hiring spree? That caused some issues...
Google's own artificially made demand, you mean? (Score:2)
Every time any action of mine has led to Google doing any AI "computation" has been unintentional. I never asked for it. Google just chose to bundle it with my search.
I usually use an address bar keyword that uses an URL with &udm=14 to avoid this, but sometimes I slip up.
And I've also noticed that Google have added an AI summary to image search has well... I never asked for a textual description of an image, that contains factual errors or pure guesses half of the time: I have my own eyes, and I'm not
Re: (Score:2)
The AI search gadget will be a very small model (distilled from a larger model) since it'll need relatively little intelligence, since it'll be fed with search results and page snippets to formulate its blurbs.
The real demand comes from Gemini, with its very expensive model and 650 million MAU- and an independent revenue stream.
Re: (Score:2)
Every time any action of mine has led to Google doing any AI "computation" has been unintentional. I never asked for it. Google just chose to bundle it with my search.
This point cannot be emphasized enough. This is just another facet of the self-inflating bubble. The desperation behind the hype is becoming more and more obvious. Hyping AI is the new "no one gets fired for buying IBM". And once it pops, every one of these chuckleheads will say, "No one could have seen it coming" and walk away with the money their fraudulent gains.
Re: (Score:2)
Having the product stuck on the page makes it appear valuable. (Inflating user counts, familiarizing the public with the brand.)
When the product appears valuable, they can sell more of it (primarily to investors).
They have to give the product away to sell it to a few gullible people.
The fact that there's enough dumb money floating around to make it a worthwhile scam is a whole problem bigger still.
Gemini 3 (Score:3)
I use one or another of the various LLM's to do a lot of coding work for me on a regular basis. Some are definitely better than others, and usually I settle on Anthopic's Claude Sonnet 4.5, it is outstanding. But Google recently launched their Antigravity IDE, which is basically a fork of vscode that has very good integration with the new Gemini 3 release.
And while Claude 4.5 is very good, I have to say that Gemini 3 is at least that and in many ways better. I could definitely switch but at present it has drawn so much interest that it's flooded. Google has it rate-limited at present, much more demand than they have been able to provision. I don't know what their "AI serving capacity" is. Maybe not so very much, their previous offerings weren't particularly special. Gemini3 is a whole different deal, and I think they are correct to ramp up the capacity ASAP.
https://www.allaboutai.com/ai-... [allaboutai.com]
"Turn a hand-drawn dashboard sketch into working HTML, CSS, and JavaScript." And it can do it.
More lies to keep the hype going (Score:2)
How repulsive.
cold hard cache (Score:2)
As the unwashed increase their use of google's ai, 90% of queries will just be previously answered. Just cache them instead of recomputing the answers.
Solid math (Score:2)
Re: (Score:2)
2^10 = 1024 (1 -> 2 -> 4 -> 8