Meta's Building an In-House AI Chip to Compete with Other Tech Giants (techcrunch.com) 17
An anonymous reader shared this report from the Verge:
Meta is building its first custom chip specifically for running AI models, the company announced on Thursday. As Meta increases its AI efforts — CEO Mark Zuckerberg recently said the company sees "an opportunity to introduce AI agents to billions of people in ways that will be useful and meaningful" — the chip and other infrastructure plans revealed Thursday could be critical tools for Meta to compete with other tech giants also investing significant resources into AI.
Meta's new MTIA chip, which stands for Meta Training and Inference Accelerator, is its "in-house, custom accelerator chip family targeting inference workloads," Meta VP and head of infrastructure Santosh Janardhan wrote in a blog post... But the MTIA chip is seemingly a long ways away: it's not set to come out until 2025, TechCrunch reports.
Meta has been working on "a massive project to upgrade its AI infrastructure in the past year," Reuters reports, "after executives realized it lacked the hardware and software to support demand from product teams building AI-powered features."
As a result, the company scrapped plans for a large-scale rollout of an in-house inference chip and started work on a more ambitious chip capable of performing training and inference, Reuters reported...
Meta said it has an AI-powered system to help its engineers create computer code, similar to tools offered by Microsoft, Amazon and Alphabet.
TechCrunch calls these announcements "an attempt at a projection of strength from Meta, which historically has been slow to adopt AI-friendly hardware systems — hobbling its ability to keep pace with rivals such as Google and Microsoft."
Meta's VP of Infrastructure told TechCrunch "This level of vertical integration is needed to push the boundaries of AI research at scale." Over the past decade or so, Meta has spent billions of dollars recruiting top data scientists and building new kinds of AI, including AI that now powers the discovery engines, moderation filters and ad recommenders found throughout its apps and services. But the company has struggled to turn many of its more ambitious AI research innovations into products, particularly on the generative AI front. Until 2022, Meta largely ran its AI workloads using a combination of CPUs — which tend to be less efficient for those sorts of tasks than GPUs — and a custom chip designed for accelerating AI algorithms...
The MTIA is an ASIC, a kind of chip that combines different circuits on one board, allowing it to be programmed to carry out one or many tasks in parallel... Custom AI chips are increasingly the name of the game among the Big Tech players. Google created a processor, the TPU (short for "tensor processing unit"), to train large generative AI systems like PaLM-2 and Imagen. Amazon offers proprietary chips to AWS customers both for training (Trainium) and inferencing (Inferentia). And Microsoft, reportedly, is working with AMD to develop an in-house AI chip called Athena.
Meta says that it created the first generation of the MTIA — MTIA v1 — in 2020, built on a 7-nanometer process. It can scale beyond its internal 128 MB of memory to up to 128 GB, and in a Meta-designed benchmark test — which, of course, has to be taken with a grain of salt — Meta claims that the MTIA handled "low-complexity" and "medium-complexity" AI models more efficiently than a GPU. Work remains to be done in the memory and networking areas of the chip, Meta says, which present bottlenecks as the size of AI models grow, requiring workloads to be split up across several chips. (Not coincidentally, Meta recently acquired an Oslo-based team building AI networking tech at British chip unicorn Graphcore.) And for now, the MTIA's focus is strictly on inference — not training — for "recommendation workloads" across Meta's app family...
If there's a common thread in today's hardware announcements, it's that Meta's attempting desperately to pick up the pace where it concerns AI, specifically generative AI... In part, Meta's feeling increasing pressure from investors concerned that the company's not moving fast enough to capture the (potentially large) market for generative AI. It has no answer — yet — to chatbots like Bard, Bing Chat or ChatGPT. Nor has it made much progress on image generation, another key segment that's seen explosive growth.
If the predictions are right, the total addressable market for generative AI software could be $150 billion. Goldman Sachs predicts that it'll raise GDP by 7%. Even a small slice of that could erase the billions Meta's lost in investments in "metaverse" technologies like augmented reality headsets, meetings software and VR playgrounds like Horizon Worlds.
Meta's new MTIA chip, which stands for Meta Training and Inference Accelerator, is its "in-house, custom accelerator chip family targeting inference workloads," Meta VP and head of infrastructure Santosh Janardhan wrote in a blog post... But the MTIA chip is seemingly a long ways away: it's not set to come out until 2025, TechCrunch reports.
Meta has been working on "a massive project to upgrade its AI infrastructure in the past year," Reuters reports, "after executives realized it lacked the hardware and software to support demand from product teams building AI-powered features."
As a result, the company scrapped plans for a large-scale rollout of an in-house inference chip and started work on a more ambitious chip capable of performing training and inference, Reuters reported...
Meta said it has an AI-powered system to help its engineers create computer code, similar to tools offered by Microsoft, Amazon and Alphabet.
TechCrunch calls these announcements "an attempt at a projection of strength from Meta, which historically has been slow to adopt AI-friendly hardware systems — hobbling its ability to keep pace with rivals such as Google and Microsoft."
Meta's VP of Infrastructure told TechCrunch "This level of vertical integration is needed to push the boundaries of AI research at scale." Over the past decade or so, Meta has spent billions of dollars recruiting top data scientists and building new kinds of AI, including AI that now powers the discovery engines, moderation filters and ad recommenders found throughout its apps and services. But the company has struggled to turn many of its more ambitious AI research innovations into products, particularly on the generative AI front. Until 2022, Meta largely ran its AI workloads using a combination of CPUs — which tend to be less efficient for those sorts of tasks than GPUs — and a custom chip designed for accelerating AI algorithms...
The MTIA is an ASIC, a kind of chip that combines different circuits on one board, allowing it to be programmed to carry out one or many tasks in parallel... Custom AI chips are increasingly the name of the game among the Big Tech players. Google created a processor, the TPU (short for "tensor processing unit"), to train large generative AI systems like PaLM-2 and Imagen. Amazon offers proprietary chips to AWS customers both for training (Trainium) and inferencing (Inferentia). And Microsoft, reportedly, is working with AMD to develop an in-house AI chip called Athena.
Meta says that it created the first generation of the MTIA — MTIA v1 — in 2020, built on a 7-nanometer process. It can scale beyond its internal 128 MB of memory to up to 128 GB, and in a Meta-designed benchmark test — which, of course, has to be taken with a grain of salt — Meta claims that the MTIA handled "low-complexity" and "medium-complexity" AI models more efficiently than a GPU. Work remains to be done in the memory and networking areas of the chip, Meta says, which present bottlenecks as the size of AI models grow, requiring workloads to be split up across several chips. (Not coincidentally, Meta recently acquired an Oslo-based team building AI networking tech at British chip unicorn Graphcore.) And for now, the MTIA's focus is strictly on inference — not training — for "recommendation workloads" across Meta's app family...
If there's a common thread in today's hardware announcements, it's that Meta's attempting desperately to pick up the pace where it concerns AI, specifically generative AI... In part, Meta's feeling increasing pressure from investors concerned that the company's not moving fast enough to capture the (potentially large) market for generative AI. It has no answer — yet — to chatbots like Bard, Bing Chat or ChatGPT. Nor has it made much progress on image generation, another key segment that's seen explosive growth.
If the predictions are right, the total addressable market for generative AI software could be $150 billion. Goldman Sachs predicts that it'll raise GDP by 7%. Even a small slice of that could erase the billions Meta's lost in investments in "metaverse" technologies like augmented reality headsets, meetings software and VR playgrounds like Horizon Worlds.
Zuck! (Score:2)
The hype may die before that (Score:4, Interesting)
I mean, it is already becoming blatantly obvious how limited this form of AI actually is. It does sound good and it does sound confident in its answers. That is probably the only thing that has really improved much and it seems to be enough to trick most people. Sure, it can solve simplistic problems (with a real chance that it is just making things up that sound plausible but are utter nonsense), but that is not really that helpful. Maybe it can actually summarize documents with reasonable reliability, but even that seems doubtful. What is seems to be able to do is find stuff better than conventional search engines. But given how utterly bad Google search, for example, is these days, that seems to be hardly much of an accomplishment.
Hence it is entirely possible the hype is over by 2025. The one reasonable use some of my friends have found is as a somewhat dumb personal assistant. Maybe that will be something that remains or maybe not. For me it will depend on customization. If I just get the generic thing, it will probably not be worth the bother. If I can customize in a way that the customization needs to be sent back and some shady company learns a lot about me, that is a no-go as well. If I can run my own and backup my own customizations, that would work, bit it does not look like that will be possible very soon.
Re: (Score:2)
The one reasonable use some of my friends have found is as a somewhat dumb personal assistant. Maybe that will be something that remains or maybe not.
Just having software that can summarize is enormously useful. If that's the only trick it ever does a good job with, it'll be a huge win. I've seen a lot of people who seem rational say it's doing that well for them. Obviously though I have to be able to run it on my device, and it has to be OSS, so it's not going to be producing any revenue anyway.
The stuff I'm really impressed with is the image generators anyway, more than the text generators. As people find spiffy techniques for automating the processes
Re: (Score:2)
I mean, it is already becoming blatantly obvious how limited this form of AI actually is. It does sound good and it does sound confident in its answers. That is probably the only thing that has really improved much and it seems to be enough to trick most people. Sure, it can solve simplistic problems (with a real chance that it is just making things up that sound plausible but are utter nonsense), but that is not really that helpful. Maybe it can actually summarize documents with reasonable reliability, but even that seems doubtful. What is seems to be able to do is find stuff better than conventional search engines. But given how utterly bad Google search, for example, is these days, that seems to be hardly much of an accomplishment.
I think this is selling the technology way too short. People come up with wrong answers all the time.
Quiz humans and they blurt out comically wrong "hallucinations" with regularity. Ask a human how to do something and they will come up with solutions that don't pass scrutiny. These facts don't make humans worthless or otherwise incapable of producing reliable solutions.
Like all endeavors what matters is process and vigor. When creating a solution one does not simply stop at the first idea to pop into th
Re: The hype may die before that (Score:2)
Re: (Score:2)
I think they're just doubling down on stupid even if that is their actual aim, but I'm not invested in Facebook and if they want to burn billions of their own money it's no skin off my back. Maybe they won't achieve anything themselves, but the de
Re: (Score:2)
I mean, it is already becoming blatantly obvious how limited this form of AI actually is.
Hence it is entirely possible the hype is over by 2025.
There are very few technologies that don't have obvious limitations. Yet, many of these technologies are nonetheless used a lot. Google search is nowhere near perfect. It doesn't have GPT's level of confidence, but it does spit out plenty of non-useful responses. Wikipedia is arguably more on-point for a specific broad topic, but its information needs to be parsed.
I find GPT to be very useful. However, I treat it like Google search and Wikipedia, i.e., resources for information that I then parse and ev
How Slashdot has fallen. (Score:2)
Dear god, how the fuck did this trash get upvoted to +5?
I'm betting money on it that you haven't even used any of the new LLMs as well as image generators.
Facebook should have equivalent of AWS (Score:1)
Re: (Score:2)
In the present economy and after the metaverse debacle, I don't think shareholders would go along with massive investments to break into cloud services. Facebook should have a mega-merger with one of the existing cloud provider. The time for their main business model is running out, but the knowhow and services they still have would combine really well as you say.
Between the Trump derangement Syndrome sufferers and the people who scream about monopoly at every opportunity, but magically become blind wheneve
Maybe we shouldn't have software companies (Score:3)
Re: (Score:2)
Teams of college students design ASICs.
So do many very small startups.
The future is analog (Score:2)
I think we'll soon see an explosion of analog TPUs in the near future. One need only look at shoestring efforts like Mythic running on ancient nodes to see the writing on the wall.