Furiosa's Energy-Efficient 'NPU' AI Chips Start Mass Production This Month, Challenging Nvidia (msn.com) 25
The Wall Street Journal profiles "the startup that is now one of a handful of chip makers nipping at the heels of Nvidia."
Furiosa's AI chip is dubbed "RNGD" — short for renegade — and slated to start mass production this month. Valued at nearly $700 million based on its most recent fundraising, Furiosa has attracted interest from big tech firms. Last year, Meta Platforms attempted to acquire it, though the startup declined the offer. OpenAI used a Furiosa chip for a recent demonstration in Seoul. LG's AI research unit is testing the chip and said it offered "excellent real-world performance." Furiosa said it is engaged in talks with potential customers.
Nvidia's graphic processing units, or GPUs, dominated the initial push to train AI models. But companies like Furiosa are betting that for the next stage — referred to as "inference," or using AI models after they're trained — their specialty chips can be competitive. Furiosa makes chips called neural processing units, or NPUs, which are a rising class of chips designed specifically to handle the type of computing calculations underpinning AI and use less energy than GPUs. [Founder/CEO June] Paik said Furiosa's chips can provide similar performance as Nvidia's advanced GPUs with less electricity usage. That would drive down the total costs of deploying AI. The tech world, Paik says, shouldn't be so reliant on one chip maker for AI computing. "A market dominated by a single player — that's not a healthy ecosystem, is it?" Paik said...
In 2024, at Stanford's prestigious Hot Chips conference, Paik debuted Furiosa's RNGD chip as a solution for what he called "sustainable AI computing" in a keynote speech. Paik presented data showing how the chip could run the then-latest version of Meta's Llama large language model with more than twice the power efficiency of Nvidia's high-end chips. Furiosa's booth was swarmed with engineers from big tech firms, including Google, Meta and Amazon.com, wanting to see a live demo of the chip. "It was a moment where we felt we could really move forward with our chip with confidence," Paik said.
Nvidia's graphic processing units, or GPUs, dominated the initial push to train AI models. But companies like Furiosa are betting that for the next stage — referred to as "inference," or using AI models after they're trained — their specialty chips can be competitive. Furiosa makes chips called neural processing units, or NPUs, which are a rising class of chips designed specifically to handle the type of computing calculations underpinning AI and use less energy than GPUs. [Founder/CEO June] Paik said Furiosa's chips can provide similar performance as Nvidia's advanced GPUs with less electricity usage. That would drive down the total costs of deploying AI. The tech world, Paik says, shouldn't be so reliant on one chip maker for AI computing. "A market dominated by a single player — that's not a healthy ecosystem, is it?" Paik said...
In 2024, at Stanford's prestigious Hot Chips conference, Paik debuted Furiosa's RNGD chip as a solution for what he called "sustainable AI computing" in a keynote speech. Paik presented data showing how the chip could run the then-latest version of Meta's Llama large language model with more than twice the power efficiency of Nvidia's high-end chips. Furiosa's booth was swarmed with engineers from big tech firms, including Google, Meta and Amazon.com, wanting to see a live demo of the chip. "It was a moment where we felt we could really move forward with our chip with confidence," Paik said.
acronym (Score:3, Funny)
RNGD is for Random Number Generator Daemon?
Re: acronym (Score:5, Funny)
It's for AI models, so Random Nonsense Generating Dummy.
No data included (Score:3)
Read the article and did not find any mention other than a "uses less electricity" mention.
The news should include a comparison including power usage per chip and server rack, AC cooling requirement power usage for a workload for this chip and a NVIDIA system.
The news editor could help with AI quality control by: "In this news article, remove all mentions of childhood, outdoor scenery, clothes, fashion trends, physical descriptions of people, emotional appeals, softening language, and vignettes."
Removing all of those would help to show that the article is 98% filler material (CEO personal profile + office decor + vignette) with little information.
Re: (Score:2)
Which part of "uses less electricity" don't you understand?
Re: (Score:1)
Re: (Score:2)
From TFS:
"the chip could run the then-latest version of Meta's Llama large language model with more than twice the power efficiency of Nvidia's high-end chips. "
Re: (Score:2)
Re: (Score:2)
It's a press release, not a tech spec.
You're not going to be buying these any time soon.
Re: (Score:2)
https://developer.furiosa.ai/l... [furiosa.ai]'
Based on "Tensor Contraction Processor" also of FuriosaAI's:
https://web.ist.utl.pt/nuno.lo... [ist.utl.pt]
"A TCP chip was designed and fabricated in 5nm technology as the second-generation product of Furiosa AI, offering
256/512/1024 TOPS (BF16/FP8 or INT8/INT4) with 256 MB
SRAM and 1.5 TB/s 48 GB HBM3 under 150 W TDP."
Re: (Score:2)
Nothing About AI Saves Energy (Score:1, Troll)
Youtube data (Score:1)
Hopefully, someone will collect metrics on Youtube videos produce by AI, categorize them into "entertainment including brainrot" and everything else and then estimate the electricity used for the ai produced entertainment and scale it in terms of how many life saving medical operations could be performed in poor countries
Does this mean (Score:5, Funny)
I might finally be able to build a decent gaming PC?
Nope (Score:2)
Re: (Score:3)
Only marginable decent.
But for old versions of Pac Man, it might be okay.
Re: (Score:2)
This is actually bad for gamers. The bottleneck is how many chips TSMC can make, new designs aren't going to change that. But at least with GPUs you will be able to buy them on the cheap once the bubble bursts, while inference chips will end up in a landfill. Now some of these chips are just generic systolic arrays doing matrix multiplication which could have uses in a desktop, but you would need to get drivers for it somehow and also no software is going to support them.
Re: (Score:2)
You can if you disregard future proofing for now. Older platforms capable of running DDR4 memory are coming back into vogue because DDR4 memory prices, while still inflated, are still roughly half the price of an equivalent amount of DDR5.
This has meant things like AMD AM4, and Intel 13th and 14th gen CPUs are back which can be paired with a decent midrange GPU.
The other meta if that PCs with 16GB of RAM would become common again, with 32GB being "high end
TSMC (Score:2)
The US AI stock bubble is hideously dependent on a few fabs in Taiwan.
Software interfaces matter too. (Score:2)
CUDA locks people into using NVIDIA hardware, and I doubt that'll change. Yes, the big guys can spend the engineering effort to rewrite software, but that'll give NVIDIA time to release their own inference optimized chips. I'd bet they're already working on some.
Re: (Score:2)
But do NPUs lock you into doing AI? After the AI bubble bursts, your CUDA/Nvidia hardware can be used for other things. While I prefer more open solutions such as OpenCL on AMD, the same OpenCL code will also run on Nvidia hardware.
I don't know about the details of NPUs, but I've got the impression that efficient AI uses very low precision arithmetic, such as 8-bit and even lower. So while it could be possible to use NPUs for general vector/matrix math, the low precision would be a deal breaker for a lot