OpenAI Unveils First Chip As Part of Broadcom Deal (cnbc.com) 20
OpenAI and Broadcom have unveiled Jalapeno, OpenAI's first custom AI chip, designed primarily to handle inference for ChatGPT and other services. It's a major step in OpenAI's plan to "build the full stack behind its models and products," says OpenAI. "By designing more of the stack ourselves, we can serve more intelligence with greater efficiency and keep pushing advanced AI toward broader access." CNBC reports: The chip with Broadcom is an ASIC, which industry experts say is less flexible than Nvidia's GPU, but is also less expensive and can be designed for specific AI tasks. OpenAI said that it designed the chip in nine months, and that it also crafted large parts of the computer system where it will be used.
The companies are calling the chip an "Intelligence Processor" and describe it as the first "AI accelerator" in a platform they're building "to make advanced AI faster, more reliable, and more accessible to more people." [...] A physical sample of the new chip will be delivered to OpenAI on Wednesday. The companies said they're aiming for initial deployment of the Jalapeno chips by the end of 2026, "expanding in the years ahead."
The companies are calling the chip an "Intelligence Processor" and describe it as the first "AI accelerator" in a platform they're building "to make advanced AI faster, more reliable, and more accessible to more people." [...] A physical sample of the new chip will be delivered to OpenAI on Wednesday. The companies said they're aiming for initial deployment of the Jalapeno chips by the end of 2026, "expanding in the years ahead."
Wait, did you say Broadcom? (Score:4, Informative)
Re: (Score:3)
Broadcom is also the manufacturer of the Internet's darling, the Raspberry Pi.
Re: (Score:3)
Not exactly. Broadcom makes the chips used in the Pi. Initially it was due to Broadcom being the cheapest but anymore I wonder why they keep doing business.
Re: (Score:3)
Broadcom is fabless, it designs chips it does not make them. Broadcom has designed SoCs for the Pi.
Re: (Score:3)
Re: (Score:3)
I wonder why they keep doing business.
All of the Pi engineers are former bcm employees, last I heard some of the pi people were even current employees. There seems to be some motion away from bcm though.
So... nvidia is boned? (Score:2)
Because why would they need nvidia when they have everything broadcom has?
Re: (Score:2)
ASICs vs NVIDIA GPUs (Score:1)
I was under the impression that ASICs were cheaper to produce while still doing a subset of tasks as well (or nearly as well) as a more general GPU, but that the power-performance ratio was about the same.
If that is so, I don't understand how its going to allow OpenAI to serve more users. Maybe the issue is that NVIDIA can't keep up with demand? So this is a way to expand the overall capacity of their data centers? By adding fleets of ASICs in addition to their fleets of NVIDIA GPUs? Not sure.
Re: ASICs vs NVIDIA GPUs (Score:2)
Targeted markets. All in one solutions work best in government contracts, for example.
Re: (Score:2)
Nvidia does mostly one thing well: Fast tensor multiplications.
If you build a transformer-optimized architecture you can probably take a few shortcuts that for example minimize what needs to be moved around in memory.
Re: (Score:1)
Nvidia does mostly one thing well: Fast tensor multiplications. If you build a transformer-optimized architecture you can probably take a few shortcuts that for example minimize what needs to be moved around in memory.
The vast majority of the compute of executing (or training) a GPT-like model is tensor (ie multi-dimensional matrixes of floats) multiplications. GPU memory is used to read/write/store tensors for the operands and results. I don't see how a special purpose ASIC can do any better than an NVIDIA GPU can on this task, (other than being cheaper per chip at the expensive of being less general/flexible).
Re: (Score:2)
Maybe we will see if OpenAI publishes something regarding that. I can definitely imagine that knowing where which parts come to lie allows for optimizations that a general purpose hardware cannot optimize that well.
Re: (Score:2)
Nvidia does mostly one thing well: Fast tensor multiplications.
If you build a transformer-optimized architecture you can probably take a few shortcuts that for example minimize what needs to be moved around in memory.
The biggest problem in AI architecture is data movement. That's why the GPU memory subsystem takes so much chip area, to achieve huge data bandwidth. That's also why GPUs do well in training and are able to handle evolving AI models better than ASICs.
Broadcom is evil (Score:2)
They should have worked with Intel
I'm disappointed
That's Jalapeño. (Score:2)
Come on, man.
I couldn't find the power specs on it (Score:2)
I bet that chip eats a metric sh*t ton of power to run.
My money is somewhere North of 10Kw
Please tank soon. (Score:1)
When the AI bubble pops there are going to be lots of skilled chip designers looking for work. Maybe there will finally be some fresh competition in the Intel and AMD duopoly in the PC CPU scene. If we could have low power, cool running, SoC units that can run Linux as well as Apple silicon handles MacOS I might finally go back to Linux.
So this is how the bubble pops (Score:2)
The irony of this causing the AI bubble to pop would be too delicious for words. You might be asking yourself: "Why would this, a chip that makes inference cheaper, hurt the AI bubble instead of helping it?" and the answer is brutally simple:
We have billions of pre-sold Blackwell GPU's sitting in warehouses waiting for install. That's on top of the billions it will take to put them into compute modules and install them for use. There are more yet in pre-sold Vera Rub
It's already outdated... (Score:2)
Next week, Habanero will be the New Standard
Week after that, Dave's Insanity chips