Nvidia Unveils $3,000 Personal AI Supercomputer (nvidia.com) 71
Nvidia will begin selling a personal AI supercomputer in May that can run sophisticated AI models with up to 200 billion parameters, the chipmaker has announced. The $3,000 Project Digits system is powered by the new GB10 Grace Blackwell Superchip and can operate from a standard power outlet.
The device delivers 1 petaflop of AI performance and includes 128GB of memory and up to 4TB of storage. Two units can be linked to handle models with 405 billion parameters. "AI will be mainstream in every application for every industry," Nvidia CEO Jensen Huang said. The system runs on Linux-based Nvidia DGX OS and supports PyTorch, Python, and Jupyter notebooks.
The device delivers 1 petaflop of AI performance and includes 128GB of memory and up to 4TB of storage. Two units can be linked to handle models with 405 billion parameters. "AI will be mainstream in every application for every industry," Nvidia CEO Jensen Huang said. The system runs on Linux-based Nvidia DGX OS and supports PyTorch, Python, and Jupyter notebooks.
That sounds like a winner, seriously. (Score:1)
Just getting that much VRAM costs more than 3k... If it's not crippled in some way I can see these flying off the shelves.
Re: (Score:2)
I suspect NVidia will be siphoning $3k out of my bank account this year :P
Re:That sounds like a winner, seriously. (Score:5, Insightful)
I suspect NVidia will be siphoning $3k out of my bank account this year :P
Per TFA, the siphoning will start at $3k
Re: (Score:1)
Yes, but the difference in prices will be storage-related, because per TFPR, "Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage." The max price shouldn't be too much higher given the difference between a plausible lower bound on provided storage (I'd say 512GB) and the price of a 4TB NVMe SSD. Unless they go full Apple and it's soldered, I guess. But even then, why would it matter? As long as local storage is significantly larger than memory, you're not going to bo
Re: (Score:2)
I mean, I'd just connect it via NFS to my fileserver regardless. Most storage needs for training have no performance constraints, and for inference, it's just about how long it takes to load the model when you change models.
Re: (Score:2)
"Most storage needs for training have no performance constraints..."
Perhaps for this device's intended use, but not generally true. Nothing says sophisticate quite like spending thousands on a dedicated AI compute module for your desk, then not caring how data is fed to it. I mean, it's there for bragging rights, not to get work done!
The real problem here is that the device runs Python AI code bases, those code bases use python libraries to import data sets, python is slow as shit, and this Nvidia processo
Re: (Score:2)
The max price shouldn't be too much higher given the difference between a plausible lower bound on provided storage (I'd say 512GB) and the price of a 4TB NVMe SSD. Unless they go full Apple and it's soldered, I guess
If they go full Apple, the price delta will be a small part of the price difference...
Re: (Score:1)
That's definitely where I was going with that. Soldering the storage makes it possible to overcharge for more of it.
Commodification of AI processing (Score:2)
Fully expect to see these evolve into general purpose compute boxes used in a typical office and cut out more expensive cloud computing.
Re:Commodification of AI processing (Score:5, Interesting)
A killer app here would be local inference servers for commercial coding tasks. A lot of companies refuse to let their code leave their office for queries to eg. Claude or whatnot. A server like this could serve a really hefty local model to dozens if not hundreds of programmers at once. At ~$3k per unit (or even more), what software development company that wasn't adamantly opposed to AI wouldn't buy one?
In addition to the open coding models out there, I could picture companies like Anthropic offering servers that are already set up for an open-weights version of one of their flagship models that's been optimized for coding, running on an inference server specifically optimized to both the hardware and the model, shipping with a full commercial license for any number of programmers, for an addon fee atop the base hardware price. Whereas those who are "on the cheap" will just install the best free open model and inference server on their own.
Any sort of conversational job too (Score:3)
1. Put AI to answer on a general questions phone support line
2. Ask "Did that answer your question?" or "Did that solve your issue?"
3. Ask "How would you rate the quality of this call today?"
And use positive and high rating answers to #2 and #3 to up the rating of the conversation for future training of the AI.
Re: (Score:2)
Because the most important problem to solve when using shitty AI coding crutches is keeping it a secret.
"what software development company that wasn't adamantly opposed to AI wouldn't buy one?"
Ones that solve the problem in other ways?
This is a product that gets a very specific CPU down to a price and form factor suitable for a single user at a desk. It's not a server for "hundreds of programmers". Perhaps "what software development company" would be a company that understands that reality.
Re: (Score:2, Insightful)
Re: (Score:3)
Just getting that much VRAM costs more than 3k... If it's not crippled in some way I can see these flying off the shelves.
I suspect it will as well, especially if you can run a server on it so a small company could setup its own secure AI system free of cloud fees to access the system's power. We are experimenting with using AI for a product and run a model on an M3Max Mac; this would be a whole new level of capability at a bargain price.
Re: (Score:2)
Re: That sounds like a winner, seriously. (Score:2)
And mailmerge? Lol.
Re:That sounds like a winner, seriously. (Score:5, Informative)
Just getting that much VRAM costs more than 3k... If it's not crippled in some way I can see these flying off the shelves.
Some more details are in this other article. [theregister.com] An excerpt:
Project Digits vaguely resembles an Intel NUC mini-PC in terms of size. Nvidia hasn’t detailed the GB10’s specs in full but has said the machine it powers delivers a full petaFLOP of AI performance. But before you get too excited about the prospect of a small form factor desktop outperforming Nvidia’s A100 tensor core GPU, know that the machine’s performance was measured on sparse 4-bit floating point workloads.
Specs we’ve seen suggest the GB10 features a 20-core Grace CPU and a GPU that packs manages a 40th the performance of the twin Blackwell GPUs used in Nvidia’s GB200 AI server.
So, 1/40th the performance of twin Blackwells. I don't suppose that counts as "crippled" but there you go.
Re: (Score:2)
But that's like a $70k piece of kit that does 40 PFLOPS at FP4 with 384GB VRAM.
So by that estimate, this would do 1 PFLOP at FP4, with 128GB VRAM, for $3k. I'm not complaining. And it's still on the same chip architecture, so efficiency should be just as good.
Re: (Score:2)
Thanks for the reply. The question was whether it's crippled. I'm going with "probably not" but let's wait for the independent specs.
I might even get one myself.
Re: (Score:2)
But can it run Crysis?
Screamer (Score:3)
It reminds me of the old "Byte" days of reporting progress in flops - like page 143 featuring this advert for a "Screamer 500" [vintageapple.org] from 1997.
Running on a 500 MHz 21164 that bursts at 1 gigaflop, a dot product kernel we use for compiler testing runs at a mindboggling 940 megaflops! ! !
They may not have been able to compete with modern performance, but "Screamer", that's a great name
For who? (Score:3, Interesting)
It's for robotics & distributed edge compute (Score:5, Informative)
So who are they trying to sell this to and for what purpose? I think this is just "make stock go up" AI bullshit before the bubble bursts.
Well, this is a small ARM board with big GPU (Blackwell is ARM) and I used their previous $3000 small ARM board (Orin AGX) for compute heavy operations in mobile, battery powered robotic platforms. The AGX is several years old, had 64GB and could do 0.275 petafiop at 60W, or about half that on 15W. https://www.ebay.com/itm/22493... [ebay.com] (Used, 2.6k usd)
This seems like an update to the years-old AGX, small portable package with 4x performance and 2x ram at the same price-point, so I imagine targets the same developer audience who needs small mobile/distributed compute.
Re: It's for robotics & distributed edge compu (Score:2)
Re: (Score:2)
It runs their cloud/server stack that's in DGOS, an Ubuntu derivative. It's much more closely related to the GB200 than the Tegra related devices like the orin.
Re: (Score:2)
Re: (Score:2)
You can probably pip install jupyterhub and make Jupyter available on it over the network.
Re: (Score:2)
Indeed, the entire CUDA related stack and any tools that use it should "just work".
Re: For who? (Score:2)
But it still won't run Crysis (Score:1)
Now in addition to money and resources being wasted on AI nonsense, they're going to lure people into impoverishing themselves even farther than they're already impoverished. People struggle to afford their rent and Nvidia wants to convince them to buy a computer that starts at $3000?
Some caveats (Score:3)
Still, it's a slick package and a lot of power. The available images don't show any active cooling, which is hard to fathom. They probably just omitted that (and heat sinks generally) from the press materials. Is it just me, or do the front and rear panels look like they're made of copper sponge?
Re: (Score:2)
FP16 should run approximately 4x slower on the nvidia personal AI, so 250 Gigaflops. FP16 is much more commonly quoted for other CPUs and GPUs.
128GB max ram? the old mac pro can do more and oth (Score:2)
128GB max ram? the old m 2 mac pro can do more and other PC hardware can go much higher.
Re:128GB max ram? the old mac pro can do more and (Score:4, Informative)
Re: (Score:2)
Quite a differentiation over Apple Silicon! /s
Re: (Score:2)
*Pending third party performance testing of course.
Re: (Score:2)
But the comment wasn't about cost, it was about an "important distinction" being unified RAM which all Apple Silicon has.
But if you're gonna make stupid comments and move the goalposts, a Mac mini with half the RAM is a far more capable device over a broad range of applications. This is a very specific device for a particular application, it does not replace a general purpose computer (well). But again, to be clear, unified memory is not a special feature, or even a good one. It is well suited to this ca
Re: (Score:2)
Apple Silicon is unified RAM.
Re: (Score:2)
Re: (Score:2)
You said it was an "Important distinction when talking about LLMs" specifically when compared to an Apple silicon Mac. You literally said that unified memory distinguished this device over a Mac, and it was "important".
But no, you didn't say it was the ONLY device with unified RAM, and neither did the person you responded to. What you said was that unified RAM distinguished this device over an M2 Mac Pro, which is not true. And now you're making bad faith arguments. It must really suck being you.
Re: (Score:2)
Re: (Score:2)
This machine is not a general purpose computer, it is a specific function device optimized for development tasks. It has a truly shitty CPU complex compared to a Mac. Your argument is very SuperKendall-esque, cherry picking details and intentionally misrepresenting both facts and previous arguments. You sound like an old Apple fanboy.
"PC's generally do NOT have unified RAM"
Not true, and that feature is not new. It's generally an inferior approach used on cheap PCs, but repopularized specifically in Appl
Re: 128GB max ram? the old mac pro can do more and (Score:2)
Re: (Score:2)
Right. But the vast majority of x86 laptops and desktops have 128 bit wide memory and would be severely bottlenecked for AI workloads. This widget should have a faster memory system (widely speculated to be 256 to 512 bits wide). No official word from nvidia ... yet.
Re: (Score:2)
Right, and tiered memory architectures have been proven inferior, right? That's why we only have cache-less, unified memory systems today.
up to 4TB of storage so only 1 m.2 slot? (Score:2)
up to 4TB of storage so only 1 m.2 slot?
does it have sata?
what is the number of pci-e lanes?
what kind of IO does it have?
have pci-e slots?
Re: (Score:3)
It's there to do your thinking for you, dude, not to store your p0rn.
Re: (Score:2)
It's there to do your thinking for you, dude, not to store your p0rn.
When it can store as much porn as our brains, then it can do our thinking for us. Until then, it's just a useless business toy. Sigh.
When the AI bubble bursts (Score:2)
can I use this machine to mine Bitcoin?
Re: (Score:2)
Re: (Score:2)
Oh that's okay. I just need a couple.
Imagine a ... (Score:2)
Nobody?
Re:Imagine a ... (Score:5, Funny)
Re: Imagine a ... (Score:1)
Re: (Score:1)
Forget beowulf or any other cluster - imagine a *physical computer lab* of these puppies!
Re: (Score:2)
I think so, Brain, but what will Natalie Portman and Cowboy Neil do with all those grits?
At least they are eating their own dogfood. (Score:2)
Look at the picture they posted to go along with this press release. Look closely. Exactly what alphabet is this obviously AI generated image using? Klingon? Hallucinations are pretty funny. AI THINKS that those shapes look like language, as though we just write shapes on paper and they have some meaning, throwing away an established alphabet.
Re: (Score:2)
Very good catch! Even the window titles, and OS-level "icons" are obviously bull.
Re: (Score:2)
"AI THINKS that those shapes look like language..."
What makes you think that AI believes those shapes are supposed to be language?
"...as though we just write shapes on paper and they have some meaning..."
That is exactly what "we" do.
"...throwing away an established alphabet."
What evidence is there that AI is even capable of that? An "established alphabet" is a fundamental component of how AI works with language, AI couldn't "throw away" its alphabet without discarding itself.
Re: (Score:2)
The AI placed that 'text' where text would go.
WE have already agreed on alphabets to use. We don't make them up each time we write a new document.
It's a bit of hyperbole, it didn't literally throw it away. But it didn't use an established alphabet. It didn't even use it's own as the 'letters' don't have re-use, each is unique. And image diffusion works quite differently from LLMs, including their use of language. So saying away it would discard itself is a little silly.
But I'm glad you had the time to
Re: (Score:2)
"The AI placed that 'text' where text would go."
You don't have any reason to believe AI was even involved in generating that image.
"WE have already agreed on alphabets to use. We don't make them up each time we write a new document."
But we did at one time. Alphabets are literally just shapes written on paper, to use your specific language. We literally write shapes on paper to create writing. You didn't say the shapes were entirely new each time, although that has historically also been true. Not all lan
What memory? (Score:2)
It's all about the GB/s and memory pool.
$3000 cover price irrelevant (Score:1)
Re: (Score:2)
LOL the GB200 has a TDP of 2700 watts, this is 2-3% of that processor. Not that a 3KW wall outlet is a problem, BEV chargers are commonly 3x more powerful than that.
You might want to take a look at the size of the product, then wonder where 3000 watts could possibly go.
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Wow stupid has no limits, huh?
No, the argument is that you could run 30 of them out of an "EV socket" because your assumption that it consumes that much power is bullshit. Get it, short bus boy?
Sorry for the part I played feeding nVidia (Score:2)
$3K plus (Score:2)
Don't forget to add on the Trump Tariffs
(for any components not made in the USA)
Power consumption? (Score:2)
At $0.17/KWhour, what will it cost to run? And keep cool?
Can it heat my house?
Re: (Score:2)
Did you see the package? Where do you think a heat exchanger of your imaginary capacity could possibly go?
What will it cost to run? Probably about as much as a desktop computer. It requires no special power source or cooling.