Meta's 'Open' Muse Glimmer Model Can Run On a Single Computer (engadget.com) 53
Meta has released Muse Glimmer, a slimmed-down open-weight AI model designed to run locally on a single GPU for agent tasks such as scheduling, file management, coding, and tool use. The release is based on Meta's closed Muse Spark 1.2 model and appears to be aimed at attracting developers who want capable AI agents without relying entirely on cloud-hosted services. Engadget reports: Facebook said that it's making the "weights" that AI systems use to choose responses available to everyone on Hugging Face along with developer documentation. The download is available for free, and users can run the model on their own PCs. The company noted that optimized integrations will land on llama.cpp and other sites, "so you can go from download to working agent in minutes."
The model is powerful for its size, according to Meta, with the "strong success rates" on benchmarks like DeepSearch QA, MCP-Atlas and SWE-Bench (which evaluates its ability write and debug code). It also supports reliable tool use, multi-step reasoning, failure recovery, multimodal input and scaffold compatibility for work with OpenClaw and other agent orchestrators. It was trained on data from over 100 languages, the company added. "Rather than centralizing superintelligence, we should distribute it widely and give every person the ability to direct it," CEO Mark Zuckerberg said in an essay accompanying Muse Glimmer's release. "This has the potential to begin a new era of personal empowerment where individuals can use this powerful new capability to reach their full potential, pursue their interests, and improve their lives and the world more than ever before."
The model is powerful for its size, according to Meta, with the "strong success rates" on benchmarks like DeepSearch QA, MCP-Atlas and SWE-Bench (which evaluates its ability write and debug code). It also supports reliable tool use, multi-step reasoning, failure recovery, multimodal input and scaffold compatibility for work with OpenClaw and other agent orchestrators. It was trained on data from over 100 languages, the company added. "Rather than centralizing superintelligence, we should distribute it widely and give every person the ability to direct it," CEO Mark Zuckerberg said in an essay accompanying Muse Glimmer's release. "This has the potential to begin a new era of personal empowerment where individuals can use this powerful new capability to reach their full potential, pursue their interests, and improve their lives and the world more than ever before."
What card? (Score:2)
So do I finally need to buy an RTX 5090?
Re:What card? (Score:4, Informative)
At what cost? (Score:2)
If you want to use it for coding or something else requiring a very large context then yea, 5090 with 32gb.
If only that was cheaper than hiring a human to do it..
Re: (Score:1)
I purchased a used Radeon Instinct MI50 with 32Gb VRAM on eBay for $360 — in December. it is quite usable with llama.cpp under FreeBSD [freshports.org] inside an older (DDR3) workstation. I use it to filter incoming email (with a plugin to SpamAssassin [github.com]).
That same model is now being sold for over $500. On the one hand, this is the first time in my life a witnessing a piece of computer hardware costing more seven month later.
But on the other limb, it is still chea
Re: (Score:2)
Re: (Score:3)
You can use it fine on 24GB with a good size context with KV cache compression. They've also designed it to use speculative decoding to boost performance instead of a MoE, so that they can pack as much capability as possible into a dense model. Seems like a winner.
Re: (Score:2)
speculative decoding gets you speed (in my experience factor about 1.8-2.0) but not RAM. A MoE saves RAM and is faster than a model with spec decoding that doesn't fit the card completely.
This model has the advantage to have quantization aware training and works well in 4 Bit. That saves you half the RAM (at cost of some quality loss, but 4 Bit is often the smallest acceptable).
Re: (Score:3)
MoEs don't save RAM (for a given quality), they increase it. You have to store all of the parameters in memory, not just the active params. But inference only uses a subset of the total params for each token, so it reduces the memory bandwidth requirements and improves token generation rate. But this comes at the cost of a higher total param count for a given quality.
Re: (Score:2)
The thing is, you can put experts in CPU RAM, if your VRAM is small and get a better Token/s rate than when you do the same with a dense model of comparable size. The best model is the one that fits into your memory. What does it help when Kimi releases a 2T model, when you don't have the hardware for it.
Re: What card? (Score:2)
A Radeon Pro R9700 is much cheaper (around â1500 where I live) and has the same VRAM. ROCm and Vulkan have evolved quite nicely and with room for improvement. I considered buying that, however, ended up choosing a Ryzen AI Max+ 395 with 128GB (around â2500) so I could experiment with bigger models.
Re:What card? (Score:5, Interesting)
And how f-d up is that? Since when does computer hardware increase in value after it was released?
Re: What card? (Score:2)
If you have a decent CPU it might run on that at half speed or so.
Re: What card? (Score:4, Informative)
This really isn't a good model for CPU. For CPU, you want a MoE with a large number of total params but a tiny number of active params. Something like DeepSeek V4 Flash 0731 if you have at least 128GB of RAM - you might get 2-3 tok/s or so on that. The goal is to minimize the memory bandwidth requirements per token, at the cost of a greater total RAM footprint.
For GPU, you're highly VRAM limited but not bandwidth limited, so your best option is generally a dense model (non-MoE) with speculative decoding to make up for the performance limitations.
Re: (Score:2)
Feels like such an announcement should include a few examples and their relative performance stats. Instead, the main linked article includes nothing of the sort, and Meta's announcement only notes it would fit in 20gb of memory, so you'd only need a 24gb or 32gb card - normal consumer level stuff, right? /s
Re: (Score:2)
It as well runs in ordinary RAM.
GPUs and TPUs are more important for training, less important for using an ANN/LLM.
Re: (Score:2)
I mean, if you want your model to be terribly slow and terribly power inefficient, sure, go ahead and run it on CPU.
Re: (Score:2)
If you only want to run it ... and not train it: the difference is tolerabel.
Does not make sense if you are "new to AI" to invest in hardware first instead of checking out what you can do.
Then I was basically aiming at Mac Minis, or other computers that have GPUs connected to the main RAM.
I am actually not sure if this laptop here has a dedicated GPU RAM, lol. It is an Acer with AMD Radeon, that should have its own GPU RAM but not more than 4GB or so.
Re: (Score:2)
Define "tolerable".
The model in question - Muse Glimmer - with DFlash/speculative decoding - will probably get you ~60 to 124 tok/s on a 3090 (a quite dated GPU). For a 5090, it's said to clock in at 233,4 tok/s. On CPU you're looking at maybe 3-5 tok/s.
If you call that "tolerable", I guess you're more patient than me? And as mentioned, you're not just wasting time, but also wasting a lot of power too - CPU is a very power-inefficient way to run ML models.
If you insist on CPU, this isn't the right kind o
Re: (Score:2)
Thanks! Those seem like valuable estimates and some decent info. Makes me wish they included some actual benchmarks even more :-)
I might have to give it a try just to see how slow it is on just CPU + memory - I've got a spare HP Z something-or-other with 128gb RAM, but the CPUs are pretty ancient.
Re: (Score:2)
If you insist on CPU,
I don't.
Just do not have the budget to get a new high end Laptop :D
This model is designed for >= 24GB GPUs.
I did not talk (or did not want to) about a specific model.
Thanx for your knowledge.
Re: (Score:2)
MSI Spark. Thank me later.
Re: (Score:3)
16GB plus some offloading for the 4-Bit version I think and people say it quantizes with little quality loss.
Last time I checked the cards recommended were: 5060 Ti for starters (16 GB best value), 3090 if you need 24GB VRAM and 5090 if you need 32 GB. The xx90 are now quite unaffordable and the 5060 Ti now isn't exactly cheap either.
Re: (Score:2)
I'm seeing 3090s go for $1000-1200, which isn't actually much more than they were a few years ago. They haven't sky rocketed in price like the 5090. Solid points on the rest.
Re: (Score:2)
The problem is, that 30xx is getting old. Usually one would expect 40xx to take its place, but my impression is that 40xx doesn't have a good place at all currently.
Features one may want to have for AI:
40xx and later: FP8
50xx and later: FP4
I am not sure if int8 and intconvrot may make 30xx more attractive again, but one also needs to see that they will fall out of driver support earlier.
Many models are now optimized for 4 Bit and RAM is the most important point to run good LLM, so the 50xx cards have a larg
Given the scarcity of RAM... (Score:2)
...reported here [slashdot.org], the fact that it runs on your computer does not mean that it will answer your questions....
Re: (Score:2)
appointments (Score:2)
But can/will it hack the scheduling system at the doctor's office and get me the appointment I need? I mean the one I really want.
Zuck Wakes Up (Score:5, Interesting)
Zuckerberg has finally realized his shop doesn't have the AI muscle to compete with the top dogs, so instead he's emulating the Chinese model of releasing open models in an effort to undercut OpenAI, Anthropic, etc.
Re: (Score:2)
Maybe Dario was mean to him.
Re: (Score:2)
- Widespread use will expose the vulnerabilities/inadequacies of the system,
- Widespread use will encourage hackers/tinkerers to solve those vulnerabilities/inadequacies,
- An attempt to gain a large market share that will encourage corporate purchases,
- An attempt to force vendor lock-in to the Meta ecosystem, that can monetize all users
Re: (Score:2)
It's hard to get lock-in effects by releasing a model. I can load this one today and Google's one tomorrow.
But they certainly will benefit from optimizations the community does (fitting a model into the VRAM of a smaller card leads to a lot of cool optimizations), and they can find talented people to hire. If someone optimized your model to need 8 GB VRAM instead of 24, they might be able to optimize your large model in a similar way even when its memory requirements are two orders of magnitude higher.
Re: (Score:2)
Meta has always been releasing open models (Llama was famously the first powerful open model). The change has actually been in the opposite direction, with Muse Spark *not* being open.
Zuck descrbed his motivation way back when, about how they got burned with Facebook on app stores, in that Apple and Google could basically bully them however they wanted, on whatever extractive terms they wanted, and there was nothing Meta could do about it. He's now paranoid about "others controlling the platform", and wan
intrests (Score:2)
ok zuck, tell us how they can use the meta pedo glasses to use this powerful new capability to pursue their interests,
Re: (Score:2)
'24 GB or 32 GB envelope' (Score:3)
Why is there no AI cube that has a general purpose CPU, plenty of ram and a GPU with 64GB+ that you can just plop on your intranet? Or is that simply not cost competitive to a cloud AI in your neighbor's back yard that has debt pouring out the wazoo and heavily discounted by the local government.
Re: (Score:2)
Strix Halo. That thing has 128 GB of RAM and fast interconnect if you buy two which is enough to run the deepseek flash model. You're currently at 10k for two, though. So if you like to tinker or want privacy that's an option. If you like it cheap you can get a lot of cloud usage for the money.
Re: (Score:2)
Not sure with the interconnect, I think I mixed it up with DGX spark. Both are interesting, but they have differences.
Re: (Score:3)
There are plenty of those. Mac Minis were hard to get for a while because the shared memory architecture is really good for inferencing, for running a model like this.
nvidia calls theirs spark, AMD calls theirs the "Halo Developer Platform" but both are just a small box with a fair bit of CPU and 128 gigs of shared memory with the GPU.
Re: (Score:2)
There are a bunch of them. People were chasing after Mac minis, which are almost cubes, just a little squashed.
Re: (Score:2)
There are plenty. They are called Mac Mini.
A hated fruit company sells them.
Actually: pretty cheap.
Re: (Score:2)
Re: (Score:2)
What you want is a DGX Spark.
It costs $4000.
And yes, there are fundamental advantages to cloud services, such as large-scale batching, little idle downtime, high speed, and hardware optimized to the specific models / serving needs. That said, one can weigh that off against sovereign control over your server...
This is a good thing (Score:2)
Local control is good
The cloud is a trap
Hopefully, affordable hardware will be available in the future and the need to use the cloud will diminish
Is this supposed to be new? (Score:1)
Re: (Score:2)
1) 30B, dense. Optimized to fit Q4 quantized in a 24GB card with a speculative decoding model as well.
2) Because reporters don't know what weights are and assume you don't know either.
3) Way better than Gemma 4 on text tasks, slightly better on multimodal. Numbers below are all: Benchmark: Muse Glimmer score Gemma 4 31B score difference
Artificial Analysis Intelligence Index: Muse Glimmer: 35 30 +5
MCP Atlas (Public): 75.5 54.2 +21.3
DeepSearch QA: 74.6 61.7 +12.9
SWE-Bench Pro: 51.2
What? (Score:2)
Re: (Score:2)
Download a Meta AI on my PC so it can do what?
Scheduling and file management — apparently, kids these days need a heavy AI box for basic computing tasks we've been doing for over half a century. I keep seeing quite a lot of "AI solutions" to other basic automation tasks, probably because the typical Windows/Mac user has no idea how to automate things beyond interactive clicking.
Make AI Open-Source, Individually Ran, and Yours (Score:2)
Obligatory Silicon Valley (Score:2)
CEO Mark Zuckerberg said in an essay accompanying Muse Glimmer's release. "This has the potential to begin a new era of personal empowerment where individuals can use this powerful new capability to reach their full potential, pursue their interests, and improve their lives and the world more than ever before."
"...but, most importantly... we're making the world a better place... through slimmed-down, open-weight AI models."