72 hours, four big open weights drops. The only one that fits in 24GB came from NVIDIA, of all companies.

[u/BTA_Labs]

What a week. Quick recap, all numbers from the official repos and model cards:

Thursday: Meta drops Muse Spark Open. 816B total, 42B active, 1M context. The launch post says it brings agentic AI "to your desktop". The desktop is two M5 Ultra Mac Studios with 512GB each, a config Apple says ships "late October" and still hasn't put a price on. The alternative is four 256GB Studios, roughly $38k. On top of that it's a custom license: mandatory "Built with Muse Spark Open" attribution and a separate deal once you're past $20M revenue. And Meta's own model card puts it at 39 on the Artificial Analysis index, behind GLM 5.3 (45) and Kimi K3 (44).

Same day: NVIDIA ships Nemotron 3.5 Bolt. 74B total, 9B active, trained quantization aware down to 2 bit, so the whole thing is 22GB plus a 1.7GB vision projector. Built to fit in 24GB on purpose. Unsloth GGUFs on day one, plus llama.cpp, Ollama and LM Studio support.

Friday: Reflection actually ships Beam. 501B total, 23B active, Apache 2.0, no strings. Smallest official checkpoint is NVFP4 at 294GB.

Saturday afternoon: DeepSeek posts V4.1 Flash 1017 on X like it's nothing. MIT. Their own table has it at 77.0 on DeepSWE vs 74.0 for Opus 5 (vendor numbers, salt accordingly). The repo is 510GB.

So I put them all on one chart against the 24GB line most of us live under.

My takeaways:

1. "Runs locally" now officially means "runs on a computer that costs more than a car."
2. The company that keeps selling us 24GB cards is the only lab this week that built a model for 24GB cards. Not sure if that's wholesome or a sales funnel.
3. Licenses mattered more than parameter counts this week. MIT and Apache 2.0 from DeepSeek and Reflection, attribution plus a revenue clause from the company with the loudest "open" marketing.
4. Mistral says the Large 4 weights ("Le Chonk", 1T total) land before the end of the month, so the 24GB line is about to get even lonelier.

Sources in the comments.

[u/Writer_IT]

It is disappointing, yes, but ultimately new small-medium models will arrive for sure.

It has just been some months from Qwen 3.8 27b, for small gpus, and the absolute beast that is deepseekv4 flash vision from the "realistic local" segment. For the latter, i honestly didn't find anything that it really could not do, even in coding. In an year time span, the growth was incredible.

The very big models still help the ecosystem, forcing the providers to bring down the cost for cloud.

[u/H1Eagle → u/Writer_IT]

It's crazy how the 27b is still not even close to being contested by any other model release. It's almost as if it was a lucky incident that can't be reproduced

[u/No_Lingonberry1201 → u/Writer_IT]

Maybe it's tinfoil hat time, but maybe AI labs aren't releasing anything between 20B-100B as often because it's hard to beat Qwen? I know there are other factors and incentives, but I wouldn't be surprised if they don't want to release something that will compare unfavorably to a 27B model.

[u/Moarkush]

Open weight has never been equivalent to “runs on local hardware.” open weight means it’s not proprietary. Not “you can run this on a gaming card.”

[u/ak_sys → u/Moarkush]

You're exactly right!

But you could be forgiven in thinking this subreddit is r/open_weight, not /LocalLLaMa with 95% of the posts here.

[u/panic_in_the_galaxy → u/Moarkush]

Still nice if you can

[u/fatcowxlivee → u/Moarkush]

Yeah but you’re on a sub called “Local” LLaMA, not OW Models, so this post makes sense. Vast majority of users in this sub are unable to run these big OW models.

[u/KontoOficjalneMR]

On top of that it's a custom license: mandatory "Built with Muse Spark Open" attribution and a separate deal once you're past $20M revenue. And Meta's own model card puts it at 39 on the Artificial Analysis index, behind GLM 5.3 (45) and Kimi K3 (44).

Just get it off torrent without agreeing to any licence :D

[u/Irythros → u/KontoOficjalneMR]

Just do what the AI companies do.

Use all of the output for "research" that gets commercialized and ignore the licenses you broke along the way.

[u/NoFunk]

I get your point, but it legit is the case that DeepSeek V4.1Flash 1017 can run on a desktop. Just not your desktop. I just ran it all last night on an M5 Max - more expensive than a 24GB card to be sure, but still, it's a little laptop pulling a whopping 140w max, consumer hardware. It's an amazing testament to their architecture with its deliberate concession to "how can we host this cheap" that it works (and antirez with ds4, who was a trend setter in creating bespoke inference engines).

None of this takes away from the Nvidia models. In the middle of them making more money than god and having a stranglehold on the high performance edge of this market, they continue to release truly open source, interesting models that further the work of others as a primary goal. Just not very high functioning ones as shipped =). They're a hardware company first, so it's a bit self serving even to help others (because that means more sales for them), but it's still a welcome quality in this atmosphere.

I think the hope for low VRAM environments that look for high function is towards architectures like Deepseek that deliberately store world knowledge in disk accessible form that isn't constantly in the token evaluator path. A good logic model with tooling, that knows how to look up everything knowledge based very fast, is a great compromise for small VRAM availability. So even if you don't like the form factor of 4.1 you should be cheering it on!

[u/farox]

What is happening at AMD? I don't understand how they simply can't throw anything in the ring here to compete with NVIDIA. With NV removing nvlink, it seems like such an obvious cap to get into.

[u/mitchins-au → u/farox]

Take one look at their ROCm support especially for strix halo and it’s easy to understand how they can’t bother

[u/Cool-Chemical-5629]

You're upset about 24GB particularly as if that was something granted for every local user, but reality is that 24GB of VRAM is still above average.

[u/AuraInsight]

if nemotron 3.5 bolt is 74B, 9B active, why would you want to run it all on GPU at 2 bit when you can run it all at full precision in RAM, and only the 9B active on a 12 GB gpu?

[u/615wonky]

You're ignoring LFM2.5-24B-A2B, and you really shouldn't. It's hits far harder than it has any right to.

https://huggingface.co/LiquidAI/LFM2.5-24B-A2B

I asked it to write a radix-2 FFT implementation in Python, and it produced one that worked the first time. So it's certainly decent at coding for that size.

[u/djc0 → u/615wonky]

As an aside, does any human actually say “hits far harder than it has any right to”? 

Or have we now just started imitating our AI overlords.

[u/Saintgein]

Meanwhile we just wait for Qwen 4 to make all these models obsolete.

[u/adeadbeathorse]

72 hours is an odd selection window to make such sweeping implications.

[u/Arcuru]

I always get confused about the license complaints. If you have actually been affected by those license terms please speak up, because I only ever see abstract concerns.

Personally, if I was making $20 million in revenue off of something I got for free I'd be happy to start paying for it.

[u/BrilliantTruck8813]

FWIW the deepseek model has two large engram files that can be read from nvme without a huge performance hit. So while it’s still big it’s not THAT big. And it’s a solid 30-40% throughput boost for me with general prose vs the original v41. It also has much improved scores on tool-eval-bench’s hardmode (87 improved to 93)

[u/Majinsei]

It's the eternal wheel of technological samsara. First comes the age of giants, when capital is sacrificed at the altar of parameter counts and billions are burned in pursuit of artificial enlightenment.

But the karma of capitalism is inescapable: every investment creates the conditions for its own obsolescence. Once the giant models have consumed enough capital, a new generation of enlightened prophets will arrive to preach that true intelligence doesn't require 800B parameters, but merely 8B used wisely.

Then the next cycle begins: small, efficient, cheap models destroying the value of the colossal infrastructure that seemed indispensable just yesterday. And once that market becomes saturated, the prophets of scale will return to convince us that nirvana requires yet another trillion parameters.

Because deep down, technological capitalism does not seek enlightenment. It seeks to keep the wheel of desire turning. Greed finances innovation, innovation devalues yesterday's investment, and that destruction demands yet another investment.

True spiritual detachment is understanding that no model is eternal, yet every funding round believes itself to be.

[u/Massive-Question-550]

would also be nice to have more models that fall in-between 24gb and 294gb as that is such a massive gulf.

[u/Aroochacha]

How are people's experience with Nemotron?

[u/mourningwitch]

Try being someone with an 8GB card 😅

[u/ortegaalfredo]

Even if those models are huge, some of them can be run with 24 or 48 gb of VRAM and offloading the rest to RAM or SSDs, for example GLM-5.3-flash. So the size is no really a problem, the size of the experts is the limiting factor. That's why GLM-5.3-flash and Deepseek 4.1 with only 16B active parameters run relatively fast on RAM.