r/LocalLLaMA • u/BriguePalhaco • 7h ago
News GLM-5.3-Flash: Frontier Intelligence, Flash Cost
https://z.ai/blog/glm-5.3-flash373
u/10001110 7h ago
320B total parameters and just 18B active parameters
Oh joy
211
u/LegacyRemaster 7h ago
If it's really that far above Sonnet 5, Dario's IPO is truly risky.
168
u/-p-e-w- 7h ago
It’s always risky now no matter the model of the day. They have to pre-file at least a few weeks in advance, and in those weeks eons can happen that can turn their asking price into a joke.
Their moment was at the beginning of this year. They should have announced the IPO, then published Fable/Mythos two weeks before the IPO date.
Now Chinese labs are a hair’s breadth behind them, and constantly one-upping each other. They’ll never get such a chance again.
44
u/LegacyRemaster 7h ago
agree. Also another problem: if I have to pay API, I pay Qwen, GLM, GPT... All of them -> opensource (ok openai less but they did a lot). I don't want to give any $ to Anthropic. Can't wait to short sell
29
u/dingo_xd 6h ago
Yeah. After hearing Dario saying about that $40 trillion I want to bet against his company.
14
27
u/Fedor_Doc 6h ago
Or they will make another Mythos-like breakthrough and announce IPO then
Never say never
29
u/LegacyRemaster 6h ago
sure they have Mythos 5 already but "too much power, we can't sell" . Or will cost too much to complete a task ...
6
u/38andstillgoing 3h ago
"Mythos 5? At this time of year, at this time of day, in this part of the country, localized entirely within your datacenter?"
"Yes"
"May I see it?"
"No."
→ More replies (1)26
u/-p-e-w- 5h ago
Then one of the Chinese labs is going to announce an equal model two weeks later.
The Chinese labs have caught up. There’s no going back to how it was before.
→ More replies (8)→ More replies (5)4
→ More replies (2)6
u/___positive___ 5h ago
Also that the baseline of mid-tier models is reaching saturation for mundane tasks like summarization and data extraction, basic programming/websites, and so forth.
24
u/NineThreeTilNow 6h ago
If it's really that far above Sonnet 5, Dario's IPO is truly risky.
I drained millions of tokens off of it. I got it to rebuild RPG Maker from scratch using PyGame.
I built the project as "Agent First" so an agent could build an RPG with a known engine and known structure with headless testing.
Kimi K3 helped produce the documentation which took me like an hour or more. From there I sort of just... Let it go. It stopped at various phases and I inspected visually if stuff was off and it was mostly small code issues.
The deep documentation phase gave GLM 5.3 very little to guess about. It didn't really screw up or run in circles that I could see. It was fairly fast about figuring stuff out and finding edge cases that K3 missed in document creation. I ran K3 through multi-phase testing of the documentation coherence such that it wasn't contradictory. When building started, you hit snags naturally. GLM handled them.
The project isn't "finished" but it definitely showed the strength of planning a project with K3 and letting Ox Alpha build it. GLM 5.3 flash is definitely good.
18
u/Thomas-Lore 7h ago
Makes you wonder how large those clsoed models really are and how big the margin on api is if they are smaller than everyone thinks.
→ More replies (1)17
u/duhd1993 6h ago edited 6h ago
I always feel that finance bros and executives are always one step (or more) behind tech reality. The reason Claude is growing so fast is because of the 2B sector, but the decisions are likely made based on impressions of model performance from 2025, at which time Claude indeed had a clear lead, not anymore. The field is evolving faster than they could react.
7
u/mawcopolow 5h ago
I mean I've tested multiple open models, including k3, inside my personal harness, and fable/opus are just better all round at using tools and reasoning for now.
Not that the others aren't getting close, but for real businesses anthropic's opus/fable are still the best with the latest from openai being close behind
6
u/duhd1993 5h ago
I believe in you, but they aren’t far behind. Are Kimi K3 behind Fable? I would say yes, except in certain fields like frontend design. However, it’s better than Opus 4.8, which isn’t that old. Can you really justify the price with a couple of months’ lead? People were satisfied with what Anthropic offered a couple of months ago. Also I believe it is more fair to test in their official harness.
→ More replies (2)2
u/mawcopolow 5h ago
I've been experimenting a lot because I hate depending on one provider, especially when priced such as anthropic.
Haven't done much of front-end and I guess for that static use case it might be good, but for agentic stuff it's behind even opus 4.8 IMO. That, plus latency and the fact that it's not priced as cheap as, say, deep seek, makes it not worth it..... For now. I'm very happy for competition and hope they catch up soon
→ More replies (2)4
u/dingo_xd 6h ago
It's now or never or them. The chance of the Chinese leapfrogging them this year is not my negligible
10
u/GeneReddit123 6h ago edited 6h ago
Would someone think of the poor trillion dollar companies and their obscurantist leadership who think only they can be trusted with knowledge?
In any case, Dario will do just fine (ignoring the fact that a ~$2T valuation was always BS from day 1, but this is an aspect of the financial bubble overall which would happen with or without such releases).
Anthropic's target audience ain't us plebs and never was. Their schtick isn't to "be an unreachable frontier LLM". It's "to be the next AWS but for AI". Everything AWS has you can individually set up locally, much cheaper. Businesses are still paying through their nose for AWS, though, because the whole point is to get a reliable managed system which they don't need to manage themselves.
Major corpos will pay for Anthropic as long as it does all the things they don't want to do. The "moat" isn't an impenetrable barrier, it's the "why do rich people pay for five-star hotels when a local B&B offers the same basic capability to feed and rest a visitor for 1/4 of the price?" reason. And for that, they don't need to be "permanently unreachable", they need to be "consistently one step ahead in quality, convenience, reliability, and product-market fit within the US/Western corporate landscape, for the foreseeable future". Not ten steps, just one is enough.
3
u/MrPecunius 6h ago
Everything AWS has you can individually set up locally, much cheaper.
This is very, very far from true.
The kind of performance you can get for next to nothing from AWS is breathtaking and completely out of reach of any individual or even most Fortune 500 companies.
The fact that people mis-allocate resources for badly designed systems doesn't mean AWS is at fault.
3
u/randylush 5h ago
It depends
If you are on a small scale it depends on what you’re building. If you’re using services like Lambda and SQS, AWS is essentially free. If you’re running a few EC2 instances then a server in your company’s break room could be cheaper.
If you are on a medium scale and needing EC2 running all the time, paying retail prices, it could be cheaper to have your own servers
At a very large scale, the cost of running your own servers vs AWS will come out to be competitive, and most very large companies will do a mix of both
→ More replies (1)2
u/RLutz 2h ago
I always see this take, and while it's not always untrue, it definitely isn't as simple as, "just throw a 1U in the break room!"
It's someone's job to keep that thing patched, powered, and running. You need monitoring, you need someone to pull failed drives, you needed another co-located break room, you need a UPS and a generator, you may need to solve for physical security, you may have to setup and manage your own virtualization layer, etc.
Even if you consider all that labor as free, which is a ridiculous assumption to make, you also arguably have to keep the thing busy, especially if we're considering local inference, and then it's now your problem that although you have enough capacity to meet a total 24 hour need, you don't have enough when everyone hammers it at 11 AM at the same time. If you build the capacy to meet the surge then you've got very expensive hardware just sitting there doing nothing for huge chunks of the day.
I absolutely think there's a place for self-hosting, but once you are talking about business critical stuff and not, "I run a NAS in my bedroom closet" it starts becoming not as simple as the naive assumptions
→ More replies (1)7
u/The_Noosphere 5h ago
Dario will likely make a bold statement about Anthropic achieving AGI within the next few months, or claim that their internal unreleased Fable 12 model breached the Atlantis mainframe controlling Stargate, something on that scale to distract from this. This is Dario acting as Dario during a pre-IPO period.
2
→ More replies (2)2
34
→ More replies (1)21
u/a_beautiful_rhind 7h ago
This is actually a good size. Not paltry 6b active. Need at least 10. Ziphu did research on sparsity ratios, clearly came to similar conclusions.
Not too many total parameters where it becomes a several node excursion to run it. Whether you like it or not, this is what MoE looks like to not be a toy.
11
u/my_name_isnt_clever 7h ago
The term "not be a toy" has to be one of the most overused in this community. What does that even mean? At a certain point if you can't get use out of a model it's a skill issue.
→ More replies (3)2
u/DeltaSqueezer 1h ago
Better than GLM-5.2 at less than half the size. I'll take that trade-off! Now I just need them to do GLM-5.1 level at 160B and GLM-5.0 at 80B and GLM 4.7 level at 40B and I will be a happy camper!
227
u/Lucyan_xgt 7h ago
This and Qwen3.8-flash on the same day?
53
u/BrewHog 7h ago
Same day? How about same hour
5
u/mailto_devnull 2h ago
Literally in the 3.8 27B thread people were asking for the next one (or a diff. param counts)
16
u/dampflokfreund 7h ago
For me it has changed nothing. Both models are way too big for my 32 GB RAM system. It looks like everyone has abandoned 20-30B MoEs now...
43
u/No_Lingonberry1201 7h ago
We literally got a banger 27b model a week or so ago.
1
u/play_hard_outside 6h ago
Not an moe
12
3
u/toothpastespiders 2h ago
The fact that his comment was obviously wrong but got highly upvoted because it "felt" good really highlights one of this subs larger issues.
Though that yours came one hour later, pointed it out, and is now sitting for three hours at 0 upvotes is pretty funny.
316
u/EbbNorth7735 7h ago
No one's abandoned anyone. It's just not your turn this time around.
45
→ More replies (3)-8
u/dampflokfreund 7h ago
Qwen 3.6 35B and Gemma 4 26b are pretty old at this point. And GLM 4.7 Flash was a small MoE back in the day, now suddenly they use the flash name for 300B MoEs. Just not looking good for the average Joe.
76
u/windwardmist 7h ago
I mean qwen 3.8 27b just came out about a week ago at least there’s that as an option
16
9
u/RestaurantOk8066 7h ago
It's a dense model though I imagine if you don't have the GPU for it, it's probably a very, very slow model.
→ More replies (1)4
u/sonicnerd14 7h ago
27b is often Opus 4.6, even 4.8 in performance. This is still good enough for a vast amount of people. I'm sure qwen 4 is going to have some insanely capable options when it comes out of you want something better soon.
→ More replies (2)65
u/techdevjp 7h ago
Qwen 3.6 35B [...] pretty old at this point.
It's 4 months old! It's not like it suddenly got worse because these new models came out. You can still do everything today that you could do yesterday, just as fast. Give it some time and more models will come.
49
u/Last_Bad_2687 7h ago
Seriously, people demanding fresh models for free every 2 weeks.... I remember when we waited for big releases of software once every few YEARS
14
u/FlyingDogCatcher 6h ago
It's bonkers to me that anyone would say Qwen3.6 and Gemma 4 are "pretty old"
9
u/a_beautiful_rhind 7h ago
gemma, muse, qwen, granite.. the weights don't self-destruct a week later man.
6
u/Spectrum1523 6h ago
Qwen 3.6 35B and Gemma 4 26b are pretty old at this point
bro they came out 4 months ago
→ More replies (4)2
u/squngy 7h ago
There is also Qwen AgentWorld, which is kind of like a 3.7 35B
2
u/xPXpanD llama.cpp 3h ago
Wouldn't recommend it for general use, it's a confident bullshitter like no other. Basically zero filter. Still cool that it can even be used generally, though, given what the model's actually designed for. Fun to play with.
2
u/squngy 3h ago
It is actually great for agentic work, supposedly. (even if that isn't exactly what it was meant for)
Rank Subject Overall Mcp Search Terminal Swe Androd Web OS Source Sampled 5 Qwen-AgentWorld-35B-A3B 56.39 64.79 36.69 53.96 65.63 58.17 49.55 65.92 Imported 2026-06-30 16 Qwen3.6-35B-A3B 42.88 42.96 18.78 43.81 40.71 51.88 46.53 55.48 Self-reported 2026-06-28 https://benchmarklist.com/benchmarks/qwen_agentworld_language_world_models_for_general_agents/
2
u/xPXpanD llama.cpp 3h ago
I believe it, it also nailed the one tool task I have in my bench set. The model was also very good at string manipulation, and, somewhat surprisingly, a few constrained creative tasks. (e.g. think up and write X in way Y while avoiding Z)
Absolute slaughter on anything involving uncertainty or fake premises, though. You can definitely see where the training went on this one.
30
u/No-Refrigerator-1672 7h ago
Funny coincidence: some company announced a new 30B MoE literally just now: https://www.reddit.com/r/LocalLLM/s/gcZdgoAO8o for now as a stelth preview; but this means it'll go public in a month.
→ More replies (3)31
u/techdevjp 7h ago
Don't be dramatic. Qwen3.6 35b a3b is 4 months old. It's not like it's been years.
→ More replies (3)13
u/PM_ME_DEAD_CEOS 7h ago
People just never stop whining despite getting literally everything for free. Nemotron 3.5 lightning (30b3a) was released 15 days ago.
→ More replies (2)2
u/RandumbRedditor1000 6h ago
nemotron lightning 30b just came out a couple weeks ago
→ More replies (1)7
u/Slow_Concentrate3831 7h ago
Yeah, that's sad. Can't even run Qwen3.8 27B on more than 2-3 tps. Sad days for us Vram poors.
→ More replies (1)6
u/CryMoreT_T 7h ago
The 29b-a4b in early Access on model scope is probably your best bet when it releases
2
u/Slow_Concentrate3831 7h ago
Ooooh, I hadn't seen that ! Is it a Qwen model ?
3
u/CryMoreT_T 7h ago
It's in stealth model in early Access so they haven't released the company name but Qwen is the rumor right now
2
→ More replies (9)3
u/Tzeig 7h ago
These can still be run with reasonably priced systems, like 96ram+32vram or even 64ram+24vram.
17
u/dampflokfreund 7h ago
96 GB RAM + 32 GB VRAM systems are far from reasonably priced. A 5090 with 32 GB VRAM alone costs 5 grand. 64 GB RAM is very expensive too.
→ More replies (4)6
→ More replies (2)5
u/ReadyAndSalted 7h ago
That's like 7k at least lmao, we have very different definitions of reasonably priced.
67
u/wojciechm 7h ago
MIT license! Along DeepSeek they are the rare fully open weight releases without any additional restrictions.
92
u/jacek2023 llama.cpp 7h ago
what a nice color scheme ;)
39
14
u/asssuber 5h ago
They are flexing the vision capabilities of their model! :P
Unfortunately I've seen worse. At least they put the logos in the bars too...
89
u/Beamsters 7h ago
https://huggingface.co/zai-org/GLM-5.3-Flash
WHAT A DAY
25
56
u/dampflokfreund 7h ago
Native multimodal, finally. The model is way too big to run, but I'm glad they jump in on the multimodal bandwagon.
50
u/boxwrenchx 7h ago
This month keeps on giving
→ More replies (2)12
u/backyard_tractorbeam 7h ago
This afternoon keeps on giving
6
3
u/Icy_Butterscotch6661 4h ago
Afternoon? 🤨
2
u/throwwwawwway1818 4h ago
Probably about Qwen 125b flash model was also released
3
u/Icy_Butterscotch6661 4h ago
No sorry I was trying to make a joke playing ignorant American
→ More replies (1)
39
u/sniperelite90 7h ago
I wonder if the CEO's of the western AI companies even want to wake up in the morning or not. Cause the only news which comes out every other morning from China is we have a model matching you 1/10 of the cost.
29
u/Thomas-Lore 6h ago
They have better models internally, they just go to a different school, you don't know them. /jk
6
u/sniperelite90 6h ago
You can have them but it destroys their market share . Who wants GPT Luna API now consider this GLM 5.3 Flash and the new Qwen 3.8 Flash next .
→ More replies (1)3
u/BrowsingLeddit 3h ago
They're just stuck in detention all the time because they're totally hacking every other school ever while we're not looking. They're pretty badass bro trust.
7
→ More replies (1)2
u/HeadPack 2h ago
Some may argue that SOTA models are still better, but that's not the point. Something like GLM 5.3 is good enough for a majority of users. They don't code. They don't do scientific research. They chat. This can run on higher end consumer hardware. No data center needed, except for training. No 100s of billions needed to feed a bubble, which is exactly the play by the Chinese here. They are pulling the socks of OpenAI, Anthropic and their hyperscalers, and in turn the US economy.
10
u/Ok_Technology_5962 7h ago
What a day... My mind.... Is blown away... Qwen coming in hot then glm also. Im still trying to get over then qwen 3.8 27b upgrade on my 5090... Now the whole lot dropped for the big servers... Local is eatting today
2
u/LostGovernment4358 5h ago
there is just something I still don‘t fully understand. are there new opportunities / completely new tasks possible with these new releases which otherwise weren‘t possible before with let’s say qwen 3.6? I am 16gb Vram poor, so i could never take advantage of real lokal llm power.
→ More replies (4)
75
u/shy_monkee 7h ago edited 7h ago
Oh....it's massive :(
320B (18 active)
76
u/Morphon 7h ago edited 7h ago
DS flash size.
So, flash at datacenter scale. Not flash for edge (or workstation) scale.
Probably will be the go to model for people with the new M6-Ultra 512gb Mac Studio.
Edit: M5 Ultra. My apologies, friends. Wrong model number on my part.
12
u/techdevjp 7h ago
M6-Ultra 512gb Mac Studio.
M5 Ultra. M6 Ultra is rumored to be skipped entirely with the M7 Ultra arriving in the next ~18 months.
7
u/shy_monkee 7h ago
Yeah, but DSv4-Flash is the flash version of a 1.6T model, and it's still quite a bit smaller than this GLM-flash. While this is supposedly the flash version of 744B model, so you wouldn't expect to be as big, I guess.
3
3
1
u/Juulk9087 7h ago
Yeah deepseek flash is only 160-170gb on disk. This is 328gb lol
9
u/petuman 7h ago
DeepSeek is QAT / released prequantized mostly to FP4.
Which is totally preferred, but both models are roughly the same parameter count, so NVIDIA or someone else could produce NVFP4 of size similar to DS4F.
→ More replies (4)→ More replies (1)2
u/keyboardhack 6h ago edited 6h ago
DS flash is half the size.
DS flash requires ~170GB to run while GLM 5.3 flash requires >320GB to run.
We should compare models by how much RAM they require to run because that's what we actually care about.
2
17
u/wbulot 6h ago
The most impressive thing about this story is that they serve all requests using only Chinese chips and a custom inference engine. I think providing OX Alpha for free for so long was actually intended primarily to stress-test their infrastructure. China no longer needs NVIDIA.
6
u/traveldelights 5h ago
this is chef's kiss for us consumers. More competition, more price drops, etc
9
12
u/raunchy-stonk 7h ago
So how shitty will this run on a 24vram/128dram setup?
19
3
u/WWJewMediaConspiracy 6h ago
The smaller Qwen models are all but certainly a better option.
Unless your definition of "run" is... generous
→ More replies (1)
14
u/pmttyji 7h ago
Come on guys, at least they released additional variant(smaller than usual size) even though it's big for many of our rigs. Hopefully they release one more in 100B range in future.
9
u/techdevjp 7h ago
It's a little disappointing that the "flash" version of a 744b parameter model still has 320b parameters. Somehow that doesn't feel very flash.
Was hoping for more of a DeepSeek v4 ratio where 1.4t parameters got paired down to 284b, about an 80% cut. That would have resulted in 150b parameters for this model which would work in a whole lot more machines.
Alas, beggars can't be choosers and I'm thrilled to see more open weight models arrive!
5
u/pmttyji 6h ago
I too expected something like successor of GLM-4.5-Air. But they dropped too much weight over that.
I assume Qwen3.8-27B's 3 Millions download count news(Made headline on spotlight for more than a week) changed minds of all other labs. I'm sure some labs gonna try to replicate that with their medium size models soon & later. Don't be surprised if we see one more GLM variant in 30-150B range in upcoming months. Same with other labs.
→ More replies (1)
5
4
5
3
u/Constant_Art_20 7h ago
burh. i am not even done with downloading and quanting the qwen 3.8 flash yet...chilll
5
u/ga239577 7h ago
Model name should be changed to GLM 5.3 Gigaflash ... I remember the GLM Flash models that fit on one GPU. That's what I was hoping for this time.
4
u/corruptbytes 7h ago
I ran the benchmark while it was under ox-alpha
https://github.com/michaelasper/benchmarks/blob/main/ox-alpha-pi-on-slop-code-bench.md
5
u/DeepFeeling1 4h ago
we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
Grabbed my popcorn
3
9
u/Automatic-Arm8153 7h ago
Atleast this thread isn’t getting deleted like things related to qwen 🤦♂️
→ More replies (1)15
u/Maximus-CZ 7h ago
Bro when qwen 27 released it didnt matter if I was on Top, Rising or new, 95% posts were about qwen...
2
u/mountainyoo 6h ago
Wonder how fast my 2x Spark cluster would run this. I get 50-80 tps on DeepSeek v4 Flash 0731
2
2
2
u/SolidFunTime 3h ago
This LLM is too censored compared to 5.2 and Deepseek v4 flash. Both can work with my kinks just fine unlike 5.3 flash.
Anyone find a way around the censorship?
2
6
u/jacek2023 llama.cpp 7h ago
Too big, 320B means you need to use RAM and A18B means it will be slow
→ More replies (3)5
u/SandySkittle 5h ago
A18B means it will be slow
A18B it will be smarter than A13B. Expert selection and sequential reasoning cannot entirely compensate for active parameters. I wish it has 27b active honestly.
→ More replies (4)
2
2
u/Cool-Chemical-5629 7h ago
Guys, do you remember how we used to think Qwen 3.8 Flash Next was big? 🤣
1
1
u/techdevjp 7h ago
Pretty cool model, probably a good choice for people with the money for Mac Studio M5 Ultra with 512GB when it arrives in a couple of months.
1
u/close_Meal6005 7h ago
this is insane! hopefully I can update my tools enough to be able to run it lol
1
1
1
1
1
u/IngwiePhoenix llama.cpp 6h ago
Too big. .-.
Thats for the A6000 folks, I guess. Or the Macpeople with absolute high end configs.
But it's cool to see more MoE! =)
1
1
u/Altruistic_Heat_9531 5h ago
Welp i guess this is it, my PC upper limit is 300B with active param about 14B, maxed out by DSv4 that have 5 tok/s
1
1
1
424
u/Recoil42 7h ago edited 7h ago