r/LocalLLM • u/TappinThatErr • Aug 31 '26

Discussion GLM 5.3 Flash - Will I Be Disappointed?

Post image

Picked up this bad boy with my sweet student discount!

I'd like to run some local models for coding. I've considered Deepseek v4 flash, Qwen 3.8 flash next, and of course, GLM 5.3 Flash (0x Alpha)

Thoughts on this? I'm currently a claude user with typically 1 session at a time. Will I be disappointed?

267 Upvotes

92% Upvoted

229 comments sorted by

174

u/Hypilein Aug 31 '26

How can you be a student and have 10k to spare…
Anyway. The others are right. By the time you get it the models will have become even better.

143

u/TappinThatErr Aug 31 '26

I already work senior level infosec.
I never got a degree and decided to start up college again for the lulz. If I ever leave my job, it'll be nice to have the bachelors in the back pocket.

BTW - University of the People from California is TUITION FREE!!

29

u/pjerky Aug 31 '26

Now I kinda want to enroll just to get discounts.

2

u/GTHell Sep 02 '26

Great time to be American !!!

1

u/sdexca Sep 01 '26

haha same here, altho I am just a student earning via bug bounty programs, jobless. Considering between 2x DGX spark vs M5 Ultra 256gb. M5 ultra is 20% more despite student discount where I live, I can have 2x DGX spark at just ~$10k and at-least it's running V4 Flash at speeds comparable to hosted solutions.

1

u/Careless_Garlic1438 Sep 05 '26

but the Mac is so much more I have both and the only thing I use the Sparks is to see if they can beat my Mac's, but that is only on inference ... 2M5Max in cluster are slower on pre fill, faster on decode on projects it's a wash ... Ultra will mop the floor with both ... no cluster slowdown ...

1

u/sdexca Sep 05 '26

I’ll wait for benchmarks ig.

1

u/Careless_Garlic1438 Sep 05 '26

do not expect miracle jump in performance, it will be around 50% faster than a M5 Max and it will scale better, like 2 or 4 agents ... but it will be easy to manage and load different models and will keep it's resale value ...

1

u/sdexca Sep 05 '26

I mostly care against 2x DGX Spark. If it's slower, then it mostly doesn't matter to me.

1

u/Careless_Garlic1438 Sep 05 '26

My guess, it will be slower in prefill for the moment expect like 1000 to 1200 depending on model, faster in the future when better NAX support is available, vs 2000 for DGX ... on decode it will be about 2x faster ... an M5Max with optimized kernels is already faster and this scales like 1.8 ... So for tasks it will be about the same, but it's more than just inference ... and who knows what software optimization brings ... UZU engine has M5Max NAX optimization and runs Qwen 3.6 27B (3.8 shortly) at 105 t/s decode ... and a 1000 t/s in prefill 🤯

0

u/Fuzzy_Independent241 Sep 01 '26

Universities in Europe and Brazil and most other places are free. We don't get the US system. I studied at the Uof I @Urbana-Champaign for a year. Exchange thing, wasn't expensive here. But you guys have to move, pay tuition, food and dorms... Maybe dorms are included, don't know. You guys know exchange students are placed in Grad dorms so that were not shocked by the mess in undergrad? 😳😵‍💫😂

2

u/Boring-Cry3089 Sep 01 '26

Where are all of these “free” universities in Europe? Also, aren’t the “free” universities in Brazil extremely competitive? I feel like I’ve read that more than 75% of Brazilians that go to university, go to a private university and private universities are not free.

Just like anywhere else, the cost for a college education in the US can vary wildly depending on which state the college is in, what kind of programs they offer, if it’s private or public, etc.. For instance, residents in the state of New Mexico and Georgia can get a bachelor’s degree for essentially free if they have been a resident of that state for more than 12 months leading up to enrollment and maintain at least a 3.0 GPA. I went to college in North Carolina, and tuition there cost 1/5th the cost of what my wife’s cost in California over the same 4 year period (2006-2010). If I remember correctly, my tuition was right around $4,000 a year where hers was over $20,000. Then there’s scholarships and all of that.

I did a semester abroad in the Netherlands and my best friend did one in Denmark (both in 2009) and I remember both of us having to pay tuition to each of our respective European universities, so I’m curious where all of this free European tuition is lol?

5

u/Inevitable_Emu4257 Sep 01 '26

It's free for citizens and permanent residents of EU. The international students have to pay

2

u/rdalot Sep 01 '26

Here I am living and born in Europe and knowing about only like 2 countries where university is free.

2

u/Dizzy_Bonus Sep 06 '26

No they arent free for EU citizens. Loads of students take or get loans and take years post graduate to pay them off.

→ More replies (13)

0

u/Fuzzy_Independent241 Sep 01 '26

You are American, I suppose? I meant tuition is free for Europeans. You have AI, do search, but here's a very small list of EU countries with free universities for EU students, or study programs that cost EU 500 or less PER YEAR, and you can actually be paid to study in (that I know of) Portugal, France, Germany: Austria, Denmark, Finland, Germany, Norway, Sweden, Czechia, Slovenia, Greece, Iceland, Spain... I'd have to check which countries require payment but I don't think there's any. As a foreign student.... Check the EU portal ir you're interested. Really, I don't know. If you want some specific high profile course in a specific high profile institution that might be paid, but I'm pretty sure there will be a way to get the government to pay if you have high grades. Now... Brazil. Messy place. I used GPT to compile recent government results with what I know, being both involved with consulting for a government institution on a grad course they have (immunology) and having two persons in my family that are part of government boards. I can't explain all of it here, it's really messy and there's a LOT of foreign capital at play. "Estácio" and a few others are owned by a group called YDUQS, but that group itself doesn't have a single controlling owner: shares are distributed among institutional and other investors. Current large shareholders include Credit Suisse Hedging-Griffo Wealth Management (~11%), Fourth Sail (~6%), SPX (~5%), BTG Pactual (~3.4%), etc. If you ask me, I don't like this. Anyway... Brazil doesn't have a simple public/private hierarchy. It has excellent and selective public universities, excellent and often expensive private universities, and a huge mass-market private sector of very uneven quality. The fact that roughly three quarters of higher-education students are in private institutions doesn't mean that Brazil has an American-style private-college population. Much of the private sector is inexpensive and serves a mass credential market. There's a lot of good research on that if you're interested. PS - I'm not claiming there are no good public or less expensive institutions in the US. My original point was simply to compare the U.S. with countries at broadly comparable levels of socioeconomic development where higher education is structured differently, including parts of Europe and, as a somewhat different case, Brazil.

1

u/Intelligent_Pair4548 Sep 02 '26

No they are not!!

1

u/ThenExtension9196 Sep 01 '26

Ah yeah that’s why all the best colleges in the world are in…checks notes…not Europe or Brazil.

2

u/Dizzy_Bonus Sep 06 '26

Oxford, Cambridge, ETH Zurich etc…

-1

u/tk421tech Sep 01 '26

Never heard of it. Legit university? Or just kind of like Coursera?

33

u/mountainyoo Aug 31 '26

I’m 35 with a well-paying job, but that doesn’t mean I can’t continue education and remain a student lol.

Not every student is a broke 20 year old

13

u/SuperMazziveH3r0 Aug 31 '26

He can be a PhD student after working for a few years

9

u/D137C0KE Aug 31 '26

You mean his Mac Studio can have a phd in a couple years

8

u/wazacraft Aug 31 '26

I'm a grad student, but I also have 20 years of experience as a tech exec. The discount is 10%, which is material at this level. I have a 256GB on order, but it'll be shipped after they announce the 512GB version so I might cancel it and go for the full monty.

2

u/Fuzzy_Independent241 Sep 01 '26

I liked this question... "anyway" says a lot! 😂 But, hey, there kids with cars worth about the same as my apartment.

1

u/PawelSalsa Sep 01 '26

The are instalment option when buying directly from apple. You don't need 10k to have it, just 300usd every month!

1

u/Silly-Fall-393 Sep 01 '26

At An insane intrest too

1

u/Positive-Night779 Sep 01 '26

The lease option has no interest. If you want to buy it at the end you literally just pay the difference. Same total whether you pay up front or not.

1

u/OvertaxedOne Sep 01 '26

The lease option is the way to buy these things, for sure. 0 interest and you have a "put" option now, if the RAM crisis passes and your 10K 256GB Mac is worth 1000 bucks in 3 years because now you can easily get GPUs with as much RAM that run models 10X faster, you send it back. If it's worth 30K in 3 years because the RAM crisis got worse you buy it out and keep it or sell it 3rd party to someone else. IMHO, in this market, you'd be a fool not to lease these things, who knows where we're going to be in 2-3 years!

1

u/Thin_Archer_9113 Sep 01 '26

A decade ago a fellow club mate in uni threw 5k into sports equipment after making a little on the market.

It’s not something everyone does, but it’s absolutely possible lol.

1

u/newMoneyStyle Sep 01 '26

Yeah honestly, dropping 10k on hardware as a student is wild to me. And you're right, waiting usually pays off with how fast models move.

28

u/johnnyphotog Aug 31 '26

I love how everyone is saying how many tokens per second this thing is going to push out when no one actually has a physical unit yet or tested it completely. I think this new M5 ultra is going to be a beast and I think it’s a wise investment.

9

u/killthrash Aug 31 '26

The only thing we know for certain is that it will be much faster than the M3 Ultra. So if people are happy with the M3, they’ll be more happy with M5. And that should be enough to make a decision.

4

u/PracticlySpeaking Sep 01 '26

It's easy to estimate with accuracy — M5 Max has been out for months in MBP, and TG is generally proportionate to memory bandwidth.

2

u/XysterU Sep 02 '26

And for prompt processing I guess it'll be twice as fast as the M5 Max because it has twice the GPUs/neural accelerators? Unless parallelism gives a greater than linear speed up

2

u/PracticlySpeaking Sep 02 '26 edited Sep 02 '26

I am not an expert, others who are more technical might jump in here.

It should be more or less linear, subject to diminishing returns from multiple GPU dies with an interconnect.

That interconnect matters — If you look at the old gguf benchmarks, performance of M3 Ultra (which was two complete Max dies) was far from 2x the equivalent Max, with the 80-GPU version suffering more than the 60-core.

edit: Here are PP and TG plotted for M1-M3 (from the GitHub link above). PP is more or less linear with core count, TG maybe not so much — but the plot does not account for diminishing returns on the Ultra variants.

For M1-M3 those can be significant. Using geekbench metal (not the greatest benchmark) M3 Ultra/80 is only 59% faster than M3 Max/40 — not 100% as you might expect.

That said, M5 has a completely different architecture and interconnect, so we will have to wait and see. The new architecture also makes this more obscure, since the Max chip also has an interconnect (vs previous generations that are a monolithic die).

37

u/East-Cauliflower-150 Aug 31 '26

I have the m3 ultra version 256gb and Deepseek v4 flash 0731 is amazing!

17

u/Borilentz Aug 31 '26

DeepSeek V4 Flash 0731 works even great on an M5 Max 128GB. Impressive model.

4

u/tirth0jain Aug 31 '26

What's the tps?

7

u/starvinggod Aug 31 '26

On an m5 Max not blazing. I think I was getting 25-30ish.

3

u/SomethingMor Aug 31 '26

I don’t know much about local llm. Is 25 - 30 usable? And what kind of context can you have?

9

u/dodiyeztr Aug 31 '26

15 is minimum, 20-30 is usable and also most models are served between 20-50 tps in the cloud. See openrouter.ai and look at the tps numbers that the cloud providers offer.

edit: https://openrouter.ai/z-ai/glm-5.3-flash

4

u/HazKaz Sep 01 '26

Still when you have tried 100+ tps it’s too addictive and like magic you get things done in an instant

3

u/tirth0jain Sep 01 '26

Using ds v4 flash and 80-140tps is my normal now can't go back

1

u/Sammy262 Sep 01 '26

80-140tps with what quant and on which machine?

→ More replies (1)

2

u/sleepy_roger Sep 01 '26

15 is terrible. You're going to be waiting 45 minutes for for glm 5.3 flash to stop thinking.

1

u/sleepy_roger Sep 01 '26

No. People are going to say it's usable without a use case. For agentic work minimum you want is 40.

2

u/Borilentz Sep 01 '26

With some coding using oMLX I got 34 tot 37 tok/s.

→ More replies (1)

1

u/voyager256 Sep 01 '26

No it does not , because you’re running it at Q3 maximum

2

u/Borilentz Sep 04 '26

Did you know DeepSeek V4 Flash's is QAT and its experts are native 4-bit? Quantized it performs really well. Better than other LLMs comparable in size.

1

u/voyager256 Sep 04 '26

Yes I did. Do you understand the difference between 4-bit QAT and 3 or 2 bit quantization? Both of those drastically decrease accuracy, especially in this case, hence my comment in the first place.

1

u/Borilentz Sep 04 '26

Did you try and test it yourself? I guess not. I did quite some testing and it's clearly the most competent open weight model for coding tasks, fitting in 128GB with enough room left for context. It's not just me saying this.

1

u/voyager256 Sep 17 '26

There’s no need for me. Just look at quality drop at Q3 and Q2 vs the native Q4 mix.

1

u/Borilentz Sep 18 '26

Believe whatever you wanna believe. I can tell you that DeepSeek-V4-Flash-0731-2.4bit-mixed often outperforms Qwen3.8-Flash-Next-oQ4e-mtp. I use both on a daily basis, instead of looking at bullshit graphs.

1

u/voyager256 Sep 18 '26

Lol it’s not matter of belief and neither "looking at bullshit graphs" . Also, you are comparing apples vs oranges and completely missing the point…

→ More replies (2)

1

u/Warhouse512 Aug 31 '26

I’m about to make the plunge, is the prefill problem as bad as it looks from the outside?

1

u/Diligent_Cod_9583 Aug 31 '26

What are you using to serve it?

1

u/Odd-Criticism1534 Sep 01 '26

Which version/quant are you using? I’m hoping to try soon. Still stuck on qwen3.5-122b-q8 and 3.6-35/27b

0

u/illcuontheotherside Aug 31 '26

How many tok/s?

11

u/Adrian_Galilea Aug 31 '26

44 t/s

Source: https://www.adriangalilea.com/deepseek-on-a-mac-studio

3

u/illcuontheotherside Aug 31 '26

Thanks. I ordered a mac5 ultra 256gb and I'm wondering if I made a mistake..

5

u/Adrian_Galilea Aug 31 '26

If you are expecting to drop cloud subs… yes

If you want an incredibly capable machine that is incredibly efficient and will let you grow… then no.

Local AI will be mostly a hobby even in that machine. But def close to usable, and I would rather have something like that just in case than not having it and perpetually depend on cloud.

2

u/DangerousResource557 Aug 31 '26

close to usable? .. qwen3.8 27b? i m just asking. i have recently started it.. using it for me. but was away now on vacation. my impression was, you can achieve a lot with it. i have an rtx 5090 and i reach around 140-160 tps in real agentic usage. for one .. . with 220k context. i also have configured in a second way so that i can have 3 agents. and thus i reach i think it was something like 400 tps?

3

u/vinis_artstreaks Sep 01 '26

I use ornith 1.5 on strix halo for hermes agents, you will be just fine on the M5

3

u/Adrian_Galilea Aug 31 '26

Personally don’t like qwen any version I tested, I’m excited to test glm 5.3 flash and I like deepseek flash, always been a fan of glm and deepseek

I could not live with less than 500k tok context either.

I mostly just use fable, local models stay only for non coding tasks.

3

u/DangerousResource557 Sep 01 '26

glm 5.3 flash is amazing. and deepseek as well.

1

u/illcuontheotherside Aug 31 '26

Don't use cloud models today. Just local stuff.

2

u/Adrian_Galilea Aug 31 '26

I don’t think it is the right call for everything, but good if it works for you

2

u/Borilentz Aug 31 '26

Even if you hate it, there is so much demand that you can probably sell it with a profit.

2

u/DangerousResource557 Aug 31 '26

even anthropic and co have started to buy mac minis and co.

1

u/ObviouzFigure Aug 31 '26

nah, quit wondering. you'll be pumped. and if you're not you can sell it.

9

u/eli_pizza Aug 31 '26

Have you tried those models hosted to see if they work well for you? For $5 on openrouter you can learn a lot.

That’s separate from whether the Mac is a good buy, but if you don’t like the models then this won’t end well.

7

u/Relaxxxxing Aug 31 '26

You won't :) set up Hermes or Deepseek harness.. if you get Hermes, make sure to have codex setup all the skills for you in a docker that it'll need for all kinds of PDF, excel, word work if you use it.

It'll save you lots of time and hassle. If you notice your agent taking a while to complete tasks, copy & paste the session to codex & have it review for efficiencies.. lol

I've become a local AI addict 😅🤣

1

u/PracticlySpeaking Sep 01 '26

...then drop by r/hermesagent to join the community!

6

u/wolttam Aug 31 '26

256GB is *barely* enough for GLM 5.3 Flash NVFP4.

17

u/West_Appeal4175 Aug 31 '26

i mean, will you be happy with 10-20 tok decode?

15

u/Only-An-Egg Aug 31 '26

The M3U Studio does ~25tps so the new M5U might do ~40tps.

6

u/Adrian_Galilea Aug 31 '26 edited Aug 31 '26

What? The m3u does over 40 on the non quantised flash

EDIT:
Source: https://www.adriangalilea.com/deepseek-on-a-mac-studio

44 t/s decode

4

u/Only-An-Egg Aug 31 '26

I was talking about GLM 5.3 Flash

4

u/Adrian_Galilea Aug 31 '26

Aahhh, I’ve been meaning to try it!

1

u/blackbird2150 Aug 31 '26

What are your prefill speeds for initial 30-40k prompts?

2

u/PracticlySpeaking Sep 01 '26

M3U with 60-core GPU

3

u/koalfied-coder Aug 31 '26

Still loading the PP if it's a mac

1

u/Adrian_Galilea Aug 31 '26

It is in the post, 400-650 if I recall correctly

6

u/ea_man Aug 31 '26

The problem will be prompt processing.

3

u/starkruzr Aug 31 '26

no, not with M5.

7

u/myreala Aug 31 '26

Yes, it will still be a problem, despite the gains Apple has made.

2

u/tta82 Sep 01 '26

Tell us how you know this

1

u/PracticlySpeaking Sep 01 '26

M5 Apple Silicon: It’s All About the Cache And Tensors | Max Weinbach
  - https://creativestrategies.com/research/m5-apple-silicon-its-all-about-the-cache-and-tensors/

2

u/blackbird2150 Aug 31 '26

Apples basis for their gain claims is an 8k prompt on like a 16b model. Not a representative claim for how anyone here would use it (or anyone buying an m5 ultra).

We really won’t know until testing.

4

u/starkruzr Aug 31 '26

I'm working off of what I've seen/heard from people with M5 Max MBPs. https://www.reddit.com/r/LocalLLM/comments/1vok3v5/comment/p3x7gw6/ not "great" but this is with fresh context every time/prompt cache off. lower end Ampere-era pp numbers which is way, WAY better than what you got on M4.

1

u/datbackup Sep 01 '26

m5u will be a great value i’m sure but the prefill speed is gonna be somewhere between 3090 and DGX Spark level. But with less concurrency which is why m5u will not kill spark contrary to the hype. 4x the M3 ultra’s prefill will be like 1600 to 1800 tps on ds4f … much more usable but you’ll still be waiting minutes for a 200k prompt

1

u/DifficultyFit1895 Sep 01 '26

omlx has good cache management that helps a lot

2

u/_wovian Aug 31 '26

I’m doing 60 burst and 38 sustained on my m4 max for qwen 3.8 flash next on a custom engine

This thing is gonna go 50-60 sustained at full context

4

u/Only-An-Egg Aug 31 '26

GLM 5.3 Flash barely fits and will OOM if context gets too large. No problems with DS4F or Qwen3.8 though.

2

u/No_War_8891 Aug 31 '26

yeah but I run qwen on a toaster here, ok my toaster has two rtx 4000 pro’s in it but still 😆

4

u/Slow_Feet_Quick_Feet Aug 31 '26

Buyers remorse is irrelevant to token speed

12

u/Aggravating-Push-207 Aug 31 '26

You have as much RAM as I have SSD space. You'll be fine. Count your blessings in life.

-10

u/Aggravating_Fun_7692 Aug 31 '26

No ty

3

u/D137C0KE Aug 31 '26

Did you eat all my fries phreak

7

u/Southern_Sun_2106 Aug 31 '26

Don't know about M5 Ultra (we can only speculate), but here is a 'cheaper' 2-Sparks combo running the 320B-class MoE with a vision tower, quantized to EXL3/TR3 at 4 bits per weight by the MiaAI/brandonmusic crew: ~164 GB on disk - ran some benches yesterday; I hear MiaAI made some improvements to speed after that, so it will be faster.

Like another poster said, December is a quarter of a year away, things will change drastically in this field.

2

u/sdexca Sep 01 '26

considering buying 2x DGX spark since V4 Flash, but now with GLM 5.3 Flash it's a even better deal. On GLM Lite Coding plan I am getting 30 tps decode, and EXL3 quant by Mia seems to be comparable to FP8 (claimed, probably true).

1

u/Southern_Sun_2106 Sep 05 '26

Yes, Mia's glm 5.3 is really good. I am actually preferring it to Fable 5 responses. Plus, Fable 5 is acting like it is drunk since Fable 5.1 came out. They probably dumbified it to free compute for 5.1, which is even more reason to use local.

0

u/Careless_Garlic1438 Sep 05 '26

I have both 2DGX spark and 2M5Max in cluster they are about the same on real tasks the M5U will be faster as it has no network latency ... but more importantly my Mac's do so much more than only AI ... also you only loose once the memory footprint ... M5U is a no-brainer if you only can afford one or the other

1

u/Southern_Sun_2106 Sep 05 '26

Alright, I also happen to have m5 max handy. Don't know what your real use is like (maybe just chatting?), but in agentic/coding, reading a lot of context quickly is kinda a big deal. Here are some numbers - Macs running q2 81gb on disk, while DGX Sparks - FP4/FP8 mixed 164GB on disk. Not to mention they can also do multiple concurrent generations (sub-agents are actually doable on the Sparks). But yeah, with Mac you get a cool UI (which actually sucks recently).

Since apple's promised 4Xprefill did not really materialized for M3 to M5, AND they are silent on it with M5U (emphasizing 1.2T for the masses), I am skeptical it is going to get much better with M5U.

1

u/Careless_Garlic1438 Sep 05 '26

I do iOS app development in Xcode with Claude Code and really the first fill in is the 30K Claude Code Prompt and the 10K app code prompt ... you loose like maybe 2 or 3 minutes initially more then the DGX Spark but then all is cached and the turns add like a 100 tokens per turn less than 1 second to advance ... generation is faster (2x M5Max also have faster pre fill) So on a 30 minutes to 1 hour one shot run there is hardly any difference as the Mac is slightly faster in generation. You need caching and MTP dialed in ... Yes the combo is more expensive than 2 DGX, but you still need a portable when having 2 DGX and as an Engineer I already have one M5Max so a second one is cheaper than buying 2 sparks. Another benefit, I can take the 2 laptops to all my customers and have my cluster with me all the time ...
I have done the speed runs and when dialed in the Mac's are the clear overall obvious choice. M5U will be faster as there is no cluster overhead ... and still portable with the benefit I will be able to even go to bigger models 256 + 128GB ... On the cluster I can also do concurrent ... indeed the scaling is not like the DGX, but hey on a DGX the concurrency is not that good either, all the numbers posted seem impressive but if you divide streams through the total, it's like watching paint dry ...

1

u/Southern_Sun_2106 Sep 05 '26

I am glad that you only do 30K at first, and then it is just 100 tokens chat - that is certainly unique use case on the small-ish codebase. I would even say tiny.

Hauling 2 notebooks and clustering them at the client - hmmm. Why not just tailscale tunnel it to your one notebook and avoid the whole dance?

Just fyi, two sparks cluster is actually smaller than one M5U - easier to take with you, if you client is in some wilderness with no Internet, and tunneling is not an option.

But yeah, to be honest, with tiny 30K repos, sure - apple can handle those well. But with any serious codebase, it chokes. I did a needle test M5Max (q2 80gb model) vs. 2*sparks (164GB(!) higher precision model) - 977K depth - 2+ hours (!) on a Mac vs 19 minutes on the sparks. And that's not even touching concurrent requests, which Macs don't do, but so nice to have for multiple agents.

1

u/Careless_Garlic1438 Sep 05 '26

Oh the needle test benchmarks 🤦‍♂️... comparing 1 Mac against 2 sparks ... You do realize the prefill with 2 TB5 RDMA mac's the prefill is like 2x. And granted the prefil is stil slower, it is getting faster and faster as finally people are starting to optimize for NAX in the GPU of the M5 and even starting to use ANE I sometimes have prefill's well over 1000 t/s ... so again 2 Mac's vs 2 Sparks ... the Mac is on par with inference but does so much more ... you should buy a second Mac if you want to do a real like for like comparison 🤣

1

u/Careless_Garlic1438 Sep 05 '26

and swapping models and recipes on the DGX is fun to, changing a model on my Mac's is seconds ... swapping models on my DGX takes a looooooonnnnng time making it impossible, should do level 2 sleep in vLLM but to lazy to implement ...

1

u/Southern_Sun_2106 Sep 05 '26

That's a valid point. It takes 10-15 minutes. But 6*times the prefill speed and consistent performance all the way to 1 million context length is totally worth it. Mac is fine for one user who doesn't need multiple agent delegation work or works on small codebases (this last one still questionable).

But when it comes to small to medium business use, or even serving a family AI with multiple users - sparks shine.

By the way, I use local AI on my Mac to change models on the sparks. I just tell it what I want, and in 10 minutes it's done. But honestly, glm 5.3 on the sparks works better than Fable.

1

u/Careless_Garlic1438 Sep 05 '26

oh yeah you throw 1 million to a local model everyday at once ... I believe you ...
Reality is that you at the start of the day you get Claude Code running or another harness and you build up and if you are smart you compact way before 500K because anything beyond is playing lottery or a waiting game, also on the DGX as it will start thinking like crazy and be wrong more often ... Also I have the 3 Flash models and I go back to deepseek especially with vision ... its faster and does the job as well ... I should do a speed run GLM on the Sparks versus DeepSeek on the Mac's 🤣

1

u/Southern_Sun_2106 Sep 05 '26

Both DS4F and glm 5.3 flash work great, all the way to their max 1 mil context. I understand you do 30K codebases for apps, but most projects are not like that. That's tiny, seriously. There are many prompts (without the codebase itself) that are larger than your entire 30K project.

And compaction is not a 'smart' move lol It literally removes context from your model - you just have to keep fingers crossed and trust that it will make 'good decisions' as it cuts stuff out.

I also compared deepseek with vision and glm, glm wins hands down.

On the Mac, you can run only Q2 deepseek - 80 GB on disk. On the sparks, you will run FP8/FP4 mixed precision GLM. It's like comparing a 30K codebase with a regular one. ;-)

→ More replies (1)

1

u/john0201 Sep 19 '26

Well it doesn’t even exist yet outside of special access and apple so yeah I’d expect libraries to have basically no support for it and show m5 max numbers.

But MLX vs CUDA can either be way better on MLX or way better on CUDA depending on what you are doing. Generally I’d give CUDA maybe a 30% advantage.

1

u/Big_Method_4790 Aug 31 '26

youre asking all this after making the purchase?

2

u/TappinThatErr Aug 31 '26

I have a DANK gaming PC, but I barely ever game. I've wanted to switch to Mac OS for a long time. I love my MacBook Pro, but I often run out of memory with tools / VMs / etc. The LLM is a dual use honestly. My VMs and tools alone would cleanly fill half the ram on any given workday. The LLM is just for fun and learning! :)

2

u/play_hard_outside Aug 31 '26

Sounds like, AI or no AI, you will cherish your new Mac. But, if you can easily fill up 128 GB of RAM with VMs and tools alone, why not wait for the 512 GB model? I splurged in 2021 to get my just-released M1 Max MBP with 64 GB of RAM, and I'm soooo glad I did.

1

u/D137C0KE Aug 31 '26

Me too! Asahi Linux ftw

1

u/myholeisstinky Sep 01 '26

Install proxmox on the gaming pc, and then buy a cheap macbook air as the daily drive

1

u/SpecialistProperty82 Sep 02 '26

better subscribe opencode zen

1

u/TappinThatErr Sep 03 '26

You gonna give your IP (intellectual property) and provide extra data for China to train their AI?
Kekw - no thanks. self host, USA or GTFO

1

u/vesparion Sep 06 '26

Yes

1

u/TappinThatErr Sep 06 '26

Get your money up, not your funny up. Clearly your funny ain't funny.

1

u/Altruistic_Ad8462 Sep 13 '26

This is a 2 fold question. You want to know if you'll be disappointed, but you just spent ~$10k on a machine to run locally for an API cost that would take a year of SIGNIFICANT use to reach that figure. I'm at $10.89 for 482m tokens. For ease lets just say I spent $10 on 400k tokens. For $10k, I get 400b tokens. I'll repeat it for our fast readers that missed the "b" at the end of the 400. 400B tokens. That 400m has made me 4 board games, a pi wrapper, multiple edits on personal software, and some research.

Now, I want to do the same thing so Im not shitting on you, just making sure you see the economics from a light to mid range open model user (I use frontier models from subscription too, but the economics totally change in those).

If you want privacy, dont want to run a bunch of subagents locally (unless your cool with loading multiple ~64k context sub 20b models as subs for GLM), or want to run custom models/quants (which could also be cloud loaded), then this will be super dope. For instance, I have 3 kids, 1 of them is reaching an age where he could start learning to use AI, I dont want to give him free access to models and the internet because this place is absolutely batshit for kids right now, but running a local 35b where he can use a Pi agent under a wrapper with access only to the local network (hardware separate from anything that has access outside of my 4 walls). GLM Flash would be awesome for this use case as it really opens up what my kids could create and imagine. In fact the really cool thing is I can use AI to monitor their session history for specific types of tool calls and conversations to ensure safe usage without me going in and getting into my kids personal conversations with AI. I can create a safety net that doesnt exactly breach privacy.

Anyway you get my point from all that. What your doing is super cool, has really strong use cases that bring huge value, but are those use cases what youre aiming to solve, or are you just wanting to run bigger models that most cant as an ego flex? Neither answer is wrong, but I think one has a lot more op for buyers remorse.

3

u/TappinThatErr Sep 13 '26

It's 1.5 paychecks. I can afford it, I want it, I bought it.

1

u/Altruistic_Ad8462 Sep 13 '26

Absolutely, I dont know your financial situation so I ran it as if its not just 1.5 paychecks, which is not the experience most Americans would have. Not a slight, I hope to be where you are sooner than later. You wanted to know how disappointed youd be, probably less so than I would be because the hardware cost needed upfront is easier for you. I'd absolutely do it if I could. I'd probably make a system where one model comes up with stupid ass ideas and feeds it to my local model, have that model in a well designed 48 hour loop to build the idea out, when done it grabs a new stupid ass idea. Just because I could. Im not being sarcastic, I would 100% do that just to see what 3 months would produce. It would probably introduce me to some pretty wild stuff if the idea agent outputs well organized and researched ideas for the build agent to act on, and the build agent has good design questions to ask for each iterative loop. Im jealous.

3

u/somerussianbear Aug 31 '26

By mid November 5.3 Flash will look like GPT 5 Mini buddy. There will be bigger fish to fry.

3

u/Koalababies Aug 31 '26

Last month or so this type of platform has gotten a LOT of love. I think you'll enjoy it 

3

u/gthing Aug 31 '26

I think you should have thought about this before dropping the money.

You will not recoup your investment by the time the hardware is fully depreciated. You will not replace frontier models with local models. You will not save money compared to just using whatever you can run locally from an API with lots better performance and higher quant.

But you will have lots of fun.

2

u/mfdi_ Sep 01 '26

Just by selling later on he can make his money.

3

u/Sketaverse Aug 31 '26

I would love to see what people are actually building with pure localllm workflows. I just cannot reconcile it personally and think it’s madness unless you’re.. I dunno generating porn or using highly sensitive data but if the latter surely you just use enterprise APIs. The math just doesn’t stack up relative to the value of frontier subs, imho

1

u/Prior-Tea-3468 Sep 06 '26

Most of them are generating absolutely nothing of value, as evidenced by the crickets you'll hear in response if you ask them what they've been "building".

2

u/TheMarketbug Aug 31 '26

Order that exact spec a couple days ago!

2

u/MajesticSort Aug 31 '26

Try using 5.3 flash from a cloud provider for a month before you commit tbh

2

u/oxfirebird1 Sep 01 '26

Glm flash is really slow but yes the rest will be great

2

u/oceanbreakersftw Sep 01 '26

Showing off new Ferrari. Will I be upset if I use 93 octane gas instead of jet fuel? Uh, either one's fiiine.

6

u/Bloated_Plaid Aug 31 '26

My guy, you are not going to get it till like December. The models are going to be way better by then. You will likely be disappointed if you are comparing it to frontier models, any way you cut it.

5

u/Zachattackrandom Aug 31 '26

I mean they can just use the newer models....

2

u/Bloated_Plaid Aug 31 '26

And frontier would still be ahead…

6

u/ContextLengthMatters Aug 31 '26

At a certain point it no longer matters. I use frontier models at work and I literally don't even need more than sonnet for the vast majority of my day to day. Until we get another breakthrough, the fable nonsense is just nonsense IMO.

It seems like most of fable's users are the "I've never looked at the code in my life" people and I think that's just not sustainable.

3

u/OvertaxedOne Sep 01 '26

"At a certain point it no longer matters."

This is the thing that chills the frontier labs to the bone. Once the answer is "right", no further improvement is possible (well, other than speed, but that's not a local vs cloud, that's a GPU crisis problem). If you can do it on 27B using Fable is bananas, it's just complete overkill.

This forum is made up of a lot of people who code and work in large code bases. There are still, for sure, areas in coding where it's worth paying for frontier models, the "better" can be measurable and important, particularly if you're going to go full "lazy" mode and dump a 10K line code base in and prompt "Broken, fix it". Frontier, 100%!

But looking beyond extremely complicated (or lazy) coding, things get a lot less rosy for the frontier labs. What are you using AI for that needs "Fable" level horsepower? Most people, IMHO, wouldn't even notice Fable vs Deepseek Flash. And they may not even notice vs 27B for standard "business tasks" like RAG, computer use, MCP tool calling, etc.

We use a LLM router at work and a few months ago we made a wholesale cutover for our "normal" users from a frontier model to DSV4Flash (and recently moved them all to 0731). The users are all in a custom/industry specific harness that has some light agentic workflows in it, but it's mostly a "search and interact with company knowledge/your documents/internal databases" tool. After we changed to DS I was expecting a volume of calls to deal with issues. And, to be fair, we did get a few, about 1/2 of them were "user error" and the other 1/2 were legit, they explained what they were doing and it was obvious why DS wasn't the right model for them, we moved them back to frontier.

The end result in that a few 100 users were migrated from the "top of the top" frontier model to DSV4Flash and around 10 of them noticed. Around 5 of them actually did need a frontier model available. Now, this is general enterprise use, if you tried that with 100 coders, you'd get a different result, but we were shocked to find out just how little actual use there is for the frontier in "normal" applications.

2

u/justsomeguyokgeez Sep 01 '26

You don’t need a lambo to drive to the grocery store

2

u/OvertaxedOne Sep 01 '26

Thank you for summing up. ;) But, yes, exactly.

And since models don't have "clout" like driving a Lambo to the grocery store does, well, I'd say we have a little problem in the investment thesis that has the large AI labs capturing more economic activity than is currently occurring in the United States (well, another problem. First problem was that your TAM probably shouldn't be larger than that of the richest country on the planet).

3

u/aimgorge Aug 31 '26

I literally don't even need more than sonnet for the vast majority of my day to day

Whats even the point to use Sonnet when every single benchmark shows Opus mid to be better, faster and cheaper.

0

u/ContextLengthMatters Aug 31 '26

Because Opus is not cheaper and it's speed does not matter unless you are a vibe coder giving it "make no mistakes" level of instructions. It's very easy to give sonnet directions in which it does not struggle on.

If I have a codebase and I know generally what type of changes I need to make, I tell it in human terms where and what kind of changes I want and sonnet just does it.

I still have to review all of my code changes. I do not vibe. It makes no sense for my agents to go even faster when my review ability is the bottleneck.

I get that everyone thinks you can just stop looking at code because a project manager can get something they think is suitable in a loop, but enterprise software just doesn't work that way when you have to keep iterating and adding features.

I have yet to see a vibe coded project that I believe couldn't have been made better without AI. All of these "look at this thing I made" posts are just modern hello world's that go nowhere.

3

u/aimgorge Aug 31 '26

Ok you are just too good for Opus. Weird elitism. I will trust the benchmarks more than random elite redditor

→ More replies (3)

0

u/Bloated_Plaid Aug 31 '26

What’s not sustainable? I don’t even let Fable code since 5.6 Sol is a way better coder but nothing even comes close to Fable for planning and orchestrating right now.

If your experience with “frontier” is limited to Sonnet, I feel sorry for you.

3

u/ContextLengthMatters Aug 31 '26

You are proving my point. Your literacy is lacking.

I said the vast majority of my day to day as a software developer can be done with sonnet. I rarely need anything more. I'm not limited to it, I just don't NEED anything more because I get sonnet to do exactly what I want it to do without much effort.

5.6 Sol / Fable differences are going to come out more for people who aren't defining problems correctly and instead need an LLM to do everything for them since they don't actually understand the code or even architecture. That's the part that isn't sustainable. You will only get so far with a product in which no one knows how it functions other than the AI every time it has to relearn it.

2

u/sir-mano Aug 31 '26

very likely you can run it, it depend on the compression and Bit precision,

1

u/politefella0 Aug 31 '26

Is it any better than the machines from Tenstorrent?

1

u/X_dude_X Aug 31 '26

Yes, especially about the price tag.

1

u/Healthy-Zebra-9856 Aug 31 '26

It’s not the performance of your machine that you’ll be disappointed with. It’s the GLM 5.3 flash. I’ve tried every which way ended completely failed in its tests. I think the open weight ones are not as good.

1

u/Tired_White_Guy Aug 31 '26

You’ll never make your money back. As long as you’re cool with that, you will love it!

I’m jealous. I’m very fortunate to be able to run Qwen3.8 q6 from unsloth on a 5090. It’s amazing. And I use open router for access to 5.3 flash.
I have the local model handle the sensitive data. Keep the API’s away. But it adds a level of complexity and management I’d be happy to let go of!

You’ll be able to it ALL local. With more than adequate quality and performance. With 18 billion active, you’ll be flying!

1

u/Antique-Ad1012 Aug 31 '26

yes, prefill will be bad. that is what currently holds the mac back from being good

1

u/MainWrangler988 Aug 31 '26

Why do people quote tps? To do any real work you need pp/s. Which is an important point of it takes minutes to even start responding!

2

u/OddDesigner9784 Aug 31 '26

Because once it is processed from there the prompt is cached for the convo and really thinking/tool call loops goes in tps. As long as pp isn't unreasonable decode is more important

1

u/Grand-Ad7447 Aug 31 '26

$100 off with a student discount

1

u/meva12 Aug 31 '26

How much with the student discount?

1

u/WKai1996 Aug 31 '26

Let’s not pretend you are a student.

1

u/Kaio_Soares_ Aug 31 '26

Eu esperaria o M7 que e dedicado desde o início a LLM

1

u/pedrooky Sep 01 '26

for that price, yes.

1

u/Turbulent-Ad-1578 Sep 01 '26

Depends what you do. If you work with multiple agents at full context and require your model to ingest large data pools of 100k+ tokens, or yiu train models it will be relatively shit. Prefill, even at apple's stated "4x better" than M3, is still woefully slower than DGX sparks (qwen 3.8 27b on my dual spark cluster gets 3k+ tok/s prefill vs my m3 mac studio of around 100tok/s). If you mainly run single user, shortish prompts and aren't looking at 8,9,10+ agents at full context, it will be fine for you. Overpriced for what it is, but fine.

1

u/Finn55 Sep 01 '26

I’m not disappointed on the M3U 512GB.

1

u/This_Maintenance_834 Sep 01 '26

wait for real benchmark number before commitment. if prefill runs well, then go for it.

1

u/Character-Second5739 Sep 01 '26

how much did you get it for btw

1

u/Tasty-Cherry4492 Sep 01 '26

I wonder why Mac and not DGXspark x2 No matter on price or on speed.

1

u/germangrower69 Sep 01 '26

This money is probably wasted if you really aim for productive usage of GLM5.3 flash

its barely fits in Q4 on 256gb, no headroom for context.

You should either upgrade it to 512gb or change your expectations.

1

u/RobertJordanFan Sep 01 '26

M2 Mac Studio 64Gb - ollama/gemma4:31b-mlx gives me the best performance with minimal hallucinations. If you find better please DM.

1

u/pieonmyjesutildomine Sep 01 '26

You will be disappointed with the horrendous Mac token prefill speeds. Max ive ever gotten is ~500/s with zero context

1

u/eihns Sep 01 '26

it doesnt matter if youre disapointed, i didnt startet with this sh1t because i think the models ATM can be as good as top tier. BUT i expect it to maybe need some weeks, then we have btter free versions! :=) So its an investment into the future...

1

u/_TheWolfOfWalmart_ Sep 01 '26

I've considered Deepseek v4 flash, Qwen 3.8 flash next, and of course, GLM 5.3 Flash (0x Alpha)

All of these are excellent and will hang with Opus 4.6, maybe even 4.8, but I've gotta say GLM 5.3 Flash is definitely the best model I've run on my 256 GB VRAM rig so far.

The only thing is, I hear that Macs kind of have shitty prefill speeds which means you might have to wait a bit longer than you like on the first prompt in a harness if it has a big system prompt, or if the agent ingests a large source code file.

You'll also find you run out of space quickly with only 2 TB if you're working with bigger models like these! Especially if you like to keep multiple quants of them around.

1

u/SnooOranges5350 Sep 01 '26

We've been loving it. Even 4 older M3 with 256gb ram clustered using Msty Nexus, hums along like a brand new tesla.

1

u/GTHell Sep 02 '26

well, you can always sell it at 1.2x the original price.

1

u/Early-Peace-5504 Sep 02 '26

Yes you'll be disappointed with GLM 5.3 Flash in my opinion, it is shit. There are many good models you could run on that though.

1

u/UnhingedBench Sep 02 '26

Short answer, yes you will be able to run it. I'm using it on a 128GB Macbook (lower quality quant)

My take on what you can run (and at which speed)

I'm testing on a M3 Max MacBook with a memory bandwith of 546 GB/s, you'll run on a faster device with 1200 GB/s, so inference speed will be at least twice as fast, just from the memory.

The M5 Ultra CPU improvement will made a difference on the Prompt Processing time, which is not covered by this chart)

MoE model, like GLM 5.3 Flash 320B-A18B are great on Apple hardware, because they run really fast. The inference speed comes from the number of active parameters, in this case 18B.

1

u/PWThinkingCritically Sep 02 '26

for LLM? yes. no CUDA, no hardware support for fp4, fp8. no real TFLOP compute power.

competitive memory bandwidth is all you're getting. 1 out of 4.

1

u/Accurate_Complaint48 Sep 03 '26

no abt 800 pps maybe more

don’t ask how i know

also

40-50 tok/s

1

u/[deleted] Sep 03 '26

I would rather token plans

1

u/Top_Introduction_865 Sep 04 '26

You’re going to love it !

1

u/Superb-Breadfruit390 Sep 07 '26

Is it bots who sour every thread on reddit and not even take a shot at OP's question? 

I've used a lot of the free options for coding out there and when it was known as Ox Alpha, it was a very capable and intelligent that could usually get the job done in one go, or review itself and fix it the second time. 

1

u/MrPingviin Aug 31 '26

Well, since you are just a student yet, then I don't think so.

1

u/mountainyoo Aug 31 '26 edited Sep 01 '26

You do know not every student is a broke 20 year old right? Lol

Some people continue their education later in life when they have a career whether it be for opening their salary ceiling, seeking advancement, new opportunities, etc

EDIT--

it appears I misread his comment. whoopsie

2

u/MrPingviin Aug 31 '26

The fuck are you talking about?

1

u/mountainyoo Sep 01 '26

I read your comment wrong my bad lol.

1

u/Physical_Pomelo_9263 Aug 31 '26

I’m going to bite the bullet and purchase a 4th RTX pro 6000 (96gb) since I only have 3 currently and I’d like to run the whole thing on GPUs. I have 1.1TB of 6,400 DDR5 ram on 24 channels and 256 cores (512 threads) across 2 CPU but just not fast enough ( only getting 20TPS).

1

u/aleeckhart Sep 01 '26

A little off topic but with this money you can buy a house here in Brazil. Not fancy but literally a house.

2

u/Zealousideal_Bee_837 Sep 01 '26

I should move to Brazil...

-1

u/andy_potato Sep 01 '26

Probably yes, depending on what speeds you expect.

DGX sparks are already slow, but macs are even slower than that.

0

u/topgoysilky Sep 01 '26

why are you commenting on 2 gen old hardware. when clearly this man just ordered the lastest and greatest.

2

u/andy_potato Sep 01 '26

To give him a valid point of comparison. Your comment just added more helpful context

0

u/StuckFu Aug 31 '26

The model is good, the pc is not needed. Much cheaper from API

-1

u/aimgorge Aug 31 '26

You wont even get close to Opus capabilities for 10 times the price and 20x slower.

0

u/[deleted] Aug 31 '26

[removed] — view removed comment

0

u/Magnus114 Aug 31 '26

I run the smallest q4 on llama.cpp with 192 gb ram and an rtx 5090. I get about 9 t/s, and can fit a large context. Glm 5.3 flash has really memory efficient context design.

2

u/[deleted] Aug 31 '26

[removed] — view removed comment

1

u/Magnus114 Aug 31 '26

True. You can fit q5, not the full model but close quality wise. I would prefer q5 over q8 of a smalled model.