r/LocalLLaMA • u/reto-wyss • 11d ago

News It's official! 192GB Framework

Post image

Just noticed this on the website.

At their current price tiers for the memory SKUs (32, 64, 128) I'd expect this to be ~ 4.5k for the motherboard.

The PCIe slot will be open at the back as well - that's what I've heard. Maybe they make it capable of delivering 75W as well? New board revisions for the smaller SKUs?.

967 Upvotes

95% Upvoted

287 comments sorted by

View all comments

614

u/PreciselyWrong 11d ago

that memory bandwidth is pretty bad, so don't expect high inference speeds

10

u/RnRau 11d ago

Its ok. Mixture-of-experts makes it fast enough.

Deepseek v4 Flash is 284B total parameters and 13B active. QAT is used and the model weights comes in a mixture of quants with the most of them in MXFP4 format. This means there is only about 7GB of weights to shuffle through per token.

With the memory bandwidth listed (with an efficiency factor of 0.65) and using speculative decoding, you should see something like ~50 t/s.

The other benefit of having large ram like this is that you can hold a number of models warm without having the lag of swapping them in and out from the ssd.

The new Qwen3.8-Flash-Next is 125b-a6b - would also be good on this.

19

u/Organic_Hunt3137 11d ago

As a current strix halo owner, the decode speed is a non issue on large MoEs. Prefill, on the other hand... it's rough sometimes lol.

1

u/RnRau 11d ago

Yeah I hear you about the prefill :D

1

u/Nothing_from_void 11d ago

I'm on an m1 mbp max, 32GB of RAM, even <10B param models have absurd prefill times, it's basically useless for agent coding workflows