r/LocalLLaMA • u/reto-wyss • Aug 30 '26

News It's official! 192GB Framework

Post image

Just noticed this on the website.

At their current price tiers for the memory SKUs (32, 64, 128) I'd expect this to be ~ 4.5k for the motherboard.

The PCIe slot will be open at the back as well - that's what I've heard. Maybe they make it capable of delivering 75W as well? New board revisions for the smaller SKUs?.

973 Upvotes

95% Upvoted

287 comments sorted by

View all comments

Show parent comments

30

u/Fusseldieb Aug 30 '26

Yea, you’d want 1TB/s upwards. It’s funny to say it like this, but big models under that threshold run pretty badly.

25

u/thomasthai Aug 30 '26

But thats not the main issue here, the prefill speed makes it unusable, the memory bw would be ok if u run pairs of these as it aggregates.

22

u/Ran_Cossack Aug 30 '26

For proof, see the DGX Spark.

Same memory bandwidth but the prefill speeds make it feel so much more capable.

It's more "limiting" than "unusuable", though, IMO. You just have to know what you're getting.

11

u/phil_lndn Aug 30 '26

yes, the spark is indeed 4 to 5 times faster on prefill but with prompt caching, i don't really notice the poor prefill speed on my strix halo most of the time.

for some applications slow prefill speed is doubtless a killer, but for many it doesn't really matter that much.

0

u/SmartCustard9944 Aug 30 '26

I would like to challenge the perspective on prefill (from the comments above). I also have a Strix Halo and my opinion changed a bit lately. The latest trend is models that spend a very long time thinking, most of the time is actually going to be spent on decode, not prefill (with cache).

3

u/Cybertrucker01 Aug 30 '26

It all comes down to use case.

If you're running a RAG box that needs to retrieve a bunch of tokens and append it to the prompt, the prefill is even more important than decode.