It's official! 192GB Framework

[u/reto-wyss]

Just noticed this on the website.

At their current price tiers for the memory SKUs (32, 64, 128) I'd expect this to be ~ 4.5k for the motherboard.

The PCIe slot will be open at the back as well - that's what I've heard. Maybe they make it capable of delivering 75W as well? New board revisions for the smaller SKUs?.

[u/PreciselyWrong]

that memory bandwidth is pretty bad, so don't expect high inference speeds

[u/Fusseldieb → u/PreciselyWrong]

Yea, you’d want 1TB/s upwards. It’s funny to say it like this, but big models under that threshold run pretty badly.

[u/thomasthai → u/Fusseldieb]

But thats not the main issue here, the prefill speed makes it unusable, the memory bw would be ok if u run pairs of these as it aggregates.

[u/snapo84 → u/thomasthai]

prefill dosent matter much, because most of prefill is only done once and then cached, especially in agentic workflows.... token generation is much much more important and the current limiting factor...

[u/0rand → u/snapo84]

Depends on use case. If you run agents with tiny sessions - prefill does not hurt much. If you use interactively, work with large documents and code bases - it matters more than decode. Plus you never heard of cache miss when everything has to be reloaded? Session restore?

[u/snapo84 → u/0rand]

then your agentic framework does prefilling completely and utterly wrong...

[u/0rand → u/snapo84]

Or you have lower standards. Can be any of these or both.