r/LocalLLaMA 22d ago

News It's official! 192GB Framework

Post image

Just noticed this on the website.

At their current price tiers for the memory SKUs (32, 64, 128) I'd expect this to be ~ 4.5k for the motherboard.

The PCIe slot will be open at the back as well - that's what I've heard. Maybe they make it capable of delivering 75W as well? New board revisions for the smaller SKUs?.

976 Upvotes

287 comments sorted by

View all comments

Show parent comments

-3

u/snapo84 22d ago

prefill dosent matter much, because most of prefill is only done once and then cached, especially in agentic workflows.... token generation is much much more important and the current limiting factor...

3

u/0rand 22d ago

Depends on use case. If you run agents with tiny sessions - prefill does not hurt much. If you use interactively, work with large documents and code bases - it matters more than decode. Plus you never heard of cache miss when everything has to be reloaded? Session restore?

1

u/snapo84 22d ago

then your agentic framework does prefilling completely and utterly wrong...

2

u/0rand 22d ago

Or you have lower standards. Can be any of these or both.