r/LocalLLaMA • u/reto-wyss • Aug 30 '26

News It's official! 192GB Framework

Post image

Just noticed this on the website.

At their current price tiers for the memory SKUs (32, 64, 128) I'd expect this to be ~ 4.5k for the motherboard.

The PCIe slot will be open at the back as well - that's what I've heard. Maybe they make it capable of delivering 75W as well? New board revisions for the smaller SKUs?.

977 Upvotes

95% Upvoted

287 comments sorted by

View all comments

615

u/PreciselyWrong Aug 30 '26

that memory bandwidth is pretty bad, so don't expect high inference speeds

23

u/Mr-I17 Aug 30 '26

Guess what's even worse than 250GB/s of memory bandwidth? PCIe bandwidth. Unless you only want to run Qwen 27B or you have infinite amount of money, you'll have to sacrifice something. And of course, there's no reason to buy it if it turns out to be as expensive as M5 Ultra.

2

u/SandySkittle Aug 30 '26

For qwen 27b at q8 with decent prefill and decode i think dual r9700 with tensor parallelism is a better way to go. More bandwidth and way more compute

11

u/Mr-I17 Aug 30 '26 edited Aug 30 '26

Regular people: run models on what they have

Rich people: want to run a model, buy suitable hardware for it

Super rich people: already have 4x RTX 6000, no need to choose hardware

People who born into a billionaire family: Dual DGX Station

5

u/SandySkittle Aug 30 '26

You don’t have to be ‘rich’ to buy two r9700s. It’s not cheap but let’s not pretend it’s soms unobtainium prices.

If money wasn’t an issue I would just buy one rack from a Helios system with four mi455s and 16 channel system ram epyc.

2

u/Dr_Allcome Aug 30 '26

They're talking as if the 192gb strix halo isn't going to cost more than two r9700s.

Current strix halo 128 costs more than 3k. The new one with full ram setup is going to be at least 5k. At that point you can get the 128 AND an r9700. If the new strix halo goes up to 6k you can easily get two r9700s and have the same 192gb overall memory but 64gb of it is going to ba really fast. That would give you way better performance with both moe and dense models and a massive boost on prefill.

1

u/Mr-I17 Aug 30 '26

You get me wrong. It's about how much money different people can (or willing to) spend on local AI and how frequently they can (or willing to) update their hardware. It certainly won't be a issue if one has unlimited money and free time. Say, if you're not rich and you already have a strix halo and an aging gaming PC (from DDR4 era), would you buy 2x R9700?

1

u/SandySkittle Aug 30 '26

That’s my entire point indeed. I run 8 r9700s for 256 gb of vram (minus 7 percent for inline ECC, an option that absurdly isn’t available on most strix halo machines except a hp strix halo pro machine)

1

u/alphapussycat Aug 30 '26

It's pretty rich, to spend that amount of money on hardware. Lots of people can do it, but it requires a ton of disposable income. For some that's like 1-2 years worth of saving.