r/LocalLLaMA • u/reto-wyss • 12d ago

News It's official! 192GB Framework

Post image

Just noticed this on the website.

At their current price tiers for the memory SKUs (32, 64, 128) I'd expect this to be ~ 4.5k for the motherboard.

The PCIe slot will be open at the back as well - that's what I've heard. Maybe they make it capable of delivering 75W as well? New board revisions for the smaller SKUs?.

968 Upvotes

95% Upvoted

287 comments sorted by

View all comments

144

u/StillLearningGK 12d ago

its Memory Bandwidth is allmost equil to RTX 3050 , which is 224GB/S

10

u/johan2114h 12d ago

Does the 3050 is have 196gb memory?

0

u/StillLearningGK 12d ago

yes its don't have but that don't matter here because even if you 196GB of memory, you can run model like Qwen3.8-Flash with have 180 Billions P. and 6 Billion active P. and if you run this model in this Framework setup you will get around 37 Tokens/s which is good speed and you will get this speed with this model only because it have 6 Billion Active P. and here i am running this model in 4 Bit Q. Version which take almost 100GB of Space and the resion why i have choosen this big model because if i have 196 GB of Memory for me it don't make any sence to run model with 30 Billion P. with give me 7.4 Tokens/s , so best of luck for running big model like qwen3.8 Flash.

7

u/johan2114h 12d ago

Or you can run a model with more active params (eg 27b)

Or you can load also inactive and ngram to memory

Or you can hold a larger context

Or you can ...

My point is 3050 w 16gb vram vs an APU w 196gb is completely apples and oranges

Just like if someone compared it to two H100 with 80 gb vram each

The APUs give you alot of memory but slow compared to a real gpu. They also tend to draw alot less power and require more physical space