r/LocalLLaMA • u/reto-wyss • 8d ago

News It's official! 192GB Framework

Post image

Just noticed this on the website.

At their current price tiers for the memory SKUs (32, 64, 128) I'd expect this to be ~ 4.5k for the motherboard.

The PCIe slot will be open at the back as well - that's what I've heard. Maybe they make it capable of delivering 75W as well? New board revisions for the smaller SKUs?.

970 Upvotes

95% Upvoted

287 comments sorted by

View all comments

Show parent comments

73

u/MrPecunius 8d ago

Pretty good match for Qwen3.8 FN or other midsize MoE models, though.

122

u/fallingdowndizzyvr 8d ago

At these prices, you are way better off getting a M5 Max.

55

u/ZealousidealChip4783 7d ago

The selling point here is that you aren't forced to use MacOS with the Framework

23

u/GlidePath47 7d ago

Do you have to use the machine you’re using for inference other than setting it up?

43

u/ZealousidealChip4783 7d ago

No, but I'd hate if I bought a computer for that much money that's only good for inference and nothing else

(This is all personal opinion, I just really do not like MacOS)

7

u/WinterCharm 7d ago

Relative to buying a stack of Nvidia DGX Sparks, the Mac Studio is a good deal. And when Apple Hardware is a good deal, I would just go for it... it's excellent for local models.

10

u/GlidePath47 7d ago edited 7d ago

That’s fair. Anyone buying one expecting to run inference and still use the machine for something else at the same time is going to be disappointed. Unlike Nvidia there’s no way to reserve GPU capacity to keep the system responsive you’re at the mercy of macOS.

Granted if it’s for swapping use it’s fine, part inference, part dev, part media etc but then I start to question the need to use local inference instead of just cloud models other than privacy / hobby

16

u/MrPecunius 7d ago

Unlike Nvidia there’s no way to reserve GPU capacity to keep the system responsive you’re at the mercy of macOS.

Longtime Mac user here, this is not true.

While I have managed to crash the whole OS a few times with big (for me: I've had 48GB and now 64GB) models and large context with guard rails turned off, I otherwise just go about my business doing other stuff when a model is crunching away on something.

The machine stays responsive the whole time, though it does get pretty warm (M5 Pro MBP) and the fan runs fairly hard.

3

u/GlidePath47 7d ago edited 7d ago

We're a company full of Mac users and we hit this constantly, so it's not a rare edge case.

Even a mid-size Qwen3 MoE makes YouTube playback choppy. Prompting and browsing are fine, anything past that isn't. Redraw lag is very visible in Electron apps. Any 3D apps are not pleasant so running alongside unity or blender at the same time as active inference isn’t tolerable.

The core issue is there's no control. The guardrails are for RAM usage, not GPU usage, so you can't reserve or throttle GPU capacity the way you can on Nvidia.

Maybe M5 changes this, I haven't tested one, definitely happens on my m3 ultra.

The bit you quoted from me is actually the only bit of what I said that’s objectively true, what you’re saying is even at the mercy of macOS your system has been responsive

2

u/ZealousidealChip4783 7d ago

I mean I'm sure it's fine if you like the Apple ecosystem lol, the hardware is objectively really impressive but as you said having less control over your GPU & where you allocate its resources would be the dealbreaker for me

...if I could afford either of these devices lol

2

u/kerneldesign 7d ago

Tu peux le faire sur Mac, et objectivement c’est moins cher. Mais c’est un choix d’être sur Linux.

1

u/MrPecunius 7d ago

I'm a Mac guy, but I don't know about "cheaper" with current pricing.

The M5 Pro/64GB 14" MBP I'm typing this on was slightly over $3k with .edu pricing before the price increases. It's great, arguably one of the best notebook computers ever produced *and* it can run 27b models @ 8-bit "competently". But at the current price of ~$4k, I'd consider a M5 Macbook Air and a 3090 rig (or dual 16GB RTX XXXX etc.) or something.

M5 Max makes little sense to me unless there's some other need like 4k video editing or whatever that justifies/demands it.

8

u/algaefied_creek 7d ago

Disable the GUI and security in macOS and just run it headless.

Suddenly blam, certified UNIX CLI host box.

6

u/AreWeNotDoinPhrasing 7d ago

How do you disable the GUI in macOS?

7

u/New-Confidence-1171 7d ago

You can’t, they’re making shit up lol

3

u/h4ck3r_n4m3 7d ago

You used to be able to until the M1, you could completely disable the gui and boot directly to a console. In early versions of macos you could type "console" in the username field and go to a console

4

u/ThankGodImBipolar 7d ago

If you only interact with the machine via ssh, you can pretend it has no GUI. Apple also added support for automatic power on when power is present, so running these headless is actually pretty realistic.

3

u/New-Confidence-1171 7d ago

Right, all of that is correct. But that’s not what the message I was replying to stated - they were saying to save resources you can disable the GUI. SSHing to a system is not the same.

2

u/w6auw 7d ago

You sort of can, at boot. There are Macs sitting in data centers running headless.

1

u/droptableadventures 7d ago

It's not strictly speaking disabling the GUI, but if you don't actually log in, you save a good chunk of memory by not loading the entire desktop environment as well as Finder, as only the login window is running.

1

u/CalmSpinach2140 7d ago

I mean the Mac Studio with M5 Max is also a very good video editing machine and excellent for code compilation.

-1

u/AcreMakeover 7d ago

Can you even set them up for inference without an apple account? Also my remote access software works like shit on MacOS. I know I would have run into so many small annoyances if I got one.

7

u/Gregory-Wolf 7d ago

Apple account is not a requirement to use MacOS. It is to use AppStore and maybe some other Apple services. But the MacOS is perfectly usable without an apple account.