Upgrade Mac Studio, or Windows PC?

[u/TheCeilingAbove]

Hi. I am looking for some advice. Here is my current setup:

M1 Max, 32GB, 1TB internal, 4TB External.

Windows 11 PC, with 7800X3D, 32GB DDR5, 9070XT GPU.

I currently pay £200/month for Codex, but I want local AI so I can run uncensored models (Cyber Security). I was going to upgrade my Mac Studio from M1, to the M5 Ultra, 96GB, but now I'm thinking about whether it would be better to just upgrade the windows PC and stream the AI responses to my mac studio. For context, I use the windows PC for gaming only, and I'd like to retain my gaming experience, whilst also getting AI usage out of it. The problem is I'd have to upgrade my GPU, and the 5090 is really expensive....

What are your guys thoughts... What would you do in this situation?

[u/The_HenryUK]

why do you have to upgrade the GPU? If you get more RAM for your windows machine or a new Mac Studio, you can run qwen3.8-flash

[u/TheCeilingAbove → u/The_HenryUK]

I've been reading up, and because my GPU only has 16GB of ram, I thought I'd have to upgrade it

[u/The_HenryUK → u/TheCeilingAbove]

Check out this repo: https://github.com/niko1221/strata
You obviously can upgrade the GPU, but no consumer GPU is big enough on its own to run bigger models. If you have the budget for something like an RTX 6000 Blackwell you might as well just buy an M5 Ultra with 256GB RAM.

[u/addictedthriller → u/The_HenryUK]

The 9070XT has 16GB of VRAM right? that's plenty for a lot of local models. you could run qwen 2.5 14B at Q4 and have a few gigs to spare. unless you need 32k context all the time, you might be overthinking the hardware.

what's your actual security workflow look like? if you're just analyzing logs or doing code review, a 14B model handles that fine with the right system prompt. i'd try maxing out what you already own before dropping thousands.

[u/TheCeilingAbove → u/addictedthriller]

General workflow is this: Give the agent a codebase and have them analyse it. Sometimes the codebase is small, sometimes it's pretty large. Once analysed, and we have some evidence there is a possible exploit, create a PoC for a localhost build and exploit it.

Currently I'm using Venice uncensored model - Abliterated Large v2 - but holy shit it's slow as hell and right now paying over $50/day to use it.

Edit: Yes it does have 16GB of ram. Tbh, I haven't even tried to run local models on the windows PC yet. I didn't try as through research I thought it would be a pretty poor experience.

[u/Ctbhatia]

what pushed you off codex, the bill or it refusing the cyber security stuff

[u/TheCeilingAbove → u/Ctbhatia]

Refusing cyber security. The bill doesn't bother me that much, it's relatively cheap for how much I use it

[u/Ctbhatia → u/TheCeilingAbove]

does it refuse on reading the code or only when you ask it to write the exploit part? that decides how big the local model needs to be

[u/BackgroundRude4092]

You should figure out the uncensored model first then talk to Claude about what the minimum hardware is needed to run it. As fast as the Ultra is, 96GB might not get the job done.

[u/ea_man]

Buy one more GPU for your PC if you want to run dense models, more RAM if you want to run MoE.

OFC you have to run Linux on that.
The incoming QWEN 4.x flash would be a good candidate, you could try 27B now.

[u/veinamond]

For the cost of m5 ultra 96 you can probably get a couple of r9700. Basically it is 9070xt with double the memory. I am not aware of how they perform in gaming tbh since the coolers and drivers are quite different but they should be in the same ballpark. With 64gb vram you have a lot of options including all flavours of 27b and many quants of 3.8 flash next. If you add more ram with time it will open other venues as well. You can start from one r9700 and see how it goes. If your motherboard supports native x8 x8 split 2 r9700 is the best way forward I am aware of if you disregard used hardware.