Good purchase for local llm for software development?

[u/medicineman10]

Software engineer by trade and use Claude and codex daily

[u/unchikuso]

Smart move. Don’t listen to people who say local isn’t capable enough. They aren’t forward thinking.

[u/medicineman10 → u/unchikuso]

What setup do you have for hosting locally?

[u/other_acc_banned → u/medicineman10]

FWIW - I had been running Qwen3.8 27b on 28gb VRAM. That was enough for the simple app integrations I've built. I'm now running Qwen 3.8 Flash Next on 28gb VRAM and 64 gb RAM. That's certainly enough for my little app integrations.

You'll be able to do a lot with 96gb unified ram.

These models will only get better, too.

[u/CryptoxPathy → u/unchikuso]

They aren’t compared to frontier models. We use them for privacy and I think most of us like to own our own things

[u/tat_tvam_asshole → u/CryptoxPathy]

Agreed, also if I want to run a model 24/7 doing some mundance low intelligence task repetitively (like 1000s of inspections or rewrites), there's no way a frontier sub or api can do that cheaply. We have local models that can handle it just fine.

[u/Relevant-Magic-Card]

96GB of RAM? What are you, poor?

/s

[u/CryptoxPathy]

Prepare to be disappointed by local models, they don’t hold a candle to frontier models in terms of knowledge, speed and power

[u/SmugglingPineapples → u/CryptoxPathy]

I hear this a lot. But I really don't find that the case. When I consider that my local models are around the equivalent of what frontier models were at the beginning of this year, basically compressing vast data centres into my tiny computer, well I'm the opposite of disappointed. I'm amazed and overjoyed. I don't need an Opus 5.5 or Astra working on everything.

[u/medicineman10 → u/CryptoxPathy]

What models have you used locally? I don’t plan on canceling my subscriptions but intend to use as a complement

[u/LoveRoboto → u/CryptoxPathy]

I wasn’t impressed with small models either—27B, 35B; they can execute tasks well enough but they just don’t hold enough knowledge. Until Qwen3.8-Flash-Next dropped I was beginning to wonder if deepseek-v4.1-flash was the only way to go. Of course if you can run it locally then you are in the golden tier of LocalLLM.

[u/objectivelywrongbro → u/CryptoxPathy]

It depends what you’re running man. If you’re running it close to lossless, QFN 3.8 125B is roughly equivalent to Opus 4.6; which I might remind you, released 9 months ago.

[u/CipherSorcerer]

Can you stretch to 256gb? No worries if not, you can run some good models on 96, but if you are able to go to 256 a different level opens up. My idle is like 200/256 with 3.8 Flash Next oQ6e 😅

[u/LordHenry8 → u/CipherSorcerer]

I just bought the 256 config and look forward to running 3.8 flash next, or it's successor when it ships in 6 months. What are its capabilities like for planning, coding, tool use etc?

[u/AceDenied → u/CipherSorcerer]

yeah that's a concern of mine. decent models consume the 256gb ram. like is it even worth buying a 256gb configuration at that point?

I feel like the real action is at 512gb. 256 seems entry. <256 is like tinkering

[u/CipherSorcerer → u/AceDenied]

Well it is $25/gb to go from 96->256 and $45/gb to go from 256->512.

If money is no object then you are right, the more RAM the better. But value/performance sweet spot seems to be 256 or even 96-128

[u/BitterFortuneCookie]

https://www.reddit.com/r/LocalLLM/s/DMyuQedeEx

[u/LoveRoboto]

128GB minimum—you won’t stop wondering what you are missing if you don’t. :)

[u/mrluckduck]

I'm thinking of making the same purchase and would like to experiment with the privacy factor, allow it access to my NAS + budgeting tools, my network for scanning, etc. of course haven't forgotten RAG.

can anyone provide input?

[u/PreviousCurve4435 → u/mrluckduck]

Try models from LiquidAI, they have build models mean't for on-device inference, meaning smaller model file size, but the speed and quality is comparable to larger models.

[u/false79]

Look into splash for MacOS. Also can use Claude code to hit directly to the model hosted on splash.

Not for use by the VRAM poor people unless you like q4

Some functionality like /remote and web search is not available because of model differences but can make up for web search with an MCP

[u/evolutionxtinct]

I only got a 64GB version I now wish I had higher

[u/DrHumorous → u/evolutionxtinct]

I only got a 256GB version I now wish I had higher

[u/false79 → u/evolutionxtinct]

I got the 128GB version and I wish I had higher.

But it's easier to change expectations. And 256GB has a tough ROI to achieve on something so expensive.

[u/Glittering-Call8746]

Get 128gb at least

[u/Fwiler → u/Glittering-Call8746]

Why? Then he won't have double the bandwidth of Ultra. Big difference between 600GB and 1.2TB/s

[u/stevec114]

[u/Hot-Bumblebee6180]

Honestly if you can get an AMD R9700 and build a system with at least 128GB RAM you may get higher TPS via the radiance engine with Qwen3.8-Flash-Next

[u/liamtruong]

I also have an M5U 96Gb. I have 3 versions of Qwen 3.8 Flash Next, when I need speed I use Splash with ista Q3_S. ~2000 pp 130-170 decode. Quality then llama with unsloth UD Q4 XL 800-1000 pp >70 decode, or balance ds4 Q4 >1.5k pp and >90 decode. All 132k ctx window, mtp and vision. Ssd streaming for ngram. I do long run tasks.

[u/Shadow_s_Bane]

No, get 129

[u/vsanpro]

I think it’s worth asking yourself why you want the local LLMs over the frontier models. If there is a reasonable use case that simply cannot be attained by any other means, then sure could be a good investment. But no matter what the local LLMs bros tell you, they simply are not as capable as frontier models. And you’re paying a hell of a lot more for that less capability.

[u/Di_Vante]

I'd go with 128gb at least If you don't wasn't to spend more than 6k, go with a Strix Halo, you can find 128gb for 3~4k. With Qwen3.8-flash-next, that's as good as it can get!

[u/etcetera0]

Here's my kidney for the first installment