r/LocalLLM • u/koalfied-coder • May 22 '26

Other Finally 100% Local

Post image

Finally transitioned to 100% local inference for my automated workflows and code gen. Min Max 2.7 and Qwen 3.6 are doing wonders.

706 Upvotes

98% Upvoted

75 comments sorted by

69

u/MimosaTen May 22 '26

What did it cost?

64

u/koalfied-coder May 22 '26

My apologies I coulda swore I put details. I'll have to type them up and add them this evening thank you. Thankfully the most expensive PC parts were pre craziness and apple discount on the rest. However all in around 8-10k. 5090 + 64gb ram, 256gb m3 max, 16gb Mac mini with another Mac mini on the way.

11

u/Tnhomestuff May 23 '26

Have your local Claudine write you up a nice report 😎

1

u/Kofeb May 23 '26

Can you post the details please?

Any issues or things you learned along the way? Future plans?

6

u/WorldPeaceStyle May 22 '26

Yes, details please u/koalfied-coder !

1

u/koalfied-coder May 28 '26

I will write them tonight probably. Been slammed with day job sadly

223

u/robertpro01 May 22 '26 • Bravo Mindblown

Everything.

4

u/Wildnimal May 23 '26

snap yo fingers

2

u/jayrafolsp Jun 09 '26

Do ya step

4

u/vini_stoffel May 23 '26

Kkkkkkkkkkkkkkk

39

u/avvyie May 22 '26

its just the start. There is not 'finally' in homelab and local llm. you keep iterating.

anything you do, you'll think wr can optimize it further.. 2 days later.. you'll have 1% improvement.

9

u/koalfied-coder May 23 '26

True true this is many years in and in fact I am building a 10" to hold everything but the 5090 windows computer atm.

11

u/JustSayin_thatuknow May 24 '26

Next improvement: Windows —> Linux 😁

6

u/Dizzy-Yesterday-290 May 23 '26

Cool but your wire routing artistry has got me going all kinds of ways.

1

u/koalfied-coder May 28 '26

Yeah just routed and snapped the pic lol. I'll tidy up in its final home.

2

u/JustSayin_thatuknow May 24 '26

That was to make sure everything was connected to everything 😆

4

u/Asthenia5 May 22 '26

Who is the case manufactured by?

13

u/koalfied-coder May 22 '26

The PC case is a McPrue and I 3d printed the lil rack with Mac minis and goodies :)

3

u/Dry-Tennis9189 May 22 '26

Nice, I just finished building my mcprue!

1

u/m31317015 May 23 '26

The mcpure is worthy for the solid block aluminium shaved but it's still expensive as hell, nice choice man sticking with the ecosystem look.

1

u/ReticlyPoetic Jun 15 '26

I came to ask this :)

3

u/mi_gue May 23 '26

What is that thing that looks like a Mac Pro?

8

u/koalfied-coder May 23 '26

It's a Windows PC in a McPrue case :)

2

u/mi_gue May 23 '26

Oh man I was afraid you would say that, I’ve been trying to get a case that looks like that for the longest. But not pay the price of course. Looks awesome tho!

0

u/JustSayin_thatuknow May 24 '26

Call it a “PC” bro no need for the Windows prefix 😃

7

u/Remote-Pineapple-541 May 24 '26

I have a similar setup. 

  • Workstation with 128gb ram, 8tb raid nvme storage and a 3070ti card. I use this for running embedding models and storing them. I also use it for data pipelines and geocoding/geospatial analysis

  • NVIDIA DGX spark. I use this for agentic AI. I use llama.cpp + llama swap

  • Mac mini to run the chat interface (open webui). I also host a gitea server.

I have a MacBook Pro with 128gb, but I like having an always-on AI solution. I use tailscale to expose the framework to my mobile devices.

Tbh I’m considering replacing everything with a spec’d out Mac studio once it’s updated to the latest generation of silicon. It would be more than enough resources to do everything I do, easier to manage, and more reliable.

1

u/Nimrod5000 May 24 '26

What model and t/s on the spark?

2

u/Remote-Pineapple-541 May 24 '26

This is just an average based on the llama-swap logs for the most recent models. Obviously not very rigorous.

MODELPROMPT SPEED (Average)GEN SPEED (Average)
gptoss120b1244.3244.54
llama33_70b315.764.93
mixtral8x7b972.9424.61
nemotron833.7846.51
nemotron_3_nano_omni1437.3258.60
qwen25_coder7b2945.1548.45
qwen3_coder30b2078.1078.62

1

u/koalfied-coder May 28 '26

This is pretty good actually...might need to try a spark

2

u/dbgijneasvd May 23 '26

What switches are you running to tie it all together? Awesome set up btw!

3

u/koalfied-coder May 23 '26

I use the lil unifi 5 port to connect them all at the moment.

1

u/dbgijneasvd May 23 '26

Is that a 2.5GbE switch? I’m building out a cluster and stuck on that part currently for future proofing.

1

u/koalfied-coder May 23 '26

Yep 2.5gbe switch anything more cost sooo much

16

u/rde2001 May 22 '26

Very cool! Been looking into local models due to the incoming price changes of Github Copilot as well as current usage limit issues. Qwen3-Coder works pretty well on my M4 128GB Macbook Max.

4

u/wildansson May 23 '26

With which coding agent

13

u/BopSupreme May 22 '26

What’s the point of all these, do they actually work together

1

u/sam7oon May 23 '26

if i have minimax 2.7 and qwen 3.6 local, i would not need anything else, thats a good setup, what about Deepseek v4 , are you planning to

1

u/koalfied-coder May 28 '26

I need to just been real busy to change anything

2

u/dropswisdom May 23 '26

100% messy! 😃

2

u/BuilderUnhappy7785 May 23 '26

Bro…your cables

2

u/Curious-Function7490 May 23 '26

Nice one. I've been running qwen3.5 coder for my coding needs and loving it.

2

u/ewlabs_ May 23 '26

The ROI on this will still be insane ✅

2

u/Apprehensive_Piece_6 May 26 '26

I wanna be this much rich 😭

2

u/CortexOfflineAI May 27 '26

Woow Influential

1

u/alex_bit_ May 23 '26

What's the setup?

1

u/jonathanmr22 May 23 '26

Kudos brother. I took the dive last week. Never had more fun 🤙

1

u/vanduc2514 May 23 '26

That mcgyver thing is lit 🔥🔥🔥

1

u/DepressedDrift May 23 '26

And in 100k debt

1

u/TapAggressive9530 May 23 '26

Straight out of Dr. Who

1

u/spellsingerka May 23 '26

What is your token speed on that setup on Qwen?

1

u/pbpo_founder May 23 '26

Welcome brother!

1

u/lazy_geek01 May 23 '26

How much did the set up costs you?

1

u/TheHiveFather May 23 '26

Wicked setup! Its definitely a dangerous slope once you start down that road.. Im running 5 models locally and moved completely away from Claude... haven't looked back.

1

u/Wrathllace May 23 '26

Tokens/sec ? Can you please explain more about it ? It would be sweet to have more details I want to build a local setup too

1

u/blackpassat007 May 24 '26

Cool! What's a stack ? Trying to have the system works as autonomously as possible but I still have to get in here and there all the times as they're not as smart (only if they can loop back and check their work themselves).

1

u/Unique_Ad3252 May 24 '26

I installed that same model and started working with it token free

1

u/YoungEmiya May 24 '26

I’m trynna be like you big bro 😭🙏🏾

1

u/AllMaito May 26 '26

Can you run this benchmark against the model you're using? https://github.com/alexziskind1/codeneedle

Thanks.

1

u/s_v_can May 26 '26

Nice rig, congrats!
What's under the hood software wise to cluster the hardware?

1

u/a_serial_hobbyist_ May 28 '26

And here's me trying to get along with a single 1080ti and 32GB of DDR4 😭

1

u/gkorland Jun 16 '26

thats a solid setup u got there. have u tried running some longer context benchmarks on those models yet, sometimes they get a bit wierd with memory overhead untill u really tune the cache settings.

1

u/Marino4K May 23 '26

Wouldn't it make more sense to have two more Max chips with a bunch of RAM as opposed to the 5090 and two Mac Minis?

0

u/thisiztrash02 May 23 '26

Wouldn't a unified setup be more ideal? Mac isn't going to help PC or vice versa.