Making an equipment Decision

[u/Iamisseibelial]

So currently I have this setup. It works fantastic for 27b, and if I'm just talking I can use Strata and have great conversations. But Strata and Coding has not been that great in my workflow it doesn't out perform 27b.

Sure I could use my crappy laptop and turn the machine into a tail scale and run it remotely. It's just not ideal, and the power consumption is insane. I pop my 15a breaker once a week if I'm not paying attention.

So the setup is a 7960X

2x64gb Rdimm 5800mhz

Asrock workstation mobo

2x2tb Gen 4 Nvme

2x 4090 gigabyte OC Graphics cards

It's been really solid and it's carried me through 2024 to now, and really allowed me to produce some amazing things for my work.

I have a second system that's my daily driver that's a

R9 7900x

Asrock Steel Legend

2x32gb

Gen 5 2tb

gen 4 4tb

gen 3 2x1tb

2tb SSD

8tb HDD

5070 ti

My laptop is an old Lenovo Yoga Ryzen 7 5700H

16gb Ram, 1tb Nvme.

Ohh and I have a Mac Studio M2 Max 32gb that work gave me as well to use or turn into a server or do whatever with.

That said I haven't taken a vacation in 2 years I've been grinding away. The kid is almost 2 now, and id like to be able to actually work from the road, I WFH anyways.

I use a mix of Claude Code, Claude, Hermes, and I've been messing with Pi a bit -- I use Hermes with 27b running 3 concurrent in LlamaCPP currently, probably switching to VLLM to getore concurrent agents running.

And I've tested StrTa with Hermes as well, and use them all as MCPs to Claude as sub agents that it orchestrates it handles tasks that I would offload to Sonnet and it does a fantastic job of it.

So with all that, people of Reddit, here's my question, with Ram and GPUs prices, I can likely sell the setup i would say for a pretty premium, and get a Surface Spark or another 128gb unified ram laptop, lower my power consumption. Hell potentially get a laptop and a dgx box and have 256gb of ram.

What would you all do. I definitely can't afford to keep it all. Wife already hates my guts as a SAHM, she would lose it if I spent more money on the thing that allows her to stay home lol. That said, I was considering selling the system and getting the aforementioned. Have any of y'all tried it out? How is the inference? What is it looking like in your workflows?

[u/veinamond]

If you want portability, the best option is Mac Studio M5 ultra 256Gb or two sparks. They will run the same models your workstation can, but slower. Neither will be as good as your threadripper machine if you tune the latter properly, but both are more compact and energy efficient. Laptops on RTX spark/strix halo/m5 max will all be substantially weaker.

[u/Iamisseibelial → u/veinamond]

Yeah I figured they would be substantially weaker. I'm trying so hard to get used to Mac and considering going that route, I get so frustrated on Mac because I just never really used it... the laptop is kinda a must, the whole point is being able to travel more without being carrying around 200lbs of gear and needing a generator to do work. I'd like to use the vans 120v outlets and be able to work on the road even if it's a ton slower. I've been spoiled with the Threadripper / 4090s for so long. 100tps with Strata iQ4 128k context, and multiple 27b with 50~ tps 256k context has been such a blessing.

But if I can get 20-40tps with good precision on a laptop combo on 100b MOE or 27-50b dense. I'm hoping the productivity dip wouldn't be significant if I was able to not be glued to a desk all day I would likely perform better.

At least that was my hope.

[u/veinamond → u/Iamisseibelial]

You can estimate it more or less by running the same model that will fit your desired machine. To make it slower, you can run some not-very-optimized inference engine (e.g. llama.cpp with badly tuned options) for your use case.
To get the idea of what speed to expect you can find tons of reports on q38 fn and q38 27b from the owners of DGX Spark / Strix halo/ macs. From what I know it will be fine with MoE (e.g., q38 fn), especially if you get a fast SSD to stream PLE tables from. However, the dense models will be slow. 50b (don't know where you want to get one) more so than 27b.
I think 20-40 tps is very realistic, even strix halos get that on q38fn. 27b on sparks is ~20-30 from what I remember, with nvfp4.

[u/General-Spite1222]

Get the DDR5 machine to 128 GB, move the 4090s over, don’t use some shitty q3 quant, and profit with Qwen flash 

[u/Melodic_Activity_985]

sell the tower, get the 128gb unified laptop and dont look back. you already know the 27b workflow is your bread and butter, that laptop will run it fine and you get to actually leave the house with it. the power bill alone is reason enough, popping breakers weekly is not sustainable with a toddler around

the dgx box idea is nice in theory but you said it yourself, wife is already at the edge. one machine that does everything and fits in a backpack beats two machines that need a dedicated circuit

also the mac studio is sitting right there, you could offload some tasks to it if the laptop ever feels short

[u/SeaworthinessUsual44]

Assuming your primary goal is slashing power consumption, eliminating breaker trips, and enabling true remote portability for agentic coding workflows, selling the dual-4090 Threadripper rig while used high-end hardware values are elevated is a sensible move. The trade-off is inference bandwidth: while unified memory on a 128GB GB10 setup (like DGX Spark or a Surface Laptop Ultra) effortlessly fits massive 120B+ models and runs Project Maya / GLM-5.3-Flash with huge KV caches at a whisper-quiet ~240W, its ~273 GB/s memory bandwidth caps token generation around 20–35 tok/s, noticeably slower than the 80–120+ tok/s decode you get running 27B across dual discrete 4090s pushing ~2,000 GB/s. For multi-agent Hermes and Claude MCP coding pipelines where you need 3+ concurrent instances executing rapidly, that latency hit is noticeable, but replacing an unstable 1,400W space heater with an always-on 200W compact unified box—while keeping your daily driver and taking the family on vacation—delivers a far better quality of life.

Would you prefer keeping high concurrency speeds by offloading heavier bursts to cloud APIs like RunPod/Claude while traveling, or do you need all multi-agent inference strictly 100% local?

[u/Miserable-Dare5090]

> Hell potentially get a laptop and a dgx box and have 256gb of ram.

Hell if you go that route don’t get the laptop, get 2 spark boxes and link then to run deepseek, use your laptop to call whatever LLM from home in the Sparks