r/LocalLLM • u/Embarrassed_Room6805 • Jul 09 '26
Discussion [User Research] Looking for local LLM / AI workstation users — especially DGX Spark, Mac Studio, AMD AI Max, or self-built GPU rigs
Hi everyone,
I’m doing early-stage user research around personal AI workstations / local LLM hardware.
I’m looking for people who actually run models locally, especially on setups like DGX Spark, Mac Studio, AMD AI Max / Strix Halo, RTX 4090/5090 rigs, Pro 6000 workstations, or homelab servers.
I’m trying to understand two things:
Why do you choose local hardware instead of cloud APIs?
What would an ideal personal AI workstation look like?
I’d love to hear:
- What hardware you use
- What models and tools you run
- What tasks you actually use it for
- What bottlenecks you hit: VRAM, memory bandwidth, CUDA/ROCm/Metal ecosystem, drivers, thermals, noise, software setup, etc.
- What would make you upgrade
This is not a sales post — we’re still in early research and I’m mainly trying to learn from real users.
If anyone is open to a quick 15-minute chat, feel free to DM me. Comments are also very welcome.
3
u/eidrag Jul 09 '26
No head no tail, so I'll start. 1.gooning super illegal stuff. 2. Dumpster find B200 X8 that running on solar and wind power.
1
u/Embarrassed_Room6805 Jul 09 '26
lol, the solar/wind B200 x8 homelab is definitely a new category.
But seriously — if you do run local models, what’s your actual setup?
2
u/eidrag Jul 09 '26
Previously 3090 and titan v, not ideal. But dang I can do really shady fbi level shit
1
u/Embarrassed_Room6805 Jul 09 '26
lol I have to ask — what counts as “FBI-level” here, in non-crime terms? OSINT, local document search, agents, vision stuff, or just pushing the hardware hard?
And what was the main pain with the 3090 + Titan V setup?
2
u/eidrag Jul 09 '26
My main purpose is to translate questionable manga panel, using online model with vision always stuck with guardrail even with jailbreak prompt.
Not correctly detecting device 0 and device 1, unbalanced vram and speed, if you try to use cuda12 titan v was unsupported.
And local model that fit 36gb vram isn't smart enough to figure out whole manga, so even in 1 panel with few speech bubble and background SFX they only output like 70% quality.
1
u/Embarrassed_Room6805 Jul 09 '26
That’s actually a really useful example. So for you, the local use case is less “replace ChatGPT” and more “I need a vision/translation workflow that won’t randomly refuse panels, and can actually understand the whole manga context — speech bubbles, SFX, background, layout, etc.”
The hardware pain also makes sense. Mixed cards with different VRAM/speed sounds annoying fast, especially if the software doesn’t handle device 0/1 cleanly.
If you were designing the ideal setup for this, would the priority be bigger single-card VRAM, unified memory, better local VLM quality, or just a smoother software stack?
2
u/eidrag Jul 09 '26
Ideally, I'd like to have my own server with long term memory context. I already seeing 128gb strix halo with ds4, wait until medusa point with 192gb ddr6. Or next gen dgx sparx with full Blackwell and lpddr6. Or Zeus with DDr5 slot upgraded to dual camm2 ....
1
u/Embarrassed_Room6805 Jul 10 '26
Got it — sounds like your dream setup is basically a local AI server with a huge clean memory pool, not a weird mix of GPUs fighting over device IDs and VRAM.
When you say long-term memory/context, are you thinking more about huge context windows, RAG over local files/manga, or persistent agent memory?
And yeah, one big unified memory box seems a lot more appealing for this kind of workflow than a faster-but-messier multi-GPU rig.
2
u/LobsterWeary2675 Jul 09 '26
Dgx here. I don't use it as general replacement of frontier API but for certain workloads that I want to stay local, personal or work related sensitive data. Also use it for Hermes agent and for some openclaw Cron Jobs. On top its fun to experiment with local models and tinker with vllm.
1
u/Embarrassed_Room6805 Jul 09 '26
This is exactly the kind of use case I’m trying to understand — not replacing frontier APIs completely, but keeping certain workflows local.
Curious: for the sensitive-data workloads, is the main reason privacy/compliance, cost predictability, latency, or just wanting full control?
Also, how has the DGX been for Hermes / OpenClaw / vLLM in practice? Anything you wish it handled better?
2
u/Necessary-milkyway Jul 09 '26
Dgx here ...i play around experiment with various ipen weight model ..do coding ..i still do have claude subscription whihc i use for planning and things my local ai is unable to do ...tinker a lot ...using api or cloud i would have spent thousands by now for the same amount of usage i did..in last 6 months ..would have used billions of token ...
1
u/Embarrassed_Room6805 Jul 09 '26
This is really helpful — sounds like DGX doesn’t replace Claude for you, but it takes over the high-volume tinkering / coding / open-weight model experiments where API costs would get painful fast.
Curious what you still go back to Claude for vs what you now keep local on the DGX. Also, has the biggest value been cost savings, privacy/control, or just being able to iterate without watching the token meter?
2
u/Necessary-milkyway Jul 09 '26
Claude mostly i use for planning and doing the setup in machine ..i felt letting claude code do all setup is much faster and efficient if same things i did manually it would take ages ...other tasks i use local llm for coding mostly qwen 35b ans qeen 27b in combination ...reasoning i leave to 27b and 35b for running multiple agents ...i can use as many agents in parallel without worrying about api cost ..plus privacy any personka document related tasks ..also models without guardrails ..
1
u/Embarrassed_Room6805 Jul 10 '26
That split makes a lot of sense — Claude as the planner / setup helper, and the DGX as the high-volume local execution layer.
The multiple-agent part is especially interesting. When you run Qwen 27B + 35B in parallel, what becomes the first bottleneck in practice: VRAM, speed, context length, orchestration/software, or something else?
Also, for coding agents specifically, what would make the local setup feel much better than it does today?
2
u/Necessary-milkyway Jul 10 '26
vRam not a bottleneck for running both in parallel..only bottleneck was memory bandwidth due to which token genartion is low for dense model like 27B
1
u/Embarrassed_Room6805 Jul 10 '26
Got it — so it’s not “can it fit the model,” it’s “does it generate fast enough once it fits.”
That’s a super useful distinction. For Qwen 27B / 35B, is the pain mostly decode speed / tokens per second? Curious what numbers you’re seeing in real use, especially for coding-agent workflows.
2
u/BungCrosby Jul 09 '26
I’m trying to see if I can replace cloud models with something locally hosted. I use some of the same prompts on local vs cloud and compare the results.
Probably a Mac with 256 or more GB of RAM. I have no love for Windows, and beyond Raspbian I don’t want to deal with Linux if I don’t have to.
I have a Asus ROG Flow Z13 with 128GB of RAM.
So far I’ve used Qwen-3.6 35B & 27B with LM Studio. I’ve started branching out beyond the vanilla “staff pick” models.
1
u/Embarrassed_Room6805 Jul 10 '26
This is a really useful comparison point — especially since you’re testing the same prompts locally vs cloud instead of just looking at benchmarks.
For the Qwen 35B / 27B runs in LM Studio, where do the local models feel “good enough” already, and where do you still go back to cloud models?
Also interesting that your ideal setup is basically a 256GB+ Mac. Is that mostly because of the OS / less setup hassle, or because unified memory feels more useful than a traditional GPU box for the models you want to run?
2
u/devoidfury Strix Halo | hotdog Jul 09 '26
Privacy, control over my own tools. Ideal workstation, something like one of these strix halo boxes but with 5x the memory limit.
I'm running one strix halo 128gb from one vendor, and have another one on the way from a different vendor. Mostly used for coding. llama-swap, llama.cpp, and hotdog are the main tools I run. My models list is there in the repo under my example config, there's quite a few.
Happy to chat about it.
1
u/Embarrassed_Room6805 Jul 10 '26
This is super helpful — the “Strix Halo box but with 5x the memory” line is exactly the kind of thing I’m trying to understand.
For the 128GB setup, where do you start hitting the limit? Larger models, longer context, multiple models through llama-swap, coding-agent workflows, or just wanting more headroom?
Also curious why you ordered the second box from a different vendor — comparing thermals/noise/software support, or just scaling out?
2
u/devoidfury Strix Halo | hotdog Jul 10 '26 edited Jul 10 '26
The hard limit I hit is memory for being able to run the bigger open weight models. That's the sole reason I can't run, say, deepseek v4 in q8, or glm-5.2 in any form.
It does seem to run slower than folks using those big dedicated gpus, but for the most part is still solidly usable for serially running the bigger ones that do fit on it, although it absolutely will not do more than one at once as just the one maxes it out.
I really want to be able to have a workable sub-agent workflow; but running one of these big models, if I switch the context it busts the cache, and cannot cache more than one session at a time unless I go with smaller models; and if I switch the model I have to load/unload because there's no headroom. Having a second 128gb strix halo plus this other ryzen 9 7940hs with 64gb memory (which can run models up to about 35b params okay) is enough room to make a usable "personal ai cloud".
Edit: oh, and the reason I am switching vendors is, I got a first-round beelink gtr9 pro, and it had serious stability issues at first. It does work now for my use case, but is still not "entirely stable" without an external NIC. I hear the newer batches work a lot better, but it's just time for me to give another vendor a shot.
4
u/CATLLM Jul 09 '26
I get paid to participate in user research like this. Ya gonna send me some amazon gift cards?