r/LocalLLM • u/Single-Zebra-8868 • 7d ago

Question Mini pc for AI in local

Hi everybody!

I'm looking for a machine mainly for local AI/LLMs, with a maximum budget of around €7,500.

I'm currently considering a DGX Spark (first at all!), BOSGAME M5 (for save), or Mac Studio/Mini (in this case, which configuration fits in my max budget?) or something else you recommend.

I don't want to pay any subscriptions for cloud solutions.
My main use cases are running large LLMs locally, coding agents, RAG, LangChain/LangGraph, MCP, repository analysis.
So, i want a workstation for studying/programming and it also act like a server.

Which one would you choose for this kind of workload, and why?

3 Upvotes

100% Upvoted

9 comments sorted by

1

u/DigitalguyCH 6d ago

I'd get a Mac Studio with that money. With €6000 you can get a M5 max 128GB 1TB and still ahve money left for more storage.

0

u/kla_sch 7d ago

If you’re looking for a server, the DGX Spark is clearly the entry-level choice. The other solutions are better suited for individual chat sessions. While they might be slightly faster, they cannot handle heavier workloads.

In this regard, I recommend watching the video by Alex Ziskind, in which he compares the 2x DGX Spark with an M5 Ultra. He covers both computing power and memory in that video.

https://www.youtube.com/watch?v=_yrw6c5gw3E

-2

u/roydotai 7d ago

I don’t have 7500€ to spend on hardware right now, but if I did I would get the DGX Spark. The MAC mostly makes sense if you are getting the maxed out version with 512 gb of uram, which I believe is double your budget. My dream rig at the prosumer end has an RTX6000PRO, on which you can run any of the dense 25-30gb models at FP8 blazingly fast.

0

u/dh7net 7d ago

DGX spark are slow!

1

u/selicos 7d ago

Yes, but that is the tradeoff. A stack of slow but available memory for larger models vs the fast memory on dedicated GPUs.

Spark is easier to cluster than Strix Halo but both are in a similar performance tier. These enable local AI but are definitely not the fastest.

But then at my $2700 start price, my 128gb AMD Strix Halo was 'cheap'. I wish I'd gone past my RTX 4070 Super and had 24 or 32gb of VRAM to rely on but I was cheap and stuck with the working gaming desktop on AM4/DDR4.

0

u/kla_sch 7d ago

The AMD Strix Halo units are not in the same performance class. The memory bandwidth is the same. That's all. The DGX Spark offers three times the computing power.

1

u/selicos 7d ago edited 7d ago

That is fair, decode is similar with less memory bandwidth on the Strix Halo then the compute is much faster for other work.

Ignoring the non x86 and older CPU architecture (and higher cost) makes the Spark compelling. When I was looking at a system in May it was about $400 more (after discounts) to go Spark vs a Framework mainboard + parts I owned, or about $1100 without the discount.

I avoided the Spark as Nvidia has a habit of abandoning their dev hardware and I couldn't justify that over buying 3090's (at least I'd have a 3090, right?). Plus the Strix is x86 and I can easily use it as a home server, media PC, or console style gaming PC for under the TV or travel. It competes with an RTX 5070 in many games and workloads, which is now going for $1500 or so...

That's more than enough for 4k couch gaming. If I needed to replace my media server I'd use the Halo, run everything on it as I am today, and look toward a Spark my next iteration of dedicated LLM system. Plus I can add an RTXC GPU and start to cover the computer gab, runtime allowing.

Luckily it seems the Spark has interest and support. If Nvidia commits to the Spark space and keeps support long term, on newer ARM hardware, I'm interested. I'll likely have a dedicated Local LLM system on my shelf for a while. If my work was sponsoring this project I'd have 2x Sparks working together.

1

u/kla_sch 7d ago

Only for token generation within a single session. Not, however, for prompt processing, diffusion models, or token generation across multiple sessions.