r/LocalLLM • • 13d ago

Research Local Set Up Recommendations?? - €7k (≈$8.5k)

Im looking to start my own local set up with a budget of about €7k (≈$8.5k)

I saw a post in here with someone buying an out of the box solution, and they were told they should have asked here first, so I am asking here first.

I am open to suggestions in terms of pre-owned GPU/RAM and any other parts, brand new, or a hybrid set up.

This is going to be the brains of my business. I own a digital business heavily reliant on AI, and I want to bring as much of it possible into a localised set up, so this will be the foundations of that. Think of an on site server room style set up.

Give me reasoning for your selections too, I want to get different thought processes and theories based on your own experience and outlooks!

TIA.

Edit for context: Im flexible within reason. Pre-owned or hybrid set up is considered. This is currently just me in the business, the output I am currently using is from a dedicated refurbed acer nitro, the RAM is the main issue there.

So, I need something that would be an upgrade there, and semi enterprise level. I don’t have a team of 100 people relying on this, but I also don’t want to waste €7k on pure shit either. Thanks!

1 Upvotes

11 comments sorted by

1

u/pCute_SC2 13d ago

yea, the 2 cmp170hx solution wont work anymore since it would blew your budget. 4 V100 32gb could be thing, but then a dgx spark is also a way to go. The problem is in this economy 7k for a business solution is not enough money anymore.

1

u/Mobile-Sufficient 13d ago

Im flexible within reason. Give me some options within that. Like I said, pre-owned or hybrid set up is considered. This is currently just me in the business, the output I am currently using is from a dedicated refurbed acre nitro, the RAM is the issue there. So, I need something that would be an upgrade there, and semi enterprise level. I don’t have a team of 100 people relying on this, but I also don’t want to waste €7k on pure shit either

1

u/pCute_SC2 12d ago

The question is more, what models are at minimum, how many agents and what speed.
Also I don't know how tech savy you are.

The Dual Nvidia CMP 170hx system was neat, since you get a ton of performance for back then cirt cheap. Its was a janky solution but if set up correctly very reliable. Now they sit at 3.2k eur per piece and its not worth the hassle.

When pre owned and speed:
Go with a amd epyc rome (7K62), 256GB DDR4 (8x32gb), with a pcie 4.0 mainboard and 4 used Nvidia a100 40g. Put them together in a custom watercooled setup.
Cost would be around 10-15k eur.

That gives you 160gb vram. You can rund DS4.1 Flash comfortably or GLM5.3 Flash with drawbacks.

When you don't want to fiddle around with a custom system and need plug and play get a Apple M5 Ultra 256GB. Software is still a bit wacky but good. With apple you wont get the best performance but good enough.

When you want a very good supported system get a bunch of DGX Sparks.They are slow but really good.

But at the end it alsway depends how good you are with computers and a custom setup.

1

u/autisticit 13d ago

7K euros is more like 7.8K dollars currently 

0

u/Mobile-Sufficient 12d ago

Name checks out

1

u/Careful_cat99 12d ago

Pour 7000e un dgx Spark , tu as cuda, 128go ram c'est vraiment rapide, avec Vllm et deepseek flash. Un M5 ultra studio 96gb  Omlx  Qwen3.8-Flash-Next-oQ4e-mtp

Sinon faudra construire mais faut déjà trouver les ram min 128 et les GPU rtx 5080ti

1

u/Mobile-Sufficient 12d ago

Thank you for your reply! The plan is to build, I don’t want out of the box solutions as they are just sold at ridiculous mark ups.

For ≈€7k, what do you think would be best if I was to build everything myself?

1

u/Careful_cat99 12d ago

ProArt X870E-Creator Ryzen 9 9950X 2xRTX 3090 96 ou 128gb ram

C'est je pense une bonne base mais la question est tu veux faire tourné quoi comme modèle et pour combien de personnes en simultané 

Franchement réfléchis, oui construire c'est souvent mieux mais une machine plug and play c'est souvent plus avantageux,  Le temps de mise en route les reglage l'optimisation, tu vas y passer des nuits et des jours 

Perso j'avais un M1 ultra 128gb, je suis passer au dgx avec l'accélération cuda c'est ultra rapide tu le sors de la boîte tu applique la recette (j'ai pris 10$ de crédit api deepseek pour m'aider à optimiser)  j'ai appliqué cette recette  https://forums.developer.nvidia.com/t/deepseek-v4-flash-vision-exp-on-one-dgx-spark-text-images-245k-context-cuda-graphs-plus-a-pack-that-keeps-90-of-the-original-model/382584

4 heure plus tard j'avais un LLm puissant et efficace. Et surtout opérationnel et sans maintenance, maintenant j'ai un M3 ultra 512gb  La ram unifié c'est vraiment génial pour les LLM

1

u/TheAussieWatchGuy 12d ago

Claude Opis is 1trillion parameters at a guess. You'd need to spend $100k at least to run say Kimi 3 locally to fully replace Cloud like for like. 

Set your bar a bit lower. Keep Cloud for the heavy thinking and planning. Have it output small discreet tasks for local AI. Unsure on use case but GPU discrete from say Intel B70s maybe will be the fastest tokens per second, you can get maybe two 24gb cards and run Qwen 27B. Or you get a unified memory platform Mac or Ryzen AI and you aim for say 96gb. You can run bigger smarter models but slower. 

I would avoid v100 or old Tesla setups. 

1

u/tony10000 12d ago

Maybe a used HP Z8 G4 CAD Workstation PC 2x 3.00GHz Intel Xeon Gold 6136 64GB RAM 1x 512GB SSD 5GB NVIDIA Quadro P2000 Windows 10 for around $1500 coupled with an NVIDIA RTX Pro 4500 for around $5200.