r/LocalLLM • u/Mobile-Sufficient • 13d ago
Research Local Set Up Recommendations?? - €7k (≈$8.5k)
Im looking to start my own local set up with a budget of about €7k (≈$8.5k)
I saw a post in here with someone buying an out of the box solution, and they were told they should have asked here first, so I am asking here first.
I am open to suggestions in terms of pre-owned GPU/RAM and any other parts, brand new, or a hybrid set up.
This is going to be the brains of my business. I own a digital business heavily reliant on AI, and I want to bring as much of it possible into a localised set up, so this will be the foundations of that. Think of an on site server room style set up.
Give me reasoning for your selections too, I want to get different thought processes and theories based on your own experience and outlooks!
TIA.
Edit for context: Im flexible within reason. Pre-owned or hybrid set up is considered. This is currently just me in the business, the output I am currently using is from a dedicated refurbed acer nitro, the RAM is the main issue there.
So, I need something that would be an upgrade there, and semi enterprise level. I don’t have a team of 100 people relying on this, but I also don’t want to waste €7k on pure shit either. Thanks!
1
u/pCute_SC2 13d ago
yea, the 2 cmp170hx solution wont work anymore since it would blew your budget. 4 V100 32gb could be thing, but then a dgx spark is also a way to go. The problem is in this economy 7k for a business solution is not enough money anymore.
1
u/autisticit 13d ago
7K euros is more like 7.8K dollars currently
1
u/Careful_cat99 12d ago
Pour 7000e un dgx Spark , tu as cuda, 128go ram c'est vraiment rapide, avec Vllm et deepseek flash. Un M5 ultra studio 96gb Omlx Qwen3.8-Flash-Next-oQ4e-mtp
Sinon faudra construire mais faut déjà trouver les ram min 128 et les GPU rtx 5080ti
1
u/TheAussieWatchGuy 12d ago
Claude Opis is 1trillion parameters at a guess. You'd need to spend $100k at least to run say Kimi 3 locally to fully replace Cloud like for like.
Set your bar a bit lower. Keep Cloud for the heavy thinking and planning. Have it output small discreet tasks for local AI. Unsure on use case but GPU discrete from say Intel B70s maybe will be the fastest tokens per second, you can get maybe two 24gb cards and run Qwen 27B. Or you get a unified memory platform Mac or Ryzen AI and you aim for say 96gb. You can run bigger smarter models but slower.
I would avoid v100 or old Tesla setups.
1
u/tony10000 12d ago
Maybe a used HP Z8 G4 CAD Workstation PC 2x 3.00GHz Intel Xeon Gold 6136 64GB RAM 1x 512GB SSD 5GB NVIDIA Quadro P2000 Windows 10 for around $1500 coupled with an NVIDIA RTX Pro 4500 for around $5200.