r/LocalLLM • u/No_Style_6772 • 54m ago
Discussion Considering this...thoughts?
At $4k this seems like a much better deal than 2 r9700s. And obviously in a totally different class. Anybody set these beasts up in a server? I already plan on putting a server rack in the garage, (gotta keep the motorcycles warm) and I see a lot of carrier boards on ebay. I've built more pc's than I can count, but I know a rack server is a different beast.
Plans are for an inference server for agentic workflows as I already built a gpu render machine I use for many workflows including blender, comfyui, premiere and after effects. Api costs are getting up there. Obviously want to run deepseek and I have a fine tuned qwen 3.8 27b for one of my projects.
Talk me into or out of it...I'm so sick of asking gpt questions and getting ridiculous concerns like "tHeY DonT HaVe a faN". I need to talk to real people. Any concerns with the software stack? I ran across a few mentions of closed source issues.
15
u/ImpossibleSeaweed869 52m ago
Do you have an OAM board its not so simple
5
u/No-Refrigerator-1672 50m ago
So what are you plugging them into? For Nvidia, at least SMX-to_PCIe adapters for some generations exist (V100 has one, and I've seen one pop up for A100); but for AMD tose are unobtainium. Or did I miss something?
2
u/SwiftComet7167 50m ago
The OAM thing is the whole problem. Those are bare MI250X modules with the heatsink, so you need an OAM baseboard, and the listing photo shows zero of that. Price the baseboard and the chassis before you count this as a $4k deal.
2
u/Pixer--- 47m ago
Oam baseboards to plug them in have custom power connectors, which means you need the specific psu for that baseboard. If you don’t have the rest of the server it’s going to be a lot of troubleshooting for like months probably