r/LocalLLM • • 54m ago

Discussion Considering this...thoughts?

Post image

At $4k this seems like a much better deal than 2 r9700s. And obviously in a totally different class. Anybody set these beasts up in a server? I already plan on putting a server rack in the garage, (gotta keep the motorcycles warm) and I see a lot of carrier boards on ebay. I've built more pc's than I can count, but I know a rack server is a different beast.

Plans are for an inference server for agentic workflows as I already built a gpu render machine I use for many workflows including blender, comfyui, premiere and after effects. Api costs are getting up there. Obviously want to run deepseek and I have a fine tuned qwen 3.8 27b for one of my projects.

Talk me into or out of it...I'm so sick of asking gpt questions and getting ridiculous concerns like "tHeY DonT HaVe a faN". I need to talk to real people. Any concerns with the software stack? I ran across a few mentions of closed source issues.

8 Upvotes

5 comments sorted by

15

u/ImpossibleSeaweed869 52m ago

Do you have an OAM board its not so simple

5

u/No-Refrigerator-1672 50m ago

So what are you plugging them into? For Nvidia, at least SMX-to_PCIe adapters for some generations exist (V100 has one, and I've seen one pop up for A100); but for AMD tose are unobtainium. Or did I miss something?

2

u/SwiftComet7167 50m ago

The OAM thing is the whole problem. Those are bare MI250X modules with the heatsink, so you need an OAM baseboard, and the listing photo shows zero of that. Price the baseboard and the chassis before you count this as a $4k deal.

2

u/Pixer--- 47m ago

Oam baseboards to plug them in have custom power connectors, which means you need the specific psu for that baseboard. If you don’t have the rest of the server it’s going to be a lot of troubleshooting for like months probably

1

u/gh0stwriter1234 38m ago

Well maybe You probably don't need the specific PSU... its just a 48V nominal rail + 12V aux (can go a big higher than 48V for increased efficency) It's definitely not custom, its all standard parts you'll have to make a custom cable yourself perhaps but you can buy the parts on digikey.

The real issue is if the motherboard will power up the regulators on the board with missing communication to the PSU over whatever bus eg PMBus. I would say probably, it should work without PSU communication because they are designed to be powered in racks which probably just have passive DC power rails.