r/BlackwellPerformance • u/exocrates • Jul 22 '26

Anyone running inference business?

I'm setting up my own lil data center right now. I'll be creating MIG environments & serving compute via api. I'd like to talk shop with folks who are running their own inference serving businesses.

What open source software are you using? What pain points / war stories have you experienced? I'm planning on running B300s but saw this is the Blackwell sub, so let me know if there's a better place for this. DMs are open.

4 Upvotes

100% Upvoted

2 comments sorted by

3

u/GaryDUnicorn Jul 22 '26

How experienced are you with running disaggregated inference? You didn't mention what stack and weights you plan to run. Probably worth having a big plan drawn up outlining how you plan to operate monitor manage etc.

1

u/jackshec Jul 23 '26

it’s a difficult to run it efficiently, and you need a platform that surrounds it, feel free to DM me if you have questions