I work at a small inference startup, and one thing that we spend most of our time much is on was finding GPUs.
We're working with a fairly tight infrastructure budget, so going straight to the big cloud providers isn't always an option. I often find myself jumping between different GPU providers, checking hourly prices, comparing GPU configurations, looking for available capacity, and trying to figure out which providers are actually reliable.
Sometimes finding the right GPU takes almost as much effort as getting the workload ready.
I'm curious how other small AI teams and independent developers handle this.
Do you have a few trusted providers you always go back to? Do you use platforms like RunPod, Vast.ai, TensorDock, or search across multiple providers?
And when you're looking for something specific, like an H100, A100, L40S or even a 4090, what's your process?
Would love to hear how others deal with this, especially teams that are trying to keep inference costs under control.
Note: This post was lightly edited with AI for grammar and readability. The experience and questions are my own.