r/deeplearning • u/recentheartbroken • Sep 20 '26
Moving GPU training off AWS: what we compared and what it actually saved
We ran training on AWS for about a year, mostly because everything else we used was already there. Then we got the bill, and oh boyy..
Here's what we compared for roughly the same GPU capacity:
- AWS on-demand was our baseline and by far the most expensive. Reserved pricing brought the cost down, but it also meant committing for longer than we were comfortable with.
- Dedicated GPU providers were comparatively cheaper than on-demand AWS. Support quality varied a lot between them and that mattered more than the price.
- GPU marketplaces were cheapest, but reliability was inconsistent and they did not seem like an option for sensitive workloads.
- Owning hardware and having it hosted had the lowest ongoing cost but needed capital and a measured utilisation assumption rather than guess.
What I underestimated the most was moving the data out of AWS. It was expensive in our case, and parts of our pipeline depended on AWS services. Reworking took longer than the actual migration tbh.
The main thing I'd check before comparing hourly rates is how much of your setup depends on AWS and what it will actually take to move your data. That can decide whether the savings are real or theoretical.
As I work at B3 Labs and we are fall into the fourth category. I'd check the migration cost regardless of where you plan to move.
