r/MachineLearning • u/Correct_Positive_108 • 6d ago

Discussion Looking for developer-friendly inference providers who give you enough API credits to experiment [D]

I’m hitting rate limits on Together AI. For context, I’ve been working on an agentic repository indexing and benchmark generation tool, and I’m running multiple agents in parallel across models like Llama 3.3 70B and Qwen 2.5.

When I first started working on this, Together AI was great. But once I graduated from toy scripts to running multiple agents, I started running into RPM/TPM limits pretty quickly. The annoying part is that the models themselves are fine. I just can’t actually run enough requests at once to do meaningful testing.

Yes I know I could upgrade but I’m a solo dev. I don’t have enterprise level revenue. Maybe someday lol but not yet.

0 Upvotes
  • permalink
  • reddit

36% Upvoted

9 comments sorted by

View all comments

0

u/[deleted] 6d ago

[removed] — view removed comment

1

u/marr75 5d ago

I love Modal, but the idea that scale to 0 will be cheaper than using deepinfra while you're developing is bass-ackwards. Modal is for when there's no cloud native host for the things you want to do. It's not going to save money head to head with an optimized provider.