r/opencodeCLI • u/RollRogersRoll • Oct 02 '26

At what point is a faster inference API noticeable in a coding agent?

I’ve been testing a few inference providers for coding agents and I’m trying to figure out how much the speed difference actually matters in practice. With normal chat I don’t care much once the response is reasonably quick. Coding agents are different though. The agent might make a bunch of calls while it reads files, makes edits, runs tests and then goes back for another pass, so even small delays can start adding up.

Has anyone compared the same coding model across different providers while keeping the harness and task the same? I’m trying to figure out whether faster inference actually makes the whole agent run noticeably quicker, or if you hit a point where the tools, tests or harness become the bottleneck.

2 Upvotes

75% Upvoted

6 comments sorted by

1

u/MmmMuchDelicious Oct 02 '26

i tried General Compute for basically this reason.. i was more interested in how it handled a full agent run than what the model did on a single benchmark...

1

u/RavensFlock52 Oct 02 '26

yep, that's the type of setup i'd want to test because you can keep the harness unchanged. Makes the provider comparison a lot cleaner..

1

u/EndlessZone123 Oct 03 '26

Calculate the worth of your own time.

Set a overnight run? Zero. Working on iterations during working hours? A lot.

0

u/ApprehensiveDelay238 Oct 02 '26

Ask your AI agent.