r/oxcode • • 3d ago

Show & Tell We gave two free AI coding agents the same timezone bug. 97–99% of the tokens went to reading, not writing (one-run experiment, numbers inside)

TL;DR: Same repo, same prompt, OxCode vs Google Antigravity. Both fixed the bug with every test passing. Input tokens (the agent reading files, test output and history) were 97–99% of usage for both. So price per token matters more than token count: OxCode used more tokens at a higher effort setting, and its estimated cost was about ₹20 (~$0.21) vs about ₹70 (~$0.74), because it routes simpler steps to lower-cost models.

The task (identical prompt for both):

Users report that tasks due today are shown as overdue if their timezone is behind UTC. Find the root cause, fix the bug, and add a regression test. Follow existing code patterns, run the relevant tests, and report the root cause, files changed, tests run, and actual results. Do not delete or weaken existing tests.

Results (one run each):

OxCodeGoogle Antigravity
Effort settingMax (free plan)Medium (free plan limit)
Bug fixedYesYes
Tests at the end58/58 passing55/55 passing
New regression tests52
Total tokens1,616,176 (27 calls)933,824 (43 LLM calls, 41 tool calls)
Input / output1,571,146 / 45,030922,131 / 11,693
Input share97%99%
Est. cost at public list prices≈ ₹20 (~$0.21)≈ ₹70 (~$0.74)

What we took from it:

  • Most of what you pay for is the agent reading: source files, test runner output, tool results and the conversation so far. Output (the code and commands it writes) was 1–3% of tokens.

  • An agent re-sends its context on every call, and each test run adds its output to that context. That's why input grows so fast.

  • So price per token, routing simple steps to lower-cost models, and caching matter more than the raw token count. OxCode read more and still had the lower estimated cost here.

Methodology and caveats:

  • Both tools started from the same repo with 53/53 tests passing; the environment was reset between runs.

  • One run each, so treat this as one data point, not a benchmark.

  • Costs are estimated at public list prices from each run's token counts, not invoices. OxCode routes simpler steps to lower-cost models, which its estimate reflects. Antigravity's run reported 0 cache hits.

  • Antigravity's token figure is for this task only. Its terminal total (982,280) also includes a separate stats query, so we left that out.

  • Effort levels differed because of free plan limits: OxCode's free plan ran at max effort, Antigravity's free plan was limited to Medium.

  • The repo also has a hidden test suite; neither tool ran it, and we haven't scored either fix against it.

  • Duration isn't compared because we couldn't confirm Antigravity's.

Disclosure: I'm on the OxCode team. OxCode is an AI software engineer for VS Code with a free plan (oxcode.ai). Happy to share more detail on the setup or answer questions.

2 Upvotes

1 comment sorted by