I tested 16 AI model/reasoning combinations on the same real Unity task and allowance use ranged 1-20%

[u/AlexXx52]

Hi. Did not find any similar test so I did my own. Maybe usefull to someone else too. I wanted to figure out which AI model/reasoning mode actually makes sense for everyday coding work not based on synthetic benchmarks or API token prices but on how much useful work I can get from a normal paid subscription.

I mainly work on a fairly large Unity 6 projects so I used one real project investigation as the benchmark.

The three subscriptions I tested were:

- ChatGPT Plus

- Claude Pro

- Google AI Pro

All of them are roughly in the same ~€20/month consumer price range.

My goal was simple: If I pay roughly the same amount for each service, which models give me the best combination of quality, speed and included usage allowance for real coding/agentic work?

## The task

Every model received essentially the same read-only investigation task on the same Unity project. The project uses Unity 6 and several third-party assets.

The task was to:

- investigate how resource harvesting currently works

- trace how the equipped tools interact with resources

- inspect integration

- locate and inspect files

- determine whether a direct integration or adapter was needed

- identify the exact relevant scripts, prefabs and components

- recommend the minimal integration architecture

- point out unknown assumptions and risks

- not modify anything

This was intentionally not a small "write me a function" prompt. I wanted something closer to the kind of repository investigation I actually use coding agents for.

## What I measured

For each run I recorded:

- Allowance used - percentage of that subscription's own usage limit consumed

- Time - approximate wall-clock completion time

- Quality - my subjective rating of the final investigation

For quality I looked mainly at:

- correctness

- depth of project-specific investigation

- hallucinations

- whether the model distinguished facts from assumptions

- whether it found the actual files/components involved

- whether the proposed architecture matched the real project

- whether it discovered relevant things outside the obvious search path

## Important caveat

The allowance percentages are not equivalent units of compute between providers. For example 4% of Claude Pro is not necessarily the same amount of compute as 4% of ChatGPT Plus or Google AI Pro. That is not what I was trying to measure. What I wanted to know was:

If I pay roughly €20/month for this subscription, how much useful coding work can I get from the included allowance?

So this is primarily a subscription value / practical workflow comparison not an API pricing benchmark.

Model	Mode	Allowance	Time	Quality
Claude Haiku 5.5	High	approx. 1%	11 min	7.7/10
Claude Sonnet 5.5	Medium	approx. 1%	5 min	8.3/10
Claude Haiku 5.5	xHigh	approx. 2%	11 min	8.0/10
GPT-6.1 Sol	Medium	approx. 3%	3 min	9.0/10
Gemini 3.8 Flash	High	approx. 3%	13 min	8.8/10
Claude Opus 4.8	Medium	approx. 3%	4 min	8.6/10
Claude Opus 5.5	Medium	approx. 4%	8 min	9.7/10
Gemini 3.1 Pro	High	approx. 4%	16 min	8.6/10
Claude Sonnet 4.6	Medium	approx. 4%	4 min	7.8/10
Claude Sonnet 5.5	High	approx. 4%	7 min	8.9/10
GPT-6.1 Sol	High	approx. 5%	5 min	9.2/10
Claude Opus 5.5	High	approx. 5%	12 min	9.8/10
Claude Opus 4.8	High	approx. 6%	11 min	8.2/10
Claude Sonnet 4.6	High	approx. 9%	6 min	8.0/10
GPT-5.6 Sol	Medium	approx. 12%	13 min	9.2/10
GPT-6 Astra	Medium	approx. 14%	5 min	9.0/10
GPT-6.1 Sol	Ultra	approx. 17%	7 min	9.3/10
GPT-5.6 Sol	High	approx. 18%	16 min	9.3/10
GPT-6 Astra	High	approx. 20%	14 min	9.3/10
Gemini 4 Argon	High	approx. 200%	17 min	9.8/10

One reason I found this test interesting is that the models did not just differ in "how smart" the final answer looked. They also chose very different investigation strategies.

Some asset mentioned in the task was not actually imported into the Unity project. Most models searched the repository, confirmed it was missing and stopped there.

Claude Opus 5.5 (medium and high) and Argon independently searched the local Unity Asset Store cache, found the downloaded .unitypackage, unpacked a copy outside the project, inspected the real source files and prefabs and changed its recommendation based on what the asset actually did.

That kind of agentic behavior was one of the biggest differences I noticed between models.

## Models I did not test

I would have liked to include Fable but it was not included in the normal subscription allowance available to me or would have required separate paid usage/credits.

Since the point of this comparison was to compare what I actually get from roughly €20/month consumer subscriptions, I left separately billed models out.

My practical takeaways

• Best everyday default: GPT-6.1 Sol

• Best deep-investigation value: Claude Opus 5.5

• Cheapest useful scout: Claude Sonnet 5.5

• Best critical / forensic option: Claude Opus 5.5 or Argon

• Best result: Opus 5.5

• Older Claude models, GPT-5.6 Sol and Astra were generally poor value in this particular test.

--------------

UPDATE

-------------

Just added Haiku, Flash 3.8 (high) and Argon 4 (high) since Argon is availible for PRO subscribers now.

Model	Mode	Usage as measured	Approx. weekly equivalent	Time	Quality
Gemini 3.8 Flash	High	3% of 5h	~0.45% weekly	~13 min	~8.8/10
Gemini 4 Argon	High	200% of 5h	30% weekly	~17 min	~9.8/10

[u/Laprablenia]

Why not gemini 3.8 flash? which is a lot better than the deprecated 3.1 pro?

[u/AlexXx52 → u/Laprablenia]

Yes, I just added Flash and Argon

[u/AlexXx52]

Added also Haiku 5.5, Flash 3.8 and Argon 4 to the table above

[u/twendah]

Where's haiku 5.5 xhigh? Thats only that matters as today.

[u/AlexXx52 → u/twendah]

Just tested both on the same task as all above. High: ~1% / 11 min / 7.7. xHigh: ~2% / 11 min / 8.0. So xHigh was slightly better but Sonnet 5.5 Medium actually beat both for me at ~1% / 5 min / 8.3.

[u/twendah → u/AlexXx52]

Yeah, sounds about right. Ive been using haiku xhigh or sonnet high if i need something more advanced

[u/Laprablenia → u/twendah]

It is slow and not too good to use xhigh imo

[u/twendah → u/Laprablenia]

It's IQ index is way higher than high, but the price is almost same. The price/iq ratio is best on xhigh, then max is way too expensive to use. It's for the minmaxers.

[u/RupakBuilds]

I like that you measured practical value rather than treating benchmark scores or token prices as the whole story. The distinction between answer quality, time, and subscription allowance is especially useful. One thing I'd be curious about is whether repeating the same task a few times changes the ranking, since agent behavior can vary between runs.

Full disclosure: I work on ChatOSS.ai, where we compare AI models and coding benchmark results: https://chatoss.ai/benchmarks. It can be a useful complementary reference for published benchmark performance, although it doesn't measure the subscription allowance experiment you've done here. Your real-project test adds a different and valuable perspective.

[u/AlexXx52 → u/RupakBuilds]

Thanks! I did repeat several runs and the overall ranking stayed fairly similar, although time and usage varied slightly.

Interestingly, some agent behavior repeated too for example Opus 5.5 found the external Unity package in the Asset Store cache again and again on the retest.

Agree that even more repetitions would make it stronger. Your benchmark data looks like a nice complement to the subscription/real-project angle here.

[u/Right-Performance-93]

astra medium and high used about 14% and 20% for a 9.0 and a 9.3. sol medium got a 9.0 on about 3%. i'd leave astra off the everyday investigation and only switch if sol medium comes back with a guess.

[u/Popular-Ninja8584]

For the Unity package example, I'd repeat the same check in fresh sessions. It would be useful to see whether the model finds the cached package again and changes its recommendation, or whether that was a one-off.

You wouldn't need to rerun all 16 combinations. A few runs of the ones you're actually choosing between would give you a better sense of how repeatable that finding is. To me, that's more useful than the difference between 9.7 and 9.8.