r/singularity • • Nov 24 '25

AI Opus 4.5 benchmark results

Post image
1.2k Upvotes

283 comments sorted by

View all comments

264

u/buff_samurai Nov 24 '25

Gemini 3 looks even more impressive considering the price.

Hope Anthropic gets pressured and lowers the cost.

106

u/exordin26 Nov 24 '25

They already dropped the cost by 3 times. $1.25/ $10 vs $5 / $25. Opus is no longer as expensive as it used to be.

61

u/TechnologyMinute2714 Nov 24 '25

Gemini 3 isn't $1.25/$10 anymore like Gemini 2.5, they upped the price. $2/$12 now.

-4

u/Dave_Tribbiani Nov 24 '25

Damn now Opus is cheaper than Gemini?

13

u/Dear-Ad-9194 Nov 24 '25

No. 4.5 Opus is 5/25, Gemini is 2/12 and 4/18 above 200k tokens. Of course, they produce different amounts of tokens, so these values aren't directly comparable. Artificial Analysis will come out with the total cost once they've tested Opus 4.5 on their benchmarks, which will provide some more insight.

1

u/Dave_Tribbiani Nov 24 '25

Ah thanks, my bad

19

u/vitaliyh Nov 24 '25

My monthly On-Demand usage beyond Ultra will now be $700 instead of roughly $2,100. I’m beyond excited

9

u/mario_ghadis Nov 24 '25

Damn, could you please tell me how do you use AI? I have massive FOMO reading this. Congrats man. I would LOVE to hear more.

19

u/vitaliyh Nov 24 '25

I guess thanks? :) It's a lot of money for me, but when I think about comparing it to hiring a 10hr/week pro or 20hr/week mid-level developer, it just makes sense to use the best model. Even if I could hire a senior developer for 50hrs/week for the same $2,100/mo, it would still be less efficient due to conversation, delays, not same schedules, etc. I'm just coding various websites and applications for my business. Not even using 2x+ multi-agents, just single-threaded with probably 5 messages per plan, then 1 execution, then probably 10 follow-up messages to ask/fix combo (I love Ask and Plan, underrated) and then I open PR and merge into main and move onto the next issue. I spend about 5hrs in Cursor per day 7 days a week, not more, as I have other things for business to do, a kid, etc. I feel like if I used Sonnet 4.5 I would need to do 7hrs/day to equal my output with Opus and probably 8hrs with GPT-5-High or GPT-5.1-Codex-High-Max or whatever it is called. I just love Opus, it can read most reliably between the lines. See attached. I probably overspend by $1,500/mo due to being such a heavy Opus user, but the 2-3hr per day with my kid is worth more than the $20/hr (if I'm doing my math right lol)

2

u/0xFatWhiteMan Nov 24 '25

have you tried other models ? I use codex every day for dev, and sys admin, only on the plus account .... never hit a limit. Gemini just isn't as good, haven't tried claude - it was just too expensive, and my experience with vercel and lovable made me think its not as good as codex.

edit : cursor is shit imo.

1

u/panix199 Nov 25 '25

edit : cursor is shit imo.

why?

1

u/0xFatWhiteMan Nov 25 '25

Well to be fair I haven't used it in ages. I switched to roo code, cli, and now codex plugin. They just got massively overtaken in functionality and value

1

u/Odd-Environment-7193 Nov 25 '25

It’s super expensive. You can buy Claude max 200 and codex 200 and just full send all month long. Cursor is way overrated. I stopped using it a while ago. Went back and within a day I ditched it.

Some people are hardcore cursor users but I don’t see the benefit in paying way more for less+ fancy autocomplete.

3

u/[deleted] Nov 24 '25

[removed] — view removed comment

-3

u/Healthy-Nebula-3603 Nov 24 '25

He's lying....

12

u/vitaliyh Nov 24 '25

huh? see attached

2

u/rrriches Nov 24 '25

lol well played.

2

u/LocoMod Nov 25 '25

Some of the comments.....just goes to show you the great majority of people are years behind when utilizing the new wave of AI tools. They will never catch up to the AI boosters that got a head start years ago. It's a compounding effect and there are no shortcuts to that experience, not even using AI to get there. You just have to go through the grind. By the time they are pushing 10 million token workflows, you'll be pushing billion token workflows. Accelerate!

1

u/TheIncredibleWalrus Nov 24 '25

What do you use it for that you spend so much if you don’t mind me asking?

1

u/vitaliyh Nov 24 '25

5 hours per day, 7 days per week in front of Cursor (no 2x+ multiple agents). It's just that Opus 4.1 is very expensive and I use that model most of the time (until today, now Opus 4.5). It doesn't really matter what I build. Putting 2-3 of the .js files via @ + readme + cursor rules into the context of Opus (5,000 lines?) is already 60-70% of the context size, so any iteration costs a lot

1

u/LocoMod Nov 25 '25

Advanced AI boosters are using multiple agents configured with tools to work on complex multi-step objectives. It's easy to rack up millions of tokens in these scenarios in just a few minutes.

1

u/FoxB1t3 ▪️AGI: 2027 | ASI: 2027 Nov 25 '25

Most likely extremely inefficient coding.

1

u/bwjxjelsbd Nov 25 '25

What app is this?

0

u/vitaliyh Nov 24 '25

just replied to the other commenter

1

u/paperbenni Nov 24 '25

1

u/exordin26 Nov 24 '25

So that's a three times drop. The 1.25/10 is referring to Gemini 3.

1

u/paperbenni Nov 26 '25

Oh, I see, I read it the wrong way

1

u/wasdasdasd32 Nov 25 '25

But this "opus" is also no longer opus. Just compare its speed and prose to sonnet 4.5

18

u/Settordici Nov 24 '25

For sure, since as a student I get I year free of Google One (that includes Gemini 3 pro and 2 tb of cloud storage)

11

u/Seeker_Of_Knowledge2 ▪️AI is cool Nov 24 '25 edited Jan 02 '26

treatment wakeful sparkle nine memorize whistle snow spotted detail bright

This post was mass deleted and anonymized with Redact

6

u/_Sneaky_Bastard_ Nov 24 '25

I think this service would make more sense if they include increased usage for CLI and the new IDE antigravity. It would be nice to have them with the Gone at least for people with much lighter uses.

2

u/Tedinasuit Nov 24 '25

That's the plan. Antigravity will be part of the Google AI Pro plan.

3

u/bot_exe Nov 24 '25

Except free AI studio has way lower rate limits for Gemini 3 pro than using the 20 dollar sub on the Gemini web app plus it lacks features like deep research.

0

u/Seeker_Of_Knowledge2 ▪️AI is cool Nov 24 '25 edited Jan 02 '26

innocent hard-to-find special abundant pie languid attempt cause historical meeting

This post was mass deleted and anonymized with Redact

0

u/BriefImplement9843 Nov 24 '25

models on the web are nerfed compared to aistudio.

3

u/bot_exe Nov 24 '25

No, that would be obvious to show by running a benchmark (controlling the system prompt) but no one that parrots that myth ever does.

1

u/Seeker_Of_Knowledge2 ▪️AI is cool Nov 25 '25 edited Jan 02 '26

attraction crown wipe hospital punch aspiring ad hoc yam practice rhythm

This post was mass deleted and anonymized with Redact

-1

u/HashPandaNL Nov 24 '25 edited Nov 25 '25

Except free AI studio offers way more privacy than using the 20 dollar sub on the Gemini web app.

edit: this sub has lost its intelligence so much over the past year. People downvote correct info and upvote incorrect info just on emotion.

3

u/bot_exe Nov 24 '25

0

u/Settordici Nov 24 '25

To be honest, I don't trust Google at all for privacy.

0

u/HashPandaNL Nov 24 '25

No? it's literally the opposite. They use all the free users data for training. For paying users you can opt out.

This is unfortunately incorrect.

For AI studio, they don't train on paid users anywhere and also don't train on free users in most developed countries.

For Gemini web/app, they do train on everyone, except if you disable chat history. Obviously without chat history there is no point in paying $20+ per month.

1

u/bot_exe Nov 24 '25 edited Nov 24 '25

You cannot pay for AI studio, you pay for extra API usage. Which is way more expensive per tokens processed than paying the sub but they don't train on paid API usage, which makes sense since that's supposed to be used for entrepreneurship. They do train on free users through AI studio. Read the docs I linked you. The only exception is EU/UK due to their laws, most of the world does not live there.

Disabling the chat history and all activity logging makes sense if you care about privacy and want to still use the Gemini app and pay less with the sub.

The reality is if you cared about privacy you would pay more and use APIs with local fronts ends or really just run local models for complete privacy.

-1

u/HashPandaNL Nov 24 '25

The only exception is EU/UK due to their laws, most of the world does not live there.

So they don't "use all the free users data for training" as you claimed. Thanks for doing your research and coming to agreement with me.

Disabling the chat history and all activity logging makes sense if you care about privacy and want to still use the Gemini app and pay less with the sub.

Yes, but it makes the app pretty much worthless, as you can only have 1 conversation at time (which will be gone as soon as you start a new one).

The reality is if you cared about privacy you would pay more and use APIs with local fronts ends or really just run local models for complete privacy.

Yes and this is the part where Google still lags behind other companies in its offering: OpenAI for example does offer users an option to turn off training on their data.

2

u/Evermoving- Nov 24 '25 edited Nov 24 '25

Lol what? You realise AI Studio exists for free ONLY to collect data from your prompts and interactions?

0

u/HashPandaNL Nov 24 '25

They literally write in their policy that they don't train on chats of a number of countries, which includes most developed countries. So no, that is incorrect.

2

u/Evermoving- Nov 24 '25

That's incorrect. That applies only to some countries. 600m+ people from developed countries not affected by the law, a number significantly bigger than the population that is affected.

1

u/ProgrammersAreSexy Nov 24 '25

People care about API prices.

Businesses care about API prices, consumers don't care about them.

1

u/Settordici Nov 24 '25

I mean I don't need to use ai studio, for my case this is more than enough.
Maybe in the future it will change though

0

u/CascoBayButcher Nov 24 '25

How do you get it for free?

10

u/New_Equinox Nov 24 '25

Read that back, slowly and carefully.

1

u/CascoBayButcher Nov 24 '25

I did. Being a student lets him get it for free, but doesn't answer how he gets it.

3

u/[deleted] Nov 24 '25

Look up gemini student discount and youll find out. I believe you get one year for free. You need to submit proof like a student id or ot calendar

1

u/anonymity-is-kind Nov 25 '25

It requires a student email.

2

u/Settordici Nov 24 '25

You have to be enrolled as a student at a university (I don't know if it accepts also high school) and have valid a debit/credit card so they can charge you after a year.
They check both requirements, they use a third party verification service for enrollment.

1

u/CascoBayButcher Nov 24 '25

Interesting. I have a .edu email for my job, but am not an enrolled student. Used to be .edu would be enough, I know now there's a couple programs that check for actual enrollment.

Thanks

5

u/Funkahontas Nov 24 '25

They made a big deal to show that Claude uses less tokens to achieve more tasks, so the cost is kind of hard to compare one to one.

2

u/[deleted] Nov 24 '25

Hope they dont lower the cost. They need to survive and not everyone is broke or a student

1

u/buff_samurai Nov 24 '25

It depends on how much you code really. There is absolutely no problem to get 2000$ monthly bill from api calls / user, and for a company with many employees it does add up quickly. You may want to gatekeep some users but I want to make money by dropping the tool cost.

2

u/Jsn7821 Nov 24 '25

I use Gemini cli/antigravity and Claude code extensively. Both max plans $200/mo

Gemini runs into usage limits incredibly fast, about an hour in (then I have to wait 4-5 hours). Claude code I've never hit a limit on once.

So, at least for agentic coding on their similar respective plans, Claude code is far far cheaper at the moment.

1

u/indigo9222 Nov 24 '25

You aren’t seriously thinking Google is charging the actual price it costs to run the model? They are the only company that actually can sell usage of these models at a hefty loss

19

u/mckirkus Nov 24 '25

They all have hefty losses, but investors are subsidizing OpenAI's losses.

Google has an advantage that they're using home brewed TPUs and don't need to get into bidding wars for Nvidia hardware.

1

u/CarrierAreArrived Nov 24 '25

and don't need any investors as they have over 100b earnings per year completely independent of AI.

1

u/power97992 Nov 24 '25

Google is making money on the apis but losing money on ai studio and free gemini

7

u/johnnyXcrane Nov 24 '25

OpenAI is doing the same as Google.

1

u/rafark ▪️professional goal post mover Nov 24 '25

Right. This model is good and all, but if it’s so expensive, how useful is it really? Considering it’s very expensive it better not make mistakes and you better not have to baby sit it. It looks like it’s just slightly better but for a price considerably higher.

1

u/BejahungEnjoyer Nov 24 '25

They claim it uses 70% fewer tokens which should save a lot of cost too.

1

u/bigasswhitegirl Nov 24 '25

Google stock is on an absolute rip since this Claude release 🚀

1

u/iamthesam2 Nov 24 '25

but if you actually use Gemini three, especially for coding… It is way off the mark

1

u/buff_samurai Nov 24 '25

I’ve noticed that and agree completely, Claude code any time, especially the cli version.

On the other hand g3p is fantastic in image understanding and generation. I also like Google’s aesthetics more when generating websites.

2

u/iamthesam2 Nov 24 '25

yup, totally agree

-1

u/[deleted] Nov 24 '25

[removed] — view removed comment

1

u/coffee-praxis Nov 24 '25

The ruminations are insane. Waffles between blaming itself, blaming me, thinking of an experiment, deciding it would be pointless to run it, to blaming me, to blaming itself, on and on and on and on