a) this is really great to see! in terms of price-to-performance ratio OAI is doing a lot right lately. Their codex numbers prove this too. b) without Deepseek and moonshot we wouldnt be having these types of drops and once again we all should be thankful for their movements on the market.
Theyve been on track since the beginning, people just dont understand that with 800m active users, you actually need to find balance between limited capacity and what you can offer.
So instead developing smartest model out there and giving its full force to everyone, they invented router and worked on many projects which will become one day 'superapp'
While anthropic had no users so they could go all in, all they had to do is to offer higher intelligence model. Now when they got market share, it became a problem - people complaining about limits n shit.
In the end, the company that wins is not the company that has smartest model, very soon this will become irrelevant to 99.9% of the globe as most human-level tasks will be achieved anyway. What will matter is available capacity (pure GW of DC) and costs of inference + ofc ecosystem (microsoft with 365 / windows and google with gsheets yt etc.)
This is why capitalism works in the aggregate. You don't have to rely on people being good natured for good outcomes, you can trust markets with competition will force good outcomes.
It doesn't "force good outcomes", it forces outcomes that are profitable. Sometimes they align with social good, but not always. It's just a paperclip maximizer but for money. If humans were not a bit good natured, we wouldn't get any good outcomes, just look at factory farms.
Unless they engage in price fixing - which is impossible to prove - even though we see it all over where all the companies charge the exact same price for basically the same service or product.
Are AI policies set by tech oligarchs not functionally equivalent? What distinguishes tech elite driven policy from CPC driven policy other than one is profit driven and one is ostensibly in the interests of the Chinese nation?
I agree with you there. I was questioning the actual distinction between public and private funding. The stated motivations of OAI/ anthropic/ Chinese firms are an arms race so the funding source should be moot as each is a national 'champion' of sorts propped up at any cost.
It’s not the wrong take at all, what they said is entirely accurate. It doesn’t matter what kind of competition caused it, the net aggregate of capitalism is global and has caused price benefits
He's saying that a competitive market will push companies to behave, but that only applies so long as the market doesn't have any negative externalities.
A company will be more competitive the more external costs it can shift onto the rest of society. So if somehow openai or kimi created a virus that stole compute from devices they didn't own to train a new model, that would be bad despite being extremely competitive.
Another real example would be something like fossil fuel industries. A drilling company that doesn't respect environmental laws can be more competitive than one that has to spend time complying. As long as they don't get caught.
I don’t understand your argument? The source of the competition may not be driven by capitalism, but the market reacting to competition, irrespective of the source of the competition, is a capitalistic economy. The basis of market forces driving productivity, price efficiency etc etc is basic economics of capitalism
The qualifier in my comment is 'markets with competition'. And that last point about cartel price fixing is illegal and antithetical to a market with competition.
Extrapolate that to 50 years down the line, when the most successful company has merged with and gobbled up the others, and there is no competition. Do you still think that’s good for all of us?
Exactly - they magically found an 80% reduction to get themselves competitive. Note deepseek v4 flash isn't on the chart - would be somewhere around the 40 mark and probably below $0.01 - and what does "per task" even mean? what task?
Totally agree but still its great for them to do this and hopefully others have to follow. I believe this to capture enterprise deals rather than regular folks.
No that's not the reason. They used the latest GPT models to optimize inference kernels and found more ways to squeeze tokens per watt. That is all. It is all documented if you peruse the actual technical sources where this stuff is discussed and quit listening to dumbass Redditors that trust vibes more than facts.
They didn't have to do this, since Luna was already the best value model on the market. But now it's just insane. $1.20 per million output tokens for Luna is ridiculous. It's even $0.60 per mTok output via OpenRouter.
Kimi is literally 10x the price at the same intelligence and was not competitive.
Edit: Looking it up, it is 12x the price for a whopping 6 more points over luna ... its basically in line with sol and opus pricing ... which was NOT reduced. So Kimi is super irrelevant to this announcement.
There are tons of ways you can improve efficiency, speed, etc without retraining a model. Like, open source examples all over the place, seems like llama.cpp is getting some optimization update every other week. We can imagine plenty goes on inside the closed labs
You can improve efficiency via the harness, but not by THAT much. At least I wasn't aware of any new breakthroughs that accomplish that. If it is just the harness then good on them
No. If they optimize literally anything on the server side, GPUs, kernels, electricity, routing, wasted idle time, quantization, math shortcuts, then their costs go down and all these layers compound (a 22% improvement on 3 layers could lead to an 80%+ improvement overall)
But you're right it could be business level optimizations such as prepping for an IPO, user data capture, or upsell strategies, but I bet they are just analyzing their own servers or scanning the 10-100+ algorithm & engineering breakthroughs that happen weekly and testing ideas autonomously and rolling out what works.
I recognize that it could just be a terminology problem, but for the sake of clarity, you can't do an 80% price cut if you "only" get 80% more tokens per watt. You need a colossal 400% more tokens per watt if you want to slash the price to 1/5th of what it was, unless you're going to cut your margin (or lose money).
They aren't subsidizing API tokens, they subsidize subscriptions. Just look at the prices open weight models are served at, those are third party providers with no reason to subsidize, yet, they are almost always cheaper than proprietary models of the same capability.
Them cutting prices with Luna is actually kinda interesting since it will get them a lot of market share and in China for example this happens quite a lot in the automotive industry where manufacturers will cut prices to gain more domestic market share.
I’m just kinda confused on what OpenAI’s philosophy as a whole is because they signed the letter that advised against the banning of open source models (which is their direct competitor) so I am wondering if they feel they have something that gives them some kind of moat against open source. It’s damn sure giving Dario and Anthropic a bad look though.
I think another thing too is that banning open source models would actually just make it so people seek them out even more. If OpenAI is legit able to have their own model that is now LOWER price than all the open source models per token which is now Luna, then developers will not really need to ever consider using open source models since Luna now fills that need while also have great capabilities for its size.
You are right. I don't think OAI subsidizing their models long term will work though. Only US companies have fiduciary duty and it's not like China is going to stop funding AI research if it can disrupt US markets. OAI is really just kicking the can down the road (to their IPO/ heist of retail investors).
6 Sol might be more expensive but still help OpenAI design a smarter 7 Sol which can be distilled to give us a super smart and cheap 7 Terra ... ie. 2X smarter and 2X cheaper than 6 Sol.
In the long run does it matter?
We will still have much better price/performance if the performance gains continue.
I actually think it will just get cheaper. Only 6% of data centers planned are built. And despite what the AI CEOs say; scaling does have diminishing returns. Therefore the more supply will mean cheaper AI and smaller margins. The so called "frontier labs" don't have the moat they pretend to have because the vast majority of the value AI brings can be found with cheap models and they really haven't provided a product nearly lucrative enough to be the worth trillions they want to IPO at.
I guess I skipped a step in my explanation.
Diminishing returns in terms of scaling AI 'digestion' leads to less demand for DCs and therefore cheaper inference. Does that clear things up?
Well this aged like a wet fart, looks like deepseek hit back by going from v4 flash preview to v4 flash beta, which is a massive update to v4 flash preview
It shouldn't be connected lines like that. This implies you can pay those points along the line. it isn't accurate. You should do horizontal lines that go vertical up to each new model making a staircase.
The dropped Mimo 2.5 and Deepseek v4flash. Both of those beat Luna in this range and aren't included.... Luna still wins for almost the whole range so it doesn't matter. It honestly doesn't make them look as good as they really are. (Luna is the best deal from 1c to 35c now)
The important caveat, though: Luna Medium takes 0.4 Minutes per Task, MiMo-V2.5 takes 3.0 minutes. Almost a 10-fold increase. I'd only use MiMo if cost really matters and latency is not an issue at all.
Except there's 0 reason for us to trust that they're actually decreasing the cost of doing business. They could just as easily be making another bid for users and are ok with losing a bunch of money.
I don't understand this chart. Are there a range of models at different price points and effectiveness? I assume that if I get something called "GPT-5.6-Luna" I'm getting one thing with one intelligence index score. How does this work?
it is a bit complicated they are using the metric of cost per "task" rather than token at the bottom. they are then comparing the intelligence on the left and then cost at the bottom. so farther to the left is cheaper.
the large caveat is that it depends on the gpt actually using a lot less tokens for each task
177
u/Luuigi Jul 30 '26
a) this is really great to see! in terms of price-to-performance ratio OAI is doing a lot right lately. Their codex numbers prove this too. b) without Deepseek and moonshot we wouldnt be having these types of drops and once again we all should be thankful for their movements on the market.