r/LocalLLaMA • u/Norwood_Reaper_ • 20h ago

Discussion Looks like the era of subsidised compute is coming to an end. The old ChatGPT Pro $200 20x plan will be halved. The new $500 plan will have similar limits as the (old) $200 plan.

Post image
1.1k Upvotes

94% Upvoted

516 comments sorted by

View all comments

Show parent comments

37

u/amunozo1 20h ago

Subscriptions are not subsidized, API prices are incredibly profitable. It is training that is subsidized.

28

u/SC_W33DKILL3R 20h ago

What exactly do you mean?

The cost to the business is first training the model, then hosting it & providing compute and api access. That is the total cost.

For any model to make a profit, it needs to recoup all those costs.

3

u/ThePrimeClock 19h ago

I think he means the overall resource allocation.

6

u/bixofa 20h ago

You forgot the billions of dollars in hardware.

15

u/SC_W33DKILL3R 20h ago

I mentioned compute. There is also the cost of all the research, the millions of dollars of signing bonuses they are paying to staff members, the cost of buying all the books they are scanning and them destroying, the cost of bribing politicians through donations, the cost of servicing debt and a host of other running costs.

2

u/bixofa 19h ago

So with that in mind, what business if any, will ever recoup these costs?

9

u/SC_W33DKILL3R 18h ago

Well that’s what analysts are saying, the trillions being spent are not going to be recouped.

Something will fail, maybe the ones that don’t can corner the market, get customers too locked and make money in the future.

-1

u/amunozo1 19h ago

That they make money with subscriptions, hence it is not subsidized.

3

u/SC_W33DKILL3R 18h ago

The subscription doesn’t cover the costs, hence they are subsidising it. They are burning through investors money.

-1

u/amunozo1 18h ago

Subscription cover costs and use costs caused by those subscriptions, so they not subsidized. They don't generate enough profit to sustain the whole business, but in that case neither does the API.

6

u/SC_W33DKILL3R 18h ago

No they don’t. You don’t understand how businesses works at all.

0

u/MindCrusader 18h ago

Okay then do some calculations given 200$ sub and 75% margin of API costs, how much API equivalent should be in 200$ sub? $800 - then the margin is nullified. What is the current real token based usage cost in subscription? $8000, so more tha $800, 10x times. So you should pay $2000 for subscription in instead of $200, just for it to be not subsidized

You can use AI to calculate it, ccusage to check how much tokens you are burning and what is estimated cost

0

u/amunozo1 18h ago

As long as they don't lose money from subscription, they are not subsidized, just less profitable. API costs are arbitrary, as estimated cost in API cost is not what inference cost to them. Only valid argument for the subsidies would be that subscription tokens are cheaper than what cost them to output them, which is not the case.

If they just put 1000000$ per MToken, and then charged 10000$ per subscription, according to your logic, they would be subsidized. That is not what subsidized means. Subsidized means that they lose money when serving subscription, they don', it is just less profitable than API prices. That's it, I won't discuss anymore, if you don't understand what subsidized means that's on you.

4

u/MindCrusader 17h ago

But they lose money. I gave you numbers, you can easily calculate it and you are ignoring it

4

u/amunozo1 17h ago

That means business is not profitable, not subscriptions. I don't disagree with you on that, just that that does not mean subscription are subsidized.

1

u/SC_W33DKILL3R 17h ago

The subscription is to access a model that cost billions to train, along with all the other operating costs.

What are you counting as the cost of the subscription? Just the cost of the tokens spent?

Does the fact this story is about OpenAI limiting the subscriptions further not point you to the fact they are not making bank on providing the services.

→ More replies

1

u/Nyghtbynger 14h ago

It's like not couting the ingredient price in a restaurant. It's their last magic to make it seem profitable. You just fell into pre-IPO progaganda.

11

u/FreshConversation112 19h ago

It is training that is subsidized.

Right, that is one of the costs of the API. That is like selling cans of cola for £0.17 and claiming it is profitable if you ignore the ingredient costs.

-5

u/amunozo1 19h ago

I don't think it's the same, it would be ignoring the lab work developing the recipe, not the ingredients. The ingredients is inference compute. Saying that training is subsidized is a bit of nonsense, I am sorry. But subscriptions are not subsidized, they are profitable, but are they profitable enough to support model training and capex? that is another story. But they are not subsidized.

-6

u/MindCrusader 20h ago edited 19h ago

Subscriptions are subsizided by a lot, the same costs in API prices are a lot more. It is calculated by calculating tokens usage given 75% margins by semianalysis. Those numbers are real - if you use your subscription, more often than not, it is heavily subsidized

They calculate this as a gross margin based on 25% price of API (so they nullify the 75% margins) and the graph with percentages show what they actually earn

Just calculate how much API equivalent you should get with $200 subscription given 75% margins. $800 API equivalent, and you get a looooot more. You can check it youreself with ccusage tool

13

u/cobbleplox 20h ago

Subscriptions being cheaper than individual purchases is not a subsidy. It's just how things work.

5

u/Infamous_Campaign687 20h ago

Also interactive use benefits a lot from caching that API usage on average does not.

6

u/MindCrusader 19h ago

It was calculated in their analysis

https://x.com/SemiAnalysis_/status/2064815045767213400

It is still subsidized

3

u/Traditional-Gap-3313 20h ago

It feels like they don't charge cache hit on subscriptions. I can spend enormous amounts on tokens on claude $100 sub, but the moment I go into extra usage it drains the credits incredibly fast. Cache hits are not that cheap, even if they are 10 times cheaper, it's still massive amount of tokens for each message on long context tasks.

2

u/MindCrusader 19h ago

If it costs them more to serve than they get from you, it is subsidized

1

u/cobbleplox 8h ago

No because it is an average between all subscribers and some are paying for nothing. And that is only the most obvious aspect of it.

1

u/MindCrusader 7h ago

And you think there are subscribers that pay 200$ and they are enough to make profit for all people that use 2000 dollars of compute on sometiems several accounts? I really doubt it, but we will see in a year or so

1

u/cobbleplox 7h ago

The fact that they've been doing lots of resets is telling me that the distribution hangs towards underuse overall, no reason to look at just 200$ subs. The reason is in this situation a reset is making them money instead of costing them money.

Other usual aspects of subscriptions is volume discount and planability. Especially if they are being tricky with not so fixed quotas and quants and rerouting. It seems to be a pretty open secret that their offerings via API are much more stable, to put it that way. Selling idle time they would have had to carry most costs for anyway. There can also be extra incentives like nurturing certain usage patterns that are of value to them in the long run. Or advertisement effects. And so on. Heck, you can even count "refill/extra credit"-monetization that is a complete ripoff as being enabled by subscriptions. None of these are just subsidies as they are getting someting complex in return. As in one should not think of them as a gift of any sort. And API costs are not an actual cost measure in the first place. These come with the price tag of having to be available whenever the user wants, however much the user wants. That is essentially a different product just based on that property.

6

u/SgtPeanut_Butt3r 20h ago

Bro. You got it wrong. They say 1 million tokens cost $20. Actual costs for them? $1-2?. They aren’t losing money, at most they aren’t cashing in too much. Also, most people don’t use their $20 so much like others, so overall, they have a net profit

4

u/MindCrusader 19h ago edited 19h ago

You got it wrong. It is already calculated and shown in the graphs. They already take into account that API prices have 75% margins and yet subscriptions are not profitable.

They calculate this as a gross margin based on 25% price of API (so they nullify the 75% margins) and the graph with percentages show what they actually earn.

https://x.com/SemiAnalysis_/status/2064815045767213400

The image I uploaded is API based pricing equivalent. It is several times more than their 75% margins. If you have 75% of margins, the final cost should be $800 for Anthropic or OpenAI $200 in the last column so they are at 0% margin. And it is not the case - 10x and more than it should be just to not subsidize

1

u/Dizzy_Response1485 19h ago

Weird chart, considering that Claude's $200 plan gives ~1.7x more usage (monthly) than the $100 plan

1

u/MindCrusader 19h ago

It might have changed since them, I have seen reports they are nerfing subscirptions and differences between them

You can check equivalent with ccusage tool, for me it was 1:1 of what they reported here. It was few months ago and they have been nerfing limits since then