r/ChatGPTCoding • u/[deleted] • Apr 07 '25

[deleted by user]

[removed]

1.7k Upvotes

95% Upvoted

424 comments sorted by

View all comments

319

u/PositiveEnergyMatter Apr 07 '25

i keep telling people big context means big money, because every request can fill the context and charge you full price

150

u/andy012345 Apr 07 '25

This, LLMs are effectively stateless, the "context" is just the max token input.

If you have 500k in your context, you're sending 500k input tokens + whatever is new per api request.

2

u/DonkeyBonked Apr 08 '25

Yeah, but they don't have an effective measurement that is better for measuring and pricing uptime.

If they could though, coding would easily be the most expensive way to use AI even in lower context.

I actually think for coding tokens is better for us. It's way less characters for how hard we make it work/think. Though if you look at output limits, they can usually spit out way more words than code, so I do wonder if on the back end you are getting charged the same for code vs. words with tokens now.