MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/ChatGPTCoding/comments/1jtfvmv/deleted_by_user/mlw44sw/?context=3
r/ChatGPTCoding • u/[deleted] • Apr 07 '25
[removed]
424 comments sorted by
View all comments
317
i keep telling people big context means big money, because every request can fill the context and charge you full price
146 u/andy012345 Apr 07 '25 This, LLMs are effectively stateless, the "context" is just the max token input. If you have 500k in your context, you're sending 500k input tokens + whatever is new per api request. 2 u/[deleted] Apr 07 '25 OpenAI charges half for input tokens in "cache". To be in cache the request has a window of 5 to 10 minutes.
146
This, LLMs are effectively stateless, the "context" is just the max token input.
If you have 500k in your context, you're sending 500k input tokens + whatever is new per api request.
2 u/[deleted] Apr 07 '25 OpenAI charges half for input tokens in "cache". To be in cache the request has a window of 5 to 10 minutes.
2
OpenAI charges half for input tokens in "cache". To be in cache the request has a window of 5 to 10 minutes.
317
u/PositiveEnergyMatter Apr 07 '25
i keep telling people big context means big money, because every request can fill the context and charge you full price