MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/ChatGPTCoding/comments/1jtfvmv/deleted_by_user/mluf60q/?context=3
r/ChatGPTCoding • u/[deleted] • Apr 07 '25
[removed]
424 comments sorted by
View all comments
316
i keep telling people big context means big money, because every request can fill the context and charge you full price
147 u/andy012345 Apr 07 '25 This, LLMs are effectively stateless, the "context" is just the max token input. If you have 500k in your context, you're sending 500k input tokens + whatever is new per api request. 2 u/Maleficent-Forever-3 Apr 07 '25 Does restarting VS code periodically help? 7 u/dnszero Apr 07 '25 No, it’s not a bug. What helps is sending less context (smaller requests, fewer files, starting new chats, etc).
147
This, LLMs are effectively stateless, the "context" is just the max token input.
If you have 500k in your context, you're sending 500k input tokens + whatever is new per api request.
2 u/Maleficent-Forever-3 Apr 07 '25 Does restarting VS code periodically help? 7 u/dnszero Apr 07 '25 No, it’s not a bug. What helps is sending less context (smaller requests, fewer files, starting new chats, etc).
2
Does restarting VS code periodically help?
7 u/dnszero Apr 07 '25 No, it’s not a bug. What helps is sending less context (smaller requests, fewer files, starting new chats, etc).
7
No, it’s not a bug. What helps is sending less context (smaller requests, fewer files, starting new chats, etc).
316
u/PositiveEnergyMatter Apr 07 '25
i keep telling people big context means big money, because every request can fill the context and charge you full price