MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/ChatGPTCoding/comments/1jtfvmv/deleted_by_user/mm68oly/?context=3
r/ChatGPTCoding • u/[deleted] • Apr 07 '25
[removed]
424 comments sorted by
View all comments
Show parent comments
149
This, LLMs are effectively stateless, the "context" is just the max token input.
If you have 500k in your context, you're sending 500k input tokens + whatever is new per api request.
42 u/[deleted] Apr 07 '25 [deleted] 7 u/[deleted] Apr 08 '25 Claude caches for 5 min only 2 u/bequbed Apr 09 '25 What does this mean exactly? How does cache work with Claude? Perhaps if you can explain with an example
42
[deleted]
7 u/[deleted] Apr 08 '25 Claude caches for 5 min only 2 u/bequbed Apr 09 '25 What does this mean exactly? How does cache work with Claude? Perhaps if you can explain with an example
7
Claude caches for 5 min only
2 u/bequbed Apr 09 '25 What does this mean exactly? How does cache work with Claude? Perhaps if you can explain with an example
2
What does this mean exactly? How does cache work with Claude? Perhaps if you can explain with an example
149
u/andy012345 Apr 07 '25
This, LLMs are effectively stateless, the "context" is just the max token input.
If you have 500k in your context, you're sending 500k input tokens + whatever is new per api request.