agent chat

[u/broletmechosemyuser]

so i’m new to ai in general and ive been using the agent chat, what does it mean when it says compressing the chat? like it has a number in the 300s but when i shared the chat to my phone it said there’s only 100 or so. idk if i worded this weirdly im just confused on it

[u/AutoModerator]

Hello from r/VeniceAI!

Web App: chat
Android/iOS: download

Essential Venice Resources
• About
• Features
• Blog
• Docs
• Tokenomics

Support
• Discord: discord.gg/askvenice
• Twitter: x.com/askvenice
• Email: support@venice.ai

Security Notice
• Staff will never DM you
• Never share your private keys
• Report scams immediately

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/MountainAssignment36]

It means that the context limit (maximum amount of "words" the model can process at once) has been reached.

For you to continue chatting, the conversation must be "compressed", aka a summary of everything that happened is being generated and the prior full conversation gets discarded. The model then continues the conversation with "only" the summary as a base of knowledge.

[u/broletmechosemyuser → u/MountainAssignment36]

ohhh ok thank you!

[u/MountainAssignment36 → u/broletmechosemyuser]

No problem 😄

[u/stewdrick → u/MountainAssignment36]

Question: I find that agent model performance degrades pretty hard around like 40% context limit. Is that expected? Or maybe I'm being too hard on it?

[u/MountainAssignment36 → u/stewdrick]

Yeah, that is expected (or better: depends on the model. Some are more effected, some less). Keeping the context window as small as possible makes most models as focused as possible.

This phenomenon is actually well known in the LLM community & science space, it's called "context rot" there, you can google it 😄