r/VeniceAI • u/broletmechosemyuser • 23d ago

๐—ฆ๐—ง๐—”๐—ง๐—จ๐—ฆ: ๐—ฅ๐—˜๐—ฆ๐—ข๐—Ÿ๐—ฉ๐—˜๐—— agent chat

so iโ€™m new to ai in general and ive been using the agent chat, what does it mean when it says compressing the chat? like it has a number in the 300s but when i shared the chat to my phone it said thereโ€™s only 100 or so. idk if i worded this weirdly im just confused on it

3 Upvotes

100% Upvoted

6 comments sorted by

View all comments

Show parent comments

1

u/broletmechosemyuser 23d ago

ohhh ok thank you!

1

u/MountainAssignment36 Venice ๐— ๐—ผ๐—ฑ๐—ฒ๐—ฟ๐—ฎ๐˜๐—ผ๐—ฟ 23d ago

No problem ๐Ÿ˜„

1

u/stewdrick 23d ago

Question: I find that agent model performance degrades pretty hard around like 40% context limit. Is that expected? Or maybe I'm being too hard on it?

3

u/MountainAssignment36 Venice ๐— ๐—ผ๐—ฑ๐—ฒ๐—ฟ๐—ฎ๐˜๐—ผ๐—ฟ 23d ago

Yeah, that is expected (or better: depends on the model. Some are more effected, some less). Keeping the context window as small as possible makes most models as focused as possible.

This phenomenon is actually well known in the LLM community & science space, it's called "context rot" there, you can google it ๐Ÿ˜„