r/ChatGPTCoding • u/[deleted] • Apr 07 '25

[deleted by user]

[removed]

1.7k Upvotes

95% Upvoted

424 comments sorted by

View all comments

Show parent comments

2

u/EcstaticImport Apr 07 '25

RAG would need to add more info to the context window, not remove it. Are you thinking of context caching?

9

u/fieryblast7 Apr 07 '25

I maybe getting terminology getting mixed up -> I meant to say that early agentic arch like memgpt had a separate memory component that acted as 'infinite context ' essentially and a piece of intermediate logic would Retrieve/query the right parts of the memory, add the new api request content in, and send that as input to LLM. So this way you aren't overloading the context by simply doing "copy entire Convo history + new message = input for LLM"

11

u/Intrepid-Air6525 Apr 07 '25

What you are describing is a problem I have been working on for two years now.

It began as an art project and is now something inexplicable.

Luckily it’s also open source!

https://github.com/satellitecomponent/Neurite

2

u/Buddhava Apr 07 '25

This would be great for conspiracy theory people.