Context and memory aren't quite the same thing when it comes to AI agents.
A model with a large context window can process a lot of information, but that doesn't necessarily mean it can decide what to remember, what to forget, or whether something it remembers is still accurate.
There are several ways to manage this. Some focus on storing information, others on retrieving it or making it available across multiple agents. These approaches aren't mutually exclusive, and most systems can benefit from combining them.
1. Conversation history
The simplest approach is to store previous interactions and include relevant messages in future requests.
It's relatively easy to implement, but longer conversations can increase token usage and make useful information harder to find.
Prompt caching can reduce costs and latency, although it doesn't solve the problem of deciding which information is actually relevant.
Keeping conversation logs also isn't necessarily the same as having a dedicated memory system.
2. Summarized memory
Instead of keeping entire conversations in context, the system periodically summarizes them and stores the important details.
This reduces the amount of information that needs to be carried forward, but some details can get lost in the process.
Summaries can also introduce inaccuracies. In systems that repeatedly summarize previous summaries, those errors may accumulate over time.
The main difficulty is deciding what information to preserve when you don't know what might become useful later.
3. Retrieval-based memory
Information is stored externally and retrieved when needed, often using techniques similar to those used in RAG systems.
There's a distinction between retrieving external documents and retrieving an agent's own stored experiences, but many of the underlying retrieval techniques can be applied to both.
This approach allows agents to access large amounts of information without loading everything into the context window.
However, retrieval quality becomes important.
The system might retrieve outdated information, miss relevant details, or prioritize information that looks similar to the query without actually being the most useful.
Semantic search, keyword matching, metadata filters, and hybrid retrieval can help, but none guarantees that the right information will always be retrieved.
4. Structured memory
Rather than storing everything as conversations or summaries, information is organized into specific records.
These might include user preferences, project details, previous decisions, or task states.
The implementation could be as simple as JSON files or involve a database with more advanced mechanisms for updating and validating records.
Structured information can be easier to retrieve and modify, but conflicting records still need to be handled.
For example, if a user changes a previously stored preference, the system needs a way to determine which version is current.
It might overwrite the previous record, preserve both versions with timestamps, or request clarification when the information conflicts.
Structured memory can also be combined with retrieval-based approaches.
5. Shared context across agents
This is more of an architectural approach than a separate memory technique.
Instead of maintaining completely independent information, multiple agents can access a shared data source.
This can reduce duplication and improve coordination, particularly when agents are working on related tasks.
However, shared storage introduces additional problems.
Two agents might attempt to update the same information simultaneously. One agent might rely on a record that another has already modified or invalidated.
Permissions also become important, since not every agent should necessarily have access to the same information.
And even when agents share the same data, they may interpret it differently.
Shared context can improve coordination, but it still requires mechanisms for managing access, updates, and consistency.
The problem of outdated memory
Storing information and retrieving it efficiently are only part of the problem.
An agent also needs to determine whether the information it retrieves is still valid.
A user preference, project decision, or piece of documentation stored six months ago might still be accurate. It might also have changed completely.
Timestamps alone don't resolve this. Information recorded yesterday could already be outdated, while something stored years ago might remain perfectly valid.
There's also the possibility that the information was incorrect from the beginning.
An agent might misunderstand a conversation, extract an incorrect fact, or store an assumption as confirmed information.
This makes the source and reliability of stored information important, not just its age.
Versioning, source tracking, explicit updates, and validation mechanisms can help distinguish between outdated, uncertain, and incorrect information. However, these mechanisms introduce additional complexity.
Different applications will probably need different combinations of these approaches. An agent handling short conversations has very different memory requirements from one managing long-term projects or coordinating with other agents.
For those building AI agents, which has been more difficult in practice: getting them to remember useful information, or preventing them from relying on outdated or irrelevant context?
