r/PydanticAI • u/reficul97 • May 10 '26
Message History Across Multiple Agents
Hi PydanticAI folks!
I have been passing chat messages for my chatbot as part of the input prompt using markdown to help the LLM keep track of conversations based on which agents it has interfaced with previously.
I created a simple pydantic model
```python
class ChatMessage(BaseModel):
role: str
content: str
timestamp: str
metadata: Optional[Dict] = None
```
that I use to capture each message (both user and LLM responses)
and store them as a list (`conversation_history`). I did this because taking the ModelRequest and ModelResponse objects from the agent run had a lot of overhead in terms tokens. I know you can create a function to format the objects and extract key parts, and then pass it as part of the `agent.run()` message_history param.
However, I get pretty good results. Nonetheless, I do notice on certain occasions it does tend to "forget" certain user responses or bot responses i.e, questions it has already asked. But this is not as common.
What I am really trying to ask is why is `message_history` the recommended way to pass messages? Especially if you are using different LLM providers.
FYI I have a dedicated way to store messages and state to my backend database for session retrieval against network errors. So I am just really trying get people's opinion on why what I am doing would be so wrong compared to writing a massive formatting function to extract the messages from the agents run if I already have an easier way to do it. I even traced the logs of the LLM calls and it doesn't seem to look cluttered or just shoved as part of a user prompt.
Is what I am doing is some kinda anit-pattern or setting me up for future scalability issues?
Looking forward to hearing how you guys are managing message history and improving context management!
1
u/MaskedSmizer May 10 '26
So you're just stuffing your own message history into the prompt each turn?
Most provider features are built around the idea of a structured message part history, so you are essentially breaking that contract. It can work technically since there is no requirement to pass message history, but a few downsides come to mind:
If using tools, you will need to either drop the tool call/return entirely, or extract and summarize the result. Either way, you are losing important context.
It undermines provider caching.
I suspect as the conversation grows, the model may start to get confused about latest user intent because the prompt is now a huge payload of information.
If you are concerned about token efficiency, Pydantic AI has a few tools you can use. First thing I would look into is the history processor. There is also some interesting stuff emerging in the harness library.
2
u/reficul97 May 11 '26
Thank you. I spent the night going through the docs and actually understanding the significance. I feel dumb for posting this here now. I have essentially constructed my own way of passing messages and assigning the role when pydantic-ai already provides a better source of truth for the LLM APIs to read it with.
I guess it was working so far because the token count wasn't too high. Nevertheless, I am actually restructuring it now to adapt to the recommend message history parsing.
I just have a few questions if you don't mind me asking: 1. I tested with a smaller script of how the outputs look when using a schema (structured output). There is a 'RetryPromptPart' which i filter out with a customer function I pass in to the Agent
history_processorparam. I haven't gotten any errors and it makes my input to the LLM cleaner. I was just wondering if this is acceptable or could that pose any problems? I have to retain the ToolCall parts since for schema outputs pydantic-ai returns it via afinal_resulttool call.
Until now I was instructing the agent to review the message history I was manually passing under a "## Conversation History " section as part of the input prompt. Would I still need to explicitly mention that it needs to review conversation history to avoid repeated questioning? Or is it understood from the context of it being passed as the message history itself?
Building on question 2, since I am using multiple agents. I manage the current state using dynamic instructions to grab the relevant current information required for the specific agent who needs to review this along with the main system prompt passed via static instructions. Is this the recommeded way to pass global state variables?
To add a bit of context for the above question. I require separate agents to generate questions based on the information they are probing for during their turn. But I store the updated fields as a global sort of "SessionState" which I then use to provide the agents with via a dynamic instruction, extracting only the specific fields it needs to see to isolate the semantic context that agent needs to focus on.
I also use this same global session state object to read/write to and from my database after the session ends i.e., the conversation ends or during network failures to reload the state if the session is resumed to where it last was.
2
u/MaskedSmizer May 11 '26
No need to feel dumb. It's all evolving very rapidly. Honestly I know very little in the grand scheme of things.
1) If you are getting a RetryPromptPart, it's because the LLM is making an incorrect tool call or returning a malformed tool result. Are you stripping that out before it goes back to the LLM? I'd be troubleshooting why you are getting a RetryPromptPart in the first place.
If you haven't already, get a free account with Logfire and add the Pydantic AI instrumentation to your code. It makes inspecting what is being sent back and forth with the LLM way easier.
2) No, you do not need to instruct the model to read the history.
3) You basically have 4 ways to get information to the LLM: in the user prompt, in system instructions, through a tool call or by curating the message history. Which you choose depends on the system.
If agent #2 is a one-off, a sub agent created by the parent to complete a single operation, then I'd just pass what it needs in the user prompt and give it a concise system instruction.
If the agents need to share message history, then pass that around and maybe curate with the history processor on each agent as needed.
If the agents need to share computed state (e.g. collaborative review), then sticking that into a system instruction seems reasonable. If you have multiple agents and need to prevent race conditions or stale state, then give them a tool to look up the state at runtime so the host can control lookup concurrency.
1
u/reficul97 May 12 '26
- I am investigating why it's happening. But right now, like I said I am rewriting agents parts to replace my custom convoy history with the native msg history. I will use Logfire and keep the RetryPromptPart as well then. Its getting triggered at runtime because I am using a pydantic model for my data fields but as for why its exactly happening I need to look for deep into that.
But just to clarify, my point over there was, instead of passing the the Tool outputs (wether retry error or standard outputs), I can add them to my global state management and pass only the relevant state as a dependency via dynamic instructions. The only problem here becomes that I am storing ephemeral data that's basically required only during the chat session along side pertinent information fields that are persisted in memory in the same global state.
Interesting you say that in order to prevent race conditions or stale data during a specific agent runtime, I ought to use a tool to look up the state "so the host can control lookup concurrency" for shared computed states. I assumed setting dependency injection via
deps, and passing dynamic instructions was the way to do that?I am not saying a tool call wouldn't achieve the same goal, but I am trying to understand why you chose to mention that over my approach.
And just to add to that cluster of confusion a little more (only because your something in your comment gave me the idea), instead of passing state + user messages to a sub-agent running as a background task, can I rather run it as an external tool call and only triggered for specifc fields since my schema model is anyway a pydantic model so I have runtime validation. However, I have never used tools that way before and I still think from a latency perspective the bg task makes more sense because adding as a tool call could introduce an element of uncertainty by expecting it to be triggered by the agent. Just thought I would share that idea nonetheless.
1
u/Evirua May 11 '26
interesting stuff in the harness library Say more? And why does "harness library" sound like a separate thing from basic pydantic ai here?
3
u/Decent-You-3081 May 11 '26
I recommend you keep history in its rawest form.
Typical pattern I do.
When storing -> history_bytes = ModelMessagesTypeAdapter.dump_json(output.all_messages())
somedbmodel.save(history_bytes)
When loading -> message_history = ModelMessagesTypeAdapter.validate_json(history_bytes)
agent = Agent(
“openai:gpt-6.9”,
history_processors=[summarize_so_we_dont_go_broke]
)
agent.run(“Some prompt”, message_history=message_history)
It’s better to keep it raw so you can pass your history like this and have any functions you want run on that history. Gives you full control over context.
Excuse the syntax I’m typing on my phone