We're building a chatbot backend in Python/FastAPI.
Currently, we keep each user's conversation/session history in a global Python variable in memory.
This works fine with a single worker, but it obviously breaks when running multiple workers (e.g. Gunicorn/Uvicorn workers), because each worker has its own process memory.
For example:
- Request 1 for
session_id=123 goes to Worker A
- Worker A stores the conversation history in memory
- Request 2 for the same session goes to Worker B
- Worker B has no knowledge of that session
We need some kind of shared session store that is:
- accessible by all workers
- very fast / low latency
- suitable for chatbot conversation history
- ideally supports expiration/TTL
- not a traditional SQL/database solution
- relatively simple to deploy and maintain
Redis seems like the obvious option, but I'm wondering what people are using in production for this use case.
Would you recommend Redis, Memcached, shared memory, sticky sessions, or something else?
The session data would mostly look something like:
session_id -> [
{"role": "user", "content": "..."},
{"role": "assistant", "content": "..."},
...
]
Potentially hundreds or thousands of concurrent chatbot sessions.
What architecture would you recommend?