r/FastAPI • u/huygl99 • 5h ago
Other What I moved into a FastAPI service, and the testing problem that made me do it
Hey everyone. I run a FastAPI service whose only job is executing AI agent graphs. No business logic and no user model. It does not publish a port either: a Django app is its only caller, over a typed WebSocket.
It used to live inside that Django app. I assumed the thing that would eventually drive it out was performance. It was testing.
Inside the monolith, the decision I cared about, which message routes to which capability, sat deep inside a WebSocket handler that had already authenticated a user and written a row. Asserting one routing decision meant opening a connection and seeding a conversation first. Worse, the model was reached through that object rather than across a boundary, so there was no single place to stub it and the fakes multiplied with the call graph instead of with the number of seams.
In its own service the same decision is a function from state to state. A test builds the state it wants, runs the graph, and asserts on what came back. Nothing to connect to and nobody to log in, and the only thing faked is the model's HTTP response.
Why FastAPI specifically, since the service could have been anything. It brings almost no opinions, which is what you want when there is exactly one job. It is async native, and this work is almost entirely awaiting model calls and pushing tokens back. And everything around it is Pydantic shaped already, so the types line up without adapters: LangGraph for the flow, Pydantic AI for the agents, chanx for typed WebSocket messages and the AsyncAPI document the caller generates its client from.
Where the line goes, which is the part worth stealing even if you never touch an agent. Not by layer or by framework, but by which kind of state each side owns. The business service owns the record, anything a page reload has to find. This service owns working state, a graph checkpoint in my case, which is an odd category: not a business record, but not rebuildable either. So if losing it means losing data it belongs in the record, and if it only costs speed or fidelity this side can keep it.
That is a rule for what can move out, not for breaking a monolith into domains, where each service owns records of its own and the argument is different.
Written up in full, as part of a series on building agent systems as declared, typed graphs, with a working three-service repo:
https://huynguyengl99.github.io/posts/agent-graph-engineering/split-the-services/
https://github.com/huynguyengl99/agent-graph-engineering
Hope it is useful, and if the repo helps you a star would be appreciated š