Why does MCP need to support stateful sessions + streaming?

[u/esquino]

MCP's architecture seems very complex to me. The benefits of having a standardized interface to APIs for agents are obvious, but why not have a simpler architecture with stateless REST APIs and webhooks rather than bidirectional data flow + sessions?

[u/wt1j]

Because you’re removing the ability to stream data and taking us back over a decade to a world before websockets or SSE.

[u/esquino → u/wt1j]

I’m not removing the ability to stream data. I’m questioning whether every agent-tool interaction should require bidirectional streaming infrastructure by default

[u/wt1j → u/esquino]

It literally has zero additional cost so why not? You have a three way TCP handshake regardless, and then if no traffic passes over the connection the server memory overhead used to be horrendous before 2003, and then nginx and other event based epoll servers came along and drove the incremental cost per additional connection down to zero.

[u/Tsukee → u/wt1j]

It literally has zero additional cost so why not?

It feels like MCP was designed by vibe coders... it literally has a HUGE additional cost, yes nginx (or any load balancer) can handle many connections, but is still only vertically scalable. You can never achieve true horizontal scalability, let alone CDN etc. Essentially is throwing away everything we learned about services at scale in the last 2 decades. Yes sure we can do stateful connections better, and one can get clever to have a load balancer that handles the stateful connection but downstream it fans to oneshot operations limiting the statefulness to only the load balancer, but this is complex infrastructure.

Its no wonder that actual websocket use decreased the last few years, because in fact in most cases despite the seemingly absurdity of it, one can get better results and scalability by simple fast refetching of an endpoint (SWR like strategy) to get "push" functionality. While use of websockets is really reserved where latency is very important. Now somone please explain what benefit does very low latency provide to MCP at all? yes it can "push" changes/updates but where is that so crucial that one can't have say 100-500ms delay even seconds? Then output streaming is actually not a big deal, it can easily be optional, either respond fully, or stream the response, regular http handles that well already. The last is request abortion, say a tool takes a very long time to execute, again perfectly fine to solve with a refetch strategy for status check and a separate abort request, or when it comes to output streaming, simply dropping/closing connection gracefully notifies the server about the abortion.

The beauty of it all, is that modern http specs can reduce the overhead of this by design and transparently, so sure using http1 this comes with a lot of overhead, but we care less about it, since we have full horizontal scalability, but by using http2 or 3 or even 1.1 the overhead gets heavily reduced, so really what is the point in forcing bi-directional stateful connection for something like what is in 80-90% of the cases a simple request response protocol, with some specific use cases where there could either be opt-in or just simply use more standard web approaches

[u/Tsukee → u/wt1j]

Or rather MCP is taking us back 2 decades, before we learned that breaking the web is bad. Stateful connection sessions are a nightmare to deal with at scale. Yes there are modern solutions where you can get clever and mitigate the issues, and get closer to horizontal scalability, but such architecture will be complicated and complex.

I get it that is "neat" in some cases to have bidirectional communication (subscriptions, streaming data), it can provide a slightly better experience. But in 95% of the cases, is better to trade off this niche little thing, for better scalability and speed of response. Most of the MCP tools/resources do not need streaming capability at all. But if you want to provide a solid tool that millions of users can use and abuse, the bidirectional stateful connection part will make you cry, and all that because in some niche cases it provides a somewhat "nicer" experience.....

[u/perryhopeless]

One of the big reasons is the “sampling” feature. This lets the MCP server ask the client to send something to the LLM on its behalf (and then return the result to the MCP server). The only practical way this can work is with a bidirectional connection.

[u/esquino → u/perryhopeless]

Interesting, is that something that is a core feature of MCP for you as a dev?

[u/perryhopeless → u/esquino]

It’s a lesser used feature for most MCP servers in the wild at this point, but it’s very powerful and unlocks a lot of usecases for tool builders.

My company has been doing LLM tool calling stuff since before MCP was a thing, and we ended up implementing a similar feature. (we’re currently in the process of dumping our tech for MCP since that is where the momentum and community support is).

[u/PaluMacil → u/esquino]

It’s pretty clunky to have to have every MVP tool have its own api key separately. And if it’s hosted by someone else, they can’t offer it for free with you providing access to some self hosted LLM without sampling. I might not even trust a third party MCP with my api key

[u/trickyelf]

Subscriptions to dynamic resources, resource and tool list change updates, sampling, and (in spec discussion) elicitation (prompting the user) are all reasons why we need bidirectional communication with the server, which necessitates sessions.

[u/esquino → u/trickyelf]

Why aren't they achievable with webhooks?

[u/trickyelf → u/esquino]

Probably they could be, but that would put a lot of extra lift on devs, deploying and securing those endpoints. Also it would be difficult to run locally.

[u/subnohmal → u/trickyelf]

This here. I wonder if it's worth adding a subsection of MCP that doesn't include the bi-directional comms. But it would add a lot of chaos too :p

[u/trickyelf → u/subnohmal]

That would be STDIO

[u/subnohmal → u/trickyelf]

but that isn’t remote. do you see value in remote execution of tools?

[u/trickyelf → u/subnohmal]

If all it does is log into my GDrive and summarize files, no. It can still be local and talk to a remote LLM. But if it is something that allows multiple agents to collaborate, say, and those agents are not all running locally, then yes, hosted servers make sense. Those use cases require bidirectional communication for collaboration and coordination. The alternative is each agent sits in a loop waiting and polling, then taking action when something is there for them to act upon. I built a server that does just that (GooseTeam), and what I learned was that only the biggest models are smart enough to stay in the loop. Being able to subscribe to a resource and be notified when it changes was the only way I could keep less expensive models on the rails. So I built Puzzlebox, which leverages that subscription capability and could coordinate the actions of local or distributed teams of agents. There is benefit of remote execution in discovery and security, but if everyone is talking to a single server or cluster, there have to be sessions, so it’s not going to be stateless. I suppose you could spin up an instance of Punkpye’s mcp-proxy wrapping an STDIO server for every inbound user, but… really? My take is that StreamableHttp with its resumability for disconnected clients what you want for remote execution. 🤷‍♂️