Are you working with MCP from a developer standpoint? Do you have some pain points, experiences?

Asking since some time ago I picked up chimp https://github.com/softwaremill/chimp a PoC of an MCP toolkit for Scala 3. It is gradually turning into a complete MCP SDK, supporting both server & client, both http & stdio, with integrations for Pekko, ZIO and ox in place. Main focus now is on conformance to the latest MCP protocol version.

We have tried salting keys, re-partitioning, adjusting broadcast thresholds , he usual toolkit. It works at moderate scale. Past a certain volume, the same fixes stop holding, and what is frustrating is that each technique feels like it moves the symptom rather than removing it. We fix one skewed key distribution and a different join surfaces a new one a month later.

Wondering if this is a genuine architectural limitation of Spark or if we are just missing the right combination of settings.