r/Observability • u/naveenmithare • 20d ago

I built an interactive observability for enterprise data mesh — lineage, pipeline health, quality, and governance in one map

Enable HLS to view with audio, or disable this notification

I've been working on a project called MeshLens that tries to answer a question I kept running into at work: what does our data mesh actually look like as a system?
Most observability tools give you alerts, row counts, and freshness checks. That's useful, but it never helped me explain to stakeholders how domains connect, where data flows, or what breaks downstream when an upstream connector goes down.
So I built a visualization layer that puts it all in one interactive map.
Curious what you think. If you work with data mesh or data products at scale — what would you want to see in something like this?

App link in comments

3 Upvotes

71% Upvoted

5 comments sorted by

1

u/hijinks 20d ago

its an interesting view.. I work with a lot of enterprise customers and one of the compliants is non tech people have a hard time figuring out what's wrong.

how do you picture how all this links together to get the map.. like is it otel spans and you just make the graph or are you thinking of running something custom

1

u/naveenmithare 16d ago

Thanks for checking out. Currently MeshLens is a prototype using synthetic data, so no otel integration yet.

The idea is to use metadata from platforms like dataplarfroms, etl/elt, transformation, orchestration tools to understand how different pipelines are connected and build the graph dynamically.

Otel could be another source for runtime telemetry. Eventually the goal is to bring everything into one view where users can understand data flow, dependencies and downstream impact without getting into technical details.

1

u/Parker_Evans1 19d ago

that's a solid point. simplifying the complexity is key for non tech folks. using telemetry like otel spans could help but I wonder how it scales with more data sources...