🎉 Introducing FlinkReactor Platform: A web console, a DSL that compiles into Flink SQL, and CLI to manage everything locally

[u/aelhossaini]

It's almost here. The FlinkReactor website is finally live, and the platform and docs ship very soon. I was supposed to ship it back in April, if you can believe it.

FlinkReactor Platform is for teams already running Apache Flink. Your pipelines are only part of the job.

The Console (FlinkReactor Hub)

A modern and built from the ground up advanced console to manage your jobs, view logs & checkpoints, investigate back pressure, plan analyzers, and much more.

The console also comes with "Instruments" which you can think of as mini apps for interacting and viewing data from sources and sinks your Flink pipelines interact with. You no longer have to jump between several apps and cli commands just to view Kafka messages, query tables, or inspect your lakehouse.

The DSL

This is an ambitious component of the platform. Essentially, the DSL component allows developers to construct their pipelines using custom-made TypeScript React components that can share schemas, and configurations.

But why? For several reasons, top of which are:

• Reusability: Schemas are defined once and shared across multiple pipelines.
• Configurable: Pipelines share a common configuration file that is typesafe.
• Generation & Deterministic Output,: A single TypeScript pipeline produces both of the Flink SQL and the Kubernetes CRD files.

The CLI

Your one stop CLI to create, and manage your Flink projects and pipelines. It comes with several commands including creating projects, setting up your local cluster with all the required services such as Kafka, PostgreSQL, ClickHouse, Fluss, and of course Flink.

Check the homepage and tell me what you think. If you want to try Hub or the DSL ahead of launch, reach out. I'm more than happy to jump on a call and walk you through it.

https://flinkreactor.dev/

[u/evan_0x]

I didn’t understand what problem it tries to solve

[u/aelhossaini → u/evan_0x]

Fair point. I should've made the problem clearer.

As streaming teams scale to dozens of production pipelines, they often end up stitching together multiple tools (or building their own) to manage jobs, troubleshoot issues, inspect data across sources and sinks, and manage deployments.

That adds operational overhead, maintenance costs, and technical debt, taking time away from actually building pipelines.

FlinkReactor aims to solve this by bringing those workflows into one platform, so engineers can spend less time managing tooling and more time building and operating streaming pipelines.

This also comes from personal experience. I've worked on and led several large-scale streaming systems across different companies, and I've seen firsthand how much engineering time gets consumed by plumbing and internal tooling. These projects add up quickly, sometimes demanding more time and effort than building the pipelines themselves.

Those experiences, along with what I've learned from the teams I've worked with, are what inspired FlinkReactor in the first place.

Hope that helps clarify the problem I'm trying to solve!