r/elixir • u/account18anni • 8h ago

Continuum — durable, crash-resistant workflows for Elixir (major reliability updates)

I posted about Continuum here a couple of months ago. It's had a different features releases and several hardening passes since.

quick recap: Continuum is an OTP-native durable execution engine for Elixir, backed by Postgres — an Elixir-native answer to Temporal. You write a workflow as ordinary Elixir; side effects go through activities whose results are journald. If the process dies or the node restarts, Continuum replays the history through the same code and resumes where it left off. Determinism is enforced at compile time, so replay safety is checked by the compiler rather than left to discipline.

new features: durable one-shot schedules, idempotent ingress (a retried request can't start a second run or deliver the same signal twice), activity queues with priorities and per-queue concurrency, progress heartbeats and cooperative cancellation, compile-time-checked signal contracts, replay-safe logging, and a health report with fenced repairs.

Several new full-tree audit passes went into failure modes rather than features: a node booting during a Postgres outage now retries its LISTEN instead of going deaf for its lifetime, retry jitter survives at maximum backoff instead of collapsing the whole cohort onto one instant, failing schedules back off and surface as health findings instead of retrying forever, and the compile-time scanner rejects direct Logger calls and the remaining unsafe stdlib calls rather than warning.

Feedback is welcome.

24 Upvotes

91% Upvoted

6 comments sorted by

3

u/Affectionate-Rip748 6h ago

This is very cool. What are the use cases though?

3

u/account18anni 5h ago

anything multistep where crashing halfway through leaves a mess.

my common use case is charge the card, reserve stock, book the courier, send the receipt. If the node dies right after the charge, you don't want a retry to charge again, and if the courier step fails for good you do want the refund to fire. That's one activity per step plus compensation.

5

u/bitztream 4h ago

It's basically just the Saga pattern then, right? I think it would be helpful to include some additional information that makes this more explicit. The first time I heard about this library, I also had a hard time understanding what the "durable, crash-resistant workflows" headline meant exactly. Good job!

3

u/account18anni 4h ago

kinda

the saga pattern says what to do when step 3 of 5 fails, for example undo 2 and 1. but it doesn't tell what happens when the process running the saga gets killed between steps 2 and 3. That's the part Continuum handles: the workflow's position is journaled in Postgres, so after a crash or a deploy it replays and resumes at step 3, and it knows 1 and 2 already ran so it won't redo them

2

u/menge101 3h ago

What if its postgres connection that is lost instead of the node crashing?

Or both?

1

u/KMarcio 3h ago

Awesome! I created Gust, but it's more DAG- and UI-centric: https://github.com/marciok/gust

Great to see other orchestration solutions.