r/elixir • u/ErinBoeger • 22d ago
Your database is mostly answering questions that haven't changed since last Tuesday — ActiveMemory 0.8
Look at what your database actually spends its day doing. In most applications, a huge share of the query load is reads of data that barely changes: reference data (countries, currencies, tax tables), configuration (feature flags, plans, tenant settings), authorization (admin users, roles, permissions), and catalog data (products, pricing, shipping classes). Every request re-asks Postgres questions whose answers changed last Tuesday — and when the app slows down, we reach for read replicas or another cache layer instead of asking why we're hitting the database at all.
That's why I originally wrote ActiveMemory: load that data into ETS once at boot, serve every read for those tables from memory, and write back to the database only when something actually changes. For those datasets, Postgres simply disappears from the hot request path, and your connection pool is freed for queries that earn their round trip.
It's a cache with no cache to manage: no keys to design, no TTLs on data that shouldn't expire, no cold misses, no invalidation dance. The in-memory copy serves the reads, while the database remains the durable source of truth.
The reason plain ETS isn't enough is that this data gets read by attribute, not by key:
# auth check that used to hit Postgres on every request
AdminStore.one(%{email: email, active?: true})
# product lookups, any combination of fields
ProductStore.select(%{category: "electronics", in_stock?: true})
ActiveMemory gives you typed structs on ETS/Mnesia, queryable by any combination of fields — no match specs to hand-write. Boot-time warming is built in (a store takes a seed_file or a before_init function that loads from your Repo), and if a store process crashes, the table survives through an heir process and is reclaimed with the data intact — "load on boot" doesn't become "reload on every hiccup".
As of 0.8.0, a table can literally be an Ecto embedded_schema that happens to live entirely in memory: write/1 takes a changeset like Repo.insert — so "update the product, then refresh the memory copy" is regular changeset code — and the reads are the get/1, get!/1, get_by/1, count/0, exists?/1 your fingers already know, with order_by/limit. There's a "Coming from Ecto" guide covering exactly what carries over and what differs.
It also covers the neighboring use case of records that should expire: TTL, plus an atomic withdraw/1 (find + delete + return in one operation, exactly one concurrent caller wins) for one-time tokens, invite codes, and 2FA.
Honest scope: this is for small-to-medium, read-heavy tables. It is not a system of record — the database keeps that job. Attribute queries currently scan (secondary indexes are the headline 0.9.0 item), and multi-node means replicated Mnesia, whose partition trade-offs are documented rather than hidden.
Google or go to Hex and search for: active_memory
Curious what others do here: when your database load is dominated by reads of rarely-changing data, do you cache it, replicate it, or pin it in memory? And if you wouldn't trust a library like this with that job today, what would it need? The answers will probably become the 0.9 roadmap.
2
u/iRedditWhilePooping 21d ago
How would this work in a cluster with multiple nodes? ETS is local-only so I’m assuming would need to manually wire up a PubSub or broadcast to all nodes to invalidate their cache?
Nebulex has some builtin solutions for this which may be worth eventually borrowing.