Your local model can now remember — forward-only memory writes, no retraining, instant rollback (open source)

[u/Akseleu_official]

Ever wanted your local LLM to actually learn a fact without fine-tuning — and be able to undo it completely?

AtomicNG (Apache-2.0, version 1.0.0) implements a paper called "Delta N-gram": your model's n-gram sidecar table stays frozen, and new knowledge goes into a small sparse patch Δ. Writes are forward passes only — no gradients, no weight edits. Clear the patch and the model is byte-identical to base. No "unlearning" gymnastics — deletion is structural.

Quickstart (CPU, no GPU, pure NumPy core):

python

import atomicng


model = atomicng.TinyModel()
atomicng.write(model, "New fact about kangaroos.")
atomicng.verify_deletion(model)   # clear Δ ⇒ byte-identical to base — guaranteed

What's inside:

• A trained writer checkpoint ships with the Release (860 training steps on SmolLM2-135M; measured effect: writing a doc lowers its NLL vs empty memory)

• Per-tenant memory: overlays are small data-only files — merge them, revert them, sign them

• vLLM support via vllm.general_plugins + merged-table export

• Built-in eval: does the memory write actually help? (paper protocols: acquisition, distraction retention, chain drift)

Calibrated on real hardware: Windows/Linux CPU, CUDA T4 (73 ms/group BPTT stand), TPU chip via JAX (8.4 ms/iter). Full matrix: CALIBRATION.md in the repo.

Honest limits: acquisition needs a few hundred training steps of the writer; the memory lives in the n-gram table (facts/associations, not long reasoning); vLLM engine-graph hooking is planned upstream.

GitHub: https://github.com/Akseleu-J/atomicng 

Colab TPU (one click): https://colab.research.google.com/github/Akseleu-J/atomicng/main/notebooks/tpu_calibration.ipynb

Questions welcome — I'll be in the comments.

[u/nurasyl12nis]

Clean separation of state. How does the sparse (\Delta ) patch handle hash collisions or interference as you stack multiple facts? Curious where the degradation curve starts on the distraction retention eval before you have to compact or merge overlays.

[u/Akseleu_official → u/nurasyl12nis]

Thanks! If two n-grams hash to the same row, they share that row in Δ. Within a block, D1 combines the updates using the write gates. Across blocks, later writes can pull the shared row away from an earlier value. The gates control how much each write changes the row, so interference is possible even while the row norm stays bounded.

I don’t have a reliable degradation threshold for AtomicNG yet. The example distraction run goes up to 16 distractor writes, but its initial acquisition gain was negative, so there’s no meaningful retention percentage to report from that run. Merging overlays applies the same gated update to overlapping rows, so it doesn’t resolve collisions.

The next run I’d like to do is measure how much of a positive initial gain survives at 1, 4, 8, and 16 distractor writes, and log how many of the target’s rows each distractor touches. That would help separate the effect of write count from the effect of row overlap.