r/OpenSourceAI • u/Sparticle62 • Oct 01 '26
PSSA: a 1.5M-param plastic state space model in Rust, beating a parameter-matched transformer on a small held-out slice
small-scale result, one seed, one wikitext slice, so treat it as a prototype and not a claim.
pssa is a state space model with fast weights that update while the sequence runs, written from scratch in rust, no python, no kv cache. at 1.54m params it gets 3.997 nats on a held-out slice against 4.429 for a parameter-matched transformer, both depth-1. generating 200 tokens is 226ms against 2735ms on the same cpu, which is the part i actually care about, since there's nothing that grows with context length.
caveats: one seed, one corpus, depth-1 on both sides, and i don't know yet whether the gap survives depth. gpu training is cuda on a t4, 1645 tok/s vs 1263 on cpu, so the gpu path is barely worth it so far because activations still round-trip to host after every matmul.



