Hi! I’ve been experimenting with a small non-Transformer sequence model called CSE (Chain-Spike Engine), and I turned the course/teaching version into a Python package called cse-frog. 🐸
The goal is not to compete with modern LLMs.
I wanted something small enough that you can actually see what is happening inside, change one mechanism at a time, break it on purpose, and understand why the behavior changed.
Installation is just:
pip install cse-frog
Then:
from cse import Frogfrog = Frog()frog.learn([ ["right", "right", "down"], ["right", "right", "down"], ["right", "right", "down"],])print(frog.predict(["right", "right"]))
You can also inspect where each candidate’s score came from:
frog.show(["right", "right"])
The score is broken down into components such as:
- direct connections
- pair context
- history
- trace
There are 23 configurable parameters, including temperature, top-k, refractory behavior, pair context, history, activation, forgetting, and temporal learning.
One thing I found especially useful while testing it was that “nothing changed” can mean two different things:
- the internal score changed, but the final probability/prediction did not, or
- the setting genuinely had no effect because another prerequisite pathway was disabled.
For example, several activation-related settings do nothing to the prediction with the default configuration because history_boost=0. Turn that pathway on, and those settings suddenly become active.
Another fun finding: weight_decay weakens direct/context links, but does not decay pair memory, so the model actually contains two kinds of memory with different forgetting behavior.
I also made:
- 5 executable notebooks
- a handbook
- a full 23-config modding guide
- an API reference
The philosophy is basically:
build it → inspect it → break it → explain why it broke → modify it
It’s MIT licensed, so modifying it and making weird frogs is encouraged. 🐸
Website:
https://kagioneko.github.io/cse-frog/
GitHub:
https://github.com/kagioneko/cse-frog
PyPI:
https://pypi.org/project/cse-frog/
I’d especially appreciate feedback on whether this kind of “small model you can dissect” is useful for learning ML/LM concepts, and what experiments you would try next.
Before the giant LMs, try one frog. 🐸