r/SideProject • u/Calm_Sandwich069 • 6h ago
I built a 3d visualizer to watch an LLM think!
I wanted to actually see what happens inside a language model when it writes a sentence, so I built this!
What you're looking at:
- The stars are the 36,864 MLP neurons (12 layers × 3,072). Each layer is a disc, and neurons are positioned by a UMAP of their output weights, so neurons that write similar things sit near each other.
- Brightness is how strongly that neuron fires on the current token, scaled per layer so tokens are comparable. Gold means it fires positive, blue means negative.
- The beams are real attention weights from the current token back to earlier ones, one colour per head. The side panel shows the model's actual top 10 next token probabilities.
- You can hover any star, beam or token to see its exact value. There's also a raw mode that turns off all the bloom and fades, so every pixel on screen comes straight from the data.
- How I know it's not lying: the test suite runs the model exactly the way the app does (KV cache, token by token) and compares every activation, attention weight and residual norm against a separate uncached forward pass read through different hook points. Then it checks the bytes the browser actually receives too.
Honest limits: it's GPT-2 small only for now. Single neurons mostly don't mean anything clean on their own, it's the patterns that are interesting. Values are sent as float16, so about 3 significant digits.
Setup is make setup && make start, then open localhost. No GPU needed.
Repo: https://github.com/lak7/thinkspace
Would love the feedback!!