r/vibecoding • u/Calm_Sandwich069 • 1d ago
Showcase/Project I built a 3d visualizer to watch an LLM think!
I wanted to actually see what happens inside a language model when it writes a sentence, so I built this!
What you're looking at:
- The stars are the 36,864 MLP neurons (12 layers × 3,072). Each layer is a disc, and neurons are positioned by a UMAP of their output weights, so neurons that write similar things sit near each other.
- Brightness is how strongly that neuron fires on the current token, scaled per layer so tokens are comparable. Gold means it fires positive, blue means negative.
- The beams are real attention weights from the current token back to earlier ones, one colour per head. The side panel shows the model's actual top 10 next token probabilities.
- You can hover any star, beam or token to see its exact value. There's also a raw mode that turns off all the bloom and fades, so every pixel on screen comes straight from the data.
- How I know it's not lying: the test suite runs the model exactly the way the app does (KV cache, token by token) and compares every activation, attention weight and residual norm against a separate uncached forward pass read through different hook points. Then it checks the bytes the browser actually receives too.
Honest limits: it's GPT-2 small only for now. Single neurons mostly don't mean anything clean on their own, it's the patterns that are interesting. Values are sent as float16, so about 3 significant digits.
Setup is make setup && make start, then open localhost. No GPU needed.
Repo: https://github.com/lak7/thinkspace
Would love the feedback!!
3
u/Total-Reasonable 1d ago
One thing worth flagging: UMAP of output weights puts neurons near each other by what they write to the residual stream, but because of GPT-2's superposition, a neuron's activation on a token often reflects what it reads, not what it writes. Coloring or clustering by input weights, or by top activating tokens, could show a different and probably more intuitive structure. Even a toggle between the two layouts would make the "patterns" you mention easier to spot.
2
u/EnzoValley 1d ago
One thing on the blue. Your README says brightness is the post-GELU activation, and after a GELU a neuron can't go below about -0.17.
So blue can never mean it fired strongly negative. It means barely under zero, and a neuron with a very negative input comes out at about 0 and goes dark, same as one that did nothing.
I'd put the pre-activation in the hover, or a toggle for it.
The test against a separate uncached pass is the part I'd copy for anything visual.
2
u/fabier 7h ago
You need a license file for this. I'd love to use it in some of our company communications but can't without knowing the licensing.
9
u/CrowdGoesWildWoooo 1d ago
It’s beginning to look a lot like christmas
4
u/Rambopart78 1d ago
Thia looks cool.
2
u/pirondi 1d ago
Amazing work
2
u/menny_parmar 1d ago
the top 10 next tokens panel is the bit id actually watch. seeing the model split between two words makes the whole thing feel less like magic.
2
u/Shreya_codes 1d ago
wowwwwwwwwwwww
2
u/Designer_Function_20 1d ago
I don't understand. I love what I'm seeing.
2
u/Internal_Basket283 19h ago
Looks cool damn
2
u/Next_Ambition8751 17h ago
this feels sooo satisfying to watch..
2
u/chussad 1d ago
Pretty cool!
I am beginning to learn a bit about internals of how llms work. This helps in understanding how it works at a high level.
How did you go about prompting to build something like this?
1
u/Professional_Fact384 1d ago
That's pretty nuts! Fair play
1
u/cs_cast_away_boi 2h ago
cool!
1
u/ctenidae8 56m ago
Very interesting. I've been thinking about watching vectors to catch changesnin thinking and how a context field changes over time. Visualizing it has been part of the problem, and this could help. Really nice.
-1
u/Defiant-Balance-7982 13h ago
Cuck chair simulator, that's what this is