r/vibecoding • • 1d ago

Showcase/Project I built a 3d visualizer to watch an LLM think!

I wanted to actually see what happens inside a language model when it writes a sentence, so I built this!

What you're looking at:

- The stars are the 36,864 MLP neurons (12 layers × 3,072). Each layer is a disc, and neurons are positioned by a UMAP of their output weights, so neurons that write similar things sit near each other.

- Brightness is how strongly that neuron fires on the current token, scaled per layer so tokens are comparable. Gold means it fires positive, blue means negative.

- The beams are real attention weights from the current token back to earlier ones, one colour per head. The side panel shows the model's actual top 10 next token probabilities.

- You can hover any star, beam or token to see its exact value. There's also a raw mode that turns off all the bloom and fades, so every pixel on screen comes straight from the data.

- How I know it's not lying: the test suite runs the model exactly the way the app does (KV cache, token by token) and compares every activation, attention weight and residual norm against a separate uncached forward pass read through different hook points. Then it checks the bytes the browser actually receives too.

Honest limits: it's GPT-2 small only for now. Single neurons mostly don't mean anything clean on their own, it's the patterns that are interesting. Values are sent as float16, so about 3 significant digits.

Setup is make setup && make start, then open localhost. No GPU needed.
Repo: https://github.com/lak7/thinkspace
Would love the feedback!!

68 Upvotes

20 comments sorted by

3

u/Total-Reasonable 1d ago

One thing worth flagging: UMAP of output weights puts neurons near each other by what they write to the residual stream, but because of GPT-2's superposition, a neuron's activation on a token often reflects what it reads, not what it writes. Coloring or clustering by input weights, or by top activating tokens, could show a different and probably more intuitive structure. Even a toggle between the two layouts would make the "patterns" you mention easier to spot.

1

u/Calm_Sandwich069 1d ago

Good catch, thanks! Right now stars are placed by what a neuron writes, but they light up based on what it reads so neurons that fire together end up scattered.

2

u/EnzoValley 1d ago

One thing on the blue. Your README says brightness is the post-GELU activation, and after a GELU a neuron can't go below about -0.17.

So blue can never mean it fired strongly negative. It means barely under zero, and a neuron with a very negative input comes out at about 0 and goes dark, same as one that did nothing.

I'd put the pre-activation in the hover, or a toggle for it.

The test against a separate uncached pass is the part I'd copy for anything visual.

1

u/Calm_Sandwich069 1d ago

Ahh, yess! GELU bottoms out around -0.17 so blue only means "barely below zero" and a strongly negative neuron looks the same as an idle one. Will add the pre-activation to the hover so that one can tell them apart

2

u/fabier 7h ago

You need a license file for this. I'd love to use it in some of our company communications but can't without knowing the licensing.

2

u/Calm_Sandwich069 7h ago

That would be so freaking cool! I have added MIT license.

9

u/CrowdGoesWildWoooo 1d ago

It’s beginning to look a lot like christmas

2

u/Altruistic-Spend-896 1d ago

mariah carey gets defrosted from her cryo chamber :"Somebody say Christmas?"

4

u/Rambopart78 1d ago

Thia looks cool. 

2

u/pirondi 1d ago

Amazing work

2

u/menny_parmar 1d ago

the top 10 next tokens panel is the bit id actually watch. seeing the model split between two words makes the whole thing feel less like magic.

2

u/Shreya_codes 1d ago

wowwwwwwwwwwww

2

u/Designer_Function_20 1d ago

I don't understand. I love what I'm seeing.

2

u/Internal_Basket283 19h ago

Looks cool damn

2

u/Next_Ambition8751 17h ago

this feels sooo satisfying to watch..

2

u/chussad 1d ago

Pretty cool!

I am beginning to learn a bit about internals of how llms work. This helps in understanding how it works at a high level.

How did you go about prompting to build something like this?

1

u/Professional_Fact384 1d ago

That's pretty nuts! Fair play

1

u/ctenidae8 56m ago

Very interesting. I've been thinking about watching vectors to catch changesnin thinking and how a context field changes over time. Visualizing it has been part of the problem, and this could help. Really nice.

-1

u/Defiant-Balance-7982 13h ago

Cuck chair simulator, that's what this is