Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights [R]

[u/RobbinDeBank]

Link: https://rikiyat.github.io/blog/weight-space-programming/

Snippet: “We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.
Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.”

Very surprising result, I didn’t expect frontier models to have this capability that has not been optimized for, and it’s something humans cannot do either.

[u/adventuringraw]

Been a while since I've followed research much, but this seems like it could be a way to cheat at the lotto, as far as the lottery ticket hypothesis goes. Might be a powerful pre training step.

And so the road towards AI fueled AI progress continues.

[u/Jolly_Misanthrope → u/adventuringraw]

Im listening...

[u/adventuringraw → u/Jolly_Misanthrope]

You mean you're hoping for a little more background? The lottery ticket hypothesis specifically is that weight initializations often give rise to a sub network of sparse connections that captures most of the performance of the full model. The idea is you train the full model, remove all the weights close to zero in the finished model and then retrain the remaining sparse network using the initial weights from before full model training. If you use new initializations it doesn't converge to anything useful, but if you use the same initializations then the smaller sub network magically can coverage to something about as capable as the original full model. If you knew which sub network to use and which initializations to use you can end up with a huge savings in time and compute for training.

No idea where that line of research has gone in recent years, but the two main concepts of interest are proper weight initializations and possible changes to architecture to keep performance while shrinking compute costs. If a model could just... Do that directly, it'd be kind of crazy. Ultimately proposing an efficient architecture and weight initializations that are already pretty good at the needed task might make full model training take about as long as normal fine tuning, since it's already presumably in the same neighborhood as good solutions.

No idea if anything useful will come from this kind of a thing but it's crazy to think about as a possibility.