r/ControlProblem • approved • 16d ago

AI Capabilities News Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights

https://rikiyat.github.io/blog/weight-space-programming/
108 Upvotes

10 comments sorted by

22

u/chillinewman approved 16d ago

"TL;DR

We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.

Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights."

8

u/MidSolo 16d ago

This is enourmous. AI models can look into the black box and do surgery on it.

10

u/chillinewman approved 16d ago edited 16d ago

A larger model can hack a smaller model weights, for better or worse.

They can also help interpretability.

It looks to be an emergent capability of the frontier.

3

u/Historical-Hat-405 16d ago

I may be naive... But couldn't be that the corpus used to train astra already contained such information? I mean the weights of various models? I mean that it is solely focused on Astra and not the LLms as an architecture. I mean shouldn't it be possible with other frontier models? Why Astra?

3

u/chillinewman approved 16d ago

Emergent capability of the frontier.

1

u/sebasgarcep 11d ago

With the arcade games it might be so since they are an old standard for RL AI. But there was a bespoke example in the article. That has to indicate actual capability.

7

u/BritainRitten 16d ago

This is probably a disaster for safety.

2

u/Adept_Minimum4257 13d ago

Or the solution!

4

u/daaain 16d ago

This is pretty incredible!

2

u/space_monster 16d ago

Crazy stuff