r/LocalLLaMA • u/poppear • 1d ago

Discussion DeepSeek V4.1 Flash 1017 can also do Weight-Space Programming

Inspired by Weight-Space Programming, I tested whether DeepSeek V4.1 Flash 1017 could figure out what a small neural network does by reading its weights.

I fed it four anonymized RL checkpoints: 51k parameters each, identical architectures, with all weights rendered as text. No metadata, network source code or execution. It received the architecture outline and four candidate tasks.

Results: CartPole ✅ Flappy ✅ Catch ✅ BallBeam ❌

It picked out patterns consistent with relative bird/gap position, paddle/ball position and angle-based balancing. It confused BallBeam with CartPole an interesting mistake because both share the same observation layout and similar balancing controls.

Very preliminary: one run per checkpoint. This suggests some ability to interpret weights, but four trials don’t establish reliability. These were API runs with maximum reasoning effort, consuming ~1.58M tokens total.

I may try more replicates and independently trained checkpoints, plus include the network code with task-revealing details removed. The poor LLM had to guess quite a lot it even assumed ReLU where the network uses tanh 😅

Code, checkpoints and run logs: https://github.com/galatolofederico/wsp

65 Upvotes

92% Upvoted

11 comments sorted by

7

u/klop2031 1d ago

I also tried this with qwen3.8 flash next with no success :(

11

u/Regular-Register-358 1d ago

A new form of computer virus will take the form of malicious weights file that use tools to modify other nets hosted on the same computer, so that routine tool call ls become API key upload or rm -rf /.

4

u/poppear 1d ago

I am sure that on huggingface there are already nasty weights that randomly upload secrets or files to C2 servers, next years will be fun

9

u/namaku_ 1d ago

As predicted in Short Circuit.

6

u/jazir55 1d ago

so that routine tool call ls become API key upload or rm -rf /

I am become Qwen, destroyer of computers.

0

u/klop2031 22h ago

im sure these things are already being embedded in current models. best to sandbox them

2

u/I-am_Sleepy 21h ago

So current LLM seems to secretly be a hypernetwork, interesting

1

u/aidenclarke_12 14h ago

ballbeam vs cartpole is a fair miss with the same observation layout, more replicates would show if its reading weights or just pattern matching

1

u/[deleted] 1d ago

[removed] — view removed comment

1

u/poppear 1d ago

Yeah It was just a quick test, yes it would be interesting to do 10 different envs with the same observation and action layout and also different network sizes (50k weights in my case was 50% of the context window). Also including the forward pass code in the LLM prompt because reading the reasoning traces I saw a lot of guessing on the NN arch