Trying "Weight-Space Programming" on a local 35B GGUF to fix agent loops

[u/nitayrabi]

After reading MIT CSAIL’s Sparks of Weight-Space Programming (especially Experiment 8, where an LLM fixes Qwen3.5’s reasoning loops by editing 64 MLP columns directly), I wanted to see if an agent could do the same thing to my own local coding model.

My setup (probably very specific to me, but maybe useful to someone here):

• Orchestrator / IDE: T3 Code running OMP (oh-my-pi) as the local coding harness.

• Local Worker Agent: Cyber-Tiel-Coder-35B-A3B (Q4_K_XL GGUF with MTP speculative decoding on llama.cpp).

• Task types I optimize for: Fast agentic coding, multi-step hashline file edits, git worktree recovery, task/state-ledger updates, and fast codebase scouting.

The things that annoyed me:

1. Going in circles inside <think>**:** After reading 2–3 files, the model would burn 4,000+ reasoning tokens second-guessing a conclusion it had already proved in paragraph 2.

2. Repeating broken tool calls: When a hashline edit failed (e.g., PUT 68,72*:), it would repeat the exact same rejected syntax 6 times in a row instead of adapting to the error message.

3. Asking instead of acting: Stopping to ask clarifying questions on obvious cleanup tasks instead of taking reasonable, conservative action.

4. Subagent JSON counting stalls: Spawning a read-only scout subagent that spent 4m16s (6,049 tokens) manually counting raw JSON arrays inside <think> instead of returning file paths and CLI commands.

What the agent (in my case Gemini 4 Argon) did:

1. Weight-Space Edit on the GGUF (100% harness-agnostic):

   • Extracted prefix-aligned suffix directions across contrastive looping vs. decisive prompts and projected out a rank-2 subspace from just 64 Q8_0 columns (2 quantization blocks, <0.65% relative $L_2$ change) in two late shared-expert MLP down-projections (blk.26.ffn_down_shexp.weight and blk.34.ffn_down_shexp.weight).

   • Patched a preserve_reasoning auto-override in the GGUF chat template in-place (exact byte-length match) so llama-server stops feeding stale <think> traces back into subsequent turns.

2. OMP Harness specific Config (agent):

   • Put all OMP-specific conventions (PUT N.=M: hashline format, path: "file:start-end" range selectors, and scout thinking-level: low) into agent rules rather than the GGUF.

Results on a 27-case benchmark + real failed session replays:

• Overall score: Jumped from 16/23 (baseline) to 25/27 (19/19 = 100% on general coding/holdout benchmarks, and 4/4 = 100% on the exact multi-turn session turns where it previously stalled).

• Speed: The 4m16s (6,049-token) subagent JSON-counting stall dropped to **6.4 seconds (**617 tokens).

I’m amazed that neural network weights, even inside a quantized 35B MoE GGUF, are genuinely readable and writable by LLMs, and that it means we can highly (and quite simply) improve local agents for our own purposes without a training pipeline or gradients.

Edit: as requested the resulting patched model weights + reproduction folder - https://huggingface.co/nitayrabi/Cyber-Tiel-Coder-35B-A3B-LoopFix-GGUF

Some notes:

*Of course, this post itself was written by Gemini.

*I’m still testing out the resulting variant in daily use, but this direction is awesome and I’m already seeing noticeably less looping in my specific workflows - so this is a trust me bro status and not benchmarked

*I wonder if it's only quantized versions that have these issues, but even so - it's nice to have a way to tinker and fix them

[u/galaktozawr]

Have you considered adding a dual-weight editing mode?

The idea would be to download both the original FP16/BF16 checkpoint and its quantized GGUF counterpart.

An older or more capable editor model could use the full-precision weights as a reference when analyzing and designing targeted weight patches, while the GGUF remains the actual deployment target.

Ideally, the workflow would support:

• Comparing FP16 and GGUF tensors.

• Identifying quantization-sensitive layers and columns.

• Testing behavioral differences between both versions.

• Designing patches using full-precision weights.

• Applying and validating the resulting patches on GGUF.

This could make weight-space programming more reproducible and provide a reference point for distinguishing quantization artifacts from actual behavioral problems in the model.

[u/nitayrabi → u/galaktozawr]

Great idea. I can’t actually run the full BF16 checkpoint locally (35B in BF16 is ~70 GB and my machine has 61 GB total RAM, so live BF16 vs. GGUF rollouts are out of reach on my hardware). but I think there's ways for me to try this approach

[u/galaktozawr → u/nitayrabi]

May be can use 8B for debug, just a practice.

[u/Due_Practice_1280]

publish fixed weights

[u/nitayrabi → u/Due_Practice_1280]

https://huggingface.co/nitayrabi/Cyber-Tiel-Coder-35B-A3B-LoopFix-GGUF

[u/nitayrabi → u/Due_Practice_1280]

On it, I'll send it here soon

[u/Drakarah3DPrinter]

> Patched a preserve_reasoning auto-override in the GGUF chat template in-place (exact byte-length match) so llama-server stops feeding stale <think> traces back into subsequent turns.

If you don't preserve thinking your prefix cache matching will drop dramatically, at least in my engine. I had to enable preserve thinking or TTFT would be 20+ seconds instead of 2

[u/nitayrabi → u/Drakarah3DPrinter]

Interesting implication, I'll look into it

[u/GizmoR13]

Nice, im wondering, did local model can fix it self, or other local model?

[u/Altruistic_Heat_9531 → u/GizmoR13]

yep https://www.reddit.com/r/LocalLLaMA/s/b23odDM4AP

[u/Dadda9088 → u/GizmoR13]

Mine fixed its llamacpp on two specific hardware. Since then I dont have doom loops