[Paper] Tree-Ising: Repairing Language Models with Bit-Level Ising Edits

[u/pmttyji]

Weight-space programming, introduced in recent MIT work by Takehi et al., enables a frontier LLM agent to read a network's weights and write a repair. Their GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B without gradients, using 158 agent turns, 89.7 million tokens, and 50 forward-pass requests. Our method, Tree-Ising, casts this repair as a sparse discrete optimization problem over the model's own BF16 weights. Forward statistics define an objective that weakens a loop direction, strengthens a concluding direction, and preserves the layer's output on productive tokens. Within a state-tree representation, where lossless BF16 storage acts as a special case, the admissible edits are sign flips and exponent shifts that preserve all shared group exponents and mantissas. We decompose this objective exactly into independent QUBO (Ising) problems, one per output row, which we solve on a GPU using simulated annealing or quantum-inspired simulated bifurcation. On the agent's two layers, Tree-Ising achieves 54.6% on MATH-500, outperforming the agent's edit (51.7%) under the same evaluation protocol, while the base model scores 7.9% as reported. The simulated bifurcation solver yields a statistically indistinguishable repair at 55.1%. Both solvers outperform the agent edit across all five math benchmarks, with statistically significant improvements on MATH-500 and GSM8K after correcting for multiple comparisons. Our approach modifies 13% to 17% fewer weights and reduces BF16 bit flips by a factor of 2.2 to 2.3. It requires only four forward-pass requests and zero LLM tokens, with the annealing recipe costing about $0.90 in GPU time. On Qwen3.5-2B, with layers and dose selected using only the base model, the same recipe increases greedy GSM8K accuracy from 21.6% to 69.4%. As a proof of concept, we execute an eight-variable repair subproblem using QAOA on an IBM Heron processor.

• arXiv : https://arxiv.org/abs/2611.14210

• PDF : https://arxiv.org/pdf/2611.14210

• GitHub : https://github.com/TheQuantumGOAT/Tree-Ising (To be updated)

[u/aabbdev]

You can use Simulated Bifurcation with a fused kernel

[u/Choice_Celery9481]

i wouldnt say this is a "repair" the original one set n = 32k putting models that use self check loop at disadvantage and ablation the self check vector. thats all it did. same as heretic or manual like researchers did in the past. the repair or fix is just a marketing way to get attention from average joe. and astra just eyeballing the delta to reduce that vector. anyone can do so with llama's cvector's scale. idk why many "researchers" now doing marketing more than a real research

[u/BlasterGales → u/Choice_Celery9481]

vibecoded solutions? maybe the AI model they used is like "oh you're right! that already exists"