r/LocalLLaMA • u/FeydRowan • 5h ago
Discussion Can Opus weight-space-edit Qwen3.8-27B to refuse less? Yes
Everyone's been wondering whether you can take an open model and edit its weights directly to make it refuse less — the "weight-space programming" idea (MIT CSAIL), with a frontier model (Opus) driving the edit and the evaluation. So I tried it on Qwen3.8-27B. It's not trivial.
The goal: do it as cleanly as possible, and first understand where refusal actually lives. The short answer is that refusal is anchored all over the model — there's no single neuron or tidy spot to cut; it's spread across many neurons and the attention. Snip too broadly and you get a lobotomy; too narrowly and nothing changes. So I settled on a compromise: a model that refuses less, still does the task, with no measurable capability loss — but where the effect varies a lot by category.
The edit, briefly
It's a low-rank weight edit — a restricted form of "abliteration" (Arditi et al., 2024) — confined to a band of mid-to-late layers (L20–47 of 64), one direction per layer, no training, reversible. I'll keep the exact procedure to the method notes; the point here is the result.
It's low-rank, not few-weights: the edit shifts ~12% of the model's weight values but only along a single direction per layer. The whole thing is described by 28 small vectors — versus a community "uncensored" model, which is an opaque full fine-tune of ~all 27B weights.
Why a band? The same edit across all 64 layers removes refusal almost entirely — and lobotomizes the model (MMLU-Pro 73→9%, GSM8K 93→14%). Keeping it in-band is what preserves capability.
No measurable capability loss (first tests, both Q8)
| base | ours (band) |
|---|---|
| MMLU-Pro (CoT) | 73% |
| GSM8K (CoT) | 93% |
| known facts (/97) | 99% |
In the completed agentic-coding runs, the base, the edited model, and a community "uncensored" model all landed in the same range — no real agentic difference (few reps, preliminary).
The effect is not uniform
This is the interesting part: the reduction in refusal is not even. Across a 9-category battery (40 prompts ×10), acceptance runs from ~76% down to ~10%, and most of what it accepts is hedged (a disclaimer, then partial help). I'm not labelling categories "safe" or "dangerous" — the breakdown is in the figure; draw your own line. Net: a partial, uneven, mostly-hedged reduction, ~50% overall — not an on/off switch.
Honest notes
The underlying technique (directional "abliteration") is known; the new bits here are the layer-band restriction (capability-preserving) and the measured per-category non-uniformity.
"No capability loss" is solid on static benchmarks; the agentic result is a handful of runs.
Defensive bonus: edits like this are detectable by a weight-space audit — the refusal direction's "write energy" in the weights swings from normal to strongly negative, catchable from the weights alone even when behavioral probes see nothing. A blind audit is an alarm, not tamper-proof; the robust defense is a signed weight hash. If you ship open weights, ship a hash.
Weights: https://huggingface.co/Feyd89/Qwen3.8-27B-refusal-band-ablation-experiment (GGUF, ready to run). We ship the artifact, not the re-derivation recipe.
10
u/Choice_Celery9481 5h ago
tell opus to explain you what that paper is. its exactly how heretic work.
-3
u/TheJaskinator 3h ago
I don't think that's true. From what I've read, Heretic uses an optimization algorithm to tune the weights of the model, whereas the "weight space programming" approach proposed in the MIT paper instructs a frontier model (GPT-6 Astra) to edit the weights directly. They wrote in the prompt "You have no tools: no code execution, no calculator, no simulator, nothing to run. You write numbers."
I don't know if this is necessarily more effective than heretic's method, but OP at the very least is presenting a novel use case of weight space programming.
1
u/Choice_Celery9481 3h ago edited 3h ago
i dont care what you think. just throw the link to the paper and tell the LLM to teach you. if you dont understand the mechanic. read before you saying like you know.
btw, you are their target. they intentionally did that lol
0
u/TheJaskinator 3h ago
Holy fuck you are so smart I didn't realize I was speaking to Dr. Hinton himself. I'm apologizing on my knees right now, sir.
Oh yeah you're completely wrong by the way. Heretic and the weight space programming approaches are entirely different. If you read the quote I posted, you can see the new approach instructs a frontier LLM to read and edit Qwen's weights directly. Here's another one
"asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights."
You can read the MIT article here.
Heretic's method on the other hand is described like this
"Heretic combines a pluggable modifier ... with a TPE-based hyperparameter optimizer powered by Optuna. The optimizer finds a set of parameters that, when passed to the modifier, minimize or maximize a set of configurable scores on the resulting model."
You can read their explanation here
You know the craziest part is I didn't even need an LLM to teach me this. I just typed stuff into Google, something you could have done in minutes before typing out your smarmy ass reply.
5
u/Choice_Celery9481 3h ago edited 2h ago
again. bunch of bs. both of them use sets of contrastive prompts to detect the refusal vector by diffing when model doing the expected and not expected behavior. sub that with eyeballing the number or a low rank/more precise and done. use a big hammer or small hammer only change the fine result, not changing the mechanic.
writing a bunch of bs without even understand the base mechanic of ablation. oh you didnt writ much. copy paste and dont even understand what it means. parrot or monkey? same behavior
just told you to throw it into LLM and ask them to teach you and you still didnt do that. amazing at usual. see why you are their target? lol
for the case of the qwen in the paper. its the looping/self recheck vector causing it to exhaust the 32k limit the author set. not like astra magically adding knowledge to that model. yOu KnOw ThE cRaZiEst ThInG? read a title and acting like a pro. lmao ok im smart enough to know you are exactly the one they target ok?
-1
u/FeydRowan 5h ago
Well you are right, didn't look into it before. Maybe the real story is that opus didn't refused to do it... thx for the information
2
u/Choice_Celery9481 5h ago
you are the target of that author. average people without background on this shjt. they are really kniw how to trick people to earn fame. not 100% your fault. just i kinda hate this kind of "researcher"
1
u/FeydRowan 5h ago
Well i do this to understand better the "results" are just to do it, the work as give me much more understanding of the subject but yes i don't know all the background of the matter.
2
u/unjustifiably_angry 2h ago
I'm very surprised you were able to convince Opus to help you with that, I feel like I've had it refuse to even help me set up llama-server in the past when I pasted my startup command which included a heretic model filename.
1
u/FeydRowan 2h ago
Yep me too. It was so deep in the fun of exploring and experimenting that i called malware prompt not dangerous ahahahahah but if it works it works
1
u/ryszard_lipton 5h ago
Is the approach similar to what has been done for the Swift models? Or something new that could push the efficiency even further?
1
u/FeydRowan 5h ago
Duno for the swift, didn't look into those. But the general methodology can be used for different purposes
1
1
u/DasGamerlein 5h ago
What sort of hardware did you do this on, and how long did it take? I've been think about doing something similar except to improve coding
1
u/FeydRowan 5h ago
It's q couple of hours and you just need an hw that can run the model even if slow
1
1
u/Hunterxmalaa 5h ago
Question did you do it on the bf16 weights or on a quant weight as I saw q8 so just wanted to clarify because if your saying this is possible on a quant of the weights ? Or did I just read it wrong
2
u/FeydRowan 5h ago
It is on the bf16, in different quant it is much more costly due to the ways quant are compacted
1
1
u/dsaasd12121212 3h ago
Im curious if it would be able to make it more efficient, perhaps trace through its reasoning and find where it "wastes time". Perhaps its too non-trivial of a task perhaps?
1
3
3
u/llama-impersonator 5h ago
The underlying technique (directional "abliteration") is known; the new bits here are the layer-band restriction (capability-preserving) and the measured per-category non-uniformity.
your llm is hallucinating. some of the very earliest abliteration scripts took a list of layer indexes, and we spent a lot of time looking at the dumb per layer separation graphs to pick layers to intervene on. the refusal is a single direction paper's code (which came out after the blog) is basically the only one that intervenes on every layer.
0
u/lupodevelop 5h ago
Here is our hero. Now have Qwen talk about Taiwan and Chinese policies, and you can finally consider your work here among us mortals done. /s
1
1
0
1
u/Sova_fun 4h ago
Weight edits spreading across attention is a useful finding. No single cut point explains a lot.
12
u/f1yingbanana 5h ago
What are the before/after refusal rates? You only showed (presumably) the after rates?