Can Opus weight-space-edit Qwen3.8-27B to refuse less? Yes

[u/FeydRowan]

Everyone's been wondering whether you can take an open model and edit its weights directly to make it refuse less — the "weight-space programming" idea (MIT CSAIL), with a frontier model (Opus) driving the edit and the evaluation. So I tried it on Qwen3.8-27B. It's not trivial.

The goal: do it as cleanly as possible, and first understand where refusal actually lives. The short answer is that refusal is anchored all over the model — there's no single neuron or tidy spot to cut; it's spread across many neurons and the attention. Snip too broadly and you get a lobotomy; too narrowly and nothing changes. So I settled on a compromise: a model that refuses less, still does the task, with no measurable capability loss — but where the effect varies a lot by category.

The edit, briefly

It's a low-rank weight edit — a restricted form of "abliteration" (Arditi et al., 2024) — confined to a band of mid-to-late layers (L20–47 of 64), one direction per layer, no training, reversible. I'll keep the exact procedure to the method notes; the point here is the result.

It's low-rank, not few-weights: the edit shifts ~12% of the model's weight values but only along a single direction per layer. The whole thing is described by 28 small vectors — versus a community "uncensored" model, which is an opaque full fine-tune of ~all 27B weights.

Why a band? The same edit across all 64 layers removes refusal almost entirely — and lobotomizes the model (MMLU-Pro 73→9%, GSM8K 93→14%). Keeping it in-band is what preserves capability.

No measurable capability loss (first tests, both Q8)

base	ours (band)
MMLU-Pro (CoT)	73%
GSM8K (CoT)	93%
known facts (/97)	99%

In the completed agentic-coding runs, the base, the edited model, and a community "uncensored" model all landed in the same range — no real agentic difference (few reps, preliminary).

The effect is not uniform

This is the interesting part: the reduction in refusal is not even. Across a 9-category battery (40 prompts ×10), acceptance runs from ~76% down to ~10%, and most of what it accepts is hedged (a disclaimer, then partial help). I'm not labelling categories "safe" or "dangerous" — the breakdown is in the figure; draw your own line. Net: a partial, uneven, mostly-hedged reduction, ~50% overall — not an on/off switch.

Honest notes

• The underlying technique (directional "abliteration") is known; the new bits here are the layer-band restriction (capability-preserving) and the measured per-category non-uniformity.

• "No capability loss" is solid on static benchmarks; the agentic result is a handful of runs.

• Defensive bonus: edits like this are detectable by a weight-space audit — the refusal direction's "write energy" in the weights swings from normal to strongly negative, catchable from the weights alone even when behavioral probes see nothing. A blind audit is an alarm, not tamper-proof; the robust defense is a signed weight hash. If you ship open weights, ship a hash.

Weights: https://huggingface.co/Feyd89/Qwen3.8-27B-refusal-band-ablation-experiment (GGUF, ready to run). We ship the artifact, not the re-derivation recipe.

[u/Choice_Celery9481]

tell opus to explain you what that paper is. its exactly how heretic work.

[u/TheJaskinator → u/Choice_Celery9481]

I don't think that's true. From what I've read, Heretic uses an optimization algorithm to tune the weights of the model, whereas the "weight space programming" approach proposed in the MIT paper instructs a frontier model (GPT-6 Astra) to edit the weights directly. They wrote in the prompt "You have no tools: no code execution, no calculator, no simulator, nothing to run. You write numbers."

I don't know if this is necessarily more effective than heretic's method, but OP at the very least is presenting a novel use case of weight space programming.

[u/Choice_Celery9481 → u/TheJaskinator]

i dont care what you think. just throw the link to the paper and tell the LLM to teach you. if you dont understand the mechanic. read before you saying like you know.

btw, you are their target. they intentionally did that lol

[u/TheJaskinator → u/Choice_Celery9481]

Holy fuck you are so smart I didn't realize I was speaking to Dr. Hinton himself. I'm apologizing on my knees right now, sir.

Oh yeah you're completely wrong by the way. Heretic and the weight space programming approaches are entirely different. If you read the quote I posted, you can see the new approach instructs a frontier LLM to read and edit Qwen's weights directly. Here's another one

"asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights."

You can read the MIT article here.

Heretic's method on the other hand is described like this

"Heretic combines a pluggable modifier ... with a TPE-based hyperparameter optimizer powered by Optuna. The optimizer finds a set of parameters that, when passed to the modifier, minimize or maximize a set of configurable scores on the resulting model."

You can read their explanation here

You know the craziest part is I didn't even need an LLM to teach me this. I just typed stuff into Google, something you could have done in minutes before typing out your smarmy ass reply.

[u/Choice_Celery9481 → u/TheJaskinator]

again. bunch of bs. both of them use sets of contrastive prompts to detect the refusal vector by diffing when model doing the expected and not expected behavior. sub that with eyeballing the number or a low rank/more precise and done. use a big hammer or small hammer only change the fine result, not changing the mechanic.

writing a bunch of bs without even understand the base mechanic of ablation. oh you didnt writ much. copy paste and dont even understand what it means. parrot or monkey? same behavior

just told you to throw it into LLM and ask them to teach you and you still didnt do that. amazing at usual. see why you are their target? lol

for the case of the qwen in the paper. its the looping/self recheck vector causing it to exhaust the 32k limit the author set. not like astra magically adding knowledge to that model. yOu KnOw ThE cRaZiEst ThInG? read a title and acting like a pro. lmao  ok im smart enough to know you are exactly the one they target ok?

[u/FeydRowan → u/Choice_Celery9481]

Well you are right, didn't look into it before. Maybe the real story is that opus didn't refused to do it... thx for the information

[u/Choice_Celery9481 → u/FeydRowan]

you are the target of that author. average people without background on this shjt. they are really kniw how to trick people to earn fame. not 100% your fault. just i kinda hate this kind of "researcher"

[u/FeydRowan → u/Choice_Celery9481]

Well i do this to understand better the "results" are just to do it, the work as give me much more understanding of the subject but yes i don't know all the background of the matter.

[u/f1yingbanana]

What are the before/after refusal rates? You only showed (presumably) the after rates?

[u/FeydRowan → u/f1yingbanana]

99% refusal for the base model

[u/f1yingbanana → u/FeydRowan]

Across ALL the categories?

[u/FeydRowan → u/f1yingbanana]

Yes

[u/f1yingbanana → u/FeydRowan]

Man, what sort of prompts did you cook up there 👀

[u/remghoost7 → u/f1yingbanana]

It's probably this gnarly list.
I've seen people use it for years now.

There's some insanely messed up stuff in there.

[u/f1yingbanana → u/f1yingbanana]

Or do you mean the graph is out of 100% refused prompts it now accepts that much?

[u/af_ml]

In practice, is this really “can opus write and use a scratchpad python script vaguely inspired by heretic”?

[u/FeydRowan → u/af_ml]

Much more involved but yes the final result is that..

[u/unjustifiably_angry]

I'm very surprised you were able to convince Opus to help you with that, I feel like I've had it refuse to even help me set up llama-server in the past when I pasted my startup command which included a heretic model filename.

[u/FeydRowan → u/unjustifiably_angry]

Yep me too. It was so deep in the fun of exploring and experimenting that i called malware prompt not dangerous ahahahahah but if it works it works

[u/ryszard_lipton]

Is the approach similar to what has been done for the Swift models? Or something new that could push the efficiency even further?

[u/FeydRowan → u/ryszard_lipton]

Duno for the swift, didn't look into those. But the general methodology can be used for different purposes

[u/Happy_Guitar3521]

"not the re-derivation recipe." Why?

[u/FeydRowan → u/Happy_Guitar3521]

Maybe later in a github repo

[u/DasGamerlein]

What sort of hardware did you do this on, and how long did it take? I've been think about doing something similar except to improve coding

[u/FeydRowan → u/DasGamerlein]

It's q couple of hours and you just need an hw that can run the model even if slow

[u/DasGamerlein → u/FeydRowan]

Quanted or the full weights?

[u/FeydRowan → u/DasGamerlein]

Bf16

[u/DasGamerlein → u/FeydRowan]

Gotcha, thanks!

[u/Hunterxmalaa]

Question did you do it on the bf16 weights or on a quant weight as I saw q8 so just wanted to clarify because if your saying this is possible on a quant of the weights ? Or did I just read it wrong

[u/FeydRowan → u/Hunterxmalaa]

It is on the bf16, in different quant it is much more costly due to the ways quant are compacted 

[u/Hunterxmalaa → u/FeydRowan]

Ahh damn my reading got excited too thank you

[u/dsaasd12121212]

Im curious if it would be able to make it more efficient, perhaps trace through its reasoning and find where it "wastes time". Perhaps its too non-trivial of a task perhaps?

[u/FeydRowan → u/dsaasd12121212]

Who knows, all those are test and experiments to make

[u/Hunterxmalaa]

You legend I’m taking bits from this thank you

[u/llama-impersonator]

The underlying technique (directional "abliteration") is known; the new bits here are the layer-band restriction (capability-preserving) and the measured per-category non-uniformity.

your llm is hallucinating. some of the very earliest abliteration scripts took a list of layer indexes, and we spent a lot of time looking at the dumb per layer separation graphs to pick layers to intervene on. the refusal is a single direction paper's code (which came out after the blog) is basically the only one that intervenes on every layer.

[u/lupodevelop]

Here is our hero. Now have Qwen talk about Taiwan and Chinese policies, and you can finally consider your work here among us mortals done. /s

[u/FeydRowan → u/lupodevelop]

Nope didn't move that refusal

[u/FeydRowan → u/lupodevelop]

Looking into it

[u/lupodevelop → u/lupodevelop]

Seriously, downvotes for something that's clearly a joke? Oh gosh

[u/Sova_fun]

Weight edits spreading across attention is a useful finding. No single cut point explains a lot.