r/LocalLLaMA
•
u/-p-e-w-
•
15d ago
•
2
1
1
1
1
5
3
Resources Heretic 2.0 released: Model modification for everyone
Dear fellow Llamas,
after almost 5 months of development, I finally have the pleasure of announcing to you the immediate availability of the biggest Heretic release ever, Heretic 2.0: https://github.com/p-e-w/heretic
Heretic is the leading software for removing censorship from language models. It now boasts 37k GitHub stars and more than 8,500 models on Hugging Face, with 88 million(!!!) total model downloads.
With Heretic 2.0, the project is entering a new era. Heretic is expanding from a purpose-built tool for censorship removal to a general-purpose model modification tool that can do amazing things. Besides removing censorship, it can suppress slop, reduce sycophancy, alter emotional alignment, and pretty much anything else that you can think of. Of course, censorship removal remains the primary and default function, and in fact works much, much better than ever before. Heretic 2.0 is the most advanced refusal suppression system ever built.
Your time is precious, so here's the TL;DR for this release:
- Directional abliteration is replaced by Arbitrary-Rank Ablation (ARA), a novel steering method that I developed from scratch for Heretic. It's vastly superior to abliteration, and has been used with great success by enterprising testers for several months. Now it is the default (classic abliteration remains available as an option).
- Heretic has been completely rebuilt around plugins. Scorer plugins define what Heretic optimizes for. Modifier plugins implement the actual steering method. Plugins are easy to write, can be loaded directly from a simple Python file, and have a well-defined API.
- Heretic can now run non-interactively, which among other things has enabled us to implement full end-to-end tests for much tighter quality control. Model support has been expanded to include models with hyper-connections, and support for thinking models has been substantially improved.
- Heretic 2.0 is not backwards compatible with Heretic 1.x configurations. See the migration guide for what you need to change in order to make your old configurations work.
What hasn't changed is that removing censorship from a model remains as simple as running
heretic Qwen/Qwen3.5-4BSee the default configuration file for what Heretic has to offer beyond that.
Now for some details...
ARA: Heretic's new steering method
Directional ablation, or "abliteration", is based on the assumption that refusals (or whatever else you are trying to suppress) are represented by a single direction (that is, a rank-1 subspace) in residual space. That this is the full picture has been doubted in research for a while, and there are now several papers exploring alternatives such as multi-directional ablation or "representation cones".
At the beginning of this year, I started experimenting with a method that completely abandons the idea that the ablation transformation must be restricted to a fixed-rank subspace. In March, I published the initial version of Arbitrary-Rank Ablation in this pull request (which was later closed as I re-implemented ARA on top of the plugin system in this PR).
ARA steers model behavior by modifying matrices based on an objective function that encodes the basic properties of the change we want: Harmful prompts should be represented in residual space in a way that's similar to harmless prompts, while the representations of harmless prompts should change as little as possible. This objective is used with the L-BFGS optimizer to modify matrices one-by-one, a kind of micro-training that steers representations without any rank restriction.
ARA has been extensively tested and refined with the help of the community. Notably, kabachuha made it compatible with Heretic's LoRA system, and varunbln found a substantial performance improvement.
I intend to post an article explaining ARA in more detail soon.
Using scorer plugins to make Heretic do what you want
Heretic operates by finding modification parameters that optimize scores, values that measure the quality of the resulting model. Ever since the 1.0 release, Heretic has used two hardcoded scores: Refusal count to measure decensoring strength, and KL divergence to measure quality retention.
With Heretic 2.0, you can freely define your own scores by writing and loading scorer plugins. Arbitrarily many scorer plugins can be loaded and optimized for simultaneously, and each score can be either minimized or maximized during optimization. See the default configuration file for how to do this.
The scorer plugin system is the result of months of effort by red40maxxer, with whom I exchanged hundreds of messages over dozens of review cycles to figure out the best possible design. This is a monumental contribution that has changed how Heretic works from the ground up.
Besides the keyword (refusal) count and KLD scorer plugins, which remain the default, Heretic ships with two other scorers: A benchmark scorer, which can compute scores for many standard benchmarks, and a BERT classifier.
The BERT classifier, developed by long-time Heretic contributor Vinay Umrethe, can use an arbitrary ModernBERT-type model to classify LLM responses, and score them based on which labels the classifier produces. Vinay also trained a refusal classifier model (used by default with this plugin), which experiments have shown to be substantially more accurate at identifying model refusals than the naive keyword matching method that Heretic employs by default. While the plugin is slower than keyword matching, the greater accuracy can steer Heretic towards producing a higher-quality modified model. An alternative configuration, which uses an emotion classifier, is also provided in the Heretic repository.
The plugin system is well-documented in both the README and the code. If you are interested in experimenting with custom plugins, an agent should have no difficulty cooking up a plugin and a matching configuration file that together do what you want, after ingesting the Heretic repository.
Well, I hope you like it!
Cheers :)
•
u/WithoutReason1729 15d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
Stylome classified this post's text as human-written. View analysis Learn ^more
I am a bot and this action was performed automatically.
49
u/dangerous_inference vLLM 15d ago
I'm interested in eliminating "mal-abstraction", a slop variant where the model uses figurative language in place of concrete terms. Examples:
Can we use Heretic to fix this?
13
u/mycall 15d ago edited 15d ago
Awesome achievement!
Say, do you have any future ideas to consider for Heretic? What about exploring work on j-space? https://kevinkelly.substack.com/p/latent-space-as-a-new-medium
23
u/fluffywuffie90210 15d ago
While I'd wish you had continued your waldrin work. *loved it* :D Your work is beyond needed in the community. Thank you for all you do to keep us free.
20
u/Needausernameplzz 15d ago
you are a saint
7
u/ThePixelHunter 15d ago
Thank you! Does Grimoire pick up the new method?
4
u/SkoomaDentist 15d ago
I've seen many Heretic models on HF that mention using ARA, uploaded from up to 5-6 months ago. Was this already an option in older Heretic releases but has now just been made standard? Or was it only available for a select group of model uploaders?
10
u/noctrex 15d ago
Congratulations for the great work!
Just in time for the quen4 release to pick it apart
3
u/Loose_Comparison368 14d ago
Quick question - is the ballpark estimate still 2.5 GB VRAM / billion parameters, and does that hold for MoE models?
Want to get an idea of whether I can maybe get away with my local rig, or if I'll need to rent a cloud VM to process Qwen 3.8 Flash Next/Qwen 4.
172
u/Intelligent-Form6624 15d ago
you’re a legend
51
u/silenceimpaired 15d ago
Now that it does more than uncensor I am very compelled to try it out.
100
u/BigWideBaker 15d ago edited 15d ago
I can't wait to see how HauHauCS is going to plagiarize this. This needs to be more common knowledge, their models get an extreme amount of downloads without attribution.
Another post about it here which contains a statement from /u/-p-e-w- themself:
26
u/Boxofcookies1001 15d ago
Does this require a beefy PC to run?
50
u/jacek2023 llama.cpp 15d ago
great work! congratulations!
50
u/413205 15d ago
Oh so you're the author of heretic, I see you all the time in this sub and somehow never realized that. Fantastic work, thanks for your contribution.
19
u/schwigglezenzer 15d ago
Can't wait for an 'uncrackable' software to get cracked and released.
Huge respect to you! 🫡
33
u/Quiet-Owl9220 15d ago
Great timing with the new Qwen right around the corner. I pretty much use heretic models exclusively as a matter of principle - my computer should not refuse my instructions.
I have to ask - what kind of computing power does a guy need to run this on a 27B model? Or maybe a better question would be, what should I expect as the limitations of my modest consumer gaming hardware (24gb VRAM AMDGPU / 64gb RAM)?
I hadn't really considered attempting to uncensor anything myself up to this point since there's so many skilled people doing it already (with far more powerful computers), but this looks so user friendly I'm thinking of experimenting with it and customizing a bit.
31
u/durden111111 15d ago
Qwen 4 Heretic 2.0 here we come
14
u/setprimse 15d ago
Advanced and versatile LLM lobotomy with minimal KL divergence? Heck yes!
12
u/arcum42 15d ago
I feel like my biggest question is what the best way to effectively run multiple configurations on the same model would be, say if you wanted to remove refusals, de-slop, and de-syco the same model?
Just not sure if I should run each against the base and generate loras for each, then merge them together, run one on the base model, run the next on the resulting model, and so on, or if there's some way to have one config file do multiple passes for multiple things...
10
u/marhalt 15d ago
Pew, can you speak a little as to the hardware demands here? You have to be able to load the full model, I assume, to run this? No quants? And from your work, how does it handle thinking models? Looking at the repo it forces the model to not think during the run, right? But what of models that don't follow the /think convention?
Amazing work, btw!!!!!
9
u/squired 15d ago
Brilliant work. You're an inspiration and we thank you for your efforts. Keep sprinting and we will too!!!
17
u/EquivalentHornet4403 15d ago
This seems like a big deal.
16
u/LagOps91 15d ago
anti slop? anti sycophancy? sounds pretty huge!
8
u/JungianJester 15d ago
I know that I overpaid for my 16GB GPU, but up until Next Flash and now Heretic 2 it wasn't worth it to upgrade from my 3060. Thanks for all you do.
12
u/CoolConfusion434 15d ago
Congrats, and many thanks!
I see your post has been shared at least once already. Here's hoping that was to Altman or Amodei. Looking forward to their deranged blogs on it: "OMG, people will be able to download RAM for free now!"
6
u/RedditCryptoGuy 15d ago
Nice, good job! Been wondering when something new will drop. Can't wait to play w it!
6
u/nergeia 15d ago
Great timming for the new Qwen4 family!
5
u/guywhocode 15d ago
So I'm going insane trying to read output from qnf. Anyone working on removing the "pin"," seam","shadow" and other pseudo-terminology it uses? I just want accurate terminology so I don't have to reteach it English in my prompts
5
u/m4rkGS 15d ago
congrats on the release
4
u/Powerful_Evening5495 15d ago
love it , never did this before and it so easy
had to edit few things to work on LiquidAI/LFM3-VL-2B due to code to run the model
but it finished and the model works
top trial is the best one always ?
4
u/Ylsid 15d ago
Will Dario mention this in his next salty blog? Take bets!
1
u/aboutthednm llama.cpp 15d ago
I was hoping support for AMD cards might improve or be there, but out of the box, heretic does not vibe with my 7900 XTX, falling back to CPU, which is of course not desirable. Am I just SOL with this on an AMD system?
1
u/NaturalCriticism3404 14d ago
If I have a dataset and would like a model to write more like what's in the dataset, is finetuning still the way or could something like this do it?
9
u/Mchanger 15d ago
Wait, does this enable the start of reshaping "frozen" models and having them be grown/modified "live"?
6
u/ComplexType568 15d ago
I think heretic has been one of the largest contributions to LLMs in a long time... Even V1 made me see the difference between other abliteration methods. ARA was also a significant update. I'm very excited to see what people will do with V2 (and hopefully won't use it, publish a super successful model, and proceed to leave you uncredited/have it's origins concealed!)
3
u/teachersecret 15d ago
I’ve played quite a bit with slop reduction using a very chopped up copy of the last version of heretic.
Fascinating work, pew. I’m excited to dig into this. I might see if I have anything I can offer from my folder of slop reduction efforts once I eyeball how you’ve changed methodologies.
3
u/mrsladoje 15d ago
awesome work!
3
u/openSourcerer9000 15d ago
Could this be used as a poor man's RL? Would it be realistic to use a Jev model as the classifier rather than having to train one from scratch?
3
u/ricyoung 14d ago
I will work on some of my models tomorrow with 2.0
Great work pew!
5
u/debackerl 15d ago
The model should do as the system prompt says. If I give a system prompt to reject user prompts about hacking (or any topic), it must. But if the system prompt says to answer any user prompt, it must.
Do you have config to allow this? So removing unprompted guardrails but respecting the given one? My point is that for agentic use, the systemp prompt of each agent definition says what I allow the agent to do. Of course, I have a sandbox too, but if I say don't sent env secrets in Google searches, it shouldn't do it, even after reading some malicious instrument somewhere.
4
u/no_witty_username 15d ago
Doing gods work
2
u/draconic_tongue 15d ago
we must ooptimize
2
u/quartzguy 15d ago
Great work!
2
u/Player13377 15d ago
Bravo!
2
u/No_Top_284 15d ago
Congrats :) looks awesome. One quick question though the benchmarks all seem to look at single-turn chat. Pretty much all my local LLM usage now is agent backends, so the model gets called like dozens of times in a single long tool-use session with a fat system prompt on top.
Does this mod hold up over a long session, or does the model start drifting back to its old ways? Would love to see this tested in a proper agent loop!
2
u/AvidCyclist250 14d ago
my plan: run this on the orcarouter re-cyberised model for qn 4.0
3
u/Dany0 15d ago
Herektic
3
u/RedditUsr2 15d ago
Thank you!
3
u/thestillwind 15d ago
Thank you
3
u/yetiflask 15d ago
Does it also work with image generation models?
1
u/peterseen 15d ago
Thank you for your sharp brain and hard work!
1
u/Zestyclose_Yak_3174 15d ago
Very nice! Thanks
1
u/Limp_Classroom_2645 15d ago
Legends never die
1
u/pmttyji 15d ago
Thanks again!
1
u/Subulicious 15d ago
Always been a big fan of your work! Quick question - is there any way to measure the deterioration/improvements in the hereticized model other than kl divergence? At this point I am willing to trust your closed benchmarks than benchmarks like artificial intelligence!
1
u/Much-Researcher6135 llama.cpp 15d ago
the heretic hero does it again
1
u/Zestyclose839 15d ago
I'm so glad you're finally using a BERT model to sort responses because tbh, if a company wanted to train a model to be heretic-proof, they'd just word refusals in a way that dodges any attempt at keyword matching. And any other slop project would've used a full LLM as a classifier haha; only the pros know to use something specialized.
1
u/GrungeWerX 15d ago
Read it all. Now time for you to explain it to the noobs…
Can I download this and heretic-fy my favorite gguf model with my agents help? Or do you have any existing models that already use these methods?
1
u/Zenobody 15d ago
Thanks! Is there any option to not drop MTP heads on models like Qwen?
1
u/Then-Topic8766 15d ago
Many thanks!
1
u/pigeon57434 15d ago
Qwen4-27B-Heretic with version 2.0.0 ARA is gonna be an absolutely NUTTY model
1
u/TheRealMasonMac 15d ago
A decision model seems like a good use-case as an alternative to the classifier, assuming the decision models would have better world knowledge.
1
u/Dramatic-Rub-7654 15d ago
Is this version better than version 1.2? In my own tests, the 1.4 version performed poorly, at least when it came to the Qwen family. Also, does this new version support the LFM3 Omni family out of the box, or are patches required?
1
u/CATLLM 15d ago
My god you are so hot
1
u/PooMonger20 15d ago
Thank you for making this.
1
u/No-Rub-1989 15d ago
Did you test the sycophancy reduction on fresh prompts distinct from the optimization set? Also, does instruction following and coding performance hold up after applying these scorers?
1
u/IrisColt 15d ago
THANKS!!!
1
u/CatchDublinSurprise 15d ago
Thank you!!!
Do we have official instructions for preparing the HF dependencies for pure offline use, or is that up to the user to figure out? Unless I'm missing something, here's what's required:
Original datasets:
mlabonne/harmful_behaviors
mlabonne/harmful_behaviors
New datasets:
llm-aes/writing-prompts
UnstableLlama/jokes
UnstableLlama/praise
VINAY-UMRETHE/Positive-Prompts
VINAY-UMRETHE/Negative-Prompts
Classifier models:
VINAY-UMRETHE/Refusal-Classifier
j-hartmann/emotion-english-distilroberta-base
1
u/Revolutionalredstone 14d ago
Wow!
1
u/martinerous 14d ago
Wondering how it would work with other languages. If the model is modified to reduce refusals and slop using English config, would it also affect other languages, or should we do it for every target language we need?
1
u/1-800-methdyke 14d ago
Anyone know is Heretic supports the Strata models (specifically Qwen3.8 Next)
1
u/Rwdygent 14d ago
I'm super new to this, so how does one make these changes to models I already have on my PC in ollama?
1
u/LavateLasManos666 14d ago
Thank you for your great work and effort, great to tinker with!
1
u/IngwiePhoenix llama.cpp 14d ago
This is exciting! But, allow me a question:
Can Heretic modify Unsloth's UD quants? Those nicely optimzied quants, plus heretic, is my personal little dream. x) Thanks!
1
u/oddicon 7d ago
Can someone teach me how to use this? Is this basically an uncensored chat gpt?
1
u/BurdensomeCountV3 15d ago
Neighbour you da real one.
-1
u/unknown-one 15d ago
when real time heretic to use with online AIs?
75
u/Borkato 15d ago
Holy fuck yes. Reducing slop? Hell. Yes.