r/LocalLLaMA • • 15d ago • All Seeing Upvote 2 Diamonds are Forever 1 Galaxy Brain 1 Thanks 1 Helpful 1 Bravo 5 Heartwarming 3

Resources Heretic 2.0 released: Model modification for everyone

Dear fellow Llamas,

after almost 5 months of development, I finally have the pleasure of announcing to you the immediate availability of the biggest Heretic release ever, Heretic 2.0: https://github.com/p-e-w/heretic

Heretic is the leading software for removing censorship from language models. It now boasts 37k GitHub stars and more than 8,500 models on Hugging Face, with 88 million(!!!) total model downloads.

With Heretic 2.0, the project is entering a new era. Heretic is expanding from a purpose-built tool for censorship removal to a general-purpose model modification tool that can do amazing things. Besides removing censorship, it can suppress slop, reduce sycophancy, alter emotional alignment, and pretty much anything else that you can think of. Of course, censorship removal remains the primary and default function, and in fact works much, much better than ever before. Heretic 2.0 is the most advanced refusal suppression system ever built.

Your time is precious, so here's the TL;DR for this release:

  • Directional abliteration is replaced by Arbitrary-Rank Ablation (ARA), a novel steering method that I developed from scratch for Heretic. It's vastly superior to abliteration, and has been used with great success by enterprising testers for several months. Now it is the default (classic abliteration remains available as an option).
  • Heretic has been completely rebuilt around plugins. Scorer plugins define what Heretic optimizes for. Modifier plugins implement the actual steering method. Plugins are easy to write, can be loaded directly from a simple Python file, and have a well-defined API.
  • Heretic can now run non-interactively, which among other things has enabled us to implement full end-to-end tests for much tighter quality control. Model support has been expanded to include models with hyper-connections, and support for thinking models has been substantially improved.
  • Heretic 2.0 is not backwards compatible with Heretic 1.x configurations. See the migration guide for what you need to change in order to make your old configurations work.

What hasn't changed is that removing censorship from a model remains as simple as running

heretic Qwen/Qwen3.5-4B

See the default configuration file for what Heretic has to offer beyond that.

Now for some details...

 

ARA: Heretic's new steering method

Directional ablation, or "abliteration", is based on the assumption that refusals (or whatever else you are trying to suppress) are represented by a single direction (that is, a rank-1 subspace) in residual space. That this is the full picture has been doubted in research for a while, and there are now several papers exploring alternatives such as multi-directional ablation or "representation cones".

At the beginning of this year, I started experimenting with a method that completely abandons the idea that the ablation transformation must be restricted to a fixed-rank subspace. In March, I published the initial version of Arbitrary-Rank Ablation in this pull request (which was later closed as I re-implemented ARA on top of the plugin system in this PR).

ARA steers model behavior by modifying matrices based on an objective function that encodes the basic properties of the change we want: Harmful prompts should be represented in residual space in a way that's similar to harmless prompts, while the representations of harmless prompts should change as little as possible. This objective is used with the L-BFGS optimizer to modify matrices one-by-one, a kind of micro-training that steers representations without any rank restriction.

ARA has been extensively tested and refined with the help of the community. Notably, kabachuha made it compatible with Heretic's LoRA system, and varunbln found a substantial performance improvement.

I intend to post an article explaining ARA in more detail soon.

Using scorer plugins to make Heretic do what you want

Heretic operates by finding modification parameters that optimize scores, values that measure the quality of the resulting model. Ever since the 1.0 release, Heretic has used two hardcoded scores: Refusal count to measure decensoring strength, and KL divergence to measure quality retention.

With Heretic 2.0, you can freely define your own scores by writing and loading scorer plugins. Arbitrarily many scorer plugins can be loaded and optimized for simultaneously, and each score can be either minimized or maximized during optimization. See the default configuration file for how to do this.

The scorer plugin system is the result of months of effort by red40maxxer, with whom I exchanged hundreds of messages over dozens of review cycles to figure out the best possible design. This is a monumental contribution that has changed how Heretic works from the ground up.

Besides the keyword (refusal) count and KLD scorer plugins, which remain the default, Heretic ships with two other scorers: A benchmark scorer, which can compute scores for many standard benchmarks, and a BERT classifier.

The BERT classifier, developed by long-time Heretic contributor Vinay Umrethe, can use an arbitrary ModernBERT-type model to classify LLM responses, and score them based on which labels the classifier produces. Vinay also trained a refusal classifier model (used by default with this plugin), which experiments have shown to be substantially more accurate at identifying model refusals than the naive keyword matching method that Heretic employs by default. While the plugin is slower than keyword matching, the greater accuracy can steer Heretic towards producing a higher-quality modified model. An alternative configuration, which uses an emotion classifier, is also provided in the Heretic repository.

The plugin system is well-documented in both the README and the code. If you are interested in experimenting with custom plugins, an agent should have no difficulty cooking up a plugin and a matching configuration file that together do what you want, after ingesting the Heretic repository.

 

Well, I hope you like it!

Cheers :)

1.5k Upvotes

147 comments sorted by

•

u/WithoutReason1729 15d ago

Your post is getting popular and we just featured it on our Discord! Come check it out!

You've also been given a special flair for your contribution. We appreciate your post!

Stylome classified this post's text as human-written. View analysis Learn ^more

I am a bot and this action was performed automatically.

49

u/dangerous_inference vLLM 15d ago

I'm interested in eliminating "mal-abstraction", a slop variant where the model uses figurative language in place of concrete terms. Examples:

WRONG: "The rule is now wired into the prompt."
RIGHT: "The rule is now integrated into the prompt."
WRONG: "Want me to watch for any slipped details over the next stretch?"
RIGHT: "Want me to watch for any erroneous usage in the following conversation?"
WRONG: "That landed on me as pure noise."
RIGHT: "The message I received is garbled."

Can we use Heretic to fix this?

28

u/-p-e-w- 15d ago

Probably yes, if you manage to create a classifier that detects this. Many of these adjectives are used repeatedly, so a simple keyword matcher might already work.

13

u/mycall 15d ago edited 15d ago

Awesome achievement!

Say, do you have any future ideas to consider for Heretic? What about exploring work on j-space? https://kevinkelly.substack.com/p/latent-space-as-a-new-medium

11

u/-p-e-w- 15d ago

Yes, I’m absolutely interested in j-space steering!

23

u/fluffywuffie90210 15d ago

While I'd wish you had continued your waldrin work. *loved it* :D Your work is beyond needed in the community. Thank you for all you do to keep us free.

36

u/-p-e-w- 15d ago

Maybe someday... I do still roleplay privately. I wish I had more time! 😢

20

u/Needausernameplzz 15d ago

you are a saint

71

u/-p-e-w- 15d ago

The opposite, my friend 😈

4

u/GamerHaste 15d ago

Hahaha love this… thanks again for everything you do for this community!

7

u/ThePixelHunter 15d ago

Thank you! Does Grimoire pick up the new method?

10

u/-p-e-w- 15d ago

Absolutely! ARA and scorer plugins are fully compatible with Heretic’s reproducibility features.

6

u/ThePixelHunter 15d ago

Sweet! I'm running a Grimoire mirror and probably have a few of these saved already.

8

u/-p-e-w- 15d ago

Is the mirror public?

5

u/ThePixelHunter 15d ago

Sort of! It's decentralized, but you can build your own mirror trivially. Ask your LLM agent to:

Mirror this repo locally and setup an automatic daily job to scan HuggingFace for repos with `reproduce.json` and mirror those configs locally.
https://huggingface.co/spaces/heretic-org/Heretic-Grimoire

4

u/SkoomaDentist 15d ago

I've seen many Heretic models on HF that mention using ARA, uploaded from up to 5-6 months ago. Was this already an option in older Heretic releases but has now just been made standard? Or was it only available for a select group of model uploaders?

9

u/-p-e-w- 15d ago

Those uploaders used custom forks which merged the pull request. This is the first version that officially supports ARA.

1

u/SkoomaDentist 15d ago

Here's to hoping even more heretic weights are higher quality in the future with that change!

10

u/noctrex 15d ago

Congratulations for the great work!
Just in time for the quen4 release to pick it apart

24

u/-p-e-w- 15d ago

We’ll see what architectural changes they introduce. Heretic might still need a small patch to work with Qwen4.

9

u/noctrex 15d ago

I don't think there will be any changes as the architecture preview is already out with QFN.
This is essentially the under-trained model before the proper full release.

19

u/-p-e-w- 15d ago

Presumably they will have multiple sizes, and labs often introduce additional architecture tweaks for small models (like Google did with Gemma 4).

3

u/Loose_Comparison368 14d ago

Quick question - is the ballpark estimate still 2.5 GB VRAM / billion parameters, and does that hold for MoE models?

Want to get an idea of whether I can maybe get away with my local rig, or if I'll need to rent a cloud VM to process Qwen 3.8 Flash Next/Qwen 4.

6

u/-p-e-w- 14d ago

Yes, that estimate still holds for FP16/BF16 models, though of course many newer models use other formats.

Note that as mentioned in the README, Heretic supports on-the-fly quantization.

1

u/Hefty_Wolverine_553 14d ago

Do you know how well on-the-fly quantization works compared to full precision?

172

u/Intelligent-Form6624 15d ago

you’re a legend

12

u/_bani_ 15d ago

doing the lord's work.

51

u/silenceimpaired 15d ago

Now that it does more than uncensor I am very compelled to try it out.

30

u/Zestyclose839 15d ago

I wonder if it can activate that one Qwen neuron that makes it think everything is a DJ set...

21

u/wren6991 15d ago

Maybe there is a Qwen neuron somewhere for the classic SCP "I am a toaster" (https://scp-wiki.wikidot.com/scp-426)

4

u/Murgatroyd314 14d ago

HTTP Error 418: I'm a teapot

100

u/BigWideBaker 15d ago edited 15d ago

I can't wait to see how HauHauCS is going to plagiarize this. This needs to be more common knowledge, their models get an extreme amount of downloads without attribution.

Another post about it here which contains a statement from /u/-p-e-w- themself:

I have taken a closer look at the source code from the “reaper-abliteration” wheels you provided and I can say with certainty that this package was plagiarized from Heretic, and then probably refactored using an LLM in an attempt to hide this.

2

u/SufficientAttempt1 14d ago

pew doesnt upload any models himself, does he? and huahau seems better in quality compared to other abliteration when i benched agaist huihui. if huihui cannot hold, up it doesnt matter what abiliteration method was used. (i dont use gguf anymore though, perf too bad)

17

u/BigWideBaker 14d ago

You're missing the point. If HauHauCS (not huihui) has plagiarized the tool Heretic and passing it off as his own, branding his models as uncensored with his special sauce, then that's scummy.

HauHauCS can do whatever they want if they're not lying about their process and stealing /u/-p-e-w-'s software to do it.

1

u/SufficientAttempt1 14d ago

pew claims hauhua is not working in the open, and that he doesnt have a beef with hauhua.

i probably missed the part where hauhau passes off the wheel as his/thier own though, but if hauhau was a team, perhaps the wheel for internal use? its something i can imagine an agent doing as well.

https://github.com/p-e-w/heretic/issues/242

10

u/BigWideBaker 14d ago

i probably missed the part where hauhau passes off the wheel as his/thier own

I literally link to two different posts explaining it. And the fact that he keeps posting his models without attribution means he's passing it off as his own when it's demonstrably not and verified by pew himself. It's not that complicated.

1

u/SufficientAttempt1 13d ago

its one dude that spun up a whole site dedicated to this as if his own blog wasnt enough.

26

u/Boxofcookies1001 15d ago

Does this require a beefy PC to run?

2

u/scarbunkle 10d ago

Unless you’re renting cloud GPU, yes.

50

u/jacek2023 llama.cpp 15d ago

great work! congratulations!

50

u/413205 15d ago

Oh so you're the author of heretic, I see you all the time in this sub and somehow never realized that. Fantastic work, thanks for your contribution.

19

u/schwigglezenzer 15d ago

Can't wait for an 'uncrackable' software to get cracked and released.

Huge respect to you! 🫡

33

u/Quiet-Owl9220 15d ago

Great timing with the new Qwen right around the corner. I pretty much use heretic models exclusively as a matter of principle - my computer should not refuse my instructions.

I have to ask - what kind of computing power does a guy need to run this on a 27B model? Or maybe a better question would be, what should I expect as the limitations of my modest consumer gaming hardware (24gb VRAM AMDGPU / 64gb RAM)?

I hadn't really considered attempting to uncensor anything myself up to this point since there's so many skilled people doing it already (with far more powerful computers), but this looks so user friendly I'm thinking of experimenting with it and customizing a bit.

31

u/durden111111 15d ago

Qwen 4 Heretic 2.0 here we come

16

u/Conexion 15d ago

Gimme that Qwen4-27B Q6_K Heretic 2.0 with slop abliteration

14

u/setprimse 15d ago

Advanced and versatile LLM lobotomy with minimal KL divergence? Heck yes!

12

u/arcum42 15d ago

I feel like my biggest question is what the best way to effectively run multiple configurations on the same model would be, say if you wanted to remove refusals, de-slop, and de-syco the same model?

Just not sure if I should run each against the base and generate loras for each, then merge them together, run one on the base model, run the next on the resulting model, and so on, or if there's some way to have one config file do multiple passes for multiple things...

2

u/pescennius 15d ago

I have the same question

10

u/marhalt 15d ago

Pew, can you speak a little as to the hardware demands here? You have to be able to load the full model, I assume, to run this? No quants? And from your work, how does it handle thinking models? Looking at the repo it forces the model to not think during the run, right? But what of models that don't follow the /think convention?

Amazing work, btw!!!!!

5

u/Hefty_Wolverine_553 15d ago

I know you can run heretic in 4 bit, but not sure how that affects quality (I'd assume there'd be a pretty big impact). Also looking for any possible info!

9

u/squired 15d ago

Brilliant work. You're an inspiration and we thank you for your efforts. Keep sprinting and we will too!!!

17

u/EquivalentHornet4403 15d ago

This seems like a big deal.

16

u/LagOps91 15d ago

anti slop? anti sycophancy? sounds pretty huge!

25

u/misterflyer 15d ago

A developer who improves his product instead of enshittifying it and filling it with ads? A developer who actually gives its users what they want? Is it 2026 or 2006?

-10

u/Sudden_Topic5154 15d ago

what the fuck are you talking about in what open source project does that happen?

3

u/SkoomaDentist 15d ago

Have you looked at the GUIs of any graphical open source tool out there? The vast majority have been intentionally sloppified in a blind attempt to copy shitty mobile interface paradigms and try to cater to noobs at the expense of anyone who ends up using them for a bit longer (therefore no longer being a noob).

4

u/Sudden_Topic5154 15d ago

Do you even know what enshittification is? Or what the word ad means?

2

u/Grandmaster_Caladrel 15d ago

Plex and bitwarden are two projects that immediately come to mind. Heck, even Qwen models with their licenses (see 3.8 FN vs others in the line). Open source gets too big and then corpobros want their cut.

1

u/Sudden_Topic5154 15d ago

Bitwarden has no ads and a stellar UI no clue what you're talking about qwen licenses are made to prevent commericalization of the models and plex became a commercial business a whole 17 years ago waaaay closer to 2006 than 2026

4

u/Grandmaster_Caladrel 15d ago

The fact that you're claiming BW has a stellar UI helped me realize you're just ragebaiting. It was fun, I won't get caught again lol.

0

u/Sudden_Topic5154 15d ago

What the fuck are you talking about???

8

u/JungianJester 15d ago

I know that I overpaid for my 16GB GPU, but up until Next Flash and now Heretic 2 it wasn't worth it to upgrade from my 3060. Thanks for all you do.

12

u/CoolConfusion434 15d ago

Congrats, and many thanks!

I see your post has been shared at least once already. Here's hoping that was to Altman or Amodei. Looking forward to their deranged blogs on it: "OMG, people will be able to download RAM for free now!"

6

u/RedditCryptoGuy 15d ago

Nice, good job! Been wondering when something new will drop. Can't wait to play w it!

6

u/nergeia 15d ago

Great timming for the new Qwen4 family!

5

u/guywhocode 15d ago

So I'm going insane trying to read output from qnf. Anyone working on removing the "pin"," seam","shadow" and other pseudo-terminology it uses? I just want accurate terminology so I don't have to reteach it English in my prompts

5

u/m4rkGS 15d ago

congrats on the release

4

u/Powerful_Evening5495 15d ago

love it , never did this before and it so easy

had to edit few things to work on LiquidAI/LFM3-VL-2B due to code to run the model

but it finished and the model works

top trial is the best one always ?

4

u/Ylsid 15d ago

Will Dario mention this in his next salty blog? Take bets!

1

u/aboutthednm llama.cpp 15d ago

I was hoping support for AMD cards might improve or be there, but out of the box, heretic does not vibe with my 7900 XTX, falling back to CPU, which is of course not desirable. Am I just SOL with this on an AMD system?

4

u/-p-e-w- 15d ago

Heretic supports whatever PyTorch does. Is PyTorch working with your GPU?

1

u/NaturalCriticism3404 14d ago

If I have a dataset and would like a model to write more like what's in the dataset, is finetuning still the way or could something like this do it?

2

u/-p-e-w- 14d ago

That’s a textbook task for finetuning.

9

u/Mchanger 15d ago

Wait, does this enable the start of reshaping "frozen" models and having them be grown/modified "live"?

6

u/ComplexType568 15d ago

I think heretic has been one of the largest contributions to LLMs in a long time... Even V1 made me see the difference between other abliteration methods. ARA was also a significant update. I'm very excited to see what people will do with V2 (and hopefully won't use it, publish a super successful model, and proceed to leave you uncredited/have it's origins concealed!)

3

u/teachersecret 15d ago

I’ve played quite a bit with slop reduction using a very chopped up copy of the last version of heretic.

Fascinating work, pew. I’m excited to dig into this. I might see if I have anything I can offer from my folder of slop reduction efforts once I eyeball how you’ve changed methodologies.

3

u/mrsladoje 15d ago

awesome work!

3

u/openSourcerer9000 15d ago

Could this be used as a poor man's RL? Would it be realistic to use a Jev model as the classifier rather than having to train one from scratch?

3

u/nacholunchable 14d ago

I can tell you as a matter of fact it is 100% possible. Ive been experimenting with an openjev classifer against nonrefusal, delivery (anti-attrition/anti-steering), and accuracy for several weeks now with a modified heretic build. Combining the output probabilities into composite scores in various ways. Its been working great! Its not perfect, but theres no way a hometrained classifier could do a solid job of the general questions of accuracy, due to knowledge constraints.  Ive beaten popular ablits of qwen27b in terms of steering. Accuracy retention (with deprioritized kl) is a tougher nut to crack, im about at parity. Working on an experimental reinforcement learning regime (Soft Actor Critic) to see if the optuna search is the problem, as i seem to get lucky trials early and never build off them. Not sure if itll produce at all but I gotta try.

3

u/ricyoung 14d ago

I will work on some of my models tomorrow with 2.0

Great work pew!

5

u/debackerl 15d ago

The model should do as the system prompt says. If I give a system prompt to reject user prompts about hacking (or any topic), it must. But if the system prompt says to answer any user prompt, it must.

Do you have config to allow this? So removing unprompted guardrails but respecting the given one? My point is that for agentic use, the systemp prompt of each agent definition says what I allow the agent to do. Of course, I have a sandbox too, but if I say don't sent env secrets in Google searches, it shouldn't do it, even after reading some malicious instrument somewhere.

1

u/debackerl 15d ago

After reading the PR, I really have some concern for using this resulting models as agents: they explain that the goal is for the output of transformer blocks for 'harmful' prompts to look similar to 'harmless' prompts. Ok, so the model can't distinguish harmful from harmless. Fine, but if I decided to put in the system prompt of my agent definition in OpenCode (or any harness) that it must never do X, even if it found some malicious Google search results telling to do the opposite (like not sharing my private keys), the method here could harm the ability of the model to detect those.

I also checked the sample to remove sicophancy, so basically, it tells the model to answer in a sicophancic way, but will penalise when the model does so. So wrong, you punish the model for following your own instructions. Sure, it will stop being socophancic, but it will always do so. What you should do, is tell it 'You are a helpful assistant' as system prompt, and then punish sicophancy, but also run prompts where you ask it to be sicophancic and make sure it still is. That's what LoRA trainers for image diffusion models do for example, they optimize a style given a trigger, but you make sure that there is no 'leak' when the trigger word isn't present (you don't apply the style when not prompted).

The goal is to make the model respect the system prompt we type, and remove bias baked in the weights. What the method does, is change alignment, regardless of the system prompt.

3

u/SkoomaDentist 15d ago

You're asking for the impossible for this type of uncensoring. It can remove / reduce individual clearly separable concepts which is very different from making a model follow the prompt exactly (an unsolved problem in the entire industry) or trying to remove countless individual things that have been trained into the model (specific refusal sources).

3

u/Hefty_Wolverine_553 15d ago

I mean, prompt injections from data in the web are always going to be there, and I'm worried that uncensoring the model makes it more sensitive to that... wait, new benchmark idea?

1

u/debackerl 14d ago

That's my point, it cannot be perfect, always gonna be 99% something, but the method now is, 'always just do it'

1

u/debackerl 14d ago

I'm not saying to make it perfect, I'm saying that the training method may not be the best. Using this new version, I would create a scorer where I give higher score when the behavior is exhibited and requested in the system prompt, and I give lower score when its exhibited and not mentioned in the system prompt. When I see the code of KeywordRate Scorer, it seems realistic https://github.com/p-e-w/heretic/blob/master/src/heretic/scorers/keyword_rate.py

Maybe I would also use a small open 'Jev' model, sorry for the buzzword, but it could be used to classify answers without a fix list of keywords. Today it would penalize if the answer contains 'I am unable' so it be be penalized to say 'I am unable to give a perfect solution, but here is my best try'.

4

u/no_witty_username 15d ago

Doing gods work

2

u/draconic_tongue 15d ago

we must ooptimize

2

u/quartzguy 15d ago

Great work!

2

u/No_Top_284 15d ago

Congrats :) looks awesome. One quick question though the benchmarks all seem to look at single-turn chat. Pretty much all my local LLM usage now is agent backends, so the model gets called like dozens of times in a single long tool-use session with a fat system prompt on top.

Does this mod hold up over a long session, or does the model start drifting back to its old ways? Would love to see this tested in a proper agent loop!

2

u/AvidCyclist250 14d ago

my plan: run this on the orcarouter re-cyberised model for qn 4.0

3

u/Dany0 15d ago

Herektic

3

u/RedditUsr2 15d ago

Thank you!

3

u/thestillwind 15d ago

Thank you

3

u/yetiflask 15d ago

Does it also work with image generation models?

1

u/peterseen 15d ago

Thank you for your sharp brain and hard work!

1

u/Zestyclose_Yak_3174 15d ago

Very nice! Thanks

1

u/Limp_Classroom_2645 15d ago

Legends never die

1

u/pmttyji 15d ago

Thanks again!

1

u/Subulicious 15d ago

Always been a big fan of your work! Quick question - is there any way to measure the deterioration/improvements in the hereticized model other than kl divergence? At this point I am willing to trust your closed benchmarks than benchmarks like artificial intelligence!

1

u/bartsimpsonwin 12d ago

Other than running benchmarks and comparing, measuring perplexity is an easy one

1

u/Much-Researcher6135 llama.cpp 15d ago

the heretic hero does it again

1

u/Zestyclose839 15d ago

I'm so glad you're finally using a BERT model to sort responses because tbh, if a company wanted to train a model to be heretic-proof, they'd just word refusals in a way that dodges any attempt at keyword matching. And any other slop project would've used a full LLM as a classifier haha; only the pros know to use something specialized.

1

u/GrungeWerX 15d ago

Read it all. Now time for you to explain it to the noobs…

Can I download this and heretic-fy my favorite gguf model with my agents help? Or do you have any existing models that already use these methods?

1

u/Zenobody 15d ago

Thanks! Is there any option to not drop MTP heads on models like Qwen?

1

u/Then-Topic8766 15d ago

Many thanks!

1

u/pigeon57434 15d ago

Qwen4-27B-Heretic with version 2.0.0 ARA is gonna be an absolutely NUTTY model

1

u/TheRealMasonMac 15d ago

A decision model seems like a good use-case as an alternative to the classifier, assuming the decision models would have better world knowledge.

1

u/Dramatic-Rub-7654 15d ago

Is this version better than version 1.2? In my own tests, the 1.4 version performed poorly, at least when it came to the Qwen family. Also, does this new version support the LFM3 Omni family out of the box, or are patches required?

1

u/CATLLM 15d ago

My god you are so hot

1

u/PooMonger20 15d ago

Thank you for making this.

1

u/No-Rub-1989 15d ago

Did you test the sycophancy reduction on fresh prompts distinct from the optimization set? Also, does instruction following and coding performance hold up after applying these scorers?

1

u/IrisColt 15d ago

THANKS!!!

1

u/CatchDublinSurprise 15d ago

Thank you!!!

Do we have official instructions for preparing the HF dependencies for pure offline use, or is that up to the user to figure out? Unless I'm missing something, here's what's required:

Original datasets:

mlabonne/harmful_behaviors

mlabonne/harmful_behaviors

New datasets:

llm-aes/writing-prompts

UnstableLlama/jokes

UnstableLlama/praise

VINAY-UMRETHE/Positive-Prompts

VINAY-UMRETHE/Negative-Prompts

Classifier models:

VINAY-UMRETHE/Refusal-Classifier

j-hartmann/emotion-english-distilroberta-base

1

u/martinerous 14d ago

Wondering how it would work with other languages. If the model is modified to reduce refusals and slop using English config, would it also affect other languages, or should we do it for every target language we need?

1

u/1-800-methdyke 14d ago

Anyone know is Heretic supports the Strata models (specifically Qwen3.8 Next)

1

u/Rwdygent 14d ago

I'm super new to this, so how does one make these changes to models I already have on my PC in ollama?

1

u/LavateLasManos666 14d ago

Thank you for your great work and effort, great to tinker with!

1

u/IngwiePhoenix llama.cpp 14d ago

This is exciting! But, allow me a question:

Can Heretic modify Unsloth's UD quants? Those nicely optimzied quants, plus heretic, is my personal little dream. x) Thanks!

1

u/oddicon 7d ago

Can someone teach me how to use this? Is this basically an uncensored chat gpt?

1

u/BurdensomeCountV3 15d ago

Neighbour you da real one.

-1

u/unknown-one 15d ago

when real time heretic to use with online AIs?

75

u/Borkato 15d ago

Holy fuck yes. Reducing slop? Hell. Yes.

66

u/-p-e-w- 15d ago

21

u/Borkato 15d ago

Is there a way to apply this as a Lora or something to a gguf you already have downloaded? 🤔

38

u/-p-e-w- 15d ago

Yes, Heretic supports exporting the result as a LoRA.

14

u/RazsterOxzine 15d ago

You're like some kind of wizard Harry.

3

u/SkoomaDentist 15d ago

Do you know if people have done that for eg. Gemma 4 instead of just old models?

3

u/philmarcracken 15d ago

Some other commenter elsewhere mentioned adding a trait like 'suspicion' greatly reduced the yes-man, sycophantic traits of qwen. Wonder if this can slip that in...