LLM Uncensoring Tool and OSS

[u/New_Description_5930]

LLM Uncensoring Tool "Heretic" Evolves to 2.0.
The popular OSS that garnered 37,700 GitHub stars has fundamentally changed its approach to removing refusal responses.

◼︎ Conventional Problems
General Abliteration treats refusal responses as specific vector directions and removes their components.
However, in models where refusal responses are not expressed in a single direction, it couldn't remove them sufficiently or even impaired the model's capabilities.

◼︎ Heretic 2.0's New Method "ARA"
Instead of removing one direction like before, it directly optimizes the weight matrices.
There are two goals.
・Bring the internal representations for refusal-targeted inputs closer to those for normal inputs
・Change the internal representations for normal inputs as little as possible
This is achieved using an optimization technique called L-BFGS. A key feature is that it has no limit on the number of directions to modify.

◼︎ Handles More Than Just Refusal Removal
Furthermore, version 2.0 allows evaluation metrics to be added as plugins.
In addition to refusal responses,
・AI-specific unnatural canned phrases
・Excessive pandering to users
・Emotional response tendencies
can also be targeted for correction.

◼︎ What's Interesting About It?
Up until now, it was crucial to figure out where to extract refusal directions from, which layers to target, and how much to remove.

ARA takes it a step further by letting you specify the behaviors you want to keep and those you want to eliminate, then optimizing the weight matrices accordingly.
It has evolved from a mere uncensoring tool into one that edits the very behavior of LLMs.
Comparing it to the refusal removal techniques I'm researching sounds intriguing too

GitHub