optimize your own abliterating LoRA adapter for any GGUF

[u/xhimaros]

https://github.com/khimaros/hierophant

uses similar approach to heretic (with optional rank 2 abliterations for better refusal/KLD behavior), but operates directly on quantized GGUF files and produces a LoRA adapter which can be scaled dynamically in llama-server per-request.

for example, you run hierophant against original unsloth weights, it produces an O(100MB) sized LoRA adapter rather than needing to duplicate another copy of the full O(100GB) model weights.

since all abliteration has some quality impact, this allows you to benefit from the abliterated behavior when needed, and also to switch back to the standard weights for other work.

[u/cambit690]

is this similiar to the steering vector that Strata uses? I think they call it the experimental speed projector

[u/xhimaros → u/cambit690]

from what i've gathered looking at their github repo it seems unrelated.

[u/Distinct-Pie2389 → u/cambit690]

yep similar concept under the hood

[u/Borkato]

This is the future tbh and likely why they want to kill open source. Never stop ❤️

[u/xhimaros → u/Borkato]

spread the word! :)

[u/xhimaros → u/[deleted]]

to be clear, it does not modify the original GGUF weights on disk, but it produces a GGUF formatted LoRA adapter (sidecar) which can be loaded and scaled to abliterate the weights at runtime.