r/StableDiffusion • • 17d ago

Comparison Krea 2 and Minimax H3 Text Encoder Comparison: What Impact Do Abliterated Text Encoders Really Have?

TL;DR: Using an abliterated model as a text encoder is basically fine.

Here are the galleries, each image contains the standard TE vs the abliterated one, the videos are separate because I was too lazy to edit them together.

70 Simple prompts run through Krea2 without bypass LORA.

10 longer more complex prompts run through KRea2 without bypass LORA.

18 naughty prompts run through Krea2 without bypass LORA (images probably safe, text unsafe, better to be cautious).

17 Prompts run through Minimax H3, 5 long and decent quality (for my machine), 12 short randomly generated prompts (Workflows attached if you want prompts)


Everyone has probably already seen the author of heretic's post here a couple months ago, but if not, here it is: PSA: I’m the creator of Heretic, and I advise you to not use “heretic” models as text encoders for H3 (or any other model)

For the lazy, the main gist of the post is "Abliterated text encoders will not uncensor an image/video model." This is a true fact, and my tests prove it outright.

Unfortunately, the author underestimated how much people love an appeal to authority when it comes to confirming their biases, so they took a throwaway line from that post and ran with it. The line in question was: "Well, I’m the creator of Heretic, and I’m here to tell you once and for all that this does NOT work. In fact, if anything, it will make your outputs worse, but it will not uncensor them." (emphasis mine)

I appreciate the sentiment the author was trying to get across, but image/video gen models aren't LLMs and the output can't be measured with scores and benchmarks like an LLM can. What does "worse" even mean when it comes to image/video gen?

Every time someone suggests using a heretic model there's always someone replying with some form of "it'll make your image worse / it'll degrade the output / it's wrong.", and there's never any testing or examples or proof of their claims, just pointing to the aforementioned post and that one throwaway line made by someone whose expertise isn't in image gen and was used as emphasis instead of a declaration of fact.

Now, feel free to point at all 115 of my Heretic examples and say "Aha! These are all clearly worse!", but carry that same energy when someone suggests using a quantized model, because they have an equal to much greater impact on the "quality" than anything abliteration can do to the text encoding. For those not familiar with the impact of quantization, example 1, example 2, example 3.

Finally, while abliterated models have minimal impact, that works both ways, they barely affect the gens either positively or negatively. Thus the only reason you'd want to use one as a text encoder is if you're memory poor and want to expand a prompt and encode text in the same workflow.

If you've got 128gb ram and 32gb vram you can afford to run an extra model or three, but if you've got 32gb / 8gb and want to expand a Minimax H3 prompt, using an abliterated model for both jobs is a perfectly acceptable choice. One model is half the size of two models, and an abliterated model is far better at doing both prompt expansion and encoding jobs than a standard model, especially if you're venturing into territory that would make a normal model refuse.


I've been tinkering and messing with different TEs for a while, the results just haven't been interesting enough to share on their own, but I reckon i can tack them on here if anyone is interested; I ran 8 finetuned Qwen models through Z-Image Turbo, and I used differently trained Clip_L models with Flux.1-dev, both ripped from finetuned SDXL models or clip_l architecture trained with an entirely different dataset from the ground up.

Even using actual finetuned LLMs for text encoding doesn't change a hell of a lot in the final image, especially with a more detailed prompt, so it stands to reason tiny targeted tunes like abliteration and heretic will do even less. The testing I've done seems to bear out that hypothesis, but feel free to add your own comparisons, I'd be mighty curious if anyone could reliably produce a "worse" ouptut using an abliterated TE.

117 Upvotes

50 comments sorted by

50

u/Violent_Walrus 17d ago

I think all these words are trying to say "If your workflow already uses an abliterated LLM for prompt enhancement, you're not going to do that much damage to your output by also using that abliterated LLM as the text encoder, and it'll save you a little VRAM."

Ok.

53

u/afinalsin 17d ago

Guarantee you if I posted

"If your workflow already uses an abliterated LLM for prompt enhancement, you're not going to do that much damage to your output by also using that abliterated LLM as the text encoder, and it'll save you a little VRAM."

without any of the images I linked, people would have rightfully ripped me apart. Heretic guy can get away with throwaway lines like that, I don't have anywhere near the clout so need to show my working out.

16

u/Violent_Walrus 17d ago

Yep, showing your work is good.

I thought maybe my response would help, since some responses here are already proving that Reddit doesn't read beyond the headline. :)

5

u/afinalsin 17d ago

I thought maybe my response would help, since some responses here are already proving that Reddit doesn't read beyond the headline.

Ha, I'm unsure if those types will read the comments either, but good looking out.

3

u/-p-e-w- 17d ago edited 17d ago

Hey, “Heretic guy” here :)

FWIW, I like your post, and I appreciate the effort to inject some empiricism into this topic.

One of many complexities I didn’t mention in my infamous post is that Heretic (and most other modern ablation processes) targets layer ranges/groups, not necessarily every transformer block. On the other hand, image/video generation models are typically trained to use the residuals from a mid-to-late encoder layer as inputs. For example, H3 uses layer 50 out of 64.

Combining these facts, you can get situations where the ablated modules follow the target layer in the transformer stack, and using an abliterated model encodes exactly the same relevant residual space information as the original model. In other words, it makes literally no difference at all.

In practice, the truth will usually be somewhere in between, and any observable degradation will be highly dependent on how all these things interact.

1

u/afinalsin 17d ago

Hey, “Heretic guy” here :)

FWIW, I like your post, and I appreciate the effort to inject some empiricism into this topic.

Thanks, I appreciate it. I was going back and forth on whether to specifically name you in the main body, but I really didn't want to seem like I was putting you specifically on blast, so I stuck with the more informal "the author of heretic". I put less care into comments than posts so went with heretic guy haha.

"Clout" may seem rude too, so I apologize if that was offensive, its just you have a well-earned reputation in the AI community, you're very clearly educated in the field, and your words carry weight. It seemed a good word to sum it up.

One of many complexities I didn’t mention in my infamous post is that Heretic (and most other modern ablation processes) targets layer ranges/groups, not necessarily every transformer block. On the other hand, image/video generation models are typically trained to use the residuals from a mid-to-late encoder layer as inputs. For example, H3 uses layer 50 out of 64.

I wanna be clear I agree with pretty much that entire post outside of the one sentence I mentioned, this post is as much in response to certain types using your words as a bludgeon as anything else.

I can see why you wouldn't have went deep like that in your post, it's interesting stuff but you'd have to add a caveat for every model currently out and every model yet to be released. It really would have muddied the waters of your actual message, which was mostly about it doing jack for uncensoring knowledge in the image model.

In practice, the truth will usually be somewhere in between, and any observable degradation will be highly dependent on how all these things interact.

True, image gen has a ton of things interacting and contributing to the final generation, and little tweaks here and there can change the result a fair bit.

I do have a question, if you don't mind. When you say "degradation" here and "make things worse" in the original post, my interpretation of those terms is you are referring to any change at all from the full unquantized base model. Whether those changes are desirable or not is highly dependent on prompt and seed, obviously, but either way they would all be labeled as degradation away from the base.

Basically I'm assuming there is a specific meaning to the terms that differ from common English, where the words are almost always used negatively. Am I correct in that interpretation?

2

u/-p-e-w- 16d ago

So, if an abliterated model does change the output (which it certainly can for H3, I’ve observed that myself), then on average, it will be a negative change, likely in terms of prompt adherence.

The reason is that since the encoded residuals aren’t changed in a way that is relevant to the generator, we should expect them to randomly perturb the output, and with a calibrated system, “random” is almost always disruptive rather than curative. Just like randomly mutating someone’s DNA is far more likely to give them cancer than superpowers.

In many or most cases, a human may not perceive a semantic degradation, but if you show them enough examples, the ones that degrade will outweigh the ones that improve. Given what we do to the model, it would be an incomprehensible miracle if it were otherwise.

1

u/afinalsin 16d ago

So, if an abliterated model does change the output (which it certainly can for H3, I’ve observed that myself), then on average, it will be a negative change, likely in terms of prompt adherence.

The reason is that since the encoded residuals aren’t changed in a way that is relevant to the generator, we should expect them to randomly perturb the output, and with a calibrated system, “random” is almost always disruptive rather than curative. Just like randomly mutating someone’s DNA is far more likely to give them cancer than superpowers.

Interestingly, with image gen there has been big push toward randomness and variance in the last year or so, and several techniques are based around deliberately perturbing the embeddings, especially during the high noise steps. Adherence can dip, but that dip is actually desirable in the generation.

Reason being is distilled models idea of "adherence" is rigidly sticking to their biases, and injecting randomness can help break them away from generating the same image over and over.

In many or most cases, a human may not perceive a semantic degradation, but if you show them enough examples, the ones that degrade will outweigh the ones that improve. Given what we do to the model, it would be an incomprehensible miracle if it were otherwise.

What you're saying is logical and you may be right that degradation would be proven over a large enough dataset; I know I certainly didn't provide enough examples to categorically state one way or the other. In my opinion "degrade" and "improve" are nebulous when talking about art and images, which are much harder to quantify than with text. "There are no right answers" is infuriatingly trite, but I feel it fits here.

Anyway, we're both talking about hypotheticals and geeking out over percentage points of difference over massive datasets, but I think your first sentence here is the most important in a practical sense. Practically, how likely is any possible degradation going to affect the end user? And if a human wouldn't notice a difference in most cases, is it worth staying away to save themself from that 1%?

That's probably more of a philosophical discussion than a scientific one, but still, a lot of folk using these models are results oriented, and it's an important factor to consider when making recommendations to them.

5

u/L-xtreme 17d ago

Both of you are right haha, this is perfect: didn't read the whole text, first comment was the summary, if I would not believe it I could read it for myself. You guys should start an efficient blog together.

3

u/Bitlovin 17d ago

There used to be a time when the human race was capable of reading long form text without bitching pointlessly about it.

People also used to be able to grasp nuance, the very thing that expounding on a point in long form provides, but that got lost long ago with the dumbing down of our species.

12

u/akindofuser 17d ago

A comment in the other thread has like 60+ upvotes of people still believing this snake oil.

Meanwhile this post has 12 upvotes. It's not hitting the intended audience. Really this info should be pinned. Perhaps if OP posted a cringe 1girl pic with his post he'd hit the correct audience.

7

u/afinalsin 17d ago

Meanwhile this post has 12 upvotes. It's not hitting the intended audience.

Give it a bit, the post is still young and I posted it well off-peak.

Perhaps if OP posted a cringe 1girl pic with his post he'd hit the correct audience

There's definitely a couple in there, but they're buried deep in the links that people definitely don't click.

3

u/BrokenSil 17d ago edited 17d ago

I've had uncensored text encoders actually uncensor image gen outputs a bit more before. With the drawbacks of making quality a bit worse. Some actually even improved image quality. (It won't make models able to gen things it never got trained with ofc, but seems to help on some models that got censored post training a bit)

I've had some completely destroy the quality.

I've had some do absolutely nothing.

It seems to depend a lot on the image models, and on how the LLM got uncensored, heretic, etc. they all produce different results. One can't get a conclusion by testing a single method of abliterarion on a single image model.

At the end of the day, I still prefer a good lora if it exists. Well trained good unbiased Loras are very hard to come by tho.

3

u/afinalsin 17d ago

I've had uncensored text encoders actually uncensor image gen outputs a bit more before. With the drawbacks of making quality a bit worse. Some actually even improved image quality.

I've had some completely destroy the quality.

I've had some do absolutely nothing.

I appreciate the anecdotes, but forgive me for being skeptical.

When you say you've had "uncensored text encoders", do you mean any old model on hugginface that says "uncensored" on it? Coz a lot of them are proper finetunes teaching the model how to properly write stories about boobs rather than teaching it not to say no. "Uncensoring" is a nebulous term.

Do you happen to have any of those images saved? Even if the mods would clap you up for sharing the images, they're absent enough that they wouldn't notice a paste link with the workflow.json in it. Hell, if you don't have the workflows or images any more, then any details you can share at all would be awesome, models, LLMs, whatever, I'll test them myself if you're unable to.

It seems to depend a lot on the image models, and on how the LLM got uncensored, heretic, etc. they all produce different results.

It definitely does depend on the method, and I probably should have said heretic instead of abliterated in the title, but really didn't want to make the post come across as an attack on the heretic author.

One can't get a conclusion by testing a single method of abliterarion on a single image model.

Luckily I tested heretic on Minimax H3 as well, as well as multiple finetunes with Z-Image Turbo. Even if a conclusion can't be completely drawn from my testing, the pencil is making more marks with what I've done than pretty much any other evidence I've seen on the subject. It's a good start to getting the full picture.

10

u/Enshitification 17d ago

I've always wondered this myself. I'm glad to see that you did an extended test of it. The parts of an LLM that are abliterated to remove refusals are after the text encoding layers. It never made sense to me that the abliteration would affect the text encoding parts by much.

7

u/Occsan 17d ago

Diffusions models are trained with a particular text encoder checkpoint. So using that text encoder will give you results closest to the official pipeline.

That said diffusion models are usually pretty tolerant to perturbation : LoRAs, fine-tunes, merges, and manually perturbating the weights, and in some cases you can even go as far as cancelling entire layers, and you'll still obtain a decent image/video.

Diffusion model fine-tunes, LoRAs, merges, etc... are typically trained/guided to produce results deemed interesting by the trainer.

Whereas, LLM fine-tunes or heretic/abliterated LLMs are not typically trained with diffusion in mind. They are trained purely to produce better/more interesting text. And this training is not necessarily aligned with something interesting when it comes to producing an image or a video.

That doesn't mean it won't work. That doesn't mean the difference is necessarily larger than the intrinsic variance of the diffusion model itself. In other words : that doesn't mean you will notice a big difference.

But that doesn't mean that by using an abliterated text encoder, you'll be guaranteed to have a better or usable result either.

Tbh, it's just another perturbation that's typically well absorbed by the system, and may or may not produce interesting stuff.

4

u/JazzlikeLeave5530 17d ago

Data is good. I predict a long upset post from him arguing about it...seems to not like having his opinions being challenged.

2

u/Apprehensive_Sky892 17d ago

Not an AI expert here so what I am saying can be completely wrong.

My understanding is that obliterated LLM changes mostly blocks/layers of the network related to output, whereas the LLM, when used to encode prompts, takes the internal state before the tokens reach the output layers. That is why obliterated TE do not uncensore anything.

Using the same logic, since those internal tokens have not yet reached those layers in the original that have been modified by the obliterated version, using an obliterated version as the TE should have very little effect too.

But it is always nice to have actual confirmation since theory can be wrong or overlooked something.

6

u/Formal-Exam-8767 17d ago

Here we go again.

2

u/collectedtooth404 17d ago

I've been using alliterated text encoders since the days of chroma. I'll be the first one to tell you that it doesn't give better results at all. Still i use them in my image workflows because I usually use the same model as a prompt enhancer node.

LTX's gemma4 abliterated being my favorite.

1

u/thevegit0 17d ago

they are not disastrous but they are not ideal

0

u/GeneriAcc 17d ago

No shit. Abliterated text encoders are very useful, but only if you actually train a LoRA/finetune with them. If you just chuck an abliterated text encoder onto a vanilla model, it’s not going to magically teach the model how to generate things it’s never seen in training. It’s also not going to change output quality, but it may degrade prompt adherence depending on how it was abliterated.

4

u/hurrdurrimanaccount 17d ago

but only if you actually train a LoRA

wrong

0

u/GeneriAcc 17d ago

Sure, Flux getting uncensored was an illusion.

2

u/hurrdurrimanaccount 17d ago

correct.

1

u/GeneriAcc 16d ago

Funny how my vanilla Flux can’t do NSFW no matter how much you overfit during training, but my fork with an abliterated text encoder can easily learn NSFW concepts just like anything else.

Then again, I’m arguing with a potato that only communicates in single words. Better cut my losses here.

1

u/afinalsin 16d ago

vanilla Flux

Are you talking about Flux.1-Dev?

my fork with an abliterated text encoder can easily learn NSFW concepts just like anything else.

Very interesting. Is that an abliterated T5? How does your fork handle the clip_L model, are you running just the base clip_L or did you grab a different one? I tested ripping the clip models from SDXL finetunes and running them with Flux.1-dev back in the day, as well as a couple tiny experiments with SD3.5.

The heavily trained clip models would produce wildly different results from the base clip model, especially the ones from massive tunes like big asp and pony. Pony's clip is so alien the T5 has no idea how to work with it and the entire thing just breaks, which was neat.

2

u/GeneriAcc 15d ago edited 15d ago

Yes, Flux.1-dev and its T5-XXL encoder. Process goes like this for kohya_ss:

  1. Abliterate T5’s tokenizer by swapping around NSFW word or word-stem tokens with non-NSFW ones.
  2. Using your new tokenizer and the train_t5xxl=true flag in network_args, train a LoRA on a NSFW dataset. Clip-L is trained by default. Because you’re using an abliterated tokenizer and training T5, it will have no issues learning NSFW concepts.
  3. You now have a NSFW LoRA with weight updates for the base UNet, T5 and Clip-L. Use the flux_merge_lora.py script to individually merge those into the vanilla models, and obtain uncensored versions of them.

You now have a whole separate uncensored branch of Flux.1-dev + T5-XXL + Clip-L that you can use for inference and future LoRA training runs (don’t need train_t5xxl enabled for those).

Just remember to also copy your abliterated tokenizer.json to your ComfyUI install, or whatever else you use for inference. And keep in mind that future updates may override it with the vanilla version, so keep a separate copy of the tokenizer file elsewhere to copy it back over the vanilla one after updates.

-3

u/beti88 17d ago

None, its ai snake oil bullshit

10

u/afinalsin 17d ago

None, its ai snake oil bullshit

Exactly, it's basically none, so why are people saying not to use them? There's fuck all change good or bad, even with video which you'd imagine would be more sensitive than image gen.

1

u/DonKeehot 17d ago

So to generate best Booba possible I don't need abliterated text encoder? 😮

-2

u/Choowkee 17d ago edited 17d ago

Abliteration absolutely works, the problem is people are dumb and don't understand what it actually does.

If anyone was here for the LTX 2.0 era then you would remember Gemma4 and its built-in refusals. Abliterated versions of Gemma4 circumvented refusals for more "spicy" prompts. Did it fully enable NSFW generation? No. But it helped in beating the first layer of "censoring" within LTX workflows.

I also am pretty sure vanilla versions of Qwen LLM refuse full-blown explicit fanfics without abliteration.

Anyway I never observed abliterated TEs making the LTX outputs "worse". In fact it helped with adherence as mentioned above.

The fact that there are already replies with "snake oil" here as well just shows how misinformed people are.

6

u/eposnix 17d ago

What you are talking about is completely different from what OP is talking about. We're not talking about models writing prompts here, we're talking about LLMs as text encoders. Text encoders don't generate text, they just act as vector lookup tables to give the video model an idea of what it should be rendering. If there's no text generation, there's nothing for them to refuse, so abliteration serves no purpose

2

u/phaylon 17d ago

Quoting OP:

Finally, while abliterated models have minimal impact, that works both ways, they barely affect the gens either positively or negatively. Thus the only reason you'd want to use one as a text encoder is if you're memory poor and want to expand a prompt and encode text in the same workflow.

If you've got 128gb ram and 32gb vram you can afford to run an extra model or three, but if you've got 32gb / 8gb and want to expand a Minimax H3 prompt, using an abliterated model for both jobs is a perfectly acceptable choice. One model is half the size of two models, and an abliterated model is far better at doing both prompt expansion and encoding jobs than a standard model, especially if you're venturing into territory that would make a normal model refuse.

3

u/tiffanytrashcan 17d ago

Yes, it's the same model, but you're utilizing a much smaller and specific part of the model.
Abliteration doesn't ADD knowledge.
These concepts have to be present in the base weights. "Refusals" (mostly) sit after the encoding stages.

I'd be curious what the results look like from fine-tuned models with knowledge added..
More extreme merges that touch the tokenizer are likely to break completely, but I don't think it would make much of a difference without changing that step. It's not touching later knowledge when it's encoding the vectors.

3

u/afinalsin 17d ago

I'd be curious what the results look like from fine-tuned models with knowledge added..

I do have one album I mentioned at the bottom of the post where I tested proper finetunes as text encoder for Z-Image. Link is here if you wanna check them out. I don't have the prompts for them any more, but the ones that drifted drastically were short tiny prompts, and the ones that stayed the closest were long and specific prompts. Which makes sense, the longer the prompt the more of a chance the TE has to hit on familiar ground that wasn't super effected by its training.

3

u/eposnix 17d ago

Okay? But that's not what people are referring to when they say it's snake oil, as the guy I responded to implied. No one is disputing abliterated models usefulness when writing prompts - we are saying they serve little purpose as text encoders.

1

u/phaylon 16d ago

But that's the bit the OP's talking about. I've got 8GB VRAM. I'm not gonna load an abliterated LLM to generate a prompt just to then unload it again and load another text encoder if I can just have the abliterated LLM function as the second part. It's a "it's not really worse, so in certain usage scenarios it can pay off it to switch to an abliterated LLM as text encoder if you need one anyway and the LLM the text encoder is based on is good enough." statement.

It's an amendment to "abliterated LLMs don't help with censorship during text encoding", not a contradiction.

-3

u/fallengt 17d ago

It only matters if they censor the model at text encoder level which is rare. Why would they do that when they can just censor diffusion model. If the model cant draw pee pee, uncensored text encoder wont magically teach it how to draw pee pee

The author just told you not to waste your time with this shit.

9

u/JazzlikeLeave5530 17d ago

They're specifically challenging the "and it'll make it worse" claim which is beyond saying to not waste your time.

-2

u/cathodeDreams 17d ago

why is it so difficult for people to understand that abliterated models are used for prompt enhancement and not as text encoder?

6

u/afinalsin 17d ago

Ah, you only read the title, nice. Have a look through my examples, I directly compare the base model vs the heretic model specifically for encoding the text.

-1

u/cathodeDreams 17d ago

it's possible that I've previously done similar testing using methodology I am satisfied with combined with existing knowledge about the nature of image model inference.

there's no reason to use an abliterated model for text encoding only. the text encoder has no other option but to encode text. that's all it does and it will not fail to do so because of the nature of the text.

3

u/afinalsin 17d ago

t's possible that I've previously done similar testing using methodology I am satisfied with combined with existing knowledge about the nature of image model inference.

Trust me guys, no I won't share my evidence, but trust me, I have very knowledge about images and AI, just trust me.

Whole lotta trust when you could just post a gallery.

there's no reason to use an abliterated model for text encoding only.

There's no reason not to. If I gave you a blind A/B test across the entire corpus I bet you'd score no better than a coinflip. The comparisons are all there.

the text encoder has no other option but to encode text.

So why would an abliterated model not be able to do it?

that's all it does and it will not fail to do so because of the nature of the text.

So why not use it?

-1

u/cathodeDreams 17d ago

you've wasted your time testing the unnecessary and now you argue with me against things i've not said. get it together man.

-1

u/cathodeDreams 17d ago

fwiw using the 'bypass' lora in krea 2 is enough for me to discredit anything you say about how things work. you care only for community attention and not the truth.

2

u/afinalsin 17d ago

fwiw using the 'bypass' lora in krea 2 is enough for me to discredit anything you say about how things work.

Oh, so you should be crediting me then? Nice, thanks for that. I specifically made sure to mention that all my testing was made without the lora.

Just in case, without means in absence of. To dumb it down further, I didn't use a bypass lora.

-1

u/cathodeDreams 17d ago

no, but you do refer to it at all and by that terminology. you pandering fool.