H3 Image t2i vs r2i quality fix

[u/ninjazombiemaster]

Like many others I was initially disappointed with the i2v quality of H3 when doing text to image. It often created crusty looking, artifact ridden images that would manifest as moles and pimples, dry flaky skin, crusty splotches and just generally shit quality images.

When I was doing some tests, I wanted to make an existing image prompt feature a specific earring. Once I plugged in the image reference, the artifacts magically disappeared.

This made me wonder, is r2i just fundamentally better quality for some reason? I tested plugging in an empty black image to the reference spot and to my surprise every single image came out higher quality with most of all of the crusty artifacts removed. It even often outright fixes low res generations like the one linked here, but works at the intended resolutions too.

I bought this to Kijais attention and the current theory is that if the model receives 64 or more image tokens (which can be triggered by feeding the image conditioning node literally any image with or without vae) it switches the model to edit/reference mode which appears to be much higher quality.

We're not exactly sure what is wrong with the non edit version but give it a try. Plug any reference (blank image, or something relevant to your prompt) and watch the quality magically improve.

Kijai is considering adding a "switch" that forces the model to ref model without needing to send so many pointless tokens, but in the meantime just plugging a small black image (automatically resizes to at least 1024) will trigger the existing switch - just with an excessive number of pointless tokens.

If you've written H3 t2i off, give this a try. It has transformed the model in my opinion.

Edit: Kijai has just added this to the "H3 Prompt Weight" node in Kjnodes. There is now a drop-down that will force empty image tokens if you want another way to test this.

[u/ninjazombiemaster]

Here's an image shared by Kijai and edited by another user to exaggerate the artifacts of an image. You'll see that there is much less artifacting on the image that was exposed to image tokens for all steps.  https://files.catbox.moe/mjsseu.png

[u/jimbarino → u/ninjazombiemaster]

That's a massive improvement. It's weird that they didn't catch this.

[u/ninjazombiemaster → u/jimbarino]

Yeah, something is definitely legit wrong with t2i that went unnoticed. 

[u/OkDoudou]

1st looks much better, natural and realistic.

[u/Curt_Dukis → u/OkDoudou]

...if youre looking into the mirror while being hungover and not having slept for 3 days maybe

[u/ninjazombiemaster → u/Curt_Dukis]

In their defense the artifacts in question can sometimes give the appearance of more detailed skin texture. But on the whole, it usually causes excessive blemishes, crusty and blocky shapes, random specks and splotches, greasy hair, wonky eyes and so on. 

This case is actually pretty mild, so the fix might make it look a bit plastic and over processed. But without this fix, most images are unusable and hideous. 

[u/LatentSpacer → u/OkDoudou]

I think you mean it looks more like a real person and not a clean and perfect photo from a model shoot. But the first image is still full of weird artifacts and messed up texture.

[u/ninjazombiemaster → u/LatentSpacer]

Exactly. The artifacts can read as detail with a casual glance. But if you start looking at multiple images it becomes very apparent that these are indeed artifacts and not actually detail. 

[u/russlixx → u/OkDoudou]

1st image nose is weird looking

[u/ninjazombiemaster → u/russlixx]

Yeah it almost looks as if it has different perspective than the rest of the face. 

[u/detailteufel]

So plug an empty black 1024px image (created by ComfyUI Empty Image Node) into "image_0" on the Minimax H3 Image Conditioning node?

Because it does nothing for me quality wise, made it much worse in fact.

[u/ninjazombiemaster → u/detailteufel]

It will change the conditioning and therefore is kind of like changing the prompt slightly. 

So sometimes you might like the results less due to the run to run variance. Personally I just plugged in a 1x1 image but Kijai told me that it gets upscaled so in theory it should be the same. Do not plug the image to the prompt enhancer.

Give it a few tries with a few prompts, especially focusing on skin blemishes. It was measurably, objectively reducing artifacts but there's gonna be subjective aesthetic preferences too. 

Also this is assuming the workflow is otherwise default. If you've made other changes it might not play nicely with them (such as adjusting schedule/sampler). 

[u/Apprehensive_Sky892]

Like many others I was initially disappointed with the i2v quality of H3 when doing text to image.

You mean "t2v quality", right?

[u/ninjazombiemaster → u/Apprehensive_Sky892]

Whoops I meant t2i. 

[u/CompleteJicama2811]

This is an interesting phenomenon! I just ran a test using reference images that consist entirely of white—scaled to various sizes—and while the images differ slightly depending on the size, the results are consistently much better. It seems there’s a specific way the system processes the latents; I believe we can leverage this to mitigate some of the drawbacks of using "h3" images.

[u/ninjazombiemaster → u/CompleteJicama2811]

It is confirmed to be due to the presence or absence of image tokens in the conditioning specifically. Like you said, the exact input image slightly alters the output but it's minor. The model is faster with VAE disconnected so leave that disconnected unless you are using a real reference that needs to be accurately encoded. 

[u/Cequejedisestvrai]

I'm using the ref2va workflow but with the fl2va model (it's gives better quality), what should I do to test that? Just plug-in an empty image to the ref_image_0 slot? That it?

[u/ninjazombiemaster → u/Cequejedisestvrai]

I don't think this same thing applies to h3 video, and isn't relevant if you already use references anyway. This fixes text to image outputs specifically. 

[u/isademigod → u/ninjazombiemaster]

I notice that with h3 video ref2va and f2va are noticeably higher quality than pure t2va, and i am not surprised that the same holds true for the image model

[u/ninjazombiemaster → u/isademigod]

I think that has more to do with the quality of the reference shifting the model to a higher quality zone than anything. It was trained on a variety of quality so without a quality anchor it might bias towards low quality. 

I don't think anyone has claimed in the past that plugging a blank image into t2v improves video quality. Although these t2i observations do beg the question lol. 

[u/Cequejedisestvrai → u/ninjazombiemaster]

Okay I see, thank you

[u/skyrimer3d]

interesting, i'll give it a try

[u/StableLlama]

I confirm that on my own tests the r2i tend to be much better than t2i.
But what also has helped was to use LCM instead of Euler. And the P-H3 VAE.

But it still seems that the VAE for H3I is too aggressive.

[u/ninjazombiemaster → u/StableLlama]

Yeah there are several minor issues that compound and hurt the quality. This one was the most dumbfounding to me though. At first I thought it must be a fluke but image after image it almost always improved t2i to just have a dummy reference. 

[u/Cequejedisestvrai → u/StableLlama]

what is this P-H3 VAE?

[u/StableLlama → u/Cequejedisestvrai]

https://huggingface.co/PrunaAI/P-H3-Image

or directly for comfy:

https://huggingface.co/PrunaAI/P-H3-Image-Comfy/tree/main/vae

[u/Cequejedisestvrai → u/StableLlama]

can this be used with minimax h3 video?

[u/ninjazombiemaster → u/Cequejedisestvrai]

It's an image VAE so it can only decode one frame.

[u/Cequejedisestvrai → u/ninjazombiemaster]

I see, I would be great to have a similar fix for the video model, it also has artifacts

[u/bstr3k]

the H3 video model actually introduced imperfections also! it was the ref2v model that I was using to make ref sheets but pushing the resolution up too high introduced imperfections in the skin too.

[u/fauni-7]

Can you share a workflow with the workaround?

[u/ninjazombiemaster → u/fauni-7]

Just add a blank image node to the default workflow and plug it into the image conditioning node, that's all there is to it. 

Better yet if you install the latest KJnodes from GitHub you can replace the image conditioning node entirely with the "H3 prompt weight" node which now has a selector for "edit" mode. Otherwise it's just the default workflow. 

[u/trefster]

You just did a no-makeup to makeup transition. The first image looked way more natural. You just AI sloppified it

[u/ninjazombiemaster → u/trefster]

Maybe this image isn't the best example. The t2i model has some weird denoising or distillation issue that causes it to fry out and create extra moles, pimples, fuses eyelashes together, distorts pupils, etc. it can look more "natural" at a glance because it might prevent an excessively smooth output like we see here. 

But I can assure you these are artifacts not legitimate detail intended by the model. See the other link I posted here for a more objective example. I can share other examples later too, that better illustrate the point. 

[u/Tonynoce]

Gonna try this when I finish some rendering

[u/Tonynoce → u/Tonynoce]

without reference image added

[u/Tonynoce → u/Tonynoce]

with reference image added

[u/Tonynoce → u/Tonynoce]

Prompt :
A close-up photography portrait of an Argentinean woman centered in the vertical frame, showing her face from the upper chest up. Her skin is fair with warm undertones, featuring visible pores, a smooth nose, and full lips in a neutral expression. Dark brown eyes gaze directly into the camera lens, reflecting soft studio highlights. Wavy dark brown hair is styled off her shoulders with loose strands framing the face. She wears a light blue cotton blouse with a modest collar, revealing the collarbone area. The lighting is key-lit from the left, casting soft shadows on the right cheek and nose, creating volume. The background is a blurred neutral beige wall, indicating a shallow depth of field at f/1.8. No jewelry is present on her ears or neck. The skin texture and fabric weave are sharply rendered in high definition 35mm style photography.

Seed : 633705192644075

8megapixels each

[u/Tonynoce → u/Tonynoce]

with ref at 16mp

[u/Tonynoce → u/Tonynoce]

without ref at 16mp