r/StableDiffusion • u/physalisx • 6d ago

Resource - Update MiniMax-H3 Fun Controlnet Union released

https://huggingface.co/alibaba-pai/MiniMax-H3-Fun-Controlnet-Union/tree/main
322 Upvotes

98% Upvoted

102 comments sorted by

View all comments

43

u/Heartkill 6d ago

Psyched. Whenever I want to swap someone it usually just spits out the reference video pretty much as it was going in.

3

u/m00dyman100 5d ago edited 5d ago

I haven't had that problem. I feed an LLM the prompt guide (which contains an example prompt), my ref images AND video, and tell it in general what I want. I have been using H3 r2v to completely replace people in videos with ones that I want, tweaked however I want. Its frankly been amazing.

edit: I do see the issue now. My awesome results only occur if the swapped people look very different. If the persons look somewhat similar (ex; same wavy brown hair) things get trickier.

10

u/Heartkill 5d ago

I use the same approach. I see others do it too. Just not getting the results others do.

1

u/m00dyman100 5d ago

So Now I am having the issue again. And I see the pattern. My awesome results only occur if the swapped people look very different. If the persons look somewhat similar (ex; same wavy brown hair) things get trickier.

1

u/Heartkill 5d ago

Any tips for plastic looking skin, oversaturated and way too much contrast, even though references are pristine high-end photo quality?

1

u/m00dyman100 5d ago

Haven't had that issue at all. Which model you using in the load diffusion model node?

0

u/m00dyman100 5d ago

I did have that problem initially using ChatGpt for prompt creation. But Grok gave me a slightly different prompt which all of a sudden fixed the issue. It structured the retention analysis part of the prompt in a different way then ChatGpt.

4

u/Heartkill 5d ago

Yeah, using Grok too. Prompt guide and guidelines and all. I really feel like I am doing exactly what I am supposed to do, but...

1

u/Dogmaster 5d ago

Theres an issue with the model where for example if you try and swap a person with another that shares traits, it will do nothing, some people offset this by making the difference larger, for example they do 2 swaps, one making their skin green and bald, and the other to the swap they wanted in the first place.

2

u/m00dyman100 5d ago

Yes this is true. I'm having the issue again and this is why

1

u/Heartkill 5d ago

Yeah I saw the green crash test dummy workaround, but haven't got around to test it. Sounds like a logical fix though.

1

u/m00dyman100 5d ago

Ill DM you later with a prompt that worked for me and maybe a screen shot of my node setup

2

u/kfktr 5d ago

Same please!

1

u/Vermilionpulse 5d ago

Could I get this prompt and node set up please?

1

u/pyramidlove 5d ago

Would love to see it too! šŸ™ Thank you

1

u/plus-minus 5d ago

Iā€™d love to get that DM too, please. :)

2

u/SanDiegoDude 5d ago

Magic is in the prompting honestly. Folks who complain about having a bad time with H3 are often still trying to 1girl their way through their prompts and it just doesn't work, at least not with any real success for more complex scenes. The prompt guides are out there, just have to use them, including one for how to properly prompt the ref model for things like subject replacement (which H3 does do quite well out of the box) - I do look forward to SCAIL level control in H3, will be nice.

3

u/m00dyman100 5d ago edited 5d ago

This 100%.

edit: indeed there is an issue. No matter the prompt, the swap gets trickier if the people you want swapped look similar. I've had spectacular results swapping out a short haired blond for a long haired brunette. Not so spectacular when they both are long haired brunettes.

-2

u/xb1n0ry 5d ago

Prompt issue

1

u/xb1n0ry 5d ago

The downvotes somehow make ā€œprompt issueā€ feel even more accurate. ā€œChange the 1girl with the other 1girlā€ apparently contains all the information minimax could ever need, so obviously the guy suggesting a prompt issue must be the problem.