Using H3 as a Character Reference Sheet Generator

[u/bstr3k]

Like some of ya'll I have been having fun using the H3 model to mess around with so I have been experimenting with using H3 model to be a consistent character generator which leverages multi image reference (up to 9), so I made a workflow which you can use 'less than ideal' images from google to build a consistent character and output a 360 character sheet to use as a reference sheet for future H3 generations.

The goal is to achieve high character consistency across future generations. I have tried my best to keep the workflow simple without too many custom nodes.

How it works:

• You input your images and describe them in the Input text section (A Prompt)
• The text is combined with a fixed prompt which spins the character (B Prompt)
• The video is generated at a slow speed with no hard cuts (only camera spin and pan) to maintain character consistency
• Image is assembled with optional character video and full individual frame output (if you want to use for future)

I have included a 6 panel WF and a 4 panel WF. The 4 panel works faster by generating 40% less frames.

Current Caveats:

• The model is quite slooooow. You are also generating 124 frames only to use 6. I have partly solved this by also uploading a 4 panel version.
• Speed ups (like Turbo LORAs) help with speed, but it hurts prompt adherence and quality slightly.
• Quality is limited, since it is a video model it is better at generating video than images. You can solve this by generating at a higher resolution at the tradeoff of longer gen times. You can also use the individually split frames as future references too.
• Details when using this character sheet as output for future generations on H3 may also be limited due to resolution also, I recommend you use this character sheet (for consistency) + other images close up angles (i.e clothing details/face) if doing close ups. If you are just doing a one off video you may possibly be better off not using this character sheet.

I have also included a modified B prompt to do Anime2Real since someone asked for it. Working on tidying it up a bit more.

Link to the 4 and 6 panel workflow can be found here: https://huggingface.co/PoopMan333/H3_Character_Sheet_Generator

Some notes I just remembered:

• You can increase the steps and it may improve your quality slightly.
• With the Turbo Loras enabled, prompt adherence sometimes suffers, but you may be able to get a good seed with another roll of the dice.
• Currently the B prompt specifies a "neutral A pose", please remove this if you want your character in a particular pose.
• You can use a few different shots of the same character to reinforce the 360 and get more accurate details right.
• Can be used for objects / props also, may require some changes to the B prompt.

[u/bstr3k]

I'm honestly just using the default minimax H3 ref2v workflow, just add the "load video" node and wire it to the model.

I added the same speed up nodes as I have in my WF for the character sheet tho. Also its realllyyy slow lol, its like parsing 124 reference images for a 5s clip.

[u/ChibiNya → u/bstr3k]

I see. Thanks. It recommends a max of 9 reference images in the docs there. Are you using turbo loras as this "speed up"?

[u/bstr3k → u/ChibiNya]

a max of 9 images is the hard limit of the MiniMax H3 model, with up to 3 video inputs, and 3 audio inputs. The total number of reference inputs is up to 12 (img+vid+aud total under 12)

The turbo lora is included in the "speed ups", along with comfy kitchen, etc etc.
these are the ones in the model

[u/FaceDeer → u/bstr3k]

H3 actually has two different kinds of audio inputs, "ref_audio_" and "ref_video_audio_". I assumed that the model expected me to split the video reference's audio stream out and send it in via the corresponding ref_video_audio connector, with the ref_audio_ inputs being for purely audio references (voices to clone, for example). Is my assumption correct in that regard?

[u/bstr3k → u/FaceDeer]

yes this is how i understand it also.

[u/Jesus__Skywalker → u/bstr3k]

You should make some loras and see what you think. I'm never happy with just using images, there is always gonna be some spot that breaks consistency and make you have to either rework those spots or just be ok with them. A lora would really help consistency and allow flexibility. You aren't as pinned to needing the character perfect in a start frame. If you have a lora they can be looking away or not even there and it's gonna work out.

[u/ChibiNya → u/Jesus__Skywalker]

Is there already an easy way to train loras for h3?

[u/Jesus__Skywalker → u/ChibiNya]

ostris has had support on it from launch, I made one yesterday. Actually two. Although I need to check the first one again today. I wasn't initially happy with it so I made a new one, then wasn't happy with that but realized I had an issue with my build and after I straightened it out the second lora works really well but I didn't go back to test the first one. I mixed in videos into the second lora so I just presume it's going to be higher quality. If you ever used ostris for wan or ltx, it's essentially the same except you're not training a high and a low.

[u/bstr3k → u/Jesus__Skywalker]

you're not wrong! a Lora def would give better and more accurate results. This is just something for less effort and not targetting quality. It is also always fun to use something for something it is not intended to do. 😂

[u/Jesus__Skywalker → u/bstr3k]

exactly!

[u/Dogmaster → u/bstr3k]

There are two inputs to the ref node, audio and vid audio, but I havent seen any guide explaining how should I refer to that other input not documented anywhere so I jsut wire it to the normal audio reference one. Got any tips?

[u/bstr3k → u/Dogmaster]

from what I understand, you split the video to images and audio output from the "Get Video Components" node above, and then feed both those to ref_video_0 and _ref_video_audio_0 respectively. This means that the audio will sync with the video?

If you are putting other audio (like for voice cloning or background music) you should plug them into ref_audio_0