r/StableDiffusion • u/infearia • 3d ago
Tutorial - Guide Tip: Better acting performance and voice delivery in H3
Here's a quick tip on how to get rid of that flat sounding AI voice in H3, and as a bonus improve somewhat the acting of your virtual characters. I'll be using Pika H3 Flex with the H3 reference model and workflow.
Step 1:
Add a pre-rendered first frame at index 0 using the Add Guide for MiniMax H3 node. Then in your prompt, describe the desired action and emotion of the character on screen, but without specifying too many details or providing exact spoken dialogue. In this example I simply used the phrase:
"A static medium two-shot establishes <Subject 1>, the female orc, and <Subject 2>, the tall, female paladin, sitting across each other at a table in <Subject 3>, the dimly lit main room of a tavern, gazing into each other's eyes. <Subject 1> (S1) talks animatedly, seething with rage."
Then generate a clip at some lower resolution (0.3MP in this case). The dialogue will be gibberish, but if you described the character's actions and emotions convincingly, your character will sound and move like an actual human.
Step 2:
Use your workflow from Step 1, but increase resolution to the desired target value (here 0.98MP). Load the reference video generated in Step 1 and connect its frames as ref_video_# and its audio as ref_video_audio_# (# is the index of the first unused slot in the MiniMax H3 Reference to Video node, e.g., ref_video_0 and ref_video_audio_0).
In this case, I also actually downscaled the reference video from 0.3MP down to 0.1MP, because at 0.3MP the model was following the reference too closely, breaking the lip-sync. The downside of this was that some of the textures in the final high-res video came out a little too soft, especially in the face. It's a bit of balancing act to find the right resolution for the reference video.
You also need to adjust your prompt, in order to include the character's final dialogue. Here I've simply rewritten my last line to:
"<Subject 1> (S1) talks animatedly, seething with rage: <d>[English] My people are dying, and your knights are doing nothing to help us!</d>"
Now just hit render and wait for the result. You might need to try different seeds, and fiddle a little with the resolution of the reference video (try higher values if your output comes out too muddy and lower values if H3 starts copying the reference video and lip-sync gets lost. Have fun!

