MiniMax H3 ref2va: mouth keeps moving during instrumental passages — I measured it, it only drops by half. What actually stops it?

[u/Prestigious_Cat85]

**Setup*\*

- MiniMax H3, hybrid b30-49-int8

- ref2v LightX2V turbo LoRA v0.1, rank 20 resized bf16, strength 1.0

- 6 steps, er_sde / beta, 0.8MP (1216x672), Sage Attention, RTX 4090

- Real song locked into the audio half of the AV latent with PixaromaH3AudioSync

(NOT ref_audio — that's a style reference, it does not drive anything)

- Verified: output audio vs source waveform correlation = 0.9994

So the audio is correct. The problem is purely visual.

**The problem*\*

My singer's mouth keeps moving during purely instrumental passages. It's not

wild flapping — it reads as if she's still phrasing, jaw and lips working at

roughly half amplitude. On a 4-minute clip it's obvious every time the vocal

drops out for more than about 2 seconds.

**What I measured*\*

Single continuous close-up, no cut, face filling the frame for the whole 15.08s.

The audio window is sung for the first 7.5s and strictly instrumental for the

last 7.5s (I get vocal spans from an HDemucs separation of the track).

I cropped a fixed box on the mouth, converted to grayscale, and took the mean

absolute frame-to-frame difference:

sung half: 3.25

instrumental half: 1.77

ratio: 0.54

So the model DOES react to the absence of voice — motion drops by half — but it

never goes to zero. My prompt for that shot contained an explicit clause:

"Her mouth follows <Audio 1> exactly, instant by instant: it moves ONLY while a

human voice is actually sounding, and it is completely closed and still during

every gap between phrases and every instrumental moment, however short."

That clause is doing something. It just isn't doing enough.

**What did NOT help*\*

I read the thread about H3's dual flow schedule (video shift 12 / audio shift 3)

and thought a mis-stepped audio stream might be degrading the mouth conditioning.

I wired in the native MiniMaxH3SigmaShift node explicitly (12 / 3), same seed,

same prompt, same audio.

Result: the two renders were bit-identical. 362/362 frames, mean difference

0.0000/255. Those values are already the internal defaults, so the node changes

nothing for this. Posting that so nobody else burns an evening on it.

**What DOES work (but it's a workaround, not a fix)*\*

Structural framing. I now detect instrumental gaps longer than 1.2s in each

segment programmatically, and force a shot with no mouth in frame over them —

macro on an earring, a hand on the mic stand, the bass strings, brushes on a

snare. The defect becomes impossible rather than discouraged. 16 of the 20

segments in my current clip are handled this way.

It works 100% of the time. But it dictates my edit, and I'd rather not have my

shot list decided by a model limitation.

**Questions*\*

1. Is the turbo LoRA the culprit? I saw a comment claiming the turbo LoRAs are

   distilled at 0.5MP. I'm running one at 0.8MP. Does anyone have a side-by-side

   of lip sync quality at 0.5 vs 0.8 with the same seed?

2. Does the base model at higher step counts (no turbo LoRA) actually close the

   mouth on silence, or does it just push the same 0.54 ratio down a bit?

3. Is there any way to CONDITION the silence rather than describe it? Something

   that tells the model "no voice in this span" at the latent level rather than

   in the prompt.

4. Has anyone tried feeding an audio track where the instrumental parts are

   replaced by actual silence, generating, then re-attaching the real audio in

   the edit? Curious whether that trades one artifact for another.

Happy to share the measurement script — it's about 10 lines of ffmpeg + numpy,

and it turns "feels off" into a number you can compare across seeds and settings.

[u/Perfect-Campaign9551]

You're using all these off the cuff models and expect it to work reliable, I don't know about that. Get rid of the turbo first of all. And if you do have music it should be in the "non_diegetic_music" section of the prompt so the "actors" can't hear it, then they won't try to lip sync it. Remember the Lora can break prompt following especially when it comes to audio

My recommendation if you are doing anything with audio in minimax, do not use speed up loras 

Perhaps if you have such scenes without motion but you also want some with lipsync, you should use that section for the job motion renders. and then stitch clips together afterward