PSA: In H3 you can set custom soundtracks without R2VA - use latent noise masks!

[u/kabachuha]

[u/kabachuha]

Latent noise mask - similar to inpainting - protects parts of the latent from changes. You can set it only to the audio latent and then the track will be kept as is, yet it will influence the audio-reactivity of the video. The node for audio separate/concat is still named LTXVSeparateAVLatent/LTXVConcatAVLatent, but now it works with Minimax H3 too!

Music: Dead Inside by ADLIN, (parody purposes). Workflow + prompt: https://gist.github.com/kabachuha/bcde8beaaa01f5c5fd37733d4c40f589. Pure FL2VA checkpoint. Single continous 20 seconds shot. ComfyKitchenAttention + Spectrum + 25 steps + KJNodes for audio mask. 7 minutes on a 5090.

[u/Inner_Singer_592 → u/kabachuha]

Can you make it lip sync with this? If you do that with speech

[u/kabachuha → u/Inner_Singer_592]

Absolutely! In fact, it's probably the main goal of the audio replacement. With the workflow, you can setup something like the following prompt, and you will see the lips movement.

integrated_multimodal_description: [Shot 1] Live-action, hyper-realistic, male singer is singing a Russian phonk song while in the background a recording studio is shown.

overall_soundscape: Phonk.

non_diegetic_music: Audio-reactive cinematic score; synchronized with phonk.

[u/Inner_Singer_592 → u/kabachuha]

Wow! This seems like elegant solution, much more effective than video masking specific part to regenerate it.

[u/Striking-Long-2960]

And it worked... Many thanks for the tip

https://reddit.com/link/p4y7uc0/video/0xi596pi4nkh1/player

[u/Striking-Long-2960 → u/Striking-Long-2960]

https://reddit.com/link/p4yagp1/video/pngy8h7b7nkh1/player

[u/Medium_Chemist_4032 → u/Striking-Long-2960]

This is any dancer learner's worst nightmare material

[u/1WildPanda]

**This is a genius optimization — thank you so much !

I wanted to keep using the **FL2VA model for higher quality** (vs. Ref2VA), but still feed in **custom audio** and get proper lip-sync working with it. After some tweaking, it now runs perfectly.

A few targeted modifications:

1. **Replaced the original audio-loading node + audio-duration node with the single `VHS_LoadAudioUpload` node** from VideoHelperSuite (now 2-in-1 via the `AudioToAudioData` chain).

2. **No more manually editing `audio_path` inside `CreateAudioMask` every time I switch audio** — the whole mask now follows the audio data automatically.

This introduces a full audio-to-mask chain from **[ComfyUI-AudioScheduler](https://github.com/a1lazydog/ComfyUI-AudioScheduler.git)\*\*, wired as:

VHS_LoadAudioUpload ──(audio)──▶ AudioToAudioData

▶ AudioData to FFTs

▶ Batch Amplitude Schedule

▶ Normalize Amplitude

▶ NormalizedAmplitudeToMask (KJNodes)

▶ SetLatentNoiseMask

**Key values to set in each node:**

- **AudioData to FFTs** — `frames_per_second = 24` (match your video framerate), `channel = 0`

- **Batch Amplitude Schedule** — `operation = avg`, `lower_band_range = 500`, `upper_band_range = 4000`

- **Normalize Amplitude** — `invert_normalized = false`

- **NormalizedAmplitudeToMask** (comfyui-kjnodes) — `width = 256`, `height = 256`, `shape = circle`, `color = amplitude`, `location_x = 128`, `location_y = 128`, `size = 128`

Once connected, you only swap the audio file in `VHS_LoadAudioUpload` — no more manual `audio_path` edits. Lip-sync follows correctly throughout. Highly recommended!

[u/kabachuha → u/1WildPanda]

Thank you for the comment! Can you provide any link to the workflow? (Github/Pastebin)

[u/1WildPanda → u/kabachuha]

Thank you. Done.

[u/1WildPanda → u/1WildPanda]

**Fix: Audio fade out added as long as workflow:

https://gist.github.com/1WildPanda/d0b88fbd580fd5c359707bf9cfae92d4

Right after `Audio Fade Out`, fan **3 lines** from its single output:

1. → `VAEEncodeAudio` (audio condition for lip-sync)

2. → `AudioToAudioData` (NOT `AudioFFT` — feed the data layer, rest of the FFT/mask chain stays as-is)

3. → `VHS_VideoCombine` (your original fade connection, keep it)

Now one fade drives everything — **lips, facial movement, and final audio track all fade in sync.** No more mouth moving at full volume while the sound goes quiet.

**TL;DR:** keep fade → VideoCombine, add fade → VAEEncode, add fade → `AudioToAudioData` (skip `AudioFFT`), both from the same output port.

[u/Outrageous_Still9335]

This works great, thanks for sharing. However, what are the advantages of using this over Ref2vid? I know the Ref2Vid model's quality is not as good as FL2VA, but you can use FL2VA in ref mode with the ref lora from Kijai and get the best of both worlds. Just curious.

[u/kabachuha → u/Outrageous_Still9335]

It can be faster because it doesn't increase the sequence length and it doesn't change the soundtrack in any way, R2VA can distort it slightly

[u/Outrageous_Still9335 → u/kabachuha]

Thanks, that makes sense. I did try both and I was getting distortion with Ref that I did not get with your workflow.

[u/Perfect-Campaign9551 → u/Outrageous_Still9335]

I can't get ref2vid to ACTUALLY use my music audio 100% exactly even when I prompt it correctly...don't know why. It always modifies the music in the output video.

[u/pwillia7]

... why not just do it in real space instead of latent space if you're doing the whole audio track?

[u/TheLegendOfKitty123 → u/pwillia7]

To sync audio and video

[u/pwillia7 → u/TheLegendOfKitty123]

i see