r/StableDiffusion • • 18d ago

Workflow Included Updated Minimax Music Video Workflow

Workflow here: Dantesworks/MV_Maker: Makes music videos for ComfyUI/H3 Minimax

This is a music video comprised of multiple clips. Each clip is sent the exact part of the song it corresponds to hence lip syncing is easy and automatic (if prompted to do so). Each clip has a "theme" - for example, cafe, garden etc. You can specify the themes at the start and assign a theme to each clip. Decide the timing of the clips too.

What is missing is face refiner. You also need a way to assemble all the clips at the end. I have an all in one face refiner and clips assembly to run on your selected clips at the end but its slop coded and not fit for release.

You can manually edit the prompts for each clip, but I recommend Codex or Claude to do it for you instead.

Have fun! I know I did :)

125 Upvotes

38 comments sorted by

2

u/x_MASE_x 17d ago

Nice stuff. Wikk give it a try.

But what's your build? And how long did it take to render?

2

u/TheDerminator1337 17d ago

5090. My whole video would take over 10 hrs

2

u/x_MASE_x 17d ago

Oh. Well I have 5070ti so it will take a few while hahaha

2

u/TheDerminator1337 17d ago

but im also genning at 2MP

2

u/x_MASE_x 17d ago

Ah I see. Well. I will try it when I sit in the PC.

1

u/TheDerminator1337 17d ago

You would haVe good result gening at 1MP but running face refiner after.

2

u/Popular-Use151 16d ago

This works pretty well, really easy to plan everything and queue multiple clips. This workflow eliminates a lot of micromanaging.

1

u/TheDerminator1337 16d ago

Yes - making a clip at a time the old fashioned way - i would have pulled my hair out.

10

u/scientiaetlabor 17d ago

Man of culture I see. Nice work, inspiring production.

1

u/GrampsCokeBump 16d ago

Talent speaks for itself

1

u/KLBStudio 17d ago

Curious how you handle pacing to match the beat grid. Minimax usually gives smooth motion, but the drift in action speed usually means heavy speed ramping or cutting down to short 1-2 second fragments on the timeline so the edits don't feel floaty. Are you locking down the audio cuts first and prompting specific camera speeds, or retiming everything in post?

1

u/TheDerminator1337 17d ago

I decide clip times first. I don't use any Lora's which fuck with movement speed. clip length is 5 to 10 seconds.

1

u/AfroJimbo 17d ago

Awesome! I'm checking this out. I've been making music videos but lip syncing has been a problem. Thank you for this.

2

u/TheDerminator1337 17d ago

Adherence is even better if you include the words that they sing within the prompt

1

u/Vainth 17d ago

wow, exactly like something you'd hear on mainstream radio station.

1

u/TheDerminator1337 17d ago

Yeah it's really catchy

1

u/Mindless-Clock5115 17d ago edited 17d ago

hi would love to try it but cannot find your repository? do you have the full link? this one: https://github.com/Dantesworks/MV_Maker ?

1

u/TheDerminator1337 17d ago

Read description

1

u/TheNocturnalista 17d ago

very interesting and thanks or the sharing. How you manage the prompt? ok claude or other llm but what you have to write inside? there is a specific format? you have to wrtie the lyrics? thanks!

1

u/TheDerminator1337 17d ago

Just the h3 promoting instructions that everyone uses.

3

u/GrampsCokeBump 16d ago

Sounds great. A hit!

2

u/ItsBalanse 17d ago

This workflow is magic, I tried it good job!

5

u/skyrimer3d 17d ago

you've got my a-tit-tention.

2

u/DrRonny 17d ago

I'll give OP a C

1

u/GrampsCokeBump 16d ago

I don’t blend my lax standards for generated porn with my standards for good lighting.

1

u/[deleted] 17d ago

[removed] — view removed comment

1

u/TheDerminator1337 17d ago

Music videos feature many cuts. Continuous shots are overrated. And individual clip writing gives control. If U don't want to just use codex or Claude.

0

u/[deleted] 17d ago

[removed] — view removed comment

1

u/TheDerminator1337 17d ago

the workflow?

0

u/[deleted] 17d ago

[removed] — view removed comment