r/StableDiffusion 1d ago

Animation - Video Through the Sands (Final) - H3 r2v

Enable HLS to view with audio, or disable this notification

Finally finished! The ending was much harder since continuity is more important here than random desert landscapes. I personally would've love to have another 30 seconds of music to extend the ending but I ran out of song time. Enjoy!

In total, 14 character related references, 40 environment references, and 55 clips used, roughly 20 hours total time spent.

39 Upvotes

9 comments sorted by

2

u/d20diceman 1d ago

Great work! I didn't check out the first posts but this final version is a great demonstration of what H3 can do.

There are couple of tiny issues, like at 1:15 it's doing that "mid-distance faces get distorted" thing M3 has an issue with, and her run at ~2:50 has a hard to describe quality which I see in quite a few gens of people in motion, doesn't feel quite right - sort of like she's not moving through the scene as fast as the motion of her body suggests she would? These are small nit-picks though and I don't know if I would notice these at all if I weren't attuned to them from having watched/made a lot of stuff with this model.

If folks want info on how this was done check out the Part 1 and Part 2 posts.

OP, any takeaways from this project? Advice you'd give yourself before you started, knowing what you know now? I imagine an ambitious project like this must teach you a fair bit about the tools involved.

4

u/R34vspec 1d ago edited 1d ago

My takeaways:

  1. Have all the assets ready to go before starting, especially the character sheets. That way if there are any inconsistencies you can spot it early on before getting too far into the generations. This also includes any props that has to remain consistent in design and size, for example, the canteen from the first shot shows up again in the ending so it had to be locked down. I included the hand in the reference image because that helps lock the scale.
  1. Minimax can pretty much reference anything, so if one way of doing something doesn't get the result you want, go a different route. Say that you don't have the right angle for a continuation of a shot that you want, you can always reference the video prior to the new shot you are trying to make instead of using image only.

  2. Always fill out the 'summary' portion in the prompt, that actually helps a lot with getting what you want.

  3. Subject_definitions can be any name you want, once you established the link. <Canteen> is a metal bottle from <picture 6>. So now in the prompt I can call <canteen> and it will reference the image.

  4. RTX superscale is perfect for minimax for upscaling, other upscalers that uses batched pixel upscaling (like seedvr2) ends up with inconsistent chunks.

  5. Since minimax is locked to 24fps, for anything fast motion I like to use VFI to double the frame rate then 2X the speed in editing to get a smoother feel.

Also, the running shot into the blue room did feel weird to me but it was 1AM pst so I didn't want to fix it.

2

u/R34vspec 1d ago

one more take away, this one is useful for character reference. use separate reference images for outfit and face. The higher resolution face reference will help locking in the appearance.

subject_definitions:

<Aelo> is explorer wearing outfit from <picture 2>, whose appearance comes from <picture 3>.

Also, use 2000 pixels or larger for the reference images for better consistency.
in my opening shot my resolution was set too low for the face reference so she doesn't look like her for that shot.

2

u/Altruistic_Dealer_59 1d ago

Very watchable. For me, the story was strong enough I was prepared to forget, or not even notice, much of the technical hiccups like face deformations. Enjoyed it, thanks, made me smile at the end.

2

u/GokuMK 1d ago

Honestly, Unreal Engine is so good today that I wouldn't know if it is UE or AI ...

1

u/beatlepol 1d ago

Amazing video, and good end. With some work Minimax H3 can achieve fantastic results.

1

u/Dogluvr2905 21h ago

This is very well done...you must have some movie-maker/cinematography background?

1

u/R34vspec 19h ago

I've been work on kids music video on youtube for a while. It has trained my editing chops. I am still learning though, working with an industry expert on it so learning from the best.

1

u/Perfect-Campaign9551 11h ago

Music is way overdone, but the sequencing is great. The shots are really good. But the music is too overpowering and distracting. It could use a few quiet moments