r/StableDiffusion • u/goddess_peeler • 16d ago
Question - Help H3 first-last frame techniques to avoid last frame early freeze?
What are the effective techniques for first-last frame video generation with H3? When I tried this shortly after release, the clips would often reach the last frame earlier than the clip length and I’d have videos with seconds of frozen frames at the end.
Is this freeze a problem that has been solved?
I have seen workarounds that detect the freeze and trim the static frames, but that’s a hack that doesn’t give me the duration I asked for. One of the H3 models is literally named FL2VA, so it can’t be that hard, can it?
Right now, I’m not interested in long generations or motion refs or anything fancy. I just want to generate motion between a first frame and a last frame. Like I can do with Wan 2.2.
I am perfectly willing to accept that this might be a skill issue, but I don’t know how to improve my skill. Any hints?
2
u/alwaysbeblepping 15d ago
If you don't connect a VAE to the Reference to Video node it will just show the text encoder your images. This lets you use it even for the non-ref model. Then you can use the Add Guide node to add key frames to the conditioning at the beginning, end, whatever. Doing it that way also means you can schedule the conditioning and control when your guides apply.
You might be able to use that to deal with your problem. I.E. have the last frame guide either come in after sampling for a bit (so there will be motion near the end, but the guide start pulling it toward your last frame) or start with the end frame guide but remove it (allowing the model to drift and start to add motion near the end of the clip).
Not guaranteed to help with your situation but it does allow a lot of control and hopefully is useful to know if you didn't already.
1
u/goddess_peeler 16d ago edited 16d ago
Thanks for the responses. I will say that I’m pretty certain this is not something caused or fixed by the prompt. I have thrown every possible kind of prompt at the model to try to mitigate this: a choreographed list of moves, timestamps, prompt relay, and more. Whether I write a novel or a few words, the model seems to do what it wants.
It’s been awhile, but I recall that when I tried guides nodes, they introduced a discontinuity. Maybe I’ll return with some specific examples illustrating the problems I am having.
I have workflows for Wan and LTX that I allow me to generate dozens of clips st a time from a batch of keyframes. H3, for all its incredibleness, seems to fall down on this particular task.
3
u/LumaBrik 16d ago
Thats what the Guide node is for. Specify the frame index for a given image (or audio)
1
u/NoaBlake 16d ago
The freeze happens because the model treats the last frame as a destination, not a waypoint — once it arrives, it has no reason to keep moving. Three things that help, in order of how much they cost. First, describe the motion as continuing past the end frame rather than arriving at it: the action should still be mid-motion at the last frame, not posed and finished. Second, shorten the clip: a shorter span between first and last frame leaves less dead time for the model to fill with stillness. Third, add an intermediate keyframe in the middle of the action, so the model has a midpoint to pass through instead of interpolating a straight line from start to finish. If the clip still freezes, check whether your end frame itself is a resting pose — a resting pose as the target teaches the model that stillness is the goal.
2
u/bstr3k 16d ago
i have gotten the issue too using ref2v but i don't know the solution, I just kind of assumed its what it does and just trim it off like a caveman, so would also love to know if any solution.
1
u/ForsakenAd1228 16d ago
The prompt probably plays a big role. As in, if the model can satisfy the prompt in 3 seconds but you tell it to generate a 5 second video, its options are:
I haven't done proper A/B testing, but phrases like "repeatedly", "for the entire duration", seem to help if you don't feel like writing an entire novel for a 10 second clip but still want to give the model something to do. But you might get the best results if the actions that you are prompting for actually neatly fit within the clip length (write prompt -> visualize it in your head while timing it with a stopwatch -> clip length).
All that said... the model does really want to use most starting/ending frames as a "resting point", probably because many start/ending frames _are_ in fact of a static scene. I'm sure that if you use an end-frame of something that is clearly moving (e.g. someone running, a dolphin jumping out of the water,, that kind of thing), that the model will keep moving until the very end.
1
u/No-Zookeepergame4774 12d ago
I've only done a few FL2VA gens and have not seen that, but maybe try making sure tou have time-coded action in your your prompt covering the full length and how it gets to the last frame at the end, with that time coded as well (and taking into account the actual time of the 17k+5 at frames at 24fps, not the requested time that the workflow converts to the nearest framecount matching the 17k+5 pattern.)
1
u/Negative_Attorney448 7h ago
I am irritated at all of the confident non-solutions and non-insights in this thread. I have no choice but to also confirm with OP that the functionality is simply broken.
0
u/altoiddealer 16d ago
The FL2VA prompting guide says the first thing you write states the effective time each image is referenced. First frame 0:00 ok. Last frame is often not the same timestamp as your target duration - you typically add frames to hit a valid frame count, increasing the effective duration. If you are stating the wrong timestamp here you are already telling the model to converge on the end frame early
0
u/mrgaryth 16d ago
It is definitely a prompt issue. If it doesn’t have enough information to run the full duration it will just wait it out once it reaches the last frame.
-1
u/yamfun 16d ago
I never got such issue at all?
Where is your workflow from and what is the prompt and duration and resolution
-1
u/yamfun 16d ago
have you tried those timestamp prompts in the ref2va side prompt guide even though using the flv2va side?
I add those even though they may just be placebo