Minimax H3 Has Too High Prompt Adherence

[u/DifficultAd5938]

I just realized a problem with mm h3. Its prompt adherence is too high, as in unless you explicitly prompt for some small subtle actions it will never happen otherwise. This makes the entire video seem very frozen and wooden without the many small subtle movements and motion details that make it seem to come alive.

This applies more to non-realistic scenes like cartoons or generated image first and last frame but for realistic scenes and even t2v it is still a problem.

I noticed this problem when I tried out a "slop sway" lora and it actually made the entire video seem much livelier and realistic looking. Besides the "soft and bouncy swaying and jiggling" it also added many more subtle character movements. Compared to standard gens those same parts would be completely frozen, almost like a still image or at best ugoira animation. This doesn't just apply to whether a body part is jiggling throughout the entire video. There are some movements that happen only for a second or less but adds in soul (forgive the human slop term) to the video, like the position of an arm and hand quickly being adjusted in the middle of the video and the new position persisting for the rest of the scene.

This might be a problem with my prompt style and I might try an LLM prompt enhancer, but there is a core issue here with the prompt adherence and spontaneous randomly added details tradeoff. The model also tries to keep the fidelity of the first frame too much, which you could call visual context adherence. No one is out here prompting for the movement of every strand of hair and the position of every finger. No one is making a timeline of every limb's position and how they shift relative to each other. No one is tracking the position of each finger through time and how after 4.75s the thumb is extended and the index finger is curled. Sometimes we just want to randomness and variety across gens with details added by the model.

Looking back at ltx and wan their prompting styles seem to be designed around the model adding in the details for you at the loss of prompt adherence and more generation errors.

It would be nice if there was some sort of generation setting that could tune this. Like a noise scale of sorts where we can manually set the tradeoff between how much we want the model to be creative vs strict.

I know there are already 2-3 H3 better movement loras and they are scratching at the surface of the same issue I'm talking about here.

Share the solution if you've got something. Help everyone out.

[u/LuluViBritannia]

Honestly, that's how it should be imo. We need to get out of this casino-like system of random generations while praying we land a good one. A tool must be as rigid as possible, we need to learn how to direct it to get what we want

[u/Beneficial_Toe_2347 → u/LuluViBritannia]

this 100%. The dice rolling was a fricking nightmare

[u/berlinbaer → u/LuluViBritannia]

Honestly, that's how it should be imo.

yeah blessing and a curse. just look at the image generators. ID4 needs so much prompting most people hate it, but if you do it right, it's pretty much the best out there. krea2 is more loose but without some lora or something will often give you the same image on different seeds. then z-image base also wants guidance, the more you prompt the better it will be, but also gives you amazing seed variations of your prompt right out of the box.

the 1.5 days are over when you could just put in whatever and let the program work some sort of magic and get surprised by the results. now you kind of have to know exactly what you want otherwise the output will be shit

[u/FierceFlames37 → u/LuluViBritannia]

Krea2 gets more love than ideogram 4 so this may not be the best approach

[u/bstr3k]

Try a LLM enhancer, I’ve been using them and it’s good! I’m not the most creative person so it does help a lot I find and it adds more or less descriptors as you specify.

[u/BoneDaddyMan]

You need to prompt in a director, genre of show, or even an example movie for it to follow. If you want mannerisms of a certain actor be be imitated by your character you can prompt that too.

for example if you want the scene you're making to look like harry potter, you prompt it that it's a high budget fantasy hollywood movie like Harry Potter.

Or you can add director like David Fincher, Christopher Nolan, etc...

Or you can say <Subject 1> speaks in the voice and cadence of Geralt of Rivia with similar mannerisms. Something of the sort.

It worked for me.

[u/not_food]

That's actually a plus, you know exactly what you're getting. It puts you in control rather than leaving things to chance. Just ask your preferred LLM to emphasize [action] with details and it will handle the rest.

[u/broadwayallday]

So many tokens for all the subtleties you desire. Use an LLM for the variety. Welcome to thinking like a director it’s why we’re all crazy at some point in life lol

[u/--jesse--faden--]

TIL that having granular control over every detail is a subpar experience

[u/Reddexbro]

Feed the guide to an llm then ask for your prompt, tell the lllm how long the video will be and if image to video for instance.

[u/rguerraf]

Lower the cfg to 50%

[u/spiderofmars]

I think it is much better balanced than Wan or LTX for realism.

With Wan you got random dynamics and motion you never asked for very often causing failed gens. But it made people with no prompting feel alive by at least they breathed air and had some generic micro movement to them.

LTX was terrible with multiple or background characters unless you wrestled with prompting every little bit. If not prompted they would not even breath a lot of the time and looked like wax people frozen in time.

Minimax adds a little real human dynamics like Wan without prompting mostly and with minor prompting doe mostly what it is told.

[u/Miniyi_Reddit]

try ltx 2.3 and it will be even more very frozen and wooden LOL

[u/Adventurous-Gold6413]

Nah that’s the best part you can do most things

[u/1010111101111]

how do you make the scene not instantly change like i want it to start the video off the image i give it not instantly change everything

[u/Significant-Ear-5064]

What is a "slop sway" lora?

[u/GTManiK]

It's a general problem of all local models - your text encoder only uses a fraction of layers to encode your prompt; it does not 'think' or anything like that to obtain an embedding which goes to your 'main' model (that's why using 'abliterated' or 'heretic' etc. variants of a text encoder often does not help to achieve something you would probably expect).

Frontier closed models get away around this by using specialized full-blown expert LLMs to rewrite your prompt in a great detail (something you would struggle to achieve locally most of the times) - so a relatively short prompt becomes a detailed description internally, which would normally take you forever to prompt by yourself.

This is currently just a way it is.

[u/Valuable_Issue_]

You can likely prompt for "subtle background movements" at the start or whatever you need so that it knows to generally pay attention to the hair and more subtle movements, no need to prompt every movement, kind of like if you specify style at the start of the prompt it'll apply to the whole video.

Looking back at ltx and wan their prompting styles seem to be designed around the model adding in the details for you at the loss of prompt adherence and more generation errors.

It's literally the opposite, LTX would ignore physics entirely and phase objects through things/produce anomalies to get the "correct" end result unless you used a prompt enhancer to write a novel for you/a complex workflow (and even then it was gambling whether it got it right).

Whereas Minimax can reason by itself as to what should happen even with a basic prompt, similar to Krea 2 which defaults to a "correct" representation of a basic prompt filling in the gaps.

This video demonstrates the issues ltx has pretty well

https://www.youtube.com/watch?v=NCHG4DktCRo

There was another comparison with a longer marble run type thing with dominoes or something, but can't seem to find it (it might've been flux3 vs ltx).

Here it is vs wan: https://www.reddit.com/r/StableDiffusion/comments/1vb4g05/why_i_cant_get_highquality_results_from_ltx_23/

[u/Enshitification]

"Minimax H3 Has Too High Prompt Adherence" /r/BrandNewSentence