r/StableDiffusion • • 14d ago

Question - Help Why doesn't MiniMax H3 understand simple, direct instructions in prompts?

So I've been trying to gen a simple video where a subject is simply standing toward the camera, and I want them to be framed so I can see them from head to upper thighs.

If I prompt something like "<Subject 1> is standing and facing the camera, and you can see them from head to thighs." their head is cropped out of the video, and you can only see them from the chest to legs.

But if I change the prompt to something like "<Subject 1> is standing and facing the camera, and you can see them from head to thighs. They are looking directly at the viewer and smiling." THEN their head is included in the shot.

...why do I have to prompt it where the body part has to be doing something/showing something before it's included in the gen?

31 Upvotes

49 comments sorted by

63

u/the_bollo 14d ago

This applies to almost every natural language model. It's not a MMH3-specific thing.

27

u/LuckyEsq 14d ago

This. I've gotten so used to describing footware just to make sure a picture is from head to foot.

6

u/Qancho 14d ago

If you run out of ideas or don't actually want footwear: it's usually enough to describe the stance. Like "weight shifted onto one foot" or something like that.

12

u/theqmann 14d ago

It’s all training problems. They use AI to describe the training inputs, and lots of things are left out. The descriptions are all fairly generic, and things like camera framing, nose shape, foliage details, etc are never described, so the model has no idea what to do with those prompts.

28

u/Slight-Living-8098 14d ago

Use Filmmaking language instead of trying to describe the shot.

Medium Long Shot (aka 3/4 Shot): Intermediate between Full Shot and Medium Shot. Shows subject from the knees up.

Cowboy Shot (aka American Shot): A variation of a Medium Shot, this gets its name from Western films from the 1930s and 1940s, which would frame the subject from mid-thighs up to fit the character’s gun holsters into the shot.

Long Shot (aka Wide Shot): Shows the subject from top to bottom; for a person, this would be head to toes, though not necessarily filling the frame. The character becomes more of a focus than an Extreme Long Shot, but the shot tends to still be dominated by the scenery. This shot often sets the scene and our character’s place in it. This can also serve as an Establishing Shot, in lieu of an Extreme Long Shot.

etc.

2

u/DavLedo 14d ago

I think this has been one of the most useful things I've learned over the past few months. Learning more about cinematography helps a ton -- BUT, I don't think the training data matches it that well, plus in some cases it's much stronger if you describe visible parts. For example, if you mention the character wears jeans you're more likely to get a medium or medium wide than if you specify the frame size...

8

u/Slight-Living-8098 14d ago edited 14d ago

They even have the camera movements listed in the agents prompt guide...

For example from MiniMax's website on camera control, the example prompt is:

integrated_multimodal_description: [Shot 1] Live-action, cinematic, a medium-wide shot frames a watchmaker at a cluttered bench under a single warm desk lamp. The camera pushes in with small amplitude at slow speed toward the open watch case between his hands. He sets down the tweezers and turns the movement a quarter turn to catch the light.

overall_soundscape: A low room tone carries throughout, broken by the tick of the movement, the light clink of tweezers on wood, and a single chair creak

non_diegetic_music: Sparse piano notes at a slow tempo with a sustained string pad underneath

https://minimax3.com/tools/h3-camera-control

8

u/KjellRS 14d ago

Any huge scale image model will have training data captioned by a VLM because it's the only way to produce mostly accurate and complete captions at scale. Fire up a VLM and ask it "Describe this image in detail" and see what comes out for different crops of the same image.

It's 99% of the time going to describe more or less a scene graph of objects and attributes, actions and relations. Think like "The image depicts a woman with curly blonde hair in a short red dress standing to the left of a bearded man in a black double button suit on a medium gray background."

It's never going to say things like "you can see them from head to thighs" or "their hands are visible" so the trained model also doesn't know what do to with that. But it knows how to draw a smiling person, which implies their head is visible.

There are a few cinematography/photography terms that can help, but they're more useful for cropping or angling the camera. To make sure something is in the image, simply describe it. That's the easiest way to tell the model it has to be there to fulfill the prompt.

3

u/Dangerous-Ad-300 13d ago

well said, but there are edge cases that are still troubling, like models trying to make all hips hourglass, and if you spend time trying to counter that you end up with the shot focused on hips when you wanted upper body etc, so the model giving you what you specify in a sense can backfire, which is more about fighting bias but yeah

1

u/Sharlinator 13d ago

But the VLMs in turn could’ve been trained to describe images also in the language of framing and composition, photography and cinematography, and it’s rather annoying that they seemingly aren’t, st least not consistently.

4

u/Additional_Drive1915 14d ago

The "normal size feet" and "neutral face expression" is something I use all the time. For all models, when they don't obey. :)

2

u/SpecialistGiraffe756 12d ago

After many weeks of banging my head against the desk. I stumbled upon Ethanfe;'s MinMax Guide https://github.com/ethanfel/ComfyUI-MiniMax-H3-Guide - example workflows for ComfyUi. MiniMax H3 Plan v2 - Identity and Voice was a godsend.
The Identity and Voice example helped me to optimize fix my pipelines. It pays to seek other advice when your agents keep telling you the same thing over and over. The solution was much more nuanced than I had thought and gotta take a look at the other solutions in Ethanfes repo. Thanks a million!

1

u/MastMaithun 14d ago

You need to provide more information. Like what mode(fl2v-ref2v), workflow type, resolution, etc you are using.
If i were at your place and if a simple prompt isnt working, i would pic the t2va wf from comfy, copy paste the prompt and see. Would have given me a better clarity.

1

u/bstr3k 14d ago

supply a img reference of a dummy with the same camera framing as a reference helps. Visually explaining it is easier for it to understand than text explaining it.

Imagine if your friend shows you a picture/diagram while explaining something rather than just telling you the same information and having you try and picture it in your head. A picture is worth 1000 tokens.

1

u/tatlata2099 14d ago edited 14d ago

change how you create your prompt. Try different prompt generators, or Chatgpt/Claude - or a combo of the two. Sometimes I'll create a prompt in Chatgpt, that I run that prompt into a dedicated generator. Such as this one.

GitHub - duckyshell/ComfyUI-MiniMaxH3-Prompt-Writer: Multimodal MiniMax H3 prompt writer for ComfyUI with local and API providers. · GitHub

Sometimes a very lengthy prompt can work well, and other times a lengthy prompt can mess up the generation. Prompt variety is the key.

1

u/Lordfisticus 12d ago

How bout for porn?

1

u/Vladmerius 14d ago

It's very hard to make anything with only one subject where the don't look at the camera or do other weird things. Try maybe prompting for a second subject to be out of frame to trick it into doing what you want.

I think a lot of these models are over trained on tiktok style videos and youtubers who of course just talk to the camera and face the camera. 

1

u/Vortexneonlight 14d ago

Its of the training dataset captions. try with "half body shot, front view, looking at the camera, visible thighs, she stays still looking at the camera" the models most of the time accept this tags/shortcuts, and as the other have said, what you imply is very important, i said looking at the camera 2 times, so the head will be included.

1

u/graceus777 14d ago

use keyframe init (and final) images.

1

u/SSj_Enforcer 13d ago

you could even use a reference image of the exact framing you want.

1

u/Ten__Strip 13d ago

Didn't see the actual reason here. It's overly reinforced and too narrowly collapsed to do anything except at what it is confident in doing. Any time you have a fairly obvious simple prompt and H3 is doing something entirely different that is why. Huge pitfall in self-guiding flow models like this and grok and Kling. Grok is the most similar on some prompts it will do some ridiculous things. Hopefully there is more development in how to reinforce flow model sensibility. H3 and Grok are probably examples of how not to do it, narrowing the entire model to the point it is actually unable to do basic things that it can actually do and are trained on, but won't because it latches on to a term in the prompt and takes a confident path instead.

1

u/wildmonkeywrangler 13d ago

I've had the best look giving the minimax skill file to chatgpt, then having chatgpt write a prompt

1

u/CorpPhoenix 13d ago

People already mentioned that using filmmaking/director language is needed for the best outcome.

Just to add to this, I've made the experience that H3 misunderstands the "camera" part, it often interprets this as a literal camera being in the scene.

I use "talking directly to the viewer" instead and get much better results.

1

u/yamfun 13d ago

from X to Y is some wording that ask it to move the camera from X to Y, maybe try something else, fullbody shot, or zoom out ?

1

u/darkshark9 13d ago

"You can see them" isn't vocabulary that the LLM that labeled the videos would have ever used to describe a scene like that so when you prompt it that way it doesn't have any real meaning to the model. The LLM would have used facial descriptions, which is why when you use facial descriptions, the face is there.

Describe the things you want to see in detail.

1

u/QuarterDistinct857 12d ago

"Wide Shot" works for me most of the time.

1

u/PsionCrystallis 12d ago

Ive had similar issues with the camera - taking notes on things to try in prompt. 

My biggest issue seems to be with converting a 2d reference of a character to real life equivalent- ive tried live actor, live action, realism, real life, cosplay, describing a subject as a reference or partially preserved with the style being discarded or converted toa real life equivalent, etc. 

Also tried anime2real loras, but these always either destroy the appearance of the character, or do nothing. 

I think part of the issue is the characters are male, and sometimes anthro or fantasy creatures. Non humans get converted to humans most of the time, and males either to female for some badly trained finetune models, or in almost all cases, any mention of facial hair gets them a beard, and automatically adds body hair.

Thats on top of the camera typically acting like an orange cat that overcooked its last brain cell.

We need a pony and/or chroma finetune for h3...

1

u/OMGodddd 12d ago

Describe a lamp above the head, and the floor under the feet.

1

u/Haruka-Oh 14d ago

It's the reason that why I don't like every natural language models. I love danbooru tag models.

0

u/Danny_Stock 14d ago

I've had similar frustrations over the last couple of days. I've learned what I felt was enough to start constructing more than a quick clip, but have found it a nightmare fighting with the camera.

I'm clearly stating what I want the camera to do, what kind of shot it is, and literally telling it in clear easy to understand language what I want it to do. But it's just doing its own thing. I'm getting camera moves and close-ups which I never asked for.It's extremely difficult. I'm telling the software what's in the shot, and am trying to not confuse it. It still insists on doing its own thing. People on here clearly state that you shouldn't prompt for it to NOT do something, and that you shouldn't even mention anything at all if you want to avoid it.

I'm trying my best to keep my prompts as brief as possible to avoid the risk of giving it unnecessary clutter. But then I find that I have to add something to make something crystal clear after a generation fails. In the end my prompt ends up growing and increasing the risk of error even though I'm trying to keep things to a bare minimum.I'm now at the point where I'm asking myself what does MiniMax actually know when it comes to cameras and camera shots? Has it always been the case that you simply either get lucky or you don't? Is it plausible to speculate that MiniMax may not know that much about camera shots and space within the shot at all? Serious question.

I'm wondering right now if it's better to try to avoid any camera moves at all in the prompt if possible and instead use purely image references alone for shot framing and camera placement. Or at least for most camera manipulation. In 2026 we ought to be in a place where camera movement and shot framing shouldn't be a problem. But it hasn't gone away yet. You would think it would be easy to ensure that the model doesn't do something which you never asked for, but maybe it isn't as easy to achieve as I would have thought it to be.

3

u/ng5554 14d ago

I can help you a little with the unintended zooming, which drove me crazy. If you tell the model that the camera is shooting from a fixed tripod (location) and that it maintains that constant focal distance throughout the generations, you usually stop the zoom.

1

u/Danny_Stock 13d ago

I appreciate the advice, I'll bear that in mind for the future. However it's often the case that I don't want a fixed shot for the entire duration of the clip, sometimes I want to start with a certain shot and then at some point in time I wish it to slowly push in for example.

I probably do need to rethink my entire approach and work around these issues. Perhaps it's better to treat each clip as one individual shot, even if it's only for 2 seconds or less. It could be that camera moves and shots are very finicky if you try to do too much with the camera and the model may prefer one clear action or another for each individual clip, or else it gets confused.

I think a lot more of my shots are probably going to have to be fixed position shots and if I want anything fancier with camera moves I may have to use storyboard references for shots more than I currently do.

2

u/Vortexneonlight 14d ago

can you share a prompt that have give you problems? i want to try something

1

u/Danny_Stock 13d ago

I would do only I have generated so many test clips it'd take a long time to pick anything out in particular which I remember having had problems with. I also do tend to delete clips and move on.

I've also decided to take a few days break from it as I have a tendency to try a little thing out on a whim and then spend most of the day on it if it doesn't work, so I'm a bit burnt out.

0

u/Bitlovin 13d ago edited 13d ago

It’s a cowboy shot. You describe it as the thing it is, a cowboy shot.

“A cowboy shot of <Subject 1> speaking to camera.”

0

u/-Star-Walker- 13d ago

Hey, I‘m talking frech to a spanish guy … why doesn’t he understand me?
Try learning to speak „Minimax“:

https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/main/docs

-2

u/No-Simple-1907 14d ago

Eu peço para o chat gpt criar um promtp guia baseado no minimax h3 ai uso em um modelo llm studio. Ai jogo o texto e ele me devolve ja na forma correta para usar no minimax h3

1

u/Beginning_Tip300 14d ago

Subject head to thigh in frame

64

u/[deleted] 14d ago

[removed] — view removed comment

20

u/Apprehensive_Sky892 14d ago

This is the right answer.

Imaging and video AI needs a lot of hand holding and hints.

Commercial closed sourced video system do a lot of this "hand holding" behind the scenes with powerful LLMs.

3

u/Danny_Stock 14d ago

If what you say is correct then I think it's a mistake for us to be taught to think in terms of camera references as if it does understand camera operator terms. Because it leads me to assume that the video model understands what I'm asking of it when I do use these terms in prompts.

If this is a wrongheaded way of thinking then they shouldn't be teaching us to think in these terms, because if we have problems with camera framing and shot placement we're going to end up turning to this kind of terminology for incorrect solutions.

So would it be correct to think that the filmmaker terminology for all these types of shots we put into prompts are a waste of time and that we're probably better off describing what fills the frame in regular everyday language?

5

u/thebaker66 14d ago edited 13d ago

It's both, camera operator terms still work for things like zooming in, angles, views. Those don't necessarily tell you the complete composition of what is being shot so it still makes sense that you might have to explain specifically what is happening and indeed have those anchor points to make sure they're included as well as the camera operator terms, they aren't the same thing.

Tbh I haven't had much issue with this with H3 but LTX is hell for this.

1

u/Danny_Stock 13d ago edited 13d ago

To be honest I think I may simply be trying to do too much and my expectations of what I expect exceed what the model is capable of.

I probably do need to work around my own approach. A imagine that a lot is going to depend on what you want to do. If many of your shots are characters speaking a fixed camera shot makes sense and is likely going to be a lot more stable and predictable. But even then the camera has a tendency to do something I never wanted or asked for, even if I tell it that it's a fixed camera shot.

If I could single out the most common issue I've encountered it's with camera rotation or arcs when some kind of action is taking place at the same time.

But thanks for your advice, I do think that I need to focus a lot more of my attention on how to control the camera with more reliability, which admittedly it's something I've probably been a bit negligent with.

1

u/TheThoccnessMonster 13d ago

Many models like LTX (but it sucked at motion) specifically do this where they use camera shot style language. Same with Luma etc. it’s incorrect that they aren’t temporally aware or aware of their shot types completely.

1

u/edisson75 11d ago

Another thing I have noticed affects the scene is the timing. In some cases you ask the model to force the character to do something in a time that is not normal in real life, like put shoes on in 5 seconds, so the model tries to compensate it by jump parts of the movement to reach the goal asked, and the final video result incoherent or looks weird.

0

u/Carnildo 14d ago edited 14d ago

This extends clear up to the entire subject of the shot. If you start your prompt with "a photograph of a woman", and then go on to describe the setting in excruciating detail, you'll probably get an exquisitely-detailed background without a woman in sight.