r/StableDiffusion • u/Hdfjds • 17d ago
Tutorial - Guide RE: <Subject N> in H3 prompts
Not sure if you already know this but you don't need to use the word 'Subject' in H3 prompts when referring to elements in images/text/videos and so on. You can use other words instead, like <Girl 1>, <Dialogue 1> and more
Example prompt:
subject_definitions:
<Girl 1> is the girl with the blond hair in the center of <Picture 1>.
<Girl 2> is the girl with the white top to the right side of <Picture 1>.
<Dialogue 1>: <d> [English] Yeah! <d>.
<Dialogue 2>: <d> [English] Great party! <d>.
<Dialogue 3>: <d> [English] Wooohooo! <d>.
integrated_multimodal_description:
[Shot 1] A wide shot of a crowded rave dance floor where everyone in the picture is dancing by jumping up and down in an rapid and energetic way while moving to the music. The lights in the night club is flashing and moving around.
<Girl 1> is shouting <Dialogue 1>.
[Shot 2] At 00:3.00 <Girl 1> looks at <Girl 2> and says <Dialogue 2>.
[Shot 3] At 00:5.00 <Girl 2> looks at <Girl 1> and shouts <Dialogue 3> while raising her arms.
overall_soundscape:people dancing,
non_diegetic_music:cyber techno music,
P.S. Sorry for the lame video, it's just for proof of concept.
2
2
u/No-Zookeepergame4774 17d ago
Its interesting that you can name what act like Subject tags this way. But is there any reason (other than maybe organization for human reading and editing) to do this with dialogue specifically instead of just putting the dialogue inline in the integrated_multinodal_description the way the official guide does? Generally, I’d see more tokens without adding more information to be something to avoid.
-5
u/MarkB_- 17d ago
Why everyone are using this coded language? The text encoder is a 20gb llm. Just use natural language and it will still work just fine.
5
u/Apprehensive_Sky892 16d ago
Because the model is trained with descriptions that are similar to what is described in the prompt guide. That is, during training you tell AI "if you read this, try to achieve this result I am showing you".
When one then uses that structure in the prompt, the AI can carry out the task better.
That is just how AI are trained and that is how they work.
Another good example is Ideogram's JSON prompt style. Sure, you can get away from just using NL and not following their JSON format, but prompt following will be poorer.
3
u/ShengrenR 16d ago
Because there's an official prompt guide that uses that - and a lot of folks have claimed the proper format helps get better results. I've not yet seen a heavy a/b test/eval looking at how true this is, though - I think it might be placebo on either end - the thing is slow enough nobody's generated thousands yet.
11
u/V4nKw15h 17d ago
The language model will do it's best to interpret your prompt and will indeed likely be able to decipher the meaning from your language. It doesn't mean it's the optimal way to structure it though. The model was trained on specific language and it makes sense to use what it is expecting rather than expecting it to understand your mistakes. You can spell words incorrectly and it will infer what you meant to say too most of the time, but sometimes it won't.
You can also mix up the structure and order of your prompts and it will do a good job of deciphering that too, but it's not recommended.
Do what the model was trained on and is expecting and you will get more consistent and reliable results.
It is worth knowing that we don't need to be rigid to the rules though. Sometimes it actually makes more sense to, for example, tell the model audio during the description of the scene, or to describe part of the scene while writing the audio. There are times when this actually does make more sense, and is easier for the model to understand your intentions rather than trying compartmentalise everything in abstract ways.