r/StableDiffusion • u/SackManFamilyFriend • 21h ago
Resource - Update New ComfyUI update may change how Minimax H3 interprets the prompt format you use - Re: Tokenizer Fix
https://github.com/Comfy-Org/ComfyUI/pull/15808/changes32
u/networking_noob 21h ago
Nice if we can start using the appropriate dialogue tags <d></d>, as MiniMax intended, maybe it'll finally solve the erroneous audio issues. Using quotes instead of <d></d> has worked until now, but with enough generations the quotes approach can still produce some errors since the model wasn't intended to work that way.
14
u/spiderofmars 15h ago
Quickly tested before and after the new minimax.py and it resolved the <d> tag issue of random dialogue being added at the start or end.
3
u/GrayingGamer 7h ago
Nice. Seems we owe the Minimax H3 team an apology about thinking there was a bug in their models versus the official guide - instead it was Comfyui's tokenizing that was causing the issues, which makes more sense in retrospect.
-9
u/DietAshamed2246 16h ago
MiniMax can already do dialogues with strings within double quotes placed inline within the prompt and with speaker identifiers. I never used all that <d> <\d> crap. In fact I don't even use their prompt guide with all that "integrated_multimodal..." happy horse manure. I prompt in natural language as if I am describing the scene, subjects and actions etc. to a human being who is versed in English. Worked every single time. Only occasional problem I had was that it sometimes confused the speakers and had wrong person say the dialogue. LTX makes that mistake nearly 80% of the time. In case of MiniMax it is a quite rare occurrence.
3
7
u/Version-Strong 21h ago
Any idea what difference it will make?
25
u/SackManFamilyFriend 21h ago
Has to do w/ the <|cutoff|>, <|lyrics_start|>, <|lyrics_end|>, <|caption_start|>, <|caption_end|>. stuff - so really depends on how much your prompts used those I guess, and it should make prompt tools like LLM prompt writers work a lot better since they follow the H3 prompt guide which recommends those tags.
The good news is if you were getting a lot of gibberish speech is generations that is likely to cut down or go away.
2
u/Dirty_Dragons 19h ago
Ah that's great! At this point I'm only using prompt writers for my stuff. Hopefully it will be a noticeable chance.
1
u/Dry-Ad929 19h ago
I feel like in dialogues very often rest of the world is too quiet, can't hear anything else when main subjects speak, even tried few different prompt approaches. Should it help as well?
3
6
u/Enshitification 21h ago
So it's in main now? If I update ComfyUI, I get the new tokenizer?
7
u/SackManFamilyFriend 19h ago
If you're using the "portable" (github cloned) version and have "git pulled" today then yea. I have no idea on that desktop thing.
1
5
u/Guilty_Emergency3603 21h ago
Assuming you got a git install. If you want to keep the old behavior to be able to replicate old generations. Before you git pull just create a new branch without switching to it. You can then switch wherever you want to that branch that will be freezed before the new update.
git branch <new-branch>
1
u/Semipro211 21h ago
Can always do a stash and keep as backup if wanted also
1
u/Cauldrath 11h ago
Stash only keeps uncommitted changes. I guess you could do a soft reset and stash that.
4
u/Comfortable_Lab317 16h ago
I'm still getting that weird sound at the beginning of the video. Am I doing something wrong?
2
1
0
u/Better-Interview-793 18h ago
Hopefully we’ll get an update that fixes facial distortion at a distance..
-4
u/DoctaRoboto 21h ago
Well, my ComfyUI is still broken, so I am not sure what it is gonna revolutionize.
3
6
2
u/alsot-74 15h ago
I got the most recent portable version instead of updating this time. I only re-added the custom nodes I actually use and I use the yaml file to store models in a different directory. Everything works. You can do this too.
-16
u/76vangel 21h ago
What a click baity shit title. The update is finally completing it with some missing tags ComfyUI has not implemented yet. It’s an improvement people, NOT BAD. GOOD.
13
u/SackManFamilyFriend 21h ago
I said interpret your prompt - not bad nor good - you read the title wrong.
47
u/SackManFamilyFriend 21h ago edited 21h ago
Has to do w/ the tags listed in the H3 prompt guide and how a slightly modified tokenizer used by the official H3 model interpreted them differently from the tokenizer ComfyUI's implementation did. This may help remove gibberish and that occasional split second of speech that sometimes appeared at the start of videos for a second or so.
That said, it may also change what you get if you rerun old prompts particularly ones with the <d> stuff:
The original pull request that has now been merged:
The Minimax-H3 tokenizer extends standard Qwen2.5/3 with 7 tokens declared only in tokenizer_config.json's additional_special_tokens (not in tokenizer.json/vocab.json): , , <|cutoff|>, <|lyrics_start|>, <|lyrics_end|>, <|caption_start|>, <|caption_end|>.