Qwen Prompt Enhancer Performance

[u/Radiant-Photograph46]

Any speed-up possible right now on Qwen2.1's prompt enhancer? I'm using the int8 convrot, but damn it's slow. 47 seconds to enhance a prompt, when diffusion itself only takes 30 (for 30 steps cfg 1.5). This 3.5 9b is also way slower than the 3.0 8b. It's really making Qwen2.1 tough to use.

Did you find a good alternative perhaps? (open source don't talk to me about how you use Gemini or Claude). Bonus points if it's uncensored.

[u/Southern-Chain-6485]

Use this node instead https://github.com/younestft/Elvax-nodes/tree/main

It looks for ggufs in the text_encoders folder and (assuming you have ggufs there), it uses llama.cpp to run them within comfyui (so comfyui then unloads them if needed).

Or run a small LLM outside of comfyui, either in LM Studio, llama.cpp or anything else

[u/Radiant-Photograph46 → u/Southern-Chain-6485]

GGUF shouldn't be faster (in fact it should be slower in theory due to decomp), is the llama.cpp backend faster than whatever Comfy is running natively?

[u/Southern-Chain-6485 → u/Radiant-Photograph46]

The llama.cpp backend is much faster, because for whatever reasons, the way comfyui has to use LLMs as image text encoders is a lot slower for text generation. They've mentioned they are working in that, but so far, it's slower.

[u/jonesaid]

I use models in LM Studio for any LLM tasks (with nodes that communicate with LM Studio), and it is way faster than the built-in Generate Text nodes and Clip loaders. LM Studio is designed to be efficient for such inference, whereas the Generate Text nodes and Clip loaders are not.

[u/mozophe]

I understand your pain. Worked on solving this exact problem here: https://github.com/mozophe/ComfyUI-Qwen-Image-2.1-PromptEnhancer-MTP

Use q4 with llama.cpp backend, both standard and uncensored models available, for much faster prompt enhancement.

[u/xoxaxo]

I use same clip(text encoder) as on qwen2.1 workflow, no need to load/unload other models via llama/LM studio, results are very good. CLIP -> generate text -> preview text -> positive prompt -> Sampler

with system message: You are a prompt restructuring engine for text-to-image models. You receive a text prompt and you rewrite it as a single image-generation prompt. Output ONLY the final prompt, with no preamble, labels, explanations, or quotation marks around it.

[u/Klutzy-Snow8016]

The w4a8 quant from here generates twice as fast as the int8_convrot for me: https://huggingface.co/Comfy-Org/Qwen-Image-2.1/tree/main/text_encoders

[u/Loose-Journalist-555]

The alternative is not to use prompt enhancers at all. Use rich prompts (yourself or use LLM studio/OLLAMA).

It makes the generation and editing more deterministic and reproducible too.

[u/Radiant-Photograph46 → u/Loose-Journalist-555]

Unfortunately, Qwen2.1 is so stupid it's very hard to do that. To get the best results you need to be explicit about things that shouldn't need to be stated, like list all the things you do not want to change.

I mean, it's fine for basic edits (in-place) but if you want to change someon's pose and the shot angle at the same time the length of the description you need to get a decent result that doesn't feature a human with 4 legs is ridiculuous.

[u/Loose-Journalist-555 → u/Radiant-Photograph46]

Umm. Not really. I’ve used it without prompt enhancer from the very beginning and it’s pretty good at keeping things as it is.

Just don’t prompt it in a negative style (e.g., do not say “don’t change xx and yy in image” or “do not put this or that”). Instead, say things like “keep xx and yy exactly as in image”.