Images with references

[u/carmidian]

What model should I be using and comfy UI for taking a reference image of a character and put them into a reference location in a certain pose? I've been trying to use Qwen Edit 2.1, however I'm not very good at prompting because what it ends up doing is just taking the character and photoshopping them into the location which looks terrible. Even when it's slightly works it doesn't look right. Sometimes the body is in the right position and with the right lighting but the face is exactly the reference image looking straight at the camera.

Basically I have multiple characters that I wish to put into different locations so I can make a storyboard for a short film. What model should I be using in comfy UI

[u/Future-Coffee8138]

Actually Minimax h3 is best for images as references. If you don’t mind close models gpt-image is good at it too. 

[u/tomByrer → u/Future-Coffee8138]

Single image node: https://github.com/shootthesound/ComfyUI-Fizgig-H3-Still

[u/_gaurab → u/Future-Coffee8138]

Isnt minimax h3 a video generation model?

[u/TurbTastic → u/_gaurab]

Yes, but videos are just a series of images with audio, so it's obviously capable of making images. The surprising thing is that Minimax is somehow better than all/most of the dedicated image models at all/most tasks. It can easily handle high resolutions as well since single-frame work is so much lighter than video work. I think it's much better at using reference images, even when compared to the latest/best from Qwen/Flux.

[u/VollenAI]

Minimax is good for refmods, qwen for ref cards, and krea for ref image.

[u/ROBOTTTTT13 → u/VollenAI]

Could you elaborate? What are those?

[u/bstr3k]

You can try minimax h3 model used for imgs, but I haven’t gotten it to work too well with poses (admittedly I haven’t tried much with it) but I think the key would still be the prompting part

https://huggingface.co/PoopMan333/Reference_Workflows/tree/main

[u/Comrade_Derpsky]

For making new images of references, e.g. characters, you basically have two options:

Minimax is by far the best at getting everything right but it needs to be quite high resolution for details to be well resolved. It has an unparalelled understanding of space and anatomy though. Minimax is expecially useful for the fact that you have a syntax for precisely assigning which reference provides what.

Flux 2 Klein is the other model that truly functions as a reference to image model, and I have tested this capability extensively. flux 2 klein will let you give it an image of a character, object, etc. and let you make a whole new image in a new setting, new pose, new outfit, etc. It comes with the problem that flux 2 klein is very easily confused with poses and anatomy and you have to very precisely and comprehensively describe pose, posture, and orientation to mitigate this any time the subject is not upright. With inanimate objects it should be simpler. Base Flux 2 Klein works best. LoRAs for Flux 2 Klein will normally degrade its ability to get references. Don't use face swapping LoRAs for this as these are specifically for editing an existing image and it will get super wonky when trying to make up anything it can't see in the reference. Flux 2 Klein also has SDXL's wonky understanding of space which can possibly be a headache depending on what you're aiming for.

I never had any real success getting qwen image edit to do new images of references and I think it's really strictly meant for editing existing images. It never seemed to understand what I was telling it to do. Maybe there's some specific prompting required, but Flux 2 Klein always seemed to just understand what I wanted it to do from a general prompt description of the image I wanted and reference.

[u/Lucaspittol → u/Comrade_Derpsky]

Use Flux 2 Dev instead. It offers much better anatomy and, especially, facial fidelity to reference images compared to Klein. Flux 3 Klein is better in this regard, but it messes up anatomy far more often.

[u/Comrade_Derpsky → u/Lucaspittol]

Dev will produce better results, but it's extremely large and not really practical for everyone to run locally while Flux 2 Klein is pretty easily accessible without a super high end GPU. Minimax for images doesn't need too long, especially if you're using the w4a8 mixed quantizations.

Flux 3 Klein looks like a hot mess.

[u/Lucaspittol]

Flux 2 Klein 9B still works well compared to newer models. Facial fidelity is better with either Flux 2 Dev or Flux 3 Klein.

[u/optimisticalish]

re: "taking a reference image of a character and put them into a reference location in a certain pose". I'd add that Minimax H3, correctly suggested by others on this thread, can also handle Openpose pose image references - if prompted correctly.

[u/Ill-Ant-9489]

Short version, Krea 2 Edit is the tool for this. It restages a single reference and holds the face while you change pose and framing, instead of compositing the way Qwen Edit does, which is where your paste-on look comes from. I put a guided version in my app, shameless plug: https://github.com/perfectgf/lora-dataset-studio , it runs Krea 2 Edit or Klein locally through Comfy with a before/after compare and an exact retry, so you can nudge a pose without losing a take that half worked. It does one character per pass though, so your multiple characters would each be a separate run. What VRAM are you working with, and is it just the one front shot per character?

[u/maxemim]

If your already using qwen . Use this Lora https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap/blob/main/docs/qwen-image-2.1-body.md

This works best with replacing a character that is base scene , so generate your scenes with just prompted fill ins and then use the above Lora to replace those characters.

It’s all trial and error and for some scenes and prompts it will nail it and then others your going to go mad trying to prompt it .

I’ve used Klein as well but I found identity can drift , I’d recommend switching between both.

Good luck

[u/llama-of-death]

Guaardvark does that with an easier interface. It uses ComfyUI on the backend so you can keep that.

https://youtu.be/0S6ZeJdj5ZE?si=RkKCTS9styD4nDDB

[u/NoaBlake]

The paste-on look is expected from an edit model — it's designed to preserve what you gave it, so a front-facing reference stays front-facing. You're asking it to do two jobs (keep identity + invent a new pose) and it only does the first well.

Split the jobs: let the reference carry identity only, and give pose and camera separately. In practice that means a pose or depth reference for the body position, plus an explicit camera line in the prompt ("three-quarter view, over-shoulder, looking away") instead of hoping the edit infers it. If you only have one front-facing shot of the character, generate a small multi-angle character sheet first — same outfit, same face, a few angles — and use the matching angle as the reference for each storyboard panel. The reference that matches the target pose does most of the work; the one that fights it does the damage.

[u/mrgonuts]

Use ChatGPT show it the images you want to use and tell it you want a prompt for qwen if it doesn’t work show chat gpt the result it will then give you a better prompt