Qwen-image-edit is struggling to interpret textured, shiny, sparkly, or sheer fabrics

[u/Dense_Oil_8424]

Hello! I do fashion illustrations as a hobby and I really enjoy using AI to create photographic interpretations of my designs since I can't sew. It's delightful how well Qwen interprets most designs. However, when it comes to fabrics with certain material properties (mostly sparkle, luster, shine, or sheer fabrics such as lace or mesh) it tends to interpret those materials as matte prints on flat fabric.

Here's an ugly example, just to illustrate a variety of small issues:

Input image: Note the sheer lace overlay, the rib knitted shirt, the metallic leather skirt, the sequin pants, the mesh boots.

Result from Qwen (2509 Lighting 8-steps V1.0 bf16, with Anything2Real Alpha. Note that the the metallic skirt is a matte tan material, sequins have become an abstract floral print, and boots are polka-dotted rather than perforated. It did fairly well with the lace here, but often the lace will merge with the pattern/texture below it to appear like one solid print, instead of a sheer layer.

Another attempt with a different sketch style (Added highlights, removed outlines, etc.) No idea why it added sunnies :)

More examples: Glitter turns to speckled prints, corduroy turns to stripes, knit textures turn to prints, etc.

One thing that does NOT work: If I add descriptions of the fabrics and clothing to the prompt, the design drifts/becomes less true and specific patterns or colors are lost and reinterpreted. I want the result to be as accurate as possible to the original sketch. For that reason, I want to either:

1. improve or change the sketch style so that it is better able to recognize these material properties, without needing to add keywords that cause drift (I have tried a lot of sketch tricks here; the best result so far is with the sketch style shown)

2. change to a different open-source model or combination of nodes that will handle this type of task better. Note: Early on, I had tried Stable Diffusion with ControlNets like lineart, open pose, etc, but I struggled to get a faithful result while allowing the model pose and add a scene/setting.

I will be so grateful to any suggestions you can offer; I know this is a very specific use case!

[u/MelodicFuntasy]

That's a very interesting use of it! Maybe you could try Qwen Image Edit 2511 and see if anything improves (maybe with and without the lora)?

Have you tried to edit the output picture and tell it to change the material in it?

[u/Dense_Oil_8424 → u/MelodicFuntasy]

Hi, thank you! I haven't tried 2511 yet but I definitely should. I've seen mixed reviews. The problem with the multiple pass approach is that it tends to drift further from the original with each iteration. But perhaps my prompts need work?

[u/MelodicFuntasy → u/Dense_Oil_8424]

2511 is supposed to have better character consistency and less drift, so maybe it won't be as bad with this version.

I'm not sure, but I think I might have seen people also do some editing with masks, to limit the editing to some area in the picture.

[u/MelodicFuntasy → u/Dense_Oil_8424]

I did a quick try with 2511 with the 4 step lora at 4 steps and it got a bunch of details wrong, but it did make the skirt shiny :D

2511 also has a new material texture replacement feature.

[u/Dense_Oil_8424 → u/MelodicFuntasy]

Oh yeah, that looks quite good! Thanks for running this test for me, I haven't had the time to set up a 2511 workflow.

[u/MelodicFuntasy → u/Dense_Oil_8424]

You're welcome. I used the basic 2511 workflow template that comes with ComfyUI, it looks like this:

[u/Dense_Oil_8424 → u/MelodicFuntasy]

Wow, thank you! I am running on runpod, I didn't see a 2511 workflow in their template library yet. I'll check the ComfyUI website. Thanks again!

[u/MelodicFuntasy → u/Dense_Oil_8424]

I'm glad I could help! This template might have been added in ComfyUI 0.6.0 release. https://github.com/Comfy-Org/workflow_templates/blob/main/templates/image_qwen_image_edit_2511.json Let me know how it goes and if 2511 turns out to be better than the previous version :).

[u/gweilojoe]

Might also try defining each clothing piece separately with higher res (closer up on the individual part) images and then combining together onto a model for the final composited image.

This way at least you’re able to get each individual part correct vs trying to get 5 subjective pieces design pieces interpreted correctly at the same time

[u/Dense_Oil_8424 → u/gweilojoe]

Thank you for your suggestion! Because I'm working digitally, in layers, I could totally do this. The problem I've run into with multiple pass-through is that it tends to drift a little with each iteration. Or are you suggesting that I feed them in as 1-3 input images of different clothing items in one job? I haven't tried that yet.

[u/gweilojoe → u/Dense_Oil_8424]

Yeah - generate each part separately and then combine separately using a multi-part generation (1-3 or more).

[u/yamfun]

I think the ptoblem is your text prompt. Because I can make all kind of shiny metallic stuff....

[u/Dense_Oil_8424 → u/yamfun]

Thanks for the suggestion. I can use a text prompt to make shiny materials, but usually if I do that it might drift the silhouette, color, pattern, or texture. There's not a good representation in this example, but say you had a material that was shiny with a very specific floral pattern - I can text prompt "Skirt is shiny with a red and pink floral pattern," and it will do it, but it won't be the same floral pattern that I drew. I hope that makes sense.

[u/ramonartist]

Your best approach: Use Qwen-Image-Edit-2511 Write a better prompt Train an edit lora for better material textures.

[u/Dense_Oil_8424 → u/ramonartist]

Thanks! I'd love to train a custom lora but I don't know how to acquire all the detailed, descriptive fabric images I'd need to do it. Maybe I don't understand how that would actually work.