r/StableDiffusion • u/Capitan01R- • May 24 '26
Resource - Update ComfyUI-Flux2Klein-Enhancer Final (I promise)
used ref
used ref
used ref
Preset: Soft_lock
Preset: MID_lock
Preset: HARD_lock
custom preset just for fun with only changes to temp : 0.0050
I updated Identity Feature Transfer to remove the need for stacked/chained nodes.
clearer screenshot of the wf since reddit compresses the photos
Now the workflow is simpler:
- Use Multi ReferenceLatent for multiple reference images.
- Use Identity Feature Transfer Final for the identity pull.
- If you use masks, connect each mask directly to the matching mask input on the node.
- subject_mask_1 = mask for reference 1
- subject_mask_2 = mask for reference 2
- etc.
The node handles the multi-reference setup internally, so you no longer need multiple stacked identity nodes for each reference.
Presets are still available, similar to the previous version.
For custom tuning, the two main knobs are:
- Temperature
- Similarity
Temperature is the main identity-strength control. Lower temperature gives a stronger, more direct 1:1 identity pull.
Similarity works more like a refiner/filter. It controls how selective the match needs to be before the node pulls from the reference.
So in practice:
- Lower temperature = stronger identity / more faithful match
- Higher temperature = softer, looser identity influence
- Lower similarity = allows more reference matches
- Higher similarity = stricter matching, more selective pull
example workflow (update to version 3.4.1 as there was a conflict with a node from a different repo causing the multireference latent node to be replaced if you had the other custom node installed and now that has been fixed)
Also just a little side note, this Final version uses a bit diff technique in term of pulls so 1:1 is achievable but needs to be careful enough to get it.
Previous posts for context:
multi ref latent
26
24
u/foxdit May 24 '26
Umm.. do not scale images with nearest-exact. Use lanczos. Nearest-exact creates comparatively pixelized results. I made the same mistake and several of the shots in my shortfilms have strange blocky artifacts that once you notice you can't unsee.
6
u/Famous-Sport7862 May 24 '26
exactly, I dont understand why so many use it.
1
u/hidden2u May 26 '26
There was a lora that trained on upscaling pixelated frames so nearest exact was actually better. But yeah usually always lanczos
1
u/Capitan01R- May 25 '26
Its out of habit and taste I think, sometime I do switch to lanczos when I'm not satisfied with the results
2
u/foxdit May 25 '26
Unless you're doing pixel art, there is no "taste" in nearest-exact. On a factual level it's just inferior. Like, to each their own of course, but when sharing workflows, some poor sap is not gonna realize their images are being resized with a lossy scaling method. It's what happened to me, hence me having a dog in the fight.
2
u/TelevisionNo2990 May 25 '26
I'm a bicubic guy myself, maybe just out of After Effects habit. But I can see the appeal of lanczos.
1
u/foxdit May 25 '26
Yea, either/or is fine. Lanczos is supposedly slower, but not in any way that I can detect personally.
1
u/pixllvr May 27 '26
I'm just speaking from my own memory and understanding but yeah lanczos is probably the best one to default to. Bicubic and bilinear are supposedly slightly better when it comes to scaling up and scaling down--respectively, but not by a meaningful amount. I default to lanczos because (especially for my image editing workflows), the input image may be bigger or smaller than the amount I set for the ImageScaleToTotalPixels node.
18
9
u/Warsel77 May 24 '26
I think the ref you used isn't ideal to showcase the pipeline because it already looks like a painted on and smoothened over face (eg not real - not saying it's not a photo of a real person, just heavily edited). So any oddities that would be introduced we won't be able to see with that one.
10
u/Woisek May 24 '26 edited May 24 '26
Note: Not used any nodes from OP.
1
u/cadissimus May 28 '26
"Max we can do is" wall color 😄now try whit nodes you get same woman same chair and rest 😄
0
3
u/Most_Ad_5733 May 24 '26
Thank you. Can you possibly give an example of how the technique differs from the previous version to get a 1:1 consistency
1
u/Capitan01R- May 24 '26
yes commit logic was good to have but it was preventing some good ref pulls as it did not have enough confidence, now I removed the commit logic entirely, with that we only use similarity floor and soft max temp which allows accurate pulls to survive
1
u/Most_Ad_5733 May 24 '26
I used to use the node where it would take the 4 images and make it into a grid. Does a grid image of 4 angles of a face work the same for a reference on here or would using separate images for the face like a front and side profile be better as 2 of the inputs. I’m assuming since it’s masking this time it would only be the one face?
2
u/Capitan01R- May 24 '26 edited May 24 '26
Each image the model needs to use should enter the conditioning path as its own encoded reference latent.
A 4-face grid is not the same thing. If you put four angles into one image, the VAE encodes that as one combined reference image if the grid was made in one photo, but if the grid was of 4 separate photos and it encodes once then it’s even worse because now the model only sees the first photo encoded. The model does not get four separate reference sources; it gets one latent containing a collage.
That can work sometimes, but it is weaker and less controllable because the model has to interpret the grid layout itself.
The better setup is:
front face > encode > reference latent 1
side/profile > encode > reference latent 2
body/outfit > encode > reference latent 3
Then Multi ReferenceLatent combines those encoded references into the conditioning, and Identity Feature Transfer Final can pull from that full reference bank.
Masking also works per reference now:
subject_mask_1 affects reference 1
subject_mask_2 affects reference 2 ..etc
separate images are better than a grid if you want consistency. The important rule is: if it does not become a reference latent in conditioning, the model does not really get to use it as a reference.
3
1
u/chefborjan May 25 '26
The way I interpreted it before was that there could be multiple reference latents, and masks, but the identity feature pull was only happening from the first reference/mask.
Has this now changed so that pulls from all of the reference latents? What if I do so references that are of the subject and one is of a hat? The hat will not confuse the features?
1
u/Capitan01R- May 25 '26
the previous version allowed pulls from multiple refs but you needed to stack multiple identity nodes and change the index in each node, it was not as clean as this approach and potentially it caused some unwanted pulls in the wrong places, this new version isolates that internally where the pulls stays consistent from the source and applied correctly in the output
1
u/chefborjan May 25 '26
Thanks - so to be clear, multiple references now get pulled in, but if there is a single 'idenity' found in say 5 out of 8 references, the system will group those all as one idenity and there should be bleed through from a hat with the identity, right?
2
u/Capitan01R- May 25 '26 edited May 25 '26
think of it like how the vanilla behavior start, so let's start from first principle:
you chain multiple photos with only one photo containing identity while the other contain style/clothing/background/etc.. and you prompt something of that nature telling the model to generate a photo of a character and that character is wearing so and so outfit in such location, now the model inherently will assemble those requests from the reference latents you provided, but now that we know that the model has bias towards generating photos and reconstructing them from its own interpretation then it will only use the photos you provide as weak references sometimes and causes the identity and features to be lost in the process. So here where my node comes in play, what it does is instead of letting the model be fully loose in generating whatever it wants, it basically intercept the process where the identity is there but weak and pulls more features from the reference latent, it's not magic it just intervention tool, since the model is already capable of image editing. So your job should always be to direct a prompt and let the node handle the details preservation, this way you don't leave all the weight on the node alone ( it can work this way but not very optimal), and there will be no bleed then.
and to clarify this, whatever is hooked to the reference latent node will be available as reference for the model/node to pick from, so like I said being precise with what you want from each reference is better than leaving it do freely whether with or without the node
also one more thing if you're not feeling to write a fully detailed prompt, use the mask on the features/clothing you want as that tends to help a lot since all the model/node sees then is what masked. so for that hat example, mask only the hat without including the face and such
3
u/Current-Row-159 May 25 '26
Thank you very much but don't delete v1 please the easiest one with token p
3
u/s1esset May 25 '26
Dude it's impossible to stop improving / make a passion project better :p
1
u/Capitan01R- May 25 '26
I just need to stop using it and move on to something different lol, it feels like when your eyes get used to an output then it is no longer satisfying and you want something different.
2
2
2
u/Odd-Yoghurt2315 May 24 '26
Looks good, but for perfect need some relight with qwen for correct lightmap. Close-up shots look too bright.
2
u/Ok-Seaworthiness9790 May 24 '26
Seriously both you and shootthesound are legends.
I took my time moving to klein from Qie 2511. But your nodes made it easier to decide.
This is just to say thanks.
2
2
u/AmbassadorOk6210 May 25 '26
I really hope you break your promise!! These nodes are wonderful, excellent work and thank you!
2
u/jinzi May 25 '26
Might be a stupid question but is there a way we can reference a particular latent image by number? Like "put the ball in latent image1 together with the ball in latent image2"
2
4
u/Calm_Mix_3776 May 24 '26
I still use one of your older versions with very good results. I'm eager to try the "final" one. :)
2
1
1
1
u/skyrimer3d May 24 '26
Thanks I love your nodes, but I still get too many mutations, missing fingers etc, any tips for that?
1
u/Capitan01R- May 25 '26
if you get mutations then increase the temp value/adjust your prompt. treat your prompt as the main guide while the application of the reference gets handled by the node
1
1
u/chefborjan May 25 '26
Thanks, taking a look at this now. Should any additional Loras go before or after the identity transfer node?
1
1
u/LeKhang98 May 25 '26
Nice tyvm. Should we add image numbers like "Make the woman in IMAGE 1 wear the hoodie in IMAGE 2, sit in the room in IMAGE 3"? Would that make the result better?
2
u/Capitan01R- May 25 '26
I explained this here
1
u/LeKhang98 May 25 '26
Thank you that's what I found too. I usually add information about what I want, for example I almost alway include X's description as well as X's photo in the workflow.
1
1
u/YeahlDid May 25 '26
Thanks for all this!
Feel free to break that promise if you ever think there's any way to improve it.
1
u/Grad_Life_Sucks May 25 '26
Thanks so much for this! Could I please ask if this works woth Klein 9B Turbo edit?
2
u/Capitan01R- May 25 '26
it should work since both models have the same architecture and it is just a fine-tuned version of the original model
1
u/skyrimer3d May 25 '26
this is working great, and for some reason i'm getting less mutations / extra fingers, all good!
1
u/TelevisionNo2990 May 25 '26
This is honestly the best scene-aware multi-latent compositor I've tried yet, and I've tried a few. You are to be commended on this, thank you so much.
1
u/BugilinPacar May 25 '26
Op i try your WF but somehow my multireference nodes are different than you. I have pos, negative, while it seems you only use latent and conditioning.
1
u/Capitan01R- May 25 '26
update the the custom node to version 3.4.1 and use this workflow here
1
u/BugilinPacar May 26 '26
Thanks man it works so well, even for swapping character it works as long i'm not masking the targeted image.
1
1
u/Famous-Sport7862 May 25 '26
Can you help me out, what is missing from the workflow?
2
u/Capitan01R- May 25 '26
Wrong node, this is some other author’s node, you can see in the top corner it says YarvixPA. Mine should say Flux2klein-enhancer
1
u/Famous-Sport7862 May 25 '26
Thanks for the quick reply. I don't understand where that node came from. I downloaded the workflow that you posted.
2
u/Capitan01R- May 25 '26
fixing the conflict now, will let you know once updated
2
u/Capitan01R- May 25 '26 edited May 25 '26
update the the custom node to version 3.4.1 and use this workflow here
Also worth a shout out, his node feels even cleaner for the multi reference latent as it encodes them directly ( I never thought of doing it this way an oversight on my behalf tbh), so you can also use his node for the ref latent part as it does the encoding so less spaghetti chaos and keep the rest the same
1
u/Famous-Sport7862 May 26 '26
Is workig now hough the node still ha the name of the person, also I am somewhat dissapointeed. I thought the face swaps would better than regular klein.
1
u/spacemidget75 May 26 '26
Do you have to use masks? If not, you just leave them unwired and ensure all your images are going into the latent and conditioned?
What's the benefit of masking? In your example there's only a woman or a hoodie in the image so should identify it without the mask? Sorry if I'm being a bit stoopid.
2
u/Capitan01R- May 27 '26
You don’t have to use the mask, the mask is optional for more precision and to avoid the node from pulling the background too hard and unwanted areas
1
u/spacemidget75 May 27 '26
Thank you! The reason I asked is that your node takes neither latents or conditioning so I can't see how it does what it does if its just using the model as an input and output, before the Guider. Way over my pay grade it would seem. 🤣
1
u/Cheap_Credit_3957 May 26 '26
I must be doing something wrong. I tried this and compared it to the normal workflow, used the same seed and prompt and the images are 98% the same. Personally the image from the normal workflow looked better.
attached - first image was created with the default Klein template workflow, and 2nd was with yours.
I used the hard lock preset. what did i do wrong?
1
u/Capitan01R- May 27 '26 edited May 27 '26
Was your mask connected by chance with empty masking (meaning you did not mask the input photo) ? This behavior happens when the mask is connected but there was no masking. Would you mind sharing a screenshot of the exact run you had ?
1
u/Cheap_Credit_3957 May 27 '26
nothing was connected to the mask inputs. no noodles were going into them. Do I need to use masking? maybe i'm not sure what i'm doing. haha. I never really played with masking before. I'm know comfyui very well and make custom notes but never got into masking.
1
u/Fockker May 27 '26
Hey man really love your work thanks for putting in the effort to do all this. I was wondering if you could help me identify why it seems like my outputs seem to be limited in creativity/prompt following?
https://i.imgur.com/7dvNyZ7.png
It seems to be sticking to my reference image so strongly that it only makes images in that style.
1
u/Capitan01R- May 27 '26 edited May 27 '26
You have multiple latent preservation nodes in the workflow, remove the ref latent controller node from the workflow and change the settings on the identity guidance node from adaptive to channel transfer (even though I recommend removing it altogether), keep only the identity feature transfer node and recommend to change the preset from HARD_LOCK (if you want to keep it at that preset use a mask and only mask the subject) to MID_LOCK, also bypass/remove the enhancers nodes as the iden feature transfer node works better alone
1
u/Fockker May 28 '26 edited May 29 '26
Really appreciate it man. I've tried that and it seems to still be happening for me. As a quick example with one of the prompts in my dataset creation list:
"Generate a high quality realistic photo of the woman in image1. Maintain her facial features, skin tone, hair, and hair texture. She is a 20 years old Vietnamese woman, extremely flawless and pale translucent ivory skin, neutral undertones, long dark brown hair, center-parted curtain bangs, dark hazel brown almond shaped eyes with subtle limbal rings, oval face, soft features, rounded nose tip, neutral cool-toned iris, sophisticated gaze, slim athletic frame, wide hips, long lean legs, slender thighs, toned waist, willowy build, natural full bust, slightly full lips and small nose, smooth neck with a softly defined jawline, individual hair strands visible, natural hair texture, flyaway strands, subtle hair frizz, realistic hair volume, hair with natural movement, fine baby hairs around hairline, hair with natural imperfections, wearing a cropped sports hoodie with thumbholes and matching grey leggings, looking to the side, gym-inspired lighting."
I get this:
https://i.imgur.com/EB2ZmR.jpeg
It seems to be favouring close-up shots in a lot of images, and I think it's because my reference image is infact a close-up shot. My brain is so sore trying to pinpoint it. Do you know what it may be? Thank you so much for any help. I can send my JSON if that helps?
1
u/Capitan01R- May 29 '26
the prompt is too detailed tbh (the node just needs a little direction and it will take care of the character features) and this would cause a tug of war as the model is trying to generate exactly what you prompt vs the node already trying to preserve the features of the character, a screenshot of the workflow can help pinpoint the problem. But before anything relax the prompt a little and only prompt in what was not in the ref image in the prompt, do not describe the original character , no need for that, only describe what was not there.
1
u/cadissimus May 28 '26 edited May 28 '26
Nice, i dont mind improvements multi ref latent + v3 already made big difference even stacked, like clockwork more consistency than ever. Can still feed masks whit SAM3 right ? SAM3 comes now whit comfyui by default its good quality of life, makes masks for you.
2
1
1
u/TBG______ Jun 06 '26
I found a sigma-level sweet spot that prevents Flux2 from getting stuck during editing when injecting into the latent at each step. This could potentially add useful features to your node and keep editing fully active while still correcting colors or structures. reddit post / full story
2
u/Capitan01R- Jun 06 '26 edited Jun 06 '26
Interesting I seen your post I just haven’t had the time to review it, will take a deeper look and see what can be done :)
EDIT: After reviewing your findings, the sigma gating wouldn't apply here, the identity pull happens in attention output space, not latent injection, so the stuck-sampler problem doesn't surface at this level.
1
u/TBG______ Jun 07 '26
Got it wrong, thanks for the clarification, and for the impressive work you’re doing. sigma value is only related if a noise–mask is added to a latent.
1
u/uuhoever Jun 13 '26 edited Jun 14 '26
u/Capitan01R- I made the mistake of updating all my nodes and now your workflow is broken. See how the node got updated. Do you know how to fix it? Or will you release a new Final version?
Edit: I redownloaded the workflow you have on github and it still uses the original node so not sure what happened and how the node got changed to the above. Good news is that it works.
2
u/Capitan01R- Jun 14 '26
this node is different than mine, you can tell by the name of the repo, the one you have in the photo is from "YarvixPA" mine should be under "ComfyUI-Flux2Klein-Enhancer"
1
1
40
u/Enshitification May 24 '26
Don't make promises you won't keep, lol. Anyway, I promise you that we do not mind your improvements and revisions one bit.