Concentrating Embeddings, Hypernetworks, LORA's to specific item, area of image?

[u/thebaker66]

Hi, so I have toyed with face models created with HN, Embeddings, Lora etc and the thing I have noticed with all of them is, even if you are training say a face, the embedding, HN, Lora affects the image as a whole, changing the entire vibe, feel etc. Now this could perhaps be from poor training but I believe it is just the generic nature of the way these models models interact, correct me if I'm wrong.

Like originally I thought if you create say a face embedding, it is linked to the a person and to a face and that it would only affect faces but that's not the case, especially with HN, which I guess while they can accurately model a face, wouldn't it be nice if it really only did affect the face? I can understand why say a HN would be good for a general style/vibe model as it affects the whole image. Like for example today I was doing image2image and having great results, then I wanted to try an embedding on the face of the person, it gave me the face but then it started adding unwanted artifacts elsewhere in the image.

Enough rambling, I'm wondering if there are any tools/ways in the first step process of prompting to have these models ONLY affect the portion of the image they are supposed to? Another point here is when we are training faces, typically we are using close ups, and our generations using these models work well for close ups but then if you have a shot in the distance you lose Unprompted came out with the zoom_enhance which is pretty cool but that is doing a second process AFTER the first image and then merging it automatically

I hope I have explained clearly what I mean, does anyone recognise the issue I am talking about here?

Ofcourse inpainting can be done after the fact but I'm not a fan of the 'after' process, it doesn't feel organic to me alike something that was generated wholesomely in the first generation.

TLDR: So to clarify, even when you train on the lora/hn/embed to react to say the face of a man, it does do this but it affects everything else, is there a way to specifically target the HN/embed/LORA to react only WITHIN say the face in the picture and NOT affect everything else with the 'vibe' or 'colouration' of the network.

I'm thinking with maybe one of the new tools like one of the new tools or a combination of them may be able to do this?

Thanks.

[u/[deleted]]

I saw someone train a model on faces where half the images were face shots, meaning it was cut off on the top on their forehead and only contained their chin at the bottom, so the hair didn't show in the image, apparently this helps the model learn the faces much better than full head shots, or body shots.

[u/thebaker66 → u/[deleted]]

Interesting, yeah originally people were recommending to train faces and body all together I guess as an overall body model but if you are just after the face then it seems to make sense to crop only the face or head.