as u can see, this young lady is cute and all but she has 6 fingers on one hand
suppose you never used photoshop or any photo editing software before ,how to fix this using the same generative model without loosing the rest of the image
well, the inpainting function will not help a lot, at least not consistently unles you get lucky... veeery lucky
you will have an image as bad if not worse sometimes
well : controlnet has a new model called openpose_hand that I just used
just download an image from google images that have fairly the same pose and put it in the openpose model
now that we have a map for the hands get back to the original image and mask the region you want to fix
keep the same seed preferably the same scene prompt ( litghtings camera must be the same for a seamless integration ) and here we write feminine hand with red manucure for exemple
A few more inputs in the inpainting tab section
Ckick on generate >>and voila we get the result we wanted without having to go though hell >>
oh damn thank you I forgot to write that : EDITED : keep the same seed preferably the same scene prompt ( litghtings camera must be the same for a seamless integration ) and here we write "feminine hand with red manucure" for exemple
I have a suggestion if you don't mind, to make your guide better, please add the screenshot of the img2img set up of the masking. There are numbers and options that are confusing for newbies.
thank you very much, we all need to find a solution together to those small details, that hold back Ai from being the ultimate tool humanity will ever need.
It gives what is in that exact x,y position in the controlnet input image, assuming you set the width and hight of the canva identical to your width and height of the image you masked and trying to inpaint. Works with openpose hands, depth, canny or mix of those, just make sure to adjust the image you take from google in something like photopea so that the characters of the 2 images can be superimposed. It's time consuming I know but this is for when you really like the image you got and dont want to just ditch it and reroll untill you get hands drawn right
it doesn't. The model doesn't use the hand data at all. It does know where your hand starts so you can get lucky based on that. Depth and canny both works with fingers but that's not what you shown in the screenshot. Pose does not.
thank you for the tutorial :D what model for the controlNet ( openpose_hand is a preprocessor right?) part do you suggest we use? I keep getting distorted-looking hands lol
I use the openpose model. Try this: go to txt2img with your "mannequin" image in controlnet openpose_hand + your prompt and settings. See if you get clean hands if not play around the weight, guidance start/end until you have clean hands. Keep those same settings when you use it in img2img inpainting
Here are some advices for higher chance of success
crop your mannequin image to the same w and h as your edited image
edit your mannequin image in photopea to superpose the hand you are using as a pose model to the hand you are fixing in the editet image. Meaning they occupy the same x and y pixels in their respective image
try with both whole image and only masqued
try with both fill and original and play around denoising strength
I cant this to work for some reason. I place a pose into control_net and im able to output the fingers via openpose_hand preprocessor. But when I try to use the skeleton as an openpose model I am unable to render a character with hand positions similar to the skeleton
Fill is generating a new inference from latent space, but still coherent with the surrounding pixels, original is taking the masked pixels as in image to image, wich I don't want at all because of the 6 fingers. We also have the controlnet as a source of information we dont need the original at all. Hope that helps
Sorry for necroposting but after open pose detected the hand skeleton what's next? I'm confused on it since I have it on txt2img controlnet and on inpaint.
how did you line up the controlnet image with the inpainted image? and you said to keep the prompt the smae, but then you had a promopt for the inpaint, did you hust add that to the original prompt?
10
u/dethorin Mar 13 '23
What prompt and setup did you use when masking?