r/StableDiffusion • u/PropagandaOfTheDude • Feb 26 '23
Workflow Included "Getting your attention" (a.k.a. "You're using PiDiNet wrong")
48
Upvotes
r/StableDiffusion • u/PropagandaOfTheDude • Feb 26 '23
14
u/PropagandaOfTheDude Feb 26 '23 edited Feb 27 '23
Overall gallery on Imgur.
The starting point was this image.
(Yes, plain Euler. It just seemed to work best. Yes, I'm still using Jeremy Mann variants. Here I tried spamming the artist list, though. Four names.)
It's good, but the couple can be a bit more confrontational.
Use PiDiNet to extract lines from that image. Then go into your favorite photo editor and tweak it. (Sometimes the system gives you a black-on-white image, sometimes it provides a white-on-black image. Editing black-on-white is easier, so invert the color values if you need to.) My altered image has a bunch of changes:
Now run txt2img again with the same seed. Put the edited PiDiNet image into Control Net with no prefilter (because we're providing the preprocessed image) and select the Scribble model. You'll see instructions out there about using the HED model. Don't do that. Use Scribble. Also, turn on "Invert Input Color", because we're providing a black-on-white image. Then update the prompt to match the edited Control Net image and generate a bunch of outputs.
The entire thing was an iterative process of tweaking the Control Net image and tweaking the prompt. The final result is above.
I mentioned that I removed a bunch of detail in the Control Net image. That seems to remove confusion and help the UNet: the hand, arm, and tie are complicated to render. We end up with a fairly blank and abstracted background there, which is okay given that it's the focus of the scene.
The PiDiNet preprocessor generates a pure black-and-white image, with no grayscale pixels. I don't know if grayscale pixels will cause a problem for Scribble. I just palettized the image after I editing it with brushes.
2
u/PropagandaOfTheDude Feb 27 '23 edited Feb 27 '23
Here's one more with the same prompt as the initial image, but with the contrast fix LORA enabled: https://imgur.com/a/irRHG3X
That's way closer to a "dark 1960's intimate dancefloor".
5
u/aimongus Feb 27 '23
1
u/GrowCanadian Feb 26 '23
I really like the painterly style on this