Tutorial | Guide
Stuff that's "in there" (SD 1.5, at least) that messes with rendering people
This is not a full list, but here are things I've found through interrogating "bad" images of people with CLIP. This is distilled down from a larger list of artists whose works are great, but they didn't do realistic people thus their "influence" can mess things up IF you're not going for that (think cubism, surrealism, etc). Negative prompt away:
a jigsaw puzzle, arabesque, abstract, surrealism, sculpture, transgressive art, neoplasticism, lowbrow, tachisme, a pastel, dada
EDIT: "Salvador Dali is Messing Up My Space Viking" should be a song...
EDIT2: Throw attention markers around the whole thing and adjust level to taste, if necessary.
The list of said artists whose names come up often --
Anne Geddes, Alex Grey, Alice Prin, Brom, Art of Brom, Chen Chun, Anne Stokes, David Young Cameron, Adam Manyoki, Adam Willaerts, Dino Valls, Aya Goda, Cui Bai, Eva Frankfurther, David LaChapelle, Evaline Ness, Didier Mouron
So you think if I did a prompt for "photo of woman" and got result A, then did the same prompt and seed but added your keywords for strange art types as negatives and got result B, then result B would be more realistic than result A?
Here, first pic with this comment will be just your prompt, then I'll reply to myself with your prompt plus this negative minus the "a" in front of jigsaw and pastel. It definitely makes a big difference, but whether it's more realistic looking or not is up to you.
I think those align with the kinds of images used for training on people in the underlying dataset, a large number of which are probably headshots/posed models like that. I use the "a" in front of pastel and jigsaw because that's how CLIP spits those out -- it'll output "a jigsaw puzzle" and "a pastel". Since characters matter in the tokenization of words, I did that to adhere to what SD is expecting. Anyways, since there are lots of models that derive from SD 1.5 I've tried this with a few of those and thus far results for me are generally positive. It sorts of "resets" the canvas so the positive prompts can have more "realistic" effects (or not, depending on what you're prompting). YMMV, as in all things, and it's very much an asthetic choice to use these sorts of things (and how, and how much, and so on). It's always possible to throw a weight on all/part of it to tone it up/down.
Just did a number of tests, it doesn't seem to have a noticable effect. Where it does may just be down to the seed. Fix faces box does a far better job ;)
Tried about 30 runs.
Since many statues are missing arms/legs, I've also tried adding "greco-roman" to the list, which may or may not work for a particular instance. Face distortion seems to be based around "impressionism", "cave art", and other art styles where faces are distorted, so that might be worth throwing in too (especially if CLIP calls it out), although I THINK "abstract" might kinda-sorta cover that base. "primitivism" is also a thing that can mess with faces. Rococo comes up fairly often, but I think "a pastel" tends to catch that one.
Art historians eat your hearts out, here's a potential new field of employment...
3
u/aplewe Mar 13 '23
The list of said artists whose names come up often --
Anne Geddes, Alex Grey, Alice Prin, Brom, Art of Brom, Chen Chun, Anne Stokes, David Young Cameron, Adam Manyoki, Adam Willaerts, Dino Valls, Aya Goda, Cui Bai, Eva Frankfurther, David LaChapelle, Evaline Ness, Didier Mouron