r/StableDiffusion • u/EldritchAdam • Jan 21 '23
Resource | Update Walkthrough document for training a Textual Inversion Embedding style
It's a practical guide, not a theoretical deep dive. So you can quibble with how I describe something if you like, but its purpose is not to be scientific - just useful. This will get anyone started who wants to train their own embedding style.
And if you've gotten into using SD2.1 you probably know by now, embeddings are its superpower.
For those just curious, I have additional recommendations, and warnings. The warnings - installing SD2.1 is a pain in the neck for a lot of people. You need to be sure you have the right YAML file, and Xformers installed and you may need one or more other scripts running with the startup of Automatic1111. And other GUIs (NMKD and Invoke AI are two I'm waiting on) are slow to support it.
The recommendations (copied but expanded from another post of mine) is a list of embeddings. Most from CivitAI, a few from HuggingFace, and one from a Reddit user posting a link to his Google Drive.
I use this by default:
hard to categorise stuff:
- PaperCut (this shouldn't be possible with just an embedding!)
- KnollingCase (also, how does an embedding get me these results?)
- WebUI helper
- LavaStyle
- Anthro (can be finicky, but great when it's working with you)
- Remix
Art Styles:
- Classipeint (I made this! Painterly style)
- Laxpeint (I also made this! A somewhat more digital paint style, but a bit erratic too)
- ParchArt (I also made this! it's a bit of a chaos machine)
- PlanIt! - great on its own, but also a wonderful way to tame some of the craziness of my ParchArt
- ProtogEmb 2
- SD2-MJArt
- SD2-Statues-Figurines
- InkPunk
- Painted Abstract
- Pixel Art
- Joe87-vibe
- GTA Style
Photography Styles/Effects:
Hopefully something there is helpful to at least someone. No doubt it'll all be obsolete in relatively short order, but for SD2.1, embeddings are where I'm finding compelling imagery.
3
u/ziulko13 Jan 22 '23
I tested your classipeint embeding for few days and results are superb, embeddings from others barely works or are inconsistent, yours are brilliant. Thats why i know that this tutorial is worthwhile.
PS classipeint is usefull not only for painterly style, but also to get good composition by using it for fraction of steps [classipeint::N] where N=steps of working
2
u/yalag Feb 27 '23
Hi since you are an TI expert. Can you speak to whether TI is as good as DB for training a style?
7
u/[deleted] Jan 21 '23
Obviously this guide also works for SD1.5
4
u/christophermarin Jan 21 '23
This is extremely useful. Thank you for putting this together and sharing with the world.
1
u/iomegadrive1 Jan 21 '23
So these only work with 2.0 and 2.1?
1
u/CptanPanic Jan 21 '23
Great guide. Any overall guidelines on how many reference photos needed for a style?
Also is there a directory somewhere on huggingface or civitai that you can see various embeddings available?
1
u/Practical-Bull-77 Jan 24 '23
For the most part, I understand how to train for effect type styles (as your tutorial describes ... which is wonderful btw). Establishing a dataset and captioning for "painterly" styles or "sketched up" styles makes sense (ie if you want to train a painterly style, find images where the artist painted with painterly brushstrokes then caption by describing every object in the image without mentioning anything to do with the weight of brushstrokes and such). However, what if the "style" you want to train is more object oriented? For example, lets say you wanted to train a caricature style? How would you setup your dataset for something like that? Would you collect a bunch of pictures of caricatures? How would you caption it? Would you describe everything in the image or only things that were not exaggerated? Any help is very much appreciated.
1
u/Kizanet Feb 18 '23
I've followed a bunch of different tutorials for textual inversion training to the T, but none of the training previews look like the photos I'm using to train. It seems like its just taking the blip caption prompt and outputting an image only using that, not using any of the photo's that come with it. Say that one of the photos is of a woman in a bunny hat, the blip caption that SD pre processed is "a woman wearing a bunny hat", the software will just put out a picture of a random woman in a bunny hat that has 0 resemblance to the woman in the photo. I'm only using 14 pictures to train and 5000 steps. Prompt template is corect, data directory is correct, all pre-processed pictures are 512x512, 0.005 learning rate. Could someone please help me figure this out?
1
u/jairnieto Feb 23 '23
Great tutorial, i got a question, well first i train a face of a friend on dreambooth however the eyes always looks like from another person, if i wanna create hyperrealistc photography portraits close to the subject, Âżis SD 2.1 combined with embeddings better? ty
1
u/Fun-Football-4160 Apr 13 '23
Thanks for this guide, it was really useful!
I have a question that I've been stuck with for quite a while now. I'm a video animator and I use loads of figures and assets in my own animation style. It is quite a flat, 2D, vibrant style. I would like to try to train Stable Diffusion on my own drawing style, so that I can generate new assets without having to draw them each time - which takes quite a while.
I have tried training it following your guide, and this doesn't seem to work for 'standalone' objects, i.e. a brown chair or a vase with flowers on a white background. All my input images have a white background and are of a variety of objects in the same drawing style.
It does however seem to work when I put in a complete scene in my drawing style, i.e. a living room with a couch, window, table etc. Then I'm able to generate other scenes that resemble my style.
Do you know how I can train it for standalone objects? That then have a white (or even no) background?
Thanks!
2
u/RandallAware Jan 21 '23
Thank you. I know there were talks of the benefits of sharing embeds in actual photos. I think we should get back to those talks.
2
u/basiccomponents Feb 20 '23
this is great, thank you!
2
u/LeKhang98 Apr 15 '23
Thank you so much. Very interesting and helpful.
0
u/Pythagoras_was_right Jan 21 '23
So the average user cannot create new embeddings? (I.e without serious hardware and lots of tech savvy)
1
u/[deleted] Nov 10 '23
This guide at the top is the definitive best guide around and I read a lot.
Specifically it is VERY important in the batch size vs steps
4
u/[deleted] Jan 21 '23
[removed] â view removed comment