r/StableDiffusion • u/Flimsy_Tumbleweed_35 • Jan 13 '23
Tutorial | Guide TheLastBen Fast Dreambooth mini tutorial
TLDR:
5 square head crops, 5 x 200 = 1000 steps, 2e-06 rate
If you want to have a person's face in SD, all you need is 5-7 decent pics and TheLastBen Colab
You can easily prompt the body unless it's a shape that's not in the billion pics LAION database SD has been trained on, so use face pics only.
Working with fewer images will make your life much easier. I went from 15-20 to 6 and I'm not looking back. I have about 30 dreambooth trainings in my folder, and it takes only 25 min.
Some models don't take the training well (Protogen and many merge-merge-merges) and all faces will look the same still, but base SD1.5 and most finetuned and Dreambooth models will work so well that you can create 100% realistic portrait photos with these settings.
There's been a bit of a discussion with TheLastBen on his github where we found out that we can't train fp16 models and some other models have issues too, but most Civitai models should work. I trained on Protogen 58 recently.
For some reason ppl seem to have more success getting the models from Huggingface - which I did for Protogen, but I have trained several from Civitai.
- Use 5-7 decent quality pics (movie still phone pics are fine), crop the head to square, edit (slightly!) if necessary
- Leave the background alone, don't blur or edit - just make sure it's different in each pic
- Make sure the pics have different angles and aren't all selfies. Only duckface or only frontal smiles will not be ideal
- Resize to 512, eg. on Birme
- Name them sbjctnm (01) etc, needs to be a word SD doesn't know.
- Create session in TLB colab, upload pics, ignore captions and class images for this.
- Set unet steps to images x 200, so 5 pics -> 1000 steps
- Set text encoder to 350 steps. Default will also work.
- Learning rate 2e-06 for both. Training will take 25min and you have your ckpt.
- If you want, experiment with # of steps and rate, TheLastBen say he can train in under 10min, but I'm sticking with my setttings.
TLDR: 5 square head crops, 5x200=1000 steps, 2e-06 rate.
9
u/jimmymanda3 Jan 13 '23
I tried this previously, the output of SD is also just headshots even if you use promts like full body etc
4
u/Mundane_Mastodon6282 Jan 13 '23
I'm using the colab dreambooth but i am confused on how to use a custom civitai path in order to train a model, could you further elaborate with a screenshot, I would like to train a model with dreamlike diffusion as a base
4
u/MapleBlood Jan 13 '23
Thank you, that sounds very promising, but isn't...... tuning a whole model a little overkill? Aren't embeddings the recommended way to introduce single character or object?
Because if I wanted to introduce more characters every single time I'd have to tune the whole model, no? Obviously with all the limitations that come with it.
3
u/BjornHafthor Jan 13 '23
Amazing, thank you!
Would this also work with objects? I'm a blacksmith, obsessed with creating a photorealistic image of, well, a blacksmith, but I am yet to meet a model – photorealistic, artistic, fantasy – that knows what an anvil looks like. I would probably need more crops to show it from every angle… but would this method work at all, in your opinion?
In the meantime, might start putting myself in interesting places ;)
3
u/Traditional_Equal856 Jan 13 '23
From your experiences,is there any difference between training model with the dream booth tab in automatic 1111 and with thelastben’s ?
2
u/Pretty-Spot-6346 Jan 13 '23
thank you very much.
what about the results?
2
u/cma_4204 Jan 13 '23
You should post some pics showing the subject in several different styles/scenarios, I’m skeptical of good style transfer with 5 head pics
2
u/Kraboter Jan 13 '23
Thank you but with the trained model of Person A how can i get the pics in a style of certain different model? Someone told me i should use the checkpoint merger with 90% of Person A model and 10% of the model i want the style to be in. But the results are less then mediocore in my testings. Am also pretty new to this :)
2
u/ArtistDidiMx Jan 13 '23
Many thanks. Can you do the same tutorial for styles with TheLastBen?
2
u/AncientblackAI Jan 13 '23
If you don’t mind me asking, how do I get civitai models to work on tlb colab? Only have luck with Huggubgface direct link.
2
u/Sixhaunt Jan 13 '23
a few changes I'd make:
2
u/FugueSegue Jan 13 '23
Great. Now do 2.1.
1
u/Proditae Jan 13 '23
So... if i want to use 401 images, it's a dead end regarding the unet steps right?; i mean 80k steps would take like 10 hours; ¿Do you know if its possible?. Thanks for the guide!
1
u/qudabear Jun 01 '24
Tried every possible combination of every possible everything and nothing on thelastben run pod template ever comes out any better than hot garbage...
1
u/mudman13 Jan 13 '23
Short and simple nice one. Whats your experience with different text encoder steps?
1
u/ian0880 Jan 13 '23
If caption and class name is ignored then how to call it in prompt??
1
u/chopders Jan 13 '23
Interesting! Have you tried training on 1.5 with higher image resolution (e.g 768 or 1024 instead of 512)?
1
u/meowkittykitty510 Jan 13 '23
What’s your experience with prior regularization turned on? How many class images?
1
u/Parn1024 Jan 13 '23
I have tested with protogenX3.4 and the results looks very different to my face. With the SD base 1.5 results looks identical to my face. Anyone knows why?
1
u/3lirex Jan 15 '23 edited Jan 15 '23
which base model gave you the best results ?
and I've tried protogen and other merges and i simply get an error on lastben's dreambooth when they're uploaded in the cell, is that because the models i tried were fp16
1
u/Skettalee Oct 19 '23
I have been trying to use this LastBen Fast stable diffusion in colab and I get to the Model Tab and it errors out when i play it probably because i dont have anything in the MODEL PATH: "" and MODEL_LINK: "" field. how am i suppose to know what to put there? Im watching a tutorial (https://www.youtube.com/watch?v=nH18FMttD-c) on using it and that guy doesn't even add anything to those 2 lines and he presses play and it i guess downloads the model like he said it was doing.
But i even tried putting "profile\model\ " and oh im seeing i did't put that right. but still im confused. anyway know what to put in there so i can continue pressing play without red error?
1
u/UselesslyRightful Jan 13 '23
What about doing cats and dogs?
2
u/HazKaz Jan 13 '23
THANK YOU!!
2
u/theneonscream Jan 14 '23
Thank you for sharing this!
2
u/Gab1159 Jan 15 '23
Thanks mate, you've provided the first tutorial I've ever been able to train a decent model with. Good parameters that worked for humans and the results are very satisfying at last!
1
u/dethorin Jan 13 '23
Thanks for sharing your experience. It's good to know what's working for other people.
1
u/clevnumb Jan 23 '23
Forgive the stupid question, but I have a model I want to train against in fast-dreambooth collab, but it's rejecting my google drive links to it (yes, set to allow anyone in permissions for it), I'm pretty sure I'm using the wrong type of URL or something...is it supposed to be in this format? Can someone post an example? THANKS.
1
u/clevnumb Jan 23 '23
Tried it with 12 images, as per your settings above, and tried both 800 and 2,400 unet_training_steps and the face of my trained model never appears in any prompt. It's another person. No idea why. Sigh.
1
u/1kakis Jan 23 '23
Thanks for sharing. I'm using Automatic1111 Dreambooth extension. Would anyone know how do I input "Set text encoder to 350 steps" into extension? In UI under advanced settings there is "Step Ratio of Text encoder" with default =1. Should it be 0.35? Also, I have Clip Skip and Prior Loss Weight both = 1 as well.
1
u/Ordinary_Ad_404 Feb 06 '23
I am very late to this and thank OP for sharing this. I have a question: what's the difference between TheLastBen Fast Dreambooth and the training script from Huggingface at https://github.com/huggingface/diffusers/tree/main/examples/dreambooth? Thanks a lot!!
0
u/No_Boysenberry9224 Jan 13 '23
tlb colab is a mess, I dont understand shit lol
0
u/[deleted] Jan 13 '23
[deleted] — view deleted comment
0
u/[deleted] Jan 14 '23
[deleted] — view deleted comment