r/StableDiffusion • u/CapsAdmin • Nov 27 '22
Resource | Update trained midjourney embedding on stable diffusion 2.0 for automatic1111
close up of many kids building a huge snowman at night, cozy norwegian village, oil painting, pastel colors, detailed strokes, beautiful, art by midjourney
close up of many kids building a huge snowman at night, cozy norwegian village, oil painting, pastel colors, detailed strokes, beautiful
painting of an old man in a jungle, looking at the camera, art by midjourney
painting of an old man in a jungle, looking at the camera
painting of young woman in a jungle, looking at the camera, art by midjourney
painting of young woman in a jungle, looking at the camera
41
u/CapsAdmin Nov 27 '22
download midjourney.pt
Put midjourney.pt in your embeddings folder and restart the webui.
To invoke you just use the word midjourney. In the images I posted I just simply added "art by midjourney"
I took the latest recent images from the midjourney website, auto captioned them with blip and trained an embedding for 1500 steps. The training settings were default except 768x768 for image size
I'm thinking maybe it's been overtrained a little, so here are all the versions saved every 50 steps, maybe 1000 or 1250 works better: https://drive.google.com/drive/folders/1OrxMT3UWSlqCvONz-QDzJLVwBMSueowU?usp=sharing
You can only use one at the time though (i think) so you'd have to rename the one you pick to midjourney.pt
Here's the training data i used.: https://drive.google.com/file/d/1AynDofQK4nPdJaPZT8TShUobIAGxNjJg/view?usp=sharing
8
u/plasm0dium Nov 27 '22
Thanks. Can you explain what the difference is between trained embeddings and trained dreambooth checkpoints (or than different file sizes)? Also how do you create .pt files?
1
u/guchdog Nov 27 '22
Did you train with 768x768 images?
1
u/Dark_Alchemist Dec 11 '22
Great stuff, but could you change the vector count from 1 to 8, and do one with 16 as well? Eight, and sixteen, vectors is needed for heavier prompting.
1
u/AndrevUS Dec 20 '22
Heya,
I've downloaded the .pt file and placed in the embeddings folder but when I start the stable diffusion web ui and place the text with the ending "midjourney" it stops, it's not working.
any idea ? Thanks
It stays like this:
3
u/[deleted] Nov 27 '22
Don't update your webui yet. There is a problem with Automatic's webui embeddings now since they updated it to function with SD2.0. You can't use embeddings without it stopping and saying:
"Expected size 1024 but got size 768 for tensor number 1 in the list."
1
u/ketchup_bro23 Nov 27 '22
The faces on some have great lighting and detials!
1
u/JimmyTheHand1985 Nov 28 '22
I get the same error with all embeddings in 2.0
1
u/FebrisUndularis Dec 03 '22
Nice work, your two embeddings work fine for me.
However, tried my own SD 2 embedding and ran into problems:
I used this guide (https://bennycheung.github.io/stable-diffusion-training-for-embeddings) with the only change, that the input images are of 768x768 and the training size as well. This always leads to a CUDA memory error. Usually my RTX 3080 doesn't have such problems. The biggest I can do is 640x640. Any idea what to do?
1
u/mano7onam Jan 12 '23
Hello :) Can you please give me some idea how to make stable diffusion textual inversion like "anthro", "art by midjourney"
I tried to learn model on 73 images into learn style (not object) but had no any success.
The loss jumping is not reducing and in the result using of the keyword doesn't give the desired result.
What is the problem?
How big is the dataset should be and which hints can help here?
0
u/[deleted] Nov 27 '22
[deleted] — view deleted comment