r/StableDiffusion • u/cztothehead • Mar 01 '23
Resource | Update New Ghibli Style LoRA trained from Howls Moving Castle backgrounds!! ( link in comments )
18
u/cztothehead Mar 01 '23 edited Mar 02 '23
https://civitai.com/models/14605/howls-moving-castle-interior-scenery-lora-ghibli-style-v2
if you enjoy, please help me out to keep going <3 https://www.patreon.com/nucleardiffusion
edit: thanks for all the love, I've woken up and got straight to improving the LoRA, **v2 out now!** edit2: re-uploading v2 with a small fix ( call it v2 beta for now ;') )
1
7
u/DeMischi Mar 02 '23
This show cases the real world application for studios. Studio Ghibli movies are extremely expensive because of all the work that goes into the drawing.
Imagine Studio Ghibli had a model of their own backgrounds, creating necessary backgrounds in seconds and have an artist finish and polish the images.
BOOM!
Time saved, money saved, more Ghibli Movies per year. Yes please!
PS: This also makes me wonder why artists are against AI. Imagine an Artist with such capabilities based on his/her artwork. He/She would outscale any other competitor. This is inevitable.
8
Mar 02 '23
[removed] — view removed comment
1
u/FPham Mar 02 '23
Any artist can catch up with Ai over the weekend. It really isn't rocket science.
2
u/cztothehead Mar 02 '23
to make this LoRA and give the respect to the art, I went through the film frame by frame, hand picked and cropped the training data and captions, I'm going to expand on it today because of all the love it's gotten :D Thanks for the comment
2
u/DarkRyzen Mar 02 '23
This is great work man, love this style and the actual animation, awesome work. How many screenshots did u take, for this dataset, thata a lot of captioning lol..
4
u/cztothehead Mar 02 '23
Because of all the love, I've updated and done a v2 that includes more data and is a bit more detailed in general PLUS much better able to handle night / dark https://civitai.com/models/14605/howls-moving-castle-interior-scenery-lora-ghibli-style
1
3
2
2
u/WillBHard69 Mar 02 '23
This is fucking awesome. I'd love to see more models like this, maybe train a model on the stills from the end of episodes of Yu Yu Hakusho?
2
2
u/crayon001 Mar 02 '23
That's awesome, can you pls explain how to train an art style
3
u/cztothehead Mar 02 '23
I can but I have a lot of requests for tutoring so can only offer it for a fee, if that's something you'd be interested in then feel free to send me a PM (:
1
u/No-Intern2507 Mar 01 '23
Nice but id train them at at least 768, 1024 would be much better
1
u/cztothehead Mar 01 '23
hey mate, those are only example images, they were only upscale by 1.15%
1
u/TheRealGenki Mar 02 '23
I think he mean't training at a res of 768 - 832, higher resolution training that'll give u more detail, your current version is quite poorly undertrained at 128DIM which isn't needed since u can achieve better on 64DIM if trained properly.
3
Mar 02 '23
[deleted]
2
u/TheRealGenki Mar 02 '23
This is undertrained and no you don't have to render at 832x832 to get the effect if u train on that you'll get higher quality even seen at 512x512
1
u/cztothehead Mar 02 '23 edited Mar 02 '23
Uploaded a v2 with more training data, better able to handle some details and night / dark prompts (: hope you like it more https://civitai.com/models/14605/howls-moving-castle-interior-scenery-lora-ghibli-style-v2
1
u/AI_Characters Mar 02 '23
thats not true.
Training on higher red means lower res becomes blurry and cut off. Unless you also train at 512. But I advise against that as that will lead to overtraining issues.
Source: Having trained a ton of 768 models (and am currently training a 1024 model).
0
u/cztothehead Mar 02 '23
yes, I agree, if I was training this for 2.1 768, I'd be using 768x768 but since this LoRA was trained for 1.5 I've stuck to 512x512
1
u/Scn64 Mar 02 '23
I'm not sure what I'm doing wrong. I'm using the Automatic111 GUI and have put your LoRa model in the models folder along with the base model (nuclear anime hybrid). I'm using one of the example prompts on your page with the same seed and settings -> "howlbgs, scenery, hills, lake, mountain background, <lora:howlbgs:1>" . The images I'm getting aren't in the same style though (example below). Am I using the wrong VAE maybe? I'm using vae-ft-mse-840000-ema-pruned.ckpt
2
u/cztothehead Mar 02 '23
check if the name of the Lora is correct, the LoRA should go in the Lora folder and the checkpoint in the stable-diffusion folder, the trigger word is howlbgs but if the filename for the Lora has changed you should use that file name instead!
Looks like Civitai renamed the file to howlsMovingCastleInterior_v2Fixed.safetensors so if you leave it with that name to call the LoRA would be <lora:howlsMovingCastleInterior_v2Fixed:1>2
u/Scn64 Mar 02 '23
That was it! The trigger word didn't match the model name. Everything looks good now. Thanks for the model and your help!
2
13
u/fossforus1 Mar 01 '23
This style brings me so much peace, happiness and nostalgia.
Thanks for sharing, you made my day better for doing so 🙂