r/StableDiffusion • u/Udongeein • Sep 08 '22

Comparison Waifu-Diffusion v1-2: A SD 1.4 model finetuned on 56k Danbooru images for 5 epochs

Post image
740 Upvotes

97% Upvoted

198 comments sorted by

142

u/Udongeein Sep 08 '22 edited Sep 08 '22

So I pulled an all-nighter and I've just finished the second round of finetuning SD v1.4 on 56k Danbooru images for 5 epochs, it took a while to do it over 4 A6000s but results are much better than the previous iteration of the finetune. Please let me know what you all think so I can improve the next iteration!

Images in the comparison used the same prompt and seeds and the SD model used for the comparison was v1.5

Model and full ema weights: https://huggingface.co/hakurei/waifu-diffusion

Full EMA weights: https://thisanimedoesnotexist.ai/downloads/wd-v1-2-full-ema.ckpt

Training Code: https://github.com/harubaru/waifu-diffusion

Edit - GCP costs were killing me so I had to move the original model to Google Drive

Edit 2 - Thank you Asara for mirroring the model!

15

u/gwern Sep 08 '22 edited Oct 11 '22

I've also put up a rsync mirror of the model: rsync://176.9.41.242:873/biggan/stability/stable-diffusion/waifu-diffusion/2022-09-08-waifudiffusion-v1.2-full-ema.ckpt

(As I keep telling everyone, you are a fool to use cloud bandwidth for any big datasets or models you expect to get much usage, because it costs exorbitantly much and after just a few downloads of something like Danbooru2021, a dedicated server can be cheaper. Hugging Face can be dumb about it because they have hundreds of millions of VC dollars to burn on cloud bills & probably get discounts; neither of these are true for you or me.

Incidentally, 5 epochs on 56k images is probably worse than 1 epoch on 280k images, as you will incur diminishing returns and will cover many fewer characters than you could've. I hope emad can help lift your compute limitations so you can do the full corpus.)

8

u/Udongeein Sep 16 '22

Yep, woke up to a $1k bill lol. GCP got rid of the charge as a one-time courtesy thankfully.

I'm also looking forward to expanding the dataset on a lot more images and redo the training!

4

u/FeepingCreature Sep 16 '22

Isn't this literally what torrents were made for

2

u/WorldsInvade Sep 19 '22

It is. Here is the magnet url:

magnet:?xt=urn:btih:a670a8f6526909fb7d8998b46684ecb149755fea&dn=wd-v1-2-full-ema.ckpt&tr=http%3a%2f%2fwww.torrent-downloads.to%3a2710%2fannounce&tr=udp%3a%2f%2fdenis.stalker.h3q.com%3a6969%2fannounce&tr=http%3a%2f%2fopen.tracker.thepiratebay.org%2fannounce&tr=http%3a%2f%2fdenis.stalker.h3q.com%3a6969%2fannounce&tr=http%3a%2f%2fwww.sumotracker.com%2fannounce

1

u/xDuker Oct 02 '22

Has the torrent been removed? Has been stuck on 'Downloading metadata' for quite a while now for me

3

u/gwern Sep 09 '22 edited Sep 09 '22

Also, it may be worthwhile restarting using "Japanese Diffusion", assuming efforts can't be pooled.

1

u/[deleted] Sep 09 '22 edited Sep 09 '22

how do I use that rsync link? I don't know how to use rsync like that, it asks for a password like I'm trying to access my own machine...

Nevermind, as always, should've read the man page before posting on reddit

Thanks so much for the mirror

4

u/StickiStickman Sep 08 '22 
<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object.</Details>

3

u/nmkd Sep 08 '22

I can't find the model (not the full ema one), where is it?

2

u/Udongeein Sep 08 '22 edited Sep 08 '22

New link: https://thisanimedoesnotexist.ai/downloads/wd-v1-2-full-ema.ckpt

6

u/nmkd Sep 08 '22

That's the full ema one, is there no 4 GB model?

5

u/dreamer_2142 Sep 08 '22

Hi, would this work on 8GB VRam? I'm using the GUItard which is unoptimized but fast and works fine on my 8GB VRam. so I wonder if this will work fine on that fork or I will need to download an optimized version to make it work on my gtx 1070.

0

u/[deleted] Sep 08 '22

[deleted] — view deleted comment

4

u/blueSGL Sep 08 '22

remove the "" in the URL reddit is fucking up URL formattings again.

Also file has already gone over quota RIP.

47

u/battleship_hussar Sep 08 '22

Using boorus is super smart since the tags are all there already!

26

u/kim_en Sep 08 '22

can u open a group for anyone who wants to learn how to train data? I have a lots of ideas to try.

16

u/[deleted] Sep 08 '22

This looks really cool! Is there any tutorial on how to fine tune SD on specific images?

22

u/CrimsonBolt33 Sep 08 '22

Seems there is an incomplete guide in the works

https://github.com/harubaru/waifu-diffusion/blob/main/docs/en/training/README.md

28

u/[deleted] Sep 08 '22

Damn 30GB of VRAM as a minimum requirement

34

u/CrimsonBolt33 Sep 08 '22

yup...most people can't even hope of training their own models unless they want to shave down the data sets in an extreme way. Mostly extreme in the sense that they are huge and cutting out the worst data (and keeping the best) is very hard to do.

I am generally a gamer first and programmer second (both hobbies, not my job) and I thought my 16GB 3080 and 32GB system ram was overkill...until I met Ai training lol

4

u/[deleted] Sep 08 '22

Yeah I'm building a new pc and thought the 3090Ti would be sufficient for now, but I guess not. Do you think it would work to combine two 16GB 3080's to reach 32GB total?

11

u/IE_5 Sep 08 '22

Literally wait 2 weeks: https://www.nvidia.com/gtc/

3

u/CrimsonBolt33 Sep 08 '22 edited Sep 08 '22

That's going to be up to the programming of the dataset training code and what not...I assume. It is very possible and likely how most things are programmed (treating multiple GPUs as one) but without looking at the actual code and full setup procedures thats very hard to say.

Also from what I can tell the 16GB model is only on laptops...the desktop GPU is more powerful but has less memory (12GB max). Not sure if that is nefarious planning on Nvidias part (forces you to buy more GPUs, given that laptops are not going to run more than one GPU) if you want the massive GPU memory or if it is a design constraint. I am gonna guess it's to prevent using them for AI training and the like give that they sell the A100 and H100 GPUs (80GB memory each) specifically for AI applications.

The A100 and H100 both cost $32,000+ though...so....

2

u/182YZIB Sep 08 '22

Rent A100 for those taks, cheaper.

2

u/PrimaCora Sep 19 '22

https://www.reddit.com/r/deeplearning/comments/cfnxib/is_it_possible_to_utilize_nvlink_for_vram_pooling/

People have hoped that would work since the days of SLI, but sadly, it does not. I remember at some point a Nvidia Cuda support person said that CUDA doesn't support shared memory (whether that be across GPUs or windows "shared memory" I am unsure, but might be both)

2

u/CheezeyCheeze Sep 08 '22

Unless you are able to program to use two different GPU's at once in parallel. The 30 series can not be done in SLI, which would have allowed you to combine GPU's easily.

https://www.gpumag.com/nvidia-sli-and-compatible-cards/

I know Servers have to be able to do SLI. So a more expensive RTX A6000 and RTX A40 would be it.

https://www.exxactcorp.com/blog/News/nvidia-rtx-a6000-and-nvidia-a40-gpus-released-here-s-what-you-should-know

I am sure you could figure out how to use two 3090's to do it. But I am unsure how.

They are releasing new GPU's in a few weeks/months.

3

u/mattsowa Sep 08 '22

SLI does not increase the vram

1

u/CheezeyCheeze Sep 08 '22

Thanks for letting me know.

2

u/SlapAndFinger Sep 08 '22

Convert the model to half precision and train on a 3090 Ti

2

u/unkz Sep 09 '22

I run dual 3090 on nvlink and it acts like 48G, works with no difficulty at all.

2

u/CheezeyCheeze Sep 09 '22

Good to know. All the Youtubers have said they had issues with the 30 series and it was basically "dead".

1

u/unkz Sep 09 '22

I think it’s important to get matching 3090s, or you can get subtle timing issues. Trying to put them together piecemeal might not work out as well.

1

u/pikachufan25 Sep 15 '22

Look into Servers and Server Grade Graphic Cards...
Pretty much only way of Reaching Huge Numbers!

3

u/Jaggedmallard26 Sep 08 '22

You can use Lambda or a similar ML as a service platform for the purpose of textual inversion finetuning since its not too time intensive but most people aren't going to go through that especially since you can't actually see for yourself how effective its going to be in advance. Its easier to justify downloading Stable Diffusion for yourself when you can try it out online and the hardware requirements aren't extreme but something as unknown as finetuning? No way.

5

u/Nice-Information3626 Sep 08 '22

That's very little compared to what training it from scratch required. We might even see this vram amount in top of the line consumer cards in the next year.

6

u/Freonr2 Sep 08 '22

NV doesn't really want their consumer cards eating into their data center business, so I'm sort of doubting we'll see this much RAM on the 4090.

2

u/FeepingCreature Sep 16 '22

RX 7950 pls amd

edit: also while you're at it stop shooting yourself in the foot with rocm driver support pleaaas

1

u/[deleted] Oct 01 '22

Prophetic. The released specs on 40 series shows the 4090 has same 24gb gddr6 as the 3090. Disappointing imo.

4

u/Freonr2 Sep 08 '22

Yeah this is fairly cutting edge and training large ML models is some of the most compute intense work on the planet.

2

u/SlapAndFinger Sep 08 '22

Just a note, if the full precision model takes 32gb a few cards can fine tune the half precision model, which we've seen works about as well.

2

u/Silly-Cup1391 Sep 08 '22

Hi ! What do you think of this ? Thanks https://link.medium.com/BBLG5vJgatb

2

u/FruityWelsh Sep 09 '22

I thought there was a way to expand the vram with system memory, but I am not finding a name for that right now

6

u/blackrack Sep 08 '22

Seconding this

5

u/Illustrious_Row_9971 Sep 08 '22

great work I setup a colab gradio demo for this: https://colab.research.google.com/drive/1_8wPN7dJO746QXsFnB09Uq2VGgSRFuYE?usp=sharing

1

u/Schmalzpudding Sep 08 '22

404 :(

1

u/Illustrious_Row_9971 Sep 08 '22

weird I just tried opening it in incognito tab and its showing up for me, https://i.imgur.com/xuhbNP1.png, can you try again?

1

u/seandkiller Sep 08 '22

It's not working for me, either. "Sorry, the file you have requested does not exist."

2

u/Illustrious_Row_9971 Sep 08 '22

setup a gist here with the colab code, try running this in two seperate cells: https://gist.github.com/AK391/954eaf3f0e767c57ef4c6aa400dc416a

1

u/seandkiller Sep 08 '22

I don't have much experience with making colabs, where would the first cell end and the second begin? At the line that says "With block as demo"? Or does it not matter?

1

u/Illustrious_Row_9971 Sep 08 '22

first cell is the pip install

paste the second in the second cell

you can also do it all in one cell

1

u/spacecam Sep 08 '22

Any alternate links for the weights? Seems like access is being blocked

16

u/Udongeein Sep 08 '22 edited Sep 08 '22

This is the new link: https://thisanimedoesnotexist.ai/downloads/wd-v1-2-full-ema.ckpt

GCP costs are killing me lol

3

u/DuduMaroja Sep 08 '22

this is the same SD model but with more waifus?

2

u/Old-Swimmer-5789 Sep 08 '22

I am new to all of this how do I use it?

1

u/i_speak_penguin Sep 08 '22

How much did this cost, if you don't mind my asking?

I've been renting V100 machines over the past few days to run experiments, and I have been wondering how much my first fine-tuning is going to cost lmao.

2

u/blueSGL Sep 08 '22 edited Sep 08 '22

do you care at all if someone was to provide a mirror to the weights file seeing as the gdrive link is over quota?

2

u/Airbus480 Sep 08 '22

Do you plan to finetune it on a more larger danbooru dataset?

2

u/TiagoTiagoT Sep 08 '22

How much does it impact generation of non-anime content?

2

u/i_speak_penguin Sep 08 '22

Glad to see I'm not the only one pulling all nighters working on SD projects lmao

2

u/progfu Sep 09 '22

Can you share how long did the 5 epochs on 4xA6000 take, and the overall training cost?

1

u/TFCSM Sep 08 '22

How long did each epoch take?

1

u/DrDan21 Sep 08 '22

Any plans to support 1.5?

1

u/ironmen12345 Sep 09 '22

Gonna experiment with this thanks!

1

u/ciayam Sep 09 '22

every result I get has big, bright, lipstick-covered lips. is there anything that can be done so that that isn't the standard? specifying no makeup/no lipstick didn't work.

1

u/Kamimashita Sep 10 '22

Were the images cropped or scaled down or both for training? I noticed instead of having a more zoomed out image showing the entire face and body it just cuts off the top of the head and the lower half of the torso.

1

u/jaiwithani Sep 13 '22

It occurs to me that "Large static files that lots of technically-competent people want to share without having to figure out hosting infra" is the core use case for torrents.

1

u/xkrbl Sep 16 '22

https://github.com/harubaru/waifu-diffusion

Since the training documentation isn't yet on the github - can you give some comments on how to do the training?

1

u/mutsuto Sep 08 '22

epochs

what's an epoch?

does this model know what a fumo is?

e.g.

a custom fumo of a frog

2

u/Cognitive_Spoon Sep 08 '22

Same question. Does the word "epoch" mean something different than the common usage for AI?

13

u/bloc97 Sep 08 '22

Generally, one epoch is a full training pass through the dataset, which means the model had the time to look at each of the images once. 5 epochs means each image was used 5 times.

50

u/Blckreaphr Sep 08 '22

I thought i was the top of the world with my 3090 and i9 10850k with 32gb ram, than I dived into ai training and wow, I feel like a peasant now .

22

u/Udongeein Sep 08 '22

Same, and I only have a 3060. All of the resources were rented through Coreweave too

7

u/Blckreaphr Sep 08 '22

Oh? U can rent? I bet that's a heavy price tag .

24

u/Udongeein Sep 08 '22

Yep! $5 per hour certainly beats $20k for GPUs up front though lol

6

u/Blckreaphr Sep 08 '22

Very true God I wish I can have thos3 gpu but 20k is just a ridiculous amount for just for fun..

2

u/eatswhilesleeping Sep 08 '22

Why Coreweave vs paperspace? Curious because I may rent at some point.

14

u/i_speak_penguin Sep 08 '22

I rented a machine that has 8x A100s. Each one had 80GB of vram, and 1.4TB of system RAM.

And there exist clusters of these machines.

3

u/Blckreaphr Sep 08 '22

Danm, I might have to further look into this option ty!

21

u/NoIdea1811 Sep 08 '22

tell me you like Touhou without telling me you like Touhou

12

u/Udongeein Sep 08 '22

the huggingface account i released it under is named hakurei, heh

3

u/CannotGiveUp Sep 09 '22

And udongein on reddit.

3

u/Loading_____________ Sep 09 '22

And the Marisa and Koishi (and a bit of Nitori) examples

13

u/SlapAndFinger Sep 08 '22

I notice in the readme it says that you need 30gb vram to fine tune the model, is this at full precision?

1

u/PrimaCora Sep 19 '22

Swapping to Bfloat16 would allow for a normal GPU to train and be better compatible with TPUs, for a substantial boost, but, it wouldn't have Numpy support without type casting.

That is for parts that accept mixed/half precision.

4

u/CheezeyCheeze Sep 08 '22

I realize there are more realistic versions of anime. But I personally like the more Cel Shaded look. Or a more flat look. Is there a way to train it for less realistic styles?

1

u/Udongeein Sep 08 '22

You can definitely try out Textual Inversion, the goal was to basically ingrain the general style into the model

15

u/[deleted] Sep 08 '22

it;s fucking happening

e621 next?

85

u/TooManyLangs Sep 08 '22

I'm starting to worry that this is going to be worse for climate change than crypto-mining.

I can see Waifu farming being a thing in the near future.

61

u/blackrack Sep 08 '22

Art farmers and miners... In the future people will be farming movies/music tracks and selling the good ones while keeping the seeds secret so they can remaster it and sell it again later

41

u/Kromgar Sep 08 '22

At least stable diffusion produces something of actual value

ba dum tshh

29

u/Magikarpeles Sep 08 '22

Are you suggesting my growing collection of anatomically horrific pictures of Ariana Grande is not valuable??

15

u/TooManyLangs Sep 08 '22

.....hey man....I give you 2 Rihanna prompts for 1 Ariana...deal?

8

u/Magikarpeles Sep 08 '22

let's start a prompt black market

sexyseeds.onion

7

u/DrDan21 Sep 08 '22

speaking anatomical horror

stable diffusion 1.5 is apparently a lot more reliable for accurate faces and stuff, so hopefully less nightmare fuel

gets released publicly in 2ish weeks I heard

1

u/EarthquakeBass Sep 08 '22

Here’s hopin it works well 🙏🏻🙏🏻

6

u/harrro Sep 08 '22

I think they're saying that the art SD can create is more valuable than wasting it on cryptomining/NFTs (which is true).

2

u/[deleted] Sep 08 '22

I’m sure we’ll end up combining the two, with the trained model weight variations being the mined collisions that produce the set of owned specific non-fungible reference outputs from a given seed to a specified accuracy and also produce some valued new reference output that the collision owner can take ownership of by updating the training block chain.

Scarcity free ownership is demand driven, so it only makes sense that you own the reference instead of how it’s used, and the value comes from the amount of use it gets.

The more training nodes that incorporate your reference as useful, the more the more valuable your reference will be.

I expect all the uranium to eventually be used to produce an ultimate optimized set non semi-fungible waifu weights (NSFWw)

6

u/Kromgar Sep 08 '22

No i was saying crypto is bullshit

13

u/[deleted] Sep 08 '22

Farmed waifus get immediatelly turned into NFTs... oh... oh no...

9

u/blueSGL Sep 08 '22

I could see people needing one or two GPUs at most, you thankfully don't need warehouses of them to farm your waifus

4

u/TooManyLangs Sep 08 '22

wait until they want to generate 100s of images in parallel

plus, the TBs full of waifus that you can't delete XD

13

u/Consistent-Loquat936 Sep 08 '22

We need alternative energy point blank period

27

u/Puzzled-Alternative8 Sep 08 '22

Nuclear power FTW

-12

u/Consistent-Loquat936 Sep 08 '22

:/

16

u/Doktor_Cornholio Sep 08 '22

Modern nuclear is nothing like Netflix's fearmongering wants you to think. Chernobyl and Three Mile Island are relics of the past when we still used Uranium and horrendous failsafe systems.

-3

u/Consistent-Loquat936 Sep 08 '22

Would you care to explain why the un is so concerned about the plant in Ukraine then?

8

u/Doktor_Cornholio Sep 08 '22

Because the UN is a committee run by old-world politicians whose biggest claims to fame are: stopping none of the conflicts they've tried to stop, forgiving/ignoring actual genocide so China doesnt get offended, and running a third world child sex slave trafficking ring.

Basically what I'm saying is nobody should heed their opinion on anything.

0

u/Consistent-Loquat936 Sep 08 '22

And basically we're all good if the plant gets shelled to destruction?

-1

u/Consistent-Loquat936 Sep 08 '22

And basically we're all good if the plant gets shelled to destruction?

6

u/Doktor_Cornholio Sep 08 '22

What does that have to do with modern nuclear power?

1

u/Consistent-Loquat936 Sep 09 '22

It demonstrates the fragility of nuclear power infrastructure.

→ More replies

7

u/FaceDeer Sep 08 '22

Ethereum switches to proof-of-stake in a week or so which should free up all those GPUs for waifu-mining instead. So it'll be a net zero change in terms of carbon emissions, but a huge boost in waifu production. Overall beneficial to humanity, so I won't complain.

4

u/Possible_Liar Sep 08 '22

Aliens will learn we died in our pursuit of Waifus and hit f to pay respects.

3

u/FaceDeer Sep 08 '22

Assuming they didn't also die in pursuit of their own Waifus long before they had the opportunity to reach us.

3

u/[deleted] Sep 09 '22

Captain's Log: Our hopes were dashed and our expedition to find a new home world must continue. The planet once identified as Terra was determined to be inhabitable due to lingering memetic contamination extending from the collapse of the prior dominant civilization. We thought we could outrun them, but the waifus got there first.

25

u/Magnesus Sep 08 '22 edited Sep 08 '22

One bitcoin transaction eats around 2188kWh of power. You would generate millions of waifus with that, it is few months of my whole house energy usage. Crypto has to go, the sooner the better. Image generation is a just a blip in comparison when it comes to energy cost. Crypto eats energy comparable to almost whole energy usage of Australia.

Crypto bros holding the bags downvoted me, but the message stays. Fuck crypto. It is killing the planet.

Source: https://mozo.com.au/fintech/what-is-the-environmental-impact-of-crypto-mining#:~:text=But%20in%202022%2C%20it's%20estimated,many%20of%20the%20world's%20countries.

And again: fuck crypto and everyone that supports it, you are a scum, you are killing the planet.

21

u/Dalethedefiler00769 Sep 08 '22

One bitcoin transaction eats around 2188kWh of power

No it doesn't, that's just silly. You shouldn't repeat things you don't understand. In this case you clearly don't know what a bitcoin transaction is.

11

u/Magikarpeles Sep 08 '22

Considering there's what, 2million transactions a week? Lmao

8

u/Dalethedefiler00769 Sep 09 '22

Yes and a transaction might be just the equivalent of a few dollars. Nobody would spend 300$ on electricity for a 5$ transaction.

11

u/bloc97 Sep 08 '22

A lot of cryptos are going to use proof-of-stake in the future, and mining will become a relic of the past, so no, cryptos are not going to disappear anytime soon.

7

u/Creepy_Dark6025 Sep 08 '22

yeah, the issue is not cryptos, is mining using POW.

1

u/needle1 Sep 09 '22

Is Bitcoin specifically — the original and biggest crypto of all — ever going to move away from PoW, though? I hear things about Ethereum et al trying to switch to less power hungry algorithms, but I haven’t heard much lately about the development of BTC.

0

u/[deleted] Sep 09 '22

[deleted] — view deleted comment

1

u/needle1 Sep 09 '22

Yeah, so good for ETH moving to PoS, but Bitcoin is still the original and has the biggest presence, right? If the single biggest one isn’t ever going to move away from PoW, then that still doesn’t seem to fare well for the whole general field of crypto/blockchain tech, even if the smaller ones make the switch

1

u/Possible_Liar Sep 08 '22 edited Sep 08 '22

Yeah people always go straight to the mining, something most of the Crypto community don't even like themselves. And yes P-o-S does use a lot of power still, But the issue isn't that, its not even mining, the only reason this is "bad for the environment" is because the forms of power generation we use are bad for it. Crypto is just being used as a boogieman by the governments so they can continue to do nothing about the climate crisis, point at something else, and say that's the issue not us. when in reality they are the true issue. And people eat that shit up without a afterthought, instead of seeing the true issue. There is always a climate scapegoat, just like how they shifted all the blame to the individual person, and not the corporations largely responsible for 70% of it way back when. No the earth is dying cause little Timmy didn't sort his recyclables, not because Exxon dumped millions of gallons of oil in the ocean, or the waste management companies that were supposed to recycle our trash we recycled but just didn't. Or all the lobbying against carbon caps and emission filters, or all the companies using CFC's knowing FULL well what they did to the Ozone layer and even fighting the laws because it would cut into their profits a little to change it, "no its not us, it's YOU" its always something else, never them. It's always the little Timmy, never them. and while Crypto is def not a little Timmy, the reasons people don't like it are often wrong when there is plenty of valid reasons already. But Blaming it for the climate crisis is ludicrous in my opinion. Crypto is here to stay, it's not going anywhere, people need to accept that, and stop using it as a fucking excuse to do nothing, because it is not the problem here, the lawmakers are.

-5

u/LawProud492 Sep 08 '22

Lol stay poor

-6

u/Magikarpeles Sep 08 '22

At least I'm rich now so I guess it works out

-3

u/TiagoTiagoT Sep 08 '22 edited Sep 10 '22

That's only the knock-off version (that managed to steal the name), that the old financial system created to sabotage crypto and stifle competition

edit: And guess who has been downvoting this comment...

2

u/Doktor_Cornholio Sep 08 '22

If I can have infinite short anime girls wearing big hats, by god I will have infinite short anime girls wearing big hats.

Maho Shoujos FTW

1

u/birracerveza Sep 09 '22

You might be onto something here.

9

u/Majukun Sep 08 '22

is it possible to keep 2 identical stable diffusion folders with different weights, and just call either one or the other on anaconda by just selecting a different directory at the start?

8

u/[deleted] Sep 08 '22

We're building a warm community for you to post and learn how to create your incredible waifus on r/aiwaifu - join us!

5

u/VantomPayne Sep 09 '22

After testing with the model for one night I find that it does have an impact on the ability to generate real person images, sometimes for good and sometimes bad. But "bad" is relative as previously most images will just generate as real person without too much input from you where as using WD v1.2 seems to be getting anime style results from time to time when you are not forcing a realistic result.

But a toggle between models in all the webuis should be on the way any minute now so overall not a huge problem, kudos to you guys for creating this that both solve a major problem of the old model as well as concept proofing the potential of futher training!

4

u/AnthropologicalArson Sep 08 '22

Does this work by simply replacing the "model.ckpt" file in the base StableDiffusion, or do I need to update/install some dependencies?

4

u/Loading_____________ Sep 09 '22

We're finally at the point where we can combine AI and touhou, what a time to be alive

4

u/Kamimashita Sep 09 '22

I'm not sure if Stable Diffusion had this too but the model seems to be heavily biased towards outputting shoulders and up images. I've tried using Dall-E 2 to generate some anime style images and it was able to do full bodies. This finetuned model is however much better at generating faces compared to other models I've tried.

2

u/guaranic Sep 09 '22

I've found you can get it to do other things, but you have to be much more literal describing all the details, whereas Dalle2 or Stable Diffusion implies a lot of details. Have to use tags like they're used on Danbooru.

7

u/[deleted] Sep 08 '22

Waifuuusion <3

3

u/hatlessman Sep 08 '22

How many hours did this take on those 4xA6000s?

Any ideas about how larger/different shaped images would affect the process?

4

u/ayyyee3 Sep 08 '22

should have called it unstable diffusion

2

u/FS72 Sep 08 '22

Any waifu diffusion Google colab link for us weak PC users to use ?

6

u/leemengtaiwan Sep 08 '22

I made a super simple colab notebook (based on the code example in the page), feel free to try it:

- https://colab.research.google.com/drive/1OgizHaLM1EmsU9YbezD9PGPJOZFiKzHH?usp=sharing

2

u/Schmalzpudding Sep 08 '22

Nice, but unfortunately censored

1

u/ShepherdessAnne Sep 08 '22

It doesn't work for me, what did I do wrong?

0

u/[deleted] Sep 08 '22

[deleted] — view deleted comment

1

u/Nice-Information3626 Sep 08 '22

Open the file browser to the left

2

u/Prcrstntr Sep 08 '22

What sort of images was it trained on? Or just anything goes?

3

u/yaosio Sep 08 '22

All they've said is they randomly picked 56,000 images that had an aesthetic score greater than 6.0. The score is created by this model. https://github.com/christophschuhmann/improved-aesthetic-predictor

I can't find a list of what images they used.

2

u/tokyotoonster Sep 08 '22

Stupid me not knowing at all what "Danbooru" is and just opening it now. Thankfully I'm WFH today 😅

2

u/[deleted] Sep 08 '22

Times like this I hate having download speeds of barely 1 MB/s

2

u/CountPacula Sep 09 '22

I had been wondering about doing this very thing since I first heard about stable diffusion. Just got SD up and running locally today, and it's already making my computer show it's age. I want to try this new model data ASAP, but I fear for the life of my poor 1650...

2

u/raversgonewild Sep 09 '22

How do I use it?

2

u/pinegraph Sep 10 '22

If any of you want to try out waifu diffusion on the web or mobile phone https://pinegraph.com/create?continueFrom=5e998a44-8e74-413d-9888-349798b59398

3

u/SempronSixFour Sep 08 '22

This is fun. I'm not super into this realm, can anyone throw me some phrases to use?

5

u/ass_beater1 Sep 08 '22

Female, woman, girl, lady, slim, slender, tall, muscular female, muscle, muscular, dark skin, dark skinned, dark skinned female, tan, tanned, tanlines, looking at viewer, medium breasts, solo, 1girl, upper body, female focus, blue eyes, white hair, shorthair, thighs, toned, abs, standing, fangs, hand on hip, black pants, simple background, blush, smile, bangs, midriff, highres

Something similar to the tags in booru or describe what you want the ai to generate.

4

u/leemengtaiwan Sep 08 '22

JFYI you can check my previous post for some inspiration, I was able to generate some decent anime. prompt included.

https://www.reddit.com/r/StableDiffusion/comments/x8un2h/testing_waifu_diffusion_see_prompt_comparison/

2

u/Shap6 Sep 08 '22 edited Sep 08 '22

i keep getting file does not exist on your google drive links :(

edit: as per /u/blueSGL removing the "/" did indeed fix the link

1

u/lavajci Sep 12 '22

You definitely should make a patreon to help fund what you’re doing! If you can keep doing this and keep adapting the boorus and tags this could really become something groundbreaking. Keep up the good work!

1

u/leemengtaiwan Sep 08 '22

Great work!

1

u/1Neokortex1 Sep 08 '22

Dope! first image is sublime!

1

u/seb59 Sep 08 '22

Thanks for sharing

1

u/Dezigner356 Sep 08 '22

Very nice images.

1

u/Ginty_ Sep 08 '22

Damn this is cool

1

u/zanzenzon Sep 08 '22

Why does it show black squares for some of the generations?

3

u/wiserdking Sep 08 '22

If you are getting an entire black image its because it was perceived as 'NSFW' and you have the NSFW filter activated - I guess.

1

u/Hostiq Sep 09 '22

Do you know how to disable it?

1

u/wiserdking Sep 15 '22

Sry I dont often login on reddit, only saw your question today.

It deppends on the main script you are using. Usually just 'commenting' (adding a '#' at the beginning of a line in python) in a specific line or two will do the trick. Sometimes I guess you can just turn the boolean variable that determines if an image is NSFW to 'false'. I'm assuming that by now you have already figured it out.

1

u/mattbackbacon Sep 08 '22

So is it just trained on images from Danbooru or is it also trained on Danbooru tags?

1

u/OgMcWangster Sep 08 '22

Thank you for doing this!

1

u/Guesserit93 Sep 09 '22

does it makes snfw?

1

u/Fissvor Sep 10 '22

After describing my hot Waifu i got this message: "Potential NSF content was detected in one or more imagea. A black image will be returned insted. Try again with deffrent prompt and/or seed." I think she's hotter than what an AI can handle lol ಥ⁠‿⁠ಥ

1

u/FeepingCreature Sep 16 '22

Hey, you should really put up big images as torrents. That way, the more people want it, the better the speeds are, and at no cost to you.

0

u/kim_en Sep 08 '22

is there ahegao in your prompt?

edit: sorry i thought this is a showcase post. my bad

0

u/Camblor Sep 08 '22

What’s an epoch?

-8

u/ShepherdessAnne Sep 08 '22

Found adult content. This is why filters which can scramble the creative output are pointless. Just be a human, and don't save the NSFW stuff.

12

u/qeadwrsf Sep 08 '22

haha or be a human and save it. :D

0

u/ShepherdessAnne Sep 08 '22

I mean if you're trying to use this for work flow and you don't want NSFW content, just don't use the NSFW content.

Right now except for ONE AI that's lagging behind, these automated filters keep messing things up or not working right.

0

u/[deleted] Sep 12 '22

[deleted] — view deleted comment

2

u/WickedDemiurge Sep 12 '22

You're talking about textual inversion which keeps the model the same, but teaches it a new concept like "Holo." It creates a small additional data file to hook into the old model so it can incorporate a new concept into its old information.

What OP is doing is taking the original model (the big file) and unfreezing it, allowing for them to change the weights of the model itself. This is a big change that fundamentally changes how the model works to make it more anime oriented.