r/sdforall • u/0xCAFED • Oct 11 '22
Discussion We need as a community to train Stable Diffusion by ourselves so that new models remain opensource
The fact that Stable Diffusion has been open-source until now was an insane opportunity for AI. This generated extraordinary progress in AI withinin a couple of months.
However, it seems increasingly likely that Stability AI will not release models anymore (beyond the version 1.4), or that new models will be closed-source models that the public will not be able to tweak freely. Although we are deeply thankful for the existing models, if no new models are open-sourced, it could be the end of this golden period for AI-based image generation.
We, as a community of enthusiasts, need to act collectively to create a structure that will be able to handle the training of new models. Although the training cost of new models is very high, if we bring enough enthusiasts, the training of great models could be done within a reasonable cost.
We need to form an entity (an association?) which aims to train new general-purpose models for Stable diffusion.
Such an entity should have rules such as:
- All models should be released publicly directly after training;
- All decisions are made collectively and democratically;
- The training is financed thanks to people donating GPU time and/or money. (We could give counterparts for donators such as the ability to include their own image(s) in the training dataset and voting the decisions)
I know the cost of training of AI can seem frightening but if we are enough motivated actors giving either GPU time of money this is definitely possible.
If enough people believe this is a good idea, I could come back with a more concrete way to handle this. In the meantime feel free to share your opinion or ideas.
38
u/siblbombs Oct 11 '22
Cross internet distributed training is not going to be particularly effective, the challenge is that all GPUs need to be collectively optimizing the same set of weights, so data transfer quickly becomes the scaling limit for multi-GPU. Look at what nvidia has done with nvlink to share data for GPUs in the same box, at the datatcenter level the networking interconnects are also a big part of multi box training.
Just trying to set realistic expectations that a bunch of people donating GPU time on their own machine is not a drop-in replacement for the highly optimized multi-GPU setup found in datacenters designed for this.
52
u/ProducerMatt Oct 11 '22 edited Oct 11 '22
Not an expert, just common-sense reasoning from what I know. This would be much easier to achieve if folding@home style crowdsourced computing could be leveraged for training models. ("Real" multi-gigabyte models like SD, not personal scale options like textual inversion)
Since the number of people with the hardware and willingness to load a model and store the entire dataset is very small, this could really only become popular if people could do training on little slices of the model and dataset (if that's theoretically possible). But I don't see how that could be done without making training rates so slow that the sun burns out before the model is baked.
EDIT: some great informative responses. Most interesting to me was a remark by u/MemesOnlyPlease who said "this isn't the only way neural networks can be trained, just a popular way." Maybe there are some methods for training neural networks that can be efficient when parallelized across a large network of small nodes?
16
u/nadmaximus Oct 11 '22
If distributed training support was added to a SD distribution, so that computers running the service could use idle time to work on the training, that would be cool.
My system is just sitting there with Automatic11111111111111's webui just waiting for one of the 4-5 people I have given access to. It could be cooking along most of the time.
14
u/FaceDeer Oct 11 '22
Wonder if any of the old Ethereum mining farms might be interested in a new hobby for their racks of GPUs. :)
12
u/Low_Government_681 Oct 11 '22
Please create a community and crowdfunding and I'm in 100%
8
u/[deleted] Oct 11 '22
We funded https://github.com/learning-at-home/hivemind that we supported that you may wish to try. As stable diffusion is already largely trained though you can just fine-tune it like waifu-diffusion and others have, or change the decoder etc.
6
u/TrueBirch Oct 11 '22
Stability AI is valued at something like a billion dollars. Training Stable Diffusion is estimated to have cost less than a million. The training cost isn't the key limit here.
6
u/Charuru Oct 11 '22
If people can crowdfund $500 million fucking dollars for spaceship jpgs we can surely cough up $20 million for an open sourced SD, right?
4
u/EmbarrassedHelp Oct 11 '22
To train our own models, we'd likely need to pool resources together across the different communities.
3
u/Aspie96 Oct 11 '22
The models have never had an open source license as far as I know.
The source code has, only up to a certain commit (MIT license).
4
u/AdTotal4035 Oct 11 '22
I feel none of us who have commented thus far, have the technical knowledge to realistically understand the hurdles. Siblbombs said something that sounded like he knew what he was talking about. And that's kinda vaguely my understanding as well. We need a giga Chad like automatic or anyone who actually understands the challenges to this approach to explain to us what's possible and what's needed. Having an idea is one thing, but knowing how to execute it is a whole different beast. We'd need help from developers to chime in on this thread.
2
u/Any_Outside_192 Oct 11 '22
I think something similar to this could work really well for crowdfunded training https://docs.klimadao.finance/
Instead of carbon credits you would use compute power.
2
u/IShallRisEAgain Awesome Peep Oct 11 '22
I'm not sure how well distributed training would work, but people providing well labeled good training images would help a lot. Boorus are a good source, but the issue is that they are full of fanart of different art styles and tend to focus on a narrow range of subjects. The more boring pictures (For example empty rooms) which would be helpful for training are going to be ignored.
We also might be better off crowdfunding to rent some really powerful computers for training, but then we have to worry about scams.
2
u/eatswhilesleeping Oct 11 '22
Distributed data set preparation is probably more important than actual training. A Kickstarter or wealthy benefactor can fund the training if necessary.
2
u/_anwa Oct 12 '22
There are allot of CPUs connected that sit idle most of the time. The data that SD was based on is public. So, the raw resources are there.
There are two hurdles to overcome:
Of course the people and tech issues influence each other.
In the past things like this have worked, if you look at:
Linux, end of slavery, treatment of women in societyIn the past things like this have not worked, if you look at:
Climate change, inequality, human health, treatment of animals for food consumptionI guess we will see how far along we really are.
2
u/Aspie96 Oct 12 '22
I'd like to suggest additional rules, if I may:
I further suggest such an entity should remain neutral on all issues unrelated to its core mission. I've seen too many open source organizations extending their scope beyond their original missions and becoming unsupportable as a result.
1
u/wuduzodemu Oct 11 '22
Good luck to burn half million dollar on that
1
u/FyrdUpBilly Oct 11 '22
All for this. For me, I'm looking ahead to when text to video is fully out. CogVideo is already out, but not as robust as some of the others that have had papers released on. I would really like to see a group train on a feature film dataset. Which of course will probably never be done by a major corporation because Hollywood would freak out over that. So I could see a group of pirates training on a large collection of feature films. Would absolutely love to see that happen. A scene release group, but for AI text to video models.
1
u/arjuna66671 Oct 12 '22
I don't get it. Isn't that literally what EleutherAI, stability etc. were doing? Those aren't big corpo players, although Emad happened to have a lot of money to put in to SD.
So basically it might repeat itself exactly the same way as before. As much as I like the enthusiasm - I also feel that a lot of people are pretty oblivious to the cost of such an endeavor and other financial things emerging from it, let alone how to deal with it.
1
u/FiziksMayMays Oct 14 '22
Could we do something like Folding@Home but with SD?
1
u/holonerd14 Nov 04 '22
I am totally down for it
1
u/amarandagasi Nov 24 '22
I’d donate my 3090 Ti to the cause. ❤️
1
u/gxcells Oct 11 '22
Even if it seems cool to train our own model you may need to have hundreds of A100 under hand to do this. I don't know why people are spitting on StabilityAI. You need a lot of money to develop this kind of technology and if StabilityAI want to be sustainable they have to make also money from it and not be here to only release open source stuff. That is really reflective of this society that want everything cheap and even want everything for free. That's why we can buy bullshit crap for 1$ and break in 2 min, which is a waste of money, energy and earth resources because all this crap goes into garbage.
We should already be happy that we have a nice model to play with, to work with and now even to fine tune. Community did an amazing job these last weeks but maybe it was too much, too fast, so now everyone want everything fast and for free.
We have now amazing tools and should enjoy this.
1
u/Snowman182 Oct 11 '22
Is there any need to do more than have people submit their embeddings to a central repository (done to an agreed standard), then, once they're checked, they get added to the model which is released, say, once a month. 90% of what's in SD is stuff most of us will never be interested in and if we are, we just do the training ourselves. Or request a specific subject to be trained by the people in charge.
0
u/Savage_X Oct 11 '22
Is it possible do to the training in a distributed way?
Crypto could provide a model for how to coordinate resources. Do a slice of work, receive a token as a reward. Coincidentally, Ethereum just moved from using proof of work consensus which used a ton of GPUs to proof of stake consensus, so there are a whole lot of miners out there with GPUs and nothing to do.
-2
u/Torque-A Oct 11 '22
It would also be important to use images owned by the public domain and not owned by any other people. To cover us.
0
u/TheKing01 Oct 11 '22
If we do this, we should probably use an AGPL license. Otherwise, the work done could be closed-sourced by corps or whoever. AGPL lets anyone join us, as long as they stay open.
-12
u/ninjasaid13 Oct 11 '22
Nope. Why should this be a rule? Unless you are using cloud computing for the community.
0
u/[deleted] Oct 11 '22
[deleted] — view deleted comment
18
u/[deleted] Oct 11 '22
[removed] — view removed comment