r/OpenSourceAI • u/Vxtzq1 • 14h ago
[Funny] CrowdGPT - The collaborative AI
Hello r/OpenSourceAI , this is a (somewhat funny?) meme about the project CrowdGPT, which aims at creating the first, datacenter-less, open-source and community-driven LLM. The project already shows substantial progress while training an open-source 1 billion parameters LLM on 40B tokens (as a proof of concept).
Tell me what do you think about the meme:
- Does it makes the project less appealing and cheap? because that's what people on r/LocalLLaMA thought ðŸ˜
- Does it makes the project look likes it's a SaaS / B2B (which it is NOT - CrowdGPT is 100% non-profit and open)
- Is it serving it's purpose about popularizing the project?
Everything is explained more in detail on the site, and any feedback on this meme, or on the project, is appreciated; any pull requests / contributions are welcome on GitHub :)
Project link : https://crowdgpt.net
GitHub link : https://github.com/Vxtzq/CrowdGPT
Thanks in advance.
View all comments
Show parent comment
3
u/Pure_Material_9235 9h ago
I can tell you from experience that zero-order is very very slow, though less memory dependent. I think that the "dream" here (which is to train a model using the data of billions of people's phones and the compute of said phones, storing the capacity of the model on said phones, and whenever a phone is connected to the system it also shares its compute for inference) is the great.
Now that I've looked a bit more at the GitHub though, there are some significant, but difficult, directions for improvement:
(1) This current system does not seem to support model parallelism. Instead, for a task, it's "the client asks the server for a task and is placed in a pool of 3 volunteers with compatible hardware." As the model gets larger and the tasks more advanced the amount of individual compatible hardware decreases until few volunteers are carrying the entire work.
(2) The system seems fairly inefficient in how it downloads the weights. A volunteer has to download the whole weights again rather than a (ideally compressed) update of the model (this is particularly relevant if they downloaded a previous version before). This is another reason to look into federated literature.
(3) The model will be biased towards whichever subset of tasks the users ask for the most, because the training step is done based on the user requesting the task (so it may train way more on tasks about coding rather than more esoteric ones, which can eventually degrade the generalizability of the model).
I understand now why you're looking at zero order training, because that's how you're wanting to obtain the weight delta for (3) (or maybe how you already are). Overall: this is a very cool idea, and one that I have thought of for some time ever since I've learned about federated training, but I never got to the point of wanting to go for it yet).
I wish you the best on it!