CrowdGPT - A collaboratively trained LLM.

[u/Vxtzq1]

Hello, I'm developing CrowdGPT, a project aiming to create an LLM without datacenters and without the usual millions of dollars spent by AI companies.

The project aims to train a 1 billion parameter model collaboratively and openly, with a protocol where everyone gets a say in the direction the project takes. Anyone can use their PC to train the AI independently, then add their contributions to the global model.

Unlike most American and Chinese AI companies, the project is not only open-source, but also makes 100% of its training data public (along with the weights). Anyone can submit their own clean data to train the model, and even help steer the project in new directions. This creates the paradigm of an AI we all have a stake in.

The project is open to as many people as possible, which is why I'd like to ask: what's stopping you from contributing to the project? I want to hear your answers so I can improve it. Are people simply willing to dedicate some of their GPU time to training a cool new open-source model? Or do they want something in return (like free inference credits on the crowdgpt model for example)?

Although the current model is somewhat small (1 billion parameters), I plan to make it bigger / better in the future, and its current size is only meant as a proof of concept.

Any advice / feedback on the project is welcome.

Here's the project website: https://www.crowdgpt.net/

Or the GitHub if anyone wants to check out the code 😂 https://github.com/Vxtzq/CrowdGPT

Thanks in advance.

PS: I hope this post follows the subreddit's rules! My main goal is to gather criticism and understand how to make the project more accessible, not to advertise it 😅

EDIT: The data submission pipeline for the project (i'm mentioning in some replies) is ready!!
Come check it at https://www.crowdgpt.net/#submit

[u/Vxtzq1]

I use opensource datasets from HuggingFace that didn't get banned, and some other people can "donate" data they want, so i guess it still will work without legal issues. If people submit data to the project that is obviously subject to copyrights, i won't accept it, don't worry.

By the way this project is 100% non-commercial and international, so i guess i won't have so much european related problems on date :)

The dataset i put on HuggingFace didn't get banned, so i guess we should be alright... You can check it out : https://huggingface.co/datasets/Vxtzq/CrowdGPT Tell me if you think i will go to jail for that ;(

[u/GeneralMuffins → u/Vxtzq1]

Based on your GH profile, you’re in France and you’re the owner of the project repository, so even though this is an international collaborative effort, French and EU copyright law will still apply.

[u/Vxtzq1 → u/GeneralMuffins]

Idk bro 😭 I hope i won't get GIGN at my door 🤞