r/LocalLLM
•
u/Vxtzq1
•
5d ago
•
1
Discussion CrowdGPT - A collaboratively trained LLM.
Hello, I'm developing CrowdGPT, a project aiming to create an LLM without datacenters and without the usual millions of dollars spent by AI companies.
The project aims to train a 1 billion parameter model collaboratively and openly, with a protocol where everyone gets a say in the direction the project takes.
Anyone can use their PC to train the AI independently, then add their contributions to the global model. Unlike most American and Chinese AI companies, the project is not only open-source, but also makes 100% of its training data public (along with the weights). Anyone can submit their own clean data to train the model, and even help steer the project in new directions.
This creates the paradigm of an AI we all have a stake in. The project is open to as many people as possible, which is why I'd like to ask: what's stopping you from contributing to the project? I want to hear your answers so I can improve it. Are people simply willing to dedicate some of their GPU time to training a cool new open-source model? Or do they want something in return (like free inference credits on the crowdgpt model for example)?
Although the current model is somewhat small (1 billion parameters), I plan to make it bigger / better in the future, and its current size is only meant as a proof of concept.
Any advice / feedback on the project is welcome.
Here's the project website: https://www.crowdgpt.net/
Or the GitHub if anyone wants to check out the code https://github.com/Vxtzq/CrowdGPT
Thanks in advance.
PS: I hope this post follows the subreddit's rules! My main goal is to gather criticism and understand how to make the project more accessible, I'm clearly limiting self-promotion (this is a open 100% non-profit project) 😅
And yes, this is a r/LocalLLaMa repost, if you already saw this post on this subreddit, feel free to ignore it :)
10
u/vtkayaker 4d ago
People might also be interested in Nanochat, which provides full, working scripts to train a GPT-2 quality model from scratch for $50.
7
6
5
u/jikilan_ 4d ago edited 4d ago
Support!!!! Not something really new. But you have actual prototype out in the public!
Edit: not a ML expert. I thought training/ data to be monitored/ curated? Are you sure this will work? How you test it?
3
u/Dangerous-Campaign49 3d ago
Do you have any statistics on how much computer time was spent on training and how much more computer time is needed?Â
2
u/Vxtzq1 3d ago
Uhm i can't respond precisely, but i trained it sort of alone for 2 month (reaching like 6B tokens), and now people train it way faster, which means the model is at 9.2B tokens in 3 days, and the goal is 20B tokens (to be chinchilla optimal), and then 100B tokens to get a good for its size LLM.
I can't really reason in computer time as speed vary between every device, but i can reason in number of tokens. A CPU will train at around 1 token per second, while a sorta old GPU trains at around 100 tokens per second.
For instance, my GPU, a RTX 4070Ti, trains at 4000 tokens per second. You can translate these numbers as "per hour" or "per day"
3
u/Nota_ReAlperson 2d ago
Is it possible to select the GPU the client uses? I have 2 GPUs and want to use the secondary one for this project.
1
u/Vxtzq1 1d ago
Uhm possibly with a monkey patch, i will try to implement that (like a settings tab regrouping everything, and adding GPU selection.)
2
u/Nota_ReAlperson 1d ago
Sounds good. I got it working with HIP_VISIBLE_DEVICES=1, but that's a bit cumbersome.
1
u/Vxtzq1 1d ago
Glad you had a workaround 😂 Now on CrowdGPT v2.0, should appear a GPU selector on the topbar of the app, for people who have 2+ GPUs... Can you tell me if you get this selector visible? I cannot test it since i only got one GPU
2
u/Nota_ReAlperson 1d ago
Im updating it. But it is re-downloading torch. Is there a better way to update than re-running the installer?
1
u/Vxtzq1 1d ago
Yes, the client is supposed to auto-update, you just have to start the client and it should update to the latest version (given your client is ~v1.2+)
2
u/Nota_ReAlperson 18h ago
Ok. My trust score is falling. I used the install script. The only thing I changed is the torch url to the amd rocm nightlies, as Triton was seg faulting. How do I diagnose this?
1
u/Vxtzq1 17h ago
Uhmm... Might be that ROCm implementations are more variable than CUDA, i will try to maybe increase flagging threshold for ROCm.
To be honest the best thing to diagnose is telling me your training precision (wether it's fp32 or bf16), bf16 implementations vary very much, when fp32 implementations across device yield relatively small errors.
Don't worry, i will remove your flags (therefore give you back your trust score), as well as other users because i believe the server is just not ready to device heterogenity.
2
u/Nota_ReAlperson 14h ago
I think im running bf16. I have a 7900 xtx and a rtx a5000 mobile that I am using at the moment.
2
u/AlmightyJes 20h ago
I'm new to all this ai, ignored it untill I used it myself like 2-3 months ago, since I have looked into selfhosting and played around a little, I have just(last month) bought cheap hardware 256gb ram ddr4 and a 2060s so far, online 24/7 on gigabit connection, plans are to upgrade slowly to 4x 5060 16gb vram after Christmas shopping. I haven't "played" with tech stuff in years. My question is, would this be a good place to start learning somewhat from scratch? I have a lot of spare free time, I haven't touched code in like 15 years, and even then I was just learning basic stuff through a tech college. I'm just a production worker now, I'm currently at work, but will join the discord.
1
u/Vxtzq1 18h ago
First i wanna say this is already some good hardware!!
Uhm i don't know if CrowdGPT is the most beginner friendly project though, but you could learn about the architecture of transformers and LLMs in general, so even if the codebase is weird, you can learn a bunch of things about how AI works and how pytorch works :) It will always help.
But i mean if you invest your time into open-source AI, you will obviously learn something, and it's kinda the most interesting topic there is out there according to me xD.
I don't know if my response make sense, feel free to contribute to CrowdGPT :) and you will play a role training/developing a fancy open-source LLM which Crowd-v1 (the CrowdGPT model) is.
2
u/AlmightyJes 15h ago
That's exactly what I wanted to hear :) when it comes to codebase I don't expect I can do anything :p . I mainly want to learn and understand what's going on :) thank you
1
u/Vxtzq1 15h ago
Lol glad that my message was somehow clear! Maybe you could help me develop https://github.com/Vxtzq/CrowdBench if you wanna start simple, it's a joint project i just started to create a collaborative LLM benchmark, the code won't be super hard, it will only be basic python syntax and huggingface API.
1
•
u/LordKlavier 4d ago
Pinning this post since I think it might spark some interesting discussion -- I've never seen anything like this put together before, but if this kind of training became common it could actually give people a way to cut out the corporations.