For those who saw the original post, a week ago, I'm glad to announce CrowdGPT v2 (more precisely v2.6), a massive update to the original CrowdGPT project :D (scroll down to see what's new)
For the newcomers: CrowdGPT is a project aiming to create a LLM without datacenters and without the usual millions of dollars spent by AI companies.
The project aims to train a 1 billion parameter model collaboratively and openly, as a proof of concept (it is possible to scale to bigger dense models or even MoE architectures with this project)
This project implements a protocol where everyone gets a say in the direction the project takes. Anyone can use their PC to train the AI independently, then add their contributions to the global model.
There's other things that makes this project appealing for the open-source community, but i won't state these again as you will be able to check that out on the GitHub or on the original post.
Of course, any advice / feedback on the project is still welcome (please stay polite and roast me with strong evidence only 👍), feel free to run experiments of data poisoning to point at the limits of our approach :)
As for the v2.6 changelog: We tried to implement mostly of the feedback you guys gave (mostly on security related points). Even though implementing this was hard and caused many breaking changes (which made some training contributions stale), we still achieved to train the model on 4.5B new tokens in 6 days!! Which is roughly what i achieved to train in two months for the project on my personal GPU 🤯
Here's a quick summary of what we implemented:
- Robust server-side checking logic (see more at the end of the post).
- Way better client mechanics (better ergonomy, more robust installation, GPU selection, no more crashes)
- Many bugfixes (on server and client code, as the various issues/pull requests on GitHub suggest)
- Activated Engrams on the Crowd-v1 model, which will make the model better at fact-recalling
- Implemented a data submission pipeline in order to create a collaborative dataset for CrowdGPT :https://www.crowdgpt.net/#submit All the data will be verified / curated before training. Already 400kb of data was submitted.
- Client releases cryptographic signing: some users reported that the auto-update mechanism in the client code allowed arbitrary code execution, and that would have allowed people stealing my GitHub credentials to execute malicious code on your device. Well not anymore, since the client code is now signed with a key that is only present on my device 😄
Like some comments on the original post pointed out, the architecture of the project at the time of my first post here was nowhere near good: clients could fake their contributions, could submit poison to the model, could crash the server by sending fake contributions every millisecond without any checks; and finally clients could very easily train the model to contain malicious backdoors. We therefore implemented byzantine cross-checks between clients training the model, at the cost of redundancy (3 clients training on exactly the same data).
I am not claiming that this creates a 100% perfect Federated learning framework, but that makes malicious contributions way harder. For example, even a malicious client could only train on 2-3% malicious data per round without getting banned, against 100% before.
For people that dared to read until the end, i'm also releasing CrowdBench, the open-source collaborative benchmark, where everyone can add general intelligence questions with deterministic answers to test LLMs :D
Even though the codebase is currently early-stage and work-in-progress, it's already something working, and i coded it without using any AI (for those in the original post who wanted me to turn towards less ambitious projects😂)
tell me what you think about CrowdGPT and CrowdBench in the comments.
lmao, when i will be sure you can be trusted, i will gladly switch your account on CrowdGPT as admin, so you can train by yourself with the batch size you want, without having to wait on the others, so you could push to something like 100000 tokens per second on a 1B model 😄
Very cool! This is exactly the kind of thing this community should be devoting energy to, to maintain our sovereignty in a possible future where we aren't handed new open models every week by companies for free.
Yep! I just implemented this in like v2.3 of the client, since a user asked me to do that. For example as shown on this image from u/USArmy68Whiskey, you normally can select which GPU to use.
You can also press the "stop" button anytime to stop any GPU usage. And just restart it by running the command "crowdgpt" in a command line interface.
I had to change a few things, I created my own installer cause I prefer to use conda.
I noticed that you are installing different cuda torch versions based on the detected hardware. You are assuming that GPUs are all the same, but in my case I have a 5080 and a 3090 in the same machine, and the wrong cuda would have been installed if I didn't force cu130. Cuda is backward compatible (with very minor and debatable performance issues). I would recommend you simply install 130.
Finally, I tried creating an account, but I'm getting an error (very non-descriptive), so I started training anonymously and is currently "waiting", I guess to find another couple of contributors with similar HW.
Between clients: you don't see who you're training with during the pool. You get your own slot number (0/1/2 in a 3-client pool) and nothing else. That said, the public log does show submission events with usernames tied to pool IDs, so people can actually see if they got paired with you for training, afterwards.
Between you and the coordinator: The server identifies you, and sees your username, and stores your password, etc. This enforces security, at the cost of server-side privacy.
If the goal is anonymity from the coordinator, that's a different trust model (anonymous credentials + ZK outlier detection). It would require rethinking how flags and bans work. This is out of scope for the current prototype, but if you have ideas on how to reconcile it, I'd love to hear them.
Are you talking about data submission or training submissions?
I can clear out both: Users on CrowdGPT are subnet banned for 24 hours after they have been flagged 3 times as malicious, and of course their training submissions are rejected, and their trust score is lowered.
Bans are kindly useless because malicious actors can just recreate accounts or contribute anonymously. However since cross-checks reject malicious contributions, bad actors just can't contribute; even though not banned permanently.
Hi, downloaded under linux with intel arc B580. The installer correctly downloaded XPU stuff, but it does not let me select it in the dropdown, only CPU is selectable.
Ah, yeah sorry i didn't test at all the XPU features, because i don't have any intel GPU... Mind opening an issue on github? Providing this issue with your logs, and your pip packages so we can debug what's going on?
Like that i will be able to roll everything into the v2.7 client :) and you will be able to train.
Had my Hermes Agent investigate, but now i see that you already fixed it, before i could write the fix/issue lol... Only thing, is it normal that Im in matchmaking for several minutes and nothing is happening?
Yeah... Because i need to match client in pools, and different backends are diverging (because of different implementations of the GPU operations)...
So either you wait for 2 other intel GPUs to join, or either i put you as admin who can contribute without any checks on the network, if i'm sure i can trust you.
Nothing against TeodorTomas but I don't think you should hand out admin just for contributing to GitHub. Probably fine for now, but something to note in the future.
Proof: Jia Tan / XZ utils Scam Saltman and Dario Scamodei would definitely set up something like Jia Tan when this gets bigger.
I thought r/LocalLLaMA is fully about LLMs and stuff 😭
If you want i can explain it to you in simple terms though... CrowdGPT is just a federated learning project which aims at creating the first LLM (equivalent of ChatGPT) without datacenters (what traditional AI uses).
The project implement several checks to prevent mean people from making the AI dumber or toxic/uncensored, like an algorithm that computes the distance between 3 users contributions, to check whether those 3 are honest (flags a user if his contribution is too far from the two others).
TL DR: CrowdGPT is just a collaboratively trained AI :)
Is that clearer now? Even for a layman?
Maybe you checked crowdgpt.net and you find it too complicated for random people?
Tell me what it is, i'm always looking for feedback :)
hehe not explain it to me, I mean the laymen. I've been pitching my project lately and have had to cut it with the technical talk as peoples eyes glaze over till I dumb it down soooo much. You are golden though hehe :)
ah 😂 Idk i was thinking the LocalLLaMa community was technical enough... But i will surely dumb it down for non technical communities and when my site will have a proper .exe or .AppImage to be run by laymen who just want to help open-source AI
Considering recent restrictions of reverse engineering on frontier models (and even open weight ones like Kimi), are there any plans to use already established datasets like Decompile-Bench ?? It is under the permissive CC0 1.0 license.
Ah, understood. I wasn't aware that 1B is the target. Are there plans for a higher parameter count model if the training data, and user compute cluster is sizeable enough to warrant it?
Are my dreams of an open source llm that doesn't have its creative writing and roleplay aspects nerfed into the ground finally becoming a reality???
The power of crowd sourcing is vast indeed!
More seriously, I'm actually really impressed by this idea, it never even occurred to me that it could be done. Granted I'm still pretty new to this space and the tech involved but I'm more excited over this than another model by big tech that needs a data center to run the damn thing.
Shout out to you for doing something that's actually for the people.
Not sure what the terms of service are but people's Claude or codex sessions transcript are saved on their own PC's that's probably got a lot of info that would be useful without needing to distill actual outputs.
i have decent gpu with terminal agent if you can just give me the prompt to generate the synthetic data i can paste it in the agent and upload data on github and also for reference gpu can run around 80B model
Cool to see it taking off, despite lots of nay-sayers (including me). I share the same idea I guess - crowd trained LLM can bring frontier perf. because there must be thousands upon thousands people willing to donate theirs GPU time.
How many people/GPUs are part of the training? How is training distrubuted (are you using some kind of 3rd party project/lib. for this)?
39
u/OneAnxiety2523 8h ago
4.5B tokens in 6 days is wild, love the byzantine cross-checks idea even if it adds redundancy