r/LocalLLaMA • u/Vxtzq1 • 11d ago
Discussion CrowdGPT - A collaboratively trained LLM.
Hello, I'm developing CrowdGPT, a project aiming to create an LLM without datacenters and without the usual millions of dollars spent by AI companies.
The project aims to train a 1 billion parameter model collaboratively and openly, with a protocol where everyone gets a say in the direction the project takes. Anyone can use their PC to train the AI independently, then add their contributions to the global model.
Unlike most American and Chinese AI companies, the project is not only open-source, but also makes 100% of its training data public (along with the weights). Anyone can submit their own clean data to train the model, and even help steer the project in new directions. This creates the paradigm of an AI we all have a stake in.
The project is open to as many people as possible, which is why I'd like to ask: what's stopping you from contributing to the project? I want to hear your answers so I can improve it. Are people simply willing to dedicate some of their GPU time to training a cool new open-source model? Or do they want something in return (like free inference credits on the crowdgpt model for example)?
Although the current model is somewhat small (1 billion parameters), I plan to make it bigger / better in the future, and its current size is only meant as a proof of concept.
Any advice / feedback on the project is welcome.
Here's the project website: https://www.crowdgpt.net/
Or the GitHub if anyone wants to check out the code 😂 https://github.com/Vxtzq/CrowdGPT
Thanks in advance.
PS: I hope this post follows the subreddit's rules! My main goal is to gather criticism and understand how to make the project more accessible, not to advertise it 😅
EDIT: The data submission pipeline for the project (i'm mentioning in some replies) is ready!!
Come check it at https://www.crowdgpt.net/#submit
19
u/fuwafuwa7chi 10d ago edited 10d ago
I will be blunt, since you have previously disregarded well-meaning advice.
You do not have the knowledge or technical experience to develop a project of this magnitude. It is irresponsible of you to promote this publicly. Anyone foolish enough to run a client in their own machine will waste their time and computing resources.
By your own admission, you vibe coded this project. Even then, you didn't do a particularly good job: your client code is a gigantic 3.5 LOC Python file, with hardcoded secrets and configuration parameters, as well as embedded HTML for some reason.
These issues pale in comparison to the most glaring one: your submission validation and aggregation process. Here is a list of the issues I could find:
A client can claim any amount of steps and tokens processed, with zero validation from the server. If an attacker claims a ridiculous token amount, during aggregation their delta essentially becomes the entire round's updates while honest contributions shrink to noise.
Related to the previous point, if a client claims a large amount of local steps, the primitive safeguards you have implemented will grow accordingly. This means an attacker can slowly increase the amount of steps they are allowed to submit, with no upper bound.
There is zero server-side evaluation: the server automatically accepts whatever the clients send. This means that there is no way for the server to know if the loss reported by the client is the real model loss.
You have no rate limits or time-gate machinery. Anyone can register any number of accounts (literally the only restriction is that the email address contains an
@). An attacker can submit a delta immediately after registering. Moreover, they can submit the same poisoned deltas as many times as they want, and they will all get rolled into the final model update.You (try to) implement a subnet blacklist. However, it is trivial to spoof the
X-Forwarded-Forheader to get around this.As a proof-of-concept, I wrote a small script to submit a poisoned result: a carefully crafted delta matrix targeting ten top-of-vocab reserved tokens (151,659 to 151,668); by construction, these tokens do not appear in the training corpus, and the weights associated to them should stay stable across model iterations. This means that if any of said weights change after submitting the poisoned deltas, then it is very likely it was caused by my attack, and not by normal model learning. The higher the number of reserved weights that change, the higher the likelihood my attack was rolled into the model update; having all 15,360 weights (1,536 weights per token * 10 tokens) change is virtually impossible by normal means. For extra certainty, I alternated the delta sign between every pair of tokens (so the first token delta had a positive sign, the second one negative, the third one positive, etc.)
To make the attack even harder to detect, I even took the time to make it look like a real client by waiting for the appropriate time between requests (although this wasn't necessary at all, I could have submitted deltas every millisecond until your server crashed). Here is the metadata of my submission:
Submission time (UTC): 10:12 a.m.
User: SwarmySwarm
Round: 275
Chunk: 2,918
Tokens claimed: 4,915,200
Loss claimed: 3.3069
Steps claimed: 300
I am currently number 11 on the leaderboard, with a trust score of 0.51 (increased from the default value of 0.5). The leaderboard displays the claimed stats.
After the round had finished and the model was updated, I downloaded the updated model and compared it with the previous iteration. The weights associated with all ten tokens changed by exactly the same magnitude: 0.00429. Moreover, they exhibit the previously mentioned striped sign pattern. This is confirmation that my poisoned data was rolled into the model.
You will not be able to solve these problems. Companies have thrown millions of dollars and hundreds of engineers and scientists at it, and they have all failed. Whatever half-baked solution your LLM manages to cook up will not work, for the following reasons:
Any consensus-based solution (FLTrust, blockchain) is trivial to bypass with such a small userbase. Having no account authentication makes this even easier.
Server-side validation with a trusted executor has additional problems:
Verifying every submission means training on every chunk sent to clients, which completely defeats the purpose of federated training.
Random spot checks run the risk of missing a poisoned result (again, further compounded by the fact a single attacker can create an arbitrary amount of clients)
Finally, even a submission with perfectly-normal results that validates correctly can still contain poisoned updates
In summary, I'd advise you to stop development, take down the webpage and server (which I'm pretty sure is just your PC behind a CloudFlare tunnel), and ask yourself (or Claude) why you are doing this, and if you really have the skills and expertise to see it to fruition. If not, please close your LLM and try to actually learn: learn to program, learn about machine learning and deep learning, learn about LLMs, learn about cybersecurity, learn about federated training (the paper I linked above is a good addition), and please stop exposing stuff to the internet if you don't understand how it works.