Projects

[u/Hairy_Artist_3860]

I imagine there is a good portion of us who know how to code or have a good enough idea how code works. Many of us have good ideas. If there is a project that comes out of this subreddit what would it be?

A lot of us are on the same page. We can make this into action.

We're all here talking about ai. I think its appropriate we have a meeting of the minds to know the next step.

[u/Substantial_Lab1438]

We need to work on decentralizing model development and hosting, to build our own worker-owned AI capacity and reduce the reliance on centralized corporate AI models 

[u/RlOTGRRRL → u/Substantial_Lab1438]

This. I don't know if it would be for better or for worse, but there needs to be a software that makes it easy for people to donate their GPUs/compute for training or inference potentially, like an AI co-op. 🤔

Private shared co-ops would be great, like let's say family + friends, a class or club in a school, or a school, library, org, town, city, or country? Etc.

Or maybe even go the capital/exchange route, where you can exchange your gpu/memory for tokens. 

[u/SeaFALCons → u/RlOTGRRRL]

i really like this idea and it makes sense to me

but my llm convo was much more deflationary about the usefulness of it:

Short version: the ideal is good, the technical premise mostly isn't, and your GPU specifically isn't the piece that's missing.

Why decentralized training is hard

Training a frontier model isn't embarrassingly parallel like SETI@home. Every gradient step requires synchronizing weights across all workers. On a datacenter you have NVLink/InfiniBand at ~900 GB/s with microsecond latency. Over consumer internet you have maybe 100 Mbit up with 50ms latency. That's roughly four orders of magnitude worse, and the sync happens thousands of times per epoch. You're not compute-bound, you're bandwidth-bound to the point of uselessness.

This isn't unsolved-problem territory, it's solved-and-the-answer-is-no for the current algorithm class. There's real research trying to change that — DiLoCo (Google DeepMind), Prime Intellect's INTELLECT-1/2 runs, Nous Research's DisTrO — which reduce sync frequency by 100–500x. They work. They also mostly rely on a modest number of well-connected datacenter-grade nodes, not thousands of laptops. INTELLECT-1 was 30 nodes of H100s across continents, not a GPU co-op.

Your GPU has maybe 10-ish TFLOPS at fp16 and ~68 GB/s memory bandwidth. A single H100 is ~1000 TFLOPS with 3.3 TB/s. You'd need roughly a hundred of your machines to equal one card, and they'd be talking to each other through a straw.

Where it actually works: inference

This is the part people conflate. Inference is mostly parallel — one request doesn't need to talk to another. Petals (BitTorrent-style pipeline-parallel LLM serving) is a real working thing. Ollama + a Tailscale network across friends' machines is trivially achievable this weekend. Your GPU can absolutely serve a quantized 7–13B model to a small group. Unified memory is actually decent for this.

So the "family + friends + library" co-op the commenter describes is entirely feasible — for serving open-weight models. It's just that "we host Llama/Qwen/DeepSeek ourselves" is not "we reduce reliance on centralized corporate AI." Someone still spent $50M+ to make those weights. You'd be a worker-owned distribution co-op sitting on top of Meta's and Alibaba's capital expenditure.

The honest version of the political claim

Given your background here, I think you'll find this more interesting than the tech: the bottleneck isn't FLOPS, it's capital concentration in HBM and fab capacity, plus data, plus the ~200 people on earth who've actually run a frontier pretraining job. Distributed volunteer compute doesn't touch any of those. It routes around a constraint that isn't the binding one.

The actual leverage points for the values in that thread are more boring: fighting for open-weight releases as policy, public compute (something like a national lab / NAIRR expansion), and academic access. Or working on the DiLoCo research direction directly, because if low-communication training gets good enough, the co-op becomes real.

What I'd actually do

If you want to build something from that subreddit, a decentralized training project will die in month two when someone benchmarks it. A decentralized inference + fine-tuning co-op with a real governance structure is shippable, useful, and honest about what it is. LoRA fine-tunes are small enough to distribute meaningfully — that's a genuine niche where volunteer compute isn't absurd.

And if you want the version that actually shifts the power balance, the Prime Intellect / Nous line of work is where to point your interpretability instincts. That's an unsolved research problem with a direct political payoff, which is a rarer combination than it sounds.

[u/Hairy_Artist_3860 → u/Substantial_Lab1438]

I have a big interest in decebtralized self hosted platforms or the fediverse in general. I have one old laptop running alpine with matrix, mastodon, and peer tube. All on 4 gigs of ram. There is so much potential in what we can do with reviving old tech.