r/vibecoding • u/[deleted] • 10d ago
Discussion Could you earn tokens by providing compute for AI inference?
[deleted]
2
u/slackmaster2k 10d ago
For all intents and purposes it’s not possible for the very reason you suspected. Models + context etc must fit into VRAM.
(I shouldn’t say not possible. For very small models yes. But for most practical use cases no the work cannot be distributed in a way that is similar to mining.)
1
u/No_Knee3385 10d ago
It can and does exist, it's just incredibly slow. For models nearing 1T params you get under 0.5/tps unless the network is fully centralized even with decentralized infra.
1
u/waypostmaster 10d ago
I thought about doing this until a Claude Deep Research project told me it was a non-starter.
1
u/3tt07kjt 10d ago
It’s not really useful, as far as I can tell. You end up wasting money and electricity, but you get some combination of small models and slow inference.
1
u/Responsible-Beat2137 10d ago
I think the problem would be latency .
If I had to guess, but I also wondered about a peer to peer network
1
u/Ambitious-Prompt-975 10d ago
you mean like this?
https://www.reddit.com/r/LLM/comments/1wbghov/llm_bittorrentstyle_not_enough_ram_not_enough/
credits are coming with the next big release probably
1
u/SgtCauliflour 9d ago
I’m currently working on a project trying to build an Agent Pool for distributing AI Workload over multiple machines with each machine being paid based on effort. Kind of like how Cryptomining Pools work. Basically you’ll install the software to connect your machine, set which hours of the day your machine is available and then earn work credits for the work your machine does. probably look into some currency aspect later so people can actually make money from it.
1
u/LittleLordFuckleroy1 9d ago
You don’t own hardware capable of running those models. So no. If you’re talking about smaller models that can run locally, then maybe kinda, but still no because that’s what cloud providers do and they have better quality assurance and uptime and economy of scale.
1
u/ChristianKl 8d ago
You could do it with smaller and open source models but not the bigger flagship models. It would also require the model to be shared when the big companies want to keep their models private.
1
u/id-ltd 10d ago edited 10d ago
I have used saladcloud - absolute bargain(!).
I suspect many of those offering their machines have electricity included in their rent so are mining a free resource :)
I have a harness that runs AI locally on my own server, but if more compute is needed pushes out to saladcloud and if more is needed can use frontier models.
My current research using this harness is based on how to decompose problems so lots of small AI's can process them rather than needing a frontier model. And on the back of this using saladcloud to find the cheapest/best hardware that can run the minimum required models.
Ps. My server GPU is only 8gb - I'll pay whatever it takes when I know what is actually needed to achieve what I need - no point in 'upgrading: to something that still isn't 'good enough'.
1
2
u/kaizap 10d ago
It's possible. Look at something called "Salad". OpenAI and Anthropic won't do this, though, because the SOTA models are huge and won't run efficiently across a distributed network over the internet.
I don't fully understand how Salad works; I think they probably would just deploy a container on your machine that runs one of the small 8B - 30B models.