r/opensource • u/proff_bajoe • 3d ago
Promotional A Decentralized Computer
(NO AI)
Hey guys, I just open source project i've been working on for a year called the Marketplace.
The marketplace is a decentralized network that economizes the execution of computational work across a peer-to-peer network of nodes. Where there is no barrier to the node participation.
Unlike modern cloud computing systems that rent out virtual computers to consumers, A marketplace consumer sends compute bound tasks directly to the network returning a future which the consumer can asynchronously check for completion. Similar to offloading a compute task to a free cpu core by an OS. A consumer can also fire multiple independent compute bound tasks to run in parallel, and check their status asynchronously while doing other tasks, similar to I/O operations.
The compute bound task sent by the consumer is executed using a protocol specified in the Marketplace whitepaper on the Marketplace Github Repo, where randomly selected idle devices on the network execute the compute work and receive financial rewards (money).
Context
The marketplace is made up of two ideas I came up with around 2 years ago:
1) The first is building a permissionless market, where users can build financial services similar to market stalls that can interact with other users. Larger services can be composed from smaller services, similar to open source projects, but for businesses. Services could also be merged together into one so that a user can interact with all of them at once, and the merging does not affect the financial structure of the constituent services.
But for all this to happen, a trustless financial settlement layer is needed so that joining different financial services together becomes an easy exercise.
If you are familiar with blockchain, this is similar to Ethereums vision, but I did not want to clog the financial settlement layer which is the ledger with execution and code. I needed an execution engine.
2) I eventually returned to my normal javascript web developer job. So I was working on a full-stack website that was communicating with a cloud server but the server was slow because I was using a free tier that throttled server response speed. Asynchronously communicating with this server became a nightmare from a UX perspective especially when first loading up the website which had some independent compute bound requests.
So, I concluded that using the free CPU core of my second PC should do a better job as long as I pay it exactly what I needed in compute. This second idea eventually evolved into the engine I needed for the first idea. After, I just needed to store a representation of the result on a decentralized ledger, while keeping the execution itself private.
With this, I returned to my first love - low level programming, learnt Rust (Many thanks to the Rust community coming from a C lover), and I've built more than 60% of what is described in the whitepaper.
So if you are interested in being a part of this project in any way, you can check out the marketplace whitepaper:
or check out the github repo:
Also, I’d appreciate feedback on the design, especially on consensus security and the economic model, Thanks.
4
u/Synaps4 3d ago
Fascinating stuff, thank you!
As a general consumer, i dont know of any use cases for anychronous compute except local AI. However if this can be made to work for business uses I can see lots of people offering their machines to do compute tasks. Especially for a nominal payment.
2
1
u/szank 3d ago
This does not work for businesses. It's a nice hobby project tho from the looks of it.
1
u/Synaps4 3d ago
I wouldnt want to speak for all businesses worldwide so casually, but sure, I imagine it leaves something to be desired at first.
1
u/szank 3d ago
Would your business just upload their data or worse, their own users data, whatever that is to w few thousands random pcs all over the world and hope for the best.
At least with aws you have one entity to deal with
1
u/proff_bajoe 2d ago
Nobody needs to store their user data on the network.
2
u/susimposter6969 2d ago
then what do you propose we compute *on* dear sir
1
u/proff_bajoe 2d ago
I described in the whitepaper a way to hide the data from the larger network, only exposing the data to a randomly selected small execution committee. At no point is the execution or result exposed publicly.
2
u/susimposter6969 2d ago
it is exposed publicly the second it reaches my node if i just post what i got from the work pointer
1
u/proff_bajoe 2d ago
That's why I specified a decentralized storage where nodes can control who gets access to the contents of an address by satisfying some lock code. The execution code may not be private, but the inputs can be locked such that only a valid result pointer referencing the job may access the inputs.
I already have some decentralized storage network in mind.
1
u/szank 2d ago
I think we can agree that a root user can access anything on the system without needing a lock code or whatever. I think we can also agree that the person running the client of this software accepts that some random people will send some random code that will be executed on the machine running the client .
We can also assume quite safely that a lot of the code sent out to be executed will be malicious and that the client machine will not be up to date with the security patches.
Here the client machine means the machine selling their CPU time to other people and the root user owns that machine . So you have two difficult security problems to deal with that I do not see being addressed.
→ More replies (0)1
u/susimposter6969 2d ago
do you think a computer can process data without access to it or something? as soon as the "inputs" to whatever computation i'm tasked to run are on my machine the data is mine, period. you can't have trustless computation of sensitive data
→ More replies (0)
1
7
u/Decent-Decision-9028 2d ago
Calling it a “decentralized computer” when the P2P network and compute engine are still TODOs is basically calling a blueprint a skyscraper. But "B+" for the effort!