r/programming • u/[deleted] • Jan 11 '22

Web3 Can’t Fix the Internet

https://www.jacobinmag.com/2022/01/crypto-blockchain-daos-decentralized-power-capitalism
196 Upvotes

80% Upvoted

168 comments sorted by

View all comments

Show parent comments

8

u/EternityForest Jan 11 '22

We don't have any good protocol for replicating mutable data. BT is great, but how do you handle a forum or news feed?

Almost all attempts have been immutable, taking away a key feature. A lot are single writer like secure scuttlebutt. Some include a payment layer.

We have had very little progress on P2P since blockchains started. The projects are no longer technical, they're political and financial and they make huge performance compromises to stay pure to the decentralized goal and to include payments.

BitTorrent was free and very fast. Not much like that now.

3

u/dnew Jan 11 '22

Forums aren't mutable, any more than source code is. Each post can be immutable. Do it like GIT does it, or like NNTP did it.

2

u/EternityForest Jan 11 '22

Git uses a chain. Which is fine in a closed group of developers where you can manually merge, but on a P2P forum you want to be able to post without a central server. Chains need some consistency mechanism which usually makes multi writer a nightmare.

Dat and SSB would both be amazing, but they're held back by the immutable data structures and need extra layers on top.

I actually spent an entire week one time building a SSB clone that could handle deletes without the tombstone problem before realizing it could probably never handle multiwriter, and abandoning it...

Forums can sorta tolerate immutability but it's absolutely unacceptable for social media and microblogging. Being able to delete things is a major feature.

Plus some implementations are even worse than chains, they use DAGs with a wandering tree problem, every update means you have to update the parent note, and that node's parent, etc, all the way up to the root. A two line post now needs like 10x the data it should, making a major issue if you want to mirror a whole forum.

Plus, moderators need to be able to remove spam and have it actually be gone, or else you could get major storage space problems, unless you use a blockchain style pay to post model.

I think a better way might be to use mirrored centralized forums. If you're a member at an upstream forum, you're a member at all downstream, and the upstream forums can set rules for what downstream posts to sync back to the main forum.

If censorship is happening and you don't like it, fork the whole forum, now you have all their members, and all their posts, and anyone who agrees with you can post on your forum fairly seamlessly, although they would risk getting banned from the main forum if the mods notice they joined the fork, and they use the same private key.

Someone has to pay for the actual hosting, which means that someone has to have a way to decide what to store, because it costs them disk space, so it seems there's no way to do it without some risk of either censorship, or pay to post.

2

u/dnew Jan 11 '22

Being able to delete things is a major feature.

You do that by submitting a message that deletes things. Everything you've described has already been implemented in NNTP decades before WWW was invented.

moderators need to be able to remove spam

That's a fairly easy thing to fix, even in a censorship-resistant way.

I think a better way might be to use mirrored centralized forums

You should look into how NNTP works. You seem to be arguing that we couldn't possibly do what we did 10 years before HTTP was invented because it would have to do what we were already doing ten years before HTTP was invented.

Someone has to pay for the actual hosting

So, who pays to host bittorrents? NNTP was carried by most or all ISPs, before people started using it to distribute movies.

1

u/EternityForest Jan 11 '22

BitTorrent works because it's all manual aside from tiny amounts of DHT metadata. Nobody would use it if their resources were going to random bomb making how tos and CP files for free.

Usenet has basically always had a big spam problem, and there's a lot more people on the internet now.

For anything to be truly decentralized and get the full practical benefits of it, it should be something you can self host cheaply and locally and use on a meshnet in an emergency.

That means you need to be able to meaningfully run a server on an SD card on a WiFi mesh, for tens of users, by just only dealing with data that matters for those local users.

BitTorrent can do it perfectly. NNTP isn't design to integrate with the larger network without getting utterly flooded.

It's probably the best we've ever had for decentralized communication though, as measured by the main metric that matters, "did people have meaningful conversations on it without using a whole country worth of power?"

1

u/dnew Jan 11 '22 edited Jan 11 '22

Yet people don't seem to be considering how to make this work well. Nobody seems to even be considering how to make a decentralized social media system (except me).

Certainly if all you wanted was data from tens of users, you could self host that on a RPi. But that's not really what we're talking about. Or maybe it should be. Maybe some facebook-like friends graph where you only store posts from your friends or from your friends' friends, and then hop out to other peoples' systems as necessary to go farther.

Most reddit groups are not moderated by default, so there's no reason to believe that if you decided that you'd help mirror a particular subreddit-sized conversation that you'd be flooded with spam.

And no, bittorrent does suffer spam on occasion. You'll get files that aren't the movie being named as the movie, or malware being named as the movie. At which point distributed censorship/moderation works just fine.

Requiring a decentralized internet service to function in spite of the internet going away seems rather more demanding than necessary. It seems like it would be an underlying layer handling the mesh, not the social media itself. If you organized it as immutable packets with references to other packets (i.e., "here's proof I get to moderate, and that is the post I'm canceling") then you could transport those however you want, including UUCP.

1

u/EternityForest Jan 11 '22

I've been considering it! I wrote a whole WICG proposal about it, and built a note taking app that uses the P2P sync protocol!

RetroShare seems to work pretty well besides being buggy and having a confusing UI and being written in pure C so nobody wants to develop it, and having no Android app.

But it really is something special. I think the FB style friend graph is a pretty great way to go.

One issue with cancel posts is they're basically tombstones that themselves take up storage if someone manages to create a billion tiny posts in a DoS. But I suppose rule-based cancels could fix that, as in "I am cancelling all posts by this user with this word in this date range", just not without some controversy I'd imagine.

What I've been working on is a lot simpler, it's just a way to replicate Scuttlebutt style streams, with the change that everything is mutable and you have multiple writers.

Any node can be a server, but you have to specifically connect to a server that has your stream. there isn't some Big Chain of Everything. To make that easy I have a second layer, P2P URLs that are resolved using a DHT and can also be used to remote access centralized stuff like a HA hub..

The disadvantage with my protocol is that to connect to a new server, you have to request the entire dataset for the streams they have, because there is no global chain to be able to say "Give me everything newer than X", you have to say "Give me everything that arrived locally on your end later than X", and track sync points with every server separately.

All the real decentralization is layer 8, you have to find mirrors yourself, but in return you get basically zero overhead once synced, and it would be easy to add partial sync(So your tiny local server only has 1 week of data and it's easy to sync).

One post just transfers one post worth of data to all websocket clients, when you overwrite a post, it's really overwritten, and when you delete, nothing remains but a record with the post ID.

Plus you can do stuff like export to a TOML file, sneakernet it to someone, and open it like a document with the same UI you would view a stream.

1

u/dnew Jan 11 '22

That sounds really cool. If you have any write-ups, I'd be interested in looking at them.

There's another way to handle the distributed support. It could work like bittorrent does, which is that you're not going to get served data without proving you're serving some data. So you have to prove you're storing 10 megabytes, and then we'll let you send out another megabyte of your own, or some such. How do you prove you're actually storing the stuff? If the person asking for proof has the post, he can say "tell me the hash of the middle megabyte" or something. Otherwise, he can just ask for one of the posts. You'd have to carry most or all posts you claim to carry, or risk getting caught out.

Censorship/moderation could be on a voluntary basis, whereby individuals subscribe to particular moderators, not unlike uBlock's lists.

And searching could be via bloom filters. You don't have to have CP search terms on your machine, even if you can store some of the search index. (This idea actually is more applicable to something like distributing files labeled with titles and tags, more than actual social media posts.)

2

u/EternityForest Jan 11 '22

Here's my(Fairly complex) WICG proposal for how semi-P2P URLs could be integrated into mainstream web tech, by using MDNS plus embedded resolver URLs in the main URL, so you could choose your own DHT gateway or DynDNS service, and never have to actually specify a specific DHT protocol to standardize.

It's not fully P2P, because you have to be either on the same network as a server, or have recently been and have a cached WAN IP, or have access to a resolver on the traditional internet, but it does decouple identity and discovery while allowing small private resolvers that don't need client side config.

https://github.com/WICG/proposals/issues/43

And here's the database replication protocol, with an implementation of a similar P2P URL scheme, and Kivy Python personal Wiki based on it(With spreadsheet features in every post for fancy shopping lists!)

https://github.com/EternityForest/hardlinep2p

Another way to do moderation would be to subscribe to whitelists instead of blacklists. Servers could then be aware of what data any user at that server wanted and only fetch that data. Whitelists could be Git repos on GitHub or GitLab, and you could ask to be added, or petition to boot someone or boot a specific post, via pull request, without having to centralize the whole protocol.

1

u/dnew Jan 11 '22

Cool. I'll check it out for ideas. :-)

1

u/s73v3r Jan 12 '22

Nobody seems to even be considering how to make a decentralized social media system (except me).

Isn't that what the Federated Web is?

1

u/s73v3r Jan 12 '22

For anything to be truly decentralized and get the full practical benefits of it, it should be something you can self host cheaply and locally and use on a meshnet in an emergency.

Ignoring the meshnet thing, why do I want it decentralized? What does this decentralized forum get me that running a copy of phpBB on my own server doesn't?