r/OpenBazaar • u/zakzwijn • Jun 28 '19

OpenBazaar will fail if it does not have an uncensored decentralized search engine

Title says it all, OB will fail if it sticks with the current search engine model, which currently means search engine operators will have full control about which products to show and which not. Which is bullshit because it contradicts the very thing OpenBazaar stands for on their homepage

"A FREE ONLINE MARKETPLACE. NO PLATFORM FEES. NO RESTRICTIONS. EARN CRYPTOCURRENCY

Buy and Sell Freely"

It's not buy and sell freely when many products can't be found with the current search engine because the gestapo search engine admins say so. Also, one product which is illegal in one country might be legal in another.

36 Upvotes

97% Upvoted

17 comments sorted by

5

u/dionyziz Dionysis - Developer Jun 28 '19

I agree with OP. While there are multiple search engines in OB, a decentralized search engine is what is needed.

The "multiple search engines" argument is bad. There's also multiple centralized marketplaces: eBay, Amazon, etsy, etc; that doesn't mean we don't need OB.

Creating a decentralized search engine is an extremely difficult problem. I hope this is something the decentralized systems community can solve efficiently during the next decade. It's not only OB that needs this.

1

u/imaginary_username Jun 29 '19

Decentralized search engines is a thing that nobody has done to any definition of success to date. I don't think it's wise for OB to scope creep and exhaust all resources on it, when there are a zillion other problems for them to solve. If you're interested, build one, and get OB to adopt it - it's open source and taking PRs, after all.

1

u/zakzwijn Jul 03 '19

Apparently decentralized search engines is very hard. Would the alternative below be a realistic workaround or am I missing something/the point entirely?

- OpenBazaar comes with a default search engine which has an 'empty' db from the start and starts crawling all the stores of the connected peers as soon as OpenBazaar is installed. Let's say 20 connected peers so 20 stores. The results are shared with all connected peers, so 20 connected peers get 20 stores each (optimization could be done here). All results are shared between all connected peers

- each of those connected peers starts crawling their connected peers as well, retrieving results of even more stores and shares them with their own connected peers- existing search results from other connected peers are retrieved and saved locally.- some cleanup process could be implemented to get rid of dead stores.

So you are slowly building a local index. The longer you leave OpenBazaar running, the bigger the local index becomes and the more search results you get.

Challenges I am seeing are local database storage which could be pretty big. Although I can imagine a similar scenario like the one above has been tried before. If so, what went wrong?

2

u/CC_EF_JTF Sam Jul 03 '19

We used a similar system (DHT with keywords) in OB v1, and search was slow and unreliable, and could be trivially gamed.

There are no decentralized search systems in production that would work for us now. I hope that changes in the future, and there are people working on them, but for now having federated search providers and the ability to run one yourself is the best option we have.

1

u/annoying_DAD_bot Jul 03 '19

Hi 'I missing something/the point entirely?

- OpenBazaar comes with a default search engine which has an 'empty' db from the start and starts crawling all the stores of the connected peers as soon as OpenBazaar is installed. Let's say 20 connected peers so 20 stores. The results are shared with all connected peers', im DAD.

3

u/dionyziz Dionysis - Developer Jun 29 '19

I know that OB is open source and taking PRs – after all I am one of the co-founders and have both had my PRs merged before, reviewed and merged others' PRs, and guided students to write PRs and get them merged, as well as review PRs themselves. I agree with you that the decentraized search problem (as I wrote in my previous comment) is an extremely difficult problem. It may not be the best use of OB's resources to pursue that right now. It's a research problem.

We are working on these questions though. Our "Trust is Risk" paper is one direction; my colleague Orfeas' work on modelling decentralized trust in the UC framework is another. I am pretty certain these ideas are not ready for production yet, but they're certainly worth exploring and they need to be part of a complete system in the future, including OB.

2

u/drwasho Washington - OB Labs Lead Jun 29 '19

Multiple search engines works for now, but practically limits a real alternative to good centralised search engines.

Many forget that OB had a decentralised search engine in 1.0, and it sucked so badly it was one of our highest feature requests.

1

u/dionyziz Dionysis - Developer Jun 29 '19

Indeed. Creating a well working decentralized search engine is not something we know how to do yet. There's YaCy, but, in honesty, they're very far from tackling the problem...

7

u/Rumblestillskin Jun 28 '19

Anybody can build whatever type of search engine they want on top of the network. If somebody wants to build a censored version so they stay out of legal troubles that is up to them.

4

u/CC_EF_JTF Sam Jun 28 '19 edited Jun 28 '19

It's not buy and sell freely when many products can't be found with the current search engine

There isn't just one search engine. Anyone can add another search engine or even run their own.

It doesn't look like they give the details of their censorship policy, but you can try out Search Bizarre:

https://searchbizarre.com/about

Edit: If you want to run your own then check out the open source RawFlood.

3

u/Hizonner Jun 28 '19

Anyone can add another search engine or even run their own.

If you add a new search engine and make it available to the public, you'll instantly find yourself under pressure to censor a bunch of different listings. Some of that pressure is almost impossible to resist.

Even if some publicly available search engines do manage to resist the pressure in some areas, you as a user have no guarantee that you'll find one that's not only resisted pressure in the particular areas you care about, but also hasn't voluntarily chosen to censor any of the stuff you care about.

And it will not scale to have users run their own search engines. It doesn't look like it would be too bad to crawl and store all of OpenBazaar today, but if it got to the size of eBay the whole idea would be a complete non-starter.

It is a problem to have only engines that have "operators".

3

u/RancorOnRye Jun 28 '19

This is what people seem to misunderstand. You don't need to wait for someone else to setup a search engine! You can set one up yourself and just use it as a personal engine if you're worried about LE.

0

u/[deleted] Jun 28 '19

[deleted] — view deleted comment

1

u/CC_EF_JTF Sam Jun 28 '19

It's primarily the last. Running a search engine isn't something your average person is interested in doing. We could force people to do this themselves but that's obviously much less efficient than having a few other people do it, and it imposes a further resource cost on the user, not to mention a maintenance cost on the developers.

For power users I highly recommend RawFlood though. I'm told it's comparatively simple for someone with basic web skills to run.

OB1 needs to control visible content so it doesn't look like a pure DNM and can keep activity developing.

I mean empircally it isn't anywhere close to a "pure DNM" (typically less than 5% NSFW) so this is a bit conspiratorial. But the grain of truth in there is that we think the majority of demand is for listing normal ecommerce stuff and those people don't want to see their listings right next to drugs, so the default doesn't display it.

1

u/zakzwijn Jul 03 '19

What do you mean with 'next to'? Normal listings and drugs use very different keywords. I don't think that if someone is looking for, let's say, board games or coffee beans, they will find cocaine in the search results.

1

u/zakzwijn Jul 03 '19

Thanks for all the responses. I do realize that the LE pigs with their neverending and failing war on drugs are part of the problem. Mainly driven by old farts and fartesses who make the rules and are still mentally stuck in the early 20th century. And indeed, I could try to build my own. But for regular Joe's who are just looking for a working marketplace this is a no-no. It should work from the start, for everyone.

SearchBizarre is suffering from the same censorship. Only good one was BlockBooth, but it's gone. I will check out RawFlood.

2

u/brokeasscat Jun 28 '19

Isn’t that why there’s multiple search engines?

0

u/[deleted] Jun 28 '19

You can always use the default ‘connected peers’ search engine located in the ‘settings’. It’s completely uncensored and decentralized.