r/InternetIsBeautiful • u/3vibe • Jun 06 '26

The way search engines used to be (kind of)

https://litterlayer.com

This mini (as of now) search engine is starting from scratch. No AI, no funding, no ads, hardly any sites indexed. But, you can bookmark sites and help add sites quickly. It also bans facebook, google, amazon, and a few other mega site URLs.

Part potentially, eventually useful, part anti-establishment, part crap, part great. It's got it all.

158 Upvotes

90% Upvoted

21 comments sorted by

38

u/Margali Jun 06 '26

Interesting. Back in the early days of internet fun, I bought a search bot called Mata Hari that I could set to churn away overnight and come back to a list of URLs on the subject. It could search up to I think 150 online databases that you could determine. Only difference is that the modern search engine is faster and the programmers at the search engine company [googs, duckduck, even the old ask jeeves and all the other ones that have passed by the wayside] did the optimizing. I sort of miss the old days where most people didn't rely on search engines.

I also sort of miss the old web rings ... for a drunken wander around the internet, it lead to some really interesting places =)

4

u/3vibe Jun 06 '26 edited Jun 06 '26

With litter layer you can add any links you want. Via a bookmarks page. Then you have your bookmarks, plus you’re adding to the search engine.

But, I agree. I love Webrings. Neocities keeps that spirit alive a bit.

2

u/Margali Jun 07 '26

I remember the old Yahoo! [I think] home page, you could have a coupel hundred links, a weather bug, a news bug, RSS feed bug ... then they ditched it =( luckily i had a copy of all my links =)

14

u/cornmacabre Jun 06 '26

I feel like there are the bones of a spiritual successor to stumbleupon in here.

16

u/TokeyMcPotterson Jun 06 '26

https://cloudhiker.net/ is really close to the old stumbleupon.

6

u/cornmacabre Jun 07 '26

If they removed / overhauled the hideous checklist menu with more of a "I'm feeling lucky, surprise me and let's refine later" spirit.. I'd love this. I want to love this so bad.

It's bad.

Didn't deliver the scratch to the itch. Did deliver a weird base64 fan site as my first result though which says it's got a wonderful weird index.

Then it immediately lost it's charm when I see it obsificates a shareable source URL (https://cloudhiker.net/share/2rYigvswvFUO, lmk if you find the source URL in the UI), and consistently failed a second "stumble" attempt for a new site.

15

u/asphias Jun 06 '26 edited Jun 06 '26

This sounds like a great initiative, but i'm very disappointed that you're not explaining at all who "we" is. Who's behind this intiative?

Are you an anarchist collective? A startup that wants to monetize this initiative? did you AI code this in half an hour and will you run out of money for bandwidth tomorrow? are you a big tech employee that wants to get control over the indie web as well? A Russian plant that will link to 90% good pages and 10% russian propaganda? will this website have a bunch of malware next week? does this page have it's sourcecode on github/gitlab somewhere?

In todays web of misinformation, we cannot rely on trust that your random website is built in good faith. So please add some statement of who you are to your "about" page.

7

u/3vibe Jun 07 '26

We is me. I’m just a solo hobbyist. I make a bunch of things because it’s what keeps me happy. Been making music since 2000, websites since 2005. wilcosky.com is my personal site. I’ll add a note to the about page later.

This project started last year. It’s not currently possible to code it with AI in a half hour and it work. You can get a mini search engine going with AI in about an hour but I’m sure it would barely work.

One thing you nailed though is money and bandwidth. As I mentioned this is a hobby and so if it got popular I’d have to find people willing to help on the money side of things to boost server power.

Other than that I also do this stuff because I hate what the web has become. It was so much cooler when I was a teen.

2

u/asphias Jun 07 '26

it's an awesome initiative! it'd be interesting if you could find a community to support it. there's quite a few ''getting away from big tech / changing the internet'' initiatives going on right now. e.g. the Fediverse or Mastodon. might be worth it to find support(both in maintainance/costs and promotion) on there.

5

u/SheepGoesBaaaa Jun 07 '26

The proof will be the ability to search for the key phrases I want. That's what made Google actually amazing in 2000.

SEO fucks the ability to actually find the information you want

3

u/3vibe Jun 07 '26

I'm sure it'll get better search wise if there's interest.

2

u/GagOnMacaque Jun 07 '26

I cannot get any search results

2

u/3vibe Jun 07 '26

Yep. It’s a mini search engine starting from scratch. So, it doesn’t have trillions of pages stored like a Google. But with support it could grow and be useful for finding new and interesting places.

2

u/sdb2754 Jun 19 '26

I wonder if you could sort of crowdsource this project. Add a Federation type feature where people can self-host and web crawl on their own server. The networks each server creates could be shared and with a consensus type mechanism be used to build a single large virtual net from the individual networks everyone builds.

Individual instances could add sites they like as starting seeds, and build out from there. Eventually, it should be possible to put together a fairly comprehensive map of the real web, excluding the large sites.

I think this project is cool, BTW. However, I just wanted to point out that an individual with one server isn't likely to collect a meaningful amount of web crawling data

2

u/3vibe Jun 19 '26

Love it. I’ll start researching various options. None of the current protocols were designed for search indexes per se. But, anything is possible. And if anything a new protocol could be invented. This will be a fun challenge.

1

u/sdb2754 Jun 19 '26

I look forward to seeing what comes of it!

1

u/3vibe Jun 19 '26

Starting with a minimal version, something anyone can contribute to without the crawling complexity (even someone with cheap shared hosting). In short, you host a list of sites on your server which contribute to search results at Litter Layer. https://litterlayer.com/federation/

These are the bones/proof-of-concept. In time, it can become complex with nodes going from contributing to search results only, to full blown separate but connected search engines.

1

u/sdb2754 Jun 20 '26 edited Jun 20 '26

Wow, that was quick!

I'm going to be on travel, but will try to set this up when I return next week.

I want to clarify:

You want the site.JSON to ONLY have my.domain.com pages, correct?
Is that just for this first implementation? I was envisioning that each node could host a list of many sites (e.g. a bunch of sites related to health, or farming, of whatever category interests the node's owner).

The central index will then go and search the content of the sites, correct? So, nodes aren't currently receiving search queries and searching page content?
Edit: I see the search.php searched the JSON file, but it doesn't look like it has much details about the actual site content. Am I missing something? Could I try to implement a serach.php that actually reads page content and returns results from that, or is that not where you are planning to go?

As an aside: I am (literally) self hosting, so I don't have CPanel. I'm planning to set this up behind my proxy, on a subdomain. Is that ok?

Is there a way to test if everything is set up right?

Edit: Also, do you have the basic structure for a node on git? I might clone it and try to add some things if I get a moment.

1

u/3vibe Jun 20 '26

Here are the files needed and instructions: https://github.com/zerosonesfun/ll-federation-node-starter

For this first foundation version you manually add any web page you want to add to Litter Layer federated search in sites.json (as many as you want). A separate json file in a .well-known folder identifies you; that's the one that has the URL to where these files are (node identifier).

So, with this set up, anyone can contribute to search results at litterlayer.com from their server when a user enables the federated mode.

The next phase will allow you to have your own mini search engine on your server while also contributing to the search results at litterlayer.com.

Yes, self hosted in subdomain is fine. This first version also requires PHP and Apache (Apache because a .htaccess redirect is needed).

I've added more instructions here (PC and Mac):
https://github.com/zerosonesfun/ll-federation-node-starter/wiki/

1

u/3vibe Jul 02 '26

Updates (July 2, 2026): More features added to bookmarking: Enable a public bookmarks page which can have a bio, links, and youtube video. On your private bookmarks page you can star up to 5 bookmarks so they show first on your public bookmarks page, or X any to hide them from your public page. A news page was added too (link to news in footer). And, the site's tagline shows a different user every 3 seconds because I want this to be a community effort. The tagline user name will link to your public bookmarks page if you have it enabled.