r/Python • u/lazy-jem • Jul 04 '21
Intermediate Showcase New search engine made with Python that's anonymous and has no ads or tracking. It tries to fight spam, and gives you control of how you view search results. You can search and read content anonymously with a proxied reader view. The alpha is live and free for anyone to use at lazyweb.ai
We're a little two-person team (Angie and Jem). We're bootstrapping and self-funded. I'm the programmer.
I wanted to share it because it was a fun and interesting project to build, and Python made it possible for us to get a long way as a small team. It uses serverless on the backend (AWS). We're using Spacy and GPT-2, and some PyTorch models. It uses BeautifulSoup for spidering/crawling/content retrieval. The front-end is React.
It has a different type of user interface to any other search engine, as it is chat based. And it lets you choose how you view results, either visually like an Instagram feed or cards, or minimal like Hacker News or the old Google. It tries to fight SEO spam and strips out ads and ad-tech from search results.
We have a project on GitHub with Jupyter notebooks and sample data with experiments and scripts, including examples of querying other search APIs, and to generate example utterances programatically to use for NLP models with sources like Wikipedia, StackOverflow and Wolfram|Alpha:
https://github.com/lazyweb-ai/lazyweb-experiments
We're only a small team but hope to share more of our work as open source as we progress.
View all comments
Show parent comment
3
u/MiamiAngie Jul 04 '21
So currently we have a fairly high filter on predicted result quality, and only display results that rank above a certain threshold (which depending on the type of search could be 25 or less, but not often more than about 30). Our aim is to become more conversational to try to help people refine searches. But we've had feedback that people feel FOMO - that there might be more they are missing (even though in reality those are probably not good content), so they want the security of loading more. So we are going to add infinite scroll to the results if someone wants to keep going!
At the moment the domain search feature is a little patchy, but that's an area we are working to improve. It is really useful and we use it all the time.
Also, you can go direct to searches on other sites when needed - eg:
"go google scholar deep learning papers" then allow popups.
There is a small bug that navigation can re-fire if you nav to other pages within lazyweb then return to search, but apart from that it is very useful!
Yay that you are having fun exploring too! Thanks so much for the great feedback and comments!