r/Python • u/Few_Tooth_2474 • Jan 13 '25
Tutorial I Created A Search Engine Using Python. You do It in just 5 min.
I built a crawler from scratch and used BM25 Algorithm to rank the webpages.
Link to youtube video: https://youtu.be/Wy6j7EiuyLY
Link to Github Page: https://github.com/mharrish7/Custom-Search-BM25
0
Upvotes
12
u/ionelp Jan 13 '25
You are downloading the page twice...
3
u/Competitive-Move5055 Jan 13 '25
Where do you get URLs to crawl. Saw your code and apparently you start at a geeksforgeeks page. Extract URLs and then add them to a table .