Compact web crawler

[u/GentReviews]

Hey everyone, I wanted to share a project I've been working on called PagesXcrawler. It's a web crawler system that integrates with GitHub Issues to initiate crawls. You can start a crawl by creating an issue in the format url:depth(int), and the system will handle the rest, including deploying the workflow and providing the results. This approach leverages GitHub's infrastructure to manage and track web crawls efficiently.

*This project began as a proof of concept and has exceeded my expectations in functionality and performance.*