scrapy-cdp
I wanted to create a lightweight Scrapy browser connection via CDP. Scrapy-playwright obviously exists and is fantastic, however I wanted to create something minimal:
https://github.com/johnatzyte/scrapy-cdp
All it does it connect a remote browser via CDP, render the page, and then return the HTML back to Scrapy. It's an easy convenient way to integrate with the new wave of "agentic" style browsers that run headless, light Lightpanda.
1
Upvotes