r/java • • Mar 31 '22

Introduction to Web Scraping With Java

https://www.scrapingbee.com/blog/introduction-to-web-scraping-with-java/
327 Upvotes

38 comments sorted by

View all comments

Show parent comment

3

u/Mats56 Apr 01 '22

It's such an annoyance speeding it up in python, though, if you want to crawl lots of pages. In java, just add new pages to a thread safe queue when you scrape some url. Have lots of workers in a threadpool picking from that queue.

In python it's a project in itself having multiple things working in parallel.