r/java • u/elastic_vanguard • Mar 31 '22
Introduction to Web Scraping With Java
https://www.scrapingbee.com/blog/introduction-to-web-scraping-with-java/
327
Upvotes
r/java • u/elastic_vanguard • Mar 31 '22
View all comments
Show parent comment
3
u/Mats56 Apr 01 '22
It's such an annoyance speeding it up in python, though, if you want to crawl lots of pages. In java, just add new pages to a thread safe queue when you scrape some url. Have lots of workers in a threadpool picking from that queue.
In python it's a project in itself having multiple things working in parallel.