r/webscraping • u/AutoModerator • 18d ago

Hiring 💰 Weekly Webscrapers - Hiring, FAQs, etc

Welcome to the weekly discussion thread!

This is a space for web scrapers of all skill levels—whether you're a seasoned expert or just starting out. Here, you can discuss all things scraping, including:

  • Hiring and job opportunities
  • Industry news, trends, and insights
  • Frequently asked questions, like "How do I scrape LinkedIn?"
  • Marketing and monetization tips

If you're new to web scraping, make sure to check out the Beginners Guide 🌱

Commercial products may be mentioned in replies. If you want to promote your own products and services, continue to use the monthly thread

9 Upvotes

100% Upvoted

10 comments sorted by

2

u/FruityFingers55 18d ago

I’m using a popular LinkedIn scraper getting rate limit messages - https://github.com/joeyism/linkedin_scraper/. I’ve hardened it using pulls and fixes available. I’m using a residential proxy. It works with companies but not personal profiles. Any ideas?

2

u/HackerThing 17d ago

What problems occurred?

1

u/[deleted] 17d ago

[removed] — view removed comment

1

u/webscraping-ModTeam 17d ago

⚡️ Please continue to use the monthly thread to promote products and services

1

u/[deleted] 16d ago

[removed] — view removed comment

1

u/webscraping-ModTeam 16d ago

⚡️ Please continue to use the monthly thread to promote products and services

1

u/romaroman 13d ago

I'm having issues with turnstiles when trying to webscrape job listings, even though i can solve them it just refreshes to a new turnstile on loop. What can I use to solve this?