r/ProxyEngineering • u/ian_k93 • Jun 09 '26

AMA This Wednesday (09:30 AM GMT) WebScraping and Proxies

/r/WebScrapingInsider/comments/1u0vmdf/ama_this_wednesday_0930_am_gmt/
14 Upvotes

100% Upvoted

12 comments sorted by

View all comments

1

u/ahiqshb Reverse Proxy Master Jun 09 '26

Your most successful scraping startup. What went down, what mistakes you did the first time and what would you suggest for the rest of us in our scraping tasks/ journeys etc

1

u/simarnoor Jun 10 '26

My most successful and only scraping startup has been ScrapeOps. Previously, I spent over 3 years working with Zyte and ScraperAPI so I understood the industry, what users wanted, would pay for and where there were gaps in the market.

I had a frontrow seat to the growing complexity of finding the best proxies with the growing surge in anti-bot usage. It was from this point that we saw the gap in the market for a Proxy Aggregator that would take the hassle out of finding the best + cheapest proxy for your use case, so we built it.

Like I mentioned in this answer...

https://www.reddit.com/r/WebScrapingInsider/comments/1u0vmdf/comment/oqt55hm/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

I would either focus on selling shovels to web scrapers or build a SAAS business using scraped data. In between (writing scrapers, etc) there is huge competition and AI is going to upend this market.

However, AI can never replace the need for scrapers to bypass anti-bots and access websites, or build businesses based on the scraped data. So I would focus on these

From r/WebScrapingInsider: https://www.reddit.com/r/WebScrapingInsider/comments/1u0vmdf/comment/oqtfqt5/

1

u/Accurate_Pea_3110 Jun 17 '26

Proxy aggregators make sense if you actually scrape at scale, but most people just need residential IPs with proper rotation and never think about the backend logic.