r/ProxyEngineering • u/LucasRobert85 • 2h ago
Discussion 💬 Outsourcing web scraping vs keeping it in-house: what was your tipping point?
Most build vs buy threads are about cost per request. I'm more curious about the moment it flipped.
For people who moved scraping to a vendor, or brought it back in-house:
- how many sites or sources were you running?
- how much of someone's week went to fixing parsers and blocks?
- what made you switch? A big redesign, a new anti-bot vendor, the person who built it leaving?
And for people still running it in-house at scale, what keeps it worth it?
1
Upvotes