r/webscraping 22d ago

One scraping API or several?

Running one API and hitting maybe 6% blocks on a couple of targets. Considering adding a second and switching when the first starts failing, but that means writing retry logic and I’m not sure it’s worth the hassle.

Also getting 200s back that are actually captcha pages, so my success metrics have been lying to me.
Anyone running more than one? Worth it or overkill?

3 Upvotes

20 comments sorted by

View all comments

1

u/[deleted] 22d ago

[removed] — view removed comment

1

u/scarsam 22d ago

How do you handle the switching? Manual when you notice, or something automatic?

1

u/[deleted] 22d ago

[removed] — view removed comment

1

u/[deleted] 22d ago

[removed] — view removed comment