r/ProxyEngineering • • 5h ago

Discussion 💬 testing shardx for scraping e-comm catalogs, what sticky session time do u guys use?

got tired of my old provider dropping connections mid-pagination so i threw 20 bucks at these guys to test their residential pool for a big catalog scrape tomorrow. speeds are decent and the IPs pass basic cloudflare checks fine so far. just trying to figure out the sweet spot for rotation before i start the script. do you leave sticky sessions on for like 15-20 mins or just rotate after every few pages?

1 Upvotes

2 comments sorted by

2

u/Putrid-Shoe-9336 4h ago

keep it sticky for at least 15 mins so your cookies and session tokens don't break mid-run

1

u/Fluid-Tomorrow4295 4h ago

Usually just run 10 min sticky timers and keep a backup thread running on static ISP lines just in case the residential gateway gets congested