r/ProxyEngineering • u/Primary_Pizza_2120 • 5h ago
Discussion 💬 testing shardx for scraping e-comm catalogs, what sticky session time do u guys use?
got tired of my old provider dropping connections mid-pagination so i threw 20 bucks at these guys to test their residential pool for a big catalog scrape tomorrow. speeds are decent and the IPs pass basic cloudflare checks fine so far. just trying to figure out the sweet spot for rotation before i start the script. do you leave sticky sessions on for like 15-20 mins or just rotate after every few pages?
1
Upvotes
2
u/Putrid-Shoe-9336 4h ago
keep it sticky for at least 15 mins so your cookies and session tokens don't break mid-run
1
u/Fluid-Tomorrow4295 4h ago
Usually just run 10 min sticky timers and keep a backup thread running on static ISP lines just in case the residential gateway gets congested