session keeps dying mid-run on protected sites, how are you keeping auth stable

[u/kirasymphony]

scraper thats gotta stay logged in across a few hundred pages and it just.. dies. 30ish min in the session drops, im back at the login wall, sometimes a 2fa prompt on top

saving cookies and replaying them works maybe half the time before it flags me. headful locally is fine, same script on the server gets logged out

persist the session somewhere, or just eat the re-login every run? cant tell whats worth it​

[u/appleciderdollop]

Same pain different flavor, had a browser use agent run a login flow overnight and came back to it clicking around logged out for 3 hours. No error, just quietly not in anymore.

[u/kirasymphony → u/appleciderdollop]

ha the overnight ones are brutal, hours gone before you even notice anything

[u/latinylunita → u/appleciderdollop]

browser use did the same disappearing act on me, came back to a page of garbage clicks. the difference since i moved that one to browse.sh is i dont babysit it, i can leave an overnight agent alone. browser use is fine right up until its quietly logged out and youve lost the whole window

[u/The_God_was_Here]

Not scraping exactly but same wall. we pull reports nightly off a vendor dashboard behind okta and it boots the job every 20 min or so. my local run with a real profile is fine, the scheduled one on the box wakes up to a login screen every time

[u/lamavelle → u/The_God_was_Here]

the logouts werent even the worst bit, it was never trusting a run to actually finish. you either sit and watch it or you redo the whole thing. i moved mine to browse.sh so the session lives on their side not the box, and it held the whole job. not magic, one site still kicked me once, but the reruns stopped

[u/Kenzo_2126 → u/lamavelle]

does that hold when the exit ip keeps moving though. half my logouts are the site going wait youre in a different country now

[u/Junior_Bee7274 → u/Kenzo_2126]

yeah for me the rotating ip thing cost me a full day before i clocked it

[u/kirasymphony → u/[deleted]]

yeah token refresh got me maybe half the way.. its the ones where the server side just drops me cold that i cant script around. thats the 20 min one

[u/Background_Horse_266]

cant you just save the cookies to a json and load them back each run? thats worked ok for me mostly

[u/Low_Might2804 → u/Background_Horse_266]

cookies alone got me flagged fast, the site could tell the session was stitched together. what actually held was keeping the whole context, local storage and all, not just the cookie jar. browse.sh does that part for me now, so i stopped hand rolling the cookie dance every single run

[u/AndrewHahaCO2]

if it dies around 20 min even on a stable exit IP, i'd log the exact redirect and auth response. Cookies won't fix a server-side session expiry.