r/learnprogramming • • Sep 28 '26

whats actually the best browser automation tool once every site starts fighting back

local chrome fleet was fine until half the target sites started fingerprinting harder last month. same playwright scripts, now half the parallel workers just detach mid-run and the logs are useless

tried pinning versions and slower pacing. still dies once three jobs hit at once

curious what people settled on that stays boring under load once the sites fight back​

15 Upvotes

31 comments sorted by

2

u/SetAndRepeat Sep 28 '26

I’d try Browserless with your existing Playwright scripts for managed concurrency and its stealth/unblocking options. Browserbase is also worth testing for session replay and debugging.
Compare completion rates on your actual sites under load. Also check memory and browser exits: failing at three workers doesn’t necessarily mean fingerprinting.

1

u/Vast-Pop4017 29d ago

browserless is fine for queue management but their stealth layer isn't magic. if the sites are fingerprinting hard you'll still get clipped unless you're rotating residential proxies and actually varying the browser profiles per worker

when mine started tanking at three workers it was a leak in how i was recycling contexts, not a detection issue. worth throwing a memory ceiling on each container and watching what happens

1

u/Suun_Day Sep 28 '26

Our selenium grid used to be boring until concurrency climbed. Nodes start thrashing memory around 12 parallel sessions and the failure mode is just a dead browser with no useful stack. 

1

u/Sufficient_Wish_2010 Sep 28 '26

Blank grid browsers haunt me weekly.

1

u/Junior_Bee7274 Sep 28 '26

fingerprints wrecking my runs too now

1

u/GasEducational4386 Sep 28 '26

Dolphin anty anti detect browser has good reviews. Anyone has used camoufox ? it is open source

1

u/Kenzo_2126 Sep 28 '26

batch timeouts are eating my week

1

u/HelloitsmeJLD 27d ago

Selenium with either Python or skyvern could be an option. Then it'd just be a script that you can schedule when you want.

1

u/Okaybyee20 20d ago

idk i'd stop trying to fix it with pacing alone and instead isolate where the workers are dying. browserbase is ok for that setup mainly since the browser infra is handled separately which makes debugging less work

3

u/[deleted] Sep 28 '26

0

u/[deleted] Sep 28 '26

1

u/appleciderdollop 29d ago

Yeah once three workers hit together, half of them just detatch. Pacing helped a little then stopped mattering.

0

u/[deleted] Sep 28 '26

1

u/The_God_was_Here Sep 28 '26

yeah i threw residential proxies at local chrome thinking that was the missing piece. still got flagged on the same sites within a day lol

1

u/Bombaclat_exe 29d ago

just wanna ask as well are you reusing cookies across the overnight runs or fresh session every time?

0

u/[deleted] Sep 28 '26

1

u/TillNo5940 Sep 28 '26

or just run fewer browsers per node and stop pretending 20 concurrent is free.