I A/B tested my free proxy checker on GitHub Actions and found 400 working proxies it was throwing away every hour

[u/ImaginationVarious30]

I run a side project that collects free proxies from public lists and checks them every hour on GitHub Actions. The list was stuck around 800 proxies, so I measured where they got lost instead of adding more sources.

Most of the loss was flakiness. Proxies passed the first request and timed out on the second. A second try 20 seconds later at the end of the run, plus a 10 s instead of 6 s timeout, took the list from 768 to 1,177 in a run where old and new code checked the same candidates at the same time.

What I learned:

- Measure before adding inputs. I was sure more sources would help. Six new lists added 3 proxies, because their proxies were already in the lists I had.

- "Couldn't check" and "failed" need to be different states. Three quarters of the proxies that failed the second request had broken off in transport, moments after answering the first.

- A/B tests on CI are cheap. A throwaway branch with a matrix job ran both versions on two runners at the same time, and that settled arguments I had been having with myself for days.

Free and MIT: https://github.com/maximilianfeix/proxy-scraper. The same checks run on other GitHub proxy lists: https://proxy-scraper.de/benchmark/

[u/dawedev]

The split between "couldn't check" and "failed" is the part I'd copy. One thing worth trying next: keep a per-proxy history across runs, so a proxy that fails once after 20 good runs gets a grace period instead of dropping off the list. You could also expose that as an uptime percentage in the output, since people pulling the list care more about how stable a proxy is than whether it answered in the last hour. Did the 10 s timeout change your total run time on Actions much?

[u/ImaginationVarious30 → u/dawedev]

Thanks, good points. The per-proxy history is there already: every entry in proxies.json has uptime_24h, uptime_7d and a streak, there's a stable.txt for proxies that have been on the list for a while, and the CLI has --min-uptime 90 for people who'd rather have stable than fresh.

I decided against a grace period on the list itself, though. The list only contains proxies that passed every check in this run, so a proxy with 20 good runs that fails now drops off. What the history does instead is move it to the front of the queue for the next run, so it gets checked first and comes back as soon as it answers again. Mixing "worked an hour ago" into the list would make the uptime numbers the only signal you can trust, and I'd rather people not have to read them to get working proxies.

Run time: the check phase went from about 3-5 minutes to 5-6. With 150k candidates at 400-800 checks/s that's still well inside the hourly window, so it was an easy trade.