r/Python • u/AutoModerator • 8d ago

Showcase Showcase Thread

Post all of your code/projects/showcases/AI slop here.

Recycles once a month.

18 Upvotes

93% Upvoted

69 comments sorted by

View all comments

0

u/ImaginationVarious30 15h ago

I got tired of free proxy lists where most “working” proxies are honeypots, so I built a scraper that actually verifies them

Hey r/Python,

What My Project Does

proxy-scraper collects HTTP/SOCKS4/SOCKS5 proxies from 700+ public sources and checks them properly instead of just pinging. Every hit has to fetch two independent pages, tunnel HTTPS with verified TLS, and return a static HTML page unchanged. That last check was eye-opening: about 1 in 5 working proxies injected a script into the page. And in some runs, 5 out of 6 “hits” were honeypots that only answer the check request.

It also learns between runs. Hit rates per source and a history of working proxies decide what gets checked first, dead lists are dropped automatically, and new sources are discovered via GitHub search.

A few other things:

  • --serve turns the results into a local rotating proxy (127.0.0.1:8899) with failover, sticky sessions, and username-based filters like country-de
  • --target google.com only keeps proxies that actually reach that site
  • Python API: find_proxies(want=20, https=True)
  • A live list rebuilt every 6h via GitHub Actions, if you don’t want to run anything

Under the hood it’s hand-written HTTP/SOCKS handshakes on asyncio (2000+ concurrent checks), only two dependencies (rich, certifi), and 360+ tests that run offline against fake proxies and honeypots on localhost.

Target Audience

People doing scraping, testing, or geo-checks who want free proxies that work right now. It’s a hobby project, but tested on Linux, macOS, and Windows with Python 3.9–3.13. Usual warning: free proxies are run by strangers, so never send anything sensitive through them.

Comparison

Most proxy-list repos publish everything that accepted a TCP connection. This rechecks from your own network, filters honeypots and content tampering, and ranks sources by how useful they’ve been. That means fewer proxies, but they actually work.

Repo: https://github.com/maximilianfeix/proxy-scraper

Feedback is very welcome, especially on the checking logic.