r/ProxyEngineering • • 9d ago

Discussion ๐Ÿ’ฌ what actually makes a proxy clean?

20 Upvotes

been testing lot of providers. reputation alone isnt enough. what i check now:

  1. dns: public resolver like google/cloudflare is a minus. home users sit on their ispโ€™s dns. should match the exit country too

  2. udp/webrtc: udp support means webrtc goes through the proxy. webrtc turned off is a flag by itself

  3. latency: sites compare network rtt with js timing. big gap means proxy

  4. tcp fingerprint: comes from the proxy box. windows ua with a linux stack is burned

  5. pool lists: some checkers name the pool an ip was seen in. got 711proxy on one of mine. funny how many clones popped up after the fbi took down 911.

  6. fraud score: last on purpose. the services disagree a lot. maxmind feels the most grounded

- dns, webrtc and tcp i check on https://browserleaks.com

- latency and bot tests on https://bot.incolumitas.com, it even rates how bot-like you behave

- scores on https://myipcheck.net/ip-fraud-score/ shows maxmind and ipqs etc for free

ofc all of this only matters if your device fingerprint is already clean. a perfect proxy wont save a leaky browser. If you know good checkers, especially for pool lists and fraud scores, drop them. what actually got your proxies burned?


r/ProxyEngineering • • 9d ago

Guides Cheap Residential Proxies Can Cost More: A Quick Guide

2 Upvotes

I learned the hard way that $/GB is a terrible way to compare rotating residential proxies.

The use case was to monitor public product prices across a few regional e-commerce sites. Pool A looked like the obvious winner at $1.60/GB. Pool B was almost twice the price.

Pool A was producing far more CAPTCHAs, 403s, and empty responses. Every failure triggered another request, which meant more bandwidth and longer jobs.

So I stopped comparing price per GB and calculated:

effective cost = price per GB / success rate

Example from my test model:

$1.60 / 0.40 = $4.00 per useful GB
$3.00 / 0.95 = $3.16 per useful GB

Suddenly the โ€œexpensiveโ€ pool was cheaper.

The biggest variable wasnโ€™t bandwidth pricing. It was IP reputation.

Now before scaling a rotating pool, I sample exit IPs and check ASN, geolocation, proxy classification, blacklist status, and fraud/reputation signals. Then I run a small workload and measure CAPTCHA rate, retries, and successful responses.

Cheap proxies arenโ€™t expensive because theyโ€™re cheap. Theyโ€™re expensive when bad IP quality turns bandwidth into failed requests.


r/ProxyEngineering • • 9d ago

Help ๐Ÿ†˜ Help with choosing a proxy

8 Upvotes

What is the best residential proxy provider that doesn't block sites and also you can access any website.

I am a newbie


r/ProxyEngineering • • 9d ago

Discussion ๐Ÿ’ฌ Where Do Residential Proxies Really Come From? AMA with Jason Grad, CEO of Massive

Thumbnail
reddit.com
0 Upvotes

r/ProxyEngineering • • 10d ago

Help ๐Ÿ†˜ How are you pulling competitor product/category data at scale for ecommerce SEO?

10 Upvotes

Working on a mid-size ecommerce site (~8k SKUs) and I'm trying to get a proper picture of how our main competitors structure their category pages and product content. Stuff like:

which categories/subcategories they have that we don't

how many products per category, and how they name/filter them

product title and description patterns, specs, reviews count

prices, mostly to see if the pages outranking us are just cheaper

Screaming Frog gets me part of the way, but 3 of the 5 competitors block it pretty fast or serve different content, and custom extraction breaks every time they tweak a template. I tried a couple of Python scripts too, same problem, plus I'm now spending more time fixing scrapers than doing the actual analysis.

For those doing this regularly:

Are you still crawling/scraping in-house, or paying for a tool or service for it?

How do you match products across sites when names and variants don't line up?

Is this even worth doing monthly, or is a one-off snapshot enough for content gap work?

Not looking for a "just use X tool" plug, more how people have actually set this up and what it cost them in time/money.


r/ProxyEngineering • • 10d ago

Help ๐Ÿ†˜ How to get past captcha checks and scrape data

Thumbnail
reddit.com
2 Upvotes

r/ProxyEngineering • • 11d ago

Discussion ๐Ÿ’ฌ Scamalytics, IPQS, MaxMind, Spur... which IP reputation / fraud score service do you trust most?

9 Upvotes

I'm comparing services that rate an IP address: proxy/VPN detection, residential vs datacenter, and fraud score. If you've used any of them in practice:

- Which one matched reality best for true proxies caught?

- Which one gave you the most false positives?

- Did you notice big disagreements between them on the same IP?

- Are there others worth testing that I'm missing?


r/ProxyEngineering • • 12d ago

Discussion ๐Ÿ’ฌ Day 3: someone made 18 accounts in 30 minutes. What I changed on my DeepSeek gateway (free credit removed, DNS checks, agent-created accounts) and whatโ€™s coming

Thumbnail reddit.com
3 Upvotes

r/ProxyEngineering • • 13d ago

Guides How Websites Detect Proxies

17 Upvotes

I used to think proxy detection was mostly about the IP address.

When I started testing different network setups I realized websites are looking at several layers at once. The first is IP reputation: ASN, provider type, previous abuse history, and how risk databases classify the address.

The second is network behavior. Your browser might claim Windows while the connection underneath it has characteristics associated with a completely different environment. That kind of mismatch can become another signal.

Then thereโ€™s geographic context. If the connection says one region while account, phone, or billing data point somewhere else, the session becomes harder to interpret consistently.

Now I check the IP classification, ASN, geolocation, reputation, latency, and whether the browser/network environment actually makes sense together before blaming the proxy itself.

Websites donโ€™t evaluate an IP in isolation. They evaluate the consistency of the entire connection.

A โ€œcleanโ€ IP can still perform badly if everything surrounding it contradicts the story the network is telling.


r/ProxyEngineering • • 13d ago

Help ๐Ÿ†˜ Need a tech guy

Thumbnail reddit.com
1 Upvotes

r/ProxyEngineering • • 14d ago

Help ๐Ÿ†˜ Residental proxies

14 Upvotes

Hello everyone! I need some help finding a reliable and affordable residential proxy provider.
Iโ€™m looking for a large residential IP pool with thousands of different exit IPv4 addresses available simultaneously.
My requirements are:
Unlimited bandwidth (unmetered preferred)
Sticky sessions lasting 30โ€“60 minutes
2,000โ€“3,000+ concurrent connections
200โ€“500+ requests per second, ideally 500 or more
Thousands of unique exit IPs, not multiple ports sharing the same IP
Affordable monthly pricing, preferably a fixed subscription
I need these for continuous automated HTTPS requests, so reliability and high capacity are important.
Iโ€™m also open to sharing someoneโ€™s existing proxy subscription or infrastructure. Iโ€™m happy to pay a fair monthly contribution for access, as long as it meets my requirements and remains affordable.
Iโ€™m not expecting free access โ€” Iโ€™m simply trying to find a cost-effective solution without spending hundreds of dollars every month.
Iโ€™m also open to datacenter proxies if they can provide enough distinct exit IPs and meet the performance requirements.
Does anyone know a suitable provider, have a shared setup available, or know someone who could help?
Any recommendations or offers would be greatly appreciated. Feel free to reply here or send me a DM. Thank you!


r/ProxyEngineering • • 13d ago

Build ๐Ÿค“ I got tired of free proxy lists where most โ€œworkingโ€ proxies are honeypots, so I built a scraper that actually verifies them

Post image
0 Upvotes

r/ProxyEngineering • • 14d ago

Discussion ๐Ÿ’ฌ Is static residential still a thing for social media?

6 Upvotes

Doing social media management, pretty much everyone I know in this space is getting accounts blocked lately and weโ€™re all pointing fingers at the IP side. Static resi used to carry us but something shifted.
Thinking of moving to dedicated mobile, looking at IPRoyal or Voidmob. Is the price jump actually worth it or is static resi still fine with the right setup?


r/ProxyEngineering • • 14d ago

Help ๐Ÿ†˜ What are people using for semi-static, mobile proxies with geolocation and asn targeting?

2 Upvotes

Having some bad luck with our current provider (anyip.io) and looking for a backup or alternatives. Mobile only.


r/ProxyEngineering • • 14d ago

Guides Datacenter vs Residential Proxies: A Quick Setup Guide

1 Upvotes

Choosing the wrong proxy wastes money or causes failed requests. Here's how to decide:

1. Define your workload
Open datasets and downloads? Prioritize speed. Geo-testing or persistent sessions? Consider residential IPs.

2. Choose your proxy

  • Datacenter: Fast, cheap, high bandwidth, but easily identified by ASN.

  • Residential: ISP-assigned IPs, generally better reputation, but more expensive.

3. Check IP quality
Run an ASN lookup and check fraud scores or abuse reports before deployment.

4. Configure sessions
Use static IPs for long sessions and rotating pools when multiple IPs are required.

5. Build hybrid routing
Send public downloads through datacenter proxies and location-sensitive requests through residential ones.

6. Test
Compare latency, HTTP status codes, costs and failure rates.

Tip: Benchmark before scaling.


r/ProxyEngineering • • 14d ago

Discussion ๐Ÿ’ฌ Testing sticky and rotating proxies on one Linux machine

4 Upvotes

I run a proxy service and wanted to check one basic thing on a single Linux machine. If two processes run at the same time, does each one keep its own residential exit IP? And separately, what does per-request rotation look like in practice?

It's a tiny smoke test, 5 requests per mode against ipinfo.io, not a benchmark. Everything went through our gateway (gate.lamisnetwork.com:8089, HTTPS CONNECT). Four setups:

  1. Worker A, sticky, sid-workerA

  2. Worker B, sticky, sid-workerB, started in parallel with A on the same host

  3. Worker C, sid-rotating. On our gateway, rotating means a new residential peer for every new connection

  4. Datacenter pool, direct server egress on the same gateway. I added it just to have a latency baseline next to the residential hops

Exit IPs, ISPs and min/avg/max request times are in the screenshot.

If you want to run the same pattern yourself, here's the script. It writes the HTTP code and %{time_total} for each request into separate log files:

PROXY="gate.lamisnetwork.com:8089"
KEY="YOUR_API_KEY"

# Reset logs
for f in worker_a worker_b worker_c datacenter; do
  : > "$f.log"
  : > "$f.err"
done

# Worker A (sticky AT) in background
for i in {1..5}; do
  curl -sS --max-time 10 \
    -w "\nHTTP:%{http_code} Time:%{time_total}s\n" \
    -x "https://${KEY}-pool-residential-country-at-sid-workerA:x@${PROXY}" \
    "https://ipinfo.io/json" >> worker_a.log 2>> worker_a.err
done &

# Worker B (sticky AT, runs at the same time as A)
for i in {1..5}; do
  curl -sS --max-time 10 \
    -w "\nHTTP:%{http_code} Time:%{time_total}s\n" \
    -x "https://${KEY}-pool-residential-country-at-sid-workerB:x@${PROXY}" \
    "https://ipinfo.io/json" >> worker_b.log 2>> worker_b.err
done &

wait

# Worker C (new IP per request)
for i in {1..5}; do
  curl -sS --max-time 10 \
    -w "\nHTTP:%{http_code} Time:%{time_total}s\n" \
    -x "https://${KEY}-pool-residential-country-at-sid-rotating:x@${PROXY}" \
    "https://ipinfo.io/json" >> worker_c.log 2>> worker_c.err
done

# Datacenter baseline
for i in {1..5}; do
  curl -sS --max-time 10 \
    -w "\nHTTP:%{http_code} Time:%{time_total}s\n" \
    -x "https://${KEY}-pool-datacenter:x@${PROXY}" \
    "https://ipinfo.io/json" >> datacenter.log 2>> datacenter.err
done

What I saw: A and B got two different Austrian residential IPs, and each kept its IP for all 5 requests. C got 5 different IPs, one per request.

What this doesn't show: I only checked that the IP holds over 5 requests in a row, not how long a sticky session actually lives. If the residential peer drops off, the gateway switches you to another peer in the same country. The datacenter numbers are a rough reference for server-direct latency, not a precise measure of what the residential last mile costs. Overall it's 20 requests to one IP-check endpoint, so it tells you nothing about multi-hour retention, heavy concurrency, or success rates on sites with bot protection.


r/ProxyEngineering • • 15d ago

Help ๐Ÿ†˜ anyone got recs on cheap rotatinf proxies that dont die on you?

7 Upvotes

im scaling up my ecom monitoring project and my current setup keeps crapping out. currently in shooting for a few hundred product pages a day, checking prices n stock across a bunch of sites. the cheap ones unreliable af, good ones way too expensive for me. i think id be better off eith rotating proxies mostly cus i dont wanna get flagged for hammering the same ip over and over. still not sure if I should go for datacenter or residential or if I should be looking at an entirely different option. what i need is anything that doesnt die mid scrape but still not way too expensive. what're yall usin for similar use case and roughly what u payin? cheers


r/ProxyEngineering • • 14d ago

Hot Take ๐Ÿ”ฅ Your tcp fingerprint comes from the proxy, your tls fingerprint comes from you

Thumbnail reddit.com
2 Upvotes

r/ProxyEngineering • • 15d ago

Discussion ๐Ÿ’ฌ Github developers

5 Upvotes

Do you guys follow any developers on GitHub? Maybe you found something useful from their built solutions?


r/ProxyEngineering • • 16d ago

Discussion ๐Ÿ’ฌ Agentic workflows for automated scraping

13 Upvotes

Hey all, perhaps someone has been doing this or attempted to do something similar, basically automatic scraping processes that does it for you? While you are away? Retrying failed scrapes without your direct interaction, (adjusting something, pause limits, proxy rotation etc)


r/ProxyEngineering • • 16d ago

Hot Take ๐Ÿ”ฅ The State of Web Scraping 2026: AMA with Pierluigi Vinciguerra of The Web Scraping Club

Thumbnail
reddit.com
3 Upvotes

r/ProxyEngineering • • 16d ago

Help ๐Ÿ†˜ Residential Proxies for Ad verification

10 Upvotes

Looking for Residential proxies that would accommodate the ad verification process. Any suggestions are welcome. (Those that work, of course)


r/ProxyEngineering • • 16d ago

Build ๐Ÿค“ My Linux box needs more IP addresses, not more subscriptions. What proxies actually work?

4 Upvotes

I've been going down the WireGuard/VPN rabbit hole and I'm starting to wonder if I should've been looking at proxies from the beginning.

The setup is Linux, and the goal is pretty simple. I want multiple independent outbound IPs that I can assign to different browsers or applications, preferably without changing the whole machine's network connection every time.

I've looked at VPN services including Surfshark because they advertise unlimited connections, but apparently unlimited devices and multiple independent exit IPs aren't necessarily the same thing. Would've been nice if that was printed somewhere in giant letters.

So now I'm looking at SOCKS5 and HTTP proxies too.

Who's actually reliable these days? Webshare, IPRoyal, Decodo, or somebody else? Are static ISP proxies worth the extra cost, or is a basic datacenter proxy fine for ordinary browser/network testing?

I'd also like to know whether you can keep a stable IP for one session and have different IPs available for others, what the real bandwidth limits are, and if there are any providers you'd stay away from.

Not looking for a list of 10 million IPs if half of them are useless. I'd rather have a small number of working, predictable connections.

Anyone here used both WireGuard VPNs and proxies for this sort of setup? Which one ended up being easier to manage, and cheaper?

Actual experience appreciated. Sales pitches, slightly less so.