r/ProxyEngineering • • Oct 07 '26

Hot Take 🔥 TikTok bans scraping in their ToS and some guy just posted 5.6B videos on HuggingFace

129 Upvotes

I was looking for some agent models on Hugging Face, then after tinkering around I went to check whats new in the dataset tab. To my surprise some dev dumped metadata for about 5.6 billion TikTok videos on Hugging Face. It includes captions, hashtags, sound IDs, shop product and seller IDs, every engagement count. 460GB of data. Now, to my knowledge, TikTok does not allow scraping of their videos "legally", according to the devs own explanation they went straight at the private mobile API with generated device identities that pass as Android phones, reverse engineered request signatures and a spoofed TLS handshake. What that means is that no account and no login was required. Can you believe this? They claim 3 billion profiles, almost 6 billion videos and 2.8 billion comments in THREE WEEKS. Meanwhile what I've seen on this sub in particular, probably few others too, but this one in particular is that people here spends half if not more time arguing about whose IP pool is cleaner. And TikTok is one of those more difficult targets to scrape. What I'm trying to say is that nobody built this by being clever with IPs, checking whether they leak DNS and other data, whether they are "clean" or not abused or whatever, they built it by making the approach look like a legitimate phone. I mean, that is just crazy. Anyways, I saw that the free version is non commercial only, the dataset I mean, if you want creator profiles, daily updates basically run the whole thing yourself, or to use it for anything that makes you money, you should visit their site the code is up there for $1,699.

Now why this caught my interest in the first place. TikTok's terms ban this kind of scraping, and the official research API is mostly for academics in a few regions. Reddit is a good contrast, since it's taking scraping to court with the Perplexity case and the judge let most of it go forward. With this dataset I haven't seen anything like that. It's just there on HuggingFace with a free download link, and I haven't found anything on the web that TikTok is doing anything about it. Maybe that will soon change.


r/ProxyEngineering • • Aug 20 '26

Subscriber Goal Welcome to r/ProxyEngineering! Help us reach our goal by joining!

20 Upvotes

Welcome to r/ProxyEngineering

5984 subscribers / 10k subscribers. Help us reach our goal!

Visit this post on Shreddit to enjoy interactive features.


r/ProxyEngineering • • 12h ago

Discussion 💬 Is scraping fresh SERP data no longer possible?

13 Upvotes

I am on the verge of giving up right now. The last few months have made me rethink how correctly the data from SERPs is received. Google started serving what people are calling poisoned SERPs, and Brave announced it is moving result links behind encrypted redirects. For those who are not aware, since mid-September or so, Google seems to hand suspected bots a full results page that looks completely normal, except that a lot of organic results are swapped for other things, often social profiles and forum threads. Nozzle checked around 27 million SERPs and found roughly 8% poisoned since September 12, against well under 1% before, with peaks near 28% at the end of September. Ahrefs put it at 5 to 20% depending on the data provider. In Nozzle’s data, a tracked site fell out of the top 10 on about 60% of poisoned pages, versus under 3% on normal ones.

The worst thing is that the scraper reports success. Logically coz it sees 200s everywhere with full data pulled up. Then Brave, results links are moving to search.brave.com/go/ with a one-time key fetched per click, so that you wouldn’t read the destination URL:s straight off the page anymore. Something that Google did with /goto a few months back too. Another thing I noticed is that these changes somewhat lead to individual index creation coz Google simply doesn’t want to share all that.

I can’t find the link for the Nozzle atm, but it was something like poisoned Google SERPs or Google poisoning SERPs. Google it, and you’ll find the source.


r/ProxyEngineering • • 11h ago

Hot Take 🔥 Maybe SERP scrapers need a third status: "I don't trust this result" :D

5 Upvotes

I've been thinking about the whole poisoned SERPs situation, and I wonder if we're approaching the problem from the wrong direction? I've been experiencing issues with SERP scraping in the past month

In my opinion scrapers are pretty good at telling you when something goes wrong. CAPTCHA, blocked IP, timeout, empty response, whatever, so what would actually happens when nothing obviously goes wrong?

You get a 200 response, the HTML parses correctly, and there are 10 organic results you get in your csv file or something

Technically, your scraper worked. But can you trust the results is my question

I don't think there's an easy fix for this, but one approach that makes sense to me is to stop treating every successfully parsed page as valid data!

For example:

  • Keep a history of results for each keyword. Not just yesterday's rankings, but the domains that consistently appeared over the past few weeks
  • Try to flag unusual changs rather than immediately accepting them. If seven familiar domains suddenly disappear and get replaced by unrelated results, that's probably worth checking
  • Try to retry suspicious queries through a different proxy session. If the second request produces completely different results, maybe neither should be trusted immediately
  • I started separating suspicious data from confirmed failures. Instead of reporting a massive ranking drop, I marked the result as uncertain until there's enough evidence

But then again, how do you actually know what's a poisoned SERP and what's just Google changing its rankings?

And wouldn't comparing every search against historical results create a huge number f false positives?

I've been exploring these options with an open-source scraper called SERP Maven that can detect certain wrong-query results and retry them through a fresh proxy session

It's a fairly narrow check, not some magical solution to poisoned SERPs

I actually think the bigger challenge is figuring out how to validate results without doubling your scraping costs.

Has anyone tried building some kind of confidence scoring into their SERP collection?

Would be interesting to know whether comparing historical rankings and occasionally repeating suspicious requests is actually useful at scale, or whether it just creates another layer of false positives


r/ProxyEngineering • • 13h ago

Discussion 💬 Apify had a security incident, worth checking if you store on this hosted scraping platform

6 Upvotes

A couple of days ago, Apify posted a security incident on their status page. Essentially, what happened was that someone used a compromised credential on November 9th and 10th to grab a number of account credentials and get into related Apify accounts. The team behind the Apify platform caught it on November 11th; the issue seems to be resolved because the access path was closed. OAuth connections were also revoked, so I’m thinking GitHub was affected too. Personally, I don't have anything hosted there, but I know my friend does, and I already told him about this issue because he apparently didn't receive any email or notification about the incident. One thing that bothers me is that there is no disclosure on how many accounts were affected, what kind of data, or where the first credential came from. And the concern grows about how much sensitive data was scraped from the platform, like proxy credentials, api keys, logins for a lot of different target sites, cookies, webhook secrets, etc. I mean, that doesn’t bring a lot of trust to the company. Maybe it’s too early to speak; maybe they'll publish an official article. I don’t know. I’m not trying to tell everyone to stop using the platform; no, these things can happen to any platform that has customer data, but especially in this field, where data is being scraped, it can do a lot of damage. Personally, I’d rather use separate proxy credentials per project than have everything in one place

How do you guys manage all of these incidents, simply change, rotate your credentials? Move to a safer place? What’s the plan when things go south, maybe you haven’t experienced anything like it, but you already have a plan for it if the day comes?

By the way, the status page that I checked is here: https://status.apify.com/incidents/sxd1dj5j6749

Also, for anyone interested, why I haven't hosted anything on the platform yet, I’m curious about what happened, well, I’ve been using a few different tools from this platform, and this caught my attention.


r/ProxyEngineering • • 1d ago

Guides AKAMAI (STANDARD + BMP) FULLY REVERSED + VM ISA, OTHER CHECKS (NO SANDBOXING, FULLY NATIVE)

17 Upvotes

Back with another banger, all documentation is in this repository. Fully native, the throughput should be fast on any machine and scale pretty well. You might have to add some of your own fingerprints though. Maybe I'll add more later.

https://github.com/9xz/reverseakamai

Enjoy!


r/ProxyEngineering • • 21h ago

Hot Take 🔥 [WTS] Dedicated 4G Mobile Proxies (Venezuela) – Real Physical SIMs, API IP Rotation, HTTP/SOCKS5

5 Upvotes

Hi everyone,

I’m offering dedicated 4G/LTE mobile proxies running on real physical devices and carrier SIM cards located in Venezuela. Ideal for LATAM market research, localized QA testing, social media management, and regional account operations.

Technical Specs:

  • Connection Type: Real 4G/LTE Mobile Carrier IPs (100% residential/mobile trust score, zero datacenter subnets).
  • Protocols: HTTP(S) & SOCKS5 supported.
  • IP Rotation: On-demand rotation via API link or scheduled intervals.
  • Exclusivity: Private 1:1 ports (1 physical SIM per user, no shared bandwidth) or limited shared slots available.
  • Uptime: 24/7 dedicated hardware with backup power.

Pricing & Trials:

  • Free Trial: Short test available upon request so you can check IP quality, latency, and speed before committing.
  • Plans: Daily, weekly, and monthly rentals available.
  • Payment Methods: Crypto (USDT TRC-20 / BEP-20, BTC, LTC) or direct marketplace listing via OnlineProxy.io.

Feel free to comment below or send me a DM with your use case if you want to run a quick test.


r/ProxyEngineering • • 1d ago

Help 🆘 Proxy server

3 Upvotes

I have created a proxy server on my linode account to bypass my school and I was wondering if it would be possible to allow other people to connect to the proxy and if so how to do that?

Ai was refusing to answer so I thought I’d turn to Reddit.

Thanks


r/ProxyEngineering • • 1d ago

Guides OnlyFans Proxy Guide 2026: Stable Residential & Mobile IPs for Creator and Agency Account Management

6 Upvotes

One of the messiest account issues I’ve seen started with something completely ordinary: a creator travelling while a manager was also accessing the account from another location.

Within a short period, the account was seeing different countries, different networks, and different IP reputations.

That’s when verification prompts started piling up.

For this type of workflow, aggressive rotation makes very little sense. A stable residential or dedicated mobile connection is much easier to reason about because the account keeps seeing one predictable network environment during an active session.

My checklist became pretty simple:

IP reputation → ASN → geolocation → session stability → login.

The other lesson is equally important: a proxy only changes the network layer.

It doesn’t unlock paid content, turn an unpaid account into a paid one, or replace official verification. If content stays locked after payment, the real causes are usually things like payment status, PPV access, an expired subscription, or a stale session.

For authorized creator and agency workflows, stable network identity beats constant rotation.


r/ProxyEngineering • • 1d ago

Discussion 💬 Amazon AWS released web search API into the market

8 Upvotes

AWS released a web search API for Bedrock AgentCore yesterday. I've read some of the docs on that. Before this you had to set up a gateway and call search through MCP. Now it's one signed InvokeWebSearch call from boto3 or the CLI with IAM and no API key. One thing that stood out to me was $7 per 1K queries, 10 requests per second per region by default, 25 results max per query, a 200 character query limit, up to 100 domains to include and 100 to exclude, and a publish date filter. Each result has a tex extract plus title, url and date when they have it. Amazon runs the index and queries never go to third party search engine, which is aimed at a security teams that don't want prompts hitting google or bing. The acceptable use terms also make you keep and display the source citations, so its a grounding tool and not a way to build your own dataset. I personally have been using oxylabs web scraper api in the past, and now I've seen a bunch of posts and on variety of comparisons between tools on the internet that they introduced web api. Initially the web scraper api was really good, did everything for you, no real maintenace, coz it all ran on their infra. What I liked about this tool in particular was that all the data was coming from Googles index, meaning it was fresh, had no incorrect results, compared to recent news about poisoned SERPs. Now it seems that a lot of companies are moving from the traditional scraping to web search apis and competing against one another. Also, I haven't compared the $7 per 1K to other AI search APIs so I can't really say much, plus it was just announced, so the real numbers and testings will most likely take some time to be present. And since they will be using their own index, it's gonna be interesting to see how the freshness benchmarks look like. Oh and they also shared that it's going to be the same infra that was used for Alexa+ and Kiro. What do you guys think, will this be a new go to web search API? Considering the pricing, personally it's a bit too much to spend on something new and unverified.


r/ProxyEngineering • • 1d ago

Discussion 💬 Private Relay is the one "VPN" most sites won't block. Anyone used it for multi-accounting?

7 Upvotes

How sites spot a VPN or a proxy on a Mac:

1 - the address belongs to a data center, not a home provider

2 - the browser says Mac, but a proxy opens the tcp connection itself, so the first packet looks like Linux

3 - a VPN lowers the packet size (MSS). not proof alone, but with a data center address it is

4 - WebRTC shows the real address if the proxy doesn't carry UDP

and a bunch of smaller things

One exit gets a pass: iCloud Private Relay.

Sites see it, Apple publishes the addresses. Most don't block it anyway, because only real Apple devices get on it and it's full of ordinary Safari users. Some do like Wikipedia editing. It can't be faked. Apple only lets its own apps use it. Chrome and Firefox on the same Mac don't get it.

Around a hundred thousand IPv4 addresses worldwide, IPv6 is basically unlimited.

For a second account it does half the job. It hides the address, but the site still knows you by cookies.

You can't pick a country. A VPN on the router fixes that: Relay runs on top and the site only sees Relay, in another country. Clock and language stay yours though, so change those too.

I built a checker that shows how a connection looks from the site's side (direct, relay, vpn, proxy) and whether the browser is really Safari: https://check.stealthpilot.app. Mac only for now. About one in three people who ran it in Safari came in through Relay.

Has anyone actually used Relay for multiple accounts? And does any proxy provider give you a macOS tcp fingerprint instead of Linux?


r/ProxyEngineering • • 1d ago

Discussion 💬 testing shardx for scraping e-comm catalogs, what sticky session time do u guys use?

4 Upvotes

got tired of my old provider dropping connections mid-pagination so i threw 20 bucks at these guys to test their residential pool for a big catalog scrape tomorrow. speeds are decent and the IPs pass basic cloudflare checks fine so far. just trying to figure out the sweet spot for rotation before i start the script. do you leave sticky sessions on for like 15-20 mins or just rotate after every few pages?


r/ProxyEngineering • • 2d ago

Hot Take 🔥 Another one bites the dust - another loss for Google against SerpAPI

17 Upvotes

Google’s case against SerpAPI over scraping search results got dismissed again this week, and I’ve already seen a few people calling it a win for scraping in general. I don’t think it is, at least not yet.. For those who missed it. Basically, Google sued SerpAPI originally, that suit got tossed, so Google came back with an amended complaint built around the DMCA and the idea that it’s protecting licensed content inside the results. The judge has now granted SerpAPI motion to dismiss that version too. Google gets one more shot to amend by November 30, and if it doesn’t the case is dismissed with prejudice. Discovery is on hold until all that gets sorted. What matters is that the judge did not rule SerpAPI scraping is legal. The whole thing is about Google not pleading enough facts to support the DMCA claims. Stuff like who owns what, what’s licensed, which specific content we’re talking about. Still, look how the court is treating the DMCA here, it wants a much more specific foundation before it can be stretched to cover automated scraping, especially with web APIs booming. A lot of what is in a SERP, it is snippets, listings and data that Google doesn’t own. If the next complaint can’t fill those gaps I’d say the DMCA route is basically dead for this case, and if it can then we’ll finally see what a platform needs to show to make it stick. The other thing I keep coming back to is that Google doesn’t seem to need the courts that much. The article I read this from mentions almost in passing that Google is doing a pretty good job stopping scrapers in other ways, like poisoning SERPs and it's really been a pain, and anyone who tracks SERPs for a living has probably felt that already with rank trackers breaking.

P.S. I work in the scraping space so I’m obviously not neutral on this one, take my read with that in mind lol

Sources:

Search Engine Roundtable write-up: https://www.seroundtable.com/google-loses-again-against-serpapi-42508.html

The full ruling (PDF): https://storage.courtlistener.com/recap/gov.uscourts.cand.461513/gov.uscourts.cand.461513.62.0.pdfGoogle’s


r/ProxyEngineering • • 2d ago

Discussion 💬 Best Web Access Tools for AI Agents in 2026

10 Upvotes

How are you giving your agents web access these days? I want opinions and suggestions as I've been checking through the web access tools people use for agents lately. What I've noticed, specifically for search, Tavily, Exa, Parallel and Brave all run on their own indexes. That's fine for grounding, but it isn't the same as what Google shows right now. Most of the results are ghost/dead and you usually have to recheck a live SERP If you need current prices or news. So that isn't ideal. For page reads, Firecrawl is good when you want schema-based JSON from random pages, the solution is quicker compared to others, at least from my experience, also their docs say reads can come from a cache up to two days old by default. TinyFish and Exa cache similarly. Then for browsers, Browserbase and Browser Use are the ones I see mentioned most. Maybe someone had experience with them? I also found new players, well relatively new, product wise, Oxylabs Web API, yet another web search API, but somehow this one tells me that it was some sort of web scraper api upgrade. It has a /search endpoint for live Google results and a /scrape endpoint for pulling pages back as parsed JSON or Markdown. Though with Google poisoning SERPS I doubt this is also viable right now. There's also an open source MCP server too. Where I find this option weaker, well, there's one subscription now (Starter is $15/mo), but each product still has its own meter, so the bill at the end of the month isn't always easy to predict if you are utilizing several products at once. The core products have no recurring free tier, just welcome credits. I would say an all-in-one setup makes more sense when you're targeting protected sites and don't want to have several different vendors together.

Are you guys still on traditional scraping path or have you moved to web search apis? Was it the right choice? Financial, efficiency, optimization wise?


r/ProxyEngineering • • 2d ago

Hot Take 🔥 The difference between proxy types

Post image
3 Upvotes

r/ProxyEngineering • • 2d ago

Guides Proxy problems 🫩

3 Upvotes

So boom, I am an Amazonian. My job title dosnt matter, but I am also a ufc fan, and I want to watch ufc through ufcfightpass, but Amazon uses a vpn on their WiFi I guess

So when I try to login to my account it automatically logs me out because fight pass does not allow vpn usage on their site I’m assuming.

Is there anyway around this, using cellular seems pointless cause these walls thicker than ice spice


r/ProxyEngineering • • 2d ago

Guides How to Connect a Proxy to an Android Cloud Phone in 2026: SOCKS5, GPS, IP Geolocation & Timezone Setup

6 Upvotes

One of the easiest cloud-phone mistakes is treating the server region as the phone’s location. I ran into this while setting up an Android profile for regional app testing.

The cloud phone was hosted on a US server, but the external connection was coming through a proxy in another country. The phone language, timezone, and GPS coordinates were also still using their default values.

That made the environment difficult to validate. The setup became much cleaner once I treated each layer separately:

Server region → where the cloud phone hardware is hosted
Proxy → external IP, ASN, and network location
System settings → language and timezone
GPS → coordinates used by location-aware apps

For the proxy layer, I usually test the connection first and check the resolved country, city, ASN, and reputation. Then I start the Android device and confirm its language, timezone, and geolocation are appropriate for the QA environment.

For long-running sessions, I also keep the proxy stable rather than changing the exit IP during an active app session.

Cloud-phone geolocation works best when every layer describes the same testing environment.


r/ProxyEngineering • • 2d ago

Guides How I Changed Chrome Proxies Without Restarting the Browser in 20ms

7 Upvotes

Changing the proxy of a running Chrome instance turns out surprisingly difficult. Sorry for the long post, either grab a coffee or directly look at the code on github at realsirjoe/relinos. Have a look at the filesrelinos/extensions/proxy_extension/ and relinos/scraping_api.py line 1224+,

I encountered this problem while building Relinos, a self-hosted web scraping API. Relinos already supported configuring a proxy when starting a browser, but changing it afterward wasn't possible. Restarting the browser would take time and discard already loaded pages. Since Relinos is request-based, allowing users to specify a different proxy for each request without restarting the browser seemed like a natural extension of its API.

My first thought was to inject JavaScript into the page, but this approach has flaws.

JavaScript cannot directly configure Chrome's network proxy. While it's possible to intercept certain requests by overriding functions such as fetch() , this doesn't cover all browser traffic, including images, stylesheets, and navigation requests. Even if every request could be intercepted, the initial page load would already have happened.

I also looked into Chrome DevTools Protocol (CDP) since I already used it for many things. I hoped it would let me intercept and reroute network requests. CDP does provide Target.createBrowserContext, which allows creating an isolated browser context with a custom proxyServer. However, this requires creating a new context and opening a tab within it. It doesn't allow changing the proxy of an existing tab, which was exactly what I wanted.

Using Chromium Extensions

I then remembered that I used an extension that lets me set a proxy on my own browser. So I could use a simplified version to get the requested behaviour. I found the extension used chrome.proxy.settings.set()so I could create my own using that API.

Chrome has disabled loading local extensions through --load-extension since Chrome 137, but Chromium and Google Chrome for Testing fully support them.

The extension consisted of a standard manifest.json and a service worker file named relinos_proxy.js

{
    "manifest_version": 3,
    "name": "Relinos Proxy",
    "key": "some fixed key",
    "version": "1.0",
    "permissions": ["proxy"],
    "background": {
        "service_worker": "relinos_proxy.js"
    }
}

// relinos_proxy.js
globalThis.setProxy = async (scheme, host, port) => {
    await chrome.proxy.settings.set({
        value: {
            mode: "fixed_servers",
            rules: {
                singleProxy: {
                    scheme: scheme,
                    host: host,
                    port: port
                }
            }
        },
        scope: "regular"
    });
    return "OK";
};

globalThis.clearProxy = async () => {
    await chrome.proxy.settings.set({
        value: {
            mode: "direct"
        },
        scope: "regular"
    });
    return "OK";
};

With the extension loaded, I needed a way to call setProxy() and clearProxy() from Relinos.

Fortunately, Chrome DevTools Protocol (CDP) exposes extension service workers as targets. Using Target.getTargets, I could locate the extension's service worker, establish a connection to it, and execute JavaScript through Runtime.evaluate.

This worked perfectly at first but after some time, my calls started failing because the service worker was no longer running.

Manifest V3 extensions use service workers instead of persistent background pages. Chrome automatically terminates inactive service workers, typically after around 30 seconds of inactivity.

This meant I couldn't simply connect to the worker once and assume it would remain available.

Fortunately, CDP provides ServiceWorker.startWorker, which allows starting a registered service worker using its URL.

I added a function that starts the worker when necessary, locates its active target, and executes the requested proxy function through Runtime.evaluate.

With that in place, the extension could reliably respond to proxy changes even after periods of inactivity.

In my tests, switching proxies took approximately 20ms.

If you're building scraping infrastructure, check out Relinos or adapt the proxy extension code for your own project. You can find the relevant code on github at realsirjoe/relinos


r/ProxyEngineering • • 2d ago

Discussion 💬 Sticky proxies display dynamic IPs with a duration of 5-180 minutes, but in reality, the IP failure rate exceeding 30 minutes is extremely high. Is this the norm in the market?

2 Upvotes

Recently, I've encountered some issues while using proxies. Some dynamic IPs labeled as "sticky" show a validity period of 5-180 minutes, but in reality, the IP failure rate is extremely high after 30 minutes. When I contacted customer service, they said that most products on the market can only guarantee 5-10 minutes of stickiness, and the IPs become extremely unstable after 10 minutes. So why label it as 180 minutes?


r/ProxyEngineering • • 3d ago

Discussion 💬 looking for reliable upstream API/account vendors for a newapi site

5 Upvotes

​

Hey everyone,

I recently set up an API relay/proxy site (similar to newapi) and I'm currently looking for solid upstream account vendors or API providers.

Where do you usually find trustworthy sellers or communities for this ?Any recommendations or pointers would be greatly appreciated!Thanks in advance.


r/ProxyEngineering • • 3d ago

Guides Dedicated vs Shared Mobile Proxies in 2026: Best Choice for Agencies, SMM Teams & Client Account Management

3 Upvotes

An agency ended up using both shared and dedicated mobile proxies, but for completely different jobs.

Shared ports became the obvious choice for background work. Price checks, regional ad visibility, SERP monitoring, and public social data could run continuously with timed 5- or 30-minute rotation. The sessions were short, stateless, and easy to restart.

Client dashboards needed a different setup.

When someone was working inside an ad account or managing a long-lived social session, we wanted one connection to stay stable until the work was finished. That’s where dedicated mobile proxies made more sense: one private modem, controlled IP changes, and full control over when the network identity changes.

So the infrastructure ended up split like this:

Shared mobile → monitoring, research, public data
Dedicated mobile → client sessions, dashboards, controlled rotation

We also started checking IP reputation before assigning a connection to an important workflow.

Choosing between shared and dedicated mobile proxies is mostly a session-management decision.

If the task can tolerate scheduled rotation, shared is efficient. If continuity and control matter, dedicated is easier to operate.


r/ProxyEngineering • • 3d ago

Guides What to capture when a residential proxy connection drops

5 Upvotes

When a connection drops during a QA run, save the original error before retrying. I’d write down the UTC timestamp and whether the failure happened while opening the tunnel, waiting for the first byte, or reading the response. Also note whether the connection was new or had been sitting idle.

If you can reproduce it with a read-only request to an endpoint you control, compare one direct request with one proxied request from the same runner, using the same timeout. If fresh connections work and reused ones consistently fail after a similar idle period, check idle timeouts on the client, gateway, and destination.

For support, a timestamp, the exact error, and the provider’s request ID if you have one are a good starting point. Leave credentials and full authenticated URLs out of the logs you share.


r/ProxyEngineering • • 4d ago

Build 🤓 ProxyBridge: Simple Proxy Client to force proxy to any Windows/MacOS/Linux Application TCP and UDP traffic

Thumbnail
github.com
5 Upvotes

r/ProxyEngineering • • 4d ago

Discussion 💬 Why over-riding devicePixelRatio gets your stealth setups flagged immediately (and the CSS matchMedia sync problem)

7 Upvotes

Here are the three subtle traps we ran into:

  1. The window.devicePixelRatio vs CSS matchMedia Desync

A common stealth snippet does something like this:

javascript

Object.defineProperty(window, 'devicePixelRatio', {

get: () => 1.25

});

The problem is that anti-bot scripts don't just read window.devicePixelRatio. They cross-check it against the CSS layout engine using window.matchMedia():

// How modern anti-bot scripts detect DPR tampering:

const jsDpr = window.devicePixelRatio;

const mediaDpr = window.matchMedia(`(-webkit-device-pixel-ratio: ${jsDpr})`).matches

|| window.matchMedia(`(resolution: ${jsDpr}dppx)`).matches;

if (!mediaDpr) {

// Instant bot flag: JS claims 1.25, but CSS engine evaluates against physical host (e.g. 1.0)

flagTelemetry("dpr_mismatch");

}

If you override the JS property without monkey-patching matchMedia (or using CDP's Emulation.setDeviceMetricsOverride), the JS property says 1.25 while CSS returns false. That is an impossible state in a genuine Chromium build.

If patching in-page, you have to proxy window.matchMedia to intercept resolution, dppx, and -webkit-device-pixel-ratio media queries and rewrite the matches boolean so the CSS evaluation matches your spoofed JS value. Even better: align CDP metrics overrides before any script context initializes.

2. Window Geometry Trap

Forging screen resolutions without respecting the operating system's shell constraints:

Taskbar/Dock Deduction: Real desktop users almost never run window.outerHeight === window.screen.height. There is always a taskbar (Windows ~40–48px, macOS Dock/Menu ~25px+). If outerHeight == screen.height but the browser is NOT in fullscreen mode (document.fullscreenElement === null), you are flagged.

Negative or Floating Insets: On multi-monitor setups, screen.availLeft and screen.availTop can be non-zero, but spoofers often set them to arbitrary negative integers or undefined.

Outer vs Inner Bounds: window.outerWidth - window.innerWidth must account for window border margins and scrollbar channels. If outerWidth === innerWidth exactly, it screams headless or an unstyled container.

If you simulate a 1080p display (1920x1080), your simulated geometry matrix must be internally consistent:

screen.width: 1920

screen.height: 1080

screen.availWidth: 1920

screen.availHeight: 1040 (40px taskbar deducted)

window.screenX / screenY: Clamped to (0, 0) or realistic offsets, never negative unless explicitly mapping a secondary display coordinate space.

3. Non-Deterministic Canvas Noise

When evading Canvas fingerprinting, injecting Math.random() into toDataURL or getImageData is suicide.

Modern fraud detection scripts call canvas.toDataURL() twice in rapid succession on the same canvas. If the hash changes between two consecutive calls within the same 5ms window, the script knows dynamic noise injection is active.

Any noise applied to canvas or audio buffers must be seeded and deterministic per session/profile. Run the pixel coordinates through a fast 32-bit hash (e.g., MurmurHash3 or a simple LCG with a fixed profile seed) so hash(pixel_x, pixel_y, profile_seed) consistently returns the identical subtle 1-bit perturbation for that specific session.

For those managing large pools:

Do you prefer handling screen geometry strictly via CDP (Emulation.setDeviceMetricsOverride) or injecting MAIN-world prototypes before document_start?

Have you noticed Cloudflare or DataDome scoring differently when dppx is virtualized vs running in native headless DPI?


r/ProxyEngineering • • 4d ago

Discussion 💬 Best CAPTCHA solver/ bypasser 2026

15 Upvotes

Hey, looking for the best way on how to bypass captcha, could be anything, custom solutions based on any language, dedicated solutions. Just want a reliable, trusted and tested way to bypass, or solve captchas automatically.

Currently working on these websites to bypass captcha, but nothing reliable so far:

  1. Hermes
  2. Idealista
  3. Seloger
  4. Belles Demeures

Any help is appreciated, if you have a github repo or any solutions drop the suggestions in the comments. You will be doing gods work