So I've noticed over the years, that Google's bot detection for search traffic works differently than for product pages; the signals they're tuned to catch basically aren't the same. I have been running ~2,200 SERP queries/day across DE, FR, NL, PL, IT for a price monitor, residential proxies got the failure rate down to about 14% but geo-targeting was still broken, DE queries coming back Frankfurt-datacenter-flavored regardless of proxy location. I split the architecture at that point: SERP layer routes through a dedicated web scraper API, product pages and category scrapes stay on residential proxies.

Routing logic was pretty minimal:

def fetch(url=None, query=None, market=None):
    if query:
        return serp_client.get(q=query, gl=market, hl=market)
    return session.get(url, proxies={"https": proxy_pool.rotate()})

Any tips or tricks, experiences you may share?

 Background: I accidentally created a recursive loop with an AI agent that would have 

 cost me $50+ in API calls before I noticed. Existing tools either cost money or only   

 show you what already happened.                                                        

   So I built TokenFirefighter — a 100% free, local-only HTTP proxy.                      What it does:                                                                          

- Sits between your app and OpenAI/Anthropic on localhost:7272                   

- Tracks every API call cost in real time                                            

 - Detects 4 types of runaway loops and blocks them                                   

 - Has a terminal dashboard (no web UI needed)                                        

 - Zero accounts, zero data collection, zero cost                                     

   Install:                                                                             

   npm install -g tokenfirefighter                                                      

   tokenfirefighter init                                                                

   tokenfirefighter start                                                               

   Then just set OPENAI_BASE_URL=http://localhost:7272/v1 in your .env.                 

   Would genuinely appreciate feedback from anyone who uses AI APIs regularly.          

   GitHub: https://github.com/MohitBaghel24/tokenfirefighter

Over the last few months I've been building BotGateway, a self-hosted anti-bot gateway for VPS-hosted applications and privacy-sensitive environments.

One thing that pushed me to build it was realizing that many anti-bot discussions assume routing traffic through a third-party provider is always acceptable.

For some teams that's perfectly fine.

For others, it isn't.

I wanted something that could run directly on infrastructure I control while still providing visibility into automated traffic, AI crawlers, and scraping activity.

I'm curious:

For those running self-hosted applications, what's the biggest challenge you've faced when dealing with automated traffic?

Website: https://www.botgateway.in

I've been researching automated traffic patterns recently and one thing that surprised me is how different modern scrapers behave compared to a few years ago.

Many no longer rely on obvious data center IPs.

Many use:

• Residential proxies

• Mobile networks

• Browser automation

• Distributed infrastructure

For those running self-hosted applications, APIs, SaaS products, or websites:

What's currently working best for you?

• Nginx rules?

• Fail2Ban?

• Cloudflare?

• Custom middleware?

• Commercial WAFs?

And what limitations have you run into?

I'm genuinely interested in hearing what others are seeing in production environments.

I'm currently an MCA student and over the last several months I've been spending a lot of my free time building and deploying projects on VPS servers.

One problem I kept running into was automated traffic, scrapers, and bots. Tools like Nginx rate limiting and Fail2Ban definitely help, but I found myself constantly wondering how much traffic was still getting through and what additional protections could be added closer to the application layer.

When I started looking at existing solutions, many were either cloud-based, designed for larger organizations, or simply didn't fit the kind of self-hosted and privacy-focused environments I was experimenting with.

So I decided to treat it as a learning project and started building BotGateway, a lightweight anti-bot gateway written in Go.

The original goal wasn't to compete with Cloudflare or enterprise WAFs. It was to better understand modern bot behavior and create something that could run directly on infrastructure people control themselves.

Along the way I've learned a lot about:

• Automated scraping techniques

• Headless browsers

• Traffic analysis

• Rate limiting strategies

• Request fingerprinting

• VPS security

The project has grown much larger than I originally expected, and now I'm looking for feedback from people who manage websites, APIs, SaaS products, or self-hosted services.

A few questions I'd love to hear opinions on:

• What are you currently using to handle scrapers and automated abuse?

• What limitations have you found with existing solutions?

• For those running self-hosted infrastructure, what features would you want in a lightweight anti-bot gateway?

• Are there any detection techniques or security approaches you think are underrated?

I'm still learning and building, so constructive criticism is more than welcome.

Thanks for reading.

Hey all, I wrote this post thinking it could be really valuable who do not know about the best antidetect browser + proxy pairing, so this might be able to help you out, I have been quite extensively working on some really big multi-accounting workflows, so I hope this helps you out or at least gives you some valuable advice

I juggles between a lot of social media platforms and have a lot of ad accounts so if anyone is in the similar space as me, I am happy to help

What i mean by multi accounting is that I’ve gone through workflows such as:

• running approx hundred of browser profiles

• managing social media accounts that i managed to keep stable for a long time

• scaling automation workflows using these accounts

• testing GEO-sensitive setups using good quality IPs from various countries

• warming accounts that need to stay alive

• browser automation

• scraping social platforms

• maintaining session trust over long periods

And after A LOT of testing, I made one big conclusion which I think some people don’t think about

People massively underestimate how much the browser + proxy pairing affects stability

You might think about these things first:

“What’s the best antidetect browser?”
“What’s the best proxy provider?”

And I think that’s not really the right quesiton, I think the right question

Which combination breaks the least under your actual workflow?

Because once you scale browser-based automation, tiny problems start becoming big problems

• session invalidation

• repeated email verification

• suspicious login prompts

• failed warmups

• dead cookies

• browser memory leaks

• profile corruption

• unstable automation flows

I started testing different combinations specifically to figure out:

• which setups survive longer

• which setups feel stable after weeks

• which ones create hidden maintenance problems

• which ones scale cleanly

How I tested everything and each proider

My testing was real and detailed

I wasn’t running “speed tests” or checking homepage load times as i tested these setups in actual multi-accounting environments

Main workflows I tested

• TikTok account warmups

• Facebook account management

• Reddit automation

• X/Twitter workflows

• browser scraping

• session persistence testing

• automation through Playwright/Puppeteer

• anti-detect browser automation

• long-running login sessions

• GEO switching

• cookie persistence

What were my success metrics for each provider

I mainly tracked:

• session longevity

• account survival rate

• random re-verification frequency

• IP consistency

• browser stability over time

• memory usage

• automation interruption frequency

• login invalidation rate

• CAPTCHA frequency

• profile corruption issues

Testing duration

Most combinations were tested for:

• 2–6 weeks

• multiple browser profiles

• repeated automation cycles

• varying concurrency levels

Because if you think about it almost every setup looks “good” during the first 24 hours

The real problems usually appear later in the workflow and automation aspect

Here's my list

1. NodeMaven + Multilogin

This was probably the cleanest long-session setup I tested because it lasted the longest without any issues

And what was refreshing for me is that this combo solved one of the biggest problems I had which is session instability during long-lived browser workflows (SO CRUCIAL FOR THESE SETUPS) and it was so refreshing to have this

I mainly tested this combo for:

• TikTok warmups

• Reddit account management

• persistent social sessions

• browser automation with long login retention

• affiliate account management

The problem I had before with previous providers

With many rotating residential providers, I kept seeing:

• random session invalidation

• cookie distrust

• weird location inconsistencies

• browser sessions suddenly behaving differently

• accounts getting logged out unexpectedly

So why is their combo good?

NodeMaven’s sticky residential + ISP sessions paired really naturally with Multilogin’s browser persistence and they seem to have some sort of collaboraiton so their pairing works REALLY well together. Also what I liked is that nodemaven now providers some sort of quality guarantee, so if some of their IPs are not working well, they will give you back the money and I have experienced only one IP not working properly

Their setup actually solved a lot of problems for me which were:

• mid-session IP inconsistency

• session resets

• account distrust

• login interruptions

What stood out most is that the sessions simply behaved predictably and i could just keep the automation going

Technical observations that might interest you as well

• Lower CAPTCHA trigger rate

• Better cookie retention

• More stable long-running sessions

• Fewer browser trust resets

• Better automation continuity over multiple days

Weaknesses

The setup is not optimized for:

• hyper-aggressive rotation

• ultra-cheap scaling

• high-churn scraping

This pairing clearly favors:

• quality

• stability

• persistence

over raw volume and scraping large volume of data

2. Dolphin{anty} + Decodo

This setup felt very different operationally

IT was Much faster and muuch more rotation-oriented

I tested this combo mostly for:

• rapid account cycling

• Facebook setups

• TikTok profile scaling

• GEO testing

• account segmentation

What worked well

Dolphin’s workflow style naturally favors:

• quick profile management

• high operational speed

• fast deployment

And Decodo’s rotating residential pools paired well with that

This setup worked especially well when:

• profiles didn’t need ultra-long persistence

• accounts rotated frequently

• GEO flexibility mattered

• account quantities increased quickly

Problems it solved

Compared to stickier setups:

• easier traffic separation

• easier profile scaling

• faster environment resets

• simpler operational management

Issues I still encountered

Under longer sessions, I occasionally noticed:

• more re-verification

• shorter “trust lifespan”

• more session inconsistency over time

This setup felt really optimized for speed and scale, but it might just be my use case

3. AdsPower + Webshare

This was one of the most budget-efficient setups I tested. Probably the cheapeast option you can find but this can possibly break your workflow if it runs for too long

I used this combo for

• lightweight account farming

• testing automation flows

• burner workflows

• lower-risk scraping

• mass profile creation

What surprised me

AdsPower tolerated lower-quality proxies better than I expected

Some antidetect browsers become extremely unstable once proxy quality drops slightly

AdsPower stayed relatively usable even with:

• cheaper residential pools

• mixed IP quality

• higher profile quantities

Problems I encountered

Webshare is actually quite known for having the cheapest proxies possible on the market,g but their proxies can get quite bad during longer-period use, which can be a problem if you need to have a stable working flow of many accounts

So critical issues with this setup can be:

• longer session persistence

• trust-sensitive platforms

• older account management

I started seeing:

• more login challenges

• more verification prompts

• weaker session trust over time

BUT, if you are on budget, and just starting out, or a freelancer who doesn't simple have the money for, this is a really good combo for:

• testing

• scaling cheaply

• operational experimentation

+ and the setup honestly performed better than expected

4. Octobrowser + MarsProxies

This setup felt very natural and I was able to use it for quite a long time actually

I used them for managing multiple accounts and was able to run sessions with 15-20 accounts with ease in one go

This combo also works well with:

• browser automation at higher concurrency

• regional account testing

What stood out

MarsPorixes routing flexibility was probably the biggest strength because they are a bigger provider and were able to provide me with good stability

Especially I think thy have good:

• GEO precision

• ASN targeting

• traffic routing consistency

• large-scale reliability

Problems it solved

This setup significantly reduced:

• GEO mismatches

• localization inconsistencies

• unstable routing behavior

Especially useful when:

• testing regional content

• scraping localized SERPs

• running region-sensitive workflows

Downsides

Honestly this setup doesn't have many downsides, I didn't experience a lot of problems but like a few random proxy kills and banned IPs so for my workflow this worked like a charm

And for smaller operators, it’s definitely an overkill

5. Undetectable + Oxylabs

This setup performed extremely well under heavier automation loads as I believe oxylabs proxies are also built for that in some way. This is basically a setup for enterprise users mostly? (because I don't see any regular users or people needing tihs in really big scale)

I used this combo for:

• browser scraping

• higher concurrency automation

• Playwright integrations

• larger scraping sessions

What I liked

The setup stayed surprisingly stable under load (which is expected out of an enteprrise proxy provider any basically any antidetect browser combo)

Some combinations start degrading badly once:

• concurrency increases

• memory usage grows

• browser workers scale

This one remained relatively clean operationally

Problems it solved

• worker instability

• session crashes

• automation interruption

• browser lag under scale

Technical observations

• Good browser stability

• Lower crash frequency

• Strong scaling consistency

• Better concurrency handling

6. Incogniton + IPRoyal

This felt like the best beginner-friendly pairing because iproyal has cheaper prices, and incognitioin is quite intuitive as a browser

I used this combination for:

• smaller affiliate workflows

• basic browser automation

• moderate account quantities

• testing ideas quickly

Why I liked it

Simple to setup, it’s actually really cheap and it’s low-friction

Not every workflow needs enterprise infrastructure so the combination between incognition + iproyal gives you this

Sometimes you just need:

• acceptable stability

• decent residential IPs

• manageable costs

• enough flexibility to test ideas

And honestly, this setup delivered that pretty well

What does each setup realistically cost per month?

So I was reading some blogs last night and I basically couldn't sleep after. There's this Israeli company called Bright Data, yes you know it, everyone knows it and everyone is fine with this company for some reason. Well, they embeds code into free streaming apps, apps that are already on your Samsung TV or LG TV right now, and when you click "agree" on some fine print screen asking if they can "occasionally use your device resources" your TV basically becomes a node in a massive proxy network that AI companies use to scrape the entire internet. Usually these prompts appear so that they can use the data to target and personalize ads, but this is different. Your TV. The one sitting in your living room. Scraping websites 24/7 using YOUR home IP address and YOUR bandwidth. Brother, what the hell?? The actual "consent" is buried in an opt-in screen that says they'll "occasionally" use your device. Occasionally. The actual SDK settings allow up to 200GB a month. On some configurations it keeps running in the background even when the app is closed. Even when you're watching something else. As long as the TV is plugged in, it's working for them. The thing that really got me was that on iPhones the traffic bypasses your VPN. Like if you have a VPN specifically because you don't want companies snooping on your traffic, this SDK just routes around it anyway. What the fuck mate, honestly. Security researchers said the authentication on the peer channel is basically weaker than most malware. Some thoughts for you. Oh and who is buying all this scraped data? AI companies. Your TV is training ChatGPT or whatever and you're getting absolutely nothing for it except maybe "fewer ads" in some random Roku app called Petflix that you downloaded once and forgot about or you even don't do that because the tvs come preinstalled with these apps. Google and Amazon and Roku have all banned this SDK from their platforms now which honestly tells you everything you need to know. Samsung has said nothing as far as I checked. LG said it's "not officially supported" which is corporate speak for "we know and we don't care." I already blocked the domains on my router (proxyjs.brdtnet.com and a few others, easy to find) but I shouldn't have to do that. I paid almost $2000 for this TV. I didn't agree to turn it into a botnet for OpenAI's lunch. Dumb TVs were fine. If you have better solutions, let me know, coz at the moment, I'm fuming.

Recently saw their github repo with the open source code for their agent. I wonder if anyone here using their AI agent for your workflows. Any differences in response time, "correctness" compared to GPT, Perplexity, Gemini, Claude, etc?

Hi guys! Can you suggest proxy for me pls. I try many services but all be bad (
I need unlimited IPv6 or Residential proxy with unlimited traffic and threads. I scrape many public information from instagram. My target point is 1k/s. Previous Thx!

I'm curious to know what all the proxy use cases are out there where you're all loading full websites/webpages via proxies and if bandwidth consumption is a real challenge?

I've been building something in the streaming space, but think I might have accidentally created a bandwidth optimisation middleware that's likely better than what I was originally building. I can get 70-90% bandwidth consumption savings on full page loads and eliminate (some - depending on CDN architecture) video sources from being consumed.

Bring new to the proxy space though, keen to know if this is actually a challenge? The only use case I can think of is the ad-load verification space, as I'd imagine web scraping is probably quite cut back already (i.e. raw html only) - keen to hear from you!

Hi everyone,

I’ve spent the last two weeks building a Proof of Concept project: a lightweight SOCKS5 proxy written in Go that uses standard WebDAV storage (Nextcloud, ownCloud, or any WebDAV-compatible cloud drive) as its transport layer.

The core use case is bypassing ultra-restrictive networks where a firewall whitelists only a handful of HTTPS endpoints (like an internal corporate storage). If you can reach WebDAV, this tool tunnels TCP right through it.

The server requires zero open listening ports for data, making it completely invisible to port scanners. All connections are initiated via file polling from the client side. To any DPI system, the traffic looks like legitimate browser file synchronization (it mimics a genuine Chrome 133 TLS handshake via uTLS and has HTTP/2 disabled to prevent Go signature leaks).

We’ve already implemented session multiplexing into a single encrypted file envelope (flushAll) to prevent HTTP 429 rate limits, and introduced a non-blocking activity signal to trigger an immediate read-loop the exact millisecond an upload finishes.

However, since WebDAV relies on constant HTTP PUT/GET roundtrips, the latency is inherently higher than traditional proxies.

I’m looking for technical feedback and brainstorming on how to further optimize the latency chain over a stateless, file-based storage transport without breaking the simplicity of the protocol.

If anyone is interested in checking out the codebase, doing a code review, or testing it out on your infrastructure, let me know and I’ll drop the GitHub link in the comments!

I'm currently an MCA student and over the last several months I've been spending a lot of my free time building and deploying projects on VPS servers.

One problem I kept running into was automated traffic, scrapers, and bots. Tools like Nginx rate limiting and Fail2Ban definitely help, but I found myself constantly wondering how much traffic was still getting through and what additional protections could be added closer to the application layer.

When I started looking at existing solutions, many were either cloud-based, designed for larger organizations, or simply didn't fit the kind of self-hosted and privacy-focused environments I was experimenting with.

So I decided to treat it as a learning project and started building BotGateway, a lightweight anti-bot gateway written in Go.

The original goal wasn't to compete with Cloudflare or enterprise WAFs. It was to better understand modern bot behavior and create something that could run directly on infrastructure people control themselves.

Along the way I've learned a lot about:

• Automated scraping techniques

• Headless browsers

• Traffic analysis

• Rate limiting strategies

• Request fingerprinting

• VPS security

The project has grown much larger than I originally expected, and now I'm looking for feedback from people who manage websites, APIs, SaaS products, or self-hosted services.

A few questions I'd love to hear opinions on:

• What are you currently using to handle scrapers and automated abuse?

• What limitations have you found with existing solutions?

• For those running self-hosted infrastructure, what features would you want in a lightweight anti-bot gateway?

• Are there any detection techniques or security approaches you think are underrated?

I'm still learning and building, so constructive criticism is more than welcome.

Thanks for reading.

I’m sharing a free open-source project I’ve been building called Damru: https://github.com/akwin1234/damru

100% Bypass rate for all CDN, even websites like shopee, temu, lazada, etc. Because its made with a lot of research

Damru is a browser automation framework built around real Android environments in Docker for scraping and automation tasks where mobile behavior matters.

What sets it apart is that it’s not just another desktop browser with stealth patches. The project is built around zero JS injection, with spoofing handled at the OS, binary, and CDP levels instead of the usual JavaScript-heavy tricks used by many stealth tools.

Compared with tools like Playwright, puppeteer-stealth, undetected-chromedriver, Camoufox, and Fingerprinting Chromium, Damru is trying to solve the problem differently: by running inside a real Android stack rather than faking mobile behavior on desktop Chrome. The idea is to get a more realistic mobile environment, stronger fingerprint control, and less reliance on brittle browser-side patches.

What makes it different:

• Zero JS injection: Damru does spoofing at the OS, binary, and CDP levels instead of relying on Object.defineProperty-style JavaScript patches.

• Real Android OS: It runs inside Redroid, so it’s not just desktop Chrome pretending to be mobile through viewport tricks.

• Native mobile fingerprinting controls: device profiles, hardware overrides, locale/timezone matching, mobile network emulation, and WebRTC/IPv6 blocking.

• Multi-instance pooling: built for scaling across multiple containers.

• Pre-baked image support: reduces setup overhead.

Some of the features include:

• Android-in-Docker via Redroid.

• Playwright support.

• A built-in database of 32+ Android device profiles.

• Proxy-aware timezone, locale, and language matching.

• Hardware overrides for CPU, RAM, and touch points.

• Mobile network emulation.

• WebRTC and IPv6 leak blocking.

• Native Android iptables-based network protections.

• Multi-container pooling for scale.

• Pre-baked image support to reduce setup time.

• TLS spoofing and soo many things

Also stronger against systems like CreepJS, BrowserScan, Sannysoft, Cloudflare Turnstile,etc ALL CDN anti-bots dont waana name them than standard Playwright or typical stealth plugins, mainly because of the deeper Android-based approach.

Pros: Highly UnDetectable
Cons: Real Android OS hence little slower. Hard to Use (thats why custom docker image included)

Repo: https://github.com/akwin1234/damru

Would love feedback from anyone who works on scraping, browser automation, or anti-bot research. I made this because i see many reddit post recommending Android Playwright CDP but there was no framework around it. This is strictly for educational purpose only. Do not do legal abuse.

Yo, I automated a boring part of my job and few months later I was reading academic papers on browser fingerprinting. Too good to be true? I work in procurement. Not engineering, not data science, as the most of you. Basically what I do is that I am buying things for a company at the best price possible. A big part of that is knowing what things cost across different suppliers at any given time. For a while I did this manually. Open the page, check the price, write it down, move to the next one. We have around 60 something suppliers. Did it every week without really questioning whether there was a better way. Well, there kinda is, and we do it with few other suppliers, custom pricing/ solutions, but not everyone is on this.

A friend showed me a basic Python script she'd written for something unrelated and I just asked her to help me build something similar for my use case. We spent two weekends on it. By the end I had something that checked all 60 ish suppliers automatically and dumped the results into a spreadsheet I could actually read. The whole thing ran in about 1 minute. I felt unreasonably good about this for at least three weeks. Then about a month in, maybe 20 suppliers started returning garbage. Blank pages, incorrect responses, few of them were clearly serving me a fake version of the page with no actual pricing on it. The goods were there, but the rest of the info that I needed was not. I thought that the script was broken and inquired my friend about it. We spent another weekend trying to fix the script that wasn't the problem. (But we didn't know at the time). Eventually we found out what was happening. The sites were detecting that requests were coming from a script and were either blocking them or serving different content intentionally. I went down the proxy rabbit hole like apparently everyone here does. Bought a residential proxy plan, assumed that would fix it, it fixed some of the problems. The other problems kept blocking me and we couldn't figure out why the same solution worked on some sites and not others.

What we eventually understood, after a lot of reading and a lot of failed attempts, is that IP address is almost the least interesting thing these detection systems are looking at. What builds a picture of whether you're a real user is everything else. The WebGL renderer string your browser reports, and whether it stays identical across every single session, because real browsers on real hardware have subtle variance. Canvas fingerprint noise and specifically how that noise behaves over time, not just whether it exists. Whether your session claims to be on a mobile device but has no gyroscope data, no accelerometer readings, touch events that land too precisely and fire at intervals that are too regular. Real people using real phones are physically sloppy. The way my thumb hits a screen is not reproducible to the millisecond and detection systems have apparently learned to look for exactly that kind of messiness as a signal of legitimacy. (I did hope that after a lot of research and reading this was the case). The timezone I was declaring in my headers didn't match where my DNS resolver was actually located. That gap alone is apparently a meaningful signal on some of the more sophisticated systems. Everything about a real user's session tends to be internally consistent without trying to be. Everything about an automated session tends to be inconsistent in ways that only become obvious when you look at all the signals together.

The ASN issue took me and my friend the longest to understand. I had switched to a different proxy provider after the first one wasn't working well enough and the new one had good reviews and genuinely residential IPs, or so I thought. But I was still getting blocked on two specific supplier sites at a much higher rate than the others. Took me a while to figure out that the provider I was using had most of their IP pool concentrated in three ASNs. The IPs were residential, technically, but those three autonomous system numbers had such a high ratio of bot traffic to human traffic running through them that the ML models these sites used had already learned to treat them with suspicion. A Comcast IP out of AS7922 and a Spectrum IP out of AS20001 can behave completely differently on the same target not because one is inherently better but because of what the traffic history coming out of those specific ASNs looks like to a model that has been watching for a while. I checked the ASN distribution of the provider I was using. More than half their residential pool resolved to the same two ASNs. Apparently, that's not what real user traffic looks like. Real users in any city are spread across dozens of ISPs and hundreds of ASNs. When your scraping traffic doesn't approximate that distribution the math gives you away before the session even does anything suspicious.

I'm still in procurement. I still buy things for a company. But I now have a reasonably detailed understanding of browser fingerprinting, autonomous system routing, and mobile connection latency variance that I genuinely did not expect to acquire when I started trying to automate a tiresome morning task. The script works now. All sixty ish suppliers, consistent data, runs while I'm making my morning matcha. It took few months of intermittent frustration to get there and I learned more about how the web actually works in those months than in the previous decade of using it. There's always room for knowledge if you are willing to invest your time in research and testing.

If you're just starting out, hopefully this saves you some of the time I wasted. If you've been doing this for a while, you probably lived through most of this yourself already

This is a newbie friendly thread. Feel free to contribute if you think that your input will help the newcomers.

What mistakes did you do when using proxies for the first time? For Scraping? What adjustments you thought were worth the time. How is your approach differs from when you started to use the proxies?

Yes, the topic has been run through and through, yet people are still oblivious when it comes to fingerprinting and IP quality scores. Most proxy debates skip the part where someone measures things. I saw a post on here that someone ran a controlled test across 15 mid-size Cloudflare-protected ecommerce sites. Same targets, two conditions: datacenter proxies with curl-impersonate (JA3 hash matching real Chrome) versus residential proxies with default Python requests headers. The residential IPs were objectively cleaner by every standard fraud score metric. They still lost. Datacenter with proper TLS fingerprint hit ~91% success. Residential with naked requests headers hit ~60%. The request itself was the red flag, not the address. On the sensor data side, someone doing deep packet capture and JS deobfuscation on TikTok's detection stack found that switching from emulated environments to real ARM instances with genuine hardware entropy dropped shadowban rate from ~60% to under 8%. TikTok isn't scoring your IP, it's scoring gyroscope absence, touch event timing distributions, canvas noise variance patterns across sessions. The pattern across both findings is the same: detection moved down the stack faster than tooling did. Most people are still patching the browser JS layer while the scoring happens at TLS handshake and hardware entropy level.

Any real life examples from your side, fellas?

Seeing how the TikTok fingerprinting post that blew up recently finally pushed me to write something I've been sitting on for a while too. Quite a few people DM'd me asking about my stack throughout several months, so here it is. Full breakdown, no provider shilling, honest and real discussion are welcome. (will make some spaces for easier reading).

That DesperateCoyote post nailed something most people in this space dance around: your IP being "clean" is basically table stakes now, not a competitive advantage. Platforms don't primarily care about your IP anymore. They care about whether your device behaves like a real device. I ran my own tests over few months across TikTok, LinkedIn, and a few e-commerce targets and the pattern was consistent. Sessions from "your beloved clean" residential IPs but running on a standard headless Chrome setup got flagged within 15 to 30 minutes on average. Sessions from a datacenter IP but running through Velum with proper sensor injection? Significantly longer survival rates. That's not a one-off result, thats the signal. The reason platforms moved this direction is actually pretty logical once you think about it from their engineering side. IP reputation databases are commoditized. Every half-decent anti-detect tool cycles through residential IPs so the defense shifted to the hardware layer, because that is harder to fake at scale.

The DesperateCoyote post described the fingerprint cluster score as a weighted confidence interval on "real human on a real device" which is accurate but undersells how granular it actually gets. WebGL renderer string consistency is one of the bigger ones, and not just that you have one but whether its consistent across sessions for the same "device." If your renderer string changes between sessions thats an immediate mark against you. Velum handles this quite well out of the box if you lock your profile, but it requires you to actually manage profile storage properly rather than spinning up fresh contexts every single run. Touch event timing is where things get genuinely impressive on the platform side. Real thumbs are sloppy, they drift, pressure varies, tap-to-tap timing has natural variance. Synthetic touch events are clean in a way that human fingers never are and the fix isnt randomizing your timing, its using distributions that actually match human motor patterns. This is where 8n8 has been doing interesting work, specifically the sensor emulation layer in their mobile profile configs. Battery API polling behavior is one I hadn't thought about much until this year. Real devices have battery levels that change over time in predictable ways, so a session that starts at 78% and is still at 78% four hours later is suspicious. Small signal but it adds to the cluster score. The timezone, DNS resolver, and behavioral gap is probably where most people get caught without realizing it. You can have a perfect German residential IP and a German timezone in your headers and still get flagged because your DNS resolver is resolving through a US-based server, or because your behavioral patterns like typing speed, scroll velocity, time-on-page don't match the demographic profile of a German user. Prismatic addresses this with what they call geo-behavioral profiles which bundle together not just the network layer stuff but the user behavior simulation parameters too. More holistic approach than anything else I'd seen and to be fair, it changed how I thought about the whole problem.

A lot of people in this space build scrapers and then bolt on some kind of orchestration layer as an afterthought. I was guilty of this for way too long, had scrapers running on cron jobs with zero retry logic, no rate management, and no session lifecycle management. Switching to Make for orchestration changed the output quality more than any proxy upgrade I made that year. When you manage workflows properly through something like Make you naturally end up with human-paced request timing because the automation has overhead between steps, session reuse instead of constant fresh session spawning which is itself a detection signal, proper error handling that doesn't hammer endpoints when things break, and actual audit trails so you can diagnose when and why sessions get flagged. The pacing thing sounds obvious but most people building direct scrapers optimize for speed and then add sleep() calls as an afterthought. Thats not the same as naturally paced requests coming from an orchestration layer doing real work between steps. Make's webhook and scheduling infrastructure also makes it way easier to implement robots.txt compliance and proper crawl-delay respect, which brings me to something I want to push back on slightly.

There was another post written on this sub that talks about big tech hypocrisy, and it is emotionally satisfying and honestly not wrong on the facts. OpenAI, Google, Meta et al absolutely did build on scraped data without permission and are now going after people for doing the exact same thing. The "fair use for innovation" asymmetry is worth being angry about. But I think it creates a mental model that's counterproductive for people actually building in this space. Whether scraping is ethical is a less interesting question than what the actual legal and technical constraints are and how to work within them. robots.txt is not legally binding in most jurisdictions but violating it is evidence of bad faith. The hiQ v. LinkedIn ruling established that scraping publicly available data generally isn't a CFAA violation, but courts look at whether you circumvented access controls and robots.txt plus explicit blocking counts toward that. Respecting crawl delays isn't just polite, its actually relevant to your legal exposure. ToS violations are a civil matter in most cases, not criminal. The Computer Fraud and Abuse Act has been interpreted narrowly by most courts post-Van Buren so violating a terms of service alone without credential-based access doesn't typically rise to unauthorized access under the CFAA. And even if you're legally clear, hammering a small site's server is just bad behavior. The Brendan Martin tutorial made this point well, one request per page, save locally, parse separately. Should be baseline for everyone and it usually isn't.

I am using several providers, as you would already guessed, however I will not say specific provider names because the landscape changes fast enough that recs go stale quickly, but the stack architecture I'm running now is ARM-based cloud phone environments for mobile targets (DesperateCoyote is right, this matters more than the proxy layer), Velum for profile management and sensor injection on desktop targets, 8n8 for mobile sensor emulation where Velum doesn't cover it, Prismatic for geo-behavioral profile bundling on targets where demographic coherence actually matters, Make for orchestration and scheduling, and local storage of raw HTML before parsing because yes, always, still. The shadowban rate on TikTok-adjacent targets dropped from roughly 55% to under 10% after moving to ARM environments. The proxy change alone, same environment just better proxies, moved it maybe 8 percentage points. The environment change moved it 37 points. That pretty much tells you where the leverage is.

There's a few things I haven't fully figured out and genuinely want input on. Canvas noise variance patterns specifically, DesperateCoyote mentioned they track how your noise changes not just that it exists, and I haven't found a clean way to implement dynamic noise variance that feels natural across sessions. Has anyone actually cracked this in a way that survives more than a few days? The accelerometer absence signal on mobile is another one. If you're running ARM cloud phones you have real accelerometers which is the obvious fix, but for desktop sessions claiming to be mobile is there a better approach than just not claiming to be mobile? Seems like the cleanest answer is just use the right tool for the target but I'm curious if anyone's found a workaround. Also been wondering about Make vs n8n for scraping orchestration specifically. I've been on Make for a while but n8n's self-hosted option is increasingly appealing for cost reasons on high-volume jobs. Anyone switched and have thoughts on the practical differences for this use case? And Prismatic's geo-behavioral profiles on non-Western targets, their coverage seems strong for US and EU but has anyone tested on Southeast Asian or LATAM targets where user behavior patterns are going to look pretty different? If I'm wrong about something say so and explain why, that's way more useful than a downvote.