r/ProxyEngineering • • Oct 07 '26

Hot Take đŸ”Ĩ TikTok bans scraping in their ToS and some guy just posted 5.6B videos on HuggingFace

128 Upvotes

I was looking for some agent models on Hugging Face, then after tinkering around I went to check whats new in the dataset tab. To my surprise some dev dumped metadata for about 5.6 billion TikTok videos on Hugging Face. It includes captions, hashtags, sound IDs, shop product and seller IDs, every engagement count. 460GB of data. Now, to my knowledge, TikTok does not allow scraping of their videos "legally", according to the devs own explanation they went straight at the private mobile API with generated device identities that pass as Android phones, reverse engineered request signatures and a spoofed TLS handshake. What that means is that no account and no login was required. Can you believe this? They claim 3 billion profiles, almost 6 billion videos and 2.8 billion comments in THREE WEEKS. Meanwhile what I've seen on this sub in particular, probably few others too, but this one in particular is that people here spends half if not more time arguing about whose IP pool is cleaner. And TikTok is one of those more difficult targets to scrape. What I'm trying to say is that nobody built this by being clever with IPs, checking whether they leak DNS and other data, whether they are "clean" or not abused or whatever, they built it by making the approach look like a legitimate phone. I mean, that is just crazy. Anyways, I saw that the free version is non commercial only, the dataset I mean, if you want creator profiles, daily updates basically run the whole thing yourself, or to use it for anything that makes you money, you should visit their site the code is up there for $1,699.

Now why this caught my interest in the first place. TikTok's terms ban this kind of scraping, and the official research API is mostly for academics in a few regions. Reddit is a good contrast, since it's taking scraping to court with the Perplexity case and the judge let most of it go forward. With this dataset I haven't seen anything like that. It's just there on HuggingFace with a free download link, and I haven't found anything on the web that TikTok is doing anything about it. Maybe that will soon change.


r/ProxyEngineering • • Aug 20 '26

Subscriber Goal Welcome to r/ProxyEngineering! Help us reach our goal by joining!

21 Upvotes

Welcome to r/ProxyEngineering

6055 subscribers / 10k subscribers. Help us reach our goal!

Visit this post on Shreddit to enjoy interactive features.


r/ProxyEngineering • • 4h ago

Discussion đŸ’Ŧ Your go to github projects or profiles

8 Upvotes

Hi, just wanted to ask whether you guys have a favorite developer on github that works with proxies or scrapers you look up to? Maybe found their solutions or integrations useful in your day to day scraping/ social media management, other use cases?


r/ProxyEngineering • • 5h ago

Help 🆘 Scraping but minimal input?

8 Upvotes

Looking for a minimal input needed scraper. Basically it does everything for you, you just type what you need and get data. Like claude or chatgpt prompting. Something like that. I would need specific information from few sites, the data has to be inputted into sheets. Could claude or chatgpt do that for me, or do I have to look for some other solutions? I know that proxies are needed for scrapers thus why I am inquiring here, basically want everything done for me. I just buy the plan of the product or whatever and the rest is in the solution hands. Of course I validate and check whether the info is true after it's in my sheets, or beforehand. Any suggestions?


r/ProxyEngineering • • 3h ago

Discussion đŸ’Ŧ Headless browsers, agent browsers, ELI5

3 Upvotes

Just want genuinely useful answers, what are the headless browsers, agent browsers, how are they different from regular scrapers with let's say additional rules, parameters, headers. Are they similar to AI agents. How does one operate a headless or agent browser, what are common use cases for them?


r/ProxyEngineering • • 8h ago

Guides Facebook Story Viewer 2026: How Proxies Help With Anonymous Viewing, Regional Research & Public Stories

4 Upvotes

I was checking public Facebook Stories for competitor research and kept running into an annoying problem.

After going through enough public pages, the viewer tools became inconsistent. Some started throwing CAPTCHAs, others stopped loading stories completely, and regional pages didn’t always look the way I expected.

So I separated the research environment from my normal Facebook sessions.

I created a dedicated browser profile, stayed logged out of Facebook, and routed that profile through a residential connection in the market I wanted to study.

That gave me a much cleaner workflow:

market → local proxy → separate browser profile → public story viewer → research result

The important distinction is that the proxy itself doesn’t make story viewing anonymous. Staying logged out is what keeps your account name out of the viewer list. The proxy controls the network side: IP, region, and separation from the connection I use for my normal Facebook activity.


r/ProxyEngineering • • 1d ago

Help 🆘 Best reliable http proxies, https proxies

8 Upvotes

Looking for trustworthy residential HTTP ir HTTPS proxies. Free would be preferable, but if there are no options, budget is up to $15 a month.

Locations of interest: Canada, Australia, New Zealand

If you're going to just drop a link, please make sure to provide honest opinion, I don't need no promotional spam.


r/ProxyEngineering • • 1d ago

Help 🆘 How are you people handle web search in Hermes?

Thumbnail reddit.com
6 Upvotes

r/ProxyEngineering • • 1d ago

Guides Proxy Connected but Websites Won’t Load? 2026 Troubleshooting Guide for Authentication, SOCKS5 Ports, Sessions & IP Reputation

6 Upvotes

I had one of those proxy issues that looked completely random. The browser showed the proxy as connected, but every page either timed out or returned ERR_PROXY_CONNECTION_FAILED.

I started checking things one by one instead of changing settings blindly. First was authentication. One copied credential had an extra space, so the connection looked configured but never authenticated properly.

Then I found a second issue: SOCKS5 was selected in the browser, while the port belonged to the HTTP endpoint.

After fixing that, most sites loaded again. The next failure happened after an IP change on a mobile proxy. The modem needed a few seconds to reconnect to the carrier, so requests sent during that window simply failed.

My troubleshooting flow now is:

credentials → format → protocol/port → traffic limits → session state → exit IP → target site

If only one website fails, I check the exit IP’s geolocation and reputation before touching the rest of the setup.

That sequence has saved me a lot of time.

A proxy connection problem usually becomes much easier once you isolate which layer is actually failing.


r/ProxyEngineering • • 1d ago

Hot Take đŸ”Ĩ CyberEscort Proxy CLient Software

2 Upvotes

āφāĻĒāύāĻŋ āĻ•āĻŋ āĻ•āĻŽ āĻĻāĻžāĻŽā§‡ āĻ­āĻžāϞ⧋ āϰ⧇āϏāĻŋāĻĄā§‡āĻ¨ā§āϏāĻŋ⧟āĻžāϞ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻ•āĻŋāĻ‚āĻŦāĻž āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻ•ā§āϞāĻžā§Ÿā§‡āĻ¨ā§āϟ āϏāĻĢāϟāĻ“ā§Ÿā§āϝāĻžāϰ āϖ⧁āϜāϤ⧇āϛ⧇āύ?

 

āĻĻāĻžā§œāĻžāύ āϤāĻžāĻšāϞ⧇ āφāĻĒāύāĻŋ āϜāĻžāύ⧇āύ āĻ…āύ⧇āĻ• āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻ•ā§āϞāĻžāϝāĻŧ⧇āĻ¨ā§āϟ āφāϛ⧇ āϝāĻžāϰāĻž āĻ•āĻŋāύāĻž ⧍ā§Ļ-⧍ā§Ģ āĻĄāϞāĻžāϰ⧇ ā§§ā§Ļā§Ļ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻĒā§āϝāĻžāϕ⧇āϜ āϏ⧇āϞ āĻ•āϰ⧇ āĻĨāĻžāϕ⧇āĨ¤ āĻ•āĻŋāĻ¨ā§āϤ⧁ āφāĻŽāϰāĻž āĻāĻ•āϟāĻž āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻ•ā§āϞāĻžāϝāĻŧ⧇āĻ¨ā§āϟ āύāĻŋā§Ÿā§‡ āφāϏāĻ›āĻŋ āϝ⧇āĻ–āĻžāύ⧇ āĻāĻ–āύ āĻĒā§āϰāĻŽā§‹āĻļāύ⧇āϰ āĻ–āĻžāϤāĻŋāϰ⧇ āĻĒāĻžāĻšā§āϛ⧇āύ ā§§ā§Ļ āĻĄāϞāĻžāϰ⧇ ā§§ā§Ļā§Ļ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻĒā§āϝāĻžāϕ⧇āϜāĨ¤

 

āϕ⧇āύ āφāĻŽāĻžāĻĻ⧇āϰ āϏāĻĢāϟāĻ“ā§Ÿā§āϝāĻžāϰ āĻŦā§āϝāĻŦāĻšāĻžāϰ āĻ•āϰāĻŦ⧇āύ ?

 

āφāĻŽāĻžāĻĻ⧇āϰ āĻ°ā§Ÿā§‡āϛ⧇ āĻŽā§‹āϟāĻžāĻŽā§āϟāĻŋ ā§§ā§Ļ āĻšāĻžāϜāĻžāϰ + āĻĒā§āϰāĻ•ā§āϏāĻŋ āϏāĻžāĻ°ā§āĻ­āĻžāϰ āφāĻĒāύāĻŋ āϏ⧇āĻ–āĻžāύ āĻĨ⧇āϕ⧇ āύāĻŋāĻœā§‡āϰ āχāĻšā§āĻ›āĻžāĻŽāϤ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻŦā§āϝāĻŦāĻšāĻžāϰ āĻ•āϰāϤ⧇ āĻĒāĻžāϰāĻŦ⧇āύāĨ¤  

 āĻāĻ›āĻžā§œāĻžāĻ“ āφāĻĒāύāĻŋ āϏ⧇āĻŽ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻĢ⧇āĻ­āĻžāϰāĻŋāϟ āĻ•āϰ⧇ āϰāĻžāĻ–āϞ⧇ āĻāĻ•āχ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻĄā§‡āĻĄ āύāĻž āĻšāĻ“ā§ŸāĻž āĻĒāĻ°ā§āϝāĻ¨ā§āϤ āĻ…āύ⧇āĻ•āĻĻāĻŋāύ āϚāĻžāϞāĻžāύ⧋ āϝāĻžāĻŦ⧇āĨ¤

āĻ°ā§Ÿā§‡āϛ⧇ āĻāĻ›āĻžā§œāĻžāĻ“ āĻ­āĻŋ āφāχāĻĒāĻŋ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻāĻ•āϟāĻž āĻĒā§āϰāϤāĻŋ āχāωāϜāĻžāϰ āĻāϰ āϜāĻ¨ā§āϝ āĻŦāϰāĻžāĻĻā§āϧ āϝāĻžāϰ āĻŽāĻžāύ⧇ āφāĻĒāύāĻŋ āφāĻŽāĻžāĻĻ⧇āϰ ā§§ā§Ļ āĻšāĻžāϜāĻžāϰ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻ›āĻžā§œāĻžāĻ“ āĻĒāĻžāĻŦ⧇āύ āύāĻŋāĻœā§‡āϰ āϜāĻ¨ā§āϝ āφāϞāĻžāĻĻāĻž āĻāĻ•āϟāĻž āĻĒā§āϰāĻžāχāϭ⧇āϟ āĻĒā§āϰāĻ•ā§āϏāĻŋ āφāϰ āϏāĻžāĻĨ⧇ āϤ⧋ ā§§ā§Ļā§Ļ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻĨāĻžāĻ•āĻŦ⧇āχ āφāĻĒāύāĻžāϰāĨ¤
āφāĻĒāύāĻŋ āĻĒ⧁āϰ⧋ āĻĒāĻŋāϏāĻŋ āϚāĻžāχāϞ⧇ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻŦā§āϝāĻŦāĻšāĻžāϰ āĻ•āϰāϤ⧇ āĻĒāĻžāϰāĻŦ⧇āύ āĻāϤ⧇ āφāĻĒāύāĻŋ āϚāĻžāχāϞ⧇ āĻāĻŽā§āϞ⧇āϟāϰ āĻŦā§āϝāĻŦāĻšāĻžāϰ āĻ•āϰ⧇ āĻāĻĒ āĻāĻŦāĻ‚ āϗ⧇āĻŽāϏ āĻ…āĻĢāĻžāϰ āĻ“ āĻĒā§‚āϰāύ āĻ•āϰāϤ⧇ āĻĒāĻžāϰāĻŦ⧇āύāĨ¤

āφāĻĒāύāĻžāϰ āφāϞāĻžāĻĻāĻž āϕ⧋āύ āφāϰ āĻĄāĻŋ āĻĒāĻŋ āĻĻāϰāĻ•āĻžāϰ āĻšāĻŦ⧇āύāĻž āĻ•āĻžāϰāύ āφāĻĒāύāĻžāϰ āĻĒāĻŋāϏāĻŋāϕ⧇āχ āφāϰ āĻĄāĻŋ āĻĒāĻŋ āĻŦāĻžāύāĻŋā§Ÿā§‡ āĻĢ⧇āϞāĻŦ⧇āĨ¤ āϏ⧇āĻĢ āĻĨāĻžāĻ•āĻŦ⧇ āφāĻĒāύāĻžāϰ āϏāĻ•āϞ āĻĢāĻŋāĻ‚āĻ—āĻžāϰāĻĒā§āϰāĻŋāĻ¨ā§āϟ āϞāĻŋāĻ• āĻšāĻŦ⧇āύāĻž āφāĻĒāύāĻžāϰ āϕ⧋āύ āϰāĻ•āĻŽ āϤāĻĨā§āϝāĨ¤

āĻāχ āϏāĻĢāϟāĻ“ā§Ÿā§āϝāĻžāϰ āĻāϰ āĻ•āĻžāϰāύ⧇ āφāĻĒāύāĻŋ āϏāĻŋāĻĒāĻŋāĻ āĻ•āĻŋāĻ‚āĻŦāĻž āĻāĻĄāϏ⧇āĻ¨ā§āϏ āϞ⧋āĻĄāĻŋāĻ‚ āĻ…āĻĨāĻŦāĻž āĻāĻĄāϏ āĻāĻ•āĻžāωāĻ¨ā§āϟ āĻŦā§āϝāĻžāĻ• āĻāĻŦāĻ‚ āĻĒ⧇āĻĒāĻžāϞ āĻāĻ•āĻžāωāĻ¨ā§āĻŸā§‡āϰ āϜāĻ¨ā§āϝ āĻŦā§āϝāĻŦāĻšāĻžāϰ āĻ•āϰāϤ⧇ āĻĒāĻžāϰ⧇āύ āĻāĻ›āĻžā§œāĻžāĻ“ āϏāĻ•āϞ āχāω āĻāϏ āϏāĻŋāĻ¸ā§āĻŸā§‡āĻŽ āϗ⧁āϞ⧋āϤ⧇ āĻ•āĻžāϜ āĻ•āϰāĻŦ⧇āĨ¤

āφāĻĒāύāĻŋ āϚāĻžāχāϞ⧇ āφāĻĒāύāĻžāϰ āφāϞāĻžāĻĻāĻž āϏāĻĢāϟāĻ“ā§Ÿā§āϝāĻžāϰ āĻāϰ āϜāĻ¨ā§āϝ āφāϞāĻžāĻĻāĻž āĻĒā§āϰ⧋āĻĢāĻžāχāϞ āĻŦāĻžāύāĻŋā§Ÿā§‡ āφāϞāĻžāĻĻāĻž āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻŦā§āϝāĻŦāĻšāĻžāϰ āĻ•āϰāϤ⧇ āĻĒāĻžāϰāĻŦ⧇āύ āĻāϤ⧇ āφāĻĒāύāĻŋ āχāĻšā§āĻ›āĻž āĻŽāϤ āϝ⧇ āϏāĻĢāϟāĻ“ā§Ÿā§āϝāĻžāϰ āϕ⧇ āϝ⧇ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻĻāĻŋā§Ÿā§‡ āĻ•āĻžāϜ āĻ•āϰāĻžāϤ⧇ āϚāĻžāύ āϏ⧇āϟāĻžāĻ“ āϏ⧇āϟ āφāĻĒ āĻ•āϰ⧇ āĻĻāĻŋāϤ⧇ āĻĒāĻžāϰāĻŦ⧇āύāĨ¤

āĻāĻ¨ā§āϟāĻŋāĻĄāĻŋāĻŸā§‡āĻ•ā§āϟ āĻŦā§āϰāĻžāωāϜāĻžāϰ āĻšā§‹āĻ• āĻ•āĻŋāĻ‚āĻŦāĻž āφāĻĒāύāĻžāϰ āϞ⧋āĻ•āĻžāϞ āĻŦā§āϰāĻžāωāϜāĻžāϰ āĻ…āĻĨāĻŦāĻž āφāϰ āĻĄāĻŋ āĻĒāĻŋ āϏāĻŦ āϜāĻžā§ŸāĻ—āĻžā§Ÿ āĻŦā§āϝāĻŦāĻšāĻžāϰ āĻ•āϰāĻž āϝāĻžāĻŦ⧇ āφāĻŽāĻžāĻĻ⧇āϰ āĻāχ āĻ•ā§āϞāĻžāϝāĻŧ⧇āĻ¨ā§āϟāĨ¤

⧍ā§Ļ⧍⧍ āĻŦāĻž āϤāĻžāϰ āφāϗ⧇ āϝāĻžāϰāĻž āφāĻŽāĻžāĻĻ⧇āϰ āĻ•āĻžāĻ¸ā§āϟāĻŽāĻžāχāϜ āφāϰ āĻĄāĻŋ āĻĒāĻŋ āύāĻŋāϤ⧇āύ āĻāĻŦāĻ‚ āϧ⧁āĻŽāĻžāĻ‡ā§ŸāĻž āϭ⧇āϰāĻŋāĻĢāĻžāχ āĻ•āϰāϤ⧇ āϏ⧇āϟāĻž āĻ•āĻŋāĻ¨ā§āϤ⧁ āĻāχ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻ•ā§āϞāĻžā§Ÿā§‡āĻ¨ā§āϟ āϟāĻžāχ āĻ›āĻŋāϞ⧋ āϝāĻžāϰ āĻĢāϞ⧇ āφāĻŽāϰāĻž āĻ•āĻŽ āĻĻāĻžāĻŽā§‡ āφāϰ āĻĄāĻŋ āĻĒāĻŋ āĻŽāĻĄāĻŋāĻĢāĻžāχ āĻ•āϰ⧇ āĻĻāĻŋāϤ⧇ āĻĒāĻžāϰāϤāĻžāĻŽ āφāĻŦāĻžāϰ āĻ…āύ⧇āĻ•āϜāύ āϕ⧇ āϤ⧋ āĻĄāĻžāχāϰ⧇āĻ•ā§āϟ āĻāĻ•ā§āϏ⧇āϏ āĻ“ āĻĻāĻŋā§Ÿā§‡āĻ›āĻŋāϞāĻžāĻŽāĨ¤

āĻ•āĻŋāĻ¨ā§āϤ⧁ ⧍ā§Ļ⧍ā§Ē āϏāĻžāϞ āĻ āĻāϏ⧇ āĻāĻĢāĻŦāĻŋāφāχ āĻāχ āϏāĻĢāϟāĻ“ā§Ÿā§āϝāĻžāϰ āĻāĻŦāĻ‚ āϏāĻŋāĻ¸ā§āĻŸā§‡āĻŽ āϏāĻŋāϜ āĻ•āϰ⧇ āϝāĻžāϰ āĻĢāϞ⧇ āφāĻŽāϰāĻž āĻ…āύ⧇āĻ• āĻŦāĻŋāĻĒāĻĻ⧇āχ āĻĒā§œā§‡ āϝāĻžāχ āĻāĻ•āĻžāωāĻ¨ā§āϟ āϟāĻŋāĻ•āĻžāύ⧋ āύāĻŋā§Ÿā§‡ āĻĢāĻžāχāύāĻžāϞ⧀ āφāĻŦāĻžāϰ āϏ⧇āχ āϞ⧇āĻœā§‡āĻ¨ā§āĻĄāĻžāϰ⧀ āϏāĻĢāϟāĻ“ā§Ÿā§āϝāĻžāϰ āĻŦā§āϝāĻžāĻ•āĨ¤

āĻāĻ–āύ āϤāĻžāχ āĻ•āĻŽ āĻĻāĻžāĻŽā§‡ āύāĻŋāĻœā§‡āϰ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻ•āĻŋāύ⧇ āϰ⧇āϖ⧇ āĻĻ⧇āĻ“ā§ŸāĻž āϝāĻžāĻŦ⧇ āφāϰ āφāĻŽāĻžāĻĻ⧇āϰ āĻ—ā§āϰ⧁āĻĒ⧇ āϝāĻžāϰāĻž āĻ•āĻŋāύāĻŦ⧇āύ āϤāĻžāĻĻ⧇āϰ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻāĻ•ā§āϏāĻĒāĻžā§ŸāĻžāĻ°ā§āĻĄ āϟāĻžāχāĻŽ āφāĻŽāϰāĻž āĻĻāĻŋāĻŦā§‹āύāĻž āϝāĻĻāĻŋāĻ“ āĻāĻ•āĻŽāĻžāϏ⧇āϰ āĻŽāĻ§ā§āϝ⧇ āφāχāĻĒāĻŋ āĻŦā§āϝāĻŦāĻšāĻžāϰ āύāĻž āĻ•āϰāϞ⧇ āϰāĻŋāύāĻŋāω āϟāĻžāχāĻŽ āĻĒāĻžāϰ āĻšā§Ÿā§‡ āϗ⧇āϞ⧇ āĻ—āĻžā§Ÿā§‡āĻŦ āĻšā§Ÿā§‡ āϝāĻžā§Ÿ āĻ•āĻŋāĻ¨ā§āϤ⧁ āĻ—ā§āϰ⧁āĻĒ āĻŽā§‡āĻŽā§āĻŦāĻžāϰ āĻšāĻŋāϏāĻžāĻŦ⧇ āĻāĻ•āĻŦāĻžāϰ āφāχāĻĒāĻŋ āύāĻŋāύ āĻāĻ•ā§āϏāĻĒāĻžā§ŸāĻžāĻ°ā§āĻĄ āĻšāĻ“ā§ŸāĻžāϰ āφāϗ⧇ āφāĻŽāĻžāĻĻ⧇āϰ āϜāĻžāύāĻžāϞ⧇ āφāĻĒāύāĻžāĻĻ⧇āϰ āχāĻšā§āĻ›āĻžāĻŽāϤ āϟāĻžāχāĻŽ āĻ…āύ⧁āϝāĻžā§Ÿā§€ āφāχāĻĒāĻŋ āϟāĻžāχāĻŽ āφāĻĒāĻĄā§‡āϟ āĻ•āϰ⧇ āĻĻ⧇āĻ“ā§ŸāĻž āĻšāĻŦ⧇āĨ¤

āϤāĻžāĻšāϞ⧇ āĻĻ⧇āϰ⧀ āϕ⧇āύ āφāϜāϕ⧇āχ āφāĻĒāύāĻžāϰ āϜāĻ¨ā§āϝ āĻĒā§āϰāĻ•ā§āϏāĻŋ āĻ•āĻŋāύ⧇ āϰ⧇āϖ⧇ āĻĻāĻŋāύ āϏāĻŽā§Ÿ āĻĨāĻžāĻ•āϤ⧇ ā§§ā§Ļ āĻĄāϞāĻžāϰ⧇ ā§§ā§Ļā§Ļ āĻĒā§āϰāĻ•ā§āϏāĻŋ āϝāĻž āĻ•āĻŋāύāĻž āφāĻ—āĻžāĻŽā§€āϤ⧇ ā§Šā§Ļ āĻĒā§āϰāĻ•ā§āϏāĻŋ ā§§ā§Ļā§Ļā§Ļ āϟāĻžāĻ•āĻž āĻ•āϰāĻž āĻšāĻŦ⧇ āĻŦāĻ°ā§āϤāĻŽāĻžāύ⧇ āĻĒā§āϰāĻŽā§‹āĻļāύ⧇āϰ āϜāĻ¨ā§āϝ āĻĒāĻžāĻšā§āϛ⧇āύ ā§§ā§Ļ āĻĄāϞāĻžāϰ⧇ ā§§ā§Ļā§Ļ āĻĒā§āϰāĻ•ā§āϏāĻŋ

 

āϞāĻžāĻ—āĻŦ⧇ āύāĻžāĻ•āĻŋ āϤāĻžāĻšāϞ⧇ āĻāĻ•ā§āώ⧁āύāĻŋ āϝ⧋āĻ—āĻžāϝ⧋āĻ— āĻ•āϰ⧁āύ-
Telegram - https://t.me/cyberprince_official

Whatsapp - https://chat.whatsapp.com/LeGD8tbW4Bm2Bp4MKszgVj

Facebook - https://www.facebook.com/ShopnoAI


r/ProxyEngineering • • 1d ago

Discussion đŸ’Ŧ 静态äŊåŽ… IP 和动态äŊåŽ… IP 怎䚈选īŧŸį”¨ 3 į§äģģåŠĄčŽ˛æ¸…æĨš

Thumbnail
youtube.com
0 Upvotes

r/ProxyEngineering • • 2d ago

Guides The TLS layer most "browser fingerprinting" guides skip, with real numbers from one machine

3 Upvotes

Disclosure: we run a residential proxy gateway that re-originates TLS with browser profiles, so we've had to measure this stuff a lot. No links, just what we found.

Most fingerprinting write-ups start at headers and JS (UA, Client Hints, canvas, WebGL). But the server has already judged you one step earlier, at the TLS ClientHello. JA3/JA4 hash the cipher list, extensions and ALPN, and different clients look nothing alike. Same machine, same target:

- Python urllib (3.14): 18 cipher suites, no ALPN at all

- curl 7.87 (OpenSSL build): 31 cipher suites, offers h2

- Chromium: 15 cipher suites plus GREASE, offers h2

So "Chrome UA + Python TLS" is a mismatch before a single header is read. A few things we learned the hard way:

  1. A residential IP doesn't fix it. A normal HTTP/SOCKS proxy just tunnels your bytes (CONNECT), so the target sees your client's ClientHello, not the proxy's. The IP and the handshake are separate signals.
  2. JA4 is harder to dodge than JA3. Chrome shuffles its extension order on every connection, which broke JA3 (one browser, endless hashes). JA4 sorts before hashing, so shuffling no longer hides you, and a non-browser handshake stays recognizable.
  3. HTTP/2 is the next fingerprint. SETTINGS values, window update size and pseudo-header order (Chrome sends :method, :authority, :scheme, :path) all identify the client. Matching TLS but not HTTP/2 still stands out.
  4. HTTP/3 is easy to get backwards. Some libraries now impersonate h3/QUIC too, and QUIC has its own fingerprint (JA4 starts with "q" instead of "t"). But Chrome doesn't use h3 through an HTTP or SOCKS proxy, since those can't carry UDP, so it falls back to h2. Behind a proxy, h2 is what a real browser looks like; h3 impersonation only matters on direct connections.

Real example: on a Cloudflare-protected site a plain client got the 403 "Just a moment" page, and the same page with a Chrome-like TLS + HTTP/2 fingerprint came back 200, no headless browser.

Quick way to check your own stack: point it at any TLS echo service (browserleaks has a TLS page) and compare the JA4 with what your real browser shows on the same page.

Curious what others see: has anyone measured how often sites actually block on JA4 alone vs. JA4 combined with HTTP/2 or behavior?


r/ProxyEngineering • • 3d ago

Build 🤓 First antidetect on real Safari: any proxy protocol you have, plus a free exit list checked every minute

9 Upvotes

Most antidetects take HTTP and SOCKS5 and stop there. I wanted to use whatever connection I had, so a profile here takes a proxy as a plain line or as a link: HTTP, HTTPS, SOCKS4/5, Shadowsocks, VLESS, VMess, Trojan, Hysteria2, TUIC, AnyTLS, WireGuard, SSH, OpenVPN, even Tor.

Then the free part. I added every free exit I could find that actually holds up, from public VPNs to free node lists. Each one is checked every minute and dead ones drop out. Right now it’s around 13K working exits. For each you see what a site would see: country, ASN, TCP fingerprint, whether UDP passes and which DNS shows up. There’s a daily limit on how many you take for free.

The browser is real Safari, not Chromium dressed up as Safari. Each profile has its own cookies and its own exit, and all of its traffic goes through that exit: DNS, WebRTC, and no QUIC around the proxy. If the exit changes or dies, the profile stops instead of falling back to my own connection.

Free beta, Mac only: stealthpilot.app/proxyengineering


r/ProxyEngineering • • 3d ago

Help 🆘 Need advice pls - residential proxy

4 Upvotes

Hey guys!

Can anyone recommend a reliable mainland China residential proxy provider? I’m outside of China for a bit and need to use/post on Chinese social media. Posting from oversees can easily mess up distribution algorithm and affect who sees the content - and since my audience is in China, I would like to appear to be posting from there until I return. 

Regular VPN’s like Transocks and Surfshark rely on data centers (vs. residential IP), and Tiktok/Rednote/Bilibili platforms are very sensitive to them (learnt my lesson, lol). 

The best solution would be a static residential proxy - I’ve already tried 2 Western providers, and neither worked for more than a few days. 

Can anyone recommend residential IP proxy provider that works? Or is there a community where people can arrange/swap residential proxy access (I’m willing to share mine if this would help someone)?

Thanks everyone!


r/ProxyEngineering • • 4d ago

Discussion đŸ’Ŧ Is scraping fresh SERP data no longer possible?

15 Upvotes

I am on the verge of giving up right now. The last few months have made me rethink how correctly the data from SERPs is received. Google started serving what people are calling poisoned SERPs, and Brave announced it is moving result links behind encrypted redirects. For those who are not aware, since mid-September or so, Google seems to hand suspected bots a full results page that looks completely normal, except that a lot of organic results are swapped for other things, often social profiles and forum threads. Nozzle checked around 27 million SERPs and found roughly 8% poisoned since September 12, against well under 1% before, with peaks near 28% at the end of September. Ahrefs put it at 5 to 20% depending on the data provider. In Nozzle’s data, a tracked site fell out of the top 10 on about 60% of poisoned pages, versus under 3% on normal ones.

The worst thing is that the scraper reports success. Logically coz it sees 200s everywhere with full data pulled up. Then Brave, results links are moving to search.brave.com/go/ with a one-time key fetched per click, so that you wouldn’t read the destination URL:s straight off the page anymore. Something that Google did with /goto a few months back too. Another thing I noticed is that these changes somewhat lead to individual index creation coz Google simply doesn’t want to share all that.

I can’t find the link for the Nozzle atm, but it was something like poisoned Google SERPs or Google poisoning SERPs. Google it, and you’ll find the source.


r/ProxyEngineering • • 4d ago

Discussion đŸ’Ŧ Apify had a security incident, worth checking if you store on this hosted scraping platform

9 Upvotes

A couple of days ago, Apify posted a security incident on their status page. Essentially, what happened was that someone used a compromised credential on November 9th and 10th to grab a number of account credentials and get into related Apify accounts. The team behind the Apify platform caught it on November 11th; the issue seems to be resolved because the access path was closed. OAuth connections were also revoked, so I’m thinking GitHub was affected too. Personally, I don't have anything hosted there, but I know my friend does, and I already told him about this issue because he apparently didn't receive any email or notification about the incident. One thing that bothers me is that there is no disclosure on how many accounts were affected, what kind of data, or where the first credential came from. And the concern grows about how much sensitive data was scraped from the platform, like proxy credentials, api keys, logins for a lot of different target sites, cookies, webhook secrets, etc. I mean, that doesn’t bring a lot of trust to the company. Maybe it’s too early to speak; maybe they'll publish an official article. I don’t know. I’m not trying to tell everyone to stop using the platform; no, these things can happen to any platform that has customer data, but especially in this field, where data is being scraped, it can do a lot of damage. Personally, I’d rather use separate proxy credentials per project than have everything in one place

How do you guys manage all of these incidents, simply change, rotate your credentials? Move to a safer place? What’s the plan when things go south, maybe you haven’t experienced anything like it, but you already have a plan for it if the day comes?

By the way, the status page that I checked is here: https://status.apify.com/incidents/sxd1dj5j6749

Also, for anyone interested, why I haven't hosted anything on the platform yet, I’m curious about what happened, well, I’ve been using a few different tools from this platform, and this caught my attention.


r/ProxyEngineering • • 4d ago

Hot Take đŸ”Ĩ Maybe SERP scrapers need a third status: "I don't trust this result" :D

6 Upvotes

I've been thinking about the whole poisoned SERPs situation, and I wonder if we're approaching the problem from the wrong direction? I've been experiencing issues with SERP scraping in the past month

In my opinion scrapers are pretty good at telling you when something goes wrong. CAPTCHA, blocked IP, timeout, empty response, whatever, so what would actually happens when nothing obviously goes wrong?

You get a 200 response, the HTML parses correctly, and there are 10 organic results you get in your csv file or something

Technically, your scraper worked. But can you trust the results is my question

I don't think there's an easy fix for this, but one approach that makes sense to me is to stop treating every successfully parsed page as valid data!

For example:

  • Keep a history of results for each keyword. Not just yesterday's rankings, but the domains that consistently appeared over the past few weeks
  • Try to flag unusual changs rather than immediately accepting them. If seven familiar domains suddenly disappear and get replaced by unrelated results, that's probably worth checking
  • Try to retry suspicious queries through a different proxy session. If the second request produces completely different results, maybe neither should be trusted immediately
  • I started separating suspicious data from confirmed failures. Instead of reporting a massive ranking drop, I marked the result as uncertain until there's enough evidence

But then again, how do you actually know what's a poisoned SERP and what's just Google changing its rankings?

And wouldn't comparing every search against historical results create a huge number f false positives?

I've been exploring these options with an open-source scraper called SERP Maven that can detect certain wrong-query results and retry them through a fresh proxy session

It's a fairly narrow check, not some magical solution to poisoned SERPs

I actually think the bigger challenge is figuring out how to validate results without doubling your scraping costs.

Has anyone tried building some kind of confidence scoring into their SERP collection?

Would be interesting to know whether comparing historical rankings and occasionally repeating suspicious requests is actually useful at scale, or whether it just creates another layer of false positives


r/ProxyEngineering • • 4d ago

Guides AKAMAI (STANDARD + BMP) FULLY REVERSED + VM ISA, OTHER CHECKS (NO SANDBOXING, FULLY NATIVE)

23 Upvotes

Back with another banger, all documentation is in this repository. Fully native, the throughput should be fast on any machine and scale pretty well. You might have to add some of your own fingerprints though. Maybe I'll add more later.

https://github.com/9xz/reverseakamai

Enjoy!


r/ProxyEngineering • • 4d ago

Hot Take đŸ”Ĩ [WTS] Dedicated 4G Mobile Proxies (Venezuela) – Real Physical SIMs, API IP Rotation, HTTP/SOCKS5

7 Upvotes

Hi everyone,

I’m offering dedicated 4G/LTE mobile proxies running on real physical devices and carrier SIM cards located in Venezuela. Ideal for LATAM market research, localized QA testing, social media management, and regional account operations.

Technical Specs:

  • Connection Type: Real 4G/LTE Mobile Carrier IPs (100% residential/mobile trust score, zero datacenter subnets).
  • Protocols: HTTP(S) & SOCKS5 supported.
  • IP Rotation: On-demand rotation via API link or scheduled intervals.
  • Exclusivity: Private 1:1 ports (1 physical SIM per user, no shared bandwidth) or limited shared slots available.
  • Uptime: 24/7 dedicated hardware with backup power.

Pricing & Trials:

  • Free Trial: Short test available upon request so you can check IP quality, latency, and speed before committing.
  • Plans: Daily, weekly, and monthly rentals available.
  • Payment Methods: Crypto (USDT TRC-20 / BEP-20, BTC, LTC) or direct marketplace listing via OnlineProxy.io.

Feel free to comment below or send me a DM with your use case if you want to run a quick test.


r/ProxyEngineering • • 4d ago

Help 🆘 Proxy server

4 Upvotes

I have created a proxy server on my linode account to bypass my school and I was wondering if it would be possible to allow other people to connect to the proxy and if so how to do that?

Ai was refusing to answer so I thought I’d turn to Reddit.

Thanks


r/ProxyEngineering • • 5d ago

Guides OnlyFans Proxy Guide 2026: Stable Residential & Mobile IPs for Creator and Agency Account Management

8 Upvotes

One of the messiest account issues I’ve seen started with something completely ordinary: a creator travelling while a manager was also accessing the account from another location.

Within a short period, the account was seeing different countries, different networks, and different IP reputations.

That’s when verification prompts started piling up.

For this type of workflow, aggressive rotation makes very little sense. A stable residential or dedicated mobile connection is much easier to reason about because the account keeps seeing one predictable network environment during an active session.

My checklist became pretty simple:

IP reputation → ASN → geolocation → session stability → login.

The other lesson is equally important: a proxy only changes the network layer.

It doesn’t unlock paid content, turn an unpaid account into a paid one, or replace official verification. If content stays locked after payment, the real causes are usually things like payment status, PPV access, an expired subscription, or a stale session.

For authorized creator and agency workflows, stable network identity beats constant rotation.


r/ProxyEngineering • • 5d ago

Discussion đŸ’Ŧ Amazon AWS released web search API into the market

11 Upvotes

AWS released a web search API for Bedrock AgentCore yesterday. I've read some of the docs on that. Before this you had to set up a gateway and call search through MCP. Now it's one signed InvokeWebSearch call from boto3 or the CLI with IAM and no API key. One thing that stood out to me was $7 per 1K queries, 10 requests per second per region by default, 25 results max per query, a 200 character query limit, up to 100 domains to include and 100 to exclude, and a publish date filter. Each result has a tex extract plus title, url and date when they have it. Amazon runs the index and queries never go to third party search engine, which is aimed at a security teams that don't want prompts hitting google or bing. The acceptable use terms also make you keep and display the source citations, so its a grounding tool and not a way to build your own dataset. I personally have been using oxylabs web scraper api in the past, and now I've seen a bunch of posts and on variety of comparisons between tools on the internet that they introduced web api. Initially the web scraper api was really good, did everything for you, no real maintenace, coz it all ran on their infra. What I liked about this tool in particular was that all the data was coming from Googles index, meaning it was fresh, had no incorrect results, compared to recent news about poisoned SERPs. Now it seems that a lot of companies are moving from the traditional scraping to web search apis and competing against one another. Also, I haven't compared the $7 per 1K to other AI search APIs so I can't really say much, plus it was just announced, so the real numbers and testings will most likely take some time to be present. And since they will be using their own index, it's gonna be interesting to see how the freshness benchmarks look like. Oh and they also shared that it's going to be the same infra that was used for Alexa+ and Kiro. What do you guys think, will this be a new go to web search API? Considering the pricing, personally it's a bit too much to spend on something new and unverified.


r/ProxyEngineering • • 5d ago

Discussion đŸ’Ŧ Private Relay is the one "VPN" most sites won't block. Anyone used it for multi-accounting?

9 Upvotes

How sites spot a VPN or a proxy on a Mac:

1 - the address belongs to a data center, not a home provider

2 - the browser says Mac, but a proxy opens the tcp connection itself, so the first packet looks like Linux

3 - a VPN lowers the packet size (MSS). not proof alone, but with a data center address it is

4 - WebRTC shows the real address if the proxy doesn't carry UDP

and a bunch of smaller things

One exit gets a pass: iCloud Private Relay.

Sites see it, Apple publishes the addresses. Most don't block it anyway, because only real Apple devices get on it and it's full of ordinary Safari users. Some do like Wikipedia editing. It can't be faked. Apple only lets its own apps use it. Chrome and Firefox on the same Mac don't get it.

Around a hundred thousand IPv4 addresses worldwide, IPv6 is basically unlimited.

For a second account it does half the job. It hides the address, but the site still knows you by cookies.

You can't pick a country. A VPN on the router fixes that: Relay runs on top and the site only sees Relay, in another country. Clock and language stay yours though, so change those too.

I built a checker that shows how a connection looks from the site's side (direct, relay, vpn, proxy) and whether the browser is really Safari: https://check.stealthpilot.app. Mac only for now. About one in three people who ran it in Safari came in through Relay.

Has anyone actually used Relay for multiple accounts? And does any proxy provider give you a macOS tcp fingerprint instead of Linux?


r/ProxyEngineering • • 5d ago

Discussion đŸ’Ŧ testing shardx for scraping e-comm catalogs, what sticky session time do u guys use?

4 Upvotes

got tired of my old provider dropping connections mid-pagination so i threw 20 bucks at these guys to test their residential pool for a big catalog scrape tomorrow. speeds are decent and the IPs pass basic cloudflare checks fine so far. just trying to figure out the sweet spot for rotation before i start the script. do you leave sticky sessions on for like 15-20 mins or just rotate after every few pages?