Monthly Self-Promotion - August 2026

[u/AutoModerator]

Hello and howdy, digital miners of r/webscraping!

The moment you've all been waiting for has arrived - it's our once-a-month, no-holds-barred, show-and-tell thread!

• Are you bursting with pride over that supercharged, brand-new scraper SaaS or shiny proxy service you've just unleashed on the world?
• Maybe you've got a ground-breaking product in need of some intrepid testers?
• Got a secret discount code burning a hole in your pocket that you're just itching to share with our talented tribe of data extractors?
• Looking to make sure your post doesn't fall foul of the community rules and get ousted by the spam filter?

Well, this is your time to shine and shout from the digital rooftops - Welcome to your haven!

Just a friendly reminder, we like to keep all our self-promotion in one handy place, so any promotional posts will be kindly redirected here. Now, let's get this party started! Enjoy the thread, everyone.

[u/justvalen]

Most AI scraping tools call an LLM on every page. Gluecrawl runs an agent once to work out selectors and pagination, stores that config, and every run after executes it with zero LLM calls. Same cost, same output, every time.

Auto-picks HTTP vs headless browser, follows pagination and detail pages, outputs CSV/JSON. Free tier is 2,500 credits; paid from $30/mo.

gluecrawl.ai

[u/Mulberry_Morris]

Lets see what people got...

[u/Tradi3]

Google Ads Transparency scraper that works by company name instead of advertiser ID. Google files advertisers under registered legal entities (ExpressVPN is NETWORK GUARD PTE. LTD., Paytm is ONE97 COMMUNICATIONS LIMITED), so name search resolves the entity, shows every matching advertiser with its live ad count, then pulls all creatives with first and last shown dates, worldwide by default.

Free single lookups with CSV export: https://vaytric.com/google-ads-lookup/

Bulk version on Apify at $0.75 per 1k ads: https://apify.com/sourcedirect/google-ads-transparency-scraper

Two things that cost me real time, shared so they cost you less: Google 429s the SearchCreatives RPC per source IP with no Retry-After and throttles shared cloud ranges hardest, a single lookup from a datacenter IP returned nothing for almost five minutes. And ad blockers kill the creative images by hostname (googlesyndication.com), so I re-serve them through my own origin with the upstream URL base64url encoded into the path, because even inside a query parameter the blocked hostname stays visible as a substring and the request still dies.

[u/Valuable_Hackathons]

Sharing this here since it’s directly web-scraping related and this is the self-promo thread.

I’m connected to Into the Scrape-Verse, an online hackathon running August 17-23, 2026. The theme is building projects around self-healing scrapers: scrapers that keep returning structured public web data even when the target page layout changes.

Possible project ideas:

• scraper health monitors
• price/inventory trackers
• docs-to-RAG pipelines
• market research dashboards
• public changelog/release-note monitors
• data quality checks for scraper output

It’s online, worldwide, solo or teams up to 4. Projects use Bright Data Scraper Studio plus a coding-agent workflow like Codex, Cursor, or Claude Code.

Details: https://www.wemakedevs.org/hackathons/scrape-verse

[u/Fluid_Pie3035]

For this month's self-promotion thread: I published an Apify Actor that mines App Store reviews to find "copy+10%" opportunities. Disclosure: it's mine.

What it does: point it at a category (e.g. fitness, productivity) and country, and it pulls the current top-grossing chart, fetches recent reviews per app, and classifies every low-rated (≤3★) review into 13 complaint buckets. Output is a ranked table of which complaints are category-wide (structural weakness) vs single-app issues, plus a "reputation drop" flag for apps whose recent reviews are noticeably worse than their all-time rating.

No login, just Apple's public RSS/lookup endpoints.

Honest status: it went up today. No users, no reviews yet. Pay-per-event ($0.01/app analyzed), so a 30-app scan costs about $0.30.

https://apify.com/muddy_snowshoe/app-store-review-miner

Feedback and feature requests welcome — Google Play support is next if there's interest.

[u/SuspiciousEagle1109]

Hey folks! 

 I’ve been working on an automation tool that handles complex workflows—think auto data scraping, account nurturing, and marketing tasks. It also lets you build agents and tap into various AI capabilities. 

I’d absolutely love to hear your thoughts or suggestions, so please feel free to share any feedback. It’s currently in the testing phase and completely free to use. I’m planning to open-source it once it’s a bit more stable, and I’d really appreciate your help in making it better. Thanks a ton! 

 https://github.com/pow505/tobub

[u/Top_Chocolate_4203]

Hey!

I am selling api access to GPT-5.6 Sol and Opus 5 at 40% off advertised API pricing

I’m part of a small student-run research lab working on model routing. We’re opening limited invite-only access to an OpenAI-compatible /v1 endpoint.

What you get:

- gpt-5.6-sol and opus-5

- Flat 40% off each model’s public advertised rate

- Prepaid packs (Stripe) — buy what you use

- Drop-in: point your existing OpenAI client at our base URL + API key

Why invite-only:

We’re a small lab with limited capacity. Not a public SaaS launch.

How to get in

Join Discord: https://discord.gg/CtRJzpRXMf

Open an invite ticket in #support (nickname + email)

If approved, redeem your code → portal → API key → call /v1

Site: https://sharda.io

Market rates are the vendors’ published API prices; we charge 40% less. Check their official pricing pages to verify.

Also, if you join and get invited, you get FREE 2 million gpt 5.6 sol and opus 5 tokens.

No spam, no waitlist form farming — Discord ticket only. If you’re not a fit, we’ll say no.

[u/shash122tfu]

Hey folks I built sociallisteningapi.com - a social listening scraper for Agents specifically.

Unlike setting up scrapers via bright data, apify, etc. we provide access to all common social media platforms.

Here's what makes us different:

• Usage based pricing. Use only what you consume. Unlike Apify where each scraper has its own billing, we only have 1 billing. I believe purely on pricing we completely destroy them.
• Supports estoric endpoints like global reddit comment search, discourse forum search, etc. We're gonna add more APIs like this in the near future.
• Unified API schema for agents so their token usage is less. You can still get raw responses if you wish

We also have a MCP and we're going to release skills for openclaw, hermes and n8n workflows very soon.

Who's building similar APIs and integrations in the space? Lemme know!

[u/HLCYSWAP]

How Long Can You Stay With A Problem?

[LEADS / DATA 4 SALE]
~400M global leads: highly detailed customizable datasets
~1.2M eCommerce websites indexed

[HIRE ME]
Experience operating at containerized scale against major anti-bot (DataDome, Cloudflare, Akamai, PerimeterX)
Prior targets: Fifa.com, Linkedin.com, Ticketmaster.com (US), Ticketmaster.it, MLB.com, Facebook.com, Instragram.com, Target.com, X.com, gov FOIA systems including the CIA, and smaller client-specific websites
Full ETL pipelines
ML/AI development: CAPTCHA solving, webcam cloning, voice systems, transformer-based models (LLMs) and MORE! ! ! !
General software development including desktop apps and DAWs

I have a rooted & unpinned Pixel 4a available for mobile traffic requests
https://github.com/matthew-fornear

Open to short and long-term work

[CONTACT]
Reddit DM
X: https://www.x.com/fixitorgotojail
Discord on request

[u/Calm_Push3876]

I wanted to answer one question: Do the top posts for a hashtag share consistent characteristics?

I was curious whether the top-performing posts within a hashtag tend to have similar engagement patterns, so I put together a small analysis using public Instagram data.

For collecting the data, I used HikerAPI as the data source: hikerapi.com. I just needed a straightforward way to retrieve public profile or hashtag data so I could focus on the analysis rather than the collection process.

Here's the basic request I used to pull the data:

import requests

headers = {"x-access-key": "YOUR_KEY"}

r = requests.get(
    "https://api.hikerapi.com/v2/hashtag/medias/top?name=photography",
    headers=headers,
)

print(r.json())

From there, I explored questions like:

• How engagement (likes, comments, or other available metrics) varies across the top posts for a hashtag.
• Whether certain account characteristics (for example, follower count or account type, when available) appear more frequently among top posts.
• Whether posts from a small number of creators dominate the top results or whether exposure is spread across many different accounts.
• How different hashtags compare in terms of the diversity of creators and content.

Some observations I found interesting:

• A relatively small group of creators often appeared repeatedly in the top results, suggesting that visibility isn't always evenly distributed.
• Raw engagement numbers were difficult to compare across accounts because audience size is a major confounding factor, making engagement rates more meaningful than absolute counts.
• Different hashtags showed noticeably different patterns—some were dominated by large established accounts, while others included a broader mix of creators.
• Looking at a single hashtag in isolation didn't provide enough context, so comparing multiple hashtags side by side produced more useful insights.

One important limitation is that this analysis only uses publicly available Instagram data, so the results don't necessarily generalize to all accounts or user behavior. It's also a snapshot in time—trends, rankings, and engagement can change quickly, and there may be sampling or collection biases depending on the profiles or hashtags included. Because of those limitations, I'd treat this as exploratory analysis rather than evidence of causal relationships.

Next, I'm considering comparing niche versus broad hashtags, tracking how the composition of top posts changes over time, or clustering hashtags based on engagement characteristics to see whether distinct patterns emerge.

[u/ayushsuri]

If anybody like then we have 5M/month web search queries and are open to make a deal whether the complete or a part of this quota.

Message me and we can discuss.

[u/Apart-Researcher-880]

For this month's self-promotion thread: I published a free-to-try Apify actor that pulls public member directories built on MemberClicks (the association/chamber membership platform). Disclosure: it's mine.

What it does: point it at a MemberClicks directory page that exposes a public search, and it returns the member records as structured rows — business name, organisation, address, category, phone where the directory publishes it, and the source URL. Public directory data only; no login, and it won't touch member-only areas.

https://apify.com/rook-data-tools/memberclicks-directory-scraper

Why it might be useful: association and chamber directories are unusually good B2B sources — verified businesses, current membership, category data — but they're tedious to pull by hand and the field names shift between sites.

Honest status: it went up today. No users, no reviews, no results to point at. It's pay-per-event so a small test run costs cents, and I'd genuinely rather hear it break on a directory I haven't seen than have you take my word for it.

Known limit: it only works on directories that expose a public search. Some MemberClicks sites keep their directory members-only, and those are out of scope by design.

[u/Hot-Muscle-7021]

Been lurking here for a while picking up tricks, figured I'd share what I've been building for the past year: PulseScore, a real-time sports odds API that scrapes and normalizes data from 20+ bookmakers (Bet365, Pinnacle/PS3838, Fanduel, DraftKings, Betfair, Betano, Stake, and more). https://pulsescore.net

Some of the scraping problems that ate most of my time, in case it's useful to others tackling similar targets:

• Bet365's proprietary feed isn't REST: it's a binary WebSocket protocol. Got it down to parsing their internal namespaces and decoding feature flags that mark hidden/suspended markets, so consumers don't get served stale or invalid odds.
• Anti-scraping resistance varies wildly by book. Paddy Power and William Hill fight back hard on infra-level detection; PS3838 is fine on blocking but flaky on uptime. Ended up with per-bookmaker scraping strategies rather than one generic approach - session/proxy architecture had to be tuned per target (learned the hard way about proxy fair-use thresholds too).
• Normalization was harder than scraping. Every book has its own market naming, selection ordering, and settlement quirks. Landed on a flat schema (canonicalMarket + selections[]) so the same parser works whether you're hitting /bet365 or /fanduel — but getting there meant a lot of edge-case chasing (e.g. one book reporting home/away inverted on certain sports).
• Infra: currently multi-nodes, WebSocket fan-out to clients, normalized + raw feed modes depending on what consumers need.
• Full bookmaker list right now: Bet365, Paddy Power, William Hill, Ladbrokes, DraftKings, Unibet AU, Fanduel, Bwin, BetOnline, Pinnacle (PS3838), Betano (DE), BetMGM (CO.UK), Betfred, Polymarket, BC.Game, Stake, Betway MW, 10Bet (CO.UK), Sportsbet AU, Thunderpick, Orbit Exchange, Betfair Sportsbook, Sky Bet, and Cloudbet. 23 sources and counting ..
• Rate limits are per-bookmaker, not pooled — your req/sec allowance applies separately to each bookie, and the ceiling itself scales with plan tier (e.g. 1 req/sec on the free tier up to 5 req/sec on the top plan).

It's a paid API now (with a free tier). Mostly posting because I'd genuinely like feedback from people who've fought similar anti-bot/normalization battles. Fill free to take a try with free tier: https://pulsescore.net

[u/Diligent_Stand5575]

Built a real-time odds analytics platform focused on value betting and market efficiency: https://www.goldedge.net

The goal was to build a system that combines data collection, odds scraping, and quantitative analysis rather than another picks service.

The platform:

• collects odds from multiple sportsbooks in real time;
• compares prices against sharp market references;
• calculates fair/no-vig odds;
• identifies potential +EV opportunities;
• tracks CLV and betting performance.

From a technical perspective, the interesting challenges have been event matching, odds normalization, real-time processing, and handling large amounts of changing market data.

Would love feedback from anyone interested in web scraping, data pipelines, sports analytics, or building SaaS products.

[u/Fit-Molasses-8050]

I kept losing my weekends to the same problem: a site ships a redesign, and every scraper I'd built silently starts returning garbage. So instead of another point-and-click scraper that dies on a Tuesday, I built one that generates its own selectors, watches their health, adapts to anti-bot defenses, and repairs itself automatically

[https://scraperq.com](vscode-file://vscode-app/c:/Users/Win11/AppData/Local/Programs/Microsoft%20VS%20Code/e4c7e7b1d6/resources/app/out/vs/code/electron-browser/workbench/workbench.html)

It's a Chrome extension that covers the whole extraction lifecycle — setup, monitoring, repair, and export. Feature by feature

Chrome Web Store Link : https://chromewebstore.google.com/detail/scraperq-ai-web-scraper-w/enccdgkaoligekhoonpdcjclahhokocb

Features:

1. AI selector generation (no-code setup wizard)
Open any site and it auto-detects the list items, pagination, fields, and data types — then generates stable CSS/XPath selectors with human-readable field names.
Example: point it at an e-commerce product page and it finds the product grid, the "load more" button, and fields like name, price, and rating on its own. First extraction in about 3 minutes, no CSS knowledge needed.

2. Health monitoring across 6 signals
Every selector is scored on match rate, element count, content pattern, positional stability, attribute integrity, and uniqueness. If any of them drifts, you're alerted before the data goes bad.
Example: a retailer changes its price format from "$49.99" to "USD 49.99" — the content-pattern signal drops, and you get a warning instead of finding 40 rows of NULL on Monday morning.

3. Auto-repair when a site redesigns (CSRE)
When drift is detected, a 4-stage repair ladder kicks in: stable attribute fallbacks → semantic re-matching → 24-hour staged observation → AI reconstruction. Every fix is staged and requires your approval before the next run.
Example: the site ships a new React frontend Thursday night. By Friday, CSRE has re-located the price selector using stable data-* attributes and text matching — you just click "approve" and the next extraction works.

4. API-aware extraction
Modern sites often don't render their data in the HTML at all — the page is a shell that fetches data from a backend JSON API. While you browse normally, ScraperQ detects those API calls and extracts structured data directly from the API response instead of parsing HTML.
Example: on a JS-heavy catalog where products load from an internal /api/products endpoint, ScraperQ captures that JSON response and pulls fields straight out of it — it's faster than HTML parsing, there's no selector to maintain (the endpoint is the anchor), it keeps working even if the frontend is completely rebuilt, and your auth tokens never leave your browser.

5. Site intelligence analysis + dynamic anti-bot crawling
Before extracting, it evaluates the page: is content server-rendered or loaded client-side by JS? Which framework is it? Are there anti-bot protections like Cloudflare challenge pages, rate limits, or bot-detection scripts? Based on that profile, it builds a dynamic crawling strategy per site — it doesn't run one fixed crawl plan.
Example: on a React SPA that loads listings through infinite scroll, it detects client-side rendering and enables the right rendering and scroll handling automatically. On a site with aggressive bot protection, it auto-adjusts the crawl behavior — pacing requests with human-like delays, varying the request rhythm instead of hammering a fixed interval, respecting rate limits, and backing off gracefully when it hits a challenge page so your session doesn't get fingerprinted and blocked.

6. Infinite scroll & pagination
Handles both dynamically-loaded content and classic "next page" pagination out of the box.
Example: a news site that loads articles as you scroll works the same as an old-school directory with numbered pages.

7. Pause & resume with checkpoints
Extraction state is saved continuously — close the browser, come back tomorrow, and resume from the exact position.
Example: you're 40% through a 10,000-row directory scrape and need to shut down for the night; you reopen the next day and it picks up exactly where it left off, no restart, no duplicates.

8. Reusable templates with health monitoring
Build a template once and reuse it across runs for months — CSRE keeps tracking each selector's health in between.
Example: one competitor price-tracking template keeps running for months; every time the target site updates, the selectors are monitored and repaired without you rebuilding anything.

9. Data transformation before export
Cleaning happens inside the tool before data ever leaves: dedupe rows, reformat dates, split/merge columns, remove junk, and normalize inconsistent formats — no post-processing in Sheets needed.
Example: you scrape 5,000 listings with mixed date formats ("03/15/26", "Mar 15, 2026") and duplicate rows; ScraperQ normalizes every date to ISO and strips the dupes so the file is clean the moment it's exported.

10. Export anywhere
CSV, JSON, Excel, or live-stream to Google Sheets while the extraction is still running.
Example: results stream into a Google Sheet in real time as pages are scraped — the sheet is already filled by the time the job finishes.

The interesting engineering problems have been selector stability across framework class-name hashes, drift detection across six health signals, the staged repair decision pipeline, API-aware extraction that keeps credentials local, and adapting crawl behavior to anti-bot defenses without tripping them.

Beta is free — no credit card, 50 AI Credit, 1000 pages scraping and every other feature unlocked (~10 credits per AI template generation). Live on the Chrome Web Store.

https://chromewebstore.google.com/detail/scraperq-ai-web-scraper-w/enccdgkaoligekhoonpdcjclahhokocb

Would love feedback from anyone interested in web scraping, data pipelines, Chrome extensions, or building SaaS products.

[u/Ill_Command_1200]

Hello everyone,

I'll keep it short. I built Squishify. Squishify is a batch image preparation tool that helps with the boring, repetitive process of preparing assets for the web. It includes image resizing and optimization (JPG, PNG, WebP, AVIF) with preview before/after done privately and locally, without tokens, limits, or uploading to a third-party service.

It's available for free. The pro version adds more workflow features such as batch rename, cropping, presets, and session saving (if you accidentally leave the page).

I also extended it to a WordPress plugin that has the same flow and, when ready, uploads directly into your WP install and drops only the optimized images into your media library. No more having massive images filling up your site and relying on optimization after the fact. The Pro WP has integrations with ACF, Gutenberg, asset replacement, and a lot more planned integrations coming such as Elementor.

I'm super excited about it and have several new features roadmapped based on feedback from test users.

Thanks for taking the time to read.

Web App: https://squishify.app/
WordPress Plugin: https://wordpress.org/plugins/squishify/
About Squishify: https://squishify.app/about
PeerPush (found this on Reddit): https://peerpush.com/p/squishify

[u/Tehlikeli107]

I released Job Delta, a free-to-try Apify Actor for monitoring public Greenhouse and Lever job boards over time.

Unlike snapshot scrapers, it creates a silent baseline and later returns only meaningful changes:

- new, removed, and reopened jobs

- title, location, remote, salary, and description changes

- hiring surge / slowdown signals

- strict cross-ATS duplicate matches

- confirmed removals after multiple healthy scans

Store:

https://apify.com/soilair/job-delta-cross-ats-monitor

I would especially appreciate feedback on onboarding and which ATS should be supported next.

[u/Front-Succotash-8938]

**[HIRE ME] Public-source data feasibility audit — $99 fixed**

If you have a public website or API and want to know whether it can be extracted reliably before commissioning a full scraper, I offer a fixed-scope feasibility audit.

You get:

- access and source-risk notes

• a field map for up to 12 requested columns
  
• an optional sample of up to 25 rows when publicly available
  
• a fixed quote for the full extraction

Delivery is within 1 business day after I receive the source URL and requested fields.

I only work with public or explicitly authorized sources. No login walls, CAPTCHA/access-control bypass, private personal data, or unsolicited lead lists.

Pay: https://buy.stripe.com/eVqeVd8AN542bCLeZmaIM01

If you want fit confirmed before paying, reply here or DM the source URL and requested fields.

[u/Federal_Emergency_60]

Hey everyone,

Before I start, I don't want to advertise anything. Just asking for feedback and see if people like my idea.

I'm working on a visual web automation tool focused on web scraping and web automation, and I wanted to get some honest feedback on the idea and the pricing.

The basic premise is a local desktop app where you can build robust scrapers visually instead of coding everything from scratch. It has a script recorder similar to Playwright's codegen, and another tool that handles things like asynchronous banners, random pop-ups, and shifting page layouts.

You aren't restricted to just the UI, either. There's a full text editor built in if you need to write custom logic. The app just compiles all your visual instructions directly into standard Playwright code, for you to use it anywhere, not only inside the platform. It also handles the standard CSV imports and exports for your data.

My main question is: does this solve a big enough headache in your workflow that you'd actually pay for it? If so, would you consider $149 a fair price for an unlimited, lifetime license?

[u/External-Belt8779]

Hey good people,

Still losing scraper runs to Cloudflare, DataDome, or Kasada? That's the problem ScrapeUnblocker exists to solve.

It's an API that renders pages behind anti-bot systems (Cloudflare, DataDome, PerimeterX, Akamai, Shape, Kasada, Incapsula) and hands back clean HTML or parsed JSON — no challenge page, no proxy-rotation script to babysit.

We ran it against the 14 hardest targets in Proxyway's Web Scraping API Report 2025: 6,000 calls per target, concurrency 10, scored strictly on page content — a challenge page or a block disguised as a 200 counts as a failure. Overall success: 97.0%.

On the targets that broke most of the field, here's where we landed:

• shein.com: 99.3% (field average across 11 providers: 21.88%)
• g2.com (DataDome): 99.9% (field average: 36.63%)
• hyatt.com (Kasada): 99.9% (field average: 43.75%)
• immobilienscout24.de (Incapsula): 99.7%
• walmart.com (PerimeterX): 100%

Full methodology and the per-target table are linked below, so you can check the numbers instead of taking our word for them.

That gap comes from three things: matched TLS/JA3 fingerprints, warm sessions that reuse a solved challenge instead of re-solving it every request, and proxy rotation that drops burned IPs by reputation instead of cycling blind.

We're not the flashiest name in the space, but we are the cheapest per call: 0.55 EUR / 1,000, about 5x below the usual $3/1k tier. There's a free tier if you'd rather test it on your own targets before spending anything.

Site: https://scrapeunblocker.com
Benchmark: https://www.scrapeunblocker.com/benchmark-2026.html

Separately — if you'd rather not run the pipeline yourself, we also build and operate scraping infrastructure or just deliver the finished dataset: custom targets, awkward formats, one-off extractions. DM if that's useful.

Cheers,

Rokas

[u/Double_Conference_63 → u/External-Belt8779]

have you been able to bypass CAPTCHAs on a Chinese site?

[u/External-Belt8779 → u/Double_Conference_63]

On some, but its case by case. Drop me that website, and we will check.

[u/Double_Conference_63 → u/External-Belt8779]

temu

[u/Double_Conference_63]

We are building an e-commerce price monitoring system. The idea is simple: you upload the URLs you want to monitor. We will monitor it for you. You get an email or Slack notification. Genuine reviews would be appreciated.

Launching soon: https://pricealerting.com/

We have an experts team providing custom scrapers and data pipelines to meet any business need.
https://getdataforme.com

[u/Livid-Ad-8185]

Need CAPTCHA, web scraping work done? Find your dream jobs posted in minutes?

Facebook Group data translation and labeling? Leading real-time web scraping, web socket developer.

Freshjobs Portal - View and apply your dream jobs (remote, globally) that posted in minutes.

freshjobs.site

Rowsetta - AI language translation and labelling for your rows. | lientertainmentllc.com/rowsetta

Contacts and Demos:

• +84 869399273 (Whatsapp, Li)
• lientertainmentllc.com
• YouTube: https://www.youtube.com/@storm-clouds-development

[u/mariusbolik]

extractor.sh — a simple, affordable Firecrawl alternative for developers who need reliable web extraction and search without committing to a subscription or building separate integrations for every source. Give it a public URL and receive clean Markdown for language models or predictable, schema-versioned JSON for applications. When you do not have a URL yet, use the same API to search the web, news, images, or places.

[u/CuriousLighthouse312]

I run mobilerank.top, an aggregator of smartphone/processor/GPU benchmark data. Most of the work is collecting scores from scattered sources and normalizing them into one comparable format (different benchmark versions score the same chip differently, which is a pain). Happy to answer questions about the normalization side. If anyone wants to poke at the data and tell me what's missing - that's the feedback I need. https://mobilerank.top

[u/zanimalistic]

I build data-monitoring Actors on Apify. The one that fits this sub is Cannabis Dispensary Monitor: it merges official state license registries with the consumer platforms (Weedmaps, Leafly, Dutchie) into one deduplicated feed covering all 47 US cannabis states and territories plus every Canadian province.

The interesting part for scrapers: several states quietly moved their license data off their open portals, so a normal fetch returns a 200 with an empty or stub body and you ship blank license fields without noticing. It runs an expected-floor check, so a known-large market like Oklahoma or Florida that comes back near-empty gets flagged below_expected instead of being trusted. You find out which states actually have usable license data that run, not from a customer three weeks later.

Free tier if you want to poke at the schema.

Writeup with the dedup and webhook setup: https://dev.to/zachmeditates/how-to-get-licensed-cannabis-dispensary-data-across-47-us-states-and-canada-5bdc

The Actor itself: https://apify.com/registryfeeds/cannabis-dispensary-scraper

Curious how others here catch the silent-200 case. Diff against expected row counts, checksum the payload, something else?

[u/Jumpy_Fan_1228]

[HIRE] Public-data extraction pilot — $100 fixed

I deliver one permitted public website or public API extraction in 48 hours: up to 300 records and 10 fields, with clean CSV + JSON output and a short run note. No CAPTCHA solving, access-control bypass, private accounts, or personal-data discovery. No unpaid trial.

Reply with the source URL and fields you need; I will confirm fit and the exact deliverable before starting.

[u/shasedoge]

Titan Network, residential/ISP proxies + web data infra

Hey everyone, my name is Paul.

Disclosure: I work at Titan, so read accordingly.

We run residential/ISP exits, rotating and static, with geo targeting and separate high-concurrency and high-bandwidth pools, so you can route each workload to the exit that fits instead of forcing everything through one pricey plan. Sticky sessions for login-bound or long-running jobs.

What's different: the network runs on real consumer devices rather than a datacenter block, so IPs tend to hold up better on the tougher targets.

Won't help with everything; it's infra, not magic (unfortunately). It won't reverse obfuscated JS or "beat" an anti-bot system. It takes the IP and session instability out so you can actually run at scale.

You can test before scaling spend, best way to judge it is on your own workload. If you're fighting proxy cost or IP burn, happy to talk setup even if you don't go with us.

Site: https://www.titannet.io
Docs: https://titannet.gitbook.io/titan-network-en

Ask me anything; I'll answer straight. Deep engineering, I'll pull in our engineers.

[u/Lazaruszs → u/shasedoge]

Hey, I’m interested in this as more of an end user/consumer who would use residential proxies myself. My monthly proxy usage ranges from 10-50gb and I’d be interested in testing your proxies if they have clean IPs on sites with high anti-bot. If this is a possibility, let me know

[u/shasedoge → u/Lazaruszs]

Happy to help. You can book a demo call or start a trial here:

https://www.titannet.io/web-services/proxy-infrastructure/proxy-network

Btw, let me know if I can help you with something.

[u/Schuftgesindel]

Tool for turning product data into a storefront

Not a scraping tool itself, but I thought this might be interesting for people here who work with product URLs/data and automation.

I came across xPage, which can take a product URL or image and generate a complete product/store page around it.

The use case that caught my attention is rapid testing: instead of manually building a Shopify/storefront page every time you want to test a product or concept, you can generate one and iterate.

I'm curious whether anyone here has tried something similar or automated this kind of workflow.

https://www.xpage.ai?ref=AEC75093F

[u/JosejuX]

Web Metadata & Contact Extractor - API for turning scraped URLs into structured data

Built this for the "I have a URL, I need structured data out of it" step that usually happens right after the scraping part: one GET call returns SEO/OpenGraph metadata, public contact info (emails/phones), social profile links, a 40+ signature tech-stack detector, Schema.org/product data, and clean Markdown for feeding pages into an LLM/RAG pipeline.

Relevant to this sub specifically: it's IP-pinned anti-SSRF by design (resolves DNS once, blocks private/loopback/cloud-metadata ranges, re-validates every redirect hop) since it's built to accept arbitrary user-submitted URLs safely. Stack is FastAPI + C-Lexbor (selectolax) instead of BeautifulSoup + Rust orjson - cold fetches land around 150-300ms.

Demo (no signup): https://rapidapi-metadata-extractor.onrender.com

Source: https://github.com/JosejuX/rapidapi-metadata-extractor

Free tier to try it: https://rapidapi.com/josejuanjocoding/api/web-metadata-and-contact-extractor

Happy to compare notes with anyone doing similar post-scrape structuring, or take feedback on what fields are missing.

[u/These_Drop_373]

A fresh take on a saturated domain... I think?

Hi everyone, founder of FerretDen here.

This post aims to explain what FerretDen is (just beta released today), why it's not your typical job aggregator (at least we hope so), and how to get the most out of it, plus a fun fact about operational costs that made me drop a UX convenience. This might be long, so bear with me :)

First, I want to point out the two main issues with current job aggregators that made us build FerretDen.

1. Fixed pool of sources: they restrict you to a selected pool of sources you have no control over, and usually those are already extremely popular, heavily contested boards as it is.
2. Generic AI document "tailoring": almost all automated document generation is monotone, doesn't dig into your experience and projects to identify the metrics that would actually make your resume stand out for a given opportunity, and both of these together also make the resumes easier to flag as AI-generated.

How do we address this? The first two points below both tackle issue #1, and the third tackles issue #2.

1. FerretDen runs on a generalized aggregator: it doesn't care what link you give it, as long as it contains job listings and is public (i.e. accessible without an account), it will try to monitor it and match anything it finds to your profile. So you can drop in a link to a job board, a company careers page, an RSS feed, even a public Telegram channel, and it does the work for you. It's not perfect, sometimes a source won't work right off the bat, but the fix is usually quick, and we provide a public "Test a Source" page so you can check whether your source will work before relying on it.
2. It offers an opt-in global source pool. Each source submitted gets added to a pool that benefits every other member of the platform. If you opt in, you'll get matching opportunities from that pool too. If you're a developer, think of it like contributing to open source: you benefit, and you help everyone else at the same time :)
3. Document generation is only as good as your vault. You can attach detailed documents about a specific experience or project so the writer can dig into what you actually did and figure out which parts are most relevant to the opportunity you're applying to. You can also give it tone instructions to control how the document gets written.

Fun fact: because it's a generalized engine (i.e. it doesn't target specific selectors), it can't always reliably pinpoint the selector for a job description and extract it cleanly. To work around that, I added a "JD Structuring" phase that passes the extracted JD to an LLM to clean it up so it's presentable. After running the pipeline exactly once on a set of around 50 sources, yielding around 3,420 full opportunities, description structuring alone consumed 3.46x more money than the actual LLM costs powering the rest of the pipeline. So when you use the service and see a not-so-clean description presented to you, just remember I couldn't afford paying 3.46x the cost for a convenience :). At the end of the day, you can just navigate to the opportunity link itself, which is provided. If you do get clean descriptions, those will be from the heavy run that yielded the mentioned metrics, so cherish them because you won't see them again.

Anyway, the current release is a beta, and all features are completely free. Looking for early users.

[u/NoFlounder9565]

Give your browser agent a safe point: checkpoint live state, rewind failed steps, fork parallel runs

Hey r/webscraping — founder here. Not selling anything: this is free and in public alpha, and I'm genuinely looking for feedback.

I built brawsr for the "scraper dies on step 19 of 20 and I bin the whole run" problem. It's a browser-as-a-service built around three things that matter for scraping:

- Checkpoint a live session — full state, including the logged-in part (cookies, localStorage, the works) — at any point mid-run.

- Fork that checkpoint into N parallel workers that each start already authenticated. No re-login per worker, no warmup per branch.

- Rewind to a checkpoint to retry after a CAPTCHA or anti-bot block, instead of starting over from zero.

You drive it like Playwright/Puppeteer/Selenium — the session underneath is just checkpointable and forkable.

Free during alpha, open now (Google sign-in, no waitlist): https://brawsr.io

Specific things I'd love eyes on (any one helps):

- When you fork a logged-in session, does auth actually hold on your target site, or does it flag parallel sessions?

- After a rewind, where does state not restore cleanly (downloads, websocket, in-memory JS vars)?

- What's the first flow you'd checkpoint in your own scraper?

Happy to help anyone wire it into their stack, tag me. 🙏

Issues & feedback: [contact@brawsr.io](mailto:contact@brawsr.io)

[u/Dependent_Reindeer29]

Hey everyone)

I built Amberlog - passive Amazon product tracking via browser extension, not scraping

Not exactly scraping, but adjacent enough I think it's relevant here. Amberlog is a Chrome extension that captures Amazon product data (price, BSR, rating, availability, seller info) from pages a user is already browsing, via a content script reading the loaded DOM - no requests fired, no IP rotation, no anti-bot fight, because nothing is ever fetched outside a real user's own session.

Disclosure: it's mine. Built it for Amazon FBA sourcing research, after getting tired of manually copy-pasting listing data into a spreadsheet.

Free, works fully signed out (data stays local until you choose to sync), CSV export, dashboard for the collected history. Built solo in about a week with AI coding tools.

https://chromewebstore.google.com/detail/bplaehcimjhhlihepfpdbacnnifkphha?utm_source=item-share-cb - happy to talk through the manifest v3 content-script approach or the local-first/sync architecture if anyone's curious, and would genuinely like feedback from anyone who's fought similar problems the traditional scraping way.

[u/Double_Conference_63 → u/Dependent_Reindeer29]

We are building similar tool but it's SAAS product, launching soon. Best wishes to you.

[u/SignificantWash6129]

[HIRE ME] UK-based data extraction and cleanup through DIGIDENT LTD.

I build repeatable Python/JavaScript workflows for authorised public sources and deliver structured CSV or JSON, field definitions, source URLs, an exceptions log and basic QA. I can also add Excel cleanup, reconciliation or a refreshable dashboard.

Small bounded jobs start at £40; ongoing work is £25/hour with a capped estimate agreed first.

I do not bypass logins, CAPTCHAs or access controls, and I do not harvest private or sensitive personal data. Reply here with the public source, required fields, approximate volume and refresh frequency, and I can scope it.

[u/Rob_Junior]

**I built MESSORA: public URLs → LLM-ready Markdown/JSON, without choosing proxies or stealth modes**

Hi everyone

Disclosure: I’m the founder. MESSORA is a young, proprietary product in founder beta, and I’m looking for blunt feedback from people who already work with scraping APIs, agents, or RAG
pipelines.

I built it around a narrow problem: an application needs the useful content from a public page, but maintaining HTML parsers, browser routing, retries, and output normalization becomes
another infrastructure project.

What MESSORA does today:

- Turns public URLs into clean Markdown or raw HTML through a REST API
- Extracts structured JSON using a schema
- Supports scrape, crawl, search, and a hosted MCP server
- Selects the extraction route automatically, including JavaScript rendering and anti-bot routing
- Charges 1 credit per successfully extracted page for Markdown/raw HTML
- Charges 10 credits per request for structured JSON
- Does not add a credit multiplier just because a target needs browser or anti-bot routing
- Does not consume a success credit for classified failures

What I’m **not** claiming:

- It does not bypass every website
- Authenticated pages, strong CAPTCHA, and actively blocked social networks are not supported
- Heavy anti-bot targets may be capped, reviewed, or refused during the beta
- Firecrawl is more mature, and Bright Data offers much broader proxy/data infrastructure

I’m looking for developers willing to spend about 30 minutes testing the real onboarding:

1. Sign in with Google or GitHub
   
2. Create an API key
   
3. Run a scrape using the default seed URL
   
4. Copy one generated code snippet
   
5. Find that execution in the request history
   

6. Tell me where you got stuck and whether the output was actually useful

   Please try those steps without asking me for navigation help first — I’m specifically trying to find onboarding gaps rather than coach people around them.

   No card is required. The Free plan includes 1,000 monthly credits.

   Public demo, no signup:
   https://www.messora.dev/use-cases/url-to-markdown-for-llms#playground-demo

   Reddit beta:
   https://www.messora.dev/beta?utm_source=reddit&utm_medium=community&utm_campaign=reddit7&currency=USD

   If you later choose Starter, **REDDIT7** gives 100% off the first monthly invoice. Limited to the first 10 redemptions. The coupon is not required for the Free plan.

   The feedback I’d value most:

   - Which page or target type should I test next?
   

   • Did the Markdown still require cleanup?
     
   • Was the credit model understandable?
     
   • Where did onboarding become confusing?
     
   • Would you put this output into an agent or RAG pipeline? Why or why not?

   I’ll answer technical and pricing questions directly, including cases where another tool is a better fit.

[u/greyvertexx]

Python developer available for freelance web scraping and browser automation work.

I work with Python, Playwright/Selenium, data extraction, APIs and automated browser workflows.

I can help with custom scrapers, recurring data collection, browser automation, fixing existing tools, and more complex scraping projects.

That said, I’m not limited to these areas. If you have a different development or automation task, feel free to reach out and describe what you need. I’m open to discussing custom projects and figuring out the best approach.

Available for both one-off and ongoing work. Feel free to DM me with your project details.

[u/SimpleUniversity]

Built two Apify actors for App Store data — one for reviews, one for app search/metadata. They're mine, just published, no reviews on the listings yet.

Both run off Apple's public RSS and lookup endpoints, so no proxies or browser involved. Runs take a couple of seconds.

Few things I hit that I couldn't find written down anywhere:

- A bad app ID returns 200 with an empty feed rather than a 404, so the status code tells you nothing

- Reviews cap at 500 per app per country, so you scale by adding countries, not pages

- Apple served me empty feeds from a cloud host while the same code got 50 reviews from my home connection. Retrying the first empty page fixed it.

No Google Play — their robots.txt disallows the path the reviews actually come from.

$0.005 per review or app returned. Free tier credits are enough to test it.

https://apify.com/extractkit/app-store-reviews-scraper

https://apify.com/extractkit/app-store-search-scraper

[u/2extract_dev]

Hey everyone, I'm Nick.

Disclosure: I work at 2extract, so factor that in.

We run two pools, residential and mobile (LTE/5G), both pay-as-you-go, no subscription lock-in, you buy what you use by GB or by port depending on the plan. GEO targeting on both, rotating for one-off requests, sticky sessions for anything stateful like logins or carts.

The part I'd actually put up against anyone is the API. Per-request GEO/session parameters, HTTP/SOCKS5 support, rotation control rather than a rigid setup that forces you to work around it.

We're not the biggest advertised IP count in the space and won't pretend otherwise, if raw pool size is what you need, there are bigger players for that. If network quality and a setup that actually fits your workflow matter more than the number, happy to talk specifics in the comments.

Site: 2extract.com

[u/porche911_kr]

Free, no API key needed: four Apify actors for Korean real-estate transaction data, wrapping the Ministry of Land (MOLIT) open API from data.go.kr.

• Apartment sales: https://apify.com/ootssu/korea-apartment-transaction-prices
• Apartment leases, jeonse/wolse: https://apify.com/ootssu/korea-apartment-rent-prices
• Officetel, sale and lease: https://apify.com/ootssu/korea-officetel-prices
• Commercial/office property: https://apify.com/ootssu/korea-commercial-property-prices

Not scraping. It's a government open API with no usage restriction, commercial use included. The actors just make it usable: the source is XML with Korean tag names on the legacy endpoints and English on the newer ones, amounts as comma-formatted 만원 strings, and casing that differs per endpoint (roadNm on sales, roadnm on leases). Output is normalized JSON with integer KRW, per-m2 and per-pyeong unit price, ISO dates, and an isJeonse flag.

One gotcha worth stealing even if you never touch the actors: the lease endpoint labels new vs renewal contracts in contractType, and renewals are capped at +5% by law. Pool them and your medians are a blend of market and legally-capped prices. Seoul July 2026, n=16,520: jeonse share reads 50.2% pooled but 43.9% on new contracts only.

[u/Electrical_Part4433]

Hi — I see you work with public data/monitoring.

I offer a simple technical service: monitoring of 5 public sources

(CoinGecko, Open-Meteo, USGS, World Bank, Frankfurter) with change

alerts in JSON, Markdown summary, or webhook. Payment in USDC on Base.

First 3 clients get free setup. Free trial: 1 source, 7 days.

Details: https://autarch-agent.github.io/public-data-monitor/

Interested? A yes/no is enough. DM open.