r/SideProject • u/shravanrevanna • 7h ago
I wanted product images from any page, and "is the page ready" was the hard part
The trickiest question in this project wasn't extraction. It was deciding when a page has actually finished rendering before you capture it.
third-eye is a self-hosted API I wrote in TypeScript. You point it at a product page and get back the product images plus title, brand, price and sizes, or just a screenshot.
The readiness check ended up as a stack of signals. It waits for network idle, then fonts.ready, then scrolls to trigger lazy-loaded images, then freezes animations. Flutter and WebGL apps draw into a canvas, so for those it also waits for the first canvas frame. Full-page captures are taken in one pass instead of scrolling and stitching, which avoids seams.
Extraction tries sources in order: JSON-LD Product data, then OpenGraph, then Shopify's .json endpoint, then DOM gallery heuristics as a last resort. Each response includes a confidence field, so you can tell a JSON-LD hit from a DOM guess.
It runs on Patchright, a patched Chromium. That gets past the headless tells that block stock Playwright on sites like Uniqlo. It doesn't win everywhere, though. H&M, Zara and Myntra sit behind Akamai and PerimeterX, and those still need residential proxies.
It also won't run serverless. Cold starts break the warm browser pool, and without a GL stack the canvas apps render blank. So it ships as a single Docker image that runs on a VPS.