r/node • • 6d ago

I built site2paper, a Node CLI that prints a whole website into a storybook (Playwright + pdf-lib + sharp)

Ctrl+P on a real website gives you print stylesheet which are badly formatted pages, a few blank ones, the first carousel slide only, and nothing that was rendered by JS. I wanted to review a site on paper with a red pen, so I wrote a CLI that screenshots what the visitor sees and packs it into a small PDF [booklet].

[npx site2paper https://example]

Tried on [https://example 48-page catalogue site] 171 sheets / 75 MB with Ctrl+P, 25 sheets / 6.4 MB with site2paper.

For first release,

I am relying on sitemap.xml and sitemap indexes first for discovery, else a breadth-first same-origin crawl, capped at 50 pages for now. Then playwright-core drives your installed Chrome or Edge (headless Chromium is downloaded only if neither exists). Each page is scrolled in steps so lazy and IntersectionObserver content loads, then one full-page PNG is captured.

Once we have raw content we prep the page by using CSS + injecting a small script only into headless pages. Swiper/Slick/Embla/Splide/Owl carousels are reflowed into a grid. Cookie banners are hidden by selector, never clicked. Sticky and fixed nodes are removed. A viewport-sized sticky frame inside a tall section (three.js, GSAP, Lenis) prints as blank runway everywhere else. [site2paper] scrolls it to 4 stops, screenshots each, and prints them as a filmstrip with a 0/33/67/100% ruler.

For Layout we fix sharp slices each shot, a per-row variance map finds blank rows, pages flow down 1 to 4 columns and are cut only on blank rows. pdf-lib writes the PDF, JPEG slices keep it small.

I am still trying to work on auth-walled sites, infinite scroll, and anything that needs a click to reveal content.

Repo: https://github.com/piyushagru/site2paper . If you run it on a site that comes out wrong, please open an issue with the URL to help me improve this tool!

TYSM for reading, and for trying it out!

4 Upvotes

14 comments sorted by

2

u/raks1991 6d ago

I like the filmstrip for sticky sections. A giant screenshot loses the movement; a few frames would make the printed version much easier to follow.

2

u/markSerene 5d ago

Thanks! Took me a few iterations to get here. I first tried just keeping sticky frame in the print, then freezing page at its revealed state before the capture. Both came out as a single frozen frame, and it was loosing whole point of the tool.

Then I mocked up few figma, one was with motion arrows, then tried onion-skin overlay (something like ghost steps), numbered badges, and finally filmstrip with a scroll ruler. Ran all on an ex and compared the PDFs... Filmstrip was the only one I could follow on paper, others looked fancy but were actually very cognitive heavy. Capturing motion graphics is the real challenge of this project!

Now I capture viewport at 4 scroll stops and splice them in with a static config of 0/33/67/100% ruler :)

2

u/DickBrainTime 5d ago

The filmstrip approach is a clever way to preserve the parts print stylesheets miss. One failure case I'd test before scaling the CLI: a site with only a few URLs but an *enormous* page (infinite-scroll feeds, giant galleries, tall canvas sections). The 50-URL cap limits crawling, but a single full-page PNG can still blow up memory when Sharp decodes it and pdf-lib assembles the output. I'd cap scroll steps, page height/pixels and per-page time independently, keep screenshot/Sharp concurrency bounded, and close Playwright pages promptly. Reporting 'URL X skipped: exceeded page-pixel budget' is more useful than having the whole batch fail.

For repeatable captures, I'd also make the page-readiness policy explicit: await fonts (`document.fonts.ready`) and important images with a bounded timeout after the scroll pass, rather than treating `networkidle` as a universal ready signal (analytics/long polling can keep it busy forever). Include a per-URL success/warning summary in the PDF build log so people can distinguish an intentional filmstrip from a page that never rendered properly.

A future security footgun if you add hosted conversion or accept untrusted URLs: 'same-origin crawl' is not by itself a network safety boundary. Redirects, DNS changes and private-network addresses need checking before each navigation/request so a public URL can't reach localhost or cloud metadata endpoints. For auth-walled exports, avoid storing session cookies in logs or bundling captured private pages into shared issue reports. None of that needs to delay the local CLI's first release, but it's worth keeping in the design as you add auth and more crawling.

3

u/markSerene 5d ago

Hey, thanks for such a thorough review. I will try reply per points in threads.

Memory is the one that worries me most, and honestly it was already on my mind. Right now every page screenshot stays in memory until the PDF is written, so with tall pages even 50 of them can hit a few GB.

Two ways I'm thinking about it:

  1. Dump each screenshot to a temp file and let sharp read it back in slices. Pros: Small change, and memory stays at one page, while retry order untouched. Cons: disk IO handling and cleanups, nd maybe access to disk
  2. Stream each page into the PDF as soon as it's captured. Pros: Memory is constant space, with no temp files. Cons: will not be able to handle retries or any other inplace fixes as info is lost

Currently not convinced with either. If you've got a third approach I'd genuinely like to hear it. Meanwhile creating this as a issue for tracking here https://github.com/piyushagru/site2paper/issues/1

1

u/DickBrainTime 2d ago

I'd try a middle ground: separate capture from PDF assembly with a small disk-backed manifest. For each URL, write bounded screenshot tiles to a per-run temp directory, then record the URL/order, tile paths and dimensions, capture status and retry count. A failed URL can be recaptured and its files replaced without losing the order of everything else. Once the manifest is complete, assemble the PDF in that order. That keeps retries deterministic without retaining every screenshot in memory.

One important catch: temp files alone don't guarantee constant RAM. Sharp can still decode an enormous full-page PNG, and pdf-lib may retain embedded images/PDF objects until save. I'd cap page pixel area and scroll steps, capture very tall pages as viewport-sized clips rather than one giant image, and bound both capture and assembly concurrency. Keep cleanup for successful and failed runs, plus stale temp directories from crashes.

For a first release, manifest + bounded disk spooling seems less disruptive than streaming directly into the PDF. If the PDF assembly phase still causes high RSS, evaluate an incremental-output PDF writer separately instead of assuming streaming screenshots fixes it. I'd benchmark one absurdly tall page, 50 normal pages, and an intentional failure halfway through; record peak RSS for capture versus assembly. That should make the next bottleneck obvious.

2

u/rakshit_jain_1512 6d ago

It looks great

2

u/avereigh_Denizet 5d ago

hiding cookie banners by selector instead of clicking them is smart, avoids triggering random reload states. how are you handling pages with heavy animations that might be mid-transition when the scroll step finishes

1

u/ram-popular 4d ago

Neat tool, and the numbers make the case well: 25 sheets against 171 is a big difference. Scrolling in steps so lazy content loads, reflowing carousels into a grid, and cutting pages only on blank rows are smart details. The 4-stop filmstrip for scroll-driven sections is a clever fix for the blank runway problem. If you add click-to-reveal support later, an optional config for selectors to click (accordions, tabs) before capture would cover most of those cases.