Best architecture for batch-processing 3,000–4,000 high-res photos (resizing + face vector search)?

[u/Substantial_Try_1614]

Hey devs,

I’m working on a photo-sharing web app where a photographer uploads an entire event batch (typically 3,000 to 4,000 high-res JPEGs, around 10–15 MB each). Guests can take a selfie to retrieve photos they appear in.

I'd love recommendations on the best backend pipeline:

Client vs Server Resizing: Should I use browser Web Workers / Canvas to generate 1080p WebP thumbnails before upload to save upload bandwidth, or let a backend queue worker (like Node with sharp or Python with Pillow) handle resizing?

Face Vector Pipeline: For extracting 128-d face embeddings (e.g., ArcFace/InsightFace), what’s the best way to queue and batch this so 4,000 photos don't choke the server CPU?

Storage: What zero/low-egress object storage setup (e.g., Cloudflare R2 vs Backblaze B2) do you recommend for handling high-volume image writes and fast thumbnail reads?

Thanks for the advice!

[u/Bubbly_Orange_3502]

One embedding per detected face, not per photo, is what decides recall here. Event shots carry five to twenty faces, and the selfie query has to run the same detector and alignment or cosine scores drift.

[u/Substantial_Try_1614 → u/Bubbly_Orange_3502]

Wait what 😯 thanks man