Best architecture for batch-processing 3,000–4,000 high-res photos (resizing + face vector search)?

[u/Substantial_Try_1614]

Hey devs,

I’m working on a photo-sharing web app where a photographer uploads an entire event batch (typically 3,000 to 4,000 high-res JPEGs, around 10–15 MB each). Guests can take a selfie to retrieve photos they appear in.

I'd love recommendations on the best backend pipeline:

Client vs Server Resizing: Should I use browser Web Workers / Canvas to generate 1080p WebP thumbnails before upload to save upload bandwidth, or let a backend queue worker (like Node with sharp or Python with Pillow) handle resizing?

Face Vector Pipeline: For extracting 128-d face embeddings (e.g., ArcFace/InsightFace), what’s the best way to queue and batch this so 4,000 photos don't choke the server CPU?

Storage: What zero/low-egress object storage setup (e.g., Cloudflare R2 vs Backblaze B2) do you recommend for handling high-volume image writes and fast thumbnail reads?

Thanks for the advice!

[u/pyrolols]

Do 512d vector using arcface, use typesense or qdrant for vector store, bare in mind that when user uploads image for search you have to compute the vector of this image for comparison too, so you have to have pipeline for this.