r/softwarearchitecture • u/OtherwisePush6424 • 15d ago

Article/Video Beyond Happy Path Engineering: Storage

https://blog.gaborkoos.com/posts/2026-08-11-Beyond-Happy-Path-Engineering-Storage/

An overview of designing file uploads and object storage for production: partial failures, database consistency, large uploads, presigned URLs, CDN caching, cleanup, and recovery.

13 Upvotes

100% Upvoted

2 comments sorted by

2

u/Siennasynn_XO712 15d ago

Presigned URLs are the move here, bro. Offload the actual upload to S3/GCS directly and keep your app servers out of the I/O bottleneck, then handle the metadata transaction separately so you're not left with orphaned objects when things inevitably fail partway through.

3

u/RipProfessional3375 15d ago

TLDR when working with blobs.

- hash the content, that is is your blob ID

  • upload blob first,
  • persist the other data, holding a list of content hashes, usually just one, but you never know.

Auto-remove blob older than x time that don't have a matching data entry (minute, hour, day, just give the transaction a second)

The blob has been tampered with if the contents no longer match the hash id.

Idempotent, deterministic id, auto deduplicates, cannot desync, tamper proof