Storage & Data Modelling

Not everything belongs in a database. Matching the storage type to the access pattern avoids a large class of scaling problems.

Object storage (S3, GCS, Vercel Blob) holds immutable blobs addressed by key. Effectively unlimited, cheap, durable, and high-latency per request. Correct for images, video, backups, and logs. Block storage is a raw disk attached to one machine — the thing databases run on. File storage offers a shared POSIX filesystem across machines, convenient and usually the slowest option.

The rule for large files: store the bytes in object storage and the metadata plus the key in your database. Storing blobs in a relational database bloats backups, wrecks buffer-cache hit rates, and drags down queries that never touch the blob.

Uploads should not proxy through your API. Issue a presigned URL so the client uploads straight to object storage; your server only signs and records. This removes your API from the bandwidth path entirely.

Modelling. Normalise until joins hurt, then denormalise deliberately, and know that a denormalised copy is a consistency obligation you have taken on. Soft deletes preserve history and mean every query must remember to filter. Append-only event tables give you an audit trail and unbounded growth to manage.

Durability is not availability. Object storage advertises eleven nines of durability and considerably fewer of availability. Those are separate promises: the first says the bytes will still exist, the second says you can fetch them right now. A design that treats a storage read as infallible has quietly bet on the weaker of the two.

Classes and lifecycle. Tiers trade retrieval cost and latency against storage price, so data written once and read almost never — old logs, raw event archives, backups past their restore window — belongs on a colder tier. Make that a lifecycle rule rather than a chore someone remembers, because storage bills grow quietly and no one looks at a bucket until it is expensive.

Deletion is a design decision. Overwriting a key in an object store is a new version rather than an edit, and versioning plus object lock is what stops a bad deploy erasing production data. The same mechanism means bytes you told a user you deleted are still there until the lifecycle rule catches up, which matters the moment you have promised anyone a deletion deadline in writing.