File Storage: Object Storage (S3 & R2)

A user picks a photo, taps upload, and a second later it shows on the page. Simple on the surface — but where did the bytes actually go? Not into your database. Not onto your web server’s disk. They went into a thing called object storage, the invisible workhorse behind every avatar, every attachment, every video, every backup on the internet. This module strips the mystery off it — and by the end you will have read the exact ~40 lines of real code that store and serve every photo on a blog you built.

The Goal

By the end of this module you can:

• Explain why files do not belong in your database or on your server’s disk — and what breaks when you put them there anyway
• Define object storage from scratch: buckets, objects, keys, metadata — a giant key-to-bytes map you talk to over HTTP
• Separate the three storage families — object vs block vs file — and say which one S3 and R2 are
• Trace an upload two ways: proxied through your server, and uploaded directly with a presigned URL — and choose between them
• Read the real upload.js and media-serving code from the Blogging Website and name what every line does
• Map Cloudflare R2 to AWS S3 one-to-one, and explain the single number (egress) that makes R2 exist
• Secure an upload: validate the file, cap its size, and decide public vs private buckets

The Lesson

Why files do not go in the database

The instinct of every beginner is: “I have a database, files are data, put the file in the database.” You can — SQL has a BLOB type (Binary Large OBject). You should not. Here is why, concretely:

• Databases are tuned for small rows, read millions of times. A 5 MB photo is 5,000× bigger than a typical row. Stuffing blobs in makes every backup, every replica, every query slower — you drag megabytes of image bytes through machinery built for kilobytes of text. (You met this machinery in How Data Is Stored.)
• You cannot put a database behind a CDN. A photo should be served from a server near the user and cached for a year. A database row cannot be cached at the edge — so blob-in-DB means every image download hits your one database, in one region. That does not scale.
• It wastes your most expensive resource. Database storage and RAM are premium. Object storage is dirt cheap. Paying database prices to hold vacation photos is like storing firewood in a bank vault.

So the rule every real system follows: the database stores a short text URL; the bytes live somewhere built for bytes. In the Blogging Website, the posts table has a cover_url column — a string like /media/uploads/3f2a....jpg. The image itself is nowhere near the database.

The second wrong place: the server’s own disk

“Fine, not the database — I will save the file to ./uploads/ on my server.” This works on your laptop and dies in production, for one reason: modern servers are stateless and disposable.

Your app runs on servers that get created and destroyed constantly — a deploy replaces them, autoscaling adds and removes them, a crash reboots them. Anything written to a server’s local disk vanishes with that server. And if two servers sit behind a load balancer, a file uploaded to server A does not exist on server B — so half your image requests 404 at random. (This is exactly why scaling means going stateless: user files must live outside any single server.)

The fix is the same as the database fix: put the bytes in one shared, durable place that every server can reach over the network. That place is object storage.

What object storage actually is

Strip away the branding and object storage is astonishingly simple. It is a giant map from a key (a string) to a value (a blob of bytes plus a little metadata), that you talk to over HTTP.

Three things make it not a filesystem and not a database, and this is the whole mental model:

1. It is flat, not a tree. There are no real folders. uploads/2026/cover.jpg is not three nested directories — it is one key that happens to contain slashes. The console draws folders for you, but under the hood it is one flat namespace of keys.
2. You get and put whole objects, over HTTP. PUT a key with bytes to store it; GET the key to read it back. There is no “open file, seek to byte 40,000, write 10 bytes.” Objects are immutable-ish: to change one, you replace the whole thing.
3. It is effectively infinite and self-healing. You never provision size. You put your first object and your billionth object the same way. The provider silently keeps multiple copies across machines (and often data centres) so hardware can die without losing your data — this is durability, and S3 advertises eleven nines of it (99.999999999%).

That is it. A bucket is a namespace, an object is key -> bytes + metadata, and the API is HTTP. Every “file storage” product on earth is a variation on this.

The family: object vs block vs file storage

Object storage is one of three storage types, and interviews love the distinction. Here is the honest separation:

The tell: block storage is a disk for one machine; file storage is a shared folder; object storage is an HTTP key-value store for the internet. When someone says “file storage” in a web-app context, they almost always mean object storage — S3 or its clones — because it is the only one that scales to internet traffic and sits behind a CDN.

The anatomy of an upload — two patterns

When a user uploads a file, the bytes have to get from their browser into the bucket. There are exactly two shapes for this, and choosing between them is a real design decision.

Pattern A — Proxy through your server. The browser sends the file to your API; your API forwards it to the bucket.

flowchart LR
    A["Browser"] -->|"multipart POST the file"| B["Your API server"]
    B -->|"validate, then PUT"| C["Object storage bucket"]
    B -->|"returns the file URL"| A

Pattern B — Presigned URL (direct upload). Your server never touches the bytes. It hands the browser a temporary, signed URL that grants permission to PUT one object directly into the bucket.

flowchart LR
    A["Browser"] -->|"1. ask for permission"| B["Your API server"]
    B -->|"2. presigned URL, expires in 5 min"| A
    A -->|"3. PUT the file straight to storage"| C["Object storage bucket"]

A presigned URL is worth pausing on, because it is a beautiful idea and it is pure authorization: your server, which holds the secret bucket credentials, computes a URL with a cryptographic signature baked in that says “the holder of this link may upload exactly one object, to exactly this key, for the next 5 minutes, and nothing else.” The browser never sees your credentials — it just gets a time-boxed, single-purpose key. Same core idea as the signed session tokens in the auth module: a signature turns an untrusted party into a temporarily-trusted one.

The real build — the Blogging Website, ~40 lines end to end

You do not have to imagine this. Here is the actual object-storage code from a blog running on Cloudflare, where the bucket binding is called MEDIA (an R2 bucket, declared once in wrangler.toml):

# wrangler.toml — bind the R2 bucket to the app as env.MEDIA
[[r2_buckets]]
binding = "MEDIA"
bucket_name = "silkys-blog-media"

The upload (src/pages/api/upload.js) is Pattern A — proxy through the server — because the files are small blog images and we want to validate them first:

export async function POST(ctx) {
  const denied = await guard(ctx, { api: true });   // must be logged in (authorization)
  if (denied) return denied;

  const env = ctx.locals.runtime.env;
  const form = await ctx.request.formData();
  const file = form.get('file');

  if (!file || typeof file === 'string')            // 1. there must be a file
    return Response.json({ error: 'no file' }, { status: 400 });
  if (!file.type.startsWith('image/'))              // 2. images only — never trust blindly
    return Response.json({ error: 'images only' }, { status: 400 });

  const ext = (file.name.split('.').pop() || 'bin').toLowerCase().replace(/[^a-z0-9]/g, '');
  const key = `uploads/${crypto.randomUUID()}.${ext}`;   // 3. a random, unguessable key

  await env.MEDIA.put(key, await file.arrayBuffer(), {    // 4. PUT the bytes into the bucket
    httpMetadata: { contentType: file.type },             //    remember the content-type as metadata
  });

  return Response.json({ url: `/media/${key}` });         // 5. hand back a short URL for the DB
}

Read it as five moves: auth-gate → require a file → validate the type → mint a random key → put the bytes with metadata → return a URL. That returned URL (/media/uploads/3f2a....jpg) is the only thing that ends up in the database. Note the key uses crypto.randomUUID() — a UUID (Universally Unique IDentifier) is a random string so astronomically unlikely to repeat that it is treated as one-of-a-kind — never the user’s filename, so two people uploading photo.jpg do not collide, and nobody can guess someone else’s key.

The serving (src/pages/media/[...key].js) reads the object back out and streams it to the browser with the right headers:

export async function GET({ params, locals }) {
  const env = locals.runtime.env;
  const obj = await env.MEDIA.get(params.key);          // fetch the object by key
  if (!obj) return new Response('Not found', { status: 404 });

  const headers = new Headers();
  headers.set('content-type', obj.httpMetadata?.contentType || 'application/octet-stream');
  if (obj.httpEtag) headers.set('etag', obj.httpEtag);  // fingerprint for cheap re-checks
  headers.set('cache-control', 'public, max-age=31536000, immutable');  // cache for a YEAR
  return new Response(obj.body, { headers });
}

Every line here is a concept you have already met on the HTTP module: the content-type header (pulled from the metadata we saved at upload time) tells the browser it is a JPEG; the etag is a fingerprint so a returning visitor can ask “changed?” and get a cheap 304; and cache-control: public, max-age=31536000, immutable says “anyone may cache this for a year and never re-check.” That last header is what lets a CDN hold the image at the edge, so the bucket is hit once and then thousands of visitors are served from a machine near them. The immutable promise is safe because the key is a random UUID — if the image ever changes, it gets a new key, so a cached copy is never wrong.

That is the entire loop: put bytes + metadata under a random key → store the short URL in the DB → get them back with caching headers → let a CDN do the heavy lifting. Forty lines. Every serious app does exactly this, just with more validation.

Walk the whole flow

R2 is S3 — mapped one to one

You are learning on Cloudflare (R2); most jobs run on AWS (S3). Good news: they are the same concept with different names, because R2 is deliberately S3-compatible — the same API, so the same code and tools work against both.

The one number that explains why R2 exists: egress fees. AWS charges you every time bytes leave S3 to the internet (roughly $0.09/GB). Serve a viral image a million times and the storage is nearly free but the egress bill is brutal. R2 charges zero egress. That single line item is Cloudflare’s entire pitch, and it is why cost-conscious builders (and this blog) reach for R2. You will meet the same egress trap again in Cost, Scaling & Security.

Security — the three rules you never skip

Uploads are a favourite attack surface. Three non-negotiables:

• Never trust the client’s content-type or filename. A file claiming image/png can be a script. For anything sensitive, sniff the real bytes (magic numbers), not the label. The blog’s startsWith('image/') check is the floor, not the ceiling.
• Cap the size. Without a limit, one request can upload a 10 GB file and blow up your bandwidth bill or memory. Enforce a max on both the client and the server.
• Choose public vs private deliberately. A blog’s cover images are public — anyone may view them, so a public bucket + long cache is correct. A user’s uploaded ID document is private — the bucket must be locked, and you serve each view through a short-lived presigned URL so only the right user, for a few minutes, can fetch it. Getting this wrong is how companies leak millions of private files from “just an S3 bucket someone left open.”

Where This Lives — The Bigger Picture

This page is one concept in a real build. It connects to:

• How Data Is Stored — why blobs wreck a database, from the storage-engine side.
• HTTP — the Content-Type, ETag, and Cache-Control headers this code sets by hand.
• Caching & Consistency — how immutable + a random key lets a CDN cache images forever, safely.
• REST Controllers — the same upload, done with MultipartFile in the Java/Spring world you are heading toward.
• Scaling & Distributed Systems — why stateless servers force user files into shared storage in the first place.
• Authentication — presigned URLs are signed, time-boxed permission: the auth idea, applied to storage.

Build This

Pick the stack you are on. Both free.

On Cloudflare (R2). First, the two tools you are about to type — because you should never run a command you cannot name:

• wrangler is Cloudflare’s command-line tool: the way you create and manage Cloudflare things (buckets, Workers, databases) by typing commands in your terminal instead of clicking around the dashboard. That wrangler.toml file in the blog is wrangler’s config file — it is how the blog told wrangler “bind an R2 bucket called MEDIA.”
• npx is a Node command that runs a tool without installing it permanently. npx wrangler … just means “fetch-and-run wrangler.” (If you use it a lot, npm i -g wrangler installs it so you can drop the npx.)

1. Create a bucket: npx wrangler r2 bucket create my-first-bucket
2. Put a file into it: npx wrangler r2 object put my-first-bucket/hello.txt --file ./hello.txt
3. Read it back: npx wrangler r2 object get my-first-bucket/hello.txt
4. In the Cloudflare dashboard, open the bucket and watch your object appear with its key, size, and content-type metadata. Notice there are no real folders — put a/b/c.txt and see it is one key, not three directories.

On AWS (S3) — the industry default, free tier 5 GB:

1. In the S3 console, create a bucket (globally unique name).
2. Upload any image through the console. Click it — read its Object URL, its metadata, its storage class.
3. Toggle “Block all public access” off for a test object and open the URL in a browser; then turn it back on and watch it 403. That toggle is the public-vs-private decision, made real.

Then trace the real thing: open the Blogging Website’s src/pages/api/upload.js and src/pages/media/[...key].js and annotate every line against the five-move / serving breakdown above. If you can explain each line out loud, you own this.

Check the concept

Check Yourself

1. Give two concrete reasons a 5 MB image should not live in a BLOB column.
2. Why does saving uploads to a server’s local disk fail once you have more than one server?
3. Define bucket, object, key, and metadata in one sentence each.
4. Object vs block vs file storage — which is S3, and what is the one-line tell for each?
5. Proxy vs presigned-URL upload: when do you pick each?
6. In the blog’s upload.js, why is the key a random UUID instead of the user’s filename?
7. Which three response headers does the media route set, and what does each do?
8. What is egress, and why does it make R2 attractive versus S3?

Explain it out loud: In two minutes, narrate what happens from the moment a user taps “upload” to the moment a visitor on the other side of the world sees that image load instantly — name every place the bytes go, and end with why the database only ever held a short string.

Why AI Can’t Do This For You

AI will happily generate an upload endpoint that saves to local disk, trusts the client’s content-type, uses the raw filename as the key, and leaves the bucket public — and it will look clean until it 404s in production, gets a poisoned “image,” collides two files, or leaks. The judgment calls here — proxy vs presigned, public vs private, what to validate, how to key objects so caching stays safe — are decisions about your system’s failure and threat model. The model does not know your files are user ID documents, or that your bucket is one careless toggle from the news. Reading the forty real lines and knowing why each one is the way it is — that is the skill that survives.

Module done? Mark it complete above, and carry the pattern into the next real build.