Docker 02 — Images, Layers & The Build Cache

You already know an image is the frozen package and a container is the running process. Now open the box: an image is not one blob — it is a stack of read-only layers, and the single most useful Docker skill is making the build cache reuse those layers so a rebuild takes 5 seconds instead of 90. Get the layer order wrong and every tiny code edit re-downloads your whole dependency tree. This module is where slow builds become fast ones.

The Goal

Explain what a Docker image actually is — a stack of content-addressed read-only layers, not a single file.
Describe how the union filesystem stacks those layers into the one filesystem your container sees, plus a thin writable layer on top.
Predict which layers a docker build will reuse from cache and which it will rebuild — and why.
Order a Dockerfile so rarely-changing instructions sit above frequently-changing ones, making the cache work for you.
Read docker history and docker image ls output to see the layers and sizes of any image.

The Lesson

An image is a stack of layers, not one file

When you build an image, every instruction in the Dockerfile produces one layer — a read-only snapshot of the filesystem changes that instruction made. COPY pom.xml . adds a layer containing one file. RUN mvn package adds a layer containing the compiled jar. The image is the ordered stack of all those layers.

FROM eclipse-temurin:21-jre   # layer 0 — the base (Java runtime)
WORKDIR /app                  # layer 1 — metadata, sets the working dir
COPY pom.xml .                # layer 2 — your dependency list
RUN mvn dependency:go-offline # layer 3 — every downloaded dependency
COPY src ./src                # layer 4 — your source code
RUN mvn package               # layer 5 — the built jar

Six instructions, six layers (give or take — some instructions like WORKDIR only add metadata). That FROM eclipse-temurin:21-jre is not magic either — it is itself a stack of layers someone else built and published. Your image just stacks more layers on top of theirs.

flowchart TD
  L5["RUN mvn package — the jar"]
  L4["COPY src ./src — your code"]
  L3["RUN dependency go offline — deps"]
  L2["COPY pom.xml — dep list"]
  L1["WORKDIR /app"]
  L0["FROM temurin 21 jre — base"]
  L5 --> L4 --> L3 --> L2 --> L1 --> L0

Read it bottom-up: the base is the foundation, each instruction stacks one more read-only layer on top.

Layers are content-addressed, cached, and shared

Each layer gets an ID derived from its content — a hash of what is inside it (this is what “content-addressed” means). Two layers with identical content have the same ID and are stored on disk exactly once.

That has a huge consequence: if you build two different images that both start FROM eclipse-temurin:21-jre, the base layers are shared on disk. You do not pay for that 200 MB twice. Pull ten images built on the same base and the base is downloaded once.

docker image ls

REPOSITORY        TAG       IMAGE ID       SIZE
splitease-api     latest    a1b2c3d4e5f6   412MB
eclipse-temurin   21-jre    9f8e7d6c5b4a   274MB

That SIZE column lies a little — it is the logical size of each image counted alone. Because splitease-api and eclipse-temurin share the base layers, the actual disk used is far less than 412 + 274. docker system df shows the real shared/reclaimable totals.

Why this matters in a real job: when your CI pushes a new image after a one-line fix, only the changed layers get uploaded to the registry — the shared base is already there. Layer sharing is why pushing a “new” image is fast and cheap. Break layer sharing (covered below) and every deploy ships hundreds of MB it did not need to.

The union filesystem stacks layers into one

Layers are separate read-only directories on disk. Your container does not see a stack — it sees one normal filesystem with /app, /usr, /etc, everything in its place. That illusion is the union (overlay) filesystem: it merges all the read-only layers into a single view, with upper layers shadowing files of the same path in lower layers.

On top of that read-only stack, a running container gets one thin writable layer. Every file the container creates or modifies at runtime — log files, an uploaded receipt, a row written to a local SQLite file — lands in that writable layer. The layers below stay untouched and read-only.

flowchart TD
  W["Writable layer — added when the container RUNS"]
  RO["Read only image layers — shared, never modified"]
  W --> RO

That thin writable layer is the heart of the next two modules. It is per-container and it is destroyed when the container is removed — which is exactly why data written inside a container vanishes, and why you will reach for volumes. We cover that in running-containers and data-and-networking. For now, hold one fact: the image is read-only; the container scribbles on a throwaway top layer.

The build cache — the core skill of this module

Here is the rule that decides whether your builds are fast or painfully slow:

When you run docker build, Docker walks the Dockerfile top to bottom. For each instruction it asks: is this exact instruction, with the exact same inputs, already built? If yes, it reuses the cached layer (CACHED in the output) and moves on. The moment one layer is invalid, that layer and every layer below it rebuild.

“Same inputs” matters. For a RUN line, the input is the command text. For a COPY src ./src, the input is the contents of the files being copied — change one .java file and that COPY layer’s input changed, so it is invalid, so it and everything below rebuild.

Cache invalidation flows downward only. A changed layer never invalidates the layers above it — only the ones below. That single fact is the whole game, and it dictates Dockerfile order: put the stuff that rarely changes (your dependency list, your dependency download) above the stuff that changes constantly (your source code). Then a normal code edit invalidates only the cheap layers near the bottom and keeps the expensive dependency download cached.

Toggle the two modes below. Same app, same one-line code edit, two different Dockerfile orderings. Watch which layers say CACHED vs REBUILD, and watch the rebuild-time counter.

Read what just happened:

deps first (good): COPY pom.xml then RUN dependency:go-offline sit above COPY src. You edited a .java file, so only COPY src and mvn package below it rebuild. The 90-second dependency download stays CACHED. Rebuild ≈ 21s.
code first (bad): COPY . . copies the whole project — including your edited file — into one early layer. That layer changes on every edit, so the dependency download below it is invalidated and re-runs every single time. Rebuild ≈ 111s, and 90 of those seconds were pure waste.

Same app. Same edit. The only difference is the order of two instructions. That is the entire lesson: order rarely-changing instructions before frequently-changing ones.

Make the layers visible: `docker history`

You do not have to guess what layers an image has — ask it.

docker history splitease-api:latest

IMAGE          CREATED         CREATED BY                                      SIZE
a1b2c3d4e5f6   2 minutes ago   RUN mvn package                                 64MB
<missing>      5 minutes ago   COPY src ./src # buildkit                       12kB
<missing>      5 minutes ago   RUN mvn dependency:go-offline                   138MB
<missing>      5 minutes ago   COPY pom.xml . # buildkit                       4kB
<missing>      6 minutes ago   WORKDIR /app                                    0B
<missing>      6 minutes ago   /bin/sh -c #(nop) ... eclipse-temurin:21-jre    274MB

Read top-to-bottom is newest-to-oldest (the reverse of the Dockerfile). Each row is one layer, with the instruction that created it and the size it added. <missing> IDs are normal — they are intermediate layers without their own tag. This is how you find the fat layer when an image is too big: scan the SIZE column. Here the dependency layer (138MB) and the base (274MB) dominate, which is expected for a JVM app.

Image vs container, one more time — now with layers

	Image	Container
What it is	A stack of read-only layers	An image plus one writable top layer + a running process
Mutable?	No — every layer is read-only	Yes — but only the thin writable top layer
On disk	Layers shared between images	Its own writable layer, destroyed on `rm`
Count	One image	Many containers from one image, each with its own writable layer

Ten containers from splitease-api share the same read-only image layers on disk and each get their own tiny writable layer. That is why starting a container is near-instant and cheap — Docker is not copying 412 MB, it is stacking one empty writable layer on top of layers that already exist.

Build This

You will build a tiny image twice, change one source line in between, and read the build output to see the cache work. No Spring Boot needed for the mechanic — a one-file Node app makes the cache hits unmistakable. (The same ordering rule is what you will apply to SplitEase’s real Dockerfile in dockerfile-deep-dive.)

1. Make a folder with three files.

package.json:

{ "name": "cache-demo", "version": "1.0.0", "dependencies": { "express": "4.19.2" } }

server.js:

const express = require('express');
const app = express();
app.get('/', (req, res) => res.send('split v1'));
app.listen(3000, () => console.log('up on 3000'));

Dockerfile — note the order: dependency manifest copied and installed before the source:

FROM node:20-slim
WORKDIR /app
COPY package.json .
RUN npm install
COPY server.js .
CMD ["node", "server.js"]

2. Build it the first time.

docker build -t cache-demo .

Everything runs fresh. npm install actually downloads express — note the rows that do real work.

3. Edit ONE source line. In server.js, change 'split v1' to 'split v2'. Do not touch package.json.

4. Build again.

docker build -t cache-demo .

Now read the output. You should see something like:

 => CACHED [2/5] WORKDIR /app
 => CACHED [3/5] COPY package.json .
 => CACHED [4/5] RUN npm install
 => [5/5] COPY server.js .
 => exporting to image

5. Inspect the layers.

docker history cache-demo:latest

Definition of done: looking at the second build’s output, you can point at each step and say why it was CACHED or rebuilt. Specifically you can explain: COPY package.json and RUN npm install were CACHED because their inputs (package.json and the install command) did not change; COPY server.js rebuilt because the file you copy changed; and anything below a rebuilt layer also rebuilds. Bonus: move COPY server.js . above RUN npm install, rebuild, and watch npm install lose its cache and re-run — proof that order is everything.

Where to Practice

What	Where	Time
Build, edit, rebuild — see CACHED in a throwaway browser VM	labs.play-with-docker.com	25 min
The build cache + layer concepts, official	docs.docker.com (Get Started → “Build with cache”)	20 min
`docker history` / `docker image ls` / `docker system df` on your own machine	your terminal	15 min
Free guided layer + cache labs	KodeKloud (free Docker labs tier)	20 min
Order the real SplitEase Dockerfile for cache	your SplitEase repo	20 min

How to Practice

Do the build-edit-rebuild loop until reading the CACHED lines is automatic: build once, change one line, build again, and before you hit enter predict out loud which steps will say CACHED. Then deliberately break it — reorder instructions so a code edit invalidates the dependency install — and watch the rebuild time balloon. The skill is not memorising; it is being able to look at any Dockerfile and instantly spot the line that kills the cache.

Check Yourself

1. What does each instruction in a Dockerfile produce?

One read-only layer — a snapshot of the filesystem changes that instruction made. The image is the ordered stack of all those layers.

2. Two images both start FROM the same base. How many copies of the base layers are on disk?

One. Layers are content-addressed (identified by a hash of their content), so identical layers are stored once and shared between images.

3. The union filesystem merges read-only layers into one view. What does a running container add on top?

One thin writable layer. Every file the container creates or changes at runtime lands there; the layers below stay read-only.

4. You edit one source file and rebuild. Which layers rebuild?

The layer that copies the changed file, and every layer below it. Cache invalidation flows downward only — layers above the change stay cached.

5. Why put COPY of the dependency manifest and the dependency install ABOVE COPY of your source code?

Because source changes constantly and dependencies rarely. With deps above source, a code edit invalidates only the cheap layers near the bottom and keeps the expensive dependency download cached. Reverse the order and every edit re-downloads everything.

6. Why is COPY . . early in a Dockerfile a classic cache-killer?

It copies the whole project — including files you edit constantly — into one early layer. That layer changes on nearly every build, invalidating every layer below it, including the expensive dependency install.

7. Which command shows you the layers of an image and the size each one added?

docker history <image>. Rows read newest-to-oldest; <missing> IDs are normal intermediate layers. Scan the SIZE column to find the fat layer.

8. The SIZE in `docker image ls` adds up to more than your actual disk usage. Why?

Because it counts each image’s logical size in isolation, but images sharing layers store those layers once. docker system df shows the real shared and reclaimable totals.

Still Unclear?

Paste any of these to Claude to go deeper:

“Here is my Dockerfile: [paste]. Walk through it line by line and tell me, for a typical one-line source code edit, exactly which layers would be CACHED and which would rebuild, and why. Then suggest a reordering that maximises cache reuse.”
“Explain content-addressed storage in Docker: how is a layer’s ID derived, and how does that let two images share a base layer on disk? Use a concrete two-image example.”
“I changed nothing but my docker build re-downloads all dependencies every time. Give me a checklist of the common reasons the build cache gets invalidated and how to diagnose each from the build output.”

Why AI Can’t Do This For You

AI can generate a Dockerfile, but it cannot watch your build output and tell you why your cache keeps missing on your machine with your edit patterns — that is reading real output against a mental model of layer invalidation, and the model only sticks once you have caused a slow rebuild and fixed it yourself. The judgment of where the change-boundary sits in a Dockerfile — which lines change daily, which change monthly — depends on how your team actually works, and you are the one who knows that. Build a slow image, feel the 90 wasted seconds, reorder two lines, and watch it drop to 5 — that muscle memory is yours, not the model’s.

Module done? Add it to today’s tracker