Skip to content
SeeItFlow
Interactive guide · 13 chapters · ~34 min read

Docker Production Engineering

An image that runs on your laptop tells you almost nothing about whether it will run safely under production traffic. This is the practical playbook for image size, security, networking, storage, health, and the common ways containers fail in production.

Mangesh MandlikLast updated:
01 · Smaller, faster

Image Optimization

A smaller image pulls faster, starts faster, and has a smaller attack surface.

Every unnecessary file in an image is bandwidth on every pull and disk on every host. The biggest wins usually come from choosing a minimal base image (-slim or -alpine variants, or distroless images), removing build-only tooling from the final image, and cleaning up package manager caches within the same RUN instruction that installed them (so the cleanup doesn't just become its own layer on top).

Minimal base images

Alpine-based images are typically much smaller than full Debian/Ubuntu images (compare the sizes for your own base image and tag). Distroless images go further, shipping only the application and its runtime — no shell, no package manager.

.dockerignore

Excludes files (node_modules, .git, local env files) from the build context, keeping builds fast and preventing accidental inclusion of secrets or bloat.

02 · Build tools stay out of production

Multi-Stage Builds

Multi-stage builds let you use a full toolchain to compile or build your app, then copy only the final artifact into a clean, minimal runtime image.

dockerfileTwo stages: one full of build tools, one lean runtime that only copies the compiled output. Manifests are copied before source so the dependency layer stays cached.
FROM node:22 AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:22-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY --from=build /app/dist ./dist
CMD ["node", "dist/server.js"]

The ordering is deliberate: layers are cached top-down, and one changed layer invalidates every layer after it. Because package*.json is copied and installed *before* COPY . ., an edit to application code reuses the cached npm ci layer and only re-runs the build; the install layer is rebuilt only when the manifests or lockfile change. Put the least frequently changing inputs first. Pin a currently supported Node LTS tag (22 here) and revisit it as the Node release schedule moves.

textIllustrative BuildKit progress for a code-only change (abridged, not captured from a real build).
#5 [build 3/6] COPY package*.json ./        CACHED
#6 [build 4/6] RUN npm ci                   CACHED   <- code-only change: deps layer reused
#7 [build 5/6] COPY . .                     (re-run)  <- source changed, cache busted from here
#8 [build 6/6] RUN npm run build            (re-run)
03 · Reducing blast radius

Container Security

A container is a process on the host — if it's compromised, the goal of good security practice is to limit what that process can do.

Run as non-root

By default, container processes run as root (UID 0) inside the container with a reduced set of Linux capabilities; without user-namespace remapping or rootless mode, that is the same UID 0 as on the host. Adding a USER instruction to run as an unprivileged user drastically reduces the impact of a container breakout.

Read-only filesystems

Running a container with --read-only prevents an attacker (or a bug) from writing anywhere except explicitly mounted, writable paths — a strong default for stateless services.

Image scanning

Automated vulnerability scanning (Trivy, Docker Scout, Snyk) checks base images and dependencies against known CVE databases as part of CI, before an image is ever deployed.

Minimizing capabilities

Linux capabilities (--cap-drop, --cap-add) let you strip privileges a container doesn't need (e.g. dropping NET_RAW if it never needs raw sockets) instead of granting the full default set.

04 · Sharing the host fairly

Resource Limits

Without limits, a single misbehaving container can starve every other container on the same host of CPU or memory.

Docker enforces resource boundaries using cgroups: --memory caps how much RAM a container can use before it's OOM-killed, and --cpus caps how many CPU cores' worth of time it can consume. In orchestrated environments (Kubernetes, ECS), these become resources.limits / resources.requests in the deployment spec.

Requests vs limits

A "request" is what the scheduler reserves for the container (guaranteed minimum); a "limit" is the hard ceiling it can't exceed. Setting requests accurately is what makes bin-packing and cluster capacity planning work.

05 · Production topologies

Container Networking in Production

Production networking layers Docker's basic bridge networks with service discovery, load balancing, and network policy.

Overlay networks

Extend container networking across multiple hosts, so containers on different machines in a cluster can still reach each other as if on the same network — the basis of Swarm's and Kubernetes' cross-node networking.

Service discovery

In orchestrated environments, containers don't reach each other by container name directly — a stable service name resolves (via DNS) to whichever healthy instance is currently running, even as containers are replaced.

Network segmentation

Separate networks for tiers (public-facing, internal services, database) so a compromised public-facing container can't directly reach the database network.

06 · Storage at scale

Persistent Storage in Production

Local named volumes work great on a single host — production usually needs storage that survives a host failure too.

Local volumes

Fast, simple, but tied to a specific host — if the host dies, the volume (and its data) may be unavailable unless separately backed up or replicated.

Network-attached / cloud volumes

Volume drivers backed by network storage (EBS, NFS, cloud block storage) let a volume follow a container even if it's rescheduled to a different host.

Backups

Volumes are not a backup strategy by themselves — regular snapshots or dumps to durable, versioned storage are still required for real disaster recovery.

07 · Seeing what happened

Docker Logging

The standard Docker logging convention is simple: write to stdout/stderr, and let the platform handle collection.

Docker captures anything a container writes to stdout/stderr via its logging driver (json-file by default, but often swapped for a driver that ships logs to a central system — Fluentd, Loki, CloudWatch, Datadog). This means the application itself shouldn't manage log files or rotation — it just prints, and the platform is responsible for storage, rotation, and searchability.

08 · Is it actually working?

Health Checks

A container can be "running" while the application inside it is completely unresponsive — health checks close that gap.

dockerfileDocker polls this command; three consecutive failures mark the container unhealthy.
HEALTHCHECK --interval=10s --timeout=3s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1

A HEALTHCHECK instruction defines a command Docker runs periodically inside the container. If it fails enough times in a row, the container is marked unhealthy — which some orchestrators (for example Docker Swarm) use as a signal to stop routing traffic to it or to replace it. Kubernetes ignores the Dockerfile HEALTHCHECK and uses its own liveness, readiness and startup probes.

09 · Automatic recovery

Restart Policies

A restart policy tells the Docker daemon what to do when a container's main process exits.

no

Never automatically restart (the default) — useful for one-off jobs that should run once and stop.

on-failure

Restart only if the process exits with a non-zero (error) status — good for services that should recover from crashes but not endlessly retry a deliberate stop.

always / unless-stopped

Restart regardless of exit status, including after a daemon restart — appropriate for long-running services that should always be up unless explicitly stopped by an operator.

10 · Naming things is hard

Image Versioning

Image tags are how you control exactly which build of your application is running where.

Immutable, explicit tags

Tag images with something unique and traceable — a semantic version (app:2.4.1) or a build/commit hash (app:sha-abc1234) — so a running container's exact code is always identifiable.

Digest pinning

For maximum reproducibility, reference an image by its content digest (app@sha256:...) rather than a tag — digests can't be reassigned, tags can.

Key takeaways
  1. 01Multi-stage builds and minimal base images are the two biggest levers for smaller, safer images.
  2. 02Non-root users, read-only filesystems, and image scanning meaningfully reduce blast radius.
  3. 03Resource limits (cgroups) prevent one container from starving its neighbors.
  4. 04Health checks distinguish 'process alive' from 'ready for traffic' — restart policies handle the former.
  5. 05Never rely on the latest tag in production — use explicit, immutable versions.
11 · Watching the fleet

Monitoring

Beyond individual container health checks, production Docker fleets need visibility into resource usage, restart counts, and request-level metrics.

cAdvisor / Docker stats

Container-level CPU, memory, network and disk I/O metrics — the raw material for dashboards and alerts.

Application metrics

Request rate, error rate, and latency exported by the application itself (often via Prometheus) — usually more actionable than infrastructure metrics alone.

Restart-count alerting

A container restarting repeatedly is a strong leading indicator of a problem, even before users notice — alerting on restart count catches issues restart policies would otherwise hide.

12 · Things that go wrong

Common Production Issues

Most Docker production incidents fall into a small, recognizable set of categories.

OOM kills

A container silently dies with exit code 137 (SIGKILL) — commonly because it exceeded its memory limit and the OOM killer fired — often mistaken for a crash rather than a resource-limit problem.

Disk pressure from logs or dangling images

Unrotated logs or an accumulation of unused images/layers (docker system prune territory) slowly fill the host's disk until everything fails at once.

Port conflicts

Two containers trying to publish the same host port fail to start — a common source of confusing 'address already in use' errors in local dev and CI.

Zombie processes / PID 1 issues

A container's main process (PID 1) has special responsibilities (reaping child processes, forwarding signals) that many application binaries don't handle correctly, leading to containers that don't shut down cleanly on docker stop.

13 · When things break

Debugging Containers

Debugging a container is debugging a process you usually can't (and shouldn't) modify in place — so the toolkit is built around observing it from outside.

bashThe four commands that answer 90% of 'why is this container misbehaving?' questions.
docker logs -f app-1
docker exec -it app-1 sh
docker inspect app-1
docker stats app-1

docker logs

The first thing to check — most application-level errors surface here immediately.

docker exec

Open a shell inside a running container to inspect files, environment variables, or run diagnostic commands live.

docker inspect

Dumps the full JSON configuration of a container — mounts, network settings, environment, restart count, exit code — invaluable for "why did this restart" questions.