Staying Secure & Up to Date – Part 7: Watchtower – Auto-Updates That Work (and Three Pitfalls That Lurk)
Watchtower is a small helper container that regularly checks whether newer images exist for your running containers and restarts them automatically. Combined with the weekly scan from part 5, you get a comfortable split:
• Watchtower keeps your latest containers fresh automatically.
• Trivy + KEV (part 5) tell you where something is still stuck.
Setup in four steps
1) The compose file (watchtower/docker-compose.yml):
services:
watchtower:
image: ghcr.io/containrrr/watchtower:latest
container_name: watchtower
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- DOCKER_API_VERSION=1.41
- WATCHTOWER_SCHEDULE=0 0 4 * * *
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_LABEL_ENABLE=true
2) Start it: docker compose up -d
3) Opt in per container – only containers carrying this label get updated:
# in the respective service, e.g. your node-exporter:
labels:
- com.centurylinklabs.watchtower.enable=true
4) Verify: docker logs watchtower (should show something like “Only
checking containers using enable label”)
LABEL_ENABLE, Watchtower
touches every container – including ones you intentionally pinned or that have
their own update mechanism (e.g. Nextcloud AIO containers). With the label you decide per service
who gets updated and who doesn’t.
Pitfall 1: The stale-source problem
The classic that caught us too: the Docker Hub image containrrr/watchtower:latest has
not been maintained for years – using it means running an ancient build forever. Use
the maintained image from the project’s own container registry instead:
image: ghcr.io/containrrr/watchtower:latest
Rule: before you use any “:latest” image, quickly check that the source is still alive (a glance at the release page/GitHub never hurts).
Pitfall 2: “client version 1.25 is too old”
Right after setup, Watchtower spins in a restart loop and spits out:
Error response from daemon: client version 1.25 is too old. Minimum supported API version is 1.40 …
This means your Docker version only accepts newer APIs – your daemon’s minimum version has been
raised. The fix is an env variable in the compose file (see above): DOCKER_API_VERSION=1.41.
Now Watchtower speaks a language the daemon accepts and runs.
Pitfall 3: “It updated EVERYTHING” or “it did NOTHING”
Two sides of the same coin:
• Without LABEL_ENABLE, Watchtower touches every container – including
managed stacks (Nextcloud AIO, tools with their own update rhythm) that have their own
cycle. Result: fighting automations, possibly a broken service. → Always use the opt-in label.
• With LABEL_ENABLE but no labels, nothing happens at all – you think
“automatically up to date”, yet no container carries a label. → After setup, run docker ps
and see which services are labelled.
And an honest limit: pinned versions (e.g. traefik:3.7.14 or
cadvisor:0.60.5) are deliberately not updated by Watchtower – a tag is a fixed state, not
“latest”. That’s exactly why the part-5 scan remains your control instrument.
Pro Corner
• Schedule instead of interval: WATCHTOWER_SCHEDULE=0 0 4 * * * (cron
syntax) is more deterministic than an interval; the project’s docs are the source of truth for the many
WATCHTOWER_* options.
• Scope concept: with --scope plus
com.centurylinklabs.watchtower.enable=false you can define exceptions; a whitelist of
container names is also possible.
• Cleanup: WATCHTOWER_CLEANUP=true removes old image states after an
update – saves space, but keep a previous image for rollback (snapshot).
• What Watchtower cannot do: check CVE scores, match KEV, or update host packages.
That’s Trivy/KEV (part 5) and apt upgrade (part 6) – the parts complement each other, they
don’t replace each other.
Why this matters: automated updates save time – but only if you control what gets automated and the source stays alive. That’s what the three pitfalls are about.
Next part: Part 8 – Routine & staying calm: your monthly check and handling alerts.