09 — Environment variables and secrets

Read this first: this lesson contains the nastiest trap in the course — a way to set a perfectly good secret and have your application receive a fraction of it, with no error anywhere. It really happened to this repository, in production. It also covers ${VAR:?} as a machine-readable contract, and why the shell environment quietly beats your env file.

Time: about 45 minutes. Assumes lesson 08.

Configuration comes from outside the image

The same image runs on your laptop, on staging, and in production. What differs is configuration, and it arrives as environment variables. That is why the toy app reads APP_SECRET, DATABASE_URL and GREETING from the environment rather than a config file baked into the image. In Spring Boot that is application.yml referring to ${APP_SECRET:} and friends — the file is in the jar, but every value in it comes from outside.

Compose gives you three ways to supply them, and the interactions between them are where the surprises live:

Source	Wins against
environment: in the compose file	the env file
--env-file / the default .env	nothing
Your actual shell environment	everything above

That last row is the one people trip over. Read it again: an exported shell variable overrides your .env file.

${VAR:?} — a requirement the machine can read

In a compose file you can mark a variable as mandatory:

    image: ghcr.io/you/greeter-api:${IMAGE_TAG:?set by deploy.sh — do not run compose up by hand}

Predict: what happens if you run docker compose up without IMAGE_TAG set?

error while interpolating services.app.image: required variable IMAGE_TAG is missing a value:
set by deploy.sh — do not run compose up by hand

Compose refuses to start and prints your message. Compare that with the alternative — a default that silently resolves to latest and deploys who-knows-what.

This is worth more than it looks. deploy/preflight.sh greps the compose files for that exact :? pattern to discover what must be configured:

for v in $(grep -ohE '\$\{[A-Z_]+:\?' "$1" 2>/dev/null | sed 's/\${//;s/:?//' | sort -u); do

So the requirement is not documented in two places that can drift. The compose file is the documentation, and the checker reads it. Related forms:

Form	Means
${VAR:?message}	Required. Fail with message if unset or empty.
${VAR:-default}	Use default if unset or empty.
${VAR}	Use it, or empty. Usually the wrong choice for anything important.

See what compose actually resolved

docker compose config

This prints the fully-resolved file: every variable substituted, every default applied. It is the first thing to run when a stack behaves as though it is configured differently from what you wrote, because it shows you what compose actually believes.

Break it on purpose: the $ that eats your secret

Here is the trap. Put a secret in a .env file, and give it a $ in the middle — as a strong random secret very well might have:

APP_SECRET=your-secret$abcdef123456

That is 24 characters. Predict: what does the container receive?

Run it and read the actual value the container got:

--- container actually receives: ---
your-secret

Eleven characters. Compose interpolated $abcdef123456 as a variable, found nothing by that name, and substituted empty. No warning. No error. The stack came up — and everything after the $ is simply gone.

Now the part that makes it genuinely dangerous. Predict: wrapping the value in double quotes fixes it, right?

It does not. Write that same value in double quotes and the container still receives:

--- container actually receives: ---
your-secret

Only single quotes protect it. With the value wrapped in single quotes instead:

--- container actually receives: ---
your-secret$abcdef123456

All 24 characters, intact. To summarise the three cases:

How the value is written in the env file	What the container receives
Unquoted	your-secret — truncated
Double quotes	your-secret — still truncated
Single quotes	your-secret$abcdef123456 — correct

(The quoted forms are described rather than printed on purpose. This site's own CI scans the published pages for anything shaped like an assigned credential, and a secret-looking variable name followed by a quoted value trips it — including in a lesson explaining the trap, which is a small illustration of lesson 22's point that a gate checking the output catches what a denylist would miss.)

Why this is worse than a broken deploy

A crash is a good outcome — it is loud. This is quiet.

In this repository the same bug hit twice, and it is documented in the code both times:

• A JWT_SECRET was truncated this way, so the application signed authentication tokens with a three-character key. Nothing failed. Logins worked. The system was simply far less secure than everyone believed.
• A bcrypt password hash was truncated, and a prefix check that only looked at the start of the string passed it straight through into production.

The toy app is built to reproduce the shape of this. Its APP_SECRET must be at least 16 characters, so a truncated secret produces:

Caused by: java.lang.IllegalStateException: APP_SECRET must be at least 16 characters (got 11).
If you set it in an env file and it contains a $, single-quote the value.

Which is a message about length, pointing nowhere near the actual mistake — a $ in an env file. That gap between symptom and cause is exactly why this is worth a lesson.

Fix it — and write the guard

The rule is simple: single-quote every value in an env file. But rules that rely on remembering are not controls, so the real deploy script checks. From deploy/deploy.sh:

check_env_interpolation() {
  local file="$1" line key val bad=""
  while IFS= read -r line || [ -n "$line" ]; do
    case "$line" in ''|'#'*) continue ;; esac
    key=${line%%=*}
    val=${line#*=}
    case "$val" in \'*\') continue ;; esac
    case "$val" in *\$[A-Za-z_]*|*\$\{*) bad="$bad $key" ;; esac
  done < "$file"
  [ -z "$bad" ] || die "these values contain an un-single-quoted \$:$bad"
}

Everything in it is from lesson 03 and lesson 02:

• ${line%%=*} strips the longest match of =* from the end, leaving the key.
• ${line#*=} strips the shortest match of *= from the start, leaving the value.
• case "$val" in \'*\') — a value that is already single-quoted is safe, skip it.
• The final case flags any remaining $ followed by a variable-looking name.
• while IFS= read -r line || [ -n "$line" ] handles a file whose last line has no trailing newline.

Note that it checks every variable, not a list of known-sensitive ones. A guard that only protects the secrets you remembered to list is a guard that will miss the next one.

The other half: OS environment beats --env-file

Predict: your .env says IMAGE_TAG=v2. You previously ran export IMAGE_TAG=v1 in this shell. Which one does compose use?

v1. The exported shell variable wins.

This is why the real deploy script never relies on the file alone when it matters — it pins the value explicitly for that one command:

IMAGE_TAG="$new" compose up -d

VAR=value command sets the variable for only that command, which is both explicit and self-cleaning.

Never commit a secret

.env belongs in .gitignore; .env.example — with placeholder values only — belongs in git. In this repository the production values live in a GitHub Secret and are written to the server by the deploy workflow. You will do the same thing in lesson 15 and after.

There is also a scanner: .github/workflows/gitleaks.yml runs on every push and pull request, precisely because "don't commit secrets" is not a control either.

Recap

• ${VAR:?message} makes a requirement machine-readable — and preflight.sh really does grep for it rather than keeping a second list.
• Compose interpolates $ inside env-file values. Double quotes do not help; only single quotes do. An unquoted $ silently truncates the value and nothing reports it.
• The shell environment overrides --env-file, so pin values you care about with VAR=value command.
• docker compose config shows you what compose actually resolved. Use it before guessing.

Next: 14 — YAML without tears.