Bash scripts that don't surprise you

Bash is everywhere in DevOps. CI pipelines, Dockerfiles, deployment scripts, cron jobs, quick automations. It's the glue that holds infrastructure together.

Bash has hidden dangers. By default, it keeps running after errors, treats undefined variables as empty, and splits words in unexpected ways. A script that works fine on your laptop might accidentally delete the wrong folder in production if a variable is empty.

Here are the habits that make my scripts boring, in the best way.

The first lines of every script

#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

What they do:

• set -e: stop the script when a command fails, instead of carrying on.
• set -u: treat undefined variables as an error, instead of an empty string.
• set -o pipefail: a pipeline fails if any command in it fails, not just the last one.
• IFS=$'\n\t': split words only on newlines and tabs, not spaces. Fewer surprises with filenames containing spaces.

Why set -u matters so much, in one example:

rm -rf "$BUILD_DIR/"*

If BUILD_DIR is accidentally unset, without set -u this becomes rm -rf /*. With set -u, the script stops with an error. That single line has saved many systems.

(set -e has some well-known edge cases, like commands in conditions or subshells. It's not perfect, but it's a big improvement over the default behavior.)

Bash assumes you know what you're doing. Good scripts don't make that assumption about anyone, including their author.

ShellCheck

ShellCheck is a linter for shell scripts, and the single most useful tool for writing better Bash. It catches unquoted variables, wrong comparisons, useless cats, portability issues and many subtle bugs, with explanations for each warning.

shellcheck deploy.sh

Add it to your editor and to CI. I treat ShellCheck warnings like compiler errors.

Habits that prevent most bugs

Quote your variables. Always. "$file", not $file. Unquoted variables get split on spaces and expanded as wildcards.

Use [[ ]] for tests in Bash, not [ ]. It's safer and supports more features.

if [[ -z "${ENVIRONMENT:-}" ]]; then
  echo "ENVIRONMENT must be set" >&2
  exit 1
fi

Provide defaults and required checks with parameter expansion:

REGION="${REGION:-eu-central-1}"       # default value
: "${API_TOKEN:?API_TOKEN is required}" # fail with a message if unset

Use local in functions so variables don't leak.

Write errors to stderr (>&2) and exit with non-zero codes on failure.

Clean up with trap, even when the script fails:

tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT

Use $(...) instead of backticks for command substitution.

Avoid parsing ls. Use globs or find instead.

A template I reuse

#!/usr/bin/env bash
set -euo pipefail

usage() {
  cat <<EOF
Usage: $(basename "$0") -e <environment> [-d]
  -e  target environment (staging|production)
  -d  dry run
EOF
}

log() { printf '%s [%s] %s\n' "$(date -u +%FT%TZ)" "${1}" "${2}" >&2; }

main() {
  local env="" dry_run=false

  while getopts ":e:dh" opt; do
    case "$opt" in
      e) env="$OPTARG" ;;
      d) dry_run=true ;;
      h) usage; exit 0 ;;
      *) usage; exit 1 ;;
    esac
  done

  [[ "$env" =~ ^(staging|production)$ ]] || { usage; exit 1; }

  log INFO "Deploying to $env (dry run: $dry_run)"
  if [[ "$dry_run" == false ]]; then
    : # real work here
  fi
}

main "$@"

It has argument parsing, validation, logging with timestamps, a dry-run mode, and a main function. Copy it, delete what you don't need.

When Bash is the wrong tool

Bash is great for gluing commands together. It's bad at almost everything else. I switch to Python (or Go) when a script:

• Is longer than about 100 lines.
• Needs real data structures (nested data, maps of lists).
• Parses JSON or YAML beyond a simple jq call.
• Talks to APIs with authentication, pagination and retries.
• Needs error handling more subtle than "stop on failure".
• Needs tests.
• Will be maintained by many people for a long time.

Python with a proper CLI library, or a small Go tool, is much easier to read, test and maintain at that point.

In CI pipelines

CI YAML often contains multi-line shell scripts. The same rules apply:

• Many CI systems don't enable set -euo pipefail by default. Add it, or set the shell options for the whole job.
• Move long scripts out of YAML into files in the repository, where ShellCheck can lint them and you can run them locally.

So, what's the one thing to do?

Add set -euo pipefail to the top of every script, and run ShellCheck on all of them. Those two habits prevent most of the disasters I've seen from shell scripts.

My take: Bash is a sharp knife. Use it for small cuts, keep it sharp with ShellCheck, and switch to a proper tool when the job gets bigger.

— Desly