Bash · Software Engineering Wiki

Cheatsheet #

Task	Snippet
Fail fast	`set -euo pipefail`
Script’s own directory	`cd "$(dirname "${BASH_SOURCE[0]}")"`
Default if unset	`${VAR:-default}`
Fail if unset	`${VAR:?message}`
Strip suffix	`${file%.txt}`
Strip directory	`${path##*/}`
Replace all	`${str//old/new}`
Lowercase	`${str,,}`
Length	`${#str}`, `${#array[@]}`
Command output	`out=$(cmd)`
Arithmetic	`(( count++ ))`, `n=$(( a * b ))`
Loop over files safely	`for f in *.log; do [ -e "$f" ] \|\| continue; done`
Read a file line by line	`while IFS= read -r line; do ...; done < file`
Array of lines	`mapfile -t lines < file`
Temp file that cleans up	`t=$(mktemp); trap 'rm -f "$t"' EXIT`
Is the command available	`command -v jq >/dev/null`
Timestamped name	`printf -v f 'dump-%(%Y%m%dT%H%M%S)T.sql' -1`

Quoting #

Unquoted expansion is split on $IFS and then glob-expanded. That is the origin of nearly every script bug involving spaces, asterisks or empty values.

rm $file        # two arguments if $file contains a space; everything if it is "*"
rm "$file"      # one argument, always

"$@"            # each positional argument, individually quoted — what you want
"$*"            # all arguments joined into one string
'$literal'      # single quotes: no expansion at all
"$(cmd)"        # command substitution, quoted

Quote every expansion unless you deliberately want splitting. shellcheck finds the ones you miss.

Strict mode #

#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

Option	Effect	Trap
`-e`	Exit on unhandled non-zero status	Ignored inside `if`, `&&`, `\|\|` and any tested command
`-u`	Unset variable is an error	Use `${VAR:-}` for optional ones
`-o pipefail`	Pipeline fails if any stage fails	Without it `false \| true` succeeds
`-x`	Trace every command	`set -x` around one block, not the whole script

set -e does not fire inside a function whose result is tested, and it does not fire for the last command of a pipeline without pipefail. For anything with real consequences, check status explicitly:

if ! output=$(risky_command 2>&1); then
  printf 'failed: %s\n' "$output" >&2
  exit 1
fi

Parameter expansion #

Expansion is string surgery without spawning sed or cut, and it is measurably faster in a loop.

file=/var/log/nginx/access.log.1

${file##*/}        # access.log.1   — longest prefix removed (basename)
${file%/*}         # /var/log/nginx — shortest suffix removed (dirname)
${file%.*}         # ...access.log  — drop last extension
${file##*.}        # 1              — last extension only

name="deploy-prod-api"
${name//-/_}       # deploy_prod_api   replace all
${name/prod/stag}  # deploy-stag-api   replace first
${name:0:6}        # deploy            substring
${name: -3}        # api               from the end (note the space)
${#name}           # 15                length
${name^^}          # DEPLOY-PROD-API   uppercase
${name,,}          # lowercase

${VAR:-default}    # use default if unset or empty
${VAR:=default}    # ...and assign it
${VAR:?message}    # abort with message if unset or empty
${VAR:+value}      # value only if VAR is set

${VAR-default} without the colon treats an empty string as set. That distinction matters for flags that may legitimately be empty.

Tests #

[[ -f "$path" ]]        # regular file exists
[[ -d "$path" ]]        # directory
[[ -s "$path" ]]        # exists and is non-empty
[[ -r "$path" ]]        # readable
[[ -n "$str" ]]         # non-empty string
[[ -z "$str" ]]         # empty string
[[ "$a" == "$b" ]]      # string equality
[[ "$a" == prefix* ]]   # glob match (unquoted pattern)
[[ "$a" =~ ^[0-9]+$ ]]  # regex, captures land in ${BASH_REMATCH[@]}
(( n > 5 ))             # arithmetic comparison
[[ "$f" -nt "$g" ]]     # newer than

Use [[ ]] in Bash: it does not word-split, handles empty variables without quoting tricks, and supports =~. [ ] is POSIX and needed only in sh scripts.

Loops #

for f in *.log; do
  [ -e "$f" ] || continue          # the glob stayed literal: no matches
  printf '%s\n' "$f"
done

while IFS= read -r line; do        # IFS= keeps leading whitespace, -r keeps backslashes
  printf '%s\n' "$line"
done < input.txt

while IFS=, read -r name port _; do
  printf '%s -> %s\n' "$name" "$port"
done < hosts.csv

find . -name '*.tmp' -print0 | while IFS= read -r -d '' f; do rm -- "$f"; done

for i in {1..10}; do :; done       # brace expansion happens before variables exist
for ((i = 0; i < n; i++)); do :; done

A pipeline into while runs the loop in a subshell, so variables set inside it are lost afterwards. Use redirection or process substitution instead:

count=0
while IFS= read -r _; do (( count++ )); done < <(grep 'ERROR' app.log)
printf '%d errors\n' "$count"      # visible here; it would be zero after a pipe

Arrays #

arr=(one two "three four")
arr+=(five)

"${arr[@]}"        # each element, separately quoted
"${arr[*]}"        # single string joined by the first character of IFS
"${#arr[@]}"       # count
"${arr[2]}"        # index
"${arr[@]:1:2}"    # slice
mapfile -t lines < file            # file into array, no trailing newlines
readarray -t pods < <(kubectl get pods -o name)

declare -A limits=([cpu]=2 [memory]=4Gi)
limits[disk]=100Gi
"${!limits[@]}"    # keys
"${limits[@]}"     # values
[[ -v limits[cpu] ]] && echo set

Associative arrays need declare -A first; without it Bash treats the subscript as arithmetic and everything lands in index 0.

Functions #

usage() {
  cat >&2 <<'EOF'
usage: deploy [-n] <env>
  -n  dry run
EOF
  exit 2
}

deploy() {
  local env=${1:?env required}     # local, or it leaks into the caller
  local -r dry=${2:-false}
  printf 'deploying %s\n' "$env"
  return 0
}

deploy prod || { printf 'failed\n' >&2; exit 1; }

Functions return an exit status, not a value: print the value and capture it with $(...), or assign to a variable the caller names. local is what stops a recursive or repeated call from clobbering state.

Traps and cleanup #

tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT          # runs on normal exit and on error
trap 'printf "interrupted\n" >&2; exit 130' INT TERM
trap 'printf "failed at line %s\n" "$LINENO" >&2' ERR

EXIT fires once, after everything else, which makes it the right place for cleanup. Set it immediately after creating the resource, not at the end of the script.

Input and options #

while getopts ":n:v" opt; do
  case $opt in
    n) name=$OPTARG ;;
    v) verbose=1 ;;
    \?) printf 'unknown option: -%s\n' "$OPTARG" >&2; exit 2 ;;
    :)  printf 'option -%s needs a value\n' "$OPTARG" >&2; exit 2 ;;
  esac
done
shift $((OPTIND - 1))

getopts handles short options only. For long options, parse $1 in a while loop with case, or accept that a wrapper script is the wrong place for a complex interface.

read -r -p 'Continue? [y/N] ' reply
[[ $reply == [yY]* ]] || exit 0

cat <<EOF          # expands variables
host=$HOSTNAME
EOF

cat <<'EOF'        # literal, no expansion
cost=$100
EOF

Redirection #

cmd > out.log 2>&1        # both streams to a file, order matters
cmd 2>&1 | tee out.log    # both streams through a pipe
cmd > /dev/null 2>&1      # discard everything
cmd 2> >(logger -t app)   # stderr to a process
exec 3< file              # open fd 3 for reading
diff <(sort a) <(sort b)  # process substitution: commands as files
printf '%s\n' "$data" | cmd   # avoid `echo` for data you did not write

2>&1 > file sends stderr to the old stdout (the terminal) and only stdout to the file. The order reads right to left.

Oneliners #

# Directory of the running script, symlinks resolved
script_dir=$(cd -- "$(dirname -- "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)

# Require commands up front
for c in jq curl kubectl; do command -v "$c" >/dev/null || { echo "need $c" >&2; exit 1; }; done

# Retry with backoff
for i in {1..5}; do cmd && break || sleep $(( 2 ** i )); done

# Run at most 8 jobs in parallel
printf '%s\n' "${hosts[@]}" | xargs -P8 -I{} ssh {} uptime

# Timeout a command that may hang
timeout 30s curl -fsS https://api.example.com/health

# Lock a script to one instance
exec 9>/var/lock/myjob.lock && flock -n 9 || exit 0

# Trim whitespace without an external command
trim() { local s=$1; s=${s#"${s%%[![:space:]]*}"}; printf '%s' "${s%"${s##*[![:space:]]}"}"; }

# Epoch to local time
date -d "@$epoch" '+%F %T'

# Sum a column
awk '{s += $3} END {print s}' file

# Most frequent values in a column
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head

# Bytes human-readable
numfmt --to=iec-i --suffix=B 1234567

# Every variable in the current shell, sorted
compgen -v | sort

# Is this an interactive shell
[[ $- == *i* ]] && echo interactive

# Print with colour only when attached to a terminal
[[ -t 1 ]] && red=$'\e[31m' rst=$'\e[0m'; printf '%sfailed%s\n' "$red" "$rst"

Run shellcheck script.sh before anything reaches CI; it catches the quoting and set -e traps above mechanically.