Idempotent Scripts and Retry-with-Backoff Logic
Design operations that are safe to re-run and add exponential backoff for flaky external calls.
What Is Idempotency and Why It Matters
Idempotency means running the same operation multiple times produces the same result as running it once. In Bash scripting, this is critical because scripts crash, networks drop, and humans re-run things by accident.
- A non-idempotent script that creates a user twice may fail or duplicate data.
- An idempotent script checks first: does this already exist?
- Idempotent scripts are safe to use in cron jobs, CI pipelines, and retry loops.
The golden rule: check before you act. Every destructive or creative operation should be guarded by a precondition test.
Guarding File and Directory Creation
The most common idempotency pattern is checking whether a resource already exists before creating it. Bash provides concise one-liners for this.
[ -d dir ]— true if directory exists[ -f file ]— true if regular file existsmkdir -p— creates directory only if absent (built-in idempotency)
Prefer built-in flags like -p and --no-clobber over manual checks when available — they are atomic and race-condition-safe.
#!/usr/bin/env bash
set -euo pipefail
CONFIG_DIR="$HOME/.myapp"
CONFIG_FILE="$CONFIG_DIR/config.ini"
# Idempotent: mkdir -p never fails if dir already exists
mkdir -p "$CONFIG_DIR"
# Idempotent: only write config if it doesn't exist yet
if [ ! -f "$CONFIG_FILE" ]; then
echo '[defaults]' > "$CONFIG_FILE"
echo 'timeout=30' >> "$CONFIG_FILE"
echo "Created $CONFIG_FILE"
else
echo "Config already exists, skipping."
fiAll lessons in this course
- Strict Mode with set -euo pipefail
- Trap Handlers for Cleanup and Signals
- Safe Temporary Files and Lock Directories
- Idempotent Scripts and Retry-with-Backoff Logic