Shell scripting is essential for DevOps engineers. Every day you'll write scripts to automate deployments, manage infrastructure, process logs, and handle system administration. Learning bash properly—not just copy-pasting—makes you significantly more productive and helps you write reliable automation that won't break at 3 AM in production.
1
Basics & Getting Started
Shebang, permissions, echo, comments
#!/bin/bash
The Shebang Line
Tells the system which interpreter to use
›
The shebang (#!) must be the first line of every script. It tells Linux which interpreter to execute the script with. #!/bin/bash means "run this using the bash interpreter located at /bin/bash".
Why it matters: Without a shebang, your system doesn't know whether to use bash, python, perl, or something else. With it, you can run ./script.sh directly instead of typing bash script.sh every time. The shebang is also called a "magic number"—it's special.
Script with shebang
#!/bin/bash
# This is a comment - bash ignores everything after #
# Author: Anji Yarra
# Purpose: Demonstrate shebang and basic script
echo "Hello DevOps World!"
echo "This is my first bash script"
echo "Date: $(date +%Y-%m-%d)"
chmod +x
Making Scripts Executable
Grant execute permission to run directly
›
By default, files don't have execute permission. Use chmod +x to add execute permission. Then you can run ./script.sh directly instead of bash script.sh. Without execute permission, Linux treats it as a regular file, not a program.
How it works: chmod = change mode (permissions). +x = add execute. You can also use chmod 755 (owner: rwx, others: rx) or chmod 700 (owner only). Once executable, bash can start the shebang interpreter automatically. This is one of the most fundamental Unix concepts.
Making a script executable
# 1. Create a script file
cat > hello.sh << 'EOF'
#!/bin/bash
echo "This script is now executable!"
EOF
# 2. Check current permissions (not executable)
ls -la hello.sh
# -rw-r--r-- user group 62 May 26 hello.sh
# 3. Make it executable
chmod +x hello.sh
# 4. Check permissions (now executable - note the x)
ls -la hello.sh
# -rwxr-xr-x user group 62 May 26 hello.sh
# 5. Run it directly (no bash needed)
./hello.sh
# This script is now executable!
# Without chmod +x, you'd get "Permission denied"
echo
Print Output to Screen
Display text and variables
›
echo writes text to stdout (your terminal). It's how scripts communicate with users. By default echo adds a newline at the end. Use -n flag to suppress it. echo can display variables, command output, and any text.
stdin/stdout/stderr: Every Linux process has three data streams. stdin = input from keyboard/files, stdout = normal output to terminal, stderr = error messages. echo writes to stdout. You can redirect: echo "text" > file.txt (save to file), echo "text" >> file.txt (append), echo "error" >&2 (send to stderr).
Echo examples
# Basic echo - prints text with newline
echo "Hello World"
# Output: Hello World
# Multiple lines
echo "Line 1"
echo "Line 2"
# Output: Line 1
# Line 2
# No newline with -n flag
echo -n "Enter your name: "
# Output: Enter your name: [cursor waits for input]
# Variables in echo (always quote to prevent word splitting)
NAME="Alice"
VERSION="3.2.1"
echo "Welcome $NAME to version $VERSION"
# Output: Welcome Alice to version 3.2.1
# Save output to file
echo "Deployment started at $(date)" > deploy.log
# Append to file
echo "Status: Success" >> deploy.log
#
Comments & Documentation
Add notes that bash ignores
›
Comments start with # and are completely ignored by bash. They're for humans to read. Good comments explain WHY you're doing something, not WHAT (the code shows that). Always add a header block explaining the script's purpose, author, and usage.
Comment best practices: Start with a header explaining purpose/author/usage. Comment complex logic explaining the approach. Don't comment obvious code. For functions, document parameters and return values. Comments are essential for production scripts—future you will thank present you.
Proper commenting in production script
#!/bin/bash
########################################
# Database Backup Script
########################################
# Author: Anji Yarra
# Purpose: Automated daily backup of production databases
# Usage: ./backup.sh [database-name]
# Cron: 0 2 * * * /usr/local/bin/backup.sh
########################################
# Configuration - easier to find and modify
DB_NAME="${1:-production_db}"
BACKUP_DIR="/backups/postgresql"
RETENTION_DAYS=30
# Stop application so database is idle for backup
systemctl stop myapp
# Create backup with timestamp for unique naming
TIMESTAMP=$(date +%Y-%m-%d_%H-%M-%S)
BACKUP_FILE="$BACKUP_DIR/db-$TIMESTAMP.sql.gz"
pg_dump "$DB_NAME" | gzip > "$BACKUP_FILE"
# Restart application immediately
systemctl start myapp
# Clean old backups to save disk space
find "$BACKUP_DIR" -name "db-*.sql.gz" -mtime +$RETENTION_DAYS -delete
echo "Backup completed: $BACKUP_FILE"
2
Variables & Data
Store, access, and use data
NAME=value
Variable Assignment
Create and store values
›
Variables store values for later use. Syntax: NAME=value with NO SPACES around =. Variable names are case-sensitive (NAME and name are different). By convention, system variables use UPPERCASE, local variables use lowercase.
Common mistake: Spaces break assignment! NAME = value fails because bash thinks you're running a command called NAME. Always use NAME=value with zero spaces. Variables can contain text, numbers, paths, or command output.
Variable assignment patterns
# String variables
USERNAME="alice"
SERVER="prod-db-01"
ENVIRONMENT="production"
# Numeric variables (stored as text, used as numbers)
PORT=5432
MAX_CONNECTIONS=200
TIMEOUT=30
# Path variables
CONFIG_DIR="/etc/myapp"
LOG_FILE="/var/log/app.log"
BACKUP_PATH="/backups/$(date +%Y-%m-%d)"
# Boolean-like (bash has no true boolean)
ENABLE_DEBUG="true"
AUTO_RESTART="false"
# Multi-line values
DEPLOYMENT_INFO="
App: MyApplication
Version: 3.2.1
Environment: Production
"
echo "Server: $SERVER on port $PORT"
$VARIABLE
Accessing Variables
Use the $ prefix to get values
›
Use $ to read a variable. Without $, bash treats the name as literal text. Always quote variables to prevent word splitting—spaces in values break your commands if unquoted. This is one of the most common bugs in shell scripts.
Word splitting: If VAR="hello world" and you use $VAR without quotes, bash splits it into two arguments: "hello" and "world". Use "$VAR" to keep it as one value. This causes subtle bugs: mkdir $DIR might fail if $DIR has spaces, but mkdir "$DIR" works perfectly.
Accessing variables correctly
# Assign values first
APP_NAME="MyApplication"
ENV="production"
VERSION="2.5.1"
# Access with $ prefix
echo "Running $APP_NAME in $ENV"
# Output: Running MyApplication in production
# Without $, treated as literal text (WRONG)
echo "APP_NAME"
# Output: APP_NAME (not MyApplication)
# CRITICAL: Always quote variables!
DIR_WITH_SPACES="/var/log/my application"
# BAD: Without quotes, splits on space
ls $DIR_WITH_SPACES # Fails or gives wrong output
# GOOD: With quotes, works correctly
ls "$DIR_WITH_SPACES" # Lists the correct directory
# Use variables in paths
CONFIG_FILE="/etc/app/$APP_NAME/config.conf"
BACKUP_DIR="/backups/$APP_NAME-$(date +%Y-%m-%d)"
mkdir -p "$BACKUP_DIR"
$(command)
Command Substitution
Run commands, save output to variables
›
Command substitution $(command) runs a command and saves its output to a variable. This is one of bash's most powerful features. Use $() syntax (modern and nestable), not backticks (deprecated and harder to read).
Why it's powerful: Instead of manually typing dates or results, TIMESTAMP=$(date) gets today's date automatically. Combined with loops and functions, this enables dynamic, flexible automation. Your script adapts to the environment instead of being hardcoded.
Command substitution examples
# Capture date/time
TODAY=$(date +%Y-%m-%d)
NOW=$(date +%H:%M:%S)
MONTH=$(date +%B)
echo "Date: $TODAY"
echo "Time: $NOW"
echo "Month: $MONTH"
# Get system information
CURRENT_USER=$(whoami)
HOSTNAME=$(hostname)
KERNEL=$(uname -r)
echo "Running as: $CURRENT_USER on $HOSTNAME"
echo "Kernel: $KERNEL"
# Count things
FILE_COUNT=$(ls -1 /tmp | wc -l)
PROCESS_COUNT=$(ps aux | wc -l)
DISK_USAGE=$(df -h / | tail -1 | awk '{print $5}')
echo "Files in /tmp: $FILE_COUNT"
echo "Running processes: $PROCESS_COUNT"
echo "Disk usage: $DISK_USAGE"
# Create timestamped filenames (very useful)
BACKUP_FILE="/backups/db-$(date +%Y%m%d_%H%M%S).sql.gz"
LOG_FILE="/var/log/deploy-$(date +%Y-%m-%d).log"
# Use in conditionals
CURRENT_HOUR=$(date +%H)
if [ "$CURRENT_HOUR" -ge 22 ]; then
echo "Late night deployment, proceed with caution"
fi
$1 $# $? $$
Special Built-in Variables
Access automatic bash values
›
Bash automatically creates special variables: $0=script name, $1/$2/$3=arguments, $#=argument count, $?=exit code of last command, $$=process ID. These enable scripts to be flexible and handle errors.
Practical use: Check $# before using $1 (prevents "argument not found" errors). Check $? after critical commands to detect failures. Use $$ for unique temp files. Arguments let one script handle multiple scenarios: deploy.sh production v2.5 vs deploy.sh staging v2.4 use identical code with different data.
Special variables in action
#!/bin/bash
# Script name: deploy.sh
# Usage: ./deploy.sh [--force]
# $0 is the script name itself
SCRIPT_NAME="$0"
echo "Running: $SCRIPT_NAME"
# $# is total argument count
if [ $# -lt 2 ]; then
echo "ERROR: Need at least 2 arguments"
echo "Usage: $SCRIPT_NAME "
exit 1
fi
# $1, $2, $3 are individual arguments
# bash script.sh prod 2.3.0 false
# $1=prod, $2=2.3.0 , $3=false are individual arguments
ENVIRONMENT="$1"
VERSION="$2"
FORCE_DEPLOY="$3"
echo "Deploying version $VERSION to $ENVIRONMENT"
# Run a command and check $? (exit code)
npm test
if [ $? -ne 0 ]; then
echo "ERROR: Tests failed, aborting deployment"
exit 1
fi
# $$ is this script's process ID (useful for temp files)
TEMP_LOG="/tmp/deploy-$$.log"
echo "Logging to: $TEMP_LOG"
npm run build >> "$TEMP_LOG" 2>&1
if [ $? -eq 0 ]; then
echo "✓ Build successful"
else
echo "✗ Build failed (see $TEMP_LOG)"
exit 1
fi
# Display all arguments
echo "Total arguments: $#"
echo "All arguments: $@"
3
Operators & Comparisons
Math, conditions, file tests, logic
$((...))
Arithmetic Operations
Do math: +, -, *, /, %
›
Use $(( )) to perform integer arithmetic. Inside, you don't need $ before variables. Supports +, -, *, /, % (modulo), ++, --. Bash normally treats everything as text, so $(( )) tells bash "treat these as numbers".
Why separate from text: Without $(( )), "5" + "3" is text concatenation, not math. Modulo (%) is useful for: checking if even/odd, wrapping counters, cycling through values. Integer arithmetic only—use bc or awk for decimals.
Arithmetic examples
# Basic math operations
A=10
B=3
SUM=$((A + B))
echo "Sum: $SUM"
# Output: Sum: 13
DIFF=$((A - B))
PRODUCT=$((A * B))
QUOTIENT=$((A / B)) # Integer division
REMAINDER=$((A % B))
echo "10 - 3 = $DIFF"
echo "10 * 3 = $PRODUCT"
echo "10 / 3 = $QUOTIENT" # 3, not 3.333
echo "10 % 3 = $REMAINDER" # 1 (the remainder)
# Increment/decrement
COUNTER=0
COUNTER=$((COUNTER + 1))
echo "Counter: $COUNTER"
# Check if even or odd using modulo
NUMBER=42
if [ $((NUMBER % 2)) -eq 0 ]; then
echo "$NUMBER is even"
else
echo "$NUMBER is odd"
fi
# Useful for loops
for ((i=1; i<=5; i++)); do
echo "Iteration $i"
done
-eq -lt -gt
Number Comparisons
Compare numbers: equal, less, greater
›
Use -eq, -ne, -lt, -le, -gt, -ge inside [ ] for number comparisons. Don't use = or < (those are for text). [ ] is the test command—it evaluates conditions and returns true/false.
Text vs number: [ "5" = "10" ] is false (comparing text), but [ 5 -lt 10 ] is true (comparing numbers). Use -eq/-lt for numbers, = for text. This matters for sorting and logic correctness.
Number comparison patterns
# Comparison operators
AGE=25
THRESHOLD=18
# Equal (-eq) and Not equal (-ne)
if [ $AGE -eq 25 ]; then
echo "Age is exactly 25"
fi
if [ $AGE -ne 18 ]; then
echo "Age is not 18"
fi
# Less than (-lt) and Less or equal (-le)
if [ $THRESHOLD -lt $AGE ]; then
echo "$THRESHOLD is less than $AGE"
fi
if [ $THRESHOLD -le $AGE ]; then
echo "$THRESHOLD is <= $AGE"
fi
# Greater than (-gt) and Greater or equal (-ge)
if [ $AGE -gt 18 ]; then
echo "Is an adult"
fi
if [ $AGE -ge 21 ]; then
echo "Can drink"
fi
# Practical: Check resource usage and alert
CPU_USAGE=85
MEMORY_USAGE=92
if [ $CPU_USAGE -gt 80 ]; then
echo "WARNING: High CPU ($CPU_USAGE%)"
fi
if [ $MEMORY_USAGE -gt 90 ]; then
echo "CRITICAL: Memory almost full ($MEMORY_USAGE%)"
fi
-f -d -e
File Tests
Check file existence and properties
›
Test files with [ -f file ] (is file?), [ -d dir ] (is directory?), [ -e path ] (exists?), [ -r ] (readable?), [ -w ] (writable?), [ -x ] (executable?), [ -s ] (non-empty?). Essential for safe, production-grade scripts.
Why test first: Never blindly access files—always test first. This prevents "file not found" errors, permission issues, and wrong file operations. Safe scripts test before reading, writing, or executing. This makes debugging much easier and prevents silent failures.
File testing patterns
# Check if file exists and is readable
CONFIG_FILE="/etc/app/config.conf"
if [ -f "$CONFIG_FILE" ]; then
echo "Config file found"
source "$CONFIG_FILE" # Load it safely
else
echo "ERROR: Config file missing"
exit 1
fi
# Check if directory exists, create if not
BACKUP_DIR="/backups"
if [ -d "$BACKUP_DIR" ]; then
echo "Backup directory exists"
else
echo "Creating backup directory..."
mkdir -p "$BACKUP_DIR"
fi
# Check if writable before backup
if [ -w "$BACKUP_DIR" ]; then
cp -r /data "$BACKUP_DIR/data-$(date +%Y-%m-%d)"
echo "✓ Backup completed"
else
echo "ERROR: Cannot write to backup directory"
exit 1
fi
# Check if file is not empty
LOGFILE="/var/log/app.log"
if [ -s "$LOGFILE" ]; then
echo "Log has content"
tail -f "$LOGFILE"
else
echo "Log file is empty"
fi
# Check if executable (script/binary)
DEPLOY_SCRIPT="/usr/local/bin/deploy.sh"
if [ -x "$DEPLOY_SCRIPT" ]; then
$DEPLOY_SCRIPT # Safe to run
else
echo "Deploy script not executable"
chmod +x "$DEPLOY_SCRIPT"
fi
&& || !
Logical Operators
AND, OR, NOT—combine conditions
›
Combine conditions: && (AND—both true), || (OR—at least one true), ! (NOT—negate). && runs right side only if left succeeds. || runs right side only if left fails. This enables powerful conditional logic and error handling.
Short-circuit evaluation: [ $A -gt 0 ] && echo "positive" only echoes if test passes. cmd1 || cmd2 runs cmd2 only if cmd1 fails. This is how you chain commands: mkdir dir || exit 1 means "make dir, if it fails then exit".
Logical operator examples
# AND (&&) - both conditions must be true
USERNAME="alice"
PASSWORD="correct123"
if [ "$USERNAME" = "alice" ] && [ "$PASSWORD" = "correct123" ]; then
echo "✓ Login successful"
else
echo "✗ Login failed"
fi
# OR (||) - at least one must be true
DAY_OF_WEEK=$(date +%A)
if [ "$DAY_OF_WEEK" = "Saturday" ] || [ "$DAY_OF_WEEK" = "Sunday" ]; then
echo "It's the weekend!"
fi
# NOT (!) - negate/invert condition
if [ ! -f "/tmp/lock" ]; then
echo "Lock file doesn't exist, safe to proceed"
touch "/tmp/lock"
fi
# Chaining commands - stop if any step fails
npm test && npm build && npm deploy
# Error handling pattern
cp important.txt backup.txt || {
echo "ERROR: Backup failed"
exit 1
}
# Check multiple conditions
FILE1="data1.txt"
FILE2="data2.txt"
FILE3="data3.txt"
if [ -f "$FILE1" ] && [ -f "$FILE2" ] && [ -f "$FILE3" ]; then
echo "All required files present, proceeding"
else
echo "Missing required files"
exit 1
fi
4
Control Flow & Loops
if/else, for, while, case statements
if then fi
If/Else Conditionals
Execute code based on conditions
›
if tests a condition. If true, execute the then block. Use elif for multiple branches, else for the fallback. Always end with fi. This is how you make scripts make decisions.
Structure: if [ condition ]; then ... elif [ condition2 ]; then ... else ... fi. The semicolon after ] is required. You can have multiple elif branches. Always quote variables: [ "$VAR" = "value" ] not [ $VAR = value ].
If/else patterns
#!/bin/bash
# Simple if/else
AGE=20
if [ $AGE -ge 18 ]; then
echo "Adult"
else
echo "Minor"
fi
# Multiple branches with elif
if [ $AGE -lt 13 ]; then
echo "Child"
elif [ $AGE -lt 18 ]; then
echo "Teenager"
elif [ $AGE -lt 65 ]; then
echo "Adult"
else
echo "Senior"
fi
# Check file and act accordingly
BACKUP_FILE="/backups/database.sql.gz"
if [ -f "$BACKUP_FILE" ]; then
echo "Backup exists"
SIZE=$(ls -lh "$BACKUP_FILE" | awk '{print $5}')
echo "Size: $SIZE"
elif [ -d "/backups" ]; then
echo "Backups directory exists but file missing"
echo "Starting backup..."
else
echo "ERROR: Backups directory missing"
mkdir -p "/backups"
fi
# Validate script arguments
if [ $# -lt 2 ]; then
echo "Usage: $0 "
exit 1
fi
ENV="$1"
VERSION="$2"
if [ "$ENV" != "prod" ] && [ "$ENV" != "staging" ]; then
echo "ERROR: Invalid environment (use prod or staging)"
exit 1
fi
echo "Deploying v$VERSION to $ENV"
for do done
For Loops
Iterate over ranges, lists, or files
›
for loops iterate over lists, ranges, or files. Common syntaxes: for i in {1..10}, for file in *.log, for ((i=0; i<10; i++)). Always end with done. This is how you automate repetitive tasks.
Three loop styles: for var in list (iterate a list), for ((init; condition; increment)) (C-style), for var in $(command) (iterate command output). Each is useful for different scenarios. Loops are essential for batch operations.
For loop patterns
# Iterate over range
for i in {1..5}; do
echo "Number: $i"
done
# Iterate over list
SERVERS="web1 web2 web3 db1 cache1"
for server in $SERVERS; do
echo "Checking $server..."
ping -c 1 "$server" > /dev/null
if [ $? -eq 0 ]; then
echo "✓ $server is UP"
else
echo "✗ $server is DOWN"
fi
done
# Iterate over files
for logfile in /var/log/*.log; do
if [ -f "$logfile" ]; then
lines=$(wc -l < "$logfile")
echo "$(basename $logfile): $lines lines"
fi
done
# C-style loop (for when you need counter)
for ((i=1; i<=3; i++)); do
echo "Attempt $i of 3"
sleep 1
done
# Iterate command output
echo "Processing all processes:"
for pid in $(ps aux | grep myapp | grep -v grep | awk '{print $2}'); do
echo "PID: $pid"
kill -0 "$pid" && echo " ✓ Still running"
done
# Practical: Backup multiple databases
DATABASES=(prod_app analytics payments)
BACKUP_DIR="/backups/$(date +%Y-%m-%d)"
mkdir -p "$BACKUP_DIR"
for db in "${DATABASES[@]}"; do
echo "Backing up $db..."
mysqldump "$db" | gzip > "$BACKUP_DIR/$db.sql.gz"
echo " ✓ $db backed up"
done
while do done
While Loops
Loop while condition is true
›
while loops run as long as a condition is true. Useful for reading files, waiting for conditions, retries. Always ensure the condition eventually becomes false to avoid infinite loops. Common pattern: read input, process it, check condition.
When to use: while is perfect for reading files line by line, waiting for events, retry logic. Beware infinite loops—always have a way out. Use break to exit early, continue to skip to next iteration.
While loop patterns
# Simple counter loop
COUNT=1
while [ $COUNT -le 5 ]; do
echo "Count: $COUNT"
COUNT=$((COUNT + 1))
done
# Read file line by line
while read line; do
echo "Processing: $line"
done < /path/to/file.txt
# Read input from user
while true; do
read -p "Enter command (or 'quit'): " cmd
if [ "$cmd" = "quit" ]; then
echo "Goodbye"
break # Exit loop
fi
eval "$cmd"
done
# Retry logic (retry up to 3 times)
RETRY_COUNT=0
MAX_RETRIES=3
while [ $RETRY_COUNT -lt $MAX_RETRIES ]; do
echo "Attempt $((RETRY_COUNT + 1)) of $MAX_RETRIES..."
if curl -s "https://api.example.com/health" > /dev/null; then
echo "✓ API is responsive"
break
fi
RETRY_COUNT=$((RETRY_COUNT + 1))
if [ $RETRY_COUNT -lt $MAX_RETRIES ]; then
echo " API not responding, retrying in 5 seconds..."
sleep 5
fi
done
if [ $RETRY_COUNT -eq $MAX_RETRIES ]; then
echo "✗ API still not responding after $MAX_RETRIES attempts"
exit 1
fi
# Monitor process and restart if needed
while true; do
if ! pgrep -x "myapp" > /dev/null; then
echo "Process died, restarting..."
/usr/local/bin/myapp &
fi
sleep 10 # Check every 10 seconds
done
case esac
Case Statements
Switch on value—cleaner than if/elif
›
case evaluates a value against patterns. Much cleaner than multiple if/elif when checking one variable. Each pattern ends with ;;. Use * as default fallback. Supports wildcards for pattern matching.
Pattern syntax: case $VAR in pattern1) code;; pattern2) code;; *) default;; esac. Patterns can use wildcards: "dev*" matches "dev", "development", "develop", etc. Very powerful for command routing and handling options.
Case statement patterns
# Simple case statement
DAY=$(date +%A)
case "$DAY" in
Monday)
echo "Start of work week"
;;
Friday)
echo "Almost weekend!"
;;
Saturday|Sunday)
echo "It's the weekend"
;;
*)
echo "Regular weekday"
;;
esac
# Handle script arguments/options
COMMAND="$1"
case "$COMMAND" in
start)
systemctl start myapp
echo "Application started"
;;
stop)
systemctl stop myapp
echo "Application stopped"
;;
restart)
systemctl restart myapp
echo "Application restarted"
;;
status)
systemctl status myapp
;;
*)
echo "Usage: $0 {start|stop|restart|status}"
exit 1
;;
esac
# Environment-specific logic
ENV="$1"
case "$ENV" in
prod|production)
DB_HOST="prod-db.example.com"
API_URL="https://api.example.com"
DEBUG="false"
;;
staging|stage)
DB_HOST="staging-db.example.com"
API_URL="https://staging-api.example.com"
DEBUG="false"
;;
dev|development|local)
DB_HOST="localhost"
API_URL="http://localhost:3000"
DEBUG="true"
;;
*)
echo "ERROR: Unknown environment: $ENV"
echo "Use: prod, staging, or dev"
exit 1
;;
esac
echo "Configuring for $ENV..."
echo " DB: $DB_HOST"
echo " API: $API_URL"
echo " Debug: $DEBUG"
5
Advanced Topics
Functions, arrays, text processing, error handling
function() {}
Functions
Reusable code blocks with parameters
›
Functions group code for reuse and organization. Define with function_name() { }. Call by name. Access arguments via $1, $2, etc. Return values with return (0-255) or echo. This is essential for maintainable scripts.
Benefits: Reduce code duplication, improve readability, enable testing of individual pieces, make scripts modular. Local variables (local VAR=value) don't affect global scope. Always document parameters and return values in comments.
Function patterns
#!/bin/bash
# Simple function
greet() {
local name="$1"
echo "Hello, $name!"
}
greet "Alice"
greet "Bob"
# Function with return value (0=success)
check_file_exists() {
local filepath="$1"
if [ -f "$filepath" ]; then
echo "File exists: $filepath"
return 0
else
echo "File not found: $filepath"
return 1
fi
}
# Use return value to decide next step
if check_file_exists "/etc/passwd"; then
echo "System file is present"
else
echo "ERROR: System file missing"
exit 1
fi
# Function returning data via echo
get_file_size() {
local file="$1"
if [ -f "$file" ]; then
du -h "$file" | awk '{print $1}'
fi
}
SIZE=$(get_file_size "/var/log/syslog")
echo "Log file size: $SIZE"
# Function with multiple parameters
deploy_service() {
local service="$1"
local version="$2"
local environment="$3"
echo "Deploying $service v$version to $environment"
echo " Downloading artifact..."
sleep 1
echo " Stopping current version..."
systemctl stop "$service"
echo " Starting v$version..."
systemctl start "$service"
echo " ✓ Deployment complete"
}
deploy_service "myapp" "2.5.1" "production"
# Error handling in functions
safe_backup() {
local source="$1"
local destination="$2"
if [ ! -d "$source" ]; then
echo "ERROR: Source directory not found: $source"
return 1
fi
if [ ! -w "$destination" ]; then
echo "ERROR: Cannot write to destination: $destination"
return 2
fi
cp -r "$source" "$destination/backup-$(date +%Y-%m-%d)"
echo "✓ Backup completed to $destination"
return 0
}
safe_backup "/data" "/backups" || {
echo "Backup failed with code $?"
exit 1
}
array=()
Arrays
Store lists of values
›
Arrays store lists of values. Create with (space-separated). Access with ${array[index]}. Use ${#array[@]} for length, ${array[@]} for all elements, ${array[@]:start:length} for slices. Useful for batch operations.
Array basics: Index starts at 0. Associative arrays use string keys instead of numbers (bash 4.0+). "${array[@]}" gives all elements, "${array[@]}" expands each element as a separate word, essential for loops.
Array patterns
# Create array with values
FRUITS=(apple banana orange grape cherry)
# Access by index (0-based)
echo "First fruit: ${FRUITS[0]}" # apple
echo "Third fruit: ${FRUITS[2]}" # orange
# Length of array
echo "Total fruits: ${#FRUITS[@]}" # 5
# Loop through array
for fruit in "${FRUITS[@]}"; do
echo "Fruit: $fruit"
done
# Add to array
FRUITS+=(watermelon)
echo "Now have ${#FRUITS[@]} fruits"
# Practical: Process multiple servers
SERVERS=(
"web1.example.com"
"web2.example.com"
"api1.example.com"
"db1.example.com"
)
for server in "${SERVERS[@]}"; do
echo "Checking $server..."
ssh "$server" "uptime"
done
# Associative array (key-value pairs)
declare -A COLORS
COLORS[primary]="blue"
COLORS[secondary]="green"
COLORS[accent]="gold"
echo "Primary color: ${COLORS[primary]}"
echo "Secondary color: ${COLORS[secondary]}"
# Loop through associative array
for color_name in "${!COLORS[@]}"; do
echo "$color_name = ${COLORS[$color_name]}"
done
# Store command outputs in array
RUNNING_PIDS=($(ps aux | grep myapp | grep -v grep | awk '{print $2}'))
if [ ${#RUNNING_PIDS[@]} -gt 0 ]; then
echo "Found ${#RUNNING_PIDS[@]} running processes"
for pid in "${RUNNING_PIDS[@]}"; do
echo " PID: $pid"
done
else
echo "No running processes found"
fi
sed & awk
Text Processing
Powerful text manipulation tools
›
sed (stream editor) and awk are powerful text processing tools. sed modifies text (find/replace, delete lines). awk processes text in columns/fields. Both use regex. Essential for log analysis, data transformation, and text manipulation.
sed vs awk: sed is for line-by-line text replacement/deletion. awk is for processing columnar data. Both support regex. sed 's/old/new/g' replaces all occurrences. awk '{print $1, $3}' prints columns 1 and 3. Combining these with pipes creates powerful data pipelines.
Text processing examples
# sed: Replace text (s = substitute, g = global)
echo "Hello World" | sed 's/World/DevOps/'
# Output: Hello DevOps
# sed: Replace in file (use -i for in-place)
sed 's/ERROR/ALERT/g' error.log
# sed: Delete lines matching pattern
sed '/^#/d' config.conf # Remove comment lines
sed '10d' file.txt # Delete line 10
# awk: Print specific columns
ps aux | awk '{print $1, $2, $11}' # user, PID, command
# awk: Sum a column
awk '{sum += $1} END {print "Total: " sum}' numbers.txt
# awk: Print lines matching condition
awk '$3 > 100 {print $0}' data.txt # Print if 3rd column > 100
# Practical: Parse /etc/passwd
awk -F: '{print $1, $3, $6}' /etc/passwd # user, UID, home
# Practical: Extract data from log
grep "ERROR" app.log | awk '{print $1, $2}' | sort | uniq -c
# Combine sed and awk in pipeline
cat deploy.log |
sed 's/\[.*\]//' | # Remove timestamps
grep "Deploy" |
awk '{print $2, $3}' | # Extract relevant fields
sort | uniq
set -e -u
Error Handling & Robustness
Make scripts fail safely and predictably
›
set -e exits script if any command fails. set -u errors on undefined variables. set -o pipefail exits if any command in a pipe fails. These prevent scripts from continuing after errors—essential for production scripts.
Why critical: Without set -e, scripts continue even when commands fail silently. This leads to partial deployments, incomplete backups, and hard-to-debug issues. set -e catches problems early. set -u prevents typos from silently using empty values.
Error handling patterns
#!/bin/bash
# Add these at the top of production scripts
set -e # Exit on error
set -u # Error on undefined variables
set -o pipefail # Pipe failures cause exit
# Now any error stops the script
mkdir -p "$BACKUP_DIR" # If this fails, script exits
cp -r /data "$BACKUP_DIR" # Won't run if mkdir failed
# Without set -e, would continue below...
# Dangerous: backup might be incomplete but script succeeds!
echo "✓ Backup completed successfully"
# Trap errors for cleanup
cleanup() {
local exit_code=$?
echo "Script failed with exit code $exit_code"
rm -f /tmp/temp-$$.log # Clean temp files
exit $exit_code
}
trap cleanup EXIT # Call cleanup on exit
# Example: Deployment with error handling
#!/bin/bash
set -e
set -u
set -o pipefail
APP_DIR="/opt/myapp"
BACKUP_DIR="/opt/backups"
echo "Stopping application..."
systemctl stop myapp
echo "Backing up current version..."
cp -r "$APP_DIR" "$BACKUP_DIR/backup-$(date +%Y-%m-%d_%H-%M-%S)"
echo "Deploying new version..."
tar -xzf /tmp/release.tar.gz -C "$APP_DIR"
echo "Starting application..."
systemctl start myapp
echo "Running health check..."
curl -f http://localhost:3000/health > /dev/null
echo "✓ Deployment successful"
Best Practices
Production-Grade Script Writing
Guidelines for reliable, maintainable scripts
›
Production scripts require care: validate inputs, log operations, handle errors, use clear variable names, quote everything, test before production, document thoroughly. These practices separate scripts that work from scripts you can trust with critical operations.
Philosophy: Assume your script will run in production at the worst possible time. Write code defensively: validate inputs, test file existence, handle errors gracefully, provide clear error messages. Future you (or someone else) will need to debug this at 3 AM.
Production script template
#!/bin/bash
########################################
# Production Database Backup Script
########################################
# Author: Anji Yarra
# Purpose: Daily automated database backup
# Usage: ./backup.sh [database-name]
# Cron: 0 2 * * * /usr/local/bin/backup.sh production_db
# Log: /var/log/backup.log
########################################
# Strict error handling
set -e # Exit on any error
set -u # Error on undefined variables
set -o pipefail # Pipe failures exit
# Configuration
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_NAME="$(basename "$0")"
readonly LOG_FILE="/var/log/backup.log"
readonly DB_NAME="${1:-production_db}"
readonly BACKUP_DIR="/backups/$(date +%Y-%m-%d)"
readonly RETENTION_DAYS=30
# Color for output
readonly GREEN='\033[0;32m'
readonly RED='\033[0;31m'
readonly NC='\033[0m'
# Logging function
log() {
local level="$1"
shift
echo "[$(date +'%Y-%m-%d %H:%M:%S')] [$level] $*" | tee -a "$LOG_FILE"
}
# Error handler
error_exit() {
log "ERROR" "$1"
exit 1
}
# Validate inputs
validate_inputs() {
if [ -z "$DB_NAME" ]; then
error_exit "Database name cannot be empty"
fi
log "INFO" "Starting backup of $DB_NAME"
}
# Create backup
create_backup() {
mkdir -p "$BACKUP_DIR"
log "INFO" "Creating backup..."
local backup_file="$BACKUP_DIR/${DB_NAME}-$(date +%H-%M-%S).sql.gz"
if mysqldump "$DB_NAME" | gzip > "$backup_file"; then
log "INFO" "✓ Backup created: $backup_file"
return 0
else
error_exit "Failed to create backup"
fi
}
# Cleanup old backups
cleanup_old_backups() {
log "INFO" "Cleaning old backups (retention: $RETENTION_DAYS days)"
find /backups -name "*.sql.gz" -mtime +$RETENTION_DAYS -delete
log "INFO" "✓ Cleanup complete"
}
# Main execution
main() {
validate_inputs
create_backup
cleanup_old_backups
log "INFO" "✓ Backup process completed successfully"
}
# Error trap
trap 'error_exit "Script terminated unexpectedly"' ERR
main "$@"