Shell scripting explained

What is a shell script?

A shell is the program that reads your terminal commands and runs them. A shell script is just those commands saved in a file so they run as a unit, the glue that automates backups, deployments, data wrangling, and the thousand small chores between bigger tools. Most scripts target bash, the common default shell. A script starts with a shebang declaring its interpreter, and you make it runnable with chmod +x:

#!/usr/bin/env bash: finds bash on the user’s PATH (more portable than a hard-coded path).

Variables and quoting

Assign with name="value" (no spaces around =) and read with $name. The golden rule: always wrap variables in double quotes, "$file", so a value with spaces or wildcards stays a single argument. Unquoted variables are the number-one cause of shell bugs.

Conditionals and loops

Tests go inside [ ... ] (or bash’s [[ ... ]]): [ -f "$f" ] (file exists), [ -d "$d" ] (directory), [ -z "$s" ] (empty string), [ "$a" = "$b" ] (string equal), [ "$n" -gt 5 ] (numeric greater-than). Loops come in for item in list; do ... done and while condition; do ... done forms, see them combined in the example below.

Pipes and redirection

Syntax	What it does
cmd > file	Write stdout to a file (overwrite)
cmd >> file	Append stdout to a file
cmd < file	Read stdin from a file
cmd 2> file	Redirect stderr (stream 2)
a | b	Pipe a’s output into b’s input
cmd 2>&1	Merge stderr into stdout

Pipes are the heart of the Unix philosophy: small tools, each doing one job, composed into a pipeline.

Essential commands

Command	Does	Command	Does
grep	Search text by pattern	find	Search the filesystem
sed	Stream-edit text	xargs	Build commands from input
awk	Field-based text processing	chmod	Change permissions
cut / sort	Slice and order lines	curl	Make HTTP requests

A real script

Compressing each log file into a backup folder, with the safety header and quoting in place:

#!/usr/bin/env bash
set -euo pipefail                 # exit on error, unset var, or failed pipe

backup_dir="$HOME/backups"
mkdir -p "$backup_dir"            # quote variables so spaces do not break them

for file in *.log; do
  if [ -f "$file" ]; then         # only regular files
    gzip -c "$file" > "$backup_dir/$file.gz"
    echo "backed up $file"
  fi
done

Common mistakes

• Unquoted variables. $file splits on spaces; write "$file".
• Spaces around =. name = "x" is wrong; it must be name="x".
• No safety header. Without set -euo pipefail a script keeps running after a failed command, often doing damage.
• Assuming bash under #!/bin/sh. On many systems /bin/sh is a leaner shell; bash-only syntax will fail. Match the shebang to the features you use.
• Parsing ls. Loop over a glob (for f in *.log) instead; ls output is for humans, not scripts.

FAQ

What is the difference between bash and sh?

sh is the POSIX shell standard; bash is a specific, widely installed shell that is a superset of it with many extra features (arrays, [[ ]] tests, process substitution). A script with #!/bin/sh should stick to POSIX features for portability; #!/usr/bin/env bash opts into bash's extras. On many systems /bin/sh is actually a smaller shell like dash, so bash-only syntax under a sh shebang will fail.

What is the shebang line?

The first line, like #!/usr/bin/env bash, that tells the OS which interpreter should run the file. #!/usr/bin/env bash finds bash on the user's PATH, which is more portable than hard-coding /bin/bash.

Why do I have to quote variables?

An unquoted $file splits on spaces and expands wildcards, so a filename like "my report.log" becomes two arguments. Always write "$file" (in double quotes) unless you specifically want splitting. It is the single biggest source of shell bugs.

What does set -euo pipefail do?

It makes a script fail loudly instead of limping on: -e exits on any command that returns an error, -u treats an unset variable as an error, and -o pipefail makes a pipeline fail if any stage fails (not just the last). It is the standard safety header for a serious bash script.

How do I make a script executable?

Run chmod +x script.sh to add the execute bit, then run it with ./script.sh. Without the execute bit you can still run it explicitly as bash script.sh.

How do I run a script on a schedule?

Use cron: add a line to your crontab describing when to run it. Build and read the schedule expression with the cron generator, then point it at your script's full path.

Related concepts

Schedule a script with the cron generator · curl to code · .htaccess generator · Multithreading · all references.