T03: Pipes & Power Tools

The terminal stops being a clunky file browser and becomes a superpower the moment you understand the pipe. Each command is a small specialist. The pipe | is duct tape: it lets you bolt specialists together into assembly lines. Five commands joined by pipes can replace a spreadsheet macro, a Python script, or an afternoon.

Standard Streams

Every program has three channels. stdin is the input stream. stdout is where normal output goes. stderr is where errors go. The shell lets you redirect each one independently.

echo "hello" > greet.txt        # stdout TO a file (overwrite)
echo "world" >> greet.txt       # stdout APPEND to a file
sort < names.txt                 # stdin FROM a file
command 2> errors.log            # stderr TO a file
command > out.log 2>&1          # stdout and stderr together
command > /dev/null 2>&1         # discard all output

The Pipe

A pipe connects one command's stdout to the next command's stdin. You read it left to right, like a sentence.

# Show the 5 largest files in this folder
ls -lS | head -n 5

# Count how many .ts files the project has
find src -name "*.ts" | wc -l

# Find every TODO in your code
grep -rn "TODO" src/ | less

# Top 10 commands you use most
history | awk '{print $2}' | sort | uniq -c | sort -rn | head

Essential Specialists

The standard kit every pro keeps sharp:

• grep pattern file - find lines matching a pattern. -r recurses, -i case-insensitive, -n line numbers
• find path -name "*.ext" - locate files by name, type, size, age
• sort / uniq - sort lines, collapse duplicates (uniq -c counts them)
• wc -l - count lines
• head -n 20 / tail -n 20 - first / last lines. tail -f follows a growing log

Advanced: cut, sed, awk for column extraction, stream editing, and text processing. Pick them up on demand.

Chaining Commands

Three operators sequence commands without pipes.

cmd1 ; cmd2          # run cmd1, then cmd2 regardless of outcome
cmd1 && cmd2         # run cmd2 ONLY if cmd1 succeeded
cmd1 || cmd2         # run cmd2 ONLY if cmd1 failed

# Practical example
npm test && git push                     # push only if tests pass
mkdir build || echo "already exists"      # fallback message

A Tiny Shell Script

When you run the same pipe every day, save it. A shell script is a text file with a #!/usr/bin/env bash header, made executable with chmod +x.

#!/usr/bin/env bash
# backup.sh - copy the project into a timestamped tarball

set -euo pipefail                       # fail fast on errors

name="$(basename "$PWD")"
stamp="$(date +%Y%m%d-%H%M%S)"
tar -czf "../${name}-${stamp}.tar.gz" .
echo "Saved ../${name}-${stamp}.tar.gz"

chmod +x backup.sh
./backup.sh                             # run it

Running Long Jobs

Some jobs take minutes. You do not want to babysit them.

long-command &                  # run in background
jobs                             # list background jobs in this shell
long-command > out.log 2>&1 &   # background + capture output
ps aux | grep node               # find running processes
kill 12345                       # ask a process to stop
kill -9 12345                    # force it to stop