Search Tech Journey

Find topics, journeys and posts

6-month learning plan3 / 130
back to blog
systemsbeginner 55m read

S003 · The Command Line — bash, pipes, grep, jq

Live in the terminal without fear. The pipes, filters, and text-crunching muscle memory that separates an engineer from a button-clicker — and the 20 commands you'll use every single day for the rest of your career.

🧰SetupM00 · Setup & Tools· Session 003 of 130 90 min

🎯 Chain shell commands with pipes, filter text with grep/awk/sed, and slice JSON with jq — well enough to answer real questions from real log files in under 60 seconds.

Why this session exists

The terminal is where every backend engineer, SRE, data engineer, and ML platform person lives. A junior dev double-clicks through a file manager to find a log; a senior dev pipes it through grep | awk | sort | uniq -c | sort -rn | head and has an answer in eight seconds. That gap is not talent. It's a handful of commands, learned once, wired into muscle memory. This session installs that muscle memory.

You will be able to
  • Navigate the filesystem, find files, and inspect processes without touching a GUI.
  • Chain commands with pipes to answer questions like ‘which endpoint had the most 500s in the last hour?’ from a raw log file.
  • Use `grep`, `sed`, and `awk` well enough that you stop reaching for Python for one-off text munging.
  • Slice, filter, and reshape JSON with `jq` — the tool every API and Kubernetes user needs.
  • Read a stranger's bash script and predict what it does before running it.

Prerequisites



(a) Intuition · 5 min

Why Unix people love pipes
🌍 Real world

Imagine a factory floor. Every station does one job — cut, weld, paint, box. Parts flow along a conveyor. To make a new product you rearrange the stations, you don't build a new factory.

The Unix shell is that factory. Each command is a station. The pipe (|) is the conveyor. To answer a new question you snap together commands you already know — you don't write a new program.

💻 Code world

Technically: every Unix command reads a stream of bytes from stdin and writes a stream of bytes to stdout. A pipe wires one command's stdout to the next command's stdin — in memory, streaming, zero disk I/O.

That is the entire Unix philosophy: do one thing well, and communicate via text streams. Fifty years later, no other OS design has aged as gracefully.

The three streams that everything runs on

Every process has three files open before it starts
  • stdin (fd 0) — where input comes from. Default: your keyboard. Redirect with `<` or a pipe.
  • stdout (fd 1) — where normal output goes. Default: your terminal. Redirect with `>` or a pipe.
  • stderr (fd 2) — where errors go. Default: your terminal too, but a SEPARATE stream. Redirect with `2>`.

A quick history so you know why the world looks like this

  1. 1971
    First Unix shell (Thompson shell)
    Ken Thompson writes `sh` for the PDP-11 — the ancestor of every shell you'll ever use.
  2. 1979
    Bourne shell (sh)
    Stephen Bourne's shell becomes the Unix standard. Its syntax survives untouched in every POSIX shell today.
  3. 1989
    Bash 1.0
    GNU's ‘Bourne Again SHell’ — a free, feature-richer sh. Ships with Linux. Becomes the default everywhere except macOS.
  4. 2007
    jq released
    Stephen Dolan writes a ‘sed for JSON’. Becomes essential the moment REST APIs eat the world.
  5. 2019
    macOS switches default to zsh
    Apple's licensing dance ends bash 3.2 as macOS default. zsh + oh-my-zsh becomes the new normal.

(b) Visual walkthrough · 15 min

How a pipe actually works

Six processes running in parallel, connected by kernel-managed in-memory buffers. No temp files, no waiting. When head has 10 lines it closes its stdin; sort -rn gets SIGPIPE and terminates; a cascade of shutdowns walks back to cat. This is why pipes are fast — the OS does the scheduling for you.

The 20 commands that do 90% of the work

Your daily toolbox

Navigation
cd · ls · pwd · tree · pushd/popd — move around the filesystem.
essential
Inspect files
cat · less · head · tail · wc · file · stat — look at file contents and metadata.
essential
Find things
find (name/mtime/size) · fd (modern find) · locate — search for files by attributes.
essential
Search inside files
grep · rg (ripgrep — always prefer) · ack — search for text inside files.
essential
Transform text
sed (stream editor) · awk (mini language) · tr · cut · sort · uniq · wc — the classic pipeline toolkit.
power
JSON / YAML / CSV
jq · yq · mlr (Miller) · csvkit — structured-data equivalents of the classic tools.
power
Processes
ps · top · htop · kill · pgrep · pkill · jobs · fg/bg · nohup — see and control what's running.
system
Network
curl · wget · ss · dig · nslookup · nc — hit URLs, resolve DNS, inspect sockets.
system
Composition
| (pipe) · > < >> 2>&1 · && || · $() (command substitution) · xargs — glue the above together.
glue

The pipeline that answers a real question

You've got an nginx access log with 4 million lines. Question: which 10 URLs got the most HTTP 500 errors in the last hour? In one line:

grep " 500 " access.log \
  | awk '$4 > "[04/Jul/2024:14:00:00" {print $7}' \
  | sort | uniq -c | sort -rn | head -10
1filter
grep " 500 "

Keep only lines with a 500 status code (spaces prevent matching timestamps that contain 500).

2extract
awk time filter + $7

Keep lines after 14:00, print field 7 (the URL). awk splits on whitespace by default.

3prep
sort

Groups identical URLs next to each other — required for uniq to work.

4count
uniq -c

Collapse runs of identical lines into a single line prefixed with the count.

5rank
sort -rn

Sort reverse-numeric — largest counts first.

6present
head -10

Just show the top 10 offenders.

That is the shape of every log-analysis pipeline you'll ever write: filter → extract → aggregate → sort → limit. Learn the shape; the tools you already have handle the rest.

The tools most beginners haven't heard of (but should use)

grep vs ripgrep (rg)

Text search

  • grep — POSIX standard, everywhere
  • rg — 5-100× faster, respects .gitignore
  • rg is smart about UTF-8 and binary files
  • Use rg for interactive search, grep in scripts (for portability)
find vs fd

File search

  • find — POSIX standard, byzantine syntax
  • fd — sane defaults, colored output
  • fd 'session-\d+' vs find . -regex './session-[0-9]+.*'
  • Use fd interactively, find in scripts
cat vs bat

File viewing

  • cat — dump raw bytes
  • bat — cat with syntax highlighting + git diff markers
  • bat is the ‘why haven't I always used this?’ tool
  • Alias `cat=bat` if you like
top vs htop vs btop

Process monitor

  • top — everywhere, ugly, keyboard-driven
  • htop — colorful, mouse-clickable, sane
  • btop — pretty graphs, resource-heavy
  • Use htop day-to-day

The mental model to hold


Common misconception
✗ What most people think

"Commands like ls *.log receive the pattern *.log and expand it themselves. The shell just launches programs."

✓ What is actually true

The shell expands globs before the program ever starts. ls receives an already-expanded list of filenames — it has no idea a * was typed. Same for ~, $VAR, {a,b}, and command substitution.

Why the myth is so sticky

Because the illusion is perfect until it breaks in exactly two ways, and both look like bugs in the program. First: find . -name *.log fails mysteriously — the shell expanded the glob against the current directory before find ran, so find got a filename instead of a pattern. Second: rm * in a huge directory dies with "Argument list too long" — the shell built a multi-megabyte argv that exceeded the kernel's execve limit. Neither makes sense unless you know expansion happens one layer earlier than you thought.

Prove it to yourself

Make the expansion visible — echo shows you exactly what the program would have received:

mkdir -p /tmp/g && cd /tmp/g && touch a.log b.log
echo *.log        # a.log b.log   <- shell already expanded it
echo '*.log'      # *.log         <- quotes suppress expansion

# the classic failure:
find . -name *.log     # breaks: find sees 'a.log'
find . -name '*.log'   # correct: find does its own matching
From first principles
Start with the question

Why can a Unix pipeline of five tiny programs beat a purpose-built script on a 200 GB file — and why does adding sort in the middle sometimes destroy that advantage entirely?

  1. 1
    A pipe is a fixed-size in-kernel ring buffer between two processes, not a file and not a copy of the data.
    forced by · the kernel needs bounded memory regardless of how much data flows through
  2. 2
    Because the buffer is bounded, a fast producer blocks when it fills and a fast consumer blocks when it empties. Backpressure is automatic and requires no cooperation between the programs.
    forced by · a blocking write on a full pipe is the only sane behaviour when you cannot grow the buffer
  3. 3
    Therefore every stage runs concurrently on a different chunk of the stream, and peak memory is O(buffer × stages) — independent of input size.
    forced by · no stage ever needs to hold the whole input to produce its next byte of output
  4. 4
    This works only for streaming operators — ones whose next output depends on a bounded window of input. grep, cut, sed, awk, head all qualify.
    forced by · if output depends on unseen future input, you cannot emit anything yet
  5. 5
    But sort cannot emit its first line until it has seen the last one. It is a blocking barrier: it must buffer everything (spilling to disk when it exceeds memory).
    forced by · the minimum element could be the final line of the file
⇒ Therefore

Therefore a pure streaming pipeline processes arbitrarily large inputs in constant memory with N-way parallelism for free, while any blocking stage collapses that into "materialise everything, then continue".

And note what this predicts: put grep before sort, never after — filtering early shrinks what the barrier must hold. It also predicts why head -1 on a huge file returns instantly while sort | head -1 does not, and why yes | head -5 terminates at all: head exits, the pipe closes, and yes dies of SIGPIPE. This is exactly the same streaming-vs-blocking distinction that governs map-side operations versus shuffles in Spark.

Mental modelEverything is a byte stream through three wires

Every process is born holding three wires: stdin (0), stdout (1), stderr (2). It does not know and cannot ask whether a wire leads to a terminal, a file, another process, or nowhere. The shell's entire job before exec is to solder those wires to the right endpoints.

So >, <, |, 2>&1 are not features of the program — they are wiring instructions to the shell, applied left to right, before the program exists.

  • Order matters because redirections apply sequentially: cmd > f 2>&1 sends both to the file; cmd 2>&1 > f sends stderr to the old stdout (your terminal) and only stdout to the file.
  • Errors go to stderr specifically so they survive being piped — that is why cmd | grep x still shows error messages.
  • Exit status is the other channel: 0 means success, non-zero means failure, and &&/||/set -e all read it. In a pipeline only the last command's status counts unless you set pipefail.
  • Text with one record per line is the universal interchange format — which is why xargs, -print0, and IFS exist to handle the case where filenames contain spaces or newlines.
🔔 Fires when you see

Fire this model the moment you see: output missing from a log file · 2>&1 that didn't work · a pipeline that reports success despite a failed middle stage · "Argument list too long" · a script that behaves differently under cron than in your terminal.

The tradeoff

You have a data-munging task. Do you write a shell pipeline, a Python script, or push it into a query engine?

Shell pipeline
+ you gain zero startup cost, constant memory on unbounded input, free stage-level parallelism, and the C tools are decades-optimised — grep/awk on a local file routinely beat naive Python by a wide margin
− you pay no types, no tests, quoting and whitespace hazards, error handling is manual, and it becomes unmaintainable past roughly a screenful
pick when the task is a one-off or a filter, the data is line-oriented text, and you can write it in under ~15 lines
Python script
+ you gain testable, typed, debuggable, real data structures, and libraries for JSON/Parquet/HTTP that shell handles badly
− you pay interpreter startup, higher per-row cost, and you must implement streaming deliberately (generators) or you will load the file into RAM
pick when the logic has branches, will be re-run by other people, needs tests, or touches formats that are not newline-delimited text
Query engine (SQL / Spark)
+ you gain declarative, a planner that reorders your filters and picks join strategies, and horizontal scale past a single machine
− you pay cluster spin-up and scheduling overhead that dominates for small data; you give up fine control; debugging moves into execution plans
pick when the data no longer fits one machine, or already lives in a table/lake and moving it out would cost more than querying it in place
What a senior engineer actually does

Prototype in the shell, ship in Python, scale in the engine — and be honest about which stage you are in. The shell's real value is the feedback loop: you can inspect 200 GB with head, wc -l, and awk in seconds and learn the shape of the data before writing any code at all.

The mistake seniors avoid in both directions: don't grow a 200-line bash script that should have been Python an hour ago, and don't spin up a cluster for a file that sort -u would have handled on a laptop.


(c) Hands-on · 25 min

Run this end-to-end. It creates a fake nginx log, then walks you through progressively harder questions about it. Save as shell-workshop.sh, chmod +x, run.

#!/usr/bin/env bash# shell-workshop.sh a hands-on tour of pipes, grep, awk, sed, jq.set -euo pipefail WORK_DIR="$HOME/projects/learning/s003-shell"LOG="$WORK_DIR/access.log" log() { printf "\n\033[1;36m %s\033[0m\n" "$*"; } log "1/10 Fresh workspace"rm -rf "$WORK_DIR" && mkdir -p "$WORK_DIR" && cd "

What each block does

Anatomy of the script

Line 4 · set -euo pipefail
Same safety net as S001. Any command failing = script exits. Prevents ‘silent partial success’.
safety
Line 15 · python3 - <<'PY' > access.log
Heredoc with quoted delimiter (no shell expansion inside) + output redirect. A common trick to embed a small program inside a bash script.
shell
Line 41 · awk '{print $7}'
awk auto-splits every line on whitespace into $1..$NF. Field 7 in nginx's default combined format is the URL.
text
Line 43 · sort | uniq -c | sort -rn
The canonical ‘count occurrences of each unique line’ pipeline. Notice `uniq` requires sorted input — that's why the first sort exists.
text
Line 51 · awk -F'[:[]'
`-F` sets the field separator. Here we split on `:` OR `[` — pulling the hour out of `[04/Jul/2024:13:47:22`.
text
Line 55 · awk with associative array
awk is a full programming language. `sums[$7]+=$NF` maintains a running sum per URL. `END {…}` runs after the last line.
text
Line 61 · sed -E 's/regex/replacement/'
Stream editor. `-E` enables extended regex. Capture groups (`\1` etc) let you rearrange fields — here, log lines to JSON.
text
Line 65 · jq -s 'group_by(.status)'
`-s` slurps stdin into a single array first. `group_by` clusters by a key; `map` transforms each group. jq is a full language too — invest one afternoon.
json
Try itAnswer three real questions from a real API in under 3 minutes

Use the public GitHub API (no auth needed for low volume) and jq:

# Q1: names of the last 10 repos user "torvalds" created
curl -s "https://api.github.com/users/torvalds/repos?sort=created&per_page=10" \
  | jq -r '.[].name'
 
# Q2: top 5 most-starred of those repos
curl -s "https://api.github.com/users/torvalds/repos?per_page=100" \
  | jq -r 'sort_by(-.stargazers_count) | .[0:5] | .[] | "\(.stargazers_count)  \(.name)"'
 
# Q3: how many are forks?
curl -s "https://api.github.com/users/torvalds/repos?per_page=100" \
  | jq '[.[] | select(.fork)] | length'
💡 Hint · `jq -r ".[] | .name"` prints raw strings (no quotes). Chain `head -5` at the end to preview. If a field is missing, `//` gives a default: `.company // "none"`.

Six aliases every senior dev has in their .bashrc

# Add to ~/.bashrc — restart shell or `source ~/.bashrc`
alias ll='ls -alF --color=auto'
alias ..='cd ..'
alias ...='cd ../..'
alias grep='grep --color=auto'
alias gs='git status -sb'
alias gl='git log --oneline --graph --all --decorate -20'

These aren't optional. They're the reason a senior dev can gs twenty times a day without their wrists hurting.


(d) Production reality · 15 min

War story Steam · Valve· 2015Users had their entire home directory deleted
🔥 What broke

The Steam Linux client shipped an update with an unquoted variable in a shell script:

rm -rf "$STEAMROOT/"*

When STEAMROOT was unset (a rare install path), the line became rm -rf /* — recursively delete every file on the system.

🧯 The fix
Valve patched the script within a day. The permanent fix would have been the two-line safety belt: set -euo pipefail catches unset variables (-u), and ${'{STEAMROOT:?not set}'} would have aborted with a message instead of expanding to nothing.
🎓 Lesson to steal
Two habits you will never regret: (1) always quote variables in shell — "$var", never bare $var; (2) always start scripts with set -euo pipefail. These two rules would have prevented this incident entirely.
Post-mortem
War story Common failure mode · SRE on-callWrong grep on a 200 GB log file
🔥 What broke

3 am pager. Junior SRE runs grep ERROR huge.log > errors.txt. Two minutes later disk full, half of production is choking on its own logs.

The problem: huge.log was 200 GB, and most lines matched. errors.txt was on the way to 190 GB.

🧯 The fix

Kill the grep. Free disk. Retry with awareness: grep -c ERROR huge.log first to see the count, then grep ERROR huge.log | head -1000 > sample.txt.

Better: use rg (ripgrep), which streams and is faster. Best: ship logs to a real search backend (Loki, Elasticsearch) and grep the index, not the raw file.

🎓 Lesson to steal
On unfamiliar files, always wc -l and du -h first. On any pipeline that writes to disk, always know the upper bound. On production hosts, always df -h before starting anything that grows.
War story A payments team · 20204 hour outage from a rogue cron
🔥 What broke

A nightly cleanup cron ran find /tmp -mtime +7 -exec rm {'{}'} \;. Innocent-looking. But find included /tmp/.mysql.sock, the Unix socket for MySQL. Killed every DB connection at 2 am.

🧯 The fix

Rewrite with a type filter: find /tmp -type f -mtime +7 -delete. Prefer -delete to -exec rm (safer, faster, no per-file fork).

Add a dry-run in every destructive cron: find ... -print for a week before switching to -delete.

🎓 Lesson to steal
Any find ... -exec rm or find ... -delete in production should be paired with (a) a type filter (-type f), (b) a specific name pattern, and (c) a prior week of -print logging. Cleanup crons cause more incidents than actual bugs.

Where this shows up in the rest of the plan

The shell is under every tool you'll use in the next six months
S004 · Reading docs
`man`, `--help`, `tldr`, `cheat` — all shell tools you learn to grep through.
S055 · CI/CD basics
GitHub Actions workflows are essentially remote bash scripts. Every skill here compounds.
S056 · Docker
Dockerfiles are shell + a few new verbs. `RUN` is bash.
S059 · Kubernetes basics
`kubectl get pods -o json | jq …` is 40% of your day as an SRE.
S090 · Observability & SRE
Log grep-fu, `journalctl`, `tail -F | awk` — all built on this session.
S120 · System design
‘Design a log-analysis pipeline’ is a real interview question. The mental model starts here.

(e) Recall + stretch · 10 min

Recall — click each to reveal · click to reveal
★ = stretch question

Explain-out-loud test

If you can't teach these three without notes, redo the session:

  1. What is a pipe, and why is it fast? (mention streams)
  2. What does set -euo pipefail prevent? (name each flag)
  3. When would you use jq instead of Python? (one concrete example)

What comes next

Hub: The 6-Month Learning Plan


Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.