S055 · HTTP Fundamentals — Verbs, Status Codes, Headers, Caching

Module M06: Backend & APIs · Session 55 of 130 · Track: SYS 🌐

What you'll be able to do after this session

• Explain HTTP to a friend in one minute, out loud, no notes.
• Recognise it when you see it in real code / systems / papers.
• Complete the hands-on exercise at the end without looking up help.

Prerequisites

If you can't answer these in 30 seconds each, redo the linked session first:

• S003 — Big-O & Complexity Analysis

Pre-read (skim before the session)

• 🎥 Intuition (5–15 min) · HTTP explained in 100 seconds — Fireship — the fastest possible mental model.
• 🎥 Deep dive (30–60 min) · HTTP crash course — Web Dev Simplified — verbs, status codes, headers, caching, and cookies end-to-end.
• 🎥 Hands-on demo (10–20 min) · curl tutorial — NetworkChuck — inspect requests/responses by hand; you'll never forget the shape again.
• 📖 Canonical article · MDN — HTTP overview — 20-minute skim; MDN is the reference the whole industry cites.
• 📖 Book chapter / tutorial · High Performance Browser Networking — Ch. 9 HTTP (free online) — Ilya Grigorik's free book; the caching + connection sections are gold.
• 📖 Engineering blog · Cloudflare — HTTP status codes explained — clear plain-English tour with real examples.

(a) Intuition · 5 min

The one-paragraph mental model. Why this concept exists, what problem it solves, and the analogy that makes it stick.

In one sentence: The protocol every web system speaks.

HTTP is basically the world's most successful letter-writing format. Client (browser, app, curl) sends a letter with a verb ("GET this thing", "POST this form", "DELETE that row") plus a return address (headers). Server reads the letter, does the work, mails back a reply with a three-digit status code (200 = ok, 404 = never heard of it, 500 = my bad, I broke), plus optional body content. Every website, mobile app, microservice call, webhook, REST API, GraphQL endpoint, and even most gRPC traffic ultimately rides on top of this same letter-writing convention.

Key mental model: HTTP is stateless and request/response. Server forgets you the moment it replies. If you want continuity (login, cart), you carry a token (cookie, JWT) on every letter. This "server has no memory between requests" is the property that lets Google run one service across a million machines — any of them can answer your next request, because there's no per-user context stuck to a particular server.

The verbs (methods) each mean something specific: GET = fetch (safe, cacheable, no side effects); POST = create / do a mutation; PUT = replace entirely; PATCH = update partially; DELETE = remove. Status code groups: 1xx informational, 2xx success, 3xx redirect, 4xx you screwed up, 5xx I screwed up. Memorising the top ~15 (200, 201, 204, 301, 302, 304, 400, 401, 403, 404, 409, 429, 500, 502, 503) covers 99% of what you'll see.

Forever gotcha: GET is supposed to be safe and idempotent — calling it 100 times must not change server state. Treating GET /delete_user?id=5 as a real endpoint is a classic bug that led to Google's crawler once mass-deleting a wiki because it followed every link. If it mutates, it must not be GET.

(b) Visual walkthrough · 15 min

Diagrams, worked examples, step-by-step visuals.

The shape of every HTTP transaction:

The must-know status codes (memorise these 15):

Headers you'll touch every week:

• Content-Type — tells the other side how to parse the body (application/json, text/html, multipart/form-data).
• Authorization — carries credentials (Bearer <jwt>, Basic <b64>).
• Accept — what the client can parse; enables content negotiation.
• Cache-Control — the caching contract (max-age=60, no-store, private, public, must-revalidate).
• ETag / Last-Modified — versioning tokens for conditional requests → 304 responses.
• Set-Cookie / Cookie — how servers persist per-user state.
• X-Forwarded-For — original client IP when behind a proxy / load balancer.

Caching decision tree:

1. Client has a cached copy with a fresh max-age? → use cache, no request.
2. Cached copy stale but has ETag? → send If-None-Match: <etag> → server returns 304 (no body) or 200 (new body).
3. No cache? → full request, full response.

That's the whole HTTP-caching model. Cloudflare, Varnish, browsers, and CDNs all implement this state machine.

HTTP versions in one line each:

• HTTP/1.1 (1997): text protocol, one request per connection (or keep-alive), head-of-line blocking. Still ~40% of traffic.
• HTTP/2 (2015): binary framing, multiplexing many streams over one TCP connection, header compression (HPACK). Massive latency win.
• HTTP/3 (2022): same as HTTP/2 but over QUIC (UDP) instead of TCP. Solves TCP head-of-line blocking; better on mobile networks.

Semantics are identical across versions — verbs, headers, statuses are the same. Only the wire encoding changed.

(c) Hands-on · 20 min

Code you type and run. Not pseudo-code — real, runnable, copy-pasteable.

# Everything below uses curl + a tiny local server. No installs beyond python3.
 
# 1. Start a real HTTP echo server in one process
python3 -m http.server 8000 --directory /tmp &
SERVER_PID=$!
sleep 1
 
# 2. First GET — verbose output shows the whole exchange
curl -v http://127.0.0.1:8000/ 2>&1 | head -30
 
# 3. Send custom headers
curl -H 'X-Trace-Id: abc123' -H 'Accept: text/html' http://127.0.0.1:8000/ -o /dev/null -w 'status=%{http_code} time=%{time_total}s\n'
 
# 4. Try a 404
curl -w 'status=%{http_code}\n' http://127.0.0.1:8000/does-not-exist
 
# 5. POST with JSON body (this server won't accept but you'll see the shape)
curl -X POST -H 'Content-Type: application/json' -d '{"name":"Alice"}' http://127.0.0.1:8000/api -w 'status=%{http_code}\n'
 
kill $SERVER_PID
 
# 6. Now hit a real API to see caching in action
curl -v https://api.github.com/repos/torvalds/linux 2>&1 | grep -Ei '^< (HTTP|etag|cache-control|last-modified)'
 
# 7. Conditional request — should return 304
ETAG=$(curl -sI https://api.github.com/repos/torvalds/linux | grep -i '^etag:' | cut -d' ' -f2 | tr -d '\r')
echo "Got ETag: $ETAG"
curl -w 'status=%{http_code}\n' -o /dev/null -H "If-None-Match: $ETAG" https://api.github.com/repos/torvalds/linux
 
# 8. Redirect chain
curl -sIL http://github.com | grep -Ei '^HTTP|^location:'
 
# 9. Rate limit inspection
curl -sI https://api.github.com/repos/torvalds/linux | grep -Ei '^x-ratelimit'

What to observe:

1. curl -v shows lines starting with > (what you sent) and < (what came back) — that's raw HTTP wire format.
2. Step 4 returns status=404 with an HTML error body — servers can put whatever they want in the 404 body.
3. Step 6 returns headers like etag: W/"..." and cache-control: public, max-age=60 — that's how GitHub tells you "safe to cache for 60s."
4. Step 7 with matching If-None-Match returns status=304 and zero body bytes — huge bandwidth win.
5. Step 8 shows HTTP/2 301 then HTTP/2 200 — the redirect chain from http://github.com to https://github.com/.
6. Step 9 shows x-ratelimit-remaining: 55 (or similar) — the rate-limit budget for your IP.

Try this modification: hit the same GitHub endpoint 60 times in a for loop and watch x-ratelimit-remaining count down. When it hits zero, watch the status code flip to 403 or 429. That's rate limiting in action — the exact same signal you'd handle in a production HTTP client with exponential backoff.

(d) Production reality · 10 min

How this shows up in real systems, gotchas, war stories, common bugs.

Every production HTTP client / server does the same handful of things: retries with exponential backoff on 5xx and 429, connection pooling (never open a new TCP connection per request — that's slow), timeouts (connect, read, total — set all three), and structured logging of method + path + status + duration + trace_id.

War stories:

• The retry stampede. Payment service returns 500 briefly during a deploy. 200 clients all retry immediately, no jitter. Second wave hits the recovering server, kills it again. Fix: exponential backoff with jitter (random 0–2× the delay). This is the single most important pattern in distributed HTTP.
• The 200 OK that isn't. Backend catches all exceptions and returns {"status": "ok"} with HTTP 200 even when the DB write failed. Client dashboards showed "success 100%" while nothing was actually saved. Fix: HTTP status must match reality. 200 = actually worked. 4xx = client's fault. 5xx = server's fault. Don't lie in status codes.
• The unbounded response body. A GET /users endpoint returned all 40 million users when no ?limit= was passed. First call OOM'd the server. Fix: enforce a max page size at the API layer, not in the docs.
• The header-injection XSS. Server echoed X-Client-Id into a Set-Cookie without sanitising newlines. Attacker injected \r\nSet-Cookie: session=hijacked and stole sessions. Fix: strip CR/LF from all header inputs; use a library that handles this.
• The cache-poisoning incident. CDN cached an authenticated GET /me response and served it to another user. Fix: authenticated responses need Cache-Control: private, no-store, and the cache key must include Authorization or the identity subkey.
• The 502 no-one owned. Nginx returned 502 → developers blamed the app team → app team said "app is fine, look at 200s in logs." Turns out Nginx's proxy_read_timeout (60s default) killed a slow query. Fix: end-to-end tracing so you can see which hop in the chain caused the 502.

How top teams handle it: every service exposes a /health endpoint (used by load balancers), a /metrics endpoint (Prometheus format), and enforces a consistent error envelope ({error: {code, message, request_id}}). RFC 7807 "problem+json" is a good standard error format. All requests carry a traceparent header (W3C Trace Context) that follows the request across every service.

Gotcha to memorise: timeouts default to infinite in most HTTP libraries. Requests without an explicit timeout are a production time bomb — a slow upstream can pin your worker thread forever, and eventually your whole pool. Set connect+read timeouts on every HTTP client call.

(e) Quiz + exercise · 10 min

1. What does "HTTP is stateless" mean, and why is it useful for scaling?
2. When would you return 201 vs 200 vs 204 from an endpoint?
3. Explain the flow of a conditional GET with If-None-Match and how it saves bandwidth.
4. Why is GET /users/delete?id=5 a dangerous URL to expose?
5. What are the three timeouts you should always set on an HTTP client?

Stretch (connects to S050 — Airflow): Design a retry policy for an Airflow task that calls a flaky external HTTP API. Which status codes should you retry, which should you not, and how do you avoid a retry stampede when the API is recovering from an outage?

Explain-out-loud test

If you can't teach these 3 things to a friend without notes, redo the session:

1. What is HTTP? (one sentence, no jargon)
2. When would you use it? (one concrete example)
3. When would you NOT use it? (one anti-pattern)

What comes next

• Next session (S056): REST API Design — Resources, Versioning, Idempotency
• Previous (S054): Governance & Cost — Lineage, PII, Attribution
• Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: (a) intuition · (b) visual walkthrough · (c) hands-on · (d) production reality · (e) quiz + exercise. Duration: 60 minutes.