S055 · HTTP Fundamentals — Verbs, Status Codes, Headers, Caching
The protocol every web system speaks, unpacked line by line. Verbs, status codes, headers, and the cache directives that decide whether your API costs $50 or $50,000 a month. With a curl-only walkthrough of a real request.
🎯 Read a raw HTTP request/response by eye, pick the right verb + status code + cache header for any endpoint, and explain how a CDN turns 10k RPS into 100 RPS at origin.
Why this session exists
HTTP is the assembly language of the web. Every API you write, every microservice call, every browser fetch, every webhook, every S3 GET — it's all HTTP. Understanding it at the wire level is the difference between "I glue Express routes together" and "I can debug why your Cloudflare edge is caching a 500". This session takes the messy real protocol (30 years of accumulated headers) and cuts it into the pieces you'll actually use every day.
- Read a raw HTTP request + response and explain every line.
- Choose the right verb (GET/POST/PUT/PATCH/DELETE) with idempotency and safety in mind.
- Pick the correct status code from the 2xx / 3xx / 4xx / 5xx families for a given outcome.
- Write `Cache-Control` + `ETag` headers that let a CDN absorb 99 % of traffic.
- Explain the difference between HTTP/1.1, HTTP/2, and HTTP/3 in one paragraph each.
Prerequisites
- S003 · Command line — you can run
curlwithout help. - S049 · Networking basics (or S061 later) — you know what TCP + a socket is.
(a) Intuition · 5 min
You mail a letter. The envelope has an address (URL), a return address (Host), a class of service (GET vs Registered), and instructions on how long the courier may keep a copy (Cache-Control). Inside the envelope is the actual message body — sometimes empty (a receipt request), sometimes a whole document (a photo upload).
The post office writes a stamp on the reply — 200 (delivered), 404 (address unknown), 500 (post office is on fire). Everything is stateless: each letter is self-contained, and the post office doesn't remember the last one you sent.
An HTTP request is a plain-text envelope: the verb + path on line one, headers on subsequent lines, an optional body after a blank line. The response mirrors it: a status code + reason on line one, headers, blank line, body.
Every load balancer, proxy, CDN, and browser in the world reads exactly that plain-text envelope. Learn to read it and 95 % of ‘why is my request failing?’ debugging becomes obvious.
The pieces you'll use every day
- Verbs — GET (read, safe), POST (create, non-idempotent), PUT (replace, idempotent), PATCH (partial update), DELETE (remove).
- Status codes — 2xx success, 3xx redirect, 4xx client error, 5xx server error. Sub-codes matter: 401 vs 403 vs 404 vs 429.
- Headers — request (Auth, Accept, Content-Type), response (Content-Type, Cache-Control, ETag, Set-Cookie), semantic (Retry-After, Location).
- Bodies — JSON for APIs, form-encoded for old browser forms, multipart for file uploads, streaming for events.
- Versions — 1.1 (text, one request per connection), 2 (binary multiplexed), 3 (UDP+QUIC, no head-of-line blocking).
The 30-year history in five events
- 1991HTTP/0.9 · Tim Berners-Lee at CERNOne-line spec. `GET /page` returned HTML. No headers, no status codes. That was it.
- 1996HTTP/1.0Added headers, status codes, POST. Web goes commercial.
- 1999HTTP/1.1Persistent connections, chunked transfer, Host header (enables virtual hosting). Ruled the web for 15 years.
- 2015HTTP/2 · SPDY becomes standardBinary framing, multiplexing, server push. Kills the ‘six connections per domain’ workaround.
- 2022HTTP/3 · QUIC over UDPBypasses TCP head-of-line blocking. Google, Cloudflare, Meta deploy at scale.
(b) Visual walkthrough · 15 min
Request/response on the wire
The three ‘layers of latency’ you see: DNS (~20 ms), TCP+TLS handshake (~50-100 ms), then the actual request. HTTP/2 and 3 keep the connection warm so subsequent requests skip 2/3 of the round trips.
The verbs table you'll refer to for years
Read a resource · safe · idempotent · cacheable
- No body (usually)
- Must not have side effects on the server
- Can be retried freely
- Ideal for CDN caching
Create / arbitrary action · not idempotent
- Body carries the payload
- May be cached only if explicitly allowed
- Never retry without an idempotency key
- Use for `POST /orders` (creates a new one)
Replace a resource · idempotent
- Body is the full new representation
- Same PUT twice = same result
- Use for `PUT /users/42` (full update)
- Safe to retry on network failure
Partial update · idempotent iff well-designed
- Body describes the changes (JSON Patch, Merge Patch)
- Widely used but under-standardised
- Prefer for `PATCH /users/42` (name change only)
- Retry policy depends on shape
Remove a resource · idempotent
- No body needed
- Second DELETE returns 404 or 204 depending on style
- Safe to retry — deleted-already is still deleted
- Consider soft delete (flag) vs hard delete (row gone)
Status codes, distilled
Which family and why
Caching: the money layer
Browser adds `If-None-Match: <etag>` on repeat visits automatically.
Full hit? Serve from edge, never bother origin. This is the 99 % path.
On miss with an ETag, CDN asks origin ‘still valid?’ Origin replies 304 (small) or 200 (full body).
Origin sends `Cache-Control: public, max-age=60, s-maxage=3600, stale-while-revalidate=86400`.
If origin is down, `stale-if-error` lets the CDN keep serving cached responses instead of returning 5xx.
"HTTP is stateless, so every request is independent. Cookies and sessions are a workaround bolted on top that break statelessness."
Statelessness is a constraint on the protocol, not on the application. The requirement is that the server can interpret any request without reference to prior requests on that connection. Cookies satisfy this: they move the state into the request itself, so every request stays self-describing. Cookies are not an escape from statelessness — they are how sessions exist while preserving it.
The myth is sticky because "stateless" is heard as "no state anywhere", and because the first thing anyone builds with cookies is a session, so cookies feel like a loophole. The distinction becomes operationally real the moment you scale out: because state travels with each request, any of ten identical servers can serve it. That property is precisely why horizontal scaling of web tiers works, and why sticky sessions are a smell rather than a feature.
Watch state travel in the request rather than living on the connection:
curl -v -c jar.txt 'https://httpbin.org/cookies/set?a=1'
curl -v -b jar.txt https://httpbin.org/cookies
# second request carries: Cookie: a=1
curl -v --http1.1 https://example.com https://example.com 2>&1 | grep -i 'reused'Two requests over one reused TCP connection are still independent at the HTTP layer, and two requests on separate connections still share a session. Connection lifetime and session identity are orthogonal.
Why is GET required to be safe and idempotent, and why does violating it break things in ways that feel unrelated to your code?
- 1HTTP was designed for a network with many intermediaries: browsers, proxies, gateways, CDNs. Each one sees requests it did not originate.forced by · the whole architecture assumes a request may be relayed and inspected by parties that know nothing about your application
- 2An intermediary must decide, from the method alone, whether it may cache a response, prefetch a URL, or retry after a timeout. It cannot read your application's intent.forced by · the method is the only universally understood signal of what a request does
- 3Therefore the method must carry a contract:
GETmeans "this has no side effects and may be repeated freely", so intermediaries may cache, prefetch and retry it without asking.forced by · optimisations like caching and prefetching are only safe if repetition is guaranteed harmless - 4If your application performs a side effect on
GET, you have not broken a style rule — you have broken the contract every intermediary is relying on, and they will keep exercising the behaviour you promised was safe.forced by · the intermediaries are correct; your handler is the party that lied - 5The resulting failures appear far from the code: a link prefetcher deletes records nobody clicked, a CDN serves a stale response for an action, a retry after a timeout doubles an effect.forced by · the actor causing the repeat is outside your system and invisible in your logs
Therefore method semantics are load-bearing infrastructure, not documentation. GET/HEAD safe; PUT/DELETE idempotent but not safe; POST neither, which is exactly why nothing caches or auto-retries it.
And note what this predicts: any operation you want to be retriable across a network timeout must be made idempotent, because the client genuinely cannot tell "request lost" from "response lost". That is the entire reason idempotency keys exist on payment APIs — POST gives no retry guarantee, so the application must supply one. The derivation tells you why the pattern had to be invented.
Every HTTP message is an envelope: a verb saying what to do, a URL saying to what, headers describing the contents and conditions, and an optional body. It must be understandable entirely on its own, because it will pass through machines that have never seen your other requests.
The status code is the reply's first line and the only part many intermediaries read. Everything else — caching, retries, compression, routing — is negotiated in headers between parties that do not share application state.
- Status classes are instructions, not decoration: 2xx succeeded, 3xx look elsewhere, 4xx the client must change something before retrying, 5xx the server failed and the same request may succeed later. Returning 200 with an error body defeats every retry policy in the stack.
- Headers are the negotiation channel:
AcceptandContent-Typefor representation,Cache-ControlandETagfor freshness,Authorizationfor identity. Anything an intermediary must act on belongs in a header, not the body — nothing reads bodies. - Connection reuse is a transport optimisation and nothing more. HTTP/1.1 keep-alive removes handshake cost; HTTP/2 multiplexes streams over one connection to remove head-of-line blocking at the HTTP layer. Neither changes request semantics.
- Idempotency is a property of the operation, not of the verb. Declaring a method idempotent obliges your handler to make it so — the protocol states the contract, your code has to honour it.
Fire this model when you see: an action behind a GET link · a client retry that double-charged someone · a proxy serving stale data · an API returning 200 for failures · someone debugging a session bug by looking at TCP connections.
Where does session state live: in a signed token the client carries, or in server-side storage keyed by an opaque session ID?
Start with server-side sessions. They are simpler, revocation is trivially correct, and a session store lookup is cheap until you are running in multiple regions. Move to tokens when cross-region latency or the availability coupling to the session store becomes the actual constraint — not because tokens are the modern-sounding choice.
The mistake worth naming: long-lived self-contained tokens with no revocation path. It looks elegant and it is a security incident waiting to be discovered, because the day you need to invalidate a session immediately, you find the architecture cannot do it at all. Whatever you choose, decide your revocation story before you ship, not after.
(c) Hands-on · 25 min
We'll run a tiny HTTP server, poke it with curl, and watch every header. No frameworks — just the stdlib, so you see exactly what HTTP looks like end-to-end.
"""
http_demo.py — A minimal HTTP server that demonstrates verbs, status codes,
ETags, cache directives, and JSON bodies.
Run:
python http_demo.py
And in another terminal:
curl -v http://127.0.0.1:8080/users/42
curl -v -X POST -d '{"name":"alice"}' -H 'Content-Type: application/json' http://127.0.0.1:8080/users
curl -v http://127.0.0.1:8080/users/999
curl -v -H 'If-None-Match: "v1-42"' http://127.0.0.1:8080/users/42
"""
from __future__ import annotations
import hashlib
import http.server
import json
from datetime import datetime, timezone
USERS: dict[int, dict] = {42: {"id": 42, "name": "alice", "email": "a@x.io"}}
NEXT_ID = 100
def make_etag(payload: dict) -> str:
raw = json.dumps(payload, sort_keys=True).encode()
digest = hashlib.md5(raw).hexdigest()[:8]
return f'"v1-{digest}"'
class Handler(http.server.BaseHTTPRequestHandler):
# Silence default logging
def log_message(self, fmt: str, *args) -> None:
print(f"[{datetime.now(timezone.utc).isoformat()}] " + fmt % args)
# ---------- helpers ----------
def _send_json(self, status: int, payload: dict, extra_headers: dict | None = None) -> None:
body = json.dumps(payload).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
for k, v in (extra_headers or {}).items():
self.send_header(k, v)
self.end_headers()
self.wfile.write(body)
def _parse_id(self) -> int | None:
parts = self.path.strip("/").split("/")
if len(parts) != 2 or parts[0] != "users":
return None
try:
return int(parts[1])
except ValueError:
return None
# ---------- verbs ----------
def do_GET(self) -> None:
uid = self._parse_id()
if uid is None:
self._send_json(400, {"error": "bad path"})
return
user = USERS.get(uid)
if not user:
self._send_json(404, {"error": "not found"})
return
etag = make_etag(user)
if_none_match = self.headers.get("If-None-Match")
if if_none_match == etag:
self.send_response(304)
self.send_header("ETag", etag)
self.end_headers()
return
self._send_json(
200, user,
extra_headers={
"ETag": etag,
"Cache-Control": "public, max-age=60, s-maxage=3600, stale-while-revalidate=86400",
},
)
def do_POST(self) -> None:
global NEXT_ID
if self.path != "/users":
self._send_json(404, {"error": "no route"})
return
length = int(self.headers.get("Content-Length", "0"))
raw = self.rfile.read(length)
try:
payload = json.loads(raw)
except json.JSONDecodeError:
self._send_json(400, {"error": "invalid json"})
return
if "name" not in payload:
self._send_json(422, {"error": "missing field: name"})
return
user = {"id": NEXT_ID, **payload}
USERS[NEXT_ID] = user
NEXT_ID += 1
self._send_json(
201, user,
extra_headers={"Location": f"/users/{user['id']}"},
)
def do_PUT(self) -> None:
uid = self._parse_id()
if uid is None:
self._send_json(400, {"error": "bad path"})
return
length = int(self.headers.get("Content-Length", "0"))
payload = json.loads(self.rfile.read(length))
payload["id"] = uid
USERS[uid] = payload
self._send_json(200, payload)
def do_DELETE(self) -> None:
uid = self._parse_id()
if uid is None or uid not in USERS:
self._send_json(404, {"error": "not found"})
return
del USERS[uid]
self.send_response(204)
self.end_headers()
def main() -> None:
server = http.server.HTTPServer(("127.0.0.1", 8080), Handler)
print("Serving on http://127.0.0.1:8080")
try:
server.serve_forever()
except KeyboardInterrupt:
print("\nBye.")
if __name__ == "__main__":
main()Try each verb with curl:
python http_demo.py # in one terminal
# 200 with a body + ETag + Cache-Control
curl -v http://127.0.0.1:8080/users/42
# 304 Not Modified (conditional GET) — take the ETag from the previous response
curl -v -H 'If-None-Match: "v1-<hash>"' http://127.0.0.1:8080/users/42
# 201 Created with Location header
curl -v -X POST -d '{"name":"bob"}' -H 'Content-Type: application/json' http://127.0.0.1:8080/users
# 422 Unprocessable Entity (validation)
curl -v -X POST -d '{}' -H 'Content-Type: application/json' http://127.0.0.1:8080/users
# 204 No Content (successful delete)
curl -v -X DELETE http://127.0.0.1:8080/users/42Anatomy of the script
What each block teaches
Extend do_GET to switch response body by Accept-Language:
lang = self.headers.get("Accept-Language", "en").split(",")[0][:2]
if lang == "de":
user = {**user, "greeting": "Hallo"}
else:
user = {**user, "greeting": "Hello"}
# BUG: no Vary header! CDN caches only one version.Then run:
curl -H 'Accept-Language: en' http://127.0.0.1:8080/users/42
curl -H 'Accept-Language: de' http://127.0.0.1:8080/users/42Fix by adding Vary: Accept-Language to extra_headers. That header tells caches "vary your key by this request header". Missing it caused a real Netflix outage where users saw the wrong-language homepage for 20 minutes.
(d) Production reality · 15 min
A single regex in a WAF rule went catastrophic on a routine deploy. Every Cloudflare edge server started spinning at 100 % CPU trying to match it — responses backed up, then 502s, then 5xx across every customer using the platform.
Because the edge was the cache layer for millions of sites, users saw complete outages, not degraded pages, for the duration.
Three practices that prevent it:
- Exponential backoff with jitter — never a fixed 1-second retry.
- Only retry idempotent verbs (GET, PUT, DELETE) or POSTs with an idempotency key.
- Respect
Retry-Afteron 429/503 — the server told you when to come back.
Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
Move on when you can teach these:
- Why is HTTP stateless, and how do sessions actually work then?
- What's the difference between PUT, PATCH, and POST for updating a user?
- What does a good Cache-Control header look like for a public product-page endpoint, and why?
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.