S064 · CDN — Edge, Cache Hierarchies, Cache-Control

Module M07: Systems & Infrastructure · Session 64 of 130 · Track: SYS 🌍

What you'll be able to do after this session

• Explain CDN to a friend in one minute, out loud, no notes.
• Recognise it when you see it in real code / systems / papers.
• Complete the hands-on exercise at the end without looking up help.

Prerequisites

If you can't answer these in 30 seconds each, redo the linked session first:

• S063 — Caching — Cache-Aside, Write-Through, TTLs, Invalidation

Pre-read (skim before the session)

• 🎥 Intuition (5–15 min) · What is a CDN? (Cloudflare) — Cloudflare's own 4-minute explainer with the pizza-delivery analogy.
• 🎥 Deep dive (30–60 min) · System Design — CDN (ByteByteGo) — anycast, PoPs, cache hierarchies, purging, all in one hour.
• 🎥 Hands-on demo (10–20 min) · Cloudflare Workers Crash Course (Fireship) — see edge compute + edge caching in ~10 minutes.
• 📖 Canonical article · MDN — HTTP Caching — the reference on Cache-Control, ETag, revalidation.
• 📖 Book chapter / tutorial · Google Web Fundamentals — HTTP Cache — hands-on tutorial with concrete header recipes.
• 📖 Engineering blog · Netflix — Open Connect Overview — how Netflix runs its own CDN because commercial ones weren't enough.

(a) Intuition · 5 min

The one-paragraph mental model. Why this concept exists, what problem it solves, and the analogy that makes it stick.

In one sentence: Move bytes close to users.

Imagine your website is a single pizza shop in New York, and customers are ordering from Sydney, Mumbai, and São Paulo. Every order takes 20 hours to deliver, the pizza is cold, and the shop is overwhelmed. Now imagine you open small identical franchises in each city, keep the popular pizzas pre-made in the freezer, and only phone the main shop for weird custom orders. Delivery drops from 20 hours to 15 minutes and the main shop can breathe. That's a CDN — Content Delivery Network — a global fleet of servers that keep copies of your content close to users, in points of presence (PoPs) scattered around the world.

Cloudflare, Fastly, Akamai, AWS CloudFront and Google Cloud CDN each run hundreds of PoPs — often in the same building as your ISP, sometimes literally inside it. When a user in Mumbai requests blog.example.com/logo.png, DNS/anycast routes them to the nearest PoP (say Mumbai). If that PoP has logo.png cached, it's served in ~10 ms. If not, the PoP fetches it from your origin (say, an S3 bucket in Virginia), caches it, and serves it. Every subsequent Mumbai user gets the fast path.

The magic ingredient is cache-hierarchy: PoP → regional shield → origin. A cold PoP asks a bigger regional cache first, so your origin only ever sees one request per file per region, no matter how many end users request it. This is called origin shielding and it's why a viral video doesn't take down your S3 bucket.

Forever-gotcha: A CDN caches whatever the origin's response headers tell it to cache. If you don't set Cache-Control, defaults kick in — and those defaults may cache private data, or refuse to cache static assets. Every production incident with "the CDN served the wrong thing" traces back to sloppy Cache-Control on the origin.

(b) Visual walkthrough · 15 min

Diagrams, worked examples, step-by-step visuals.

Key HTTP cache headers — the ones you must know:

Worked example — cache-miss vs cache-hit flow:

1. First user in Mumbai requests /logo.png. Mumbai PoP: MISS.
2. Mumbai PoP checks APAC shield: MISS.
3. Shield fetches from origin (Virginia, ~250 ms). Origin returns 200 with Cache-Control: public, max-age=86400.
4. Shield caches it, returns to Mumbai PoP.
5. Mumbai PoP caches it, returns to user. Response header: X-Cache: MISS from mumbai; HIT from shield.
6. Second Mumbai user requests /logo.png a minute later. Mumbai PoP: HIT. ~10 ms.
7. 24 h later the entry expires. Next user triggers a revalidation — Mumbai PoP asks shield "still valid?" If ETag matches, shield returns 304 and PoP resets its TTL without transferring the bytes.

Worked example — invalidation. You deploy logo.png with a bug fix. Two options:

• Purge: call the CDN API purge_url("/logo.png"). All PoPs drop the entry within seconds. Real but eventually consistent (a few seconds of skew).
• Versioned URL: rename to logo.v2.png and update HTML. The old URL keeps serving the old (cached) file; the new URL is a fresh miss. This is how every modern frontend build (Vite, webpack) works — the hashed filename is the cache-busting trick.

(c) Hands-on · 20 min

Code you type and run. Not pseudo-code — real, runnable, copy-pasteable.

A local walkthrough of Cache-Control and ETag behaviour using curl against a real CDN-fronted asset, plus a tiny Flask origin so you can see the round trip.

# Part A: inspect a real CDN response
curl -sI https://cdn.jsdelivr.net/npm/react@18/package.json | grep -iE 'cache-control|age|etag|x-cache|via|x-served'
 
# Look for:
#   cache-control: public, max-age=... immutable
#   age: <how long this PoP has held it>
#   etag: "<hash>"
#   x-cache: HIT / MISS
 
# Part B: revalidation
ETAG=$(curl -sI https://cdn.jsdelivr.net/npm/react@18/package.json | awk -F'"' '/^etag/i{print $2}')
echo "etag=$ETAG"
curl -sI -H "If-None-Match: "$ETAG"" https://cdn.jsdelivr.net/npm/react@18/package.json | head -1
# Should say: HTTP/2 304

Now a tiny origin that emits proper cache headers, so you can watch how a CDN would behave in front of it:

# origin.py — pip install flask
from flask import Flask, make_response, request
import hashlib, time
app = Flask(__name__)
 
CONTENT = b"hello from origin
"
ETAG = '"' + hashlib.md5(CONTENT).hexdigest() + '"'
 
@app.get("/asset.txt")
def asset():
    # Conditional revalidation
    if request.headers.get("If-None-Match") == ETAG:
        return ("", 304)
    resp = make_response(CONTENT)
    resp.headers["Cache-Control"] = "public, max-age=60, s-maxage=3600, stale-while-revalidate=30"
    resp.headers["ETag"] = ETAG
    resp.headers["X-Generated-At"] = str(time.time())
    return resp
 
@app.get("/user")
def user():
    resp = make_response(f"secret for user {request.args.get('id')}
")
    resp.headers["Cache-Control"] = "private, no-store"
    return resp
 
if __name__ == "__main__":
    app.run(port=5001)

Run python3 origin.py and in another terminal:

curl -sI http://127.0.0.1:5001/asset.txt
curl -sI -H 'If-None-Match: "..."' http://127.0.0.1:5001/asset.txt   # paste the ETag
curl -sI http://127.0.0.1:5001/user?id=42

Checklist — what to observe:

1. /asset.txt returns Cache-Control: public, max-age=60, s-maxage=3600, … — browsers cache 60 s, CDNs cache 1 h.
2. With the correct If-None-Match, the origin returns 304 Not Modified and no body — that's bandwidth saved.
3. /user returns Cache-Control: private, no-store — CDNs will refuse to cache it, so different users won't see each other's data.
4. On the jsDelivr response, notice age: grows with each request from the same PoP and x-cache: HIT — that's a live CDN cache in action.
5. Change max-age=60 to max-age=0, must-revalidate and see how every request triggers a revalidation round-trip.

Try this modification: add a route /logo.png?v=1 and /logo.png?v=2 that both serve different bytes. Confirm they cache as separate entries. This is the "versioned URL" cache-busting pattern used by every SPA build tool.

(d) Production reality · 10 min

How this shows up in real systems, gotchas, war stories, common bugs.

War story #1 — the private data leak. A newspaper site put Cache-Control: public, max-age=3600 on every response, including the logged-in user's "My Account" page. The CDN cached user A's account HTML and served it to user B who happened to hit the same PoP. GDPR incident, PR nightmare. Rule: default all authenticated endpoints to Cache-Control: private, no-store and only opt in for pure static content.

War story #2 — the Vary trap. A site sent Vary: User-Agent. That splits the cache by exact UA string — thousands of variants per URL. Hit rate collapsed to near zero, origin traffic went 100x. Rule: only Vary on things with a small set of values (Accept-Encoding, a stable feature-flag cookie).

War story #3 — the immutable footgun. A team shipped Cache-Control: max-age=31536000, immutable on app.js. Then found a bug. Because browsers literally skip the revalidation check with immutable, users had to hard-refresh for a week. Fix: only use immutable on hashed filenames (app.abc123.js) that will get a new name on the next build.

Common CDN gotchas at scale:

• Purge storms: deploying = purging 10 k URLs at once = cold caches everywhere = origin gets slammed. Use "soft purge" (mark stale, revalidate on next request) or trickle purges.
• Origin shielding matters: without it, every PoP misses independently and you get N requests to origin per cold asset. With it, 1 request per region.
• Anycast quirks: BGP changes can flip which PoP serves a user mid-session. Session-affinity via cookies only, never via IP.
• HTTPS certs at edge: SNI, cert renewal, and per-hostname configs are non-trivial. Cloudflare handles this transparently; DIY (nginx + Let's Encrypt) is a whole ops role.
• Log tail: CDN access logs are your single biggest source of user-behaviour data. Netflix, Cloudflare and Fastly all built entire analytics products on it.

Netflix runs its own CDN (Open Connect) with boxes literally installed inside ISPs — commercial CDNs weren't cost-effective at their scale. The write-up is worth reading top to bottom.

(e) Quiz + exercise · 10 min

1. What's the difference between max-age and s-maxage?
2. Explain what ETag + If-None-Match gets you that plain TTL doesn't.
3. Why is versioned filenames (app.abc123.js) better than API-based purge for a fast-moving SPA?
4. What is "origin shielding" and what problem does it solve?
5. Name one thing you should never set Cache-Control: public on.

Stretch problem (connects to S063 — Caching): map each of the four caching pattern (cache-aside, write-through, write-back, read-through) from S063 onto CDN behaviour. Which one does a CDN implement by default? Which one would you want if you were running a CDN in front of a POST-heavy API (bad idea — but explain why)? ~8 lines.

Explain-out-loud test

If you can't teach these 3 things to a friend without notes, redo the session:

1. What is CDN? (one sentence, no jargon)
2. When would you use it? (one concrete example)
3. When would you NOT use it? (one anti-pattern)

What comes next

• Next session (S065): Docker — Images, Layers, Dockerfile, Networking
• Previous (S063): Caching — Cache-Aside, Write-Through, TTLs, Invalidation
• Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: (a) intuition · (b) visual walkthrough · (c) hands-on · (d) production reality · (e) quiz + exercise. Duration: 60 minutes.