S064 · CDN — Edge, Cache Hierarchies, Cache-Control
How your static asset travels 40 ms to Sydney instead of 400 ms — edges, origins, cache hierarchies, and the four Cache-Control directives you'll set every day.
🎯 Set correct Cache-Control on every response, understand origin shield + cache hierarchies, and configure a Cloudflare-style CDN in front of an origin — with purge working.
Why this session exists
A CDN is the highest-leverage infrastructure decision you'll make: turn it on and your global P95 drops from 400 ms to 40 ms overnight, your origin's egress bill drops 90%, and you gain DDoS protection almost for free. Turn it on incorrectly and you cache personal data at the edge, or your users see yesterday's home page for a month. This session teaches both sides — the mental model AND the exact Cache-Control incantations.
- Explain edge · shield · origin hierarchy, and what problems each layer solves.
- Set Cache-Control, ETag, and Vary correctly on static assets and API responses.
- Purge a CDN cache surgically without nuking every object.
- Diagnose the top-4 CDN bugs: caching authenticated responses, missing Vary, over-long TTLs, and cache-key mismatches.
Prerequisites
- S015 · HTTP basics — headers, methods, status codes.
- S062 · Load balancers — reverse-proxy mental model.
- S063 · Caching strategies — cache-aside, TTL, invalidation.
(a) Intuition · 5 min
There is one Library of Congress in Washington DC that owns every book. If everyone on Earth had to fly there to read one page, reading would collapse.
Instead, thousands of local bookstores keep copies of the popular titles. You walk in, grab a copy, done. If the store doesn't have what you want, they order it from the central library — usually once — and now every subsequent local reader gets a copy from the store too.
Bookstores stock the top 1% of titles because that 1% accounts for 90% of demand. The long tail still exists at the central library; you just wait longer.
Your origin is the Library of Congress — one place (or two, one per region) that has every asset. Your CDN edges are the local bookstores — hundreds or thousands of PoPs (points of presence) around the world, each holding whatever's been recently requested nearby.
The first request from Sydney for /logo.png travels to your origin in Virginia (~200 ms), gets cached at the Sydney PoP, and every subsequent Sydney user gets it in 15 ms. This is why static-heavy sites feel identical everywhere on Earth: the bytes never leave the reader's continent.
What a CDN actually gives you
- Latency reduction — bytes come from ~50 km away, not ~5000 km. Global P95 drops 10×.
- Origin offload — 95%+ of requests never touch your server. Egress bandwidth bill drops accordingly.
- DDoS absorption — the edge fleet has 200+ Tbps aggregate. Attacks that would flatten your origin bounce off.
- TLS termination at edge — one place to rotate certs; clients get sub-100 ms TLS handshake.
- Programmable edge — modern CDNs run your code (Cloudflare Workers, Lambda@Edge) 30 ms from the user.
A quick history so the products make sense
- 1998Akamai founded · MIT spin-offFirst commercial CDN. Solves ‘the internet is slow’ for early web publishers.
- 2008CloudFront launchesAWS makes CDN a self-service commodity. Prices collapse.
- 2010Cloudflare launchesFree tier + DDoS protection. Onboards millions of sites in a few years.
- 2017Cloudflare Workers · edge computeV8 isolates at the edge. Your code runs 30 ms from any user, no cold starts.
- 2020QUIC + HTTP/3 rolloutUDP-based, 0-RTT resumption. CDNs deploy globally before origins do — another edge win.
(b) Visual walkthrough · 15 min
The cache hierarchy — three tiers
Why the middle tier exists. Without a shield, every one of the 200 edge PoPs misses independently and hits your origin. On a cold cache after purge, that's 200 simultaneous fetches for one asset. The shield collapses those 200 misses into 1 fetch to origin. Cloudflare calls this "tiered caching"; Fastly calls it "origin shield"; CloudFront calls it just "origin shield" too. Turn it on — always.
Anatomy of a single cached response
The four headers that decide EVERYTHING about caching
The Cache-Control directives you need memorised
Hashed filename · cache forever
- Cache-Control: public, max-age=31536000, immutable
- Filename includes content hash: /app-a3f9c2.js
- New deploy = new hash = new URL — no invalidation needed
- Standard for JS/CSS bundles, fonts, hashed images
Short TTL · validators
- Cache-Control: public, max-age=60, must-revalidate
- Or: no-cache (fetch, but 304 is allowed)
- ETag with content hash so 304 responses are cheap
- Standard for the shell that loads your hashed assets
Short TTL · Vary carefully
- Cache-Control: public, max-age=30, s-maxage=300
- s-maxage overrides max-age at the CDN specifically
- Vary: Accept-Encoding, Accept-Language
- Good fit: product catalogues, public feeds
Never cache at CDN
- Cache-Control: private, no-store
- ‘private’ = browser MAY cache; ‘no-store’ = don't even
- Skip this and you'll serve user A's data to user B
- Set explicitly on every /api/me, /account, /admin route
The 90/9/1 rule you can hold in your head
Static assets + long-TTL API responses. Served in 20 ms from the nearest PoP.
Warm asset that expired at edge but is still fresh at the regional shield. ~60 ms.
Cold or purged content. ~150 ms. This is where your origin bandwidth bill lives.
During a cache purge, tiered caching collapses N edge misses → 1 shield fetch → 1 origin fetch.
"A CDN is for static assets — images, CSS, JavaScript. Dynamic and personalised content can't be cached, so a CDN can't help there."
Static caching is the smallest benefit. A CDN terminates TLS close to the user, which removes multiple round trips from every request including uncacheable ones; it keeps warm, congestion-window-mature connections to your origin so your dynamic content rides an already-fast pipe; it absorbs volumetric attacks; and with edge compute it can assemble personalised responses from cached fragments. Even a pure pass-through for a 100% dynamic API is usually meaningfully faster.
The myth is sticky because caching is the feature CDNs are sold on, and it is the one you configure explicitly, so it becomes the mental summary of the whole product. The connection-level benefits are invisible: nothing in a dashboard says "you saved two round trips on TLS negotiation", so the largest win for dynamic traffic is the one nobody attributes.
Measure where the time actually goes on an uncacheable request, with and without the edge:
curl -w 'dns:%{time_namelookup} connect:%{time_connect} tls:%{time_appconnect} ttfb:%{time_starttransfer} total:%{time_total}\n' \
-o /dev/null -s https://your-api.example.com/dynamic
# compare the gap between time_appconnect and time_starttransfer
# (origin processing) with everything before it (connection setup).
# On a distant origin, setup often dominates - and that is exactly
# the part an edge POP removes, cacheable or not.Why does physical distance to a server matter so much, when a request is only a few kilobytes and links are gigabit?
- 1Signals in fibre travel at roughly two-thirds the speed of light, so distance imposes a hard minimum latency that no engineering can reduce.forced by · it is a physical constant, not an implementation detail — London to Sydney is about 80 ms one way at best, and real routes are longer than great-circle distance
- 2Establishing a connection is not one exchange. TCP needs a handshake, TLS needs its own, and only then does the request go out.forced by · each layer must confirm the peer before trusting it, and confirmation requires a message and a reply
- 3Therefore the fixed cost of a request is several round trips, and each round trip pays the full distance penalty twice.forced by · latency multiplies by the number of sequential exchanges, and none of them can be skipped or parallelised
- 4TCP slow start then compounds this: a new connection cannot use available bandwidth immediately, so a large response takes several further round trips to ramp up.forced by · the sender must probe capacity incrementally, as derived in the networking session
- 5So the only way to reduce this is to shorten the distance over which those round trips occur — terminate the connection near the user and let the long-haul segment run over a pre-established, already-warmed connection.forced by · you cannot reduce the number of round trips below the protocol's minimum, so you must reduce the cost of each one
Therefore a CDN's primary mechanism is moving the connection setup close to the user, not storing files. Caching is a further optimisation layered on top of that.
And note what this predicts: any protocol change that removes a round trip should produce a benefit comparable to moving the server closer. That is exactly what TLS 1.3 (one round trip instead of two) and QUIC/HTTP-3 (combining transport and crypto handshakes, with 0-RTT resumption) deliver. It also predicts that the benefit of an edge is largest for chatty, small-payload, high-latency workloads — mobile users on distant networks — and smallest for a single bulk transfer, where the ramp cost is amortised across many megabytes.
Your origin stays where it is. The CDN puts a front door in every city: the user's expensive handshake happens metres away, and the long journey to the origin happens over a connection the CDN already established and warmed. If the answer is already at the front door, the journey never happens at all.
Cache keys are the whole configuration problem. A cache key that includes something varying per user gives you a 0% hit rate; a key that omits something important serves one user's content to another.
- The origin controls caching via headers, and the two axes are separate:
Cache-Control: max-agefor browsers,s-maxagefor shared caches, andstale-while-revalidateto decouple user latency from revalidation. Setting onlymax-agemeans you cannot purge from users' browsers — one reason long browser TTLs are dangerous and long edge TTLs are safe. - Normalise the cache key aggressively: strip tracking query parameters, restrict
Varyto headers that genuinely change the response. Every additional dimension in the key multiplies the number of cached variants and divides your hit rate. - Cache-busting belongs in the URL. Immutable, content-hashed filenames with a very long TTL are strictly better than short TTLs plus purging, because they make the cache key change when the content does — no invalidation required.
- Edge compute changes the shape of what is cacheable: cache the page, personalise a fragment at the edge. This converts "this page is personalised so nothing is cacheable" into "99% of this page is cacheable", which is usually the difference that matters.
Fire this model when you see: a hit rate near zero on assets that should never change · users in one region reporting slowness · a deploy where old assets are still being served · one user seeing another's personalised content · an origin taking traffic it should never see.
How long should the edge TTL be for content that changes unpredictably?
Use immutable URLs for anything produced by a build, and long TTL plus purge for content managed through a CMS or admin interface where the write path is small and known. Reserve short TTLs for content that changes continuously and where purging every change would be its own volume problem.
The mistake worth avoiding is a uniform TTL policy across all content. Different content has genuinely different change rates and different staleness costs, and one number cannot be right for all of it. Set the policy per content class, and — critically — keep browser TTLs short even where edge TTLs are long: you can purge an edge, but you can never reach into a million browsers to correct a mistake.
(c) Hands-on · 25 min
We'll build a tiny origin server that sets Cache-Control correctly for each route type, run a local Varnish (a widely-used HTTP cache) in front of it to simulate a CDN edge, and watch the behaviour with curl. Then we'll do a targeted purge and see it work.
What each block does
Anatomy of the origin + VCL
Add gzip encoding when the client requests it, but forget the Vary: Accept-Encoding header. Then:
# Client 1 asks for gzip:
curl -sH "Accept-Encoding: gzip" http://127.0.0.1:8080/api/catalogue | file -
# Cache now stores the gzipped body under key `/api/catalogue`.
# Client 2 does NOT ask for gzip:
curl -s http://127.0.0.1:8080/api/catalogue
# You'll see raw gzip bytes — the browser would show garbage.Add Vary: Accept-Encoding and the cache keys become (/api/catalogue, gzip) vs (/api/catalogue, identity) — two separate entries, correct content to each client. This is one of the top-3 CDN bugs in the wild.
(d) Production reality · 15 min
On December 25 2015 Steam was under DDoS. Their CDN (a bespoke setup) had cache rules tuned for the attack. A misconfiguration made the edge cache authenticated pages — including account details, phone numbers, and last four digits of credit cards.
Users refreshing the store page were served the last cached response, regardless of who they were logged in as.
Set-Cookie, Authorization, or a session cookie is never cacheable at the edge — enforced in VCL, not left to origin discipline.Cache-Control: public, max-age=86400 and no purge was issued. Users see the old page all day. Support inbox explodes.POST /purge_cache) with the list of changed URLs. Or use tag-based purging: Cache-Tag: home-v2 on the response, purge by tag on deploy. Fastly and Cloudflare Enterprise both support this natively.Common failure modes to internalise
- Caching set-cookie responses — the cache serves user A's session cookie to user B. Every LB/CDN default rule should refuse to cache responses with
Set-Cookie. - Missing Vary — same URL returns different bodies based on
Accept-Encoding,Accept-Language, orAuthorization. Cache stores one, serves it to everyone. Fix:Varyheader lists which request headers partition the cache. - Cache key too coarse — a URL like
/api/products?category=x&sort=y— different query strings should be different cache entries. Most CDNs cache by full URL including query string by default; verify yours. - Cache-Control set from framework defaults — Django/Express/etc. often ship
Cache-Control: no-cacheon every response until you configure otherwise. You'll get 0% cache hit ratio and wonder why the CDN "isn't working."
Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
- How does a CDN reduce global P95 latency by 10×? (edge PoPs + hierarchy)
- Which single header decides whether a response gets cached at the edge, and what are the four values you'll set most often?
- What are two ways to invalidate a CDN cache, and when do you prefer versioned URLs over purge?
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.