S063 · Caching — Cache-Aside, Write-Through, TTLs, Invalidation

Module M07: Systems & Infrastructure · Session 63 of 130 · Track: SYS ⚡

What you'll be able to do after this session

• Explain Caching to a friend in one minute, out loud, no notes.
• Recognise it when you see it in real code / systems / papers.
• Complete the hands-on exercise at the end without looking up help.

Prerequisites

If you can't answer these in 30 seconds each, redo the linked session first:

• S055 — HTTP Fundamentals — Verbs, Status Codes, Headers, Caching

Pre-read (skim before the session)

• 🎥 Intuition (5–15 min) · Caching — System Design Basics (ByteByteGo) — 8-minute animation of cache-aside vs write-through vs write-back.
• 🎥 Deep dive (30–60 min) · Redis Caching Strategies (Redis official) — the canonical patterns straight from the Redis team.
• 🎥 Hands-on demo (10–20 min) · Redis Crash Course (Web Dev Simplified) — spin up Redis and cache real API calls in 20 min.
• 📖 Canonical article · AWS — Caching Best Practices — Amazon's reference on cache patterns, TTLs, eviction.
• 📖 Book chapter / tutorial · Redis docs — Client-side caching — the modern tutorial including tracking and invalidation.
• 📖 Engineering blog · Facebook — Scaling Memcache at Facebook — the classic paper on lease keys, thundering herds, and cache invalidation at planet scale.

(a) Intuition · 5 min

The one-paragraph mental model. Why this concept exists, what problem it solves, and the analogy that makes it stick.

In one sentence: The oldest performance trick, done right.

You already cache things every day. When you memorise your best friend's phone number instead of pulling out your contacts app each time, that's a cache: a small, fast copy of something whose original is slower or further away. Computers do exactly the same thing — a cache is a fast store (RAM, SSD, edge node) that holds copies of things whose source of truth is slow (a database, a disk, a remote API).

Why bother? Because the ratio between fast and slow is enormous. A CPU cache hit is ~1 ns. RAM is ~100 ns. A local SSD is ~100 µs. A cross-region database call is ~50 ms. A cache turns "50 ms" into "0.5 ms" for the 90% of requests that ask for the same thing. At scale, this is the single biggest lever you have on latency and cost.

But caches are dangerous in a specific way: they can lie. A cache holds a copy; if the original changes and the cache doesn't know, you serve stale data. The three classic patterns handle this trade-off differently: cache-aside (app checks cache, falls back to DB, populates cache — most flexible), write-through (every write goes to cache and DB — always consistent, slower writes), and write-back / write-behind (write to cache, flush to DB async — fastest, risky on crash).

Forever-gotcha: There are only two hard things in computer science: cache invalidation and naming things (Phil Karlton). Deciding when to remove or refresh a cached entry is genuinely one of the hardest problems in engineering. TTLs are a compromise, not a solution.

(b) Visual walkthrough · 15 min

Diagrams, worked examples, step-by-step visuals.

The three canonical patterns:

Pattern comparison:

Eviction policies (when the cache fills up):

• LRU (Least Recently Used) — default for most caches. Good general-purpose.
• LFU (Least Frequently Used) — better when a few keys are hot forever.
• FIFO — simple, rarely right.
• TTL-only — no size cap, just expiry. Only safe with bounded key space.

Worked example — cache-aside for a user profile:

1. GET /users/42 arrives.
2. App: cache.get("user:42") → miss.
3. App: SELECT * FROM users WHERE id=42 → row.
4. App: cache.set("user:42", row, ex=300) — TTL 5 min.
5. Returns row to client.
6. Next 1000 requests within 5 min → cache hit, no DB touch.
7. User updates their email → app writes DB and cache.delete("user:42"). Next read repopulates.

Worked example — the thundering herd. A hot key with TTL 60 s expires at 12:00:00. Between 12:00:00 and 12:00:01, 10 000 requests all miss, all hit the DB, and all try to repopulate. DB melts. Fix: use a per-key lock (only one recomputes; others wait) or "probabilistic early expiration" (some requests refresh a few seconds early to smooth the spike).

(c) Hands-on · 20 min

Code you type and run. Not pseudo-code — real, runnable, copy-pasteable.

A minimal cache-aside implementation with Redis + a fake slow DB. Requires pip install redis and redis-server running (or docker run -p 6379:6379 redis).

# cache_aside.py
import time, json, redis, hashlib
 
r = redis.Redis(host="127.0.0.1", port=6379, decode_responses=True)
TTL = 30  # seconds
 
def slow_db_read(user_id: int) -> dict:
    # Pretend this is a 200ms query to a distant DB.
    time.sleep(0.2)
    return {"id": user_id, "name": f"User {user_id}", "fetched_at": time.time()}
 
def get_user(user_id: int) -> dict:
    key = f"user:{user_id}"
    cached = r.get(key)
    if cached is not None:
        return {"src": "cache", **json.loads(cached)}
    row = slow_db_read(user_id)
    r.set(key, json.dumps(row), ex=TTL)
    return {"src": "db", **row}
 
def update_user(user_id: int, new_name: str):
    # Write to DB first (source of truth) …
    # (real code would UPDATE the DB here)
    # … then invalidate the cache
    r.delete(f"user:{user_id}")
    print(f"invalidated user:{user_id}")
 
if __name__ == "__main__":
    # First call: miss + slow
    t0 = time.time(); print(get_user(42), f"({time.time()-t0:.3f}s)")
    # Second call: hit + fast
    t0 = time.time(); print(get_user(42), f"({time.time()-t0:.3f}s)")
    # Invalidate, then a fresh miss
    update_user(42, "New Name")
    t0 = time.time(); print(get_user(42), f"({time.time()-t0:.3f}s)")
    # Ttl expiry demo
    print("sleeping past TTL…"); time.sleep(TTL + 1)
    t0 = time.time(); print(get_user(42), f"({time.time()-t0:.3f}s)")

Run: python3 cache_aside.py.

Checklist — what to observe:

1. First read takes ~0.2 s (DB), reports src=db.
2. Second read takes <0.005 s (cache hit), reports src=cache.
3. After update_user, the delete forces the next read to hit the DB again — this is invalidation-on-write.
4. After sleeping past 30 s, the key expires and the next read is another DB hit — this is TTL-based expiry.
5. Open redis-cli and run TTL user:42 between calls — watch the countdown.

Try this modification: add a negative cache. If slow_db_read returns None (user doesn't exist), cache the None with a shorter TTL (say 5 s). This prevents an attacker from DoS-ing your DB by requesting a million random non-existent user IDs — a real-world attack called cache penetration.

(d) Production reality · 10 min

How this shows up in real systems, gotchas, war stories, common bugs.

War story #1 — Facebook's thundering herd. In the Memcache paper, Facebook describes issuing "leases" on cache misses: the first requester gets a lease token and is allowed to recompute; concurrent requesters wait or serve stale. Without leases, a popular post going viral would kill their DB every time its cache entry expired.

War story #2 — the invalidation race. Classic bug: (1) app reads DB, (2) another app updates DB and invalidates cache, (3) first app writes stale value into cache. Cache is now wrong for hours. Fix: version numbers in the cache value, or "compare-and-set" via Redis WATCH/MULTI. Better: never repopulate on write; always invalidate and let the next reader repopulate.

War story #3 — GitHub's edge cache. GitHub famously had incidents where an API endpoint got a Cache-Control: public header by mistake and CDNs cached user-specific data. One user saw another user's private repo list. Rule: default to Cache-Control: private, no-store and opt in only when you're sure the response is user-agnostic.

Common gotchas and how top teams handle them:

• Cache penetration: attacker requests keys that don't exist → every request hits DB. Fix: negative caching, or a Bloom filter in front of the DB.
• Cache stampede (aka dogpile): hot key expires, N requesters recompute simultaneously. Fix: locks, "single-flight" pattern, or early probabilistic expiration (Netflix EVCache).
• Cache avalanche: many keys with the same TTL expire at the same moment (e.g. warmup). Fix: add jitter — ttl = 300 + random(0, 60).
• Stale-while-revalidate: serve slightly stale content to the user while refreshing in the background. Standard Cache-Control directive, dramatically smooths latency spikes. Vercel/Cloudflare built products around this.
• Client-side caching (Redis 6+): the server notifies clients when a key they cached changes. Cuts round-trips further, at the cost of extra invalidation complexity.

The biggest players (Meta, Netflix, Twitter) all have internal papers/talks on cache invalidation because it never fully "solves" — it's always a tuning problem between freshness, latency, and cost. When you hear a senior engineer say "the outage was a cache issue", it's almost always invalidation, not the cache itself.

(e) Quiz + exercise · 10 min

1. Explain cache-aside in 4 steps. When would you pick it over write-through?
2. What is a "thundering herd" on a cache and name two mitigations.
3. Why is cache.delete usually safer than cache.set(new_value) on a write?
4. Give a real-world case where LFU beats LRU eviction.
5. What does stale-while-revalidate do and why is it good for user-perceived latency?

Stretch problem (connects to S061 — TCP/IP + DNS): DNS itself is a cache system, with TTLs, invalidation problems, and stampedes. Map each concept from this session (cache-aside, TTL, stampede, invalidation) onto its DNS equivalent. Which cache-invalidation problem does DNS not have a good solution for, and how do modern CDNs (Cloudflare, Fastly) work around it? ~10 lines.

Explain-out-loud test

If you can't teach these 3 things to a friend without notes, redo the session:

1. What is Caching? (one sentence, no jargon)
2. When would you use it? (one concrete example)
3. When would you NOT use it? (one anti-pattern)

What comes next

• Next session (S064): CDN — Edge, Cache Hierarchies, Cache-Control
• Previous (S062): Networking II — Load Balancers L4 vs L7, Reverse Proxies
• Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: (a) intuition · (b) visual walkthrough · (c) hands-on · (d) production reality · (e) quiz + exercise. Duration: 60 minutes.