Day 26 — Caching Strategies — CDN, Application Cache, Cache-Aside, Read-Through, Write-Through

Caches are amplifiers — they multiply throughput, reduce cost, and shave latency. They also amplify bugs (staleness, inconsistency, herd effects). Picking the pattern that matches the consistency need is the senior skill.

🧠 Concept

Why it matters & the mental model.

1. The cache hierarchy

Each layer has its own TTL and invalidation story. The closer to the user, the cheaper and the staler.

2. Cache-aside (lazy loading)

• Read: try cache; on miss, read DB, write to cache, return.
• Write: update DB, invalidate (or update) cache.
• Pros: simple, cache only what's read.
• Cons: stale window after writes; cold-start every cache miss hits DB.

3. Read-through

• Cache is a transparent wrapper over DB; on miss it loads behind the scenes.
• Same characteristics as cache-aside but encapsulated.
• Examples: Spring @Cacheable, Django's cache_page.

4. Write-through

• Write goes to cache + DB synchronously.
• Pros: cache always fresh.
• Cons: writes pay both latencies; cache holds data that may never be read (wastes memory).

🛠 Deep Dive

Internals, code, architecture.

5. Write-behind (write-back)

• Write to cache; DB updated asynchronously.
• Pros: fastest writes.
• Cons: data loss window if cache dies before flush; ordering issues.
• Use only where loss tolerance is explicit (counters, analytics).

6. Cache-aside + invalidation: getting it right

Three patterns:

• TTL-only: write to DB, let cache expire. Simple, bounded staleness.
• DEL on write: write to DB, DEL the cache key. Next read populates fresh.
• Update on write: write to DB, SET cache directly. Fastest reads, but reasoning about race conditions is harder.

The race: two writers update DB then both write cache → last writer wins; with TTL-only or DEL, you avoid this class of bug.

7. Cache stampede / thundering herd

A hot key expires; thousands of requests miss simultaneously → all hit DB → DB falls over. Fixes:

• Probabilistic early expiration: refresh before TTL with probability rising near expiry.
• Request coalescing / singleflight: only one request rebuilds; others wait and share the result.
• Soft TTL + background refresh: serve stale-but-revalidate, refresh async.

8. Hot key problem

Single key serves 90% of traffic (a celebrity post). Mitigations:

• Client-side cache with TTL + invalidation messages (Redis tracking).
• Replicate the key to N shards, randomise read path.
• CDN at the edge for read-mostly data with HTTP Cache-Control headers.

🚀 In Practice

Trade-offs, exercises, what to ship today.

9. Eviction policies

• LRU (Least Recently Used): default.
• LFU (Least Frequently Used): better for stable distributions, worse on bursts.
• TinyLFU + W-TinyLFU (Caffeine): admission control prevents pollution from one-off scans.
• FIFO: ring buffer; cheap; surprisingly good in some workloads (FIFO Queue research, 2023).

10. Negative caching

Cache "not found" too — otherwise every miss for a non-existent key hammers the DB. TTL shorter than positive cache.

11. CDN-specific

• Cache-Control: public, max-age=3600, s-maxage=86400, stale-while-revalidate=300.
• Vary header for content negotiation (accept-language, user-agent).
• Cache keys: URL + selected headers; keep keys stable to avoid fragmenting.
• Purges: instant (slow, costly) vs versioned URLs (fast: change ?v=2).

12. What to take away

"How do you cache X?" Strong answers: pattern (cache-aside / read-through), TTL strategy, invalidation, stampede mitigation, one hot-key story. Bonus: name negative caching and edge CDN config.

Resources

• 🎥 ByteByteGo — Caching strategies
• 📖 AWS — Caching strategies
• 📖 Redis docs — Client-side caching
• 📖 Cloudflare — How CDN works

Practice Problem: LFU Cache (Hard)