R13 · Week 13 Recall & Drill

Weekly revision · Week 13 · Covers 5 sessions from Mon–Fri.

Sessions covered

• S061 — Networking I — TCP/IP, DNS, Sockets
• S062 — Networking II — Load Balancers L4 vs L7, Reverse Proxies
• S063 — Caching — Cache-Aside, Write-Through, TTLs, Invalidation
• S064 — CDN — Edge, Cache Hierarchies, Cache-Control
• S065 — Docker — Images, Layers, Dockerfile, Networking

90-min structure

Blank-page reconstruction · 30 min

S061 · Networking I

1. Narrate the three-packet handshake, saying what each packet establishes.
2. Explain why the transport is a byte stream, and the bug that creates when you write your own protocol.
3. Say what a record's time-to-live controls, and why lowering it during an outage helps nobody.

Gotcha you probably forgot: a large number of sockets lingering in the post-close wait state is usually not a bug in the peer — it is the side that closed first holding the connection open so late duplicate packets cannot be misattributed to a new connection reusing the same port pair. The real fix is connection reuse rather than tuning the wait down: if you were not opening a fresh connection per request, the state would not accumulate.

S062 · Networking II

1. Give a one-sentence rule for choosing transport-level versus application-level balancing.
2. Explain why fewest-connections often beats round-robin.
3. Describe connection draining in one sentence.

Gotcha you probably forgot: a health check that only proves the process is listening will happily keep sending traffic to a backend that is saturated or has lost its database connection. A useful health check exercises the dependency path the request actually needs — and should distinguish liveness, meaning restart me, from readiness, meaning stop sending me traffic but do not kill me.

S063 · Caching

1. Explain cache-aside in one sentence, and give one workload where writing through is a better fit.
2. Define a stampede and give one line of defence.
3. Say why caching the absence of a result matters.

Gotcha you probably forgot: populating a large cache all at once gives every entry the same expiry, so they all expire simultaneously and the entire load lands on the origin in one instant. The fix is to jitter the expiry — add a random spread — so expirations are distributed rather than synchronised. This bites hardest right after a deploy that warms the cache.

S064 · CDN & Edge

1. Distinguish the client-facing maximum age from the shared-cache maximum age.
2. Explain what declaring variance on encoding does, and what breaks without it.
3. Say why an intermediate shield tier exists between edge and origin.

Gotcha you probably forgot: a hit ratio far below expectation is usually a cache-key problem, not a policy problem. Query parameters that do not affect the response, cookies included in the key, or variance declared on a header that takes many values will each fragment one logical object into many cache entries, none of which get enough traffic to stay warm.

S065 · Docker

1. State the difference between an image and a container in one sentence.
2. Explain why instruction ordering in a build file matters.
3. Say why multi-stage builds produce smaller images.

Gotcha you probably forgot: deleting a file in a later layer does not shrink the image. Layers are additive and the earlier layer still contains the bytes, so a secret or a build artefact removed in a subsequent instruction is still present in the image and extractable. Do not create it in that layer in the first place — use a separate build stage and copy only what you need.

Hands-on drill · 30 min

Task: reproduce a framing bug, an uneven-load surprise, a stampede, and a layer-cache miss.

mkdir -p ~/projects/w13-drill && cd ~/projects/w13-drill

Step 1 — the byte-stream framing bug (9 min)

# framing.py
import socket
import struct
import threading
 
MESSAGES = [b"hello", b"a much longer second message", b"third"]
 
def naive_server(sock):
    """Assumes one recv == one message. This is the bug."""
    conn, _ = sock.accept()
    with conn:
        got = []
        while len(got) < len(MESSAGES):
            chunk = conn.recv(4096)
            if not chunk:
                break
            got.append(chunk)
        print("naive server received", len(got), "chunks:", got)
 
def framed_server(sock):
    """Length-prefixed: read exactly the declared number of bytes."""
    def recv_exactly(conn, n):
        buf = b""
        while len(buf) < n:
            part = conn.recv(n - len(buf))
            if not part:
                raise ConnectionError("peer closed mid-message")
            buf += part
        return buf
 
    conn, _ = sock.accept()
    with conn:
        got = []
        for _ in MESSAGES:
            (length,) = struct.unpack("!I", recv_exactly(conn, 4))
            got.append(recv_exactly(conn, length))
        print("framed server received", len(got), "messages:", got)
 
def run(server_fn, framed):
    srv = socket.socket()
    srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    srv.bind(("127.0.0.1", 0))
    srv.listen(1)
    port = srv.getsockname()[1]
    t = threading.Thread(target=server_fn, args=(srv,))
    t.start()
 
    c = socket.create_connection(("127.0.0.1", port))
    for m in MESSAGES:
        c.sendall(struct.pack("!I", len(m)) + m if framed else m)   # no delay: writes coalesce
    c.close()
    t.join()
    srv.close()
 
run(naive_server, framed=False)
run(framed_server, framed=True)

Expected outcome: the naive server very often receives fewer chunks than there were sends, because consecutive writes coalesce into one segment — three sends can arrive as one read. The framed server always recovers exactly three messages regardless of how the bytes were split or merged. Message boundaries are something you impose; the transport only promises ordered bytes. Note the naive result may vary between runs, and that non-determinism is itself the lesson: the bug passes in testing and fails under load.

Step 2 — round-robin does not equalise work (6 min)

# balance.py
import random
 
rng = random.Random(7)
# Realistic cost distribution: most requests cheap, a few very expensive.
costs = [rng.choice([1, 1, 1, 1, 1, 2, 3, 40]) for _ in range(3000)]
 
def simulate(strategy, backends=4):
    load = [0] * backends
    for i, c in enumerate(costs):
        if strategy == "round_robin":
            b = i % backends
        else:                                    # least-loaded: pick the least busy
            b = min(range(backends), key=lambda x: load[x])
        load[b] += c
    return load
 
for s in ("round_robin", "least_loaded"):
    load = simulate(s)
    print(f"{s:<14} per-backend work={load}  spread={max(load)-min(load)}")

Expected outcome: round-robin gives every backend an almost identical request count while the work spread between the busiest and idlest backend is substantial, because the expensive requests do not distribute evenly. The least-loaded strategy narrows the spread sharply. That gap is the argument for connection- or load-aware balancing whenever request cost varies — which is always.

Step 3 — stampede and synchronised expiry (8 min)

# stampede.py
import random
import threading
import time
 
origin_calls = {"n": 0}
lock = threading.Lock()
cache: dict[str, tuple[float, str]] = {}
 
def origin(key):
    with lock:
        origin_calls["n"] += 1
    time.sleep(0.05)                 # the expensive thing you are protecting
    return f"value-for-{key}"
 
def get_unprotected(key, ttl=1.0):
    hit = cache.get(key)
    if hit and hit[0] > time.time():
        return hit[1]
    val = origin(key)
    cache[key] = (time.time() + ttl, val)
    return val
 
single_flight: dict[str, threading.Lock] = {}
sf_guard = threading.Lock()
 
def get_protected(key, ttl=1.0):
    hit = cache.get(key)
    if hit and hit[0] > time.time():
        return hit[1]
    with sf_guard:
        kl = single_flight.setdefault(key, threading.Lock())
    with kl:                          # only one filler per key; others wait and reuse
        hit = cache.get(key)
        if hit and hit[0] > time.time():
            return hit[1]
        val = origin(key)
        cache[key] = (time.time() + ttl, val)
        return val
 
for label, fn in (("unprotected", get_unprotected), ("single-flight", get_protected)):
    cache.clear(); single_flight.clear(); origin_calls["n"] = 0
    threads = [threading.Thread(target=fn, args=("hot-key",)) for _ in range(50)]
    for t in threads: t.start()
    for t in threads: t.join()
    print(f"{label:<14} origin calls for 50 concurrent readers: {origin_calls['n']}")
 
# Synchronised expiry vs jittered expiry.
now = time.time()
same = [now + 3600 for _ in range(10_000)]
jittered = [now + 3600 * random.uniform(0.8, 1.2) for _ in range(10_000)]
print("entries expiring in the same second — fixed TTL:", sum(1 for e in same if abs(e - same[0]) < 1))
print("entries expiring in the same second — jittered:", sum(1 for e in jittered if abs(e - same[0]) < 1))

Expected outcome: without protection, a cold key with fifty concurrent readers produces many origin calls — every reader misses before anyone fills. With single-flight, one call serves all of them. Then the expiry comparison shows every fixed-expiry entry expiring in the same second while jittered entries spread out, which is the same failure at a different scale: a synchronised expiry is a scheduled stampede.

Step 4 — layer cache ordering (7 min)

# Dockerfile.bad — dependencies reinstall on every source change
FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install --no-cache-dir -r requirements.txt
CMD ["python", "app.py"]

# Dockerfile.good — dependency layer is cached until requirements change
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
RUN useradd --create-home appuser
USER appuser
CMD ["python", "app.py"]

printf 'requests\n' > requirements.txt
printf 'print("v1")\n' > app.py
 
for f in Dockerfile.bad Dockerfile.good; do
  docker build -q -f "$f" -t "w13-${f##*.}" . >/dev/null
  printf 'print("v2")\n' > app.py                      # source changed, deps did not
  echo "--- rebuild with $f ---"
  docker build -f "$f" -t "w13-${f##*.}" . 2>&1 | grep -Ei 'cached|installing|running' | head -5
  printf 'print("v1")\n' > app.py
done

Expected outcome: after touching only the source file, the second build file reuses the cached dependency-install layer while the first re-runs the install. Every instruction invalidates the cache for itself and everything after it, so ordering from least- to most-frequently-changing is the whole optimisation. If you do not have a container runtime available, read both files and write down which instruction invalidates the cache in each — the reasoning is the point, not the timing.

Gap analysis + next week preview · 10 min

• Did the naive framing server give a different result across runs? Write down why that non-determinism makes framing bugs so dangerous in testing.
• Was the round-robin work spread larger than you expected? That number is the argument you will make in your next design review.
• Could you list the four cache failure modes without notes? They are the ones you will actually meet.

Next week (S066–S070) stays in systems and moves up the stack: container orchestration with pods, deployments, and services; infrastructure as code; continuous integration and delivery pipelines; monitoring and alerting; and incident response. The container model from S065 is the unit everything in orchestration schedules, and the caching and balancing ideas from this week are what those platforms configure for you.

Part of the 6-month evergreen learning plan.