R14 · Week 14 Recall & Drill

Weekly revision · Week 14 · Covers 5 sessions from Mon–Fri.

Sessions covered

• S066 — Kubernetes I — Pods, Deployments, Services
• S067 — Kubernetes II — ConfigMaps, Secrets, HPA, Network Policies
• S068 — Azure Cloud — Identity, Storage, Networking, App Service
• S069 — Infrastructure as Code — Terraform / Bicep Basics
• S070 — CAP & PACELC — the Actual Trade-Offs

90-min structure

Blank-page reconstruction · 30 min

S066 · Kubernetes I

1. Define pod, deployment, and service in one sentence each, and say why you rarely create the first directly.
2. Distinguish a readiness probe from a liveness probe by what each causes when it fails.
3. Explain the two rolling-update parameters that control surge and unavailability.

Gotcha you probably forgot: a label selector mismatch produces a service with no endpoints and no error message anywhere. The service is created successfully, the workloads are running successfully, and traffic simply goes nowhere. Whenever something is "deployed but unreachable", check the endpoint list before checking anything else — an empty one localises the fault instantly.

S067 · Kubernetes II

1. Say when to use plain configuration versus a secret, and give the two injection methods.
2. Name the two things the autoscaler needs that people forget to provide.
3. Say what an empty policy targeting a set of workloads does.

Gotcha you probably forgot: secrets are base64-encoded, not encrypted. Encoding is a transport convenience with no confidentiality whatsoever — anyone who can read the object can read the value. Encryption at rest for these objects is a separate cluster-level configuration, and without it the values sit in the cluster's datastore in effectively plain form.

S068 · Cloud Platform

1. Name the four pillars with a flagship service in each.
2. Explain the difference between a standalone service principal and an identity attached to a running workload.
3. Explain a private endpoint in one sentence.

Gotcha you probably forgot: availability zones protect against a datacentre-level fault — power, cooling, a fabric failure in one building — but not against a regional control-plane problem, a bad configuration rolled out region-wide, or a subscription-level quota exhaustion. Multi-zone is not multi-region, and the failure modes that actually cause long outages are frequently the ones zones do not cover.

S069 · Infrastructure as Code

1. Say in one sentence what the planning step does.
2. Explain what happens when you rename a resource block, and how to avoid the destructive outcome.
3. Contrast the two collection constructs and say which is safer.

Gotcha you probably forgot: secrets end up in state in plaintext. Any sensitive attribute the provider returns is recorded there, which is why the backend must be encrypted and access-controlled, and why committing the state file to version control is a security incident rather than a style mistake — it publishes every credential the deployment touched.

S070 · CAP & PACELC

1. State CAP in one precise sentence.
2. Say why choosing consistency and availability while ignoring partitions is not a real option.
3. Explain what PACELC adds that CAP omits.

Gotcha you probably forgot: "strongly consistent" is not one thing across systems. One vendor's strong read may mean reading from a leader within a single replica group, while another's means externally-consistent global ordering. The words match and the guarantees do not, so read the actual definition in the documentation before assuming two systems offer the same thing.

Hands-on drill · 30 min

Task: implement the three ideas that carry this week — a reconciliation loop, a plan-and-apply diff, and a partition with two policies.

mkdir -p ~/projects/w14-drill && cd ~/projects/w14-drill

Step 1 — write a reconciler (10 min)

# reconcile.py
import itertools
import random
 
rng = random.Random(3)
counter = itertools.count(1)
 
class Cluster:
    """Actual state. Things break on their own, exactly like the real thing."""
    def __init__(self):
        self.pods: dict[str, str] = {}         # name -> image
 
    def create(self, image):
        name = f"pod-{next(counter)}"
        self.pods[name] = image
        return name
 
    def delete(self, name):
        self.pods.pop(name, None)
 
    def chaos(self):
        if self.pods and rng.random() < 0.4:
            victim = rng.choice(list(self.pods))
            self.delete(victim)
            return victim
        return None
 
def reconcile(cluster, desired_replicas, desired_image):
    """One pass: observe, diff, act. Never 'start N pods' imperatively."""
    actions = []
    # Wrong image is a difference too — replace, do not ignore.
    for name, image in list(cluster.pods.items()):
        if image != desired_image:
            cluster.delete(name)
            actions.append(f"delete {name} (image drift)")
    while len(cluster.pods) < desired_replicas:
        actions.append(f"create {cluster.create(desired_image)}")
    while len(cluster.pods) > desired_replicas:
        victim = sorted(cluster.pods)[-1]
        cluster.delete(victim)
        actions.append(f"delete {victim} (excess)")
    return actions
 
c = Cluster()
desired, image = 3, "app:v1"
for tick in range(8):
    if tick == 4:
        image = "app:v2"                        # a deploy: change the declaration only
    killed = c.chaos()
    acts = reconcile(c, desired, image)
    print(f"tick {tick}: killed={killed or '-':<8} actions={acts or ['(none)']} "
          f"-> {len(c.pods)} pods on {image}")

Expected outcome: the loop converges to the declared replica count after every random deletion, and the version change is expressed purely as a change to the desired state rather than as a sequence of restart commands. Note that the same code handles first deploy, self-healing, scaling, and rollout — that convergence property is the whole reason the platform is declarative, and it is also why a stuck rollout is diagnosed by asking what difference the controller sees, not by looking for a failed command.

Step 2 — plan before apply (10 min)

# plan.py
DESIRED = {
    "storage.main":  {"type": "storage", "tier": "standard", "replication": "zonal"},
    "vault.secrets": {"type": "vault", "purge_protection": True},
    "db.primary":    {"type": "sql", "sku": "GP_Gen5_4"},
}
STATE = {                                  # the authoritative mapping, not a cache
    "storage.main":  {"id": "/sub/x/storage/main", "type": "storage",
                      "tier": "standard", "replication": "local"},
    "db.primary":    {"id": "/sub/x/sql/primary", "type": "sql", "sku": "GP_Gen5_2"},
    "vault.old":     {"id": "/sub/x/vault/old", "type": "vault", "purge_protection": True},
}
FORCES_REPLACE = {"sql": {"sku"}, "storage": set(), "vault": set()}
 
def plan(desired, state):
    out = []
    for addr, want in desired.items():
        have = state.get(addr)
        if not have:
            out.append(("create", addr, {}))
            continue
        diff = {k: (have.get(k), v) for k, v in want.items() if have.get(k) != v}
        if diff:
            replace = bool(set(diff) & FORCES_REPLACE.get(want["type"], set()))
            out.append(("replace" if replace else "update", addr, diff))
    for addr in state:
        if addr not in desired:
            out.append(("destroy", addr, {}))
    return out
 
for action, addr, diff in plan(DESIRED, STATE):
    marker = {"create": "+", "update": "~", "replace": "-/+", "destroy": "-"}[action]
    print(f"{marker:>3} {addr:<16} {action:<8} {diff or ''}")
print("\nRead the -/+ and - lines before typing yes. That is the entire discipline.")

Expected outcome: one create, one in-place update, one replacement, and one destroy. Two things to sit with. First, the replacement is triggered by changing an attribute the provider cannot alter in place — that is the class of change that quietly destroys a production database, and the plan is the only place it is visible before it happens. Second, the destroy comes from an address present in state but absent from the configuration: rename a block without telling the tool, and this is exactly what you get — a destroy plus a create, rather than a rename.

Step 3 — a partition, two policies (10 min)

# partition.py
class Replica:
    def __init__(self, name):
        self.name, self.value, self.version = name, "v0", 0
 
    def write(self, value):
        self.version += 1
        self.value = value
 
def run(policy):
    a, b = Replica("A"), Replica("B")
    log = []
 
    # Healthy: writes replicate, both agree.
    a.write("v1"); b.value, b.version = a.value, a.version
    log.append(f"healthy: A={a.value} B={b.value}")
 
    # Partition begins. A client writes to A, another reads from B.
    if policy == "CP":
        # Minority side refuses rather than serve possibly-stale data.
        a.write("v2")
        log.append("partition: write to A accepted (majority side)")
        log.append("partition: read from B REFUSED — unavailable, but never stale")
    else:  # AP
        a.write("v2")
        b.write("v3")     # both sides accept; they now disagree
        log.append(f"partition: A={a.value} B={b.value} — both available, diverged")
 
    # Heal.
    if policy == "CP":
        b.value, b.version = a.value, a.version
        log.append(f"healed: B catches up -> {b.value}; no conflict to resolve")
    else:
        winner = max((a, b), key=lambda r: (r.version, r.name))
        log.append(f"healed: conflict resolved by rule -> {winner.value}; "
                   f"the other write is LOST unless the data type merges")
    return log
 
for policy in ("CP", "AP"):
    print(f"--- {policy} ---")
    for line in run(policy):
        print("   ", line)

Expected outcome: the consistency-favouring run has an interval where reads simply fail, and heals with nothing to reconcile. The availability-favouring run answers every request and heals with a genuine conflict where one write disappears unless the data type is designed to merge. Neither is correct in general — the point is that the cost is paid either in refused requests or in lost writes, and choosing not to decide means the system decides for you. Write down which cost your current project can actually absorb.

Gap analysis + next week preview · 10 min

• In Step 1, did you handle the version change as a state difference rather than as a restart command? If your instinct was to write a rollout procedure, the declarative model has not landed.
• Did you notice the destroy line in the plan output and connect it to a renamed block? That is the single most common way people delete production resources.
• Which cost can your current system actually absorb — refused requests or lost writes? Write the answer down; it is a real design commitment.

Next week (S071–S075) continues into distributed systems proper: consensus and replication; partitioning and sharding strategies; message queues and event-driven architecture; distributed transactions and the saga pattern; and idempotency and exactly-once delivery at the system level. The CAP and PACELC framing from S070 is the lens for every one of those.

Part of the 6-month evergreen learning plan.