Search Tech Journey

Find topics, journeys and posts

back to blog
systemsintermediate 32m read

R02 · Week 2 Recall & Drill

Week 2 revision: the iterator protocol behind for-loops, argument grammar and closures, data structure Big-O choices, class attribute lookup, and inheritance vs composition.

🧠SoftwareRevision · Week 2· Session 002 of 130 90 min

🎯 Rebuild Week 2 from a blank page: iteration as a live cursor, defaults evaluated once, containers chosen by access pattern, attributes resolved by lookup chain, and re-use chosen by substitutability.

Weekly revision · Week 2 · Covers 5 sessions from Mon–Fri.

Sessions covered

By the end of this revision you can
  • Explain that Python's for-loop is built on the iterator protocol, not on indices, and predict what mutating a collection mid-loop does.
  • Write a full function signature using positional-only, keyword-only, defaults, *args, and **kwargs on purpose rather than by habit.
  • Trace LEGB name resolution through nested functions and fix the late-binding closure bug without running the code.
  • Choose between list, tuple, dict, set, and frozenset with a Big-O justification, and reach for Counter, defaultdict, or deque on reflex.
  • Describe what happens when Python evaluates obj.attr — instance dict, then class, then the MRO — and why a mutable class attribute is shared state.
  • Apply substitutability, not taxonomy, as the test for whether inheritance is the right tool.

90-min structure

BlockMinutesWhat you do
Warm-up recall5Name all five sessions and one sentence each, no tabs.
Blank-page reconstruction30Work the per-session prompts below.
Hands-on drill30The event-aggregator drill in this post.
Quiz + misconception15Answer before revealing.
Gap analysis + preview10Write the gaps down. Skim next week.

Blank-page reconstruction · 30 min

S006 · Control Flow

  1. Write the four control-flow shapes and say which one you reach for when you need to destructure the value you are branching on, not just compare it.
  2. Convert [x for x in items if pred(x)] into a generator expression and state precisely what you trade away.
  3. Write the one-liner that turns a list of dicts into a dict keyed by each item's id field.

Gotcha you probably forgot: for item in items: if cond: items.remove(item) silently skips elements. The loop holds a live cursor into the object; removing an element shifts everything after it down by one, and the cursor advances past the shifted element. Build a new list, or iterate over a copy.

S007 · Functions

  1. Write out the full signature grammar in order — positional-only marker, positional-or-keyword, *args, keyword-only, **kwargs — and say what the bare * does.
  2. State LEGB in one sentence, then say what nonlocal does that global does not.
  3. Write a minimal decorator that measures elapsed time, and name the one helper from functools that stops it from destroying the wrapped function's name and docstring.

Gotcha you probably forgot: funcs = [lambda: i for i in range(3)] gives three functions that all return 2. Closures capture the variable, not the value at creation time, and by the time you call them the loop has finished. The standard fix is a default argument that binds at definition time: lambda i=i: i.

S008 · Data Structures

  1. Draw the Big-O table for list, dict, and set across the four operations: index, append, membership, delete-by-value.
  2. Name the collections helper you would reach for to tally occurrences, to group into buckets, and to push and pop from both ends efficiently.
  3. Explain why list.pop(0) is slow and what you use instead.

Gotcha you probably forgot: for k, v in d.items(): if v is None: del d[k] raises RuntimeError: dictionary changed size during iteration. The view object is live. Collect the keys first — for k in [k for k, v in d.items() if v is None]: del d[k] — or build a filtered dict.

S009 · Classes & Objects

  1. Narrate what happens when Python evaluates p = Point(3, 4), from the call through object creation to __init__.
  2. List the dunder methods that make a class Pythonic, and say which one you must define alongside __eq__ if you want instances usable in a set.
  3. Explain the difference between @staticmethod, @classmethod, and @property in one sentence each.

Gotcha you probably forgot: class Cart: items = [] puts one list on the class object, and every instance reads through to that same list. Instances only get their own copy when you assign to the attribute — appending is a mutation, not an assignment, so all carts share the contents. Initialise mutable attributes inside __init__.

S010 · Inheritance & Composition

  1. Write the four re-use tools in order of preference and give the one-line test for each.
  2. Explain what the MRO is and why Python needs a defined one at all.
  3. State the Liskov Substitution Principle in one sentence, then use it to argue whether Square should subclass Rectangle.

Gotcha you probably forgot: inheritance is a substitutability claim, not a taxonomy claim. "An admin is a kind of user" is not sufficient justification. If the subclass narrows what the parent allowed — rejecting inputs the parent accepted, or requiring setup the parent did not — every caller written against the parent is now potentially broken.


Hands-on drill · 30 min

Task: build a streaming event aggregator that exercises all five sessions in one file. No pandas, no third-party libraries — stdlib only.

Step 1 — generate an event file (5 min)

mkdir -p ~/projects/w2-drill && cd ~/projects/w2-drill
python - <<'PY'
import random, json
actions = ["view", "click", "purchase", "signup", "search", "logout"]
with open("events.jsonl", "w") as f:
    for i in range(200_000):
        rec = {
            "user_id": f"u{random.randint(1, 900)}",
            "ts": 1_700_000_000 + i,
            "action": random.choice(actions),
        }
        f.write(json.dumps(rec) + "\n")
PY
wc -l events.jsonl

Expected outcome: 200000 events.jsonl.

Step 2 — write the aggregator (15 min)

The constraint that makes this a real drill: one pass, never hold all events in memory. That forces generators (S006), a decorator (S007), the right containers (S008), and a class with dunders (S009).

# aggregate.py
import functools
import json
import time
from collections import Counter
from typing import Iterator, Protocol
 
 
def timed(fn):
    @functools.wraps(fn)          # S007 — keeps __name__ and __doc__
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        try:
            return fn(*args, **kwargs)
        finally:
            elapsed = time.perf_counter() - start
            print(f"[timed] {fn.__name__} took {elapsed:.3f}s")
    return wrapper
 
 
def read_events(path: str) -> Iterator[dict]:
    """Lazy — yields one dict at a time, never builds a list."""
    with open(path) as f:
        for line in f:
            yield json.loads(line)
 
 
class Sink(Protocol):             # S010 — Protocol, not ABC: no subclassing needed
    def feed(self, event: dict) -> None: ...
 
 
class ActionTally:
    """Counts actions. Composed into the report, not inherited from."""
    def __init__(self) -> None:
        self.counts: Counter[str] = Counter()   # S008 — Counter, not manual dict
 
    def feed(self, event: dict) -> None:
        self.counts[event["action"]] += 1
 
    def __len__(self) -> int:
        return len(self.counts)
 
    def __repr__(self) -> str:
        return f"ActionTally(distinct={len(self)}, total={sum(self.counts.values())})"
 
 
class LastSeen:
    """Per-user last event timestamp. Plain dict is right here — O(1) writes."""
    def __init__(self) -> None:
        self.latest: dict[str, int] = {}
 
    def feed(self, event: dict) -> None:
        uid = event["user_id"]
        ts = event["ts"]
        if ts > self.latest.get(uid, -1):       # S008 — .get with default, no membership test
            self.latest[uid] = ts
 
    def __len__(self) -> int:
        return len(self.latest)
 
    def __repr__(self) -> str:
        return f"LastSeen(users={len(self)})"
 
 
class Report:
    """Composition (S010): Report HAS sinks, it is not a kind of sink."""
    def __init__(self, *sinks: Sink) -> None:
        self.sinks = sinks
 
    @timed
    def run(self, path: str) -> None:
        for event in read_events(path):         # S006 — one live cursor, streaming
            for sink in self.sinks:
                sink.feed(event)
 
 
if __name__ == "__main__":
    tally, seen = ActionTally(), LastSeen()
    Report(tally, seen).run("events.jsonl")
    print(tally)
    print(seen)
    print("top 3 actions:", tally.counts.most_common(3))
python aggregate.py

Expected outcome: a [timed] line, then ActionTally(distinct=6, total=200000), then LastSeen(users=900), then the three most common actions with counts near 33,000 each — the six actions are drawn uniformly, so they should land close to a sixth of the total. Distinct actions must be exactly 6 and users exactly 900 by construction.

Step 3 — break it on purpose (10 min)

Prove the gotchas rather than trusting them:

# gotchas.py
# 1. Mutating while iterating
nums = [1, 2, 3, 4, 5, 6]
for n in nums:
    if n % 2 == 0:
        nums.remove(n)
print("after remove-in-loop:", nums)   # NOT [1, 3, 5]
 
# 2. Late-binding closures
funcs = [lambda: i for i in range(3)]
print("late binding:", [f() for f in funcs])
fixed = [lambda i=i: i for i in range(3)]
print("fixed:", [f() for f in fixed])
 
# 3. Shared class attribute
class Cart:
    items = []
a, b = Cart(), Cart()
a.items.append("apple")
print("shared class attr:", b.items)

Expected outcome: the first print shows [1, 3, 5, 6] — the 6 survives because the cursor skipped it after 4 was removed. The second shows [2, 2, 2] then [0, 1, 2]. The third shows ['apple'] on an instance that never appended anything. If any of those surprises you, re-read that session before moving on.


Common misconception
✗ What most people think

"A dict lookup is constant time, so a dict is always the fast option — and x in my_list versus x in my_set is just a style preference."

✓ What is actually true

Hash lookup is constant time on average, amortised, and only over a decent hash function. Membership in a list is a linear scan that compares every element. At ten elements the difference is noise; at ten million it is the difference between an instant answer and a coffee break. The rule is not "dicts are fast" — it is "pick the container from the access pattern": repeated membership means set, keyed lookup means dict, ordered positional access means list, and push/pop at both ends means deque.


Week 2 recall · click to reveal
★ = stretch question

Gap analysis + next week preview · 10 min

  • Which of the three deliberate breakages in Step 3 actually surprised you? That is your weakest session this week.
  • Did the aggregator hold the whole file in memory anywhere? Check that you never called list() on the generator — if peak memory scaled with file size, the streaming lesson has not landed.
  • Can you justify every container choice in aggregate.py out loud, with Big-O? If not, redo the S008 decision matrix from memory.

Next week (S011–S015) finishes the Python core and steps into complexity: errors, exceptions, and debugging with pdb; modules, packages, virtualenvs, pip and uv; testing with pytest and the TDD loop; type hints, mypy, dataclasses and pydantic; and finally Big-O notation, which turns this week's informal "lists are slow for membership" intuition into a formal tool you will use for the rest of the plan.


Part of the 6-month evergreen learning plan.