Dinesh/ Blog
learning

R02 · Week 2 Recall & Drill

Week 2 revision: the iterator protocol behind for-loops, argument grammar and closures, data structure Big-O choices, class attribute lookup, and inheritance vs composition.

20 min readChecking device speech…
Lesson preparation & details

Level: intermediate

By the end, you should be able to

  • Explain that Python's for-loop is built on the iterator protocol, not on indices, and predict what mutating a collection mid-loop does.
  • Write a full function signature using positional-only, keyword-only, defaults, \*args, and \*\*kwargs on purpose rather than by habit.
  • Trace LEGB name resolution through nested functions and fix the late-binding closure bug without running the code.
  • Choose between list, tuple, dict, set, and frozenset with a Big-O justification, and reach for Counter, defaultdict, or deque on reflex.
  • Describe ordinary attribute lookup including data-descriptor precedence, the instance dictionary and the class MRO; explain why a mutable class attribute can be shared state.
  • Apply substitutability, not taxonomy, as the test for whether inheritance is the right tool.

Bring with you

Sources checked: . A source check does not change the publication or update date.

Editorial review: · What review means

In this article · 19 sections

Assessment scope

Attempt the recall prompts before reading the worked drill and answer explanations. The worked examples below restore the combined teaching task, with explicit contracts and checks; a local model is distinguished from a reproduced external service. The minute allocations are adjustable practice suggestions, not evidence of learning speed.

Sessions covered

By the end of this revision you can

  1. Explain that Python's for-loop is built on the iterator protocol, not on indices, and predict what mutating a collection mid-loop does.

  2. Write a full function signature using positional-only, keyword-only, defaults, *args, and **kwargs on purpose rather than by habit.

  3. Trace LEGB name resolution through nested functions and fix the late-binding closure bug without running the code.

  4. Choose between list, tuple, dict, set, and frozenset with a Big-O justification, and reach for Counter, defaultdict, or deque on reflex.

  5. Describe ordinary attribute lookup including data-descriptor precedence, the instance dictionary and the class MRO; explain why a mutable class attribute can be shared state.

  6. Apply substitutability, not taxonomy, as the test for whether inheritance is the right tool.


90-min structure

Treat this as an adjustable practice budget: 5 minutes to recall the five subjects, 30 for the blank-page questions, 30 for the worked drill, 15 to compare answers and test a counterexample, and 10 to record a gap and plan an unseen retest. The solutions and transfer exercises can take longer; the goal is correct reasoning, not racing a timer.

Blank-page reconstruction · 30 min

S006 · Control Flow

  1. Write the four control-flow shapes and say which one you reach for when you need to destructure the value you are branching on, not just compare it.
  2. Convert [x for x in items if pred(x)] into a generator expression and state precisely what you trade away.
  3. Write the one-liner that turns a list of dicts into a dict keyed by each item's id field.

S007 · Functions

  1. Write out the full signature grammar in order — positional-only marker, positional-or-keyword, *args, keyword-only, **kwargs — and say what the bare * does.
  2. State LEGB in one sentence, then say what nonlocal does that global does not.
  3. Write a minimal decorator that measures elapsed time, and name the one helper from functools that stops it from destroying the wrapped function's name and docstring.

S008 · Data Structures

  1. Draw the Big-O table for list, dict, and set across the four operations: index, append, membership, delete-by-value.
  2. Name the collections helper you would reach for to tally occurrences, to group into buckets, and to push and pop from both ends efficiently.
  3. Explain why list.pop(0) is slow and what you use instead.

S009 · Classes & Objects

  1. Narrate what happens when Python evaluates p = Point(3, 4), from the call through object creation to __init__.
  2. List the dunder methods that make a class Pythonic, and say which one you must define alongside __eq__ if you want instances usable in a set.
  3. Explain the difference between @staticmethod, @classmethod, and @property in one sentence each.

S010 · Inheritance & Composition

  1. Write the four re-use tools in order of preference and give the one-line test for each.
  2. Explain what the MRO is and why Python needs a defined one at all.
  3. State the Liskov Substitution Principle in one sentence, then use it to argue whether Square should subclass Rectangle.

Hands-on drill · 30 min

Build a one-pass event aggregator with two independently owned sinks. A lazy reader bounds the number of event objects retained at once; it does not bound the number of distinct users stored by the report. For N events, U users and A actions, expected processing cost is O(N) with a fixed number of sinks and ordinary hash behavior; retained state is O(U+A), plus the largest input line. Large or adversarial keys change those assumptions.

Step 1 — generate an event file (5 min)

Use the three JSON-lines records below instead of an unseeded 200,000-row sample. The exact action counts are view=2 and click=1; user u1's last-seen time is 9 even though its second event arrives with timestamp 4. Negative timestamps are valid integers here, so a sentinel of -1 would be incorrect. The fixture is passed as an iterable; a file object could supply the same interface without loading the file.

Step 2 — write the aggregator (15 min)

The policy is fail-fast with a line number for malformed JSON or invalid fields. Previous events may already have reached sinks when an error occurs: this is not a transaction. An atomic batch would need staging or rollback. Type hints document the protocol but do not enforce it at runtime; explicit validation below performs that job for events.

import functools
import json
import time
from collections import Counter
from typing import Protocol
 
def timed(fn):
    @functools.wraps(fn)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        try:
            return fn(*args, **kwargs)
        finally:
            print(f"{fn.__name__}: {time.perf_counter() - start:.6f}s")
    return wrapper
 
def read_events(lines):
    for number, line in enumerate(lines, 1):
        try:
            event = json.loads(line)
            if (not isinstance(event, dict)
                or not isinstance(event.get("user_id"), str)
                or not isinstance(event.get("action"), str)
                or type(event.get("ts")) is not int):
                raise ValueError("invalid event fields")
        except (ValueError, TypeError) as exc:
            raise ValueError(f"line {number}: invalid event") from exc
        yield event
 
class Sink(Protocol):
    def feed(self, event: dict) -> None: ...
 
class ActionTally:
    def __init__(self):
        self.counts = Counter()
    def feed(self, event):
        self.counts[event["action"]] += 1
    def __len__(self):
        return len(self.counts)
    def __repr__(self):
        return f"ActionTally(distinct={len(self)}, total={sum(self.counts.values())})"
 
class LastSeen:
    def __init__(self):
        self.latest = {}
    def feed(self, event):
        uid, ts = event["user_id"], event["ts"]
        if uid not in self.latest or ts > self.latest[uid]:
            self.latest[uid] = ts
 
class Report:
    def __init__(self, *sinks: Sink):
        self.sinks = sinks
    @timed
    def run(self, lines):
        for event in read_events(lines):
            for sink in self.sinks:
                sink.feed(event)
 
rows = [
    '{"user_id":"u1","ts":9,"action":"view"}',
    '{"user_id":"u1","ts":4,"action":"click"}',
    '{"user_id":"u2","ts":-5,"action":"view"}',
]
tally, seen = ActionTally(), LastSeen()
Report(tally, seen).run(iter(rows))
assert tally.counts == Counter(view=2, click=1)
assert seen.latest == {"u1": 9, "u2": -5}
assert repr(tally) == "ActionTally(distinct=2, total=3)"
empty = ActionTally(); Report(empty).run(iter(()))
assert len(empty) == 0 and empty.counts is not tally.counts
assert Report.run.__name__ == "run"
for invalid in ('{', '[]', '{"user_id":"u","ts":true,"action":"x"}'):
    try:
        list(read_events([invalid]))
    except ValueError as exc:
        assert str(exc).startswith("line 1:")
    else:
        raise AssertionError("malformed record accepted")

Report composes sinks rather than inheriting their state. A new sink needs a compatible feed behavior; the Protocol supports static structural checking, not a guarantee of semantic substitutability. A base class is appropriate when there is a genuinely shared invariant or implementation and subclasses preserve its contract. Composition is a preference to reason about, not a theorem that inheritance always fails.

Step 3 — break it on purpose (10 min)

Adjacent removable values reveal the live-index list-iterator bug. The original alternating list [1,2,3,4,5,6] actually becomes [1,3,5] under the loop below; it did not demonstrate the claimed surviving 6. This counterexample uses [2,4,6]: after removing 2, index 1 refers to 6, so 4 is skipped.

nums = [2, 4, 6]
for n in nums:
    if n % 2 == 0:
        nums.remove(n)
assert nums == [4]
assert [n for n in [2, 4, 6] if n % 2] == []
alternating = [1, 2, 3, 4, 5, 6]
for n in alternating:
    if n % 2 == 0:
        alternating.remove(n)
assert alternating == [1, 3, 5]
funcs = [lambda: i for i in range(3)]
fixed = [lambda i=i: i for i in range(3)]
assert [f() for f in funcs] == [2, 2, 2]
assert [f() for f in fixed] == [0, 1, 2]
class SharedCart:
    items = []
a, b = SharedCart(), SharedCart()
a.items.append("apple")
assert b.items == ["apple"]
class Cart:
    def __init__(self):
        self.items = []
a, b = Cart(), Cart(); a.items.append("apple")
assert b.items == []

Recall answers: iteration, signatures and object contracts

A for loop calls iter and then next until StopIteration. The mutation behavior belongs to the particular iterator: list iterators use indices, dict/set size changes can raise, and custom iterators may snapshot or reject changes. Do not generalize the list behavior to all iterators. A generator expression is lazy and normally single-pass; a list comprehension is eager and reusable. A dict comprehension keyed by id overwrites earlier duplicate IDs, so uniqueness must be a contract or explicitly checked.

def f(a, /, b=0, *args, flag=False, **kwargs) has positional-only a, positional-or-keyword b, extra positional args, keyword-only flag, and extra keywords. A bare * introduces keyword-only arguments without collecting extras. LEGB is local, enclosing-function, global, builtins for ordinary name lookup; nonlocal rebinds an existing enclosing-function name, while global selects the module scope. Defaults evaluate when def executes. functools.wraps preserves useful wrapper metadata; the timer above measures wall-clock elapsed duration, not an asymptotic complexity proof.

Lists give O(1) indexed access and amortized O(1) append, but membership and value removal are O(N). Dict/set lookup is expected O(1) under ordinary hashing, not a worst-case guarantee or an O(1) cost for arbitrarily expensive key hashing/equality. Counter tallies, defaultdict(list) groups, and deque provides efficient operations at both ends; middle deque access is not list-like O(1).

For ordinary object.__getattribute__, lookup gives data descriptors in the class MRO priority over the instance dictionary, then non-data descriptors/class attributes. __getattr__ is fallback after failure; custom hooks can change behavior. A property is a descriptor, so the oversimplified 'instance first, then class' rule is wrong. __new__ creates an instance and __init__ initializes an appropriate returned instance. staticmethod supplies no automatic first argument; classmethod receives the class; a property mediates attribute access. Defining __eq__ without a corresponding __hash__ in the class makes __hash__ = None, so instances are unhashable. With value equality, equal objects must hash equally and a hash must remain stable while used as a key. Leave mutable value objects unhashable; a frozen dataclass is not automatically usable as a key if its fields are unhashable.

Substitutability retest: if a mutable rectangle interface promises independent width and height setters, a square that forces them equal violates that behavioral contract. An immutable area-only interface has a different contract and may permit both. Name the caller's promises rather than deciding from geometric taxonomy alone.

Payment-provider transfer answer: compose a provider with a narrow charge/refund protocol when implementations need independent setup and state. Specify accepted amounts, currency, refund eligibility and failure behavior as well as signatures: a structural type checker cannot prove those promises. A base class can still be sound when it shares implementation without narrowing callers' accepted inputs or weakening their guarantees.

Complete the control-flow and container decision table

The recall prompt's four common shapes are if/elif/else for conditions, for for iteration, while for repeated testing, and match/case for structural pattern matching (Python 3.10+). This is a useful selection, not an exhaustive grammar. (x for x in items if pred(x)) is the lazy counterpart of the list comprehension. {item['id']: item for item in items} builds the ID lookup; later duplicate IDs overwrite earlier ones.

ContainerIndexed/key accessAddMembershipRemove
listO(1) by indexamortized O(1) appendO(n)O(n) by value or from front; O(1) end pop
tupleO(1) by indeximmutable; concatenation copiesO(n)no mutation
dictexpected O(1) by keyamortized expected O(1) assignmentexpected O(1) for keysexpected O(1) by key; value search O(n)
setno positional indexamortized expected O(1) addexpected O(1)expected O(1) discard/remove
frozensetno positional indeximmutableexpected O(1)no mutation

These bounds assume bounded-cost hashing/equality and ordinary dynamic-array/hash-table implementations, not adversarial keys. Use a tuple for a fixed ordered record and a frozenset for an immutable unordered collection of hashable values. Both can be keys only when their contained values are hashable. A dict has insertion order, not integer positional indexing; a set has no sequence-order contract.

A useful reuse decision order is a function for a stateless transformation, composition for owned collaborators, a narrow protocol for replaceable behavior, and inheritance for an actual shared implementation/contract. These tools overlap and this is a heuristic, not a mandatory universal ordering. MRO is Python's C3 linearization of base classes: it preserves local base ordering and monotonicity; super() follows that order, not simply 'my parent'.

# Signatures, enclosing names, descriptor precedence and diamond lookup.
def signature(a, /, b=0, *args, flag=False, **kwargs):
    return a, b, args, flag, kwargs
assert signature(1, 2, 3, flag=True, note='x') == (1, 2, (3,), True, {'note':'x'})
 
def counter():
    value = 0
    def bump():
        nonlocal value
        value += 1
        return value
    return bump
c = counter()
assert (c(), c()) == (1, 2)
 
class Root:
    label = 'root'
class Left(Root): pass
class Right(Root):
    label = 'right'
class Diamond(Left, Right): pass
assert Diamond.__mro__ == (Diamond, Left, Right, Root, object)
assert Diamond().label == 'right'  # not Root via a naive depth-first walk
 
class WithProperty:
    @property
    def value(self):
        return 7
obj = WithProperty()
obj.__dict__['value'] = 99
assert obj.value == 7  # data descriptor wins over instance dictionary

Dictionary-mutation repair: the original deletion gotcha needs a different repair from list index reasoning. Materialize the keys to delete before changing the live dictionary; do not assume a dict iterator shifts like a list iterator. defaultdict(list) creates a bucket on missing-key indexing, whereas get does not invoke the default factory.

from collections import defaultdict
values = {'keep': 1, 'drop': None, 'also_drop': None}
for key in [key for key, value in values.items() if value is None]:
    del values[key]
assert values == {'keep': 1}
groups = defaultdict(list)
for user, action in [('u1', 'view'), ('u2', 'click'), ('u1', 'click')]:
    groups[user].append(action)
assert dict(groups) == {'u1': ['view', 'click'], 'u2': ['click']}
assert groups.get('absent') is None and 'absent' not in groups

Exercise and solution: feed an invalid second event after a valid first event. read_events raises ValueError('line 2: invalid event'); the first event remains in each sink because run is not transactional. Two independent Reports need independently constructed sinks. Reusing a sink deliberately accumulates results across calls, even though each reader itself is one-pass.

Answer constraints and corrections

Streaming input alone does not imply constant total memory: grouping by an unbounded number of keys grows with the key set. Defaults evaluate once; lambda i=i captures the current value. Mutable value-equality objects should ordinarily remain unhashable rather than inventing a hash that changes.

Gap analysis + next week preview · 10 min

Score each original recall question separately: explain unaided; explain with a hint; or not yet. A fluent verbal answer is not a successful implementation. Choose one failed concept, write a counterexample that distinguishes the mistaken rule from the correct one, and schedule an unseen retest. Keep successes as well as failures in the log so the plan does not become a list of increasingly familiar problems.

For the drill, require an explicit input contract, a correctness argument or domain assumption, one ordinary case, one boundary case and an independent expected result. A chart, compile success or passing sample alone does not meet that rubric. Use the stated input contracts and named runtime when interpreting a result; finite tests and timing samples are not universal correctness or performance guarantees.

Next week (S011–S015) finishes the Python core and steps into complexity: errors, exceptions, and debugging with pdb; modules, packages, virtualenvs, pip and uv; testing with pytest and the TDD loop; type hints, mypy, dataclasses and pydantic; and finally Big-O notation, which turns this week's informal "lists are slow for membership" intuition into a formal tool you will use for the rest of the plan.

Corrective source notes

The cited passages support these named claims. The algebra, counterexamples and executable fixtures supply the lesson-specific reasoning; finite checks are not universal benchmarks.

Pause / Recall / Apply

Can you explain it without the page?

Close the example. Reconstruct the core idea, then change one assumption. Mark complete when you’re ready; you can always undo it.

Stored in this browser only. No account, no sync. Clearing browser data removes your record.