Testing with pytest — TDD Workflow

Review boundary and corrected contracts

pytest rewrites assertions in collected tests to explain failures. Coverage is execution evidence, not behavioral adequacy. Yield teardown cannot undo setup that failed before reaching yield; register cleanup immediately or use context managers. Package-install/network examples are not run.

The examples below teach the stated input contracts; fragments and deliberately incorrect counterexamples are exercises, not production implementations. Complexity counts elementary operations unless a bit-cost or allocation caveat is stated. Practice times are suggestions, not measured performance or hiring guarantees.

Learning objective: Write a real test suite with pytest — parametrize, fixtures, mocks, and coverage — and use the red → green → refactor loop to add a feature without fear.

Why this session exists

Writing tests before or alongside code makes assumptions inspectable when a change or refactor arrives. Untested code is a liability that compounds daily: a change breaks something unrelated, a coworker refactors and doesn't know a corner case existed, you deploy Friday and page yourself at midnight. Tests are the seatbelt. This session teaches you the seatbelt-clicking motion until it's automatic.

You will be able to

• Explain the difference between a unit test, an integration test, and an end-to-end test with a concrete example.

• Write pytest tests with assert, parametrize, and fixture — and know when to reach for each.

• Mock an external dependency (HTTP, filesystem, clock) with pytest-mock / monkeypatch.

• Run tests with coverage, read the report, and identify branches you forgot.

• Do a full red → green → refactor cycle on a small feature without leaving the terminal.

Prerequisites

• S012 · Modules, Packages, Virtualenvs — you need a src/ layout package to test against.
• Basic Python: functions, classes, exceptions.

Pre-read · skim before the session (30 min total)

Further reading: Pytest · Get Started

Further reading: Effective Python Testing With pytest

Further reading: The Test Pyramid

Optional archived videos

These are retained original resource URLs, not independently verified titles, speakers, durations or technical evidence. Use the official readings above for API contracts.

• Original video 1 — optional background; availability and attribution unverified.
• Original video 2 — optional background; availability and attribution unverified.
• Original video 3 — optional background; availability and attribution unverified.

(a) Intuition · 5 min

Tests are the airport metal detector, not the TSA line

Real: Imagine an airport with no metal detector. Every passenger clears TSA by promising they didn't pack a knife. That's untested code — a promise, not evidence. You'd never board that plane. Now imagine the detector beeps only on knives and only when there's really a knife. Fast, cheap, boring. That's what a good test suite feels like: it beeps only when you actually broke something, and it takes 3 seconds.

 
    A test is an assertion: "given this input, my code should produce this output." assert add(2, 3) == 5. If it stops being true, pytest beeps. If nothing beeps, the discovered tests passed; omitted cases and weak assertions can still hide defects.
    The magic isn't the framework — it's that you now have an executable specification. Refactor in small steps; tests can catch the regressions their assertions cover.
  

The three levels of tests — the pyramid

Test pyramid · what to write, in what ratio

• Unit tests (illustrative 70%) — one function, no I/O, milliseconds. Cheap to write, cheap to run, fail with a laser-precise message.

• Integration tests (illustrative 25%) — one component talking to a real dependency (DB, HTTP). Slower but proves the wiring works.

• End-to-end tests (illustrative 5%) — the whole system, user-shaped inputs. Expensive, flaky, invaluable for critical happy paths.

The mistake juniors make: too much E2E, too little unit

Every hour of E2E flakes is an hour of ‘is my code broken or is the test broken?’ archaeology. E2E is dessert, not the meal. Write dozens of unit tests, a few integrations, one or two E2E per user journey.

These percentages are a discussion aid, not an industry requirement. Choose the mix from failure risks, feedback time and dependencies; E2E tests can be reliable when isolated.

The red → green → refactor loop

1. Red: write one observable expectation and run it. Inspect that it fails for the missing behaviour, not an import error or bad fixture.
2. Green: implement the smallest change that satisfies that expectation. Run the whole relevant suite, not just the new case.
3. Refactor: change structure without changing the contract, rerunning after each edit. A mutant that deliberately breaks the contract should still fail.
4. Repeat: add the next behaviour, including errors and boundaries. The stats() exercise below supplies a complete red test and green implementation; no fixed cycle time is promised.

A quick history

Python's standard-library unittest arrived in Python 2.1 (2001), not 2003; it is still a supported framework rather than merely a compatibility layer. Its xUnit-style classes and pytest's function/fixture style are alternative interfaces. Today property-based generation and branch coverage complement example tests: neither replaced examples nor became a universal requirement in a particular year. The earlier timeline's JUnit-first, pytest-default and adoption-date claims lacked supporting evidence and are not retained as facts. See the Python 2.1 release notes.

(b) From test discovery to a checked result · 15 min

How pytest collects and runs your tests

A plain assertion only helps if the runner discovers and executes its test. Trace the route from matching filenames through fixture resolution to the test call, so a missing test and a failing test remain different problems.

Collection defaults can be overridden. rootdir identifies configuration/node IDs; it is not itself a command to collect every file beneath it. Collection/import errors, fixture setup/teardown errors, skips and expected failures are distinct from the simplified pass/fail path shown.

The fixture branch supplies the conditions in which the test runs; the assertion determines what it checks. That split explains why fixtures have their own setup and teardown lifecycle, and why simply executing a line is not evidence that its behaviour was verified.

The lifecycle of a fixture

Define with @pytest.fixture

Desc: A plain function that yields or returns a value. Name = the fixture's name.

Test requests it by name

Desc: Add a parameter with the fixture's name; pytest injects the value.

Scope decides re-use

Desc: scope='function' (default) rebuilds per test; scope='module' once per file; scope='session' once per pytest run.

yield → teardown runs after

Desc: After a successful yield, teardown is attempted even when the test fails. If this fixture raises before yielding, its post-yield cleanup is skipped; previously successful fixtures still tear down. Register cleanup as soon as acquisition succeeds.

The tools that live around pytest

The modern Python testing stack

Name: pytest

Desc: The runner discovers and runs tests. Common exit codes are 0 for collected tests passing; 1 failures; 2 interruption; 3 internal error; 4 usage error; 5 no tests collected. Newer pytest documentation also defines 6 for exceeding the maximum warnings limit; check the installed version. Treat no-tests as different from a successful suite.

Name: pytest-cov

Desc: Wraps coverage.py. pytest --cov=mypkg --cov-branch --cov-report=term-missing requests statement and branch coverage; plain --cov does not enable branch coverage by itself.

Name: pytest-mock

Desc: A friendlier unittest.mock — one fixture mocker instead of nested decorators.

Name: hypothesis

Desc: Property-based testing. You describe the shape of valid input; Hypothesis generates and shrinks cases; its configurable example budget is not exhaustive coverage.

Name: pytest-xdist

Desc: Runs tests in parallel across cores. Can reduce suite time when tests are independent and worker/setup overhead permits; speedup is measured, not guaranteed.

Name: tox / nox

Desc: Runs the suite across multiple Python versions in isolated envs. What CI uses under the hood.

unittest vs pytest — same problem, different taste

Name: pytest

Tagline: Modern, minimal

• Plain assert x == y

• Rich failure messages via introspection

• Function fixtures, not setUp/tearDown

• Huge plugin ecosystem

• Use this by default

Name: unittest (stdlib)

Tagline: Java-flavoured, verbose

• self.assertEqual(x, y)

• Class-based, setUp/tearDown methods

• In stdlib — no install

• Fine for tiny scripts

• Legacy code you inherit

The mental model

A test is a scientific claim

Arrange the world → Act on it → Assert what should be true. If you can't split your test into those three sections, your test is doing too much. That's the AAA pattern, and it's all you need.

Myth: "High coverage means well tested. If we're at 90% line coverage, the code is verified — and a test that runs the code without crashing proves it works."

Truth: Coverage measures which lines executed, not which behaviours were checked. A test suite with no assertions at all can reach 100% coverage. Coverage is a reliable negative signal — uncovered lines are definitely untested — and a nearly worthless positive one.

Why: Coverage identifies code and branch destinations that the run did not execute. Raising coverage alone does not show that assertions detect wrong results: a test that only calls a function can cover every line while ignoring its return value. Add a meaningful assertion and deliberately break the implementation to check whether the test notices.

Test: the weak test below executes the return but does not pin the result; the strong test checks ordinary and boundary inputs.

def discount(price, pct):
    return price - price * pct / 100
 
def test_discount_weak():
    discount(100, 10)
 
def test_discount_real():
    assert discount(100, 10) == 90
    assert discount(100, 0) == 100
    assert discount(0, 50) == 0

Question: Why do pytest fixtures use dependency injection — a function that declares a parameter name and receives an object — instead of the obvious setUp() method? Derive why the indirection earns its keep.

Premise: A test needs preconditions (a temp dir, a database, a client) and those preconditions must be torn down whether the test passes or fails.

Because: leaked state makes the next test's result depend on this test's outcome

Premise: Different tests in the same file need different subsets of preconditions, and building all of them for every test is wasteful — sometimes prohibitively so, if one is a database.

Because: setup cost is paid per test, and test suites are run constantly

Premise: setUp() runs before each test method, but it can call composable helpers and register cleanup. Fixtures make the requested subset explicit in each test signature rather than requiring every test to use the same setup path.

Because: named dependencies reduce manual orchestration; unittest also supports helpers, context managers and addCleanup

Premise: If instead each precondition is a named, independently-defined provider, and a test requests the ones it needs by name, then each test builds exactly its own subset.

Because: naming the dependency is the minimal way to express "I need this one, not the others"

Premise: And because providers can themselves request other providers, the framework can resolve a dependency graph, deduplicate shared nodes within a scope, and tear down in reverse order automatically.

Because: declared dependencies let pytest order setup and reverse teardown; manually composed setup must encode that order itself

Yield-fixture teardown runs after a successful yield and in reverse setup order. If setup fails before yield, that fixture's post-yield code does not run; acquire resources with immediate cleanup registration or context managers.

Name: Arrange–Act–Assert on a contract, not an implementation

Picture: A test is three blocks: put the world in a known state, perform exactly one action, assert on the observable outcome. If you cannot point at the single Act line, the test is testing more than one thing and its failure will not tell you which.The deeper rule is what you assert on: the contract (given this input, this output; given this bad input, this error) rather than the mechanism (these internal methods were called in this order). Contract tests tend to survive refactors. Call assertions are appropriate when the interaction itself is the contract, such as sending exactly one charge request.

• A failing test must name its own cause. If you have to open a debugger to find out what broke, the test was too broad or asserted too little.

• Tests should not accidentally depend on order. Parallel execution also requires isolated resource names or deliberately serial scheduling. Shared mutable state — a session fixture, a real database, a module-level cache — is the usual culprit when they aren't.

• Mock at the boundary you don't own (network, clock, filesystem, cloud SDK). Mocking your own internals couples the test to the code and guarantees false failures on refactor.

• Test the edges, not the middle: empty, one, many, duplicate, null, wrong type, boundary values. Bugs live at boundaries because that's where the branches are.

Trigger: Fire this model the moment you see: a test named test_it_works · assertions on mock call counts and nothing else · a test that fails only in CI or only when run with others · a refactor that broke 40 tests but no behaviour · a bug report for code that had "full coverage".

Decision: Your data pipeline needs testing. Unit tests with fake data, integration tests against a real store, or production data-quality assertions?

Name: Unit tests on transform functions

Gain: milliseconds to run, deterministic, no infrastructure, and they pin business logic precisely — every edge case you can enumerate becomes a permanent regression guard

Cost: they verify your logic against your assumptions about the data; schema drift, nulls and duplicates absent from fixtures can still fail in production

Pick: always, for any non-trivial transform — this is the cheapest layer and the fastest feedback loop you will ever have

Name: Integration tests against a real store

Gain: catches what unit tests structurally cannot: connection handling, SQL dialect quirks, type coercion, partitioning behaviour, permissions

Cost: slow, needs provisioned infrastructure and credentials, flaky under concurrency, and expensive enough that people start skipping them locally

Pick: the code's job is the interaction — a query, an upsert, a partition write — where a unit test would only be asserting on a mock's configuration

Name: Data-quality assertions in production

Gain: the only layer that tests the data you actually received; catches upstream schema changes, volume anomalies, and null spikes that no pre-deploy test could have anticipated

Cost: detects rather than prevents — the bad data has already arrived; thresholds need tuning or they become alert noise that gets muted, at which point they are worse than absent

Pick: the pipeline consumes data produced by a team that can change it without telling you, i.e. essentially every production pipeline

Verdict: These are not alternatives; they cover overlapping but different failure risks. Unit tests catch your bugs, integration tests catch interface bugs, quality checks catch the world's bugs . A useful allocation is: heavy unit coverage on transforms, a thin integration suite over the store interactions you actually depend on, and unconditional quality gates (row counts, null rates, key uniqueness, freshness) on every input and output. Choose thresholds from the data contract and route failures to an owner; a gate with no response policy does not protect a pipeline.

(c) Hands-on · 25 min

You're going to test a tiny in-memory URL shortener — enough surface for real patterns (parametrize, fixtures, mocking a clock, exceptions). Copy this into tdd_demo.sh and run it.

#!/usr/bin/env bash
# tdd_demo.sh — a small package + real pytest suite.
set -euo pipefail
 
WORK_DIR="$HOME/projects/learning/s013/tinyurl"
log() { printf "\033[1;36m▶ %s\033[0m\n" "$*"; }
 
log "1/6 Fresh package"
WORK_DIR=$(mktemp -d "${TMPDIR:-/tmp}/pytest-workshop.XXXXXX")
mkdir -p "$WORK_DIR/src/tinyurl" "$WORK_DIR/tests"
cd "$WORK_DIR"
 
cat > pyproject.toml <<'TOML'
[project]
name = "tinyurl"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = []
 
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
 
[tool.hatch.build.targets.wheel]
packages = ["src/tinyurl"]
 
[tool.pytest.ini_options]
addopts = "-ra -q"
testpaths = ["tests"]
TOML
 
log "2/6 Source: a URL shortener with an expiry clock"
cat > src/tinyurl/__init__.py <<'PY'
from .store import Shortener, LinkExpired, LinkNotFound
__all__ = ["Shortener", "LinkExpired", "LinkNotFound"]
PY
 
cat > src/tinyurl/store.py <<'PY'
"""Tiny in-memory URL shortener with TTL support."""
from __future__ import annotations
from dataclasses import dataclass, field
from time import time
from typing import Callable
import secrets
import string
 
_ALPHABET = string.ascii_letters + string.digits
 
class LinkNotFound(KeyError): ...
class LinkExpired(RuntimeError): ...
 
@dataclass
class _Entry:
    url: str
    expires_at: float | None  # unix seconds; None = forever
 
@dataclass
class Shortener:
    """Return short codes for URLs. `clock` is injectable for tests."""
    clock: Callable[[], float] = field(default=time)
    _links: dict[str, _Entry] = field(default_factory=dict)
 
    def shorten(self, url: str, ttl_seconds: int | None = None, length: int = 6) -> str:
        if not url.startswith(("http://", "https://")):
            raise ValueError(f"URL must start with http(s)://, got {url!r}")
        if type(length) is not int or length <= 0:
            raise ValueError("length must be a positive integer")
        if ttl_seconds is not None and (type(ttl_seconds) is not int or ttl_seconds < 0):
            raise ValueError("ttl_seconds must be a nonnegative integer or None")
        for _ in range(10):
            code = "".join(secrets.choice(_ALPHABET) for _ in range(length))
            if code not in self._links:
                break
        else:
            raise RuntimeError("code space collision: retry budget exhausted")
        expires_at = None if ttl_seconds is None else self.clock() + ttl_seconds
        self._links[code] = _Entry(url=url, expires_at=expires_at)
        return code
 
    def resolve(self, code: str) -> str:
        try:
            entry = self._links[code]
        except KeyError as e:
            raise LinkNotFound(code) from e
        if entry.expires_at is not None and self.clock() >= entry.expires_at:
            raise LinkExpired(code)
        return entry.url
PY
 
log "3/6 Tests: unit, parametrize, fixture, mocking a clock"
cat > tests/conftest.py <<'PY'
"""Shared fixtures. pytest auto-imports conftest.py in the test tree."""
import itertools
import pytest
from tinyurl import Shortener
 
class FakeClock:
    """A clock you can advance from tests."""
    def __init__(self, start: float = 1_000_000.0):
        self._now = start
    def __call__(self) -> float:
        return self._now
    def advance(self, seconds: float) -> None:
        self._now += seconds
 
@pytest.fixture
def fake_clock() -> FakeClock:
    return FakeClock()
 
@pytest.fixture
def shortener(fake_clock: FakeClock) -> Shortener:
    return Shortener(clock=fake_clock)
PY
 
cat > tests/test_shortener.py <<'PY'
import pytest
from tinyurl import LinkExpired, LinkNotFound, Shortener
 
def test_shorten_returns_a_code_of_default_length(shortener: Shortener):
    code = shortener.shorten("https://example.com")
    assert len(code) == 6
    assert shortener.resolve(code) == "https://example.com"
 
@pytest.mark.parametrize("bad_url", ["", "example.com", "ftp://x", "javascript:alert(1)"])
def test_shorten_rejects_non_http_urls(shortener: Shortener, bad_url: str):
    with pytest.raises(ValueError, match="must start with http"):
        shortener.shorten(bad_url)
 
def test_resolve_unknown_code_raises(shortener: Shortener):
    with pytest.raises(LinkNotFound):
        shortener.resolve("nope42")
 
def test_ttl_expiry(shortener: Shortener, fake_clock):
    code = shortener.shorten("https://example.com", ttl_seconds=60)
    fake_clock.advance(59)
    assert shortener.resolve(code) == "https://example.com"   # not yet
    fake_clock.advance(2)
    with pytest.raises(LinkExpired):
        shortener.resolve(code)                                # now expired
 
def test_no_ttl_never_expires(shortener: Shortener, fake_clock):
    code = shortener.shorten("https://example.com")
    fake_clock.advance(10**9)
    assert shortener.resolve(code) == "https://example.com"
 
def test_length_parameter(shortener: Shortener):
    assert len(shortener.shorten("https://example.com", length=10)) == 10
 
def test_monkeypatch_secrets(monkeypatch, shortener: Shortener):
    """Prove monkeypatch works: force secrets.choice to return 'x'."""
    import secrets as s
    monkeypatch.setattr(s, "choice", lambda alphabet: "x")
    assert shortener.shorten("https://example.com", length=4) == "xxxx"
PY
 
log "4/6 Venv + install"
uv venv .venv --python 3.12
# shellcheck disable=SC1091
source .venv/bin/activate
uv pip install -e . pytest pytest-cov
 
log "5/6 Run the suite (green run)"
pytest
 
log "6/6 Coverage report — which lines survived unchallenged?"
pytest --cov=tinyurl --cov-branch --cov-report=term-missing

The shell script is an optional installation recipe, not something this review ran. The tested core is the extracted Python files. URL validation here checks only the prefix, not host validity or safe navigation. Codes are unique within this instance because collisions retry at most ten times; exhaustion raises without overwriting a link. TTL zero expires immediately; negative/noninteger TTL and nonpositive/noninteger lengths are rejected. Storage is in-memory, not concurrent or persistent; no production URL-service security claim is made.

What each block does

Anatomy of the test suite

Name: conftest.py

Desc: pytest auto-imports this. Fixtures defined here are available to every test in the folder without an import. Standard convention.

Name: @pytest.fixture · shortener

Desc: Depends on fake_clock — pytest wires the graph for you. Each test gets a fresh Shortener; no shared state.

Name: @pytest.mark.parametrize

Desc: Runs the same test body once per input. 4 bad URLs → 4 distinct test cases in the report. One test body, many claims.

Name: pytest.raises(match=...)

Desc: Asserts the exception type AND that its message matches a regex. Prevents accidentally passing on the wrong exception.

Name: FakeClock injection

Desc: The real gold: production code accepts clock as a dependency. Tests inject a controllable one. TTL tests advance a fake clock rather than sleeping for 60 seconds; no microsecond timing claim is needed.

Name: monkeypatch.setattr

Desc: Built-in fixture that swaps an attribute for the duration of one test, then restores. Perfect for third-party functions you don't own.

Name: --cov-report=term-missing

Desc: Prints exactly which line numbers were not executed. If it says store.py 25 miss, you know where to add a test.

What to observe

ClassName: ml-4 list-decimal space-y-1

The parametrize test shows as 4 separate cases in the report — test_shorten_rejects_non_http_urls[], [example.com], [ftp://x], [javascript:alert(1)]. Read the actual coverage report rather than assuming 100%. Force a collision, test exact expiry and remove the URL guard as a negative control: the rejection tests must fail. Fixtures compose: shortener depends on fake_clock. Pytest builds the graph automatically. The FakeClock trick is the pattern for everything time-dependent — payments, sessions, caches, rate limits.

Goal: Do a full red → green → refactor cycle

Hint: After you make it green, run coverage again. Did the new line get covered? If not, add a test that exercises the failure path (e.g. shortening the same URL twice — should it return the same code?).

Add a new behaviour: shortener.stats() returns {"total": int, "active": int, "expired": int}.

Loop:

# 1. RED — write the test first
cat >> tests/test_shortener.py <<'PY'
 
def test_stats_counts_active_and_expired(shortener, fake_clock):
    shortener.shorten("https://a.com")                     # forever
    c = shortener.shorten("https://b.com", ttl_seconds=10) # expires
    fake_clock.advance(11)
    stats = shortener.stats()
    assert stats == {"total": 2, "active": 1, "expired": 1}
PY
 
pytest   # RED: AttributeError: 'Shortener' object has no attribute 'stats'
 
# 2. GREEN — implement the minimum. Add to Shortener class in src/tinyurl/store.py:
#    def stats(self) -> dict[str, int]:
#        now = self.clock()
#        expired = sum(1 for e in self._links.values()
#                      if e.expires_at is not None and now >= e.expires_at)
#        return {"total": len(self._links), "active": len(self._links) - expired, "expired": expired}
pytest   # GREEN
 
# 3. REFACTOR — extract `_is_expired(entry, now)` used by both resolve() and stats().
#    Re-run tests after each edit. Stay green.

A complete green implementation and refactor seam

Insert this method into Shortener after observing the missing-method failure. It reads the clock once and classifies each retained entry, including immediately expired links; an empty store returns all zeros.

def stats(self) -> dict[str, int]:
    now = self.clock()
    expired = sum(1 for e in self._links.values()
                  if e.expires_at is not None and now >= e.expires_at)
    return {"total": len(self._links), "active": len(self._links) - expired,
            "expired": expired}

For the refactor, define this module-level helper; in resolve, replace the expiry predicate with _is_expired(entry, self.clock()), and in stats replace it with _is_expired(e, now). The helper takes time as data so one stats call retains a consistent snapshot.

def _is_expired(entry, now: float) -> bool:
    return entry.expires_at is not None and now >= entry.expires_at

Test empty, active, exact-deadline and mixed stores before and after the transformation. Deliberately changing >= to > must fail the exact-deadline test; otherwise the suite has not pinned the boundary.

(d) Production reality · 15 min

Slow CI and flaky-test ownership

Illustrative mechanism, not the earlier unsourced GitHub incident: record per-test duration and failure causes before splitting a suite across workers. Isolate shared files, ports and records; otherwise concurrency creates flakes. A quarantine needs an owner, expiry and replacement checks for critical behaviour. Do not make a failing payment/security check non-blocking merely because it is intermittent. Keep required integration checks separate from fast feedback, and investigate a reproducible seed rather than blindly retrying.

Fake an HTTP boundary where it is looked up

A real network call can fail because of credentials, rate limits or remote changes. Keep deterministic unit tests offline; separately test the integration contract against a controlled service. This uses stdlib mocks (the optional pytest-mock plugin offers mocker.Mock/mocker.patch with automatic cleanup):

from unittest.mock import Mock
 
def fetch_names(get, url):
    response = get(url, timeout=2)
    response.raise_for_status()
    return [row["name"] for row in response.json()["data"]]
 
def test_fetch_names():
    response = Mock()
    response.json.return_value = {"data": [{"name": "Ada"}]}
    get = Mock(return_value=response)
    assert fetch_names(get, "https://example.test/users") == ["Ada"]
    get.assert_called_once_with("https://example.test/users", timeout=2)
    response.raise_for_status.assert_called_once_with()

Patch the name in the consumer's module: after from client import get, patch consumer.get, not an unrelated client.get. Test errors as well as success: make raise_for_status raise and assert it propagates instead of parsing JSON. For filesystem work, tmp_path provides a test-owned directory; for environment variables use monkeypatch.setenv, not permanent shell changes. Recorded HTTP fixtures can become stale and must not contain tokens.

Time and teardown are observable dependencies

Two real clock reads need not be equal. The shortener's injected FakeClock makes 59 seconds, exactly 60, and later expiry deterministic. If refactoring expiry into a helper, pass a single now to all comparisons in stats() so one snapshot does not straddle a changing clock.

This fixture registers cleanup immediately after acquisition. If subsequent setup fails, pytest still runs that finalizer; code placed only after an unreached yield would not run.

import pytest
 
@pytest.fixture
def resource(request):
    state = {"open": True}
    request.addfinalizer(lambda: state.update(open=False))
    # Further setup may fail here; cleanup is already registered.
    yield state

Where this shows up in the rest of the plan

Root: Once you can test, everything else you build is safer

Application: Typing + dataclasses

Desc: A configured checker reports some type mismatches before execution; tests check chosen runtime behaviours. Neither is a full correctness proof.

Application: FastAPI

Desc: TestClient runs your API in-process — an in-process ASGI request/response without a real network socket. Test the same way.

Application: Airflow

Desc: DAG unit tests + task integration tests keep your pipelines from rotting silently.

Application: ML models

Desc: Model tests: schema, shape, invariants (‘prediction should not depend on user_id’). Same pytest patterns.

Application: CI/CD

Desc: GitHub Actions runs pytest on every PR. Passing required checks is one merge gate alongside review; a red build needs investigation.

Application: Observability

Desc: Test that alerts fire. Yes, you should test your alerts. Same red/green loop.

(e) Recall + stretch · 10 min

Self-check (answer follows each question).

Recall — click each to reveal

Q: Give a concrete one-line example each of a unit test, an integration test, and an E2E test.

A: Unit: assert add(2, 3) == 5. Integration: call your DB layer against a real Postgres in Docker and assert the row was inserted. E2E: drive a browser, click through checkout, assert the confirmation email arrives.

Q: What is the AAA pattern, and why does it matter?

A: Arrange (set up inputs) → Act (call the code under test) → Assert (check the outcome). A test that mixes these three is doing too much and will be hard to name and hard to debug when it fails.

Q: What does @pytest.fixture buy you over just calling a helper function?

A: Automatic dependency injection, scope-based re-use (function/module/session), teardown via yield, and composability — one fixture can depend on another and pytest resolves the graph.

Q: You need to test code that reads the current time. What are two acceptable approaches?

A: 1. Inject the clock: production code accepts a clock: Callable[[], float] defaulting to time.time; tests pass a fake. 2. Freeze time with freezegun / time-machine — monkeypatches datetime.now for the test's lifetime.

Q: Why is pytest.raises(ValueError, match="must start with http") better than just pytest.raises(ValueError)?

A: Because a generic ValueError could come from anywhere in the call stack — a typo, a slice, an unrelated line. Matching the message pins the assertion to the specific failure you meant to test.

Q: What does pytest.mark.parametrize do, and when should you reach for it vs writing separate tests?

A: It runs the same test body once per input tuple. Reach for it when the assertion is identical and only the input varies (e.g. 10 bad URLs, all should raise). Write separate tests when the assertion itself differs.

Q: You add a feature and coverage drops from 96% to 88%. What should you do first?

A: Run pytest --cov=mypkg --cov-branch --cov-report=term-missing. Missing statements and missing branch destinations are different: a covered if can still lack one outcome. Read the report and add assertions for meaningful missing behaviours, not just calls that touch lines.

Q: Why is a test that hits the real internet a bad unit test?

A: It's slow (100+ ms per call), flaky (network hiccups fail the build), non-deterministic (real APIs change), and creates cross-team dependencies (Stripe rate-limits your CI). Mock the HTTP layer for unit tests; keep real calls in a separate integration tier.

Q: In one sentence, what is the difference between monkeypatch and pytest-mock?

A: monkeypatch is a built-in pytest fixture for swapping attributes/env vars for one test; pytest-mock wraps unittest.mock in a friendlier mocker fixture for building mock objects with call assertions.

Q: Stretch — write a hypothesis property test for our Shortener.shorten that asserts every produced code is 6 chars from the alphabet.

Stretch: true

A: Construct state inside each generated example; a function-scoped pytest fixture is shared across Hypothesis examples and triggers its fixture health check. This optional test requires Hypothesis and was not executed in this review.

from hypothesis import given, strategies as st
import string
from tinyurl import Shortener
 
@given(url=st.sampled_from(["https://a.com", "http://b.io"]))
def test_code_shape_property(url):
    shortener = Shortener()
    code = shortener.shorten(url)
    assert len(code) == 6
    assert all(c in string.ascii_letters + string.digits for c in code)
    assert shortener.resolve(code) == url

Explain-out-loud test

1. What is the test pyramid, and why not just write only E2E tests? (name the trade-off in one sentence)
2. What is a fixture, and how is it different from a helper function? (name two things fixtures do)
3. You inherited a codebase with no tests and 100k lines. What is the first test you write? (defend the choice)

What comes next

Next: S014 · Typing and dataclasses.

Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.

Explain-out-loud answer guide

1. A pyramid favours fast isolated tests while reserving broader tests for wiring and user journeys; no percentage proves adequacy.
2. Fixtures declare dependencies and manage reuse/teardown; a helper alone leaves that lifecycle orchestration to its caller.
3. Start with a deterministic regression for a high-risk observed bug or critical public behaviour, then expand around the code you change. Do not chase a coverage number before pinning an expectation.

Corrective source notes

• How to write and report assertions in tests - pytest documentation — pytest supports ordinary Python assertions and reports assertion failures. Checked 2026-10-07.
• How to use fixtures - pytest documentation — If a yield fixture fails before yielding, its post-yield teardown does not run. Checked 2026-10-07.

Additional primary references: collection defaults, rootdir, exit codes, monkeypatch, branch coverage, Hypothesis API and fixture health check.

These citations support the named claims, not blanket certification of the lesson. Worked proofs are derived in the text; execution scope and finite-test limits are recorded separately.