S011 · Errors, Exceptions & Debugging with pdb
The mindset shift that turns ‘print statement warrior’ into ‘engineer who kills bugs in minutes’. Try/except done right, custom exceptions, tracebacks decoded, pdb keystrokes memorised, and the modern replacements (icecream, ipdb, structlog).
🎯 Handle exceptions on purpose, read any traceback bottom-to-top in 30 seconds, and use pdb well enough that print-statement debugging feels slow.
Why this session exists
Errors are how Python tells you what went wrong. Most beginners treat them as ‘the thing that stops my program’ and reach for a broad except: to make it go away. Seniors treat them as information — each exception has a type, a message, a traceback, and a location that together tell you exactly where and why. This session teaches you to read that information, to raise exceptions on purpose (custom types, context), to catch them precisely, and to drop into pdb when the stack trace alone isn't enough. By the end you'll debug in minutes what used to take hours.
- Read any Python traceback bottom-to-top and locate the bug in 30 seconds.
- Raise the RIGHT exception for the situation — built-in when it fits, custom when it doesn't.
- Use try/except/else/finally correctly, including chained exceptions with `raise X from Y`.
- Drive `pdb` with `n / s / c / l / p / w / u / d / b / q` — no cheat sheet needed.
- Know when a print-statement is right (fast!) and when pdb is right (surgical), and use the modern tools (icecream, ipdb, breakpoint()).
Prerequisites
- S007 · Functions — stack frames are function calls.
- S009 · Classes & Objects — exceptions are classes.
(a) Intuition · 5 min
Imagine a mail room. When a delivery goes wrong — wrong address, damaged parcel — the mail room doesn't burn down. It puts the parcel in an envelope with a note (‘recipient unknown, returning to sender’) and passes it back up the chain until someone knows what to do.
Python exceptions are that envelope. They carry a type, a message, and a full trail (traceback) of who passed them along. Your job as a dev is to read the envelope, decide which layer of code knows what to do with it, and catch it exactly there — no earlier, no broader.
Formally: an exception is an object (subclass of BaseException) raised via raise. Raising unwinds the stack until a matching except is found; if none, the interpreter prints the traceback and exits. That's it.
The bugs happen when devs catch too broadly (except:) or too eagerly (except Exception: right at the boundary). Both hide the real problem. Catch narrow, catch late, and always preserve the traceback.
The three rules that separate junior from senior error handling
- Catch the narrowest type you can handle. `except ValueError:` beats `except Exception:` beats `except:`. Broad catches hide bugs.
- Catch as late as you can — at the layer that has enough context to decide what to do. Catching in a helper and swallowing gives the caller no chance.
- Preserve the traceback. Use `raise NewError(...) from original` to chain exceptions; NEVER `except Exception as e: raise MyError(str(e))` — that discards the original stack.
A quick history
- 1991Python 0.9.0 — exceptionsString-based at first. Everything is a raise-catchable string.
- 2000Python 2.0 — class-based exceptionsException class hierarchy replaces strings. You can `except SubclassOfException:`.
- 2008Python 3.0 — `raise X from Y`Explicit chaining. Preserves the original traceback under __cause__.
- 2018breakpoint() builtin (3.7)PEP 553 — `breakpoint()` replaces `import pdb; pdb.set_trace()`. Respects PYTHONBREAKPOINT for IDE plugins.
- 2022ExceptionGroup (3.11)PEP 654 — first-class ‘multiple exceptions from concurrent code’. asyncio TaskGroups use this.
(b) Visual walkthrough · 15 min
The exception hierarchy (the parts you'll actually touch)
The rule: catch Exception, never BaseException. Catching BaseException silences KeyboardInterrupt — your Ctrl-C stops working. Catching Exception is fine at the top of a task or main().
Reading a traceback bottom-to-top
It's the exception type + message. That tells you WHAT went wrong.
That's WHERE it went wrong — the innermost frame.
Each ‘File …’ line above shows how you got here. Usually the top 2-3 frames are your code; below that is stdlib/library.
That's usually the fix site — even if the exception was raised deeper.
try / except / else / finally — what each block does
The risky code
- Runs top-to-bottom
- Stops the moment an exception is raised
- Jumps to matching except (or unwinds if none)
The recovery
- Runs only if X is raised
- Multiple except clauses = pattern match top-down
- Use `except X as e:` to name the object
The ‘it worked’ branch
- Runs only if NO exception was raised in try
- Great for code that must run only on success
- Avoids wrapping too much in try
The cleanup
- ALWAYS runs — success, exception, return
- Perfect for closing files, connections, locks
- Modern replacement: context managers (`with`)
The pdb keystroke cheat sheet
12 commands = 95% of your debugging
The mental model to hold
"Exceptions are for errors, so the defensive thing to do is wrap risky code in try/except Exception and log it. Catching more is safer than catching less."
A broad except is the opposite of safe: it converts a loud, localised failure into a silent wrong answer that surfaces somewhere else, much later, with no stack trace pointing at the cause. Catch only the exceptions you can actually handle — meaning you have a specific recovery action, not just a log line.
Because it works in the demo. The script stops crashing, the log gets a line, and the shift ends. What it hides is that except Exception swallows KeyError from your own typo, AttributeError from a None you didn't expect, and MemoryError — bugs, not conditions — and the pipeline continues with a half-built result. In batch data work this is the worst possible failure mode: a crashed job is retried, but a job that "succeeded" while quietly dropping every malformed record writes bad data downstream and nobody looks. The narrow case where the myth is true — a top-level handler in a long-running service that must not die — is real, which is exactly why the habit spreads to places where it isn't.
Watch a broad catch eat a typo and report success:
rows = [{'id': 1, 'value': 10}, {'id': 2}]
total = 0
for r in rows:
try:
total += r['vlaue'] # typo, not a data problem
except Exception as e:
continue # 'handled'
print('done, total =', total) # done, total = 0
# narrow it and the bug names itself immediately:
# except KeyError as e: raise ValueError(f'row {r["id"]} missing {e}') from eWhy does a stack trace read bottom-up — the line that actually failed printed last? And why is raise ... from e more than cosmetic? Derive both from how exceptions propagate.
- 1Calling a function pushes a frame recording the caller's position, so control can return. The call stack is therefore a record of how you got here.forced by · a return address must be stored somewhere, and a stack is the natural structure for nesting
- 2When an exception is raised, the runtime walks that stack outward looking for a handler, popping frames as it goes.forced by · if the current frame can't handle it, only an ancestor can — there is nowhere else to look
- 3Each popped frame is appended to the traceback, so the traceback accumulates in the order frames were unwound: innermost failure first, outermost caller last.forced by · the exception object travels outward and records where it has been
- 4Python then prints it reversed — "most recent call last" — so the failing line is the final line of output, closest to your cursor.forced by · terminal output scrolls; the most-needed line should be where your eye already is
- 5If a handler raises a new exception, the original would be lost, so Python attaches it as
__cause__(explicit, viafrom) or__context__(implicit) and prints both chains.forced by · the new exception explains what the caller should do; the original explains why, and debugging needs both
Therefore the traceback is the unwinding path itself, and reading strategy follows: start at the bottom for the immediate failure, scan upward for the first frame in your code, ignore the library frames in between.
And note what this predicts: raise NewError(...) from e preserves the original trace under "The above exception was the direct cause", while a bare raise NewError(...) inside an except shows it under "During handling ... another exception occurred" — a subtle difference that tells a future reader whether the second failure was intentional translation or a bug in your handler. It also predicts why except: pass is uniquely destructive: it discards the only record of the path that produced the error.
Picture an exception as a package thrown outward through the call stack, stopping at the first frame that declared it will accept that type. Every frame it passes through stamps its address on the package. When it reaches the top with no taker, the runtime prints the stamps and stops the program.
So designing error handling is really designing catch altitude: how far out should this travel before someone deals with it? Too low and you handle things you don't understand; too high and you lose the context needed to recover.
- Catch at the level that has enough context to decide. A parser knows a row is malformed; only the caller knows whether to skip it, quarantine it, or fail the batch.
- Always catch the narrowest type.
except Exceptionis legitimate at exactly one place: a top-level boundary that logs withlogger.exceptionand re-raises or exits non-zero. - Use
finallyor context managers for cleanup — they run whether or not an exception passed through, which is the only guarantee that closes files and releases locks on the failure path. - Debugging order: read the traceback bottom-up, reproduce it deterministically, bisect the input, then use a debugger (
breakpoint()) — printing is a fallback, not a method.
Fire this model the moment you see: except: pass · a job that "succeeded" with suspiciously few output rows · a log line saying "error occurred" with no type or trace · a retry loop with no backoff and no exception filter · an error message naming a library file you have never opened.
A batch job hits a malformed record on row 4 million of 10 million. Fail fast, skip and continue, or quarantine and continue?
Fail fast during development and on anything authoritative; quarantine in production ingestion. What separates the two in practice is not the code path but the observability: skipping is only acceptable when the skip count is a first-class metric with a threshold that pages someone.
And keep the distinction sharp between the two error classes. Dirty input is an expected condition and deserves a policy. A KeyError from your own code is a bug and deserves a crash — routing it through the same tolerant path is how a pipeline ends up emitting zeros for a month.
(c) Hands-on · 25 min
Save as s011_errors.py, run.
"""s011_errors.py — exceptions, debugging, and the modern toolkit."""
from __future__ import annotations
from contextlib import contextmanager
from typing import Iterable
import json
import logging
import traceback
logging.basicConfig(
level=logging.INFO,
format="%(levelname)s [%(name)s] %(message)s",
)
log = logging.getLogger("s011")
# =========================
# Part 1 · try / except / else / finally
# =========================
print("--- Part 1 · try/except/else/finally ---")
def parse_int_safe(s: str) -> int | None:
try:
n = int(s)
except ValueError:
log.warning("not an int: %r", s)
return None
else:
log.info("parsed cleanly: %r → %d", s, n)
return n
finally:
log.debug("attempted parse of %r", s)
print(parse_int_safe("42"))
print(parse_int_safe("nope"))
# =========================
# Part 2 · Custom exceptions with context
# =========================
print("\n--- Part 2 · custom exceptions ---")
class DomainError(Exception):
"""Base for all domain errors in this app."""
class OrderError(DomainError):
def __init__(self, order_id: str, reason: str) -> None:
super().__init__(f"order {order_id!r}: {reason}")
self.order_id = order_id
self.reason = reason
class OrderNotFound(OrderError):
def __init__(self, order_id: str) -> None:
super().__init__(order_id, "not found")
try:
raise OrderNotFound("O-1234")
except OrderError as e:
log.error("caught %s (id=%s reason=%s)", type(e).__name__, e.order_id, e.reason)
# Callers can catch broadly (DomainError) or narrowly (OrderNotFound) as needed.
# =========================
# Part 3 · Exception chaining — `raise ... from ...`
# =========================
print("\n--- Part 3 · chaining ---")
def load_order(raw: str) -> dict:
try:
return json.loads(raw)
except json.JSONDecodeError as e:
raise OrderError("<unknown>", "malformed JSON") from e
try:
load_order("{not-json")
except OrderError as e:
log.error("chained: %s", e)
log.error("root cause: %s", e.__cause__) # the original JSONDecodeError
# NEVER lose the original traceback. `raise NewError(...) from e` preserves it.
# =========================
# Part 4 · Context managers replace try/finally
# =========================
print("\n--- Part 4 · context managers ---")
@contextmanager
def open_and_lock(path: str):
log.info("acquiring %s", path)
f = open(path, "w")
try:
yield f
finally:
log.info("releasing %s", path)
f.close()
with open_and_lock("/tmp/s011_demo.txt") as f:
f.write("hello from context manager\n")
# Same guarantee as try/finally, expressed once at the resource layer.
# =========================
# Part 5 · What to do with an exception you can't handle
# =========================
print("\n--- Part 5 · logging vs re-raising ---")
def process_all(items: Iterable[dict]) -> list[dict]:
"""Log per-item errors but keep processing. Return successes only."""
out: list[dict] = []
for i, item in enumerate(items):
try:
out.append(process_one(item))
except DomainError:
log.exception("skipping item %d", i) # logs message + traceback
return out
def process_one(item: dict) -> dict:
if "id" not in item:
raise OrderError("<missing>", "no id field")
return {"id": item["id"], "ok": True}
print(process_all([{"id": 1}, {"broken": True}, {"id": 3}]))
# =========================
# Part 6 · breakpoint() — pdb without the import dance
# =========================
print("\n--- Part 6 · breakpoint() ---")
def buggy_average(values: list[float]) -> float:
total = sum(values)
# Uncomment the next line to drop into pdb here:
# breakpoint()
return total / len(values) # ZeroDivisionError if values is empty
try:
print(buggy_average([]))
except ZeroDivisionError:
log.exception("buggy_average failed — traceback:")
# If you set breakpoint() above, pdb would open here.
# Try: n (next), p total (print), p values, l (list), q (quit).
# =========================
# Part 7 · Reading a traceback like a pro
# =========================
print("\n--- Part 7 · traceback inspection ---")
def a(): b()
def b(): c()
def c(): raise ValueError("boom!")
try:
a()
except ValueError:
print("--- traceback ---")
traceback.print_exc()
print("--- summary ---")
for frame in traceback.extract_tb(traceback.sys.exc_info()[2]):
print(f" {frame.filename}:{frame.lineno} in {frame.name}: {frame.line}")
# You now know where + how you got there, programmatically.
# `logging.exception` and `logger.error(..., exc_info=True)` capture the same.
# =========================
# Part 8 · Modern niceties
# =========================
print("\n--- Part 8 · modern tools ---")
# icecream (pip install icecream): print with variable names + source line.
# from icecream import ic
# ic(some_var) # → "ic| some_var: value"
# ipdb (pip install ipdb): pdb with IPython niceties (colors, tab-completion).
# breakpoint() will pick up ipdb if PYTHONBREAKPOINT=ipdb.set_trace
# structlog (pip install structlog): structured logging (JSON) instead of f-strings.
# Vital in production so log aggregators can grep by field.
log.info("End of session — sample structured line", extra={"session": "s011", "ok": True})What each block teaches
Anatomy of the exercises
Save this to debug_me.py and run it:
def find_max(items):
m = items[0]
for x in items:
if x > m:
m = x
return m
def summarise(rows):
return {"max": find_max(rows), "count": len(rows)}
breakpoint()
print(summarise([])) # will crash — figure out why using pdbAt the pdb prompt:
n(next) to step to the print line.s(step) to dive INTOsummarise.l(list) to see the surrounding code.p rowsto see the argument.sagain to step intofind_max.p items— see what you have.p items[0]— now you know the exception.w(where) to see the stack.uto go back up tosummarise, inspect state.qto quit.
Now fix find_max to raise a friendlier ValueError when items is empty.
(d) Production reality · 15 min
An engineer wraps a flaky third-party call in try: ... except: pass. It ‘works’ in staging. In prod under real load, the third-party call also raises MemoryError and OperationalError from the DB it hits internally — both silenced. Bug reports come in for months as ‘some orders never appear’.
Catch the SPECIFIC exception you know how to handle: except requests.Timeout:. Log everything else with log.exception(...). Never except:, and never silence without at least a log line.
except clause should answer two questions: WHICH exception do I know how to handle here? WHAT do I do with it? If either answer is ‘all of them’ / ‘nothing’, you're papering over a bug.A Celery task wraps its whole body in try/except Exception, logs the error message, and returns None. From Celery's perspective, the task succeeded. Nobody notices for weeks. Then a customer complains their invoice never arrived.
Either RE-RAISE the exception (Celery will mark the task failed, retry, and alert) or explicitly return a failure result the caller understands. Silently returning None from a wrapped block is a distributed-systems anti-pattern.
A junior debugging a state-machine bug adds print(state) at 12 places. Runs the flow. State is fine at all 12 points. Bug still there. Adds more prints. Six hours in, still not found.
breakpoint() at the start of the problematic step. Step through with `n` and `s`. When state changes unexpectedly, `w` to see the call stack, `u` to inspect the caller. Bug found in 4 minutes.
Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
If you can't teach these three without notes, redo the session:
- Why is
except:almost always wrong? (name what it catches that you don't want) - What does
raise X from Ydo differently fromraise X? (one word: chaining) - List the six pdb commands you'll actually use. (n, s, c, w, u, d — say what each does)
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.