S027 · Recursion — Call Stack, Base Case, Worked Examples
The mental model for solving problems by solving smaller versions of themselves — plus the tricks (memoisation, tail-form, iterative rewrites) that make it survive production.
🎯 Write recursive solutions confidently — with a real base case, a shrinking argument, and the memoisation instinct that turns exponential brute force into polynomial time.
Why this session exists
Recursion is not a language feature; it's a way of thinking. Once you can see a problem as the same problem on a smaller input, huge swaths of DSA collapse: trees, graphs, divide-and-conquer, dynamic programming, backtracking, parsers, and the entire functional-programming toolkit. This session gives you the mental model AND the escape hatches (memoisation, iterative rewrite, tail-form) that keep recursion from crashing in Python, where the call stack is small and there's no tail-call optimisation.
- State the three parts of every recursion (base case, recursive case, shrinking argument) and identify them in any code you read.
- Write factorial, Fibonacci, tree traversal, and merge-sort recursively without notes.
- Turn a naive exponential recursion into a linear one with a one-line @lru_cache.
- Convert a recursive function into an iterative one using an explicit stack — and know when you have to.
- Recognise recursion crashing (RecursionError) and its two fixes: raise sys.setrecursionlimit, or rewrite iteratively.
Prerequisites
- S026 · Stacks & Queues — the call stack is a stack.
- S013 · Functions & Scope — recursion is a special case of function calling itself.
(a) Intuition · 5 min
You want to count the books on a giant shelf. You could count them all yourself (tedious). Or: you tell the person next to you ‘count the LEFT half, tell me the number; I'll count the right half; we add.’ They face the same problem — but on half the shelf — so they hand it to two more people. The shelf shrinks by half each time; eventually someone has a shelf of one book and can just answer ‘1’.
That's recursion: assume you already have the answer for a smaller version of the problem, then combine it. The base case is when the problem is small enough to answer directly.
A recursive function calls itself with a smaller / simpler input and combines the result. Three ingredients: base case (small enough to answer directly), recursive case (reduce to smaller problem + combine), and a shrinking argument (why we always get closer to the base case).
Miss the base case → infinite recursion → RecursionError. Miss the shrinking argument → same crash. Miss the recursive case → not recursion.
The three-part recipe
- Base case — the smallest input(s) where you can answer directly, no further recursion. Usually 0, 1, empty list, single node.
- Recursive case — assume the function works on smaller inputs; call it and combine the results.
- Shrinking argument — every recursive call must be on a STRICTLY smaller input (fewer elements, smaller number, smaller subtree). If it isn't, you loop forever.
Timeline — why recursion is everywhere
- 1958LISP invents recursive functionsJohn McCarthy makes recursion (not iteration) the default way to loop. Half of modern programming language design descends from this choice.
- 1960ALGOL 60 makes recursion mainstreamFirst widely-used procedural language to support recursion. Every serious language since assumes it works.
- 1969Divide-and-conquer is formalisedAho, Hopcroft, Ullman's algorithms textbook codifies mergesort, quicksort, FFT — all recursive.
- 1985SICP‘Structure and Interpretation of Computer Programs’ argues that recursion is the natural shape of most algorithms. Two generations of CS students learn Scheme because of it.
- 2005Python popularises decorators@lru_cache turns any pure recursive function into a memoised polynomial-time one with one line.
- 2017React Hooks (2018)React's Fiber reconciler proves the point in reverse: sometimes recursion is the wrong shape and you must rewrite with an explicit work stack.
(b) Visual walkthrough · 15 min
factorial(4) — trace the call stack
Each downward arrow is a call (frame pushed). Each upward arrow is a return (frame popped). The stack is at most O(n) deep — this is where memory dies for deep recursion.
Naive Fibonacci is EXPONENTIAL — see it
Look at how many times fib(3) and fib(2) are recomputed. On fib(50), you'd redo the same subproblems ~600 million times. Memoisation caches each (fib(k) → value) → each fib(k) computed once → linear time.
The four shapes of recursion
One recursive call per invocation. factorial, list-length, linked-list reverse.
Two recursive calls per invocation. Naive Fibonacci, tree traversals, mergesort.
Function A calls B, B calls A. Common in parsers (expression/term/factor).
Recursive call plus ‘undo’ on failure. N-queens, sudoku solver, permutations (S034).
Recursion vs iteration — pick your side
The natural shape
- Tree / graph traversals
- Divide-and-conquer (mergesort, quicksort, FFT)
- Backtracking (permutations, N-queens)
- Grammar parsers
The safe shape
- Very deep or unbounded recursion
- Linear scans (loops are just clearer)
- Hot performance paths (Python overhead per call)
- Anywhere you need pausable execution
The DP trick
- Overlapping subproblems (Fibonacci, edit distance)
- Pure function of its arguments
- @lru_cache is the one-line fix
- Turns O(2^n) into O(n)
The escape hatch
- Depth > 1000 (Python) or > 10k (Java)
- Need to pause/resume/cancel work
- React Fiber, generators, coroutines
- ‘Iterative DFS’ = recursion with your own stack
"Recursion is just a prettier way to write a loop. Anything recursive can be rewritten as a loop, so it's a style choice."
Every recursion can be made iterative, but only tail recursion converts to a plain loop for free. General recursion converts to a loop plus an explicit stack — you do not remove the stack, you just move it from the call frames onto the heap. The style choice is where the stack lives, not whether there is one.
Because the examples used to teach it — factorial, sum, countdown — are all tail-recursive or trivially linear, so the loop version really is a straight swap. Then you meet tree traversal or backtracking, where you must return to a node after processing a child, and discover you need to remember where you were. That memory is the stack, and no rewrite makes it vanish. In Python this matters twice over: there is no tail-call optimisation at all, so even the easy case still consumes a frame.
Python does not optimise tail calls — the frames are real and they run out:
import sys
print(sys.getrecursionlimit()) # 1000 by default
def countdown(n):
if n == 0: return 'done'
return countdown(n - 1) # tail call - still allocates a frame
try:
countdown(10000)
except RecursionError as e:
print('RecursionError:', e)
# the loop version uses O(1) memory
n = 10000
while n: n -= 1
print('done')Why does naive recursive Fibonacci take exponential time, while the identical recurrence with memoisation takes linear time? The recurrence did not change — so what did?
- 1The definition
fib(n) = fib(n-1) + fib(n-2)describes a tree of calls, not a chain.forced by · each call spawns two children, so the call graph branches - 2A binary tree of depth n has on the order of 2ⁿ nodes, and each node does O(1) work.forced by · the node count of a branching process compounds multiplicatively with depth
- 3But the number of distinct arguments ever passed is only n — the values 0 through n.forced by · every argument is derived by subtracting 1 or 2, so the reachable argument set is bounded by n
- 4So the exponential tree contains at most n distinct subproblems. Everything beyond those n is recomputation of an answer already derived elsewhere in the tree.forced by · the same subproblem is reachable by many different paths down the tree — the subproblems overlap
- 5Caching each argument's result the first time collapses every repeat into an O(1) lookup, so total work becomes (number of distinct subproblems) × (work per subproblem).forced by · a memo turns the call tree into a call DAG, and a DAG is traversed once per node
Therefore memoisation is not a speed trick — it is the recognition that the recursion tree was a DAG all along, and the tree shape was an artefact of forgetting.
And note what this predicts: memoisation helps only when subproblems overlap. Apply it to merge sort or to a plain tree traversal, where every call sees a distinct input, and you will pay the cache cost for zero hits. That single test — "do the subproblems repeat?" — is the entire boundary between divide-and-conquer and dynamic programming.
Do not trace recursion. You will run out of head-stack before the machine does. Instead: assume a working version of the function already exists for every smaller input, and your only job is to (a) handle the input so small there is nothing to do, and (b) combine the smaller answers into yours.
If both parts are correct and every call strictly shrinks the input, the whole thing is correct by induction. The machine's stack is an implementation detail you were never meant to simulate by hand.
- Base case first, and make it the case where the answer is obvious — usually empty or size 1, not size 0-is-impossible.
- Every recursive call must move strictly toward the base case, or you get infinite descent, not a bug you can spot by reading.
- Write the combine step as if the calls already returned the right answer. Do not verify them mentally.
- Depth costs memory. Python's limit is ~1000 frames; depth proportional to n is a production hazard, depth proportional to log n is fine.
Fire this model the moment you see: a tree or nested JSON · a directory walk · a problem defined in terms of itself · "all permutations / combinations / subsets" · a parser · a DAG of task dependencies (Airflow) · a query plan · anything where the shape of the data is itself recursive.
The algorithm is naturally recursive. Ship the recursion, or convert it to an explicit-stack loop?
RecursionError for a segfault that takes the process down with no tracebackDefault to recursion when depth is logarithmic, and to an explicit stack when depth can scale with untrusted input size. The failure mode that matters is the second one: recursive parsers over user-supplied nested data are a genuine denial-of-service surface, because the attacker chooses your depth.
In data engineering specifically, the recursion you write is usually over a schema (bounded, yours) while the recursion you must harden is over data (unbounded, theirs). Sort your recursive code into those two buckets and the decision makes itself.
(c) Hands-on · 25 min
Save as recursion.py. Zero dependencies.
"""recursion.py — the interview canon for recursion + memoisation.
Run: python recursion.py
"""
from __future__ import annotations
from functools import lru_cache
import time
import sys
# ------------------------------------------------------------------
# 1) Factorial — the ‘hello world’ of recursion
# ------------------------------------------------------------------
def factorial(n: int) -> int:
if n <= 1: # base case
return 1
return n * factorial(n - 1) # recursive case, shrinks by 1
# ------------------------------------------------------------------
# 2) Fibonacci — naive vs memoised
# ------------------------------------------------------------------
def fib_naive(n: int) -> int:
if n < 2:
return n
return fib_naive(n - 1) + fib_naive(n - 2) # exponential — see it below
@lru_cache(maxsize=None)
def fib_memo(n: int) -> int:
if n < 2:
return n
return fib_memo(n - 1) + fib_memo(n - 2) # linear — because each n computed once
# ------------------------------------------------------------------
# 3) List sum — recursive is fine but iteration is clearer
# ------------------------------------------------------------------
def sum_list(a: list[int]) -> int:
if not a:
return 0
return a[0] + sum_list(a[1:]) # careful — a[1:] copies! O(n^2) memory
def sum_list_iter(a: list[int]) -> int:
total = 0
for x in a:
total += x
return total
# ------------------------------------------------------------------
# 4) Mergesort — the classic divide-and-conquer
# ------------------------------------------------------------------
def merge_sort(a: list[int]) -> list[int]:
if len(a) <= 1: # base case
return a
mid = len(a) // 2
left = merge_sort(a[:mid]) # recursive on smaller
right = merge_sort(a[mid:])
return _merge(left, right)
def _merge(left: list[int], right: list[int]) -> list[int]:
out: list[int] = []
i = j = 0
while i < len(left) and j < len(right):
if left[i] <= right[j]:
out.append(left[i]); i += 1
else:
out.append(right[j]); j += 1
out.extend(left[i:])
out.extend(right[j:])
return out
# ------------------------------------------------------------------
# 5) Binary tree — the recursive definition matches the code exactly
# ------------------------------------------------------------------
class TreeNode:
__slots__ = ("val", "left", "right")
def __init__(self, val, left=None, right=None):
self.val = val
self.left = left
self.right = right
def tree_depth(root: TreeNode | None) -> int:
if root is None: # base case
return 0
return 1 + max(tree_depth(root.left), tree_depth(root.right))
def inorder(root: TreeNode | None) -> list:
if root is None:
return []
return inorder(root.left) + [root.val] + inorder(root.right)
# ------------------------------------------------------------------
# 6) Iterative DFS on a tree — recursion with your own explicit stack
# ------------------------------------------------------------------
def inorder_iter(root: TreeNode | None) -> list:
"""Same output as inorder() but uses an explicit stack — no recursion, no depth limit."""
out, stack, node = [], [], root
while node or stack:
while node: # descend the left spine
stack.append(node)
node = node.left
node = stack.pop()
out.append(node.val)
node = node.right
return out
# ------------------------------------------------------------------
# 7) The perf demo — see exponential vs linear
# ------------------------------------------------------------------
def perf_demo() -> None:
n = 30
t = time.perf_counter(); fib_naive(n); dt_naive = time.perf_counter() - t
fib_memo.cache_clear()
t = time.perf_counter(); fib_memo(n); dt_memo = time.perf_counter() - t
print(f"fib({n}) naive : {dt_naive*1000:.1f} ms {'(' + str(2**n) + ' calls)' if False else ''}")
print(f"fib({n}) memo : {dt_memo*1000:.4f} ms <- {dt_naive/dt_memo:.0f}x faster")
if __name__ == "__main__":
print("factorial(5) :", factorial(5))
print("sum_list([1..10]) :", sum_list(list(range(1, 11))))
print("merge_sort([3,1,4,1,5,9,2,6]) :", merge_sort([3, 1, 4, 1, 5, 9, 2, 6]))
# Build a small tree: 1
# / \
# 2 3
# / \
# 4 5
root = TreeNode(1, TreeNode(2, TreeNode(4), TreeNode(5)), TreeNode(3))
print("tree_depth :", tree_depth(root))
print("inorder recursive :", inorder(root))
print("inorder iterative :", inorder_iter(root))
perf_demo()
# Recursion limit — comment out unless you're brave
# sys.setrecursionlimit(2000)
# print("factorial(1500) last 5 digits :", str(factorial(1500))[-5:])Anatomy of the script
What each function teaches
def count_paths_with_sum(root: TreeNode | None, target: int) -> int:
from collections import defaultdict
counts = defaultdict(int)
counts[0] = 1
def dfs(node, running: int) -> int:
if node is None:
return 0
running += node.val
found = counts.get(running - target, 0)
counts[running] += 1
found += dfs(node.left, running) + dfs(node.right, running)
counts[running] -= 1 # undo — this is backtracking
return found
return dfs(root, 0)
# Test on the tree from the main script:
# tree_paths_with_sum(root, 6) should return 1 (path 1 -> 2 -> 3 doesn't exist; 1 -> 2 -> 4 = 7; but 1+2+3 doesn't exist either)The takeaway: recursion + a mutable running state = backtracking. You must undo your mutation on the way back up.
(d) Production reality · 15 min
Naive recursive JSON parser: for each nested object, recurse. Attacker sends a JSON blob with 100,000 levels of nested brackets. Parser recurses 100k times → segfault → server crashes.
CVEs for exactly this pattern have hit Go's encoding/json, Python's json, Ruby's JSON, and Java's Jackson at various points.
DEFAULT_JSON_LEVELS = 200 internally) and either uses an iterative parser or a small explicit-stack machine. If you're writing a parser, cap the depth.React's original reconciler was a recursive tree walk — beautiful, but blocking. Rendering a large tree hogged the main thread for tens of ms, freezing scroll and animation.
RecursionError: maximum recursion depth exceeded. Job crashes at 3 AM, wakes on-call.sys.setrecursionlimit(20_000) — buys headroom but doesn't fix the underlying issue; (2) rewrite with an explicit stack — no depth limit. Option 2 is the right long-term fix; option 1 is the ‘get pager off’ patch.Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
Teach these three, one minute each, no notes:
- What are the three parts of a correct recursion, and what breaks if any is missing?
- Why does memoisation turn Fibonacci from exponential to linear?
- Give one real-world case where recursion is the RIGHT tool and one where it's the WRONG tool.
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.