S031 · Sorting — Merge, Quick, and When to Trust the Built-in
The three sorts everyone should know (merge, quick, heap), why Python's sorted() beats them all, and the interview questions that ask you to implement them anyway.
🎯 Own mergesort, quicksort, and heapsort — plus understand why every real language ships hybrid sorts like TimSort or Introsort that you should almost always use instead.
Why this session exists
You will almost never implement a sort in production. But you WILL be asked about them in every algorithms interview, and — more importantly — you'll debug production issues where knowing which sort your language uses (and its stability, worst case, and memory profile) is the difference between shipping a fix in minutes vs days. This session teaches the three foundational sorts, the two production hybrids (TimSort and Introsort), and the ‘which sort should I actually pick?’ decision tree.
- Implement mergesort and quicksort correctly, from memory, in Python.
- Explain stable vs unstable sort and give one case where stability matters.
- State the worst-case and average-case time and space for each sort in the family.
- Know that Python's sorted() is TimSort — a hybrid mergesort/insertion — and roughly what it does.
- Choose the right sort (or non-sort) for a given constraint: memory, latency, streamed data.
Prerequisites
- S027 · Recursion — merge and quick are recursive divide-and-conquer.
- S029 · Heaps — heapsort is heap + repeated pop.
- S023 · Arrays & Strings — the substrate everything sorts.
(a) Intuition · 5 min
You have 50 shuffled cards. Merge sort: split into two piles, sort each pile (magic recursion), then merge them by repeatedly taking the smaller top card. Quicksort: pick a random card, split into ‘smaller than it’ and ‘bigger than it’ piles, sort each pile. Insertion sort: walk down the deck, inserting each card into its correct place in a sorted sub-deck to the left.
For 10 cards, any of these works and you can't tell the difference. For 10 million cards, the choice of algorithm decides whether it finishes before lunch or before the sun swallows Earth.
Three foundational sorts: mergesort (O(n log n) guaranteed, stable, needs O(n) memory), quicksort (O(n log n) average / O(n²) worst, in-place, unstable), heapsort (O(n log n) guaranteed, in-place, unstable). No single sort wins every dimension — every real language ships a HYBRID.
Python's sorted() is TimSort — mergesort with insertion-sort for small chunks and a clever ‘detect existing runs’ trick. It's stable, adaptive (fast on already-sorted or nearly-sorted data), and hard to beat.
The three tradeoff axes you must know
- Time complexity — worst case matters MORE than average case (in production, worst case is what pages you at 3am).
- Space complexity — in-place (O(1) or O(log n) stack) vs O(n) auxiliary. In-place is critical when memory is tight.
- Stability — does the sort preserve the relative order of equal-key items? Matters when you sort by one field and want ties broken by original order.
Timeline — 70 years of ‘just sort the array’
- 1945John von Neumann invents mergesortDuring the EDVAC project. First described merge sort as one of the demonstration programs for a stored-program computer.
- 1959ShellsortDonald Shell publishes the first practical improvement over O(n²) sorts.
- 1961Tony Hoare invents quicksort23 years old, working on machine-translated Russian → English. Later gets a Turing Award for a family of related contributions.
- 1964Heapsort · J. W. J. WilliamsThe first O(n log n) worst-case sort that runs in-place. Never beats quicksort in practice due to cache behaviour.
- 2002TimSort · Tim PetersWritten for Python — mergesort optimised for real-world data (partially-sorted input). Becomes the default in Java 7, Android, Swift, Rust's slice::sort_by.
- 2020pdqsort in Rust / C++ standard proposalsPattern-Defeating Quicksort — quicksort with introsort fallback and branchless partitioning. Faster than TimSort on integer arrays, unstable.
(b) Visual walkthrough · 15 min
Mergesort — divide, sort each half, merge
Quicksort — pivot + partition
The comparison table you should memorise
Guaranteed O(n log n)
- Stable ✅
- O(n) extra memory ❌
- Predictable worst case ✅
- Best when: you need stability or guaranteed performance
O(n log n) avg, O(n²) worst
- Unstable ❌
- In-place (O(log n) stack) ✅
- Fastest constants on random data ✅
- Best when: memory is tight, and you trust random pivots
Guaranteed O(n log n)
- Unstable ❌
- In-place ✅
- Cache-unfriendly (jumps around array) ❌
- Best when: you need O(1) extra memory AND worst-case guarantee — rarely picked
Real-world hybrid
- Stable ✅
- O(n log n) worst case ✅
- O(n) best case on sorted data ✅
- Best when: you're using Python's sorted() — which is always
The decision tree — which sort do I actually pick?
Use sorted() / list.sort() (TimSort). Stop reading — that's the answer 99% of the time.
std::sort (Introsort — quicksort with heapsort fallback for worst case) for speed. std::stable_sort for stability.
Mergesort if you need stability or predictable time. Quicksort if you need in-place. Never implement bubble sort seriously.
External merge sort — sort chunks in memory, merge chunks on disk with a k-way heap merge (this is what MapReduce does).
Insertion sort is often fastest — low constants, no recursion overhead. TimSort literally does this internally for small runs.
DON'T sort. Use a size-K heap → O(n log K) instead of O(n log n). See S029.
Where sorting shows up in real systems
Sorting in the wild
"Quicksort is O(n log n) and mergesort is O(n log n), so they're equivalent — pick either."
Quicksort is O(n log n) average and O(n²) worst case, in-place, unstable. Mergesort is O(n log n) worst case, needs O(n) auxiliary memory, and is stable. Those three differences — worst case, memory, stability — decide every real sorting choice. The shared average complexity decides none of them.
Because complexity tables list one number per algorithm and students memorise the row, not the footnotes. The footnotes are the entire content. Stability in particular looks like trivia until you sort a dataset by one key and then another and discover the first sort's ordering was destroyed — at which point "stable" is the only property you care about.
Stability is not an abstract property — it changes your output:
rows = [('b', 2), ('a', 1), ('c', 2), ('d', 1)]
# sort by value, then by nothing else: stable sort keeps original relative order
print(sorted(rows, key=lambda r: r[1]))
# [('a',1), ('d',1), ('b',2), ('c',2)] <- a before d, b before c: input order preserved
# multi-key sorting by chaining relies ENTIRELY on stability
by_name = sorted(rows, key=lambda r: r[0])
by_value = sorted(by_name, key=lambda r: r[1])
print(by_value) # sorted by value, ties broken by name - only works because sort is stableWhy can no comparison-based sort beat O(n log n)? This is stated as a law, but it is a counting argument you can reconstruct in four steps.
- 1A sort of n distinct items must be able to produce any of the n! possible orderings, because any of them could be the correct answer.forced by · the algorithm cannot know the input in advance; if some permutation were unreachable, that input would be sorted incorrectly
- 2A comparison-based algorithm learns about the input only through yes/no comparisons, so its execution is a binary decision tree.forced by · each comparison has exactly two outcomes, branching the algorithm's state in two
- 3Each distinct permutation must be a distinct leaf of that tree, so the tree needs at least n! leaves.forced by · two different correct answers cannot be produced by the identical sequence of decisions
- 4A binary tree with n! leaves has height at least log₂(n!), and the worst-case number of comparisons is the tree's height.forced by · a binary tree of height h has at most 2^h leaves, so h ≥ log₂(leaves)
- 5By Stirling's approximation, log₂(n!) ≈ n log₂n − n·log₂e, which is Θ(n log n).forced by · n! grows super-exponentially, and its logarithm grows as n log n
Therefore Ω(n log n) is not a limit of our cleverness — it is an information-theoretic floor. You need roughly n log n bits of information to identify one permutation out of n!, and each comparison yields at most one bit.
And note what this predicts: the bound applies only to comparison-based sorts. Sorts that extract more than one bit per operation escape it entirely — counting sort and radix sort read the key's structure directly and run in O(n + k) or O(n·d). That is not a loophole, it is the derivation telling you exactly which assumption to drop, and it is why sorting integer keys or fixed-width strings at scale is done with radix, not quicksort.
Every efficient sort is divide-and-conquer, and there are only two places to put the effort. Mergesort splits blindly down the middle (trivial split) and does all the real work combining two sorted halves (expensive merge). Quicksort does all the work up front partitioning around a pivot (expensive split) and then the combine is nothing — the pieces are already in the right regions.
That single choice explains everything downstream. A blind split always halves, so mergesort's depth is guaranteed log n. A pivot-based split only halves if the pivot is near the median, so quicksort's depth is a gamble — and a bad pivot on sorted input gives you depth n.
- Mergesort: guaranteed O(n log n), stable, O(n) extra memory, sequential access — which is why it is the basis of every external / distributed sort.
- Quicksort: average O(n log n) with the best constant factor of any comparison sort, in-place, unstable, O(n²) on adversarial or already-sorted input without randomised pivots.
- Heapsort: guaranteed O(n log n) and in-place, but unstable and cache-hostile — the safety net, rarely the first choice.
- Real library sorts are hybrids. Python's Timsort is mergesort that detects existing sorted runs, so nearly-sorted input costs O(n) — and real data is very often nearly sorted.
Fire this model the moment you see: ORDER BY in a query plan · a Spark shuffle (it is a distributed mergesort) · sort_values on a large DataFrame · a merge join · "top K" · a window function that needs an ordered partition · deduplication · any operation whose cost suddenly jumps and whose plan says "sort".
You must sort a dataset far larger than memory. In-memory sort per chunk then merge, or push the sort into the storage layer?
sortThe first question a senior engineer asks is not "which sort?" but "why am I sorting at all?". A very large fraction of production sorts exist to compute a top-K, a median, or a dedup — all of which have direct algorithms that never materialise a total order and never shuffle.
When the sort is genuinely required, the leverage is in moving it to write time. Sorting once at ingest and reading many times is strictly better than sorting on every query, and it is what every columnar format and storage engine is built around.
(c) Hands-on · 25 min
Save as sorting.py. Zero deps beyond stdlib.
"""sorting.py — implementations + benchmarks for the three foundational sorts.
Run: python sorting.py
"""
from __future__ import annotations
import heapq
import random
import time
# ------------------------------------------------------------------
# 1) Merge sort — the interview classic
# ------------------------------------------------------------------
def merge_sort(a: list[int]) -> list[int]:
if len(a) <= 1:
return a
mid = len(a) // 2
left = merge_sort(a[:mid])
right = merge_sort(a[mid:])
return _merge(left, right)
def _merge(left: list[int], right: list[int]) -> list[int]:
out: list[int] = []
i = j = 0
while i < len(left) and j < len(right):
if left[i] <= right[j]:
out.append(left[i]); i += 1
else:
out.append(right[j]); j += 1
out.extend(left[i:])
out.extend(right[j:])
return out
# ------------------------------------------------------------------
# 2) Quicksort — in-place with random pivot (avoids worst case on sorted input)
# ------------------------------------------------------------------
def quicksort(a: list[int], lo: int = 0, hi: int | None = None) -> None:
if hi is None:
hi = len(a) - 1
if lo >= hi:
return
p = _partition(a, lo, hi)
quicksort(a, lo, p - 1)
quicksort(a, p + 1, hi)
def _partition(a: list[int], lo: int, hi: int) -> int:
# Random pivot — SWAP to end. Prevents O(n^2) on already-sorted input.
pivot_idx = random.randint(lo, hi)
a[pivot_idx], a[hi] = a[hi], a[pivot_idx]
pivot = a[hi]
i = lo - 1
for j in range(lo, hi):
if a[j] <= pivot:
i += 1
a[i], a[j] = a[j], a[i]
a[i + 1], a[hi] = a[hi], a[i + 1]
return i + 1
# ------------------------------------------------------------------
# 3) Heap sort — build heap, repeatedly extract min
# ------------------------------------------------------------------
def heap_sort(a: list[int]) -> list[int]:
heap = list(a) # copy so we don't mutate input
heapq.heapify(heap) # O(n)
return [heapq.heappop(heap) for _ in range(len(heap))] # n * O(log n) = O(n log n)
# ------------------------------------------------------------------
# 4) Insertion sort — perfect for small or almost-sorted lists
# ------------------------------------------------------------------
def insertion_sort(a: list[int]) -> list[int]:
a = list(a)
for i in range(1, len(a)):
x = a[i]
j = i - 1
while j >= 0 and a[j] > x:
a[j + 1] = a[j]
j -= 1
a[j + 1] = x
return a
# ------------------------------------------------------------------
# 5) External merge — sorted chunks + k-way heap merge
# ------------------------------------------------------------------
def external_merge(chunks: list[list[int]]) -> list[int]:
"""Merge already-sorted chunks in O(n log k)."""
return list(heapq.merge(*chunks)) # stdlib's k-way merge
# ------------------------------------------------------------------
# 6) Stability demo — TimSort preserves equal-key order; naive quicksort does not
# ------------------------------------------------------------------
def stability_demo() -> None:
people = [("Alice", 30), ("Bob", 25), ("Carol", 30), ("Dan", 25), ("Eve", 30)]
# Sort by age with sorted() — stable, preserves name order within each age group
by_age_stable = sorted(people, key=lambda p: p[1])
print("stable (TimSort):", by_age_stable)
# Now shuffle then sort — same output because we've re-sorted
print("(if we sorted by NAME first then age, we'd get age-primary, name-secondary sorted)")
# ------------------------------------------------------------------
# 7) Benchmark our sorts vs Python's sorted() on 100k random ints
# ------------------------------------------------------------------
def benchmark() -> None:
n = 100_000
data = [random.randint(0, 1_000_000) for _ in range(n)]
for name, fn in [
("merge_sort", lambda d: merge_sort(d)),
("quicksort", lambda d: (quicksort(d[:]), d)[0] or sorted(d)), # returns None; use sorted for verify
("heap_sort", lambda d: heap_sort(d)),
("sorted() TimSort", lambda d: sorted(d)),
]:
d = data[:]
t = time.perf_counter()
if name == "quicksort":
quicksort(d)
result = d
else:
result = fn(d)
dt = time.perf_counter() - t
assert result == sorted(data)
print(f"{name:20s} n={n} : {dt*1000:.1f} ms")
if __name__ == "__main__":
print("merge_sort [3,1,4,1,5,9,2,6]:", merge_sort([3, 1, 4, 1, 5, 9, 2, 6]))
a = [3, 1, 4, 1, 5, 9, 2, 6]; quicksort(a); print("quicksort in-place:", a)
print("heap_sort [3,1,4,1,5,9,2,6]:", heap_sort([3, 1, 4, 1, 5, 9, 2, 6]))
print("insertion [3,1,4,1,5,9,2,6]:", insertion_sort([3, 1, 4, 1, 5, 9, 2, 6]))
print("external_merge:", external_merge([[1, 4, 7], [2, 5, 8], [3, 6, 9]]))
stability_demo()
benchmark()Anatomy of the script
What each function teaches
def quickselect(a: list[int], k: int) -> int:
"""Return the k-th smallest (0-indexed) — average O(n), worst O(n^2)."""
import random
a = list(a)
lo, hi = 0, len(a) - 1
while lo <= hi:
pivot_idx = random.randint(lo, hi)
a[pivot_idx], a[hi] = a[hi], a[pivot_idx]
pivot = a[hi]
i = lo - 1
for j in range(lo, hi):
if a[j] <= pivot:
i += 1
a[i], a[j] = a[j], a[i]
a[i + 1], a[hi] = a[hi], a[i + 1]
p = i + 1
if p == k:
return a[p]
elif p < k:
lo = p + 1
else:
hi = p - 1
return -1
print(quickselect([3, 2, 1, 5, 6, 4], 1)) # 2nd smallest = 2
print(quickselect([3, 2, 1, 5, 6, 4], 4)) # 5th smallest = 5Use case: ‘find the median’ in one pass (k = n//2), which the heap-based two-heap method also solves but quickselect is O(n) instead of O(n log n).
(d) Production reality · 15 min
Before Java 7, Arrays.sort(Object[]) was mergesort. Fine, but not fast enough for real workloads on partially-sorted data (which is the common case in real programs).
McIlroy showed that for any quicksort with a deterministic pivot rule, you can construct an input that forces O(n²) behaviour. Attackers exploited this against Apache and other services that sorted user data — DoS via crafted requests.
user_id. But two users hash to the same bucket after some transformation → unstable sort produces DIFFERENT ordering on different runs → downstream model training gets slightly different training set → metrics drift between reruns of ‘the same’ pipeline.sorted() is stable by default. In pandas, use df.sort_values(kind='mergesort') (or the newer kind='stable'). Numpy's default is quicksort (unstable) — you have to opt in with kind='mergesort'.Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
Teach these three, one minute each, no notes:
- Explain mergesort to a friend using a deck of cards.
- When would you pick quicksort over mergesort — and vice versa?
- Why should you almost always just call
sorted()instead of implementing anything?
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.