Search Tech Journey

Find topics, journeys and posts

6-month learning plan31 / 130
back to blog
pythonbeginner 50m read

S031 · Sorting — Merge, Quick, and When to Trust the Built-in

The three sorts everyone should know (merge, quick, heap), why Python's sorted() beats them all, and the interview questions that ask you to implement them anyway.

🧩DSAM03 · Data Structures & Algorithms· Session 031 of 130 90 min

🎯 Own mergesort, quicksort, and heapsort — plus understand why every real language ships hybrid sorts like TimSort or Introsort that you should almost always use instead.

Why this session exists

You will almost never implement a sort in production. But you WILL be asked about them in every algorithms interview, and — more importantly — you'll debug production issues where knowing which sort your language uses (and its stability, worst case, and memory profile) is the difference between shipping a fix in minutes vs days. This session teaches the three foundational sorts, the two production hybrids (TimSort and Introsort), and the ‘which sort should I actually pick?’ decision tree.

You will be able to
  • Implement mergesort and quicksort correctly, from memory, in Python.
  • Explain stable vs unstable sort and give one case where stability matters.
  • State the worst-case and average-case time and space for each sort in the family.
  • Know that Python's sorted() is TimSort — a hybrid mergesort/insertion — and roughly what it does.
  • Choose the right sort (or non-sort) for a given constraint: memory, latency, streamed data.

Prerequisites

  • S027 · Recursion — merge and quick are recursive divide-and-conquer.
  • S029 · Heaps — heapsort is heap + repeated pop.
  • S023 · Arrays & Strings — the substrate everything sorts.


(a) Intuition · 5 min

Sorting is what you do when your deck of cards is out of order
🌍 Real world

You have 50 shuffled cards. Merge sort: split into two piles, sort each pile (magic recursion), then merge them by repeatedly taking the smaller top card. Quicksort: pick a random card, split into ‘smaller than it’ and ‘bigger than it’ piles, sort each pile. Insertion sort: walk down the deck, inserting each card into its correct place in a sorted sub-deck to the left.

For 10 cards, any of these works and you can't tell the difference. For 10 million cards, the choice of algorithm decides whether it finishes before lunch or before the sun swallows Earth.

💻 Code world

Three foundational sorts: mergesort (O(n log n) guaranteed, stable, needs O(n) memory), quicksort (O(n log n) average / O(n²) worst, in-place, unstable), heapsort (O(n log n) guaranteed, in-place, unstable). No single sort wins every dimension — every real language ships a HYBRID.

Python's sorted() is TimSort — mergesort with insertion-sort for small chunks and a clever ‘detect existing runs’ trick. It's stable, adaptive (fast on already-sorted or nearly-sorted data), and hard to beat.

The three tradeoff axes you must know

Every sort is a point in a 3D trade-off space
  • Time complexity — worst case matters MORE than average case (in production, worst case is what pages you at 3am).
  • Space complexity — in-place (O(1) or O(log n) stack) vs O(n) auxiliary. In-place is critical when memory is tight.
  • Stability — does the sort preserve the relative order of equal-key items? Matters when you sort by one field and want ties broken by original order.

Timeline — 70 years of ‘just sort the array’

  1. 1945
    John von Neumann invents mergesort
    During the EDVAC project. First described merge sort as one of the demonstration programs for a stored-program computer.
  2. 1959
    Shellsort
    Donald Shell publishes the first practical improvement over O(n²) sorts.
  3. 1961
    Tony Hoare invents quicksort
    23 years old, working on machine-translated Russian → English. Later gets a Turing Award for a family of related contributions.
  4. 1964
    Heapsort · J. W. J. Williams
    The first O(n log n) worst-case sort that runs in-place. Never beats quicksort in practice due to cache behaviour.
  5. 2002
    TimSort · Tim Peters
    Written for Python — mergesort optimised for real-world data (partially-sorted input). Becomes the default in Java 7, Android, Swift, Rust's slice::sort_by.
  6. 2020
    pdqsort in Rust / C++ standard proposals
    Pattern-Defeating Quicksort — quicksort with introsort fallback and branchless partitioning. Faster than TimSort on integer arrays, unstable.

(b) Visual walkthrough · 15 min

Mergesort — divide, sort each half, merge

Quicksort — pivot + partition

The comparison table you should memorise

Merge sort

Guaranteed O(n log n)

  • Stable ✅
  • O(n) extra memory ❌
  • Predictable worst case ✅
  • Best when: you need stability or guaranteed performance
Quicksort

O(n log n) avg, O(n²) worst

  • Unstable ❌
  • In-place (O(log n) stack) ✅
  • Fastest constants on random data ✅
  • Best when: memory is tight, and you trust random pivots
Heapsort

Guaranteed O(n log n)

  • Unstable ❌
  • In-place ✅
  • Cache-unfriendly (jumps around array) ❌
  • Best when: you need O(1) extra memory AND worst-case guarantee — rarely picked
TimSort (Python, Java)

Real-world hybrid

  • Stable ✅
  • O(n log n) worst case ✅
  • O(n) best case on sorted data ✅
  • Best when: you're using Python's sorted() — which is always

The decision tree — which sort do I actually pick?

11
Python or Java?

Use sorted() / list.sort() (TimSort). Stop reading — that's the answer 99% of the time.

22
C++?

std::sort (Introsort — quicksort with heapsort fallback for worst case) for speed. std::stable_sort for stability.

33
Doing it manually because interview / exercise?

Mergesort if you need stability or predictable time. Quicksort if you need in-place. Never implement bubble sort seriously.

44
Data > memory?

External merge sort — sort chunks in memory, merge chunks on disk with a k-way heap merge (this is what MapReduce does).

55
Data is small (<50 items)?

Insertion sort is often fastest — low constants, no recursion overhead. TimSort literally does this internally for small runs.

66
Only need top-K?

DON'T sort. Use a size-K heap → O(n log K) instead of O(n log n). See S029.

Where sorting shows up in real systems

Sorting in the wild

Database ORDER BY
Every SQL ORDER BY not covered by an index runs an in-memory or external sort. Postgres uses quicksort + external merge sort for large result sets.
database
Log aggregation
Merging N sorted log streams by timestamp is a k-way merge with a heap (S029 pattern), not a re-sort of the concatenation.
streaming
MapReduce shuffle stage
The shuffle-and-sort phase is external mergesort at petabyte scale. This is where all the runtime goes in a Hadoop / Spark job.
big data
TCP reassembly
Out-of-order packets held in a priority queue keyed by sequence number until the missing packets arrive — a variant of insertion sort.
networking
Auction / bidding systems
Ad tech auction results sorted by bid amount within a strict latency budget — high-frequency sorting with tight tail latency.
ad tech

Common misconception
✗ What most people think

"Quicksort is O(n log n) and mergesort is O(n log n), so they're equivalent — pick either."

✓ What is actually true

Quicksort is O(n log n) average and O(n²) worst case, in-place, unstable. Mergesort is O(n log n) worst case, needs O(n) auxiliary memory, and is stable. Those three differences — worst case, memory, stability — decide every real sorting choice. The shared average complexity decides none of them.

Why the myth is so sticky

Because complexity tables list one number per algorithm and students memorise the row, not the footnotes. The footnotes are the entire content. Stability in particular looks like trivia until you sort a dataset by one key and then another and discover the first sort's ordering was destroyed — at which point "stable" is the only property you care about.

Prove it to yourself

Stability is not an abstract property — it changes your output:

rows = [('b', 2), ('a', 1), ('c', 2), ('d', 1)]

# sort by value, then by nothing else: stable sort keeps original relative order
print(sorted(rows, key=lambda r: r[1]))
# [('a',1), ('d',1), ('b',2), ('c',2)]  <- a before d, b before c: input order preserved

# multi-key sorting by chaining relies ENTIRELY on stability
by_name  = sorted(rows, key=lambda r: r[0])
by_value = sorted(by_name, key=lambda r: r[1])
print(by_value)   # sorted by value, ties broken by name - only works because sort is stable
From first principles
Start with the question

Why can no comparison-based sort beat O(n log n)? This is stated as a law, but it is a counting argument you can reconstruct in four steps.

  1. 1
    A sort of n distinct items must be able to produce any of the n! possible orderings, because any of them could be the correct answer.
    forced by · the algorithm cannot know the input in advance; if some permutation were unreachable, that input would be sorted incorrectly
  2. 2
    A comparison-based algorithm learns about the input only through yes/no comparisons, so its execution is a binary decision tree.
    forced by · each comparison has exactly two outcomes, branching the algorithm's state in two
  3. 3
    Each distinct permutation must be a distinct leaf of that tree, so the tree needs at least n! leaves.
    forced by · two different correct answers cannot be produced by the identical sequence of decisions
  4. 4
    A binary tree with n! leaves has height at least log₂(n!), and the worst-case number of comparisons is the tree's height.
    forced by · a binary tree of height h has at most 2^h leaves, so h ≥ log₂(leaves)
  5. 5
    By Stirling's approximation, log₂(n!) ≈ n log₂n − n·log₂e, which is Θ(n log n).
    forced by · n! grows super-exponentially, and its logarithm grows as n log n
⇒ Therefore

Therefore Ω(n log n) is not a limit of our cleverness — it is an information-theoretic floor. You need roughly n log n bits of information to identify one permutation out of n!, and each comparison yields at most one bit.

And note what this predicts: the bound applies only to comparison-based sorts. Sorts that extract more than one bit per operation escape it entirely — counting sort and radix sort read the key's structure directly and run in O(n + k) or O(n·d). That is not a loophole, it is the derivation telling you exactly which assumption to drop, and it is why sorting integer keys or fixed-width strings at scale is done with radix, not quicksort.

Mental modelSplit-then-work vs. work-then-split

Every efficient sort is divide-and-conquer, and there are only two places to put the effort. Mergesort splits blindly down the middle (trivial split) and does all the real work combining two sorted halves (expensive merge). Quicksort does all the work up front partitioning around a pivot (expensive split) and then the combine is nothing — the pieces are already in the right regions.

That single choice explains everything downstream. A blind split always halves, so mergesort's depth is guaranteed log n. A pivot-based split only halves if the pivot is near the median, so quicksort's depth is a gamble — and a bad pivot on sorted input gives you depth n.

  • Mergesort: guaranteed O(n log n), stable, O(n) extra memory, sequential access — which is why it is the basis of every external / distributed sort.
  • Quicksort: average O(n log n) with the best constant factor of any comparison sort, in-place, unstable, O(n²) on adversarial or already-sorted input without randomised pivots.
  • Heapsort: guaranteed O(n log n) and in-place, but unstable and cache-hostile — the safety net, rarely the first choice.
  • Real library sorts are hybrids. Python's Timsort is mergesort that detects existing sorted runs, so nearly-sorted input costs O(n) — and real data is very often nearly sorted.
🔔 Fires when you see

Fire this model the moment you see: ORDER BY in a query plan · a Spark shuffle (it is a distributed mergesort) · sort_values on a large DataFrame · a merge join · "top K" · a window function that needs an ordered partition · deduplication · any operation whose cost suddenly jumps and whose plan says "sort".

The tradeoff

You must sort a dataset far larger than memory. In-memory sort per chunk then merge, or push the sort into the storage layer?

External merge sort
+ you gain works at any scale with bounded memory: sort chunks that fit, spill them, then merge k sorted runs with a heap; access is fully sequential, which is what disks and object stores are fastest at
− you pay every byte is written and re-read at least once (often twice), so it is I/O-bound; and in a distributed setting the merge phase is a shuffle, the single most expensive primitive you have
pick when the data genuinely exceeds memory and must be globally ordered — the default, and what Spark does when you call sort
Sort at write time / keep data pre-sorted
+ you gain the read-time sort disappears entirely; enables range pruning, min/max skipping and merge joins with no shuffle; amortises the cost across many reads
− you pay writes become more expensive and more complex, and the sort key is now a schema decision you cannot cheaply change later — a wrong key is expensive to undo
pick when the dataset is read far more often than written and one key dominates the query pattern — clustered indexes, Z-ordering, LSM trees and partitioned tables are all this bet
Avoid the total order — sample, bucket, or approximate
+ you gain skips the O(n log n) and the shuffle completely; range-partitioning by sampled boundaries gives per-partition ordering at a fraction of the cost, and approximate quantiles answer most "sort to find the top/median" questions directly
− you pay you no longer have a globally sorted result, and approximate answers carry an error bound you must be able to defend to whoever consumes the number
pick when the sort was a means to an end — top-K, percentiles, dedup — rather than the deliverable itself, which is the majority of cases
What a senior engineer actually does

The first question a senior engineer asks is not "which sort?" but "why am I sorting at all?". A very large fraction of production sorts exist to compute a top-K, a median, or a dedup — all of which have direct algorithms that never materialise a total order and never shuffle.

When the sort is genuinely required, the leverage is in moving it to write time. Sorting once at ingest and reading many times is strictly better than sorting on every query, and it is what every columnar format and storage engine is built around.


(c) Hands-on · 25 min

Save as sorting.py. Zero deps beyond stdlib.

"""sorting.py — implementations + benchmarks for the three foundational sorts.
 
Run:  python sorting.py
"""
from __future__ import annotations
import heapq
import random
import time
 
 
# ------------------------------------------------------------------
# 1) Merge sort — the interview classic
# ------------------------------------------------------------------
def merge_sort(a: list[int]) -> list[int]:
    if len(a) <= 1:
        return a
    mid = len(a) // 2
    left = merge_sort(a[:mid])
    right = merge_sort(a[mid:])
    return _merge(left, right)
 
 
def _merge(left: list[int], right: list[int]) -> list[int]:
    out: list[int] = []
    i = j = 0
    while i < len(left) and j < len(right):
        if left[i] <= right[j]:
            out.append(left[i]); i += 1
        else:
            out.append(right[j]); j += 1
    out.extend(left[i:])
    out.extend(right[j:])
    return out
 
 
# ------------------------------------------------------------------
# 2) Quicksort — in-place with random pivot (avoids worst case on sorted input)
# ------------------------------------------------------------------
def quicksort(a: list[int], lo: int = 0, hi: int | None = None) -> None:
    if hi is None:
        hi = len(a) - 1
    if lo >= hi:
        return
    p = _partition(a, lo, hi)
    quicksort(a, lo, p - 1)
    quicksort(a, p + 1, hi)
 
 
def _partition(a: list[int], lo: int, hi: int) -> int:
    # Random pivot — SWAP to end. Prevents O(n^2) on already-sorted input.
    pivot_idx = random.randint(lo, hi)
    a[pivot_idx], a[hi] = a[hi], a[pivot_idx]
    pivot = a[hi]
    i = lo - 1
    for j in range(lo, hi):
        if a[j] <= pivot:
            i += 1
            a[i], a[j] = a[j], a[i]
    a[i + 1], a[hi] = a[hi], a[i + 1]
    return i + 1
 
 
# ------------------------------------------------------------------
# 3) Heap sort — build heap, repeatedly extract min
# ------------------------------------------------------------------
def heap_sort(a: list[int]) -> list[int]:
    heap = list(a)                   # copy so we don't mutate input
    heapq.heapify(heap)              # O(n)
    return [heapq.heappop(heap) for _ in range(len(heap))]   # n * O(log n) = O(n log n)
 
 
# ------------------------------------------------------------------
# 4) Insertion sort — perfect for small or almost-sorted lists
# ------------------------------------------------------------------
def insertion_sort(a: list[int]) -> list[int]:
    a = list(a)
    for i in range(1, len(a)):
        x = a[i]
        j = i - 1
        while j >= 0 and a[j] > x:
            a[j + 1] = a[j]
            j -= 1
        a[j + 1] = x
    return a
 
 
# ------------------------------------------------------------------
# 5) External merge — sorted chunks + k-way heap merge
# ------------------------------------------------------------------
def external_merge(chunks: list[list[int]]) -> list[int]:
    """Merge already-sorted chunks in O(n log k)."""
    return list(heapq.merge(*chunks))       # stdlib's k-way merge
 
 
# ------------------------------------------------------------------
# 6) Stability demo — TimSort preserves equal-key order; naive quicksort does not
# ------------------------------------------------------------------
def stability_demo() -> None:
    people = [("Alice", 30), ("Bob", 25), ("Carol", 30), ("Dan", 25), ("Eve", 30)]
    # Sort by age with sorted() — stable, preserves name order within each age group
    by_age_stable = sorted(people, key=lambda p: p[1])
    print("stable  (TimSort):", by_age_stable)
    # Now shuffle then sort — same output because we've re-sorted
    print("(if we sorted by NAME first then age, we'd get age-primary, name-secondary sorted)")
 
 
# ------------------------------------------------------------------
# 7) Benchmark our sorts vs Python's sorted() on 100k random ints
# ------------------------------------------------------------------
def benchmark() -> None:
    n = 100_000
    data = [random.randint(0, 1_000_000) for _ in range(n)]
 
    for name, fn in [
        ("merge_sort",  lambda d: merge_sort(d)),
        ("quicksort",   lambda d: (quicksort(d[:]), d)[0] or sorted(d)),  # returns None; use sorted for verify
        ("heap_sort",   lambda d: heap_sort(d)),
        ("sorted() TimSort", lambda d: sorted(d)),
    ]:
        d = data[:]
        t = time.perf_counter()
        if name == "quicksort":
            quicksort(d)
            result = d
        else:
            result = fn(d)
        dt = time.perf_counter() - t
        assert result == sorted(data)
        print(f"{name:20s} n={n} : {dt*1000:.1f} ms")
 
 
if __name__ == "__main__":
    print("merge_sort [3,1,4,1,5,9,2,6]:", merge_sort([3, 1, 4, 1, 5, 9, 2, 6]))
    a = [3, 1, 4, 1, 5, 9, 2, 6]; quicksort(a); print("quicksort in-place:", a)
    print("heap_sort  [3,1,4,1,5,9,2,6]:", heap_sort([3, 1, 4, 1, 5, 9, 2, 6]))
    print("insertion  [3,1,4,1,5,9,2,6]:", insertion_sort([3, 1, 4, 1, 5, 9, 2, 6]))
    print("external_merge:", external_merge([[1, 4, 7], [2, 5, 8], [3, 6, 9]]))
    stability_demo()
    benchmark()

Anatomy of the script

What each function teaches

merge_sort
Textbook divide-and-conquer. Recurses into halves, merges. O(n log n) time, O(n) extra memory. Stable.
d-and-c
quicksort + _partition
RANDOM pivot is the key: without it, sorted input triggers O(n²). Real production libraries use ‘median of three’ or introspective sort (introsort) to bound worst case.
in-place
heap_sort
heapify (O(n)) + n pops (each O(log n)) = O(n log n). In-place semantically but this version returns a new list for clarity.
heap
insertion_sort
O(n²) worst but O(n) on already-sorted data. TimSort uses this for runs \< 64 elements because constants are tiny.
small
external_merge
heapq.merge — the stdlib k-way merger. When your data doesn't fit in memory, this is the shape of the answer.
external
stability_demo
Sorting by (age) preserves name order within equal ages — that's what ‘stable’ means. Chain-sortable via multiple stable passes.
stability
benchmark
Real numbers: on 100k random ints, sorted() (TimSort) is typically 10× faster than our hand-rolled Python sorts. The C implementation matters.
perf
Try itImplement quickselect — find the k-th smallest in O(n) average
def quickselect(a: list[int], k: int) -> int:
    """Return the k-th smallest (0-indexed) — average O(n), worst O(n^2)."""
    import random
    a = list(a)
    lo, hi = 0, len(a) - 1
    while lo <= hi:
        pivot_idx = random.randint(lo, hi)
        a[pivot_idx], a[hi] = a[hi], a[pivot_idx]
        pivot = a[hi]
        i = lo - 1
        for j in range(lo, hi):
            if a[j] <= pivot:
                i += 1
                a[i], a[j] = a[j], a[i]
        a[i + 1], a[hi] = a[hi], a[i + 1]
        p = i + 1
        if p == k:
            return a[p]
        elif p < k:
            lo = p + 1
        else:
            hi = p - 1
    return -1
 
print(quickselect([3, 2, 1, 5, 6, 4], 1))    # 2nd smallest = 2
print(quickselect([3, 2, 1, 5, 6, 4], 4))    # 5th smallest = 5

Use case: ‘find the median’ in one pass (k = n//2), which the heap-based two-heap method also solves but quickselect is O(n) instead of O(n log n).

💡 Hint · Same partition as quicksort, but only recurse into the side containing index k. Skip the other side entirely → average O(n), worst O(n²). This is what numpy.partition does under the hood.

(d) Production reality · 15 min

War story Java 7 · 2011· 2011entire JVM ecosystem
🔥 What broke

Before Java 7, Arrays.sort(Object[]) was mergesort. Fine, but not fast enough for real workloads on partially-sorted data (which is the common case in real programs).

🧯 The fix
Java 7 switched to TimSort (Python's algorithm, ported by Joshua Bloch). Real workloads (mostly-sorted or run-heavy input) got 2–5× faster. Android, Swift, Rust's stable sort, Chrome's V8 all followed.
🎓 Lesson to steal
TimSort is the single most-adopted algorithm of the 21st century. It won on the strength of ‘real data is not random’ — a lesson that keeps repeating in performance engineering.
Post-mortem
War story Adversarial quicksort attacks · 2003· 2003every language shipping a naive quicksort
🔥 What broke

McIlroy showed that for any quicksort with a deterministic pivot rule, you can construct an input that forces O(n²) behaviour. Attackers exploited this against Apache and other services that sorted user data — DoS via crafted requests.

🧯 The fix
Modern quicksorts use RANDOM pivots or ‘median of three’ or full Introsort (falls back to heapsort when recursion depth exceeds 2·log₂(n)). Every real language now defends against this.
🎓 Lesson to steal
Textbook quicksort is a security bug when the input is user-controlled. Random pivot or Introsort is not optional.
Post-mortem
War story Common failure mode · every ML data pipelinereproducibility bug
🔥 What broke
A team sorts feature rows by user_id. But two users hash to the same bucket after some transformation → unstable sort produces DIFFERENT ordering on different runs → downstream model training gets slightly different training set → metrics drift between reruns of ‘the same’ pipeline.
🧯 The fix
Always use STABLE sort when reproducibility matters. Python's sorted() is stable by default. In pandas, use df.sort_values(kind='mergesort') (or the newer kind='stable'). Numpy's default is quicksort (unstable) — you have to opt in with kind='mergesort'.
🎓 Lesson to steal
Stability is invisible until it's a bug. When outputs must be deterministic across runs (ML, checksums, exports), use stable sort explicitly.

Where this shows up in the rest of the plan

Sorting is a building block for half of what's coming
S032 · Binary Search
Requires sorted input. Sort + binary search is a common combo.
S033 · Dynamic Programming
Some DP problems require sorting first (LIS, activity selection).
S037 · SQL Joins
Sort-merge join sorts both tables before merging. Alternative to hash join.
S048 · Log Aggregation
Merging N sorted log streams by timestamp = external merge with a heap.
S055 · MapReduce / Spark
The ‘shuffle’ phase is external mergesort at petabyte scale.
S097 · A/B Testing Ranking
Ranking treatment/control by lift, with tied breaking by name → chained stable sorts.

(e) Recall + stretch · 10 min

Recall — click to reveal · click to reveal
★ = stretch question

Explain-out-loud test

Teach these three, one minute each, no notes:

  1. Explain mergesort to a friend using a deck of cards.
  2. When would you pick quicksort over mergesort — and vice versa?
  3. Why should you almost always just call sorted() instead of implementing anything?

What comes next

Hub: The 6-Month Learning Plan


Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.