Search Tech Journey

Find topics, journeys and posts

6-month learning plan42 / 130
back to blog
data engineeringintermediate 55m read

S042 · Transactions & ACID — Isolation Levels, MVCC

The single feature that lets banks, e-commerce, and every serious business run on a database. Learn ACID, the four isolation levels, MVCC, and why ‘SERIALIZABLE’ is not the answer you want.

🗃️DatabasesM04 · Databases & SQL· Session 042 of 130 90 min

🎯 Reason about concurrency anomalies (dirty read, non-repeatable read, phantom, write skew) and pick the right isolation level per query — not per database.

Why this session exists

Every real system has more than one user at a time. The moment two transactions touch the same row, physics wins and something races. ACID is the four-letter promise a database makes about what "correct" means under concurrency — and isolation levels are the dial you turn to trade correctness for throughput. Most engineers never learn where that dial actually sits by default (spoiler: not where you think), so they ship silent bugs that only show up under load. Ninety minutes here saves years of "why did we double-charge that customer?" post-mortems.

You will be able to
  • Spell out A, C, I, D and give a concrete example of a bug each one prevents.
  • Rank the four SQL isolation levels and name the anomaly each one still allows.
  • Explain MVCC in one sentence and draw why Postgres readers never block writers.
  • Pick the right isolation level for a bank transfer vs an analytics query vs a like-count increment.
  • Recognise write skew and know why SERIALIZABLE (not REPEATABLE READ) is the only thing that stops it.

Prerequisites

  • S040 — Joins & Set Operations (you need to be comfortable reading multi-table SQL).
  • S041 — Indexes — B-Tree Intuition (helps for understanding row locks vs range locks).


(a) Intuition · 5 min

Two chefs, one recipe card
🌍 Real world

Two chefs share a single recipe card that reads: "Flour = 500 g. Add 100 g. Write new total." Chef A reads 500, calculates 600, and is interrupted. Chef B reads the still-500 card, writes 600, and walks off. Chef A finally writes 600. You added 200 g of flour but the card says 600. That is a lost update, and every database in existence has a story about it.

ACID is the kitchen's operating discipline: "only one chef holds the card at a time" (isolation), "if the fire alarm goes off before you finish, nothing you wrote counts" (atomicity), "when you're done, the card can't say ‘650.7 g’ if the rule is integers" (consistency), and "even if the building burns down, the card survives" (durability).

💻 Code world

Translate that to a bank: two ATM withdrawals debit the same account of ₹10,000. Without isolation, both read balance = ₹10k, both write ₹9k after taking ₹1k each. The customer paid ₹2k and the bank recorded ₹1k. Multiply by a million transactions per day.

Transactions are the primitive that wraps "read, decide, write" into a unit the database can defend. Isolation levels are the language you use to tell the database how much defense you need, because more defense = less throughput.

ACID in one line each

The four promises
  • Atomicity — a transaction is all-or-nothing. Half a bank transfer never happens, even if the power dies mid-COMMIT.
  • Consistency — the database moves from one valid state to another. Constraints, foreign keys, and triggers hold at commit boundaries.
  • Isolation — concurrent transactions look like they ran one after another (to some agreed degree). This is the dial you tune.
  • Durability — once COMMIT returns, the data survives crashes, power loss, and the CFO tripping over the server cord.

A quick history so you know why the world looks like this

  1. 1976
    Jim Gray defines transactions
    IBM Research paper ‘Granularity of Locks in a Shared Data Base’ invents the transaction concept.
  2. 1983
    ‘ACID’ coined
    Härder & Reuter's paper ‘Principles of Transaction-Oriented Database Recovery’ gives the acronym its final form.
  3. 1992
    SQL-92 standardises 4 isolation levels
    READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ, SERIALIZABLE. Every vendor implements them differently.
  4. 1995
    Oracle ships MVCC
    Multi-Version Concurrency Control: readers see a snapshot; writers never block readers. Postgres follows in 1996.
  5. 2008
    Postgres adds SERIALIZABLE via SSI
    ‘Serializable Snapshot Isolation’ — the first practical serializable that doesn't kill throughput.
  6. 2012
    Google Spanner
    External consistency across continents using GPS + atomic clocks (TrueTime). ACID at planet scale is real.

(b) Visual walkthrough · 15 min

The four anomalies, ordered by nastiness

1RU only
Dirty read

You read a value another transaction wrote but hasn't committed yet. It rolls back — you acted on data that never existed.

2RC
Non-repeatable read

You read row X, another txn updates row X and commits, you read row X again — different value. Same query, same transaction, two answers.

3RR (SQL-92)
Phantom read

You SELECT WHERE status='pending', someone INSERTs a new pending row and commits, you re-SELECT — a new row appeared out of nowhere.

4up to SI
Write skew

Two transactions each read the same set, each check ‘I'm still safe to write’, and both commit — together they violate an invariant. The subtle one.

MVCC — the model that made Postgres eat the world

In one sentence: each transaction sees a snapshot of the database as it was when the transaction started. Writers never block readers. Readers never block writers. That single design decision is why Postgres and Oracle beat MySQL's older locking model for OLTP workloads for two decades.

Isolation level cheat-sheet (what Postgres actually does)

READ COMMITTED

Postgres default. Each statement gets a fresh snapshot.

  • Blocks dirty reads
  • Allows non-repeatable reads (each statement re-snapshots)
  • Fast, forgiving, right answer 80% of the time
  • Use for: web request handlers, most CRUD
REPEATABLE READ

Postgres = snapshot isolation. The whole txn sees one snapshot.

  • Blocks dirty + non-repeatable reads
  • Blocks phantoms (in Postgres, not SQL-92 REPEATABLE READ)
  • Allows write skew
  • Use for: reports, complex reads inside one transaction
SERIALIZABLE

SSI — real serial execution, statistically.

  • Blocks everything, including write skew
  • Postgres detects serialization conflicts at commit and aborts one txn (retry loop needed)
  • Costs 10-25 % throughput
  • Use for: money movement, inventory, constraints across rows

Common misconception
✗ What most people think

"My database is ACID compliant, so my transactions are fully isolated — concurrent transactions behave as if they ran one after another."

✓ What is actually true

Full serialisability is almost never the default. Postgres and most engines default to READ COMMITTED; MySQL InnoDB defaults to REPEATABLE READ. Both permit anomalies that serialisability forbids — non-repeatable reads, phantoms, and write skew. The "I" in ACID is a dial with several settings, and you are almost certainly not on the strongest one.

Why the myth is so sticky

Because ACID is marketed as a single guarantee and the letter I is stated without qualification, so isolation sounds binary. It is not: the SQL standard defines four levels precisely because full isolation is expensive, and vendors default to a weaker level so that ordinary workloads are fast. The anomalies are rare under light load, which means they surface first in production under concurrency — the least convenient possible time.

Prove it to yourself

Check your actual level, and reproduce write skew with two concurrent sessions:

SHOW transaction_isolation;   -- Postgres: 'read committed' by default

-- Write skew: rule is 'at least one doctor must stay on call'.
-- Two doctors go off call simultaneously; each check passes, rule is violated.
-- Session A:                     Session B:
BEGIN;                         -- BEGIN;
SELECT COUNT(*) FROM oncall     -- SELECT COUNT(*) FROM oncall
  WHERE on_call = true;  -- 2   --   WHERE on_call = true;  -- 2
UPDATE oncall SET on_call=false -- UPDATE oncall SET on_call=false
  WHERE name = 'alice';         --   WHERE name = 'bob';
COMMIT;                        -- COMMIT;
-- Result: zero doctors on call. Both transactions committed. No error.
-- SERIALIZABLE would abort one of them; REPEATABLE READ does not.
From first principles
Start with the question

Why does durability require a write-ahead log, when the database could simply write the changed data pages to disk before acknowledging the commit?

  1. 1
    A transaction typically modifies a few rows scattered across many different pages of the table and its indexes.
    forced by · rows are placed by key or by insertion order, not by which transaction touches them
  2. 2
    Writing those pages directly means many random writes, scattered across the storage device, before the commit can return.
    forced by · each page lives at its own location determined by the table's layout
  3. 3
    Random writes are far slower than sequential ones, and worse, a multi-page write is not atomic — a crash midway leaves some pages updated and some not, with no record of which.
    forced by · the device guarantees atomicity only at roughly the page level, not across a set of pages
  4. 4
    So instead, append a compact record of what changed to a single sequential log file, and fsync only that. One sequential append, one flush.
    forced by · sequential appends are the fastest write pattern available, and a single file gives one clear ordering
  5. 5
    Once the log record is durably on disk, the change is recoverable — the actual data pages can be written lazily in the background, and a crash is repaired by replaying the log.
    forced by · the log is the authoritative record of intent; the data pages are a materialised cache of it
⇒ Therefore

Therefore the WAL converts many random synchronous writes into one sequential synchronous write, and simultaneously provides the atomicity that page writes cannot. Durability and atomicity fall out of the same mechanism.

And note what this predicts: commit latency is bounded by one fsync, which is why grouping many small transactions into one is such a large throughput win, and why synchronous_commit = off is so tempting and so dangerous. It also predicts the architecture of nearly everything downstream — Kafka's partitions, LSM-tree storage engines, Delta and Iceberg transaction logs, and database replication are all the same insight: the log is the database, and the tables are a view over it.

Mental modelThe log is the truth

Do not picture the database as tables that get edited in place. Picture an append-only log of intentions — "set x to 5", "delete row 9" — that is written durably and in order, plus a set of data pages that are merely a fast materialised view of replaying that log.

Commit means "my records are durably in the log". Rollback means "my records are marked abandoned and will never be applied". Recovery means "replay the log from the last checkpoint". Every ACID property is a consequence of that one structure.

  • Atomicity and durability come from the log. Isolation comes from concurrency control — locking or MVCC — and is a separate, tunable mechanism.
  • Under MVCC, readers see a consistent snapshot and never block writers. This is why "reads don't block writes" is true in Postgres and why old row versions accumulate until vacuumed.
  • The four levels in ascending strictness: READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ, SERIALIZABLE — each eliminating one more anomaly class at more cost.
  • Consistency (the C) is the odd one out: it is not enforced by the engine beyond declared constraints. It is your invariants, and it is only as strong as the constraints you actually declared.
🔔 Fires when you see

Fire this model the moment you see: a check-then-act pattern (read a balance, then update it) · "this worked in test but corrupts under load" · a uniqueness rule enforced in application code instead of by a constraint · a deadlock in the logs · a long-running transaction blocking vacuum · an idempotency requirement in a pipeline · exactly-once semantics in a streaming system.

The tradeoff

A workflow must read a value, compute from it, and write it back, with correctness under concurrency. Pessimistic locking, optimistic concurrency, or SERIALIZABLE isolation?

Pessimistic — SELECT ... FOR UPDATE
+ you gain the row is locked for the duration, so the update is guaranteed to see and win; no retry logic needed, and behaviour is easy to reason about
− you pay concurrent transactions block, throughput collapses under contention on hot rows, and lock ordering mistakes across multiple rows cause deadlocks
pick when contention is genuinely high on specific rows and the critical section is short — inventory decrement, seat booking, account balance
Optimistic — version column with a compare-and-set
+ you gain no locks held, so readers and non-conflicting writers run at full speed; conflicts are detected at write time and are cheap when rare
− you pay you must implement retry, make the operation idempotent, and handle the pathological case where a hot row causes repeated retries and effectively livelocks
pick when conflicts are rare — many concurrent transactions touching mostly different rows, the common shape in web and pipeline workloads
SERIALIZABLE isolation
+ you gain the engine guarantees the outcome equals some serial order, eliminating write skew and phantoms without you having to identify the anomaly yourself
− you pay meaningful overhead from predicate/SSI tracking, and transactions are aborted with serialisation failures, so you still need retry logic — you just get it enforced rather than designed
pick when the invariant spans multiple rows or rows that do not exist yet (write skew, phantoms), where row locks cannot express the constraint
What a senior engineer actually does

Match the mechanism to the invariant. If the rule is about one row, a row lock or a version column expresses it exactly. If the rule is about a set of rows — "at least one must remain", "no overlapping bookings" — then no row-level mechanism can enforce it, and you need SERIALIZABLE or an explicit constraint like an exclusion constraint. That distinction is what write skew is teaching you.

And prefer declared constraints over transactional cleverness wherever possible. A unique index enforces uniqueness under every isolation level, at every concurrency, forever, with no retry logic — it is strictly stronger than any application-level check-then-insert, and it is one line of DDL.


(c) Hands-on · 25 min

Two psql sessions demonstrating each anomaly with your own eyes. Save as acid_demo.sh, run against a local Postgres (docker run --rm -p 5432:5432 -e POSTGRES_PASSWORD=demo postgres:16).

#!/usr/bin/env bash
# acid_demo.sh — reproduce dirty-read (can't in Postgres), non-repeatable read,
# phantom, and write skew across two psql sessions.
set -euo pipefail
 
export PGPASSWORD=demo
PSQL="psql -h localhost -U postgres -v ON_ERROR_STOP=1"
 
$PSQL <<'SQL'
DROP TABLE IF EXISTS accounts;
CREATE TABLE accounts (id INT PRIMARY KEY, owner TEXT, balance INT);
INSERT INTO accounts VALUES (1,'alice',1000),(2,'bob',1000);
 
DROP TABLE IF EXISTS on_call;
CREATE TABLE on_call (doctor TEXT PRIMARY KEY, is_on_shift BOOLEAN);
INSERT INTO on_call VALUES ('alice',TRUE),('bob',TRUE);
SQL
 
echo "--- Demo 1: non-repeatable read at READ COMMITTED ---"
(
  $PSQL <<'SQL' &
BEGIN ISOLATION LEVEL READ COMMITTED;
SELECT balance FROM accounts WHERE id=1;   -- 1000
SELECT pg_sleep(3);
SELECT balance FROM accounts WHERE id=1;   -- ??? see what happens
COMMIT;
SQL
  sleep 1
  $PSQL -c "UPDATE accounts SET balance=1500 WHERE id=1;"
  wait
)
 
echo "--- Demo 2: phantom read at READ COMMITTED ---"
(
  $PSQL <<'SQL' &
BEGIN ISOLATION LEVEL READ COMMITTED;
SELECT count(*) FROM accounts WHERE balance >= 1000;
SELECT pg_sleep(3);
SELECT count(*) FROM accounts WHERE balance >= 1000;
COMMIT;
SQL
  sleep 1
  $PSQL -c "INSERT INTO accounts VALUES (3,'carol',5000);"
  wait
)
 
echo "--- Demo 3: write skew (only SERIALIZABLE catches this) ---"
# Rule: at least one doctor MUST be on shift.
# Two txns each read the set, each see 2 on shift, each take themselves off.
# Result: zero doctors on shift. That's write skew.
(
  $PSQL <<'SQL' &
BEGIN ISOLATION LEVEL REPEATABLE READ;   -- try both RR and SERIALIZABLE
SELECT count(*) FROM on_call WHERE is_on_shift;   -- 2
SELECT pg_sleep(2);
UPDATE on_call SET is_on_shift=FALSE WHERE doctor='alice';
COMMIT;
SQL
  $PSQL <<'SQL' &
BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT count(*) FROM on_call WHERE is_on_shift;   -- 2
SELECT pg_sleep(2);
UPDATE on_call SET is_on_shift=FALSE WHERE doctor='bob';
COMMIT;
SQL
  wait
)
 
echo "--- Final state (should be at least one TRUE; watch it be all FALSE) ---"
$PSQL -c "SELECT * FROM on_call;"

What each block does

Anatomy of the script

BEGIN ISOLATION LEVEL …
Sets the level for THIS transaction only. You can mix levels in one app — pick per operation, not per database.
sql
pg_sleep(3)
Deliberate pause so the second session's write lands in the window. In real code, replace with the natural latency of the request.
demo
Demo 1 · non-repeatable read
At READ COMMITTED, each statement gets a fresh snapshot. Second SELECT sees 1500. Bump to REPEATABLE READ and it stays 1000.
anomaly
Demo 2 · phantom
New row satisfies the WHERE clause. In Postgres, REPEATABLE READ actually blocks this (unlike SQL-92 spec). In MySQL InnoDB you need gap locks.
anomaly
Demo 3 · write skew
Both txns pass the check, both commit, invariant broken. REPEATABLE READ allows this. Change to SERIALIZABLE and one txn aborts with SQLSTATE 40001.
critical
SQLSTATE 40001 retry loop
Serializable failures are expected — your app MUST catch them and retry. This is the tax you pay for correctness.
app-code
Try itFeel the throughput cost of SERIALIZABLE

Install pgbench (ships with Postgres). Run the standard TPC-B benchmark at two isolation levels and diff the TPS:

pgbench -i -s 10 -h localhost -U postgres
pgbench -c 16 -j 4 -T 30 -h localhost -U postgres            # default RC
pgbench -c 16 -j 4 -T 30 -h localhost -U postgres -M prepared --define="isolation_level=serializable"

You'll typically see a 10-25 % TPS drop at SERIALIZABLE plus some retry churn. That's the real cost — measure it before you flip the switch on a hot table.

💡 Hint · Use `pgbench -c 32 -j 4 -T 30 -M prepared --custom-scripts` twice — once at read committed, once at serializable. The ratio surprises people.

(d) Production reality · 15 min

War story Kyle Kingsbury · Jepsen· 2019dozens of databases tested
🔥 What broke

Jepsen ran isolation tests against MongoDB, YugabyteDB, FaunaDB, CockroachDB, and others. Almost every one claimed "serializable" and almost every one exhibited anomalies under network partitions or clock skew.

The most infamous: MongoDB advertised "causal consistency" but showed dirty reads under specific replica-set failover conditions until v4.2.

🧯 The fix

Databases patched. Jepsen wrote 40+ public reports. The industry lesson: never trust an isolation level claim without a Jepsen test. Even ACID-claiming databases lie under load.

🎓 Lesson to steal
Read the Jepsen report of any distributed DB before you put money on it. Free education: jepsen.io/analyses.
Post-mortem
War story A large European bank · 2015reported by Peter Bailis in ‘Feral Concurrency Control’ paper
🔥 What broke

An account-transfer service ran at READ COMMITTED and used the classic pattern: SELECT balance, check >= amount in Java, UPDATE balance -= amount. Two concurrent withdrawals both saw balance = ₹10k, both passed the check, both committed. Customer overdrew by ₹9k and the bank ate the loss.

🧯 The fix

Two options: (a) wrap in SERIALIZABLE + retry loop, or (b) rewrite as a single atomic update: UPDATE accounts SET balance = balance - 1000 WHERE id = ? AND balance >= 1000 and check rowcount == 1. Both work; the atomic-UPDATE version is faster.

🎓 Lesson to steal
"Read-modify-write" outside the database is a bug. Push the check into the WHERE clause or use SERIALIZABLE. Never do a Java-side if.
War story GitHub · 2018· 201824-hour incident on GitHub.com
🔥 What broke

A 43-second MySQL replica partition triggered Orchestrator failover. When the split healed, both regions had accepted writes to overlapping rows. Reconciling the divergent MVCC histories took the site into degraded mode for 24 hours.

🧯 The fix

GitHub rewrote failover to require a strict quorum (block writes to any region that loses quorum) and moved key services to a single-primary topology. Post-mortem is legendary reading.

🎓 Lesson to steal
ACID guarantees are per-primary. Multi-primary + partition = you WILL diverge. Either accept degraded writes or ship a real consensus protocol.
Post-mortem

Where this shows up in the rest of the plan

Concurrency correctness lives here — everything above builds on it
S043 · Query planning
EXPLAIN reveals whether your isolation level forced a table lock or a range lock.
S044 · NoSQL landscape
Most NoSQL stores relax I in ACID. Now you know what you're giving up.
S049 · Stream processing
Exactly-once semantics is a distributed cousin of SERIALIZABLE. Same problem, different plane.
S078 · Consistency & consensus
Paxos and Raft are the distributed generalisation of the single-node WAL.
S086 · Sagas & compensating actions
When one txn can't span services, you fake atomicity with compensating writes.
S120 · System design — payments
Idempotency keys + SERIALIZABLE + retry loop = the actual answer to ‘don't double-charge’.

(e) Recall + stretch · 10 min

Recall — click each to reveal · click to reveal
★ = stretch question

Explain-out-loud test

  1. ACID in 30 seconds — the four letters, one example each.
  2. Write skew in one example — the on-call doctors story is fine.
  3. Why READ COMMITTED is the right default — one throughput argument, one correctness workaround.

What comes next

Hub: The 6-Month Learning Plan


Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.