S035 · The Relational Model — Tables, Keys, Normalisation
Codd's 55-year-old idea that still runs the world: split entities into tables, link by keys, store every fact exactly once. Learn primary vs foreign keys, the first three normal forms with worked examples, when to deliberately denormalise for OLAP, and the specific data bugs (update anomalies, natural-key collisions, missing FK integrity) that this discipline prevents.
🎯 Design a normalised schema for a real-world domain, enforce integrity with primary and foreign keys, and know when to break the rules for OLAP.
Why this session exists
Every OLTP database in the world — Postgres, MySQL, SQL Server, Oracle, SQLite on your phone — runs on the relational model Ted Codd published in 1970. Even so-called NoSQL stores reinvent primary keys, secondary indexes, and normalisation trade-offs under new names. The single largest source of data bugs in a career is the same duplicated fact stored in two places, which then disagree. This session gives you the discipline to prevent that class of bug for the next decade.
- Define primary key, foreign key, and referential integrity in one sentence each.
- Take a denormalised spreadsheet-style table and normalise it to 3NF, step by step.
- Explain when to deliberately denormalise (OLAP star schemas) and the trade-offs.
- Choose between surrogate (bigint / UUID) and natural (email, ISBN) keys with a rationale.
- Enforce integrity with NOT NULL, UNIQUE, CHECK, and REFERENCES in a CREATE TABLE statement.
Prerequisites
- S008 — Data Structures: list, tuple, dict, set — a table is a list of dicts.
- S023 — Arrays & Lists — the row-store representation.
(a) Intuition · 5 min
An online store keeps every purchase in one giant spreadsheet: customer_name, customer_email, customer_city, product_name, product_price, qty, order_date. If a customer updates their email, you must change it in every row where they appear. Miss one → some orders now belong to a ghost. If a product price changes, historical orders lose their true price. The sheet is fighting you.
The relational model fixes this by storing each real-world entity in its own table and linking them by IDs. Customers live in customers with a unique customer_id; products in products; orders reference both by ID. Change an email? One row. Add a new product column? One place. The mathematical guarantee: the same fact isn't stored twice, so it cannot disagree with itself.
- Primary key — a column that uniquely identifies each row (a passport number for the row).
- Foreign key — a column pointing to another table's primary key (a reference the database enforces).
- Normalisation — the discipline of splitting tables until no non-key fact is duplicated. First 3 normal forms cover 95 % of OLTP work.
- 1970Codd's paperTed Codd (IBM Research) publishes ‘A Relational Model of Data for Large Shared Data Banks’. Turns commercial databases upside down.
- 1974System R prototypeIBM builds the first relational engine. Its query language ‘SEQUEL’ becomes SQL.
- 1979Oracle v2 shipsLarry Ellison ships the first commercial relational DB — before IBM. Never version 1 (they thought nobody would trust it).
- 1996MySQL & Postgres go mainstreamFree relational DBs power the dot-com boom. Codd's model wins.
- 2010NoSQL wave + reversalWeb-scale shops drop SQL for MongoDB, Cassandra. By 2020 many quietly re-add relational features (secondary indexes, joins, transactions).
(b) Visual walkthrough · 15 min
The denormalised table (bad)
| order_id | customer_name | customer_email | product_name | price | qty | order_date |
|---|---|---|---|---|---|---|
| 1 | Alice | a@x.com | T-shirt | 500 | 2 | 2026-01-15 |
| 2 | Alice | a@x.com | Mug | 250 | 1 | 2026-01-20 |
| 3 | Bob | b@x.com | T-shirt | 500 | 1 | 2026-01-22 |
| 4 | Alice | alice@new.com | Cap | 300 | 1 | 2026-02-05 |
Row 4 shows Alice's email was updated — but rows 1 & 2 still hold the old one. This is called an update anomaly. Delete Alice's last order and you also lose her contact info: a deletion anomaly. Add a new customer with no orders and you can't record them: an insertion anomaly.
The normalised (3NF) design
Now Alice's email lives in one row. All her past orders reflect the new email when joined. Notice unit_price_cents is copied onto each order line — that's an intentional exception (price snapshots must not change when the product's current price does).
The normal forms — memorise the first three, cite the rest
Atomic values
- Each cell holds one value, not a list
- No CSV in a column
- No repeating groups (`phone1`, `phone2`, `phone3`)
- Fix: pull repeating groups into a child table
Full dependency on the whole key
- Applies only when PK is composite
- Every non-key column depends on the whole PK, not part of it
- Example: `(order_id, line_no) → product_price` OK; `(order_id, line_no) → customer_name` NOT OK
- Fix: split partial dependencies into a parent table
No transitive dependencies
- No non-key column depends on another non-key column
- Example: `zipcode → city → state` violates 3NF
- Fix: extract into a `zipcodes(zip PK, city, state)` lookup
- This is the target for 95 % of OLTP schemas
Key taxonomy — the choices that matter
The key decisions you'll make on every table
Worked normalisation — courses & instructors
Start: courses(course_code PK, course_title, instructor_id, instructor_name, instructor_dept).
All cells atomic — yes.
Single-column PK — trivially 2NF.
`instructor_name` and `instructor_dept` depend on `instructor_id`, not on `course_code`. Transitive dependency — 3NF violated.
Split into `courses(course_code PK, course_title, instructor_id FK)` and `instructors(instructor_id PK, name, dept)`. All non-key columns now depend only on the local PK.
The great trade-off — OLTP vs OLAP
Postgres · MySQL · your app DB
- Reads join a few rows across a handful of tables
- Writes touch one fact in one place
- Referential integrity enforced by the DB
- Storage cheap, correctness paramount
- Star schemas here would create update nightmares
Snowflake · BigQuery · Databricks
- Reads scan billions of rows; joins get expensive
- Writes are append-only batch loads (ETL)
- Star / snowflake / One Big Table designs win
- Duplication is acceptable because data is derived
- Facts live in ‘fact tables’, context in ‘dimension tables’
"A relation is a table. Rows, columns, a name — same thing, different vocabulary."
A relation is a set of tuples: unordered, duplicate-free, with attributes identified by name rather than position. A SQL table is a bag: it permits duplicates, exposes column order, and allows NULLs. SQL deliberately broke the relational model in those three places, and nearly every confusing SQL behaviour traces back to one of them.
Because the leak is invisible until it bites. As long as you have a primary key and no NULLs, a table behaves exactly like a relation and the distinction seems academic. Then a join fans out and produces duplicates, or a NOT IN with a NULL returns zero rows, or UNION silently deduplicates while UNION ALL does not — and each of those is precisely a place where the bag/set gap shows through.
Three one-liners, each showing a place SQL is not relational:
-- 1. bag, not set: duplicates are legal and UNION vs UNION ALL differ
SELECT 1 UNION SELECT 1; -- 1 row (set semantics)
SELECT 1 UNION ALL SELECT 1; -- 2 rows (bag semantics)
-- 2. NULL is not a value: three-valued logic
SELECT NULL = NULL; -- NULL, not TRUE
SELECT 1 WHERE 1 NOT IN (2, NULL); -- 0 rows, not 1 row
-- 3. column order is part of the table, not of a relation
SELECT * FROM t; -- order is positional and schema-dependentWhy does normalisation eliminate update anomalies? "Reduce redundancy" is the slogan, but why does redundancy cause anomalies specifically, rather than just wasting disk?
- 1A functional dependency X → Y says: knowing X determines exactly one Y. That is a statement about the real world the data models.forced by · facts have structure — a department has one location, an order has one customer
- 2If X → Y holds and X is not a key of the table, then X repeats across many rows, and so Y is physically stored many times.forced by · a non-key value can appear in many rows, and each row carries its own copy of Y
- 3Now the same fact exists in n places, but the database only enforces constraints it knows about — and it has no idea those n copies must agree.forced by · the DBMS enforces keys and declared constraints, not undeclared functional dependencies
- 4Therefore any update that does not touch all n copies leaves the database in a state that is internally contradictory yet perfectly legal.forced by · consistency of the copies is maintained by application discipline, and discipline is not a constraint
- 5Decomposing so that every determinant is a key puts each fact in exactly one row, and the key constraint then makes duplication structurally impossible.forced by · uniqueness of the key is enforced by the engine, converting a convention into an invariant
Therefore normalisation is not about saving space — it is about moving correctness from application code into engine-enforced constraints. BCNF is exactly the statement "every determinant is a key", which is exactly "no fact is stored twice".
And note what this predicts: the anomaly risk is proportional to the number of writers, not the number of rows. A denormalised table written by exactly one deterministic pipeline has no update anomaly, because there is no path to inconsistent partial updates. That is the entire justification for denormalised star schemas in analytics — and it also tells you the precise condition under which that justification evaporates.
Picture the database as a filing system where every fact is written on exactly one card. To change a fact, you find one card and edit it. If a fact appears on three cards, changing it is a three-step transaction that you might get two-thirds through — and the filing system will not stop you.
Normalisation is the process of splitting the cards until each holds exactly one fact keyed by one thing. Joins are how you reassemble the cards on demand. The whole tradeoff of relational design is: pay at write time (keep cards separate, join on read) or pay at read time (pre-join the cards, own the duplication).
- A key is a promise the engine enforces. Everything you do not declare as a key or constraint is a promise only your code makes.
- NULL means "unknown", not "empty". Any comparison to unknown is unknown, which is why
NOT INwith NULLs returns nothing andNOT EXISTSdoes not. - Row order is never guaranteed without
ORDER BY. Any observed ordering is an artefact of the current plan and will change. - Normalise until it hurts, denormalise until it works — but denormalise only where you control every writer.
Fire this model the moment you see: the same value stored in two tables · a "status" column updated by more than one job · a join that unexpectedly multiplies row counts · a NOT IN returning zero rows · a report that disagrees with another report · a schema with no declared foreign keys.
Design the schema normalised and join on read, or denormalised and duplicate on write?
The decision is really about who writes. Normalisation buys protection from uncoordinated writers; if there is exactly one writer and it is a deterministic, replayable pipeline, you are buying insurance against a risk you do not have. That is why the same engineer correctly normalises the OLTP source and correctly denormalises the warehouse — the workloads differ in writer count, not in taste.
What is not negotiable either way is declaring your keys. Even in a denormalised warehouse, knowing the grain of a table — the exact set of columns that makes a row unique — is what lets you reason about whether a join will fan out. Most "the numbers are wrong" incidents are a grain misunderstanding, and grain is a relational concept that survives every denormalisation.
(c) Hands-on · 25 min
You can run this against SQLite (built into Python — no install needed). Save as rel_lab.py, run python3 rel_lab.py.
"""rel_lab.py — feel the relational model in your fingers."""
from __future__ import annotations
import sqlite3
conn = sqlite3.connect(":memory:")
conn.execute("PRAGMA foreign_keys = ON") # SQLite quirk — off by default
cur = conn.cursor()
# ---------- 1. Define a normalised schema with real constraints ----------
schema = """
CREATE TABLE customers (
customer_id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
email TEXT NOT NULL UNIQUE
);
CREATE TABLE products (
product_id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
price_cents INTEGER NOT NULL CHECK (price_cents > 0)
);
CREATE TABLE orders (
order_id INTEGER PRIMARY KEY,
customer_id INTEGER NOT NULL REFERENCES customers(customer_id),
product_id INTEGER NOT NULL REFERENCES products(product_id),
qty INTEGER NOT NULL CHECK (qty > 0),
unit_price_cents INTEGER NOT NULL, -- snapshot at order time
order_date TEXT NOT NULL
);
CREATE INDEX idx_orders_customer ON orders(customer_id);
CREATE INDEX idx_orders_product ON orders(product_id);
"""
cur.executescript(schema)
# ---------- 2. Seed data ----------
cur.executemany("INSERT INTO customers VALUES (?,?,?)", [
(1, "Alice", "a@x.com"),
(2, "Bob", "b@x.com"),
])
cur.executemany("INSERT INTO products VALUES (?,?,?)", [
(10, "T-shirt", 50000),
(20, "Mug", 25000),
(30, "Cap", 30000),
])
cur.executemany("INSERT INTO orders VALUES (?,?,?,?,?,?)", [
(1, 1, 10, 2, 50000, "2026-01-15"),
(2, 1, 20, 1, 25000, "2026-01-20"),
(3, 2, 10, 1, 50000, "2026-01-22"),
(4, 1, 30, 1, 30000, "2026-02-05"),
])
# ---------- 3. Show Alice's orders — email lives in ONE place ----------
print("Alice's orders (single-source email):")
for row in cur.execute("""
SELECT o.order_id, c.email, p.name, o.qty, o.unit_price_cents, o.order_date
FROM orders o
JOIN customers c ON o.customer_id = c.customer_id
JOIN products p ON o.product_id = p.product_id
WHERE c.name = 'Alice'
ORDER BY o.order_id;
"""):
print(" ", row)
# ---------- 4. Update Alice's email ONE row; all joins reflect it ----------
cur.execute("UPDATE customers SET email = 'alice@new.com' WHERE customer_id = 1")
# ---------- 5. Change product price; historical orders KEEP their snapshot ----------
cur.execute("UPDATE products SET price_cents = 60000 WHERE product_id = 10")
print("\nAfter email + price change:")
for row in cur.execute("""
SELECT o.order_id, c.email, p.name AS product,
p.price_cents AS current_price,
o.unit_price_cents AS paid_price
FROM orders o
JOIN customers c ON o.customer_id = c.customer_id
JOIN products p ON o.product_id = p.product_id
WHERE c.name = 'Alice'
ORDER BY o.order_id;
"""):
print(" ", row)
# ---------- 6. Referential integrity: try to insert a bad order ----------
print("\nAttempt: insert order for non-existent customer_id=999")
try:
cur.execute("INSERT INTO orders VALUES (99, 999, 10, 1, 50000, '2026-03-01')")
except sqlite3.IntegrityError as e:
print(" ✓ blocked by FK:", e)
# ---------- 7. UNIQUE constraint: duplicate email rejected ----------
print("\nAttempt: insert duplicate email")
try:
cur.execute("INSERT INTO customers VALUES (3, 'Carol', 'alice@new.com')")
except sqlite3.IntegrityError as e:
print(" ✓ blocked by UNIQUE:", e)
# ---------- 8. CHECK constraint: negative qty rejected ----------
print("\nAttempt: insert order with qty = -5")
try:
cur.execute("INSERT INTO orders VALUES (100, 1, 10, -5, 50000, '2026-03-01')")
except sqlite3.IntegrityError as e:
print(" ✓ blocked by CHECK:", e)
# ---------- 9. Aggregate — revenue per customer ----------
print("\nRevenue per customer:")
for row in cur.execute("""
SELECT c.name,
SUM(o.qty * o.unit_price_cents) / 100.0 AS revenue_rupees
FROM customers c
LEFT JOIN orders o ON c.customer_id = o.customer_id
GROUP BY c.customer_id
ORDER BY revenue_rupees DESC;
"""):
print(" ", row)
conn.close()Anatomy of the script
Extend the script with:
CREATE TABLE categories (
category_id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE
);
ALTER TABLE products ADD COLUMN category_id INTEGER REFERENCES categories(category_id);Insert three categories (Apparel, Drinkware, Accessories), tag your three products, then write:
SELECT cat.name AS category,
SUM(o.qty * o.unit_price_cents) / 100.0 AS revenue_rupees
FROM orders o
JOIN products p ON o.product_id = p.product_id
JOIN categories cat ON p.category_id = cat.category_id
GROUP BY cat.category_id
ORDER BY revenue_rupees DESC;The whole query is 5 lines because the schema is right. Every extra dimension you add (city, month, campaign) is the same shape — add a lookup table, add an FK, join, group.
(d) Production reality · 15 min
The team skipped foreign key constraints during MVP ‘for performance’ (a saving of ~5 microseconds per insert). Two years later, an audit found ~4 % of payments.user_id values pointed at users that had been deleted or never fully created (race condition on user signup).
Emergency: nightly reconciliation job to flag orphan payments. Long term: back-fill missing users where possible, hard-delete orphan payments, ADD the FK constraint, add a not-null trigger on the user creation path.
The original schema used patient_email as the primary key of the patients table. Twins with a shared parent email → PK collision at admission. Later, a patient changed their email at a routine visit — 12 years of medical history silently disconnected from the new record.
Migrate to patient_id BIGSERIAL PRIMARY KEY. All 40 tables that referenced patient_email got a new FK column, old constraints were dropped and re-created, joins were rewritten. Six months of engineering. Meanwhile, keep email as UNIQUE (not PK), so it can be changed without breaking references.
Analysts were writing 20-way JOIN queries across a fully-3NF warehouse to answer basic revenue questions. Queries took hours; costs were high; different analysts computed different numbers because they joined tables differently.
Adopt a Data Modelling discipline: keep the operational stores in 3NF (fast writes) and materialise into denormalised fact + dimension tables in the warehouse. One canonical revenue fact table per business event, one dimension table per entity. Now every analyst joins the same shape and gets the same numbers.
Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
- What is the relational model? (one sentence)
- What is a primary key vs a foreign key? (both in one breath)
- Why do OLTP systems normalise and OLAP systems denormalise? (one sentence each)
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.