S045 · Data Modelling — Dimensional, Data Vault, OBT
Kimball vs Inmon vs One Big Table. The three modelling schools that dominate 30 years of data warehouses — when each wins, when each fails, and what modern cloud warehouses changed.
🎯 Design a small star schema from scratch, explain when to reach for Data Vault, and defend OBT (One Big Table) as a legitimate choice in a cloud warehouse.
Why this session exists
If you don't pick a modelling paradigm, one picks you — usually the worst one, incrementally, as ad-hoc tables pile up until the warehouse is unusable. Every serious analytics team eventually converges on dimensional modelling (Kimball), Data Vault, or One Big Table (OBT). Each has strong opinions about denormalisation, change tracking, and query economics. This session teaches you the three languages so you can read anyone's dbt project cold and know which world you're in.
- Draw a star schema and name its parts (fact, dimension, surrogate key, grain).
- Explain SCD types 1/2/3/6 and pick the right one for a real column.
- Describe Data Vault's hub/link/satellite pattern in one paragraph and say when it beats Kimball.
- Argue for OBT in Snowflake/BigQuery/Databricks without being laughed at by a Kimball veteran.
- Spot the classic modelling mistakes: multi-grain facts, snowflaked dimensions where a flat one wins, no surrogate keys.
Prerequisites
- S040 — Joins & Set Operations (star = many joins; you must be fluent).
- S043 — Query Planning (understanding why cloud engines love flat joins).
(a) Intuition · 5 min
You run a small retail business with 100 000 receipts. Three archivists come to organise them.
Kimball creates one big "sales" folder (facts) plus small side folders for customers, products, stores, dates (dimensions). Every receipt in the sales folder has short pointers to the side folders. To answer "how much did we sell of blue widgets last Tuesday in Hyderabad?" you open sales, look up widget→blue in products, look up Tuesday in dates, look up Hyderabad in stores, count.
Inmon / Data Vault keeps every raw fact ever ingested and every relationship between facts, timestamped. Nothing is deleted, nothing is overwritten. To answer the same question, you re-run history for that day. Slower but perfectly auditable — banks and regulators love this.
OBT jams customer, product, store, date, and receipt all into one giant table with 200 columns. Rows are billions long. To answer the question, you scan the giant table with predicates. Sounds crazy? Snowflake and BigQuery are architected exactly to make this cheap.
In SQL, Kimball = SELECT … FROM fact_sales f JOIN dim_customer c JOIN dim_product p … — six joins in a query is normal, and the query planner loves it because the dims are small and cached.
OBT = SELECT … FROM fact_sales_wide WHERE product_color='blue' AND day_of_week='Tue' … — no joins, one scan. Columnar storage + predicate pushdown makes this competitive with the joins version. Storage cost is higher; query complexity for analysts is lower.
The four ideas that unlock everything
- Grain — the exact level of detail one fact row represents (‘one row per order line’ vs ‘one row per order’). Never mix grains in one fact.
- Dimension — the ‘by what?’ answer. Customer, product, date, store. Small, wide, denormalised.
- Fact — the ‘how much?’ answer. Big, narrow, additive when possible.
- Slowly Changing Dimension (SCD) — how you record that a customer moved from Hyderabad to Bangalore last Tuesday. Type 1/2/3/6 = four ways to handle this.
A quick history so you know why the world looks like this
- 1992Inmon's ‘Building the Data Warehouse’Top-down: build an enterprise-wide normalised warehouse first, then data marts from it.
- 1996Kimball's ‘The Data Warehouse Toolkit’Bottom-up: build dimensional data marts per business process, conform dimensions across them.
- 2000SCD Types formalisedKimball's 40 techniques crystallise. Every book after references them.
- 2013Data Vault 2.0 · Dan LinstedtFormalised for regulated industries where auditability > query speed.
- 2016BigQuery + Snowflake go mainstreamColumnar cloud warehouses make wide, denormalised tables cheap to scan.
- 2020OBT and ‘activity schema’ risedbt + Snowflake + Fivetran make ‘just denormalise everything’ a legitimate strategy.
(b) Visual walkthrough · 15 min
A canonical star schema
The three modelling schools side by side
Star / snowflake schema; query-first.
- One fact per business process (sales, shipments, refunds…)
- Conformed dimensions shared across facts
- SCD tracks change over time
- Optimised for BI tools + fast ad-hoc analysis
Top-down normalised EDW + data marts.
- 3NF enterprise-wide model first
- Data marts derived from EDW
- High upfront investment, strong lineage
- Rare in modern startups; common in F500
Insert-only hubs / links / satellites.
- Hub = business key (customer_id)
- Link = relationship (customer × order)
- Satellite = descriptive attrs + effective dates
- Perfect audit trail; heavy join count on read
Flatten everything into one wide table.
- Zero joins on read
- Loves columnar storage (BigQuery, Snowflake)
- Higher storage, easier analyst experience
- Bad fit for OLTP; great for BI + ML feature tables
Slowly Changing Dimensions — the four types you actually see
SCD types (Ralph Kimball's numbering)
"Star schemas are a legacy technique from when storage was expensive and joins were slow. With modern columnar engines you can just query the normalised source directly."
Dimensional modelling was never primarily about performance. It is about semantics: defining an agreed grain, a single conformed definition of "customer" and "date" across every fact, and a place to store history so that last year's report still reproduces last year's numbers. No amount of compute makes those problems go away, because they are organisational, not computational.
Because the performance argument was the one written down loudest in the 1990s, and it was true then, so when the performance constraint relaxed it looked like the whole rationale had evaporated. What remains is the harder half: two teams querying the same normalised source will define "active customer" differently, both will be defensible, and both numbers will reach a leadership deck. A conformed dimension is the mechanism that prevents that, and it has no compute equivalent.
Slowly changing dimensions are the clearest case — the source table simply cannot answer the question:
-- Source system: customer.region is overwritten in place.
-- A customer moves from 'South' to 'North' in June.
-- Question: what were January's sales by region?
-- Against the live source, joining on customer_id:
-- January's sales are now attributed to 'North'.
-- Last month's report no longer reproduces. Nothing errored.
-- SCD Type 2 dimension keeps history:
-- dim_customer(customer_sk, customer_id, region, valid_from, valid_to, is_current)
-- (101, 55, 'South', '2023-01-01', '2024-06-01', false)
-- (102, 55, 'North', '2024-06-01', '9999-12-31', true)
-- The fact stores customer_sk (the surrogate key), captured at event time,
-- so January's rows point at row 101 forever. History is now immutable.Why must a fact table have a single, explicitly declared grain — and why is mixing grains the most damaging error in dimensional modelling?
- 1Grain is the precise definition of what one row represents: "one order line, per order, per product".forced by · a fact table is a set of measurements, and a measurement is meaningless without knowing what it measures
- 2Aggregating a measure means summing across rows, and that is only valid if every row measures a comparable, non-overlapping thing.forced by · adding a per-line amount to a per-order amount double-counts the line inside the order
- 3A dimension can be joined to a fact only if the dimension's key is unique at or above the fact's grain; otherwise the join fans out and multiplies the measures.forced by · this is the join fan-out result — m×n rows per key value
- 4If rows of different grains coexist in one table, then no single aggregation rule is correct for the whole table, and no filter can reliably separate them unless a discriminator column exists and every query uses it.forced by · correctness now depends on every consumer knowing an unwritten rule
- 5And the failure is silent: a
SUMover mixed grains returns a number, not an error, and that number is plausible.forced by · the engine has no notion of grain; it enforces types and keys, never semantics
Therefore grain must be declared before any column is chosen, and every measure in the table must be true at exactly that grain.
And note what this predicts: measures split into three classes by how they aggregate. Additive measures (revenue, quantity) sum across every dimension. Semi-additive measures (account balance, inventory on hand) sum across everything except time — you cannot add Monday's balance to Tuesday's. Non-additive measures (ratios, percentages, distinct counts) cannot be summed at all and must be recomputed from their components. Classify every measure you build; that single label prevents most metric bugs that reach a dashboard.
A star schema is one central table of measurements — numeric, immutable, arriving continuously, huge — surrounded by tables of context: who, what, where, when. Facts answer "how much"; dimensions answer every "by what".
Every business question has the same shape: pick a measure, pick the dimensions to slice it by, pick a filter. The star exists so that shape maps onto exactly one join per dimension, and so that the meaning of each dimension is agreed once rather than re-derived per query.
- Declare the grain in a sentence before designing anything. Every column must be true at that grain or it does not belong in the table.
- Use surrogate keys in facts, not natural keys. Surrogates are what let a dimension carry history (SCD2) and survive a source system re-keying its own data.
- Conformed dimensions — one
dim_date, onedim_customershared across all facts — are what makes two facts comparable. Without them you have several unrelated stars. - Prefer more dimension attributes over fewer. Wide, denormalised dimensions are cheap (they are small) and every attribute is a slicing option you do not have to add later.
Fire this model the moment you see: two dashboards reporting different values for the same metric · "can we see this broken down by X?" where X is not stored · a report that no longer reproduces last quarter's number · a source system overwriting an attribute in place · a table where nobody can state what one row means · a metric definition living inside BI tool logic rather than in the warehouse.
A dimension attribute changes over time — customer region, product category, employee manager. Overwrite it, or version it?
is_current when they want today's viewDefault to Type 2 for any attribute a report groups by, and Type 1 only for genuine corrections. The test is a single question: was the old value ever true? If yes, it is history and destroying it is data loss. If no, it was an error and overwriting is correct.
The failure mode to watch for is Type 2 done halfway — history stored in the dimension, but facts joined on the natural key rather than the surrogate. That gives you all the storage cost and complexity of versioning with none of the benefit, plus a fan-out that multiplies every measure by the number of versions. If you commit to Type 2, the fact must carry the surrogate key captured at event time, and that is the part reviewers should check.
(c) Hands-on · 25 min
Build a tiny star schema + SCD-type-2 dimension in Postgres. Save as star_schema.sql.
-- star_schema.sql — build the minimum viable Kimball warehouse.
DROP SCHEMA IF EXISTS mart CASCADE;
CREATE SCHEMA mart;
-- ============================================================
-- Dimension: dim_date (pre-populated calendar table)
-- ============================================================
CREATE TABLE mart.dim_date (
date_key INT PRIMARY KEY,
full_date DATE NOT NULL UNIQUE,
year INT NOT NULL,
quarter INT NOT NULL,
month INT NOT NULL,
day INT NOT NULL,
day_of_week TEXT NOT NULL,
is_weekend BOOLEAN NOT NULL
);
INSERT INTO mart.dim_date
SELECT
TO_CHAR(d, 'YYYYMMDD')::INT AS date_key,
d::DATE AS full_date,
EXTRACT(YEAR FROM d)::INT AS year,
EXTRACT(QUARTER FROM d)::INT AS quarter,
EXTRACT(MONTH FROM d)::INT AS month,
EXTRACT(DAY FROM d)::INT AS day,
TO_CHAR(d, 'Day') AS day_of_week,
EXTRACT(DOW FROM d) IN (0,6) AS is_weekend
FROM generate_series('2024-01-01'::DATE, '2026-12-31'::DATE, '1 day') d;
-- ============================================================
-- Dimension: dim_customer (SCD Type 2)
-- ============================================================
CREATE TABLE mart.dim_customer (
customer_key BIGSERIAL PRIMARY KEY, -- surrogate
customer_id TEXT NOT NULL, -- business key
name TEXT NOT NULL,
email TEXT,
city TEXT,
segment TEXT,
effective_from TIMESTAMPTZ NOT NULL,
effective_to TIMESTAMPTZ NOT NULL DEFAULT '9999-12-31',
is_current BOOLEAN NOT NULL DEFAULT TRUE,
UNIQUE (customer_id, effective_from)
);
-- Insert initial row
INSERT INTO mart.dim_customer (customer_id, name, email, city, segment, effective_from)
VALUES ('C001', 'Alice', 'alice@ex.com', 'Hyderabad', 'gold', '2024-01-01');
-- Simulate Alice moving to Bangalore on 2024-06-01 — SCD Type 2 update
BEGIN;
UPDATE mart.dim_customer
SET effective_to = '2024-06-01', is_current = FALSE
WHERE customer_id = 'C001' AND is_current;
INSERT INTO mart.dim_customer (customer_id, name, email, city, segment, effective_from)
VALUES ('C001', 'Alice', 'alice@ex.com', 'Bangalore', 'gold', '2024-06-01');
COMMIT;
-- ============================================================
-- Dimension: dim_product (SCD Type 1)
-- ============================================================
CREATE TABLE mart.dim_product (
product_key BIGSERIAL PRIMARY KEY,
sku TEXT UNIQUE NOT NULL,
name TEXT NOT NULL,
category TEXT NOT NULL,
color TEXT
);
INSERT INTO mart.dim_product (sku, name, category, color) VALUES
('SKU-1','LED Bulb','electronics','warm'),
('SKU-2','USB Cable','electronics','black');
-- ============================================================
-- Fact: fact_sales — grain = one row per order line
-- ============================================================
CREATE TABLE mart.fact_sales (
sale_id BIGSERIAL PRIMARY KEY,
sale_date_key INT NOT NULL REFERENCES mart.dim_date(date_key),
customer_key BIGINT NOT NULL REFERENCES mart.dim_customer(customer_key),
product_key BIGINT NOT NULL REFERENCES mart.dim_product(product_key),
qty INT NOT NULL,
amount NUMERIC(10,2) NOT NULL,
discount NUMERIC(10,2) NOT NULL DEFAULT 0
);
-- Insert a sale on 2024-03-15 (Alice was in Hyderabad) and 2024-08-10 (Alice was in Bangalore)
INSERT INTO mart.fact_sales (sale_date_key, customer_key, product_key, qty, amount)
SELECT 20240315,
(SELECT customer_key FROM mart.dim_customer WHERE customer_id='C001' AND '2024-03-15'::DATE BETWEEN effective_from AND effective_to),
(SELECT product_key FROM mart.dim_product WHERE sku='SKU-1'),
3, 15.00;
INSERT INTO mart.fact_sales (sale_date_key, customer_key, product_key, qty, amount)
SELECT 20240810,
(SELECT customer_key FROM mart.dim_customer WHERE customer_id='C001' AND '2024-08-10'::DATE BETWEEN effective_from AND effective_to),
(SELECT product_key FROM mart.dim_product WHERE sku='SKU-2'),
2, 6.00;
-- ============================================================
-- The query that proves SCD Type 2 works
-- ============================================================
SELECT d.full_date, c.city AS city_at_time_of_sale, p.name, f.qty, f.amount
FROM mart.fact_sales f
JOIN mart.dim_date d ON d.date_key = f.sale_date_key
JOIN mart.dim_customer c ON c.customer_key = f.customer_key
JOIN mart.dim_product p ON p.product_key = f.product_key
ORDER BY d.full_date;
-- Alice's March sale shows Hyderabad; August sale shows Bangalore.What each block does
Anatomy of the schema
Extend dim_customer with a Type 6 hybrid: keep the type 2 rows AND add a current_city column that always mirrors the current row's city on every historic row.
ALTER TABLE mart.dim_customer ADD COLUMN current_city TEXT;
UPDATE mart.dim_customer d
SET current_city = (SELECT city FROM mart.dim_customer WHERE customer_id = d.customer_id AND is_current);Now the same fact table can answer both "revenue by customer's city at sale time" (c.city) and "revenue by customer's current city" (c.current_city) without any change to the fact.
(d) Production reality · 15 min
Airbnb had 200+ dashboards computing "bookings" 200+ different ways. Same metric, different definitions of "booked" (created vs confirmed vs paid), different grains (per-listing vs per-guest), different timezones. Executive reviews became arguments about whose number was right.
Built Minerva, a metric-definition layer: one dimensional model + one canonical definition per metric + generated queries feeding every dashboard. Any change to "bookings" propagates automatically. Numbers now agree.
A retail bank ran a Kimball warehouse with Type 1 dimensions for customer address. When a regulator asked "what address did we send this statement to on 2019-03-14?", the warehouse could only produce the current address. Fine ensued.
Migrated address + PII dimensions to Data Vault satellites (insert-only, timestamped). Every historical query is now reproducible. Reporting is slower — dozens of joins for a monthly statement — but auditability is absolute.
A B2B SaaS analytics team maintained 40 conformed dimensions and 12 facts in Snowflake, with dozens of dbt models joining them for every dashboard. Compile times ballooned, cost per query stayed high, and analysts hated the join complexity.
They kept the star as the ‘layer 2’ intermediate, then materialised a single wide table per business process at the mart layer — one row per event, 300 columns denormalised. Downstream analysts write 2-line queries; Snowflake's columnar engine reads only the columns needed. Query cost dropped 40 %.
Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
- What is grain? — one sentence + one bug example.
- SCD Type 2 — the four columns and the two-statement update.
- When OBT beats star — columnar economics in one sentence.
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.