S036 · SQL Basics — SELECT, WHERE, ORDER BY, LIMIT
The universal data language. Learn the five verbs that answer 90 % of single-table questions, the execution order that isn't the written order, three-valued logic with NULLs, the operators every dashboard uses, and the two mistakes (SELECT *, string-concat SQL) that cause the most production incidents.
🎯 Write correct, fast, safe single-table SQL using SELECT / FROM / WHERE / ORDER BY / LIMIT — and know exactly what the DB does when it sees your query.
Why this session exists
SQL has outlived every programming trend since 1974 because it is the right level of abstraction for tabular data: declarative (say what you want, not how), relational (tables + set operations), and backed by a query planner that decides the execution strategy. Every ORM you'll ever use emits SQL under the hood; every BI tool writes SQL; every data warehouse is a SQL engine. Learning it directly — not through an ORM — is the highest-leverage data skill you can pick up.
- Write SELECT / FROM / WHERE / ORDER BY / LIMIT queries for any single-table question.
- State the true execution order of a SQL query and explain why you can't use SELECT aliases in WHERE.
- Use three-valued NULL logic correctly — IS NULL, COALESCE, and the traps of `= NULL` and `!= x`.
- Read an EXPLAIN plan well enough to know if a query is scanning or seeking.
- Avoid the two production classics: SELECT * and string-concatenated SQL (injection).
Prerequisites
- S035 — The Relational Model — you must know what a table, row, and key are.
(a) Intuition · 5 min
Cooking (imperative code): "Open the fridge, grab the tomatoes, wash them, chop them, heat oil, sauté onions, add tomatoes, simmer, salt, serve." Every step spelled out. Miss one, no dinner.
Ordering (declarative SQL): "One tomato curry, please." The kitchen decides which pan, which stove, which chef. You describe the outcome; the system picks the path.
SQL is ordering. SELECT name FROM users WHERE age > 30 ORDER BY name LIMIT 10 tells the database WHAT you want. The query planner picks the index, join strategy, and scan order to serve it fastest. That's the reason SQL has beaten every replacement for 50 years — the abstraction is at the right level.
Five verbs — SELECT, FROM, WHERE, ORDER BY, LIMIT — answer 90 % of single-table questions. Add JOIN (next session) and you can answer nearly any business analytics question ever asked, using the same syntax on SQLite, Postgres, Snowflake, and BigQuery.
- Declarative — describe the result, not the algorithm; the query planner finds a plan.
- Set-oriented — every operation transforms one set of rows into another; no explicit loops.
- Portable — the SQL-92 core runs on every database, and 90 % of what you write is standard.
- 1974SEQUEL · IBM System RChamberlin & Boyce publish SEQUEL — later renamed SQL because ‘SEQUEL’ was trademarked.
- 1986SQL-86 (ANSI SQL)First standardised version. Guarantees SELECT / WHERE / ORDER BY / LIMIT work the same across vendors.
- 1992SQL-92The dialect most of what you write today conforms to. Adds JOIN keyword and NULL semantics.
- 2003SQL:2003 · window functionsOVER (PARTITION BY …) — a game-changer we'll cover in S039.
- todaySQL runs the worldPostgres, MySQL, SQLite, Oracle, MSSQL, Snowflake, BigQuery, Databricks SQL, DuckDB. Same 5 verbs everywhere.
(b) Visual walkthrough · 15 min
The example table (we'll reuse in the next 5 sessions)
| id | name | dept | salary | hired |
|---|---|---|---|---|
| 1 | Alice | Eng | 120000 | 2020-03-01 |
| 2 | Bob | Eng | 95000 | 2021-06-15 |
| 3 | Carol | HR | 80000 | 2019-01-10 |
| 4 | Dan | Eng | 140000 | 2018-09-20 |
| 5 | Eve | Sales | 110000 | 2022-02-01 |
| 6 | Frank | Sales | 60000 | 2023-11-05 |
| 7 | Grace | HR | NULL | 2024-05-12 |
The execution order visualised
The WHERE-clause operators cheat sheet
Ordered comparisons
- `=`, `<>` / `!=` — equality
- `>`, `>=`, `<`, `<=`
- `BETWEEN a AND b` — inclusive range
- Dates & timestamps compare naturally
Membership tests
- `IN (…)` / `NOT IN (…)` — set membership
- `LIKE 'A%'` — % = any chars, _ = one char
- `ILIKE` (Postgres) — case-insensitive LIKE
- `~` / `~*` (Postgres) — regex match
Three-valued logic
- `IS NULL` / `IS NOT NULL` — never use `= NULL`
- `AND` binds tighter than `OR`
- Parenthesise `OR` groups always
- `COALESCE(x, default)` picks the first non-null
The three-valued logic trap
Grace is silently dropped. Any comparison with NULL yields UNKNOWN, not TRUE. To include NULLs explicitly: salary != 60000 OR salary IS NULL. This one gotcha causes more silently-wrong reports than any other SQL feature.
The three tiny rules that separate good from bad SQL
Couples your app to schema changes; reads more bytes than needed; kills prepared-statement caches.
`SELECT SUM(price * qty) AS revenue` — makes queries readable, joinable, and debuggable.
Use parameterised queries: `cur.execute("... WHERE id = ?", (uid,))`. This is the SQL-injection fix. Every driver supports it.
Pagination — the pattern every product uses
Two ways to paginate, and when to use each
"SQL executes top to bottom: SELECT first, then FROM, then WHERE. That's the order it's written."
The logical evaluation order is FROM → WHERE → GROUP BY → HAVING → SELECT → DISTINCT → ORDER BY → LIMIT. SELECT is evaluated second to last. That is why a column alias defined in SELECT cannot be used in WHERE — at the time WHERE runs, the alias does not exist yet.
Because the written order puts SELECT first, and reading order is a very strong prior. The rule stays hidden until you write SELECT price * qty AS total ... WHERE total > 100, get "column total does not exist", and cannot see why — the error names a column you can plainly see two lines above. The clue is that ORDER BY can use the alias, because it runs after SELECT. That asymmetry is the evaluation order made visible.
One alias, two clauses, opposite outcomes — and that difference is the whole rule:
-- FAILS: WHERE runs before SELECT, so 'total' does not exist yet
SELECT price * qty AS total
FROM line_items
WHERE total > 100;
-- WORKS: ORDER BY runs after SELECT
SELECT price * qty AS total
FROM line_items
ORDER BY total DESC;
-- The fix: repeat the expression, or wrap in a subquery / CTE
SELECT price * qty AS total
FROM line_items
WHERE price * qty > 100;Why is SQL declarative at all? Why does the language refuse to let you specify how to get the rows, when every other language you write is explicit about that?
- 1Relational algebra operators — select, project, join, union — are closed: each takes relations and returns a relation.forced by · closure is what allows operators to compose arbitrarily without special cases
- 2Closure plus algebraic laws (join is commutative and associative, selections push through joins) means many different operator orderings compute the identical result.forced by · these are provable equivalences over sets, exactly like arithmetic identities
- 3Those equivalent orderings can differ in cost by orders of magnitude — filtering before a join versus after can change the intermediate size by a factor of thousands.forced by · cost depends on intermediate cardinality, which the order controls, while the final answer does not depend on order at all
- 4Which ordering is cheapest depends on data statistics, index availability, memory and cluster shape — all of which change over time and none of which the query author knows at writing time.forced by · the optimal plan is a function of runtime state, not of query text
- 5Therefore the only way to always get a good plan is to have the engine choose it at execution time, which requires the query to state what and not how.forced by · if the query specified the procedure, the engine would be forbidden from improving it
Therefore SQL's declarativeness is the price of letting the optimiser rewrite your query, and the payoff is that the same query text stays fast as data grows, indexes appear, and statistics change.
And note what this predicts: when the optimiser has bad statistics, it makes bad choices, and no amount of rewriting the SELECT list helps — because the SELECT list is not what it is optimising. It also predicts that the biggest performance wins come from giving the optimiser better information (accurate stats, useful indexes, selective predicates it can push down) rather than from cleverer SQL. Go read a plan for a slow query and check its estimated versus actual row counts; that gap is almost always the real bug.
Read every query as a pipeline where each stage consumes a relation and emits a relation. FROM produces the raw set. WHERE filters rows. GROUP BY collapses many rows into one per group. HAVING filters those groups. SELECT chooses and computes columns. ORDER BY and LIMIT act on the finished result.
Once you see the pipeline, the confusing rules become obvious rather than memorised: WHERE cannot see aggregates because grouping has not happened yet; HAVING can, because it runs after; aliases are available only downstream of SELECT.
WHEREfilters rows before grouping;HAVINGfilters groups after. Putting a non-aggregate condition inHAVINGworks but processes more rows than necessary.- Nothing is ordered until
ORDER BY. Any ordering you observe without it is a coincidence of the current plan. - NULL propagates through comparisons and is skipped by aggregates —
COUNT(col)ignores NULLs whileCOUNT(*)does not. SELECT *in production code is a defect: it breaks when the schema changes, defeats columnar projection pushdown, and hides which columns you actually depend on.
Fire this model the moment you see: "column does not exist" on an alias you just defined · an aggregate rejected in WHERE · results in an order you did not ask for · a DISTINCT added to hide duplicate rows · a query that is slow and you are tempted to change the SELECT list to fix it.
You need to filter rows before an expensive join. Put the predicate in the WHERE clause, or in the JOIN ... ON clause?
WHERE, so mixing both styles across a codebase invites confusion about which one mattersFor inner joins the choice is stylistic — the optimiser produces the same plan. For outer joins it is a correctness decision, and it is one of the most common silent bugs in analytical SQL: someone adds a WHERE filter to a LEFT JOIN query, row counts drop, and it looks like the data changed rather than the join semantics.
The habit worth building: whenever you write LEFT JOIN, immediately ask where each predicate belongs, and if the answer is not obvious, pre-filter in a CTE and remove the ambiguity entirely. Then check the row count against the left table's count — a LEFT JOIN that reduces rows is either a bug or a fan-out you need to explain.
(c) Hands-on · 25 min
Save as sql_basics.py, run python3 sql_basics.py.
"""sql_basics.py — SELECT / WHERE / ORDER BY / LIMIT + the classic NULL gotchas."""
from __future__ import annotations
import sqlite3
conn = sqlite3.connect(":memory:")
cur = conn.cursor()
cur.executescript("""
CREATE TABLE employees (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
dept TEXT NOT NULL,
salary INTEGER, -- nullable ON PURPOSE for demo
hired TEXT NOT NULL
);
INSERT INTO employees VALUES
(1, 'Alice', 'Eng', 120000, '2020-03-01'),
(2, 'Bob', 'Eng', 95000, '2021-06-15'),
(3, 'Carol', 'HR', 80000, '2019-01-10'),
(4, 'Dan', 'Eng', 140000, '2018-09-20'),
(5, 'Eve', 'Sales', 110000, '2022-02-01'),
(6, 'Frank', 'Sales', 60000, '2023-11-05'),
(7, 'Grace', 'HR', NULL, '2024-05-12');
""")
def run(label: str, sql: str, params: tuple = ()) -> None:
print(f"\n-- {label} --")
print(sql.strip())
for row in cur.execute(sql, params):
print(" ", row)
# 1. Basic SELECT + WHERE + ORDER BY + LIMIT
run("Top 2 Eng earners over 100k",
"""SELECT name, salary FROM employees
WHERE dept = 'Eng' AND salary > 100000
ORDER BY salary DESC LIMIT 2;""")
# 2. Precedence bug — AND binds tighter than OR
run("Precedence BUG · name starts with A or E, in HR or Sales, 80k-120k",
"""SELECT name, dept, salary FROM employees
WHERE salary BETWEEN 80000 AND 120000
AND dept IN ('HR','Sales')
AND name LIKE 'A%' OR name LIKE 'E%';""")
# Same query with parentheses — the intended answer
run("Precedence FIXED · with parentheses around the OR",
"""SELECT name, dept, salary FROM employees
WHERE salary BETWEEN 80000 AND 120000
AND dept IN ('HR','Sales')
AND (name LIKE 'A%' OR name LIKE 'E%');""")
# 3. NULL handling — the classic trap
run("Employees with UNKNOWN salary (IS NULL)",
"SELECT name FROM employees WHERE salary IS NULL;")
run("Wrong way: `= NULL` returns zero rows (silent bug)",
"SELECT name FROM employees WHERE salary = NULL;")
run("`salary != 60000` — does Grace appear? (three-valued logic)",
"SELECT name, salary FROM employees WHERE salary != 60000;")
run("`salary != 60000 OR salary IS NULL` — explicit NULL handling",
"SELECT name, salary FROM employees WHERE salary != 60000 OR salary IS NULL;")
# 4. COALESCE for a default
run("Show salary as 0 when NULL, sort ascending",
"SELECT name, COALESCE(salary, 0) AS salary FROM employees ORDER BY salary ASC;")
# 5. DISTINCT + LIMIT/OFFSET pagination
run("Distinct departments alphabetically",
"SELECT DISTINCT dept FROM employees ORDER BY dept;")
run("Page 2 of employees, 3 per page (LIMIT 3 OFFSET 3)",
"SELECT id, name FROM employees ORDER BY id LIMIT 3 OFFSET 3;")
# 6. Keyset pagination — the O(log n) way
run("Keyset pagination — next page after id=3",
"SELECT id, name FROM employees WHERE id > ? ORDER BY id LIMIT 3;", (3,))
# 7. Parameterised query — the ANTI-SQL-INJECTION pattern
run("Safe: parameterised query", "SELECT name FROM employees WHERE dept = ?", ('Eng',))
# 8. EXPLAIN — see what the planner is doing
run("EXPLAIN plan for the top-earners query",
"""EXPLAIN QUERY PLAN
SELECT name FROM employees
WHERE dept='Eng' ORDER BY salary DESC LIMIT 2;""")
# Add an index and re-EXPLAIN
cur.execute("CREATE INDEX ix_dept_salary ON employees(dept, salary DESC)")
run("EXPLAIN after CREATE INDEX",
"""EXPLAIN QUERY PLAN
SELECT name FROM employees
WHERE dept='Eng' ORDER BY salary DESC LIMIT 2;""")
conn.close()Anatomy of the script
Add a city column and 20 rows spread across 3 cities. Then write:
-- First page: no filter
SELECT id, name, city FROM employees ORDER BY id LIMIT 5;
-- Next page: replace ? with the last id you saw
SELECT id, name, city FROM employees WHERE id > ? ORDER BY id LIMIT 5;Compare it to the OFFSET version:
SELECT id, name, city FROM employees ORDER BY id LIMIT 5 OFFSET 15;On a 20-row table you won't feel the difference. On a 100M-row table, the OFFSET query walks through 15,000,015 rows and throws 15,000,010 away. The keyset one does a single index seek and reads 5 rows. Same output, orders of magnitude different cost.
(d) Production reality · 15 min
The profile page ran SELECT * FROM transactions WHERE user_id = ?. Innocent — until a data engineer added a 2 KB raw_response_json column to the table for a compliance audit. Every profile page load now dragged 4 MB back to the app server. The DB became network-bound; the app became CPU-bound serialising JSON it never used.
One PR: enumerate the columns the page actually uses. Latency dropped back to 40 ms within the deploy. Add a lint rule that fails CI on any SELECT * in application code.
Weekly finance report ran SELECT SUM(refund_amount) FROM orders WHERE status = 'refunded'. It looked right — but the store-credit refunds stored NULL in refund_amount. SUM silently ignores NULLs. The report was 8 % low for 52 weeks before anyone noticed.
SUM(COALESCE(refund_amount, 0)) and, going forward, a NOT NULL DEFAULT 0 on the column. Add a check to CI that no monetary column is nullable without a written comment explaining why.
An engineer wrote cur.execute(f"SELECT * FROM users WHERE email = '{email}'"). A user set their email to ' OR '1'='1'; DROP TABLE users; --. The DB happily returned every user and then dropped the table.
Always use parameterised queries: cur.execute(\"SELECT * FROM users WHERE email = ?\", (email,)). The driver sends the SQL and the parameter separately; the parameter is never interpreted as SQL. Every language, every driver, every ORM supports this.
Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
- What is SQL, in one sentence? (the word "declarative" must appear)
- What's the actual execution order? (all seven clauses, in order)
- What are the two production classics that this session says never to do? (name both)
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.