S056 · REST API Design — Resources, Versioning, Idempotency

Module M06: Backend & APIs · Session 56 of 130 · Track: SYS 🧬

What you'll be able to do after this session

• Explain REST API design to a friend in one minute, out loud, no notes.
• Recognise it when you see it in real code / systems / papers.
• Complete the hands-on exercise at the end without looking up help.

Prerequisites

If you can't answer these in 30 seconds each, redo the linked session first:

• S055 — HTTP Fundamentals — Verbs, Status Codes, Headers, Caching

Pre-read (skim before the session)

• 🎥 Intuition (5–15 min) · What is REST API? — IBM Technology — 9 minutes, whiteboard, plain English.
• 🎥 Deep dive (30–60 min) · REST API Design Best Practices — freeCodeCamp — full course covering nouns, versioning, pagination, filtering.
• 🎥 Hands-on demo (10–20 min) · Build a REST API with FastAPI in 15 minutes — click-along to a working, documented API.
• 📖 Canonical article · Google API Design Guide — Resource-oriented design — Google's public guide; the gold standard reference.
• 📖 Book chapter / tutorial · Microsoft — REST API guidelines — public MS internal standard; used for Azure & Graph APIs.
• 📖 Engineering blog · Stripe — Designing robust and predictable APIs with idempotency — the canonical write-up on idempotency keys.

(a) Intuition · 5 min

The one-paragraph mental model. Why this concept exists, what problem it solves, and the analogy that makes it stick.

In one sentence: APIs that scale past v1.

REST is basically "design your API as if it were a filing cabinet." Every "thing" your system knows about — a user, an order, a document — is a resource with a stable URL like /users/42 or /orders/2025/09/abc123. Verbs are the standard HTTP ones (GET / POST / PUT / PATCH / DELETE), and status codes are the standard HTTP ones. That's it. There's no REST framework, no REST library — REST is a convention that says "if you organise your URLs and verbs this way, every human and every tool on Earth will already know how to use your API."

The alternative is a "RPC-style" API where every action gets its own verb-URL (POST /createUser, POST /updateEmail, POST /deleteUserById). This works, but every new action needs new docs, every client needs a custom SDK, and CDN caching gets awkward because you can't cache POSTs. REST's trick: constrain yourself to a tiny vocabulary (resources + 5 verbs) and get a huge ecosystem for free — OpenAPI, Swagger docs, Postman, CDNs, browser dev tools, generated clients.

The three things you must design well or you'll regret it: (1) resource modelling — what are your nouns, and how do they nest? (2) versioning — how will v2 exist without breaking v1 clients? (3) idempotency — what happens when a client retries because the response never arrived?

Forever gotcha: PUT replaces the entire resource; PATCH updates specific fields. Sending PUT /users/42 with {name: "Alice"} and forgetting the email field will delete the email. This bug ships to production somewhere every single week.

(b) Visual walkthrough · 15 min

Diagrams, worked examples, step-by-step visuals.

Resource-modelling table — the "orders" domain done right:

Use plural nouns. Use kebab-case or snake_case, be consistent. Never verbs in URLs (/getUser is not REST; /users/{id} is).

Idempotency — the flow that saves your bacon:

Without the idempotency key, the retry creates two payments. Stripe pioneered this pattern; every serious payments/orders API now supports it.

Versioning — the three schools:

1. URL versioning (/v1/users, /v2/users) — most popular, simplest for humans and caches. Downside: can't evolve one endpoint without a full version bump.
2. Header versioning (Accept: application/vnd.myco.v2+json) — Github/Stripe use variants. Cleaner URLs, but harder to explore.
3. Date-based versioning (Stripe-Version: 2024-06-20) — Stripe's approach; clients pin to a snapshot date. Server maintains a translation layer to keep old clients working forever.

Pagination — three flavours in one table:

Cursor-based is what Twitter, Slack, and Stripe use. Prefer it for anything serious.

Filtering, sorting, sparse fields — all query-string:

GET /orders?status=paid&sort=-created_at&fields=id,total

That's the standard convention across Google APIs, GitHub, JSON:API.

(c) Hands-on · 20 min

Code you type and run. Not pseudo-code — real, runnable, copy-pasteable.

Build a real, versioned, idempotent, paginated REST API in ~70 lines of FastAPI.

python -m venv .venv && source .venv/bin/activate
pip install "fastapi[standard]==0.115.0" uvicorn

# api.py
from fastapi import FastAPI, HTTPException, Header, Response, status
from pydantic import BaseModel
from uuid import uuid4
from typing import Optional
import base64, json
 
app = FastAPI(title="Orders API", version="1.0.0")
 
# --- in-memory stores (real code uses Postgres) ---
orders: dict[str, dict] = {}
idempotency_cache: dict[str, dict] = {}
 
class OrderIn(BaseModel):
    customer_id: int
    amount_cents: int
 
class OrderOut(OrderIn):
    id: str
    status: str
 
# ---- Create with idempotency ----
@app.post("/v1/orders", status_code=201, response_model=OrderOut)
def create_order(
    body: OrderIn,
    response: Response,
    idempotency_key: Optional[str] = Header(default=None, alias="Idempotency-Key"),
):
    if idempotency_key and idempotency_key in idempotency_cache:
        cached = idempotency_cache[idempotency_key]
        response.headers["Location"] = f"/v1/orders/{cached['id']}"
        return cached
    oid = f"ord_{uuid4().hex[:12]}"
    order = {"id": oid, "status": "created", **body.model_dump()}
    orders[oid] = order
    if idempotency_key:
        idempotency_cache[idempotency_key] = order
    response.headers["Location"] = f"/v1/orders/{oid}"
    return order
 
# ---- Get one ----
@app.get("/v1/orders/{oid}", response_model=OrderOut)
def get_order(oid: str):
    if oid not in orders:
        raise HTTPException(404, {"error": {"code": "not_found", "message": f"Order {oid} not found"}})
    return orders[oid]
 
# ---- List with cursor pagination ----
@app.get("/v1/orders")
def list_orders(limit: int = 20, cursor: Optional[str] = None):
    if limit < 1 or limit > 100:
        raise HTTPException(400, {"error": {"code": "invalid_limit", "message": "limit must be 1-100"}})
    all_ids = sorted(orders.keys())
    start = 0
    if cursor:
        decoded = json.loads(base64.b64decode(cursor).decode())
        start = decoded["offset"]
    page = all_ids[start:start + limit]
    next_cursor = None
    if start + limit < len(all_ids):
        next_cursor = base64.b64encode(json.dumps({"offset": start + limit}).encode()).decode()
    return {"data": [orders[i] for i in page], "next_cursor": next_cursor}
 
# ---- PATCH (partial) ----
@app.patch("/v1/orders/{oid}", response_model=OrderOut)
def patch_order(oid: str, patch: dict):
    if oid not in orders:
        raise HTTPException(404)
    orders[oid].update({k: v for k, v in patch.items() if k in {"amount_cents", "status"}})
    return orders[oid]
 
# ---- DELETE (idempotent by nature) ----
@app.delete("/v1/orders/{oid}", status_code=204)
def cancel_order(oid: str):
    orders.pop(oid, None)  # No 404 on repeat — idempotent

Run: uvicorn api:app --reload --port 8000

Test with curl:

# Create with idempotency key
KEY=$(uuidgen)
curl -X POST http://127.0.0.1:8000/v1/orders \
  -H "Idempotency-Key: $KEY" -H "Content-Type: application/json" \
  -d '{"customer_id":42,"amount_cents":9900}' -w '\nstatus=%{http_code}\n'
 
# Repeat SAME key — should return same order, still 201
curl -X POST http://127.0.0.1:8000/v1/orders \
  -H "Idempotency-Key: $KEY" -H "Content-Type: application/json" \
  -d '{"customer_id":42,"amount_cents":9900}' -w '\nstatus=%{http_code}\n'
 
# List with pagination
curl 'http://127.0.0.1:8000/v1/orders?limit=1'
 
# Auto-generated OpenAPI docs
open http://127.0.0.1:8000/docs

What to observe:

1. First POST returns 201 + a Location: /v1/orders/ord_... header — that's the REST convention.
2. Second POST with same idempotency key returns the same order id — no duplicate.
3. GET on a missing id returns 404 with {error: {code, message}} — consistent error envelope.
4. /docs opens Swagger UI, auto-generated from Pydantic models.
5. Cursor pagination hides the underlying storage — client only knows "next_cursor" and passes it back.

Try this modification: add rate limiting — return 429 Too Many Requests with a Retry-After: 5 header if a single IP hits POST /v1/orders more than 10 times in 60 seconds. That completes the four "must-have" behaviours of a production REST API: idempotency, pagination, structured errors, and rate limiting.

(d) Production reality · 10 min

How this shows up in real systems, gotchas, war stories, common bugs.

Every serious API at scale ends up implementing the same set of primitives: versioning strategy, idempotency keys on all state-changing writes, cursor pagination, structured errors, rate limiting, request IDs for tracing, and an OpenAPI/JSON-Schema contract as the source of truth (so SDKs and docs auto-generate).

War stories:

• The double-charge Christmas Day. E-commerce site had POST /orders with no idempotency key. Mobile client on flaky 3G retried on timeout. Customers were charged 2–5×. Manual refunds took a week; press picked it up. Fix: idempotency keys mandatory on all POSTs that create money-touching resources. Cache the key for at least 24h.
• The v1 that couldn't die. Company shipped /v1/users with a required field they later regretted. Ten years later, they still support v1 because "some clients are IoT devices in the wild with no update path." Fix: from day one, publish a deprecation policy ("v1 will sunset 24 months after v2 is GA") and enforce it with 410-Gone responses.
• The offset-pagination timebomb. GET /products?offset=1000000 did LIMIT 20 OFFSET 1000000 on Postgres → 8-second query. Bot pagination brought the DB down. Fix: switch to cursor pagination or cap offset at 10,000.
• The chatty API. Mobile app made 40 REST calls to render one screen (/users/me, /users/me/orders, /orders/1/items, /orders/1/items/1/product, ...). Cold-start latency = 12 seconds on 3G. Fix: either GraphQL, or a BFF (Backend For Frontend) that composes one screen-shaped endpoint.
• The security scan finding. GET /users?email=foo@bar.com returned different results (or timing) for existing vs non-existing emails → account enumeration. Fix: normalize responses, add auth.
• The versioning-through-headers surprise. Team versioned via Accept: application/vnd.foo.v2+json. New JS devs hit the API with default Accept: */* → got v1 forever, missed features. Fix: URL versioning is friendlier for humans exploring your API in the browser.

How top teams handle it: OpenAPI spec is committed to git, CI blocks breaking changes (using tools like oasdiff or openapi-diff), SDKs auto-generate from the spec (openapi-generator), and every response carries X-Request-Id that flows through structured logs for one-click debugging. Google's AIP and Microsoft's public API guidelines are worth bookmarking as reference.

Gotcha to memorise: breaking changes are: renaming a field, changing a type, adding a required field, tightening validation, removing an endpoint. Everything else (adding optional fields, adding new endpoints, adding new enum values with a documented default) is safe. Ship a linter in CI that enforces this — otherwise clients WILL break silently.

(e) Quiz + exercise · 10 min

1. What's the difference between PUT and PATCH, and what bug happens if you conflate them?
2. Design the URLs for a resource "books" and its sub-resource "reviews" — include the create, list, get-one, and delete endpoints.
3. What is an idempotency key and why does it need to be sent by the client (not generated by the server)?
4. Give three examples of a breaking change and three examples of a non-breaking change to a REST API.
5. Why is cursor pagination better than offset pagination for large collections?

Stretch (connects to S055 — HTTP): Your REST API returns 500 on 0.1% of requests due to a flaky downstream. Design a client-side retry policy (with backoff, jitter, max-attempts) and explain how it interacts with your idempotency key strategy.

Explain-out-loud test

If you can't teach these 3 things to a friend without notes, redo the session:

1. What is REST? (one sentence, no jargon)
2. When would you use it? (one concrete example)
3. When would you NOT use it? (one anti-pattern)

What comes next

• Next session (S057): GraphQL — Schema, Resolvers, N+1, When to Pick It
• Previous (S055): HTTP Fundamentals — Verbs, Status Codes, Headers, Caching
• Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: (a) intuition · (b) visual walkthrough · (c) hands-on · (d) production reality · (e) quiz + exercise. Duration: 60 minutes.