S056 · REST API Design — Resources, Versioning, Idempotency
The design decisions that separate an API you can still maintain at v4 from one you have to burn down. Resources vs actions, versioning strategies that actually work, and idempotency keys as a first-class citizen. With a full working design for a real ‘orders’ API.
🎯 Design a resource-oriented API for a real domain (orders + payments), pick a versioning strategy you can defend, and specify idempotency + pagination so clients survive network failures without duplicating data.
Why this session exists
Every API starts clean at v1. Two years later it has three versioning schemes, four ways to paginate, and a POST endpoint called /doStuff. REST design is the discipline that keeps that from happening — a small set of conventions (resources, verbs, status codes, versioning) that scale from three endpoints to three hundred without collapsing. Stripe, Twilio, GitHub, and Shopify all built billion-dollar businesses partly by getting these decisions right. This session teaches the decisions.
- Model a domain as resources instead of actions, and name them consistently.
- Choose between URL versioning, header versioning, and evolutionary compat with a defensible argument.
- Design cursor pagination that survives inserts, deletes, and jumps in the underlying data.
- Add idempotency keys to a POST endpoint so clients can safely retry.
- Write an OpenAPI spec that generates docs, client SDKs, and mock servers from one file.
Prerequisites
- S055 · HTTP Fundamentals — verbs, status codes, headers.
- S045 · Data modelling (dimensional) — helps for choosing resources vs sub-resources.
(a) Intuition · 5 min
Cities that are easy to navigate follow a consistent grid: numbered streets, avenues at right angles, addresses that increment predictably. Newcomers find any building without asking.
Cities that grew organically (looking at you, historic city centres) are charming but have three ways to spell every road and require locals to guide you. Beautiful, but you cannot scale a delivery service on top of them.
REST is the street grid for APIs. Resources are the buildings (/orders/42), verbs are the actions (GET, POST), and status codes are the standard signage. A newcomer can guess ‘DELETE /orders/42 probably cancels order 42’ without reading a single doc page.
Non-REST or RPC-style APIs work fine at small scale (‘/api?op=cancelOrder&id=42’), but every new endpoint requires new documentation because there's no shared grammar.
The design decisions that make or break an API
- Resource modelling — nouns not verbs, plural collections, sub-resources for compositions.
- Versioning strategy — URL segment, custom header, or evolutionary compat. Pick one and commit.
- Idempotency — POST is not idempotent by default; add an Idempotency-Key header for anything the client will retry.
- Pagination — offset for small lists, cursor for anything that could grow past 10k rows.
- Error format — one envelope shape across the whole API, with a machine-readable code + human message.
- Spec-first — write OpenAPI before code so SDKs, docs, and mocks all stay in sync.
How the industry converged
- 2000Fielding's REST dissertationFormalises the architectural style. Ignored for years while SOAP + XML dominate.
- 2005Rails ‘resources’ scaffoldDHH's Rails router makes REST verbs the default idiom. Adoption explodes.
- 2011Stripe launches with a public, versioned REST APIIdempotency keys, pinned versions, deprecation policy — the template for the next decade.
- 2014Swagger becomes OpenAPIVendor-neutral spec format. SDK generation and docs auto-derived from one YAML.
- 2020gRPC + GraphQL crowd inREST is no longer the only option, but ‘default’ for public APIs remains REST + OpenAPI.
(b) Visual walkthrough · 15 min
Resource tree for a real ‘orders’ API
Notice :cancel — Google's convention for actions that don't fit CRUD. Prefer this to POST /cancelOrder because it keeps the resource identity in the URL.
CRUD → HTTP mapping
POST /orders
- Body = new order
- 201 Created + Location header
- May accept Idempotency-Key
- Returns the created resource
GET /orders/{id}
- No body
- 200 OK + full resource
- Cacheable with ETag
- 404 if not found
GET /orders?limit=&cursor=
- Cursor pagination for growth
- Filters as query params
- Consider server-side default limit
- Return next_cursor in response
PUT /orders/{id}
- Body = full resource
- 200 OK or 204 No Content
- Idempotent by contract
- Rarely used outside admin tooling
PATCH /orders/{id}
- Body = JSON Merge Patch or JSON Patch
- 200 OK + updated resource
- Idempotent if well-designed
- Prefer for user-visible mutations
DELETE /orders/{id}
- No body
- 204 No Content
- Second DELETE = 404 (or 204 in ‘tolerant’ style)
- Consider soft-delete for audit
Versioning strategies
Pick one — do not mix
Pagination models
Simple. Fails as data grows (‘page 500 of a live table’ = duplicates + skips). Fine for small admin lists.
Server returns `next_cursor` opaque token. Stable under concurrent inserts. Default for anything that grows.
Cursor's simple cousin: use a monotonic primary key. Fast on indexed columns. Twitter, Slack.
Best for event streams. Combine with cursor for pages within a window.
"REST means using HTTP verbs correctly and having nice plural nouns in the URL. If I have GET /users/123 instead of /getUser?id=123, my API is RESTful."
URL aesthetics are the least important part. REST's actual constraints are: uniform interface, statelessness, cacheability, layered system, and client–server separation. The pressure that matters in practice is resource modelling — deciding what your nouns are — and evolvability: how the API changes without breaking clients you cannot redeploy.
The myth is sticky because URL shape is the visible, checkable part — it fits in a lint rule and a code review comment, so it becomes the proxy for quality. It's also genuinely correlated with good design early on. It breaks down at the first operation that isn't a noun: "retry this job", "merge these accounts", "send the invoice". Teams that believe REST is about nouns spend hours inventing fake resources for these instead of accepting that some operations are operations.
Take the hardest endpoint in your API and ask which constraint it actually satisfies:
# Is the response cacheable? Does it say so?
curl -sI https://api.example.com/v1/users/123 | grep -i 'cache-control\|etag'
# Is it stateless? Does it work with no prior request and no server memory?
curl -s -H 'Authorization: Bearer TOKEN' https://api.example.com/v1/users/123
# Does a conditional request actually save bandwidth?
curl -sI -H 'If-None-Match: "abc123"' https://api.example.com/v1/users/123Most "RESTful" APIs return no ETag, no Cache-Control, and 200 on every conditional request. Pretty URLs, none of the constraints.
Why must a public API version its contract rather than just "not breaking things"? Careful engineering sounds sufficient — it isn't.
- 1Once an API is public, clients exist that you cannot enumerate, cannot contact, and cannot redeploy.forced by · anyone with a token can integrate; there is no registry of who parsed your JSON last Tuesday
- 2Clients depend on properties you never intended to promise: field order, the presence of a field they ignore, the exact shape of an error, a numeric ID being numeric.forced by · observable behaviour becomes the contract regardless of what your documentation says (Hyrum's Law)
- 3Therefore "is this change breaking?" cannot be answered by inspecting your own code — it depends on client behaviour you cannot observe.forced by · breakage is defined at the consumer, and you have no visibility into consumer parsing logic
- 4The only way to make change safe is to keep the old observable behaviour reachable while the new one exists, so clients migrate on their own schedule.forced by · if old and new cannot coexist, every change forces synchronous global migration, which is impossible with unknown clients
- 5That requires an explicit selector — a version in the path, a header, or a date — because the server must know which contract this particular request expects.forced by · coexistence needs disambiguation; the server cannot infer intent from the request alone
Therefore versioning is not bureaucracy, it is the mechanism that decouples your deploy schedule from your clients'. Its real purpose is not "supporting old clients" but "letting you ship at all".
And note what this predicts: additive changes need no version bump, because adding a field cannot break a client that ignores unknown fields — which is exactly why "clients must tolerate unknown fields" appears in every mature API guideline. It also predicts the cheapest possible versioning strategy: make almost every change additive, and you will need a version bump so rarely that the cost of supporting two versions never arises. Teams that version constantly usually have a modelling problem, not a versioning problem.
Design the nouns first: what things exist, what identifies them, how they contain each other. The endpoints then fall out almost mechanically — a collection, an item, and sub-collections for relationships. If designing the URL is hard, the resource model is wrong, not the URL.
The API is a vocabulary a stranger must be able to guess. If they know /orders/{id} exists, they should be able to guess /orders/{id}/items without reading anything.
- Collections are plural, items are collection-plus-identifier, relationships are sub-collections. Never encode an action as a path segment on a noun — if you need an action, either it is a sub-resource with its own lifecycle (
POST /jobs/{id}/retries) or you accept a verb and move on. - Pagination is not optional, and it is cursor-based. Offset pagination on a mutating collection silently skips and duplicates rows as items are inserted, and it degrades on large offsets because the database must still scan what it skips.
- Errors are part of the contract: consistent shape, a stable machine-readable code, a human message, and a correlation ID. Clients switch on the code — so changing a code is a breaking change even though nothing in your schema moved.
- Every mutating endpoint that a client might retry needs an idempotency key. The client cannot distinguish a lost request from a lost response, so it will retry, and without a key it will double-create.
Fire this model when you see: a URL containing a verb · an endpoint returning a different shape depending on a query parameter · offset pagination over a live table · an error response that is a bare string · a "v2" that changed one field.
A client needs a resource plus three of its relationships. Do you let them fetch each separately, embed everything by default, or offer field selection?
?expand=customer,items&fields=id,total), so payload size matches actual requirements and clients evolve independently of the server.Start with strict resources and add a small, explicit set of expansions when a real client demonstrates a real round-trip problem. Do not build general field selection speculatively — it is a query language in disguise, and if you find yourself building the whole thing, that is the signal to evaluate GraphQL rather than reinvent it badly.
The tell that you chose wrong: a client making 50 sequential calls to render one screen, or a response payload where 90% of the bytes are ignored by every caller. Instrument response size and per-screen request count; both are cheap to measure and both tell you which side of this tradeoff is currently hurting.
(c) Hands-on · 25 min
We'll design a small orders API with FastAPI — the fastest way to see resource modelling, idempotency, cursor pagination, and OpenAPI generation working together.
Try it:
pip install fastapi uvicorn
uvicorn orders_api:app --reload
# create an order with an idempotency key
KEY=$(uuidgen)
curl -X POST http://127.0.0.1:8000/v1/orders \
-H 'Content-Type: application/json' \
-H "Idempotency-Key: $KEY" \
-d '{"customer_id": 42, "items": [{"sku":"A1","qty":2}]}'
# retry the exact same call — same order returned, no duplicate
curl -X POST http://127.0.0.1:8000/v1/orders \
-H 'Content-Type: application/json' \
-H "Idempotency-Key: $KEY" \
-d '{"customer_id": 42, "items": [{"sku":"A1","qty":2}]}'
# cursor pagination
curl 'http://127.0.0.1:8000/v1/orders?limit=1'Then open http://127.0.0.1:8000/docs — the Swagger UI is generated for free from the Pydantic models. That's your API reference, mock server, and SDK-generator source of truth, all from one file.
Anatomy of the script
What each block teaches
Add a Deprecation header when a legacy code path is used:
from datetime import datetime, timedelta, timezone
@app.get("/v1/orders/{order_id}", response_model=Order)
def get_order(order_id: str, accept_version: str = Header(default="v1", alias="Accept-Version")):
order = ORDERS.get(order_id)
if not order:
raise HTTPException(status_code=404, detail={"code": "resource_missing", "message": f"No such order: {order_id}"})
headers = {}
if accept_version == "v0":
sunset = (datetime.now(timezone.utc) + timedelta(days=90)).strftime("%a, %d %b %Y %H:%M:%S GMT")
headers["Deprecation"] = "true"
headers["Sunset"] = sunset
headers["Link"] = '</docs/migration>; rel="deprecation"'
return JSONResponse(content=order.model_dump(), headers=headers)Now any client can programmatically detect they're on a deprecated path — automated migration reports become trivial.
(d) Production reality · 15 min
Payments must never charge twice. But the network is unreliable — a client can't tell if a failed request actually created the charge or not. Retrying naively risks duplicates; not retrying risks lost data.
Every early payments API punted this to the customer: "if unsure, query manually before retrying". Awful UX, error-prone integrations.
Two patterns that prevent the trap:
- Evolutionary compat (Google): never break existing fields, only add. Semantically-versioned changes stay backwards-compatible for years.
- Date-pinned versions (Stripe): the server evolves continuously; each request pins to a snapshot. Clients migrate by bumping one string.
Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
Move on when you can teach these without notes:
- Why do we model APIs around resources instead of actions?
- What is an idempotency key and when is it non-negotiable?
- Why is offset pagination a trap once your data grows?
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.