Dinesh/ Blog
systems

Designing for Scale · Requirements to Architecture

Convert prompts into testable requirements, capacity constraints and failure contracts through four worked designs and a traceable review method.

24 min read
Checking device speech…
Lesson preparation & details

Level: advanced

Designing for scale · Lesson 1 of 20 ↗
Loading this browser’s progress…

By the end, you should be able to

  • Bound behavior, load and failure requirements
  • Trace architectural mechanisms to falsifiable requirements
  • Compare four designs with consistent capacity arithmetic
  • Explain acknowledgement, consistency and recovery boundaries

Bring with you

Editorial review: · What review means

In this article · 24 sections

The problem this post solves

“Design a system to share short notes” is a prompt, not a specification. Before selecting a database, determine who writes and reads, what acknowledgement means, how stale a read may be, what failures must be survived and which costs are unacceptable. A technically competent diagram can still describe the wrong product.

The architecture is a consequence of constraints and choices among feasible alternatives. Requirements do not mechanically select one vendor. They make a choice reviewable: someone can ask whether the design satisfies a named objective, and identify the observation that would disprove it.

First principles

A cache trades repeated work for invalidation and memory. Asynchronous processing trades response immediacy for queueing and status semantics. Multi-region replication trades failure/geographic benefits against coordination, cost and operations. None is “correct” independently of a requirement. Starting with a familiar box list hides those decisions.

A useful initial split is functional versus quality/operational requirements, not an exhaustive taxonomy of exactly two kinds. Functional requirements name behavior: create, retrieve, expire, bill or authorize. Quality requirements specify latency, availability, durability, capacity, accessibility, privacy or cost under stated conditions. Constraints also include deployment environment, regulation and organizational capabilities.

Both kinds shape architecture. A transfer requiring an atomic debit and credit is not merely the same nouns as an eventually consistent social counter. The original slogan “features barely constrain architecture” was too strong. Quality is not polish added after features, and behavior is not irrelevant to structure.

The core model

The core model
Figure text equivalent — labels and relationships
  • Node “Ambiguous prompt” (square).
  • Node “Requirements and exclusions” (square).
  • Node “Capacity and failure budgets” (square).
  • Node “Candidate architecture and operation paths” (square).
  • Node “Deep dive into weakest contract” (square).
  • Node “Tests risks and decisions” (square).
  • “Ambiguous prompt” leads to “Requirements and exclusions”. Line: normal; ends: arrow point.
  • “Requirements and exclusions” leads to “Capacity and failure budgets”. Line: normal; ends: arrow point.
  • “Capacity and failure budgets” leads to “Candidate architecture and operation paths”. Line: normal; ends: arrow point.
  • “Candidate architecture and operation paths” leads to “Deep dive into weakest contract”. Line: normal; ends: arrow point.
  • “Deep dive into weakest contract” leads to “Tests risks and decisions”. Line: normal; ends: arrow point.
  • “Tests risks and decisions” leads to “Requirements and exclusions”: revise assumptions. Line: dotted; ends: arrow point.
  • “Capacity and failure budgets” leads to “Requirements and exclusions”: infeasible objective. Line: dotted; ends: arrow point.
  • “Candidate architecture and operation paths” leads to “Capacity and failure budgets”: missing mechanism. Line: dotted; ends: arrow point.

This four-step frame—requirements, capacity, high-level design, deep dive—supports iteration. Finding that a freshness target demands unaffordable hot storage is progress: revise the target or budget before implementation. The final test/decision record makes the loop explicit.

Step one in practice: bounding the problem

Who uses it, and when? DAU helps model daily user actions; concurrent connections size connection state; registered users size account records; scheduled jobs and bots also generate load. Do not replace one with another. A product launch can change population suddenly, so “population is always the slowest corner” is not a safe assumption.

What do they do, and in what proportion? Separate operations and payloads. Read/write ratio matters, but a large analytical read can cost more than many append writes. Count retries, fanout and background compaction as work too. State a range for uncertain usage rather than treating a remembered industry multiplier as a measurement.

What must not happen? Translate aspirations into invariants and failure domains. For chat, distinguish accepted, durably stored, delivered to a device and read by a person. For billing, distinguish a usage event, an aggregate and a finalized invoice. “Never lose data” needs an acknowledgement boundary, retention, recovery-point objective and named failure scenarios; no finite design promises survival of every conceivable failure.

The load triangle is population × frequency × payload, with operation cost and fanout beside it. A new notification can multiply actions; a video feature can multiply bytes; neither needs more users to overload the service. Measure the corner that changed instead of assuming all overload is a traffic spike.

Writing requirements that constrain

Replace “fast” with a measurable statement: “Over a rolling day, at least 99% of eligible authorized note reads complete within 200 ms at the client boundary for a specified workload and regions.” Define eligible requests, timeout/cancellation treatment, payload range and how measurements are collected. Google SRE's SLO guidance treats an SLI as a defined measurement, often good events divided by total eligible events, rather than an adjective.

If the target covers only the hot 90% of traffic, define that population independently and report the excluded tail. Do not let the system reclassify slow requests as ineligible after observing them. A 200 ms target does not logically require a cache or categorically forbid an ocean crossing; evaluate path latency, work and failure requirements. A cache is one possible mechanism.

Percentiles need population and composition care. If twenty independent calls each have a 1% probability of exceeding a threshold, the chance that at least one exceeds it is 1 - 0.99**20 ≈ 18.21%. This is not most page loads, and does not by itself establish the page's p99. Calls may be correlated, sequential or partially optional; the page also has other work. End-to-end measurement remains necessary.

Capacity and constraints

Compute only what might change a decision: operation rates, occupied work, retained bytes, transfer by link, hot-set memory and recovery time. Use 86,400 seconds/day for reproducible arithmetic, then round presentation appropriately. A peak factor is an assumption about a time window and workload; utilization headroom is a separate policy.

A short decision table can say: the retained dataset may fit one node, but restore must finish within two hours; cached eligible reads reduce origin work, but revoked notes must disappear within one minute; asynchronous export is allowed five minutes, but an accepted creation must survive a zone loss. Numbers and qualitative invariants both constrain the design.

How much estimation? A genuinely small system may need one paragraph and a simple design. A design-stage review should focus on thresholds and sensitivity. A purchase/recovery plan needs a more detailed model with measured inputs. Skipping irrelevant arithmetic is disciplined; skipping load and failure assumptions is not.

Architecture

Architecture
Figure text equivalent — labels and relationships
  • Node “Clients” (square).
  • Node “Optional edge for eligible content” (square).
  • Node “Service and admission” (square).
  • Node “Optional read cache” (cylinder).
  • Node “Authoritative durable store” (cylinder).
  • Node “Deferred-work queue” (square).
  • Node “Workers” (square).
  • Node “Replicas and recovery copies” (cylinder).
  • “Clients” leads to “Optional edge for eligible content”. Line: normal; ends: arrow point.
  • “Optional edge for eligible content” leads to “Service and admission”. Line: normal; ends: arrow point.
  • “Service and admission” leads to “Optional read cache”. Line: normal; ends: arrow point.
  • “Service and admission” leads to “Authoritative durable store”. Line: normal; ends: arrow point.
  • “Service and admission” leads to “Deferred-work queue”. Line: normal; ends: arrow point.
  • “Deferred-work queue” leads to “Workers”. Line: normal; ends: arrow point.
  • “Workers” leads to “Authoritative durable store”. Line: normal; ends: arrow point.
  • “Authoritative durable store” leads to “Replicas and recovery copies”. Line: normal; ends: arrow point.

Trace one write and one read, not merely the boxes. For a note creation, authorize, validate, assign identity, commit durably under the stated policy and then acknowledge. A retry with the same operation identity must not create inconsistent duplicates. For a read, authorize under the current policy, select a version permitted by the consistency contract and return or explicitly fail. Cache hits still need the relevant authorization/invalidation contract.

A queue exists because a particular operation tolerates delay and has a recovery path. A replica may support reads or availability; it is not a backup against accidental deletion if it faithfully replicates the deletion. Backups need restore tests. A load balancer can itself be a failure dependency. Remove unjustified complexity, but do not delete a component merely because one justification disappears if another still applies.

Choosing the deep dive

Choose the most uncertain, consequential contract: hot-key fanout, cross-region acknowledgement, seat ownership, deletion propagation or model-serving admission. A timeboxed interview may permit one deep dive; a production design needs all critical risks examined. Personal interest is not a substitute for risk selection.

For each deep dive, write the state before the operation, the durable transition, acknowledgement, retry identity, failure between steps and recovery. That sequence makes “we use a queue” into an actual design.

Failure modes

Remembered template: the diagram is unchanged across radically different invariants. Challenge each box with the requirement it satisfies. Average-only latency: tails and fanout disappear. Define percentiles and end-to-end measurement. Population confusion: DAU is treated as concurrent connections or all stored accounts. Keep separate counts. Peak equals average: bursts and expensive requests have no admission plan. Unbounded retention: “all history instantly queryable” imports unknown storage and indexing cost. Implicit consistency: caches and replicas accidentally decide which stale reads users see.

A requirement can also be overconstrained: promising a global synchronous write with an unrealistically small deadline may be infeasible across the chosen geography. State the tradeoff and ask which product property can change instead of quietly weakening the acknowledgement.

Trade-offs that matter

Bound the current problem, then identify plausible changes that cross a design threshold. “Split the connection tier when memory or connection churn exceeds the tested safe envelope” is more useful than an arbitrary universal 200,000-connection rule. Monitor the load-bearing assumption and include migration lead time.

Record exclusions: for example, “regional feed reads may be up to 60 seconds stale; account revocation may not.” Exclusions are decisions, not omissions. Requirements for privacy, auditability and correctness can dominate capacity even at 50 transactions/s. A design can be small in throughput and still difficult.

Four worked cases

All figures here are synthetic inputs. Each case follows requirements → arithmetic → conditional architecture → failure test, so similar QPS does not imply similar design.

Requirements: create, resolve, expire and count resolutions. Assume resolution p99 below 50 ms at the edge, creation below 300 ms, and resolution through a region outage under a defined replica-freshness policy. A code must never resolve to a different owner's URL after retirement. Define whether newly acknowledged codes must already be visible in another region.

Numbers: 1 million creations/day and 500 million resolutions/day give 11.57 writes/s and 5,787.04 reads/s average, or 17,361.11 reads/s at an assumed 3× read peak. The ratio is 500:1. At 500 logical bytes/row, growth is 500 MB/day or 182.5 GB per 365-day year, before indexes, copies and logs. A few-GB hot set is a separate assumption to validate.

Choice: repeated eligible reads may justify caching. The logical storage volume suggests evaluating a single primary before sharding, but throughput, restore time and indexes can change that conclusion. Full regional replication may be useful for this small logical dataset; placement and acknowledgement must satisfy the outage contract. Random codes with atomic uniqueness checks can be correct, as can a counter with collision-free allocation; wrong-resolution prevention does not force a monotonic counter. Preserve tombstones or another no-reuse policy.

Failure test: create in region A, lose A before replication, then resolve in B. If the requirement promises visibility of every acknowledged creation, asynchronous replication alone cannot meet it. Also test expired cached redirects and no-reuse behavior.

Case B — Group chat

Requirements: send, receive, fetch history and report delivery state. Define per-room ordering and what “delivered” means. Suppose durable acceptance must survive one zone loss, while historical reads may lag by seconds. Offline devices need cursor-based replay and deduplication.

Numbers: 10 million DAU × 40 messages/day = 400 million writes/day, 4,629.63/s average and 13,888.89/s at 3×. If room size eight includes the sender, seven other recipients imply about 97,222 deliveries/s at peak; including a sender echo yields 111,111.11/s. At 1 KB logical bytes/message, growth is 400 GB/day and 146 TB/year before overhead. DAU is not simultaneous socket count.

Choice: partitioning/retention deserve early analysis, but one year's volume does not prove partitioning is required on day one. room_id is a natural ordering key; a very hot room can still overload one partition. Separate connection handling when memory/churn and message processing scale differently. A durable replicated log can itself be the commit/acknowledgement boundary; “never acknowledge an enqueue” incorrectly excludes such logs.

Failure test: crash after durable append but before client acknowledgement. A retried message ID must not become two visible messages. Test sequence recovery, offline replay and the explicitly supported failure domain.

Case C — News feed

Requirements: post, follow and render ranked candidates. Assume feed p95 below 300 ms and up to one minute of freshness lag for ordinary posts, while deletion/privacy enforcement has a stricter policy. Whether dropping a post is acceptable is a product decision, not a universal characteristic of feeds.

Numbers: 100 million DAU × 20 loads/day = 2 billion reads/day, 23,148.15/s average and 69,444.44/s at 3×. Five million posts/day is 57.87/s average; the read/post ratio is 400:1. A single author with ten million followers can create ten million materialized-feed updates.

Choice: the freshness allowance makes precomputation possible, but does not mandate it. Compare push cost for ordinary authors with pull/merge cost for high-fanout authors and active readers. A hybrid can bound fanout work, provided read-time merge, deduplication and pagination meet the latency budget. Cache authority must not override blocked/deleted content policy.

Failure test: a celebrity post arrives during a worker outage. Measure queue age and candidate freshness; define how reads degrade and how backlog drains. Delete that post and test both stale feed IDs and media access.

Case D — Metering and billing

Requirements: ingest usage, aggregate by account/period and issue reproducible invoices. Duplicate delivery must not double-count; accepted usage must survive the named failures. Aggregation may lag by minutes. Late corrections and policy/rate changes need an auditable invoice revision process.

Numbers: 2 billion events/day gives 23,148.15/s average, 69,444.44/s at 3× and 400 GB/day at 200 logical bytes/event. The throughput resembles the feed, but the invariant is different.

Choice: retain stable event identity, a durable source record, versioned aggregation/rate rules and transactionally guarded accounting effects. “Exactly once” is an effect at a named boundary, not a magic property of every message transport. Replaying raw events must not create duplicate charges. Retention and privacy rules determine what evidence can be retained and for how long; immutable does not mean lawless indefinite storage.

Failure test: crash between receiving an event and advancing a consumer checkpoint. Reprocess and prove totals unchanged; apply a late correction with an auditable version instead of mutating a historical invoice silently.

The requirement that decides the shape

Throughput constrains resources; correctness, observation and freshness constrain permitted behavior. Both affect topology. One minute of staleness can permit asynchronous views and aggressive caching. Zero-staleness requirements do not prohibit caches or batching; those mechanisms must participate in the relevant consistency protocol.

The original numeric staleness/cost plot was an illustrative curve, not measured evidence. Its useful mechanism is conditional: relaxing freshness can let many reads reuse one computation. The gain depends on update rate, key popularity, invalidation and recomputation cost, not a universal 1,000-to-4 cost ratio. Ask what breaks at one second, ten seconds and sixty; preserve the answer as a product decision.

Tracing the boxes back

MechanismRequirement it may satisfyA test that can falsify the choice
Edge cacheReusable content near usersCold misses or invalidation violate the latency/revocation budget
Service admissionBounded expensive workRetry burst exceeds CPU/memory or dependency quota
Durable storeDefined acknowledgement and recoveryA supported failure loses an acknowledged write
Read cache/replicaEligible read latency and scaleReturned version violates consistency or authorization
Queue/workersDeferred work and recoveryBacklog age exceeds deadline or duplicate delivery repeats an effect
Backup/restoreRecovery from corruption/deletionRestore misses RPO/RTO or lacks required keys/configuration

The forward trace asks why a component exists. The reverse trace asks where every requirement is enforced. A diagram with no answer for deletion, accessibility or auditability is incomplete even if its QPS arithmetic is correct.

Numbers worth memorising

Keep exact unit anchors: 86,400 seconds/day; 1 million/day ≈ 11.57/s; 100 million/day ≈ 1,157.41/s; 1 billion/day ≈ 11,574.07/s. Dividing by 100,000 underestimates by 13.6%. One KB × one million/day is one GB/day in decimal units, or 365 GB per 365-day year.

Peak factors, SSD latency, network round trips and safe TB/node are measurements or scenario inputs, not constants. Physical propagation imposes a distance-dependent lower bound; a claimed 150 ms cross-continent RTT is not a universal law. Benchmark the relevant path and include protocol/queueing/processing time. A cache or replica can reduce remote trips only for operations whose consistency/security contract permits it.

For n independent calls with a 1% probability of exceeding a threshold, the chance of at least one exceedance is 1 - 0.99**n: about 9.56% for ten, 18.21% for twenty. Hedging can reduce some tails but adds work and may worsen overload; apply budgets and cancel redundant work where possible.

Reproducible arithmetic and fanout assumptions

from fractions import Fraction
 
per_day = 86400
assert round(500_000_000/per_day*3, 2) == 17361.11
assert 1_000_000 * 500 * 365 == 182_500_000_000
messages_peak = Fraction(10_000_000*40*3, per_day)
assert round(float(messages_peak*7), 2) == 97222.22
assert 400_000_000*1000*365 == 146_000_000_000_000
assert 2_000_000_000/5_000_000 == 400
assert round(1-0.99**20, 4) == 0.1821
assert round(1-0.99**10, 4) == 0.0956
assert abs((1-86400/100000)-0.136) < 1e-12
print("four-case capacity, recipient count and independent-tail arithmetic passed")

This validates arithmetic, not independence, workload realism or machine capacity. For fully correlated call failures, the chance of at least one exceedance can remain 1%; different dependence structures yield different page behavior.

Where this frame breaks

Correctness-first systems: start with invariants, audit and failure models before QPS. Hard real-time systems: worst-case execution, scheduling and bounded communication may matter more than a percentile; not all trading/control systems share one requirement, and queues/caches are not categorically disqualified. Unknown products: choose a simple reversible design and observable migration triggers rather than inventing precision. Small systems: one host can be a valid answer if availability/recovery constraints permit it; small load does not waive those constraints.

Companion deep report

The public lesson now includes the four worked cases and arithmetic directly. A private companion PDF is not required to establish coverage, and its availability/content was not verified here. An optional resource-link decision belongs to the parent editor, not to this lesson's technical acceptance.

Practice and worked answers

  1. “No lost messages”—which four questions? Which acknowledgement means accepted, which failures must be survived, how long history is retained, and how retries/offline delivery reconcile.
  2. Does 182.5 GB/year prove no sharding is needed for years? No. It is logical storage only; throughput, indexing, recovery, overhead and growth may bind sooner.
  3. Does a 400:1 feed ratio mandate fanout-on-write? No. Follower distribution, active readers, freshness and write amplification decide; compare a hybrid under explicit workloads.
  4. Does every component need a capacity number? No. Security, auditability and failure-domain requirements can justify it. It still needs a traceable constraint and verification plan.
  5. How should the four-case review be assessed? Check six dimensions: bounded behavior, honest arithmetic, clear acknowledgement, consistency/access policy, failure recovery and falsifiable tests. For each, identify a concrete missing decision rather than awarding points for vendor names.

What to carry forward

Bound the problem, estimate what constrains it, trace operations through the architecture and examine the weakest contract. Then reverse the trace: every requirement needs an enforcing mechanism and a test. The numerical program runs locally; no regional service, database fleet, traffic benchmark or production review was reproduced.

Sources and verification boundary

Primary documentation was reviewed against saved raw snapshots on 8 October 2026. Publication review date follows the corrective contract (7 October 2026); the evidence records retain actual access timestamps. Tagged/versioned sources bind API claims to those versions; mutable sources are access-date snapshots. Local programs and design scenarios are separate from HTTP evidence.

No paid service, model download, cloud write, network security probe or deployed service was used. Scoped source compilation and offline static rendering are not global application, browser-interaction or release acceptance.

Pause / Recall / Apply

Can you explain it without the page?

Close the example. Reconstruct the core idea, then change one assumption. Mark complete when you’re ready; you can always undo it.

Stored in this browser only. No account, no sync. Clearing browser data removes your record.