Capacity estimates that survive an overload question
Derive traffic, storage, bandwidth, working set and backlog with explicit units, sensitivity analysis and exact retained laboratory programs.
Original publication date unrecordedUpdated
Lesson preparation & details
Level: intermediate
By the end, you should be able to
- Estimate traffic, storage, transfer and cache working set
- Separate peak load, headroom and queue recovery
- Build a decision ledger with sensitivity ranges
- Test stable-flow arithmetic without claiming a load benchmark
Bring with you
Sources checked: . A source check does not change the publication or update date.
Editorial review: · What review means
In this article · 18 sections
The problem this post solves
Capacity arithmetic earns its place when a different answer could change a decision. “We need a cache” is a preference; “the eligible hot objects need 40 GB including overhead, and origin read work exceeds the tested budget” is a hypothesis that can be checked. An estimate is not a benchmark, and a long page of multiplication is not an architecture.
This lesson retains the handbook-service worksheet and queue program, then develops the broader original questions: read/write work, retention and replication, bandwidth, cache hit-rate curves, uncertainty and architectural thresholds. Every scenario number is an assumption unless identified as a unit conversion.
First principles
At design time, some inputs are unknown, but existing systems, prototypes and vendor limits may still provide measurements. Use those where relevant; rough arithmetic does not inherently beat measurement. Its advantage is letting you vary assumptions cheaply before committing to an expensive implementation.
Keep enough precision internally to check units and boundaries, then present a sensible range. One significant figure is often useful for a whiteboard estimate, but not for an integer worker ceiling, a quota or a tight budget. A 99.9% availability objective must not be rounded to 100%. Decisions can change at any operational limit, not only powers of ten.
A day has 86,400 seconds. Dividing daily volume by 100,000 instead underestimates the exact average by 13.6%. Do not silently claim that an unrelated peak multiplier corrects this approximation. One million seconds is about 11.57 days; a 365-day year has 31,536,000 seconds. Decimal GB means 10^9 bytes; GiB means 2^30 bytes. Powers of two are useful mental anchors, not interchangeable unit definitions.
The core model
Figure text equivalent — labels and relationships
- Node “Active population and background jobs” (square).
- Node “Actions by operation” (square).
- Node “Average and peak operation rates” (square).
- Node “Retained writes and storage overhead” (square).
- Node “Payload bandwidth and fanout” (square).
- Node “Occupied time and work in flight” (square).
- Node “Access distribution and hot objects” (square).
- Node “Cache capacity and miss load” (square).
- Node “Decision ledger and sensitivity tests” (square).
- “Active population and background jobs” leads to “Actions by operation”. Line: normal; ends: arrow point.
- “Actions by operation” leads to “Average and peak operation rates”. Line: normal; ends: arrow point.
- “Actions by operation” leads to “Retained writes and storage overhead”. Line: normal; ends: arrow point.
- “Average and peak operation rates” leads to “Payload bandwidth and fanout”. Line: normal; ends: arrow point.
- “Average and peak operation rates” leads to “Occupied time and work in flight”. Line: normal; ends: arrow point.
- “Retained writes and storage overhead” leads to “Access distribution and hot objects”. Line: normal; ends: arrow point.
- “Access distribution and hot objects” leads to “Cache capacity and miss load”. Line: normal; ends: arrow point.
- “Payload bandwidth and fanout” leads to “Decision ledger and sensitivity tests”. Line: normal; ends: arrow point.
- “Occupied time and work in flight” leads to “Decision ledger and sensitivity tests”. Line: normal; ends: arrow point.
- “Cache capacity and miss load” leads to “Decision ledger and sensitivity tests”. Line: normal; ends: arrow point.
Requests per second
For operation i, average arrival rate is active users times actions per user per day divided by 86,400, plus relevant scheduled, retry and system-generated work. Registered users matter for durable account storage; concurrent users matter for connection memory. DAU alone does not describe either.
The handbook scenario has 20,000 DAU and twelve questions each per day: 240,000 requests/day or about 2.78/s. An assumed peak factor of six produces 16.67/s. Peak depends on measurement interval and workload shape: a one-second burst and a five-minute average peak are different inputs. There is no universal two-to-four multiplier and no law that peak/mean can never be near 1.2.
Split operation types before sizing. If 95% of 1,000 requests/s are cache-eligible reads and 5% are writes, the initial rates are 950 reads/s and 50 writes/s. At an 80% hit rate, the read origin still sees 190 reads/s, plus all write work and cache-fill/invalidation work. Reads may be expensive scans; writes may be cheap append-only records. “Reads cheap, writes expensive” is not a capacity model.
A margin on average can accidentally cover a peak, but it hides the actual assumption. In a product peaking at 3× mean, provisioning 1.2× mean supplies 40% of peak demand before any spare-capacity target. State the peak scenario and utilization target separately.
Storage growth
For a retained event stream, start with writes/day × retained days × logical bytes/write. Then itemize metadata, indexes, replica copies, logs, compression, backups, temporary compaction and rebuild space. Multipliers belong to the boundary they actually describe: a measured physical-store size may already include indexes or replicas. Multiplying it again double-counts.
In the protected worksheet below, 600 bytes/event, 30 days and three copies give 12.96 GB. That intentionally excludes other overhead. For an expanded scenario, suppose each logical event has 100 bytes of metadata and a 200-byte index entry per copy. Then 240,000 × 30 × (600 + 100 + 200) × 3 = 19.44 GB, before backups and temporary space. Compression should be estimated separately for structures with different compressibility.
“Keep forever” creates linear raw growth under constant ingestion, not inherently compound growth. Rising ingestion can accelerate it. Require explicit retention, deletion and recovery objectives; raw events, aggregates and backups may have different retention rules. A storage total does not alone decide partitioning: write throughput, hot keys, index size and restore duration may dominate.
Bandwidth
Multiply rates by payload sizes at a named link. The handbook response path is 16.67/s × 12,000 bytes ≈ 200,000 bytes/s, or 1.6 megabits/s before protocol overhead. If a question fans out to three 100 KB internal responses, the internal network budget differs from the client response budget. Uploads, replication, cache fills and backup/rebuild traffic are separate flows.
Media can increase payload substantially without increasing request count. A hypothetical 12 KB response becoming a 12 MB object multiplies transferred bytes by 1,000. That is a scenario, not a universal text/image/video size rule. A CDN may reduce origin bandwidth and geographic latency for eligible reusable objects; private or rapidly changing responses may require different cache policy. Bandwidth alone does not prove that a CDN is mandatory.
Working set
Cache sizing asks how many bytes must remain resident to achieve a target hit rate for the actual access distribution. A “hot 1%” without object sizes, time window or hit-rate target is incomplete. Include key/entry overhead, allocator slack, replication and churn. A workload with uniform one-time reads may gain little from caching even if the entire corpus fits.
Suppose ten equally sized objects account for 70% of requests and ninety others account for 30%. An ideal static cache holding the first ten achieves at most that 70% share under the stated distribution. It is not a guarantee for LRU under churn or a system where large objects distort byte occupancy. A trace-based hit-rate/size curve makes the assumption inspectable; popularity shifts and tenant isolation change it.
Translate users into work
If a successful handbook request occupies one worker for a mean 0.24 seconds, mean work in flight at the assumed peak is 16.67 × 0.24 = 4. Under a stable flow with consistent boundaries, Little's law relates long-run average population, throughput and time. It is not a p99 concurrency formula or a model of an unbounded overload.
Four single-request workers sit at the ideal capacity boundary. At a target utilization of 60%, the worksheet asks for ceil(4 / 0.60) = 7. This is a starting point for representative testing, not a host recommendation. CPU time, blocked I/O, service-time variance and dependency limits determine whether the worker abstraction is meaningful.
A reproducible worksheet
The exact existing program is retained. It reports logical event payload with three copies, not a complete infrastructure bill.
from math import ceil
users, questions, peak_factor = 20_000, 12, 6
seconds_per_day = 86_400
daily = users * questions
average_rps = daily / seconds_per_day
peak_rps = average_rps * peak_factor
mean_seconds = 0.24
in_flight = peak_rps * mean_seconds
workers = ceil(in_flight / 0.60)
storage = daily * 600 * 30 * 3
response_bytes_per_second = peak_rps * 12_000
assert round(peak_rps, 2) == 16.67
assert round(in_flight, 6) == 4
assert workers == 7
assert storage == 12_960_000_000
assert round(response_bytes_per_second) == 200_000
print(workers, f"{storage / 10**9:.2f} GB", f"{storage / 2**30:.2f} GiB")Do not silently exchange bits and bytes or decimal and binary units. The 12.96 GB total is about 12.07 GiB. If the platform bill charges transfer, requests and retained bytes separately, the unit labels also determine which costs can be compared.
Queues buy time, not capacity
Accepted work arriving at 30 jobs/s against 20 completions/s accumulates 600 jobs in one minute. If arrivals fall to 10, spare capacity is ten jobs/s and recovery takes another minute. At arrival equal to service, the existing backlog never drains in this fluid model.
def drain_seconds(backlog, arrival, service):
if min(backlog, arrival, service) < 0:
raise ValueError("nonnegative rates and backlog required")
if backlog == 0:
return 0.0
return backlog / (service - arrival) if service > arrival else float("inf")
backlog = (30 - 20) * 60
assert backlog == 600
assert drain_seconds(backlog, 10, 20) == 60
assert drain_seconds(backlog, 20, 20) == float("inf")
print(backlog, drain_seconds(backlog, 10, 20))The protected function assumes finite real-valued rates and a finite backlog after its nonnegative check; it is not an input-validation API for NaN, infinity or arbitrary types. Its simple constant-rate model does not capture batch service, priorities or service-time variance.
Queue limits must cover count, bytes and age. At 20 completions/s, a 600-job FIFO backlog implies about 30 seconds of work ahead of a newly admitted job if arrivals do not overtake it. That may be unacceptable for interactive answers even though storage can hold the queue. Bound retries too: a retry storm increases offered work while the original queue is already full. Background reindexing and an interactive answer need different admission policies.
Capacity and constraints
A decision ledger pairs assumptions with a threshold and a verification action:
| Quantity | Scenario result | Consequence and test |
|---|---|---|
| Peak request rate | 16.67/s | Test request-cost distribution, not only this rate |
| Mean occupied work | 4 slots | Start with seven single-request workers at 60% modeled utilization; measure tails |
| Retained event storage | 12.96 GB payload copies; 19.44 GB expanded scenario | Check indexes/logs/backups and restore time before selecting a storage tier |
| Client response traffic | 1.6 Mb/s payload | Add protocol/fanout/burst costs; no automatic CDN conclusion |
| Queue recovery | 600 jobs / 10 spare jobs/s = 60 s | Compare with oldest-job and user-deadline budgets |
| Cache | Determined by hit-rate/byte curve | Measure origin miss work and invalidation rather than guessing a hot percentage |
For uncertain inputs, optimistic-only estimates are defensible only with cheap reversible scale-out and explicit failure acceptance. Pessimistic-only estimates may protect a severe downside but waste money and complexity. Usually carry a low/base/high range. If the same architecture works throughout it, say so; if it changes category, identify the trigger and lead time for the change.
Sensitivity, working set and queue stability
This additional complete program varies a genuinely load-bearing parameter and computes a small ideal cache bound. The original worksheet remains unchanged above.
from math import ceil
def worker_count(rate, occupied, utilization):
if rate < 0 or occupied < 0 or not 0 < utilization < 1:
raise ValueError("nonnegative work and utilization between zero and one required")
return ceil(rate * occupied / utilization)
def ideal_hit_rate(counts, entries):
if type(entries) is not int or entries < 0 or any(c < 0 for c in counts):
raise ValueError("nonnegative counts and integer capacity required")
total = sum(counts)
return sum(sorted(counts, reverse=True)[:entries])/total if total else 0.0
assert [worker_count(50/3, s, 0.60) for s in (0.12, 0.24, 0.48)] == [4, 7, 14]
assert worker_count(2000, 0.05, 0.50) == 200
counts = [70]*10 + [10/3]*90
assert abs(ideal_hit_rate(counts, 10)-0.70) < 1e-12
assert ideal_hit_rate(counts, 0) == 0
assert ideal_hit_rate([], 10) == 0
assert 240_000 * 30 * 900 * 3 == 19_440_000_000
assert abs(1-86_400/100_000-0.136) < 1e-12
print("service-time sensitivity: 4, 7, 14 workers; ideal hot-ten hit rate: 70%")The most fragile assumption here is occupied time: doubling it doubles modeled work and takes the worker ceiling from seven to fourteen. In another system the binding assumption may be retention, per-item size or skew. Record that in monitoring and capacity review rather than leaving it buried in a spreadsheet.
Architecture
Figure text equivalent — labels and relationships
- Node “Peak rate and work cost” (square).
- Node “Service concurrency and admission” (square).
- Node “Retained bytes and restore time” (square).
- Node “Storage layout and recovery plan” (square).
- Node “Transfer by link and geography” (square).
- Node “Origin and edge policy” (square).
- Node “Hit-rate curve and staleness” (square).
- Node “Cache memory and invalidation” (square).
- Node “Representative workload tests” (square).
- “Peak rate and work cost” leads to “Service concurrency and admission”. Line: normal; ends: arrow point.
- “Retained bytes and restore time” leads to “Storage layout and recovery plan”. Line: normal; ends: arrow point.
- “Transfer by link and geography” leads to “Origin and edge policy”. Line: normal; ends: arrow point.
- “Hit-rate curve and staleness” leads to “Cache memory and invalidation”. Line: normal; ends: arrow point.
- “Service concurrency and admission” leads to “Representative workload tests”. Line: normal; ends: arrow point.
- “Storage layout and recovery plan” leads to “Representative workload tests”. Line: normal; ends: arrow point.
- “Origin and edge policy” leads to “Representative workload tests”. Line: normal; ends: arrow point.
- “Cache memory and invalidation” leads to “Representative workload tests”. Line: normal; ends: arrow point.
Each arrow is a conditional justification, not a mandatory product. Availability may justify replicas even when capacity does not. Security or a failure-domain requirement may justify a component without a throughput number. The decision ledger should contain those qualitative constraints too; the original rule that every choice without a number is unjustified was too narrow.
Failure modes
Estimating without deciding: arithmetic never reaches a threshold or acceptance test. Add its consequence or stop computing it. False precision: five-digit outputs conceal factor-of-three inputs. Present ranges and retain exact units. Sizing on averages: bursts or expensive requests exhaust the fleet. Test arrival and cost distributions together. Forgetting multipliers: indexes/backups/temporary space exceed the payload calculation. Account by storage boundary rather than a magic multiplier. Unbounded retention: the design commits to indefinitely increasing cost without an owner. Averaging away skew: one celebrity key or tenant overloads a shard while aggregate utilization looks safe.
Trade-offs that matter
A genuinely small service may fit one machine with headroom, but recovery, security and availability still need explicit treatment. A large service should focus on operational boundaries: when the primary cannot meet write latency, when restore takes too long, when origin misses exceed a dependency quota, and how quickly scale-out becomes available. Those thresholds are workload/hardware contracts, not universal rules such as “shard above five TB.”
An honest estimate says: this is the current scenario, this is the uncertain input, this is the decision, and this observation would invalidate it. It need not hedge every sentence into uselessness or pretend that an unmeasured capacity is established fact.
Exercises
- Double mean occupied time to 0.48 s. Work in flight becomes eight; divide by 0.60 and round up to fourteen workers.
- Drain 600 jobs at arrival 19/s and service 20/s. Recovery takes 600 seconds. A small net spare rate produces a long recovery despite being below peak.
- Only 16.67 requests/s but memory is exhausted—how? Large per-request buffers, model state, connection state or an unbounded queue can dominate memory. RPS does not specify bytes in flight.
- At 2,000/s and mean occupancy 0.05 s, what is average concurrent work? One hundred slots under stable-model assumptions; at 50% target utilization, 200 modeled slots. That does not assert 200 threads or a p99 result.
- The cache hit rate is 90%, so origin needs only 10% of old capacity? Only if misses have comparable cost, writes/background work are unchanged and cold-cache/recovery scenarios are acceptable. A miss can be much more expensive than an average pre-cache request.
What to carry forward
Derive operation rates, retained bytes, transfer and hot-set occupancy with units; keep peak and spare capacity separate; name the most sensitive assumption; and test the architectural boundary it affects. The programs verify arithmetic and simple queue/cache models. They do not measure a production service, host capacity or cloud cost.
Sources and verification boundary
Primary documentation was reviewed against saved raw snapshots on 8 October 2026. Those source-access timestamps describe evidence provenance, not the article's original publication date. Tagged/versioned sources bind API claims to those versions; mutable sources are access-date snapshots. Local programs and design scenarios are separate from HTTP evidence.
- Google SRE: Load Balancing with Client Side Throttling — SRE Book, 2016.
- Google SRE - Continuous Improvement To Get Reliability — SRE Workbook, 2018.
No paid service, model download, cloud write, network security probe or deployed service was used. Scoped source compilation and offline static rendering are not global application, browser-interaction or release acceptance.
Pause / Recall / Apply
Can you explain it without the page?
Close the example. Reconstruct the core idea, then change one assumption. Mark complete when you’re ready; you can always undo it.
Stored in this browser only. No account, no sync. Clearing browser data removes your record.