Designing for Scale · Estimation That Constrains

The problem this post solves

Capacity estimation has a reputation problem. Done badly it is theatre: a column of impressive-looking multiplications that produces a number nobody uses, followed by an architecture that would have been drawn identically without it.

Done well it is the opposite of theatre. It is the mechanism that converts an argument about taste into an argument about arithmetic. "I think we need a cache" is a preference. "The working set is roughly forty gigabytes and the read rate is fifty thousand per second, so serving from disk would need this many spindles" is a constraint, and constraints end debates.

The distinguishing test is brutal and worth applying every single time: a number is worth computing only if a different answer would have changed a decision. Everything else is decoration.

First principles

A small set of anchors makes the arithmetic fast enough to do out loud. A day has 86,400 seconds, which is close enough to 100,000 that dividing daily volume by 100,000 gives a usable per-second rate — deliberately slightly low, which is then corrected by the peak factor. A million seconds is roughly eleven and a half days. A year is roughly 30 million seconds.

Powers of two matter for sizing: a thousand is roughly 2 to the tenth, a million roughly 2 to the twentieth, a billion roughly 2 to the thirtieth. So a billion records at a hundred bytes each is roughly a hundred gigabytes, and that calculation should take seconds rather than a spreadsheet.

The core model

Four quantities carry almost all the decision-making weight. Each one has a specific choice hanging off it, and computing one without naming its choice is the decorative failure mode.

Requests per second

Take daily active users, multiply by actions per user per day, divide by seconds in a day. That gives the average. Nobody serves the average.

Real traffic has a diurnal shape, and the ratio of peak to average is the number that actually sizes the fleet. For a product concentrated in one time zone this ratio is commonly in the range of two to four; for a globally distributed product it is flatter. The important discipline is to state which you assumed and why, because a design provisioned to average will be under-provisioned during every peak, and peak is when failures are most expensive.

The read/write split then divides that total, and the two halves are sized entirely differently. Reads scale horizontally with replicas and caches, cheaply. Writes scale with partitioning and coordination, expensively. A system with a hundred-to-one read/write ratio and one with a one-to-one ratio at the same total throughput are not remotely the same engineering problem.

Storage growth

Writes per day, times bytes per write, times retention. The trap is bytes per write: it must include indexes, replication factor and metadata, not just the payload the user typed.

A record whose visible content is a hundred bytes may consume several times that once identifiers, timestamps, index entries and a replication factor of three are accounted for. Estimating storage from payload alone routinely undercounts by a factor of three to ten, which is exactly the kind of error that moves the answer across a decision boundary.

Retention is where unbounded requirements hide. "Keep everything forever" turns a bounded storage figure into a permanently growing one and quietly commits to a cost that compounds. If nobody has stated a retention window, that is a requirements gap, not an estimation detail.

Bandwidth

Requests per second times average response size. This is the number that decides whether static and media content can be served from the origin at all, or whether an edge tier is load-bearing rather than an optimisation.

Bandwidth is also where media features detonate budgets. Text responses live in the kilobyte range; images in the hundreds of kilobytes; video in megabytes per second of playback. A feature that adds media to a text product can raise bandwidth by three orders of magnitude while leaving requests per second almost unchanged — which is why sizing on request count alone is dangerous.

Working set

Total data is rarely the interesting number. The interesting number is how much of it is hot enough to be requested frequently, because that is what a cache must hold to be effective.

Access in most systems is heavily skewed: a small fraction of keys attracts a large fraction of traffic. That skew is what makes caching economically viable, and its steepness determines the cache tier size. A cache sized to the hot fraction can serve the large majority of reads from memory; a cache sized to a small fraction of the hot set thrashes and returns little benefit for its cost.

The estimate that matters is therefore not "how much data is there" but "how much data must be resident to satisfy the target hit rate", and the honest version of that sentence includes the fact that the answer depends on a skew assumption which should be stated and later measured.

Capacity and constraints

The output of estimation should be a short table that a reviewer can attack. Every row pairs a quantity with the decision it forces, because that pairing is what makes the estimate reviewable rather than ornamental.

The rows worth having are usually: peak requests per second, and therefore the size and shape of the service tier; storage at retention, and therefore whether the data fits on one node or must be partitioned; bandwidth, and therefore whether an edge tier is required; and working set, and therefore the cache size and its cost.

Anything that does not force a decision belongs in a footnote or nowhere.

The second output, which is more valuable and more often skipped, is a sensitivity note: which assumption is the design most fragile to. If halving the read/write ratio would invalidate the caching strategy, that sentence belongs in the document. It converts a hidden fragility into a monitored one and tells whoever operates the system later which metric is actually load-bearing.

That third option deserves emphasis because it reframes the whole exercise. The question is not "what is the right number" — it is "over what range of numbers does this architecture remain correct". A design that holds across an order of magnitude is robust. A design that only works at one specific point estimate is fragile, and knowing that is worth more than the point estimate itself.

Architecture

Estimation feeds the design at four specific joints, and it is worth being explicit about which number lands where.

The right-hand column is the architecture. Every arrow is a justification. When a reviewer asks why the cache tier is sized as it is, the answer is a number and an assumption, not a preference.

Failure modes

Estimating without deciding. The most common failure: a page of arithmetic followed by an architecture that ignores it. The fix is mechanical — write the consequence next to every number, and delete numbers with no consequence.

False precision. Carrying many significant figures through a chain of guesses. It signals to reviewers that the estimator does not know which parts are uncertain, which undermines the numbers that are actually solid.

Sizing on averages. Provisioning to mean load guarantees under-provisioning at peak, which is when outages are most visible and most costly.

Forgetting the multipliers. Replication factor, index overhead and metadata routinely multiply naive storage estimates several-fold. Omitting them produces answers that are wrong in the direction that looks cheapest, which is the direction most likely to be believed.

Unbounded retention. Treating "keep everything" as a given rather than as a requirement with a cost attached. It converts a fixed storage number into a compounding one.

Ignoring the tail of the distribution. Estimating with means when the underlying distribution is heavily skewed. Hot keys, celebrity accounts and viral items are not outliers to be waved away; they are usually the specific case the architecture must survive, and averaging them out removes the hardest part of the problem from the analysis.

Trade-offs that matter

Time spent estimating is time not spent designing, and both are bounded in any real review. The resolution is to estimate only along the axes where the design might change category.

If the system is obviously small — a few thousand users, kilobytes of data — the correct estimate is one sentence saying so, followed by a single-node design and an explicit note about the scale at which that stops being true. Elaborate arithmetic to prove a small system is small is a waste, but the note about where it breaks is not.

If the system is obviously large, the estimate should focus on the boundaries: at what point does the data stop fitting on one node, at what point does the read rate exceed what replicas can absorb, at what point does bandwidth force an edge tier. Those thresholds are more durable than any point estimate, because they remain true as the product's actual numbers move around inside them.

There is also a tension between honesty and confidence. An estimate hedged into uselessness helps nobody, but one presented with unearned certainty is worse, because it will be quoted back later as though it were measured. The workable middle is to state the number, state the assumption it rests on, and state what would have to be true for it to be badly wrong.

What to carry forward

Estimation is not about being right. It is about being constraining — producing numbers that rule things out, at a precision matched to the uncertainty of the inputs, fast enough to try several scenarios.

Three habits capture most of the value. Compute to one significant figure. Write the forced decision beside every number. Name the assumption the design is most sensitive to.

The next question the numbers raise is where the data should actually live, since storage growth and access pattern together do most of the work in narrowing that choice.