Probability — Random Variables, Distributions, Expectation

Review boundary and corrected contracts

Bayes requires a positive evidence probability; the rare-disease example yields exactly one half for the given rates. The elementary CLT is for iid observations with finite nonzero variance. Zero covariance is sufficient for variance additivity without independence. Distribution names are modeling assumptions, not laws for every arrival process.

The examples below teach the stated input contracts; fragments and deliberately incorrect counterexamples are exercises, not production implementations. Complexity counts elementary operations unless a bit-cost or allocation caveat is stated. Practice times are suggestions, not measured performance or hiring guarantees.

Learning objective: Model an uncertain outcome as a random variable, pick the right distribution, compute expectation and variance, and apply Bayes' rule to a real diagnostic-test problem.

Why this session exists

Every modern engineering discipline is drowning in uncertainty. A/B tests, ML model outputs, load forecasts, tail latencies, credit-card fraud detection, LLM sampling — all of them are probability calculations wearing engineering clothes. The difference between "I have a feeling this is a lot" and "the p95 is 340 ms with 95% CI [335, 345]" is fluency in random variables, distributions, and Bayes' rule. This session gets you the vocabulary, and — crucially — the intuition to spot when someone (or you) is using it wrongly.

You will be able to

• Define a random variable, its distribution, and its PMF/PDF/CDF — and translate between them.

• Recognise Bernoulli / binomial / normal / exponential / Poisson in the wild and pick the right one.

• Compute expectation E[X] and variance Var(X) — both from formula and by simulation.

• State and apply Bayes' rule to update beliefs given evidence (the medical-test canonical example).

• Explain the Central Limit Theorem in one sentence and know why it underlies everything from confidence intervals to A/B tests.

Prerequisites

• S016 · Discrete math — counting/combinatorics underlies discrete probability.
• S017 · Vectors — you'll compute expected values as dot products with probabilities.
• High-school algebra, integration is optional (we'll use NumPy for anything continuous).

Pre-read · skim before the session (35 min total)

Further reading: Seeing Theory — a visual introduction to probability and statistics

Further reading: Arbital · Bayes' rule guide

Further reading: numpy.random — the modern Generator API

(a) Intuition · 5 min

A random variable is a labelled roulette wheel

A random variable X maps an outcome ω in a probability sample space to a number: X(ω). Its distribution is the induced probability rule, not X itself. For a roulette-like prize model, X may pay 0 with probability 3/4 and 4 with probability 1/4. Then E[X]=1 and Var(X)=3. Mean and variance summarize this model but do not determine all tail probabilities.

Calling rng.binomial(n=10, p=0.3) draws one sample from a specified distribution. A generator is a computational sampler, not the mathematical definition of a random variable. Simulated averages can approach theoretical moments under suitable assumptions; a finite run never proves equality.

The five distributions you'll see 100× a year

Meet the family

• Bernoulli(p) — one flip of a biased coin. Outcome 0 or 1. Foundation of every yes/no event.

• Binomial(n, p) — n independent same-p trials; count successes. Clicks/defects fit only if these assumptions are reasonable.

• Poisson(λ) — nonnegative integer count with mean λ. A homogeneous Poisson process models independent increments at constant rate; bursts or dependent earthquakes need not fit it.

• Normal(μ, σ) — a bell-shaped continuous model, with σ the standard deviation in this lesson (not variance). It can approximate standardized sums under CLT assumptions, not every measurement.

• Exponential(λ) — waiting time in a homogeneous Poisson process of positive rate λ. Requests/failures fit only if that process assumption is justified.

Correlation is not independence, and independence is not causation

Independence means joint probabilities factor into marginal probabilities. Zero covariance is weaker. For X uniform on −1,0,1 and Y=X², Cov(X,Y)=0 by symmetry, yet Y is determined by X. Sampling design, clustering and time dependence decide which independence assumptions are plausible; neither correlation nor independence alone establishes causation.

A quick history

The historical topics form a useful map: Pascal–Fermat gambling questions motivate counting outcomes; Bayes and Laplace motivate inference from evidence; limit theorems explain standardized sums; Kolmogorov's measure-theoretic framework formalizes probability spaces and random variables. Numerical posterior computation, including Markov chain Monte Carlo, makes models with difficult normalizers practical, but correlated MCMC draws are not automatically iid observations. Modern token sampling draws from a model's categorical distribution; temperature rescales logits and top-k/top-p truncate then renormalize. These mechanisms—not claims of sole invention, “ignored for 150 years,” or universal probabilistic explanations—are the point of the timeline.

(b) Describing outcomes and updating beliefs · 15 min

The distribution families above describe different kinds of uncertainty. Before choosing one, separate two questions: what values can the outcome take, and how do we turn its distribution into a probability?

PMF vs PDF vs CDF

Follow the branch for a count, such as clicks, or for a continuous measurement, such as waiting time. The branches meet at the CDF because either kind of variable can answer a threshold question: how likely is an outcome at or below this value?

The distinction to carry forward is probability at a point versus density over an interval. A discrete PMF assigns probability to each possible value; for a continuous variable, probability comes from area under the PDF, not the curve's height. The CDF accumulates either kind up to a threshold. The diagram covers discrete and absolutely continuous cases; mixed distributions can contain both point masses and a density, and a CDF always exists even when no PDF does.

For X uniform on [0,0.5], f(x)=2 inside the interval: a density may exceed 1, but P(0.1≤X≤0.3)=2(0.2)=0.4 and P(X=0.2)=0. For a discrete Bernoulli(0.3), F(x)=0 below 0, 0.7 from 0 inclusive to 1 exclusive, and 1 at or above 1. PMF jumps and PDF areas both agree with F(b)−F(a)=P(a<X≤b).

The five distributions at a glance

Distribution cheat sheet

Name: Bernoulli(p)

Desc: P(X=1)=p, P(X=0)=1-p. Mean=p, Var=p(1-p). Example: does this email open?

Name: Binomial(n, p)

Desc: Sum of n independent Bernoulli(p) with the same p, integer n≥0 and 0≤p≤1. P(X=k)=C(n,k)pᵏ(1−p)ⁿ⁻ᵏ, with boundary cases interpreted by the distribution. Mean=np, Var=np(1−p).

Name: Poisson(λ)

Desc: P(X=k)=exp(−λ)λᵏ/k! for integer k≥0, λ≥0 (λ=0 is a point mass at 0). Mean=Var=λ. A rate r over interval t gives λ=rt for a homogeneous Poisson process; check independent increments and constant rate.

Name: Normal(μ, σ)

Desc: Continuous bell curve for σ>0, μ real; mean=μ and variance=σ². NumPy accepts σ=0 as a degenerate constant sampler. A normal approximation to a mean requires its sampling assumptions.

Name: Exponential(λ)

Desc: For positive rate λ, density λexp(−λx) on x≥0; mean=1/λ, variance=1/λ². Survival P(X>t)=exp(−λt) implies P(X>s+t|X>s)=P(X>t). NumPy takes scale=1/λ, not rate.

Expectation and variance in one paragraph

E[X] = the long-run average value of X

Desc: For discrete X: E[X] = Σ x · P(X=x). For continuous X: E[X] = ∫ x · f(x) dx. Just a weighted average with weights = probabilities.

Var(X) = E[(X - E[X])²]

Desc: Average squared deviation from the mean. Measures spread. Standard deviation σ = √Var(X).

Linearity: E[aX + bY] = aE[X] + bE[Y]

Desc: For finite expectations, linearity holds without independence. Do not cancel undefined/infinite moments as if they were real numbers.

For independent X, Y: Var(X + Y) = Var(X) + Var(Y)

Desc: With finite second moments, Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y). Zero covariance suffices; independence implies it but is not necessary. For Y=X, the sum variance is 4Var(X), not 2Var(X).

Bayes' rule — the update formula

Expectation and variance summarise a distribution you already have. For events with P(B)>0, P(A|B)=P(A∩B)/P(B). Since P(A∩B)=P(B|A)P(A) when P(A)>0, Bayes follows. A positive test requires the total evidence denominator, not just the sensitivity. If P(B)=0 this elementary conditional probability is undefined.

P(A | B) = P(B | A) · P(A) / P(B)
        =    likelihood × prior     /   evidence

Name: Prior P(A)

Tagline: What you believed before

• Base rate of the hypothesis

• Prevalence of the disease

• Fraction of spam in your inbox

Name: Likelihood P(B|A)

Tagline: How well does A explain B?

• Sensitivity of the test

• Probability of this word given it's spam

• Model's assigned probability to the data

Name: Evidence P(B)

Tagline: How likely is B overall?

• Total probability across all hypotheses

• P(B) = Σ P(B|Aᵢ) P(Aᵢ) for mutually exclusive exhaustive hypotheses with defined conditional terms — the normaliser

• Often the hardest thing to compute directly

Name: Posterior P(A|B)

Tagline: Updated belief

• What you should now believe

• Reuse the posterior as a prior only with the appropriate likelihood for the new evidence conditional on earlier evidence; multiplying independent-test likelihoods requires conditional independence given the hypothesis

• This is Bayesian updating under the stated probabilistic model, not a definition of every learning algorithm

The Central Limit Theorem, in one sentence

Bayes' rule changes beliefs when evidence arrives; averaging changes the distribution of the quantity we measure. That second operation explains why the normal family matters even when individual observations are not bell-shaped.

Why the bell curve is everywhere

For independent and identically distributed (iid) Xᵢ with finite mean μ and variance 0<σ²<∞, Zₙ=√n(X̄ₙ−μ)/σ converges in distribution to N(0,1). Without centering and scaling, X̄ₙ instead concentrates near μ. Independence alone for nonidentical variables is not enough; additional conditions control dominant terms. A 95% normal-approximation interval uses standard error, not raw population standard deviation, and needs an appropriate estimator/sampling model. It is not a formula for every statistic.

Myth: “99% accurate means an alert has 99% probability of being true.” Sensitivity is P(+|D), specificity is P(−|not D), and overall accuracy also depends on prevalence. None alone equals P(D|+).

base = 0.001
tpr = 0.99
fpr = 0.01
n = 1_000_000
tp = n * base * tpr       # 990
fp = n * (1-base) * fpr   # 9990
print(tp / (tp+fp))      # 0.0901639..., about 9.0%

For the medical example at 1% prevalence, a population of 10,000 has 100 diseased and 9,900 healthy people in expectation. Sensitivity/specificity of 99% yield 99 true and 99 false positives: posterior 99/(99+99)=1/2 exactly for those specified rates. This is arithmetic under a model, not a clinical recommendation.

Why standardizing a sum matters

For the iid finite-variance setting just stated, independence gives Var(ΣXᵢ)=nσ², hence Var(X̄)=σ²/n. This explains shrinking spread, but variance alone does not prove a normal shape. A proof route uses characteristic functions: for centered unit-variance W, φ_W(t)=1−t²/2+o(t²). Independence makes the characteristic function of (ΣWᵢ)/√n equal to [φ_W(t/√n)]ⁿ, which tends to exp(−t²/2), the normal characteristic function. The continuity theorem then gives convergence in distribution. The finite-second-moment assumption justifies the expansion; a third moment is not required for the basic CLT.

For an average of n independent Exponential(1) draws, the exact variance is 1/n and Pearson kurtosis is 3+6/n. Kurtosis approaching 3 is consistent with this known gamma-family limit, not by itself proof of normality. The exponential is skewed but not heavy-tailed in the usual moment-generating-function sense. Infinite-variance Cauchy averages do not satisfy this CLT; heavy-tailed finite-variance distributions can converge slowly. Correlation can enlarge or shrink the variance: it does not always imply overconfidence. Quantiles such as p99 answer different questions from means and require their own uncertainty analysis.

Name: Base rate first, then evidence

Picture: Posterior odds equal prior odds times the likelihood ratio when the denominators are positive. With prior probability 0.001, odds are 1:999; a likelihood ratio of 10 gives posterior odds 10:999 and probability 10/1009≈0.00991. Multiplying the probability itself by 10 is not the exact update.

• P(A|B) and P(B|A) are different quantities, though they may coincide numerically. Label both condition and event.

• Accuracy alone can hide rare-class failures. Report precision and recall at your production base rate, and be explicit about which errors you are choosing to make.

• Positive correlation often makes an independence-based error bar too narrow; negative correlation can have the opposite effect. Independence violations do not universally move uncertainty in one direction — correlated failures, shared upstream dependencies, repeated measures on the same subject.

• Expected value alone is not a decision. Variance and tail behaviour matter, especially when one tail is catastrophic and unrecoverable rather than merely expensive.

Trigger: Fire this model the moment you see: a rare-event classifier reporting high accuracy · alert fatigue on a monitoring system · "the model is 95% confident" · an A/B test that reached significance in a day · anything described as a false positive rate without a base rate attached.

Decision: Setting the threshold on a fraud/anomaly detector: optimise for precision, for recall, or calibrate the probabilities and decide per case?

Name: High precision (few false alarms)

Gain: a higher fraction of alerts are true positives at the evaluated base rate; combine that rate with review costs and capacity

Cost: raising a threshold can reduce recall, but not necessarily below one half. Delayed labels and audited samples are needed to estimate missed events

Pick: each alert triggers expensive manual review or a customer-visible action, and reviewer trust is the binding constraint — the usual case when a human is in the loop

Name: High recall (catch almost everything)

Gain: few misses, which is essential when a single missed case is catastrophic or legally consequential

Cost: lowering a threshold can increase false alerts and review workload; measure investigation/completion rates rather than assuming collapse to zero

Pick: a miss is far more costly than an investigation and you have the capacity — human or automated — to actually process the volume

Name: Calibrated probabilities + cost-based decision

Gain: separates the model (estimate the probability) from the policy (decide what to do), so thresholds become an explicit business decision per segment, and you can route high-confidence cases to automation and mid-range ones to humans

Cost: calibration must be checked on representative held-out data and monitored under drift. Platt scaling/isotonic regression are options, not mandatory cures; costs and operational constraints still need estimates

Pick: error costs differ across segments, or you need to defend the threshold to a regulator or a stakeholder, or volume forces tiered automated handling

Verdict: Calibrate whenever you can, because a calibrated score lets the threshold be a decision with an owner rather than an artefact of whoever trained the model. An uncalibrated score forces every downstream consumer to guess what 0.8 means, and they will all guess differently. An operational failure to measure: when precision is too low, humans stop reading the alerts, and the system's real-world recall can fall below offline recall if true alerts are not acted on. Any threshold decision for a human-facing system is therefore partly a decision about attention, not just about statistics — and that means monitoring the investigation rate alongside the model metrics.

(c) Hands-on · 25 min

You'll simulate five distributions, compare finite sample moments with theoretical values, apply Bayes' rule to a diagnostic-test scenario, and watch CLT emerge. Save as probability_demo.py.

"""probability_demo.py — the five distributions + Bayes + CLT."""
from __future__ import annotations
import numpy as np
 
rng = np.random.default_rng(42)
N = 200_000  # fixed bounded experiment size, not a guaranteed error tolerance
 
# ── 1. Five distributions: theory vs simulation ────────────────────────
def bernoulli(p: float) -> tuple[float, float]:
    samples = rng.binomial(1, p, size=N)
    return float(samples.mean()), float(samples.var())
 
def binomial(n: int, p: float) -> tuple[float, float]:
    samples = rng.binomial(n, p, size=N)
    return float(samples.mean()), float(samples.var())
 
def poisson(lam: float) -> tuple[float, float]:
    samples = rng.poisson(lam, size=N)
    return float(samples.mean()), float(samples.var())
 
def normal(mu: float, sigma: float) -> tuple[float, float]:
    samples = rng.normal(mu, sigma, size=N)
    return float(samples.mean()), float(samples.var())
 
def exponential(lam: float) -> tuple[float, float]:
    if not np.isfinite(lam) or lam <= 0:
        raise ValueError("rate must be positive and finite")
    samples = rng.exponential(1.0 / lam, size=N)
    return float(samples.mean()), float(samples.var())
 
def demo_distributions() -> None:
    experiments = [
        ("Bernoulli(0.3)",     bernoulli(0.3),        0.30, 0.30*0.70),
        ("Binomial(10, 0.4)",  binomial(10, 0.4),     10*0.40, 10*0.40*0.60),
        ("Poisson(3.7)",       poisson(3.7),          3.70, 3.70),
        ("Normal(0, 2)",       normal(0, 2),          0.00, 4.00),
        ("Exponential(0.5)",   exponential(0.5),      1/0.50, (1/0.50)**2),
    ]
    print(f"{'distribution':<22} {'sample mean':>13} {'true mean':>10}  "
          f"{'sample var':>12} {'true var':>10}")
    for name, (m, v), tm, tv in experiments:
        print(f"{name:<22} {m:>13.4f} {tm:>10.4f}  {v:>12.4f} {tv:>10.4f}")
 
# ── 2. Bayes' rule: the canonical medical-test example ─────────────────
def bayes_medical_test() -> None:
    """
    A disease has prevalence 1%. Test sensitivity = 99% (P(+|D)).
    Test specificity = 99% (P(-|¬D), so false-positive P(+|¬D) = 1%).
    You test positive. What's the probability you actually have the disease?
    """
    p_disease = 0.01
    p_pos_given_disease = 0.99
    p_pos_given_healthy = 0.01
 
    p_pos = (p_pos_given_disease * p_disease
             + p_pos_given_healthy * (1 - p_disease))
    posterior = (p_pos_given_disease * p_disease) / p_pos
    print(f"P(disease | positive test) = {posterior:.4f} ≈ {posterior*100:.1f}%")
 
    # Estimate the same conditional proportion in a simulated million people
    n = 1_000_000
    has_disease = rng.random(n) < p_disease
    tests_positive = np.where(has_disease,
                              rng.random(n) < p_pos_given_disease,
                              rng.random(n) < p_pos_given_healthy)
    empirical = has_disease[tests_positive].mean()
    print(f"Simulation over {n} people: {empirical:.4f}")
 
# ── 3. Central Limit Theorem: sums of exponentials → normal ────────────
def central_limit_theorem_demo() -> None:
    """Compare finite exponential averages with the CLT prediction."""
    for n in [1, 5, 30, 200]:
        # Draw 20 000 experiments, each = mean of n Exponential(1) samples
        samples = rng.exponential(1.0, size=(20_000, n)).mean(axis=1)
        s = samples
        # Pearson kurtosis: standardized fourth moment, exact theory 3 + 6/n
        kurt = ((s - s.mean())**4).mean() / (s.var()**2)
        print(f"n={n:>3}  mean={s.mean():.3f}  var={s.var():.3f}  "
              f"kurtosis={kurt:.2f}  (normal is ~3.00)")
 
# ── 4. Confidence-interval sanity check via CLT ────────────────────────
def confidence_interval_demo() -> None:
    """Estimate coverage of an approximate Wald interval at a specified p and n."""
    true_p = 0.30
    trials = 1000
    n = 500
    hits_in_95_ci = 0
    for _ in range(trials):
        samples = rng.binomial(1, true_p, size=n)
        p_hat = samples.mean()
        se = np.sqrt(p_hat * (1 - p_hat) / n)   # standard error (CLT)
        lo, hi = p_hat - 1.96 * se, p_hat + 1.96 * se
        if lo <= true_p <= hi:
            hits_in_95_ci += 1
    print(f"True p was inside 95% CI in {hits_in_95_ci}/{trials} trials "
          f"({100*hits_in_95_ci/trials:.1f}%) — expect ≈95%")
 
if __name__ == "__main__":
    print("── DISTRIBUTIONS: sample vs theoretical ─────────")
    demo_distributions()
    print("\n── BAYES: medical test paradox ──────────────────")
    bayes_medical_test()
    print("\n── CLT: sums of exponentials become normal ──────")
    central_limit_theorem_demo()
    print("\n── Approximate Wald CI coverage experiment ──────")
    confidence_interval_demo()

Run it:

uv run --with numpy python probability_demo.py

Representative seeded output from this exact program is recorded below. The shared generator and call order matter; rerunning the same supported environment with the same seed does not introduce fresh randomness.

── DISTRIBUTIONS: sample vs theoretical ─────────
distribution             sample mean  true mean    sample var   true var
Bernoulli(0.3)                0.2998     0.3000        0.2099     0.2100
Binomial(10, 0.4)             3.9974     4.0000        2.3920     2.4000
Poisson(3.7)                  3.7006     3.7000        3.6903     3.7000
Normal(0, 2)                 -0.0023     0.0000        4.0063     4.0000
Exponential(0.5)              1.9989     2.0000        4.0130     4.0000
 
── BAYES: medical test paradox ──────────────────
P(disease | positive test) = 0.5000 ≈ 50.0%
Simulation over 1000000 people: 0.5012
 
── CLT: sums of exponentials become normal ──────
n=  1  mean=0.999  var=0.994  kurtosis=8.72  (normal is ~3.00)
n=  5  mean=1.001  var=0.199  kurtosis=3.97  (normal is ~3.00)
n= 30  mean=0.998  var=0.033  kurtosis=3.24  (normal is ~3.00)
n=200  mean=1.000  var=0.005  kurtosis=3.06  (normal is ~3.00)
 
── Approximate Wald CI coverage experiment ──────
True p was inside 95% CI in 957/1000 trials (95.7%) — expect ≈95%

What each block does

Anatomy of the code

Name: demo_distributions

Desc: For each family, draw 200k samples and compare sample mean/var to the closed-form theoretical values. Compare against distribution-specific sampling error, not a guaranteed number of matching decimals.

Name: bayes_medical_test

Desc: With exactly 1% prevalence, 99% sensitivity and 99% specificity, the posterior is 1/2. The million-person simulation estimates that value with sampling error; overall “accuracy” is not the required input.

Name: central_limit_theorem_demo

Desc: Average n Exponential(1) draws. The exact Pearson kurtosis 3+6/n goes from 9 toward 3. The estimated fourth moment fluctuates; this one moment is not a normality test.

Name: confidence_interval_demo

Desc: 1000 fake ‘surveys’ of 500 people each. Compute a 95% CI on the sample proportion; count how often it contains the true p. Should be ~950 out of 1000. This is what ‘95% confidence’ means, operationally.

What to observe

Compare sample moments with theory using absolute error for a zero mean; a relative “1% of zero” guarantee makes no sense. At these medical rates true and false positives are equally frequent, not one vastly outnumbering the other. Pearson kurtosis and the coverage count are finite random estimates.

The Wald interval p̂±1.96√(p̂(1−p̂)/n) is an approximation. With n=500 and p=0.3 it often has coverage near 95%, but if all observed outcomes are zero it collapses to [0,0] and excludes every positive p. Consider Wilson or exact binomial intervals for small counts, and distinguish a confidence procedure's long-run coverage from probability assigned to a fixed parameter after observation.

Sampler domains: Bernoulli/binomial require 0≤p≤1; binomial uses a nonnegative integer n; Poisson uses finite λ≥0 (NumPy has upper numerical limits); normal uses finite μ and σ≥0; exponential requires finite λ>0. Samples are scalar arrays of shape (N,); each CLT experiment allocates (20_000,n), then reduces axis 1 to 20,000 means. At n=200, the float64 draw array is about 32 MB. Runtime is proportional to draws, plus 1000×500 Bernoulli draws for the coverage experiment; this is a small CPU demonstration, not a convergence proof.

Goal: Rebuild the intuition that ‘rare event, positive test’ is often a false alarm

Hint: With 1% prevalence and 99% sensitivity, raising specificity to 99.9% gives 0.0099/(0.0099+0.00099)=10/11≈90.91%. Follow-up evidence must use its conditional likelihood; two copies of the same result are not independent evidence. Medical follow-up depends on clinical guidance, not this toy model.

Change the disease prevalence and see how the posterior changes:

for prev in [0.0001, 0.001, 0.01, 0.05, 0.10, 0.50]:
    p_pos = 0.99 * prev + 0.01 * (1 - prev)
    posterior = (0.99 * prev) / p_pos
    print(f"prevalence {prev*100:>5.2f}%  →  P(disease|+) = {posterior*100:>5.1f}%")

Exercise answer: the six posterior percentages are approximately 0.98, 9.02, 50.00, 83.90, 91.67 and 99.00. The nonlinear denominator accounts for false positives as prevalence changes. These follow from the specified rates, not a general claim about doctors.

(d) Production reality · 15 min

Repeated looks need a stopping rule

Imagine 20 independent tests of truly null hypotheses, each rejecting with probability 0.05. The probability of at least one rejection is 1−0.95²⁰≈0.642, not 0.05. Repeated looks at accumulating data are dependent, so this number is not their exact error rate; it demonstrates why a per-look bound alone is insufficient. Precommit to the sample size or use an appropriate sequential method. A simple finite-look protection allocates α/20 to each of 20 looks: the union bound controls any false rejection by α without independence. Always Valid Inference develops stopping-aware alternatives. The archived Netflix attribution and “will eventually” assertion are not established by this toy calculation.

Cluster structure changes the unit of information

If each of m independent household responses is copied k times, the km rows still contain only m independent responses. Their mean has variance σ²/m, not σ²/(km); the naive standard error is too small by √k. For k=9, that is exactly a factor of 3 in this constructed model, not a measured incident. Resampling individual rows repeats the same mistake; resample independent households (or use a justified cluster model). For time series use a suitable block/dependence method. “When unsure, bootstrap” is not a substitute for the sampling design.

Rare-event metrics and an explicit cost threshold

At prevalence 0.001, an always-negative classifier is 99.9% accurate but has recall zero and undefined precision because it never alerts. A nonempty alert stream's precision is TP/(TP+FP), and recall is TP/(TP+FN). The earlier 990/9990 count makes both denominators visible.

If a calibrated p estimates fraud, acting has expected false-positive cost (1−p)C_FP and not acting has missed-fraud cost pC_FN in a two-action, zero-correct-cost model. Act when p>C_FP/(C_FP+C_FN). For costs 1 and 9, the threshold is 0.1; use “do not act” on an exact tie under this strict-inequality policy. Capacity constraints, nonzero correct-action costs and calibration drift change the decision, so accuracy or calibration alone cannot choose it.

Where this shows up in the rest of the plan

Root: Probability is the language of uncertainty

Label: Statistics — CLT, hypothesis testing

Desc: The applied side of what we set up here.

Label: Naive Bayes classifier

Desc: Literally Bayes' rule applied to text features. Still a strong baseline.

Label: Logistic regression

Desc: Model outputs probabilities via sigmoid; loss = negative log-likelihood.

Label: Language models sample from a distribution

Desc: GPT/Claude output a probability distribution over the next token. Temperature, top-k, top-p all reshape it.

Label: Reinforcement learning

Desc: Expected reward over stochastic environments — the ‘E’ everywhere in the Bellman equation.

Label: Bayesian A/B testing

Desc: Posterior distributions over conversion-rate differences; no p-values in sight.

(e) Recall + stretch · 10 min

Self-check (answer follows each question).

Recall — questions with written answers

Q: In one sentence, what is a random variable?

A: A random variable is a measurable function on a probability sample space. A random-number generator is a computational way to sample a modeled distribution, not the mathematical definition.

Q: Match each to a distribution: number of heads in 10 coin flips; time until next request; number of earthquakes this year; a single yes/no event; measurement error.

A: Heads → Binomial only for independent same-p flips; a yes/no outcome → Bernoulli. Exponential waits and Poisson counts fit homogeneous Poisson-process assumptions. Normal measurement error is a modeling choice to check, not a fact implied by the word “error”; earthquake clustering can violate a Poisson model.

Q: State the definition of expectation E[X] for a discrete random variable.

A: E[X] = Σ x · P(X = x). Weighted average of possible values, weights = probabilities.

Q: For X ~ Bernoulli(0.3), what are E[X] and Var(X)?

A: E[X] = 0.3, Var(X) = 0.3 · 0.7 = 0.21. General: E[X] = p, Var(X) = p(1-p).

Q: State Bayes' rule.

A: P(A|B)=P(B|A)P(A)/P(B), with P(B)>0 and the likelihood conditional defined. The evidence includes both hypothesis and complement contributions; a zero denominator is not posterior zero.

Q: You have a rare-disease test with 99% sensitivity, 99% specificity, and 1% disease prevalence. You test positive. Roughly what's P(disease)?

A: Exactly 50% for the stipulated rates: true and false positives each have probability 0.0099; dividing 0.0099 by 0.0198 gives 1/2. A simulation is only approximately 50%.

Q: State the Central Limit Theorem in plain English.

A: For iid observations with finite nonzero variance, the standardized sample mean converges in distribution to a standard normal. Independent but non-identical observations require additional conditions; n=30 is not a universal threshold.

Q: When does variance add without a covariance term?

A: Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y), when the moments exist. Zero covariance is sufficient. Independence implies zero covariance but is not necessary.

Q: What's the difference between a 95% confidence interval and a Bayesian 95% credible interval?

A: Frequentist 95% CI: over repeated samples, a correctly calibrated 95% interval procedure contains the fixed parameter with probability 0.95 (approximately for a Wald interval). Bayesian 95% credible interval: the posterior distribution assigns 95% probability to the parameter being in this range. Different semantics; often similar numerically for large samples.

Q: Stretch — an email is flagged spam if a Naive-Bayes model gives P(spam | words) > 0.8. Explain how Bayes' rule turns per-word probabilities into a full posterior.

Stretch: true

A: Under conditional feature independence given class, the class score is P(c)ΠᵢP(wᵢ|c). Normalize the spam and nonspam scores by their sum, provided it is positive. Work with log probabilities to avoid underflow and use a justified smoothing model for unseen features. With prior 0.2, one feature of likelihood 0.5 under spam and 0.1 under nonspam gives 0.1/(0.1+0.08)=5/9. Duplicating that same dependent feature and pretending independence gives 25/29, a spurious confidence increase. Misspecified independence can harm both ranking and calibration; the model is a baseline to evaluate, not a guarantee or a description of modern autoregressive language models.

Explain-out-loud test

1. What is expectation, in one sentence, with one example?
2. State Bayes' rule and name the four pieces.
3. Why is a positive medical test on a rare disease often not proof of the disease?

Explain-out-loud answer guide

1. Expectation is a probability-weighted average when it exists: a prize of 4 with probability 1/4 and 0 otherwise has mean 1, though no draw pays 1.
2. Bayes divides likelihood times prior by the positive total evidence probability; the result is the posterior. Odds form multiplies prior odds by a likelihood ratio.
3. Many healthy people can produce false positives even at a small false-positive rate. At the stated 1% prevalence, 99 true and 99 false positives per 10,000 give a posterior of one half.

Optional archived videos

These are retained resource identities, not governing evidence. Playback, duration and suitability were not verified. Use the written examples even if a video is unavailable.

• Bayes theorem, the geometry of changing beliefs — archived label, unverified.
• Central Limit Theorem, clearly explained — archived label, unverified.
• The Main Ideas behind Probability Distributions — archived label, unverified.

What comes next

Next: S022 · Statistics — CLT, Hypothesis Testing, Confidence Intervals

Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.

Corrective source notes

• Conditional probability — event conditioning and positive denominator.

• Central limit theorem — centered, normalized iid sums with finite variance.

• NumPy Generator — sampling interface; distribution-specific numerical limits still apply.

• 3.3 Two Basic Rules of Probability - Introductory Statistics 2e | OpenStax — Conditional probability distinguishes joint probability from probability under a condition; Bayes calculations must include base rates. Checked 2026-10-07.

• asa — P-values do not measure the probability a studied hypothesis is true or the probability data arose by random chance alone. Checked 2026-10-07.

These citations support the named claims, not blanket certification of the lesson. Worked proofs are derived in the text; execution scope and finite-test limits are recorded separately.