Quantization — INT8, INT4, GPTQ, AWQ

Smaller storage is not automatically faster or equally accurate

Quantization represents values with a smaller code set plus scales/metadata. Seven billion weights at four bits have a raw payload of 3.5 decimal GB, about 3.26 GiB, before scales, zero points, packing alignment and other tensors. The corresponding 16-bit payload is 14 GB, about 13.04 GiB. Neither is total runtime memory: activations, KV cache, workspace and allocator reserve remain.

The prerequisites are tensor shapes, precision/rounding, LoRA's frozen-base distinction and KV-cache budgeting. You will compute affine quantization, test group scales, derive output-error weighting, distinguish GPTQ/AWQ/LLM.int8/NF4/FP8 and newer methods, and use a complete local-artifact benchmark reference. Only small CPU arithmetic is executed. No pretrained model, calibration corpus, benchmark dataset or GPU kernel is downloaded/run here.

LLM.int8(), Dettmers et al. v2 (2022) describes systematic activation outliers and a mixed-precision decomposition: most dimensions use vector-wise INT8, while outlier feature dimensions use 16-bit matmul. That is the source-supported origin, not an invented account of the author's lab evening. Weight outliers and activation outliers are related but different objects; one histogram does not establish the behavior of every model.

Affine and symmetric quantization by hand

For positive scale s, integer zero point z and code limits,

For an affine min/max calibration, choose bounds that include zero, set s=(upper−lower)/(qmax−qmin), and round/clip z=qmin−lower/s. Including zero matters for all-positive or all-negative tensors. A constant/zero range needs a defined policy rather than division by zero. State the rounding convention; NumPy/PyTorch nearest rounding ties to even in the examples here.

For lower=−1, upper=2 and unsigned four-bit codes 0..15, s=.2 and z=5. Values [−1,0,.3,2] map approximately to [0,5,6 or 7,15] at the half-step .3 depending on binary representation/tie policy, reconstructing −1,0,.2 or .4,2. Use a non-tie .31 if you want the unambiguous code 7 and reconstruction .4. A value 3 saturates at 15 and reconstructs 2, so its error is 1: the half-step bound does not survive clipping.

Symmetric signed INT8 often uses z=0, s=max(abs(x))/127 and codes −127..127. For x=[−.9,−.1,.05,.7,1.2], s=1.2/127 and q=[−95,−11,5,74,127]. Reconstruction is [−.897638,−.103937,.047244,.699213,1.2]; max absolute error .003937. “Under .5% error” is meaningless without a denominator: a small element can have much larger relative error. Uniform signed four-bit storage has −8..7, but a symmetric quantizer may deliberately use −7..7.

Executable affine check. This independent scalar fixture makes clipping and the rounding rule observable; it is not a packed kernel. The caller supplies finite bounds including zero. Equal bounds are allowed only at zero and use scale one/zero point zero. Positive and negative constant calibration tensors must instead include zero when forming bounds. Intermediate values must remain representable in floating point; this small fixture is not an arbitrary-magnitude numeric library.

import math
 
def affine(values, lower, upper, bits=4):
    values = list(values)
    if type(bits) is not int or not 2 <= bits <= 8:
        raise ValueError('integer bit width 2..8 required')
    if not values or not all(math.isfinite(v) for v in values):
        raise ValueError('nonempty finite values required')
    if not math.isfinite(lower) or not math.isfinite(upper) or not lower <= 0 <= upper:
        raise ValueError('finite ordered bounds including zero required')
    qmax = 2**bits-1
    scale = (upper-lower)/qmax if upper != lower else 1.
    if not math.isfinite(scale) or scale <= 0:
        raise ValueError('scale underflow or overflow')
    zero = min(qmax, max(0, round(-lower/scale)))
    # Clamp before dividing: an out-of-range finite value must saturate.
    codes = [min(qmax, max(0, round(min(upper,max(lower,v))/scale)+zero)) for v in values]
    return codes, [scale*(q-zero) for q in codes], scale, zero
 
q, restored, scale, zero = affine([-1.,0.,.31,2.,3.], -1., 2.)
assert q == [0,5,7,15,15] and zero == 5
assert abs(restored[2]-.4) < 1e-12 and restored[-1] == 2.
assert abs(3.-restored[-1]) > scale/2  # clipping invalidates half-step bound
assert affine([0.,0.],0.,0.)[:2] == ([0,0],[0.,0.])
assert affine([2.,2.],0.,2.)[0] == [15,15]
# Exact binary half steps, with a zero point of zero: ties round to even.
assert affine([.125,.375],0.,3.75)[0] == [0,2]
print('affine codes, zero range, clipping and exact tie controls passed')

Rounding the zero point can move the representable endpoints by up to half a step relative to calibration bounds. The textbook half-step rounding guarantee concerns values in the representable unclipped interval, not every value seen after distribution shift. The zero-range policy only reconstructs zero; applying that degenerate calibration to nonzero inputs loses their information. Compare outputs and task errors as well as the code range.

Per-tensor, per-channel and per-group scales

A single scale is cheap but one large range can consume precision for many small values. Per-output-row scales isolate rows; groups along the input dimension isolate smaller neighborhoods. Group size 128 and one FP16 scale per group add 16/128=.125 bits/weight, yielding 4.125 bits before zero points, padding and additional metadata. Adding a four-bit zero point per group gives 4.15625 bits under that particular packing. An INT8 array holding four-bit codes is still one byte per entry until packed.

The following exact protected program implements inspectable symmetric groupwise round-trip behavior, including zero groups and invalid input rejection. It does not implement GPTQ/AWQ, packed INT4 kernels or GPU acceleration.

import numpy as np
 
def quantize_groups(w, group_size=4):
    w = np.asarray(w, dtype=np.float64)
    if w.ndim != 2 or group_size <= 0 or w.shape[1] % group_size:
        raise ValueError('require 2D divisible groups')
    if not np.isfinite(w).all():
        raise ValueError('finite weights required')
    grouped = w.reshape(w.shape[0], -1, group_size)
    maximum = np.max(np.abs(grouped), axis=-1, keepdims=True)
    scale = np.where(maximum == 0, 1., maximum/7.)
    q = np.clip(np.rint(grouped/scale), -7, 7).astype(np.int8)
    reconstructed = (q*scale).reshape(w.shape)
    return q, scale, reconstructed
 
w = np.array([[0.,0.,0.,0.], [-1.,-.25,.25,1.]])
q, scale, restored = quantize_groups(w)
np.testing.assert_array_equal(restored[0], w[0])
assert np.max(np.abs(restored[1]-w[1])) <= scale[1,0,0]/2 + 1e-12
assert q.min() >= -7 and q.max() <= 7
print('zero-group handling and rounding bound pass')

Outlier exercise with an answer: take one row [.1,.2,.3,.4,.1,.2,.3,5.]. With one group of eight, scale is 5/7≈.7143; several ordinary weights round to zero or .7143. With groups of four, the first group's scale is .4/7≈.05714 and its small weights retain finer resolution; the last group still contains the outlier. Group size one reconstructs a nonzero scalar exactly in this symmetric scheme because its code is ±7 and its own scale stores its magnitude—but scale metadata can erase the compression benefit. Granularity trades representation quality against storage/kernel cost; “per-tensor is always catastrophic” is not a theorem.

Why output error differs from weight error

For column-vector storage W (out,in) and calibration columns X (in,samples), Y=WX. Quantization error is (W−W_hat)X, so a sensitive input direction can amplify a small weight error. For one row w, squared reconstruction error is (w−w_hat)XXᵀ(w−w_hat)ᵀ; its Hessian in w_hat is 2XXᵀ. If samples are rows instead, the covariance expression transposes. Do not mix conventions.

A simple example: equal weight errors [.1,.1] applied to input [10,0] produce output error 1, while input [0,.1] produces .01. Identical weight MSE does not imply identical functional error. Across a deep autoregressive model, changed logits can alter sampled tokens and all subsequent prefixes. Calibrate with permitted representative languages, domains and sequence lengths, with evaluation labels held out; generic short English text cannot establish multilingual or long-context fidelity.

GPTQ and AWQ: different ways to protect the function

GPTQ: compensate remaining weights

Frantar et al., GPTQ v2 use approximate second-order information for layerwise weight quantization. Quantizing one weight/column introduces a constrained error; remaining unquantized values can be adjusted using an inverse-Hessian-related update. In the elementary optimal-brain-quantization step, an error (w_j−q_j) is distributed to remaining coordinates in proportion to an inverse-Hessian row/column divided by its diagonal. GPTQ's efficient implementation batches/lazily applies updates and uses a stable factorization; merely copying a single formula is not the full algorithm.

For a toy inverse Hessian [[1,.5],[.5,1]], first weight .3 rounded to .25 creates error .05; the compensating change to the second coordinate is −.05×.5/1=−.025 under this convention. That illustrates using input correlation instead of independently rounding every weight. Singular calibration covariance requires damping/handling; calibration choice and activation-order/packing settings affect the artifact. Do not promise a fixed thirty-minute quantization duration or parallelize all layers while ignoring changed upstream activations.

AWQ: activation-aware scaling

Lin et al., AWQ v1 identify salient weights using activation information and search per-channel scaling to protect them. For a positive diagonal S,

Multiplying weight columns by S and dividing corresponding activation rows by S preserves the unquantized product. After rounding, Q(WS)S^{-1}X need not equal WX; evaluate the quantization error to select scales. “Shrink salient weights” and “enlarge salient weights” are not interchangeable descriptions without identifying which operand and convention is used.

Check the function, not only weight MSE. This tiny matrix example uses the group quantizer defined above. The first assertion verifies the exact change of variables; the next measures the function after rounding instead of assuming equivalence survives quantization. The final control gives equal weight MSE but unequal output error.

W = np.array([[.1,.2,.3,5.]])
X = np.array([[10.,0.],[0.,.1],[1.,1.],[.01,.02]])
S = np.array([4.,1.,1.,1.])
np.testing.assert_allclose((W*S) @ (X/S[:,None]), W@X, atol=1e-12)
_, _, roundtrip = quantize_groups(W)
_, _, scaled_roundtrip = quantize_groups(W*S)
plain_error = np.sum(((W-roundtrip)@X)**2)
scaled_error = np.sum((W@X-scaled_roundtrip@(X/S[:,None]))**2)
assert scaled_error < plain_error  # selected illustrative fixture, not an AWQ guarantee
np.testing.assert_allclose(np.mean(np.array([.1,0.])**2),
                           np.mean(np.array([0.,.1])**2))
assert abs(np.array([.1,0.])@np.array([10.,.1])) == 1.
assert abs(np.array([0.,.1])@np.array([10.,.1])) < .011
print('output squared errors', plain_error, scaled_error)

This manually selected S demonstrates the scaling mechanism, not AWQ's calibration search or a full GPTQ implementation. On another input direction it can worsen error; evaluate candidate scales on representative held-out inputs and keep the baseline. All examples use unpacked CPU arrays, so none measures INT4 bandwidth, GPU kernel speed, perplexity or language quality.

Inverse scaling can be fused into a compatible preceding linear/norm operation, but residual branches, shared activations and nonlinearities constrain where it can move. Verify the whole graph; do not insert an arbitrary diagonal through every nonlinearity. AWQ is not simply GPTQ with a faster kernel, and it does not prove universal immunity to calibration shift. Separate the algorithm, checkpoint layout and serving kernel when comparing speed.

LLM.int8, NF4 and FP8 are not the same four-bit story

LLM.int8 keeps outlier feature dimensions in higher-precision products and quantizes the rest, with vector-wise normalization. It does not imply every activation is plain INT8, zero overhead or fixed slower-than-AWQ throughput.

bitsandbytes NF4 uses a nonuniform four-bit codebook motivated by normally distributed weights. QLoRA v1 backpropagates through a frozen quantized base into LoRA adapters, adds double quantization of quantization constants and paged optimizers for memory spikes. NF4 is not uniform INT4 and not Blackwell's native FP4 arithmetic format. Transformers 4.45.2's bitsandbytes guide distinguishes compute dtype, NF4 and nested quantization; support depends on actual backend/hardware.

FP8 is floating point with exponent and mantissa bits, not eight-bit integer scaling. Transformer Engine v2.0 explains E4M3's precision/range versus E5M2's wider range and scaling recipes. Outliers, scaling and sensitive layers do not disappear. Hopper-compatible low-precision tensor arithmetic can help when the relevant kernel/shape uses it; it does not double all inference throughput or guarantee a .1-point quality delta. FP8 KV cache and weight precision are separate choices, not required to match.

A method map instead of an unsupported benchmark scoreboard

smooth activation transfer in SmoothQuant v1 , SqueezeLLM v1, QuIP# v1, AQLM v1 and SpinQuant v1 support these mechanisms. The QuaRot authors' repository identifies its exact paper/implementation; HQQ's repository defines its method and supported backends. OmniQuant v1 learns weight clipping and equivalent transforms, while Atom v1 emphasizes fine-grained/mixed-precision low-bit serving with batching. These are not reproduced benchmark results. A method with a sparse residual or codebooks must count them in its memory total.

BitNet and two-bit limits

Ma et al., BitNet b1.58 v1 (2024) describe ternary weights −1,0,+1 and a quantization-aware training recipe. Log2(3)≈1.585 is an information measure; practical packing plus scales is not automatically 1.585 bits per stored parameter. A dot product with ternary weights can select/add/subtract activation terms, e.g. weights [−1,0,1] and activations [2,5,7] give −2+0+7=5. Activation quantization, scales, accumulators, memory and kernels still cost work. Assuming an arithmetic operation costs one tenth as much does not imply a tenfold end-to-end energy reduction. No unsupported 70B BitNet result is asserted.

QuIP# uses randomized Hadamard processing and an E8 lattice codebook, not uniform INT2 with a renamed scale. It also uses fine-tuning to improve fidelity. Two-bit 70B raw weights would already be 17.5 GB before overhead/cache, so a headline fit on 24 GB needs an actual peak inventory. Marlin's implementation concerns a specific optimized FP16×INT4 path; it is not evidence of universal INT2 support, 90% bandwidth on any shape, or every runtime automatically selecting it.

W8A8, W4A16 and W4A8

The notation says nominal weight bits W and activation bits A, but still needs integer/float formats, accumulation dtype, scale granularity and exceptions. W4A16 commonly means weight-only compression with higher-precision activation arithmetic. W8A8 also quantizes activations, potentially using low-precision compute kernels. W4A8/W4A4 require precise hardware/kernel support; their names do not mandate one vendor generation or guarantee a quality loss. KV-cache precision is a third axis.

Small-batch decode can be dominated by weight bandwidth, so reducing weight bytes may help. Larger-batch/prefill work may be more compute-intensive, so dequantization can outweigh saved traffic. Long-context decode can be cache-dominated. None of these regimes is fixed by “batch 1” or “long prompt” alone; profile the intended service.

Complete local-artifact load and benchmark reference

This is an unexecuted GPU procedure, not the former invented A100/WikiText throughput table. The loading API is pinned to Transformers 4.45.2. GPTQ and AWQ artifacts additionally require the compatible backend and packing version recorded by their producer; the tagged GPTQ guide and AWQ guide describe those contracts. For a historical replication, separately lock and verify backend/CUDA versions before use rather than assuming newer archived packages work unchanged.

Save this as bench_local.py. Run a fresh process for each approved local base/GPTQ/AWQ artifact to avoid two-model residency and allocator carryover. Use identical tokenizer, prompt bytes, generation budget and hardware. LOCAL_MODEL and LOCAL_TEXTS_JSONL are already-staged permitted inputs; no Hub or dataset fetch is performed.

# Reference only: CUDA required; no downloads or actual benchmark run in this review.
import json
import math
import os
import statistics
import time
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
 
path = os.environ["LOCAL_MODEL"]
tok = AutoTokenizer.from_pretrained(path, local_files_only=True, trust_remote_code=False)
model = AutoModelForCausalLM.from_pretrained(path, local_files_only=True,
    trust_remote_code=False, device_map="cuda:0", torch_dtype=torch.float16)
model.eval()
prompt = "Explain why a checksum and a load test check different things."
inputs = tok(prompt, return_tensors="pt").to("cuda:0")
eosc = model.generation_config.eos_token_id
eos = set(eosc if isinstance(eosc, list) else ([] if eosc is None else [eosc]))
 
def count_generated(output):
    emitted = output[0, inputs.input_ids.shape[1]:].tolist()
    for i, token in enumerate(emitted):
        if token in eos:
            return i + 1
    return len(emitted)
 
with torch.inference_mode():
    for _ in range(2):
        model.generate(**inputs, max_new_tokens=64, do_sample=False,
                       pad_token_id=tok.pad_token_id if tok.pad_token_id is not None else tok.eos_token_id)
    torch.cuda.synchronize()
    torch.cuda.reset_peak_memory_stats()
    times, counts = [], []
    for _ in range(5):
        torch.cuda.synchronize()
        start = time.perf_counter()
        out = model.generate(**inputs, max_new_tokens=64, do_sample=False,
                             pad_token_id=tok.pad_token_id if tok.pad_token_id is not None else tok.eos_token_id)
        torch.cuda.synchronize()
        times.append(time.perf_counter()-start)
        counts.append(count_generated(out))
    nll, targets = 0., 0
    with open(os.environ["LOCAL_TEXTS_JSONL"], encoding="utf-8") as f:
        for line in f:
            text = json.loads(line)["text"]
            ids = tok(text, return_tensors="pt", truncation=False).input_ids[0]
            # Explicit non-overlapping 256-token windows; no label/prompt ambiguity.
            for start in range(0, len(ids), 256):
                chunk = ids[start:start+256].unsqueeze(0).to("cuda:0")
                if chunk.shape[1] < 2:
                    continue
                result = model(input_ids=chunk, labels=chunk, use_cache=False)
                n = chunk.shape[1]-1
                nll += result.loss.item()*n
                targets += n
    if targets == 0:
        raise ValueError("no scored evaluation tokens")
print(json.dumps({"model": path, "transformers_contract": "4.45.2",
    "device": torch.cuda.get_device_name(), "seconds": times,
    "generated_tokens": counts, "aggregate_tokens_per_second": sum(counts)/sum(times),
    "median_request_seconds": statistics.median(times),
    "peak_allocated_bytes": torch.cuda.max_memory_allocated(),
    "peak_reserved_bytes": torch.cuda.max_memory_reserved(),
    "nll_sum": nll, "scored_tokens": targets, "perplexity": math.exp(nll/targets)}, indent=2))

The reported generation time includes prefill plus decoding; it is not TTFT or isolated per-token decode latency. Obtain those separately with timestamped token streaming/profiling in the chosen runtime, without changing the sampler. The peak above covers generation and evaluation; record separate phase peaks if comparing serving capacity. The chosen windowing drops cross-window context; use exactly the same rule and corpus hash for all artifacts and never compare this PPL directly to a paper using different windows. Report cold load separately, then repeat at real batch/context/output lengths. These procedures retain the original benchmark objective without pretending a benchmark happened.

Reproduce quantization only with a complete calibration contract

For GPTQ, the tagged Transformers API accepts GPTQConfig(bits=4, dataset=list_of_strings, tokenizer=tok, group_size=128) and passes it to from_pretrained for a local unquantized checkpoint. Specify calibration sample selection, lengths, dataset hash and backend/kernel options, then save the complete output directory. For AWQ, the historical AutoAWQ 0.2.6 API provides the following complete quantize/save portion, with the same already-staged calibration texts. This is unexecuted and requires its separately compatible locked environment, not simultaneous installation into the current CPU environment.

# Reference only: AutoAWQ 0.2.6 API, local approved artifacts.
import json
import os
from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer
base = os.environ["LOCAL_BASE"]
tok = AutoTokenizer.from_pretrained(base, local_files_only=True, trust_remote_code=False)
with open(os.environ["LOCAL_CALIBRATION_JSONL"], encoding="utf-8") as f:
    calibration = [json.loads(line)["text"] for line in f]
if not calibration:
    raise ValueError("empty calibration corpus")
model = AutoAWQForCausalLM.from_pretrained(base, trust_remote_code=False)
model.quantize(tok, quant_config={"zero_point": True, "q_group_size": 128,
                                 "w_bit": 4, "version": "GEMM"}, calib_data=calibration)
model.save_quantized("awq-local-output")
tok.save_pretrained("awq-local-output")

For GPTQ, this complete Transformers 4.45.2 reference uses the same local calibration format. The compatible AutoGPTQ/Optimum/CUDA versions are part of the artifact producer's environment lock; the loader API does not prove that arbitrary backend versions compose.

# Reference only: historical Transformers 4.45.2 GPTQ API, no execution here.
import json
import os
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig
base = os.environ["LOCAL_BASE"]
tok = AutoTokenizer.from_pretrained(base, local_files_only=True, trust_remote_code=False)
with open(os.environ["LOCAL_CALIBRATION_JSONL"], encoding="utf-8") as f:
    calibration = [json.loads(line)["text"] for line in f]
if not calibration:
    raise ValueError("empty calibration corpus")
config = GPTQConfig(bits=4, dataset=calibration, tokenizer=tok, group_size=128)
model = AutoModelForCausalLM.from_pretrained(base, local_files_only=True,
    trust_remote_code=False, device_map="auto", quantization_config=config)
model.save_pretrained("gptq-local-output")
tok.save_pretrained("gptq-local-output")

For bitsandbytes NF4, the complete load configuration below saves a calibrated-quantizer pass but not evaluation. It does not work on every backend or imply a fixed relative speed. Keep the exact base and adapter/quantizer configuration for reconstruction.

# Reference only: Transformers 4.45.2 API; compatible bitsandbytes/CUDA lock required.
import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
base = os.environ["LOCAL_BASE"]
tok = AutoTokenizer.from_pretrained(base, local_files_only=True, trust_remote_code=False)
config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                           bnb_4bit_compute_dtype=torch.bfloat16,
                           bnb_4bit_use_double_quant=True)
model = AutoModelForCausalLM.from_pretrained(base, local_files_only=True,
    trust_remote_code=False, device_map="cuda:0", quantization_config=config)
model.eval()

The benchmark program can be factored to accept that loaded model for an NF4 comparison, then run in a separate process from the baseline. Record bitsandbytes/backend versions and the generated quantization configuration along with the model hash.

Failure cases, acceptance and all worked answers

Long context: test the intended context and retrieval/position tasks; weight and KV quantization are separate. Keeping KV in BF16 can isolate a cause but cannot repair arbitrary weight errors. Math/code: evaluate actual correctness and error categories, not only generic perplexity. Multilingual: audit calibration coverage and compare each language. Merged LoRA: compare unmerged validated function against merged/requantized function. Kernel fallback: inspect runtime/backend and measured shapes; a group-size change alone does not guarantee a specific kernel. FP8 cache: casts/scales can help or hurt independently of weight precision; there is no blanket ten-percent mismatch penalty.

A decision procedure is: identify state/cache/compute bottleneck; retain an unquantized reference; select a supported representation/kernel; fit/calibrate only permitted data; test reconstruction and task regressions; benchmark the actual batch/context; keep rollback artifacts. If full precision already meets constraints or quantization fails the task gate, do not quantize. A smaller/distilled model is another candidate and is not automatically worse on every task.

1. Scale/zero point? Use the affine equations with declared bounds including zero, a positive-scale zero-range policy, rounding and saturation. The symmetric INT8 example above gives s=1.2/127 and max error about .003937.
2. Scale per what? Tensor, output channel or input-dimension group. Smaller groups can isolate outliers but add metadata and constrain kernels; group=1 is a poor compression limit despite low reconstruction error.
3. GPTQ versus AWQ? Second-order reconstruction compensation versus activation-aware channel scaling/protection. Both need an artifact/runtime contract; neither guarantees universal quality preservation.
4. Why activation quantization needs extra care? Activations depend on input/context and can shift outside calibration ranges. Weights are static, but their errors still matter through the network. Wide accumulators do not justify an independence/zero-mean error assumption for every weight/input.
5. Why FP8 on compatible hardware? It offers a floating dynamic-range/precision tradeoff with supported low-precision kernels. It remains quantization, not a reason to say “do not quantize,” and must pass numeric/task tests.
6. BitNet's attraction and limitation? Ternary weight products can use signed additions; training/packing/scales and hardware decide actual benefit. It is not a lossless post-training conversion of arbitrary FP16 weights.
7. AWQ stretch proof? (WS)(S^{-1}X)=W(SS^{-1})X=WX for invertible diagonal S. Fusing S inverse is graph-dependent; rounding breaks the equality and must be measured.
8. NF4 WikiText stretch? Use an already licensed local corpus with a recorded tokenizer/windowing, measure token-weighted NLL for baseline and compressed artifact, exponentiate, and report the difference. No numeric result is supplied because no model/dataset/GPU run occurred.
9. Energy stretch? For weights [−1,0,1], signed addition gives 5 on [2,5,7]. Even if that arithmetic were tenfold cheaper, memory movement, other layers and scales remain. Under an explicit toy model where replaceable arithmetic is 60% of energy and everything else is 40%, the new energy fraction is .6/10+.4=.46, so the improvement is about 2.17×, not 10×. Actual fractions require measurement; this is a worked assumption, not a hardware energy benchmark.

Spaced review: tomorrow recompute affine saturation and 4.125-bit metadata; in a week compare full-precision versus compressed behavior at a declared context. Bring back the quantizer and kernel identity plus an evaluation protocol, not fabricated PPL/tokens/sec. Pruning and distillation next change model structure or training targets rather than merely the stored number representation.

Previous: ← DL S066 · Next: DL S068 →

Original optional resource links

The original optional video/reading destinations are retained here for continuity, not as current API, price, benchmark or publication-date evidence. The versioned primary citations in the teaching above govern the corrected claims. These supplementary destinations were not all fetched or watched; a missing or changed resource does not replace the local derivations, procedures and answers.

• https://arxiv.org/abs/2208.07339
• https://www.youtube.com/watch?v=v1oHf1KV6kM
• https://www.youtube.com/watch?v=mNE_d-C82lI
• https://huggingface.co/docs/transformers/main/en/quantization
• https://arxiv.org/abs/2306.00978
• https://github.com/AutoGPTQ/AutoGPTQ
• https://github.com/NVIDIA/TransformerEngine
• https://arxiv.org/abs/2402.17764
• https://arxiv.org/abs/2402.04396
• https://github.com/IST-DASLab/marlin
• https://arxiv.org/abs/2211.10438
• https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/
• https://arxiv.org/abs/2401.06118