CIFAR-10 training: a complete local pipeline and controlled experiments

A training pipeline is a chain of contracts: dataset identity, disjoint splits, transformations, logits and labels, optimizer state, scheduler clock, evaluation mode and checkpoint selection. CIFAR-10 is small enough to reason about the whole chain without pretending that a single accuracy number explains it.

What a reproducible CIFAR result would mean

The official CIFAR page defines60,00032×32 color images in10 classes:50,000 training and10,000 test images. Airplane,automobile,bird,cat,deer,dog,frog,horse,ship,truck are the label order in the standard distribution. The original93%-in-twenty-minutes goal is an experiment target, not a guaranteed or measured result of this lesson.

ResNet Strikes Back reports80.4% top1 for a vanilla ResNet50 at224×224 on ImageNet validation without extra data or distillation under its more demanding training recipe. That is neither ResNet18 on CIFAR nor proof that SGD always beats Adam, that EMA supplies a fixed gain, or that architecture is irrelevant. Improved recipe and scaling studies teach a better lesson: compare training conditions before crediting a new block.

You will build a complete local-data-only CIFAR pipeline with deterministic validation, a CIFAR-adapted ResNet18, parameter groups, optional warmup followed by cosine, checkpoint selection and final test evaluation. Read DL025–026 for operator/model shapes and DL020–024 for regularization, normalization and optimization. No network download, pretrained weights or GPU is required by the default recipe; its full dataset path is opt-in and unexecuted in this audit.

Dataset, normalization and split ownership

RandomCrop32 with reflection padding4 samples a32×32 view from a40×40 padded image. HorizontalFlip models a plausible natural-photo symmetry, not a theorem for all labels. Digits, printed text and medical laterality require different choices. Cropping can remove the object, even if the class is normally flip-invariant.

ToTensor on uint8 imagery maps pixels to floating[0,1] and CHW. Normalize then applies $(x_c-\mu_c)/\sigma_c$; it is not division by255 a second time. The commonly copied mean(.4914,.4822,.4465) and std(.2470,.2435,.2616) are reference training statistics, not magical ReLU bounds. Normalized inputs are not guaranteed to remain inside[−2,2]. This recipe computes moments only from the selected training portion, so the validation subset does not help choose preprocessing.

Training and validation need different dataset views over the same official training bytes, because changing the transform on one shared dataset also changes it for every Subset referring to it. Never tune on the official test split. Test-time augmentation is a separate declared protocol with an aggregation rule and compute budget, not random training transforms accidentally left on.

import copy, math, time
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader, Subset, TensorDataset
 
def channel_stats_uint8(images):
    if images.dtype != np.uint8 or images.ndim != 4 or images.shape[-1] != 3:
        raise ValueError('expected uint8 NHWC RGB')
    total=np.zeros(3); square=np.zeros(3); count=0
    for start in range(0,len(images),512):
        batch=images[start:start+512].astype(np.float64)/255
        total += batch.sum(axis=(0,1,2)); square += (batch*batch).sum(axis=(0,1,2))
        count += np.prod(batch.shape[:3])
    if not count: raise ValueError('empty training split')
    mean=total/count; std=np.sqrt(np.maximum(square/count-mean**2,0))
    if np.any(std==0): raise ValueError('constant channel')
    return tuple(mean),tuple(std)
 
def local_cifar_loaders(root, batch_size=128, seed=27):
    # Requires existing torchvision0.24 and trusted, already-present CIFAR files.
    from torchvision import datasets, transforms as T
    raw=datasets.CIFAR10(root,train=True,download=False)
    order=torch.randperm(len(raw),generator=torch.Generator().manual_seed(seed)).tolist()
    val_ids,train_ids=order[:5000],order[5000:]
    mean,std=channel_stats_uint8(raw.data[train_ids])
    base=[T.ToTensor(),T.Normalize(mean,std)]
    train_tfm=T.Compose([T.RandomCrop(32,padding=4,padding_mode='reflect'),
                         T.RandomHorizontalFlip()]+base)
    eval_tfm=T.Compose(base)
    train_ds=datasets.CIFAR10(root,train=True,download=False,transform=train_tfm)
    val_ds=datasets.CIFAR10(root,train=True,download=False,transform=eval_tfm)
    test_ds=datasets.CIFAR10(root,train=False,download=False,transform=eval_tfm)
    generator=torch.Generator().manual_seed(seed)
    train_dl=DataLoader(Subset(train_ds,train_ids),batch_size=batch_size,shuffle=True,
                        generator=generator,num_workers=0,pin_memory=False,drop_last=False)
    val_dl=DataLoader(Subset(val_ds,val_ids),batch_size=512,num_workers=0)
    test_dl=DataLoader(test_ds,batch_size=512,num_workers=0)
    manifest={'train_ids':train_ids,'validation_ids':val_ids,'mean':mean,'std':std,
              'class_to_idx':train_ds.class_to_idx,'seed':seed}
    return train_dl,val_dl,test_dl,manifest

No download=True path exists. Torchvision's standard CIFAR reader expects its already extracted official pickle files; only use trusted files with the official checksums, never arbitrary untrusted pickles. Record their hashes with the split manifest when running the real experiment. Missing data or torchvision should fail visibly, not silently replace CIFAR with synthetic images.

num_workers=0 and no pinning are conservative CPU settings. For an independently authorized accelerator run, measure workers, pinned memory and transfer overlap on that machine. More workers can increase memory and contention; neither four workers nor pinning guarantees60% less idle time. Windows spawn/fork differences and stochastic worker seeds must be part of a resumed-run contract.

The model: keep small images spatially useful

An ImageNet stem's7×7 stride2 convolution plus max pool downsamples32 to8 before most blocks. The CIFAR variant uses3×3 stride1 and no initial pool: stage resolutions32,16,8,4, then global average pooling and a10-class head. It is roughly11.2M parameters, not100k.

import torch.nn as nn
import torch.nn.functional as F
from typing import Type
 
class BasicBlock(nn.Module):
    def __init__(self, in_c, out_c, stride=1):
        super().__init__()
        self.conv1 = nn.Conv2d(in_c, out_c, 3, stride, 1, bias=False)
        self.bn1   = nn.BatchNorm2d(out_c)
        self.conv2 = nn.Conv2d(out_c, out_c, 3, 1, 1, bias=False)
        self.bn2   = nn.BatchNorm2d(out_c)
        self.short = (nn.Sequential(
                        nn.Conv2d(in_c, out_c, 1, stride, bias=False),
                        nn.BatchNorm2d(out_c))
                      if stride != 1 or in_c != out_c else nn.Identity())
 
    def forward(self, x):
        out = F.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        return F.relu(out + self.short(x))
 
class ResNet18_CIFAR(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.stem = nn.Sequential(
            nn.Conv2d(3, 64, 3, 1, 1, bias=False),
            nn.BatchNorm2d(64), nn.ReLU(inplace=True),
        )
        self.layer1 = self._make(64,  64,  2, stride=1)
        self.layer2 = self._make(64,  128, 2, stride=2)
        self.layer3 = self._make(128, 256, 2, stride=2)
        self.layer4 = self._make(256, 512, 2, stride=2)
        self.pool = nn.AdaptiveAvgPool2d(1)
        self.fc   = nn.Linear(512, num_classes)
 
    def _make(self, in_c, out_c, n, stride):
        blocks = [BasicBlock(in_c, out_c, stride)]
        blocks += [BasicBlock(out_c, out_c, 1) for _ in range(n-1)]
        return nn.Sequential(*blocks)
 
    def forward(self, x):
        x = self.stem(x)
        x = self.layer1(x); x = self.layer2(x)
        x = self.layer3(x); x = self.layer4(x)
        return self.fc(self.pool(x).flatten(1))

The projected shortcut must match channel count and stride. BatchNorm behavior is controlled by train/eval; no_grad() alone does not select stored statistics. A batch size of one with nontrivial H×W does not automatically make every BatchNorm output zero.

Optimizer groups and an explicit schedule clock

SGD with momentum.9,initial lr.1 and decay5e−4 is a reasonable baseline to test, not a universal best choice. Momentum accumulates directional history; Nesterov uses the configured look-ahead-style update. In PyTorch SGD, weight decay is an L2 gradient term; AdamW applies decoupled decay and is not interchangeable with Adam plus L2.

Excluding one-dimensional normalization/bias parameters from decay is a chosen policy, not a mathematical commandment or fixed1–2% accuracy improvement. Check every trainable parameter appears exactly once. This dimensionality heuristic is appropriate for this CNN, not every architecture's positional or scalar parameter.

Cosine is a schedule with an initial/minimum LR and horizon, plus any warmup choices—not only one hyperparameter. The function below assigns the LR used at the start of each epoch. Warmup3 gives factors1/3,2/3,1; with at least two cosine-phase epochs, that phase starts at1 and reaches zero in the last epoch. A one-epoch cosine phase uses factor1: one point cannot represent both endpoints. The default50-epoch,3-warmup recipe has47 cosine-phase epochs. Other endpoint conventions are valid if logs and checkpoint resumes use the same one.

def decay_groups(model, weight_decay=5e-4):
    decay=[]; no_decay=[]
    for name,p in model.named_parameters():
        if p.requires_grad:
            (no_decay if p.ndim <= 1 or name.endswith('.bias') else decay).append(p)
    ids=[id(p) for p in decay+no_decay]
    assert len(ids)==len(set(ids))==sum(p.requires_grad for p in model.parameters())
    return [{'params':decay,'weight_decay':weight_decay},
            {'params':no_decay,'weight_decay':0.0}]
 
def lr_factor(epoch, epochs, warmup=0):
    if not 0 <= warmup < epochs or not 0 <= epoch < epochs:
        raise ValueError('invalid scheduler position')
    if epoch < warmup: return (epoch+1)/warmup
    span=epochs-warmup
    phase=(epoch-warmup)/max(1,span-1)
    return .5*(1+math.cos(math.pi*phase))
 
def set_epoch_lr(opt, epoch, epochs, base_lr=.1, warmup=0):
    value=base_lr*lr_factor(epoch,epochs,warmup)
    for group in opt.param_groups: group['lr']=value
    return value

For comparison with CosineAnnealingLR, call its step() at its documented post-optimizer/epoch point and record which LR the log displays. Do not execute a second empty epoch loop merely to add evaluation. This lesson uses one unambiguous epoch loop below.

Training, evaluation and checkpoint selection

Cross-entropy expects logits(B,10) and integer targets(B,) in0–9. This helper rejects ignored or out-of-range labels: PyTorch's default ignore_index=-100 would otherwise exclude a label from the loss denominator while this loop still counted it in accuracy and example weighting. Masked objectives require the explicit numerator/denominator accounting taught in DL024. Average loss per example, not per batch: the final short batch must not count as much as a full batch. Evaluation restores the prior top-level mode after measuring, so it cannot silently leave the next training step in eval mode. This restoration assumes a uniform train/eval policy; it does not preserve deliberately mixed child modes such as frozen BatchNorm inside an otherwise training model. Such fine-tuning needs a per-module mode snapshot or an explicitly reapplied freezing policy.

PyTorch2.9's current AMP API is torch.autocast with torch.amp.GradScaler, rather than the old torch.cuda.amp namespace. Autocast chooses eligible operation dtypes, not all tensors uniformly. FP16 scaling protects small backward values but cannot fix a forward overflow. BF16's larger exponent range often avoids the need for scaling but has fewer mantissa bits; neither “same numerics” nor “no downside” is correct.

def run_epoch(model, loader, optimizer=None, device='cpu', amp=False,
              scaler=None, clip_norm=None):
    device=torch.device(device); training=optimizer is not None
    if amp and device.type != 'cuda':
        raise ValueError('this recipe enables AMP only for an explicitly selected CUDA run')
    if training and amp and scaler is None:
        raise ValueError('persistent scaler required for FP16 training')
    was_training=model.training; model.train(training)
    loss_sum=correct=count=0
    try:
        for xb,yb in loader:
            xb,yb=xb.to(device),yb.to(device)
            if yb.dtype != torch.long or yb.ndim != 1:
                raise ValueError('integer class targets required')
            if training: optimizer.zero_grad(set_to_none=True)
            with torch.set_grad_enabled(training):
                with torch.autocast(device_type=device.type,dtype=torch.float16,enabled=amp):
                    logits=model(xb)
                    if logits.ndim != 2 or logits.shape[0] != len(yb):
                        raise ValueError('expected batch-by-class logits')
                    if ((yb < 0) | (yb >= logits.shape[1])).any():
                        raise ValueError('targets must name a class; ignored labels are not supported')
                    loss=F.cross_entropy(logits,yb)
                if not torch.isfinite(loss): raise FloatingPointError('nonfinite loss')
                if training:
                    if amp:
                        scaler.scale(loss).backward()
                        if clip_norm is not None:
                            scaler.unscale_(optimizer)
                            torch.nn.utils.clip_grad_norm_(model.parameters(),clip_norm,
                                                           error_if_nonfinite=True)
                        scaler.step(optimizer); scaler.update()
                    else:
                        loss.backward()
                        if clip_norm is not None:
                            torch.nn.utils.clip_grad_norm_(model.parameters(),clip_norm,
                                                           error_if_nonfinite=True)
                        optimizer.step()
            loss_sum+=loss.item()*len(yb)
            correct+=(logits.argmax(1)==yb).sum().item(); count+=len(yb)
    finally:
        model.train(was_training)
    if not count: raise ValueError('empty loader')
    return {'loss':loss_sum/count,'accuracy':correct/count,'count':count}
 
def train_cifar_local(root, epochs=50, warmup=3, seed=27):
    if epochs <= warmup: raise ValueError('epochs must exceed warmup')
    torch.manual_seed(seed); np.random.seed(seed)
    train_dl,val_dl,test_dl,manifest=local_cifar_loaders(root,seed=seed)
    model=ResNet18_CIFAR()  # CPU, random initialization; no resource provisioning
    opt=torch.optim.SGD(decay_groups(model),lr=.1,momentum=.9,nesterov=True)
    best_loss=float('inf'); best=None; history=[]; start=time.perf_counter()
    for epoch in range(epochs):
        lr=set_epoch_lr(opt,epoch,epochs,warmup=warmup)
        train_metrics=run_epoch(model,train_dl,opt)
        val_metrics=run_epoch(model,val_dl)
        history.append({'epoch':epoch,'lr_used':lr,'train':train_metrics,
                        'validation':val_metrics,'elapsed_seconds':time.perf_counter()-start})
        if val_metrics['loss'] < best_loss:
            best_loss=val_metrics['loss']
            best={'model':copy.deepcopy(model.state_dict()),
                  'optimizer':copy.deepcopy(opt.state_dict()),'epoch':epoch,
                  'torch_rng':torch.get_rng_state().clone(),
                  'loader_rng':train_dl.generator.get_state().clone(),
                  'numpy_rng':np.random.get_state(),'manifest':manifest,
                  'schedule':{'epochs':epochs,'warmup':warmup,'base_lr':.1}}
    model.load_state_dict(best['model'])
    final_test=run_epoch(model,test_dl)  # one official test read, after selection
    return model,history,final_test,best

Persist returned checkpoint and manifest only to a chosen local output path in your own experiment. A best-validation checkpoint supports model selection; an exact last-step resume additionally needs the current optimizer, scheduler position, augmentation/data-loader RNG and environment versions. Save the latest checkpoint separately if resuming rather than selecting. Loading state from unknown files is a trust decision; use supported weights-only loading where applicable and do not execute arbitrary downloaded objects.

zero_grad(set_to_none=True) avoids materializing zero gradients, but forgetting backward is not guaranteed to raise a NoneType error: optimizers can skip parameters with absent gradients. Check a parameter delta. Clipping follows backward and, under scaling, unscale before clip. With clip_norm supplied, this helper deliberately raises on a nonfinite total norm before scaler.step; it does not then reach GradScaler's usual skip-and-adjust path. Without clipping, the AMP branch leaves invalid-gradient skipping to GradScaler. The default non-AMP, unclipped branch checks loss finiteness but is not a general finite-gradient guard. A nonfinite gradient is not repaired by scaling an infinity to a finite norm.

Bounded CPU tests of the exact training functions

These tests intentionally use synthetic labels. They verify mechanics, not generalization. The exact full ResNet18_CIFAR forward and one synthetic training update are also checked; a tiny model makes repeated optimizer/evaluation controls cheap.

torch.manual_seed(27)
net=ResNet18_CIFAR().eval()
assert sum(p.numel() for p in net.parameters()) == 11173962
with torch.no_grad(): assert net(torch.randn(2,3,32,32)).shape == (2,10)
# One actual CIFAR-ResNet update on synthetic data tests model/loop integration.
full_batch=DataLoader(TensorDataset(torch.randn(2,3,32,32),torch.tensor([0,1])),batch_size=2)
full_before=net.fc.weight.detach().clone()
full_metrics=run_epoch(net,full_batch,torch.optim.SGD(decay_groups(net),lr=.001))
assert full_metrics['count']==2 and not torch.equal(full_before,net.fc.weight)
print('full CIFAR-ResNet synthetic update',full_metrics)
assert lr_factor(0,8,2)==.5 and lr_factor(1,8,2)==1
assert lr_factor(2,8,2)==1 and lr_factor(7,8,2)==0
assert lr_factor(0,1,0)==1 and lr_factor(3,4,3)==1  # one cosine point
pixels=np.array([[[[0,64,255]]],[[[255,192,0]]]],dtype=np.uint8)
mu,sd=channel_stats_uint8(pixels)
np.testing.assert_allclose(mu,[.5,128/255,.5])
np.testing.assert_allclose(sd,[.5,64/255,.5])
small=nn.Sequential(nn.Flatten(),nn.Linear(3*4*4,10))
xx=torch.randn(7,3,4,4); yy=torch.arange(7)%10
loader=DataLoader(TensorDataset(xx,yy),batch_size=3)
small.train(); before=small[1].weight.detach().clone()
metrics=run_epoch(small,loader,torch.optim.SGD(decay_groups(small),lr=.05))
assert metrics['count']==7 and not torch.equal(before,small[1].weight)
state=copy.deepcopy(small.state_dict())
eval_metrics=run_epoch(small,loader)
assert small.training and all(torch.equal(state[k],v) for k,v in small.state_dict().items())
with torch.no_grad(): direct=F.cross_entropy(small(xx),yy).item()
assert abs(eval_metrics['loss']-direct)<1e-6
bad=DataLoader(TensorDataset(xx,yy.float()),batch_size=3)
try:
    run_epoch(small,bad); raise AssertionError('float labels accepted')
except ValueError:
    pass
# Cross-entropy's default ignore_index must not silently change our denominator.
ignored=yy.clone(); ignored[0]=-100
bad_ignored=DataLoader(TensorDataset(xx,ignored),batch_size=3)
try:
    run_epoch(small,bad_ignored); raise AssertionError('ignored label accepted')
except ValueError:
    pass
assert small.training  # error path also restores the prior top-level mode
# An absent optimizer step really leaves these parameters unchanged.
small.zero_grad(set_to_none=True); snapshot=small[1].weight.detach().clone()
F.cross_entropy(small(xx),yy).backward()
assert torch.equal(snapshot,small[1].weight)
print('synthetic training/evaluation controls',metrics,eval_metrics)

Read logs as hypotheses, not diagnoses by percentage

Uniform10-class logits give loss $\log(10)=2.302585\ldots$. Loss near this number is a useful clue, not proof every prediction is uniform or the network never learned. Inspect logits, targets, gradient norms and parameter changes, and overfit one tiny real training batch before a long run.

• Not progressing: try an LR range, inspect disconnected/frozen parameters, verify step calls and image-label alignment. A high LR can produce divergence, but NaNs can also start in data or a forward operation; FP16 underflow is not the explanation for every NaN.
• Training strong, validation weak: evaluate training data under the same deterministic transform as validation before interpreting the gap. Then investigate overfitting, distribution shift, duplicates, labels and class balance. Missing augmentation is one possibility, not “almost certainly” the cause of a10-point gap.
• Both mediocre: insufficient optimization, preprocessing mismatch or model capacity can all matter.55% is not a universal broken-pipeline signature. A deliberately frozen backbone is not a bug if linear probing is the planned experiment.
• Inference differs: inspect eval mode, preprocessing, class mapping and BN buffers. Check that evaluation neither updates buffers nor alters model selection by touching the test set.

Choose extra epochs if the controlled learning curve is still improving, more capacity if optimization/pipeline checks indicate a representation limit, or regularization when the training/validation gap warrants it. None is the only available lever. Early stopping and schedule length interact; a20-epoch ablation should use its own declared horizon, not accidentally compare an unfinished50-epoch schedule.

Ablation protocol and modern extensions

Run baseline, no-momentum, constant-LR, no-crop/flip and no-decay variants with identical splits and several matched seeds. When momentum is0, also disable Nesterov (PyTorch requires positive momentum and zero dampening for it). Rank validation accuracy/loss with uncertainty and keep test data sealed until the design is frozen. Record wall time, parameter count, transform policy and LR actually used. There is no prefilled expected trajectory or guaranteed ranking.

Mixup/CutMix and RandAugment belong to DL029's executable contracts. EMA averages model parameters over steps; floating BN buffers need a stated copy/average/recalibration policy, and integer counters should not be blindly averaged. Stochastic depth drops residual branches during training and rescales retained branches; it is not ordinary elementwise dropout. Label smoothing changes the target distribution. These mechanisms can interact, so adding them all is not a controlled explanation.

At larger scale, avoid the original parameter-bucket mistakes: CIFAR ResNet18 is about11.2M, ImageNet ResNet18 about11.7M, ViT-Base about86M, and ViT-Large is not generically a1B model. DINOv2's family is not synonymous with10B+. Model size alone does not mandate an optimizer, image dataset or fixed accuracy band.

• AdamW decouples decay from the adaptive update. Lion keeps a momentum state and uses a sign update. Sophia periodically estimates diagonal curvature and clips preconditioned updates; memory depends on its estimator/state choices, not just its name.
• Schedule-free methods combine iterates to avoid a predefined decay schedule, but still require LR/averaging/mode contracts. The paper is not proof of a particular private production deployment.
• torch.compile can trade compilation overhead and graph specialization for faster repeated work. Break-even, shapes and graph breaks matter; a free20–40% improvement is not guaranteed.
• BF16/FP16/FP8 have different range/precision/scaling requirements. FP8 paper formats and supported accelerator kernels do not imply every CNN trains twice as fast or that arbitrary FP8 casting works.
• MAE/DINOv2/CLIP/SigLIP/EVA02 change the pretraining signal, not just the last line of this supervised pipeline. Large vision experiments such as ViT22B require their own data, scaling and sharding/checkpoint systems; this CPU pipeline is a conceptual foundation, not a reproduction of them.

Worked recall and next experiment

1. Recipe from memory: CIFAR stem,SGD(.1,.9,Nesterov),chosen decay groups,crop/flip and train-only normalization,declared warmup/cosine,logits-CE,deterministic validation,best checkpoint,one final test. AMP is optional and off in the CPU recipe.
2. Why exclude BN/bias decay? It is a tunable regularization policy for scale/shift parameters with different roles, not immunity to overfitting or a required accuracy increment.
3. Three log families: failed progress,train/validation divergence,and broadly poor performance. Each has multiple causes; show a falsifying check rather than memorizing2.30/55%/80% as diagnoses.
4. Cosine versus steps: cosine gives smooth scheduled decay; steps give piecewise drops. Compare under equal budget; cosine still has start,min,horizon and optional warmup choices.
5. What does GradScaler do? Scale loss for backward,unscale before checking/clipping,skip invalid optimizer steps and adjust scale. An explicit fail-fast check can raise before that skip path; this helper's optional nonfinite-norm error does exactly that. BF16's exponent range reduces one motivation, but finite range/roundoff remain.
6. Why revisit architecture claims? Recipe-controlled and equal-tuning-budget comparisons answer different questions; report both where possible. Do not infer every old result was spurious.
7. Why crop/flip rather than arbitrary rotation? Match task invariance. CIFAR has no digit6/9 classes, so that is an analogy, not its literal label explanation. Modest rotations may be a legitimate measured policy.
8. Largest LR mistake? LR can dominate progress, but wrong labels or splits can dominate everything. There is no rule that all other errors cost less than1%.

Stretch: add Mixup/CutMix then RandAugment one change at a time, logging clean validation accuracy,NLL and calibration;94–95% may be a goal, never a promised result. Tomorrow reconstruct the loop, and next week explain a recorded curve using interventions rather than inventing which knob caused its shape.

Sources and execution boundary

Primary sources: official CIFAR, torchvision0.24 CIFAR reader, PyTorch2.9 optimizer contracts, AMP examples, ResNet Strikes Back, Revisiting ResNets, AdamW, Lion, Sophia, schedule-free, FP8.

Only exact synthetic CPU tests are executed here; the full local CIFAR recipe and optional CUDA AMP branch are source-reviewed, not run. No dataset, weights or dependencies were downloaded. Previous: architectures · Next: transfer.