Training a Decoder on Local Text — Tiny Shakespeare Recipe

A complete local-text experiment, with honest boundaries

A small character-level decoder is a useful way to inspect tokenization, shifted targets, causal loss, optimizer state and generation. Tiny Shakespeare is one possible already-local corpus, not data bundled or downloaded by this audit. This lesson supplies a complete recipe for it and executes the same contracts on a clearly labeled artificial text fixture. No Shakespeare prose sample, 1.48 validation loss or laptop/GPU runtime is presented as a result of this run.

Use the complete gpt_lab.py from S040. Required background is training loops, cross-entropy, masking and train/eval modes. Outcomes: construct character vocabulary and disjoint streams, sample valid next-token windows, train with a defined LR clock, evaluate token-weighted NLL, save/reload trusted state, select a checkpoint, generate with explicit sampling policy and diagnose data/optimization failures.

The original narrative's outputs and timings were illustrative inventions, not evidence. A small model can learn spelling and formatting without being a reliable assistant. Character-level modeling is not intrinsically incapable of generalization; byte/character and subword approaches make different vocabulary/sequence-length tradeoffs. Tokenization does not mechanically force the model to learn only spelling or make BPE universally superior.

Tokenization, split identity and sampling windows

Read the local file as UTF-8, retain its byte hash and preserve whitespace/case. A character is a Python Unicode code point here, not necessarily a grapheme cluster or UTF-8 byte. Build sorted vocabulary from training text only. This strict recipe rejects held-out characters absent from the training alphabet instead of silently changing vocab or using an untrained unknown class. Alternatives are a declared fixed byte alphabet or an explicitly trained UNK token; whichever is chosen must be serialized.

Split the original stream before sampling windows. The common 90/10 split gives a development validation tail, not an untouched final test. Here 80/10/10 reserves a final test as well. Contiguous pieces prevent overlapping windows across split boundaries but do not prove different documents or eliminate repeated passages; for a serious corpus split by document/work and audit duplicate passages before training.

import math,copy,hashlib,json,io,os
from pathlib import Path
import torch
from torch import nn
from torch.nn import functional as F
from dataclasses import asdict
from gpt_lab import GPT,GPTConfig
 
class CharCorpus:
    def __init__(self,text,context,raw_sha256=None):
        self.text=text;self.context=context
        a=int(.8*len(text));b=int(.9*len(text));parts={'train':text[:a],'val':text[a:b],'test':text[b:]}
        self.itos=sorted(set(parts['train']));self.stoi={c:i for i,c in enumerate(self.itos)}
        if not self.itos:raise ValueError('empty training alphabet')
        self.data={}
        for name,part in parts.items():
            unknown=set(part)-set(self.itos)
            if unknown:raise ValueError(f'{name} has unseen characters: {sorted(unknown)!r}; choose a declared tokenizer policy')
            if len(part)<=context:raise ValueError('each split needs context+1 characters')
            self.data[name]=torch.tensor([self.stoi[c] for c in part],dtype=torch.long)
        self.manifest={'raw_sha256':raw_sha256 or hashlib.sha256(text.encode('utf8')).hexdigest(),
            'characters':len(text),'vocabulary':self.itos,'offsets':{'train':[0,a],'val':[a,b],'test':[b,len(text)]},
            'split_hashes':{k:hashlib.sha256(v.encode('utf8')).hexdigest() for k,v in parts.items()},'context':context}
    @classmethod
    def local(cls,path,context):
        raw=Path(path).read_bytes()
        return cls(raw.decode('utf8'),context,hashlib.sha256(raw).hexdigest())
    def encode(self,text):
        if any(c not in self.stoi for c in text):raise ValueError('prompt contains unseen character')
        return torch.tensor([[self.stoi[c] for c in text]],dtype=torch.long)
    def decode(self,ids):return ''.join(self.itos[i] for i in ids)
    def batch(self,split,batch_size,generator):
        if split not in self.data or batch_size<1:raise ValueError('split/batch contract')
        data=self.data[split];T=self.context
        starts=torch.randint(len(data)-T,(batch_size,),generator=generator)
        x=torch.stack([data[i:i+T] for i in starts])
        y=torch.stack([data[i+1:i+T+1] for i in starts])
        return x,y
 
@torch.no_grad()
def evaluate(model,corpus,split):
    mode=model.training;model.eval();total=0.;count=0
    try:
        data=corpus.data[split];T=corpus.context
        # Fixed non-overlapping target chunks; include final short chunk.
        for start in range(0,len(data)-1,T):
            length=min(T,len(data)-1-start)
            x=data[start:start+length][None];y=data[start+1:start+length+1][None]
            _,loss=model(x,y);total+=float(loss)*length;count+=length
        if count==0:raise ValueError('no evaluation targets')
        return total/count,count
    finally:model.train(mode)

For a length-N stream, starts are 0…N−T−1 inclusive because targets need T+1 characters. torch.randint(N-T,...) has an exclusive upper bound and includes the final legal start. Input and target shapes are (B,T), with y[:,t]=the next character after x[:,t]. Targets are not shifted again inside GPT. Training windows overlap within a split and positions within a window are correlated; B×T labels are not B×T independent observations.

For larger already-tokenized corpora, NumPy memmap can read windows without loading the whole file, but converting slices to int64/batching/device transfer copies data. “Zero-copy end to end” was inaccurate. Store token dtype, endianness, tokenizer/vocab hash, document boundaries and lengths with the file; avoid crossing document boundaries unless separators make that a deliberate objective.

Cross-entropy, perplexity and the uniform reference

N counts valid target tokens, not batches. Cross-entropy takes logits, combines log-softmax/NLL stably and averages over selected targets. Perplexity is the exponential of mean surprisal, equivalently the geometric mean inverse probability of observed tokens. Calling it an “effective branching factor” is an analogy, not proof the model considered exactly that many alternatives at every position or a bound on coherent sample quality.

Exactly uniform logits yield log(V) loss and V perplexity. A random initialized model need not have uniform logits; class-frequency biases, tied embeddings, scale and finite samples change initial loss. Deviations are a diagnostic prompt, not a logically exact bug assertion. At V=65, log(65)≈4.174387; exp(1.48)≈4.392946. The second number illustrates a transformation, not a target achieved here. Token-level perplexity cannot be compared across different tokenizers without a common unit such as bits per byte/character and a declared evaluation procedure.

uniform=torch.zeros(4,65,dtype=torch.float64)
y=torch.tensor([0,12,33,64])
assert math.isclose(F.cross_entropy(uniform,y).item(),math.log(65),rel_tol=1e-12)
assert math.isclose(math.exp(1.48),4.392945680918757,rel_tol=1e-12)
# Nonuniform random logits need NOT have log(V) loss.
nonuniform=uniform.clone();nonuniform[:,0]=5.
assert abs(F.cross_entropy(nonuniform,y).item()-math.log(65))>.01
print('uniform reference and nonuniform negative control: pass')

Complete trainer, schedule, selection and resume

The LR clock below is the number of completed optimizer updates. Warmup uses (step+1)/warmup, so the first update is not accidentally zero. Cosine reaches its floor by the last planned update and stays there if inspected later; warmup=0 is supported. A single-update run uses the floor by this explicit endpoint convention. This is a stated convention rather than an off-by-one imitation of a different script.

The CPU recipe uses FP32, AdamW's reviewed matrix/gain groups, finite checks and clipping. Validation is deterministic and does not consume the training sampler generator or dropout RNG. Best weights are deep-copied; last state retains optimizer, RNG, step, config, corpus identity and history. Selecting best weights and resuming last state are different operations.

def learning_rate(step,total,peak=3e-3,floor=3e-4,warmup=5):
    if total<1 or step<0 or not 0<=warmup<total or not 0<=floor<=peak:raise ValueError('schedule contract')
    if step>=total-1:return floor  # also defines the single-update edge case
    if warmup and step<warmup:return peak*(step+1)/warmup
    progress=min(1.,max(0.,(step-warmup)/max(1,total-1-warmup)))
    return floor+.5*(peak-floor)*(1+math.cos(math.pi*progress))
 
class Trainer:
    def __init__(self,corpus,cfg,total=60,batch_size=8,seed=41):
        if cfg.vocab_size!=len(corpus.itos) or cfg.block_size!=corpus.context:raise ValueError('model/data mismatch')
        torch.manual_seed(seed);self.corpus=corpus;self.cfg=cfg
        self.model=GPT(cfg);self.opt=self.model.configure_optimizers(lr=.003)
        self.generator=torch.Generator().manual_seed(seed+1)
        self.total=total;self.batch_size=batch_size;self.step=0
        self.best=float('inf');self.best_weights=None;self.history=[]
    def update(self):
        if self.step>=self.total:raise ValueError('planned updates exhausted')
        self.model.train();x,y=self.corpus.batch('train',self.batch_size,self.generator)
        lr=learning_rate(self.step,self.total,warmup=min(5,self.total-1))
        for group in self.opt.param_groups:group['lr']=lr
        self.opt.zero_grad(set_to_none=True);_,loss=self.model(x,y)
        if not torch.isfinite(loss):raise FloatingPointError('nonfinite loss before update')
        loss.backward();norm=nn.utils.clip_grad_norm_(self.model.parameters(),1.,error_if_nonfinite=True)
        self.opt.step();self.step+=1
        return {'step':self.step,'batch_nll':float(loss.detach()),'lr':lr,'gradient_norm':float(norm)}
    def assess(self):
        train,ntrain=evaluate(self.model,self.corpus,'train');val,nval=evaluate(self.model,self.corpus,'val')
        record={'step':self.step,'train_nll':train,'val_nll':val,'train_targets':ntrain,'val_targets':nval}
        self.history.append(record)
        if val<self.best:self.best=val;self.best_weights=copy.deepcopy(self.model.state_dict())
        return record
    def state(self):
        return copy.deepcopy({'config':asdict(self.cfg),'manifest':self.corpus.manifest,'model':self.model.state_dict(),
            'optimizer':self.opt.state_dict(),'sampler_rng':self.generator.get_state(),'torch_rng':torch.get_rng_state(),
            'total':self.total,'batch_size':self.batch_size,'step':self.step,'best':self.best,
            'best_weights':self.best_weights,'history':self.history})
    def restore(self,state):
        if state['config']!=asdict(self.cfg) or state['manifest']!=self.corpus.manifest:raise ValueError('checkpoint model/corpus mismatch')
        if state['total']!=self.total or state['batch_size']!=self.batch_size:raise ValueError('training schedule/batch mismatch')
        self.model.load_state_dict(state['model']);self.opt.load_state_dict(state['optimizer'])
        self.generator.set_state(state['sampler_rng']);torch.set_rng_state(state['torch_rng'])
        self.step=state['step'];self.best=state['best'];self.best_weights=copy.deepcopy(state['best_weights']);self.history=copy.deepcopy(state['history'])
    def selected_model(self):
        if self.best_weights is None:raise ValueError('assess before selecting')
        selected=copy.deepcopy(self.model);selected.load_state_dict(self.best_weights);selected.eval();return selected
 
def save_trusted(path,state):
    path=Path(path);temporary=path.with_name(path.name+'.tmp')
    torch.save(state,temporary);os.replace(temporary,path)
 
def run_local(path,output,steps=5000,width=384,layers=6,heads=6,context=256,batch_size=64):
    """Opt-in local corpus run; no network, auto-install or automatic GPU use."""
    corpus=CharCorpus.local(path,context)
    cfg=GPTConfig(block_size=context,vocab_size=len(corpus.itos),n_embd=width,n_layer=layers,n_head=heads,dropout=.2)
    trainer=Trainer(corpus,cfg,total=steps,batch_size=batch_size);trainer.assess()
    while trainer.step<steps:
        record=trainer.update()
        if trainer.step%max(1,steps//10)==0 or trainer.step==steps:
            print(json.dumps(trainer.assess()));save_trusted(output,trainer.state())
    selected=trainer.selected_model();test_nll,test_count=evaluate(selected,corpus,'test')
    prompt='\n' if '\n' in corpus.stoi else corpus.itos[0]
    result=selected.generate(corpus.encode(prompt),100,temperature=.8,top_k=min(40,len(corpus.itos)),generator=torch.Generator().manual_seed(90))
    report={'manifest':corpus.manifest,'config':asdict(cfg),'history':trainer.history,
        'test_nll':test_nll,'test_targets':test_count,'sample':corpus.decode(result[0].tolist()),'limits':'local corpus result only; not an upstream benchmark match'}
    Path(str(output)+'.json').write_text(json.dumps(report,indent=2))
    return report

Checkpoint selection must not perturb the next training update. Constructing a new randomly initialized GPT just to overwrite its weights consumes the global CPU RNG, changing later dropout even when validation itself is deterministic. selected_model therefore copies the existing model before loading best weights. The copy is independent, keeps tied weights tied and leaves the trainer's RNG and parameter tensors untouched.

Evaluation here resets context and learned positions at each fixed chunk. It counts every next-character target once (except the first character in each split), but the first target of each chunk sees only one preceding character. This is a chunk-reset NLL, not a full-context/sliding-window estimate; compare runs with the same context/reset policy. For a length-10 stream and context 4, target chunks have lengths 4,4,1, so aggregate nine token losses rather than averaging three batch means.

To run on an existing input.txt, save this lesson as train_text.py beside gpt_lab.py, import run_local, and call run_local('input.txt','local-last.pt',...) with an intentional CPU-sized configuration. Defaults express the original six-layer experiment but may be slow on CPU; no runtime promise is made. The function logs validation, stores last resumable state and writes a report with best-state test result and sample. To resume, reconstruct the same corpus/config/total/batch, read your own trusted checkpoint using torch.load(path,map_location='cpu',weights_only=True), call restore, then continue update/assess. Restricted loading reduces deserialization risk but is not a license to load arbitrary untrusted files.

For optional CUDA AMP, explicitly select a supported dtype/device; put forward/loss under autocast, use GradScaler for FP16 when appropriate, unscale before clipping, then step/update the scaler. Advance an update-based scheduler only under the chosen skipped-update policy. BF16 often avoids FP16-range issues but is not a NaN cure or universal 2× gain. This CUDA branch is guidance, not invoked code; the reproducible executed contract remains CPU FP32.

Executed synthetic training and exact continuation

The bundled text below is generated repetition, not Shakespeare, a natural-language benchmark or pretrained data. It tests state transitions, loss finiteness, deterministic same-runtime continuation, vocabulary round trips and generation. Repeated patterns across splits are deliberate for this contract and do not claim a challenging generalization test.

def copied_span_rates(sample,training_text,lengths=(20,40,80)):
    result={}
    for k in lengths:
        count=max(0,len(sample)-k+1)
        result[k]=None if count==0 else sum(sample[i:i+k] in training_text for i in range(count))/count
    return result
 
def loss_svg(history,path):
    if len(history)<2:return
    width,height=600,300;maxstep=max(x['step'] for x in history) or 1
    maxloss=max(x[key] for x in history for key in ('train_nll','val_nll')) or 1
    parts=['<svg xmlns="http://www.w3.org/2000/svg" width="600" height="300">','<rect width="100%" height="100%" fill="white"/>','<text x="50" y="20">Synthetic CPU NLL (not Shakespeare)</text>']
    parts.append('<path d="M50 40 V250 H550" stroke="black" fill="none"/>')
    for fraction in (0.,.5,1.):
        parts.append(f'<text x="8" y="{254-200*fraction}" font-size="12">{maxloss*fraction:.2f}</text>')
        parts.append(f'<text x="{46+500*fraction}" y="266" font-size="12">{maxstep*fraction:.0f}</text>')
    for key,color in [('train_nll','blue'),('val_nll','red')]:
        points=' '.join(f"{50+500*r['step']/maxstep:.2f},{250-200*r[key]/maxloss:.2f}" for r in history)
        parts.append(f'<polyline points="{points}" fill="none" stroke="{color}" stroke-width="2"/>')
    parts.extend(['<text x="50" y="285" fill="blue">train</text><text x="130" y="285" fill="red">validation</text>','<text x="320" y="285">optimizer updates →</text></svg>'])
    Path(path).write_text(''.join(parts))
 
if __name__=='__main__':
    text=('red blue\nblue red\n'*60)
    corpus=CharCorpus(text,8)
    cfg=GPTConfig(block_size=8,vocab_size=len(corpus.itos),n_embd=16,n_layer=1,n_head=2,dropout=.1)
    trainer=Trainer(corpus,cfg,total=30,batch_size=4);trainer.assess()
    for _ in range(10):trainer.update()
    trainer.assess();checkpoint=trainer.state()
    # Serialize our own state in memory, then execute uninterrupted next update.
    buffer=io.BytesIO();torch.save(checkpoint,buffer);buffer.seek(0)
    record=trainer.update();expected=copy.deepcopy(trainer.model.state_dict())
    resumed=Trainer(corpus,cfg,total=30,batch_size=4)
    resumed.restore(torch.load(buffer,weights_only=True,map_location='cpu'))
    again=resumed.update()
    assert record==again
    assert all(torch.equal(expected[k],resumed.model.state_dict()[k]) for k in expected)
    while resumed.step<resumed.total:
        resumed.update()
        if resumed.step%10==0:resumed.assess()
    selected=resumed.selected_model();nll,n=evaluate(selected,corpus,'test')
    assert n==len(corpus.data['test'])-1 and math.isfinite(nll)
    prompt='red';assert corpus.decode(corpus.encode(prompt)[0].tolist())==prompt
    output=selected.generate(corpus.encode(prompt),60,temperature=.8,top_k=4,generator=torch.Generator().manual_seed(12))
    sample=corpus.decode(output[0].tolist())
    assert copied_span_rates('abcd','abcd',(4,5))=={4:1.,5:None}
    assert learning_rate(29,30)==3e-4 and learning_rate(100,30)==3e-4
    assert all(math.isfinite(row['val_nll']) for row in resumed.history)
    print('synthetic only: exact continuation pass; history',json.dumps(resumed.history))
    print('synthetic test nll/count/sample',nll,n,repr(sample))
    print('synthetic copied-span diagnostic',copied_span_rates(sample,text[:int(.8*len(text))]))
    if os.environ.get('LOSS_FIGURE_PATH'):loss_svg(resumed.history,os.environ['LOSS_FIGURE_PATH'])

The temperature exercise should return measured samples, not promised “LORD” loops. This helper uses the selected checkpoint and a private generator per temperature, keeping the seed and prompt fixed. The same random seed does not force the same tokens after distributions change.

def temperature_probe(model,corpus,prompt,temperatures=(.2,1.3),new_tokens=24,seed=12):
    samples=[]
    for temperature in temperatures:
        generated=model.generate(corpus.encode(prompt),new_tokens,temperature=temperature,
            generator=torch.Generator().manual_seed(seed))
        samples.append({'temperature':temperature,'seed':seed,'prompt':prompt,
            'sample':corpus.decode(generated[0].tolist())})
    return samples
 
if __name__=='__main__':
    rng_before=torch.get_rng_state().clone()
    sampler_before=resumed.generator.get_state().clone()
    chosen=resumed.selected_model()
    assert torch.equal(rng_before,torch.get_rng_state())
    assert chosen.head.weight is chosen.token.weight
    assert chosen.token.weight is not resumed.model.token.weight
    samples=temperature_probe(chosen,corpus,'red')
    assert torch.equal(rng_before,torch.get_rng_state())
    assert torch.equal(sampler_before,resumed.generator.get_state())
    assert all(len(row['sample'])==27 for row in samples)
    print('measured synthetic temperature probe, not Shakespeare',samples)

Read loss curves without overdiagnosing

Evaluate train and validation in the same eval mode and reduction; a training batch loss with dropout enabled is not directly comparable to a deterministic held-out loss. Falling training NLL alongside rising validation NLL is evidence consistent with overfitting, but the gap is not a numerical measure of “amount memorized.” Distribution shift, regularization, split difficulty and noise also affect it. Equal train/validation loss does not prove leakage; inspect offsets, hashes, duplicate documents and batch provenance.

Initial loss far from log(V) warrants checking logits, labels, vocabulary, scale and reductions. A much lower value is not by itself proof of target leakage; nonuniform data and informative initialization can change it. Test causal invariance and deliberate target-copy controls. NaNs require locating the first nonfinite input, logits, loss, gradient or optimizer state; excessive LR, invalid masks, overflow and bad data are possibilities. Clipping only helps certain large finite gradients. Checkpoint the last finite state rather than treating clipping or BF16 as a universal remedy.

A plateau is not uniquely “LR too small” or “model too small.” Compare a tiny-batch fit, gradient/optimizer membership, correct shift, schedule and held-out split. Repetition can come from learned distributions, context policy, data duplication or sampling; raising temperature is not guaranteed to cure it. Record prompt IDs, model mode, temperature, top-k, seed and actual sampled output rather than placing invented Bard dialogue beside a training claim.

Copied-span rates count all length-k windows, including the last legal one; there are length−k+1 windows. Long verbatim overlap is a useful memorization signal, but repeated boilerplate/short alphabet patterns and corpus search coverage matter. A single pass over a huge corpus can still memorize rare sequences; “one epoch means cannot memorize” is false.

Choose an intervention from evidence: better/more relevant data, capacity changes, regularization, learning-rate changes or early stopping. More data can hurt through quality/shift, regularization can improve validation rather than only lower a ceiling, and the best observed validation checkpoint is not guaranteed the best future/test checkpoint. Keep final test untouched while tuning.

Correct token budgets and modern training context

The original budget arithmetic was wrong:

• 5000 updates×64 sequences×256 positions = 81,920,000 token presentations, not 200M.
• For an illustrative 10M-parameter model, 20N=200,000,000, so the ratio is 0.4096, not20×.
• If a model sees15T tokens with8B parameters, D/N=1875 tokens/parameter, while D/(20N)=93.75, not1875× the20N reference.

Chinchilla studies compute-optimal allocation across model sizes and data budgets in its training regime. The widely quoted20 tokens/parameter is a rough reference, not a theorem or an overfitting threshold. Repeating a tiny corpus81.92M times in token presentations is not the same as81.92M unique training tokens. Muennighoff et al.'s Scaling Data-Constrained Language Models specifically studies decreasing value of repeated data; it is not simply a paper proving inference-cost dominance. Inference demand can motivate more training on a smaller model, but quantify the serving/training cost objective separately.

Muon changes matrix update geometry; it is not guaranteed15% faster loss reduction and was not substituted here. Data mixing/curation changes training distributions and needs controlled ablations; a private model's quality cannot be attributed “mostly” to curation without evidence. BitNet b1.58 studies ternary-weight training; it is not a one-hour post-hoc cast of arbitrary GPT weights or an established TinyShakespeare result. Quantization-aware training, activations/scales and kernels are part of that method.

Simple JSON/stdout logging is sufficient for this lab; trackers are optional, not proof of professionalism. Checkpointing is included despite a small run, because exact continuation is a teaching objective. No distributed launch, paid service, dataset download or automatic accelerator selection is hidden in the recipe. A KV cache can reduce redundant inference work, but it is not a guaranteed100× speedup and requires position/cache equivalence checks beyond this lesson.

Practice and worked answers

1. Initial loss. Uniform logits give log(V), about4.174387 for65 classes. Random initialization only approximates that under small similar logits; it is a diagnostic reference, not an architecture-independent exact assertion.
2. Batch construction. Choose i from0…N−T−1, take x=data[i:i+T], y=data[i+1:i+T+1], stack B windows. Split before sampling; verify no boundary crossing and no second shift in the model.
3. Failure fingerprints. NaN around any step requires tracing nonfinite values and checking LR/mask/precision, not blindly clipping. Equal validation/training losses require split/provenance inspection, not an automatic leakage verdict.
4. Perplexity. exp(1.48)≈4.392946 is a geometric inverse-likelihood measure per character under this tokenizer. A uniform65-class predictor has65. Neither value alone certifies prose coherence or absence of memorization.
5. When not warmup/cosine? A validated short fine-tuning or constant-LR baseline may use another schedule. Match the optimizer-update clock, total budget, resume behavior and validation; no task universally requires or forbids warmup.
6. **Budget.**81,920,000 presentations divided by200,000,000 is0.4096. Tiny-corpus repetition, capacity, optimizer and validation behavior explain potential overfitting more directly than comparing this rough scaling-law ratio.

For a controlled exercise, change exactly one hyperparameter, predict which curve/metric may change, rerun with recorded seeds and inspect actual samples plus overlap. Review the sampler and loss normalization tomorrow, and reproduce exact continuation after a week.

Sources, optional resources and limits

Primary: nanoGPT trainer, character preparation code, character config, CrossEntropyLoss2.9, AMP2.9, Chinchilla, data-constrained scaling, Llama3 report, BitNet. Exact source passages and raw hashes are recorded separately from local arithmetic/runtime outputs.

Optional original GPT video, tokenizer video, and raw corpus location are retained as references; the dataset was not fetched. Next steps: previous · next: KV cache.

Executed: bundled synthetic CPU training, exact same-runtime checkpoint continuation, full split token accounting, generation and math/negative controls. Unexecuted: TinyShakespeare/other external corpus training, pretrained benchmark matching, GPU AMP/compile, distributed runs, audible or visual application behavior, build/deploy. Source publication metadata is not end-to-end release acceptance.