S112 · Encoder (BERT), Decoder (GPT), Enc-Dec (T5) — When Each

Module M13: NLP & Transformers · Session 112 of 130 · Track: LLM 🎭

What you'll be able to do after this session

• Explain Encoder (BERT), Decoder (GPT), Enc-Dec (T5) to a friend in one minute, out loud, no notes.
• Recognise it when you see it in real code / systems / papers.
• Complete the hands-on exercise at the end without looking up help.

Prerequisites

If you can't answer these in 30 seconds each, redo the linked session first:

• S111 — Full Transformer Architecture — Encoder + Decoder

Pre-read (skim before the session)

• 🎥 Intuition (5–15 min) · BERT, GPT, T5 in 10 minutes — CodeEmporium — Fast whiteboard tour of the three families and when each wins.
• 🎥 Deep dive (30–60 min) · Illustrated GPT-2 walkthrough — Jay Alammar (via StatQuest) — StatQuest's 30-min deep dive on decoder-only design choices.
• 🎥 Hands-on demo (10–20 min) · Fine-tune BERT for text classification — hands-on — 10-minute Hugging Face demo — clone, run, done.
• 📖 Canonical article · BERT: Pre-training of Deep Bidirectional Transformers — The original encoder-only paper — masked LM + NSP.
• 📖 Book chapter / tutorial · Exploring Transfer Learning with T5 (Google Research) — T5's 'everything is text-to-text' pitch, from the authors.
• 📖 Engineering blog · The Illustrated BERT / ELMo / GPT-2 — Alammar side-by-side diagrams — the clearest comparison anywhere.

(a) Intuition · 5 min

The one-paragraph mental model. Why this concept exists, what problem it solves, and the analogy that makes it stick.

In one sentence: The three families of LLMs.

The Transformer block is a LEGO piece. Depending on how you stack it, you get three totally different animals: BERT (encoder-only), GPT (decoder-only), and T5 (encoder + decoder). Same piece, different assembly, different jobs.

Think of it like a workshop. BERT is the reader — hand it a document and it produces a rich vector for every word using full bidirectional context. Great for classification, NER, embeddings, search. It cannot write new text because it was never trained to predict "the next token". GPT is the writer — it only knows how to guess the next word, one at a time, left to right. That's why it can chat, complete code, and hallucinate confidently. T5 is the translator — it reads with an encoder (like BERT) and writes with a decoder (like GPT), so it's ideal when the input and output are different sequences (translate, summarize, question-answer with an explicit passage).

Gotcha to carry forever: the architecture determines the superpower. Encoder-only ⇒ bidirectional context, no generation. Decoder-only ⇒ generation, but every token only sees the past. Enc-dec ⇒ both, at the cost of ~2× parameters. The industry mostly consolidated on decoder-only for LLMs (2020+) because scaling laws favored it, and because you can always prepend the input to the prompt and let the decoder "read" it.

(b) Visual walkthrough · 15 min

Diagrams, worked examples, step-by-step visuals.

Worked example — sentiment classification "This movie is great!"

• BERT approach: prepend [CLS], tokenize, run through 12 encoder blocks. Take the [CLS] vector (768-dim), pass through a tiny Linear(768→2) classifier head. Output: [pos, neg] logits. One forward pass, done.
• GPT approach: prompt the model with "Review: This movie is great! Sentiment:" and let it generate one token. Hope it says " positive". You are now doing NLP through English, not vectors.
• T5 approach: input "sst2 sentence: This movie is great!", decoder generates the literal string "positive". It was fine-tuned that way — everything is text-to-text.

Rule of thumb: if the output is a label → encoder. If the output is free text → decoder. If the input and output are both sequences of different lengths and the input is fixed → enc-dec.

(c) Hands-on · 20 min

Code you type and run. Not pseudo-code — real, runnable, copy-pasteable.

Load one model from each family in ~30 lines and see them behave differently:

# pip install transformers torch
from transformers import (
    AutoTokenizer, AutoModel,
    AutoModelForCausalLM, AutoModelForSeq2SeqLM,
    pipeline,
)
import torch
 
# --- 1. BERT (encoder-only): produces embeddings, cannot generate ---
tok_b = AutoTokenizer.from_pretrained("bert-base-uncased")
bert  = AutoModel.from_pretrained("bert-base-uncased")
ids = tok_b("Transformers are elegant.", return_tensors="pt")
with torch.no_grad():
    out = bert(**ids)
print("BERT [CLS] vector shape:", out.last_hidden_state[:, 0, :].shape)  # [1, 768]
 
# --- 2. GPT-2 (decoder-only): generates text ---
tok_g = AutoTokenizer.from_pretrained("gpt2")
gpt   = AutoModelForCausalLM.from_pretrained("gpt2")
prompt = tok_g("The transformer architecture is", return_tensors="pt")
gen = gpt.generate(**prompt, max_new_tokens=20, do_sample=False)
print("GPT-2:", tok_g.decode(gen[0], skip_special_tokens=True))
 
# --- 3. T5 (enc-dec): text-to-text ---
tok_t = AutoTokenizer.from_pretrained("t5-small")
t5    = AutoModelForSeq2SeqLM.from_pretrained("t5-small")
inp = tok_t("translate English to German: I love cats.", return_tensors="pt")
gen = t5.generate(**inp, max_new_tokens=20)
print("T5:", tok_t.decode(gen[0], skip_special_tokens=True))
 
# --- Bonus: try to make BERT generate — it fails gracefully ---
try:
    bert.generate(**ids)  # will raise: BERT has no generate method
except AttributeError as e:
    print("BERT cannot generate:", e)

Checklist:

1. BERT prints a [1, 768] vector — that's your embedding.
2. GPT-2 completes the sentence with 20 new tokens.
3. T5 outputs "Ich liebe Katzen." (or similar German).
4. The BERT .generate() call raises AttributeError — proof that architecture ≠ capability.
5. Note the model sizes: bert-base 110M, gpt2 124M, t5-small 60M — similar order.

Try this: swap bert-base-uncased for sentence-transformers/all-MiniLM-L6-v2 and use both .encode() on two sentences, then compute cosine similarity. That's how semantic search works.

(d) Production reality · 10 min

How this shows up in real systems, gotchas, war stories, common bugs.

The 2018–2020 era ran on BERT (Google Search used it for query understanding starting late 2019), T5 (Google's internal go-to for translation/summarization), and GPT-2/3 (OpenAI's decoder-only bet). By 2023 the market voted with its GPUs: decoder-only won for generation and the encoder tower is now used mostly as a feature extractor for retrieval (embedding models) rather than as a stand-alone product.

War stories. (1) Team ships BERT for a chatbot — it can classify intent perfectly but cannot produce a reply. They bolt on a template engine and call it AI. Six months later they migrate to GPT and delete the templates. (2) Team uses GPT to classify support tickets by prompting "Classify: {ticket} \n Category:". Works at 82% accuracy for $0.02 per ticket. Then someone fine-tunes DeBERTa-v3 (encoder-only) on 5000 labeled tickets — hits 94$0.00003 per ticket. The right architecture matters. (3) T5 for summarization — cheap and controllable because the output vocab is constrained to English tokens. But it can't do in-context learning: you must fine-tune. In 2024 the trend is: if you have <10k labels, use an encoder + a linear head; if you have 10k–1M, fine-tune T5/FLAN-T5; if you have >1M or want few-shot, use a decoder-only LLM with prompts + LoRA.

How top teams choose. Anthropic and OpenAI ship decoder-only. Cohere and Voyage ship encoder-only embedding models. Google ships all three (Gemini = decoder, text-embedding-004 = encoder, Vertex Translation = enc-dec). Meta open-sources Llama (decoder). Microsoft's OneNote Copilot uses a decoder-only backbone but an encoder-only reranker in front. Modern stacks are hybrids.

(e) Quiz + exercise · 10 min

1. Which family can perform in-context learning from a prompt without any fine-tuning?
2. What is the training objective of BERT vs GPT vs T5 (one line each)?
3. Why can't BERT generate text?
4. If you have 500 labeled examples and need to classify support emails, which family should you pick and why?
5. Why did the industry consolidate on decoder-only for LLMs (2020–2024)?

Stretch (connects to S111 — Full Transformer): Redraw the S111 architecture diagram three times — once with the decoder erased (BERT), once with the encoder + cross-attention erased (GPT), once unchanged (T5). For each, name one production model you'd deploy.

Explain-out-loud test

If you can't teach these 3 things to a friend without notes, redo the session:

1. What is Encoder (BERT), Decoder (GPT), Enc-Dec (T5)? (one sentence, no jargon)
2. When would you use it? (one concrete example)
3. When would you NOT use it? (one anti-pattern)

What comes next

• Next session (S113): LLM Sampling — Greedy, Beam, Top-k, Top-p, Temperature
• Previous (S111): Full Transformer Architecture — Encoder + Decoder
• Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: (a) intuition · (b) visual walkthrough · (c) hands-on · (d) production reality · (e) quiz + exercise. Duration: 60 minutes.