S121 · Multi-Agent Orchestration — LangGraph, CrewAI Patterns

Module M14: LLMs & Applications · Session 121 of 130 · Track: LLM 🕸️

What you'll be able to do after this session

• Explain Multi-Agent Orchestration to a friend in one minute, out loud, no notes.
• Recognise it when you see it in real code / systems / papers.
• Complete the hands-on exercise at the end without looking up help.

Prerequisites

If you can't answer these in 30 seconds each, redo the linked session first:

• S120 — LLM Agents — Function Calling, Tools, Planning

Pre-read (skim before the session)

• 🎥 Intuition (5–15 min) · What are AI Agents? — IBM's plain-English intro to why multi-agent setups exist.
• 🎥 Deep dive (30–60 min) · LangGraph: Multi-Agent Workflows — LangChain's own walkthrough of graph-based agent orchestration.
• 🎥 Hands-on demo (10–20 min) · Build a CrewAI Multi-Agent System — end-to-end CrewAI research crew in ~15 min.
• 📖 Canonical article · LangGraph Docs — Multi-agent Systems — the reference for supervisor / swarm / hierarchical patterns.
• 📖 Book chapter / tutorial · Anthropic — Building Effective Agents — canonical taxonomy of orchestrator-workers, routing, parallelization.
• 📖 Engineering blog · Microsoft — AutoGen: Multi-Agent Conversation Framework — MSR's take on conversational multi-agent orchestration.

(a) Intuition · 5 min

The one-paragraph mental model. Why this concept exists, what problem it solves, and the analogy that makes it stick.

In one sentence: When one agent isn't enough.

Imagine you're running a small newsroom. One person can't research, fact-check, write, edit, and publish all at once — not well, anyway. So you hire a research reporter, a fact-checker, an editor, and a publisher. Each has a narrow job, a clear handoff, and a boss (the managing editor) who decides who does what next. That's multi-agent orchestration in one paragraph: instead of asking one giant LLM prompt to do everything, you split the work across specialised agents and add a coordinator that routes tasks between them.

Why does this exist? A single agent with 20 tools and a 10,000-token system prompt gets confused, hallucinates tool calls, and loses track of the goal halfway through. Multiple small agents — each with 2-3 tools and a tight role — stay focused, are cheaper to debug, and let you swap models per role (a cheap model for extraction, a smart one for reasoning). Frameworks like LangGraph (state-machine style), CrewAI (role-based), and AutoGen (conversational) all give you the plumbing: shared state, message passing, handoffs, and human-in-the-loop checkpoints.

The gotcha you must carry forever: multi-agent is not automatically better than single-agent — it's just different failure modes. You trade "one confused agent" for "five agents in a coordination deadlock, each waiting for the other, burning $0.20 per turn". Start with one agent + tools. Only split when a single agent measurably fails.

(b) Visual walkthrough · 15 min

Diagrams, worked examples, step-by-step visuals.

Three canonical topologies dominate real deployments. Pick based on who decides what runs next.

Worked example — supervisor pattern for a research assistant:

The key insight: the supervisor is just another LLM call that reads state and returns the name of the next agent (or END). LangGraph models this as a directed graph; each node is an agent, edges are conditional based on state.

Worked example — CrewAI role play: Same task, but instead of a supervisor, each agent has a role, goal, and backstory. The framework runs them sequentially (or in parallel) based on Task definitions. Less flexible than LangGraph, but 10 lines of code to prototype.

(c) Hands-on · 20 min

Code you type and run. Not pseudo-code — real, runnable, copy-pasteable.

Below: a minimal LangGraph supervisor with 2 worker agents (researcher + writer) using OpenAI. Runs in ~30 seconds on gpt-4o-mini.

# pip install langgraph langchain-openai
import os
from typing import TypedDict, Annotated, Literal
from langgraph.graph import StateGraph, END
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
 
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
 
class State(TypedDict):
    task: str
    research: str
    draft: str
    next: str
 
def researcher(state: State) -> dict:
    prompt = f"Research this topic in 3 bullet points: {state['task']}"
    res = llm.invoke([HumanMessage(content=prompt)]).content
    return {"research": res}
 
def writer(state: State) -> dict:
    prompt = f"Write a 100-word summary using these notes:\n{state['research']}\nTask: {state['task']}"
    res = llm.invoke([HumanMessage(content=prompt)]).content
    return {"draft": res}
 
def supervisor(state: State) -> dict:
    if not state.get("research"):
        return {"next": "researcher"}
    if not state.get("draft"):
        return {"next": "writer"}
    return {"next": "END"}
 
def route(state: State) -> Literal["researcher", "writer", "END"]:
    return state["next"]
 
g = StateGraph(State)
g.add_node("supervisor", supervisor)
g.add_node("researcher", researcher)
g.add_node("writer", writer)
g.set_entry_point("supervisor")
g.add_conditional_edges("supervisor", route,
    {"researcher": "researcher", "writer": "writer", "END": END})
g.add_edge("researcher", "supervisor")
g.add_edge("writer", "supervisor")
app = g.compile()
 
result = app.invoke({"task": "Why is KV cache important for LLM serving?"})
print("--- RESEARCH ---\n", result["research"])
print("--- DRAFT ---\n", result["draft"])

Observe when you run:

1. Total token cost printed by OpenAI dashboard — usually ~$0.002 for this run.
2. The graph visits supervisor 3 times (before researcher, before writer, at END).
3. state accumulates — each node returns a partial dict, LangGraph merges it.
4. If you comment out the writer branch, execution loops forever — supervisors need a real END condition.
5. Latency: sequential, so ~total = sum of node latencies. Parallel branches would need add_edge fan-out.

Try this modification: Add a critic node that reviews draft and either returns {"next": "writer"} (revise) or {"next": "END"}. Cap max revisions at 2 to prevent infinite loops.

(d) Production reality · 10 min

How this shows up in real systems, gotchas, war stories, common bugs.

**War story #1 — the $400 overnight bill.** A startup shipped a CrewAI research crew where the supervisor and 3 workers all used GPT-4. Each user query triggered ~15 LLM calls (supervisor decides, worker acts, worker returns, supervisor decides again…). One viral tweet, 2,000 users, one night →$400 in OpenAI charges. Fix: route supervisor to gpt-4o-mini (cheap, deterministic routing), keep GPT-4 only for the actual reasoning node. 10× cost reduction, same quality.

War story #2 — the deadlock. A team built a "collaborative" swarm where Agent A always handed off to B and B always handed off to A. No explicit exit. It ran until the token limit blew. Fix: every graph needs (a) a max-step counter enforced by the framework, (b) at least one node that can emit END, (c) an eval that runs the graph on 50 golden tasks and asserts termination.

Common bugs top teams handle for:

• State bloat — appending full chat history to shared state means turn 20 costs 20× turn 1. Compress with a summariser node or use LangGraph's checkpointer + selective read.
• Hidden tool retries — an agent that fails a tool call, then the supervisor sends it back, then it fails again. Wrap tools with circuit breakers.
• Non-determinism during eval — same input, different agent path. Set temperature=0 for routing decisions; only creative nodes get temperature.
• Silent handoff failures — agent returns "handing off to writer" as text instead of actually calling the handoff tool. Use structured outputs (JSON schema / function calling) for all handoff decisions.

Anthropic's own guidance ("Building Effective Agents"): start with the simplest workflow that works. Chain → route → parallelize → orchestrator-workers → autonomous. Only climb the ladder when you have evidence the current tier fails.

(e) Quiz + exercise · 10 min

1. What are the three canonical multi-agent topologies, and when would you pick each?
2. In a LangGraph supervisor pattern, what does the supervisor node actually return, and why is that important?
3. Give one concrete failure mode that gets worse when you split a single agent into three agents.
4. Why should the supervisor typically use a cheaper/smaller model than the worker nodes?
5. What's the difference between CrewAI's role-based orchestration and LangGraph's state-machine orchestration in terms of flexibility vs code-lines?

Stretch problem (connects to S120): In S120 you built a single agent with function calling. Convert that agent into a 2-agent LangGraph: one agent that only plans (decides which tools to call) and one that only executes (actually invokes them). Compare accuracy and token cost on 10 tasks. When does splitting help? When does it hurt?

Explain-out-loud test

If you can't teach these 3 things to a friend without notes, redo the session:

1. What is Multi-Agent Orchestration? (one sentence, no jargon)
2. When would you use it? (one concrete example)
3. When would you NOT use it? (one anti-pattern)

What comes next

• Next session (S122): LLM Evaluation — LLM-as-Judge, RAGAS, Golden Sets
• Previous (S120): LLM Agents — Function Calling, Tools, Planning
• Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: (a) intuition · (b) visual walkthrough · (c) hands-on · (d) production reality · (e) quiz + exercise. Duration: 60 minutes.