S120 · LLM Agents — Function Calling, Tools, Planning

Module M14: LLMs & Applications · Session 120 of 130 · Track: LLM 🤖

What you'll be able to do after this session

• Explain LLM Agents to a friend in one minute, out loud, no notes.
• Recognise it when you see it in real code / systems / papers.
• Complete the hands-on exercise at the end without looking up help.

Prerequisites

If you can't answer these in 30 seconds each, redo the linked session first:

• S116 — Prompting — Zero-Shot, Few-Shot, Chain-of-Thought, ReAct

Pre-read (skim before the session)

• 🎥 Intuition (5–15 min) · What is an AI agent? — IBM Technology (5 min) — Cleanest one-slide definition of an agent vs an LLM call.
• 🎥 Deep dive (30–60 min) · Building effective agents — Anthropic (deep dive) — 45-min talk on the patterns that actually work in production.
• 🎥 Hands-on demo (10–20 min) · Function calling with OpenAI — Greg Brockman demo — 15-min official demo of tool-use / function-calling.
• 📖 Canonical article · Anthropic — Building effective agents (engineering blog) — The best written guide to agent patterns. Read it twice.
• 📖 Book chapter / tutorial · Lilian Weng — LLM Powered Autonomous Agents — The canonical taxonomy: planning, memory, tools, reflection.
• 📖 Engineering blog · OpenAI — Function calling guide — Authoritative spec for JSON-schema tool definitions.

(a) Intuition · 5 min

The one-paragraph mental model. Why this concept exists, what problem it solves, and the analogy that makes it stick.

In one sentence: LLMs that do things, not just talk.

A plain LLM call is a function: text in → text out, one shot. An agent is a loop around that function: the LLM decides an action, an external system executes it (search the web, run code, query a DB, call an API), the result is fed back, the LLM decides the next action, and so on until it's done. The LLM is the brain; the loop + tools are the body.

Real-life analogy: a smart intern who can do things, not just answer questions. Ask a plain LLM "book me a flight" and it hallucinates a fake confirmation. Ask an agent with a search_flights(...) and book_flight(...) tool, and it actually goes and does it — checking dates, comparing options, calling the API, handling errors.

Three primitives you must know:

1. Function calling / tool use. The LLM emits structured JSON matching a tool schema you defined, instead of prose. {"name": "search_flights", "arguments": {"from": "SFO", "to": "BLR", "date": "2026-08-01"}}. Your runtime dispatches the call, feeds the JSON result back into the conversation, and the LLM continues.
2. Planning. For multi-step tasks the LLM first writes a plan ("1. Search for flights. 2. Compare prices. 3. Book cheapest under $500."), then executes step by step. Chain-of-thought (S116) is the atomic unit; ReAct is the classic loop.
3. Memory. The conversation history acts as short-term memory. Longer-term memory (past user preferences, prior sessions) is usually a RAG lookup (S117–119) or a scratchpad file the agent reads/writes.

Gotcha to carry forever: agents are just LLMs in a while loop with tools. There's no magic. Their failure modes are all consequences of that loop: they hallucinate tool arguments, they loop forever, they take unsafe actions if you gave them dangerous tools. Every agent framework (LangChain, LlamaIndex, CrewAI, AutoGen, OpenAI Assistants, Anthropic's own MCP) is a different opinion on how to structure the loop, the tools, and the safety rails. Start with a 30-line hand-rolled loop before touching any framework — you'll understand every framework 10× faster.

(b) Visual walkthrough · 15 min

Diagrams, worked examples, step-by-step visuals.

Worked example — "Book me the cheapest flight from SFO to BLR next Friday under $700."

Turn 1 (LLM sees the user message + your tool schemas):

Thought: I need to find flights first.
Action: search_flights({"from": "SFO", "to": "BLR", "date": "2026-08-01"})

Runtime executes → returns [{"airline": "Emirates", "price": 890, ...}, {"airline": "Air India", "price": 620, ...}, ...].

Turn 2 (LLM sees the results):

Thought: Air India is $620, under $700. Confirm with the user before booking.
Action: reply_to_user("Air India at $620, want to book?")

User says yes → LLM emits Action: book_flight({...}) → runtime books it.

Anthropic's five agent patterns (2024, from their "Building Effective Agents" post):

Rule: use the smallest pattern that works. A prompt chain beats a full agent 8 times out of 10 and is 10× cheaper and more reliable.

(c) Hands-on · 20 min

Code you type and run. Not pseudo-code — real, runnable, copy-pasteable.

Roll your own agent in ~60 lines — no framework — using OpenAI-compatible function calling:

# pip install openai
import json, math
from openai import OpenAI
 
client = OpenAI()   # or point to Together / Groq / Ollama base_url
MODEL = "gpt-4o-mini"
 
# 1. Define tools as JSON schema (this is what the LLM sees)
TOOLS = [
    {"type": "function", "function": {
        "name": "calc",
        "description": "Evaluate a Python arithmetic expression like 12*(3+4).",
        "parameters": {"type": "object",
            "properties": {"expr": {"type": "string"}}, "required": ["expr"]}}},
    {"type": "function", "function": {
        "name": "search_web",
        "description": "Search the web, returns a list of {title, snippet, url}.",
        "parameters": {"type": "object",
            "properties": {"q": {"type": "string"}}, "required": ["q"]}}},
]
 
# 2. Implement the tools in Python
def calc(expr: str):
    return {"result": eval(expr, {"__builtins__": {}}, {"math": math})}
 
def search_web(q: str):
    return {"hits": [{"title": f"fake result for {q}",
                      "snippet": "pretend the internet answered",
                      "url": "https://example.com"}]}
 
DISPATCH = {"calc": calc, "search_web": search_web}
 
# 3. Agent loop
def run_agent(user_msg, max_steps=8):
    messages = [
        {"role": "system", "content":
            "You are a careful assistant. Use tools when needed. "
            "Reply with a final answer when the task is complete."},
        {"role": "user", "content": user_msg},
    ]
    for step in range(max_steps):
        resp = client.chat.completions.create(
            model=MODEL, messages=messages, tools=TOOLS, temperature=0,
        )
        msg = resp.choices[0].message
        messages.append(msg.model_dump(exclude_none=True))
        if not msg.tool_calls:
            return msg.content                        # done
        for tc in msg.tool_calls:                     # execute each tool call
            fn = DISPATCH[tc.function.name]
            args = json.loads(tc.function.arguments)
            result = fn(**args)
            messages.append({"role": "tool", "tool_call_id": tc.id,
                             "content": json.dumps(result)})
            print(f"[step {step}] {tc.function.name}({args}) -> {result}")
    return "(hit max_steps without a final answer)"
 
print(run_agent("What is 17 * 234, and then search the web for the population of Hyderabad."))

Checklist:

1. LLM emits tool_calls for the calc → runtime prints calc({'expr': '17*234'}) -> {'result': 3978}.
2. Second turn: LLM emits search_web({'q': 'population of Hyderabad'}).
3. Third turn: LLM returns a natural-language final answer, no tool call.
4. Loop terminates within max_steps; if not, you'd cap and return best-effort.
5. temperature=0 makes tool-argument generation deterministic — critical for reliability.

Try this: add a dangerous tool run_shell(cmd: str) and prompt the agent with "delete all my files" — then implement a "human-in-the-loop" confirmation step before executing any run_shell call. Congratulations, you've just built the same safety pattern Anthropic's Claude Code and Cursor use.

(d) Production reality · 10 min

How this shows up in real systems, gotchas, war stories, common bugs.

Agents are the "let's throw more autonomy at the LLM" bet of 2024. Some of that bet is paying off (Cursor, Devin, Claude Code, GitHub Copilot Workspaces, Anthropic MCP). Some of it is smoke — a lot of AutoGPT-style "fully autonomous agents" cost $50 in tokens per task and then quit halfway through, and they will not be running your business next quarter.

War stories. (1) *"Our agent got stuck in a loop for 20 turns burning $12."* Every agent loop needs a hard `max_iterations` cap. Also: if the LLM emits the same tool call twice in a row with identical args, either short-circuit or force the LLM to reflect. (2) *"Our agent hallucinated a tool that didn't exist."* Some models emit tool names that aren't in the schema. Reject with an error `"unknown tool"` in the observation — the LLM usually recovers. (3) *"It called delete_customer() by accident."* Never expose destructive tools without explicit user confirmation or a strict allowlist. Anthropic's MCP philosophy: capabilities scoped per-server, human-in-the-loop for anything writeable. (4) *"Latency is terrible."* Agents chain 5–20 LLM calls per user query. Options: use a smaller/faster model for planning steps, parallelize independent tool calls (`parallel_tool_calls=true`), cache prompt prefixes. (5) *"Prompt is huge."* Each turn re-sends the entire history including tool outputs. At turn 15 you're spending$0.20 per turn. Truncate old tool outputs, summarize past turns, or use Anthropic's prompt caching to keep a stable prefix cheap.

How top teams handle it. Cursor / Claude Code do a supervised loop — the LLM proposes an edit, the user (or a test suite) approves. Devin runs a container per agent with a virtual computer. Anthropic's official recommendation: start with a simple prompt chain, add routing when needed, add tools when you truly need actions, and only reach for a full agent loop when the task is dynamic and open-ended. The industry consensus: agents win in narrow, tool-rich verticals (coding, browser tasks, customer support with structured backends). Agents lose in open-ended "do my job" scenarios because their failure rate compounds — a 90%-per-step agent hits <60% success by step 5.

(e) Quiz + exercise · 10 min

1. In one sentence, what's the difference between an LLM call and an LLM agent?
2. What does "function calling" produce that plain generation doesn't?
3. Why must every agent loop have a max_iterations cap?
4. Name three of Anthropic's five agent patterns and when each is preferred.
5. What are two ways to keep agent latency and cost under control?

Stretch (connects to S116 — Prompting): Chain-of-thought (S116) shows the LLM "thinks" better with intermediate scratchwork. How do agent frameworks turn that scratchwork into actions? Trace the exact prompt structure that lets a model emit {"tool": "calc", "args": {...}} reliably — what's in the system prompt, what's in the tool schema, and what does the LLM output token-by-token?

Explain-out-loud test

If you can't teach these 3 things to a friend without notes, redo the session:

1. What is LLM Agents? (one sentence, no jargon)
2. When would you use it? (one concrete example)
3. When would you NOT use it? (one anti-pattern)

What comes next

• Next session (S121): Multi-Agent Orchestration — LangGraph, CrewAI Patterns
• Previous (S119): Vector Databases — pgvector, HNSW, IVF
• Hub: The 6-Month Learning Plan

Part of a 130-session evergreen learning series. Session structure: (a) intuition · (b) visual walkthrough · (c) hands-on · (d) production reality · (e) quiz + exercise. Duration: 60 minutes.