S120 · LLM Agents — Function Calling, Tools, Planning
How an LLM stops just talking and starts doing — the function-calling protocol, tool schemas, planning strategies (ReAct, Plan-Execute, Reflexion), and the failure modes every agent framework fights.
🎯 Build a small LLM agent with function calling that plans, executes tools, handles errors, and stops when done — using the same primitives every production framework uses.
Why this session exists
"Agents" is the most-hyped and least-understood word in the LLM stack. Under the hood every agent — LangChain, LlamaIndex, CrewAI, AutoGen, OpenAI's Assistants, Anthropic's tool use — is the same three-part loop: LLM plans, LLM calls a tool, we execute the tool and hand back the result. This session strips agents down to the primitives, teaches you the failure modes (infinite loops, tool schema mismatch, planning drift), and shows you exactly what a production agent needs beyond the demo.
- Explain function calling / tool use as a protocol on top of a chat completion, in one paragraph.
- Write a JSON schema for a tool that the LLM will call correctly on the first try.
- Implement a ReAct-style agent in ~60 lines using OpenAI's tool_calls or Anthropic's tool_use.
- Diagnose the four failure modes: schema mismatch, infinite loop, hallucinated tools, planning drift.
- Pick between ReAct, Plan-and-Execute, and Reflexion patterns for a new problem.
Prerequisites
- S116 · Prompting — you need the ReAct pattern mental model.
- S113 · LLM Sampling — tool calls should almost always run at temperature=0.
- S118 · Retrieval — retrieval is often ONE of an agent's tools.
(a) Intuition · 5 min
Picture an executive assistant who is smart, cheap, and infinitely patient — but has no arms. They can read, they can talk, they can plan. They cannot pick up a phone, open a browser, or file a document.
You solve this by giving them a whiteboard and a list of things a coworker will do for them if written correctly: 'CALL_CLIENT(name=Alice, message=...) → ok/failed', 'LOOKUP_ORDER(id=A-4821) → {…}'. The assistant writes commands on the whiteboard, the coworker executes each one, writes back the result, and the assistant reads it and decides the next step.
An LLM agent is that assistant. The whiteboard is the tool-call protocol (OpenAI's tool_calls, Anthropic's tool_use, Google's function_calling — all the same idea). Your code is the coworker. The LLM writes JSON like {"name":"get_weather","arguments":{"city":"Tokyo"}}; you parse it, run the function, and send the result back as the next message.
Loop until the LLM says 'I'm done' (usually by generating a final assistant message with no more tool_calls). That's an agent. Everything else — LangChain, LlamaIndex, CrewAI, AutoGen — wraps this loop with retries, logging, and pretty tracing.
- The LLM — decides which tool to call and with what arguments (planning).
- The tools — Python functions you registered, each with a JSON schema.
- The controller loop — your code that parses tool_calls, executes them, feeds results back.
- The stopping rule — max_steps, or 'no more tool_calls', or 'called finish()'.
- The memory — the growing message history (system + user + tool_call + tool_result + …).
Timeline of agent-making
- 2022ReAct paper · Yao et al.Reasoning + Acting in prompts. First reliable pattern for LLM tool use.
- 2023OpenAI function_callingGPT-3.5/4 shipped native tool_calls API. Made agents production-usable without regex parsing.
- 2023LangChain / LlamaIndex peakFramework explosion — thousands of stars, hundreds of integrations. Also: 'agents are unreliable' meme peaks.
- 2024Anthropic tool_use + Claude 3Cleaner tool-use protocol; strong reasoning makes multi-step agents more reliable.
- 2024Reflexion + Plan-ExecuteNew patterns — plan first, execute, then reflect and re-plan. Better on hard multi-step tasks.
- 2025Reasoning models (o1, Claude thinking)Long CoT baked into the model. Agents get more reliable because the LLM plans internally before emitting a tool call.
(b) Visual walkthrough · 15 min
The agent loop, drawn
The tool schema — what the LLM actually sees
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city. Returns temperature in Celsius and a short condition string.",
"parameters": {
"type": "object",
"properties": {
"city": { "type": "string", "description": "City name, e.g. 'Tokyo' or 'San Francisco'" },
"units": { "type": "string", "enum": ["celsius", "fahrenheit"], "default": "celsius" }
},
"required": ["city"]
}
}
}get_weather beats gw. The LLM decides which tool to call based on name + description; make both self-explanatory.
'Get the current weather for a city.' + example city names. LLMs pattern-match hard on examples in descriptions.
Use enum where possible; the LLM is much better at picking from a fixed set than free-typing.
The LLM will hallucinate values for missing required params instead of asking. Make required explicit.
Tool failures should return {'error': '...'} strings, not throw. The LLM needs to READ the failure to plan around it.
The three planning patterns
Think → Act → Observe · loop
- The default; every OpenAI/Anthropic tool_call loop is ReAct
- Cheap: one LLM call per step
- Drifts on tasks with >5–7 steps
- Best for reactive, well-scoped tasks
Make a full plan first, then execute steps
- LLM first outputs a list of steps ('1. Search for X. 2. Read result. 3. Compute Y.')
- Executor runs each step; can re-plan when a step fails
- Better on multi-hop research or code tasks
- Costs more tokens up front; wins on complex tasks
Execute, then critique + retry
- After an attempt, an LLM critique produces a 'lessons learned' note
- Next attempt uses that note as memory
- Big gains on coding/math benchmarks (HumanEval +20 pts)
- Expensive: N×–3N× cost of a single attempt
The message log — how it grows
What ends up in the messages array
"An agent that can call tools and loop will eventually solve the task. If it fails, I need a better model or more iterations."
Error compounds multiplicatively across steps. At 95% per-step reliability, a 20-step trajectory succeeds about 36% of the time; at 10 steps it is 60%. Adding iterations without adding verification makes things worse, because a wrong intermediate result is fed forward as fact and the agent then reasons confidently from a corrupted premise. Reliability comes from shortening trajectories and checking steps, not from more looping.
Because the myth is true in the demo regime everyone starts in: 3-step tasks, where 0.95³ is 86% and failures look like flukes that a retry fixes. The exponential only bites past roughly ten steps, which is exactly where useful autonomous work begins. It is also masked by the fact that the agent's narration stays fluent and confident even after it has gone off the rails.
Compute the reliability budget for your actual trajectory length before designing anything:
for p in (0.90, 0.95, 0.99):
for n in (5, 10, 20, 50):
print(f'p={p} steps={n:2d} success={p**n:.3f}')
# Then invert it: what per-step reliability do you need for
# 90% end-to-end success at n steps?
import math
for n in (5, 10, 20, 50):
print(f'{n:2d} steps needs p >= {0.90 ** (1/n):.4f}')
# 50 steps needs 99.8% per step. That is the real constraint.Why does ReAct interleave reasoning with acting, rather than planning fully upfront and then executing? Plan-then-execute is how humans do project work.
- 1A plan is a sequence of actions chosen under the information available when the plan was written.forced by · that is what planning means — committing to future choices now
- 2In a tool-using setting, most of the information needed to choose action k is produced by actions 1 through k−1. The search result determines which page to fetch; the query result determines which filter to apply.forced by · the environment is only partially observable, and tools are how you observe it
- 3So a full upfront plan must guess the outcomes of every intermediate step. Each guess has a failure probability, and the guesses compound exactly like the step reliability above.forced by · an unobserved outcome is a prediction, and predictions about tool results are unreliable
- 4Worse, a plan produced before observation is brittle in a specific way: when step 3 returns something unexpected, steps 4–10 are now premised on a false state, and the agent has no mechanism to notice because it is following the plan rather than reading the world.forced by · plan-following decouples action selection from current observation
- 5Interleaving fixes this by making every action conditional on all prior observations: the "plan" is recomputed at each step from the actual state, so an unexpected result changes the next action immediately rather than after ten wasted steps.forced by · replanning at every step is the cheapest possible form of error detection
Therefore ReAct is closed-loop control and plan-then-execute is open-loop control. Closed loop wins exactly when the environment is unpredictable, which is always true for tool use over real systems.
And note what this predicts: plan-then-execute should win back its advantage when the environment is predictable and steps are expensive — because then upfront planning enables parallelism and avoids paying for a reasoning pass before every action. That is precisely the regime where plan-and-execute architectures are used, and why the two patterns coexist rather than one replacing the other.
An agent is a control loop: observe → decide → act → observe. The LLM is the controller, tools are the actuators, and tool outputs are the sensors. What makes it hard is that the controller is stochastic and the sensor readings enter as text in a context window, where they are indistinguishable from the agent's own speculation.
Everything that makes agents reliable is a control-engineering move: shorten the loop, add feedback, bound the state, and put a hard limit on the actuators.
- Reliability is pn. Reduce n (fewer steps) before trying to raise p (better model).
- Tool descriptions are the API contract the model programs against. Ambiguous parameter descriptions are the most common cause of wrong tool calls — far more common than wrong tool choice.
- Tools must return errors as actionable text ("date must be YYYY-MM-DD, got 03/04/24"), not stack traces. The error message is the only feedback channel the controller has.
- Always bound the loop: max steps, max wall time, max spend. An unbounded agent loop is an unbounded bill and an unbounded blast radius.
Fire this model the moment you see: an agent looping on the same failing call · a task requiring more than ten tool calls · tool outputs dumped raw into context · an agent with write access and no confirmation step · someone proposing "just let it retry".
You are automating a multi-step workflow. Hard-coded pipeline, constrained agent with a fixed tool set, or fully autonomous agent?
Push work down this list, not up. Anything you can express as a pipeline should be one, because determinism is worth more than flexibility in anything you have to operate at 3am. Use the agent for the part of the problem that is genuinely variable, and wrap it in deterministic scaffolding on both sides.
The concrete design rule that follows from pn: every step you can move out of the agent's trajectory and into fixed code raises end-to-end reliability multiplicatively. A ten-step agent reduced to a three-step agent inside a pipeline is not a small optimisation — at 95% per step it takes you from 60% to 86%.
(c) Hands-on · 25 min
Build a real agent with 3 tools and a step limit. Uses OpenAI's tool_calls, but the pattern maps 1:1 to Anthropic tool_use.
"""mini_agent.py — a ~120-line LLM agent with function calling.
Requires: pip install openai
Set OPENAI_API_KEY. Model can be gpt-4o-mini for cheap smoke tests.
"""
from __future__ import annotations
import json
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
MODEL = "gpt-4o-mini"
MAX_STEPS = 8
# ---------- 1. Tools — plain Python functions ----------
def get_weather(city: str, units: str = "celsius") -> dict:
"""Fake weather — swap for a real API in prod."""
fake = {"tokyo": 22, "london": 12, "san francisco": 17}.get(city.lower())
if fake is None:
return {"error": f"unknown city {city!r}"}
temp = fake if units == "celsius" else fake * 9 / 5 + 32
return {"temp": temp, "units": units, "condition": "partly cloudy"}
def convert_temp(value: float, from_unit: str, to_unit: str) -> float:
if from_unit == to_unit:
return value
if from_unit == "celsius" and to_unit == "fahrenheit":
return value * 9 / 5 + 32
if from_unit == "fahrenheit" and to_unit == "celsius":
return (value - 32) * 5 / 9
return float("nan")
def finish(answer: str) -> str:
"""Emit the final answer and stop."""
return answer
TOOLS_TABLE = {
"get_weather": get_weather,
"convert_temp": convert_temp,
"finish": finish,
}
# ---------- 2. JSON schemas the LLM sees ----------
TOOLS_SCHEMA = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city. Returns temperature and condition.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, e.g. 'Tokyo'"},
"units": {"type": "string", "enum": ["celsius", "fahrenheit"], "default": "celsius"},
},
"required": ["city"],
},
},
},
{
"type": "function",
"function": {
"name": "convert_temp",
"description": "Convert temperature between celsius and fahrenheit.",
"parameters": {
"type": "object",
"properties": {
"value": {"type": "number"},
"from_unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
"to_unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["value", "from_unit", "to_unit"],
},
},
},
{
"type": "function",
"function": {
"name": "finish",
"description": "Return the final answer to the user and stop. ALWAYS call this when done.",
"parameters": {
"type": "object",
"properties": {"answer": {"type": "string"}},
"required": ["answer"],
},
},
},
]
# ---------- 3. The controller loop ----------
def run_agent(user_msg: str) -> str:
messages = [
{"role": "system",
"content": ("You are a concise assistant with tools. "
"Reason briefly, call tools when useful, and ALWAYS call "
"finish(answer=...) with your final answer. Do not narrate.")},
{"role": "user", "content": user_msg},
]
for step in range(MAX_STEPS):
resp = client.chat.completions.create(
model=MODEL,
messages=messages,
tools=TOOLS_SCHEMA,
tool_choice="auto",
temperature=0,
)
msg = resp.choices[0].message
messages.append(msg) # keep assistant turn in history
if not msg.tool_calls:
print(f"[step {step}] LLM stopped without finish; returning content")
return msg.content or "(empty)"
for tc in msg.tool_calls:
name = tc.function.name
try:
args = json.loads(tc.function.arguments)
except json.JSONDecodeError as e:
result = {"error": f"invalid JSON arguments: {e}"}
else:
fn = TOOLS_TABLE.get(name)
if fn is None:
result = {"error": f"unknown tool {name!r}"}
else:
try:
result = fn(**args)
except TypeError as e:
result = {"error": f"bad args: {e}"}
print(f"[step {step}] {name}({args}) -> {result}")
if name == "finish":
return args["answer"]
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": json.dumps(result),
})
return "(hit max_steps without finish)"
if __name__ == "__main__":
for q in [
"What's the weather in Tokyo, and what is that in Fahrenheit?",
"Is it warmer in London or San Francisco right now (report in F)?",
"What's the weather on Mars?", # should gracefully say unknown
]:
print(f"\n=== {q}")
print(">>", run_agent(q))Anatomy of the agent
What each block does
Try three experiments:
# 1. Set MAX_STEPS = 2. Watch agents get truncated.
# 2. Delete the 'ALWAYS call finish' line from system. Some queries hang.
# 3. Change get_weather to raise ValueError instead of returning {"error":...}.
# The loop crashes. That's why tools return, not raise.Each experiment is 30 seconds and teaches a durable lesson about production agents.
(d) Production reality · 15 min
Pre-function-calling, developers coaxed GPT-3.5 to emit JSON with prompt tricks and regex parsers. Every third response was 'Sure! Here's the JSON: {...' — parseable by luck.
OpenAI shipped native function_calling: JSON schema in, guaranteed structured tool_call out. Under the hood it's constrained decoding — token probabilities are masked to enforce schema.
Reliability jumped from ~85% to >99% on tool calls. Every LLM provider has since copied the interface.
An agent given a search tool and a browse tool searches for a topic, browses the top result, gets confused, searches again with slightly different terms, browses, gets confused, searches again... 40 minutes and $8 later.
Non-negotiable production controls: max_steps (5–10 typical), max_wall_clock (60s), max_cost (measure input+output tokens). Log every step. Any agent without all three will burn money in production within a week.
Anthropic's 'Building Effective Agents' post makes this the first recommendation: prefer explicit workflows over open-ended agents unless dynamic planning is really required.
Teams reached for 'agents' as a default for every new problem. Most were slower, less reliable, and more expensive than a simple deterministic workflow that hardcoded the sequence of steps.
Anthropic's public advice: use workflows (predefined step sequences with LLM calls at each step) whenever the flow is known. Reserve agents for problems where the sequence genuinely depends on runtime decisions — dynamic routing, self-correction, open-ended research.
Where this shows up next
(e) Recall + stretch · 10 min
Explain-out-loud test
- What is the loop that runs inside every LLM agent?
- What are the three limits you must enforce in production?
- When should you use a workflow instead of an agent?
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.