Chapter 4: Tool Use & Function Calling
How agents reach out of the LLM and into the world. Function calling fundamentals, JSON-mode and tool schemas, parallel tool calls, the Model Context Protocol (MCP) for portable tools, and the discipline of handling tool results and errors gracefully.
Chapter Overview
An LLM that cannot call tools is a calculator that cannot add. Tool use — also called function calling — is the API by which an agent asks the controller to execute a function and feed the result back into the context. It is the single most important capability the major LLM providers added in 2023, and it is what turns a chatbot into an agent.
Before function calling, agents emitted tool calls as raw text (Action: search['query']) and the controller parsed the string. Brittle: every escape character was a bug, every model update changed the format. Function calling moves the parsing into the inference layer: the model is trained to emit a structured tool-use block, the API guarantees valid JSON matching your schema, and the controller just dispatches the call.
This chapter covers:
- Function calling fundamentals — the JSON-schema contract between model and controller
- Tool schemas & JSON mode — designing schemas the model can follow reliably
- Parallel tool calls — fan out independent tools, wait on the slowest
- MCP — the Model Context Protocol for portable, multi-vendor tool servers
- Tool result handling & errors — what to put in the observation, how to surface failures
Chapter Roadmap
Click any topic to jump in
Function Calling
The schema-validated contract between LLM and controller. The single most important agent capability.
What the model emits, and how many it emits at once
Schemas & JSON Mode
How to design schemas the model picks reliably. JSON mode vs function calling vs structured output.
Parallel Tool Calls
Fan out independent tools, wait on the slowest. 4× wall-clock speedup is typical.
MCP
The portable tool protocol — write tools once, run them in any MCP-aware agent.
Result Handling
Be terse, be honest about errors, be structured. Observations are the binding context constraint.
Function calling is the contract: you describe your tools to the model in a JSON schema, the model decides when to call them, and the API guarantees the model's response will be a valid call (or a final text answer). It replaces the old 'parse the model's output for Action: tool_name(args)' loop with a typed, schema-validated mechanism.
In this topic
The Three-Step Loop
Every function-calling cycle has the same shape:
- You send a request to the LLM with the user message + a list of tool schemas (each: name, description, JSON-schema for parameters).
- The LLM responds with EITHER (a) a regular text completion (it is done) OR (b) a tool-use block containing the chosen tool name and a JSON object matching the schema for that tool's parameters.
- You execute the tool, then send a follow-up request with the original messages + the tool-use block + a tool-result block containing whatever the function returned.
The LLM then either calls another tool or produces a final text answer. The loop repeats until you get text instead of a tool call.
In the Anthropic API: messages alternate user / assistant and tool results come as user messages with a tool_result content block. In the OpenAI API: tool results come as tool role messages. Same shape, different wire format.
Tool Schemas — the Contract
A tool schema looks like this (Anthropic / OpenAI use the same JSON-Schema dialect):
{
"name": "get_weather",
"description": "Fetch current weather for a city.",
"input_schema": {
"type": "object",
"properties": {
"city": { "type": "string", "description": "City name (e.g. 'San Francisco')" },
"units": { "type": "string", "enum": ["metric", "imperial"], "default": "metric" }
},
"required": ["city"]
}
}
Four rules for schemas the model can use reliably:
- Description is the prompt. The model picks tools based on description text. Be specific: 'Search the customer support knowledge base. Use ONLY for support questions, NOT general web search.'
- Parameter descriptions matter as much as the tool description. The model will hallucinate parameter values if it doesn't know what they mean.
- Use enums where possible. "status": {"enum": ["open", "closed"]} constrains the model to valid values.
- Required vs optional. Mark optional anything with a sensible default. The fewer required parameters, the less chance the model picks one tool because another's required field is intimidating.
An agent has 5 tools but the model never calls one of them. What's likely wrong with the schema?