Loading...
How agents reach out of the LLM and into the world. Function calling fundamentals, JSON-mode and tool schemas, parallel tool calls, the Model Context Protocol (MCP) for portable tools, and the discipline of handling tool results and errors gracefully.
An LLM that cannot call tools is a calculator that cannot add. Tool use — also called function calling — is the API by which an agent asks the controller to execute a function and feed the result back into the context. It is the single most important capability the major LLM providers added in 2023, and it is what turns a chatbot into an agent.
Before function calling, agents emitted tool calls as raw text (Action: search['query']) and the controller parsed the string. Brittle: every escape character was a bug, every model update changed the format. Function calling moves the parsing into the inference layer: the model is trained to emit a structured tool-use block, the API guarantees valid JSON matching your schema, and the controller just dispatches the call.
This chapter covers:
Click any topic to jump in
The schema-validated contract between LLM and controller. The single most important agent capability.
What the model emits, and how many it emits at once
How to design schemas the model picks reliably. JSON mode vs function calling vs structured output.
Fan out independent tools, wait on the slowest. 4× wall-clock speedup is typical.
The portable tool protocol — write tools once, run them in any MCP-aware agent.
Be terse, be honest about errors, be structured. Observations are the binding context constraint.
Function calling is the contract: you describe your tools to the model in a JSON schema, the model decides when to call them, and the API guarantees the model's response will be a valid call (or a final text answer). It replaces the old 'parse the model's output for Action: tool_name(args)' loop with a typed, schema-validated mechanism.
Every function-calling cycle has the same shape:
The LLM then either calls another tool or produces a final text answer. The loop repeats until you get text instead of a tool call.
In the Anthropic API: messages alternate user / assistant and tool results come as user messages with a tool_result content block. In the OpenAI API: tool results come as tool role messages. Same shape, different wire format.
A tool schema looks like this (Anthropic / OpenAI use the same JSON-Schema dialect):
{
"name": "get_weather",
"description": "Fetch current weather for a city.",
"input_schema": {
"type": "object",
"properties": {
"city": { "type": "string", "description": "City name (e.g. 'San Francisco')" },
"units": { "type": "string", "enum": ["metric", "imperial"], "default": "metric" }
},
"required": ["city"]
}
}
Four rules for schemas the model can use reliably:
An agent has 5 tools but the model never calls one of them. What's likely wrong with the schema?