Tool calling is what turns a language model into something that can act. It is also where a great deal of agent cost and unreliability originates, and almost all of that traces back to how the tools were described rather than how they were implemented.
What actually happens
The model does not execute anything. It emits a structured request naming a function and arguments. Your code reads it, decides whether to run it, runs it, and returns the result as a new message. Every security boundary sits in your code, not the model.
The loop
Step 3 is the one to internalise. The model produced text requesting a call. Nothing obliges you to make it. Validation, authorisation, rate limiting and sanity checks all belong there — a model that has been manipulated into requesting delete_all_orders() is only dangerous if your code runs it without asking.
A tool definition
Everything the model uses to decide is in the description fields. The name is a label; the description is the instruction. Most tool-selection problems are description problems.
Writing descriptions that work
| Include | Because |
|---|---|
| What it does | The baseline |
| When to use it | Selection depends on this more than on capability |
| When not to use it | The single most effective addition when tools are similar |
| What it returns | Lets the model plan the next step |
| Side effects | "This sends an email" changes how cautiously it is used |
| Argument formats | Prevents invented IDs and wrong date formats |
✅ Negative guidance disambiguates better than positive guidance
With search_docs and search_tickets, adding more detail about what each does rarely helps — both descriptions get longer and remain similar.
Adding one clause about what each is not for resolves it immediately: "Searches published documentation. Not for customer-specific history — use search_tickets." The model now has a discriminating signal rather than two overlapping ones.
The token cost
| Tools | Tokens per request | At 100k requests |
|---|---|---|
| 3 | ~900 | 90M input tokens |
| 8 | ~2,400 | 240M |
| 15 | ~4,500 | 450M |
| 30 | ~9,000 | 900M |
These are sent every turn, including turns where no tool is used. In a ten-turn conversation with fifteen tools, you have paid for those definitions ten times.
Two mitigations matter. Prompt caching is the large one — tool definitions are byte-identical across requests, which is exactly what caching is designed for, and providers typically discount cached input heavily. And filtering: if the current task is billing, do not send the deployment tools.
How many tools is too many
Selection accuracy falls as the list grows — noticeably past ten to fifteen, sharply past thirty. The model is choosing among descriptions, and more options mean more chances for two of them to look equally plausible.
⚠️ Prefer fewer, higher-level tools
An agent with read_file, write_file, list_dir, search_files, copy_file, move_file and delete_file has seven ways to be confused.
One run_command tool with a constrained allowlist is often both cheaper and more reliable — the model already knows shell commands from training, and you get one place to enforce policy instead of seven.
Schema design
- Enums over free strings.
"status": {"enum": ["open","closed","pending"]}eliminates a class of invalid call. - Minimise required parameters. Every required field is something the model may invent rather than ask for.
- Flat over nested. Deep objects produce more malformed calls.
- Explicit formats in descriptions. "ISO date, YYYY-MM-DD" prevents three other formats.
- Say when to ask rather than guess. "Never invent an order ID — ask the customer" genuinely works.
Returning results
Tool output goes back to the model as text and costs tokens. Returning a raw API response is expensive and unhelpful.
Return what the model needs to decide the next step. If it later needs the detail, that is what a second tool call is for.
Errors deserve the same care: return them as results, not exceptions. A tool that returns "No order found with ID A-9999. Check the ID or use find_order by customer name" lets the model recover. A thrown exception ends the run.
Guarding execution
🚨 Tool calls are model output, and model output is influenceable
If your agent reads documents, emails or web pages, a malicious instruction inside that content can steer it into calling a tool you did not intend. This is indirect prompt injection, and the tool layer is where it becomes consequential.
Defend in code: allowlist what each tool can touch, require confirmation for destructive or outbound actions, apply least privilege to credentials, and log every call with its arguments. Never rely on a system prompt instructing the model to refuse — instructions are not a security control.
Designing or debugging a schema?
Format and validate JSON in your browser — useful for checking tool definitions and inspecting the calls a model produces.
Open the JSON Formatter →Summary
- The model requests; your code decides. Every control lives on your side.
- Descriptions drive selection, not names. Say when not to use each tool.
- Definitions are resent every turn — often the largest hidden cost.
- Cache the definitions and filter tools by task.
- Accuracy degrades past 10–15 tools. Fewer, higher-level ones work better.
- Enums and patterns prevent invalid arguments; descriptions prevent invented ones.
- Shape results for the model, do not forward raw API responses.
- Return errors as results so the model can recover.
Frequently Asked Questions
How does an LLM actually call a tool?
It does not. The model emits structured output naming a function and its arguments, your code reads that, executes the function, and passes the result back as a new message. The model never touches your system — it produces a request, and your code decides whether to honour it.
Why does my model pick the wrong tool?
Almost always because the descriptions do not distinguish them clearly. The model selects on the description text, not the function name, so two tools with similar descriptions are effectively a coin flip. State explicitly what each is for and, where they are close, what each is not for.
How much do tool definitions cost?
Every definition is sent on every request, so a typical tool with a few documented parameters costs 200 to 400 tokens per call. Fifteen tools can mean 4,000 input tokens before the user's message is even considered — frequently the largest single line in an agent's bill.
How many tools can a model handle?
Selection accuracy degrades noticeably past roughly 10 to 15 tools, and sharply past 30. If you need more, filter by task before the request or group them behind a smaller number of higher-level tools rather than exposing everything at once.
Should tool results be returned as JSON?
Usually not raw JSON. The model reads the result as text, so a compact readable summary costs fewer tokens and is easier to reason about than a deeply nested API response. Return what the model needs to decide the next step, not everything the API returned.