Defining Tools for LLMs

A model cannot call your order API. It has no code and no network access — it only emits tokens. Function calling works by handing the model a menu: a name, a description, and a JSON Schema of parameters, from which it produces a structured tool_calls block that your code executes. Every word of that menu is read by the model, so the definition is the whole contract.

TOOLS = [{
    "type": "function",
    "function": {
        "name": "search_orders",
        "description": (
            "Search a customer's order history. Use when the user names an order "
            "number, asks about a recent purchase, or wants delivery status. "
            "Returns up to 20 orders. Not for inventory levels or product pricing."
        ),
        "parameters": {
            "type": "object",
            "properties": {
                "customer_id": {"type": "string",
                                "description": "Customer ID from the session, e.g. c_88213"},
                "query": {"type": "string",
                          "description": "Order number or product name fragment"},
                "status": {"type": "string",
                           "enum": ["pending", "shipped", "delivered", "cancelled"],
                           "description": "Fulfillment status filter; omit for all"},
            },
            "required": ["customer_id", "query"],
            "additionalProperties": False,
        },
    },
}]

Send it with a conversation and the model answers with a call rather than prose:

resp = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=TOOLS)
msg = resp.choices[0].message
# msg.tool_calls[0].function.name      -> "search_orders"
# msg.tool_calls[0].function.arguments -> '{"customer_id": "c_88213", "query": "88213"}'

Note that arguments arrives as a string, not a dict — json.loads it before you dispatch, and treat the result as untrusted input.

The description earns the most attention per character: it is where you state when to use the tool and when not to, which is what keeps a near-neighbor tool from stealing the call.

Tool parameters use the same JSON Schema vocabulary as structured outputs — type, properties, required, enum, additionalProperties — so the design rules from Schema-Constrained Generation transfer directly. What changes is the audience: here the schema shapes arguments the model invents, and the model writes them with far less context than it had when it wrote your last structured reply.

Four decisions decide how often argument generation succeeds:

1. Required versus optional. Every optional property is an open decision the model can get wrong. Mark required what your handler cannot run without (customer_id, query), and let the rest default in your code.
2. Enums for closed sets. "status": "completed" is not in your database vocabulary and will not match anything. ["pending", "shipped", "delivered", "cancelled"] makes the wrong strings unsamplable in strict mode.
3. A description per property. These are read mid-generation. "customer_id" with no description invites 88213 (the order number) instead of c_88213; the example in the description removes the ambiguity.
4. One stable shape. Deeply nested parameter objects and anyOf alternatives multiply failure points the same way they do in output schemas.

Model-generated arguments are untrusted input, exactly like a request from a browser:

import json
from jsonschema import validate, ValidationError

def parse_args(name: str, raw: str) -> dict:
    args = json.loads(raw)                      # the model emitted a string
    validate(instance=args, schema=SCHEMAS[name])  # raises ValidationError on drift
    return args

model arguments: '{"customer_id": "c_88213", "status": "completed"}'
ValidationError: 'query' is a required property

Catch that error and return it as structured feedback for the next round rather than raising — the loop in Running Multi-Step Tool Flows needs a payload it can put back in the transcript so the model can add the missing field itself.

The instinct when adding capability is to add tools. Selection accuracy does not scale with that instinct:

Three failure patterns cause most of it, and all three are definition problems rather than model problems:

• Near-duplicate names. search_orders, find_order, and order_lookup that accept overlapping arguments force the model to guess. Collapse them into one tool with a mode parameter — one decision instead of three.
• Names without verbs. orders tells the model nothing about intent. search_orders, issue_refund, and get_policy name the action; the verb is what gets matched against the user's request.
• Descriptions that only describe. "Searches orders" restates the name. The description above works because it lists triggering situations and an exclusion ("not for inventory levels or product pricing"), which is the line that prevents a wrong call.

Mitigations once the menu grows past a handful:

• Prune the list per request — send only the subset relevant to the current task instead of the full catalog.
• Group near-duplicate tools under one parameterized tool to shrink the decision space.
• Log the chosen tool together with the alternatives that were available, so systematic mis-selection shows up in aggregate rather than in one-off complaints (Tool Registry Design covers ownership, versions, and where these definitions live).

There is also a legitimate "no tool" outcome. When the structured result is two short fields with no external data involved, a schema-bound JSON response is stricter and cheaper than a tool round trip — see Schema-Constrained Generation. A tool earns its place when it must reach something the model does not have.

Once the definitions are in place, the remaining work is running them: the loop in Running Multi-Step Tool Flows executes the calls and feeds results back.