Running Multi-Step Tool Flows

A support agent is asked one question that no single model call can answer:

user: "Was order 88213 refunded?"

The model cannot answer it — the fact lives in your database. So the conversation does not go user → answer. It goes user → model asks for permission to act → your code runs the tool → the result goes back in → the model continues, possibly asking for a second tool → eventually the model produces prose with no pending calls. That final condition is the only stop signal you get.

        user message
              │
              ▼
      ┌───────────────┐  tool_calls present?   ┌────────────────┐
      │  model call   ├──────── yes ──────────►│  run each tool │
      └───────────────┘                        └───────┬────────┘
              ▲                                        │
              │      role:"tool" results, one per call  │
              └───────────────◄────────────────────────┘
              │
              │  no tool_calls
              ▼
        final assistant text ──► user

Two rules govern the message sequence, and violating either returns an API error rather than a wrong answer:

1. The assistant turn carrying tool_calls must be appended verbatim. The next request has to show the model what it asked for; a transcript that skips it is an orphaned set of results.
2. Every result is its own role: "tool" message with a tool_call_id matching the call it answers. Results are strings — json.dumps whatever your function returned.

The transcript for the question above looks like this:

[user]      Was order 88213 refunded?
[assistant] tool_calls=[search_orders(customer_id="c_88213", query="88213")]
[tool]      [{"id": "88213", "total": 62.40, "status": "refunded"}]
[assistant] tool_calls=[get_refund_policy(order_id="88213")]
[tool]      {"window_days": 30, "method": "original payment"}
[assistant] "Yes — order 88213 was refunded $62.40 to the original payment method."

Three model round trips, two tool executions, one answer. Your job is to make that cycle run without a human watching it.

The whole engine is a while-shaped loop with a hard cap. The stop test is whether the model returned any tool calls:

import json
from openai import OpenAI

client = OpenAI()
MAX_TURNS = 6

def dispatch(call) -> str:
    try:
        fn = TOOL_IMPL[call.function.name]
        result = fn(**json.loads(call.function.arguments))
    except Exception as err:                     # a raised tool must not kill the run
        return json.dumps({"error": type(err).__name__, "message": str(err)})
    return json.dumps(result, default=str)       # tool content is always a string

def run(user_request: str) -> str:
    messages = [{"role": "user", "content": user_request}]

    for _ in range(MAX_TURNS):
        msg = client.chat.completions.create(
            model="gpt-4o-mini", messages=messages, tools=TOOLS,
        ).choices[0].message

        messages.append(msg)                     # assistant turn must precede its results
        if not msg.tool_calls:                   # no pending calls → the model is finished
            return msg.content

        for call in msg.tool_calls:              # batched calls: one reply per call id
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": dispatch(call),
            })

    raise RuntimeError("turn cap reached without a final answer")

Executed against the refund question:

> run("Was order 88213 refunded?")
turn 1: tool_calls=[search_orders(customer_id="c_88213", query="88213")]
        tool → [{"id": "88213", "total": 62.40, "status": "refunded"}]
turn 2: tool_calls=[get_refund_policy(order_id="88213")]
        tool → {"window_days": 30, "method": "original payment"}
turn 3: no tool_calls
return  "Yes — order 88213 was refunded $62.40 to the original payment method."

Four lines deserve the attention:

• messages.append(msg) before the if. The assistant turn with its tool_calls has to be in history for the following request; appending results without it produces a 400 about unmatched tool calls.
• if not msg.tool_calls is the termination condition, not a fixed number of steps. None and an empty list both mean the model has stopped asking to act.
• Batched calls are normal. One assistant message can request several tools; execute each (concurrently when they are independent) and append one role: "tool" message per tool_call_id.
• dispatch converts exceptions into payloads. If a tool raises and you let it propagate, the whole conversation is lost; if you send the error back as a tool message, the model can adjust its arguments and continue — the subject of Shaping Tool Results for Models.

The loop is deliberately dumb. Everything intelligent happens in the model; everything safe — caps, dispatch, error containment — happens in code you can unit test without an API key.

Left unattended, this loop fails in four characteristic ways:

• It never stops. Without MAX_TURNS, a confused model re-requests the same tool with the same arguments until the token budget is gone — each round is a full re-prompt with a longer transcript, so cost grows with every turn, not linearly with the answer. An iteration cap is mandatory; goal checks and per-task token budgets are what let a well-behaved run exit early instead of always hitting the cap (Stopping Conditions and Budgets).
• A tool exception ends the conversation. raise inside dispatch discards every prior turn. Return a structured error payload instead: the model reads it, corrects the arguments, and the run continues (Error Recovery in Agent Runs).
• Mismatched or missing tool_call_id. The provider rejects the request outright. Appending results out of order, replying to only some of a batch, or hand-editing the assistant message all trip this — build results mechanically from the tool_calls list rather than by hand.
• A silent loop. The model stops calling tools and returns a confident answer assembled from partial data. No API error will ever surface this; you catch it by logging each turn — inputs, calls, arguments, results — and replaying failed runs offline.

One design choice remains: independent calls in a batch. Serial execution is simpler to trace; concurrent execution (asyncio.gather over the batch) cuts wall-clock latency when the second call does not depend on the first. The rule for correctness is unchanged either way — one role: "tool" message per call id, all of them appended before the next model request.

Which result content goes into those messages decides whether the next turn is useful or wasted. A raw HTML page or an exception traceback handed back verbatim costs tokens and steers the model badly; shaping that payload is the next topic, Shaping Tool Results for Models.