OpenAI SDK Essentials

The hand-rolled httpx loop from Your First LLM API Call works, but it already needs three things you have not written yet: credential resolution, retries, and a timeout policy. The SDK ships all three, and the first decision is where its configuration comes from.

from openai import OpenAI

client = OpenAI(            # api_key defaults to $OPENAI_API_KEY
    timeout=30.0,           # budget for one attempt, including reads
    max_retries=2,          # attempts the SDK makes before raising
)

reply = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Classify: my card was charged twice"}],
)
print(reply.choices[0].message.content)

Input: Classify: my card was charged twice

Output: {"category": "billing", "urgency": "medium"}

Credential resolution is a two-step rule: an explicit api_key argument wins, otherwise the client reads the OPENAI_API_KEY environment variable. If neither is present, the constructor raises immediately with a message naming both options — you find out at startup, not on the first user request. Pass api_key explicitly only in tests that load a fixture key; production code should depend on the environment so no key can be committed by accident.

Three configuration points matter beyond the key:

• Build one client per process. The client wraps a connection pool. Constructing it inside a request handler throws away keep-alive connections and pays a fresh TLS handshake every call, which shows up as added latency — Latency Profiling shows how to catch it.
• base_url points the same client at a gateway, proxy, or self-hosted compatible endpoint without touching call sites.
• default_headers, organization, and project come from client options or the OPENAI_ORG_ID / OPENAI_PROJECT_ID environment variables when one deployment must bill a specific project.

A 429 at 3 a.m. should not wake anyone up. The SDK's retry helper answers three questions on its own: which responses are worth retrying, how long to wait, and when to give up.

attempt 1  ->  429   sleep 0.8s   (honors retry-after when the provider sends one)
attempt 2  ->  429   sleep 1.7s   (exponential backoff with jitter)
attempt 3  ->  200   return       default max_retries=2, so three attempts total

The distinction that saves you from a bad afternoon is which statuses are retryable:

So the loop in the previous topic collapses to one line of configuration. Override the budget per client, or for a single call when one path can afford to wait longer:

patient = client.with_options(max_retries=5)
reply = patient.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Summarize the incident report"}],
)

When to turn retries down. Retries are not free: each attempt re-sends the full prompt, so a retried call bills the prompt tokens twice, and three attempts on a 10-second generation can blow a 2-second interactive budget. Set max_retries=0 when you are writing your own recovery loop, when the call streams (a stream that dies mid-answer cannot be replayed — see Streaming Responses), or when the SDK's total worst-case wait exceeds the latency target you promised the user. The full policy — circuit breakers, jitter formulas, budget caps — lives in Rate Limiting and Retries.

Free-text output forces you to strip markdown fences, hunt for the first brace, and hope the model did not add a commentary sentence. Parsing failures like that are the most common source of bugs in LLM features, and the SDK can eliminate most of them.

Three output modes, increasing in strictness:

schema = {
    "name": "ticket_classification",
    "strict": True,
    "schema": {
        "type": "object",
        "properties": {
            "category": {"type": "string", "enum": ["billing", "bug", "question"]},
            "urgency": {"type": "string", "enum": ["low", "medium", "high"]},
        },
        "required": ["category", "urgency"],
        "additionalProperties": False,
    },
}

reply = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Ticket: charges are duplicated on invoice 88"}],
    response_format={"type": "json_schema", "json_schema": schema},
)
print(reply.choices[0].message.content)

Input: Ticket: charges are duplicated on invoice 88

Output: {"category": "billing", "urgency": "medium"} — a string you can pass straight to json.loads, then to a validator.

A strict schema turns "did the model feel like producing JSON today" into a contract: missing keys and wrong types are rejected rather than discovered downstream. The additionalProperties: False line and the required list are what make that contract airtight; omit them and the model may slip in extra fields you never see. The SDK's client.chat.completions.parse helper wraps this pattern with a Pydantic model so the schema is derived from your type instead of hand-written — JSON and Pydantic Basics walks through it.

Two limits worth knowing. Structured output constrains the shape, not the truth: a schema with an enum of urgency levels cannot stop the model from picking the wrong one, so grounding and evaluation still do that job — Schema-Constrained Generation covers the tradeoffs, and JSON mode in JSON Mode Outputs is the lighter option when you only need parseable text.

A summarization feature that costs nothing visible until the monthly bill arrives is a feature with an expiration date. Every non-streaming response carries usage, and reading it at the call site is the whole of cost attribution.

reply = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Summarize ticket 88"}],
)
u = reply.usage
record_event(
    tenant="acme",
    feature="ticket-summary",
    input_tokens=u.prompt_tokens,
    output_tokens=u.completion_tokens,
)

Two numbers, two prices: prompt_tokens cover the instructions and history you resupply on every turn, while completion_tokens cover what the model wrote — and output tokens usually cost several times more per token than input. Re-sending a 4,000-token conversation to generate a 40-token reply means the prompt dominates the bill, which is why trimming history pays for itself. Token Usage Metering builds the aggregation layer on top of this event, and Token Cost Estimation turns counts into currency before you ship.

Streaming changes where usage lives. With stream=True the usage block only appears if you ask for it with stream_options={"include_usage": True}, and it arrives on the final chunk — which has an empty choices array. Code written against the non-streaming shape will raise IndexError on that chunk, so guard it:

last = None
for chunk in stream:
    if chunk.choices:          # final usage chunk has choices == []
        last = chunk
usage = chunk.usage           # read from the terminal chunk, not from choices

Log the counts alongside a trace id rather than the raw completion text, and you keep PII out of the billing pipeline while still being able to answer "why did this tenant's spend triple" — Trace LLM Calls shows the layout.