Schema-Constrained Generation

Your triage dashboard declares four categories. Week two shows twelve rows:

Nothing crashed. Every response parsed. The damage came from allowing free text where a closed set was required, so a plain string field let "Billing" through as legally as "billing". Patching that in application code means maintaining a synonym map forever. Binding it in the request means the bad strings never exist:

import json
from openai import OpenAI

SCHEMA = {
    "name": "ticket_classification",
    "strict": True,
    "schema": {
        "type": "object",
        "properties": {
            "category": {"type": "string",
                         "enum": ["billing", "shipping", "account"],
                         "description": "Owning team for this ticket"},
            "priority": {"type": "string", "enum": ["low", "med", "high"]},
            "summary": {"type": "string"},
        },
        "required": ["category", "priority", "summary"],
        "additionalProperties": False,
    },
}

client = OpenAI()
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": ticket}],
    response_format={"type": "json_schema", "json_schema": SCHEMA},
)
payload = json.loads(resp.choices[0].message.content)

With strict: true, a token continuation producing "Billing" or a trailing space is not among the legal next steps, so it cannot be sampled. Note what strict mode demands in return: every property listed in required and additionalProperties: false. A key the model may omit or a property it may invent would break the grammar, so optional data is modeled as an explicitly nullable field rather than an absent one.

Each layer of the stack catches a different class of defect:

A model emits one token per step from a vocabulary of roughly a hundred thousand candidates. Constrained decoding narrows that choice before sampling rather than filtering after:

JSON Schema ──► grammar automaton ──► allowed-token mask (rebuilt each step)
                                            │
model logits ───────────────────────────────┼──► masked logits ──► sampled token
                                            │
                     disallowed tokens set to -inf, so they never win

Because the mask applies before sampling, an invalid continuation is unreachable, not "generated then discarded". There is no parse-fail-retry cycle for syntax or shape at all — contrast the round trips each structural defect costs in JSON Mode Outputs, where repair is always one attempt too late.

Three consequences follow from paying for that guarantee:

• The schema is input tokens on every request. A 900-token schema on a high-volume endpoint is a line item, not a footnote, and providers that compile the grammar server-side add pre-processing latency before the first token.
• Constraints are per-field, not relational. No JSON Schema construct says refund_amount <= order_total or "the quoted text must appear in the retrieved document". Those live in code after generation.
• Support is not uniform. Not every provider or model version implements binding, and strict-mode rules have shifted across API versions. Keep the JSON-mode validate-and-retry path as a fallback so a provider migration does not rewrite your output layer.

Depth is a second cost paid at generation time. Each nesting level is a decision the model must hold open while it emits siblings, so a legal schema can still fail often purely because of its shape, which the final chapter takes up directly.

Declaring the schema as a dict works, but two sources of truth — the dict and the Python type that consumes it — drift apart silently. Deriving the request schema from a Pydantic model removes that seam:

from typing import Literal
from pydantic import BaseModel, Field
from openai import OpenAI

class Classification(BaseModel):
    category: Literal["billing", "shipping", "account"] = Field(
        description="Owning team for this ticket")
    priority: Literal["low", "med", "high"]
    summary: str

client = OpenAI()

def classify(ticket: str) -> Classification:
    completion = client.beta.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": ticket}],
        response_format=Classification,   # schema generated from the model class
    )
    message = completion.choices[0].message
    if message.parsed is None:
        raise ValueError(f"refusal or early stop: {message.refusal or 'no parsed output'}")
    return message.parsed

input:  "I was charged twice for order 88213 and nobody has replied."
output: Classification(category='billing', priority='high',
                       summary='Duplicate charge on order 88213, no agent reply')

Line by line on what is non-obvious:

• Literal[...] rather than str is the whole point of chapter one. A plain str generates "type": "string" in the schema and "Billing" becomes samplable again; a Literal generates an enum.
• response_format=Classification serializes the class into the json_schema block, so the validator and the contract cannot disagree about field names.
• message.parsed returns the instance directly — skip json.loads entirely. It is None when the model refuses or stops before the object closes, which is why the guard precedes use.
• Pydantic's own @model_validator rules run after parsing. Their errors describe cross-field problems the schema could not express, and they should be fed back through the same corrective retry path described in JSON Mode Outputs.

This pattern is the practical core of Pydantic Basics; here it is what turns a declared contract into a typed object your code can trust.

Schema validity is not generation reliability. A schema can pass every linter and still fail repeatedly because the shape itself is hard to satisfy token after token.

Flatten before nesting hurts. Four or five object levels deep, the errors compound:

{"customer": {"id": "c_88", "address": {"line1": "12 Rue Victor", "zip": "75011"}}}

The same information, produced reliably:

{"customer_id": "c_88", "address_line1": "12 Rue Victor", "address_zip": "75011"}

Descriptions carry weight a type cannot: {"type": "string", "description": "ISO 8601 date of the incident, not the report date"} prevents a whole class of field mix-ups.

What the schema cannot say. Rules that compare two fields live in code:

from datetime import date

def check_invariants(p: dict) -> list[str]:
    problems = []
    start = date.fromisoformat(p["start_date"])
    end = date.fromisoformat(p["end_date"])
    if start >= end:
        problems.append(f"start_date {start} must precede end_date {end}")
    if p.get("refund_amount", 0) > p["order_total"]:
        problems.append(f"refund_amount {p['refund_amount']} exceeds {p['order_total']}")
    return problems

Treat those failures like schema failures: state them explicitly and push them back through the retry loop, because the model usually repairs a cross-field mistake once it is named.

Drift runs in two directions. Your schema evolves — adding a required field breaks older consumers, so version the schema name (ticket_classification_v2) and migrate consumers before flipping strict mode. Independently, provider model versions change how reliably the same schema is satisfied. Guard both with ten to twenty representative prompts per schema, including adversarial ones, executed whenever the schema or model version changes: Regression Suites shows how to wire that into CI so drift surfaces before your users do.