Python Data Structures for API Work

A summary pipeline died at 02:00 with one line: KeyError: 'total_tokens'. The code read response["usage"]["total_tokens"] on every call, and one response had no usage key at all — the provider returned an error envelope instead of a completion. Nothing was wrong syntactically. The code had simply never met a payload shaped like the one that arrived.

Until you parse it into something typed, every model interaction lives in nested dicts, and dict access is where shape assumptions hide.

The envelope you receive

response (dict)
├── "id"            "chatcmpl-9x2..."
├── "model"         "gpt-4o-mini"
├── "choices"       [ ... ]            ← a list, even for a single result
│   └── [0]
│       ├── "message"        {"role": "assistant", "content": "..."}
│       └── "finish_reason"  "stop"
├── "usage"         ← ABSENT on error envelopes
│   ├── "prompt_tokens"      41
│   └── "completion_tokens"  128
└── "error"         ← ABSENT on successes
    ├── "message"   "Rate limit reached for requests per min..."
    └── "type"      "rate_limit_error"

Two branches, one dict type. Reading the happy path with [] and the error path with .get() — or the reverse — is how KeyError reaches production at an hour nobody is watching.

Build the request, read the response deliberately

import httpx, os

def ask(prompt: str) -> dict:
    response = httpx.post(
        "https://api.openai.com/v1/chat/completions",
        headers={"Authorization": "Bearer " + os.environ["OPENAI_API_KEY"]},
        json={
            "model": "gpt-4o-mini",
            "messages": [{"role": "user", "content": prompt}],
        },
        timeout=30.0,
    )
    return response.json()      # httpx already ran json.loads; the result is a plain dict

data = ask("How many tokens is this sentence?")
print(data["choices"][0]["message"]["content"])
print(data["usage"]["total_tokens"])

Input: How many tokens is this sentence?

Output:

This sentence is eleven tokens long.
41

Three lines carry assumptions worth naming:

• json={...} — httpx serializes the dict and sets Content-Type for you. Passing a pre-built string instead skips that machinery and quietly changes the contract.
• response.json() — the payload becomes a nested dict the moment it lands. No schema exists yet; every ["..."] below is a promise you wrote yourself.
• choices[0] then ["content"] — a list index into a dict inside a list. Both come from the diagram: choices is always a list, message is always a dict.

Branch on the envelope instead of guessing

def answer(data: dict) -> str:
    if "error" in data:
        err = data["error"]
        raise RuntimeError(f"{err['type']}: {err['message']}")
    choice = data["choices"][0]
    if choice["finish_reason"] != "stop":
        raise RuntimeError(f"truncated: {choice['finish_reason']}")
    return choice["message"]["content"]

"error" in data reads better than truthiness on data.get("error"): an error envelope is a different contract, not a falsy value. Once the string is out, handing it to a typed record — the next chapter — removes the bracket guessing entirely. The transport around this payload is Async HTTP with Python, and the responsibility attached to every prompt is covered in Responsible AI Foundations.

A retrieval script returned None for every document title for a week. One character caused it: doc.get("scource") — a misspelled key that a dict happily accepts and answers with None. Dicts cannot defend against that. They carry no field list, no types, and no constructor that fails when a guaranteed key is missing.

A dataclass is the smallest Python construct that fixes all three.

From anonymous dict to named record

from dataclasses import dataclass, field, asdict

@dataclass
class Chunk:
    doc_id: str
    text: str
    tokens: int = 0                                     # omitted fields land here
    tags: list[str] = field(default_factory=list)        # mutable default needs a factory

    @classmethod
    def from_api(cls, raw: dict) -> "Chunk":
        return cls(
            doc_id=raw["id"],                            # [] on purpose: a missing id is a bug
            text=raw["text"],
            tokens=raw.get("usage", {}).get("total_tokens", 0),
        )

chunk = Chunk.from_api({
    "id": "doc-42",
    "text": "Retries need jitter so synchronized clients do not stampede.",
    "usage": {"total_tokens": 57},
})
print(chunk.tokens, chunk.doc_id)

Output: 57 doc-42

The mixed access is deliberate: raw["id"] for fields the contract guarantees, .get() with a default for fields the provider may omit. Inside the class, chunk.doc_id is checked by your editor and by mypy, so the scource typo becomes an immediate AttributeError instead of a silent None.

default_factory=list is not stylistic — a bare tags: list[str] = [] raises ValueError at class creation, because one list object would otherwise be shared by every instance.

Sending the record back out

import json
print(json.dumps(asdict(chunk), ensure_ascii=False))

Output:

{"doc_id": "doc-42", "text": "Retries need jitter so synchronized clients do not stampede.", "tokens": 57, "tags": []}

asdict() recurses through nested dataclasses and lists, so one call covers the whole record. Hand-writing chunk.__dict__ works until you add a field that must not travel over the wire — then every call site needs editing.

Which container for which payload

Coordinates are the clean tuple case: point = (51.5007, -0.1246) unpacks into two names, cannot be appended to by accident, and serializes to a JSON array just like a list. A {"lat": ..., "lon": ...} pair costs more to write and more to read for values that are never accessed by name.

Keep dicts at the boundary — where JSON enters and leaves — and convert right after parsing. Doing that conversion with a declared schema instead of a hand-written from_api is what JSON Handling and Pydantic Models automates.

Formatting 200 model responses with a loop grows predictably: a for block, three nested ifs, an append — and eventually a condition applied to the wrong variable, because the block is eight lines long and the review had two minutes. Comprehensions collapse filter and projection into one expression where the scope of the condition sits next to the variable it constrains.

Filter and project in a single pass

documents = [
    {"id": "d1", "title": "Rate limits", "tokens": 812},
    {"id": "d2", "title": "Chunking", "tokens": 190},
    {"id": "d3", "title": "Embeddings", "tokens": 1204},
]

titles = [d["title"] for d in documents if d["tokens"] <= 1000]
print(titles)

Output: ['Rate limits', 'Chunking']

The loop it replaces:

titles = []
for d in documents:
    if d["tokens"] <= 1000:
        titles.append(d["title"])

Same result, three statements, and the if now sits four lines from the for it belongs to. The comprehension states the intent directly — keep these, then take that — instead of making the reader reconstruct intent from mutation order.

Message lists from raw turns

Chat payloads are lists of dicts, which is where dict comprehensions earn their place:

turns = [("user", "summarize"), ("assistant", "done"), ("user", "next")]

messages = [{"role": role, "content": text} for role, text in turns]
by_role = {role: text for role, text in turns}

print(messages[0], by_role["assistant"])

Output: {'role': 'user', 'content': 'summarize'} done

Unpacking in the clause (for role, text in turns) removes the indexing entirely, so a swapped tuple fails at build time instead of silently producing {"role": "summarize", "content": "user"}.

A comprehension also does one traversal where the naive version does two — filter, then map:

active = [c["doc_id"] for c in chunks if c["tags"] and "archived" not in c["tags"]]
print(len(active))

Output: 186

When to fall back to the loop

Comprehensions are the wrong tool the moment the body does more than transform:

• More than two clauses — for a in rows for x in a.cells stays readable; a third clause is a signal to write the nesting as real loops with names.
• Side effects — logging, counters, network calls. A comprehension that calls logger.info() hides work behind an expression that looks pure.
• Compound conditions — three ands plus a not belongs in a named helper: if is_suitable(d).
• Early exit — comprehensions always run to completion; a loop can break.

For the batch itself — one request per document, overlapped rather than queued — the iteration moves from lists to coroutines in Async HTTP with Python.