Messages, Roles, and Turn Structure

The model has no memory of your chat. Whatever it knows about the conversation arrives inside the request you are sending right now — so the messages array is not a log of the past, it is the model's entire present. Three roles fill it, and each one does a different job:

• system — standing rules that apply to every turn: who the assistant is, what format to answer in, what to refuse.
• user — inputs and questions, the turns that drive the conversation forward.
• assistant — what the model said before, re-supplied so it does not contradict itself.

Because nothing persists server-side between calls, the array grows on every exchange:

call 1                         call 2                         call 3
-----                          -----                          -----
[system,                       [system,                       [system,
 user:  "2+2?"      ---->       user:  "2+2?",        ---->     user:  "2+2?",
 assistant: "4"]                assistant: "4",                 assistant: "4",
                                user:  "times 3?"]              assistant: "4",
                                                                 user:  "times 3?",
                                                                 assistant: "12"]

The code that sends call 2:

from openai import OpenAI

client = OpenAI()

history = [
    {"role": "system", "content": "Answer with a single number, no prose."},
    {"role": "user", "content": "What is 2 + 2?"},
    {"role": "assistant", "content": "4"},
    {"role": "user", "content": "Now times 3?"},
]

reply = client.chat.completions.create(model="gpt-4o-mini", messages=history)
print(reply.choices[0].message.content)

Input: the four-turn array above.

Output: 12

Two consequences follow from the array being the whole world. First, cost: turn 50 pays for turns 1 through 49 on every request, which is why history size — not answer length — dominates the bill in long chats. Second, control: whatever sits in the array is what the model conditions on, so a stale or contradictory assistant turn produces a stale or contradictory answer. The system entry is the durable part; keeping it pinned while older dialogue expires is the job of Conversation History. The full request/response contract this array plugs into is covered in Your First LLM API Call.

A classifier that answers "Certainly! The ticket appears to be about billing..." when your parser expects billing is not misbehaving — it was never told what an answer looks like. The array gives you a lever for that: an assistant turn you authored yourself. It is not history, it is conditioning.

messages = [
    {"role": "system", "content": "Classify support tickets."},
    {"role": "user", "content": "Ticket: the app crashes on export"},
    {"role": "assistant", "content": "bug"},          # example of the format
    {"role": "user", "content": "Ticket: charged twice for invoice 88"},
]

reply = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    temperature=0,
)
print(reply.choices[0].message.content)

Input: a billing ticket after a bug example turn.

Output: billing

Written assistant turns do two different work. Side by side with a user turn they are a few-shot example — the model pattern-matches the pair and continues the pattern. Placed to demonstrate a schema, they are a format constraint without any parsing machinery. The wider pattern lives in Few-Shot Examples.

The turn order has to survive the endpoint you are talking to:

• Anthropic requires strict alternation, starting with user. Two user turns in a row return 400: roles must alternate. End the array with an assistant turn and generation continues from it — a prefill that pins the first words of the reply.
• OpenAI's chat endpoint accepts consecutive same-role turns. Stacked user messages are merged in practice, which keeps a multi-paragraph paste working without ceremony.

When a client appends two messages before sending — a pasted document followed by the actual question — the portable fix is one short assistant placeholder between them:

turns = [{"role": "user", "content": document},
         {"role": "assistant", "content": "Understood."},
         {"role": "user", "content": question}]

That single acknowledgement restores alternation on strict endpoints and, on lenient ones, reads as the model agreeing to the context — behavior you can verify in Zero-Shot Instructions by removing it and comparing answers.

At some point the array stops fitting — a 300k-token context is a ceiling, not a suggestion — and something has to go. The choice is a product decision disguised as an engineering one, because trimming old turns changes model behavior the moment you do it: an instruction you dropped is an instruction the model no longer follows.

Three retention strategies, with what each actually costs:

A minimal drop-oldest policy that stays valid:

def fit(history: list[dict], budget: int = 4000) -> list[dict]:
    system = history[0] if history[0]["role"] == "system" else None
    turns = history[1:] if system else history
    kept = [system] if system else []

    total = len(system["content"]) // 4 if system else 0   # ~4 chars/token
    while turns and total + len(turns[0]["content"]) // 4 > budget:
        turns.pop(0)                      # drop oldest turn first
    if turns and turns[0]["role"] != "user":
        turns.pop(0)                       # a strict endpoint needs user first
    return kept + turns

Input: 40 turns of history with a 4,000-token budget.

Output: system prompt + the trailing window, opening on a user turn.

Two lines carry the policy. Dropping whole turns keeps role alternation intact — trimming the array to an assistant-first sequence is how a trimmed conversation starts failing with roles must alternate on strict endpoints. And the system entry never enters the loop: standing rules survive every trim, because a chatbot that forgets its safety instructions after 30 turns is worse than one that forgets what the user said in turn 3.

Nothing here depends on the provider — the array format is the shared substrate, so the same policy ports between the endpoints in OpenAI SDK Essentials and Anthropic SDK Essentials. Once the array is well-formed, its other input knob — how stochastic the next turn sounds — is Sampling Parameters and Determinism.