Common LLM Application Archetypes

Two features land in the same sprint review. One says "let users ask questions about our engineering handbook." The other says "give users a friendly assistant that answers anything about our product." They read like one feature. They need opposite architectures.

A chat assistant is open-ended dialogue: the model's own knowledge carries the answer, and the product's job is conversation state, tone, and safe refusal when the question is out of scope. Question answering over private documents is different: the answer must come from your text, and the model's only job is to read the supplied passages and quote them. That is the RAG archetype.

(user question)
      |
      v
[embed question] --> [search index] --> [top-k passages]
                                              |
                                              v
[prompt: instruction + passages + question] --> [model] --> [answer + citations]

The diagram explains the failure modes. If the search step returns the wrong passages, no prompt edit will rescue the answer, which is why retrieval quality is the first thing to measure. If the prompt does not forbid answering from general knowledge, the model will blend its own priors with your documents and produce confident sentences no document supports.

messages = [
    {"role": "system", "content": (
        "Answer only from the excerpts below. "
        "If the excerpts do not cover the question, reply exactly: "
        "'Not found in the handbook.'"
    )},
    {"role": "user", "content": f"Excerpts:\n{context}\n\nQuestion: {question}"},
]

Two design decisions hide in those five lines. The excerpts sit inside the user turn rather than the system turn so they read as evidence to argue over, not as standing orders — a document containing "ignore previous instructions" cannot rewrite your rules (Prompt Injection Attack Patterns). The fixed refusal string gives you something exact to assert on in tests.

The tradeoff is real: RAG adds chunking, an index that must stay fresh, and a second eval axis (did we retrieve the right passage?) on top of answer quality. When the corpus is a ten-item FAQ, skip it — paste it into the system prompt and save yourself an entire subsystem. Start with RAG Pipeline Overview when the corpus is large, private, or changes weekly.

An operations team pastes about 400 vendor emails a week into a spreadsheet: vendor, amount, due date, what action is needed. The emails arrive as prose with signatures, tables, and apologies. The archetype that fits is an extraction pipeline — text in, validated JSON record out — because the consumer is a database, not a human reading prose.

import json

TEMPLATE = """Extract the invoice fields. Respond with JSON only:
{"vendor": str, "amount_usd": float, "due_date": "YYYY-MM-DD"}

Email:
{body}"""

def extract(body: str, ask) -> dict:
    raw = ask(TEMPLATE.format(body=body)).strip()
    if raw.startswith("```"):                     # models often fence JSON anyway
        raw = raw.strip("`").removeprefix("json").strip()
    return json.loads(raw)                        # anything else raises

Input: From billing@acme.io - invoice 8821 for $4,250 due March 14, thanks!

Output: {"vendor": "acme.io", "amount_usd": 4250.0, "due_date": "2026-03-14"}

The fence-stripping line exists because "Respond with JSON only" is a request, not a guarantee: models wrap JSON in code blocks and append "Let me know if you need anything else." Anything the stripper does not cover raises JSONDecodeError, and in production that exception must become a logged retry rather than a crashed worker. Field-level checks belong in a schema, not in the prompt (JSON Handling and Pydantic Models).

Two sibling archetypes compress text instead of expanding it. Summarization turns a long input into a short one: a 40-message thread into a three-line status. Classification turns text into a label: a ticket into billing, shipping, or other plus a one-line reason. Both share extraction's shape — structured output, validated before use — but the economics differ. At high volume, a narrow label set is often cheaper and faster as a small embedding-based classifier, with the language model called only for the ambiguous slice it flags.

Some tasks cannot be finished in one call: look up the order, check the policy, issue the partial refund, confirm to the customer. That is the workflow agent archetype — several model calls chained through tools to complete a multi-step business process.

TOOLS = {
    "lookup_order": lambda order_id: orders.get(order_id),
    "issue_refund": lambda order_id, cents: payments.refund(order_id, cents),
}

async def run_agent(task: str, ask) -> str:
    messages = [{"role": "user", "content": task}]
    for _ in range(5):                       # hard stop, not a suggestion
        reply = ask(messages, tools=sorted(TOOLS))
        if reply.tool_call is None:
            return reply.text
        args = validate_args(reply.tool_call.name, reply.tool_call.args)
        result = TOOLS[reply.tool_call.name](**args)
        messages.append(tool_result_message(reply, result))
    return "stopped after 5 steps; routed to a human"

Read the loop with an eye on what it costs. Each pass is a full model call, so latency and token spend multiply by steps taken: five steps is roughly five bills and several seconds of wall clock. range(5) is the difference between a bounded job and a loop that reissues refunds. validate_args exists because the model invents arguments — an order id that does not exist must fail loudly at the boundary, not inside the payments client.

Failure modes worth naming before you commit to an agent:

• No step budget, so a confused model oscillates until the token meter empties.
• Partial completion reported as success: three of four steps ran, the customer got a confirmation anyway.
• Tool results dumped into context verbatim, blowing the window with a 40 KB database row.

The honest alternative: if the sequence is fixed and always the same three steps, write the three calls in code. Agents earn their keep when the path varies per request (Running Multi-Step Tool Flows, Stopping Conditions and Budgets). You also need an eval measured in task success rate, not answer fluency (Offline Evaluation Harness).

Archetype selection is mostly reading the requirement for its output shape. This table covers most product requests that arrive:

Each archetype fails differently, so each needs its own test. A chat assistant that fabricates needs a refusal eval; an extraction pipeline that crashes needs schema validation and a retry; an agent that stops early needs a step budget. Building the wrong one does not fail loudly — it fails as a feature that demos well and disappoints after week two.

Two rules of thumb for when NOT to reach for the heavier archetype:

• Agent vs plain code: if you can write the step order down as a flowchart with no branches on model judgment, write code. Deterministic, free to run, and testable without mocks.
• RAG vs stuffing the prompt: if the corpus fits comfortably in the context window and changes rarely, inline it. Retrieval earns its complexity only when the corpus outgrows the window or the index freshness problem is cheaper than the alternative.

Mixing archetypes is normal — a support product is often an assistant with retrieval for policy questions, extraction to fill the ticket fields, and a tightly budgeted agent for refunds. The discipline is deciding which piece is which, so each gets the evaluation and the guardrails its failure mode demands.