AI Engineering vs ML Engineering

A retail team opens two tickets on the same morning. Ticket one: the weekly demand forecast is off by 18% for one SKU family, and the buying team wants to know whether to retrain. Ticket two: the support assistant told a customer their order shipped when it had not, and the support lead wants the bot fixed before Monday.

Both tickets get filed as "the model is wrong," but they belong to different professions. The forecast is classical ML engineering: a model trained on structured rows (history, price, seasonality) that outputs a number, where the fix lives in features, labels, and retraining. The support answer is AI engineering: a general-purpose language model that was never trained on your order system, wired into the product by prompts, retrieved context, and output checks. Nobody retrains it. The fix lives in the application code around it.

Where the boundary sits

The distinction matters most when something breaks at 2 a.m. A drifting forecast needs retraining plus monitoring of feature distributions. A fabricated shipping status needs a grounded retrieval path, a refusal for questions the system cannot answer, and a validation step before the UI renders any text. The wrong fix is expensive in both directions: retraining a language model to repair a wiring bug burns GPU budget and ships nothing, while patching a prompt over a leaking training set guarantees the bug returns next quarter.

Scan an LLM product team's job postings for "train models" and you are usually looking at a research group, not the team shipping features. The daily work of AI engineering is a three-station loop.

1. Prompting — change the instructions, the message structure, or the few-shot examples, then rerun.
2. Retrieval — change which documents or records land in the context window, because most factual errors are missing context rather than a weak model.
3. Evaluation — rerun a fixed set of representative inputs, so a prompt edit is a measurement rather than an opinion.

The evaluation station is what keeps the other two honest. A minimal version:

def score_batch(cases, ask):
    passed = 0
    for question, expected in cases:
        answer = ask(question)            # your own call into the model
        passed += expected.lower() in answer.lower()
    return passed, len(cases)

cases = [
    ("What is our return window?", "30 days"),
    ("Do you ship to Norway?", "yes"),
    ("Can I change my address after ordering?", "support"),
]

print(score_batch(cases, ask=prompt_v3))   # (2, 3) -> v3 fails the address question
print(score_batch(cases, ask=prompt_v4))   # (3, 3)

Three cases is a demo. A working loop holds 50 to 300 labeled inputs and runs on every prompt change. Across a real week it looks like this: prompt v3 scores 154 out of 200 (77%), you rewrite the instruction that forces the model to quote its source, prompt v4 scores 171 out of 200 (85.5%). That is a change you can defend in code review. "It reads better" is not.

Training still appears in two places: fine-tuning a small model for a narrow, high-volume task, and training an evaluator for judgment calls you make hundreds of times a day. Both come after the loop works. They are optimizations you earn, not the starting point.

Strip away the tooling names and the two roles fail in the same ways and defend against them with the same three practices.

Data hygiene. An ML engineer rejects rows with impossible values before training so the model never learns from garbage. An AI engineer strips customer identifiers out of prompt payloads before the request leaves the building and pulls only the fields the feature needs, as covered in Responsible AI Foundations. Same instinct: the quality and the legality of the input decide the quality of everything downstream.

Testing discipline. ML work asserts on metrics computed from a held-out split (Train, Validation, and Test Splits). AI work asserts on parsing, formatting, and behavior over fixed cases with the model mocked out (Testing AI Code). Neither role ships a change that was only eyeballed.

Monitoring. The ML engineer watches feature distributions and prediction error for drift (Production Monitoring and Drift Detection). The AI engineer watches latency, error rate, spend per request, and eval pass rate over live traffic. Different charts, identical purpose: detect degradation before a customer files the ticket.

Failure modes show up when teams blur the boundary:

• A team labels an LLM bug a "model problem," schedules a fine-tune, and ships nothing for six weeks while the actual fix was one retrieval change.
• Model output is concatenated into a query with no schema check, so a malformed string becomes a runtime error. That fix belongs in JSON Handling and Pydantic Models, not in a prompt.
• Classical ML predictions reach the UI with no confidence threshold, so a low-certainty guess renders as a fact.

In an interview, the fastest signal you can send is naming which side of the boundary a problem sits on before you propose a fix. "This is a data problem, so I would not touch the model first" is a senior answer; a tool-list answer is not.