Query Planning at the Gateway

Anatomy of a Query Plan

A client operation is a document; a query plan is what the gateway compiles it into before any subgraph sees traffic. The plan is a small tree with three node kinds — Fetch (one GraphQL request to one subgraph), Sequence (children run one after another), and Parallel (children run concurrently) — plus a requires list on fetches that consume entity representations.

Take an operation where two branches are independent:

query ProductPage($id: ID!) {
  product(id: $id) { id name reviews { rating author { displayName } } }
  me { id }
}

QueryPlan
├── [0] Parallel
│   ├── [0.0] Fetch products   query ($id: ID!) { product(id: $id) { id name } }
│   └── [0.1] Fetch accounts   query { me { id } }
└── [1] Sequence
    ├── [1.0] Fetch reviews     requires: [{ __typename: "Product", id }]
    │         _entities(representations) { ... on Product { reviews { id rating author { id } } } }
    └── [1.1] Fetch accounts    requires: [{ __typename: "User", id }]
              _entities(representations) { ... on User { displayName } }

Every line has a precise meaning:

• The generated operation inside a fetch is what actually travels — a normalized document with only the fields that subgraph owns, never the client's original query.
• requires lists the representations that must exist before the fetch can run, which is why [1.0] waits for [0.0] and [1.1] waits for [1.0].
• path (implicit in the tree position) tells the merge step where each partial result is spliced back into the response.
• Branches with no data dependency land in Parallel: me does not depend on the product chain, so it runs alongside it rather than behind it.

Count the fetches: four. Count the leaf fields: eight. Fetch count follows subgraph boundaries crossed, not field count — which is exactly why adding a subgraph to a hot operation costs latency and merging two subgraphs into one usually saves it.

How the Gateway Decides What Runs When

Planning is dependency analysis over field ownership. A fetch becomes runnable the moment every input in its requires set has been produced by an earlier fetch, and the planner schedules whatever is runnable concurrently.

Five behaviors fall out of that rule:

1. Root fields in the same subgraph collapse into one fetch. If topProducts and bestSellers both live in catalog, the gateway sends one document, not two. Splitting them would double the round trip for no benefit.
2. Entity hops chain. Reviews cannot be requested until the catalog fetch has returned the Product keys that identify what to ask for. This is where sequential depth comes from, and it is why cross-subgraph nesting in the client's query turns into latency.
3. @requires adds an inbound edge. When a subgraph computes a field from data it does not own, the planner schedules the producing fetch first and threads the data into the representations. A single @requires can turn an otherwise parallel plan into a sequence of three.
4. Conditionals are resolved, not forwarded. @skip/@include become conditional branches in the plan, and fragments are inlined into each subgraph's operation, so a subgraph never receives a fragment it cannot resolve.
5. Variables are passed through selectively. Each fetch receives only the variables its generated operation references; the rest stay at the gateway.

The cache-friendly consequence: a plan depends on the shape of an operation — the normalized document and operation name — not on variable values. The same document with id: "p1" and id: "p999" shares one plan. That property is what makes plan caching safe, and what makes a plan worth inspecting when an operation misbehaves: the plan is stable, so the problem is in the data or in a subgraph, not in the request you happened to send.

Operating the Gateway

Plan cache. The first execution of a new operation shape pays a planning cost, so cold gateways show a latency spike on every new client release. Track plan-cache hit rate; a sudden drop means clients shipped new documents (or an attacker is spraying documents to force re-planning — a reason to pair the gateway with persisted queries). Cache entries are tiny and bounded by the number of distinct shapes, which is finite and measurable.

Entity cache. The gateway already holds entity representations during a plan; keeping them briefly turns repeated reads into memory hits. The key must include subgraph, __typename, the key fields, and the requested field set — two queries asking for different fields of the same product are different entries. Keep TTLs short and invalidate on mutations to that entity, or you will ship stale prices.

Partial failure is normal, not exceptional. When one subgraph errors or times out, the gateway returns HTTP 200 with entries in errors[] and null wherever nullability allows:

{
  "data": { "product": { "name": "Bottle", "reviews": null } },
  "errors": [{ "path": ["product", "reviews"], "extensions": { "code": "SUBGRAPH_ERROR" } }]
}

That is a feature — the page still renders — but it means alerting on HTTP status alone will show a green service while half the fields are null. Alert on error entries per operation and per subgraph instead, and set per-fetch timeouts so one slow database cannot hold the whole operation open.

Plan exposure. Apollo Explorer and equivalent tooling render the plan for any operation. Use it as the first step of triage: "which subgraph served this field, and how many hops did it take?" answers in seconds what a code archaeology session would take an afternoon. If you can print the plan for a production operation, you can prove ownership rather than argue about it.