Batching Versus Request Caching

One Library, Two Distinct Mechanisms

DataLoader ships two behaviors behind one load call, and conflating them is the most common misunderstanding of the library. Batching collapses concurrent loads into one statement. The memoized cache deduplicates sequential loads of the same key. They solve different shapes of the same problem.

const loader = new DataLoader(batchUsers);

// Three distinct keys issued in one tick -> batching
const [a, b, c] = await Promise.all([loader.load(1), loader.load(2), loader.load(3)]);
// 1 statement: batch [1, 2, 3]

// Three distinct keys in separate ticks -> neither mechanism applies
await loader.load(1);
await loader.load(2);
await loader.load(3);
// 3 statements: each dispatch found a single-key queue

// The same key in separate ticks -> the cache
await loader.load(1);
await loader.load(1);
// 1 statement total: the second call returned the first call's promise

The practical consequence: batching needs concurrency, the cache needs nothing but repetition. A resolver that awaits unrelated work between loads loses batching while keeping the cache — which is why a system can look "cached" in traces and still issue one query per row. Neither mechanism is a product-level cache: the memo exists to deduplicate work inside one operation, and freshness across operations is a different problem with a different owner.

A timeline you can reason from

tick   load calls              queue        dispatch
────   ──────────────────────   ──────────   ─────────────────────────
 t1    load(1) load(2) load(3)  [1,2,3]      next tick -> batch [1,2,3] -> 1 SQL
 t2    load(1)                  [1]          memo hit, no dispatch
 t3    load(4) load(4)          [4]          dup collapses -> batch [4] -> 1 SQL
 t4    (none)                   []           nothing

Read row by row: the queue is per tick, the memo is per request. Row t2 is the cache doing sequential deduplication; rows t1 and t3 are batching doing concurrent deduplication. Total: 2 statements for 6 load calls — a number no single mechanism would produce on its own.

What the Request Cache Hides From You

The memo map stores promises, and a promise resolves once. Within a single request, the first load(7) fixes what every later load(7) will observe — including after the row has changed.

const resolvers = {
  Mutation: {
    renameUser: async (_, { id, name }) => {
      const updated = await db.update('users', { name }, id);
      userLoader.clear(id);          // without this, later reads in this request
      return updated;                // still see the pre-write value
    },
    audit: async (_, __, ctx) => {
      const before = await userLoader.load(7);   // primes the cache
      await db.update('users', { name: 'x' }, 7);
      const after = await userLoader.load(7);    // same promise -> old row
      return { before, after };                  // identical objects
    },
  },
};

Three properties make this behavior both useful and sharp:

• It never crosses requests. A fresh map per request means no user ever observes another user's memoized row, and nothing persists after the response is sent. Request scoping is what makes the cache safe — the failure mode of sharing it is covered in request-scoped loaders.
• It deduplicates in-flight work. Two resolvers loading the same key share one pending promise rather than racing two queries, so you cannot observe two different values for one key inside a request.
• It hides writes from itself. Any read after a write in the same request needs clear(key) on the path that wrote, or the read returns the old row with no error and no log line.

The design tradeoff is deliberate: serving a millisecond-old value inside one request is almost always acceptable, and buying correctness with a cache clear at every write point would reintroduce the coordination cost the loader removed. When a request genuinely needs read-after-write within itself, clear the key or prime it with the row you just wrote.

Layering a Real Cache Beside the Loader

If data must stay fresh across requests or across users, the request memo is the wrong tool — not a weak one, the wrong one. Each layer answers a different freshness question:

The loader still participates: it batches the misses that fall through to Redis, so a warm cache costs one batched round trip to Redis and zero to Postgres. That is the standard shape — the memo absorbs repetition inside a request, Redis absorbs it across requests, and the database absorbs only what both missed.

Two rules keep the layers honest:

1. Never put freshness-sensitive data in a shared cache without a version or TTL. A price, a balance, or an access flag cached for five minutes across users is a support ticket; a product image cached for five minutes is fine. Choose per data class, not per system.
2. Invalidate at the write, not at the read. Updating Redis inside a resolver after a successful commit keeps the read path simple; polling or lazy expiry moves staleness into the user's hands.

Interview framing: state what each layer deduplicates — request memo for concurrent and repeated loads within one operation, shared cache for cross-request repeats, HTTP cache for identical documents — and say where you would put a field like balance versus avatarUrl. For the response-level side of this (document and field caching), see response caching strategy.