Data-Driven API Testing

A tester on your team owns POST /orders — customer email, coupon code, currency — and the bug tracker keeps adding permutations: unicode domains, plus-addressing, a 254-character address, an expired coupon, a coupon minted for the wrong currency. Copy-pasting the request into the collection sixty times creates sixty places to fix when the auth header changes, and nobody runs a sixty-request regression by hand. Data-driven testing removes that duplication: one request definition, a dataset that supplies the per-case values, and a runner that executes the definition once per row. The collection stops being a list of cases and becomes a program that reads a dataset.

Postman's native format is JSON: each element of the top-level array is one iteration, and its keys become template variables.

[
  { "case": "valid-basic", "email": "buyer@shop.com", "coupon": "SAVE10", "expectedStatus": 201 },
  { "case": "no-at-sign", "email": "buyershop.com", "coupon": "SAVE10", "expectedStatus": 422 },
  { "case": "expired-coupon", "email": "buyer@shop.com", "coupon": "OLD99", "expectedStatus": 422 },
  { "case": "wrong-currency", "email": "buyer@shop.com", "coupon": "SAVE10-EUR", "expectedStatus": 422 }
]

The body then reads {"email": "{{email}}", "coupon": "{{coupon}}"} and the runner substitutes per iteration. In the GUI you select the file under Runner → Data; headlessly the same switch is newman run orders.postman_collection.json --iteration-data cases.json, which is why unattended runs from Automating Postman Collections matter — a dataset earns nothing until the collection runs without a human clicking through it. Inside scripts, pm.iterationData.get("case") reads the current row when you need the value in logic rather than in the template.

CSV is the second supported format and it carries the trap that catches almost everyone the first time: CSV has no types. A JSON 25 is a number; a CSV cell containing 25 arrives as the string "25", and an API validating its request schema rejects it — or, worse, accepts a string where the documented contract says integer. Coerce deliberately in a pre-request script (pm.iterationData.set("limit", Number(pm.iterationData.get("limit")))) or keep typed cases in JSON. CSV has two more edges: a field containing a comma or newline must be quoted with embedded quotes doubled, and an empty cell arrives as "" rather than absence — so a "missing coupon" row silently sends coupon: "", a different input from the one you meant to test. Use a sentinel such as __ABSENT__ and strip it in the pre-request script.

Commit the dataset next to the collection. When a new contract case lands in a row and the expected status changes in the same diff, the reviewer sees coverage move with the code instead of discovering it in a nightly run. That pairing is the seed of the versioned contracts you build in JSON Schema Validation.

One scope question trips people up coming from Automating Postman Collections: row values live in the data scope, which shadows environment values for that iteration only. Keep per-case inputs (email, coupon) in the dataset and per-stage values (baseUrl, tenant) in the environment, so switching from staging to production re-points the run without touching a row — mixing the two is how a live coupon code ends up committed inside a CSV.

Interviewers ask how you scale API coverage; the answer that lands is "one request, a dataset, expectations in rows, files in git" — not "we wrote more requests."

A data-driven run of 140 iterations fails. The report says AssertionError: expected 422 to equal 201 and the stack points at your test script. Which case was it? Which input produced it? If answering requires cross-referencing iteration numbers against a spreadsheet, the dataset is failing you at the exact moment it should be earning its keep. The fix is structural: the data row carries the expectation, and the assertion carries the row's identity.

{ "case": "unicode-domain", "email": "buyer@shop.co.jp", "expectedStatus": 201, "expectedField": "id" }
{ "case": "wrong-currency", "coupon": "SAVE10-EUR", "expectedStatus": 422, "expectedCode": "COUPON_CURRENCY_MISMATCH" }

const row = pm.iterationData;
const body = pm.response.json();

pm.test(`[${row.get("case")}] status ${pm.response.code} == ${row.get("expectedStatus")}`, function () {
  pm.expect(pm.response.code).to.equal(Number(row.get("expectedStatus")));
});

pm.test(`[${row.get("case")}] error code`, function () {
  if (row.get("expectedCode")) pm.expect(body.error.code).to.equal(row.get("expectedCode"));
});

The report now reads [wrong-currency] status 422 == 422 in green and [expired-coupon] status 200 == 422 in red. The failing case named itself, and nobody opens the CSV. The template literal in the test name is the trick that makes this work in CI: JUnit XML and Allure carry test names into the dashboard, not stack traces, so the name is the unit of triage. A message that only exists in console.log never reaches the report your team reads.

Two ways this design rots, both worth an explicit guard. First, type coercion: the expected status arrives as a string from CSV, so to.equal("422") against the number 422 fails on every row and the suite cries wolf until someone weakens the assertion. Convert once, at the boundary, with Number(...). Second, optional expectations: a missing expectedField must mean "do not check", never undefined == undefined passing vacuously — guard with if (row.get("expectedCode")), as above, so a row that forgets a check is visible in review rather than silently green.

Give every row a unique, human case name (limit-overflow, not case17), and keep the dataset one row per meaningful reason. If two rows differ only in a value that does not change the expectation, you are padding; if one row tests two unrelated behaviors, split it. Reviewers should be able to read the dataset top to bottom as a specification of the endpoint's contract — that readable diff is exactly what makes boundary rows in the next chapter reviewable.

Runner mechanics finish the picture. newman run c.json --iteration-data cases.json --reporters cli,junit --bail executes one iteration per row and writes JUnit XML whose <testcase name> carries your templated case name — the artifact your dashboard parses, which is why the name, not the message, is where the diagnostic belongs. Leave --bail off in the nightly run: stopping at the first failing row hides the other 139 results and turns one bug into three investigation rounds.

Lint the dataset itself before running it, because a malformed row fails silently rather than loudly: jq -r '.[].case' cases.json | sort | uniq -d finds duplicate case names, and a quick check that every element carries case and expectedStatus catches the row where a missing expectation makes the assertion compare against NaN — a red build that points at the wrong cause. A dataset with a typo is worse than no dataset: it manufactures a confident-looking failure that sends someone hunting the API for a bug it never had.

Create and read endpoints take the dataset treatment easily. The endpoints where data-driven testing pays for itself are the list endpoints — GET /orders?sort=createdAt&dir=desc&limit=25&cursor=abc — because their behavior is a function of parameters, and parameters have edges. A list endpoint exercised with one happy limit=25 call has verified almost nothing: the bugs live at 0, at the maximum, one past the maximum, and in the interaction between sort and filter.

Build one dataset per parameter, pinning each edge and its neighbors:

[
  { "case": "default-page", "params": "", "expected": 200 },
  { "case": "limit-one", "params": "limit=1", "expected": 200 },
  { "case": "limit-max", "params": "limit=100", "expected": 200 },
  { "case": "limit-overflow", "params": "limit=101", "expected": 422 },
  { "case": "limit-zero", "params": "limit=0", "expected": 422 },
  { "case": "limit-negative", "params": "limit=-5", "expected": 422 },
  { "case": "limit-not-a-number", "params": "limit=ten", "expected": 422 }
]

The rows you cannot express as a single request matter most. Walk every page: a script requests limit=10 until nextCursor is null, then asserts the union of pages equals the reported total — catching the classic offset bugs where rows inserted between requests cause items to be skipped or duplicated at page boundaries. Assert id uniqueness across the walk with new Set(allIds).size === allIds.length; an unstable sort (ties in createdAt with no id tiebreaker) makes adjacent pages overlap, a defect invisible to any single-page test. Cross-check filter and sort: one row filters status=shipped and asserts every returned item matches, another sorts total:asc and asserts monotonicity — verifying the content of the ordering rather than only the 200.

The counterpart rows pin the API's rejection contract: an unknown sort=hairColor should be 422 (or explicitly ignored — the dataset records which the API chose), a filter value outside the enum likewise, and limit=ten proves type validation happens server-side. Also record the null-ordering row: where do rows with total: null land in an ascending sort? Two databases disagree, and the answer is part of the contract.

Two more dimensions belong in the list-endpoint dataset because they fail quietly in production. Filter content: one row requests status=shipped and asserts that every item in every page carries shipped — an endpoint that applies the filter to the first page only and forgets it on the cursor walk is green on single-page tests forever. Cross-parameter interactions: sort=total:asc alone is trivially correct, while status=shipped&sort=total:asc exercises the query path where filtering and ordering compose, which is where index choices and NULLS FIRST defaults actually surface. Budget the rows deliberately: a full cartesian product of four sort fields × three filters × eight limits is 96 rows of mostly duplicated paths, while one dimension per row with five interaction rows lands near 40 and tells you more. Every extra row is pipeline time paid on every pull request, so make each one name a distinct reason to fail.

Keep the matrix honest: vary one dimension per row instead of the full cartesian product, which explodes cost without adding paths. Roughly 120 rows covering edges, defaults, and cross-checks beat 900 rows of random valid values — every row must earn its runtime. This discipline, expectations-in-data plus boundary thinking, is the same one you will reuse one level up when JSON Schema Validation turns the payload shape itself into an assertion.