Custom Matchers and Assertions

The build is red and the log shows AssertionError with a 200-line JSON dump of the whole order response. Somewhere around line 187 the email differs. You open the payload, scroll, note the value, and go looking for the test to understand what it meant. Multiply that by every flaky triage session on the team and you have the real cost: a weak assertion converts a five-second failure into a ten-minute investigation. Teams feel this as arithmetic: twenty failures a week at ten minutes each to localize is a part-time job spent reading logs, and the cheapest quality improvement available is writing the message the first time.

The assertion itself is not wrong — the output is. Compare the two shapes:

Four properties make a failure message self-sufficient: name the exact path (users[3].email, not "response"), show expected versus actual with the formatting the app uses, include the context (method, URL, correlation id) so a failure can be replayed, and assert the smallest thing that proves the behavior rather than the whole object — whole-object comparison belongs to contract tests where the diff is the point. Scoped comparison is the discipline: toMatchObject when you care about a subset of keys, a snapshot when the payload is reviewed deliberately, and single-field asserts for behavior. The trap shows up in the log — pytest truncates long container diffs with ... Full diff truncated (N lines) ..., and jest collapses unequal objects to expect(received).toEqual(expected) — so the field that mattered ends up behind an ellipsis.

In pytest you get these properties by writing an assertion helper instead of repeating the logic per file:

def assert_field_error(resp, field, code):
    body = resp.get_json()
    errors = body.get('fieldErrors') or {}
    actual = errors.get(field)
    assert actual == code, (
        f"expected fieldErrors[{field!r}] == {code!r}, got {actual!r}; "
        f"all errors present: {errors}"
    )

A failure now reads expected fieldErrors['zip'] == 'zip_invalid', got None; all errors present: {'email': 'email_taken'} — it tells you the server rejected the email but never validated the postal code, which is two findings in one sentence. That is the whole goal: the message should let the person on triage rotation decide "product bug, test bug, or environment" without opening an IDE. There is a hidden bonus to asserting scalar values: pytest rewrites plain asserts and shows the evaluated expression, so assert data['total'] == 42.00 fails with assert 4200 == 42.00 — cents versus dollars diagnosed without a single print statement. Choose the smallest assertion and the framework formats the evidence for you.

Interviewers ask this as "how do you make failures debuggable?" Answer with the mechanism, not vibes: assertions name the path, carry expected and actual, include the request id, and verify one behavior; then say where you enforce it — a shared helper so the wording cannot drift per test file. The next step is exactly that: promoting helpers like assert_field_error into matchers the whole team spells the same way, which is where Design Patterns in Test Code hands off. Once the wording is uniform, the remaining question is how many failures one test run should report — chapter 3 answers it.

A helper function works until every file re-implements a slightly different version: one checks fieldErrors, another checks errors.fields, a third forgets to handle a missing map and crashes on None. A custom matcher collapses that into one registration that produces one wording, available in every test the moment the runner starts.

expect.extend({
  hasFieldError(received: ApiBody, field: string, code: string) {
    const errors = received.fieldErrors ?? {};
    const actual = errors[field];
    const pass = actual === code;
    return {
      pass,
      message: () =>
        pass
          ? `expected field "${field}" to have no error, but got ${JSON.stringify(actual)}`
          : `expected fieldErrors["${field}"] to be "${code}", got ${actual ?? 'missing'}\nfull map: ${JSON.stringify(errors, null, 2)}`,
    };
  },
});

The contract between matcher and runner is small but strict. pass answers "does the subject satisfy the matcher"; the runner inverts it when you chain .not, and it calls message() to explain the outcome. Because the runner flips pass, the matcher must supply a message for both branches — write one sentence for each, or .not.hasFieldError('zip', 'zip_invalid') will print a message that claims the opposite of what happened.

Usage reads as domain language instead of plumbing:

expect(body).hasFieldError('email', 'email_taken');
expect(body).not.hasFieldError('zip', 'zip_invalid');

Once the matcher reads as domain language, the difference from a plain helper is concrete:

TypeScript needs one declaration so the compiler knows the matcher exists:

declare global {
  namespace jest {
    interface Matchers<R> {
      hasFieldError(field: string, code: string): R;
    }
  }
}

Registration happens once, in the runner's setup file — setupFilesAfterEnv: ['<rootDir>/tests/jest.setup.ts'] for Jest, and for Playwright you extend test or expect in a module that every test imports. Playwright's expect.extend only accepts synchronous matchers: an async check becomes a test.step around a request instead, or you await the data first and pass the parsed body. Vitest mirrors Jest's API — the same expect.extend object plus a declare module 'vitest' block for types — so a matcher moves between runners with almost no edits. Name matchers after the domain rule (hasFieldError, not hasErrorMap): the failure report is where the next reader learns the API.

Centralization is the maintenance argument. Put matchers in tests/matchers/, document them in a table in the README (hasFieldError, matchesInvoice, hasSecurityHeaders), and require a test for each matcher — matchers are code, and a bug in one silently weakens every assertion using it. The test proves both branches and the negation:

describe('hasFieldError matcher', () => {
  it('passes when the field carries the code', () => {
    expect({ fieldErrors: { zip: 'zip_invalid' } }).hasFieldError('zip', 'zip_invalid');
  });

  it('reports the full error map when it fails', () => {
    expect(() => expect({ fieldErrors: { email: 'email_taken' } })
      .hasFieldError('zip', 'zip_invalid'))
      .toThrow(/full map/);
  });

  it('inverts cleanly under .not', () => {
    expect({ fieldErrors: {} }).not.hasFieldError('zip', 'zip_invalid');
  });
});
``` Keep the set small: **eight to ten matchers that encode domain rules** pay for themselves; fifty micro-matchers (`toBeWithinTwoSeconds` variants) become an API the team must memorize.

The failure output is now the same on every laptop and on CI, which is what lets you rely on it when validating whole responses. That is the setup for soft assertions, and the extension model you just used generalizes to the runner itself in [Fixtures, Plugins, and Extensibility](/topic/qa-fixtures-plugin-extensibility).

A hard assertion ends the test on the first mismatch. That is correct behavior for a broken precondition and wasteful behavior for a six-field validation: the run reports one defect, you fix it, CI reruns, and a second defect surfaces — six cycles to learn what one response already knew. Soft assertions collect failures and report them together at the end of the test.

Playwright ships them as expect.soft, keeping the same matcher API:

expect.soft(res.status()).toBe(200);
expect.soft(body.customer.email).toBe(seed.email);
expect.soft(body.lines).toHaveLength(2);
expect.soft(body.currency).toBe('USD');
await expect.soft(page.getByTestId('total')).toHaveText('$42.00');

If three of those fail, the report lists three failures with their individual messages, and the test is still marked failed at the end. AssertJ has offered this in Java since 3.9 with SoftAssertions.assertSoftly(soft -> { ... }), whose lambda keeps every assertion inside one scoped block:

SoftAssertions.assertSoftly(soft -> {
    soft.assertThat(inv.getLines()).hasSize(2);
    soft.assertThat(inv.getCurrency()).isEqualTo("USD");
    soft.assertThat(inv.getCustomer().getEmail()).isEqualTo(seed.email());
});
``` Pytest core deliberately has none — soft checking needs the test to continue after a failed assert, so the community plugin `pytest-check` provides `check.equal(a, b, 'msg')`, which records failures and keeps going until teardown:

```python
import pytest_check as check

def test_invoice_payload_matches_seed(client, seed):
    body = client.get(f'/invoices/{seed.id}').get_json()
    check.equal(body['currency'], 'USD')
    check.equal(len(body['lines']), 2)
    check.equal(body['customer']['email'], seed.email)

Every check failure is recorded with its message, the test runs to the end, and pytest reports the list together.

Soft does not mean unconditional. Two rules keep the signal clean:

• Hard between stages, soft within one response. If login returns 401, every body assertion after it is noise — one red precondition beats five cascading errors. Once a stage passes, the fields it produced can be validated softly in one sweep.
• Never continue past a state you depend on. Asserting the cart total softly after the checkout call failed tells you nothing the first failure did not. One interaction with retries is worth internalizing: each retry attempt reports its own full set of soft failures, so a flaky-retry dashboard shows a stable defect list instead of a single random mismatch — and if the same soft failure leads every attempt, that is a flake to fix upstream, not a validation gap to keep aggregating.

Placement in the report matters as much as collection: wrap each logical group in a step (Playwright test.step, Allure steps) so a reviewer sees validate invoice lines — 2 of 5 failed instead of a flat list. Two mechanics are worth knowing before you build on this: deferred is not suppressed — every soft expectation runs, the ones that failed are listed, and the test still ends red (if all of them pass, the test is green); and soft checks do not halt execution, so keep awaits hard for anything that must succeed before the next line runs. The math is why teams adopt it — six hard cycles to expose six defects versus one run listing all of them — and it changes test naming too: a test that owns a validation group should be called validate_invoice_payload, not invoice line count. The same discipline from Design Patterns in Test Code applies — one behavior per group, and a name that reads as a sentence.

Interviewers ask the practical version: "a response has eight required fields — how do you verify it?" A strong answer layers the tools: hard-assert the status and the correlation id, soft-assert the eight fields through one domain matcher, and explain the trade-off — you trade an early exit for a complete defect list per run, which shortens the fix-rerun loop on API-heavy suites. Where those matchers get their data, sessions, and databases is the next topic: Fixtures, Plugins, and Extensibility.