Designing the Capstone Framework

An interviewer asks: "How is your automation framework organized?" The answer that ends the conversation is "there's a folder of tests." The answer that keeps it going names three layers and the rule that holds them apart — each layer answers a different question, and no layer reaches into another's internals.

Take checkout. The UI layer proves a shopper can complete the flow: Playwright drives /checkout, fills the form, clicks Place order, and waits for the confirmation screen. The API layer proves the server recorded it: GET /api/orders/{id} returns status: "CONFIRMED" and total: 42.50 inside 300 ms. The data layer proves the money is right, because the other two can both lie to you — a parameterized SQL query joins order_items and recomputes the tax that the UI merely displays and the API merely echoes.

capstone/
├─ config/env.ts      the ONLY module that reads process.env
├─ fixtures/          test.extend() — customers, orders, sessions
├─ ui/pages/          CheckoutPage.ts — locators + user intents
├─ api/clients/       OrdersApi.ts — typed request/response pairs
├─ data/queries/      orderSql.ts — parameterized SQL, no string concat
└─ tests/
   ├─ ui/             checkout.spec.ts      ~42s per file
   ├─ api/            orders.spec.ts        ~3s per file
   └─ e2e/            place-order.spec.ts   spans all three layers

The dependency rule is the design: tests → ui | api | data → fixtures → config. Nothing imports sideways. CheckoutPage.ts never calls OrdersApi, and data/queries never waits on a DOM element. That is not tidiness for its own sake — it decides what breaks when the product changes. A renamed button invalidates UI specs only. A changed response field breaks the API client's type at compile time. A schema migration fails the data queries loudly. When layers reach into each other, every product change detonates the whole suite and triage turns into archaeology.

Each layer also earns its cost by the question it answers:

Because API and data checks are cheap, push verification downward: create the order through OrdersApi.create(...) instead of clicking through five screens, then spend the one expensive UI test on the interaction that only exists in a browser. A suite of 400 tests where 350 are UI tests is a maintenance liability; the same coverage expressed as 80 UI + 250 API + 70 data runs in a quarter of the time and breaks a quarter as often.

The e2e/ directory is the exception that proves the rule: those specs compose the layers instead of merging them. place-order.spec.ts drives the UI to place the order, calls OrdersApi.get(id) to assert the server's view, then runs one SQL query to confirm the persisted total — three assertions, each taken at the cheapest layer that can see the truth. That composition is also the clean answer to the interview staple "how do you decide what to test at which level?": automate at the highest layer only when the risk lives in the interaction, and assert at the lowest layer that can see the state.

Playwright can host all three layers in one runner — request fixtures for API calls, pg for SQL — so you ship one report and one command, npx playwright test, instead of three toolchains glued together. Consolidating is a decision worth recording, and keeping those boundaries enforceable as the team grows is exactly what Framework Architecture and Ownership covers. Once the layers exist and the boundaries hold, Wiring the Capstone into CI is where the suite starts running on every pull request.

The staging hostname changes from staging.acme.dev to staging2.acme.dev. In a sloppy framework you grep twelve files, miss the one hard-coded in helpers/login.ts, and find the leftover three days later when an auth test mysteriously passes. In a disciplined one you edit a single value — because process.env is read in exactly one file, and everything else receives configuration as a parameter.

// config/env.ts — the only module allowed to touch process.env
import { z } from 'zod';

const Schema = z.object({
  BASE_URL: z.string().url(),
  API_KEY: z.string().min(20),
  DB_URL: z.string().startsWith('postgres://'),
  RETRIES: z.coerce.number().int().min(0).default(2),
});

const parsed = Schema.safeParse(process.env);
if (!parsed.success) {
  throw new Error(
    'Invalid test config: ' +
    parsed.error.issues.map(i => `${i.path[0]}: ${i.message}`).join(', ') +
    '\nCopy .env.example to .env and fill every value.'
  );
}
export const env = parsed.data;

The payoff is the failure message. A missing BASE_URL used to surface as net::ERR_NAME_NOT_RESOLVED on the first navigation — a browser error about the network, four steps away from the real cause. Now the suite dies in 40 ms with Invalid test config: BASE_URL: Required, which names the fix. Failing at the config boundary converts an afternoon of debugging into a typo correction, and it is the single cheapest reliability win in the whole framework.

Config discipline extends to which environments exist at all. BASE_URL must never default: a framework that silently falls back to production because .env was not copied is one missing variable away from writing test orders into live data. The schema refuses defaults for anything with blast radius — only RETRIES gets one — and CI injects values explicitly per workflow (env: BASE_URL: ${{ vars.STAGING_URL }}) instead of trusting runner state. The committed .env.example documents every knob, so the schema doubles as the environment contract and a new engineer sees exactly what a run needs.

Fixtures are how configuration gets delivered. Playwright's test.extend defines named resources that are created before a test and torn down after it:

export const test = base.extend<{ customer: Customer; api: OrdersApi }>({
  customer: async ({ request }, use) => {
    const res = await request.post('/api/customers', { data: signupPayload() });
    const customer = await res.json();                     // 200 OK in ~120ms
    await use(customer);                                   // test runs here
    await request.delete(`/api/customers/${customer.id}`); // deterministic cleanup
  },
});

A test that needs a shopper declares customer in its signature. No shared beforeEach chain that logs the same account in 400 tests, no ordering dependency between specs, no orphaned rows after a crash because teardown runs in the fixture rather than in your test body. Scope is a real choice: fixtures default to per-test isolation, while test.describe.configure({ mode: 'serial' }) or a worker-scoped fixture covers genuinely shared resources such as a seeded product catalog that would take 20 seconds to rebuild per test.

Page objects absorb selector churn, and the rule that keeps them honest is: a page object exposes intents and data, never raw elements. await checkout.placeOrder() survives the button text changing from Place Order to Complete purchase; page.click('#btn-3') copied into thirty specs does not. Locators inside the page object go through stable queries — page.getByTestId('place-order') backed by data-testid in the markup — because long CSS chains and nth=3 are the same churn wearing a different hat.

Config, fixtures, and page objects are one mechanism seen from three angles: push volatility — environments, accounts, and the DOM — behind stable interfaces, so a test body reads like the requirement it protects and nothing else. When that holds, the framework is ready for Wiring the Capstone into CI, and it was assembled on the ground laid by Framework Architecture and Ownership.

A reviewer opens your capstone repository with one question before reading a single spec: what were the trade-offs? Code shows what you built. It cannot show what you rejected, or why. That record is the Architecture Decision Record — a short markdown file kept next to the code, one per decision that would be expensive to reverse.

# ADR-004: Playwright Test as the single runner
Status: Accepted · 2026-03-11
Context:  UI, API and data specs must share one report; CI needs
          sharding; the team is four engineers on Node 20.
Decision: Playwright Test runs tests/ui, tests/api and tests/e2e.
Alternatives:
  Cypress        — strong runner, but API checks need cy.request
                   workarounds and there is no native sharding.
  Selenium+JUnit — parallel JVM stack for a Node team; new CI
                   images and a hiring constraint for no gain.
Consequences: + one report, --shard=4 works, trace viewer for triage;
               - the team must learn a new locator model.

Three parts do the work. Context states the forces that made the choice non-obvious. Decision names the choice in one sentence. Consequences admits what you gave up, because a decision with no downside is a decision nobody evaluated. Interviewers probe exactly this shape: "Why Playwright rather than Cypress?" has no defensible answer without the context line, and "we preferred it" ends the discussion. Keep ADRs immutable — when a decision is reversed, add a new ADR that supersedes the old one so the history survives instead of being edited away.

The README is the other half of the contract, and it has an acceptance test: a new hire reaches first green without asking anyone, in under fifteen minutes. Write it as commands you have actually run on a clean machine, in the order that fails loudly when skipped:

git clone git@github.com:acme/capstone-automation.git
cd capstone-automation
nvm use                        # .nvmrc pins Node 20.11
npm ci                         # 41s with a warm cache
cp .env.example .env           # BASE_URL, API_KEY, DB_URL
docker compose up -d db        # Postgres 16, healthy in ~5s
npx playwright install --with-deps
npm run test:smoke             # 12 passed (6.4s)

Omit npm ci and the config module throws its Invalid test config error; skip Docker and the data layer's first query fails with ECONNREFUSED 127.0.0.1:5432. Under the commands: what passing output looks like, the three most common errors with their fixes, where reports land (npx playwright show-report), who maintains the framework, and how to propose a change. That last section links back to the contribution rules from Framework Architecture and Ownership, so there is one source of truth rather than two competing ones.

Both artifacts need a hygiene rule or they rot like code does. Keep ADRs in docs/adr/ with padded numbering (0001-record-architecture-decisions.md ships as the template, 0004-playwright-over-cypress.md is the first real one) so the directory listing is the index and the git log is the history. Treat the README as a tested contract: the contribution guide requires that any change to setup steps updates the README in the same PR, and the update is verified by a colleague following it cold. Documentation that nothing validates rots on the same schedule as untested code — the difference is only whether something fails loudly when it does.

Version control discipline ties it together. Trunk-based development with short-lived branches, conventional commits (feat(fixtures): add guest customer fixture), and tagged framework releases:

git tag -a framework-v1.4.0 -m "OrdersApi v2 contract, guest fixture"
git push --tags

Tags matter because product teams consume this framework as a package — a pinned @acme/automation@1.4.0 plus a readable diff between releases beats "pull main and hope". ADRs explain the decisions, the README makes the repository runnable on day one, and the tags make progress legible. Together they answer the capstone's real question: could this survive you leaving? Once it can, Wiring the Capstone into CI puts the suite on every pull request.