Test Levels and Test Types

A price-discount bug reaches production. The unit test for the discount function passes. The integration test with the pricing service passes. The system test of the cart passes. Acceptance testing signed off last Thursday. Where was the test that should have caught it? Almost always, the answer is a boundary between levels — and that is why levels matter more than any individual test.

A test level is a set of tests with a specific scope, objectives, and position in the lifecycle. The four canonical levels:

Unit tests verify the smallest testable pieces. They are fast and deterministic because everything external — the database, the clock, HTTP calls — is replaced by fakes. Their contract: if the function's logic is wrong, a unit test fails. Their blind spot: the function can be perfectly correct while wiring it to the wrong neighbor breaks the product.

Integration tests cover exactly that wiring. Does the order service persist rows the way the database schema actually expects? Do serialized fields match between producer and consumer? The modern refinement is the split: contract integration tests (consumer-driven pacts, schema checks) run fast in CI against lightweight fakes, while a smaller suite runs against a real containerized Postgres or a staging Kafka, because production-like state exposes behaviors mocks hide — nullability, timeouts, transaction isolation.

System tests treat the deployed application as one black box through its real interfaces — UI, API, message consumers — and verify complete features: register, add to cart, apply coupon, pay, receive confirmation email. Only here do you see the assembled behavior, which is why environment drift (a test environment missing the email service) makes system testing silently skip what it claims to verify.

Acceptance testing asks whether the built system satisfies the business rule the feature was funded for: can a premium customer actually use the negotiated contract pricing in the demo environment? In agile teams this becomes product-owner walkthroughs of acceptance criteria on the sprint increment; in BDD setups, acceptance criteria are executable scenarios that double as the tests.

The interview-tested fact: defects get cheaper as the level drops and rarer as the level rises. A logic error caught by a unit test costs a fix inside one file; the same error escaping to production drags integration data, system test cycles, and a hotfix release behind it. When production shows a discount bug that every level missed, the productive question is which boundary it slipped through — often an untested contract between two services that both had green suites.

Junior testers classify tests by tool — 'the Selenium tests', 'the Postman tests'. Interviewers and test strategists classify by intent: what question does this suite answer? Four intent categories cover most of what you will meet.

Functional testing verifies that a feature behaves according to its specification: the login API returns 200 with valid credentials and 401 with an invalid token; the export produces the correct CSV columns. Functional tests can live at any level — a unit test of a discount function is functional; so is a system test of checkout.

Regression testing answers a different question: did recent changes break existing behavior? It re-runs previously executed tests — ideally the automated ones — against the new build. Regression is an intent, not a suite: after a fix to the discount engine, re-running the full checkout flow is regression testing even though the flow is also functional. The economics argue for automation here more than anywhere else: regression runs repeatedly (every build, every release), so its cost is multiplied by every future sprint, while manual regression is the number one source of 'we don't have time to test' crises.

Smoke testing is the shallow, wide pass that answers: is this build stable enough to deserve deeper testing? A good smoke suite is 10 to 20 checks taking under 30 minutes — the app starts, login works, the three core journeys complete at happy-path level, the database connects, the payment sandbox responds. Smoke tests run on every candidate build, often as the last CI stage before a deploy to the test environment. The classic anti-pattern is launching a six-hour system test cycle on a build where the login API is down; forty minutes in, you learn what smoke testing would have told you immediately.

Sanity testing is the narrow, deep follow-up after a small change: the team changed only the tax calculation, so you verify tax scenarios — correct rate per region, rounding, invoice totals — and skip everything untouched. Sanity is what you run when a hotfix or a localized change needs targeted confirmation, not a full regression. The distinction interviewers probe: smoke asks 'is the build sane at all?', sanity asks 'does this specific change work, given the build is otherwise known-good'?

A practical sequence for release day: smoke first; on green, full regression; targeted functional testing for changed features throughout; sanity passes after each hotfix. Note that none of these names a tool — the same Playwright spec can be serving smoke or regression intent depending on which suite it is placed in, and putting a broken-world test in the smoke suite because 'it's automated' defeats the purpose. Intent decides suite membership; automation is an implementation detail.

Consider the discount bug again with forensics instead of blame. The unit test passed because it tested the discount function in isolation — with a mocked tax service that always returned 200. The integration test passed because it exercised the discount service against a real database but never called the tax service at all. System testing passed because the environment's tax service stub returns a flat rate for every region. Acceptance passed because the demo customer has no regional tax rules configured. Every level was green, and the product was broken.

This is the standard failure shape: each level tested its slice correctly while the seams between slices went untested. Reading escapes this way produces concrete repairs:

1. Map the defect to the smallest level that could have caught it. For the discount bug, that was integration: discount service calling the tax service with a regional rule. That level had a gap.
2. Ask why the gap existed. The integration suite mocked the tax response — a decision made for speed. Legitimate, but it silently defined the seam as out of scope.
3. Close the gap at the cheapest level that sees it. Add a contract test asserting the tax API's regional-rule behavior, plus one system test exercising a regional tax end to end. Do not fix the escape by adding system tests everywhere — that raises the cost of every future run.
4. Check the level's charter, not just its cases. Gaps hide in definitions: if 'integration' means only service-to-database, the service-to-service seam is homeless. Give every seam an owning level in the test strategy.

Two quantitative habits make this repeatable. First, track escape level: for each production defect, record the earliest level that could have detected it and whether that level actually did. A quarterly review showing five escapes from the integration boundary makes the case for contract-testing investment better than any argument. Second, watch for environment gaps: a system suite that runs against a stubbed payment gateway verifies the stub as much as the product; listing every stub in your test environment is a five-minute audit that frequently finds untested seams.

Interviewers love this scenario because it tests process maturity, not trivia. The answer skeleton is: identify the smallest level that could have caught it → treat the miss as a level-boundary or charter problem → close it with tests at that level → feed the data back into the strategy. Mentioning that you would not respond by inflating the slowest, most expensive suite is the differentiator, because it shows you understand why levels exist: each one buys specific information at a specific cost, and a defect that crosses all of them means the buying plan had a hole, not that you needed to buy everything.

A worked exercise makes the loop concrete: take the last quarter's production defects and build a three-column retrospective — defect, earliest level that could have caught it, level that actually did. The gap column is your backlog. Teams that run this exercise once usually find the same distribution: a cluster of requirement defects that no test level could catch (those route to refinement practice), a cluster owned by the integration boundary (those route to contract tests), and one or two genuinely unforeseeable events (those become documented residual risk). The exercise takes an hour and typically re-prioritizes the next quarter of test investment more accurately than any strategy debate.

For how these levels are deliberately balanced against business-facing testing, see Agile Testing and Shift-Left.