Does the test suite cover the original problem?

Start from the accepted behavior.

A patch passes its suite. Before saying the original problem is covered, find the assertion that would fail if that problem returned. Read its setup and follow the path it exercises. A test called “downloads selected files” may only check that a request succeeded.

Start with the accepted task, not a guess about what the code ought to do. State the behavior in observable terms and link its source. If the requirement is unresolved, leave that row open rather than writing an expectation that quietly decides it.

Google’s engineering review guidance asks reviewers to examine whether tests are useful, including whether they would fail when the code is broken. The audit below applies that question to an existing suite. It is a proposed worksheet, not a coverage formula or a claim that one check proves a feature correct.

Inspect a fictional download suite.

This entire example is fictional: the contract, test descriptions and possible defect are invented for the exercise. No suite was executed and no customer failure is being reported.

The accepted bulk-download contract says the package must contain each selected file identity exactly once and exclude unselected files. Two selected files with the same display name must both be preserved. With no selection, the handler must not create a package. Output ordering and the exact naming convention are not specified.

The existing suite has three checks. One uses files with different names and asserts that a package was created. Another asserts that the handler passes the selected IDs to a packager replaced by a mock returning fixed package bytes. The third verifies that an empty selection does not call the packager. Read those assertions as three different pieces of evidence.

Describe a defect these assertions could miss.

Suppose the real packager accidentally keeps only one file for each display name. Selected IDs A and B have the same name but different fixture contents. One disappears from the output, while a package is still produced.

The first described check has no equal-name fixture and does not inspect membership. The second never runs the real packager. The empty-selection check exercises a different case. The proposed defect could therefore leave these described checks green. That is a logical blind spot in the fictional setup, not a measured mutation-test result.

In a real audit, follow the actual assertions before reaching that conclusion. Another test may already exercise the relevant path. To establish whether the suite detects a particular defect, use an isolated, permitted change or controlled faulty fixture, run the relevant checks, and retain the actual result. Confirm the failure concerns the intended behavior rather than a broken setup. Restore the original code and verify the final change separately.

Map each claim to an observation.

Use a small matrix for the acceptance claim under review. Include a counterexample: an output that violates the requirement while still satisfying a weaker assertion. This helps reveal whether the test can distinguish acceptable behavior from the defect you care about.

For the fictional case, prepare selected files A and B with the same display name and distinct fixture content tokens. Add unselected file C with a third token. A proposed check should exercise the real selection and packaging path, inspect the resulting entries, and establish that A and B are each represented once while C is absent. Merely asserting that the output has two entries would still allow the wrong two files.

Choose an observable identity or content check appropriate to the accepted format. If exact entry names matter, obtain the naming policy before asserting a particular suffix or path. The requirement to preserve both files does not choose that naming scheme.

Keep the boundary and result explicit.

The mocked call check is still useful: it can catch the handler forwarding the wrong IDs. It simply cannot establish behavior inside the dependency it replaces. Keep it when it protects a real responsibility, and add the narrowest real path that can observe the missing behavior. A full browser journey is unnecessary if a focused packaging integration can establish the contract in scope.

Record the inspected revision, fixture, assertion, relevant test command and observed result. A proposed check stays proposed until it runs. A passing result covers its inputs and boundary; it does not establish every archive format, caller, deployment or failure mode.

If a check fails, inspect why before calling it a production defect. The requirement, fixture or test setup may be wrong. If it passes, retain the remaining limits. Neither a green total nor a coverage percentage replaces the behavior-to-assertion trace.

Acceptance audit — fictional bulk download
Source: accepted exercise contract; preserve selected file identities.
Fixture to add: selected A/B share a display name; distinct contents; C unselected.
Current evidence: package-created assertion; mocked packager call; empty-selection check.
Possible miss: real packager collapses equal display names.
Owning boundary: selection-to-package output; exercise the real packager.
Proposed observation: A/B represented once each; C absent; inspect content, not only count.
Open policy: exact naming and output order; do not invent expectations.
Evidence owner: task engineer with the packaging/test maintainer.
Execution: no real test run supplied by this example.
Next: inspect existing assertions, then run the smallest relevant check.
Refresh when: acceptance, packaging path, fixture or assertions change.

Audit one important claim before expanding the suite.

Choose one acceptance behavior that matters to the change. Ask the engineer to point to its source, the test setup, the actual assertion and the boundary exercised. Then ask for one plausible defect that might satisfy the assertion anyway. Investigate that specific gap before generating a larger list of tests.

A team lead can have the task owner and test maintainer review this map alongside the change. The immediate decision is whether the stated acceptance claim has support and which focused evidence is missing. An exposed gap does not show that users encountered the defect or that the whole suite has little value.

The accepted task, repository tests and inspectable fixture outputs may be enough. Retrieval helps when the original reason is scattered, but no context service is required for this audit. Keep the result with the change so the next reviewer can recover what was checked, what remains open and why the assertion exists.