1.0 KiB
For live_gui tests, verification must be batch — not isolated
Isolated-Pass Verification Fallacy (Added 2026-06-09)
A test that "passes when run after test X but fails in isolation" is a fragile test, not a fragile fixture. The flip side is also true: a test that "passes in isolation but fails in batch" is failing — its failure is masked by isolation. The only verification that matters for live_gui tests (or any test that depends on shared subprocess state) is the batch run in the suite the test will ship in.
Rule: For any live_gui test or any test that depends on shared subprocess state, do NOT commit a fix that you have only verified in isolation. The fix must pass in the batched run that includes the tests that share the subprocess. Run the batch first. If the test fails in batch, your fix is incomplete. Per the existing Live_gui Test Fragility (Authoring-Side) rule above, the bisect requires both directions. If you only run in isolation, you cannot tell "test needs work" from "real app bug."