Comprehensive audit of 393 test files + the run_tests_batched runner.
Findings:
- 6 skip markers (4 same root cause: Gemini 503 in summarize.summarise_file)
- 60 files use time.sleep (38 live_gui — the banned anti-pattern)
- ~12-14 one-shot phase tests are cruft (verifying completed phases)
- 3 redundant test clusters (history: 5 files, theme: 6, markdown: 5)
- 27 live_gui tests are high-value test engine upgrade candidates
- ~44 live_gui tests are fine with the current Hook API
- ~10 new test capabilities enabled by the test engine (docking, focus, resize, keyboard, screenshots)
- The core batch is 245 files (62% of suite) — needs criticality-based splitting
Proposes a 3-dimension ordering taxonomy: (criticality, fixture, subsystem)
with 6 criticality levels (C0-smoke through C5-stress). The live_gui tier
mixes C0/C3/C4/C5 — splitting by criticality enables fast-fail + targeted
verification.
Recommends 4-track sequence: test_engine_integration → cruft_cleanup →
ordering_taxonomy → test_engine_migration.
ed
2b392b1f76