manual_slop/conductor/tracks/test_stabilization_20260302/spec.md

Specification: Test Suite Stabilization & Consolidation (test_stabilization_20260302)

Overview

The goal of this track is to stabilize and unify the project's test suite. This involves resolving pervasive asyncio lifecycle errors, consolidating redundant testing paradigms (specifically manual GUI subprocesses), ensuring artifact isolation in ./tests/artifacts/, implementing functional assertions for currently mocked-out tests, and updating documentation to reflect the finalized verification framework.

Architectural Constraints: Combating Mock-Rot

To prevent future testing entropy caused by "Green-Light Bias" and stateless Tier 3 delegation, this track establishes strict constraints:

• Ban on Aggressive Mocking: Tests MUST NOT use unittest.mock.patch to arbitrarily hollow out core infrastructure (e.g., the App lifecycle or async loops) just to achieve exit code 0.
• Mandatory Centralized Fixtures: All tests interacting with the GUI or AI client MUST use the centralized app_instance or live_gui fixtures defined in conftest.py.
• Structural Testing Contract: The project workflow must enforce that future AI agents write integration tests against the live state rather than hallucinated mocked environments.

Functional Requirements

• Asyncio Lifecycle Stabilization:
  • Resolve RuntimeError: Event loop is closed across the suite.
  • Implement ThreadPoolExecutor for blocking calls in GUI-bound tests.
  • Audit and fix fixture cleanup in conftest.py.
• Paradigm Consolidation (from testing_consolidation_20260302):
  • Refactor integration/visual tests to exclusively use the live_gui pytest fixture.
  • Eliminate all manual subprocess.Popen calls to gui_2.py in the tests/ and simulation/ directories.
  • Update legacy tests (e.g., test_gui_events.py, test_gui_diagnostics.py) that still import the deprecated gui_legacy.py to use gui_2.py.
  • Completely remove gui_legacy.py from the project to eliminate confusion.
• Artifact Isolation & Discipline:
  • All test-generated files (temporary projects, mocks, sessions) MUST be isolated in ./tests/artifacts/.
  • Prevent leakage into conductor/tracks/ or project root.
• Enhanced Test Reporting:
  • Implement structured, sectioned logging in ./tests/logs/ with timestamps (consolidating VerificationLogger outputs).
• Assertion Implementation:
  • Replace pytest.fail placeholders with full functional implementation.
• Simulation Regression Fixes:
  • Debug and resolve test_context_sim_live entry count issues.
• Documentation Updates:
  • Update Readme.md (Testing section) to explain the new log/artifact locations and the --enable-test-hooks requirement.
  • Update docs/guide_simulations.md to document the centralized pytest usage instead of standalone simulator scripts.

Acceptance Criteria

• Full suite run completes without RuntimeError: Event loop is closed warnings.
• No subprocess.Popen calls to gui_2.py exist in the test codebase.
• No test files import gui_legacy.py.
• gui_legacy.py has been deleted from the repository.
• All test artifacts are isolated in ./tests/artifacts/.
• All tests previously marked with pytest.fail now have passing functional assertions.
• Simulation tests pass with correct entry counts.
• Readme.md and docs/guide_simulations.md accurately reflect the new testing infrastructure.