diff --git a/conductor/tracks.md b/conductor/tracks.md index 405cc48..0a22fd2 100644 --- a/conductor/tracks.md +++ b/conductor/tracks.md @@ -6,6 +6,9 @@ This file tracks all major tracks for the project. Each track has its own detail ## Current Tracks +- [ ] **Track: Test Suite Stabilization & Consolidation** +*Link: [./tracks/test_stabilization_20260302/](./tracks/test_stabilization_20260302/)* + --- ## Completed / Archived diff --git a/conductor/tracks/test_stabilization_20260302/index.md b/conductor/tracks/test_stabilization_20260302/index.md new file mode 100644 index 0000000..341517e --- /dev/null +++ b/conductor/tracks/test_stabilization_20260302/index.md @@ -0,0 +1,5 @@ +# Track test_stabilization_20260302 Context + +- [Specification](./spec.md) +- [Implementation Plan](./plan.md) +- [Metadata](./metadata.json) diff --git a/conductor/tracks/test_stabilization_20260302/metadata.json b/conductor/tracks/test_stabilization_20260302/metadata.json new file mode 100644 index 0000000..dd6cd66 --- /dev/null +++ b/conductor/tracks/test_stabilization_20260302/metadata.json @@ -0,0 +1,8 @@ +{ + "track_id": "test_stabilization_20260302", + "type": "chore", + "status": "new", + "created_at": "2026-03-02T22:09:00Z", + "updated_at": "2026-03-02T22:09:00Z", + "description": "Comprehensive Test Suite Stabilization & Consolidation. Fixes asyncio errors, resolves artifact leakage, and unifies testing paradigms." +} diff --git a/conductor/tracks/test_stabilization_20260302/plan.md b/conductor/tracks/test_stabilization_20260302/plan.md new file mode 100644 index 0000000..a567484 --- /dev/null +++ b/conductor/tracks/test_stabilization_20260302/plan.md @@ -0,0 +1,30 @@ +# Implementation Plan: Test Suite Stabilization & Consolidation (test_stabilization_20260302) + +## Phase 1: Infrastructure & Paradigm Consolidation +- [ ] Task: Initialize MMA Environment `activate_skill mma-orchestrator` +- [ ] Task: Setup Artifact Isolation Directories + - [ ] Create `./tests/artifacts/` and `./tests/logs/` with appropriate `.gitignore`. +- [ ] Task: Migrate Manual Launchers to `live_gui` Fixture + - [ ] Refactor `tests/visual_mma_verification.py` to use the `live_gui` fixture. + - [ ] Audit `simulation/` and `tests/` for other manual subprocess launchers and refactor. +- [ ] Task: Conductor - User Manual Verification 'Phase 1: Infrastructure & Consolidation' (Protocol in workflow.md) + +## Phase 2: Asyncio Stabilization & Logging +- [ ] Task: Audit and Fix `conftest.py` Loop Lifecycle + - [ ] Ensure all fixtures correctly handle loop cleanup and task cancellation. +- [ ] Task: Resolve `Event loop is closed` in Core Test Suite + - [ ] Update identified files to pass active loops and use `ThreadPoolExecutor`. +- [ ] Task: Implement Centralized Sectioned Logging Utility +- [ ] Task: Conductor - User Manual Verification 'Phase 2: Asyncio & Logging' (Protocol in workflow.md) + +## Phase 3: Assertion Implementation & Simulation Fixes +- [ ] Task: Replace `pytest.fail` with Functional Assertions + - [ ] Focus on `api_events`, `execution_engine`, `token_usage`, and `agent_capabilities`. +- [ ] Task: Resolve Simulation Entry Count Regressions + - [ ] Fix entry count assertions in `test_context_sim_live` and align mocks. +- [ ] Task: Conductor - User Manual Verification 'Phase 3: Assertions & Sims' (Protocol in workflow.md) + +## Phase 4: Final Verification & Artifact Audit +- [ ] Task: Full Suite Validation & Warning Cleanup +- [ ] Task: Final Artifact Isolation Verification +- [ ] Task: Conductor - User Manual Verification 'Phase 4: Final Verification' (Protocol in workflow.md) diff --git a/conductor/tracks/test_stabilization_20260302/spec.md b/conductor/tracks/test_stabilization_20260302/spec.md new file mode 100644 index 0000000..e9f8eef --- /dev/null +++ b/conductor/tracks/test_stabilization_20260302/spec.md @@ -0,0 +1,29 @@ +# Specification: Test Suite Stabilization & Consolidation (test_stabilization_20260302) + +## Overview +The goal of this track is to stabilize and unify the project's test suite. This involves resolving pervasive `asyncio` lifecycle errors, consolidating redundant testing paradigms (specifically manual GUI subprocesses), ensuring artifact isolation in `./tests/artifacts/`, and implementing functional assertions for currently mocked-out tests. + +## Functional Requirements +- **Asyncio Lifecycle Stabilization:** + - Resolve `RuntimeError: Event loop is closed` across the suite. + - Implement `ThreadPoolExecutor` for blocking calls in GUI-bound tests. + - Audit and fix fixture cleanup in `conftest.py`. +- **Paradigm Consolidation (from testing_consolidation_20260302):** + - Refactor integration/visual tests to exclusively use the `live_gui` pytest fixture. + - Eliminate all manual `subprocess.Popen` calls to `gui_2.py` in the `tests/` and `simulation/` directories. +- **Artifact Isolation & Discipline:** + - All test-generated files (temporary projects, mocks, sessions) MUST be isolated in `./tests/artifacts/`. + - Prevent leakage into `conductor/tracks/` or project root. +- **Enhanced Test Reporting:** + - Implement structured, sectioned logging in `./tests/logs/` with timestamps. +- **Assertion Implementation:** + - Replace `pytest.fail` placeholders with full functional implementation. +- **Simulation Regression Fixes:** + - Debug and resolve `test_context_sim_live` entry count issues. + +## Acceptance Criteria +- [ ] Full suite run completes without `RuntimeError: Event loop is closed` warnings. +- [ ] No `subprocess.Popen` calls to `gui_2.py` exist in the test codebase. +- [ ] All test artifacts are isolated in `./tests/artifacts/`. +- [ ] All tests previously marked with `pytest.fail` now have passing functional assertions. +- [ ] Simulation tests pass with correct entry counts. diff --git a/conductor/tracks/testing_consolidation_20260302/index.md b/conductor/tracks/testing_consolidation_20260302/index.md deleted file mode 100644 index 38368bb..0000000 --- a/conductor/tracks/testing_consolidation_20260302/index.md +++ /dev/null @@ -1,5 +0,0 @@ -# Track testing_consolidation_20260302 Context - -- [Specification](./spec.md) -- [Implementation Plan](./plan.md) -- [Metadata](./metadata.json) \ No newline at end of file diff --git a/conductor/tracks/testing_consolidation_20260302/metadata.json b/conductor/tracks/testing_consolidation_20260302/metadata.json deleted file mode 100644 index 84eae50..0000000 --- a/conductor/tracks/testing_consolidation_20260302/metadata.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "track_id": "testing_consolidation_20260302", - "type": "chore", - "status": "new", - "created_at": "2026-03-02T00:00:00Z", - "updated_at": "2026-03-02T00:00:00Z", - "description": "Consolidate divergent simulation tests to uniformly use the pytest live_gui fixture and remove redundant subprocess launcher scripts." -} \ No newline at end of file diff --git a/conductor/tracks/testing_consolidation_20260302/plan.md b/conductor/tracks/testing_consolidation_20260302/plan.md deleted file mode 100644 index 2e286ac..0000000 --- a/conductor/tracks/testing_consolidation_20260302/plan.md +++ /dev/null @@ -1,16 +0,0 @@ -# Implementation Plan: Testing & Simulation Consolidation - -Architecture reference: [docs/guide_simulations.md](../../../docs/guide_simulations.md) - ---- - -## Phase 1: Migrate Manual Launchers to Pytest Fixtures -Focus: Remove `subprocess.Popen` from visual verification scripts and convert them to proper pytest tests. - -- [ ] Task 1.1: Refactor `tests/visual_mma_verification.py` to be a standard pytest function: `def test_visual_mma_verification(live_gui):`. Remove all `subprocess.Popen` and directory changing logic. -- [ ] Task 1.2: Audit `tests/` for any other file containing `subprocess.Popen` pointing to `gui_2.py` and refactor them similarly. - -## Phase 2: Consolidate Simulation Scripts -Focus: Ensure the `simulation/` directory integrates cleanly with the pytest framework or serves a distinct non-testing purpose. - -- [ ] Task 2.1: Audit the `simulation/` directory. If scripts there are just tests in disguise, move them into `tests/` and wrap them in the `live_gui` fixture. If they are intended as standalone interactive demos, clearly document their purpose and ensure they don't duplicate `conftest.py` logic unnecessarily. \ No newline at end of file diff --git a/conductor/tracks/testing_consolidation_20260302/spec.md b/conductor/tracks/testing_consolidation_20260302/spec.md deleted file mode 100644 index 1037e00..0000000 --- a/conductor/tracks/testing_consolidation_20260302/spec.md +++ /dev/null @@ -1,16 +0,0 @@ -# Track Specification: Testing & Simulation Consolidation - -## Overview -Currently, the codebase has redundant testing paradigms. Some tests (`tests/visual_sim_gui_ux.py`) properly use the `live_gui` fixture managed by `pytest` in `conftest.py`. However, other visual verification scripts (like `tests/visual_mma_verification.py` and potentially files in `simulation/`) reinvent the wheel by manually opening subprocesses with `subprocess.Popen` to launch the GUI. This fragmentation causes tech debt and test flakiness. - -## Current State Audit -1. **Redundant Subprocess Launching**: `tests/visual_mma_verification.py` manually spawns `gui_2.py` via `subprocess.Popen` instead of using the `conftest.py` `live_gui` fixture. -2. **Simulation Redundancy**: The `simulation/` directory contains `sim_base.py`, `workflow_sim.py`, etc., that also use `ApiHookClient` but may be reinventing pytest workflows outside of the standard test runner. - -## Desired State -- All "visual" or "integration" testing scripts that interact with the live GUI via `ApiHookClient` MUST use the `live_gui` pytest fixture and be executed via `pytest`. -- Any standalone scripts in `tests/` that manually spawn `subprocess.Popen` for `gui_2.py` must be rewritten as standard pytest functions taking the `live_gui` argument. - -## Technical Constraints -- No tests should manually spawn `gui_2.py`. They must rely on `conftest.py`. -- Keep testing framework unified strictly under `pytest`. \ No newline at end of file