chore(conductor): Add new track 'Test Suite Curation and Organization'

This commit is contained in:
2026-02-25 20:42:43 -05:00
parent 3a6a53d046
commit edae93498d
5 changed files with 77 additions and 0 deletions

View File

@@ -45,6 +45,9 @@ This file tracks all major tracks for the project. Each track has its own detail
--- ---
- [ ] **Track: Test Suite Curation and Organization**
*Link: [./tracks/test_curation_20260225/](./tracks/test_curation_20260225/)*
--- ---

View File

@@ -0,0 +1,5 @@
# Track test_curation_20260225 Context
- [Specification](./spec.md)
- [Implementation Plan](./plan.md)
- [Metadata](./metadata.json)

View File

@@ -0,0 +1,8 @@
{
"track_id": "test_curation_20260225",
"type": "chore",
"status": "new",
"created_at": "2026-02-25T20:42:00Z",
"updated_at": "2026-02-25T20:42:00Z",
"description": "Review all tests that exist, some like the mma are conductor only (gemini cli, not related to manual slop program) and must be blacklisted from running when testing manual_slop itself. I think some tests are failing right now. Also no curation of the current tests has been done. They have been made incremetnally, on demand per track needs and have accumulated that way without any second-pass conslidation and organization. We problably can figure out a proper ordering, either add or remove tests based on redundancy or lack thero-of of an openly unchecked feature or process. This is important to get right now before doing heavier tracks."
}

View File

@@ -0,0 +1,28 @@
# Implementation Plan: Test Suite Curation and Organization
This plan outlines the process for categorizing, organizing, and curating the existing test suite using a central manifest and exhaustive review.
## Phase 1: Research and Inventory
- [ ] Task: Initialize MMA Environment `activate_skill mma-orchestrator`
- [ ] Task: Inventory all existing tests in `tests/` and mapping them to categories
- [ ] Task: Identify failing and redundant tests through a full execution sweep
- [ ] Task: Conductor - User Manual Verification 'Phase 1: Research and Inventory' (Protocol in workflow.md)
## Phase 2: Manifest and Tooling
- [ ] Task: Design and create `tests.toml` manifest file
- [ ] Task: Implement a test runner wrapper or `pytest` configuration to respect the manifest and categories
- [ ] Task: Verify that Conductor/MMA tests can be explicitly excluded from default runs
- [ ] Task: Conductor - User Manual Verification 'Phase 2: Manifest and Tooling' (Protocol in workflow.md)
## Phase 3: Curation and Consolidation
- [ ] Task: Fix all identified non-redundant failing tests
- [ ] Task: Consolidate redundant tests into single, comprehensive test files
- [ ] Task: Remove obsolete or deprecated test files
- [ ] Task: Standardize test naming conventions across the suite
- [ ] Task: Conductor - User Manual Verification 'Phase 3: Curation and Consolidation' (Protocol in workflow.md)
## Phase 4: Final Verification
- [ ] Task: Execute full test suite by category using the new manifest
- [ ] Task: Verify 100% pass rate for all non-blacklisted tests
- [ ] Task: Generate a final test coverage report
- [ ] Task: Conductor - User Manual Verification 'Phase 4: Final Verification' (Protocol in workflow.md)

View File

@@ -0,0 +1,33 @@
# Specification: Test Suite Curation and Organization
## Overview
The current test suite for **Manual Slop** and the **Conductor** framework has grown incrementally and lacks a formal organization. This track aims to curate, categorize, and organize existing tests, specifically blacklisting Conductor-specific (MMA) tests from manual_slop's test runs. We will use a central manifest for test management and perform an exhaustive review of all tests to eliminate redundancy.
## Functional Requirements
- **Test Categorization:** Tests will be categorized into:
- Manual Slop Core/GUI
- Conductor/MMA
- MCP/Integrations
- Simulation/Workflows
- **Central Manifest:** Implement a `tests.toml` (or similar) manifest file to define test categories and blacklist specific tests from the default `manual_slop` test run.
- **Blacklisting:** Ensure that Conductor-only tests (e.g., MMA related) do not execute when running tests for the `manual_slop` application itself.
- **Exhaustive Curation:** Review all existing tests in `tests/` to:
- Fix failing tests.
- Identify and merge redundant tests.
- Remove obsolete tests.
- Ensure consistent naming conventions.
## Non-Functional Requirements
- **Clarity:** The `tests.toml` manifest should be easy to understand and maintain.
- **Reliability:** The curation must result in a stable, passing test suite for each category.
## Acceptance Criteria
- A central manifest (`tests.toml`) is created and used to manage test execution.
- Running `manual_slop` tests successfully ignores all blacklisted Conductor/MMA tests.
- All failing tests are either fixed or removed (if redundant).
- Each test file is assigned to at least one category in the manifest.
- Redundant test logic is consolidated.
## Out of Scope
- Writing new feature tests (unless required to consolidate redundancy).
- Major refactoring of the test framework itself (beyond the manifest).