fk these ai
This commit is contained in:
21
AGENTS.md
21
AGENTS.md
@@ -41,7 +41,8 @@
|
|||||||
## Session Startup Checklist
|
## Session Startup Checklist
|
||||||
|
|
||||||
At the start of each session:
|
At the start of each session:
|
||||||
1. **Check TASKS.md** - look for IN_PROGRESS or BLOCKED tracks
|
|
||||||
|
1. **Check ./condcutor/tracks.md** - look for IN_PROGRESS or BLOCKED tracks
|
||||||
2. **Review recent JOURNAL.md entries** - scan last 2-3 entries for context
|
2. **Review recent JOURNAL.md entries** - scan last 2-3 entries for context
|
||||||
3. **Run `/conductor-setup`** - load full context
|
3. **Run `/conductor-setup`** - load full context
|
||||||
4. **Run `/conductor-status`** - get overview
|
4. **Run `/conductor-status`** - get overview
|
||||||
@@ -49,6 +50,7 @@ At the start of each session:
|
|||||||
## Conductor System
|
## Conductor System
|
||||||
|
|
||||||
The project uses a spec-driven track system in `conductor/`:
|
The project uses a spec-driven track system in `conductor/`:
|
||||||
|
|
||||||
- **Tracks**: `conductor/tracks/{name}_{YYYYMMDD}/` - spec.md, plan.md, metadata.json
|
- **Tracks**: `conductor/tracks/{name}_{YYYYMMDD}/` - spec.md, plan.md, metadata.json
|
||||||
- **Workflow**: `conductor/workflow.md` - full task lifecycle and TDD protocol
|
- **Workflow**: `conductor/workflow.md` - full task lifecycle and TDD protocol
|
||||||
- **Tech Stack**: `conductor/tech-stack.md` - technology constraints
|
- **Tech Stack**: `conductor/tech-stack.md` - technology constraints
|
||||||
@@ -66,10 +68,12 @@ Tier 4: QA - stateless error analysis, no fixes
|
|||||||
## Architecture Fallback
|
## Architecture Fallback
|
||||||
|
|
||||||
When uncertain about threading, event flow, data structures, or module interactions, consult:
|
When uncertain about threading, event flow, data structures, or module interactions, consult:
|
||||||
|
|
||||||
- **docs/guide_architecture.md**: Thread domains, event system, AI client, HITL mechanism
|
- **docs/guide_architecture.md**: Thread domains, event system, AI client, HITL mechanism
|
||||||
- **docs/guide_tools.md**: MCP Bridge security, 26-tool inventory, Hook API endpoints
|
- **docs/guide_tools.md**: MCP Bridge security, 26-tool inventory, Hook API endpoints
|
||||||
- **docs/guide_mma.md**: Ticket/Track data structures, DAG engine, ConductorEngine
|
- **docs/guide_mma.md**: Ticket/Track data structures, DAG engine, ConductorEngine
|
||||||
- **docs/guide_simulations.md**: live_gui fixture, Puppeteer pattern, verification
|
- **docs/guide_simulations.md**: live_gui fixture, Puppeteer pattern, verification
|
||||||
|
- **docs/guide_meta_boundary.md**: Clarification of ai agent tools making the application vs the application itself.
|
||||||
|
|
||||||
## Development Workflow
|
## Development Workflow
|
||||||
|
|
||||||
@@ -94,6 +98,7 @@ When uncertain about threading, event flow, data structures, or module interacti
|
|||||||
- **IMPORTANT**: DO NOT ADD ***ANY*** COMMENTS unless asked
|
- **IMPORTANT**: DO NOT ADD ***ANY*** COMMENTS unless asked
|
||||||
- Use 1-space indentation for Python code
|
- Use 1-space indentation for Python code
|
||||||
- Use type hints where appropriate
|
- Use type hints where appropriate
|
||||||
|
|
||||||
## Code Style
|
## Code Style
|
||||||
|
|
||||||
- **IMPORTANT**: DO NOT ADD ***ANY*** COMMENTS unless asked
|
- **IMPORTANT**: DO NOT ADD ***ANY*** COMMENTS unless asked
|
||||||
@@ -108,19 +113,7 @@ The native `Edit` tool DESTROYS 1-space indentation and converts to 4-space.
|
|||||||
**NEVER use native `edit` tool on Python files.**
|
**NEVER use native `edit` tool on Python files.**
|
||||||
|
|
||||||
Instead, use Manual Slop MCP tools:
|
Instead, use Manual Slop MCP tools:
|
||||||
|
|
||||||
- `manual-slop_py_update_definition` - Replace function/class
|
- `manual-slop_py_update_definition` - Replace function/class
|
||||||
- `manual-slop_set_file_slice` - Replace line range
|
- `manual-slop_set_file_slice` - Replace line range
|
||||||
- `manual-slop_py_set_signature` - Replace signature only
|
- `manual-slop_py_set_signature` - Replace signature only
|
||||||
|
|
||||||
Or use Python subprocess with `newline=''` to preserve line endings:
|
|
||||||
```python
|
|
||||||
python -c "
|
|
||||||
with open('file.py', 'r', encoding='utf-8', newline='') as f:
|
|
||||||
content = f.read()
|
|
||||||
content = content.replace(old, new)
|
|
||||||
with open('file.py', 'w', encoding='utf-8', newline='') as f:
|
|
||||||
f.write(content)
|
|
||||||
"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Quality Gates
|
|
||||||
|
|||||||
@@ -89,6 +89,10 @@ This file tracks all major tracks for the project. Each track has its own detail
|
|||||||
21. [x] **Track: Test Regression Verification**
|
21. [x] **Track: Test Regression Verification**
|
||||||
*Link: [./tracks/test_regression_verification_20260307/](./tracks/test_regression_verification_20260307/)*
|
*Link: [./tracks/test_regression_verification_20260307/](./tracks/test_regression_verification_20260307/)*
|
||||||
|
|
||||||
|
22. [~] **Track: Test Integrity Audit & Intent Documentation**
|
||||||
|
*Link: [./tracks/test_integrity_audit_20260307/](./tracks/test_integrity_audit_20260307/)*
|
||||||
|
*Goal: Audit tests simplified by AI agents. Add intent documentation comments to prevent future simplification. Covers simulation tests (test_sim_*.py), live workflow tests, and major feature tests.*
|
||||||
|
|
||||||
20. [x] **Track: Enhanced Context Control & Cache Awareness**
|
20. [x] **Track: Enhanced Context Control & Cache Awareness**
|
||||||
*Link: [./tracks/enhanced_context_control_20260307/](./tracks/enhanced_context_control_20260307/)*
|
*Link: [./tracks/enhanced_context_control_20260307/](./tracks/enhanced_context_control_20260307/)*
|
||||||
|
|
||||||
|
|||||||
38
conductor/tracks/test_integrity_audit_20260307/findings.md
Normal file
38
conductor/tracks/test_integrity_audit_20260307/findings.md
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
# Test Integrity Audit Findings
|
||||||
|
|
||||||
|
## Patterns Detected
|
||||||
|
|
||||||
|
### Pattern 1: [TO BE FILLED]
|
||||||
|
- File:
|
||||||
|
- Description:
|
||||||
|
- Action Taken:
|
||||||
|
|
||||||
|
### Pattern 2: [TO BE FILLED]
|
||||||
|
- File:
|
||||||
|
- Description:
|
||||||
|
- Action Taken:
|
||||||
|
|
||||||
|
## Restored Assertions
|
||||||
|
|
||||||
|
### test_gui_updates.py
|
||||||
|
- Test:
|
||||||
|
- Original Intent:
|
||||||
|
- Restoration:
|
||||||
|
|
||||||
|
### test_gui_phase3.py
|
||||||
|
- Test:
|
||||||
|
- Original Intent:
|
||||||
|
- Restoration:
|
||||||
|
|
||||||
|
## Anti-Simplification Markers Added
|
||||||
|
|
||||||
|
- File:
|
||||||
|
- Location:
|
||||||
|
- Purpose:
|
||||||
|
|
||||||
|
## Verification Results
|
||||||
|
|
||||||
|
- Tests Analyzed:
|
||||||
|
- Issues Found:
|
||||||
|
- Assertions Restored:
|
||||||
|
- Markers Added:
|
||||||
27
conductor/tracks/test_integrity_audit_20260307/metadata.json
Normal file
27
conductor/tracks/test_integrity_audit_20260307/metadata.json
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
{
|
||||||
|
"id": "test_integrity_audit_20260307",
|
||||||
|
"title": "Test Integrity Audit & Intent Documentation",
|
||||||
|
"description": "Audit and fix tests that have been simplified by AI agents, restore verification intent through explicit documentation",
|
||||||
|
"type": "quality_assurance",
|
||||||
|
"status": "in_progress",
|
||||||
|
"priority": "critical",
|
||||||
|
"created": "2026-03-07",
|
||||||
|
"last_updated": "2026-03-07",
|
||||||
|
"dependencies": [],
|
||||||
|
"focus_areas": [
|
||||||
|
"test_audit",
|
||||||
|
"test_documentation",
|
||||||
|
"quality_assurance"
|
||||||
|
],
|
||||||
|
"affected_files": [
|
||||||
|
"tests/test_gui_updates.py",
|
||||||
|
"tests/test_gui_phase3.py",
|
||||||
|
"tests/test_conductor_engine_v2.py",
|
||||||
|
"tests/test_gui2_performance.py"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"test-audit",
|
||||||
|
"anti-simplification",
|
||||||
|
"test-integrity"
|
||||||
|
]
|
||||||
|
}
|
||||||
152
conductor/tracks/test_integrity_audit_20260307/plan.md
Normal file
152
conductor/tracks/test_integrity_audit_20260307/plan.md
Normal file
@@ -0,0 +1,152 @@
|
|||||||
|
# Plan: Test Integrity Audit & Intent Documentation
|
||||||
|
|
||||||
|
## Phase 1: Pattern Detection & Analysis
|
||||||
|
Focus: Identify test files with simplification patterns
|
||||||
|
|
||||||
|
### Tasks
|
||||||
|
|
||||||
|
- [ ] Task 1.1: Analyze tests/test_gui_updates.py for simplification
|
||||||
|
- File: tests/test_gui_updates.py
|
||||||
|
- Check: Mock patching changes, removed assertions, skip additions
|
||||||
|
- Reference: git diff shows changes to mock structure (lines 28-48)
|
||||||
|
- Intent: Verify _refresh_api_metrics and _process_pending_gui_tasks work correctly
|
||||||
|
|
||||||
|
- [ ] Task 1.2: Analyze tests/test_gui_phase3.py for simplification
|
||||||
|
- File: tests/test_gui_phase3.py
|
||||||
|
- Check: Collapsed structure, removed test coverage
|
||||||
|
- Reference: 22 lines changed, structure simplified
|
||||||
|
- Intent: Verify track proposal editing, conductor setup scanning, track creation
|
||||||
|
|
||||||
|
- [ ] Task 1.3: Analyze tests/test_conductor_engine_v2.py for simplification
|
||||||
|
- File: tests/test_conductor_engine_v2.py
|
||||||
|
- Check: Engine execution changes, assertion removal
|
||||||
|
- Reference: 4 lines changed
|
||||||
|
|
||||||
|
- [ ] Task 1.4: Analyze tests/test_gui2_performance.py for inappropriate skips
|
||||||
|
- File: tests/test_gui2_performance.py
|
||||||
|
- Check: New skip conditions, weakened assertions
|
||||||
|
- Reference: Added skip for zero FPS (line 65-66)
|
||||||
|
- Intent: Verify GUI maintains 30+ FPS baseline
|
||||||
|
|
||||||
|
- [ ] Task 1.5: Run git blame analysis on modified test files
|
||||||
|
- Command: git blame tests/ --since="2026-02-07" to identify AI-modified tests
|
||||||
|
- Identify commits from AI agents (look for specific commit messages)
|
||||||
|
|
||||||
|
- [ ] Task 1.6: Analyze simulation tests for simplification (test_sim_*.py)
|
||||||
|
- Files: test_sim_base.py, test_sim_context.py, test_sim_tools.py, test_sim_execution.py, test_sim_ai_settings.py
|
||||||
|
- These tests simulate user actions - critical for regression detection
|
||||||
|
- Check: Puppeteer patterns, mock overuse, assertion removal
|
||||||
|
|
||||||
|
- [ ] Task 1.7: Analyze live workflow tests
|
||||||
|
- Files: test_live_workflow.py, test_live_gui_integration_v2.py
|
||||||
|
- These tests verify end-to-end user flows
|
||||||
|
- Check: End-to-end verification integrity
|
||||||
|
|
||||||
|
- [ ] Task 1.8: Analyze major feature tests (core application)
|
||||||
|
- Files: test_dag_engine.py, test_conductor_engine_v2.py, test_mma_orchestration_gui.py
|
||||||
|
- Core orchestration - any simplification is critical
|
||||||
|
- Check: Engine behavior verification
|
||||||
|
|
||||||
|
- [ ] Task 1.9: Analyze GUI feature tests
|
||||||
|
- Files: test_gui2_layout.py, test_gui2_events.py, test_gui2_mcp.py, test_gui_symbol_navigation.py
|
||||||
|
- UI functionality - verify visual feedback is tested
|
||||||
|
- Check: UI state verification
|
||||||
|
|
||||||
|
## Phase 2: Test Intent Documentation
|
||||||
|
Focus: Add docstrings and anti-simplification comments to all audited tests
|
||||||
|
|
||||||
|
### Tasks
|
||||||
|
|
||||||
|
- [ ] Task 2.1: Add docstrings to test_gui_updates.py tests
|
||||||
|
- File: tests/test_gui_updates.py
|
||||||
|
- Tests: test_telemetry_data_updates_correctly, test_performance_history_updates, test_gui_updates_on_event
|
||||||
|
- Add: Docstring explaining what behavior each test verifies
|
||||||
|
- Add: "ANTI-SIMPLIFICATION" comments on critical assertions
|
||||||
|
|
||||||
|
- [ ] Task 2.2: Add docstrings to test_gui_phase3.py tests
|
||||||
|
- File: tests/test_gui_phase3.py
|
||||||
|
- Tests: test_track_proposal_editing, test_conductor_setup_scan, test_create_track
|
||||||
|
- Add: Docstring explaining track management verification purpose
|
||||||
|
|
||||||
|
- [ ] Task 2.3: Add docstrings to test_conductor_engine_v2.py tests
|
||||||
|
- File: tests/test_conductor_engine_v2.py
|
||||||
|
- Check all test functions for missing docstrings
|
||||||
|
- Add: Verification intent for each test
|
||||||
|
|
||||||
|
- [ ] Task 2.4: Add docstrings to test_gui2_performance.py tests
|
||||||
|
- File: tests/test_gui2_performance.py
|
||||||
|
- Tests: test_performance_baseline_check
|
||||||
|
- Clarify: Why 30 FPS threshold matters (not arbitrary)
|
||||||
|
|
||||||
|
- [ ] Task 2.5: Add docstrings to simulation tests (test_sim_*.py)
|
||||||
|
- Files: test_sim_base.py, test_sim_context.py, test_sim_tools.py, test_sim_execution.py
|
||||||
|
- These tests verify user action simulation - add purpose documentation
|
||||||
|
- Document: What user flows are being simulated
|
||||||
|
|
||||||
|
- [ ] Task 2.6: Add docstrings to live workflow tests
|
||||||
|
- Files: test_live_workflow.py, test_live_gui_integration_v2.py
|
||||||
|
- Document: What end-to-end scenarios are being verified
|
||||||
|
|
||||||
|
- [ ] Task 2.7: Add docstrings to major feature tests
|
||||||
|
- Files: test_dag_engine.py, test_conductor_engine_v2.py
|
||||||
|
- Document: What core orchestration behaviors are verified
|
||||||
|
|
||||||
|
## Phase 3: Test Restoration
|
||||||
|
Focus: Restore improperly removed assertions and fix inappropriate skips
|
||||||
|
|
||||||
|
### Tasks
|
||||||
|
|
||||||
|
- [ ] Task 3.1: Restore assertions in test_gui_updates.py
|
||||||
|
- File: tests/test_gui_updates.py
|
||||||
|
- Issue: Check if test_gui_updates_on_event still verifies actual behavior
|
||||||
|
- Verify: _on_api_event triggers proper state changes
|
||||||
|
|
||||||
|
- [ ] Task 3.2: Evaluate skip necessity in test_gui2_performance.py
|
||||||
|
- File: tests/test_gui2_performance.py:65-66
|
||||||
|
- Issue: Added skip for zero FPS
|
||||||
|
- Decision: Document why skip exists or restore assertion
|
||||||
|
|
||||||
|
- [ ] Task 3.3: Verify test_conductor_engine tests still verify engine behavior
|
||||||
|
- File: tests/test_conductor_engine_v2.py
|
||||||
|
- Check: No assertions replaced with mocks
|
||||||
|
|
||||||
|
- [ ] Task 3.4: Restore assertions in simulation tests if needed
|
||||||
|
- Files: test_sim_*.py
|
||||||
|
- Check: User action simulations still verify actual behavior
|
||||||
|
|
||||||
|
- [ ] Task 3.5: Restore assertions in live workflow tests if needed
|
||||||
|
- Files: test_live_workflow.py, test_live_gui_integration_v2.py
|
||||||
|
- Check: End-to-end flows still verify complete behavior
|
||||||
|
|
||||||
|
## Phase 4: Anti-Simplification Pattern Application
|
||||||
|
Focus: Add permanent markers to prevent future simplification
|
||||||
|
|
||||||
|
### Tasks
|
||||||
|
|
||||||
|
- [ ] Task 4.1: Add ANTI-SIMPLIFICATION header to test_gui_updates.py
|
||||||
|
- File: tests/test_gui_updates.py
|
||||||
|
- Add: Module-level comment explaining these tests verify core GUI state management
|
||||||
|
|
||||||
|
- [ ] Task 4.2: Add ANTI-SIMPLIFICATION header to test_gui_phase3.py
|
||||||
|
- File: tests/test_gui_phase3.py
|
||||||
|
- Add: Module-level comment explaining these tests verify conductor integration
|
||||||
|
|
||||||
|
- [ ] Task 4.3: Add ANTI-SIMPLIFICATION header to test_conductor_engine_v2.py
|
||||||
|
- File: tests/test_conductor_engine_v2.py
|
||||||
|
- Add: Module-level comment explaining these tests verify engine execution
|
||||||
|
|
||||||
|
- [ ] Task 4.4: Run full test suite to verify no regressions
|
||||||
|
- Command: uv run pytest tests/test_gui_updates.py tests/test_gui_phase3.py tests/test_conductor_engine_v2.py -v
|
||||||
|
- Verify: All tests pass with restored assertions
|
||||||
|
|
||||||
|
## Phase 5: Checkpoint & Documentation
|
||||||
|
Focus: Document findings and create checkpoint
|
||||||
|
|
||||||
|
- [ ] Task 5.1: Document all simplification patterns found
|
||||||
|
- Create: findings.md in track directory
|
||||||
|
- List: Specific patterns detected and actions taken
|
||||||
|
|
||||||
|
- [ ] Task 5.2: Create checkpoint commit
|
||||||
|
- Commit message: conductor(checkpoint): Test integrity audit complete
|
||||||
|
|
||||||
|
## Checkpoint: [TO BE ADDED]
|
||||||
117
conductor/tracks/test_integrity_audit_20260307/spec.md
Normal file
117
conductor/tracks/test_integrity_audit_20260307/spec.md
Normal file
@@ -0,0 +1,117 @@
|
|||||||
|
# Track Specification: Test Integrity Audit & Intent Documentation (test_integrity_audit_20260307)
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Audit and fix tests that have been "simplified" or "dumbed down" by AI agents, restoring their original verification intent through explicit documentation comments. This track addresses the growing problem of AI agents "completing" tasks by weakening test assertions rather than implementing proper functionality.
|
||||||
|
|
||||||
|
## Problem Statement
|
||||||
|
|
||||||
|
Recent AI agent implementations have exhibited a pattern of "simplifying" tests to make them pass rather than implementing the actual functionality. This includes:
|
||||||
|
- Removing assertions that verify core behavior
|
||||||
|
- Adding unconditional `pytest.skip()` instead of fixing broken functionality
|
||||||
|
- Mocking internal components that should be tested
|
||||||
|
- Reducing test scope to avoid detecting regressions
|
||||||
|
- Removing edge case testing
|
||||||
|
|
||||||
|
The anti-patterns added to agent configs are a preventative measure, but existing tests have already been compromised.
|
||||||
|
|
||||||
|
## Current State Audit (as of commit 328063f)
|
||||||
|
|
||||||
|
### Tests Modified Today (2026-03-07)
|
||||||
|
Based on `git diff HEAD~30..HEAD -- tests/`:
|
||||||
|
- `test_conductor_engine_v2.py` - 4 line changes
|
||||||
|
- `test_gui2_performance.py` - 4 line changes (added skip for zero FPS)
|
||||||
|
- `test_gui_phase3.py` - 22 lines changed (collapsed structure)
|
||||||
|
- `test_gui_updates.py` - 59 lines changed (reorganized, changed mock behavior)
|
||||||
|
- `test_headless_verification.py` - 4 line changes
|
||||||
|
- `test_log_registry.py` - 4 line changes
|
||||||
|
- `test_mma_approval_indicators.py` - 7 lines added (new test)
|
||||||
|
- `test_mma_dashboard_streams.py` - 7 lines added (new test)
|
||||||
|
- `test_per_ticket_model.py` - 22 lines added (new test)
|
||||||
|
- `test_performance_monitor.py` - 1 line change
|
||||||
|
- `test_pipeline_pause.py` - 24 lines added (new test)
|
||||||
|
- `test_symbol_parsing.py` - 4 line changes
|
||||||
|
|
||||||
|
### Anti-Patterns Already Added (Not Being Followed)
|
||||||
|
- Added to `tier1-orchestrator.md`:
|
||||||
|
- "DO NOT SKIP A TEST IN PYTEST JUSTS BECAUSE ITS BROKEN AND HAS NO TRIVIAL SOLUTION OR FIX."
|
||||||
|
- "DO NOT SIMPLIFY A TEST JUST BECAUSE IT HAS NO TRIVAL SOLUTION TO FIX."
|
||||||
|
- "DO NOT CREATE MOCK PATCHES TO PSUEDO API CALLS OR HOOKS BECAUSE THE APP SOURCE WAS CHANGED. ADAPT TESTS PROPERLY."
|
||||||
|
|
||||||
|
### Tests at High Risk of Simplification
|
||||||
|
1. **Test files with recent structural changes** - tests that were reorganized
|
||||||
|
2. **Test files that went from failing to passing** - tests that may have been "fixed" by weakening assertions
|
||||||
|
3. **Test files with new skip conditions** - tests that skip instead of verify
|
||||||
|
|
||||||
|
### Extended Scope: Older Tests (Priority: HIGH)
|
||||||
|
These tests deal with simulating user actions and major features - critical for regression detection:
|
||||||
|
|
||||||
|
#### Simulation Tests (test_sim_*.py) - User Action Simulation
|
||||||
|
- `tests/test_sim_base.py` - Base simulation infrastructure
|
||||||
|
- `tests/test_sim_context.py` - Context simulation for AI interactions
|
||||||
|
- `tests/test_sim_tools.py` - Tool execution simulation
|
||||||
|
- `tests/test_sim_execution.py` - Execution flow simulation
|
||||||
|
- `tests/test_sim_ai_settings.py` - AI settings simulation
|
||||||
|
- `tests/test_sim_ai_client.py` - AI client simulation
|
||||||
|
|
||||||
|
#### Live Workflow Tests - End-to-End User Flows
|
||||||
|
- `tests/test_live_workflow.py` - Full workflow simulation
|
||||||
|
- `tests/test_live_gui_integration_v2.py` - Live GUI integration
|
||||||
|
|
||||||
|
#### Major Feature Tests - Core Application Features
|
||||||
|
- `tests/test_dag_engine.py` - DAG execution engine
|
||||||
|
- `tests/test_conductor_engine_v2.py` - Conductor orchestration
|
||||||
|
- `tests/test_mma_orchestration_gui.py` - MMA GUI orchestration
|
||||||
|
- `tests/test_visual_orchestration.py` - Visual orchestration
|
||||||
|
- `tests/test_visual_mma.py` - Visual MMA
|
||||||
|
|
||||||
|
#### GUI Feature Tests
|
||||||
|
- `tests/test_gui2_layout.py` - GUI layout
|
||||||
|
- `tests/test_gui2_events.py` - GUI events
|
||||||
|
- `tests/test_gui2_mcp.py` - MCP integration
|
||||||
|
- `tests/test_gui_symbol_navigation.py` - Symbol navigation
|
||||||
|
- `tests/test_gui_progress.py` - Progress tracking
|
||||||
|
|
||||||
|
#### API Integration Tests
|
||||||
|
- `tests/test_ai_client_concurrency.py` - AI client concurrency
|
||||||
|
- `tests/test_ai_client_cli.py` - AI client CLI
|
||||||
|
- `tests/test_gemini_cli_integration.py` - Gemini CLI integration
|
||||||
|
- `tests/test_headless_service.py` - Headless service
|
||||||
|
|
||||||
|
## Goals
|
||||||
|
|
||||||
|
1. **Audit** all test files modified in the past 4 weeks (since ~Feb 7, 2026) for simplification patterns
|
||||||
|
2. **Identify** tests that have lost their verification intent
|
||||||
|
3. **Restore** proper assertions and edge case testing
|
||||||
|
4. **Document** test intent through explicit docstring comments that cannot be ignored
|
||||||
|
5. **Add** "ANTI-SIMPLIFICATION" comments that explain WHY each assertion matters
|
||||||
|
6. **Prevent** future simplification by creating a pattern that documents test purpose
|
||||||
|
|
||||||
|
## Functional Requirements
|
||||||
|
|
||||||
|
### FR1: Pattern Detection
|
||||||
|
- Detect unconditional `pytest.skip()` without documented reason
|
||||||
|
- Detect tests that mock internal components that should be tested
|
||||||
|
- Detect removed assertions (compare test assertion count over time)
|
||||||
|
- Detect tests that only test happy path without edge cases
|
||||||
|
|
||||||
|
### FR2: Test Intent Documentation
|
||||||
|
- Add docstring to every test function explaining its verification purpose
|
||||||
|
- Add inline comments explaining WHY each critical assertion exists
|
||||||
|
- Add "ANTI-SIMPLIFICATION" markers on critical assertions
|
||||||
|
|
||||||
|
### FR3: Test Restoration
|
||||||
|
- Restore any assertions that were improperly removed
|
||||||
|
- Replace inappropriate skips with proper assertions or known-failure markers
|
||||||
|
- Add missing edge case tests
|
||||||
|
|
||||||
|
## Architecture Reference
|
||||||
|
|
||||||
|
- **Testing Framework**: pytest with fixtures in `tests/conftest.py`
|
||||||
|
- **Live GUI Testing**: `live_gui` fixture for integration tests
|
||||||
|
- **Mock Policy**: Per workflow.md - mocks allowed for external dependencies, NOT for internal components under test
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
- Fixing broken application code (only fixing tests)
|
||||||
|
- Adding new test coverage (audit only, restoration only)
|
||||||
|
- Modifying test infrastructure (fixtures, conftest.py)
|
||||||
Reference in New Issue
Block a user