From 959e7f30a3b657dfcf8890796e989b080d64e899 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Tue, 2 Jun 2026 19:18:36 -0400 Subject: [PATCH] docs(simulations): add workspace isolation, mock_app/app_instance fixtures, and Test Areas by Subsystem table --- docs/guide_simulations.md | 70 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/docs/guide_simulations.md b/docs/guide_simulations.md index b2602f0d..d79bc41c 100644 --- a/docs/guide_simulations.md +++ b/docs/guide_simulations.md @@ -83,6 +83,40 @@ def reset_ai_client() -> Generator[None, None, None]: Runs automatically before every test. Resets the `ai_client` module state and defaults to a safe model, preventing state pollution between tests. +### Workspace Isolation (autouse) + +```python +@pytest.fixture(autouse=True) +def isolate_workspace(tmp_path_factory, monkeypatch) -> Generator[None, None, None]: + # Redirects the path resolution layer to a temp directory + # Prevents tests from writing to the user's actual project + ... +``` + +This autouse fixture ensures every test runs against an isolated `tmp_path` workspace. It `monkeypatch`-es `src.paths` so that any code path resolving a project directory (e.g., `manual_slop.toml` lookup, conductor directory resolution, log directory) is redirected to a fresh temp directory per test. Without this, tests could mutate the user's actual `manual_slop.toml` or conductor tracks directory. + +This is the primary mechanism for satisfying the **Artifact Isolation** rule in the Structural Testing Contract. + +### Path Reset (autouse) + +```python +@pytest.fixture(autouse=True) +def reset_paths() -> Generator[None, None, None]: + # Forces `src/paths.py` to re-resolve from environment / config on next access + ... +``` + +Pairs with `isolate_workspace` to fully reset the path subsystem. After a test that creates a project config, the next test gets a clean slate. + +### `mock_app` and `app_instance` Fixtures + +For unit tests that need a partially-mocked `App` (without the full live_gui launch), two additional fixtures are available: + +- **`mock_app`** — Returns an `App` instance with key subsystems (event queue, comms log, file cache) mocked. Use for testing GUI logic in isolation. +- **`app_instance`** — Returns a real `App` instance with disk-backed state, but without launching the render loop. Use for testing the full controller logic. + +These are scoped per-test (not session) and run faster than `live_gui` for unit-level testing. + ### Process Cleanup ```python @@ -299,6 +333,42 @@ assert perf['input_lag_ms'] < 100 --- +## Test Areas by Subsystem + +Beyond the Puppeteer pattern, the test suite covers distinct subsystems with their own fixtures and assertions. The key areas: + +| Area | Key test files | Approach | +|---|---|---| +| **MMA orchestration** | `test_conductor_engine_v2.py`, `test_conductor_engine_abort.py`, `test_conductor_abort_event.py`, `test_orchestration_logic.py`, `test_parallel_execution.py` | Unit tests of `ConductorEngine`, `WorkerPool`, `run_worker_lifecycle`. Use mock providers and direct conductor invocation. | +| **MMA dashboard** | `test_mma_dashboard_refresh.py`, `test_mma_dashboard_streams.py`, `test_mma_node_editor.py`, `test_mma_orchestration_gui.py`, `test_mma_concurrent_tracks_sim.py` | GUI-level tests with `live_gui` + `ApiHookClient`. Verify dashboard refresh, stream telemetry, node editor interaction. | +| **Discussion** | `test_discussion_takes.py`, `test_discussion_takes_gui.py`, `test_discussion_metrics.py`, `test_discussion_compression.py`, `test_gui_discussion_tabs.py` | Take branching, per-response token metrics, history compression. | +| **Context composition** | `test_context_composition_*.py` (Phase 1-6), `test_context_preview_button.py`, `test_view_presets.py`, `test_custom_slices_annotations.py` | Decoupled context panel, view presets, custom slices, preview button. | +| **RAG** | `test_rag_engine.py`, `test_rag_integration.py`, `test_rag_gui_presence.py`, `test_rag_phase4_stress.py`, `test_rag_visual_sim.py` | Vector store lifecycle, integration with `ai_client.send`, GUI presence, stress testing. | +| **Beads** | `test_beads_client.py`, `test_mcp_client_beads.py`, `test_gui_dag_beads.py` | BeadsClient CRUD, MCP tool registration, DAG visualization with Beads-backed tickets. | +| **External MCP** | `test_external_mcp.py`, `test_external_mcp_e2e.py`, `test_external_mcp_hitl.py`, `test_mcp_config.py` | Server lifecycle, end-to-end with real processes, HITL approval flow. | +| **Hot reload** | `test_hot_reloader.py`, `test_hot_reload_integration.py` | Module invalidation, state preservation, integration with rendering. | +| **C/C++ AST** | `test_ts_c_tools.py`, `test_ts_cpp_tools.py`, `test_mcp_ts_integration.py` | Tree-sitter AST tools dispatch, definitions, signatures, updates. | +| **Personas & tool bias** | `test_persona_manager.py`, `test_persona_models.py`, `test_tool_bias.py`, `test_bias_efficacy.py`, `test_bias_integration.py` | Persona CRUD, bias engine, prompt generation effects. | +| **Provider-specific** | `test_deepseek_provider.py`, `test_minimax_provider.py`, `test_gemini_cli_adapter.py`, `test_gemini_cli_adapter_parity.py`, `test_gemini_metrics.py` | Per-provider behavior, parity checks, Gemini cache metrics. | +| **Workspace profiles** | `test_workspace_manager.py`, `test_workspace_profiles_sim.py` | Profile save/load, scope inheritance, auto-switch (when integrated). | +| **History (undo/redo)** | `test_history.py`, `test_history_manager.py`, `test_history_management.py`, `test_undo_redo_sim.py` | Non-provider undo/redo, snapshot jumping. | + +**Convention:** Subsystem-specific test files are named `test__.py`. Integration tests with `live_gui` end in `_sim.py` or `_integration.py`. End-to-end tests with real processes end in `_e2e.py`. + +--- + +## Headless Service Tests + +The application also runs in headless mode (without GUI) as a decoupled FastAPI/Uvicorn service. These tests verify the headless path: + +- `test_headless_service.py` — Basic service lifecycle, route registration. +- `test_headless_simulation.py` — End-to-end MMA simulation via the headless service (no GUI launch). +- `test_headless_verification.py` — Full run with error + QA interceptor verification. + +The headless service uses the **Remote Confirmation Protocol** for HITL: when an AI action requires approval, the service blocks on an HTTP endpoint and waits for an external orchestrator (typically a CLI script) to POST a decision. The protocol is documented in [guide_tools.md](guide_tools.md#the-apiask-protocol-synchronous-hitl-via-http). + +--- + ## Supporting Analysis Modules ### `file_cache.py` — ASTParser (tree-sitter)