docs(simulations): add workspace isolation, mock_app/app_instance fixtures, and Test Areas by Subsystem table
This commit is contained in:
@@ -83,6 +83,40 @@ def reset_ai_client() -> Generator[None, None, None]:
|
||||
|
||||
Runs automatically before every test. Resets the `ai_client` module state and defaults to a safe model, preventing state pollution between tests.
|
||||
|
||||
### Workspace Isolation (autouse)
|
||||
|
||||
```python
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolate_workspace(tmp_path_factory, monkeypatch) -> Generator[None, None, None]:
|
||||
# Redirects the path resolution layer to a temp directory
|
||||
# Prevents tests from writing to the user's actual project
|
||||
...
|
||||
```
|
||||
|
||||
This autouse fixture ensures every test runs against an isolated `tmp_path` workspace. It `monkeypatch`-es `src.paths` so that any code path resolving a project directory (e.g., `manual_slop.toml` lookup, conductor directory resolution, log directory) is redirected to a fresh temp directory per test. Without this, tests could mutate the user's actual `manual_slop.toml` or conductor tracks directory.
|
||||
|
||||
This is the primary mechanism for satisfying the **Artifact Isolation** rule in the Structural Testing Contract.
|
||||
|
||||
### Path Reset (autouse)
|
||||
|
||||
```python
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_paths() -> Generator[None, None, None]:
|
||||
# Forces `src/paths.py` to re-resolve from environment / config on next access
|
||||
...
|
||||
```
|
||||
|
||||
Pairs with `isolate_workspace` to fully reset the path subsystem. After a test that creates a project config, the next test gets a clean slate.
|
||||
|
||||
### `mock_app` and `app_instance` Fixtures
|
||||
|
||||
For unit tests that need a partially-mocked `App` (without the full live_gui launch), two additional fixtures are available:
|
||||
|
||||
- **`mock_app`** — Returns an `App` instance with key subsystems (event queue, comms log, file cache) mocked. Use for testing GUI logic in isolation.
|
||||
- **`app_instance`** — Returns a real `App` instance with disk-backed state, but without launching the render loop. Use for testing the full controller logic.
|
||||
|
||||
These are scoped per-test (not session) and run faster than `live_gui` for unit-level testing.
|
||||
|
||||
### Process Cleanup
|
||||
|
||||
```python
|
||||
@@ -299,6 +333,42 @@ assert perf['input_lag_ms'] < 100
|
||||
|
||||
---
|
||||
|
||||
## Test Areas by Subsystem
|
||||
|
||||
Beyond the Puppeteer pattern, the test suite covers distinct subsystems with their own fixtures and assertions. The key areas:
|
||||
|
||||
| Area | Key test files | Approach |
|
||||
|---|---|---|
|
||||
| **MMA orchestration** | `test_conductor_engine_v2.py`, `test_conductor_engine_abort.py`, `test_conductor_abort_event.py`, `test_orchestration_logic.py`, `test_parallel_execution.py` | Unit tests of `ConductorEngine`, `WorkerPool`, `run_worker_lifecycle`. Use mock providers and direct conductor invocation. |
|
||||
| **MMA dashboard** | `test_mma_dashboard_refresh.py`, `test_mma_dashboard_streams.py`, `test_mma_node_editor.py`, `test_mma_orchestration_gui.py`, `test_mma_concurrent_tracks_sim.py` | GUI-level tests with `live_gui` + `ApiHookClient`. Verify dashboard refresh, stream telemetry, node editor interaction. |
|
||||
| **Discussion** | `test_discussion_takes.py`, `test_discussion_takes_gui.py`, `test_discussion_metrics.py`, `test_discussion_compression.py`, `test_gui_discussion_tabs.py` | Take branching, per-response token metrics, history compression. |
|
||||
| **Context composition** | `test_context_composition_*.py` (Phase 1-6), `test_context_preview_button.py`, `test_view_presets.py`, `test_custom_slices_annotations.py` | Decoupled context panel, view presets, custom slices, preview button. |
|
||||
| **RAG** | `test_rag_engine.py`, `test_rag_integration.py`, `test_rag_gui_presence.py`, `test_rag_phase4_stress.py`, `test_rag_visual_sim.py` | Vector store lifecycle, integration with `ai_client.send`, GUI presence, stress testing. |
|
||||
| **Beads** | `test_beads_client.py`, `test_mcp_client_beads.py`, `test_gui_dag_beads.py` | BeadsClient CRUD, MCP tool registration, DAG visualization with Beads-backed tickets. |
|
||||
| **External MCP** | `test_external_mcp.py`, `test_external_mcp_e2e.py`, `test_external_mcp_hitl.py`, `test_mcp_config.py` | Server lifecycle, end-to-end with real processes, HITL approval flow. |
|
||||
| **Hot reload** | `test_hot_reloader.py`, `test_hot_reload_integration.py` | Module invalidation, state preservation, integration with rendering. |
|
||||
| **C/C++ AST** | `test_ts_c_tools.py`, `test_ts_cpp_tools.py`, `test_mcp_ts_integration.py` | Tree-sitter AST tools dispatch, definitions, signatures, updates. |
|
||||
| **Personas & tool bias** | `test_persona_manager.py`, `test_persona_models.py`, `test_tool_bias.py`, `test_bias_efficacy.py`, `test_bias_integration.py` | Persona CRUD, bias engine, prompt generation effects. |
|
||||
| **Provider-specific** | `test_deepseek_provider.py`, `test_minimax_provider.py`, `test_gemini_cli_adapter.py`, `test_gemini_cli_adapter_parity.py`, `test_gemini_metrics.py` | Per-provider behavior, parity checks, Gemini cache metrics. |
|
||||
| **Workspace profiles** | `test_workspace_manager.py`, `test_workspace_profiles_sim.py` | Profile save/load, scope inheritance, auto-switch (when integrated). |
|
||||
| **History (undo/redo)** | `test_history.py`, `test_history_manager.py`, `test_history_management.py`, `test_undo_redo_sim.py` | Non-provider undo/redo, snapshot jumping. |
|
||||
|
||||
**Convention:** Subsystem-specific test files are named `test_<subsystem>_<aspect>.py`. Integration tests with `live_gui` end in `_sim.py` or `_integration.py`. End-to-end tests with real processes end in `_e2e.py`.
|
||||
|
||||
---
|
||||
|
||||
## Headless Service Tests
|
||||
|
||||
The application also runs in headless mode (without GUI) as a decoupled FastAPI/Uvicorn service. These tests verify the headless path:
|
||||
|
||||
- `test_headless_service.py` — Basic service lifecycle, route registration.
|
||||
- `test_headless_simulation.py` — End-to-end MMA simulation via the headless service (no GUI launch).
|
||||
- `test_headless_verification.py` — Full run with error + QA interceptor verification.
|
||||
|
||||
The headless service uses the **Remote Confirmation Protocol** for HITL: when an AI action requires approval, the service blocks on an HTTP endpoint and waits for an external orchestrator (typically a CLI script) to POST a decision. The protocol is documented in [guide_tools.md](guide_tools.md#the-apiask-protocol-synchronous-hitl-via-http).
|
||||
|
||||
---
|
||||
|
||||
## Supporting Analysis Modules
|
||||
|
||||
### `file_cache.py` — ASTParser (tree-sitter)
|
||||
|
||||
Reference in New Issue
Block a user