Test Workspace Paths — Hard Rule

TL;DR

Test workspaces live in the project tree under tests/artifacts/. Conftest creates them. No env vars. No CLI args. No tmp_path_factory. No %TEMP%. No runner changes. The user must be able to find every test workspace by looking in tests/artifacts/.

The Rule

When creating a test workspace, fixture, or scratch directory for any test infrastructure:

# CORRECT — conftest creates the path
from datetime import datetime
_RUN_ID = datetime.now().strftime("%Y%m%d_%H%M%S")
_RUN_WORKSPACE = Path(f"tests/artifacts/live_gui_workspace_{_RUN_ID}")

@pytest.fixture(scope="session")
def live_gui(request):
    temp_workspace = _RUN_WORKSPACE
    ...

# WRONG — env vars
import os
WORKSPACE = os.environ.get("LIVE_GUI_WORKSPACE", "tests/artifacts/live_gui_workspace")

# WRONG — CLI args
def pytest_addoption(parser):
    parser.addoption("--workspace", action="store", default="tests/artifacts/live_gui_workspace")

# WRONG — tmp_path_factory (lives in %TEMP%, not in project tree)
def live_gui(request, tmp_path_factory):
    temp_workspace = tmp_path_factory.mktemp("live_gui_workspace")
    # Creates: C:\Users\<user>\AppData\Local\Temp\pytest-of-<user>\pytest-N\live_gui_workspace0
    # User CANNOT FIND THIS from the project tree.

Why This Rule Exists

This rule was added 2026-06-09 after a 4-day agent churn on workspace paths. The chain of decisions:

Original conftest: temp_workspace = Path("tests/artifacts/live_gui_workspace"). Sims worked. User could find the workspace. This was correct.
Phase 3 of test_infrastructure_hardening_20260609: agent changed it to tmp_path_factory.mktemp("live_gui_workspace"). The user did not catch this for 2 days. It moved the workspace to %TEMP%/pytest-of-<user>/... which:
- The user cannot find from the project tree
- The sims (which compute os.path.abspath("tests/artifacts/...") from the project root) could not find the workspace either
- Caused test_extended_sims.py::test_context_sim_live to fail with "stale ui - ops disabled" because the sim's project path didn't match the controller's active_project_path
- The agent then spent 2 more days trying to fix the sim timing, the MMA state, the RAG state, the watchdog — none of which were the actual cause
The user caught the regression. Their feedback: "we should be using a folder in ./tests/" — i.e., the project tree, not the system temp dir.
The agent tried Path("tests/artifacts/live_gui_workspace") (no timestamp). That solved the sim issue but was per-session, not per-run. Per-test pollution is desirable (it exposes fragility), so per-run isolation is what we want.
The user pushed back on adding CLI args: "have conftest make it, conftest is the right place." The agent then tried env vars as an indirection layer.
The user rejected env vars: "env vars are hidden global state, pass it to conftest directly." Conftest is the source of truth.
Final solution: conftest creates a per-run timestamped folder under tests/artifacts/. One source of truth. No indirection. The user must be able to find every test workspace by looking in tests/artifacts/.

Forbidden Patterns (Hard Bans)

1. `tmp_path_factory` for test infrastructure workspaces

tmp_path_factory is for pytest's own test isolation (e.g., when a unit test needs a temp dir to write a file). It is NOT for test infrastructure workspaces (e.g., the live_gui subprocess's CWD).

2. Environment variables for test paths

Env vars are hidden global state. The user has explicitly banned them.

3. CLI args for test paths

The conftest is the right place. CLI args add a layer of indirection between the runner and the test, and they require the runner to be modified to pass them.

3.7 KiB Raw Permalink Blame History