manual_slop/conductor/code_styleguides/test_sandbox.md

Test Sandbox Hardening — Hard Rule

TL;DR

The Manual Slop test suite runs under a 4-layer sandbox that prevents any pytest invocation from writing files outside ./tests/. The root-cause fix removes the historical SLOP_CONFIG env-var fallback in favor of an explicit --config CLI flag. Any test that needs a config file must point at one inside ./tests/artifacts/.

The 4-Layer Model

Layer	Mechanism	Where	Default-on?
Layer 1	Python runtime file-I/O guard (sys.addaudithook)	tests/conftest.py:_sandbox_audit_hook	Yes
Layer 2	isolate_workspace autouse + pyproject.toml --basetemp	tests/conftest.py + pyproject.toml	Yes
Layer 3	OS-level restricted-token PowerShell wrapper	scripts/run_tests_sandboxed.ps1	Opt-in
Layer 4	Static audit script (CI gate)	scripts/audit_test_sandbox_violations.py	Yes (informational) / opt-in (--strict)

Layer 1 + Layer 2 + Layer 4 are file-presence-on = enabled (delete the relevant file to disable). Layer 3 requires explicit invocation.

The --config CLI Flag (replaces SLOP_CONFIG)

The historical SLOP_CONFIG env var has been removed from src/paths.py. The CLI flag --config <path> is now the ONLY supported mechanism for overriding the default <project_root>/config.toml location.

sloppy.py

# Use the default <project_root>/config.toml
uv run python sloppy.py

# Override
uv run python sloppy.py --config /path/to/your/config.toml

sloppy.py calls paths.set_config_override(Path(args.config).resolve()) AFTER parse_args() and BEFORE any from src.gui_2 import App import. This is the only way to override the config path in production.

tests/conftest.py

tests/conftest.py parses sys.argv for --config at MODULE BODY (BEFORE any src/ import). If --config is not passed, conftest auto-defaults to tests/artifacts/_isolation_workspace_<RUN_ID>/config_overrides.toml (which lives inside ./tests/artifacts/, so the Layer 1 guard allows writes to it).

# Module body in tests/conftest.py (BEFORE any src/ import)
def _parse_config_arg(argv: list[str]) -> Path | None:
    for i in range(1, len(argv)):
        arg = argv[i]
        if arg == "--config" and i + 1 < len(argv):
            return Path(argv[i + 1]).resolve()
        if arg.startswith("--config="):
            return Path(arg.split("=", 1)[1]).resolve()
    return None

_config_override_arg = _parse_config_arg(sys.argv)
if _config_override_arg is None:
    _config_override_arg = _ISOLATION_WORKSPACE / "config_overrides.toml"

from src import paths as _paths  # noqa: E402
_paths.set_config_override(_config_override_arg)

The fixture also auto-generates a placeholder config_overrides.toml (with ai.provider, projects, gui.show_windows) so src/ code that reads the config at startup does not crash.

The --basetemp Rule

pyproject.toml sets addopts = "--basetemp=tests/artifacts/_pytest_tmp". This redirects pytest's tmp_path and tmp_path_factory fixtures (which default to %TEMP%\pytest-of-<user>\ on Windows) into ./tests/artifacts/. This is what allows the Layer 1 allowlist to be a single rule: "anything under ./tests/ is allowed."

Layer 1 Audit Hook Contract

tests/conftest.py:_sandbox_audit_hook is a sys.addaudithook callback. It fires on every open() call. Behavior:

• Reads (mode r, rb): pass through, no check
• Writes (mode contains w, a, x, +): check path
• Allowed if path resolves under:
  • <project_root>/tests/
  • Path contains .pytest_cache, __pycache__, .coverage, .slop_cache, or .ruff_cache
  • Original path string starts with \\.\ (Windows device namespace) or /dev/ (Unix device namespace)
• Blocked otherwise: raises RuntimeError("TEST_SANDBOX_VIOLATION: attempted to write to <path>...")

How to fix a violation:

• Move the write under <project_root>/tests/ (use tmp_path, tests/artifacts/_<name>/, etc.)
• For pytest internal files (cache, log): check if the path is in the allowlist; if not, open an issue to add it

Layer 2 Workspace Convention (config_overrides.toml)

Tests that need a config.toml should use the auto-generated tests/artifacts/_isolation_workspace_<RUN_ID>/config_overrides.toml. The naming convention config_overrides.toml (instead of config.toml) signals that this file is an override for tests, not the production config.

Tests CAN pass --config /some/other/path.toml explicitly; conftest will honor it. But the default is fine for most cases.

Layer 3 Opt-in OS-Level Wrapper

scripts/run_tests_sandboxed.ps1 is the Windows-only restricted-token + Job Object wrapper for paranoid users. It mirrors scripts/tier2/run_tier2_sandboxed.ps1:

# Dry-run (no actual sandbox; just prints what would happen)
pwsh -File scripts/run_tests_sandboxed.ps1 -WhatIf

# Run the full suite in the sandbox
pwsh -File scripts/run_tests_sandboxed.ps1

# Run a specific test path
pwsh -File scripts/run_tests_sandboxed.ps1 -TestPath tests/test_paths.py

# Override config explicitly
pwsh -File scripts/run_tests_sandboxed.ps1 -ConfigPath /some/path/config.toml

The wrapper:

1. Acquires a restricted token via .NET DuplicateTokenEx
2. Sets cwd to <project_root>
3. Invokes uv run python -m pytest $TestPath --basetemp=tests/artifacts/_pytest_tmp [--config=...]
4. Forwards pytest exit code

Layer 4 Static Audit

scripts/audit_test_sandbox_violations.py scans tests/test_*.py for hardcoded paths that would corrupt user files:

• Path("manual_slop.toml"), Path("config.toml"), Path("credentials.toml"), Path("presets.toml"), etc.
• open("manual_slop.toml", "w") and similar write-mode calls
• Path("C:/projects/...") and Path("C:\\projects\\...")
• Path("tests/artifacts/...") literal (violates workspace_paths.md; should use a fixture)
• tempfile.mkdtemp(), tempfile.mkstemp() (without dir=)

Default mode (informational) exits 0 and lists violations. --strict mode (CI gate) exits 1 on any violation.

# Informational
uv run python scripts/audit_test_sandbox_violations.py

# CI gate
uv run python scripts/audit_test_sandbox_violations.py --strict

Why This Rule Exists

The user has lost "important sample data" multiple times over the past month because tests have written to manual_slop.toml, manual_slop_history.toml, personas.toml, presets.toml, tool_presets.toml, or credentials.toml at the top of the repo. The root cause was the silent SLOP_CONFIG env-var fallback in src/paths.py — any test could set the env var and have paths.get_config_path() return a project-root file.

This track fixes that and adds defense in depth.

Forbidden Patterns (Hard Bans)

1. SLOP_CONFIG env var

Setting SLOP_CONFIG no longer affects paths.get_config_path(). Use --config instead.

2. tempfile.mkdtemp() / tempfile.mkstemp() without dir=

These default to %TEMP%, which the Layer 1 guard blocks. Use:

• tempfile.mkdtemp(dir="tests/artifacts/") (explicit under tests)
• tmp_path pytest fixture (resolves under --basetemp)
• tmp_path_factory.mktemp("name") (same)

3. Writing to <project_root>/*.toml or <project_root>/*.ini

The Layer 1 guard raises TEST_SANDBOX_VIOLATION on any write to a top-level TOML/INI file. Move the file under tests/artifacts/.

4. Path(__file__).parent.parent / "config.toml"

This pattern is a .. traversal to the project root. Flagged by Layer 4 static audit.

Audit Enforcement

• Layer 4 runs as a pre-commit hook + CI gate (--strict mode)
• Layer 1 fires at pytest runtime; cannot be bypassed without deleting tests/conftest.py:_sandbox_audit_hook
• Layer 2 is enforced by pyproject.toml addopts; cannot be overridden per-invocation

See Also

• conductor/code_styleguides/workspace_paths.md — the existing test-workspace rule (extended by this track)
• conductor/code_styleguides/feature_flags.md — file-presence = enabled convention
• conductor/tech-stack.md §"pyproject.toml pytest addopts" — dated note explaining --basetemp
• scripts/audit_no_temp_writes.py — pattern reference for Layer 4 audit
• scripts/tier2/run_tier2_sandboxed.ps1 — pattern reference for Layer 3 wrapper
• conductor/tracks/test_sandbox_hardening_20260619/ — this track's spec + plan + state
• conductor/tracks/workspace_path_finalize_20260609/ — prior track that established tests/artifacts/ workspace pattern