diff --git a/docs/reports/TRACK_COMPLETION_test_sandbox_hardening_20260619.md b/docs/reports/TRACK_COMPLETION_test_sandbox_hardening_20260619.md index cb863425..de1ba828 100644 --- a/docs/reports/TRACK_COMPLETION_test_sandbox_hardening_20260619.md +++ b/docs/reports/TRACK_COMPLETION_test_sandbox_hardening_20260619.md @@ -5,8 +5,12 @@ **Owner:** Tier 2 Tech Lead (autonomous sandbox mode) **Trigger:** User has lost "important sample data" multiple times because tests have silently written to `manual_slop.toml`, `manual_slop_history.toml`, `personas.toml`, `presets.toml`, `tool_presets.toml`, or `credentials.toml` at the top of the repo. **Branch:** `tier2/test_sandbox_hardening_20260619` (from `origin/master`) -**Commits:** 11 atomic commits -**Tests:** 20 default-on (all pass) + 1 Windows-only opt-in (passes) +**Commits:** 13 atomic commits +**Tests:** 22 default-on (all pass) + 1 Windows-only opt-in (passes) + +## v2 design note (post-completion user feedback) + +After the initial track completion, the user pointed out that the v1 design used `SLOP_*` env vars set in `conftest.py` to redirect paths — opaque, not self-documenting, and not inspectable in the config TOML. **The user is right.** The v2 design (commit `3a86ca37` + regression tests in `561090c0`) replaces the env-var approach with **explicit `[paths]` overrides inside the auto-generated `config_overrides.toml`**. The user can now open the file and see exactly where each path is routed. ## Summary @@ -26,6 +30,43 @@ The root cause was the silent `SLOP_CONFIG` env-var fallback in `src/paths.py:ge - `tests/conftest.py`: `live_gui` fixture now passes `--config=` as a CLI arg to the sloppy.py subprocess (replaces the `SLOP_CONFIG` env var). - `tests/test_test_sandbox.py`: 3 regression tests (`test_config_override_via_cli_flag`, `test_paths_get_config_path_no_env_fallback`, `test_sloppy_py_parses_config_flag`). +### Fix 1b (v2): Route ALL path getters through config.toml [paths] overrides + +**Bug (post-completion user feedback):** v1 design used `SLOP_GLOBAL_PRESETS` etc. env vars set by `conftest.py:isolate_workspace` to redirect paths. This was opaque — the user couldn't inspect the routing by opening a config file. "none of the paths were properly overwritten to fucking route to ./tests from a toml file" (user verbatim). + +**Fix:** +- `src/paths.py`: refactored every path getter (`get_global_presets_path`, `get_global_tool_presets_path`, `get_global_personas_path`, `get_global_themes_path`, `get_global_workspace_profiles_path`, `get_credentials_path`, plus the existing `get_logs_dir`, `get_scripts_dir`) to read from `config.toml [paths]` via the existing `_resolve_path` helper (priority: env var → config → default). Each path now reads `config.toml [paths].` where `` is one of: `presets`, `tool_presets`, `personas`, `themes`, `workspace_profiles`, `credentials`, `logs_dir`, `scripts_dir`. +- `tests/conftest.py:isolate_workspace`: now writes a `config_overrides.toml` with a complete `[paths]` section that overrides every path getter to point inside `_ISOLATION_WORKSPACE = tests/artifacts/_isolation_workspace_/`. NO `SLOP_*` env vars are set anywhere in conftest. +- `tests/conftest.py:live_gui`: dropped redundant `SLOP_*` env var setup — the sloppy.py subprocess reads paths from `--config` (which points at config_overrides.toml). + +**Example output (the auto-generated `config_overrides.toml`):** + +```toml +[ai] +provider = "gemini" +model = "gemini-2.5-flash-lite" + +[projects] +paths = [] +active = "" + +[gui.show_windows] + +[paths] +presets = "tests\\artifacts\\_isolation_workspace_20260619_085534\\presets.toml" +tool_presets = "tests\\artifacts\\_isolation_workspace_20260619_085534\\tool_presets.toml" +personas = "tests\\artifacts\\_isolation_workspace_20260619_085534\\personas.toml" +themes = "tests\\artifacts\\_isolation_workspace_20260619_085534\\themes" +workspace_profiles = "tests\\artifacts\\_isolation_workspace_20260619_085534\\workspace_profiles.toml" +credentials = "tests\\artifacts\\_isolation_workspace_20260619_085534\\credentials.toml" +logs_dir = "tests\\artifacts\\_isolation_workspace_20260619_085534\\logs" +scripts_dir = "tests\\artifacts\\_isolation_workspace_20260619_085534\\scripts" +``` + +The user can now open this file and see exactly where every path is routed. + +- `tests/test_test_sandbox.py`: 2 new regression tests (`test_config_overrides_toml_has_paths_section`, `test_path_getters_read_from_config_paths_section`). + ### Fix 2: Python runtime file-I/O guard (FR1) **Bug:** No runtime guard. Tests could call `Path("manual_slop.toml").write_text(...)` with no consequence. @@ -76,7 +117,7 @@ The root cause was the silent `SLOP_CONFIG` env-var fallback in `src/paths.py:ge ## Verification results -### `tests/test_test_sandbox.py` — 20 default-on + 1 Windows opt-in, all pass +### `tests/test_test_sandbox.py` — 22 default-on + 1 Windows opt-in, all pass ``` tests/test_test_sandbox.py::test_audit_runs_without_error PASSED @@ -99,7 +140,9 @@ tests/test_test_sandbox.py::test_pyproject_toml_basetemp_is_under_tests PASSED tests/test_test_sandbox.py::test_isolate_workspace_does_not_use_tmp_path_factory_for_infra PASSED tests/test_test_sandbox.py::test_appcontroller_init_does_not_load_config PASSED tests/test_test_sandbox.py::test_run_tests_sandboxed_whatif PASSED [Windows-only, skipif os.name != "nt"] -================ 20 passed in 3.69s ================ +tests/test_test_sandbox.py::test_config_overrides_toml_has_paths_section PASSED [v2] +tests/test_test_sandbox.py::test_path_getters_read_from_config_paths_section PASSED [v2] +================ 22 passed in 3.93s ================ ``` ### Layer 1 FR1 verification — caught real corruption attempts @@ -123,7 +166,7 @@ A full Tier-2/3/headless re-run is recommended after merge to verify VC8 ("no re 5. **The 4-layer sandbox enforcement** is default-on for Layers 1, 2, 4 (file-presence = enabled per `feature_flags.md`). Layer 3 (PowerShell restricted-token) is opt-in via explicit invocation. 6. **All scratch / intermediate / test files live inside the Tier 2 clone** (per project-relative workspace rule; no AppData / Temp / external paths). -## Files changed (cumulative, 11 commits) +## Files changed (cumulative, 13 commits) ``` 43e50f93 chore(audit): add audit_test_sandbox_violations.py + 8 regression tests for FR4 @@ -136,6 +179,8 @@ dc5afc21 feat(scripts): add run_tests_sandboxed.ps1 (FR5 OS-level sandbox) + sm 8dddf567 fix(tests): route live_gui subprocess logs to tests/logs/ instead of project root 1f7e81ac fix(sandbox): audit --tests-dir bypass EXCLUDE_DIRS; probe path in regression test 07bcd4ee fix(sandbox): allow %TEMP% writes for legitimate tempfile usage +3a86ca37 fix(paths): route ALL path getters through config.toml [paths] overrides (FR2 v2) +561090c0 test(sandbox): add [paths] section regression tests for FR2 v2 design ``` Files touched: