Private
Public Access
docs(reports): TRACK_COMPLETION_tier2_sandbox_hardening_20260617
End-of-track report for the 4 sandbox bugs hit by the first Tier 2 run (send_result_to_send_20260616) and the audit infrastructure added to prevent regression. 5 fixes (4 bugs + 1 audit) shipped as 6 atomic commits on master. See the report for: - Per-fix description, root cause, and file:line refs - Live clone state after the fixes - 38 default-on + 3 opt-in test inventory - 4 conventions established - Next steps for the user (re-run, merge review branch, etc.) - Known follow-ups NOT in this track
This commit is contained in:
@@ -0,0 +1,158 @@
|
||||
# Tier 2 Sandbox Hardening — Post-Ship Track Report
|
||||
|
||||
**Track:** `tier2_sandbox_hardening_20260617` (post-ship follow-up to `tier2_autonomous_sandbox_20260616`)
|
||||
**Shipped:** 2026-06-17
|
||||
**Owner:** Tier 1 Orchestrator (interactive)
|
||||
**Trigger:** First real Tier 2 run (`send_result_to_send_20260616`) hit 4 separate sandbox bugs that halted autonomous ops.
|
||||
**Commits:** 6 atomic commits on `master`
|
||||
**Tests:** 38 default-on (all pass) + 3 opt-in (all pass with `TIER2_SANDBOX_TESTS=1`)
|
||||
|
||||
## Summary
|
||||
|
||||
The first Tier 2 sandbox run (`send_result_to_send_20260616`, shipped earlier this week) hit four separate bugs that prevented autonomous execution:
|
||||
|
||||
1. OpenCode session-level `permission.read`/`write` did not allow the sandbox clone path (the clone inherited the main repo's `opencode.json` via `git clone`, which has no `read`/`write` keys at the top level).
|
||||
2. The MCP server was launched from the MAIN repo's `scripts/mcp_server.py` (also inherited via `git clone`), so its allowlist = main repo's `project_root` + main repo's `mcp_paths.toml` (which allowlists `gencpp`). Tier 2 calls to `manual-slop_read_file` on clone paths were rejected with "Allowed base directories are: gencpp, manual_slop".
|
||||
3. The Tier 2 agent wrote an audit JSON to `C:\Users\Ed\AppData\Local\Temp\` via shell redirection, triggering the OpenCode session's "ask" prompt for paths outside the project root, which halted ops mid-track.
|
||||
4. The top-level `model` field was inherited as `zai/glm-5` instead of the Tier 2 model `minimax-coding-plan/MiniMax-M3`.
|
||||
|
||||
All four are fixed. The sandbox now has a 3-layer enforcement stack (OpenCode session permission + MCP server config + bash deny rules) plus a default-on regression test that fails CI if any script under `./scripts/` writes to `%TEMP%`.
|
||||
|
||||
## What changed
|
||||
|
||||
### Fix 1: Top-level OpenCode permission allowlist (commit `9cd85364`)
|
||||
|
||||
**Bug:** The Tier 2 clone's `opencode.json` was a `git clone` of the main repo's, which has `permission.edit: ask, permission.bash: ask` and **no** `permission.read`/`write` keys. The `setup_tier2_clone.ps1` merge logic only updated the `tier2-autonomous` agent block — it never patched the top-level `permission`. OpenCode's default-agent access check uses the top-level, so any read of `C:\projects\manual_slop_tier2\**` was rejected (falling back to the user's project allowlist of `gencpp` + `manual_slop`).
|
||||
|
||||
**Fix:**
|
||||
- `conductor/tier2/opencode.json.fragment`: added a top-level `permission` block with `read`/`write` = `*` deny + allowlist of the sandbox clone + app-data dirs. Top-level `bash` is `*` deny + allowlist of safe git commands + `uv run python scripts/{run_tests_batched.py, tier2/*}` + basic shell utilities. The four hard-ban git commands remain denied.
|
||||
- `scripts/tier2/setup_tier2_clone.ps1`: merge now also overwrites the top-level `permission` from the fragment.
|
||||
- `tests/test_tier2_slash_command_spec.py`: added `test_config_fragment_has_top_level_permission` (default-on) and renamed the stale `_main` test to `_master`.
|
||||
|
||||
### Fix 2: MCP server pointed at clone, `mcp_paths.toml` reset (commit `fd5175bf`)
|
||||
|
||||
**Bug:** Follow-up to Fix 1. OpenCode's session-level `permission.read` is one layer, but the MCP server has its own allowlist = `project_root` (parent of the script) + `extra_dirs` from `mcp_paths.toml` at that project root. The clone inherited the main repo's `mcp.manual-slop.command` via `git clone` (pointing at `C:\projects\manual_slop\scripts\mcp_server.py` with `PYTHONPATH=C:\projects\manual_slop\src`), so the MCP server was using the MAIN repo's `project_root` + the main repo's `mcp_paths.toml` (`extra_dirs=['C:/projects/gencpp']`).
|
||||
|
||||
**Fix:**
|
||||
- `scripts/tier2/setup_tier2_clone.ps1`: now overrides the clone's `mcp.manual-slop.command` to point at `$Tier2ClonePath\scripts\mcp_server.py` and `mcp.manual-slop.environment.PYTHONPATH` to `$Tier2ClonePath\src`. Replaces the clone's `mcp_paths.toml` with `extra_dirs = []`.
|
||||
- `tests/test_tier2_setup_bootstrap.py`: added `test_setup_script_overrides_mcp_server` (opt-in).
|
||||
|
||||
### Fix 3: Top-level model = MiniMax-M3 (commit `3ec601d4`)
|
||||
|
||||
**Bug:** The clone's `opencode.json` inherited the main repo's top-level `model: zai/glm-5` via `git clone`. The `tier2-autonomous` agent had its own `model: minimax-coding-plan/MiniMax-M3` override (so the agent itself was using the right model), but any other agent path or sub-spawn would have used `zai/glm-5`.
|
||||
|
||||
**Fix:**
|
||||
- `conductor/tier2/opencode.json.fragment`: added `model: "minimax-coding-plan/MiniMax-M3"` at the top level.
|
||||
- `scripts/tier2/setup_tier2_clone.ps1`: merge now overrides `model` from the fragment.
|
||||
- Tests: `test_config_fragment_has_top_level_model` (default-on) and `test_setup_script_overrides_model` (opt-in).
|
||||
|
||||
### Fix 4: %TEMP% writes denied (commit `03c9df84`)
|
||||
|
||||
**Bug:** The Tier 2 agent wrote `audit_exception_handling.py` output to `C:\Users\Ed\AppData\Local\Temp\audit_initial.json` via shell redirection. This is outside the sandbox allowlist. OpenCode's session-level guard fires the "ask" prompt for paths outside the project root — no answer in an autonomous session, so ops halted mid-track.
|
||||
|
||||
**Fix (3 layers):**
|
||||
- `conductor/tier2/opencode.json.fragment`: added bash deny rule `"*AppData\\Local\\Temp\\*": "deny"` to BOTH the top-level `permission.bash` and the `tier2-autonomous` agent's `permission.bash`. The agent physically cannot run shell commands targeting the global Temp dir.
|
||||
- `conductor/tier2/agents/tier2-autonomous.md`: added a "Temp files" convention telling the agent to use `C:\Users\Ed\AppData\Local\manual_slop\tier2\` for scratch / audit-output files, NOT `%TEMP%`.
|
||||
- `conductor/tier2/commands/tier-2-auto-execute.md`: same convention in the slash command.
|
||||
- `tests/test_tier2_slash_command_spec.py`: added `test_agent_denies_temp_writes` and `test_config_fragment_denies_temp_writes` (default-on).
|
||||
- Also: cleaned up the leaked `audit_initial.json` + `audit.json` + `audit_after*.json` from `%TEMP%` (leftovers from prior runs).
|
||||
|
||||
### Fix 5: Structural enforcement — no-temp-writes audit (commit `7baef97d`)
|
||||
|
||||
**Bug:** The previous fixes rely on the agent following instructions and the bash deny rules catching the path. If a future script in `./scripts/` uses `tempfile.gettempdir()` or `os.environ['TEMP']`, the script itself would write to `%TEMP%` regardless of the agent's behavior. No structural guard existed.
|
||||
|
||||
**Fix (the new audit):**
|
||||
- `scripts/audit_no_temp_writes.py`: the canonical audit. Same shape as `scripts/audit_exception_handling.py` (--json for machine output, --strict for the CI gate). Patterns cover `tempfile.*`, `gettempdir`, `mkstemp`, `NamedTemporaryFile`, `TemporaryFile`, `os.environ['TEMP']`, `$env:TEMP`, `%TEMP%`, `/tmp/`, `TempDir`, etc. Excludes `scripts/tier2/artifacts/` (throw-away archive) and itself.
|
||||
- `tests/test_no_temp_writes.py`: default-on regression test. Calls the audit with `--strict` and asserts exit 0. If a new script under `./scripts/` ever uses `%TEMP%`, the test fails and CI breaks.
|
||||
|
||||
**Current state: CLEAN.** No script under `./scripts/**` (excluding the throw-away archive) emits to `%TEMP%`.
|
||||
|
||||
### Pre-existing uncommitted changes (NOT touched)
|
||||
|
||||
- `config.toml`, `manualslop_layout.ini`, `project_history.toml` — unrelated working tree drift from prior session(s). The user can commit or discard separately.
|
||||
|
||||
## Live clone state (after this session)
|
||||
|
||||
The Tier 2 clone at `C:\projects\manual_slop_tier2\` was re-bootstrapped after each fix. Current state:
|
||||
|
||||
- `mcp.manual-slop.command` → `C:\projects\manual_slop_tier2\scripts\mcp_server.py` (was `C:\projects\manual_slop\...`)
|
||||
- `mcp.manual-slop.environment.PYTHONPATH` → `C:\projects\manual_slop_tier2\src` (was `C:\projects\manual_slop\src`)
|
||||
- `mcp_paths.toml` → `extra_dirs = []` (was `extra_dirs = ["C:/projects/gencpp"]`)
|
||||
- Top-level `model` → `minimax-coding-plan/MiniMax-M3` (was `zai/glm-5`)
|
||||
- Top-level `permission.read` / `write` → deny `*`, allow sandbox clone + app-data dirs (was empty)
|
||||
- Top-level `permission.bash` → deny `*`, allowlist of safe git + test runner + tier2 scripts; deny `*AppData\Local\Temp\*` and the four hard-ban git commands
|
||||
- `tier2-autonomous.agent.permission` → unchanged (allow-edit, allow-all-bash with the 4 git denies, deny-all-read with sandbox allowlist, deny-all-write with sandbox allowlist, deny `*AppData\Local\Temp\*`)
|
||||
|
||||
## Test inventory (38 default-on + 3 opt-in)
|
||||
|
||||
| File | Count | Status |
|
||||
|---|---|---|
|
||||
| `tests/test_no_temp_writes.py` | 1 | default-on, passes |
|
||||
| `tests/test_tier2_slash_command_spec.py` | 16 | default-on, all pass (was 13) |
|
||||
| `tests/test_failcount.py` | 17 | default-on, all pass |
|
||||
| `tests/test_tier2_setup_bootstrap.py` | 3 | opt-in (`TIER2_SANDBOX_TESTS=1`), all pass |
|
||||
|
||||
## Conventions established in this session
|
||||
|
||||
1. **Top-level OpenCode `permission.read`/`write` is the source of truth** for the default-agent access check. The agent's own `permission.read`/`write` block is a per-agent override but does not replace the top-level.
|
||||
2. **The MCP server has its own allowlist**, separate from OpenCode's session-level permission. The MCP server is launched from `$Tier2ClonePath\scripts\mcp_server.py` with `PYTHONPATH=$Tier2ClonePath\src`, and the clone's `mcp_paths.toml` is reset to `extra_dirs = []` on bootstrap.
|
||||
3. **Temp files go in `C:\Users\Ed\AppData\Local\manual_slop\tier2\`**, NOT `%TEMP%`. Enforced by:
|
||||
- bash deny rule `*AppData\Local\Temp\*` (agent + top-level)
|
||||
- agent prompt + slash command convention note
|
||||
- `scripts/audit_no_temp_writes.py` + `tests/test_no_temp_writes.py` (CI gate)
|
||||
4. **Top-level `model` is `minimax-coding-plan/MiniMax-M3`** (the Tier 2 model), not the main repo's `zai/glm-5`.
|
||||
|
||||
## Files changed (cumulative, 6 commits)
|
||||
|
||||
```
|
||||
9cd85364 fix(tier2): top-level permission allowlist - sandbox paths now enforced
|
||||
fd5175bf fix(tier2): override MCP server path + reset mcp_paths.toml in clone
|
||||
3ec601d4 fix(tier2): override top-level model to MiniMax-M3
|
||||
03c9df84 fix(tier2): deny %TEMP% writes - use app-data dir for temp files
|
||||
7baef97d feat(audit): add no-temp-writes audit + regression test
|
||||
```
|
||||
|
||||
Files touched:
|
||||
- `conductor/tier2/opencode.json.fragment` (4 of 5 fixes)
|
||||
- `conductor/tier2/agents/tier2-autonomous.md` (temp file convention)
|
||||
- `conductor/tier2/commands/tier-2-auto-execute.md` (temp file convention)
|
||||
- `scripts/tier2/setup_tier2_clone.ps1` (4 of 5 fixes: top-level permission, MCP server, model, mcp_paths.toml)
|
||||
- `scripts/audit_no_temp_writes.py` (new, 108 lines)
|
||||
- `tests/test_no_temp_writes.py` (new, 35 lines)
|
||||
- `tests/test_tier2_slash_command_spec.py` (3 new tests + 1 rename)
|
||||
- `tests/test_tier2_setup_bootstrap.py` (2 new tests)
|
||||
|
||||
## Next steps for the user
|
||||
|
||||
1. **Re-run the Tier 2 track.** Launch the Tier 2 (Sandboxed) shortcut and retry the in-flight track. The sandbox should now be fully autonomous — no "ask" prompts, no ACCESS DENIED.
|
||||
2. **Decide merge on the review branch.** The `send_result_to_send_20260616` review branch still needs the user's merge decision (separate from this fix work). See `conductor/tracks/send_result_to_send_20260616/TRACK_COMPLETION_send_result_to_send_20260616.md` for the track completion report.
|
||||
3. **Optionally wire the audit into pre-commit.** `scripts/audit_no_temp_writes.py --strict` is the CI gate. If the project has a pre-commit hook setup, add it there. Currently it's only run as a default-on pytest test.
|
||||
4. **Optionally clean up pre-existing working-tree drift.** The `config.toml`, `manualslop_layout.ini`, and `project_history.toml` uncommitted changes from prior sessions can be committed or discarded.
|
||||
|
||||
## Known follow-ups (NOT in this track)
|
||||
|
||||
- **AppContainer / Job Object hardening.** The Windows restricted token + ACLs are "v1" defense. A future track could add proper AppContainer isolation.
|
||||
- **Repo-wide LF standardization.** The repo has a mix of CRLF and LF. A future track could normalize to LF; the agent prompt's "preserve existing line endings" convention is the current workaround.
|
||||
- **Parallel Tier 2 runs.** The current sandbox assumes one Tier 2 run at a time (the app-data dir is shared). A future track could add per-run isolation.
|
||||
- **Recover the accidentally-deleted `fable_review_20260617/`.** The 4 files were swept up in Tier 2's "wrong folder" commit `e2e57036` from the `send_result_to_send_20260616` run. Recovery is via the `fable_review_20260617` track's git history (or a follow-up).
|
||||
|
||||
## Verification commands
|
||||
|
||||
```bash
|
||||
# Apply the new sandbox fixes to the live clone
|
||||
pwsh -NoProfile -File C:\projects\manual_slop\scripts\tier2\setup_tier2_clone.ps1 `
|
||||
-MainRepoPath C:\projects\manual_slop `
|
||||
-Tier2ClonePath C:\projects\manual_slop_tier2
|
||||
|
||||
# Run the new + updated tests (38 default-on, all pass)
|
||||
uv run python -m pytest tests/test_no_temp_writes.py tests/test_tier2_slash_command_spec.py tests/test_failcount.py
|
||||
|
||||
# Run the opt-in tests (3 more, with TIER2_SANDBOX_TESTS=1)
|
||||
$env:TIER2_SANDBOX_TESTS=1
|
||||
uv run python -m pytest tests/test_tier2_setup_bootstrap.py
|
||||
|
||||
# Run the new audit
|
||||
uv run python scripts/audit_no_temp_writes.py --strict
|
||||
```
|
||||
|
||||
End of report.
|
||||
Reference in New Issue
Block a user