Private
Public Access
0
0

feat(directives): harvest 5 directives from docs/guide_testing.md (sandbox overview, live_gui session-scoped, defer-not-catch, narrow tests, AST method visibility)

This commit is contained in:
2026-07-02 23:56:26 -04:00
parent 782530ba6d
commit a758f0a4c9
10 changed files with 99 additions and 0 deletions
@@ -0,0 +1,7 @@
# ast_verify_class_methods_after_edit
## v1
**Why this iteration:** Lifted verbatim from `docs/guide_testing.md:834-842 (§Pattern: Indentation-Driven Method Visibility)`.
**Source:** `docs/guide_testing.md:834-842 (§Pattern: Indentation-Driven Method Visibility)`
**Lifted:** 2026-07-02 (Phase A expansion harvest; user directive 2026-07-02)
@@ -0,0 +1,11 @@
## Pattern: Indentation-Driven Method Visibility
**The bug:** A class method defined with the right intent (2-space indent) may be parsed as nested inside a previous function if indentation is off by even one space. The file passes syntactically (imports OK) but the method is **not** on the class — `hasattr(App, 'method_name')` returns `False`. Any production code that calls `app.method_name` falls through to `__getattr__`, which delegates to the controller (which also doesn't have the method), and a cryptic `AttributeError` is raised at runtime.
**How to detect:**
- Use AST to list all App methods: `uv run python -c "import ast; tree = ast.parse(open('src/gui_2.py').read()); [print(item.name) for n in ast.walk(tree) if isinstance(n, ast.ClassDef) and n.name == 'App' for item in n.body if isinstance(item, ast.FunctionDef)]"`
- The skeleton via `manual-slop_py_get_skeleton` should show the method as a class member.
**How to fix:** Re-indent the affected method to 2-space class level. Run the failing test to confirm.
**Prevention:** When reorganizing a class body, run the AST check above immediately after the edit. This catches the issue in <1 second vs. finding it via failing live_gui tests minutes later.
@@ -0,0 +1,7 @@
# defer_not_catch_for_native_crashes
## v1
**Why this iteration:** Lifted verbatim from `docs/guide_testing.md:789-813 (§Known Gotchas: Early-Render C-Level Crashes)`.
**Source:** `docs/guide_testing.md:789-813 (§Known Gotchas: Early-Render C-Level Crashes)`
**Lifted:** 2026-07-02 (Phase A expansion harvest; user directive 2026-07-02)
@@ -0,0 +1,23 @@
## Defer-Not-Catch Pattern for Native Crashes
`imgui-bundle` (and similar native extension libraries) expose C-level functions that can crash the Python process with a Windows access violation (`0xc0000005`) or a SIGSEGV on Linux. **These crashes are not catchable from Python**`try/except Exception` does not intercept native access violations, only Python exceptions.
Symptoms:
- The `sloppy.py` subprocess disappears without a Python traceback.
- The pytest output shows `pytest.fail("Hook server did not start in 15s")` (the subprocess died during startup).
- Windows Event Viewer shows `Faulting module: _imgui_bundle.cp311-win_amd64.pyd` with exception code `0xc0000005`.
**Fix pattern: defer-not-catch.** Track a one-shot ready flag in instance state; return early on the first call, only invoking the C function on subsequent calls:
```python
def _capture_workspace_profile(self, name: str) -> models.WorkspaceProfile:
if not getattr(self, "_ini_capture_ready", False):
self._ini_capture_ready = True
return models.WorkspaceProfile(name=name, docking_layout=b"", ...)
ini = imgui.save_ini_settings_to_memory()
return models.WorkspaceProfile(name=name, docking_layout=ini.encode("utf-8") if isinstance(ini, str) else ini, ...)
```
The first call (during initial startup) returns a safe empty profile and flips the flag; subsequent calls (when the user actually clicks Save Profile) invoke the C function.
**Sentinel type contract.** The early-return sentinel value must match the type contract of the downstream consumer. For `WorkspaceProfile.ini_content: str`, the sentinel must be `""` (str), not `b""` (bytes) — `tomli_w` rejects bytes (`TypeError: Object of type 'bytes' is not TOML serializable`).
@@ -0,0 +1,7 @@
# live_gui_session_scoped_no_restart
## v1
**Why this iteration:** Lifted verbatim from `docs/guide_testing.md:753-787 (§Known Gotchas: Authoring Robust live_gui Tests)`.
**Source:** `docs/guide_testing.md:753-787 (§Known Gotchas: Authoring Robust live_gui Tests)`
**Lifted:** 2026-07-02 (Phase A expansion harvest; user directive 2026-07-02)
@@ -0,0 +1,11 @@
## live_gui session-scoped fixture contract
`live_gui` is a **session-scoped** fixture. All tests in a session share the same `sloppy.py` subprocess. The subprocess is **not** restarted between tests; its internal state (Fonts, DisplaySize, internal caches, current theme, current workspace profile, current discussion, current MMA track) **accumulates** from the previous test.
**This is a test-authoring contract, not a fixture bug.** A test that passes when run after test X but fails when run in isolation is a fragile test. Robust `live_gui` tests must:
1. **Not assume clean state.** Before invoking an operation, explicitly verify the precondition via the Hook API (e.g. `client.get_value("show_my_window")`, `client.get_mma_status()`, `client.get_session()`). Do not assume a previous test set the state.
2. **Use the wait-for-ready pattern, not fixed sleeps.** `time.sleep(1)` is not enough for ImGui to stabilize in the first few render frames; use `wait_for_event` with a generous timeout, or poll `client.get_status()` until ImGui reports `ready`.
3. **Reset state explicitly if the test depends on it.** Reset relevant state via Hook API in a `try/finally` so the next test starts from a known baseline.
4. **Test both in the full suite AND in isolation before merging.** If a test passes in the full suite but fails in isolation, the test is fragile — fix the test, don't add a warmup comment.
5. **Use `get_value`/`wait_for_event` to assert ready, not just to assert success.**
@@ -0,0 +1,7 @@
# no_real_io_during_tests
## v1
**Why this iteration:** Lifted verbatim from `docs/guide_testing.md:9-16 (Overview)`.
**Source:** `docs/guide_testing.md:9-16 (Overview)`
**Lifted:** 2026-07-02 (Phase A expansion harvest; user directive 2026-07-02)
@@ -0,0 +1,10 @@
## The 4 test infrastructure principles
1. **No real I/O during tests** — every test gets a sandboxed workspace via the `isolate_workspace` autouse fixture.
2. **No real AI calls** — tests use mock providers, reset session state, and never hit the network.
3. **GUI tests launch a real app** — the `live_gui` session fixture starts `sloppy.py --enable-test-hooks` so integration tests can drive the actual app via the Hook API.
4. **Tests are categorized by marker** — unit, integration, strict, clean_install, docker — so CI can opt in to expensive tests.
The autouse `isolate_workspace` fixture (1 of 7 conftest fixtures) gives every test a fresh, isolated workspace so it cannot pollute the user's real `manual_slop.toml`, `presets.toml`, etc. The session-scoped `live_gui` fixture starts `sloppy.py --enable-test-hooks` once per session so integration tests can drive the actual app via the Hook API.
All test-generated artifacts (logs, temporary workspaces, mock outputs) MUST be written to `tests/artifacts/` or `tests/logs/` (gitignored).
@@ -0,0 +1,7 @@
# test_narrow_not_kitchen_sink
## v1
**Why this iteration:** Lifted verbatim from `docs/guide_testing.md:817-829 (§Pattern: Narrow Test Paths)`.
**Source:** `docs/guide_testing.md:817-829 (§Pattern: Narrow Test Paths)`
**Lifted:** 2026-07-02 (Phase A expansion harvest; user directive 2026-07-02)
@@ -0,0 +1,9 @@
## Pattern: Narrow Test Paths vs. Kitchen-Sink Functions
**Anti-pattern: calling a kitchen-sink function.** A test that does `gui_2.render_main_interface(app_instance)` requires mocking 50+ imgui/imscope methods because `render_main_interface` dispatches to dozens of nested render functions. Adding a single mock for `imscope.window` (to return a tuple) just reveals the next un-mocked dependency (e.g. `imgui.begin` returning bool where a 2-tuple is expected). The test never reaches its assertion.
**Better pattern: test the narrow function.** Most render flows have a dedicated sub-function (e.g. `render_prior_session_view`, `render_preset_manager_window`, `render_theme_panel`). Refactor the test to call the narrow function directly with mocks scoped to what that function actually uses.
**When to refactor vs. add mocks:**
- If the test intent is verify push/pop balance in the prior-session render path, call the narrow function.
- If the test intent is verify the whole GUI render path is correct, accept the 50+ mock cost (and ensure all mocks are correct).