From f09cd4a733426b29db2c44adf4eb244203a8955a Mon Sep 17 00:00:00 2001 From: Ed_ Date: Sat, 6 Jun 2026 21:45:27 -0400 Subject: [PATCH] conductor: doc final sync for sub-tracks 2 (partial), 3, 4 + conftest fix --- conductor/tracks.md | 10 +++++++--- conductor/tracks/startup_speedup_20260606/plan.md | 14 +++++++++++++- .../tracks/startup_speedup_20260606/state.toml | 7 ++++--- 3 files changed, 24 insertions(+), 7 deletions(-) diff --git a/conductor/tracks.md b/conductor/tracks.md index d32e8675..3ccd79c4 100644 --- a/conductor/tracks.md +++ b/conductor/tracks.md @@ -149,13 +149,17 @@ User review surfaced five outstanding UI issues, each previously attempted witho ## Remaining Backlog (Phases 3 & 4) -0. [x] **Track: Sloppy.py Startup Speedup** `[track-created: cd4fb045] [phase-1-2-done: f9a01258] [phase-3-done: 51c054ec] [phase-4-done: 3849d304] [phase-5a-done: 78d3a1db] [phase-5b-done: 69d098ba] [phase-5c-done: 48c96499] [phase-5d-done: de6b85d2] [phase-5-done: 515a3029] [phase-6-partial-done: 85d18885] [sub-track-1-done: 253e1798] [post-shipping-fix-1: 8c4791d0] [post-shipping-fix-2: 88fc42bb] [post-shipping-fix-3: 52ea2693] [COMPLETE 2026-06-07]` +0. [x] **Track: Sloppy.py Startup Speedup** `[track-created: cd4fb045] [phase-1-2-done: f9a01258] [phase-3-done: 51c054ec] [phase-4-done: 3849d304] [phase-5a-done: 78d3a1db] [phase-5b-done: 69d098ba] [phase-5c-done: 48c96499] [phase-5d-done: de6b85d2] [phase-5-done: 515a3029] [phase-6-partial-done: 85d18885] [sub-track-1-done: 253e1798] [post-shipping-fix-1: 8c4791d0] [post-shipping-fix-2: 88fc42bb] [post-shipping-fix-3: 52ea2693] [sub-track-3-done: 8fea8fe9] [sub-track-4-done: f3d071e0] [conftest-atexit-fix: 8957c9a5] [sub-track-2-partial: ae3b433e] [COMPLETE 2026-06-07]` *Link: [./tracks/startup_speedup_20260606/](./tracks/startup_speedup_20260606/), Spec: [./tracks/startup_speedup_20260606/spec.md](./tracks/startup_speedup_20260606/spec.md), Plan: [./tracks/startup_speedup_20260606/plan.md](./tracks/startup_speedup_20260606/plan.md)* *Goal: Reduce sloppy.py startup time. Main Thread Purity Invariant. 9 phases, 57 tasks. 44 TDD tests added (all passing). 7 main thread purity tests enforce invariant for 6 refactored files.* - *Final measured: import src.ai_client 161ms (was 1800ms; 91% reduction / 1638ms saved). import src.gui_2 341ms (was 1770ms; 81% reduction / 1429ms saved). Total ~3067ms saved on the 2 big files. 63 audit violations remain (was 67 baseline) - all 6 refactored files contribute 0 new violations.* + *Final measured: import src.ai_client 161ms (was 1800ms; 91% reduction / 1638ms saved). import src.gui_2 341ms (was 1770ms; 81% reduction / 1429ms saved). Total ~3067ms saved on the 2 big files. 62 audit violations remain (was 63 after Sub-track 2 partial; was 67 baseline) - all 6 refactored files contribute 0 new violations.* *Sub-track 1 (Phase 6 full completion) at 253e1798: 15 ad-hoc threading.Thread() call sites migrated to self.submit_io(...); ZERO new threading.Thread() in src/; only 5 domain-specific exempt sites remain (HookServer HTTP/WS, asyncio loop, WorkerPool, CPU monitor).* + *Sub-track 3 (Hook API warmup endpoints) at 8fea8fe9: GET /api/warmup_status and GET /api/warmup_wait?timeout=N. 7 tests (5 unit + 2 live_gui). All pass.* + *Sub-track 4 (GUI status indicator) at f3d071e0: render_warmup_status_indicator() + _on_warmup_complete_callback() + App._post_init registration. 6 tests (5 unit + 1 live_gui). All pass.* + *Conftest atexit fix at 8957c9a5: registers a non-blocking pool shutdown via atexit. Fixes the run_tests_batched.py hang between batches (ThreadPoolExecutor.__del__ was blocking on shutdown(wait=True) for stuck warmup jobs).* + *Sub-track 2 (audit violations) PARTIAL at ae3b433e: 1 of 63 violations fixed (tomli_w in src/models.py). 62 remain (pydantic in models.py; tree_sitter in file_cache.py; websockets/cost_tracker/session_logger in api_hooks.py; 48 in app_controller.py + gui_2.py; 4 in sloppy.py). These are large refactors (especially gui_2.py with 24 violations and app_controller.py with 24) that exceed the scope of a single sub-track; addressed as future work.* *3 post-shipping bugfix commits: 8c4791d0 (real bug: _ensure_gemini_client UnboundLocalError + test_discussion_compression deepseek mock adaptation); 88fc42bb (spec convention: 7 sites in src/ai_client.py use _require_warmed('google.genai') + .types parent lookup instead of leaf); 52ea2693 (conftest: use AppController.wait_for_warmup(timeout=60.0) instead of direct import google.genai — user-corrected jank workaround).* - *Pre-existing test failures (unrelated, user will address): test_api_generate_blocked_while_stale (ui_global_preset_name AttributeError); test_rag_large_codebase_verification_sim (RAG retrieval). 3 follow-up sub-tracks remain: (1) migrate remaining audit violations in src/models.py + sloppy.py, (2) add dedicated /api/warmup_status + /api/warmup_wait endpoints, (3) GUI status bar + completion toast.* + *Pre-existing test failures (unrelated, user will address): test_api_generate_blocked_while_stale (ui_global_preset_name AttributeError); test_rag_large_codebase_verification_sim (RAG retrieval).* 0c. [~] **Track: Test Batching Refactor** `[track-created: b7a97374]` *Link: [./tracks/test_batching_refactor_20260606/](./tracks/test_batching_refactor_20260606/), Spec: [./tracks/test_batching_refactor_20260606/spec.md](./tracks/test_batching_refactor_20260606/spec.md), Plan: [./tracks/test_batching_refactor_20260606/plan.md](./tracks/test_batching_refactor_20260606/plan.md) (to be authored by writing-plans skill)* diff --git a/conductor/tracks/startup_speedup_20260606/plan.md b/conductor/tracks/startup_speedup_20260606/plan.md index 5915992b..a3fead27 100644 --- a/conductor/tracks/startup_speedup_20260606/plan.md +++ b/conductor/tracks/startup_speedup_20260606/plan.md @@ -303,7 +303,19 @@ The user confirmed: *"I'll address those bugs later, tests were prob too fragile - [x] **Warmup completion notification works** — `controller.is_warmup_done()` returns True within 10s of startup; Hook API diagnostics endpoint exposes `warmup_status` (commit `b464d1fe`); conftest uses `wait_for_warmup(timeout=60.0)` to ensure warmup completes before tests run - [x] **User action latency is zero for warmup-dependent operations** — manual smoke test switching providers / opening palette / rendering NERV is instant (all heavy SDKs are in `sys.modules` by the time the user makes their first action) -**Status:** Track SHIPPED at `12cec6ae` (Phase 9 checkpoint); sub-track 1 (Phase 6 full completion) SHIPPED at `253e1798`. 3 post-shipping bugfix commits applied (`8c4791d0`, `88fc42bb`, `52ea2693`). **Final ship commit: `253e1798`.** +**Status:** Track SHIPPED at `12cec6ae` (Phase 9 checkpoint); sub-track 1 (Phase 6 full completion) SHIPPED at `253e1798`. 3 post-shipping bugfix commits applied (`8c4791d0`, `88fc42bb`, `52ea2693`). + +**Sub-track work after track SHIP (2026-06-07):** + +- **Sub-track 3 (Hook API warmup endpoints) at `8fea8fe9`:** Added `GET /api/warmup_status` and `GET /api/warmup_wait?timeout=N` endpoints in `src/api_hooks.py`. Added `get_warmup_status()` and `get_warmup_wait(timeout)` methods in `src/api_hook_client.py`. 7 tests in `tests/test_api_hooks_warmup.py` (5 unit + 2 live_gui). All pass. + +- **Sub-track 4 (GUI status indicator) at `f3d071e0`:** Added `render_warmup_status_indicator(app)` and `_on_warmup_complete_callback(app, status)` module-level functions in `src/gui_2.py`. Registered callback in `App._post_init`. 6 tests in `tests/test_gui_warmup_indicator.py` (5 unit + 1 live_gui). All pass. + +- **Conftest atexit fix at `8957c9a5`:** Registered an `atexit` handler that captures the `_io_pool` reference via closure and calls `shutdown(wait=False)` at process exit. Fixes the `run_tests_batched.py` hang between batches (where `ThreadPoolExecutor.__del__ -> shutdown(wait=True)` was blocking on stuck warmup jobs). + +- **Sub-track 2 (audit violations) PARTIAL at `ae3b433e`:** Removed top-level `import tomli_w` from `src/models.py`; now loaded on-demand in `save_config()`. 1 of 63 audit violations fixed. 62 remain (pydantic in models.py; tree_sitter in file_cache.py; websockets/cost_tracker/session_logger in api_hooks.py; 48 in app_controller.py + gui_2.py; 4 in sloppy.py). The remaining violations are large refactors that exceed the scope of a single sub-track. + +**Final ship commit: `253e1798`.** After sub-track work, the latest commit is `ae3b433e`. --- diff --git a/conductor/tracks/startup_speedup_20260606/state.toml b/conductor/tracks/startup_speedup_20260606/state.toml index 4cbe35c3..98bd7777 100644 --- a/conductor/tracks/startup_speedup_20260606/state.toml +++ b/conductor/tracks/startup_speedup_20260606/state.toml @@ -129,9 +129,10 @@ test_failure_notes = "Pre-existing failures unrelated to this work: 1) test_api_ # for the original 9-phase plan. These can be picked up in separate # tracks. sub_track_1_phase_6_full = { status = "completed", commit_sha = "253e1798", description = "Bulk ad-hoc thread migration (Phase 6 completion): 15 sites migrated to self.submit_io(...). ZERO new threading.Thread() in src/." } -sub_track_2_audit_violations = { status = "pending", commit_sha = "", description = "Migrate remaining 63 audit violations in src/models.py (tomli_w, pydantic) + sloppy.py (win32gui, src.* indirect). Out of scope for this track." } -sub_track_3_warmup_endpoints = { status = "pending", commit_sha = "", description = "Add dedicated /api/warmup_status and /api/warmup_wait?timeout=N Hook API endpoints + register in _gettable_fields. Builds on Phase 7 minimal (b464d1fe) which only added warmup field to existing diagnostics endpoint." } -sub_track_4_gui_status_toast = { status = "pending", commit_sha = "", description = "GUI status bar indicator + completion toast. Deferred from Phase 7 (T7B). Pure ImGui work in src/gui_2.py + toast system. Polls warmup_status each frame; on completion, queue toast via existing toast system." } +sub_track_2_audit_violations = { status = "partial", commit_sha = "ae3b433e", description = "Migrate 63 audit violations. PARTIAL (1/63 done): tomli_w removed from src/models.py. 62 violations remain: pydantic in models.py, tree_sitter in file_cache.py, websockets/cost_tracker/session_logger in api_hooks.py, 48 in app_controller.py + gui_2.py, 4 in sloppy.py. The remaining violations are large refactors (especially gui_2.py and app_controller.py) that exceed the scope of a single sub-track; addressed as future work." } +sub_track_3_warmup_endpoints = { status = "completed", commit_sha = "8fea8fe9", description = "Add dedicated /api/warmup_status and /api/warmup_wait?timeout=N Hook API endpoints + register in _gettable_fields. Builds on Phase 7 minimal (b464d1fe) which only added warmup field to existing diagnostics endpoint. 7 tests added (5 unit + 2 live_gui), all pass." } +sub_track_4_gui_status_toast = { status = "completed", commit_sha = "f3d071e0", description = "GUI status bar indicator + completion toast. 6 tests added (5 unit + 1 live_gui), all pass. Polls warmup_status each frame; on completion, shows 3s transient 'ready' tag in status_success color. No separate toast window (state transition is the notification)." } +conftest_atexit_fix = { status = "completed", commit_sha = "8957c9a5", description = "Register atexit handler that calls _io_pool.shutdown(wait=False) at process exit. Fixes the run_tests_batched.py hang between batches where ThreadPoolExecutor.__del__ was blocking on shutdown(wait=True) for stuck warmup jobs." } [ad_hoc_threads] # Filled by Phase 6 T6.1 audit and completed in sub-track 1 (253e1798)