ed/manual_slop
Private
Public Access
0
0
Fork 0
Code Activity

conductor(plan): flip track from lazy-loading to proactive warmup

Browse Source 
Architectural shift driven by user clarification: lazy-loading on first
use causes user-perceptible lag when the user-triggered action (e.g.
provider switch) propagates to a controller method that triggers the
first import. The fix is to pre-import heavy modules on a bg thread
at startup and have functions access them via _require_warmed().

Old design (rejected):
  - from google import genai inside _send_gemini (lazy on first call)
  - First user action that triggers this pays the cost; UI feels laggy

New design (this commit):
  - Top-level heavy imports REMOVED from main-thread-reachable files
  - AppController.__init__ submits warmup jobs to _io_pool (4 threads,
    named 'controller-io-N')
  - Each warmup worker imports its module and updates a thread-safe
    warmup_status dict
  - Functions access modules via _require_warmed(name), which assumes
    the module is in sys.modules (warmed at startup)
  - When all jobs complete, _warmup_done_event is set and registered
    on_warmup_complete callbacks fire
  - GUI shows status indicator + toast when warmup completes
  - Hook API exposes /api/warmup_status and /api/warmup_wait
  - Tests can call controller.wait_for_warmup() before exercising
    warmup-dependent functionality

Phase 2 now bundles job pool + warmup (T2.3+T2.4 add warmup tests +
implementation). Phases 3-5 do 'remove top-level imports' instead of
'lazy-load'. Phase 7 is the notification surface (Hook API + GUI).
Definition of Done includes warmup-completion criteria, the
'no function-body imports' check, and an end-to-end 'provider switch
is INSTANT' smoke test.

No code changes; this is a planning update only.
This commit is contained in:
ed
2026-06-06 13:45:05 -04:00
parent ca254bac41
commit f2f5ee1197
4 changed files with 597 additions and 269 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conductor/tracks/startup_speedup_20260606/state.toml
+64 -44
View File
@@ -10,13 +10,13 @@ last_updated = "2026-06-06"
[phases]
phase_1 = { status = "in_progress", checkpoint_sha = "", name = "Audit + Benchmark + Foundation" }
phase_2 = { status = "pending", checkpoint_sha = "", name = "Job Pool Foundation (no new threads)" }
phase_3 = { status = "pending", checkpoint_sha = "", name = "Lazy-load AI provider SDKs" }
phase_4 = { status = "pending", checkpoint_sha = "", name = "Lazy-load FastAPI in HookServer" }
phase_5 = { status = "pending", checkpoint_sha = "", name = "Lazy-load feature-gated GUI modules" }
phase_2 = { status = "pending", checkpoint_sha = "", name = "Job Pool + Warmup Foundation" }
phase_3 = { status = "pending", checkpoint_sha = "", name = "Remove top-level SDK imports (ai_client)" }
phase_4 = { status = "pending", checkpoint_sha = "", name = "Remove top-level FastAPI imports" }
phase_5 = { status = "pending", checkpoint_sha = "", name = "Remove top-level feature-gated GUI imports" }
phase_6 = { status = "pending", checkpoint_sha = "", name = "Migrate ad-hoc threads to _io_pool" }
phase_7 = { status = "pending", checkpoint_sha = "", name = "Enforcement: runtime audit hook" }
phase_8 = { status = "pending", checkpoint_sha = "", name = "Hook API + Diagnostics" }
phase_7 = { status = "pending", checkpoint_sha = "", name = "Warmup Notification (Hook API + GUI)" }
phase_8 = { status = "pending", checkpoint_sha = "", name = "Enforcement: runtime audit hook" }
phase_9 = { status = "pending", checkpoint_sha = "", name = "Verify + Checkpoint" }
[tasks]
@@ -26,42 +26,41 @@ t1_2 = { status = "pending", commit_sha = "", description = "Audit src/gui_2.py
t1_3 = { status = "pending", commit_sha = "", description = "Add StartupProfiler and instrument init" }
t1_4 = { status = "pending", commit_sha = "", description = "Write scripts/audit_main_thread_imports.py (static CI gate)" }
t1_5 = { status = "pending", commit_sha = "", description = "Commit baseline + audit script" }
# Phase 2: Job Pool Foundation
# Phase 2: Job Pool + Warmup Foundation
t2_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_app_controller_io_pool.py" }
t2_2 = { status = "pending", commit_sha = "", description = "Green: add _io_pool ThreadPoolExecutor + submit_io helper to AppController" }
t2_3 = { status = "pending", commit_sha = "", description = "Confirm T2.1 tests pass + full suite still passes" }
t2_4 = { status = "pending", commit_sha = "", description = "Commit T2" }
# Phase 3: Lazy-load AI Provider SDKs
t3_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_ai_client_lazy_imports.py" }
t3_2 = { status = "pending", commit_sha = "", description = "Green: move provider SDK imports into _send_* funcs" }
t2_2 = { status = "pending", commit_sha = "", description = "Green: add _io_pool ThreadPoolExecutor to AppController" }
t2_3 = { status = "pending", commit_sha = "", description = "Red: tests/test_warmup_mechanism.py" }
t2_4 = { status = "pending", commit_sha = "", description = "Green: implement _submit_warmup_jobs, _compute_warmup_list, _warmup_one, warmup_status, is_warmup_done, wait_for_warmup, on_warmup_complete" }
t2_5 = { status = "pending", commit_sha = "", description = "Confirm T2.1 + T2.3 tests pass" }
t2_6 = { status = "pending", commit_sha = "", description = "Commit T2" }
# Phase 3: Remove top-level SDK imports
t3_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_ai_client_no_top_level_sdk_imports.py" }
t3_2 = { status = "pending", commit_sha = "", description = "Green: remove top-level SDK imports from src/ai_client.py; add _require_warmed helper; update _send_* to use it" }
t3_3 = { status = "pending", commit_sha = "", description = "Fix existing test_ai_client.py breakage" }
t3_4 = { status = "pending", commit_sha = "", description = "Confirm T3.1 tests PASS" }
t3_5 = { status = "pending", commit_sha = "", description = "Commit T3" }
t3_6 = { status = "pending", commit_sha = "", description = "Update tracks.md T3 row" }
# Phase 4: Lazy-load FastAPI
t4_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_hook_server_lazy_fastapi.py" }
t4_2 = { status = "pending", commit_sha = "", description = "Green: move fastapi imports into HookServer methods" }
# Phase 4: Remove top-level FastAPI imports
t4_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_hook_server_no_top_level_fastapi.py" }
t4_2 = { status = "pending", commit_sha = "", description = "Green: remove fastapi imports from src/api_hooks.py; use _require_warmed" }
t4_3 = { status = "pending", commit_sha = "", description = "Fix existing test_api_hooks.py breakage" }
t4_4 = { status = "pending", commit_sha = "", description = "Confirm T4.1 tests PASS" }
t4_5 = { status = "pending", commit_sha = "", description = "Commit T4" }
# Phase 5A: Command Palette
t5a_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_command_palette_lazy.py" }
t5a_2 = { status = "pending", commit_sha = "", description = "Green: lazy-load in src/commands.py" }
# Phase 5: Remove top-level feature-gated GUI imports
t5a_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_command_palette_no_top_level_import.py" }
t5a_2 = { status = "pending", commit_sha = "", description = "Green: remove from src/commands.py; use _require_warmed" }
t5a_3 = { status = "pending", commit_sha = "", description = "Fix existing test_command_palette.py" }
t5a_4 = { status = "pending", commit_sha = "", description = "Commit T5A" }
# Phase 5B: NERV Theme
t5b_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_theme_nerv_lazy.py" }
t5b_2 = { status = "pending", commit_sha = "", description = "Green: lazy-load in src/theme_2.py" }
t5b_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_theme_nerv_no_top_level_import.py" }
t5b_2 = { status = "pending", commit_sha = "", description = "Green: remove from src/theme_2.py; use _require_warmed" }
t5b_3 = { status = "pending", commit_sha = "", description = "Fix existing test_theme_2.py + test_theme_nerv.py" }
t5b_4 = { status = "pending", commit_sha = "", description = "Commit T5B" }
# Phase 5C: Markdown Table
t5c_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_markdown_helper_lazy.py" }
t5c_2 = { status = "pending", commit_sha = "", description = "Green: lazy-load in src/markdown_helper.py" }
t5c_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_markdown_helper_no_top_level_import.py" }
t5c_2 = { status = "pending", commit_sha = "", description = "Green: remove from src/markdown_helper.py; use _require_warmed" }
t5c_3 = { status = "pending", commit_sha = "", description = "Fix existing test_markdown_helper.py" }
t5c_4 = { status = "pending", commit_sha = "", description = "Commit T5C" }
# Phase 5D: gui_2 feature-gated imports
t5d_1 = { status = "pending", commit_sha = "", description = "Run audit_gui2_imports.py and collect feature-gated list" }
t5d_2 = { status = "pending", commit_sha = "", description = "Apply TDD pattern per feature-gated import" }
t5d_2 = { status = "pending", commit_sha = "", description = "Apply same pattern per feature-gated import in src/gui_2.py; numpy in bg_shader" }
t5d_3 = { status = "pending", commit_sha = "", description = "Run full GUI test suite; fix" }
t5d_4 = { status = "pending", commit_sha = "", description = "Commit per feature group" }
# Phase 6: Migrate ad-hoc threads
@@ -69,26 +68,29 @@ t6_1 = { status = "pending", commit_sha = "", description = "Audit threading.Thr
t6_2 = { status = "pending", commit_sha = "", description = "Refactor each ad-hoc thread to use controller.submit_io" }
t6_3 = { status = "pending", commit_sha = "", description = "Run full test suite; fix" }
t6_4 = { status = "pending", commit_sha = "", description = "Commit per migration; final grep shows zero new spawns" }
# Phase 7: Enforcement - Runtime Audit Hook
t7_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_main_thread_purity.py" }
t7_2 = { status = "pending", commit_sha = "", description = "Confirm test passes after Phase 3-5" }
t7_3 = { status = "pending", commit_sha = "", description = "Wire into CI as @pytest.mark.slow gating test" }
t7_4 = { status = "pending", commit_sha = "", description = "Commit T7" }
# Phase 8: Hook API + Diagnostics
t8_1 = { status = "pending", commit_sha = "", description = "Add /api/startup_profile endpoint" }
t8_2 = { status = "pending", commit_sha = "", description = "Add /api/io_pool_status endpoint" }
t8_3 = { status = "pending", commit_sha = "", description = "Add startup profile + io_pool status to Diagnostics panel" }
t8_4 = { status = "pending", commit_sha = "", description = "Update docs/guide_api_hooks.md" }
t8_5 = { status = "pending", commit_sha = "", description = "Tests for endpoints + profiler round-trip" }
t8_6 = { status = "pending", commit_sha = "", description = "Commit T8" }
# Phase 7: Warmup Notification (Hook API + GUI)
t7a_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_api_hooks_warmup.py" }
t7a_2 = { status = "pending", commit_sha = "", description = "Green: add /api/warmup_status and /api/warmup_wait endpoints" }
t7a_3 = { status = "pending", commit_sha = "", description = "Register warmup_status in _gettable_fields" }
t7a_4 = { status = "pending", commit_sha = "", description = "Commit T7A" }
t7b_1 = { status = "pending", commit_sha = "", description = "Add status-bar indicator in src/gui_2.py that polls warmup_status each frame" }
t7b_2 = { status = "pending", commit_sha = "", description = "Register on_warmup_complete callback that shows toast on success" }
t7b_3 = { status = "pending", commit_sha = "", description = "Update docs for status bar + hook API" }
t7b_4 = { status = "pending", commit_sha = "", description = "Commit T7B" }
# Phase 8: Enforcement - Runtime Audit Hook
t8_1 = { status = "pending", commit_sha = "", description = "Red: tests/test_main_thread_purity.py" }
t8_2 = { status = "pending", commit_sha = "", description = "Confirm test passes after Phase 3-5" }
t8_3 = { status = "pending", commit_sha = "", description = "Wire into CI as @pytest.mark.slow gating test" }
t8_4 = { status = "pending", commit_sha = "", description = "Commit T8" }
# Phase 9: Verify + Checkpoint
t9_1 = { status = "pending", commit_sha = "", description = "Re-run benchmark; diff vs baseline" }
t9_2 = { status = "pending", commit_sha = "", description = "Re-run audit_main_thread_imports.py; exit 0" }
t9_3 = { status = "pending", commit_sha = "", description = "Run test_main_thread_purity.py; pass" }
t9_4 = { status = "pending", commit_sha = "", description = "Run live_gui test batch; confirm wait_for_server passes" }
t9_5 = { status = "pending", commit_sha = "", description = "Manual smoke (normal, test-hooks, headless modes)" }
t9_6 = { status = "pending", commit_sha = "", description = "Phase checkpoint commit + git note" }
t9_7 = { status = "pending", commit_sha = "", description = "Update tracks.md; archive track" }
t9_3 = { status = "pending", commit_sha = "", description = "Run test_warmup_mechanism.py; warmup completes and notifications fire" }
t9_4 = { status = "pending", commit_sha = "", description = "Run test_main_thread_purity.py; pass" }
t9_5 = { status = "pending", commit_sha = "", description = "Run live_gui test batch; confirm wait_for_server passes" }
t9_6 = { status = "pending", commit_sha = "", description = "Manual smoke: provider-switch is INSTANT after warmup" }
t9_7 = { status = "pending", commit_sha = "", description = "Phase checkpoint commit + git note" }
t9_8 = { status = "pending", commit_sha = "", description = "Update tracks.md; archive track" }
[verification]
# To be filled at Phase 9
@@ -98,13 +100,31 @@ baseline_gui_2_ms = 0
after_gui_2_ms = 0
baseline_app_controller_ms = 0
after_app_controller_ms = 0
warmup_completes_within_seconds = 0
warmup_modules_in_sys_modules = 0
provider_switch_latency_ms_after_warmup = 0
live_gui_passed = 0
live_gui_failed = 0
audit_main_thread_violations = 0
io_pool_max_workers = 4
io_pool_thread_name_prefix = "controller-io"
new_threading_thread_calls = 0
function_body_heavy_imports = 0
[ad_hoc_threads]
# Filled in Phase 6 T6.1 audit
# Format: {file = "src/foo.py", line = 42, current_target = "lambda", proposed_target = "controller.submit_io(...)"}
[warmup_list]
# Filled in Phase 2 T2.4 implementation
google_genai = true
anthropic = true
openai = true
requests = true
src_command_palette = true
src_theme_nerv = true
src_theme_nerv_fx = true
src_markdown_table = true
numpy = true
fastapi = "conditional" # only when enable_test_hooks or web_host
fastapi_security_api_key = "conditional"