conductor(track): create startup_speedup_20260606 track for sloppy.py startup latency

Fulfills the existing backlog entry at conductor/tracks.md:152
(2026-06-05 root-cause analysis of live_gui wait_for_server timeouts).

Main Thread Purity Invariant: the main thread (entering immapp.run())
must never import a module heavier than imgui_bundle and the lean
gui_2 skeleton. Enforced by:
  - static gate: scripts/audit_main_thread_imports.py (CI)
  - runtime hook: tests/test_main_thread_purity.py (sys.addaudithook)

Threading constraint: no new threading.Thread(...) calls in src/.
All background work goes through AppController._io_pool
(ThreadPoolExecutor, max_workers=4, thread_name_prefix='controller-io').

9 phases, 57 tasks: audit+baseline, job pool, lazy-load SDKs, lazy-load
FastAPI, lazy-load feature-gated GUI, migrate ad-hoc threads, runtime
enforcement, hook API + diagnostics, verify+checkpoint.

Expected savings: ~2000-2400ms off main-thread import cost.
Target: import src.ai_client < 50ms (from ~1800ms), live_gui fixtures
no longer time out at wait_for_server(timeout=15).

conductor/tracks/startup_speedup_20260606/plan.md

@@ -0,0 +1,232 @@
+# Plan: Sloppy.py Startup Speedup
+
+**Track:** `startup_speedup_20260606`
+**Spec:** [./spec.md](./spec.md)
+**Status:** In progress
+**Started:** 2026-06-06
+
+---
+
+## Phase 1: Audit + Benchmark + Foundation
+
+- [ ] **T1.1** Capture baseline with `scripts/benchmark_imports.py --runs=3 --color=never > docs/startup_baseline_20260606.txt`
+- [ ] **T1.2** Write `scripts/audit_gui2_imports.py` (AST walker): for each `import X` in `src/gui_2.py`, classify as `first-frame` (reachable from `main()` / `render_main_window` etc.) vs `feature-gated` (inside an `if/elif` branch that requires user action). Commit audit results to `docs/startup_audit_20260606.md`.
+- [ ] **T1.3** Add `src/startup_profiler.py` with `StartupProfiler` class (context manager `phase(name)`). Wire into `AppController.__init__` and `App.__init__` at 8 major init points. (No new test; verify via manual run + diagnostics panel.) `[T1.3]`
+- [ ] **T1.4** Write `scripts/audit_main_thread_imports.py` (static gate, fails CI). AST-walks the import graph reachable from `sloppy.py`, collects all top-level `import X` / `from X import Y`, compares against an allowlist. Exits non-zero with file:line:module on violation. Allowlist: `sys.stdlib_module_names` + the lean gui_2 skeleton list from `spec.md:2.1` (`imgui_bundle`, `defer`, `src.imgui_scopes`, `src.theme_2` (default theme only), `src.theme_models`, `src.paths`, `src.models`, `src.events`).
+- [ ] **T1.5** Commit baseline + audit script: `git add . && git commit -m "conductor(startup): baseline measurements + main thread import audit script"` + git note
+
+**Phase 1 checkpoint:** Baseline established. Static gate exists. All three import classes (first-frame, feature-gated, background-safe) documented.
+
+---
+
+## Phase 2: Job Pool Foundation (the "no new threads" rule)
+
+The user constraint: no new `threading.Thread(...)` per task, per import, per
+ad-hoc job. The codebase gets ONE shared `ThreadPoolExecutor` on `AppController`,
+named `_io_pool`, used by any subsystem that needs background work.
+
+- [ ] **T2.1 (Red)** `tests/test_app_controller_io_pool.py`:
+  - `test_app_controller_has_io_pool`: instantiate `AppController`, assert `hasattr(controller, '_io_pool')` and it's a `ThreadPoolExecutor`
+  - `test_io_pool_uses_named_threads`: submit a job, assert the executing thread name starts with `controller-io`
+  - `test_io_pool_size_is_4`: assert `_io_pool._max_workers == 4`
+  - `test_io_pool_shuts_down_on_close`: call `controller.shutdown()`, assert the pool is shut down
+  - Confirm FAIL (no `_io_pool` yet)
+- [ ] **T2.2 (Green)** In `src/app_controller.py`:
+  - Add `from concurrent.futures import ThreadPoolExecutor` at top
+  - In `__init__`, after the asyncio loop starts and BEFORE the existing HookServer block: `self._io_pool = ThreadPoolExecutor(max_workers=4, thread_name_prefix="controller-io")`
+  - In `shutdown()` (already exists in `App.shutdown` for the GUI; ensure the AppController has a matching shutdown that calls `self._io_pool.shutdown(wait=False)`)
+  - Add `controller.submit_io(fn, *args)` helper: `return self._io_pool.submit(fn, *args)` (with a docstring saying "use this instead of `threading.Thread` for new background work")
+- [ ] **T2.3** Run T2.1 tests; confirm PASS
+- [ ] **T2.4** Commit: `feat(app_controller): add shared _io_pool ThreadPoolExecutor` + git note
+
+**Phase 2 checkpoint:** `AppController` owns a 4-thread named pool. `controller.submit_io(fn)` is the sanctioned way to do background work. Existing ad-hoc threads still exist (will be migrated in Phase 5).
+
+---
+
+## Phase 3: Lazy-load AI Provider SDKs (TDD)
+
+- [ ] **T3.1 (Red)** Write `tests/test_ai_client_lazy_imports.py`:
+  - `test_ai_client_does_not_import_genai_at_module_level`: spawn fresh subprocess, `import src.ai_client`, assert `'google.genai' not in sys.modules` (or `google.genai` in modules but `_gemini_client` is `None`)
+  - `test_ai_client_does_not_import_anthropic_at_module_level`
+  - `test_ai_client_does_not_import_openai_at_module_level`
+  - `test_ai_client_does_not_import_requests_at_module_level`
+  - Confirm tests FAIL (proves the imports are currently eager)
+- [ ] **T3.2 (Green)** In `src/ai_client.py`:
+  - Remove `from google import genai` from top
+  - Remove `import anthropic` from top
+  - Remove `import openai` from top
+  - Remove `import requests` from top
+  - Add lazy imports inside `_send_gemini`, `_send_anthropic`, `_send_deepseek`, `_send_minimax`
+  - Provider client globals stay as `None` until first `_ensure_<provider>_client()` call
+- [ ] **T3.3** Run existing `tests/test_ai_client.py`; fix any breakage. Most likely issue: tests that rely on top-level import side effects need a fixture that triggers lazy init.
+- [ ] **T3.4** Re-run T3.1 tests, confirm PASS
+- [ ] **T3.5** Commit: `git commit -m "refactor(ai_client): lazy-load provider SDKs to defer ~1800ms off main thread"` + git note
+- [ ] **T3.6** Update `conductor/tracks.md` T3 row with SHA
+
+**Phase 3 checkpoint:** `import src.ai_client` < 50ms cold. All 273 existing tests still pass.
+
+---
+
+## Phase 4: Lazy-load FastAPI in HookServer (TDD)
+
+- [ ] **T4.1 (Red)** Write `tests/test_hook_server_lazy_fastapi.py`:
+  - `test_hook_server_does_not_import_fastapi_at_module_level`: subprocess test
+  - `test_hook_server_does_not_import_fastapi_security_at_module_level`
+  - Confirm FAIL
+- [ ] **T4.2 (Green)** In `src/api_hooks.py`:
+  - Remove `from fastapi import ...` from top
+  - Remove `from fastapi.security.api_key import APIKeyHeader` from top
+  - Add lazy imports inside the methods that need them (FastAPI app construction, route registration)
+- [ ] **T4.3** Run existing `tests/test_api_hooks.py`; fix breakage
+- [ ] **T4.4** Confirm T4.1 tests PASS
+- [ ] **T4.5** Commit: `git commit -m "refactor(api_hooks): lazy-load fastapi to defer ~470ms off main thread"` + git note
+
+**Phase 4 checkpoint:** `from src.api_hooks import HookServer` does not import fastapi.
+
+---
+
+## Phase 5: Lazy-load Feature-gated GUI Modules (TDD per module)
+
+### 5A: Command Palette
+
+- [ ] **T5A.1 (Red)** `tests/test_command_palette_lazy.py`: `from src.commands import COMMANDS` (or whatever the eager import is) does not import `src.command_palette`. Confirm FAIL.
+- [ ] **T5A.2 (Green)** In `src/commands.py`: move `from src.command_palette import ...` inside the command functions that open the palette (`_open_command_palette`, `_toggle_command_palette`).
+- [ ] **T5A.3** Run `tests/test_command_palette.py`; fix.
+- [ ] **T5A.4** Commit: `refactor(commands): lazy-load command_palette to defer 244ms`
+
+### 5B: NERV Theme
+
+- [ ] **T5B.1 (Red)** `tests/test_theme_nerv_lazy.py`: `from src.theme_2 import *` (or whatever) does not import `src.theme_nerv` or `src.theme_nerv_fx`. Confirm FAIL.
+- [ ] **T5B.2 (Green)** In `src/theme_2.py`: move `from src.theme_nerv import ...` and `from src.theme_nerv_fx import ...` inside `apply_nerv_theme()` (or whichever function activates the theme).
+- [ ] **T5B.3** Run `tests/test_theme_2.py` and `tests/test_theme_nerv.py`; fix.
+- [ ] **T5B.4** Commit: `refactor(theme): lazy-load nerv theme to defer 485ms off non-nerv path`
+
+### 5C: Markdown Table
+
+- [ ] **T5C.1 (Red)** `tests/test_markdown_helper_lazy.py`: `from src.markdown_helper import MarkdownRenderer` does not import `src.markdown_table`. Confirm FAIL.
+- [ ] **T5C.2 (Green)** In `src/markdown_helper.py`: move `from src.markdown_table import ...` inside the table-detection branch of `render()`.
+- [ ] **T5C.3** Run `tests/test_markdown_helper.py`; fix.
+- [ ] **T5C.4** Commit: `refactor(markdown): lazy-load markdown_table to defer 250ms off non-table markdown`
+
+### 5D: GUI module feature-gated imports
+
+- [ ] **T5D.1** Run `scripts/audit_gui2_imports.py` (built in T1.2); collect list of feature-gated imports in `src/gui_2.py`
+- [ ] **T5D.2** For each feature-gated import, apply the same TDD pattern (5A-5C). Group into 1-2 atomic commits per logical feature.
+- [ ] **T5D.3** Run full GUI test suite; fix.
+- [ ] **T5D.4** Commit per feature group
+
+**Phase 5 checkpoint:** Feature-gated imports are lazy. Default-theme / non-palette / non-table path is lean.
+
+---
+
+## Phase 6: Migrate Ad-hoc Threads to `_io_pool`
+
+The codebase has several ad-hoc `threading.Thread(...)` calls. Per the user
+constraint, these should migrate to `controller.submit_io(fn)`. **This phase
+audits and migrates them, but does NOT add new prefetch threads** (the heavy
+SDKs are lazy-only per spec §2.2 Layer 3).
+
+- [ ] **T6.1** Audit: `grep -rn "threading.Thread(" src/` to find all ad-hoc thread spawns. Document each in `state.toml` (a new `[ad_hoc_threads]` section).
+- [ ] **T6.2** For each ad-hoc thread in `src/log_pruner.py`, `src/project_manager.py`, etc., refactor to use `controller.submit_io(fn)` instead. Wrap the callable body in a try/except (the pool's default behavior is to surface exceptions via the Future; preserve existing error logging).
+- [ ] **T6.3** Run full test suite; fix.
+- [ ] **T6.4** Per-migration commit (or grouped by subsystem if 3+ threads in one file). Final commit: `refactor: migrate ad-hoc threads to AppController._io_pool` + git note.
+
+**Phase 6 checkpoint:** `grep -rn "threading.Thread(" src/` shows ZERO new spawns after this phase (existing project scaffolding threads like `HookServer` and `MMA WorkerPool` are exempt — they're domain-specific).
+
+---
+
+## Phase 7: Enforcement (Runtime Audit Hook)
+
+The static gate (T1.4) catches known imports at audit time. This phase adds
+empirical enforcement: a test that spawns `sloppy.py` and verifies NO heavy
+import happens on the main thread at runtime.
+
+- [ ] **T7.1 (Red)** `tests/test_main_thread_purity.py`:
+  - `test_headless_startup_no_heavy_imports_on_main`: spawn `uv run python sloppy.py --headless --enable-test-hooks` with a `sitecustomize.py` shim that installs `sys.addaudithook` to log every `import` event with the calling thread. The hook writes to a temp file as JSON-L.
+  - Wait for headless server ready (5s timeout via `ApiHookClient`).
+  - Read the audit log. Assert: no event with `thread_name == "MainThread"` for any module in the heavy denylist (`google.genai`, `anthropic`, `openai`, `fastapi`, `requests`, `numpy`, `tkinter`, `psutil`, `pydantic`, `tree_sitter_*`, `src.command_palette`, `src.theme_nerv`, `src.theme_nerv_fx`, `src.markdown_table`, `src.ai_client.send_*`-direct).
+  - Kill subprocess. Confirm FAIL (current state imports these on main).
+- [ ] **T7.2** Once Phase 3-5 land and the static gate passes, this test should start passing. If it doesn't, debug and add more lazy imports.
+- [ ] **T7.3** Wire `test_main_thread_purity.py` into CI as a gating test (it'll be slow, ~10s, so mark with `@pytest.mark.slow` and only run in batched CI).
+- [ ] **T7.4** Commit: `test: empirical main-thread purity check via sys.audit hook` + git note
+
+**Phase 7 checkpoint:** CI fails if a future commit re-introduces a heavy main-thread import.
+
+---
+
+## Phase 8: Hook API + Diagnostics
+
+- [ ] **T8.1** Add `/api/startup_profile` endpoint in `src/api_hooks.py` returning `controller.startup_profiler.snapshot()`
+- [ ] **T8.2** Register `startup_profile` in `_gettable_fields`
+- [ ] **T8.3** Add a "Startup Profile" section to the Diagnostics panel (`src/gui_2.py` `_render_diagnostics` or similar). Show: phase name, duration, % of total.
+- [ ] **T8.4** Add `/api/io_pool_status` endpoint returning `{max_workers, active_threads, queued, completed}` so the user can see the job pool is alive.
+- [ ] **T8.5** Update `docs/guide_api_hooks.md` with both new endpoints.
+- [ ] **T8.6** Tests: extend `tests/test_api_hooks.py` + new `tests/test_startup_profiler.py` + new `tests/test_io_pool_endpoint.py`.
+- [ ] **T8.7** Commit: `feat(diagnostics): expose startup profile and io_pool status via Hook API` + git note
+
+**Phase 8 checkpoint:** User can see per-phase startup cost + job-pool liveness in the GUI.
+
+---
+
+## Phase 9: Verify + Phase Checkpoint
+
+- [ ] **T9.1** Re-run `scripts/benchmark_imports.py --runs=3`. Save to `docs/startup_after_20260606.txt`. Diff against T1.1 baseline; confirm:
+  - `import src.ai_client` < 50ms
+  - `import src.gui_2` < 500ms
+  - `import src.app_controller` < 300ms (includes `_io_pool` creation; should still be < 300ms)
+- [ ] **T9.2** Re-run `scripts/audit_main_thread_imports.py` (T1.4). Confirm exit 0. No violations.
+- [ ] **T9.3** Run `live_gui` test batch (per `conductor/workflow.md:147-150`: max 4 test files per batch, long timeout):
+  - `uv run pytest tests/test_live_gui_*.py --timeout=60 -v` in batches
+  - Confirm `wait_for_server(timeout=15)` does not time out
+- [ ] **T9.4** Manual smoke:
+  - `uv run sloppy.py` (normal mode): time-to-first-frame
+  - `uv run sloppy.py --enable-test-hooks` (test mode): time-to-first-frame
+  - `uv run sloppy.py --headless` (headless): time-to-server-ready
+- [ ] **T9.5** Phase checkpoint commit: `conductor(checkpoint): Phase 9 complete - sloppy.py startup speedup track` + git note with full verification report
+- [ ] **T9.6** Update `conductor/tracks.md`: mark track complete, link to archived folder
+
+**Phase 9 checkpoint:** All verification criteria in `spec.md:6` met.
+
+---
+
+## Definition of Done
+
+- [ ] All Phase 1-9 tasks checked
+- [ ] All tests pass (273+ existing + new TDD tests including `test_main_thread_purity`)
+- [ ] `uv run ruff check .` and `uv run mypy --explicit-package-bases .` clean (per `mma-tier2-tech-lead` skill)
+- [ ] `uv run python scripts/audit_main_thread_imports.py` exits 0
+- [ ] `docs/startup_baseline_20260606.txt` and `docs/startup_after_20260606.txt` archived
+- [ ] Phase 9 git note contains: baseline diff, audit script result, runtime audit hook result, full test batch results, manual smoke timings, file inventory
+- [ ] Track moved to `conductor/tracks/archive/`
+- [ ] **NO new `threading.Thread(...)` calls in `src/`** (verified by `grep -rn "threading.Thread(" src/`)
+
+---
+
+## Notes for Tier 3 Workers
+
+- **Always use 1-space indentation for Python code.** Confirm via `uv run python -c "import ast; ..."` AST check if you do any class-body reorganization (the "Indentation-Driven Class Method Visibility" pitfall in `conductor/workflow.md`).
+- **Test fixtures**: `isolate_workspace`, `reset_paths`, `reset_ai_client`, `vlogger`, `kill_process_tree`, `mock_app`, `live_gui` — see `docs/guide_testing.md`.
+- **Subprocess tests for module-level imports**: spawn `uv run python -c "..."` and inspect `sys.modules` after the import. Pattern:
+  ```python
+  result = subprocess.run(
+      [sys.executable, "-c", "import sys; import src.ai_client; import json; print(json.dumps(sorted(sys.modules.keys())))"],
+      capture_output=True, text=True
+  )
+  assert 'google.genai' not in result.stdout
+  ```
+- **For new background work**: use `controller.submit_io(fn, *args)`, NOT `threading.Thread(target=fn).start()`. The user constraint is "no new threads."
+- **Atomic commits per task.** No batching. If a task touches 3 files, commit all 3 in one commit but the commit message describes the task.
+- **The `_io_pool` is a daemon executor by default in Python 3.9+; non-daemon workers in 3.8.** Check `pyproject.toml` for `requires-python`. Either way, the pool is shut down on `AppController.shutdown()`.
+
+---
+
+## Cross-References
+
+- Spec: [./spec.md](./spec.md)
+- Original backlog entry: `conductor/tracks.md:152`
+- Benchmark tool: `scripts/benchmark_imports.py`
+- Lazy pattern templates: `src/app_controller.py:241-271` (RAG + MMA)
+- Threading constraints: `docs/guide_architecture.md:43-67`
+- Architectural Invariant: `spec.md:2.1`
+- Job pool spec: `spec.md:2.2 Layer 2`
+- Hot reload constraints: `docs/guide_hot_reload.md:295-312`