manual_slop/conductor/tracks/startup_speedup_20260606/spec.md

Track: Sloppy.py Startup Speedup

Status: Active Initialized: 2026-06-06 Owner: Tier 2 Tech Lead Priority: High (regression blocker — live_gui fixtures time out at wait_for_server(timeout=15))

---

1. Problem Statement

uv run sloppy.py --enable-test-hooks startup latency has crept up. live_gui tests time out at wait_for_server(timeout=15). Root cause is too much work on the main thread before immapp.run() returns and the GUI becomes interactive:

• 5 AI provider SDKs (google.genai, anthropic, openai, requests, ...) eagerly imported at src/ai_client.py module top-level, even though only one is the active provider at runtime
• imgui_bundle transitively pulls numpy and 9 other heavy modules at the top of src/gui_2.py and 9 sibling files
• NERV theme, command palette, markdown table extensions are loaded eagerly even though they are feature-gated
• AppController.__init__ does all subsystem construction synchronously on the thread that will become the main GUI thread (path manager, presets, personas, context presets, tool presets, history, workspace, RAG, hook server)

The architecture is already correct: AI calls go through the asyncio worker thread, so the call is non-blocking. The imports are still synchronous on the main thread, and that is what the user sees as "sloppy.py is slow to open."

1.1 Measurement Baseline (from scripts/benchmark_imports.py)

Cold-start subprocess timings, median of 3 runs, 85 unique import paths:

module	time	files	classification
google.genai	~955ms	1	defer (provider SDK, default)
openai	~445ms	1	defer (provider SDK)
anthropic	~430ms	1	defer (provider SDK)
src.markdown_table	~250ms	1	defer (feature-gated)
src.theme_nerv	~245ms	1	defer (feature-gated)
imgui_bundle	~245ms	10	KEEP (ImGui hot path)
src.command_palette	~244ms	1	defer (feature-gated)
src.theme_nerv_fx	~240ms	1	defer (feature-gated)
fastapi (+ security.api_key)	~470ms combined	1	defer (only --enable-test-hooks or web mode)
requests	~92ms	3	defer (deepseek/minimax only)
numpy	~65ms	2	keep (bg_shader; optional in gui_2)
pydantic	~70ms	1	keep (models.py is loaded by everyone)
tree_sitter_*	~25ms each	1	keep (file_cache)

Estimated main-thread import cost today (worst case, all paths): ~2500-3000ms (1.0s SDKs + 1.0s web/fastapi + 0.5s GUI extras + ~0.5s transitives).

Estimated main-thread import cost after this track: ~500-600ms (imgui_bundle + lean gui_2 + pydantic models). Net savings ~2000-2400ms.

---

2. Approach

The architecture is already correct. The fix is systematic application of the lazy-load + shared-job-pool patterns the codebase already uses for RAGEngine (get_rag_engine in src/app_controller.py:244-249) and MultiAgentConductor (get_mma_conductor in src/app_controller.py:266-271).

2.1 Architectural Invariant: Main Thread Purity

The main thread (the one that enters immapp.run()) must NEVER import a module heavier than imgui_bundle and the lean gui_2 skeleton. Every heavy import is loaded by the asyncio worker thread, the AppController's shared job pool, or the MMA WorkerPool. This invariant is enforced by an audit script (CI gate) and a runtime audit-hook test that fails if a heavy import is observed on the main thread at startup.

Concretely, the main thread's import chain is allowed to contain:

• All import X statements transitively reachable from src/gui_2.py whose accumulated import time is < 50ms
• The modules: imgui_bundle, defer, src.imgui_scopes, src.theme_2 (default theme only), src.theme_models, src.paths, src.models, src.events
• Anything in sys.stdlib_module_names

Everything else — provider SDKs, FastAPI, NERV theme, command palette, markdown table extensions, the full src.ai_client provider list, numpy/psutil/ tree_sitter_* if used by lazy code paths — must be loaded by a background mechanism that does not run on the main thread.

2.2 Four layers of protection

Layer 1 — Pure lazy loading (the load-bearing wall, non-negotiable)

Move heavy imports from module top-level into the function body that needs them:

# BEFORE (src/ai_client.py, current)
from google import genai
import anthropic
import openai
# ... 5 provider SDKs loaded unconditionally

# AFTER
def _send_gemini(md_content, user_message, ...):
    from google import genai  # 955ms, paid once, on the first call's thread
    ...

def _send_anthropic(...):
    import anthropic
    ...

Main-thread cost: zero. First call still pays the latency, but it happens on the asyncio worker thread (per guide_architecture.md:215-234), so the GUI never sees it.

Layer 2 — Shared job pool on AppController (no new threads per task)

The codebase already has these dedicated / shared threads:

• AppController._loop_thread — asyncio worker (DEDICATED to the AI event loop, do not use for arbitrary work)
• WorkerPool (in src/multi_agent_conductor.py) — 4-thread pool for MMA workers (DEDICATED to MMA, do not pollute with imports or I/O)
• HookServer thread — DEDICATED to the FastAPI server
• Ad-hoc threading.Thread calls — used for one-off tasks; the user wants to MINIMIZE these

User constraint: no new daemon threads per import prefetch, per I/O task, per log-prune. We add ONE shared ThreadPoolExecutor to AppController named _io_pool, and any subsystem that needs background work submits jobs to it. This includes:

• Initial RAG index warm-up (if applicable)
• Log pruning (currently a one-shot thread — refactor to use the pool)
• Disk-bound subsystem initialization (e.g., TOML re-read on persona switch)
• Any other ad-hoc I/O

# In AppController.__init__
from concurrent.futures import ThreadPoolExecutor

self._io_pool = ThreadPoolExecutor(
    max_workers=4,
    thread_name_prefix="controller-io",
)

Threads created by this track: 4 (the pool). Not 4+1 per job, not 1 per import, not 1 per subsystem. Just 4 long-lived threads that all background work shares. Future work that needs a bg thread should controller._io_pool.submit(fn).

Layer 3 — NO prefetch of the heaviest SDKs (deliberate)

The original Phase 5 of this plan proposed a import-prefetch daemon thread that warms google.genai (~955ms) on a background thread. This has been explicitly rejected for the heavy SDKs, and the reasoning is sound:

• A 955ms import on a background thread holds the GIL for ~10-50ms at a time during C extension init. Each hold stalls the main thread's render loop.
• The user pays 955ms total either way: prefetch = 955ms of background stutter
  • instant first call; lazy-only = 955ms of stutter on the first call only, with the GUI fully interactive in between.
• Prefetching wastes the import cost when the user never uses that provider (e.g., default is Gemini but the user actually only uses Anthropic).

Rule: heavy SDKs (google.genai, anthropic, openai, fastapi) are lazy-only, never prefetched. Lighter modules (themes, command palette, markdown table) MAY be optionally warmed on the _io_pool if profiling shows they're commonly used, but it's not a hard requirement and the default is "don't warm."

Layer 4 — Worker-process isolation (future, out of scope)

The codebase already runs gemini_cli and external MCP servers as subprocesses for this exact reason. A future track could move google.genai / anthropic into their own worker processes, communicating via the existing SyncEventQueue. This track does NOT do this — Layer 1+2+3 is sufficient for the current problem.

2.3 Threading constraints (verified empirically)

The user's question: "if I import in the app controller's thread, will it block the GUI's thread?" The answer is:

Scenario	Blocks GUI?
Module top-level import of heavy X, then main imports X	YES (X's import is in main's chain)
Lazy import of X inside a function called from the asyncio thread	NO (asyncio thread blocks, not main)
Lazy import of X inside a function called from the main thread	YES (first call only; the function caller blocks)
_io_pool worker importing X while main thread renders	NO direct block, but GIL contention causes micro-stutters (~5-50ms each). Acceptable because the pool is capped at 4 threads.
_io_pool worker imports X; main thread later imports X (same module)	YES (main blocks on per-module import lock until worker finishes). This is why Layer 1 must come first.
Spawning a new threading.Thread for each import prefetch	Wasteful (thread creation ~1-5ms each; thread count explodes). Use the _io_pool instead.

This means: Layer 1 is non-negotiable. Even with the _io_pool, if the heavy import is also in the main thread's import chain, the main thread will block on the import lock the moment it tries to use the module. Layer 1 removes the heavy imports from the main thread's chain; Layer 2 reuses threads efficiently; Layer 3 deliberately avoids prefetching the heaviest.

2.4 Enforcement: the "main thread purity" audit

Two enforcement mechanisms, both required:

Static: scripts/audit_main_thread_imports.py (CI gate)

1. AST-walk the import graph reachable from sloppy.py (the main entry). For each .py file in the graph, collect top-level import X and from X import Y statements.

2. Compare against an allowlist of "main-thread-safe" modules (stdlib + imgui_bundle + the lean gui_2 skeleton list from §2.1). Any non-allowlist import is a violation.

3. Exit non-zero with a clear message naming the file, line, and heavy module.

4. Run as part of CI (uv run python scripts/audit_main_thread_imports.py) and as a pre-commit hook.

Runtime: tests/test_main_thread_purity.py (TDD, empirical)

1. Spawn uv run python sloppy.py --headless --enable-test-hooks as a subprocess, with a sys.addaudithook callback that logs every import event with the calling thread.

2. Wait for the headless server to be ready (or 5s timeout).

3. Read the audit log. Assert: every import event with threading.current_thread() is threading.main_thread() was for a module in the allowlist.

4. Kill the subprocess.

This is the empirical enforcement: it proves the invariant holds at runtime, not just at static analysis time.

---

3. Architectural Changes

3.1 Per-file import plan

src/ai_client.py (the biggest win: ~1800ms)

Top-level today: from google import genai, import anthropic, import openai, import requests (used by deepseek/minimax).

After:

• Drop from google import genai from top — lazy in _send_gemini()
• Drop import anthropic from top — lazy in _send_anthropic()
• Drop import openai from top — lazy in _send_deepseek() and _send_minimax()
• Drop import requests from top — lazy in those two providers' HTTP code
• Provider client objects (_gemini_client, _anthropic_client, etc.) stay as module globals but are now None until first use
• The _send_* functions check their provider client is initialized and call a new _ensure_<provider>_client() lazy initializer (extracted from the current top-level logic)

Result: ~1800ms off the main thread. First AI call still pays it, but on the asyncio worker.

src/app_controller.py (FastAPI in headless/web only)

Top-level today: from fastapi import ..., from fastapi.security.api_key import ... (only needed if --enable-test-hooks or --web-host).

After:

• Drop these from top — lazy inside HookServer.__init__ (which is itself lazy in the controller: if enable_test_hooks: from src.api_hooks import HookServer; ...)

Result: ~470ms off the main thread for non-test, non-web launches. Critical because live_gui tests launch with --enable-test-hooks but the FastAPI work can be deferred until the asyncio loop is ready.

src/commands.py and src/command_palette.py (command palette lazy)

Top-level today: from src.command_palette import ... at src/commands.py:1.

After:

• Lazy in each _*_command() function in src/commands.py that actually opens the palette
• The CommandRegistry decorator can keep module-level function references, but the body of the command does the heavy import

Result: ~244ms off if user doesn't open palette during the first session.

src/theme_2.py and src/theme_nerv.py / src/theme_nerv_fx.py (NERV theme lazy)

Top-level today: NERV modules imported at src/theme_2.py module top.

After:

• Lazy in apply_nerv_theme() (the function that activates NERV)
• The default theme path stays lean (uses only src/theme_2.py + src/theme_models.py)

Result: ~485ms off if user doesn't pick NERV theme (the default path).

src/markdown_helper.py (markdown table lazy)

Top-level today: from src.markdown_table import ... at src/markdown_helper.py:1.

After:

• Lazy in _render_table_block() (or wherever GFM table detection happens)
• The first markdown render that hits a table pays the 250ms; subsequent hits are cached in sys.modules

Result: ~250ms off the first markdown render that lacks tables (typical).

src/imgui_scopes.py, src/gui_2.py, src/bg_shader.py (KEEP imgui_bundle)

These MUST keep import imgui_bundle at top — the ImGui render loop is the hot path and needs the module on first frame. There is no way to defer this without breaking the render loop.

What CAN be deferred inside src/gui_2.py:

• import numpy (only needed for bg_shader; the GUI itself doesn't need numpy on the first frame)
• Other feature-gated imports

src/gui_2.py direct heavy imports (audit)

We will use AST to audit which import X statements at src/gui_2.py top-level are reachable from the first-frame render path (render_main_window, render_main_menu_bar, etc.) and which are feature-gated. Feature-gated ones move inside the function that gates them.

3.2 Job pool scaffolding

New code in src/app_controller.py:

from concurrent.futures import ThreadPoolExecutor

# In AppController.__init__, after the asyncio loop starts:
self._io_pool = ThreadPoolExecutor(
 max_workers=4,
 thread_name_prefix="controller-io",
)

def submit_io(self, fn, *args, **kwargs):
    """Submit a background job to the shared I/O pool. Use this instead of
    threading.Thread for new background work.
    
    Returns a concurrent.futures.Future. Caller can .result() if they need
    to block, or .add_done_callback for fire-and-forget with error handling.
    """
    return self._io_pool.submit(fn, *args, **kwargs)

In AppController.shutdown() (or wherever lifecycle cleanup lives): self._io_pool.shutdown(wait=False). Non-blocking because the pool's workers are daemon threads and will die with the process anyway.

3.3 Startup timing instrumentation

Add src/startup_profiler.py:

class StartupProfiler:
    """Records wall-clock time spent in each named init phase.
    
    Cheap (no I/O). Stored on AppController.startup_profile for later inspection
    via the Hook API (`GET /api/startup_profile`) and the Diagnostics panel.
    """
    _phases: list[tuple[str, float, float]]  # (name, start, duration_ms)
    
    @contextmanager
    def phase(self, name: str) -> Iterator[None]:
        t0 = time.perf_counter()
        yield
        self._phases.append((name, t0, (time.perf_counter() - t0) * 1000))

Used at every major init step in AppController.__init__ and App.__init__.

---

4. Phases

Phase 1: Audit + Benchmark + Foundation (Day 1)

• T1.1: Run scripts/benchmark_imports.py and capture baseline
• T1.2: AST-audit every import X in src/*.py to map which is reachable from the first-frame render path vs feature-gated
• T1.3: Add StartupProfiler to src/app_controller.py and instrument current init
• T1.4: Add scripts/audit_main_thread_imports.py (static gate)
• T1.5: Commit baseline + audit script

Phase 2: Job Pool Foundation (Day 1) — the "no new threads" rule

• T2.1 (TDD Red): Write tests/test_app_controller_io_pool.py asserting AppController has a _io_pool: ThreadPoolExecutor with 4 workers, named controller-io-*
• T2.2 (Green): Add self._io_pool = ThreadPoolExecutor(max_workers=4, thread_name_prefix="controller-io") to AppController.__init__. Add submit_io(fn, *args) helper. Wire shutdown into controller.shutdown().
• T2.3: Verify T2.1 tests pass + full suite still passes

Phase 3: Lazy-load AI provider SDKs (Day 2)

• T3.1 (TDD Red): Write tests/test_ai_client_lazy_imports.py asserting import src.ai_client does NOT import any provider SDK
• T3.2 (Green): Move from google import genai / import anthropic / import openai / import requests into their respective _send_* functions
• T3.3: Verify existing tests/test_ai_client.py still passes
• T3.4: Commit, re-run benchmark, expect import src.ai_client < 50ms

Phase 4: Lazy-load FastAPI in HookServer (Day 2)

• T4.1 (TDD Red): Write tests/test_hook_server_lazy_fastapi.py asserting from src.api_hooks import HookServer does NOT import fastapi
• T4.2 (Green): Move from fastapi import ... inside the methods that need them
• T4.3: Verify existing tests/test_api_hooks.py still passes
• T4.4: Commit

Phase 5: Lazy-load feature-gated GUI modules (Day 3)

• T5.1: Lazy-load src.command_palette in src/commands.py
• T5.2: Lazy-load src.theme_nerv and src.theme_nerv_fx in src/theme_2.py
• T5.3: Lazy-load src.markdown_table in src/markdown_helper.py
• T5.4: Audit and lazy-load feature-gated imports in src/gui_2.py
• T5.5: Run all GUI tests; fix any circular imports
• T5.6: Commit per task

Phase 6: Migrate ad-hoc threads to _io_pool (Day 4)

• T6.1: Audit: grep -rn "threading.Thread(" src/ to find all ad-hoc thread spawns (excluding HookServer and WorkerPool which are domain-specific)
• T6.2: Refactor each ad-hoc thread to use controller.submit_io(fn) instead
• T6.3: Per-migration commit
• T6.4: Final grep -rn "threading.Thread(" src/ shows ZERO new spawns (the grep result should be identical to the T6.1 audit list, no new entries)

Phase 7: Enforcement — Runtime Audit Hook (Day 4)

• T7.1 (TDD Red): tests/test_main_thread_purity.py — spawn sloppy.py --headless --enable-test-hooks with a sys.addaudithook shim, verify no heavy import happens on the main thread
• T7.2: Once Phase 3-5 land, this test should start passing. Wire into CI.
• T7.3: Commit

Phase 8: Hook API + Diagnostics (Day 5)

• T8.1: Add /api/startup_profile endpoint
• T8.2: Add /api/io_pool_status endpoint
• T8.3: Add to _gettable_fields and the Diagnostics panel
• T8.4: Document in docs/guide_api_hooks.md
• T8.5: Tests + commit

Phase 9: Verify + Checkpoint (Day 5)

• T9.1: Re-run scripts/benchmark_imports.py; confirm import src.gui_2 and import src.ai_client are now < 100ms each
• T9.2: Re-run scripts/audit_main_thread_imports.py; exit 0
• T9.3: Run tests/test_main_thread_purity.py; pass
• T9.4: Run full live_gui test batch; wait_for_server(timeout=15) no longer times out
• T9.5: Manual smoke test: uv run sloppy.py and uv run sloppy.py --enable-test-hooks both feel snappier
• T9.6: Phase checkpoint commit with full verification report

---

5. Risks and Mitigations

Risk	Likelihood	Impact	Mitigation
Lazy import inside a hot path adds latency on every call	Med	Med	Always gate the import with sys.modules check OR use module-level sentinel
First AI call on the asyncio thread blocks for ~955ms while google.genai imports	High	Low	The user already paid this latency budget; happens on the asyncio worker, not main. Document the expected first-call pause.
Lazy import surfaces circular import that was hidden by top-level ordering	Med	Med	Phase 1 audit catches this; defer each lazy import to the test phase
Test fixtures import the heavy module before main code, breaking assumptions	Low	Low	reset_ai_client and isolate_workspace fixtures already lazy-reset
Hot reload of a now-lazy module doesn't trigger	Low	Med	Update HotReloader.HOT_MODULES to register the lazy module's gate function
_io_pool worker importing a heavy module holds GIL and stutters GUI	Med	Low	The pool is capped at 4 threads; stutter is bounded; user sees responsive UI before any stutter
A future commit re-introduces a heavy import on the main thread	Med	High	Static gate (audit_main_thread_imports.py, CI) + runtime audit hook (test_main_thread_purity.py) catch this

Hot Reload consideration

src/hot_reloader.py registers modules at import time. Lazy-loaded modules (imported inside functions) are NOT registered. The hot-reload workflow needs:

• Either: register the lazy module with a callback that forces a re-import via importlib.reload
• Or: explicitly trigger the lazy import on hot-reload trigger

This is a small follow-up task; the lazy import itself doesn't break hot reload (it just means you have to invoke the gate function once to materialize the module before reload can take effect).

---

6. Verification Criteria

The track is complete when:

• import src.ai_client cold start < 50ms (down from ~1800ms)
• import src.gui_2 cold start < 500ms (down from ~3000ms)
• import src.app_controller cold start < 300ms (down from ~700ms)
• uv run sloppy.py --enable-test-hooks reaches immapp.run() in < 1.5s
• live_gui.wait_for_server(timeout=15) passes for all 273+ tests
• scripts/audit_main_thread_imports.py exits 0 (no heavy imports on main)
• tests/test_main_thread_purity.py passes (runtime audit hook confirms invariant)
• scripts/benchmark_imports.py shows no new red entries in the top-20
• First AI call latency on the asyncio thread is < 1500ms (pays the SDK load once, then the user has a snappy first call forever after). Main thread sees ZERO of this cost.
• No regressions in the existing 272/273 passing tests
• grep -rn "threading.Thread(" src/ shows ZERO new spawns after Phase 6 migration (only the existing project scaffolding threads like HookServer and WorkerPool remain, and they're domain-specific)
• Startup profile + io_pool status visible in /api/startup_profile, /api/io_pool_status, and the Diagnostics panel

---

7. Out of Scope

• Process-isolation of heavy SDKs (Layer 4 in §2.2) — future track
• imgui_bundle lazy loading — fundamentally impossible (ImGui hot path)
• Importing on the main thread for the lean gui_2 skeleton (~300ms unavoidable)
• pydantic lazy loading (used by src/models.py which is imported by 16 files; the cost is already amortized and deferring it would cascade)
• Prefetch / warm-up of the heavy SDKs in the background (Layer 3 in §2.2 is deliberately the "do nothing" layer; the user pays the import cost once on first use, on the asyncio thread, not in the background)

---

8. Cross-References

• conductor/tracks.md line 152 — original backlog entry that this track fulfills
• docs/guide_architecture.md:43-67 — thread domains (asyncio worker is the right place for heavy work)
• docs/guide_architecture.md:880-898 — Architectural Invariants (single-writer principle; this track respects it)
• docs/guide_app_controller.md:241-271 — existing get_rag_engine / get_mma_conductor lazy patterns (the templates this track replicates)
• docs/guide_hot_reload.md:295-312 — what is/isn't safe to hot-reload (lazy-loaded modules need a small follow-up)
• conductor/workflow.md — TDD Red-Green-Refactor protocol + atomic per-task commits + git notes
• scripts/benchmark_imports.py — the measurement tool built in this conversation