Sub-track 2C: 4 violations cleared. Removed 4 top-level imports (websockets, websockets.asyncio.server.serve, src.cost_tracker, src.session_logger). Runtime access via _require_warmed() at 4 use sites (L107 session_logger GET, L311 cost_tracker.estimate_cost, L412 session_logger POST, L855 websockets.exceptions.ConnectionClosed, L871 websockets.asyncio.server.serve). File already had 'from __future__ import annotations' so type hints (WebSocketServer) are strings.
ALSO: Added 'src.module_loader' to LEAN_ALLOWLIST in scripts/audit_main_thread_imports.py. The module is a 59-line pure-stdlib helper (only importlib + sys + typing imports); allowing its import at top level is consistent with the existing 'src.paths' / 'src.models' / 'src.config' allowlist entries.
Tests: 3 new in tests/test_api_hooks_no_top_level_heavy.py; 14 existing in test_websocket_server.py + test_hooks.py + test_api_hooks_warmup.py. All 17 pass.
GOTCHA: First edit attempt on src/api_hooks.py imports section failed because I forgot to include the '# TODO(Ed): Eliminate these?' comment line in old_string. Re-anchored on the exact 17-line block including the comment. (User will note: I also used the native 'edit' tool on the test file this turn, which the workflow says destroys 1-space indentation. Switched to manual-slop_edit_file.)
Design for removing 30 confirmed-unused one-off scripts from
scripts/. Net effect: scripts/ shrinks from 56 -> 26 files
(54% reduction). All deletions are hard deletes via 5 atomic
per-category commits; git log is the restore path.
26 KEEPS documented by category (CI gates, MMA, MCP, test runner,
ImGui linter, audit/scaffolding, tool-call bridge, Docker, borderline
utility). 30 DELETES grouped by category: one-shot indent fixers
(10), one-shot transform scripts (6), superseded entropy audits (4),
one-shot migrators/repros (6), tool-call aliases and legacy tool
discovery (4).
No new CI gate added. Follow-up unused_scripts_audit_20260607
recorded in the spec. Plan (writing-plans) will produce 5 phases
(one per category).
Sub-track 2B: 4 violations cleared. Added 'from __future__ import annotations' + TYPE_CHECKING import for tree_sitter/tree_sitter_python/tree_sitter_cpp/tree_sitter_c. Runtime access via _require_warmed() in ASTParser.__init__. 6 new tests in tests/test_file_cache_no_top_level_tree_sitter.py. All 25 tests pass (6 new + 19 existing).
Sub-track 2B: 4 violations cleared. Added 'from __future__ import annotations' + TYPE_CHECKING import for tree_sitter/tree_sitter_python/tree_sitter_cpp/tree_sitter_c. Runtime access via _require_warmed() in ASTParser.__init__. 6 new tests in tests/test_file_cache_no_top_level_tree_sitter.py. All 25 tests pass (6 new + 19 existing).
run_tests_batched.py hangs at the end of a batch when the pytest
subprocess fails to exit cleanly. Two hang chains have been observed:
1. ThreadPoolExecutor.__del__ -> shutdown(wait=True) joining a
blocked worker during interpreter finalization
(concurrent.futures._python_exit, pool __del__, etc.).
2. The session-scoped \live_gui\ fixture teardown hanging in
client.reset_session() (HTTP call to hook server) or
kill_process_tree(process.pid) / process.wait(timeout=2)
(waiting for the sloppy.py subprocess to die on Windows).
A previous atexit-based fix (commit 8957c9a5) attempted to preempt
chain #1, but verified empirically that atexit handlers do NOT fire
at all when a pool worker is blocked in user code (see
src/io_pool.py module docstring for the full analysis). The
atexit-based fix is therefore ineffective, and was removed from
the conftest in this commit.
Solution: a daemon-thread watchdog that unconditionally calls
os._exit(0) after 30s. If pytest exits cleanly first, the thread
is killed when the process tears down (daemon=True). If pytest
hangs, the watchdog kicks in and the batched runner can move to
the next batch. Same pattern as
src/app_controller.py:_install_sigint_exit_handler (the production
Ctrl+C fix); the difference is the trigger (time-based vs. SIGINT).
Files:
- tests/conftest.py: replaced the ineffective atexit-based fix
with the daemon-thread watchdog. Header comment documents both
hang chains and explains why atexit was abandoned.
- tests/test_conftest_watchdog.py: 3 static regression tests that
verify the watchdog is registered as a daemon thread with a
timeout in the 25-35s range. Static checks (not subprocess) so
the test itself isn't recursively bound by the watchdog.
Sub-track 2A of startup_speedup_20260606: clears 1 of 61 main-thread audit violations (pydantic in src/models.py).
Removed top-level 'from pydantic import BaseModel' (line 50) and the two static class definitions (GenerateRequest, ConfirmRequest). Replaced with PEP 562 module-level __getattr__ that materializes the pydantic classes on first access via pydantic.create_model() + _require_warmed('pydantic').
Pattern matches the lazy-proxy convention from sub-tracks 5A (command_palette), 5B (theme_nerv), 5C (markdown_table), 5D (gui_2 dead imports).
Result:
- pydantic NOT in sys.modules after 'import src.models' (verified via subprocess test)
- GenerateRequest and ConfirmRequest are accessible via 'from src.models import X' (proxy triggers pydantic import + caches class in globals())
- Pydantic validation works: GenerateRequest() raises ValidationError on missing 'prompt'
- Audit script: 60 violations (was 61)
- Existing test_project_switch_persona_preset.py: 8/9 pass; the 1 failure is the pre-existing ui_global_preset_name issue (unrelated)
Files changed:
- src/models.py: removed 1 import, 2 class defs; added 2 factory fns + 1 __getattr__
- tests/test_models_no_top_level_pydantic.py: new (7 tests; all pass)
Per user instruction, all implementation work is performed by the Tier 2 tech lead directly. The 'sub-track 2A' naming follows the sub-track 2 (audit violations) parent in the track plan.
Phase 9 was shipped at 12cec6ae and the 9-phase core plan is done, but the [COMPLETE 2026-06-07] tag was applied prematurely. Sub-track 2 (audit violations) remains partial at ae3b433e with 61 violations remaining: pydantic in models.py (1), tree_sitter in file_cache.py (4), api_hooks.py (4), sloppy.py (5), app_controller.py (23), gui_2.py (24). Reopening the track to finish sub-track 2 in 6 per-file sub-tracks (2A-2F).
Bug: on Python installs where the tkinter package imports but the
filedialog sub-module fails to load (e.g., missing Tcl/Tk runtime,
embedded Python), every call to filedialog.askopenfilename raised
'AttributeError: module tkinter has no attribute filedialog' at the
frame the Project Settings window's 'Add Project' button was clicked.
Fix: _LazyModule._resolve() now catches AttributeError on the
getattr() attempt, falls back to importlib.import_module('tkinter.filedialog')
(which surfaces the real ImportError cleanly), and finally falls back
to a new _FiledialogStub class that exposes askopenfilename,
askopenfilenames, askdirectory, asksaveasfilename returning safe
empty sentinels (str and tuple). The stub sets available=False so
future UI can detect it and offer an ImGui-based path input.
Tests:
- tests/test_lazymodule_filedialog_fallback.py: 5 unit tests using
a deliberately-missing sub-module to deterministically exercise
the fallback path on any Python install
- tests/test_live_gui_filedialog_regression.py: live_gui smoke test
that opens the Project Settings window via the Hook API and
asserts no AttributeError in the running app's log
Ctrl+C in sloppy.py's terminal would hang the process when a worker of
the shared 4-thread I/O pool was mid-task in user code (e.g. a long-
running Gemini/Anthropic HTTP request). The hang chain:
1. SIGINT delivered to main thread
2. Python raises KeyboardInterrupt (default handler)
3. Exception propagates out of main()
4. Interpreter finalization begins
5. ThreadPoolExecutor.__del__ runs shutdown(wait=True)
6. shutdown(wait=True) joins all worker threads
7. The blocked worker never returns -> hang
An atexit-based fix (mirroring the conftest fix at 8957c9a5) was
attempted first: register pool.shutdown(wait=False) at pool creation.
Verified empirically that this DOES NOT WORK — atexit handlers do not
fire at all when a pool worker is blocked in user code. The hang still
occurs in ThreadPoolExecutor.__del__ -> shutdown(wait=True).
Production fix: a SIGINT handler installed by AppController.__init__
that drains the pool non-blockingly and calls os._exit(0), bypassing
the broken finalization chain. One wire covers all three modes
(GUI/headless/web) since they all create an AppController.
Files:
- src/app_controller.py: new module-level _install_sigint_exit_handler
helper called from __init__; one-line docstring at the function
level documents the rationale.
- tests/test_app_controller_sigint.py: new test file with 2 regression
tests (unit: handler is installed on main thread; subprocess: handler
exits within 2s when invoked with a blocked worker).
- tests/test_io_pool.py: module docstring updated to explain the
reverted atexit approach and point readers at the production fix.
Best-effort: signal.signal may fail on non-main threads (some conftest
warmup paths); failure is swallowed. The conftest's own atexit fix at
8957c9a5 covers the test fixture's normal-exit path.
Mid-session expansion that was left dirty. Adds 3 main-thread phase
markers so the timeline answers 'which phase dominated' instead of
just 'how long total':
New attrs (all Optional[float], stamped lazily):
- _appcontroller_init_done_ts: set by mark_gui_run_started() on its
first call (post-init, pre-anything)
- _gui_run_started_ts: set by mark_gui_run_started() at the start of
App.run() (pre-imgui-bundle C++ init)
New property:
- cold_start_ts: reads sloppy._SLOPPY_COLD_START_TS so the timeline
covers from Python-start to first-frame, not just AppController-init
to first-frame (the gap is the main-thread module import chain)
New method:
- mark_gui_run_started(ts=None): called by App.run() before the
imgui bundle setup. Idempotent (safe to call multiple times).
Lazily captures _appcontroller_init_done_ts on first call.
startup_timeline() now exposes 4 new precomputed deltas:
- appcontroller_init_ms: init → AppController done
- gui_setup_ms: AppController done → gui_run_started (imgui init)
- first_render_ms: gui_run_started → first frame
- module_imports_ms: cold_start → init_start
- cold_start_to_first_frame_ms: full Python-start → first-frame
mark_first_frame_rendered() now also logs the 3-phase breakdown in
the stderr line, e.g.:
[startup] first frame at 1830.2ms after init [init=33ms,
gui_setup=0ms, first_render=1797ms] (rendered 6.5ms AFTER warmup done)
The leftover print(f'[startup] RunnerParams() init: ...') referenced
_t which was deleted when the block was converted to a
with startup_profiler.phase() context. Would have raised NameError
on the full native GUI path. Replaced with a comment; the phase()
above already logs the same info.
Replaces the buggy custom _t = time.time(); print instrumentation with
the proper StartupProfiler context manager.
Phases added to App.__init__:
- app_init_AppController
- app_init_history_perfmon
Phases added to App.run() (else branch = native GUI):
- theme_load_from_config
- imgui_bundle_import (the C++ extension import chokepoint)
- RunnerParams_init
Note: a leftover print(f'[startup] RunnerParams() init: ...') line in
App.run() still references a stale _t variable. Needs a follow-up
edit to remove (will raise NameError if reached on the full native
GUI path; silent on the webhost/headless paths).
Replaces ad-hoc print() timing with the proper StartupProfiler.phase()
context manager. The phases cover the actual chokepoints the user
wanted to measure (NOT src/* imports — those are benchmark_imports.py's
job):
- argv_parse: argparse setup
- defer_sugar: defer.sugar install
- web_host_imports: imgui_bundle + api_hooks
- gui_2_import_webhost: from src.gui_2 import App
- app_construct: App() instance creation
- hello_imgui_run: the C++ imgui bundle init (the actual bottleneck)
- headless_imports: from src.app_controller import AppController
- appcontroller_construct_headless: AppController() + warmup submit
- appcontroller_run: asyncio loop
- gui_2_main_import: from src.gui_2 import main
- main_call: the legacy main() entry
Combined with the existing StartupProfiler singleton, every phase now
emits [startup] <name>: <ms>ms to stderr in real time, so the user
can grep for chokepoints in a real uv run.
- startup_profiler: StartupProfiler = StartupProfiler() at module bottom
so sloppy.py can import it without circular imports.
- phase() context manager now writes a [startup] <name>: <ms>ms line to
stderr in its finally block. Live visibility of every measured phase.
The Critical Anti-Patterns list now has 2 new HARD rules:
1. NEVER run git restore / git checkout -- <file> / git reset without
EXPLICIT user permission in the same message. They destroyed
user in-progress src/* edits twice in one session (2026-06-07).
2. No giant edits: if manual-slop_edit_file new_string exceeds ~20 lines,
STOP and split it. Large blocks hide indentation bugs.
Also:
- Strengthened Session-Learned rule 4 to a HARD BAN
- Added rule 6 'Stop profiling the wrong thing' (don't re-benchmark
src/* imports; benchmark_imports.py is authoritative; the missing
metrics are on imgui_bundle init + hello_imgui.run() + first frame)
Captures the 5 patterns that burned the most time in the
startup_speedup_20260606 sub-track 4 work:
1. ALWAYS use manual-slop_edit_file, not custom scripts
(custom scripts fail silently on indent/EOL/whitespace drift)
2. The decorator-orphan pitfall
(inserting before 'def foo' leaves @property decorating YOUR new method)
3. ast.parse() is not enough
(semantic errors aren't caught; import + instantiate + call after every edit)
4. The git restore trap
(don't run git status/restore while a user is mid-conversation)
5. Small verified edits beat big scripts
(edit_workflow says 3-10 lines; if you write 200 lines of script, wrong tool)
Also adds 2 new anti-patterns to the Critical list in AGENTS.md and
3 new sections to conductor/edit_workflow.md (decorator-orphan,
ast.parse-not-enough, set_file_slice-is-literal).
Adds per-AppController startup timing instrumentation to answer
'did the warmup block the first frame?'
AppController.__init__ records _init_start_ts at entry (cold-start anchor).
WarmupManager.on_complete callback stamps _warmup_done_ts.
App.render_main_interface (gui_2.py) calls mark_first_frame_rendered()
on its first call, which stamps _first_frame_ts and logs the timeline.
New public API on AppController:
- init_start_ts (property): float
- warmup_done_ts (property): Optional[float]
- first_frame_ts (property): Optional[float]
- mark_first_frame_rendered(ts=None): idempotent; logs to stderr
- startup_timeline() -> dict with all timestamps + precomputed deltas:
warmup_ms, first_frame_after_init_ms, first_frame_after_warmup_ms
Stderr log on warmup done:
[startup] warmup done in 1186.2ms (first frame rendered Nms BEFORE/AFTER)
Stderr log on first frame:
[startup] first frame at Xms after init (warmup took Yms) (rendered Zms BEFORE/AFTER warmup done)
Hook API:
- GET /api/startup_timeline
- ApiHookClient.get_startup_timeline() -> dict
5 new tests in test_warmup_canaries.py covering all the new methods.
All 18 canary tests + 10 api_hooks tests + 6 gui_indicator tests pass.
Script scripts/apply_startup_timeline.py is included as a reference
for the multi-edit pattern (the proper MCP-equivalent tools will be
added later per the edit_workflow doc).
Per module: prints a one-line summary to stderr when the import
completes or fails:
[warmup 1] google.genai on controller-io_0 (id=18636): 1218.6ms
[warmup 2] anthropic on controller-io_1 (id=5500): 1148.3ms
[warmup 3] openai on controller-io_2 (id=34376): 1144.2ms
...
When the entire warmup completes, prints an aggregate:
[warmup done] 9 modules: 9 completed (sum of per-module elapsed: 3591.7ms)
If ANY canary ran on the main thread (main-thread-purity violation),
the per-module line is tagged with [MAIN-THREAD] AND a final WARNING
is printed:
[warmup WARNING] N module(s) loaded on the MAIN THREAD: google.genai
Default is log_to_stderr=True so production runs get the observability
for free. Tests opt out via WarmupManager(pool, log_to_stderr=False)
in the _build_warmup helper.
5 new tests (4 stderr logging + 1 quiet). All 13 canary tests pass.
Use case: 'did my heavy import run on the GUI thread when it shouldnt
have?' is now answered by grepping stderr for [warmup ...] [MAIN-THREAD]
lines. No hook server required.
Adds a canary record for each module submitted to the warmup, tracking:
canary_id, module, thread_name, thread_id, submit_ts, start_ts,
end_ts, elapsed_ms, status, error.
Surface:
- WarmupManager.canaries() returns list[dict] (defensive copy)
- AppController.warmup_canaries() returns list[dict] (delegation)
- GET /api/warmup_canaries Hook API endpoint
- ApiHookClient.get_warmup_canaries() returns list[dict]
Example: the warmup of google.genai records a 1187ms canary on
thread controller-io_0 with thread_id 50420, canary_id 1.
11 new tests (8 unit in test_warmup_canaries + 3 in test_api_hooks_warmup).
All pass; live_gui smoke test confirms endpoint returns real data.
Sub-track 2 of startup_speedup_20260606. Removes the top-level
'import tomli_w' from src/models.py and moves it inside save_config().
tomli_w (~30ms cold load) is now loaded only when the user saves
config, not on every src.models import.
This drops the audit violation count from 63 to 62.
Pydantic BaseModel (the other src/models.py violation) is left for
a future sub-track: deferring a class base requires a metaclass or
proxy pattern that's higher risk for the small (~50ms) saving.
3 new tests in tests/test_models_no_top_level_tomli_w.py:
- tomli_w NOT in sys.modules after import src.models
- save_config() still works (because tomli_w loads on-demand)
- save_config() actually triggers the import on first call
17 existing model tests pass (test_persona_models, test_bias_models,
test_context_presets_models, test_per_ticket_model, test_file_item_model).
Fixes the run_tests_batched.py hang that occurs after batch 4.
The original conftest (commit 52ea2693) stored _warmup_app_controller
at module scope for the entire pytest session. When pytest exits,
GC of the AppController triggers ThreadPoolExecutor.__del__ ->
shutdown(wait=True). If warmup hasn't fully completed by then, the
shutdown blocks indefinitely, causing the batched test runner to
hang at the subprocess.run boundary.
Fix: register an atexit handler that captures the _io_pool reference
directly (default argument) and shuts it down with wait=False. The
pool reference is captured by closure, surviving even after the
AppController is GC'd. shutdown() is idempotent so the subsequent
shutdown(wait=True) in __del__ is a no-op.
This is part of sub-track 4 (warmup notification) cleanup; the
conftest's wait_for_warmup behavior is preserved, only the
exit-hang is fixed.
Sub-track 4 of startup_speedup_20260606. Adds per-frame GUI feedback
during the AppController's background warmup:
- render_warmup_status_indicator(app): module-level render fn called
from render_main_interface. Shows 'Warming up... (N/M)' in warning
color while pending, 'Imports: K failed' in error color on failure,
or 'All imports ready (M modules)' in success color for 3 seconds
after completion. Hidden otherwise.
- _on_warmup_complete_callback(app, status): thread-safe callback
registered with controller.on_warmup_complete() in App._post_init.
Records timestamp + lock-protected toast list.
- App._post_init: registers the callback.
6 new tests in tests/test_gui_warmup_indicator.py:
- 2 importable-checks (function exists)
- 3 callback-logic tests (timestamp, failures, thread-safety)
- 1 live_gui smoke test (controller exposes warmup_status)
All additive; no breaking changes to existing content. Derived from gaps
observed during the 2026-06-06 planning session (5 tracks spec'd +
planned end-to-end).
**AGENTS.md (1 new section, 16 lines):**
- Compaction Recovery - explicit recovery path for a new agent
picking up mid-track (read the digest, check state.toml, run audits,
resume from next unchecked task). Cross-references the
workflow-level 'Compaction Recovery' section.
**conductor/workflow.md (6 new sections, 145 lines):**
- Planning Session Workflow - documents the brainstorming -> spec ->
plan flow used 5x this session; mandates spec approval before plan;
notes the plan is the only artifact the implementer reads.
- Track Dependencies and Execution Order - verify the blocked_by
chain in metadata.json before starting; topological sort gives the
recommended execution order (recorded in PLANNING_DIGEST).
- State.toml Template - canonical structure (meta / blocked_by /
blocks / phases / tasks / verification / track-specific) so future
tracks have a consistent shape.
- Per-Task Decision Protocol - small decisions (cosmetic) decide
yourself; large decisions (architectural) STOP and report; regressions
STOP and report. The boundary is 'does this require a new spec or
plan update?'.
- Documentation Refresh Protocol - after a track ships, identify
affected guides (grep for renamed/moved symbols), update them, add
new guides for new modules, add styleguides for new conventions.
The 'post-tracks documentation' pattern is repeatable; tracks that
only update code are incomplete.
- Audit Script Policy - whenever a track introduces a new convention
that can be statically checked, add an audit script in scripts/
with --help / --json / strict modes. The audit + CI gate pair is
the convention-enforcement mechanism; 3 existing audits
(audit_main_thread_imports, audit_weak_types, check_test_toml_paths)
are the precedent.
All sections reference existing project files (brainstorming skill,
writing-plans skill, audit scripts, tracks.md, the existing 5 new
tracks' spec.md files, PLANNING_DIGEST_20260606.md).
No code changes. Documentation only. ~160 lines total added.
Sub-track 3 of startup_speedup_20260606. Builds on the Phase 7 minimal
work at b464d1fe which only added warmup_status to /api/gui/diagnostics.
New dedicated endpoints:
- GET /api/warmup_status -> controller.warmup_status() (cheap, lock-guarded)
- GET /api/warmup_wait?timeout=N -> controller.wait_for_warmup(timeout)
then returns the final status. Default 30s.
Both callable from external clients via ApiHookClient.get_warmup_status()
and ApiHookClient.get_warmup_wait(timeout=30.0).
7 new tests in tests/test_api_hooks_warmup.py (5 unit + 2 live_gui).
All 7 pass.
Single-session planning digest that captures:
- The 5 tracks fully specced + planned (test_batching, qwen_llama_grok,
data_oriented_error_handling, data_structure_strengthening,
mcp_architecture_refactor)
- Cross-cutting design themes (data-oriented, audit-driven, per-track
commit + git note, out-of-scope-by-default)
- The audit + data foundation (scripts/audit_weak_types.py; 430 -> 60
finding; 0 strong patterns; 26 unique type strings; 86% concentrated
in 6 files)
- The dependency graph + recommended execution order
- Follow-up tracks already planned in spec §12.1 of each track
- Recommended future tracks (post-tracks documentation is the top pick)
- Risks, open questions, and a complete file index
This is the kind of reference document that:
- Future planners consult to understand the codebase's current state
- The implementing agent uses to coordinate across tracks
- The user reviews as a digest of the planning work
Written in the project's docs/reports/ directory alongside the existing
Phase 5 reports (PHASE5_STABILISATION_REPORT.md, MUTATION_MATRIX_PHASE5.md, etc.).
~25 tasks across 7 phases, each with explicit Red-Green-Refactor TDD steps:
- Phase 1 (1.1-1.5): Foundation. 3-layer security module (8 unit tests
returning Result[Path]); SubMCP Protocol + MCPController class (6 unit
tests). Controller added ALONGSIDE the existing 45 functions in
mcp_client.py (no removal yet).
- Phase 2 (2.1-2.4): Backward compat. git mv mcp_client.py to
mcp_client_legacy.py; create new mcp_client.py as a slim shim
re-exporting 45+ old symbols. 12 legacy shim tests verify the surface.
The 4 existing test files + src/app_controller.py:61 still work.
- Phase 3 (3.1-3.4): FileIOMCP extracted (9 tools, 10 unit tests).
- Phase 4 (4.1-4.4): PythonMCP extracted (14 tools, 14 unit tests).
- Phase 5 (5.1-5.5): CMCP, CppMCP, WebMCP, AnalysisMCP extracted
(4 sub-MCPs, 18 unit tests; pattern mirrors Phase 3/4).
- Phase 6 (6.1-6.3): ExternalMCP extracted from mcp_client_legacy.
Class name preserved (ExternalMCPManager).
- Phase 7 (7.1-7.5): Update dispatch() in the legacy shim to use the
new controller (inverted-dict O(1) lookup); update docs; manual
smoke test; archive the track.
Each sub-MCP follows the same template (class with name / description
/ tools / invoke; security check for path-taking tools; Result wrapping
in invoke(); delegation to legacy functions for the actual implementation).
The sub-MCPs are thin adapters in v1; a future track can move the
implementations into the sub-MCP files directly.
Self-review at the end maps every spec section to a task (no gaps),
confirms zero placeholders, and verifies type/method-name consistency
across phases (SubMCP Protocol, MCPController class, Result[str,
ErrorInfo], _resolve_and_check all defined in Phase 1; used
consistently across Phases 3-6).
Track + metadata + state + tracks.md registration for the 2,205-line
mcp_client.py split into a slim controller + 6 native sub-MCPs + 1
external sub-MCP.
Key design decisions (per user feedback):
- Naming convention: mcp_<type>.py for native MCPs (mcp_file_io.py,
mcp_python.py, mcp_c.py, mcp_cpp.py, mcp_web.py, mcp_analysis.py).
- ExternalMCPManager class name preserved (moves to mcp_external.py).
- Sub-MCP shape: class with name / description / tools / invoke().
- MCPController: holds ALL_SUB_MCPS list, inverted-dict tool lookup,
3-layer security (extracted to mcp_client_security.py), schema
aggregation.
- Each invoke() returns Result[str, ErrorInfo] (from
data_oriented_error_handling_20260606).
- Backward compat: mcp_client_legacy.py re-exports all 45+ old
symbols; the 4 existing test files + src/app_controller.py:61
direct call continue to work.
DSL future (per user notes on APL/K/Cosy): NOT in this track.
Documented in spec §12.1 as the mcp_dsl_20260606 follow-up.
Sub-MCP architecture is the natural unit to pair with a DSL emitter.
7 phases. ~22 task slots. New tests: 9 (one per sub-MCP + controller +
security + legacy). Modified tests: 4 (existing mcp_* tests must
pass unchanged).
Blocked by: data_oriented_error_handling_20260606, data_structure_strengthening_20260606.
Blocks: mcp_dsl_20260606 (future DSL track).
Phase 6 of startup_speedup_20260606 was partial: ~13 ad-hoc
threading.Thread spawns remained in src/app_controller.py and
2 in src/gui_2.py. This commit migrates all of them to
self.submit_io(...) (the shared _io_pool wrapper from Phase 2).
ZERO new threading.Thread() spawns in src/ (excluding the
5 domain-specific threads already exempt per spec):
- api_hooks.py:739 HookServer HTTP server (domain-specific)
- api_hooks.py:818 WebSocketServer (domain-specific)
- app_controller.py _loop_thread (asyncio event loop, DEDICATED)
- multi_agent_conductor.py WorkerPool (domain-specific)
- performance_monitor.py CPU monitor (continuous, domain-specific)
Sites migrated (15 total):
app_controller.py:
- 1289 _task in _sync_rag_engine
- 1480 _run in _rebuild_rag_index
- 2078-2079 do_fetch in _fetch_models (dropped stored ref)
- 2218-2219 queue_fallback in _run_event_loop
- 2229 _handle_request_event in _process_event_queue
- 2828-2833 _do_project_switch in _switch_project (stored as Future)
- 3455 worker in _handle_md_only
- 3477 worker in _handle_compress_discussion
- 3516 worker in _handle_generate_send
- 3784 _bg_task in _cb_plan_epic
- 3825 _bg_task in _cb_accept_tracks
- 3844 engine.run in _cb_start_track (track_id case)
- 3855 engine.run in _cb_start_track (reload case)
- 3866 _start_track_logic lambda in _cb_start_track (idx case)
- 3939 engine.run in _start_track_logic
gui_2.py:
- 1129 _stats_worker in _update_context_file_stats
- 3507 worker in _check_auto_refresh_context_preview
Stored-ref migration (Phase 6 partial work):
- self.models_thread (declared L960, assigned L2078):
No external readers. Dropped the declaration and the assignment;
replaced the .start() with self.submit_io(do_fetch).
- self._project_switch_thread (declared L868, assigned L2828):
Read by test_project_switch_persona_preset.py:21 for
.is_alive() polling. The test's _wait_for_switch helper now uses
the public is_project_stale() flag instead -- the Future from
submit_io isn't directly exposed, but the in_progress flag
already tracks lifecycle correctly. Dropped the declaration;
replaced the .start() with self.submit_io(self._do_project_switch, path).
Test impact:
- test_project_switch_persona_preset.py::_wait_for_switch:
Updated to poll ctrl.is_project_stale() instead of the
_project_switch_thread attribute. The new API is cleaner
(one public method instead of two coupled attributes) and
works with the io_pool background-thread model.
Effectiveness:
- Per-spawn cost: ~1-5ms saved (thread creation)
- 4 long-lived threads eliminated; all background work now shares
the 4-worker _io_pool
- When 4 long-lived threads were active simultaneously, the new
pool backpressure causes them to queue; future work can be
backpressured explicitly
TESTS: 19+39 = 58 tests touching migrated code paths all pass.
The 1 remaining failure (test_api_generate_blocked_while_stale:
'AppController' object has no attribute 'ui_global_preset_name')
is pre-existing and unrelated to this work (per the user's note
that they will address separately).
ed
r00tz