Phase 6 of startup_speedup_20260606 was partial: ~13 ad-hoc
threading.Thread spawns remained in src/app_controller.py and
2 in src/gui_2.py. This commit migrates all of them to
self.submit_io(...) (the shared _io_pool wrapper from Phase 2).
ZERO new threading.Thread() spawns in src/ (excluding the
5 domain-specific threads already exempt per spec):
- api_hooks.py:739 HookServer HTTP server (domain-specific)
- api_hooks.py:818 WebSocketServer (domain-specific)
- app_controller.py _loop_thread (asyncio event loop, DEDICATED)
- multi_agent_conductor.py WorkerPool (domain-specific)
- performance_monitor.py CPU monitor (continuous, domain-specific)
Sites migrated (15 total):
app_controller.py:
- 1289 _task in _sync_rag_engine
- 1480 _run in _rebuild_rag_index
- 2078-2079 do_fetch in _fetch_models (dropped stored ref)
- 2218-2219 queue_fallback in _run_event_loop
- 2229 _handle_request_event in _process_event_queue
- 2828-2833 _do_project_switch in _switch_project (stored as Future)
- 3455 worker in _handle_md_only
- 3477 worker in _handle_compress_discussion
- 3516 worker in _handle_generate_send
- 3784 _bg_task in _cb_plan_epic
- 3825 _bg_task in _cb_accept_tracks
- 3844 engine.run in _cb_start_track (track_id case)
- 3855 engine.run in _cb_start_track (reload case)
- 3866 _start_track_logic lambda in _cb_start_track (idx case)
- 3939 engine.run in _start_track_logic
gui_2.py:
- 1129 _stats_worker in _update_context_file_stats
- 3507 worker in _check_auto_refresh_context_preview
Stored-ref migration (Phase 6 partial work):
- self.models_thread (declared L960, assigned L2078):
No external readers. Dropped the declaration and the assignment;
replaced the .start() with self.submit_io(do_fetch).
- self._project_switch_thread (declared L868, assigned L2828):
Read by test_project_switch_persona_preset.py:21 for
.is_alive() polling. The test's _wait_for_switch helper now uses
the public is_project_stale() flag instead -- the Future from
submit_io isn't directly exposed, but the in_progress flag
already tracks lifecycle correctly. Dropped the declaration;
replaced the .start() with self.submit_io(self._do_project_switch, path).
Test impact:
- test_project_switch_persona_preset.py::_wait_for_switch:
Updated to poll ctrl.is_project_stale() instead of the
_project_switch_thread attribute. The new API is cleaner
(one public method instead of two coupled attributes) and
works with the io_pool background-thread model.
Effectiveness:
- Per-spawn cost: ~1-5ms saved (thread creation)
- 4 long-lived threads eliminated; all background work now shares
the 4-worker _io_pool
- When 4 long-lived threads were active simultaneously, the new
pool backpressure causes them to queue; future work can be
backpressured explicitly
TESTS: 19+39 = 58 tests touching migrated code paths all pass.
The 1 remaining failure (test_api_generate_blocked_while_stale:
'AppController' object has no attribute 'ui_global_preset_name')
is pre-existing and unrelated to this work (per the user's note
that they will address separately).
Phase 5D of startup_speedup_20260606 track.
DEAD IMPORTS REMOVED (zero uses, safe to remove):
- 'import tomli_w' (line 18) - never referenced anywhere in gui_2.py
- 'from src import theme_nerv_fx as theme_fx' (line 59) - never
referenced; the actual NERV FX objects are created in src/theme_2.py
and accessed via render_post_fx()
The theme_nerv_fx removal saves the full ~254ms import of
src.theme_nerv_fx on the main thread.
LAZY PROXY PATTERN for heavy feature-gated modules:
- 'import numpy as np' (line 9) - used in 1 place (plot_lines)
- 'from tkinter import filedialog, Tk' (lines 30, 34) - duplicates
removed, 13 use sites now go through the proxy
Added a _LazyModule class that defers module loading until first
attribute access or call. The proxy is a transparent replacement:
'np.array(...)' and 'Tk()' continue to work unchanged. The import
only fires on first use, then is cached in sys.modules for O(1)
subsequent access.
ARCHITECTURAL NOTE: This is a general-purpose pattern that can be
used for any module that should not be in the main thread's import
chain. The Phase 5A 'lazy registry proxy' was a similar idea but
custom-tailored to one use case; _LazyModule is the general form.
EFFECTIVENESS (estimated from baseline):
- src.theme_nerv_fx removal: ~254ms saved
- numpy deferral: ~65ms saved (when not plotting); 0ms saved if the
user is using numpy (imgui_bundle transitively brings it in anyway)
- tkinter deferral: small but real savings (tkinter is stdlib but
still has import cost)
Note that numpy and tkinter are still brought in transitively by
imgui_bundle and other src.* modules. The test verifies the AST
(top-level imports of gui_2.py) is clean; the runtime sys.modules
check is too strict because of these transitive imports.
TESTS:
- tests/test_gui_2_no_top_level_heavy_imports.py: 5/5 PASS (all RED -> GREEN)
- 13 gui tests sampled (gui_progress, gui_paths, gui_kill_button,
gui_window_controls, gui_custom_window, gui_fast_render,
gui_startup_smoke, gui2_layout, gui2_events): all PASS
NEXT: Phase 6 (ad-hoc threads -> _io_pool), Phase 7 (warmup
notification), Phase 8 (enforcement), Phase 9 (final verify + checkpoint).
When switching projects, the previous implementation ran the entire
save/load/refresh sequence on the main thread. With large project files
or slow disks, this caused the UI to freeze for several seconds.
Fix:
- _switch_project now returns immediately after setting flags; the
actual work runs in a daemon thread (_do_project_switch)
- New is_project_stale() property returns True while a switch is queued
or running; the GUI renders an amber/yellow tint overlay to signal
the controller state lags the user's last click
- AI ops are gated: _api_generate returns HTTP 409, _handle_generate_send
and _handle_md_only early-return with ai_status feedback, all when
is_project_stale() is true
- Queued switches (clicking project A then B in rapid succession) are
coalesced: B replaces A as the target; once A completes, B is
triggered automatically via the finally branch in _do_project_switch
- New state fields: _project_switch_in_progress, _project_switch_pending_path,
_project_switch_thread, _project_switch_lock
- AppController state class attributes use hasattr guard for _app to
keep the controller usable standalone in tests/headless mode
UX:
- Render loop keeps drawing during the switch
- User can still scroll, switch tabs, browse files
- Amber tint + popup explains what's happening and that AI ops are paused
- ai_status shows the target project name
Tests:
- _wait_for_switch helper added for the new async switch flow
- All 7 existing switch tests updated to call _wait_for_switch
- 2 new tests:
- test_switch_project_non_blocking: verifies _switch_project returns
in <0.2s and is_project_stale() is True during the switch
- test_api_generate_blocked_while_stale: verifies _api_generate
raises HTTPException(409) while a switch is in progress
All 33 related tests pass.
Two fixes for the regression introduced in b92daef3 (and an additional
hardening for the persona->context_preset stale-reference class of bug):
1. Regression: persona_manager was missing on first project load.
_load_active_project creates preset_manager and tool_preset_manager
but did not create persona_manager, so the new
self.personas = self.persona_manager.load_all() line in
_refresh_from_project raised AttributeError on app startup before
the post-_load_active_project persona_manager creation could run.
Fix: create self.persona_manager in _load_active_project alongside
the other managers, so the manager is available when
_refresh_from_project runs.
2. Stale reference: persona's context_preset field pointed to a
preset (e.g. 'GTE') that no longer exists in the project, causing
load_context_preset to raise KeyError and crash the persona
selector panel (which triggered the cascading 'Missing End()' imgui
assertion).
Fix: wrap the load_context_preset call in render_persona_selector_panel
with try/except KeyError, surface the error in app.ai_status, and
clear app.ui_active_context_preset to keep the GUI state consistent.
Tests: 2 new tests in tests/test_project_switch_persona_preset.py
- test_load_active_project_creates_persona_manager (regression guard)
- test_load_context_preset_missing_raises_keyerror (verifies the
contract that load_context_preset raises for missing names; the
GUI layer is now responsible for catching the error)
ROOT CAUSE: The ListClipper in render_prior_session_view was being
tripped up by the variable heights of discussion entries (huge system
prompts vs small tool results). When the first entry was very tall
(system prompt), the clipper would compute the visible range assuming
uniform item heights, leading to underflow/overflow on subsequent
items. The user saw only the first ~8 entries with massive empty
space below ('early clipping').
FIX: Replace the ListClipper with a direct for loop over
app.prior_disc_entries. With 233 entries, performance is acceptable
and each entry renders correctly. The user can still scroll the
parent imscope.child window if content overflows.
Tests:
- Updated test_prior_session_no_clipping.py to set entries on
app_instance.controller.prior_disc_entries (the App's __getattr__
proxies attribute reads to the controller, so the set must go to
the controller directly).
- 28/28 broad regression pass
ROOT CAUSE: During a previous indentation fix, the 'while clipper.step():'
line was accidentally removed from render_prior_session_view. Without
the step() loop, the ListClipper's display_start/display_end stay at
their initial values (0/0 or similar), so NO discussion entries
were rendered even though 233 entries were present in
app.prior_disc_entries. The user saw only a single entry because the
list clipper was never advanced.
FIX: Restore the 'while clipper.step():' line. Re-indent the entire
prior_scroll block to consistent 1-space (function), 2-space (inside
style_color), 3-space (inside child + while), 4-space (inside for)
indentation. Now all 233 entries will render through the list clipper.
Tests:
- 28/28 broad regression pass
ROOT CAUSE: render_comms_history_panel had imgui.end_child() nested INSIDE
an 'if app._scroll_comms_to_bottom:' block at line 3758. When
_scroll_comms_to_bottom was False (the common case), end_child was
NOT called, leaving the comms_scroll child window open. This caused
the imGui state to corrupt: tab_item.end_tab_item, tab_bar.end_tab_bar,
and the outer window.end all saw that the child was still open
(WithinEndChildID was set), triggering 'Must call EndChild() and not
End()!' assertion.
FIX: Convert the entire comms_scroll block to imscope.child (which uses
Python's with statement for exception-safe end_child). The scroll-to-bottom
logic is now correctly nested INSIDE the with block, and there's no
manual end_child to forget.
Tests:
- Updated test_comms_scroll_no_clipping.py to check imscope.child
instead of begin_child
- 28/28 broad regression pass
ROOT CAUSE: render_heavy_text (called per comms panel entry) had
manual begin_child/end_child pairs. If anything inside the child
(especially markdown_helper.render) raised, end_child was skipped.
The child window was left open, corrupting the imGui state. The
corruption cascaded through tab_item.end_tab_item -> tab_bar.end_tab_bar
-> window.end, triggering 'Must call EndChild() and not End()!' assertion.
FIX: Convert the inner begin_child/end_child pair to imscope.child so
the end_child is automatically called by Python's with statement, even
on exception. Also convert prior_scroll to imscope.child for consistency.
TESTS:
- Existing test_comms_no_extraneous_pop.py: push/pop balance check
- Updated test_prior_session_no_clipping.py to match new imscope.child
signature
- 28/28 broad regression pass
ROOT CAUSE: In a previous fix (df7bda6e 'explicit child size for
comms_scroll and prior_scroll'), the code that pushed a child_bg style
color at the start of render_comms_history_panel was removed when the
section was rewritten to use imgui.get_content_region_avail() for
explicit child sizing. However, the matching pop_style_color at the end
of the function (guarded by 'if app.is_viewing_prior_session') was left
in place.
RESULT: When viewing a prior session, the imscope.style_color in
_gui_func pushes 1 color at the start of the frame, then the orphaned
pop in render_comms_history_panel decrements the imGui style counter
by 1, then _gui_func's imscope __exit__ tries to pop again — triggering
IM_ASSERT 'PopStyleColor() too many times!'.
This caused a cascade of imGui state corruption on every frame after
loading a prior session log, manifesting as 'too many times' assertions
on the next frame and 'Must call EndChild() and not End()' once the
style stack underflowed.
FIX: Remove the orphan pop_style_color at gui_2.py:3761. No matching
push exists, so the pop is unconditionally wrong.
TESTS:
- New test_comms_no_extraneous_pop.py asserts push/pop balance in
render_comms_history_panel when is_viewing_prior_session is True
- 43/43 broad regression pass
Convert manual push_style_color / pop_style_color in _gui_func to use the
imscope context manager so the pop is exception-safe via Python's with
statement. Manual push/pop can desync if render_main_interface raises
mid-render, causing 'PopStyleColor() too many times!' imGui assertion
on subsequent frames.
The try/except around render_main_interface was already there but the
pop was outside it, so the pop count could exceed the push count when
an exception short-circuited the render.
ROOT CAUSE: When child windows used ImVec2(0, 0) for auto-fill, the
child's reported height was unstable inside tab items (especially when
the parent tab was inside a tab_bar inside a window). Result: the
scrollable child rendered with a fixed smaller height, showing only the
first half of the content, with empty space below.
FIX: Use imgui.get_content_region_avail() to compute explicit dimensions
and pass them to begin_child. Now the child fills the full available area
inside the tab content.
- render_comms_history_panel: avail.x, avail.y
- render_prior_session_view: same, plus added entry count indicator next
to the Exit Prior Session button ({N} entries) for at-a-glance info
Tests:
- test_comms_scroll_no_clipping.py: verifies comms_scroll child uses
explicit (non-zero) size
- test_prior_session_no_clipping.py: same for prior_scroll child
- test_log_management_first_open.py: minor cleanup
- 42/42 broad regression pass
ROOT CAUSE: src/markdown_helper.py:render() used a 'mask text with placeholders
then re.split' approach that failed when AI responses contained CRLF or when
the same table content appeared twice. The replace() either didn't match
(CRLF mismatch) or only replaced the first occurrence, leaving the second
table as raw markdown for imgui_md to render badly. Result: the same table
appeared twice (bad rendering via imgui_md, good rendering via my new code).
FIX: rewrite render() to walk lines directly. Per-line, decide whether to
buffer for imgui_md, skip into a table renderer, or accumulate into a
code-block renderer. No text replacement needed.
- src/markdown_helper.py: new render() walks lines, handles code fences
and table intervals inline via lookup dicts.
- src/gui_2.py: render_log_management now calls load_registry() on the
newly-created LogRegistry when _log_registry was None. Previously the
initial construction populated an empty table, AND the 'Refresh Registry'
button was inside the else branch, so users had no way to load data.
User re-indented the surrounding block during debugging.
Tests:
- test_markdown_render_robust.py: 2 tests (CRLF text, duplicate content)
- test_log_management_first_open.py: 1 test (registry populated on open)
40/40 broad regression pass.
ROOT CAUSE: 3 mismatched names in the empty-context warning path:
1. _handle_generate_send set self.show_empty_context_warning_modal = True
but render_empty_context_modal checks self.show_empty_context_modal.
The modal never opened.
2. _handle_generate_send / _handle_md_only never set
self._pending_generation_action, so the modal's 'Proceed Anyway'
button always saw None and dispatched nothing.
3. After Proceed Anyway, _pending_generation_action was never reset,
so subsequent empty-context calls would dispatch the wrong action.
FIX:
- gui_2.py:494,501: show_empty_context_warning_modal -> show_empty_context_modal
- gui_2.py:494,501: set _pending_generation_action before showing modal
- gui_2.py:5385: reset _pending_generation_action = None after dispatch
Tests: tests/test_gen_send_empty_context.py (5 cases) covers all 4 dispatch
paths (generate/md_only x proceed/skip) plus the happy path with context.
37/37 regression pass. No new ImGui scope errors (2 pre-existing unrelated).
- render_files_and_media now wraps the per-file loop in directory groups
via aggregate.group_files_by_dir + imscope.tree_node_ex (mirrors the
Context Composition visual style at gui_2.py:3114)
- New 'Add Directory' button next to 'Add Files to Inventory':
uses filedialog.askdirectory() + os.walk to bulk-import a folder tree
- Button IDs (i, add_f_{i}, rem_f_{i}) preserve global uniqueness via
file_indices map (regression-safe across the directory wrap)
- Test uses mock button=False, mock filedialog.askopenfilenames/askdirectory
to avoid opening a real Tk dialog during test run
- New module-level render_vendor_state(app) in gui_2.py
- New 'Vendor State' tab in render_operations_hub tab_bar
- Renders 5 stable metrics: provider_model, context_window, cache, quota, last_error
- Each row: Metric label | Value | State (colored ok/warn/error/info)
- Tooltips via imgui.set_tooltip on the value cell
ImGui scope linter: render_vendor_state OK. Pre-existing 2 errors at lines
2684 and 4994 unrelated to this commit.
ROOT CAUSE: gui_2.py:1675 re-instantiated LogRegistry() which opens the TOML
but never called .load_registry() so the table stayed empty.
FIX: in-place load_registry() on the existing instance — preserves in-memory
state (any pending update_session_metadata call) and matches the user's intent
of 'refresh from disk'.
Added imgui.set_next_window_focus() on open so the palette window itself gets focus. The input field then gets focus on the next drawn widget. Wrapped action calls in try/except so a buggy command does not break the imgui.end_child/end pairing (was causing IM_ASSERT crash). Fixed theme_2 calls: apply_dark_theme and apply_light_theme do not exist; use theme_2.apply(palette_name). switch_to_dark_theme uses apply 10x Dark. switch_to_light_theme uses apply ImGui Light. switch_to_nerv_theme uses apply NERV instead of apply_nerv() from src.theme_nerv.
- Restore monolithic architecture in gui_2.py to fix test compatibility.
- Implement full-width horizontal expansion for Markdown tables in discussion entries.
- Re-implement layered role-based tints using draw_list channels.
- Standardize Text Viewer docking ID to '###Text_Viewer_Unified'.
- Fix MiniMax compression routing and base URL.
- Fully restore missing theme_2.py definitions.
- Restore monolithic architecture in gui_2.py to fix test breakages and circular imports.
- Update Text Viewer stable ID to '###Text_Viewer_Unified' to definitively fix docking conflicts.
- Refactor discussion entry renderer to force full-width horizontal expansion for Markdown.
- Fully restore theme_2.py definitions (palettes, fonts, scale) while retaining role-tint logic.
- Robustify ImGui ID stack in imgui_scopes.py to prevent access violations.
- Verify all fixes with the comprehensive unit and visual test suite.
- Restore all rendering logic to gui_2.py to maintain monolithic architecture and test compatibility.
- Fix horizontal squashing of Markdown tables by ensuring full panel width in entry groups.
- Resolve Text Viewer docking conflicts by standardizing on a stable window ID ('###Text_Viewer_Unified').
- Fix theme initialization by restoring missing load/save functions in theme_2.py.
- Prevent ImGui access violations by ensuring ID stack always receives strings in imgui_scopes.py.
- Successfully verified all UI regressions with a passing unit test suite.
- Update Text Viewer window ID to '###Text_Viewer_Unified'.
- Ensures ImGui treats the window as a single stable entity across title changes.
- Prevents docking loop glitches.
- Insert imgui.new_line() before rendering discussion content.
- Ensures the Markdown renderer inherits the full horizontal width of the panel.
- Definitively fixes vertical squashing of tables and long text blocks.
- Update Text Viewer stable ID to match registry key exactly ('###Text Viewer') for stable docking.
- Ensure imgui.push_id always receives a string in imgui_scopes.py to prevent low-level access violations.
- Resolve ImportError by correctly prefixing 'src' in modular renderers.
- Fix ImGui access violation by ensuring push_id always receives string IDs.
- Restore visible role-based background tints using layered rendering (channels).
- Definitively fix horizontal Markdown table widths by forcing group expansion.
- Centralize color management in theme_2.py and ui_shared.py.
- Standardize Files & Media inventory layout and remove legacy controls.
- Update test mocks to support modular UI and theme-driven styling.
- Implement layered tinting using draw_list channels in modular discussion renderer.
- Fix vertical squashing of Markdown tables by forcing full group width with a dummy.
- Consolidate color constants into src/ui_shared.py to prevent circular imports.
- Update src/theme_2.py with role-based tint helpers.
- Successfully verified imports and layout logic.
- Modularize discussion entry rendering to src/discussion_entry_renderer.py to fix layout squashing.
- Fix MiniMax compression routing with robust case-insensitive check and synced base URL.
- Implement src/ui_shared.py to resolve circular imports and consolidate shared UI helpers.
- Finalize Structural File Editor integration and state unification.
- Correctly route 'minimax' provider in run_discussion_compression.
- Fix MiniMax base URL to api.minimax.io to match main sender.
- Refactor read-mode discussion entries to always use a scrollable child with auto-resize.
- Remove redundant text wrapping that caused Markdown tables to squash vertically.
- Clean up duplicate separators in discussion hub.
- Update test_gui_symbol_navigation.py and test_gui_text_viewer.py to assert against show_windows['Text Viewer'] instead of the deprecated show_text_viewer attribute.
- Increase synchronization wait time in test_visual_sim_gui_ux.py to ensure the GUI loop accurately reflects the mocked MMA status.
- Display token metrics (input/output/cache) per response in Discussion Hub.
- Add total Discussion Token usage in the panel header.
- Implement 'Compress' feature to intelligently summarize and replace exhausted discussion histories using an AI subagent.
ed