docs(app-controller): add guide_app_controller.md

2026-06-02 23:35:29 -04:00
parent a58a2fd887
commit 6415e84994
1 changed files with 447 additions and 0 deletions
@@ -0,0 +1,447 @@
+# `src/app_controller.py` — Headless Orchestrator & State Hub
+
+[Top](../README.md) | [Architecture](guide_architecture.md) | [MMA](guide_mma.md) | [Testing](guide_testing.md)
+
+---
+
+## Overview
+
+`src/app_controller.py` (~166KB) is the **headless controller** that owns the application's state and business logic. It decouples the GUI (`gui_2.py`) from the underlying subsystems (AI, presets, personas, RAG, history, MMA, paths, hot reload).
+
+When `--enable-test-hooks` is passed, the controller also spins up the HookServer so external tests/scripts can drive the running app.
+
+---
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────┐
+│  gui_2.py (App)                                   │
+│  - Pure immediate-mode UI                        │
+│  - Reads app_state for rendering                  │
+│  - Calls controller methods for mutations         │
+└─────────────────┬───────────────────────────────┘
+                  │ delegates to
+                  ▼
+┌─────────────────────────────────────────────────┐
+│  app_controller.py: AppController                 │
+│  - State container (AppState)                     │
+│  - Subsystem coordination (presets, personas, ...) │
+│  - Headless mode: skips GUI init, starts hook     │
+│    server on port 8999                            │
+│  - Provides _predefined_callbacks and             │
+│    _gettable_fields for the Hook API              │
+└─────────────────┬───────────────────────────────┘
+                  │ owns/uses
+                  ▼
+┌─────────────────────────────────────────────────┐
+│  Subsystems                                       │
+│  - PresetManager (src/presets.py)                  │
+│  - PersonaManager (src/personas.py)                │
+│  - ContextPresetManager (src/context_presets.py)   │
+│  - ToolPresetManager (src/tool_presets.py)         │
+│  - ToolBiasEngine (src/tool_bias.py)               │
+│  - RAGEngine (src/rag_engine.py)                   │
+│  - HistoryManager (src/history.py)                 │
+│  - WorkspaceManager (src/workspace_manager.py)     │
+│  - HookServer (src/api_hooks.py)                   │
+│  - HotReloader (src/hot_reload.py)                 │
+│  - PathManager (src/paths.py)                      │
+└─────────────────────────────────────────────────┘
+```
+
+---
+
+## The `AppController` Class
+
+### `__init__(self, enable_test_hooks: bool = False)`
+
+Initializes the controller. Key state:
+
+```python
+class AppController:
+    def __init__(self, enable_test_hooks: bool = False):
+        # 1. Path resolution (src/paths.py)
+        self.paths = PathManager()
+
+        # 2. State container
+        self.app_state = AppState()
+
+        # 3. Subsystem managers
+        self.presets = PresetManager(self.paths)
+        self.personas = PersonaManager(self.paths)
+        self.context_presets = ContextPresetManager(self.paths)
+        self.tool_presets = ToolPresetManager(self.paths)
+        self.tool_bias = ToolBiasEngine()
+        self.history = HistoryManager(self.paths)
+        self.workspace = WorkspaceManager(self.paths)
+        self.rag_engine = RAGEngine(self.paths)  # Lazy
+
+        # 4. Hook API surface
+        self._predefined_callbacks: dict[str, Callable] = {}
+        self._gettable_fields: dict[str, str] = {}
+
+        # 5. AI client (lazy)
+        self.ai_client = None
+
+        # 6. MMA conductor (lazy)
+        self.mma_conductor = None
+
+        # 7. Sync event queue (daemon <-> UI bridge)
+        self.event_queue = SyncEventQueue()
+
+        # 8. Optional hook server
+        if enable_test_hooks:
+            self.hook_server = HookServer()
+            self.hook_server.start()
+```
+
+The `App` (in `gui_2.py`) then reads `controller.app_state`, `controller.presets`, etc. for rendering.
+
+### `register_hooks(app: App)`
+
+Called by `gui_2.py` after instantiation. The controller populates the predefined callbacks and gettable fields that the Hook API can invoke.
+
+```python
+def register_hooks(self, app: 'App') -> None:
+    """Register App methods as predefined callbacks for the Hook API."""
+    self._predefined_callbacks['_toggle_command_palette'] = app._toggle_command_palette
+    self._predefined_callbacks['_open_command_palette'] = app._open_command_palette
+    # ... etc, many more ...
+    self._gettable_fields['show_command_palette'] = 'show_command_palette'
+    self._gettable_fields['current_provider'] = 'current_provider'
+    # ... etc ...
+```
+
+This is the **only** bridge between the GUI's app methods and the external Hook API. If a method is not in `_predefined_callbacks`, external callers cannot invoke it.
+
+### Subsystem Coordination Methods
+
+The controller has methods that span multiple subsystems:
+
+- `reload_presets()` — re-reads preset TOML files from disk
+- `reload_personas()` — same for personas
+- `reload_context_presets()` — validates files exist
+- `apply_persona(persona_name, target)` — switches model, system prompt, and tool weights for a target
+- `dispatch_mma_track(track_id)` — kicks off a multi-agent track
+- `reset_session()` — clears discussion history, resets UI state, etc.
+- `save_state_to_disk()` / `load_state_from_disk()` — for historical session replay
+- `inject_context_files(paths)` — adds files to the active context composition
+
+---
+
+## The `AppState` Dataclass
+
+`app_state` is a flat dataclass holding all GUI-visible state. Examples:
+
+```python
+@dataclass
+class AppState:
+    current_provider: str = "gemini"
+    current_model: str = "gemini-3-flash-preview"
+    temperature: float = 0.7
+    top_p: float = 0.95
+    max_output_tokens: int = 8192
+    system_prompt: str = ""
+    discussion_history: list[DiscussionEntry] = field(default_factory=list)
+    context_files: list[ContextFileEntry] = field(default_factory=list)
+    context_screenshots: list[str] = field(default_factory=list)
+    show_command_palette: bool = False
+    show_preset_manager: bool = False
+    show_persona_editor: bool = False
+    show_context_preview: bool = False
+    show_diagnostics: bool = False
+    show_mma_dashboard: bool = False
+    # ... many more
+```
+
+The `App` reads from `app_state` for rendering and writes back via setter methods. All setters are exposed to the Hook API via `_gettable_fields` and other "settable" registries.
+
+---
+
+## Preset & Persona Management
+
+### `PresetManager` (in `src/presets.py`)
+
+The controller delegates preset CRUD to `PresetManager`. The controller itself only coordinates when presets change (re-apply to active session, update system prompt, etc.).
+
+```python
+# In controller
+def on_preset_changed(self, new_preset_name: str) -> None:
+    preset = self.presets.get(new_preset_name)
+    self.app_state.system_prompt = preset.full_text  # Base + persona
+    self.app_state.temperature = preset.temperature
+    self.app_state.top_p = preset.top_p
+    self.app_state.max_output_tokens = preset.max_output_tokens
+```
+
+### `PersonaManager` (in `src/personas.py`)
+
+Consolidates model settings + system prompt + tool weights into a single named entity.
+
+```python
+# In controller
+def apply_persona(self, persona_name: str, target: str = "tier3") -> None:
+    persona = self.personas.get(persona_name)
+    if persona.model:
+        self.app_state.current_model = persona.model
+    if persona.system_prompt:
+        self.app_state.system_prompt = persona.system_prompt
+    if persona.tool_weights:
+        self.tool_bias.apply_weights(persona.tool_weights, target=target)
+    if persona.bias_profile:
+        self.tool_bias.apply_profile(persona.bias_profile)
+```
+
+`target` is the MMA tier ("tier1", "tier2", "tier3", "tier4"). This is how MMA agents get isolated cognitive load.
+
+### `ContextPresetManager` (in `src/context_presets.py`)
+
+Saves/loads complete context compositions (files, screenshots, view modes). Validates that referenced files still exist on load.
+
+### `ToolPresetManager` (in `src/tool_presets.py`)
+
+Manages tool enable/disable + weights. Persisted to `tool_presets.toml`.
+
+### `ToolBiasEngine` (in `src/tool_bias.py`)
+
+Applies weights and global bias profiles. Generates the **"Tooling Strategy"** section appended to system prompts.
+
+---
+
+## History Management
+
+`HistoryManager` (in `src/history.py`) implements the **non-provider undo/redo** system.
+
+```python
+def on_ui_state_change(self) -> None:
+    """Called when the UI changes (e.g., text input). Pushes a snapshot."""
+    snapshot = self.history.capture(self.app_state)
+    self.history.push(snapshot)
+    self.app_state.can_undo = self.history.can_undo()
+    self.app_state.can_redo = self.history.can_redo()
+```
+
+Snapshots include:
+- All text inputs (system prompt, AI input, code blocks)
+- Model parameters (Temperature, Top-P, Max Output Tokens)
+- Context (files, screenshots)
+- Discussion history (for discussion mutations)
+
+Capacity is fixed (default: 50 snapshots). Older entries are evicted.
+
+### Branching History ("Takes")
+
+`HistoryManager` also tracks **timeline branching**. When the user reverts and then takes a new action, a new "take" is created. The full history graph is preserved for back-navigation.
+
+---
+
+## RAG Engine Integration
+
+`RAGEngine` (in `src/rag_engine.py`) is owned by the controller but **lazy-loaded** on first use:
+
+```python
+def get_rag_engine(self) -> RAGEngine:
+    if self._rag_engine is None:
+        from src.rag_engine import RAGEngine
+        self._rag_engine = RAGEngine(self.paths)
+    return self._rag_engine
+```
+
+The GUI exposes a RAG settings panel that calls `controller.get_rag_engine().set_provider(...)`, `set_chunk_size(...)`, etc.
+
+### RAG Lifecycle
+
+1. **Indexing**: `controller.index_project()` walks the project workspace, chunks files, embeds them, writes to ChromaDB (or external MCP).
+2. **Search**: `controller.search_context_files(query)` returns top-k fragments with source paths.
+3. **Injection**: Fragments are prepended to the AI's prompt via `ai_client.send(...)`. The controller orchestrates the flow.
+
+---
+
+## MMA Conductor Integration
+
+`MultiAgentConductor` (in `src/multi_agent_conductor.py`) is also lazy-loaded:
+
+```python
+def get_mma_conductor(self) -> 'MultiAgentConductor':
+    if self._mma_conductor is None:
+        from src.multi_agent_conductor import MultiAgentConductor
+        self._mma_conductor = MultiAgentConductor(self)
+    return self._mma_conductor
+```
+
+The controller passes itself into the conductor so workers can access presets/personas/RAG during execution.
+
+### Dispatch Flow
+
+```
+controller.dispatch_mma_track(track_id)
+  -> conductor.load_track(track_id)
+  -> conductor.start_workers(track)
+  -> workers run in parallel via WorkerPool
+  -> workers call back into controller (presets, personas, etc.)
+  -> results pushed to controller.app_state.discussion_history
+  -> conductor emits events to controller.event_queue
+```
+
+The `event_queue` is consumed by the GUI on the main thread to update display.
+
+---
+
+## Hot Reload
+
+The controller can hot-reload Python modules while preserving state. This is critical for GUI iteration:
+
+```python
+def hot_reload(self, module_name: str) -> None:
+    """Reload a module and re-apply its render functions to the app."""
+    from src.hot_reload import HotReloader
+    reloader = HotReloader(self.app)
+    reloader.reload(module_name)
+```
+
+`gui_2.py` registers all its render functions with the reloader at startup. On reload, the reloader swaps the function references without losing app state.
+
+See **[docs/guide_hot_reload.md](guide_hot_reload.md)** for the full mechanism.
+
+---
+
+## The `SyncEventQueue`
+
+A `queue.Queue`-based bridge between the daemon threads (AI workers, MMA workers) and the GUI main thread.
+
+```python
+class SyncEventQueue:
+    def put(self, event: Event) -> None: ...
+    def get_nowait(self) -> Event | None: ...
+    def get_all(self) -> list[Event]: ...
+```
+
+The GUI polls `controller.event_queue.get_all()` once per frame and dispatches events to render functions.
+
+### Event Types
+
+- `MMA_TICKET_COMPLETED`
+- `MMA_LOG_MESSAGE`
+- `AI_RESPONSE_CHUNK`
+- `AI_RESPONSE_COMPLETE`
+- `TOOL_CALL_STARTED`
+- `TOOL_CALL_COMPLETED`
+- `PERSONA_APPLIED`
+- `WORKSPACE_LOADED`
+- `RAG_INDEX_COMPLETE`
+- `HOOK_CALLBACK_RECEIVED`
+
+The controller translates subsystem-specific events into these generic types.
+
+---
+
+## The Headless Mode
+
+When `sloppy.py` is launched with `--headless`, the controller is instantiated without an `App`:
+
+```python
+# In sloppy.py --headless
+controller = AppController(enable_test_hooks=True)
+# ... run server-only logic ...
+# No GUI, no ImGui context, but full subsystem access
+```
+
+This is the **Headless Backend Service** mode. The controller still listens on `:8999` and serves all Hook API endpoints. Tests and external scripts can drive the headless service.
+
+---
+
+## Hook API Surface (Defined Here)
+
+The controller is the **single source of truth** for what the Hook API can do. Three registries:
+
+### `_predefined_callbacks: dict[str, Callable]`
+
+Maps hook name → App method. Populated by `register_hooks(app)`.
+
+```python
+self._predefined_callbacks['_toggle_command_palette'] = app._toggle_command_palette
+self._predefined_callbacks['_open_command_palette'] = app._open_command_palette
+self._predefined_callbacks['save_context_preset'] = app.save_context_preset
+self._predefined_callbacks['load_context_preset'] = app.load_context_preset
+# ... ~30 entries
+```
+
+### `_gettable_fields: dict[str, str]`
+
+Maps hook name → AppState field name. Used by `get_value` Hook API action.
+
+```python
+self._gettable_fields['show_command_palette'] = 'show_command_palette'
+self._gettable_fields['current_provider'] = 'current_provider'
+self._gettable_fields['current_model'] = 'current_model'
+# ... etc
+```
+
+### `_action_handlers: dict[str, Callable]`
+
+Maps action name (e.g., `"click"`, `"set_value"`, `"custom_callback"`) → handler.
+
+```python
+self._action_handlers['click'] = self._handle_click
+self._action_handlers['set_value'] = self._handle_set_value
+self._action_handlers['custom_callback'] = self._handle_custom_callback
+# ... etc
+```
+
+The HookServer in `src/api_hooks.py` consumes these registries to route incoming requests.
+
+---
+
+## Testing
+
+The controller is tested via the `live_gui` fixture (full integration) and targeted unit tests.
+
+### Unit Tests
+
+- `tests/test_app_controller_init.py` — instantiation, subsystem wiring
+- `tests/test_preset_manager.py` — preset CRUD
+- `tests/test_persona_manager.py` — persona CRUD + application
+- `tests/test_context_presets.py` — context preset CRUD + file validation
+- `tests/test_history.py` — undo/redo
+- `tests/test_workspace_manager.py` — workspace profile CRUD
+
+### Integration Tests (live_gui)
+
+Use the `ApiHookClient` to drive the controller and verify state mutations.
+
+```python
+def test_apply_persona(live_gui):
+    client = ApiHookClient()
+    client.push_event("custom_callback", {"callback": "apply_persona", "args": ["code-reviewer"]})
+    time.sleep(0.5)
+    model = client.get_value("current_model")
+    assert "code" in model.lower() or model == "code-reviewer-model"
+```
+
+---
+
+## Common Pitfalls
+
+1. **Don't instantiate `AppController` twice in the same process**: The singleton holds RAG engine, MMA conductor, hook server. A second instance would conflict.
+2. **Don't read `app_state` from a daemon thread without locking**: Use `event_queue` for cross-thread communication.
+3. **Always go through the controller for subsystem changes**: Don't call `self.presets.save(...)` from the GUI directly; call `controller.save_preset(...)` so the event is broadcast.
+4. **When adding a new Hook API callback, register it in BOTH `_predefined_callbacks` AND `register_hooks`**: The latter is what populates the registry from an App instance.
+
+---
+
+## See Also
+
+- **[guide_architecture.md](guide_architecture.md)** — Threading and event flow
+- **[guide_mma.md](guide_mma.md)** — How MMA workers use the controller
+- **[guide_ai_client.md](guide_ai_client.md)** — How `ai_client` integrates
+- **[guide_api_hooks.md](guide_api_hooks.md)** — The Hook API the controller exposes
+- **[guide_hot_reload.md](guide_hot_reload.md)** — How the controller supports state-preserving reloads
+- **[guide_history.md](guide_history.md)** — Undo/redo (planned, not yet written)
+- **`src/presets.py`, `src/personas.py`, `src/context_presets.py`, `src/tool_presets.py`, `src/tool_bias.py`** — Subsystem managers
+- **`src/history.py`** — `HistoryManager`
+- **`src/rag_engine.py`** — `RAGEngine`
+- **`src/multi_agent_conductor.py`** — `MultiAgentConductor`
+- **`src/hot_reload.py`** — `HotReloader`
+- **`src/api_hooks.py`** — `HookServer` (uses the controller's registries)
+- **`src/paths.py`** — `PathManager`