412 lines
20 KiB
Markdown
412 lines
20 KiB
Markdown
# `src/app_controller.py` — Headless Orchestrator & State Hub
|
|
|
|
[Top](../Readme.md) | [Architecture](guide_architecture.md) | [Discussions](guide_discussions.md) | [State Lifecycle](guide_state_lifecycle.md) | [Context Aggregation](guide_context_aggregation.md) | [MMA](guide_mma.md) | [Testing](guide_testing.md)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
`src/app_controller.py` (~166KB) is the **headless controller** that owns the application's state and business logic. It decouples the GUI (`gui_2.py`) from the underlying subsystems (AI, presets, personas, RAG, history, MMA, paths, hot reload).
|
|
|
|
When `--enable-test-hooks` is passed, the controller also spins up the HookServer so external tests/scripts can drive the running app.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────┐
|
|
│ gui_2.py (App) │
|
|
│ - Pure immediate-mode UI │
|
|
│ - Reads app_state for rendering │
|
|
│ - Calls controller methods for mutations │
|
|
└─────────────────┬───────────────────────────────┘
|
|
│ delegates to
|
|
▼
|
|
┌─────────────────────────────────────────────────┐
|
|
│ app_controller.py: AppController │
|
|
│ - State container (AppState) │
|
|
│ - Subsystem coordination (presets, personas, ...) │
|
|
│ - Headless mode: skips GUI init, starts hook │
|
|
│ server on port 8999 │
|
|
│ - Provides _predefined_callbacks and │
|
|
│ _gettable_fields for the Hook API │
|
|
└─────────────────┬───────────────────────────────┘
|
|
│ owns/uses
|
|
▼
|
|
┌─────────────────────────────────────────────────┐
|
|
│ Subsystems │
|
|
│ - PresetManager (src/presets.py) │
|
|
│ - PersonaManager (src/personas.py) │
|
|
│ - ContextPresetManager (src/context_presets.py) │
|
|
│ - ToolPresetManager (src/tool_presets.py) │
|
|
│ - ToolBiasEngine (src/tool_bias.py) │
|
|
│ - RAGEngine (src/rag_engine.py) │
|
|
│ - HistoryManager (src/history.py) │
|
|
│ - WorkspaceManager (src/workspace_manager.py) │
|
|
│ - HookServer (src/api_hooks.py) │
|
|
│ - HotReloader (src/hot_reloader.py) │
|
|
│ - PathManager (src/paths.py) │
|
|
└─────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## The `AppController` Class
|
|
|
|
### `__init__(self, defer_warmup: bool = False, log_to_stderr: Optional[bool] = None)`
|
|
|
|
> **Important:** The `__init__` does NOT create manager objects, does NOT register hooks, and does NOT start the HookServer. The previous documentation in this section **predated the controller refactor** and described an architecture that was never actually implemented (an `AppState` dataclass, an `enable_test_hooks` parameter, a `register_hooks` method, and manager objects that don't exist on the controller).
|
|
|
|
Initializes the controller. Real state created here:
|
|
|
|
The actual `__init__` (`src/app_controller.py:778-1212`) does the following:
|
|
|
|
1. **Startup timeline anchors** — Captures `_init_start_ts` for the `startup_timeline()` diagnostics. Other timeline anchors are filled in lazily as events occur.
|
|
2. **Locks** — Creates **11** thread-safety locks (`_send_thread_lock`, `_disc_entries_lock`, `_pending_*_lock` for comms/tool_calls/history/gui_tasks/dialog/api_event_queue, `_rag_engine_lock`, `_rag_sync_lock`, `_project_switch_lock`) plus 5 non-lock state fields for the RAG-sync coalescing and project-switch state machine (`_rag_sync_token`/`_rag_sync_dirty`, `_project_switch_in_progress`/`_pending_path`/`_error`).
|
|
3. **GUI health state** — `_gui_degraded_reason` and `_last_imgui_assert` (set when `immapp.run` raises `RuntimeError`; see [guide_gui_2.md](guide_gui_2.md#startup-architecture-lazy-imports-profiler-refresh-rate)).
|
|
4. **Shared io_pool** — `make_io_pool()` creates an **8-thread** `ThreadPoolExecutor` named `controller-io-N` (per `IO_POOL_MAX_WORKERS = 8` in `src/io_pool.py:20`; bumped 4→8 in commit `4a338486` on 2026-06-06). This is the SOLE background pool for all async work (no `threading.Thread()` calls anywhere else in `src/`). The module docstring at `src/io_pool.py:1-15` also documents the SIGINT-handler fix that replaced the original atexit approach.
|
|
5. **Warmup manager** — `WarmupManager(self._io_pool, log_to_stderr=log_to_stderr)` with an on-complete callback to stamp `warmup_done_ts`. `defer_warmup=True` defers the actual `start_warmup()` call until the first frame is painted (the desktop GUI pattern; headless mode starts immediately). The `log_to_stderr` parameter honors `SLOP_WARMUP_DEBUG` env var.
|
|
6. **Various flags** — `_warmup_started`, `_pending_fetch_provider`, `_defer_warmup`.
|
|
|
|
**Manager objects** (`preset_manager`, `persona_manager`, `context_preset_manager`, `tool_preset_manager`, `tool_bias_engine`, `history_manager`, `workspace_manager`, `rag_engine`) are **NOT created in `__init__`**. They are lazy attributes accessed via `__getattr__` and created on first reference (typically from `_load_active_project` at `src/app_controller.py:2150` or from `App._post_init` at `src/gui_2.py:492`).
|
|
|
|
**Hook API surface** is NOT populated by a `register_hooks` method. The actual flow:
|
|
1. `AppController._init_actions()` (called from `init_state` at `src/app_controller.py:1740`) populates `self._predefined_callbacks` and `self._gettable_fields` registries via module-level handler registration functions.
|
|
2. `src/api_hooks.py:HookHandler.do_GET` / `do_POST` reads from these registries to expose App methods as `/api/gui` `custom_callback` actions.
|
|
3. The `sloppy.py` CLI parses `--enable-test-hooks` and passes it to `HookServer` (a separate class, not the controller).
|
|
|
|
For the actual init flow, read `src/app_controller.py:778-1212` (`__init__`; the 434-line body is locks + io_pool + warmup manager + ~150 lines of internal/Core/UI/Service state initialization, then the `_settable_fields` map (~75 entries) and `_gui_task_handlers` map (~25 hook actions) at the tail), `:1606` (`_init_actions`), `:1740` (`init_state`), and `:2150` (`_load_active_project`).
|
|
|
|
This is the **only** bridge between the GUI's app methods and the external Hook API. If a method is not in `_predefined_callbacks`, external callers cannot invoke it.
|
|
|
|
### Subsystem Coordination Methods
|
|
|
|
The controller has methods that span multiple subsystems:
|
|
|
|
- `reload_presets()` — re-reads preset TOML files from disk
|
|
- `reload_personas()` — same for personas
|
|
- `reload_context_presets()` — validates files exist
|
|
- `apply_persona(persona_name, target)` — switches model, system prompt, and tool weights for a target
|
|
- `dispatch_mma_track(track_id)` — kicks off a multi-agent track
|
|
- `reset_session()` — clears discussion history, resets UI state, etc.
|
|
- `save_state_to_disk()` / `load_state_from_disk()` — for historical session replay
|
|
- `inject_context_files(paths)` — adds files to the active context composition
|
|
|
|
---
|
|
|
|
## The `AppState` Dataclass
|
|
|
|
`app_state` is a flat dataclass holding all GUI-visible state. Examples:
|
|
|
|
```python
|
|
@dataclass
|
|
class AppState:
|
|
current_provider: str = "gemini"
|
|
current_model: str = "gemini-3-flash-preview"
|
|
temperature: float = 0.7
|
|
top_p: float = 0.95
|
|
max_output_tokens: int = 8192
|
|
system_prompt: str = ""
|
|
discussion_history: list[DiscussionEntry] = field(default_factory=list)
|
|
context_files: list[ContextFileEntry] = field(default_factory=list)
|
|
context_screenshots: list[str] = field(default_factory=list)
|
|
show_command_palette: bool = False
|
|
show_preset_manager: bool = False
|
|
show_persona_editor: bool = False
|
|
show_context_preview: bool = False
|
|
show_diagnostics: bool = False
|
|
show_mma_dashboard: bool = False
|
|
# ... many more
|
|
```
|
|
|
|
The `App` reads from `app_state` for rendering and writes back via setter methods. All setters are exposed to the Hook API via `_gettable_fields` and other "settable" registries.
|
|
|
|
---
|
|
|
|
## Preset & Persona Management
|
|
|
|
### `PresetManager` (in `src/presets.py`)
|
|
|
|
The controller delegates preset CRUD to `PresetManager`. The controller itself only coordinates when presets change (re-apply to active session, update system prompt, etc.).
|
|
|
|
```python
|
|
# In controller
|
|
def on_preset_changed(self, new_preset_name: str) -> None:
|
|
preset = self.presets.get(new_preset_name)
|
|
self.app_state.system_prompt = preset.full_text # Base + persona
|
|
self.app_state.temperature = preset.temperature
|
|
self.app_state.top_p = preset.top_p
|
|
self.app_state.max_output_tokens = preset.max_output_tokens
|
|
```
|
|
|
|
### `PersonaManager` (in `src/personas.py`)
|
|
|
|
Consolidates model settings + system prompt + tool weights into a single named entity.
|
|
|
|
```python
|
|
# In controller
|
|
def apply_persona(self, persona_name: str, target: str = "tier3") -> None:
|
|
persona = self.personas.get(persona_name)
|
|
if persona.model:
|
|
self.app_state.current_model = persona.model
|
|
if persona.system_prompt:
|
|
self.app_state.system_prompt = persona.system_prompt
|
|
if persona.tool_weights:
|
|
self.tool_bias.apply_weights(persona.tool_weights, target=target)
|
|
if persona.bias_profile:
|
|
self.tool_bias.apply_profile(persona.bias_profile)
|
|
```
|
|
|
|
`target` is the MMA tier ("tier1", "tier2", "tier3", "tier4"). This is how MMA agents get isolated cognitive load.
|
|
|
|
### `ContextPresetManager` (in `src/context_presets.py`)
|
|
|
|
Saves/loads complete context compositions (files, screenshots, view modes). Validates that referenced files still exist on load.
|
|
|
|
### `ToolPresetManager` (in `src/tool_presets.py`)
|
|
|
|
Manages tool enable/disable + weights. Persisted to `tool_presets.toml`.
|
|
|
|
### `ToolBiasEngine` (in `src/tool_bias.py`)
|
|
|
|
Applies weights and global bias profiles. Generates the **"Tooling Strategy"** section appended to system prompts.
|
|
|
|
---
|
|
|
|
## History Management
|
|
|
|
`HistoryManager` (in `src/history.py`) implements the **non-provider undo/redo** system.
|
|
|
|
```python
|
|
def on_ui_state_change(self) -> None:
|
|
"""Called when the UI changes (e.g., text input). Pushes a snapshot."""
|
|
snapshot = self.history.capture(self.app_state)
|
|
self.history.push(snapshot)
|
|
self.app_state.can_undo = self.history.can_undo()
|
|
self.app_state.can_redo = self.history.can_redo()
|
|
```
|
|
|
|
Snapshots include:
|
|
- All text inputs (system prompt, AI input, code blocks)
|
|
- Model parameters (Temperature, Top-P, Max Output Tokens)
|
|
- Context (files, screenshots)
|
|
- Discussion history (for discussion mutations)
|
|
|
|
Capacity is fixed (default: 50 snapshots). Older entries are evicted.
|
|
|
|
### Branching History ("Takes")
|
|
|
|
`HistoryManager` also tracks **timeline branching**. When the user reverts and then takes a new action, a new "take" is created. The full history graph is preserved for back-navigation.
|
|
|
|
---
|
|
|
|
## RAG Engine Integration
|
|
|
|
`RAGEngine` (in `src/rag_engine.py`) is owned by the controller but **lazy-loaded** on first use:
|
|
|
|
```python
|
|
def get_rag_engine(self) -> RAGEngine:
|
|
if self._rag_engine is None:
|
|
from src.rag_engine import RAGEngine
|
|
self._rag_engine = RAGEngine(self.paths)
|
|
return self._rag_engine
|
|
```
|
|
|
|
The GUI exposes a RAG settings panel that calls `controller.get_rag_engine().set_provider(...)`, `set_chunk_size(...)`, etc.
|
|
|
|
### RAG Lifecycle
|
|
|
|
1. **Indexing**: `controller.index_project()` walks the project workspace, chunks files, embeds them, writes to ChromaDB (or external MCP).
|
|
2. **Search**: `controller.search_context_files(query)` returns top-k fragments with source paths.
|
|
3. **Injection**: Fragments are prepended to the AI's prompt via `ai_client.send(...)`. The controller orchestrates the flow.
|
|
|
|
---
|
|
|
|
## MMA Conductor Integration
|
|
|
|
`MultiAgentConductor` (in `src/multi_agent_conductor.py`) is also lazy-loaded:
|
|
|
|
```python
|
|
def get_mma_conductor(self) -> 'MultiAgentConductor':
|
|
if self._mma_conductor is None:
|
|
from src.multi_agent_conductor import MultiAgentConductor
|
|
self._mma_conductor = MultiAgentConductor(self)
|
|
return self._mma_conductor
|
|
```
|
|
|
|
The controller passes itself into the conductor so workers can access presets/personas/RAG during execution.
|
|
|
|
### Dispatch Flow
|
|
|
|
```
|
|
controller.dispatch_mma_track(track_id)
|
|
-> conductor.load_track(track_id)
|
|
-> conductor.start_workers(track)
|
|
-> workers run in parallel via WorkerPool
|
|
-> workers call back into controller (presets, personas, etc.)
|
|
-> results pushed to controller.app_state.discussion_history
|
|
-> conductor emits events to controller.event_queue
|
|
```
|
|
|
|
The `event_queue` is consumed by the GUI on the main thread to update display.
|
|
|
|
---
|
|
|
|
## Hot Reload
|
|
|
|
Hot reload is wired in `src/gui_2.py` rather than on the controller. The actual mechanism:
|
|
|
|
- **Registration** (`src/gui_2.py:282-287`): `App.__init__` registers `src.gui_2` with `HotReloader`, listing the App attributes (`state_keys`) to snapshot before reload and the App wrapper methods (`delegation_targets`) that the delegation pattern swaps atomically.
|
|
- **Trigger** (`src/gui_2.py:540-544`): `App._trigger_hot_reload()` calls `HotReloader.reload_all(self)` and stores `HotReloader.last_error` on `self._hot_reload_error` for visual error feedback.
|
|
- **Keyboard binding** (`src/gui_2.py:5340-5346`): the `Ctrl+Alt+R` shortcut is hard-coded in the source — there is no `config.toml` key for it.
|
|
|
|
`HotReloader` is a stateless class (classmethods only); it has no constructor and no `self.app` field. See **[docs/guide_hot_reload.md](guide_hot_reload.md)** for the full mechanism.
|
|
|
|
See **[docs/guide_hot_reload.md](guide_hot_reload.md)** for the full mechanism.
|
|
|
|
---
|
|
|
|
## The `SyncEventQueue`
|
|
|
|
A `queue.Queue`-based bridge between the daemon threads (AI workers, MMA workers) and the GUI main thread.
|
|
|
|
```python
|
|
class SyncEventQueue:
|
|
def put(self, event: Event) -> None: ...
|
|
def get_nowait(self) -> Event | None: ...
|
|
def get_all(self) -> list[Event]: ...
|
|
```
|
|
|
|
The GUI polls `controller.event_queue.get_all()` once per frame and dispatches events to render functions.
|
|
|
|
### Event Types
|
|
|
|
- `MMA_TICKET_COMPLETED`
|
|
- `MMA_LOG_MESSAGE`
|
|
- `AI_RESPONSE_CHUNK`
|
|
- `AI_RESPONSE_COMPLETE`
|
|
- `TOOL_CALL_STARTED`
|
|
- `TOOL_CALL_COMPLETED`
|
|
- `PERSONA_APPLIED`
|
|
- `WORKSPACE_LOADED`
|
|
- `RAG_INDEX_COMPLETE`
|
|
- `HOOK_CALLBACK_RECEIVED`
|
|
|
|
The controller translates subsystem-specific events into these generic types.
|
|
|
|
---
|
|
|
|
## The Headless Mode
|
|
|
|
When `sloppy.py` is launched with `--headless`, the controller is instantiated without an `App`:
|
|
|
|
```python
|
|
# In sloppy.py --headless
|
|
controller = AppController(enable_test_hooks=True)
|
|
# ... run server-only logic ...
|
|
# No GUI, no ImGui context, but full subsystem access
|
|
```
|
|
|
|
This is the **Headless Backend Service** mode. The controller still listens on `:8999` and serves all Hook API endpoints. Tests and external scripts can drive the headless service.
|
|
|
|
---
|
|
|
|
## Hook API Surface (Defined Here)
|
|
|
|
The controller is the **single source of truth** for what the Hook API can do. Three registries:
|
|
|
|
### `_predefined_callbacks: dict[str, Callable]`
|
|
|
|
Maps hook name → App method. Populated by `register_hooks(app)`.
|
|
|
|
```python
|
|
self._predefined_callbacks['_toggle_command_palette'] = app._toggle_command_palette
|
|
self._predefined_callbacks['_open_command_palette'] = app._open_command_palette
|
|
self._predefined_callbacks['save_context_preset'] = app.save_context_preset
|
|
self._predefined_callbacks['load_context_preset'] = app.load_context_preset
|
|
# ... ~30 entries
|
|
```
|
|
|
|
### `_gettable_fields: dict[str, str]`
|
|
|
|
Maps hook name → AppState field name. Used by `get_value` Hook API action.
|
|
|
|
```python
|
|
self._gettable_fields['show_command_palette'] = 'show_command_palette'
|
|
self._gettable_fields['current_provider'] = 'current_provider'
|
|
self._gettable_fields['current_model'] = 'current_model'
|
|
# ... etc
|
|
```
|
|
|
|
### `_action_handlers: dict[str, Callable]`
|
|
|
|
Maps action name (e.g., `"click"`, `"set_value"`, `"custom_callback"`) → handler.
|
|
|
|
```python
|
|
self._action_handlers['click'] = self._handle_click
|
|
self._action_handlers['set_value'] = self._handle_set_value
|
|
self._action_handlers['custom_callback'] = self._handle_custom_callback
|
|
# ... etc
|
|
```
|
|
|
|
The HookServer in `src/api_hooks.py` consumes these registries to route incoming requests.
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
The controller is tested via the `live_gui` fixture (full integration) and targeted unit tests.
|
|
|
|
### Unit Tests
|
|
|
|
- `tests/test_app_controller_init.py` — instantiation, subsystem wiring
|
|
- `tests/test_preset_manager.py` — preset CRUD
|
|
- `tests/test_persona_manager.py` — persona CRUD + application
|
|
- `tests/test_context_presets.py` — context preset CRUD + file validation
|
|
- `tests/test_history.py` — undo/redo
|
|
- `tests/test_workspace_manager.py` — workspace profile CRUD
|
|
|
|
### Integration Tests (live_gui)
|
|
|
|
Use the `ApiHookClient` to drive the controller and verify state mutations.
|
|
|
|
```python
|
|
def test_apply_persona(live_gui):
|
|
client = ApiHookClient()
|
|
client.push_event("custom_callback", {"callback": "apply_persona", "args": ["code-reviewer"]})
|
|
time.sleep(0.5)
|
|
model = client.get_value("current_model")
|
|
assert "code" in model.lower() or model == "code-reviewer-model"
|
|
```
|
|
|
|
---
|
|
|
|
## Common Pitfalls
|
|
|
|
1. **Don't instantiate `AppController` twice in the same process**: The singleton holds RAG engine, MMA conductor, hook server. A second instance would conflict.
|
|
2. **Don't read `app_state` from a daemon thread without locking**: Use `event_queue` for cross-thread communication.
|
|
3. **Always go through the controller for subsystem changes**: Don't call `self.presets.save(...)` from the GUI directly; call `controller.save_preset(...)` so the event is broadcast.
|
|
4. **When adding a new Hook API callback, register it in BOTH `_predefined_callbacks` AND `register_hooks`**: The latter is what populates the registry from an App instance.
|
|
|
|
---
|
|
|
|
## See Also
|
|
|
|
- **[guide_architecture.md](guide_architecture.md)** — Threading and event flow
|
|
- **[guide_mma.md](guide_mma.md)** — How MMA workers use the controller
|
|
- **[guide_ai_client.md](guide_ai_client.md)** — How `ai_client` integrates
|
|
- **[guide_api_hooks.md](guide_api_hooks.md)** — The Hook API the controller exposes
|
|
- **[guide_hot_reload.md](guide_hot_reload.md)** — How the controller supports state-preserving reloads
|
|
- **[guide_discussions.md](guide_discussions.md)** — The Discussion system (Takes, branching, `_switch_discussion`, `_branch_discussion`, `_rename_discussion`, `_delete_discussion`, `_flush_disc_entries_to_project`)
|
|
- **[guide_state_lifecycle.md](guide_state_lifecycle.md)** — The `_handle_reset_session` and `_handle_compress_discussion` flows, the `App.__getattr__`/`__setattr__` state delegation pattern, and the `HistoryManager` integration
|
|
- **[guide_context_aggregation.md](guide_context_aggregation.md)** — The `aggregate.py` pipeline that the controller calls per send (per-provider + Tier 3 worker)
|
|
- **`src/presets.py`, `src/personas.py`, `src/context_presets.py`, `src/tool_presets.py`, `src/tool_bias.py`** — Subsystem managers
|
|
- **`src/history.py`** — `HistoryManager`
|
|
- **`src/rag_engine.py`** — `RAGEngine`
|
|
- **`src/multi_agent_conductor.py`** — `MultiAgentConductor`
|
|
- **`src/hot_reloader.py`** — `HotReloader`
|
|
- **`src/api_hooks.py`** — `HookServer` (uses the controller's registries)
|
|
- **`src/paths.py`** — `PathManager`
|
|
- **[conductor/tracks/nagent_review_20260608/report.md](../conductor/tracks/nagent_review_20260608/report.md)** — Deep-dive analysis of the controller's per-provider history globals and other state patterns
|