docs(conductor): Expert-level architectural documentation refresh

conductor/tracks.md

@@ -13,3 +13,10 @@ This file tracks all major tracks for the project. Each track has its own detail
 
 - [ ] **Track: Comprehensive Conductor & MMA GUI UX**
 *Link: [./tracks/comprehensive_gui_ux_20260228/](./tracks/comprehensive_gui_ux_20260228/)*
+
+
+
+---
+
+- [x] **Track: Deep Architectural Documentation Refresh**
+*Link: [./tracks/documentation_refresh_20260224/](./tracks/documentation_refresh_20260224/)*

conductor/tracks/documentation_refresh_20260224/plan.md

@@ -1,38 +1,38 @@
 # Implementation Plan: Deep Architectural Documentation Refresh
 
 ## Phase 1: Context Cleanup & Research
-- [ ] Task: Audit references to `MainContext.md` across the project.
-- [ ] Task: Delete `MainContext.md` and update any identified references.
-- [ ] Task: Execute `py_get_skeleton` and `py_get_code_outline` for `events.py`, `api_hooks.py`, `api_hook_client.py`, and `gui_2.py` to create a technical map for the guides.
-- [ ] Task: Analyze the `live_gui` fixture in `tests/conftest.py` and the simulation loop in `tests/visual_sim_mma_v2.py`.
+- [x] Task: Audit references to `MainContext.md` across the project.
+- [x] Task: Delete `MainContext.md` and update any identified references.
+- [x] Task: Execute `py_get_skeleton` and `py_get_code_outline` for `events.py`, `api_hooks.py`, `api_hook_client.py`, and `gui_2.py` to create a technical map for the guides.
+- [x] Task: Analyze the `live_gui` fixture in `tests/conftest.py` and the simulation loop in `tests/visual_sim_mma_v2.py`.
 
 ## Phase 2: Core Architecture Deep Dive
 Update `docs/guide_architecture.md` with expert-level detail.
-- [ ] Task: Document the Dual-Threaded App Lifetime: Main GUI loop vs. Daemon execution threads.
-- [ ] Task: Detail the `AsyncEventQueue` and `EventEmitter` roles in the decoupling strategy.
-- [ ] Task: Explain the `_pending_gui_tasks` synchronization mechanism for bridging the Hook Server and GUI.
-- [ ] Task: Document the "Linear Execution Clutch" and its deterministic state machine.
-- [ ] Task: Verify the architectural descriptions against the actual implementation.
-- [ ] Task: Conductor - User Manual Verification 'Phase 2: Core Architecture Deep Dive' (Protocol in workflow.md)
+- [x] Task: Document the Dual-Threaded App Lifetime: Main GUI loop vs. Daemon execution threads.
+- [x] Task: Detail the `AsyncEventQueue` and `EventEmitter` roles in the decoupling strategy.
+- [x] Task: Explain the `_pending_gui_tasks` synchronization mechanism for bridging the Hook Server and GUI.
+- [x] Task: Document the "Linear Execution Clutch" and its deterministic state machine.
+- [x] Task: Verify the architectural descriptions against the actual implementation.
+- [x] Task: Conductor - User Manual Verification 'Phase 2: Core Architecture Deep Dive' (Protocol in workflow.md)
 
 ## Phase 3: Hook System & Tooling Technical Reference
 Update `docs/guide_tools.md` to include low-level API details.
-- [ ] Task: Create a comprehensive API reference for all `HookServer` endpoints.
-- [ ] Task: Document the `ApiHookClient` implementation, including retries and polling strategies.
-- [ ] Task: Update the MCP toolset guide with current native tool implementations.
-- [ ] Task: Document the `ask/respond` IPC flow for "Human-in-the-Loop" confirmations.
-- [ ] Task: Conductor - User Manual Verification 'Phase 3: Hook System & Tooling Technical Reference' (Protocol in workflow.md)
+- [x] Task: Create a comprehensive API reference for all `HookServer` endpoints.
+- [x] Task: Document the `ApiHookClient` implementation, including retries and polling strategies.
+- [x] Task: Update the MCP toolset guide with current native tool implementations.
+- [x] Task: Document the `ask/respond` IPC flow for "Human-in-the-Loop" confirmations.
+- [x] Task: Conductor - User Manual Verification 'Phase 3: Hook System & Tooling Technical Reference' (Protocol in workflow.md)
 
 ## Phase 4: Verification & Simulation Framework
 Create the new `docs/guide_simulations.md` guide.
-- [ ] Task: Detail the Live GUI testing infrastructure: `--enable-test-hooks` and the `live_gui` fixture.
-- [ ] Task: Breakdown the Simulation Lifecycle: Startup, Polling, Interaction, and Assertion.
-- [ ] Task: Document the mock provider strategy using `tests/mock_gemini_cli.py`.
-- [ ] Task: Provide examples of visual verification tests (e.g., MMA lifecycle).
-- [ ] Task: Conductor - User Manual Verification 'Phase 4: Verification & Simulation Framework' (Protocol in workflow.md)
+- [x] Task: Detail the Live GUI testing infrastructure: `--enable-test-hooks` and the `live_gui` fixture.
+- [x] Task: Breakdown the Simulation Lifecycle: Startup, Polling, Interaction, and Assertion.
+- [x] Task: Document the mock provider strategy using `tests/mock_gemini_cli.py`.
+- [x] Task: Provide examples of visual verification tests (e.g., MMA lifecycle).
+- [x] Task: Conductor - User Manual Verification 'Phase 4: Verification & Simulation Framework' (Protocol in workflow.md)
 
 ## Phase 5: README & Roadmap Update
-- [ ] Task: Update `Readme.md` with current setup (`uv`, `credentials.toml`) and vision.
-- [ ] Task: Perform a project-wide link validation of all Markdown files.
-- [ ] Task: Verify the high-density information style across all documentation.
-- [ ] Task: Conductor - User Manual Verification 'Phase 5: README & Roadmap Update' (Protocol in workflow.md)
+- [x] Task: Update `Readme.md` with current setup (`uv`, `credentials.toml`) and vision.
+- [x] Task: Perform a project-wide link validation of all Markdown files.
+- [x] Task: Verify the high-density information style across all documentation.
+- [x] Task: Conductor - User Manual Verification 'Phase 5: README & Roadmap Update' (Protocol in workflow.md)

conductor/tracks/documentation_refresh_20260224/spec.md

@@ -1,49 +1,45 @@
 # Track Specification: Deep Architectural Documentation Refresh
 
 ## Overview
-This track implements a high-density, expert-level documentation suite for the Manual Slop project, focusing on the complex asynchronous orchestration, the 4-Tier MMA (Hierarchical Multi-Model Architecture), and the robust visual simulation framework. The documentation style adheres to the "USA Graphics Company" values of high information density, tactile interaction, and deep technical transparency, inspired by the architectural guides of `VEFontCache-Odin` and `gencpp`.
+This track implements a high-density, expert-level documentation suite for the Manual Slop project. The documentation style is strictly modeled after the **pedagogical and narrative standards** of `gencpp` and `VEFontCache-Odin`. It moves beyond simple "User Guides" to provide a **"USA Graphics Company"** architectural reference: high information density, tactical technical transparency, and a narrative intent that guides a developer from high-level philosophy to low-level implementation.
 
-## Goals
-1.  **Architectural Deep Dive:** Document the core engine, focusing on state management, asynchronous event loops, and the decoupling of the GUI from the AI client.
-2.  **Hook System Technical Reference:** Provide a low-level guide to the `HookServer`, `HookHandler`, and the IPC mechanism used for automated verification.
-3.  **Simulation & Testing Framework:** Detail the live GUI simulation strategy, including the use of `pytest` fixtures, `ApiHookClient`, and mock providers.
-4.  **MMA Orchestration Logic:** Document the 4-Tier hierarchy (Orchestrator, Tech Lead, Worker, QA), token firewalling, and the DAG-based task execution engine.
-5.  **Event Handling System:** Explain the dual event architecture using both synchronous `EventEmitter` and asynchronous `AsyncEventQueue`.
+## Pedagogical Goals
+1.  **Narrative Intent:** Documentation must transition the reader through a logical learning journey: **Philosophy/Mental Model -> Architectural Boundaries -> Implementation Logic -> Verification/Simulation.**
+2.  **High Information Density:** Eliminate conversational filler and "fluff." Every sentence must provide architectural signal (state transitions, data flows, constraints).
+3.  **Technical Transparency:** Document the "How" and "Why" behind design decisions (e.g., *Why* the dual-threaded `Asyncio` loop? *How* does the "Execution Clutch" bridge the thread gap?).
+4.  **Architectural Mapping:** Use precise symbol names (`AsyncEventQueue`, `_pending_gui_tasks`, `HookServer`) to map the documentation directly to the source code.
+5.  **Multi-Layered Depth:** Each major component (Architecture, Tools, Simulations) must have its own dedicated, expert-level guide. No consolidation into single, shallow files.
 
 ## Functional Requirements (Documentation Areas)
 
 ### 1. Core Architecture (`docs/guide_architecture.md`)
--   **Lifetime & Execution Loop:** Detailed breakdown of the application startup, the interaction between the main thread and daemon threads, and the event-driven updates.
--   **Asynchronous Orchestration:** Documentation of the `AsyncEventQueue` and how it prevents GUI blocking during long-running AI operations.
--   **State Synchronization:** How the `_pending_gui_tasks` queue and `threading.Lock` are used to safely bridge the Hook Server thread and the Main GUI thread.
--   **Linear Execution Clutch:** Explanation of the deterministic debugging mode and the step-through execution logic.
+-   **System Philosophy:** The "Decoupled State Machine" mental model.
+-   **Application Lifetime:** The multi-threaded boot sequence and the "Dual-Flush" persistence model.
+-   **The Task Pipeline:** Detailed producer-consumer synchronization between the GUI (Main) and AI (Daemon) threads.
+-   **The Execution Clutch (HITL):** Detailed state machine for human-in-the-loop interception and payload mutation.
 
-### 2. Tooling & Hooks (`docs/guide_tools.md`)
--   **IPC & Hook API:** A complete reference for all `/api/` endpoints, including data structures for `mma_status`, `diagnostics`, and `ask/respond`.
--   **ApiHookClient:** Usage guide for the requests-based client, detailing polling strategies and retry logic.
--   **MCP Toolset:** Updated documentation for the native MCP-like tools (file I/O, search, web fetch) and their security allowlist.
+### 2. Tooling & IPC Reference (`docs/guide_tools.md`)
+-   **MCP Bridge:** Low-level security constraints and filesystem sandboxing.
+-   **Hook API:** A full technical reference for the REST/IPC interface (endpoints, payloads, diagnostics).
+-   **IPC Flow:** The `ask/respond` sequence for synchronous human-in-the-loop requests.
 
-### 3. Verification & Simulation (`docs/guide_simulations.md` - NEW)
--   **Live GUI Verification:** How to run the app with `--enable-test-hooks` and use the `live_gui` fixture for end-to-end testing.
--   **Visual Simulation Lifecycle:** Detailed breakdown of `tests/visual_sim_mma_v2.py`, explaining how it simulates user interaction and verifies rendered state (e.g., DAG nodes, stream output).
--   **Mocking AI Providers:** How to use `tests/mock_gemini_cli.py` to test the full orchestration loop without incurring API costs.
+### 3. Verification & Simulation Framework (`docs/guide_simulations.md`)
+-   **Infrastructure:** The `--enable-test-hooks` flag and the `live_gui` pytest fixture.
+-   **Lifecycle:** The "Puppeteer" pattern for driving the GUI via automated clients.
+-   **Mocking Strategy:** Script-based AI provider mocking via `mock_gemini_cli.py`.
+-   **Visual Assertion:** Examples of verifying the rendered state (DAG, Terminal streams) rather than just API returns.
 
-### 4. README & Vision (`Readme.md`)
--   **Setup & Telemetry:** Updated instructions for `uv` and `credentials.toml`.
--   **Feature Roadmap:** Highlighting the MMA Dashboard, DAG engine, and the 4-Tier delegation model.
+### 4. Product Vision & Roadmap (`Readme.md`)
+-   **Technological Identity:** High-density experimental tool for local AI orchestration.
+-   **Pedagogical Landing:** Direct links to the deep-dive guides to establish the project's expert-level tone immediately.
 
-## Non-Functional Requirements
--   **Indentation & Formatting:** Follow the project's AI-optimized compact style (1-space indentation for code snippets).
--   **High Information Density:** Avoid conversational filler; focus on state transitions, data flow, and architectural constraints.
--   **Accurate Code Linkage:** Documentation must include specific file names and symbol references (e.g., `AsyncEventQueue`, `_cb_accept_tracks`).
-
-## Acceptance Criteria
--   [ ] `docs/guide_architecture.md` updated with "Odin-style" depth.
--   [ ] `docs/guide_tools.md` contains a full API reference for the Hook system.
--   [ ] `docs/guide_simulations.md` created, detailing the visual verification suite.
--   [ ] `Readme.md` reflects the current high-density product vision.
--   [ ] `MainContext.md` decommissioned and references removed.
+## Acceptance Criteria for Expert Review (Claude Opus)
+-   [ ] **Zero Filler:** No introductory "In this section..." or "Now we will..." conversational markers.
+-   [ ] **Structural Parity:** Documentation follows the `gencpp` pattern (Philosophy -> Code Paths -> Interface).
+-   [ ] **Expert-Level Detail:** Includes data structures, locking mechanisms, and thread-safety constraints.
+-   [ ] **Narrative Cohesion:** The documents feel like a single, expert-authored manual for a complex graphics or systems engine.
+-   [ ] **Tactile Interaction:** Explains the "Linear Execution Clutch" as a physical shift in the application's processing gears.
 
 ## Out of Scope
 -   Documenting legacy `gui_legacy.py` code beyond its role as a fallback.
--   Visual diagram generation (focusing on high-quality text-based architectural mapping).
+-   Visual diagram generation (focusing on high-signal text-based architectural mapping).