prep doc track.

conductor/tracks/documentation_refresh_20260224/plan.md

@@ -1,34 +1,38 @@
-# Implementation Plan: Documentation Refresh and Context Cleanup
+# Implementation Plan: Deep Architectural Documentation Refresh
-
-This plan follows the project's standard task workflow to modernize documentation and decommission redundant context files.
-
-## Phase 1: Context Cleanup
-Permanently remove redundant files and update project-wide references.
 
+## Phase 1: Context Cleanup & Research
 - [ ] Task: Audit references to `MainContext.md` across the project.
-- [ ] Task: Write failing test that verifies the absence of `MainContext.md` and related broken links.
 - [ ] Task: Delete `MainContext.md` and update any identified references.
-- [ ] Task: Verify that all internal links remain functional.
+- [ ] Task: Execute `py_get_skeleton` and `py_get_code_outline` for `events.py`, `api_hooks.py`, `api_hook_client.py`, and `gui_2.py` to create a technical map for the guides.
-- [ ] Task: Conductor - User Manual Verification 'Context Cleanup' (Protocol in workflow.md)
+- [ ] Task: Analyze the `live_gui` fixture in `tests/conftest.py` and the simulation loop in `tests/visual_sim_mma_v2.py`.
 
-## Phase 2: Core Documentation Refresh
+## Phase 2: Core Architecture Deep Dive
-Update the Architecture and Tools guides to reflect recent architectural changes.
+Update `docs/guide_architecture.md` with expert-level detail.
+- [ ] Task: Document the Dual-Threaded App Lifetime: Main GUI loop vs. Daemon execution threads.
+- [ ] Task: Detail the `AsyncEventQueue` and `EventEmitter` roles in the decoupling strategy.
+- [ ] Task: Explain the `_pending_gui_tasks` synchronization mechanism for bridging the Hook Server and GUI.
+- [ ] Task: Document the "Linear Execution Clutch" and its deterministic state machine.
+- [ ] Task: Verify the architectural descriptions against the actual implementation.
+- [ ] Task: Conductor - User Manual Verification 'Phase 2: Core Architecture Deep Dive' (Protocol in workflow.md)
 
-- [ ] Task: Audit `docs/guide_architecture.md` against current code (e.g., `EventEmitter`, `ApiHookClient`, Conductor).
+## Phase 3: Hook System & Tooling Technical Reference
-- [ ] Task: Update `docs/guide_architecture.md` with current Conductor-driven architecture and dual-GUI structure.
+Update `docs/guide_tools.md` to include low-level API details.
-- [ ] Task: Audit `docs/guide_tools.md` for toolset accuracy.
+- [ ] Task: Create a comprehensive API reference for all `HookServer` endpoints.
-- [ ] Task: Update `docs/guide_tools.md` to include API hook client and performance monitoring documentation.
+- [ ] Task: Document the `ApiHookClient` implementation, including retries and polling strategies.
-- [ ] Task: Verify documentation alignment with actual implementation.
+- [ ] Task: Update the MCP toolset guide with current native tool implementations.
-- [ ] Task: Conductor - User Manual Verification 'Core Documentation Refresh' (Protocol in workflow.md)
+- [ ] Task: Document the `ask/respond` IPC flow for "Human-in-the-Loop" confirmations.
+- [ ] Task: Conductor - User Manual Verification 'Phase 3: Hook System & Tooling Technical Reference' (Protocol in workflow.md)
 
-## Phase 3: README Refresh and Link Validation
+## Phase 4: Verification & Simulation Framework
-Modernize the primary project entry point and ensure documentation integrity.
+Create the new `docs/guide_simulations.md` guide.
+- [ ] Task: Detail the Live GUI testing infrastructure: `--enable-test-hooks` and the `live_gui` fixture.
+- [ ] Task: Breakdown the Simulation Lifecycle: Startup, Polling, Interaction, and Assertion.
+- [ ] Task: Document the mock provider strategy using `tests/mock_gemini_cli.py`.
+- [ ] Task: Provide examples of visual verification tests (e.g., MMA lifecycle).
+- [ ] Task: Conductor - User Manual Verification 'Phase 4: Verification & Simulation Framework' (Protocol in workflow.md)
 
-- [ ] Task: Audit `Readme.md` for accuracy of setup instructions and feature highlights.
+## Phase 5: README & Roadmap Update
-- [ ] Task: Write failing test (or link audit) that identifies outdated setup steps or broken links.
+- [ ] Task: Update `Readme.md` with current setup (`uv`, `credentials.toml`) and vision.
-- [ ] Task: Update `Readme.md` with `uv` setup, current project vision, and feature lists (Conductor, GUI 2.0).
+- [ ] Task: Perform a project-wide link validation of all Markdown files.
-- [ ] Task: Perform a project-wide link validation of all Markdown files in `./docs/` and the root.
+- [ ] Task: Verify the high-density information style across all documentation.
-- [ ] Task: Verify setup instructions by performing a manual walkthrough of the Readme steps.
+- [ ] Task: Conductor - User Manual Verification 'Phase 5: README & Roadmap Update' (Protocol in workflow.md)
-- [ ] Task: Conductor - User Manual Verification 'README Refresh and Link Validation' (Protocol in workflow.md)
----
-[checkpoint: (SHA will be recorded here)]

conductor/tracks/documentation_refresh_20260224/spec.md

@@ -1,38 +1,49 @@
-# Specification: Documentation Refresh and Context Cleanup
+# Track Specification: Deep Architectural Documentation Refresh
 
 ## Overview
-This track aims to modernize the project's documentation suite (Architecture, Tools, README) to reflect recent significant architectural additions, including the Conductor framework, the development of `gui_2.py`, and the API hook verification system. It also includes the decommissioning of `MainContext.md`, which has been identified as redundant in the current project structure.
+This track implements a high-density, expert-level documentation suite for the Manual Slop project, focusing on the complex asynchronous orchestration, the 4-Tier MMA (Hierarchical Multi-Model Architecture), and the robust visual simulation framework. The documentation style adheres to the "USA Graphics Company" values of high information density, tactile interaction, and deep technical transparency, inspired by the architectural guides of `VEFontCache-Odin` and `gencpp`.
 
-## Functional Requirements
+## Goals
-1.  **Architecture Update (`docs/guide_architecture.md`):**
+1.  **Architectural Deep Dive:** Document the core engine, focusing on state management, asynchronous event loops, and the decoupling of the GUI from the AI client.
-    -   Incorporate descriptions of the Conductor framework and its role in spec-driven development.
+2.  **Hook System Technical Reference:** Provide a low-level guide to the `HookServer`, `HookHandler`, and the IPC mechanism used for automated verification.
-    -   Document the dual-GUI structure (`gui.py` and `gui_2.py`) and their respective development stages.
+3.  **Simulation & Testing Framework:** Detail the live GUI simulation strategy, including the use of `pytest` fixtures, `ApiHookClient`, and mock providers.
-    -   Detail the `EventEmitter` and `ApiHookClient` as core architectural components.
+4.  **MMA Orchestration Logic:** Document the 4-Tier hierarchy (Orchestrator, Tech Lead, Worker, QA), token firewalling, and the DAG-based task execution engine.
-2.  **Tools Update (`docs/guide_tools.md`):**
+5.  **Event Handling System:** Explain the dual event architecture using both synchronous `EventEmitter` and asynchronous `AsyncEventQueue`.
-    -   Refresh documentation for the current MCP toolset.
+
-    -   Add documentation for the API hook client and automated GUI verification tools.
+## Functional Requirements (Documentation Areas)
-    -   Update performance monitoring tool descriptions.
+
-3.  **README Refresh (`Readme.md`):**
+### 1. Core Architecture (`docs/guide_architecture.md`)
-    -   Update setup instructions (e.g., `uv`, `credentials.toml`).
+-   **Lifetime & Execution Loop:** Detailed breakdown of the application startup, the interaction between the main thread and daemon threads, and the event-driven updates.
-    -   Highlight new features: Conductor integration, GUI 2.0, and automated testing capabilities.
+-   **Asynchronous Orchestration:** Documentation of the `AsyncEventQueue` and how it prevents GUI blocking during long-running AI operations.
-    -   Ensure the high-level project vision aligns with the current state.
+-   **State Synchronization:** How the `_pending_gui_tasks` queue and `threading.Lock` are used to safely bridge the Hook Server thread and the Main GUI thread.
-4.  **Context Cleanup:**
+-   **Linear Execution Clutch:** Explanation of the deterministic debugging mode and the step-through execution logic.
-    -   Permanently remove `MainContext.md` from the project root.
+
-    -   Update any internal references pointing to `MainContext.md`.
+### 2. Tooling & Hooks (`docs/guide_tools.md`)
+-   **IPC & Hook API:** A complete reference for all `/api/` endpoints, including data structures for `mma_status`, `diagnostics`, and `ask/respond`.
+-   **ApiHookClient:** Usage guide for the requests-based client, detailing polling strategies and retry logic.
+-   **MCP Toolset:** Updated documentation for the native MCP-like tools (file I/O, search, web fetch) and their security allowlist.
+
+### 3. Verification & Simulation (`docs/guide_simulations.md` - NEW)
+-   **Live GUI Verification:** How to run the app with `--enable-test-hooks` and use the `live_gui` fixture for end-to-end testing.
+-   **Visual Simulation Lifecycle:** Detailed breakdown of `tests/visual_sim_mma_v2.py`, explaining how it simulates user interaction and verifies rendered state (e.g., DAG nodes, stream output).
+-   **Mocking AI Providers:** How to use `tests/mock_gemini_cli.py` to test the full orchestration loop without incurring API costs.
+
+### 4. README & Vision (`Readme.md`)
+-   **Setup & Telemetry:** Updated instructions for `uv` and `credentials.toml`.
+-   **Feature Roadmap:** Highlighting the MMA Dashboard, DAG engine, and the 4-Tier delegation model.
 
 ## Non-Functional Requirements
--   **Link Validation:** All internal documentation links must be verified as valid.
+-   **Indentation & Formatting:** Follow the project's AI-optimized compact style (1-space indentation for code snippets).
--   **Code-Doc Alignment:** Architectural descriptions must accurately reflect the current code structure.
+-   **High Information Density:** Avoid conversational filler; focus on state transitions, data flow, and architectural constraints.
--   **Clarity & Brevity:** Documentation should remain concise and targeted at expert-level developers.
+-   **Accurate Code Linkage:** Documentation must include specific file names and symbol references (e.g., `AsyncEventQueue`, `_cb_accept_tracks`).
 
 ## Acceptance Criteria
--   [ ] `MainContext.md` is deleted from the project.
+-   [ ] `docs/guide_architecture.md` updated with "Odin-style" depth.
--   [ ] `docs/guide_architecture.md` is updated and reviewed for accuracy.
+-   [ ] `docs/guide_tools.md` contains a full API reference for the Hook system.
--   [ ] `docs/guide_tools.md` is updated and reviewed for accuracy.
+-   [ ] `docs/guide_simulations.md` created, detailing the visual verification suite.
--   [ ] `Readme.md` setup and feature sections are current.
+-   [ ] `Readme.md` reflects the current high-density product vision.
--   [ ] All internal links between `Readme.md` and the `./docs/` folder are functional.
+-   [ ] `MainContext.md` decommissioned and references removed.
 
 ## Out of Scope
--   Automated documentation generation (e.g., Sphinx, Doxygen).
+-   Documenting legacy `gui_legacy.py` code beyond its role as a fallback.
--   In-depth documentation for features still in early prototyping stages.
+-   Visual diagram generation (focusing on high-quality text-based architectural mapping).
--   Creating new video or visual walkthroughs.

conductor/workflow.md

@@ -35,35 +35,35 @@ All tasks follow a strict lifecycle:
    - Take the code generated by the Worker and apply it.
    - **CRITICAL:** Run the tests and confirm that they fail as expected. This is the "Red" phase of TDD. Do not proceed until you have failing tests.
 
-4. **Implement to Pass Tests (Green Phase):**
+5. **Implement to Pass Tests (Green Phase):**
    - **Pre-Delegation Checkpoint:** Ensure current progress is staged or committed before delegating.
    - **Code Style:** ALWAYS explicitly mention "Use exactly 1-space indentation for Python code" when prompting a sub-agent.
    - **Delegate Implementation:** Do NOT write the implementation code directly. Spawn a Tier 3 Worker (`python scripts/mma_exec.py --role tier3-worker "[PROMPT]"`) with a highly specific prompt to write the minimum amount of application code necessary to make the failing tests pass. (If repeating due to failures, pass `--failure-count X` to switch to a more capable model).
    - Take the code generated by the Worker and apply it.
    - Run the test suite again and confirm that all tests now pass. This is the "Green" phase.
 
-5. **Refactor (Optional but Recommended):**
+6. **Refactor (Optional but Recommended):**
    - With the safety of passing tests, refactor the implementation code and the test code to improve clarity, remove duplication, and enhance performance without changing the external behavior.
    - Rerun tests to ensure they still pass after refactoring.
 
-6. **Verify Coverage:** Run coverage reports using the project's chosen tools. For example, in a Python project, this might look like:
+7. **Verify Coverage:** Run coverage reports using the project's chosen tools. For example, in a Python project, this might look like:
    ```powershell
    pytest --cov=app --cov-report=html
    ```
    Target: >80% coverage for new code. The specific tools and commands will vary by language and framework.
 
-7. **Document Deviations:** If implementation differs from tech stack:
+8. **Document Deviations:** If implementation differs from tech stack:
    - **STOP** implementation
    - Update `tech-stack.md` with new design
    - Add dated note explaining the change
    - Resume implementation
 
-8. **Commit Code Changes:**
+9. **Commit Code Changes:**
    - Stage all code changes related to the task.
    - Propose a clear, concise commit message e.g, `feat(ui): Create basic HTML structure for calculator`.
    - Perform the commit.
 
-9. **Attach Task Summary with Git Notes:**
+10. **Attach Task Summary with Git Notes:**
    - **Step 9.1: Get Commit Hash:** Obtain the hash of the *just-completed commit* (`git log -1 --format="%H"`).
    - **Step 9.2: Draft Note Content:** Create a detailed summary for the completed task. This should include the task name, a summary of changes, a list of all created/modified files, and the core "why" for the change.
    - **Step 9.3: Attach Note:** Use the `git notes` command to attach the summary to the commit.
@@ -72,11 +72,11 @@ All tasks follow a strict lifecycle:
      git notes add -m "<note content>" <commit_hash>
      ```
 
-10. **Get and Record Task Commit SHA:**
+11. **Get and Record Task Commit SHA:**
     - **Step 10.1: Update Plan:** Read `plan.md`, find the line for the completed task, update its status from `[~]` to `[x]`, and append the first 7 characters of the *just-completed commit's* commit hash.
     - **Step 10.2: Write Plan:** Write the updated content back to `plan.md`.
 
-11. **Commit Plan Update:**
+12. **Commit Plan Update:**
     - **Action:** Stage the modified `plan.md` file.
     - **Action:** Commit this change with a descriptive message (e.g., `conductor(plan): Mark task 'Create user model' as complete`).
 
@@ -379,7 +379,7 @@ A task is complete when:
 To emulate the 4-Tier MMA Architecture within the standard Conductor extension without requiring a custom fork, adhere to these strict workflow policies:
 
 ### 1. Active Model Switching (Simulating the 4 Tiers)
-- **Activate MMA Orchestrator Skill:** To enforce the 4-Tier token firewall, the agent MUST invoke `activate_skill mma-orchestrator` at the start of any implementation phase.
+- **Mandatory Skill Activation:** As the very first step of any MMA-driven process, including track initialization and implementation phases, the agent MUST activate the `mma-orchestrator` skill (`activate_skill mma-orchestrator`). This is crucial for enforcing the 4-Tier token firewall.
 - **The MMA Bridge (`mma_exec.py`):** All tiered delegation is routed through `python scripts/mma_exec.py`. This script acts as the primary bridge, managing model selection, context injection, and logging.
 - **Model Tiers:**
     - **Tier 1 (Strategic/Orchestration):** `gemini-3.1-pro-preview`. Focused on product alignment, setup (`/conductor:setup`), and track initialization (`/conductor:newTrack`).