manual_slop/conductor/tracks/hook_api_expansion_20260308/spec.md

Specification: Expanded Hook API & Headless Orchestration

Overview

This track aims to transform the Manual Slop Hook API into a comprehensive control plane for headless use. It focuses on exposing all relevant internal states (worker traces, AST metadata, financial metrics, concurrency telemetry) and providing granular control over the application's lifecycle, MMA pipeline, and context management. Additionally, it introduces a WebSocket-based streaming interface for real-time event delivery.

Functional Requirements

1. Comprehensive State Exposure (GET Endpoints)

• Worker Traces & Logs: Expose detailed real-time logs and thought traces for every active agent tier via /api/mma/workers.
• AST & Aggregation State: Expose the current AST cache metadata, file dependency maps, and the state of the context aggregator via /api/context/state.
• Financial & Token Metrics: Provide detailed, track-specific token usage history and cost breakdowns via /api/metrics/financial.
• Concurrency & Threading Telemetry: Expose internal event queue depths, active thread counts, and background task statuses via /api/system/telemetry.
• Full State Dump: Enhance /api/gui/state to provide an exhaustive snapshot of all internal controllers and managers.

2. Deep Control Surface (POST Endpoints)

• Worker Lifecycle Control:
  • /api/mma/workers/spawn: Manually trigger a worker spawn with a specific prompt and context.
  • /api/mma/workers/kill: Abort specific running sub-agents by UID.
• Pipeline Flow Control:
  • /api/mma/pipeline/pause: Suspend the entire MMA execution loop.
  • /api/mma/pipeline/resume: Resume the MMA execution loop.
• Context & DAG Management:
  • /api/context/inject: Programmatically inject full file content or AST skeletons into the active discussion.
  • /api/mma/dag/mutate: Add, remove, or modify task dependencies within the active implementation track.

3. Real-time Telemetry (WebSocket)

• WebSocket Gateway: Implement a WebSocket server (on a secondary port or path) to stream application events.
• Event Streaming: Stream all events currently available via /api/events, plus new high-frequency telemetry (FPS, CPU, worker progress tokens) in real-time.
• Client Discovery: Support an initial "handshake" that allows clients to subscribe to specific event categories (e.g., mma, gui, system).

4. Headless Refinement

• Lifecycle Mapping: Ensure that all actions performed via the Hook API are correctly reflected in the GUI (maintaining the "manual" in Manual Slop).
• Error Propagation: Improve error reporting in the API to return descriptive JSON payloads for all 4xx and 5xx responses.

Non-Functional Requirements

• Thread Safety: Maintain the strict GUI thread trampoline pattern for all state-mutating actions.
• Performance: Ensure the WebSocket server and expanded logging do not degrade main loop responsiveness.
• Security: Maintain the port-based access control; ensure the API is only bound to 127.0.0.1 by default.

Acceptance Criteria

• A separate frontend can fully replicate the MMA dashboard using only Hook API endpoints.
• Real-time event streaming is functional via WebSockets.
• Workers can be programmatically spawned and killed via REST calls.
• Task dependencies in the DAG can be modified via the API.
• AST and file aggregation metadata are visible to external clients.

Out of Scope

• Multi-user authentication or session isolation.
• Remote filesystem access (relying strictly on existing MCP tools).
• Rewriting the core event loop (building on top of existing AsyncEventQueue).