chore(conductor): Add new track 'Expanded Hook API & Headless Orchestration'
This commit is contained in:
@@ -21,6 +21,10 @@ This file tracks all major tracks for the project. Each track has its own detail
|
|||||||
*Link: [./tracks/tool_bias_tuning_20260308/](./tracks/tool_bias_tuning_20260308/)*
|
*Link: [./tracks/tool_bias_tuning_20260308/](./tracks/tool_bias_tuning_20260308/)*
|
||||||
*Goal: Influence agent tool selection via a weighting system. Implement semantic nudges in tool descriptions and a dynamic "Tooling Strategy" section in the system prompt. Includes GUI badges and sliders for weight adjustment.*
|
*Goal: Influence agent tool selection via a weighting system. Implement semantic nudges in tool descriptions and a dynamic "Tooling Strategy" section in the system prompt. Includes GUI badges and sliders for weight adjustment.*
|
||||||
|
|
||||||
|
4. [ ] **Track: Expanded Hook API & Headless Orchestration**
|
||||||
|
*Link: [./tracks/hook_api_expansion_20260308/](./tracks/hook_api_expansion_20260308/)*
|
||||||
|
*Goal: Maximize internal state exposure and provide comprehensive control endpoints (worker spawn/kill, pipeline pause/resume, DAG mutation) via the Hook API. Implement WebSocket-based real-time event streaming.*
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### GUI Overhauls & Visualizations
|
### GUI Overhauls & Visualizations
|
||||||
|
|||||||
5
conductor/tracks/hook_api_expansion_20260308/index.md
Normal file
5
conductor/tracks/hook_api_expansion_20260308/index.md
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
# Track hook_api_expansion_20260308 Context
|
||||||
|
|
||||||
|
- [Specification](./spec.md)
|
||||||
|
- [Implementation Plan](./plan.md)
|
||||||
|
- [Metadata](./metadata.json)
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"track_id": "hook_api_expansion_20260308",
|
||||||
|
"type": "feature",
|
||||||
|
"status": "new",
|
||||||
|
"created_at": "2026-03-08T14:17:00Z",
|
||||||
|
"updated_at": "2026-03-08T14:17:00Z",
|
||||||
|
"description": "Expanded Hook API & Headless Orchestration - Maximizing state exposure and providing comprehensive control endpoints for headless use, including WebSocket event streaming."
|
||||||
|
}
|
||||||
44
conductor/tracks/hook_api_expansion_20260308/plan.md
Normal file
44
conductor/tracks/hook_api_expansion_20260308/plan.md
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
# Implementation Plan: Expanded Hook API & Headless Orchestration
|
||||||
|
|
||||||
|
## Phase 1: WebSocket Infrastructure & Event Streaming
|
||||||
|
- [ ] Task: Implement the WebSocket gateway.
|
||||||
|
- [ ] Integrate a lightweight WebSocket library (e.g., `websockets` or `simple-websocket`).
|
||||||
|
- [ ] Create a dedicated `WebSocketServer` class in `src/api_hooks.py` that runs on a separate port (e.g., 9000).
|
||||||
|
- [ ] Implement a basic subscription mechanism for different event channels.
|
||||||
|
- [ ] Task: Connect the event queue to the WebSocket stream.
|
||||||
|
- [ ] Update `AsyncEventQueue` to broadcast events to connected WebSocket clients.
|
||||||
|
- [ ] Add high-frequency telemetry (FPS, CPU) to the event stream.
|
||||||
|
- [ ] Task: Write unit tests for WebSocket connection and event broadcasting.
|
||||||
|
- [ ] Task: Conductor - User Manual Verification 'Phase 1: WebSocket Infrastructure' (Protocol in workflow.md)
|
||||||
|
|
||||||
|
## Phase 2: Expanded Read Endpoints (GET)
|
||||||
|
- [ ] Task: Implement detailed state exposure endpoints.
|
||||||
|
- [ ] Add `/api/mma/workers` to return the status, logs, and traces of all active sub-agents.
|
||||||
|
- [ ] Add `/api/context/state` to expose AST cache metadata and file aggregation status.
|
||||||
|
- [ ] Add `/api/metrics/financial` to return track-specific token usage and cost data.
|
||||||
|
- [ ] Add `/api/system/telemetry` to expose internal thread and queue metrics.
|
||||||
|
- [ ] Task: Enhance `/api/gui/state` to provide a truly exhaustive JSON dump of all internal managers.
|
||||||
|
- [ ] Task: Update `api_hook_client.py` with corresponding methods for all new GET endpoints.
|
||||||
|
- [ ] Task: Write integration tests for all new GET endpoints using `live_gui`.
|
||||||
|
- [ ] Task: Conductor - User Manual Verification 'Phase 2: Expanded Read Endpoints' (Protocol in workflow.md)
|
||||||
|
|
||||||
|
## Phase 3: Comprehensive Control Endpoints (POST)
|
||||||
|
- [ ] Task: Implement worker and pipeline control.
|
||||||
|
- [ ] Add `/api/mma/workers/spawn` to manually initiate sub-agent execution via the API.
|
||||||
|
- [ ] Add `/api/mma/workers/kill` to programmatically abort running workers.
|
||||||
|
- [ ] Add `/api/mma/pipeline/pause` and `/api/mma/pipeline/resume` to control the global MMA loop.
|
||||||
|
- [ ] Task: Implement context and DAG mutation.
|
||||||
|
- [ ] Add `/api/context/inject` to allow programmatic context injection (files/skeletons).
|
||||||
|
- [ ] Add `/api/mma/dag/mutate` to allow modifying task dependencies through the API.
|
||||||
|
- [ ] Task: Update `api_hook_client.py` with corresponding methods for all new POST endpoints.
|
||||||
|
- [ ] Task: Write integration tests for all new control endpoints using `live_gui`.
|
||||||
|
- [ ] Task: Conductor - User Manual Verification 'Phase 3: Comprehensive Control Endpoints' (Protocol in workflow.md)
|
||||||
|
|
||||||
|
## Phase 4: Headless Refinement & Verification
|
||||||
|
- [ ] Task: Improve error reporting.
|
||||||
|
- [ ] Refactor `HookHandler` to catch and wrap all internal exceptions in JSON error responses.
|
||||||
|
- [ ] Task: Conduct a full headless simulation.
|
||||||
|
- [ ] Create a specialized simulation script that replicates a full MMA track lifecycle (planning, worker spawn, DAG mutation, completion) using ONLY the Hook API.
|
||||||
|
- [ ] Task: Final performance audit.
|
||||||
|
- [ ] Ensure that active WebSocket clients and large state dumps do not cause GUI frame drops.
|
||||||
|
- [ ] Task: Conductor - User Manual Verification 'Phase 4: Headless Refinement & Verification' (Protocol in workflow.md)
|
||||||
50
conductor/tracks/hook_api_expansion_20260308/spec.md
Normal file
50
conductor/tracks/hook_api_expansion_20260308/spec.md
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
# Specification: Expanded Hook API & Headless Orchestration
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
This track aims to transform the Manual Slop Hook API into a comprehensive control plane for headless use. It focuses on exposing all relevant internal states (worker traces, AST metadata, financial metrics, concurrency telemetry) and providing granular control over the application's lifecycle, MMA pipeline, and context management. Additionally, it introduces a WebSocket-based streaming interface for real-time event delivery.
|
||||||
|
|
||||||
|
## Functional Requirements
|
||||||
|
|
||||||
|
### 1. Comprehensive State Exposure (GET Endpoints)
|
||||||
|
- **Worker Traces & Logs:** Expose detailed real-time logs and thought traces for every active agent tier via `/api/mma/workers`.
|
||||||
|
- **AST & Aggregation State:** Expose the current AST cache metadata, file dependency maps, and the state of the context aggregator via `/api/context/state`.
|
||||||
|
- **Financial & Token Metrics:** Provide detailed, track-specific token usage history and cost breakdowns via `/api/metrics/financial`.
|
||||||
|
- **Concurrency & Threading Telemetry:** Expose internal event queue depths, active thread counts, and background task statuses via `/api/system/telemetry`.
|
||||||
|
- **Full State Dump:** Enhance `/api/gui/state` to provide an exhaustive snapshot of all internal controllers and managers.
|
||||||
|
|
||||||
|
### 2. Deep Control Surface (POST Endpoints)
|
||||||
|
- **Worker Lifecycle Control:**
|
||||||
|
- `/api/mma/workers/spawn`: Manually trigger a worker spawn with a specific prompt and context.
|
||||||
|
- `/api/mma/workers/kill`: Abort specific running sub-agents by UID.
|
||||||
|
- **Pipeline Flow Control:**
|
||||||
|
- `/api/mma/pipeline/pause`: Suspend the entire MMA execution loop.
|
||||||
|
- `/api/mma/pipeline/resume`: Resume the MMA execution loop.
|
||||||
|
- **Context & DAG Management:**
|
||||||
|
- `/api/context/inject`: Programmatically inject full file content or AST skeletons into the active discussion.
|
||||||
|
- `/api/mma/dag/mutate`: Add, remove, or modify task dependencies within the active implementation track.
|
||||||
|
|
||||||
|
### 3. Real-time Telemetry (WebSocket)
|
||||||
|
- **WebSocket Gateway:** Implement a WebSocket server (on a secondary port or path) to stream application events.
|
||||||
|
- **Event Streaming:** Stream all events currently available via `/api/events`, plus new high-frequency telemetry (FPS, CPU, worker progress tokens) in real-time.
|
||||||
|
- **Client Discovery:** Support an initial "handshake" that allows clients to subscribe to specific event categories (e.g., `mma`, `gui`, `system`).
|
||||||
|
|
||||||
|
### 4. Headless Refinement
|
||||||
|
- **Lifecycle Mapping:** Ensure that all actions performed via the Hook API are correctly reflected in the GUI (maintaining the "manual" in Manual Slop).
|
||||||
|
- **Error Propagation:** Improve error reporting in the API to return descriptive JSON payloads for all 4xx and 5xx responses.
|
||||||
|
|
||||||
|
## Non-Functional Requirements
|
||||||
|
- **Thread Safety:** Maintain the strict GUI thread trampoline pattern for all state-mutating actions.
|
||||||
|
- **Performance:** Ensure the WebSocket server and expanded logging do not degrade main loop responsiveness.
|
||||||
|
- **Security:** Maintain the port-based access control; ensure the API is only bound to `127.0.0.1` by default.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
- [ ] A separate frontend can fully replicate the MMA dashboard using only Hook API endpoints.
|
||||||
|
- [ ] Real-time event streaming is functional via WebSockets.
|
||||||
|
- [ ] Workers can be programmatically spawned and killed via REST calls.
|
||||||
|
- [ ] Task dependencies in the DAG can be modified via the API.
|
||||||
|
- [ ] AST and file aggregation metadata are visible to external clients.
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
- Multi-user authentication or session isolation.
|
||||||
|
- Remote filesystem access (relying strictly on existing MCP tools).
|
||||||
|
- Rewriting the core event loop (building on top of existing `AsyncEventQueue`).
|
||||||
Reference in New Issue
Block a user