feat(conductor): Archive External MCP, Project-Specific Conductor, and GUI Path Config tracks

This commit is contained in:
2026-03-12 20:10:05 -04:00
parent 5a8a91ecf7
commit befb480285
13 changed files with 437 additions and 23 deletions

View File

@@ -0,0 +1,5 @@
# Track external_mcp_support_20260308 Context
- [Specification](./spec.md)
- [Implementation Plan](./plan.md)
- [Metadata](./metadata.json)

View File

@@ -0,0 +1,8 @@
{
"track_id": "external_mcp_support_20260308",
"type": "feature",
"status": "new",
"created_at": "2026-03-08T14:00:00Z",
"updated_at": "2026-03-08T14:00:00Z",
"description": "Add support for external MCP servers (Local Stdio and Remote SSE/WS) with flexible configuration and lifecycle management."
}

View File

@@ -0,0 +1,42 @@
# Implementation Plan: External MCP Server Support
## Phase 1: Configuration & Data Modeling [checkpoint: 4ba1bd9]
- [x] Task: Define the schema for external MCP server configuration. [1c863f0]
- [x] Update `src/models.py` to include `MCPServerConfig` and `MCPConfiguration` classes.
- [x] Implement logic to load `mcp_config.json` from global and project-specific paths.
- [x] Task: Integrate configuration loading into `AppController`. [c09e0f5]
- [x] Ensure the MCP config path is correctly resolved from `config.toml` and `manual_slop.toml`.
- [x] Task: Write unit tests for configuration loading and validation. [c09e0f5]
- [x] Task: Conductor - User Manual Verification 'Phase 1: Configuration & Data Modeling' [4ba1bd9]
## Phase 2: MCP Client Extension [checkpoint: 828fadf]
- [x] Task: Implement `ExternalMCPManager` in `src/mcp_client.py`. [828fadf]
- [x] Add support for managing multiple MCP server sessions.
- [x] Implement the `StdioMCPClient` for local subprocess communication.
- [x] Implement the `RemoteMCPClient` for SSE/WebSocket communication (stub).
- [x] Task: Update Tool Discovery. [828fadf]
- [x] Implement `list_external_tools()` to aggregate tools from all active external servers.
- [x] Task: Update Tool Dispatch. [828fadf]
- [x] Modify `mcp_client.dispatch()` and `mcp_client.async_dispatch()` to route tool calls to either native tools or the appropriate external server.
- [x] Task: Write integration tests for stdio and remote MCP client communication (using mock servers). [828fadf]
- [x] Task: Conductor - User Manual Verification 'Phase 2: MCP Client Extension' [828fadf]
## Phase 3: GUI Integration & Lifecycle [checkpoint: 3b2588a]
- [x] Task: Update the **Operations** panel in `src/gui_2.py`. [3b2588a]
- [x] Create a new "External Tools" section.
- [x] List discovered tools from active external servers.
- [x] Add a "Refresh External MCPs" button to reload configuration and rediscover tools.
- [x] Task: Implement Lifecycle Management. [3b2588a]
- [x] Add the "Auto-start on Project Load" logic to start servers when a project is initialized.
- [x] Add status indicators (e.g., color-coded dots) for each external server in the GUI.
- [x] Task: Write visual regression tests or simulation scripts to verify the updated Operations panel. [3b2588a]
- [x] Task: Conductor - User Manual Verification 'Phase 3: GUI Integration & Lifecycle' [3b2588a]
## Phase 4: Agent Integration & HITL [checkpoint: f4c5a0b]
- [x] Task: Update AI tool declarations. [f4c5a0b]
- [x] Ensure `ai_client.py` includes external tools in the tool definitions sent to Gemini/Anthropic.
- [x] Task: Verify HITL Approval Flow. [f4c5a0b]
- [x] Ensure that calling an external tool correctly triggers the `ConfirmDialog` modal.
- [x] Verify that approved external tool results are correctly returned to the AI.
- [x] Task: Perform a final end-to-end verification with a real external MCP server. [f4c5a0b]
- [x] Task: Conductor - User Manual Verification 'Phase 4: Agent Integration & HITL' [f4c5a0b]

View File

@@ -0,0 +1,39 @@
# Specification: External MCP Server Support
## Overview
This feature adds support for integrating external Model Context Protocol (MCP) servers into Manual Slop. This allows agents to utilize tools from a wide ecosystem of MCP servers (like those for databases, APIs, or specialized utilities) alongside the application's native tools.
## Functional Requirements
- **Server Protocol Support:**
- **Local Stdio:** Support for launching local subprocesses and communicating via JSON-RPC over stdio.
- **Remote (SSE/WS):** Support for connecting to remote MCP servers via Server-Sent Events or WebSockets.
- **Flexible Configuration:**
- The path to the MCP configuration file (e.g., `mcp_config.json`) must be configurable globally in `config.toml`.
- Support for per-project overrides in `manual_slop.toml` to specify a project-specific MCP configuration.
- **Lifecycle Management:**
- Provide a "Auto-start on Project Load" checkbox for each configured MCP server (or as a global/per-project default).
- If enabled, the server starts automatically when the project is loaded.
- If disabled, servers should be initialized on-demand or via a manual "Start" button in the GUI.
- **Tool Discovery & UI Integration:**
- Automatically discover tools from configured servers using the MCP `listTools` capability.
- Display discovered tools in a unified "External Tools" section within the **Operations** panel.
- Ensure external tools are correctly mapped to agent tool declarations (Gemini, Anthropic, etc.).
- **Security & HITL:**
- All tool calls from external MCP servers must adhere to the application's existing Human-in-the-Loop (HITL) approval mechanism.
- Users must review and approve the parameters of any external tool call before execution.
## Non-Functional Requirements
- **Performance:** Asynchronous communication with MCP servers to avoid blocking the main GUI thread.
- **Robustness:** Gracefully handle server connection failures, timeouts, and process crashes.
- **Scalability:** Support for multiple external MCP servers simultaneously.
## Acceptance Criteria
- [ ] Users can specify an MCP config path in `config.toml` and `manual_slop.toml`.
- [ ] The GUI lists tools from configured external servers in a unified section.
- [ ] Selecting an external tool correctly generates a tool call for the AI.
- [ ] Approving an external tool call successfully executes it via the MCP bridge and returns the output to the AI.
- [ ] Local stdio servers and remote SSE/WS servers are both functional.
## Out of Scope
- Support for MCP Resources or Prompts (focusing strictly on Tools for this track).
- Managing installation of external MCP server dependencies (e.g., `npm install`, `pip install`).

View File

@@ -0,0 +1,15 @@
{
"track_id": "gui_path_config_20260308",
"title": "GUI Path Configuration in Context Hub",
"status": "pending",
"created": "2026-03-08",
"priority": "high",
"owner": "tier2-tech-lead",
"description": "Add path configuration UI to Context Hub. Allow users to view and edit configurable paths (conductor, logs, scripts) directly from the GUI.",
"dependencies": ["conductor_path_configurable_20260306"],
"out_of_scope": [
"Per-project path configuration",
"Runtime path switching without restart",
"Path validation"
]
}

View File

@@ -0,0 +1,88 @@
# Plan: GUI Path Configuration in Context Hub
## Phase 1: Path Info Display
Focus: Show current path resolution in GUI
- [x] Task 1.1: Add path info functions to paths.py [d237d3b]
- WHERE: src/paths.py
- WHAT: Add functions to get path resolution source (default/env/config)
- HOW: Return tuple of (resolved_path, source)
- SAFETY: New functions, no modifications
- [x] Task 1.2: Create path display helper [d237d3b]
- WHERE: src/paths.py
- WHAT: Function to get all paths with resolution info
- HOW: Returns dict of path_name -> (resolved, source)
- SAFETY: New function
## Phase 2: Context Hub Panel
Focus: Add Path Configuration panel to GUI
- [x] Task 2.1: Add Paths tab to Context Hub [d237d3b]
- WHERE: src/gui_2.py (Context Hub section)
- WHAT: New tab/section for path configuration
- HOW: Add ImGui tab item, follow existing panel patterns
- SAFETY: New panel, no modifications to existing
- [x] Task 2.2: Display current paths [d237d3b]
- WHERE: src/gui_2.py (new paths panel)
- WHAT: Show resolved paths and their sources
- HOW: Call paths.py functions, display in read-only text
- SAFETY: New code
- [x] Task 2.3: Add path text inputs [d237d3b]
- WHERE: src/gui_2.py (paths panel)
- WHAT: Editable text inputs for each path
- HOW: ImGui input_text for conductor_dir, logs_dir, scripts_dir
- SAFETY: New code
- [x] Task 2.4: Add browse buttons [d237d3b]
- WHERE: src/gui_2.py (paths panel)
- WHAT: File dialog buttons to browse for directories
- HOW: Use existing file dialog patterns in gui_2.py
- SAFETY: New code
## Phase 3: Persistence
Focus: Save path changes to config.toml
- [x] Task 3.1: Add config write function [d237d3b]
- WHERE: src/gui_2.py or new utility
- WHAT: Write [paths] section to config.toml
- HOW: Read existing config, update paths section, write back
- SAFETY: Backup before write, handle errors
- [x] Task 3.2: Add Apply button [d237d3b]
- WHERE: src/gui_2.py (paths panel)
- WHAT: Button to save changes
- HOW: Call config write function, show success/error message
- SAFETY: Confirmation dialog
- [x] Task 3.3: Add Reset button [d237d3b]
- WHERE: src/gui_2.py (paths panel)
- WHAT: Reset paths to defaults
- HOW: Clear custom values, show confirmation
- SAFETY: Confirmation dialog
## Phase 4: UX Polish
Focus: Improve user experience
- [x] Task 4.1: Add restart warning [d237d3b]
- WHERE: src/gui_2.py (paths panel)
- WHAT: Show warning that changes require restart
- HOW: Text label after Apply
- SAFETY: New code
- [x] Task 4.2: Add tooltips [d237d3b]
- WHERE: src/gui_2.py (paths panel)
- WHAT: Explain each path and resolution order
- HOW: ImGui set_tooltip on hover
- SAFETY: New code
## Phase 5: Tests
Focus: Verify GUI path configuration
- [x] Task 5.1: Test path display [d237d3b]
- WHERE: tests/test_gui_paths.py (new file)
- WHAT: Verify paths panel shows correct values
- HOW: Mock paths.py, verify display
- SAFETY: New test file

View File

@@ -0,0 +1,72 @@
# Track Specification: GUI Path Configuration in Context Hub
## Overview
Add path configuration UI to the Context Hub in the GUI. Allow users to view and edit configurable paths (conductor, logs, scripts) directly from the application without manually editing config.toml or environment variables.
## Current State Audit
### Already Implemented
- `src/paths.py`: Path resolution with env var and config.toml support
- `config.toml [paths]` section: Global path configuration
- Context Hub panel in GUI (`gui_2.py`)
### Gaps to Fill
- No GUI to view/edit paths
- Users must edit config.toml manually
- No visibility into current path resolution
## Goals
1. Add Path Configuration panel to Context Hub
2. Display current resolved paths (read-only)
3. Allow editing paths via text inputs
4. Persist changes to config.toml
5. Show path resolution source (default/env/config)
## Functional Requirements
### UI Panel Location
- Context Hub → "Paths" tab/section
### UI Elements
| Element | Type | Description |
|---------|------|-------------|
| Conductor Dir | Text input + browse button | Path to conductor directory |
| Logs Dir | Text input + browse button | Path to logs directory |
| Scripts Dir | Text input + browse button | Path to scripts directory |
| Resolution Info | Label | Shows source: "default" / "env:SLOP_*" / "config.toml" |
| Apply Button | Button | Save changes to config.toml |
| Reset Button | Button | Reset to defaults |
### Path Resolution Display
Show how each path is resolved:
```
Conductor: /path/to/custom (config.toml)
Logs: ./logs/sessions (default)
Scripts: /env/path (env: SLOP_SCRIPTS_DIR)
```
### Persistence
- Changes written to `config.toml [paths]` section
- App restart required for changes to take effect (show warning)
- Backup existing config.toml before writing
## Architecture Reference
- **Paths module**: `src/paths.py` - path resolution functions
- **Context Hub**: `gui_2.py` - existing Context Hub panel
- **Config I/O**: `project_manager.py` - TOML read/write utilities
- **Config location**: `paths.get_config_path()` - config file location
## Out of Scope
- Per-project path configuration (separate track)
- Runtime path switching without restart
- Path validation/creation
## Non-Functional Requirements
- Follow existing GUI code style (ImGui/Dear PyGui patterns)
- Show confirmation dialog before writing config
- Display current resolved paths on panel open
- Handle missing config.toml gracefully (create new section)

View File

@@ -0,0 +1,15 @@
{
"track_id": "project_conductor_dir_20260308",
"title": "Project-Specific Conductor Directory",
"status": "pending",
"created": "2026-03-08",
"priority": "high",
"owner": "tier2-tech-lead",
"description": "Make conductor directory per-project. Each project TOML can specify custom conductor dir for isolated track/state management.",
"dependencies": ["conductor_path_configurable_20260306"],
"out_of_scope": [
"GUI path configuration",
"Runtime path switching",
"Track migration between projects"
]
}

View File

@@ -0,0 +1,58 @@
# Plan: Project-Specific Conductor Directory
## Phase 1: Extend paths.py
Focus: Add project-specific path resolution
- [x] Task 1.1: Add project-aware conductor path functions [48e2ed8]
- WHERE: src/paths.py
- WHAT: Add optional project_path parameter to get_conductor_dir, get_tracks_dir, get_track_state_dir
- HOW: If project_path provided, resolve relative to project root; otherwise use global
- SAFETY: Maintain backward compatibility with no-arg calls
- [x] Task 1.2: Add project conductor path resolution [48e2ed8]
- WHERE: src/paths.py
- WHAT: New function `_resolve_project_conductor_dir(project_path)` that reads from project TOML
- HOW: Load project TOML, check `[conductor].dir` key
- SAFETY: New function, no side effects
## Phase 2: Update project_manager.py
Focus: Use project-specific paths for track operations
- [x] Task 2.1: Update save_track_state to use project conductor dir [3999e9c]
- WHERE: src/project_manager.py (around line 240)
- WHAT: Pass project base_dir to paths.get_track_state_dir()
- HOW: Get base_dir from project_path, call paths with project_path param
- SAFETY: Maintain existing function signature compatibility
- [x] Task 2.2: Update load_track_state to use project conductor dir [3999e9c]
- WHERE: src/project_manager.py (around line 252)
- WHAT: Load track state from project-specific directory
- HOW: Same as above
- [x] Task 2.3: Update get_all_tracks to use project conductor dir [3999e9c]
- WHERE: src/project_manager.py (around line 297)
- WHAT: List tracks from project-specific directory
- HOW: Accept optional project_path param
## Phase 3: Update app_controller.py
Focus: Pass project path to track operations
- [x] Task 3.1: Update track creation to use project conductor dir [3999e9c]
- WHERE: src/app_controller.py (around line 1907, 1937)
- WHAT: Pass active_project_path to track path functions
- HOW: Get active_project_path, pass to paths.get_tracks_dir()
- SAFETY: Use existing active_project_path attribute
## Phase 4: Tests
Focus: Verify project-specific behavior
- [x] Task 4.1: Write test for project-specific conductor dir [48e2ed8]
- WHERE: tests/test_project_paths.py (new file)
- WHAT: Create mock project with custom conductor dir, verify tracks saved there
- HOW: Mock project_manager, verify path resolution
- SAFETY: New test file
- [x] Task 4.2: Test backward compatibility [3999e9c]
- WHERE: tests/test_project_paths.py
- WHAT: Verify global paths still work without project_path
- HOW: Call functions without project_path, verify defaults

View File

@@ -0,0 +1,73 @@
# Track Specification: Project-Specific Conductor Directory
## Overview
Make the conductor directory per-project instead of global. Each project TOML can specify its own `conductor_dir` path, allowing separate track/state management per project. This enables using Manual Slop with multiple independent projects without track/ticket cross-pollution.
## Current State Audit
### Already Implemented
- `src/paths.py`: Global path resolution via env vars and config.toml
- `paths.get_conductor_dir()`: Returns global conductor directory
- `config.toml [paths]` section: Global path overrides
### Gaps to Fill
- No per-project conductor directory support
- Tracks are always loaded from global conductor dir
- No way to isolate tracks between projects
## Goals
1. Allow projects to specify custom conductor directory in their TOML
2. Load track state from project-specific conductor dir
3. Create new tracks in project-specific directory
4. Maintain backward compatibility with global conductor
## Functional Requirements
### Per-Project Conductor Path
| Project TOML Key | Description | Default |
|-----------------|-------------|---------|
| `conductor.dir` | Path to conductor directory | `"conductor"` (relative to project root) |
### Example Project TOML
```toml
[project]
name = "MyCProject"
path = "/path/to/my-c-project"
[conductor]
# This project's tracks will be in /path/to/my-c-project/my_tracks/
dir = "my_tracks"
```
### Path Resolution Order
1. Project TOML `[conductor].dir` (project-specific)
2. Environment variable `SLOP_CONDUCTOR_DIR` (global override)
3. Config.toml `[paths].conductor_dir` (global default)
4. Fallback to `"conductor"`
### API Changes
- `paths.get_conductor_dir(project_path: str = None) -> Path`: Add optional project_path param
- `paths.get_tracks_dir(project_path: str = None) -> Path`: Returns project-specific tracks dir
- `paths.get_track_state_dir(track_id: str, project_path: str = None) -> Path`
### Integration Points
- `project_manager.py`: Pass project_path to path functions when saving/loading tracks
- `app_controller.py`: Use active project's conductor dir for track operations
## Architecture Reference
- **Existing paths.py**: `src/paths.py` - current global path resolution
- **Project loading**: `project_manager.py:load_project()` - loads project TOML
- **Active project**: `app_controller.py:active_project_path` - current project
## Out of Scope
- GUI path configuration (separate track)
- Runtime path switching (paths resolved at project load)
- Migrating existing tracks between projects
## Relationship to Track 0
This extends conductor_path_configurable_20260306:
- Track 0: Global path configuration (done/impro track: Per-projectved)
- This path override

View File

@@ -10,29 +10,25 @@ This file tracks all major tracks for the project. Each track has its own detail
### Architecture & Backend
1. [x] **Track: External MCP Server Support**
*Link: [./tracks/external_mcp_support_20260308/](./tracks/external_mcp_support_20260308/)*
*Goal: Add support for external MCP servers (Local Stdio and Remote SSE/WS) with flexible configuration and lifecycle management (including auto-start on project load).*
2. [ ] **Track: RAG Support**
1. [ ] **Track: RAG Support**
*Link: [./tracks/rag_support_20260308/](./tracks/rag_support_20260308/)*
*Goal: Add support for RAG (Retrieval-Augmented Generation) using local vector stores (Chroma/Qdrant), native vendor retrieval, and external RAG APIs. Implement indexing pipeline and retrieval UI.*
3. [x] **Track: Agent Tool Preference & Bias Tuning**
2. [x] **Track: Agent Tool Preference & Bias Tuning**
*Link: [./tracks/tool_bias_tuning_20260308/](./tracks/tool_bias_tuning_20260308/)*
*Goal: Influence agent tool selection via a weighting system. Implement semantic nudges in tool descriptions and a dynamic "Tooling Strategy" section in the system prompt. Includes GUI badges and sliders for weight adjustment.*
4. [x] **Track: Expanded Hook API & Headless Orchestration**
3. [x] **Track: Expanded Hook API & Headless Orchestration**
*Link: [./tracks/hook_api_expansion_20260308/](./tracks/hook_api_expansion_20260308/)*
*Goal: Maximize internal state exposure and provide comprehensive control endpoints (worker spawn/kill, pipeline pause/resume, DAG mutation) via the Hook API. Implement WebSocket-based real-time event streaming.*
5. [ ] **Track: Codebase Audit and Cleanup**
4. [ ] **Track: Codebase Audit and Cleanup**
*Link: [./tracks/codebase_audit_20260308/](./tracks/codebase_audit_20260308/)*
6. [ ] **Track: Expanded Test Coverage and Stress Testing**
5. [ ] **Track: Expanded Test Coverage and Stress Testing**
*Link: [./tracks/test_coverage_expansion_20260309/](./tracks/test_coverage_expansion_20260309/)*
7. [ ] **Track: Beads Mode Integration**
6. [ ] **Track: Beads Mode Integration**
*Link: [./tracks/beads_mode_20260309/](./tracks/beads_mode_20260309/)*
*Goal: Integrate Beads (git-backed graph issue tracker) as an alternative backend for MMA implementation tracks and tickets.*
@@ -103,18 +99,6 @@ This file tracks all major tracks for the project. Each track has its own detail
---
### Path Configuration
1. [x] **Track: Project-Specific Conductor Directory**
*Link: [./tracks/project_conductor_dir_20260308/](./tracks/project_conductor_dir_20260308/)*
*Goal: Make conductor directory per-project. Each project TOML can specify custom conductor dir for isolated track/state management.*
2. [x] **Track: GUI Path Configuration in Context Hub**
*Link: [./tracks/gui_path_config_20260308/](./tracks/gui_path_config_20260308/)*
*Goal: Add path configuration UI to Context Hub. Allow users to view and edit configurable paths directly from the GUI.*
---
### Manual UX Controls
1. [x] **Track: Saved System Prompt Presets**
@@ -168,6 +152,9 @@ This file tracks all major tracks for the project. Each track has its own detail
### Completed / Archived
- [x] **Track: External MCP Server Support** (Archived 2026-03-12)
- [x] **Track: Project-Specific Conductor Directory** (Archived 2026-03-12)
- [x] **Track: GUI Path Configuration in Context Hub** (Archived 2026-03-12)
- [x] **Track: True Parallel Worker Execution (The DAG Realization)**
- [x] **Track: Deep AST-Driven Context Pruning (RAG for Code)**
- [x] **Track: Visual DAG & Interactive Ticket Editing**

View File

@@ -48,6 +48,13 @@ from src.paths import get_config_path
CONFIG_PATH = get_config_path()
def _clean_nones(data: Any) -> Any:
if isinstance(data, dict):
return {k: _clean_nones(v) for k, v in data.items() if v is not None}
elif isinstance(data, list):
return [_clean_nones(v) for v in data if v is not None]
return data
def load_config() -> dict[str, Any]:
with open(CONFIG_PATH, "rb") as f:
return tomllib.load(f)
@@ -55,6 +62,7 @@ def load_config() -> dict[str, Any]:
def save_config(config: dict[str, Any]) -> None:
import tomli_w
import sys
config = _clean_nones(config)
sys.stderr.write(f"[DEBUG] Saving config. Theme: {config.get('theme')}\n")
sys.stderr.flush()
with open(CONFIG_PATH, "wb") as f:

View File

@@ -200,7 +200,7 @@ def live_gui() -> Generator[tuple[subprocess.Popen, str], None, None]:
temp_workspace.mkdir(parents=True, exist_ok=True)
# Create minimal project files to avoid cluttering root
(temp_workspace / "manual_slop.toml").write_text("[project]\nname = 'TestProject'\n", encoding="utf-8")
(temp_workspace / "manual_slop.toml").write_text("[project]\nname = 'TestProject'\n\n[conductor]\ndir = 'conductor'\n", encoding="utf-8")
(temp_workspace / "conductor" / "tracks").mkdir(parents=True, exist_ok=True)
# Create a local config.toml in temp_workspace
@@ -209,6 +209,10 @@ def live_gui() -> Generator[tuple[subprocess.Popen, str], None, None]:
'projects': {
'paths': [str((temp_workspace / 'manual_slop.toml').absolute())],
'active': str((temp_workspace / 'manual_slop.toml').absolute())
},
'paths': {
'logs_dir': str((temp_workspace / "logs").absolute()),
'scripts_dir': str((temp_workspace / "scripts" / "generated").absolute())
}
}
import tomli_w