docs(beads): new guide for Beads mode covering architecture, mock client, MCP tools, and MMA integration roadmap

2026-06-02 19:40:15 -04:00
parent 3ad6615ec2
commit 941b459bc8
1 changed files with 359 additions and 0 deletions
@@ -0,0 +1,359 @@
+# Beads Mode (Dolt-Backed Issue Tracking)
+
+[Top](../README.md) | [MMA](guide_mma.md) | [Tools & IPC](guide_tools.md) | [Simulations](guide_simulations.md)
+
+---
+
+## Overview
+
+Beads is a [Dolt](https://github.com/dolthub/dolt)-backed issue tracking system used as a first-class, project-specific alternative to Manual Slop's markdown-based ticket tracking. Beads stores the task graph in a local `.beads/` Dolt repository, allowing tracks and tickets to be versioned alongside the code in git.
+
+This guide covers:
+
+1. **Architecture** — Where Beads fits in the manual_slop stack
+2. **Components** — `BeadsClient`, the Dolt repository, the bead data model
+3. **MCP Tool Integration** — `bd_create`, `bd_list`, `bd_ready`, `bd_update`
+4. **MMA Integration (Roadmap)** — How Beads mode would integrate with the ConductorEngine
+5. **Testing** — Current state and limitations
+
+**Current state (as of 2026-06-02):** The `BeadsClient` is a **mock implementation** that uses a `.beads_mock/` directory with a JSON file. The real Dolt-backed Beads integration is **roadmap**, not yet implemented. The MCP tools, however, are fully wired and functional against the mock. When the real Beads is integrated, the MCP tool signatures and the `BeadsClient` public API will remain stable; only the backing storage changes.
+
+---
+
+## Architecture
+
+Beads is a **per-project** issue tracking layer that lives alongside Manual Slop's other per-project state (conductor directory, log directory, project TOML).
+
+```
+manual_slop_project/
+├── .beads/                   # Real Beads: Dolt repository (planned)
+│   ├── .dolt/                # Dolt internal storage (when implemented)
+│   └── config.json           # Beads configuration
+├── .beads_mock/              # Current mock: flat JSON storage
+│   └── beads.json            # List of bead objects
+├── conductor/                # Manual Slop's own track storage
+│   └── tracks/
+├── manual_slop.toml          # Project configuration
+└── ... (source files)
+```
+
+**Lifecycle**:
+- A project may opt into Beads mode by setting `[beads] enabled = true` in `manual_slop.toml`.
+- When enabled, `BeadsClient.init_repo()` is called on project load to ensure the `.beads/` (or `.beads_mock/`) directory exists.
+- All track/ticket state for that project flows through `BeadsClient` instead of the conductor's `conductor/tracks/<id>/` markdown files.
+- The MMA dashboard's Visual DAG reads from `BeadsClient.list_beads()` instead of in-memory `TrackDAG`.
+
+**Why Beads?** The Markdown-based track storage (in `conductor/tracks/`) works well for human-authored planning but is awkward for AI-driven workflows. Beads provides:
+- **Programmatic CRUD** — agents can `bd_create`, `bd_update`, `bd_ready` without parsing markdown.
+- **Dependency graphs** — beads have explicit `depends_on` relationships with cycle detection.
+- **Versioning** — the Dolt repo is git-trackable, so the task graph has the same history guarantees as the code.
+- **External tooling** — the `bd` CLI works outside Manual Slop for human review.
+
+---
+
+## Components
+
+### `Bead` (Data Model)
+
+```python
+@dataclass
+class Bead:
+    id: str
+    title: str
+    description: str
+    status: str = "active"
+```
+
+| Field | Type | Purpose |
+|---|---|---|
+| `id` | `str` | Unique identifier. Format: `bead-<N>` (sequential, mock) or Dolt-generated hash (real). |
+| `title` | `str` | Short, human-readable title. |
+| `description` | `str` | Full description. May include multi-line text, file references, and acceptance criteria. |
+| `status` | `str` | One of `active`, `in_progress`, `closed`, `blocked`. |
+
+**Future fields** (planned for the real Dolt integration):
+- `depends_on: List[str]` — explicit dependency graph
+- `assignee: Optional[str]` — Tier or human owner
+- `priority: int` — 0-9 ranking
+- `created_at: datetime`, `updated_at: datetime`
+- `tags: List[str]` — for filtering
+- `acceptance_criteria: List[str]` — structured completion conditions
+
+### `BeadsClient` (`src/beads_client.py`)
+
+The Python client for the Beads backend.
+
+```python
+class BeadsClient:
+    def __init__(self, working_dir: Path):
+        self.working_dir = Path(working_dir)
+        self.repo_dir = self.working_dir / ".beads_mock"
+        self.beads_file = self.repo_dir / "beads.json"
+```
+
+**Construction**: Takes a `working_dir` (typically the project base directory). The mock client derives the storage path as `<working_dir>/.beads_mock/beads.json`. The real client will derive `<working_dir>/.beads/` for Dolt.
+
+**Public Methods**:
+
+```python
+def init_repo(self) -> None:
+    """Create the storage directory and an empty beads.json if not present."""
+
+def is_initialized(self) -> bool:
+    """Check whether the storage directory exists."""
+
+def create_bead(self, title: str, description: str) -> str:
+    """Create a new bead. Returns the new bead's ID."""
+
+def update_bead(self, bead_id: str, status: str) -> bool:
+    """Update the status of an existing bead. Returns True if found, False otherwise."""
+
+def list_beads(self) -> List[Bead]:
+    """Return all beads as a list of Bead objects."""
+```
+
+**Internal Storage (mock)**:
+
+```python
+def _read_beads(self) -> List[dict]:
+    if not self.beads_file.exists():
+        return []
+    return json.loads(self.beads_file.read_text(encoding="utf-8"))
+
+def _write_beads(self, beads: List[dict]) -> None:
+    self.beads_file.write_text(json.dumps(beads, indent=1), encoding="utf-8")
+```
+
+The mock uses a single JSON file with a list of bead objects. The real client will use Dolt SQL tables (`beads`, `dependencies`).
+
+**Thread Safety**: The mock client's `_read_beads` / `_write_beads` are not internally synchronized. The caller (typically `mcp_client.dispatch` from a worker thread) is expected to serialize access. The real Dolt-backed client will use Dolt's transaction system.
+
+### Beads CLI (`bd`)
+
+When the real integration is in place, the `bd` CLI (provided by the [Beads](https://github.com/steveyegge/beads) project) is the primary user-facing tool for managing beads outside Manual Slop. The Python client is a thin wrapper around `bd` subprocess calls or a direct Dolt connection.
+
+**CLI usage (planned)**:
+```bash
+bd create "Fix RAG race condition" "Vector store concurrent writes need a lock"
+bd list
+bd ready       # List beads with no unresolved dependencies
+bd update bead-3 in_progress
+```
+
+The Python `BeadsClient` is intended to be a programmatic equivalent of the `bd` CLI for use by agents.
+
+---
+
+## MCP Tool Integration
+
+The four Beads operations are exposed as MCP tools in `src/mcp_client.py:1474-1494`. The dispatch checks `tool_name.startswith("bd_")` and routes to `BeadsClient` methods.
+
+### Tool Inventory
+
+| Tool | Parameters | Maps to | Returns |
+|---|---|---|---|
+| `bd_create` | `title: str`, `description: str` | `BeadsClient.create_bead` | The new bead's ID |
+| `bd_list` | (none) | `BeadsClient.list_beads` | Formatted list: `ID: <id>, Status: <status>, Title: <title>` |
+| `bd_update` | `bead_id: str`, `status: str` | `BeadsClient.update_bead` | Success/failure indicator |
+| `bd_ready` | (none) | (filtered `list_beads`) | Beads with no unresolved dependencies (mock: returns all `active` beads) |
+
+### Dispatch Flow
+
+```python
+# In src/mcp_client.py:dispatch
+if tool_name.startswith("bd_"):
+    if not _primary_base_dir:
+        return "ERROR: no active workspace to run beads tools."
+    bclient = BeadsClient(_primary_base_dir)
+    if tool_name == "bd_list":
+        beads = bclient.list_beads()
+        if not beads:
+            return "No beads found."
+        return "\n".join([f"ID: {b.id}, Status: {b.status}, Title: {b.title}" for b in beads])
+    elif tool_name == "bd_create":
+        bead_id = bclient.create_bead(title=tool_input["title"], description=tool_input["description"])
+        return f"Created bead {bead_id}"
+    elif tool_name == "bd_update":
+        success = bclient.update_bead(tool_input["bead_id"], tool_input["status"])
+        return f"Updated {tool_input['bead_id']}" if success else f"Bead {tool_input['bead_id']} not found"
+    elif tool_name == "bd_ready":
+        # Mock: return all active beads. Real: filter by resolved dependencies.
+        beads = [b for b in bclient.list_beads() if b.status == "active"]
+        ...
+```
+
+### Tool Schemas (registered in `get_tool_schemas`)
+
+```python
+{
+    "name": "bd_create",
+    "description": "Create a new bead in the active Beads repository",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "title": {"type": "string"},
+            "description": {"type": "string"}
+        },
+        "required": ["title", "description"]
+    }
+}
+# Similar for bd_update, bd_list, bd_ready
+```
+
+### Required Base Directory
+
+The dispatch requires `_primary_base_dir` to be set (i.e., a project is loaded). If not, the tools return `"ERROR: no active workspace to run beads tools."` This prevents agents from accidentally reading/writing to a global Beads repo.
+
+---
+
+## MMA Integration (Roadmap)
+
+**Current state**: The `ConductorEngine` does **not** yet consult or write to Beads. Tracks and tickets are still stored in `conductor/tracks/<id>/` markdown files.
+
+**Planned Integration**:
+
+### 1. `ConductorEngine.parse_json_tickets` → Beads
+
+When Beads mode is active, the JSON tickets ingested from Tier 2's output would be forwarded to `BeadsClient.create_bead`:
+
+```python
+def parse_json_tickets(self, json_str: str) -> None:
+    tickets = json.loads(json_str)
+    if self.beads_enabled:
+        for ticket in tickets:
+            self.beads_client.create_bead(
+                title=ticket["description"],
+                description=json.dumps(ticket, indent=2)  # Full ticket as description
+            )
+    else:
+        # Existing markdown-based path
+        ...
+```
+
+### 2. `ConductorEngine.save_track_state` → Beads
+
+Track state persistence would write to Beads instead of `state.toml`:
+
+```python
+def save_track_state(self, track_id: str, state: TrackState) -> None:
+    if self.beads_enabled:
+        # Update the corresponding bead's status
+        for ticket in state.tickets:
+            self.beads_client.update_bead(ticket.id, ticket.status)
+    else:
+        # Existing markdown path
+        ...
+```
+
+### 3. Visual DAG → Beads
+
+The MMA dashboard's Visual DAG would query `BeadsClient.list_beads()` instead of the in-memory `TrackDAG`:
+
+```python
+def get_dag_tickets(self) -> List[Ticket]:
+    if self.beads_enabled:
+        beads = self.beads_client.list_beads()
+        return [self._bead_to_ticket(b) for b in beads]
+    else:
+        return self.track.tickets
+```
+
+### 4. Context Compaction for Beads
+
+Completed beads would be summarized and archived to free context window space (see `test_aggregate_beads.py:test_build_beads_compaction` for the test pattern).
+
+### Configuration
+
+```toml
+[beads]
+enabled = true
+backend = "mock"  # or "dolt" when implemented
+auto_archive_completed = true
+```
+
+When `backend = "dolt"`, the real Beads client is used (when implemented). When `backend = "mock"`, the current JSON storage is used. This allows projects to migrate incrementally.
+
+---
+
+## Activation
+
+To enable Beads mode for a project:
+
+1. Add to `manual_slop.toml`:
+   ```toml
+   [beads]
+   enabled = true
+   backend = "mock"  # "dolt" once available
+   ```
+
+2. Restart Manual Slop (or reload the project).
+
+3. The GUI's MMA Dashboard gains a "Beads" tab showing the current beads.
+
+4. Agents can now use the `bd_*` MCP tools.
+
+**Note**: As of 2026-06-02, the conductor's track/ticket storage does **not** yet auto-migrate to Beads. Tracks created before Beads activation remain in `conductor/tracks/`. Tracks created after Beads activation may be written to Beads (depending on which integration milestone is shipped).
+
+---
+
+## Testing
+
+### Unit Tests
+
+- `tests/test_beads_client.py` — `BeadsClient` lifecycle (init, create, update, list) against a temp directory
+- `tests/test_mcp_client_beads.py` — End-to-end MCP dispatch for `bd_create`, `bd_list`, `bd_update`, `bd_ready`
+
+### Integration Tests
+
+- `tests/test_gui_dag_beads.py` — Loads beads into the Visual DAG, verifies nodes and edges
+- `tests/test_aggregate_beads.py` — Context compaction for completed beads
+
+### Test Pattern
+
+```python
+def test_bd_create_dispatch(tmp_path, live_gui):
+    # Set up a project with Beads enabled
+    project_dir = tmp_path / "test_project"
+    project_dir.mkdir()
+    (project_dir / "manual_slop.toml").write_text("[beads]\nenabled = true\nbackend = \"mock\"\n")
+    
+    # Trigger dispatch via the MCP client
+    response = live_gui[1].call_tool("bd_create", {
+        "title": "Test Bead",
+        "description": "Created in test"
+    })
+    
+    # Verify the bead was created
+    assert "Created bead bead-" in response
+```
+
+The mock backend's filesystem-based storage makes tests fast and deterministic. Tests use `tmp_path` for isolation, satisfying the **Artifact Isolation** rule in the Structural Testing Contract.
+
+---
+
+## Limitations
+
+1. **Mock Backend**: The current `BeadsClient` uses JSON files, not Dolt. There's no SQL query support, no transaction isolation, no proper concurrent write handling.
+
+2. **No Dependency Graph**: The mock's `list_beads()` returns all beads. The `bd_ready` tool is a placeholder that returns all `active` beads regardless of dependencies. The real Dolt integration will use a `dependencies` table to implement true ready-queue semantics.
+
+3. **No Versioning**: The mock has no history. Beads created today are the same as beads created yesterday. The real Dolt integration will provide point-in-time queries and diffs.
+
+4. **No External Sync**: The mock doesn't sync with the `bd` CLI. Changes via `bd` CLI on the same `.beads_mock/` directory would be lost on next write from the Python client (and vice versa).
+
+5. **Conductor Coexistence**: Until the planned integration lands, beads and conductor tracks are two parallel systems. Users must choose one or the other per project (or accept that some data lives in both).
+
+6. **No Schema Migration**: When the real Dolt integration lands, a migration tool will be needed to convert existing `.beads_mock/beads.json` files to Dolt SQL tables. This tool is not yet built.
+
+---
+
+## Future Work
+
+- **Dolt Integration** — Replace the mock with a real Dolt-backed client. Use Dolt's Python client or shell out to the `dolt` CLI.
+- **Dependency Graph** — Add a `dependencies` table; implement `bd_ready` with proper dependency resolution.
+- **Conductor Coexistence** — Allow tracks to be either markdown-based or Beads-based per project.
+- **External CLI Sync** — Detect `bd` CLI invocations and reconcile state.
+- **Migration Tool** — Convert `.beads_mock/beads.json` to Dolt.
+- **Real-Time Updates** — Beads created by external `bd` invocations appear in the GUI in real time via Dolt's event log.
+
+See [guide_mma.md#beads-integration-roadmap](guide_mma.md#beads-integration-roadmap) for the broader MMA integration plan.