From 941b459bc8529594c99d17cbf76f9ccee9719a2b Mon Sep 17 00:00:00 2001 From: Ed_ Date: Tue, 2 Jun 2026 19:40:15 -0400 Subject: [PATCH] docs(beads): new guide for Beads mode covering architecture, mock client, MCP tools, and MMA integration roadmap --- docs/guide_beads.md | 359 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 359 insertions(+) create mode 100644 docs/guide_beads.md diff --git a/docs/guide_beads.md b/docs/guide_beads.md new file mode 100644 index 00000000..2a5429b7 --- /dev/null +++ b/docs/guide_beads.md @@ -0,0 +1,359 @@ +# Beads Mode (Dolt-Backed Issue Tracking) + +[Top](../README.md) | [MMA](guide_mma.md) | [Tools & IPC](guide_tools.md) | [Simulations](guide_simulations.md) + +--- + +## Overview + +Beads is a [Dolt](https://github.com/dolthub/dolt)-backed issue tracking system used as a first-class, project-specific alternative to Manual Slop's markdown-based ticket tracking. Beads stores the task graph in a local `.beads/` Dolt repository, allowing tracks and tickets to be versioned alongside the code in git. + +This guide covers: + +1. **Architecture** — Where Beads fits in the manual_slop stack +2. **Components** — `BeadsClient`, the Dolt repository, the bead data model +3. **MCP Tool Integration** — `bd_create`, `bd_list`, `bd_ready`, `bd_update` +4. **MMA Integration (Roadmap)** — How Beads mode would integrate with the ConductorEngine +5. **Testing** — Current state and limitations + +**Current state (as of 2026-06-02):** The `BeadsClient` is a **mock implementation** that uses a `.beads_mock/` directory with a JSON file. The real Dolt-backed Beads integration is **roadmap**, not yet implemented. The MCP tools, however, are fully wired and functional against the mock. When the real Beads is integrated, the MCP tool signatures and the `BeadsClient` public API will remain stable; only the backing storage changes. + +--- + +## Architecture + +Beads is a **per-project** issue tracking layer that lives alongside Manual Slop's other per-project state (conductor directory, log directory, project TOML). + +``` +manual_slop_project/ +├── .beads/ # Real Beads: Dolt repository (planned) +│ ├── .dolt/ # Dolt internal storage (when implemented) +│ └── config.json # Beads configuration +├── .beads_mock/ # Current mock: flat JSON storage +│ └── beads.json # List of bead objects +├── conductor/ # Manual Slop's own track storage +│ └── tracks/ +├── manual_slop.toml # Project configuration +└── ... (source files) +``` + +**Lifecycle**: +- A project may opt into Beads mode by setting `[beads] enabled = true` in `manual_slop.toml`. +- When enabled, `BeadsClient.init_repo()` is called on project load to ensure the `.beads/` (or `.beads_mock/`) directory exists. +- All track/ticket state for that project flows through `BeadsClient` instead of the conductor's `conductor/tracks//` markdown files. +- The MMA dashboard's Visual DAG reads from `BeadsClient.list_beads()` instead of in-memory `TrackDAG`. + +**Why Beads?** The Markdown-based track storage (in `conductor/tracks/`) works well for human-authored planning but is awkward for AI-driven workflows. Beads provides: +- **Programmatic CRUD** — agents can `bd_create`, `bd_update`, `bd_ready` without parsing markdown. +- **Dependency graphs** — beads have explicit `depends_on` relationships with cycle detection. +- **Versioning** — the Dolt repo is git-trackable, so the task graph has the same history guarantees as the code. +- **External tooling** — the `bd` CLI works outside Manual Slop for human review. + +--- + +## Components + +### `Bead` (Data Model) + +```python +@dataclass +class Bead: + id: str + title: str + description: str + status: str = "active" +``` + +| Field | Type | Purpose | +|---|---|---| +| `id` | `str` | Unique identifier. Format: `bead-` (sequential, mock) or Dolt-generated hash (real). | +| `title` | `str` | Short, human-readable title. | +| `description` | `str` | Full description. May include multi-line text, file references, and acceptance criteria. | +| `status` | `str` | One of `active`, `in_progress`, `closed`, `blocked`. | + +**Future fields** (planned for the real Dolt integration): +- `depends_on: List[str]` — explicit dependency graph +- `assignee: Optional[str]` — Tier or human owner +- `priority: int` — 0-9 ranking +- `created_at: datetime`, `updated_at: datetime` +- `tags: List[str]` — for filtering +- `acceptance_criteria: List[str]` — structured completion conditions + +### `BeadsClient` (`src/beads_client.py`) + +The Python client for the Beads backend. + +```python +class BeadsClient: + def __init__(self, working_dir: Path): + self.working_dir = Path(working_dir) + self.repo_dir = self.working_dir / ".beads_mock" + self.beads_file = self.repo_dir / "beads.json" +``` + +**Construction**: Takes a `working_dir` (typically the project base directory). The mock client derives the storage path as `/.beads_mock/beads.json`. The real client will derive `/.beads/` for Dolt. + +**Public Methods**: + +```python +def init_repo(self) -> None: + """Create the storage directory and an empty beads.json if not present.""" + +def is_initialized(self) -> bool: + """Check whether the storage directory exists.""" + +def create_bead(self, title: str, description: str) -> str: + """Create a new bead. Returns the new bead's ID.""" + +def update_bead(self, bead_id: str, status: str) -> bool: + """Update the status of an existing bead. Returns True if found, False otherwise.""" + +def list_beads(self) -> List[Bead]: + """Return all beads as a list of Bead objects.""" +``` + +**Internal Storage (mock)**: + +```python +def _read_beads(self) -> List[dict]: + if not self.beads_file.exists(): + return [] + return json.loads(self.beads_file.read_text(encoding="utf-8")) + +def _write_beads(self, beads: List[dict]) -> None: + self.beads_file.write_text(json.dumps(beads, indent=1), encoding="utf-8") +``` + +The mock uses a single JSON file with a list of bead objects. The real client will use Dolt SQL tables (`beads`, `dependencies`). + +**Thread Safety**: The mock client's `_read_beads` / `_write_beads` are not internally synchronized. The caller (typically `mcp_client.dispatch` from a worker thread) is expected to serialize access. The real Dolt-backed client will use Dolt's transaction system. + +### Beads CLI (`bd`) + +When the real integration is in place, the `bd` CLI (provided by the [Beads](https://github.com/steveyegge/beads) project) is the primary user-facing tool for managing beads outside Manual Slop. The Python client is a thin wrapper around `bd` subprocess calls or a direct Dolt connection. + +**CLI usage (planned)**: +```bash +bd create "Fix RAG race condition" "Vector store concurrent writes need a lock" +bd list +bd ready # List beads with no unresolved dependencies +bd update bead-3 in_progress +``` + +The Python `BeadsClient` is intended to be a programmatic equivalent of the `bd` CLI for use by agents. + +--- + +## MCP Tool Integration + +The four Beads operations are exposed as MCP tools in `src/mcp_client.py:1474-1494`. The dispatch checks `tool_name.startswith("bd_")` and routes to `BeadsClient` methods. + +### Tool Inventory + +| Tool | Parameters | Maps to | Returns | +|---|---|---|---| +| `bd_create` | `title: str`, `description: str` | `BeadsClient.create_bead` | The new bead's ID | +| `bd_list` | (none) | `BeadsClient.list_beads` | Formatted list: `ID: , Status: , Title: ` | +| `bd_update` | `bead_id: str`, `status: str` | `BeadsClient.update_bead` | Success/failure indicator | +| `bd_ready` | (none) | (filtered `list_beads`) | Beads with no unresolved dependencies (mock: returns all `active` beads) | + +### Dispatch Flow + +```python +# In src/mcp_client.py:dispatch +if tool_name.startswith("bd_"): + if not _primary_base_dir: + return "ERROR: no active workspace to run beads tools." + bclient = BeadsClient(_primary_base_dir) + if tool_name == "bd_list": + beads = bclient.list_beads() + if not beads: + return "No beads found." + return "\n".join([f"ID: {b.id}, Status: {b.status}, Title: {b.title}" for b in beads]) + elif tool_name == "bd_create": + bead_id = bclient.create_bead(title=tool_input["title"], description=tool_input["description"]) + return f"Created bead {bead_id}" + elif tool_name == "bd_update": + success = bclient.update_bead(tool_input["bead_id"], tool_input["status"]) + return f"Updated {tool_input['bead_id']}" if success else f"Bead {tool_input['bead_id']} not found" + elif tool_name == "bd_ready": + # Mock: return all active beads. Real: filter by resolved dependencies. + beads = [b for b in bclient.list_beads() if b.status == "active"] + ... +``` + +### Tool Schemas (registered in `get_tool_schemas`) + +```python +{ + "name": "bd_create", + "description": "Create a new bead in the active Beads repository", + "parameters": { + "type": "object", + "properties": { + "title": {"type": "string"}, + "description": {"type": "string"} + }, + "required": ["title", "description"] + } +} +# Similar for bd_update, bd_list, bd_ready +``` + +### Required Base Directory + +The dispatch requires `_primary_base_dir` to be set (i.e., a project is loaded). If not, the tools return `"ERROR: no active workspace to run beads tools."` This prevents agents from accidentally reading/writing to a global Beads repo. + +--- + +## MMA Integration (Roadmap) + +**Current state**: The `ConductorEngine` does **not** yet consult or write to Beads. Tracks and tickets are still stored in `conductor/tracks/<id>/` markdown files. + +**Planned Integration**: + +### 1. `ConductorEngine.parse_json_tickets` → Beads + +When Beads mode is active, the JSON tickets ingested from Tier 2's output would be forwarded to `BeadsClient.create_bead`: + +```python +def parse_json_tickets(self, json_str: str) -> None: + tickets = json.loads(json_str) + if self.beads_enabled: + for ticket in tickets: + self.beads_client.create_bead( + title=ticket["description"], + description=json.dumps(ticket, indent=2) # Full ticket as description + ) + else: + # Existing markdown-based path + ... +``` + +### 2. `ConductorEngine.save_track_state` → Beads + +Track state persistence would write to Beads instead of `state.toml`: + +```python +def save_track_state(self, track_id: str, state: TrackState) -> None: + if self.beads_enabled: + # Update the corresponding bead's status + for ticket in state.tickets: + self.beads_client.update_bead(ticket.id, ticket.status) + else: + # Existing markdown path + ... +``` + +### 3. Visual DAG → Beads + +The MMA dashboard's Visual DAG would query `BeadsClient.list_beads()` instead of the in-memory `TrackDAG`: + +```python +def get_dag_tickets(self) -> List[Ticket]: + if self.beads_enabled: + beads = self.beads_client.list_beads() + return [self._bead_to_ticket(b) for b in beads] + else: + return self.track.tickets +``` + +### 4. Context Compaction for Beads + +Completed beads would be summarized and archived to free context window space (see `test_aggregate_beads.py:test_build_beads_compaction` for the test pattern). + +### Configuration + +```toml +[beads] +enabled = true +backend = "mock" # or "dolt" when implemented +auto_archive_completed = true +``` + +When `backend = "dolt"`, the real Beads client is used (when implemented). When `backend = "mock"`, the current JSON storage is used. This allows projects to migrate incrementally. + +--- + +## Activation + +To enable Beads mode for a project: + +1. Add to `manual_slop.toml`: + ```toml + [beads] + enabled = true + backend = "mock" # "dolt" once available + ``` + +2. Restart Manual Slop (or reload the project). + +3. The GUI's MMA Dashboard gains a "Beads" tab showing the current beads. + +4. Agents can now use the `bd_*` MCP tools. + +**Note**: As of 2026-06-02, the conductor's track/ticket storage does **not** yet auto-migrate to Beads. Tracks created before Beads activation remain in `conductor/tracks/`. Tracks created after Beads activation may be written to Beads (depending on which integration milestone is shipped). + +--- + +## Testing + +### Unit Tests + +- `tests/test_beads_client.py` — `BeadsClient` lifecycle (init, create, update, list) against a temp directory +- `tests/test_mcp_client_beads.py` — End-to-end MCP dispatch for `bd_create`, `bd_list`, `bd_update`, `bd_ready` + +### Integration Tests + +- `tests/test_gui_dag_beads.py` — Loads beads into the Visual DAG, verifies nodes and edges +- `tests/test_aggregate_beads.py` — Context compaction for completed beads + +### Test Pattern + +```python +def test_bd_create_dispatch(tmp_path, live_gui): + # Set up a project with Beads enabled + project_dir = tmp_path / "test_project" + project_dir.mkdir() + (project_dir / "manual_slop.toml").write_text("[beads]\nenabled = true\nbackend = \"mock\"\n") + + # Trigger dispatch via the MCP client + response = live_gui[1].call_tool("bd_create", { + "title": "Test Bead", + "description": "Created in test" + }) + + # Verify the bead was created + assert "Created bead bead-" in response +``` + +The mock backend's filesystem-based storage makes tests fast and deterministic. Tests use `tmp_path` for isolation, satisfying the **Artifact Isolation** rule in the Structural Testing Contract. + +--- + +## Limitations + +1. **Mock Backend**: The current `BeadsClient` uses JSON files, not Dolt. There's no SQL query support, no transaction isolation, no proper concurrent write handling. + +2. **No Dependency Graph**: The mock's `list_beads()` returns all beads. The `bd_ready` tool is a placeholder that returns all `active` beads regardless of dependencies. The real Dolt integration will use a `dependencies` table to implement true ready-queue semantics. + +3. **No Versioning**: The mock has no history. Beads created today are the same as beads created yesterday. The real Dolt integration will provide point-in-time queries and diffs. + +4. **No External Sync**: The mock doesn't sync with the `bd` CLI. Changes via `bd` CLI on the same `.beads_mock/` directory would be lost on next write from the Python client (and vice versa). + +5. **Conductor Coexistence**: Until the planned integration lands, beads and conductor tracks are two parallel systems. Users must choose one or the other per project (or accept that some data lives in both). + +6. **No Schema Migration**: When the real Dolt integration lands, a migration tool will be needed to convert existing `.beads_mock/beads.json` files to Dolt SQL tables. This tool is not yet built. + +--- + +## Future Work + +- **Dolt Integration** — Replace the mock with a real Dolt-backed client. Use Dolt's Python client or shell out to the `dolt` CLI. +- **Dependency Graph** — Add a `dependencies` table; implement `bd_ready` with proper dependency resolution. +- **Conductor Coexistence** — Allow tracks to be either markdown-based or Beads-based per project. +- **External CLI Sync** — Detect `bd` CLI invocations and reconcile state. +- **Migration Tool** — Convert `.beads_mock/beads.json` to Dolt. +- **Real-Time Updates** — Beads created by external `bd` invocations appear in the GUI in real time via Dolt's event log. + +See [guide_mma.md#beads-integration-roadmap](guide_mma.md#beads-integration-roadmap) for the broader MMA integration plan.