3.5 KiB
3.5 KiB
Track Specification: Pipeline Pause/Resume (pipeline_pause_resume_20260306)
Overview
Add global pause/resume for entire DAG execution pipeline. Allow user to freeze all worker activity and resume later without losing state.
Current State Audit
Already Implemented (DO NOT re-implement)
Execution Loop (multi_agent_conductor.py)
ConductorEngine.run(): Async loop that processes tickets- Loop continues until: All complete OR all blocked OR error
- No pause mechanism
Execution Engine (dag_engine.py)
ExecutionEngine.tick(): Returns ready tasksauto_queueflag: Controls automatic task promotion- No global pause state
GUI State (gui_2.py)
mma_status: "idle" | "planning" | "executing" | "done"- No paused state
Gaps to Fill (This Track's Scope)
- No way to pause execution mid-pipeline
- No way to resume from paused state
- No visual indicator for paused state
Architectural Constraints
State Preservation
- Running workers MUST complete before pause takes effect
- Paused state MUST preserve all ticket statuses
- No data loss on resume
Atomic Operation
- Pause MUST be atomic (all-or-nothing)
- No partial pause state
Non-Blocking
- Pause request MUST NOT block GUI thread
- Pause signaled via threading.Event
Architecture Reference
Key Integration Points
| File | Lines | Purpose |
|---|---|---|
src/multi_agent_conductor.py |
80-150 | ConductorEngine.run() - add pause check |
src/dag_engine.py |
50-80 | ExecutionEngine - add pause state |
src/gui_2.py |
~170 | State for pause flag |
src/gui_2.py |
2650-2750 | _render_mma_dashboard() - add pause button |
Proposed Pause Pattern
# In ConductorEngine:
self._pause_event: threading.Event = threading.Event()
def pause(self) -> None:
self._pause_event.set()
def resume(self) -> None:
self._pause_event.clear()
# In run() loop:
async def run(self):
while True:
if self._pause_event.is_set():
await asyncio.sleep(0.5) # Wait while paused
continue
# Normal processing...
Functional Requirements
FR1: Pause Button
- Button in MMA dashboard
- Disabled when no execution active
- Click triggers
engine.pause()
FR2: Resume Button
- Button in MMA dashboard (replaces pause when paused)
- Disabled when not paused
- Click triggers
engine.resume()
FR3: Visual Indicator
- Banner or icon when paused
mma_statusshows "paused"- Ticket status preserved
FR4: State Display
- Show which workers were running when paused
- Show pending tasks that will resume
Non-Functional Requirements
| Requirement | Constraint |
|---|---|
| Response Time | Pause takes effect within 500ms |
| No Data Loss | All state preserved |
| Visual Feedback | Clear paused indicator |
Testing Requirements
Unit Tests
- Test pause stops task spawning
- Test resume continues from correct state
- Test state preserved across pause
Integration Tests (via live_gui fixture)
- Start execution, pause, verify workers stop
- Resume, verify execution continues
- Verify no state loss
Out of Scope
- Per-ticket pause (all-or-nothing only)
- Scheduled pause
- Pause during individual API call
Acceptance Criteria
- Pause button freezes pipeline
- Resume button continues execution
- Visual indicator shows paused state
- Worker states preserved
- No data loss on resume
mma_statusincludes "paused"- 1-space indentation maintained