Files
manual_slop/conductor/archive/pipeline_pause_resume_20260306/spec.md
2026-03-08 13:29:53 -04:00

3.5 KiB

Track Specification: Pipeline Pause/Resume (pipeline_pause_resume_20260306)

Overview

Add global pause/resume for entire DAG execution pipeline. Allow user to freeze all worker activity and resume later without losing state.

Current State Audit

Already Implemented (DO NOT re-implement)

Execution Loop (multi_agent_conductor.py)

  • ConductorEngine.run(): Async loop that processes tickets
  • Loop continues until: All complete OR all blocked OR error
  • No pause mechanism

Execution Engine (dag_engine.py)

  • ExecutionEngine.tick(): Returns ready tasks
  • auto_queue flag: Controls automatic task promotion
  • No global pause state

GUI State (gui_2.py)

  • mma_status: "idle" | "planning" | "executing" | "done"
  • No paused state

Gaps to Fill (This Track's Scope)

  • No way to pause execution mid-pipeline
  • No way to resume from paused state
  • No visual indicator for paused state

Architectural Constraints

State Preservation

  • Running workers MUST complete before pause takes effect
  • Paused state MUST preserve all ticket statuses
  • No data loss on resume

Atomic Operation

  • Pause MUST be atomic (all-or-nothing)
  • No partial pause state

Non-Blocking

  • Pause request MUST NOT block GUI thread
  • Pause signaled via threading.Event

Architecture Reference

Key Integration Points

File Lines Purpose
src/multi_agent_conductor.py 80-150 ConductorEngine.run() - add pause check
src/dag_engine.py 50-80 ExecutionEngine - add pause state
src/gui_2.py ~170 State for pause flag
src/gui_2.py 2650-2750 _render_mma_dashboard() - add pause button

Proposed Pause Pattern

# In ConductorEngine:
self._pause_event: threading.Event = threading.Event()

def pause(self) -> None:
 self._pause_event.set()

def resume(self) -> None:
 self._pause_event.clear()

# In run() loop:
async def run(self):
 while True:
  if self._pause_event.is_set():
   await asyncio.sleep(0.5)  # Wait while paused
   continue
  # Normal processing...

Functional Requirements

FR1: Pause Button

  • Button in MMA dashboard
  • Disabled when no execution active
  • Click triggers engine.pause()

FR2: Resume Button

  • Button in MMA dashboard (replaces pause when paused)
  • Disabled when not paused
  • Click triggers engine.resume()

FR3: Visual Indicator

  • Banner or icon when paused
  • mma_status shows "paused"
  • Ticket status preserved

FR4: State Display

  • Show which workers were running when paused
  • Show pending tasks that will resume

Non-Functional Requirements

Requirement Constraint
Response Time Pause takes effect within 500ms
No Data Loss All state preserved
Visual Feedback Clear paused indicator

Testing Requirements

Unit Tests

  • Test pause stops task spawning
  • Test resume continues from correct state
  • Test state preserved across pause

Integration Tests (via live_gui fixture)

  • Start execution, pause, verify workers stop
  • Resume, verify execution continues
  • Verify no state loss

Out of Scope

  • Per-ticket pause (all-or-nothing only)
  • Scheduled pause
  • Pause during individual API call

Acceptance Criteria

  • Pause button freezes pipeline
  • Resume button continues execution
  • Visual indicator shows paused state
  • Worker states preserved
  • No data loss on resume
  • mma_status includes "paused"
  • 1-space indentation maintained