refinement of upcoming tracks
This commit is contained in:
@@ -1,21 +1,129 @@
|
||||
# Track Specification: Pipeline Pause/Resume (pipeline_pause_resume_20260306)
|
||||
|
||||
## Overview
|
||||
Add global pause/resume for entire DAG execution pipeline. Allow user to freeze all worker activity and resume later.
|
||||
Add global pause/resume for entire DAG execution pipeline. Allow user to freeze all worker activity and resume later without losing state.
|
||||
|
||||
## Current State Audit
|
||||
|
||||
### Already Implemented (DO NOT re-implement)
|
||||
|
||||
#### Execution Loop (multi_agent_conductor.py)
|
||||
- **`ConductorEngine.run()`**: Async loop that processes tickets
|
||||
- **Loop continues until**: All complete OR all blocked OR error
|
||||
- **No pause mechanism**
|
||||
|
||||
#### Execution Engine (dag_engine.py)
|
||||
- **`ExecutionEngine.tick()`**: Returns ready tasks
|
||||
- **`auto_queue` flag**: Controls automatic task promotion
|
||||
- **No global pause state**
|
||||
|
||||
#### GUI State (gui_2.py)
|
||||
- **`mma_status`**: "idle" | "planning" | "executing" | "done"
|
||||
- **No paused state**
|
||||
|
||||
### Gaps to Fill (This Track's Scope)
|
||||
- No way to pause execution mid-pipeline
|
||||
- No way to resume from paused state
|
||||
- No visual indicator for paused state
|
||||
|
||||
## Architectural Constraints
|
||||
- **State Preservation**: Worker state MUST be preserved across pause.
|
||||
- **Atomic Operation**: Pause/Resume MUST be atomic.
|
||||
|
||||
### State Preservation
|
||||
- Running workers MUST complete before pause takes effect
|
||||
- Paused state MUST preserve all ticket statuses
|
||||
- No data loss on resume
|
||||
|
||||
### Atomic Operation
|
||||
- Pause MUST be atomic (all-or-nothing)
|
||||
- No partial pause state
|
||||
|
||||
### Non-Blocking
|
||||
- Pause request MUST NOT block GUI thread
|
||||
- Pause signaled via threading.Event
|
||||
|
||||
## Architecture Reference
|
||||
|
||||
### Key Integration Points
|
||||
|
||||
| File | Lines | Purpose |
|
||||
|------|-------|---------|
|
||||
| `src/multi_agent_conductor.py` | 80-150 | `ConductorEngine.run()` - add pause check |
|
||||
| `src/dag_engine.py` | 50-80 | `ExecutionEngine` - add pause state |
|
||||
| `src/gui_2.py` | ~170 | State for pause flag |
|
||||
| `src/gui_2.py` | 2650-2750 | `_render_mma_dashboard()` - add pause button |
|
||||
|
||||
### Proposed Pause Pattern
|
||||
|
||||
```python
|
||||
# In ConductorEngine:
|
||||
self._pause_event: threading.Event = threading.Event()
|
||||
|
||||
def pause(self) -> None:
|
||||
self._pause_event.set()
|
||||
|
||||
def resume(self) -> None:
|
||||
self._pause_event.clear()
|
||||
|
||||
# In run() loop:
|
||||
async def run(self):
|
||||
while True:
|
||||
if self._pause_event.is_set():
|
||||
await asyncio.sleep(0.5) # Wait while paused
|
||||
continue
|
||||
# Normal processing...
|
||||
```
|
||||
|
||||
## Functional Requirements
|
||||
- **Pause Button**: Freeze all worker activity.
|
||||
- **Resume Button**: Continue from paused state.
|
||||
- **Visual Indicator**: Show pipeline is paused.
|
||||
- **State Display**: Show which workers are paused.
|
||||
|
||||
### FR1: Pause Button
|
||||
- Button in MMA dashboard
|
||||
- Disabled when no execution active
|
||||
- Click triggers `engine.pause()`
|
||||
|
||||
### FR2: Resume Button
|
||||
- Button in MMA dashboard (replaces pause when paused)
|
||||
- Disabled when not paused
|
||||
- Click triggers `engine.resume()`
|
||||
|
||||
### FR3: Visual Indicator
|
||||
- Banner or icon when paused
|
||||
- `mma_status` shows "paused"
|
||||
- Ticket status preserved
|
||||
|
||||
### FR4: State Display
|
||||
- Show which workers were running when paused
|
||||
- Show pending tasks that will resume
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
| Requirement | Constraint |
|
||||
|-------------|------------|
|
||||
| Response Time | Pause takes effect within 500ms |
|
||||
| No Data Loss | All state preserved |
|
||||
| Visual Feedback | Clear paused indicator |
|
||||
|
||||
## Testing Requirements
|
||||
|
||||
### Unit Tests
|
||||
- Test pause stops task spawning
|
||||
- Test resume continues from correct state
|
||||
- Test state preserved across pause
|
||||
|
||||
### Integration Tests (via `live_gui` fixture)
|
||||
- Start execution, pause, verify workers stop
|
||||
- Resume, verify execution continues
|
||||
- Verify no state loss
|
||||
|
||||
## Out of Scope
|
||||
- Per-ticket pause (all-or-nothing only)
|
||||
- Scheduled pause
|
||||
- Pause during individual API call
|
||||
|
||||
## Acceptance Criteria
|
||||
- [ ] Pause button freezes pipeline.
|
||||
- [ ] Resume button continues execution.
|
||||
- [ ] Visual indicator shows paused state.
|
||||
- [ ] Worker states preserved.
|
||||
- [ ] No data loss on resume.
|
||||
- [ ] Pause button freezes pipeline
|
||||
- [ ] Resume button continues execution
|
||||
- [ ] Visual indicator shows paused state
|
||||
- [ ] Worker states preserved
|
||||
- [ ] No data loss on resume
|
||||
- [ ] `mma_status` includes "paused"
|
||||
- [ ] 1-space indentation maintained
|
||||
|
||||
Reference in New Issue
Block a user