# Implementation Plan: True Parallel Worker Execution (true_parallel_worker_execution_20260306)

## Phase 1: Verify Existing Implementation
Focus: Understand current parallel execution and add pool management

- [ ] Task 1.1: Initialize MMA Environment
    - Run `activate_skill mma-orchestrator` before starting

- [ ] Task 1.2: Verify current parallel execution
    - WHERE: `src/multi_agent_conductor.py` `ConductorEngine.run()` method (lines ~80-150)
    - What: Confirm parallel spawning exists
    - how: Use `manual-slop_py_get_definition` on `ConductorEngine.run`
    - finding: Parallel execution ALready implemented using threading.Thread
    - note: All ready tickets spawn immediately, no limit

- [ ] Task 1.3: Identify what's missing
    - where: `src/multi_agent_conductor.py`
    - what: Document gaps for pool management
    - gaps:
      - No max_workers limit (all ready spawn)
      - No worker tracking (threads launched but not tracked)
      - No configuration (can't set pool size)

## Phase 2: Worker Pool Management
Focus: Add configurable worker pool with tracking

- [ ] Task 2.1: Add worker pool configuration
    - where: `src/multi_agent_conductor.py` or new module
    - what: Add `_worker_pool` class or configuration
    - how:
      ```python
      class WorkerPool:
       def __init__(self, max_workers: int = 4):
        self.max_workers = max_workers
        self._active: dict[str, threading.Thread] = {}
        self._lock = threading.Lock()
       
       def spawn(self, target, Callable, args: tuple) -> threading.Thread:
          with self._lock:
           if len(self._active) >= self.max_workers:
            return None  # Pool full
           t = threading.Thread(target=target, args=args, daemon=True)
           t.start()
           with self._lock:
            self._active[t.ident] = t
           return t
       
       def join_all(self, timeout: float = None) -> None:
        with self._lock:
           threads = list(self._active.values())
        for t in threads:
           t.join(timeout=timeout)
        with self._lock:
           self._active.clear()
       
       def get_active_count(self) -> int:
        with self._lock:
           return len(self._active)
      ```
    - code style: 1-space indentation

- [ ] Task 2.2: Add max_workers config to config.toml
    - where: `config.toml`
    - what: Add `[mma] max_workers = 4`
    - how: Add section to config.toml
    - safety: Default to 4 if missing

- [ ] Task 2.3: Integrate pool with ConductorEngine
    - where: `src/multi_agent_conductor.py` `ConductorEngine.__init__`
    - what: Initialize WorkerPool instead of raw thread spawning
    - how: Replace direct thread creation with `self.pool.spawn()`
    - safety: Check return value for pool-full case

## Phase 3: Thread Safety
Focus: Ensure safe concurrent access

- [ ] Task 3.1: Add locks to tier_usage updates
    - where: `src/multi_agent_conductor.py`
    - what: Protect tier_usage updates with lock
    - how:
      ```python
      self._tier_lock = threading.Lock()
      
      def update_tier_usage(self, tier: str, usage: dict) -> None:
       with self._tier_lock:
           self.tier_usage[tier]["input"] += usage.get("input", 0)
           self.tier_usage[tier]["output"] += usage.get("output", 0)
      ```
    - safety: Already updated via comms_log, this adds explicit lock

- [ ] Task 3.2: Verify no race conditions
    - where: `tests/test_parallel_execution.py` (new)
    - what: Write tests for concurrent status updates
    - how: Create multiple threads updating same ticket, verify atomic updates

## Phase 4: Testing & Verification
Focus: Verify all functionality

- [ ] Task 4.1: Write unit tests for worker pool
    - where: `tests/test_parallel_execution.py` (new)
    - what: Test pool limits, worker tracking
    - how: Mock worker functions, verify pool behavior

- [ ] Task 4.2: Write integration test
    - where: `tests/test_parallel_execution.py`
    - what: Test with live_gui
    - how: Create multiple independent tickets, verify parallel execution
    - verify: All 4 complete independently

- [ ] Task 4.3: Conductor - Phase Verification
    - run: `uv run pytest tests/test_parallel_execution.py -v --timeout=60`
    - verify no race conditions
    - verify pool limits enforced

## Implementation Notes

### Current Parallel Pattern (existing)
```python
# In ConductorEngine.run():
threads = []
for ticket in to_run:
    t = threading.Thread(target=run_worker_lifecycle, args=(...), daemon=True)
    threads.append(t)
    t.start()

for t in threads:
    t.join()  # Blocks until ALL complete
```
- **Issue**: No limit on concurrent workers
- **Issue**: No tracking of individual threads
- **Issue**: join() blocks the main loop

### Proposed Changes
1. Wrap thread management in `WorkerPool` class
2. Add `max_workers` config option
3. Track active workers with thread references
4. Non-blocking: Return immediately when pool full

### Files Modified
- `src/multi_agent_conductor.py`: Add WorkerPool, update ConductorEngine.run()
- `config.toml`: Add `[mma] max_workers = 4`
- `tests/test_parallel_execution.py`: New test file

### Code Style Checklist
- [ ] 1-space indentation throughout
- [ ] CRLF line endings on Windows
- [ ] No comments unless documenting API
- [ ] Type hints on all public methods
