From 7a7298730db142ab9ebea76a02b872c2b1c8b26c Mon Sep 17 00:00:00 2001 From: Ed_ Date: Wed, 6 May 2026 14:44:54 -0400 Subject: [PATCH] docs(perf): Document identified bottlenecks --- .../bottlenecks.md | 37 +++++++++++++++++++ .../plan.md | 2 +- 2 files changed, 38 insertions(+), 1 deletion(-) create mode 100644 conductor/tracks/data_oriented_optimization_20260312/bottlenecks.md diff --git a/conductor/tracks/data_oriented_optimization_20260312/bottlenecks.md b/conductor/tracks/data_oriented_optimization_20260312/bottlenecks.md new file mode 100644 index 0000000..4edfb6f --- /dev/null +++ b/conductor/tracks/data_oriented_optimization_20260312/bottlenecks.md @@ -0,0 +1,37 @@ +# Identified Bottleneck Targets: Data-Oriented Python Optimization Pass + +## Target 1: Context Aggregation Logic (`src/aggregate.py`) +- **Bottleneck:** O(N*M) membership checks in `build_tier3_context` and `build_tier1_context`. +- **Symptom:** As the number of focus files and total project files increase, context building becomes slower. +- **Heuristic Violation:** "Less Python does, the better." Iterative string matching in a loop is expensive in Python. +- **Proposed Fix:** Pre-calculate a set of focus paths and use O(1) lookups. + +## Target 2: DAG Graph Operations (`src/dag_engine.py`) +- **Bottleneck:** Recursive DFS in `has_cycle` and `topological_sort`. +- **Symptom:** Risk of `RecursionError` on very deep graphs; function call overhead for every node visit. +- **Heuristic Violation:** Deep recursion is a "More Python" approach. +- **Proposed Fix:** Implement iterative versions of DFS using an explicit stack. + +## Target 3: Transitive Blocking Propagation (`src/dag_engine.py`) +- **Bottleneck:** O(N^2) or O(N*D) stable-loop in `cascade_blocks`. +- **Symptom:** Repeated iteration over the entire ticket list until no more changes occur. +- **Heuristic Violation:** Redundant iterations. +- **Proposed Fix:** Use a more efficient propagation algorithm (e.g., propagating only from modified nodes or using a topological traversal). + +## Target 4: Orchestrator Main Loop (`src/multi_agent_conductor.py`) +- **Bottleneck:** Nested imports inside `ConductorEngine.run` loop. +- **Symptom:** Repeatedly calling `import` and searching the module cache every second. +- **Heuristic Violation:** Unnecessary JIT/interpreter work. +- **Proposed Fix:** Move all imports to the top of the file. + +## Target 5: Orchestrator Idle Overhead (`src/multi_agent_conductor.py`) +- **Bottleneck:** Unnecessary `tick()` and `cascade_blocks()` calls in the main loop when no tasks are running or finished. +- **Symptom:** CPU waste in the background thread. +- **Heuristic Violation:** "The less Python does, the better." Don't recalculate what hasn't changed. +- **Proposed Fix:** Only trigger a DAG tick when a significant state change occurs (e.g., a ticket is completed). + +## Target 6: Simulation Typing Latency (`simulation/user_agent.py`) +- **Bottleneck:** Character-by-character `time.sleep` in `simulate_typing`. +- **Symptom:** Extremely slow simulations for large inputs. +- **Heuristic Violation:** Excessive blocking in a loop. +- **Proposed Fix:** Batch typing or provide a toggle to disable jitter for performance-oriented simulations. diff --git a/conductor/tracks/data_oriented_optimization_20260312/plan.md b/conductor/tracks/data_oriented_optimization_20260312/plan.md index 991563d..1896bbf 100644 --- a/conductor/tracks/data_oriented_optimization_20260312/plan.md +++ b/conductor/tracks/data_oriented_optimization_20260312/plan.md @@ -10,7 +10,7 @@ - [x] Task: Run profiling scenarios (especially utilizing simulations) to generate baseline metrics. (83afc90) - [x] Task: Audit `src/` (e.g., `dag_engine.py`, `multi_agent_conductor.py`, `aggregate.py`) against the new guidelines, cross-referencing with profiling data to identify bottlenecks. (7dc91dd) - [x] Task: Audit `simulation/` files against the new guidelines to ensure the test harness is performant and non-blocking. (05db5bd) -- [~] Task: Compile a list of identified bottleneck targets to refactor. +- [x] Task: Compile a list of identified bottleneck targets to refactor. (1294619) - [ ] Task: Conductor - User Manual Verification 'Phase 2: Audit and Profiling (`src/` and `simulation/`)' (Protocol in workflow.md) ## Phase 3: Targeted Optimization and Refactoring