ed/manual_slop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

second attempt in call graph track

Browse Source
This commit is contained in:
Edward R. Gonzalez
2026-05-07 22:32:09 -04:00
parent 12439ac639
commit 2d48c07760
1 changed files with 73 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conductor/tracks/ai_interaction_call_graph_20260507/SEQUENCE_DIAGRAM.md
+73 -49
View File
@@ -1,62 +1,86 @@
# AI Interaction Loop: Sequence Diagram
# AI Interaction Pipeline: Intensive Technical Trace
This diagram traces the flow of a user message from the GUI thread to the background AI worker thread, including the multi-turn tool-calling loop and final response hand-off.
This document provides a low-level technical trace of the AI interaction loop, following a pipeline-oriented architectural model. It identifies thread context switches, data transformation overhead, and synchronization bottlenecks.
## 1. Sequence Diagram: Asynchronous Interaction Pipeline
```mermaid
sequenceDiagram
participant UI as gui_2.py (Main Thread)
participant AC as app_controller.py (Event Thread)
participant AW as app_controller.py (AI Worker Thread)
participant AI as ai_client.py (Unified SDK)
participant MCP as mcp_client.py (Read-Only Tools)
participant SR as shell_runner.py (PowerShell)
autonumber
participant UI as gui_2.py (Main/Render Thread)
participant EV as app_controller.py (Event Dispatcher)
participant WK as ai_client.py (Worker Thread Pool)
participant AI as ai_client.py (Provider Pipeline)
participant MCP as mcp_client.py (FileSystem Pipeline)
participant SR as shell_runner.py (Subprocess Pipeline)
Note over UI, SR: 1. Initiation
UI->>AC: event_queue.put("user_request", payload)
AC->>AW: threading.Thread(target=_handle_request_event).start()
Note over UI, SR: 2. Request Processing
AW->>AI: send(md_context, prompt, history, ...)
AI->>AI: Prompt Assembly (Context + History + SysPrompt)
loop Tool-Call Loop (r_idx <= MAX_TOOL_ROUNDS)
AI->>Vendor: Provider SDK Request
Vendor-->>AI: Tool Call(s) OR Text
alt Tool Call(s) Detected
AI->>AW: _execute_tool_calls_concurrently()
Note over UI, WK: [Phase A: Request Initiation]
UI->>EV: SyncEventQueue.put("user_request", dict)
Note right of UI: Data: Raw Prompt + Context Pointers
EV->>EV: polling loop (event_queue.get())
EV->>WK: threading.Thread(target=_handle_request_event).start()
Note right of EV: Context Switch: Event Thread -> AI Worker Thread
Note over WK, AI: [Phase B: Context Synthesis & Generation]
WK->>AI: ai_client.send(md_content, history)
AI->>AI: _build_chunked_context_blocks()
Note right of AI: Perf: O(N) string concatenation + regex scans
AI->>Vendor: Provider API Request (HTTPS/JSON)
Note right of AI: Bottleneck: Network Latency (1-30s)
Vendor-->>AI: ToolCall(s) or StopReason
Note over AI, SR: [Phase C: Multi-Turn Tool Execution Loop]
loop MAX_TOOL_ROUNDS (r_idx <= 10)
alt Tool Use Detected
AI->>WK: _execute_tool_calls_concurrently()
alt Read-Only Tool
AW->>MCP: call_tool(args)
MCP-->>AW: output
else Mutating / PS Tool
AW->>UI: _confirm_and_run(script) [MODAL]
Note over UI: User Approval Required
UI-->>AW: Approved
AW->>SR: run_powershell(script)
SR-->>AW: stdout/stderr
alt Read-Only (MCP)
WK->>MCP: read_file / list_dir / search
MCP-->>WK: stdout_string
else Mutating (Shell)
WK->>EV: _pending_gui_tasks.append(approval_modal)
Note over UI: UI Polling Detects Task
UI->>UI: Render ImGui Popup (Wait for HITL)
Note over UI: User Approval Interaction
UI-->>WK: threading.Condition.notify()
Note right of WK: Resume AI Worker Thread
WK->>SR: run_powershell(script)
SR->>OS: Subprocess Spawn (powershell.exe)
OS-->>SR: stdout/stderr (JSON-L Stream)
SR-->>WK: COMBINED_OUTPUT_STRING
end
AW-->>AI: Tool Result(s)
AI->>AI: Append Result to History
else Final Text Returned
AI-->>AW: Final Response Text
WK-->>AI: Aggregate Tool Results
AI->>AI: _reread_file_items() (Context Refresh)
Note right of AI: Perf: IO Bound (File MTime Scans)
AI->>Vendor: Follow-up Prompt (with Tool Result)
else Terminal Text
AI-->>WK: Final AI Response Text
end
end
Note over UI, SR: 3. Completion
AW->>AC: event_queue.put("response", {"text": resp, "status": "done"})
AC->>AC: _pending_gui_tasks.append(response_task)
loop Every Frame
UI->>AC: _process_pending_gui_tasks()
AC-->>UI: response_task
UI->>UI: Update AI History & Reset Status
Note over WK, UI: [Phase D: Result Synchronization]
WK->>EV: SyncEventQueue.put("response", result)
EV->>EV: _pending_gui_tasks.append(response_obj)
loop Every Frame (~16.6ms)
UI->>EV: _process_pending_gui_tasks()
Note right of UI: Data Copy: Controller State -> UI History Buffer
UI->>UI: Update Rendering State (Markdown/Syntax Highlight)
end
```
### Key Technical Details
- **Thread Separation:** The UI remains responsive by delegating the blocking `ai_client.send()` call to a daemon thread.
- **Human-in-the-Loop:** `_confirm_and_run` uses a `threading.Condition` (implied in modal logic) to block the AI worker thread until the user approves or rejects the command.
- **Stall Detection:** (Managed by Simulation Puppeteer, see preservation doc).
- **Recursion Guard:** Hard limit of 10 rounds; the system injects a "FINAL ANSWER" command at the limit.
## 2. Technical Performance Audit
### 2.1 Threading & Synchronization
- **Context Switches:** The pipeline traverses four distinct execution contexts: Main Thread -> Event Thread -> Daemon Worker -> Subprocess.
- **Lock Contention:** `_pending_gui_tasks_lock` is acquired twice per AI response turn (once by background thread to append, once by UI thread to process).
- **Blocking Sites:** `ai_client.send` blocks the dedicated `WK` thread. `_confirm_and_run` blocks the `WK` thread using a `Condition` variable waiting on UI input.
### 2.2 Data Transformation Costs
- **Context Bloat:** `md_content` is a monolithic string. During synthesis, this string is often copied or chunked (`_chunk_text`), increasing memory pressure on the Python heap.
- **Serialization Overhead:** Every tool call involves: Python dict -> JSON String -> Subprocess Stdin -> (Tools) -> Subprocess Stdout -> JSON String -> Python dict.
### 2.3 Curation Targets (Intensive)
1. **Reduce Memory Copies:** The monolithic Markdown context should be handled as a stream or a shared buffer to avoid redundant copies between `aggregate` and `ai_client`.
2. **Deterministic Status Polling:** Replace string-based status polling (`ai_status`) with an enum-based state machine to reduce regex comparisons in the simulator and UI.
3. **Subprocess Pooling:** `shell_runner` spawns a new process for every script. For high-frequency tool use, a persistent PowerShell session could reduce overhead.