manual_slop/conductor/archive/tool_usage_analytics_20260306/plan.md

Implementation Plan: Tool Usage Analytics (tool_usage_analytics_20260306)

Reference: Spec | Architecture Guide

Phase 1: Data Collection

Focus: Add tool execution tracking

• Task 1.1: Initialize MMA Environment

  • Run activate_skill mma-orchestrator before starting

• Task 1.2: Add tool stats state

  • WHERE: src/app_controller.py or src/gui_2.py
  • WHAT: Add _tool_stats: dict[str, dict] state
  • HOW:

    self._tool_stats: dict[str, dict] = {}
    # Structure: {tool_name: {"count": 0, "total_time_ms": 0, "failures": 0}}
    

  • CODE STYLE: 1-space indentation

• Task 1.3: Hook into tool execution

  • WHERE: src/ai_client.py in tool execution path
  • WHAT: Track tool name, time, success/failure
  • HOW:

    start_time = time.time()
    try:
     result = mcp_client.dispatch(name, args)
     success = True
    except Exception:
     success = False
    finally:
     elapsed_ms = (time.time() - start_time) * 1000
     # Update stats via callback or direct update
    

  • SAFETY: Don't impact tool execution performance

Phase 2: Aggregation Logic

Focus: Calculate derived metrics

• Task 2.1: Implement stats update function

  • WHERE: src/app_controller.py
  • WHAT: Function to update tool stats
  • HOW:

    def _update_tool_stats(self, tool_name: str, elapsed_ms: float, success: bool) -> None:
     if tool_name not in self._tool_stats:
      self._tool_stats[tool_name] = {"count": 0, "total_time_ms": 0.0, "failures": 0}
     self._tool_stats[tool_name]["count"] += 1
     self._tool_stats[tool_name]["total_time_ms"] += elapsed_ms
     if not success:
      self._tool_stats[tool_name]["failures"] += 1
    

• Task 2.2: Calculate average time and failure rate

  • WHERE: src/gui_2.py in render function
  • WHAT: Derive avg_time and failure_rate from stats
  • HOW:

    for tool, stats in self._tool_stats.items():
     count = stats["count"]
     avg_time = stats["total_time_ms"] / count if count > 0 else 0
     failure_rate = (stats["failures"] / count * 100) if count > 0 else 0
    

Phase 3: Visualization

Focus: Display analytics in GUI

• Task 3.1: Add analytics panel
  • WHERE: src/gui_2.py in MMA Dashboard or Operations
  • WHAT: Table showing tool stats
  • HOW:

    if imgui.collapsing_header("Tool Usage Analytics"):
     if imgui.begin_table("tool_stats", 4):
      imgui.table_setup_column("Tool")
      imgui.table_setup_column("Count")
      imgui.table_setup_column("Avg Time (ms)")
      imgui.table_setup_column("Failure %")
      imgui.table_headers_row()
      for tool, stats in sorted(self._tool_stats.items(), key=lambda x: -x[1]["count"]):
       imgui.table_next_row()
       imgui.table_set_column_index(0)
       imgui.text(tool)
       # ... other columns
      imgui.end_table()
    

Phase 4: Reset on Session Clear

Focus: Clear stats on new session

• Task 4.1: Clear stats on session reset
  • WHERE: src/gui_2.py or src/app_controller.py reset handler
  • WHAT: Clear _tool_stats dict
  • HOW: self._tool_stats.clear()

Phase 5: Testing

Focus: Verify all functionality

• Task 5.1: Write unit tests

  • WHERE: tests/test_tool_analytics.py (new file)
  • WHAT: Test stats accumulation, avg calculation
  • HOW: Mock tool execution, verify stats update

• Task 5.2: Conductor - Phase Verification

  • Run: uv run pytest tests/test_tool_analytics.py -v
  • Manual: Verify analytics panel displays in GUI