hopefully done refining

This commit is contained in:
2026-03-06 16:14:31 -05:00
parent 88e27ae414
commit 1294104f7f
20 changed files with 1736 additions and 734 deletions

View File

@@ -1,25 +1,107 @@
# Implementation Plan: Tool Usage Analytics (tool_usage_analytics_20260306)
## Phase 1: Data Collection
- [ ] Task: Initialize MMA Environment
- [ ] Task: Verify tool_log_callback
- WHERE: src/ai_client.py
- WHAT: Check existing logging
> **Reference:** [Spec](./spec.md) | [Architecture Guide](../../../docs/guide_architecture.md)
## Phase 2: Aggregation
- [ ] Task: Implement usage aggregation
- WHERE: src/gui_2.py or new module
- WHAT: Count tools, avg times, failures
- HOW: Process tool_log entries
- SAFETY: Efficient data structures
## Phase 1: Data Collection
Focus: Add tool execution tracking
- [ ] Task 1.1: Initialize MMA Environment
- Run `activate_skill mma-orchestrator` before starting
- [ ] Task 1.2: Add tool stats state
- WHERE: `src/app_controller.py` or `src/gui_2.py`
- WHAT: Add `_tool_stats: dict[str, dict]` state
- HOW:
```python
self._tool_stats: dict[str, dict] = {}
# Structure: {tool_name: {"count": 0, "total_time_ms": 0, "failures": 0}}
```
- CODE STYLE: 1-space indentation
- [ ] Task 1.3: Hook into tool execution
- WHERE: `src/ai_client.py` in tool execution path
- WHAT: Track tool name, time, success/failure
- HOW:
```python
start_time = time.time()
try:
result = mcp_client.dispatch(name, args)
success = True
except Exception:
success = False
finally:
elapsed_ms = (time.time() - start_time) * 1000
# Update stats via callback or direct update
```
- SAFETY: Don't impact tool execution performance
## Phase 2: Aggregation Logic
Focus: Calculate derived metrics
- [ ] Task 2.1: Implement stats update function
- WHERE: `src/app_controller.py`
- WHAT: Function to update tool stats
- HOW:
```python
def _update_tool_stats(self, tool_name: str, elapsed_ms: float, success: bool) -> None:
if tool_name not in self._tool_stats:
self._tool_stats[tool_name] = {"count": 0, "total_time_ms": 0.0, "failures": 0}
self._tool_stats[tool_name]["count"] += 1
self._tool_stats[tool_name]["total_time_ms"] += elapsed_ms
if not success:
self._tool_stats[tool_name]["failures"] += 1
```
- [ ] Task 2.2: Calculate average time and failure rate
- WHERE: `src/gui_2.py` in render function
- WHAT: Derive avg_time and failure_rate from stats
- HOW:
```python
for tool, stats in self._tool_stats.items():
count = stats["count"]
avg_time = stats["total_time_ms"] / count if count > 0 else 0
failure_rate = (stats["failures"] / count * 100) if count > 0 else 0
```
## Phase 3: Visualization
- [ ] Task: Render analytics
- WHERE: src/gui_2.py
- WHAT: Charts and tables
- HOW: imgui tables, plot_lines
- SAFETY: Handle empty data
Focus: Display analytics in GUI
## Phase 4: Verification
- [ ] Task: Test analytics
- [ ] Task: Conductor - Phase Verification
- [ ] Task 3.1: Add analytics panel
- WHERE: `src/gui_2.py` in MMA Dashboard or Operations
- WHAT: Table showing tool stats
- HOW:
```python
if imgui.collapsing_header("Tool Usage Analytics"):
if imgui.begin_table("tool_stats", 4):
imgui.table_setup_column("Tool")
imgui.table_setup_column("Count")
imgui.table_setup_column("Avg Time (ms)")
imgui.table_setup_column("Failure %")
imgui.table_headers_row()
for tool, stats in sorted(self._tool_stats.items(), key=lambda x: -x[1]["count"]):
imgui.table_next_row()
imgui.table_set_column_index(0)
imgui.text(tool)
# ... other columns
imgui.end_table()
```
## Phase 4: Reset on Session Clear
Focus: Clear stats on new session
- [ ] Task 4.1: Clear stats on session reset
- WHERE: `src/gui_2.py` or `src/app_controller.py` reset handler
- WHAT: Clear `_tool_stats` dict
- HOW: `self._tool_stats.clear()`
## Phase 5: Testing
Focus: Verify all functionality
- [ ] Task 5.1: Write unit tests
- WHERE: `tests/test_tool_analytics.py` (new file)
- WHAT: Test stats accumulation, avg calculation
- HOW: Mock tool execution, verify stats update
- [ ] Task 5.2: Conductor - Phase Verification
- Run: `uv run pytest tests/test_tool_analytics.py -v`
- Manual: Verify analytics panel displays in GUI