refinement of upcoming tracks
This commit is contained in:
@@ -1,32 +1,169 @@
|
||||
# Implementation Plan: Cost & Token Analytics Panel (cost_token_analytics_20260306)
|
||||
|
||||
## Phase 1: Panel Setup
|
||||
- [ ] Task: Initialize MMA Environment
|
||||
- [ ] Task: Create cost panel structure
|
||||
- WHERE: src/gui_2.py
|
||||
- WHAT: New panel for cost display
|
||||
- HOW: Add _render_cost_panel method
|
||||
- SAFETY: Non-blocking updates
|
||||
> **Reference:** [Spec](./spec.md) | [Architecture Guide](../../../docs/guide_architecture.md)
|
||||
|
||||
## Phase 2: Cost Calculations
|
||||
- [ ] Task: Integrate cost_tracker
|
||||
- WHERE: src/gui_2.py
|
||||
- WHAT: Use cost_tracker.estimate_cost
|
||||
- HOW: Call with model and token counts
|
||||
- SAFETY: Cache expensive calculations
|
||||
- [ ] Task: Track session totals
|
||||
- WHERE: src/gui_2.py or app_controller
|
||||
- WHAT: Accumulate cost over session
|
||||
- HOW: Maintain running total
|
||||
- SAFETY: Thread-safe updates
|
||||
## Phase 1: Foundation & Research
|
||||
Focus: Verify existing infrastructure
|
||||
|
||||
## Phase 3: UI Implementation
|
||||
- [ ] Task: Render cost breakdown
|
||||
- WHERE: src/gui_2.py
|
||||
- WHAT: Show per-model and per-tier costs
|
||||
- HOW: imgui tables
|
||||
- SAFETY: Handle zero/empty states
|
||||
- [ ] Task 1.1: Initialize MMA Environment
|
||||
- Run `activate_skill mma-orchestrator` before starting
|
||||
|
||||
## Phase 4: Verification
|
||||
- [ ] Task: Test cost calculations
|
||||
- [ ] Task: Conductor - Phase Verification
|
||||
- [ ] Task 1.2: Verify cost_tracker.py implementation
|
||||
- WHERE: `src/cost_tracker.py`
|
||||
- WHAT: Confirm `MODEL_PRICING` dict and `estimate_cost()` function
|
||||
- HOW: Use `manual-slop_py_get_definition` on `estimate_cost`
|
||||
- OUTPUT: Document exact MODEL_PRICING structure for reference
|
||||
|
||||
- [ ] Task 1.3: Verify tier_usage in ConductorEngine
|
||||
- WHERE: `src/multi_agent_conductor.py` lines ~50-60
|
||||
- WHAT: Confirm tier_usage dict structure and update mechanism
|
||||
- HOW: Use `manual-slop_py_get_code_outline` on ConductorEngine
|
||||
- SAFETY: Note thread that updates tier_usage
|
||||
|
||||
- [ ] Task 1.4: Review existing MMA dashboard
|
||||
- WHERE: `src/gui_2.py` `_render_mma_dashboard()` method
|
||||
- WHAT: Understand existing tier usage table pattern
|
||||
- HOW: Read method to identify extension points
|
||||
- OUTPUT: Note line numbers for table rendering
|
||||
|
||||
## Phase 2: State Management
|
||||
Focus: Add cost tracking state to app
|
||||
|
||||
- [ ] Task 2.1: Add session cost state
|
||||
- WHERE: `src/gui_2.py` or `src/app_controller.py` in `__init__`
|
||||
- WHAT: Add session-level cost tracking state
|
||||
- HOW:
|
||||
```python
|
||||
self._session_cost_total: float = 0.0
|
||||
self._session_cost_by_model: dict[str, float] = {}
|
||||
self._session_cost_by_tier: dict[str, float] = {
|
||||
"Tier 1": 0.0, "Tier 2": 0.0, "Tier 3": 0.0, "Tier 4": 0.0
|
||||
}
|
||||
```
|
||||
- CODE STYLE: 1-space indentation
|
||||
|
||||
- [ ] Task 2.2: Add cost update logic
|
||||
- WHERE: `src/gui_2.py` in MMA state update handler
|
||||
- WHAT: Calculate costs when tier_usage updates
|
||||
- HOW:
|
||||
```python
|
||||
def _update_costs_from_tier_usage(self, tier_usage: dict) -> None:
|
||||
for tier, usage in tier_usage.items():
|
||||
cost = cost_tracker.estimate_cost(
|
||||
self.current_model, usage["input"], usage["output"]
|
||||
)
|
||||
self._session_cost_by_tier[tier] = cost
|
||||
self._session_cost_total += cost
|
||||
```
|
||||
- SAFETY: Called from GUI thread via state update
|
||||
|
||||
- [ ] Task 2.3: Reset costs on session reset
|
||||
- WHERE: `src/gui_2.py` or `src/app_controller.py` reset handler
|
||||
- WHAT: Clear cost state when session resets
|
||||
- HOW: Set all cost values to 0.0 in reset function
|
||||
|
||||
## Phase 3: Panel Implementation
|
||||
Focus: Create the GUI panel
|
||||
|
||||
- [ ] Task 3.1: Create _render_cost_panel() method
|
||||
- WHERE: `src/gui_2.py` after other render methods
|
||||
- WHAT: New method to display cost information
|
||||
- HOW:
|
||||
```python
|
||||
def _render_cost_panel(self) -> None:
|
||||
if not imgui.collapsing_header("Cost Analytics"):
|
||||
return
|
||||
|
||||
# Total session cost
|
||||
imgui.text(f"Session Total: ${self._session_cost_total:.4f}")
|
||||
|
||||
# Per-tier breakdown
|
||||
if imgui.begin_table("tier_costs", 3):
|
||||
imgui.table_setup_column("Tier")
|
||||
imgui.table_setup_column("Tokens")
|
||||
imgui.table_setup_column("Cost")
|
||||
imgui.table_headers_row()
|
||||
for tier, cost in self._session_cost_by_tier.items():
|
||||
imgui.table_next_row()
|
||||
imgui.table_set_column_index(0)
|
||||
imgui.text(tier)
|
||||
imgui.table_set_column_index(2)
|
||||
imgui.text(f"${cost:.4f}")
|
||||
imgui.end_table()
|
||||
|
||||
# Per-model breakdown
|
||||
if self._session_cost_by_model:
|
||||
imgui.separator()
|
||||
imgui.text("By Model:")
|
||||
for model, cost in self._session_cost_by_model.items():
|
||||
imgui.bullet_text(f"{model}: ${cost:.4f}")
|
||||
```
|
||||
- CODE STYLE: 1-space indentation, no comments
|
||||
|
||||
- [ ] Task 3.2: Integrate panel into main GUI
|
||||
- WHERE: `src/gui_2.py` in `_gui_func()` or appropriate panel
|
||||
- WHAT: Call `_render_cost_panel()` in layout
|
||||
- HOW: Add near token budget panel or MMA dashboard
|
||||
- SAFETY: None
|
||||
|
||||
## Phase 4: Integration with MMA Dashboard
|
||||
Focus: Extend existing dashboard with cost column
|
||||
|
||||
- [ ] Task 4.1: Add cost column to tier usage table
|
||||
- WHERE: `src/gui_2.py` `_render_mma_dashboard()`
|
||||
- WHAT: Add "Est. Cost" column to existing tier usage table
|
||||
- HOW:
|
||||
- Change `imgui.table_setup_column()` from 3 to 4 columns
|
||||
- Add "Est. Cost" header
|
||||
- Calculate cost per tier using current model
|
||||
- Display with dollar formatting
|
||||
- SAFETY: Handle missing tier_usage gracefully
|
||||
|
||||
- [ ] Task 4.2: Display model name in table
|
||||
- WHERE: `src/gui_2.py` `_render_mma_dashboard()`
|
||||
- WHAT: Show which model was used for each tier
|
||||
- HOW: Add "Model" column with model name
|
||||
- SAFETY: May not know per-tier model - use current_model as fallback
|
||||
|
||||
## Phase 5: Testing
|
||||
Focus: Verify all functionality
|
||||
|
||||
- [ ] Task 5.1: Write unit tests for cost calculation
|
||||
- WHERE: `tests/test_cost_panel.py` (new file)
|
||||
- WHAT: Test cost accumulation logic
|
||||
- HOW: Mock tier_usage, verify costs calculated correctly
|
||||
- PATTERN: Follow `test_cost_tracker.py` as reference
|
||||
|
||||
- [ ] Task 5.2: Write integration test
|
||||
- WHERE: `tests/test_cost_panel.py`
|
||||
- WHAT: Test with live_gui, verify panel displays
|
||||
- HOW: Use `live_gui` fixture, trigger API call, check costs
|
||||
- ARTIFACTS: Write to `tests/artifacts/`
|
||||
|
||||
- [ ] Task 5.3: Conductor - Phase Verification
|
||||
- Run: `uv run pytest tests/test_cost_panel.py tests/test_cost_tracker.py -v`
|
||||
- Manual: Verify panel displays in GUI
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Thread Safety
|
||||
- tier_usage is updated on asyncio worker thread
|
||||
- GUI reads via `_process_pending_gui_tasks` - already synchronized
|
||||
- No additional locking needed
|
||||
|
||||
### Cost Calculation Strategy
|
||||
- Use current model for all tiers (simplification)
|
||||
- Future: Track model per tier if needed
|
||||
- Unknown models return 0.0 cost (safe default)
|
||||
|
||||
### Files Modified
|
||||
- `src/gui_2.py`: Add cost state, render methods
|
||||
- `src/app_controller.py`: Possibly add cost state (if using controller)
|
||||
- `tests/test_cost_panel.py`: New test file
|
||||
|
||||
### Code Style Checklist
|
||||
- [ ] 1-space indentation throughout
|
||||
- [ ] CRLF line endings on Windows
|
||||
- [ ] No comments unless requested
|
||||
- [ ] Type hints on new state variables
|
||||
- [ ] Use existing `vec4` colors for consistency
|
||||
|
||||
Reference in New Issue
Block a user