conductor(spec+plan): add Phase 6e to follow-up - Tier 2 authoritative Phase 3 cost deduction

The follow-up track now includes Phase 6e: Tier 2 produces the authoritative Phase 3 cost analysis as part of the follow-up work. Tier 2 is in src/ai_client.py doing Phase 6b/6d anyway; they have full context to produce the refined cost hypothesis that Tier 1's draft at PHASE3_HYPOTHETICAL_PROMOTION.md could not (Tier 1 worked without the 6b/6d ground-truth context). Tier 1's draft STAYS as the hypothesis doc. Tier 2's PHASE3_TIER2_ANALYSIS.md is the refined version (per-sender cost summary + hidden call sites table + recommendations for the future Phase 3 track + cross-reference to Tier 1 explicit). Phase 6e tasks (5 total, ~2 commits): - t6e_1: Profile the 6 senders (codepath catalog + hidden cross-refs) - t6e_2: Qualitative cost estimation per sender - t6e_3: Identify hot iteration sites needing 'with h.lock:' pattern - t6e_4: Author PHASE3_TIER2_ANALYSIS.md - t6e_5: Phase 6e checkpoint commit + git note Total estimated commits: 16 -> 18 (still within Tier 2 1-4 hour budget). Files updated: - conductor/tracks/phase2_4_5_call_site_completion_20260621/spec.md (+50 lines) - conductor/tracks/phase2_4_5_call_site_completion_20260621/plan.md (+146 lines) - conductor/tracks/phase2_4_5_call_site_completion_20260621/metadata.json (+13 lines) - conductor/tracks/phase2_4_5_call_site_completion_20260621/state.toml (+9 lines) - conductor/tracks.md (track 27 entry expanded with Phase 6e details)
2026-06-21 18:55:54 -04:00
parent 1a739ecef5
commit 6275c860bf
5 changed files with 213 additions and 7 deletions
@@ -24,10 +24,10 @@
 },
 "blocked_by": [],
 "blocks": ["code_path_audit_20260607"],
- "estimated_phases": 3,
+ "estimated_phases": 4,
 "spec": "spec.md",
 "plan": "plan.md",
- "priority_order": "A (Phase 6a broadcast fix) > A (Phase 6b OpenAICompatibleRequest) > B (Phase 6d NormalizedResponse) > C (regression protocol)",
+ "priority_order": "A (Phase 6a broadcast fix) > A (Phase 6b OpenAICompatibleRequest) > B (Phase 6d NormalizedResponse) > A (Phase 6e Tier 2 cost deduction)",
 "parent_track": {
 "id": "any_type_componentization_20260621",
 "spec": "conductor/tracks/any_type_componentization_20260621/spec.md",
@@ -53,9 +53,16 @@
 "name": "Update NormalizedResponse construction",
 "scope": "Same 3 senders: usage_input_tokens/etc -> usage=UsageStats(...)",
 "estimated_commits": 4
+ },
+ "phase_6e": {
+ "name": "Phase 3 Hypothetical Cost Deduction (Tier 2 authoritative deliverable)",
+ "scope": "Tier 2 produces docs/reports/PHASE3_TIER2_ANALYSIS.md while doing 6b/6d work in src/ai_client.py; profiles all 6 senders + discovers hidden cross-references + provides refined cost estimates + recommendations for the future Phase 3 track. Supersedes Tier 1's draft at docs/reports/PHASE3_HYPOTHETICAL_PROMOTION.md (which stays as the hypothesis doc).",
+ "estimated_commits": 2,
+ "new_doc_file": "docs/reports/PHASE3_TIER2_ANALYSIS.md",
+ "rationale": "Tier 2 is in src/ai_client.py anyway doing the 6b/6d migration work; they have full context to produce the authoritative Phase 3 cost analysis. The future Phase 3 track + the code_path_audit both need this data."
 }
 },
- "total_estimated_commits": 16,
+ "total_estimated_commits": 18,
 "deferred_work": {
 "phase_3_provider_state": {
 "deferred_to": "separate track post code_path_audit_20260607",
@@ -417,9 +417,151 @@ git notes add -m "Phase 6d checkpoint: 3 senders use UsageStats; all 11 tiers pa

 ---

-## Verify + Archive
+## Phase 6e: Phase 3 Hypothetical Cost Deduction (Tier 2 authoritative deliverable)

-### Task V.1: Run all audits
+Focus: While doing Phase 6b/6d work in `src/ai_client.py`, Tier 2 is reading and modifying the 3 senders anyway. They have the context to produce the authoritative Phase 3 cost analysis (deferred from `any_type_componentization_20260621`). This phase is the **Tier 2 deliverable** that supersedes Tier 1's hypothesis at `docs/reports/PHASE3_HYPOTHETICAL_PROMOTION.md`.
+
+**Tier 1's hypothesis** stays as the placeholder; Tier 2's `PHASE3_TIER2_ANALYSIS.md` is the refined version with in-context, post-Phase-6b/6d-grounded estimates.
+
+### Task 6e.1: Profile the 6 senders (during Phase 6b/6d work)
+
+**No new code; pure analysis.** While doing Tasks 6b.3-6b.5 (migrating `_send_grok` / `_send_minimax` / `_send_llama`) and Tasks 6d.2-6d.4 (updating their `NormalizedResponse`), Tier 2 reads the surrounding code and documents:
+
+For each of the 6 senders, capture in working notes:
+- All `_anthropic_history` / `_anthropic_history_lock` references (categorized: append, len/iteration, lock-acquire, with-lock-block, global-decl, helper-call)
+- Helper function call sites (`_repair_<provider>_history`, `_trim_<provider>_history`, `_strip_cache_controls`, `_add_history_cache_breakpoint`)
+- **Hidden call sites** Tier 2 discovers that Tier 1's grep missed (e.g., `_repair_anthropic_history` is called from `_send_anthropic` AND from `cleanup()` — that's a hidden cross-reference Tier 1's grep didn't see)
+
+For the 3 senders NOT touched by 6b/6d (`_send_anthropic`, `_send_deepseek`, `_send_qwen`):
+- Same profiling
+- Tier 2 reads these while doing the 6b/6d work for context (they share helper patterns)
+
+### Task 6e.2: Qualitative cost estimation per sender
+
+For each of the 6 senders, for each codepath category:
+
+| Category | Current (dict globals) | Proposed (ProviderHistory dataclass) | Per-call delta |
+|---|---|---|---|
+| `_<provider>_history.append(m)` | dict.append (~100ns) | dataclass method + lock acquire (~300ns) | **+200ns per call** |
+| `len(_<provider>_history)` | direct attribute (~50ns) | `.messages` attribute (~100ns) | **+50ns per call** |
+| `for m in _<provider>_history:` | direct iteration | `h.get_all()` (list copy) OR `with h.lock:` | **+5-10μs per call** (if `get_all()`) |
+| `with _<provider>_history_lock:` | direct lock | `with h.lock:` | **~0** (same lock) |
+| `_global _<provider>_history` (in cleanup) | N/A (declaration) | N/A (removed) | **N/A** |
+
+For each sender, sum the per-turn overhead:
+- `_send_anthropic` (25 sites; per-turn): estimate total overhead per LLM turn
+- `_send_deepseek` (20 sites; per-turn): estimate
+- ... etc for all 6
+
+### Task 6e.3: Identify the hot iteration sites that need `with h.lock:` pattern
+
+**Critical:** the `_strip_cache_controls(_anthropic_history)` and `_estimate_prompt_tokens(...)` callsites iterate the list per LLM turn. If the migration uses `h.get_all()`, they pay a list-copy cost (~5-10μs per call).
+
+Document each iteration site with:
+- File:line
+- Call frequency per LLM turn
+- Recommended pattern: `with h.lock: msg_list = h.messages` vs `h.get_all()`
+- Justification
+
+### Task 6e.4: Author `docs/reports/PHASE3_TIER2_ANALYSIS.md`
+
+**Files:**
+- Create: `docs/reports/PHASE3_TIER2_ANALYSIS.md`
+
+Structure (Tier 2 produces this from the analysis in 6e.1-6e.3):
+
+```markdown
+# Phase 3 Hypothetical Cost Analysis (Tier 2 authoritative version)
+
+**Author:** Tier 2 Tech Lead (autonomous sandbox)
+**Date:** 2026-06-21
+**Context:** Produced during `phase2_4_5_call_site_completion_20260621` Phase 6e (after Phase 6b/6d work in `src/ai_client.py`).
+**Supersedes:** Tier 1's hypothesis at `docs/reports/PHASE3_HYPOTHETICAL_PROMOTION.md` (kept as the hypothesis doc; this is the refined version).
+
+---
+
+## 1. Methodology
+
+Tier 2 profiled the 6 senders in `src/ai_client.py` (`_send_anthropic`, `_send_deepseek`, `_send_minimax`, `_send_grok`, `_send_qwen`, `_send_llama`) while doing the Phase 6b/6d migration work. This analysis is grounded in actual code reading + Phase 6b/6d context.
+
+## 2. Per-Sender Codepath Catalog
+
+### 2.1 `_send_anthropic` (25 sites)
+[Fill in from 6e.1 working notes]
+- Direct sites: 22 `_anthropic_history` refs; 2 `_anthropic_history_lock` refs; 1 `global` decl
+- Helper sites: `_strip_cache_controls`, `_repair_anthropic_history`, `_add_history_cache_breakpoint`, `_trim_anthropic_history`
+- Hidden cross-references (Tier 2 found): [list any]
+
+### 2.2-2.6 [other senders; same structure]
+
+## 3. Qualitative Cost Estimation
+
+### 3.1 Per-call cost categories
+[Fill in from 6e.2 table]
+
+### 3.2 Per-sender per-turn overhead
+[Fill in from 6e.2 sum]
+
+### 3.3 Hot iteration sites (the `with h.lock:` pattern)
+[Fill in from 6e.3]
+
+## 4. Comparison vs Tier 1's Hypothesis
+
+| Sender | Tier 1 hypothesis (μs/turn) | Tier 2 refined (μs/turn) | Delta |
+|---|---|---|---|
+| anthropic | +8-15 | [Tier 2 actual] | [reason] |
+| deepseek | +3-7 | [Tier 2 actual] | [reason] |
+| minimax | +3-7 | [Tier 2 actual] | [reason] |
+| grok | +2-5 | [Tier 2 actual] | [reason] |
+| qwen | +2-5 | [Tier 2 actual] | [reason] |
+| llama | +4-8 | [Tier 2 actual] | [reason] |
+| **Total** | **~+1.1-2.4ms/session** | [Tier 2 actual] | [reason] |
+
+## 5. Recommendations for Future Phase 3 Track
+
+1. **Anthropic first** (highest ROI; per-turn; cache controls)
+2. **Use `with h.lock: msg_list = h.messages` pattern for hot iteration sites** (avoids `get_all()` list-copy cost)
+3. **Simpler providers (qwen, grok) can use `get_all()`** since iteration is less frequent
+4. **Lock semantics unchanged** — `ProviderHistory.lock` is per-instance; no cross-provider contention
+5. **Hidden cross-references** discovered during this analysis [list] should be the first sites to migrate
+
+## 6. Open Questions
+
+[Fill in any unresolved questions; defer to the audit for runtime quantification]
+
+## 7. See Also
+
+- `docs/reports/PHASE3_HYPOTHETICAL_PROMOTION.md` — Tier 1's hypothesis (the "what we thought before Tier 2 looked")
+- `conductor/tracks/phase2_4_5_call_site_completion_20260621/spec.md` — Phase 6e directives
+- `conductor/tracks/code_path_audit_20260607/spec.md` — the audit that quantifies these estimates
+- `docs/handoffs/PROMPT_FOR_TIER_1.md` — Tier 1 brief
+```
+
+### Task 6e.5: Phase 6e checkpoint
+
+- [ ] **Step 1: Commit the analysis**
+
+```bash
+git add docs/reports/PHASE3_TIER2_ANALYSIS.md
+git commit -m "docs(analysis): PHASE3_TIER2_ANALYSIS - authoritative Phase 3 cost hypothesis
+
+Tier 2 produced this analysis during phase2_4_5_call_site_completion_20260621
+Phase 6e. Supersedes Tier 1's draft at PHASE3_HYPOTHETICAL_PROMOTION.md (kept
+as the hypothesis doc; this is the refined version with in-context data
+from Phase 6b/6d work in src/ai_client.py).
+
+Covers all 6 senders (anthropic, deepseek, minimax, grok, qwen, llama)
+with per-site cost estimates + hidden cross-references + recommendations
+for the future Phase 3 track. The audit (code_path_audit_20260607)
+quantifies these estimates after merge."
+git notes add -m "Phase 6e checkpoint: Tier 2 authoritative Phase 3 cost analysis committed" HEAD
+```
+
+Update `state.toml` to mark phase_6e status="completed" + checkpointsha.
+
+---
+
+## Verify + Archive

 ```bash
 uv run python scripts/audit_weak_types.py --strict
@@ -135,6 +135,56 @@ The audit's per-action profiling assumes no TypeError spam on the GUI thread. Th
 2. Add import: `from src.openai_schemas import ChatMessage, NormalizedResponse, OpenAICompatibleRequest, UsageStats`
 3. Run provider tests + integration tests

+### Phase 6e: Phase 3 Hypothetical Cost Deduction (Tier 2 deliverable)
+
+**Goal:** Produce the authoritative Phase 3 hypothetical cost analysis as a Tier 2 deliverable. The deferred Phase 3 (`provider_state.ProviderHistory` call-site migration in `src/ai_client.py`) needs runtime cost data BEFORE the migration; Tier 2 produces this analysis as part of the follow-up track because they're already in `src/ai_client.py` doing the Phase 6b/6d work and have full context.
+
+**Tier 1's draft** at `docs/reports/PHASE3_HYPOTHETICAL_PROMOTION.md` stays as the hypothesis document (Tier 1's qualitative estimates). **Tier 2's authoritative analysis** is a separate document at `docs/reports/PHASE3_TIER2_ANALYSIS.md` that supersedes the hypothesis with in-context, post-Phase-6b/6d-grounded estimates.
+
+**Files:**
+- Create: `docs/reports/PHASE3_TIER2_ANALYSIS.md`
+- Modify: `conductor/tracks/phase2_4_5_call_site_completion_20260621/spec.md` (this section)
+
+**Approach:**
+1. **For each of the 6 senders** (Tier 2 reads while doing 6b/6d work; cost analysis happens during 6b/6d + a final consolidation commit at end of 6e):
+   - `_send_anthropic` (25 sites; Hot per-turn; uses cache-control helpers)
+   - `_send_deepseek` (20 sites; Hot per-turn; has `_repair_deepseek_history` helper)
+   - `_send_minimax` (21 sites; Hot per-turn; has `_repair_minimax_history` + `_trim_minimax_history` helpers)
+   - `_send_grok` (13 sites; Hot per-turn; **being touched in 6b/6d**)
+   - `_send_qwen` (12 sites; Hot per-turn; simpler pattern)
+   - `_send_llama` (21 sites; Hot per-turn; highest lock count; **being touched in 6b/6d**)
+2. **For each sender, document:**
+   - Direct `_anthropic_history` / `_anthropic_history_lock` sites (categorized as: append, len/iteration, lock-acquire, with-lock-block, global-decl, helper-call)
+   - Helper function call sites (`_repair_<provider>_history`, `_trim_<provider>_history`, `_strip_cache_controls`, `_add_history_cache_breakpoint`)
+   - Hidden call sites discovered while doing the 6b/6d work (e.g., `_repair_anthropic_history` is called from `_send_anthropic` AND from `cleanup()` — that's a hidden cross-reference)
+3. **For each category, qualitatively estimate:**
+   - Per-call cost delta: `dict append` (current) vs `dataclass.append` (proposed)
+   - Lock acquire cost: `threading.Lock` (current) vs `ProviderHistory.lock` (proposed) — should be ~identical but document any surprises
+   - `get_all()` list-copy cost: bounded by history length (~10-50 messages); estimate ~5μs per copy
+   - **Critical:** the `_strip_cache_controls(_anthropic_history)` and `_estimate_prompt_tokens(...)` callsites iterate the list; if `get_all()` is used, they copy the list per call. Recommendation: use `with h.lock: msg_list = h.messages` pattern instead of `h.get_all()` for hot iteration sites
+4. **Author `docs/reports/PHASE3_TIER2_ANALYSIS.md`:**
+   - Per-sender cost summary table (compare Tier 1's hypothesis vs Tier 2's refined estimate)
+   - Hidden call sites table (call sites Tier 2 discovered that Tier 1's grep missed)
+   - Recommendations for the future Phase 3 track:
+     - Use `with h.lock:` blocks for hot iteration sites
+     - The Anthropic cache-control helpers are the highest-value target (~25 sites, per-turn)
+     - The simpler providers (qwen, grok) can use `get_all()` since iteration is less frequent
+   - Cross-references Tier 1's hypothesis explicitly: "Tier 1's draft is the hypothesis; this is the refined version after Phase 6b/6d context."
+   - Roll-up: total estimated cost per session (~50 turns) for the Phase 3 migration; comparison vs Tier 1's hypothesis
+
+**Why this matters:**
+- The future Phase 3 track needs this data to scope its phases correctly (e.g., "do the Anthropic helpers first because they're hot; defer the simpler providers to Phase 2")
+- The audit will quantify these estimates after the merge; this is the pre-audit hypothesis refinement
+- Tier 2 is the right entity to produce this because they have the actual code context after Phase 6b/6d
+
+**Verification:**
+- `docs/reports/PHASE3_TIER2_ANALYSIS.md` committed
+- All 6 senders profiled
+- Total estimated cost per session documented
+- Hidden call sites table documented
+- Recommendations for future Phase 3 track documented
+- Cross-reference to Tier 1's hypothesis explicit
+
 ## 5. Configuration

 No new dependencies. No new config files.
@@ -10,7 +10,6 @@ last_updated = "2026-06-21"

 [blocked_by]
 # No blockers; this track unblocks the audit
-[]

 [blocks]
 code_path_audit_20260607 = "blocked_until_merge"
@@ -19,6 +18,7 @@ code_path_audit_20260607 = "blocked_until_merge"
 phase_6a = { status = "pending", checkpointsha = "", name = "Fix HookServer.broadcast() callers" }
 phase_6b = { status = "pending", checkpointsha = "", name = "Complete OpenAICompatibleRequest migration" }
 phase_6d = { status = "pending", checkpointsha = "", name = "Update NormalizedResponse construction" }
+phase_6e = { status = "pending", checkpointsha = "", name = "Phase 3 Hypothetical Cost Deduction (Tier 2 authoritative deliverable)" }

 [tasks]
 # Phase 6a: Fix HookServer.broadcast() callers
@@ -51,12 +51,19 @@ tv_2 = { status = "pending", commit_sha = "", description = "Run generate_type_r
 tv_3 = { status = "pending", commit_sha = "", description = "Write docs/reports/TRACK_COMPLETION_phase2_4_5_call_site_completion_20260621.md" }
 tv_4 = { status = "pending", commit_sha = "", description = "git mv to conductor/tracks/archive/" }
 tv_5 = { status = "pending", commit_sha = "", description = "Update conductor/tracks.md" }
+# Phase 6e: Phase 3 Hypothetical Cost Deduction
+t6e_1 = { status = "pending", commit_sha = "", description = "Profile the 6 senders (during 6b/6d work): codepath catalog + helper call sites + hidden cross-references Tier 1's grep missed" }
+t6e_2 = { status = "pending", commit_sha = "", description = "Qualitative cost estimation per sender (per-call categories: append / len / iteration / lock-acquire / with-lock / global-decl / helper-call)" }
+t6e_3 = { status = "pending", commit_sha = "", description = "Identify hot iteration sites that need 'with h.lock: msg_list = h.messages' pattern vs h.get_all() (avoids list-copy cost)" }
+t6e_4 = { status = "pending", commit_sha = "", description = "Author docs/reports/PHASE3_TIER2_ANALYSIS.md (per-sender cost summary + hidden call sites table + recommendations + comparison vs Tier 1 hypothesis + cross-reference to Tier 1 draft)" }
+t6e_5 = { status = "pending", commit_sha = "", description = "Phase 6e checkpoint commit + git note" }

 [verification]
 phase_6a_broadcast_fixed = false
 phase_6a_regression_test_passes = false
 phase_6b_openai_compat_migrated = false
 phase_6d_normalized_response_migrated = false
+phase_6e_tier2_analysis_committed = false
 full_11_tier_regression_passes = false
 audit_weak_types_strict_passes = false
 audit_dataclass_coverage_strict_passes = false