From 3669ce590cb153b21cb2ababd40ac7eee350e9e0 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Sun, 21 Jun 2026 15:53:28 -0400 Subject: [PATCH] conductor(plan): author plan.md for any_type_componentization_20260621 The spec.md was approved 2026-06-21 without a plan.md (the metadata.json noted 'plan.md (to be authored by writing-plans skill after spec approval)'). This plan mirrors the state.toml's per-task ledger and specifies the TDD protocol, tier-3 delegation conventions, hard bans, failcount contract, and per-phase verification commands. Plan structure: 7 phases, 61 tasks, ~50 atomic commits per the spec. Reads all 13 conductor/code_styleguides/*.md per the agent mandate. --- .../plan.md | 1688 ++--------------- 1 file changed, 131 insertions(+), 1557 deletions(-) diff --git a/conductor/tracks/any_type_componentization_20260621/plan.md b/conductor/tracks/any_type_componentization_20260621/plan.md index 8c67566d..3265ae36 100644 --- a/conductor/tracks/any_type_componentization_20260621/plan.md +++ b/conductor/tracks/any_type_componentization_20260621/plan.md @@ -1,1557 +1,131 @@ -# Any-Type Componentization Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Promote 5 fat-struct candidates (89 sites) from `dict[str, Any]` / `list[dict[...]]` / module-globals to `dataclass(frozen=True)` definitions following the `src/vendor_capabilities.py` pattern. Add a `--strict` CI gate (`scripts/audit_dataclass_coverage.py`). Add styleguide §12 "When to Promote TypeAlias to dataclass". - -**Architecture:** 7 phases (1 scaffolding + 5 per-candidate + 1 verify/archive). Each phase is independent; cross-phase coupling is explicitly deferred. New module per P1/P2 candidate (`src/mcp_tool_specs.py`, `src/openai_schemas.py`, `src/provider_state.py`); P2/P3 candidates use inline dataclasses in existing files. New dataclasses expose `from_dict(cls, data: Metadata) -> Result[Self, ErrorInfo]` and `to_dict(self) -> Metadata` per the data-oriented convention. - -**Tech Stack:** Python 3.11+ stdlib (`dataclasses`, `threading`, `typing`). No new dependencies. New module additions follow `src/vendor_capabilities.py:64-76` template. - -**Reference Files:** -- `docs/reports/ANY_TYPE_AUDIT_20260621.md` — input audit (the "why") -- `conductor/tracks/any_type_componentization_20260621/spec.md` — the design (the "what") -- `src/vendor_capabilities.py:64-76` — the reference pattern -- `conductor/code_styleguides/type_aliases.md` — to extend with §12 -- `conductor/code_styleguides/error_handling.md` — `Result[T]` convention for `from_dict()` - -**Code Style:** 1-space indentation, CRLF line endings, no comments in source code, type hints mandatory (per `conductor/workflow.md` Code Style section). - ---- - -## File Structure - -``` -src/ - type_aliases.py # MODIFIED (Phase 0): + JsonPrimitive + JsonValue - vendor_capabilities.py # UNCHANGED (reference) - mcp_tool_specs.py # NEW (Phase 1): ToolParameter + ToolSpec + registry - openai_schemas.py # NEW (Phase 2): ToolCall + ChatMessage + UsageStats - provider_state.py # NEW (Phase 3): ProviderHistory + _PROVIDER_HISTORIES - mcp_client.py # MODIFIED (Phase 1): 8 sites - openai_compatible.py # MODIFIED (Phase 2): 17 sites - ai_client.py # MODIFIED (Phase 2+3): 41 sites - log_registry.py # MODIFIED (Phase 4): 7 sites + inline dataclasses - session_logger.py # MODIFIED (Phase 4): Session consumers - log_pruner.py # MODIFIED (Phase 4): Session consumers - gui_2.py # MODIFIED (Phase 4): Log Management panel - api_hooks.py # MODIFIED (Phase 5): WebSocketMessage + 16 sites - -scripts/ - audit_dataclass_coverage.py # NEW (Phase 0) - audit_dataclass_coverage.baseline.json # NEW (Phase 6) - -conductor/ - code_styleguides/ - type_aliases.md # MODIFIED (Phase 0): §12 - -tests/ - test_audit_dataclass_coverage.py # NEW (Phase 0) - test_mcp_tool_specs.py # NEW (Phase 1) - test_openai_schemas.py # NEW (Phase 2) - test_provider_state.py # NEW (Phase 3) - test_log_registry.py # MODIFIED (Phase 4): extend - test_api_hooks.py # MODIFIED (Phase 5): extend - -docs/ - type_registry/ # AUTO-GENERATED (Phase 6) - reports/TRACK_COMPLETION_*.md # NEW (Phase 6) -``` - ---- - -## Phase 0: Shared Scaffolding (3 tasks, ~3 commits) - -Focus: JsonValue TypeAlias + dataclass-coverage audit + styleguide §12. Additive only; no behavior change. - -### Task 0.1: Add JsonValue TypeAlias - -**Files:** -- Modify: `src/type_aliases.py` -- Test: `tests/test_type_aliases.py` (extend existing; 2 new tests) - -- [ ] **Step 1: Write failing tests** (extend `tests/test_type_aliases.py`) - -```python -def test_json_primitive_alias_resolves_to_union() -> None: - from src.type_aliases import JsonPrimitive - import typing - # Should resolve to a union of basic types - assert typing.get_origin(JsonPrimitive) is typing.Union - -def test_json_value_alias_is_recursive() -> None: - from src.type_aliases import JsonValue - # JsonValue must accept list[JsonValue] and dict[str, JsonValue] - # Use get_type_hints with include_extras=False to verify - import typing - hints = typing.get_type_hints(JsonValue, include_extras=False) - # The alias should contain 'list' and 'dict' in its string representation - assert "list" in str(JsonValue) - assert "dict" in str(JsonValue) -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `uv run pytest tests/test_type_aliases.py::test_json_primitive_alias_resolves_to_union tests/test_type_aliases.py::test_json_value_alias_is_recursive -v` -Expected: FAIL (NameError: cannot import name 'JsonPrimitive') - -- [ ] **Step 3: Add TypeAliases to `src/type_aliases.py`** (1-space indent) - -Append after the existing aliases (after line 19): -```python -JsonPrimitive: TypeAlias = str | int | float | bool | None - -JsonValue: TypeAlias = JsonPrimitive | list["JsonValue"] | dict[str, "JsonValue"] -``` - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `uv run pytest tests/test_type_aliases.py -v` -Expected: PASS (all 10 original + 2 new = 12 tests) - -- [ ] **Step 5: Commit** - -```bash -git add src/type_aliases.py tests/test_type_aliases.py -git commit -m "feat(type_aliases): add JsonPrimitive + JsonValue recursive TypeAliases" -``` - -### Task 0.2: Create `scripts/audit_dataclass_coverage.py` - -**Files:** -- Create: `scripts/audit_dataclass_coverage.py` -- Test: `tests/test_audit_dataclass_coverage.py` - -- [ ] **Step 1: Write failing tests** - -Create `tests/test_audit_dataclass_coverage.py`: -```python -"""Tests for scripts/audit_dataclass_coverage.py.""" -import json -import subprocess -import sys -from pathlib import Path - -REPO_ROOT = Path(__file__).parent.parent - -def test_audit_script_runs_without_error() -> None: - """Audit script should run in informational mode and exit 0.""" - result = subprocess.run( - [sys.executable, "scripts/audit_dataclass_coverage.py"], - capture_output=True, text=True, cwd=REPO_ROOT, - ) - assert result.returncode == 0, f"stdout: {result.stdout}\nstderr: {result.stderr}" - assert "Anonymous Any audit" in result.stdout - -def test_audit_json_mode_produces_valid_json() -> None: - """--json mode should print machine-readable report.""" - result = subprocess.run( - [sys.executable, "scripts/audit_dataclass_coverage.py", "--json"], - capture_output=True, text=True, cwd=REPO_ROOT, - ) - assert result.returncode == 0 - report = json.loads(result.stdout) - assert "total_any" in report - assert "files_with_findings" in report - -def test_audit_strict_mode_exits_nonzero_when_regression() -> None: - """--strict mode should exit 1 when current count > baseline.""" - # Create a temporary baseline with a very low count (1) - import tempfile - with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: - json.dump({"total_any": 1, "files_with_findings": 1}, f) - baseline_path = f.name - try: - result = subprocess.run( - [sys.executable, "scripts/audit_dataclass_coverage.py", "--strict", - "--baseline", baseline_path], - capture_output=True, text=True, cwd=REPO_ROOT, - ) - assert result.returncode == 1, f"expected exit 1, got {result.returncode}" - finally: - Path(baseline_path).unlink() - -def test_audit_human_readable_output_includes_summary() -> None: - """Informational mode output should include summary stats.""" - result = subprocess.run( - [sys.executable, "scripts/audit_dataclass_coverage.py"], - capture_output=True, text=True, cwd=REPO_ROOT, - ) - assert "Total Any findings" in result.stdout - assert "Files with findings" in result.stdout -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `uv run pytest tests/test_audit_dataclass_coverage.py -v` -Expected: FAIL (FileNotFoundError: scripts/audit_dataclass_coverage.py) - -- [ ] **Step 3: Implement `scripts/audit_dataclass_coverage.py`** - -Mirror the structure of `scripts/audit_weak_types.py` (read it first for the exact regex patterns and Finding dataclass shape). Key requirements: - -```python -"""Audit anonymous Any type usage across src/. - -Counts `Any` annotations in src/**/*.py that are NOT: -- TypeAlias targets (intentional) -- `@dataclass(frozen=True)` fields (those are structural) -- Pattern 3/4/5 per docs/reports/ANY_TYPE_AUDIT_20260621.md §2.2 (SDK holders, __getattr__, generic serialization) - -Modes: - - default: informational report (exit 0) - - --json: machine-readable (exit 0) - - --strict: CI gate (exit 1 if total_any > baseline.total_any) - - --baseline : baseline JSON (default: scripts/audit_dataclass_coverage.baseline.json) -""" -from __future__ import annotations -import ast -import json -import re -import sys -from dataclasses import dataclass, asdict -from pathlib import Path - -REPO_ROOT = Path(__file__).parent.parent -SRC_DIR = REPO_ROOT / "src" -DEFAULT_BASELINE = REPO_ROOT / "scripts" / "audit_dataclass_coverage.baseline.json" - -@dataclass -class Finding: - file: str - line: int - category: str - snippet: str - -@dataclass -class Report: - total_any: int - files_with_findings: int - by_category: dict[str, int] - by_file: dict[str, int] - -def _is_pattern_3_4_5(node: ast.AST, file_text: str) -> bool: - """Exclude SDK client holders, __getattr__, generic serialization. - Approximation: skip Any in: - - module-level vars named __client / __chat / __cache - - function bodies that contain 'def __getattr__' in the same file - - function signatures with '_serialize_for_api' or '_resolve_log_ref' in the name - """ - # Module-level SDK holders - if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name): - name = node.target.id - if re.match(r"^_\w+_(client|chat|cache)$", name): - return True - # Function signatures (Pattern 5) - if isinstance(node, ast.arg): - arg_text = ast.unparse(node.annotation) if node.annotation else "" - if "Any" in arg_text and any( - keyword in (node.arg or "") for keyword in ("serialize", "resolve_log_ref") - ): - return True - return False - -def scan_file(path: Path) -> list[Finding]: - """Scan a single .py file for anonymous Any annotations.""" - text = path.read_text(encoding="utf-8") - try: - tree = ast.parse(text) - except SyntaxError: - return [] - findings = [] - for node in ast.walk(tree): - if _is_pattern_3_4_5(node, text): - continue - # Detect `Any` in annotations - if isinstance(node, ast.AnnAssign) and node.annotation: - ann_text = ast.unparse(node.annotation) - if ann_text == "Any" or ann_text.startswith("Any[") or ann_text.endswith(" | Any") or ann_text.startswith("Any |"): - findings.append(Finding( - file=str(path.relative_to(REPO_ROOT)), - line=node.lineno, - category="any_standalone", - snippet=ann_text[:60], - )) - # Detect `dict[str, Any]` / `list[dict[str, Any]]` etc. - if isinstance(node, ast.arg) and node.annotation: - ann_text = ast.unparse(node.annotation) - if "Any" in ann_text: - findings.append(Finding( - file=str(path.relative_to(REPO_ROOT)), - line=node.lineno, - category="any_in_container", - snippet=ann_text[:60], - )) - return findings - -def scan_all() -> Report: - """Scan all .py files under src/.""" - all_findings: list[Finding] = [] - for py_file in SRC_DIR.rglob("*.py"): - all_findings.extend(scan_file(py_file)) - by_category: dict[str, int] = {} - by_file: dict[str, int] = {} - for f in all_findings: - by_category[f.category] = by_category.get(f.category, 0) + 1 - by_file[f.file] = by_file.get(f.file, 0) + 1 - files_with_findings = len(by_file) - return Report( - total_any=len(all_findings), - files_with_findings=files_with_findings, - by_category=by_category, - by_file=by_file, - ) - -def print_human_report(report: Report) -> None: - print("=== Anonymous Any Audit: src ===") - print(f"Total Any findings: {report.total_any}") - print(f"Files with findings: {report.files_with_findings}") - print("\nBy category:") - for cat, count in sorted(report.by_category.items(), key=lambda x: -x[1]): - print(f" {cat:30s} {count}") - -def main() -> int: - import argparse - parser = argparse.ArgumentParser() - parser.add_argument("--json", action="store_true") - parser.add_argument("--strict", action="store_true") - parser.add_argument("--baseline", default=str(DEFAULT_BASELINE)) - args = parser.parse_args() - report = scan_all() - if args.json: - print(json.dumps(asdict(report), indent=2)) - elif args.strict: - baseline_path = Path(args.baseline) - if not baseline_path.exists(): - print(f"STRICT ERROR: baseline not found at {baseline_path}", file=sys.stderr) - return 1 - baseline = json.loads(baseline_path.read_text()) - if report.total_any > baseline.get("total_any", 0): - print(f"STRICT FAIL: {report.total_any} Any sites > baseline {baseline['total_any']}", file=sys.stderr) - return 1 - print(f"STRICT OK: {report.total_any} Any sites <= baseline {baseline['total_any']}") - else: - print_human_report(report) - return 0 - -if __name__ == "__main__": - sys.exit(main()) -``` - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `uv run pytest tests/test_audit_dataclass_coverage.py -v` -Expected: PASS (4/4) - -- [ ] **Step 5: Commit** - -```bash -git add scripts/audit_dataclass_coverage.py tests/test_audit_dataclass_coverage.py -git commit -m "feat(audit): dataclass-coverage audit script with --strict CI gate" -``` - -### Task 0.3: Add styleguide §12 + generate baseline - -**Files:** -- Modify: `conductor/code_styleguides/type_aliases.md` (append §12) -- Create: `scripts/audit_dataclass_coverage.baseline.json` - -- [ ] **Step 1: Read existing styleguide structure** - -Run: `uv run python -c "print(open('conductor/code_styleguides/type_aliases.md').read()[-500:])"` -Expected: shows the end of the file; identify where §11 ends (or where to append) - -- [ ] **Step 2: Append §12 to styleguide** - -Append to `conductor/code_styleguides/type_aliases.md`: -```markdown - ---- - -## When to Promote `TypeAlias` to `dataclass` - -A `TypeAlias` like `Metadata: TypeAlias = dict[str, Any]` is a **rename** — the -underlying shape is unchanged. This is appropriate when: - -- The shape is **truly open** (extra keys allowed; the dict is a bag) -- The shape is **self-describing** (caller reads `entry.get("path")` without - needing to know which keys are required) -- The shape is **transient** (JSON-serialized, then deserialized; no - in-memory struct invariants) - -Promote to `dataclass(frozen=True)` when: - -- The shape has **a known set of required fields** with **specific types** - (e.g., a chat completion's `usage: UsageStats` with 4 int fields) -- Multiple sites access the same fields with **string keys** - (`payload["usage"]["input_tokens"]` × 5 sites = 5× the bug surface) -- The shape is **stable across serialization boundaries** (the on-disk / - on-wire format is documented and won't change per-call) -- The shape is **shared across multiple modules** (the same schema is used - by `ai_client.py` and `openai_compatible.py` and `api_hooks.py`) - -The reference pattern is `src/vendor_capabilities.py`. When in doubt, follow -that template: `frozen=True` dataclass + module-level registry + factory -functions. - -The fat-struct candidates identified in -[`docs/reports/ANY_TYPE_AUDIT_20260621.md`](../../docs/reports/ANY_TYPE_AUDIT_20260621.md) -(§3) are the canonical worked examples. -``` - -- [ ] **Step 3: Generate baseline** - -Run: `uv run python scripts/audit_weak_types.py --json > /tmp/audit_pre.json` -Then create baseline manually: -```bash -uv run python -c " -import json, subprocess -result = subprocess.run(['uv', 'run', 'python', 'scripts/audit_dataclass_coverage.py', '--json'], capture_output=True, text=True) -report = json.loads(result.stdout) -baseline = { - 'total_any': report['total_any'], - 'files_with_findings': report['files_with_findings'], - 'generated_at': '2026-06-21', - 'note': 'Baseline for --strict mode. Re-generate when a new track intentionally reduces the count.' -} -with open('scripts/audit_dataclass_coverage.baseline.json', 'w') as f: - json.dump(baseline, f, indent=2) -print('baseline:', baseline['total_any'], 'Any sites in', baseline['files_with_findings'], 'files') -" -``` -Expected output: `baseline: ~210 Any sites in ~25 files` - -- [ ] **Step 4: Verify strict mode passes** - -Run: `uv run python scripts/audit_dataclass_coverage.py --strict` -Expected: `STRICT OK: Any sites <= baseline ` exit 0 - -- [ ] **Step 5: Commit** - -```bash -git add conductor/code_styleguides/type_aliases.md scripts/audit_dataclass_coverage.baseline.json -git commit -m "docs(styleguide): add §12 When to Promote TypeAlias to dataclass + audit baseline" -``` - ---- - -## Phase 1: src/mcp_tool_specs.py (P1, 8 sites) - -Focus: Convert 45 tool specs from `list[dict[str, Any]]` to `list[ToolSpec]`. Update 6 call sites across 3 files. - -### Task 1.1: Write failing tests for ToolSpec module - -**Files:** -- Create: `tests/test_mcp_tool_specs.py` - -- [ ] **Step 1: Write tests** - -```python -"""Tests for src/mcp_tool_specs.py.""" -from src.mcp_tool_specs import ToolParameter, ToolSpec, get_tool_spec, tool_names, get_tool_schemas, register, _REGISTRY - -def test_all_45_tools_registered() -> None: - """All MCP tools from mcp_client.MCP_TOOL_SPECS must be registered.""" - names = tool_names() - assert len(names) == 45, f"expected 45 tools, got {len(names)}" - -def test_get_tool_spec_returns_correct_spec() -> None: - """get_tool_spec('py_remove_def') must return the expected spec.""" - spec = get_tool_spec("py_remove_def") - assert isinstance(spec, ToolSpec) - assert spec.name == "py_remove_def" - assert spec.description # non-empty - -def test_tool_names_matches_specs() -> None: - """tool_names() must match the set of ToolSpec.name values.""" - names = tool_names() - specs = get_tool_schemas() - assert names == {s.name for s in specs} - -def test_tool_spec_parameters_are_immutable_tuple() -> None: - """ToolSpec.parameters must be a tuple (immutable, matching frozen=True).""" - spec = get_tool_spec("py_remove_def") - assert isinstance(spec.parameters, tuple) - -def test_tool_spec_is_frozen() -> None: - """ToolSpec must be frozen (immutable).""" - spec = get_tool_spec("py_remove_def") - try: - spec.description = "modified" # type: ignore[misc] - assert False, "ToolSpec should be frozen" - except Exception: - pass # expected - -def test_register_new_tool() -> None: - """register() must add a new ToolSpec to the registry.""" - initial_count = len(tool_names()) - test_spec = ToolSpec(name="test_tool", description="test", parameters=()) - register(test_spec) - assert "test_tool" in tool_names() - assert len(tool_names()) == initial_count + 1 - -def test_tool_parameter_enum_is_optional() -> None: - """ToolParameter.enum must default to None (most params have no enum).""" - spec = get_tool_spec("py_remove_def") - for p in spec.parameters: - assert p.enum is None or isinstance(p.enum, list) - -def test_tool_names_subset_of_agent_tool_names() -> None: - """Cross-module invariant: tool_names() ⊆ models.AGENT_TOOL_NAMES.""" - from src import models - mcp_names = tool_names() - agent_names = set(models.AGENT_TOOL_NAMES) - assert mcp_names.issubset(agent_names), f"MCP-only tools not in AGENT_TOOL_NAMES: {mcp_names - agent_names}" -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `uv run pytest tests/test_mcp_tool_specs.py -v` -Expected: FAIL (ModuleNotFoundError: No module named 'src.mcp_tool_specs') - -### Task 1.2: Create `src/mcp_tool_specs.py` skeleton - -- [ ] **Step 3: Create empty module** - -Create `src/mcp_tool_specs.py`: -```python -"""Module-level abstraction layer for MCP tool specifications. - -Mirrors src/vendor_capabilities.py:64-76 template: - frozen dataclass + module-level registry + factory functions. -""" -from __future__ import annotations -from dataclasses import dataclass, field -from typing import Optional - -@dataclass(frozen=True) -class ToolParameter: - name: str - type: str # "string" | "integer" | "boolean" | "object" | "array" - description: str - required: bool = False - enum: Optional[list[str]] = None - -@dataclass(frozen=True) -class ToolSpec: - name: str - description: str - parameters: tuple[ToolParameter, ...] - category: str = "file" - -_REGISTRY: dict[str, ToolSpec] = {} - -def register(spec: ToolSpec) -> None: - _REGISTRY[spec.name] = spec - -def get_tool_spec(name: str) -> ToolSpec: - return _REGISTRY[name] - -def tool_names() -> set[str]: - return set(_REGISTRY.keys()) - -def get_tool_schemas() -> list[ToolSpec]: - return list(_REGISTRY.values()) -``` - -- [ ] **Step 4: Run tests to verify which now pass** - -Run: `uv run pytest tests/test_mcp_tool_specs.py -v` -Expected: PASS for `test_tool_spec_parameters_are_immutable_tuple`, `test_tool_spec_is_frozen`, `test_register_new_tool`, `test_tool_parameter_enum_is_optional`. FAIL for the rest (empty registry). - -### Task 1.3: Migrate MCP_TOOL_SPECS to ToolSpec registry - -- [ ] **Step 5: Read source dicts** - -Run: `uv run python -c " -import ast, json -tree = ast.parse(open('src/mcp_client.py').read()) -for n in ast.walk(tree): - if isinstance(n, ast.AnnAssign) and isinstance(n.target, ast.Name) and n.target.id == 'MCP_TOOL_SPECS': - for elt in n.value.elts: - print(elt.values[0].value) # tool name -"` -Expected: list of 45 tool names - -- [ ] **Step 6: Generate migration script** - -The 45 tool specs need to be converted from dict to ToolSpec. Use this helper script (`scripts/_migrate_mcp_specs.py`, throwaway; delete after Phase 1): -```python -"""One-time migration: convert MCP_TOOL_SPECS dicts to ToolSpec instances.""" -import ast -from pathlib import Path - -src = Path("src/mcp_client.py").read_text(encoding="utf-8") -tree = ast.parse(src) -specs = [] -for n in ast.walk(tree): - if isinstance(n, ast.AnnAssign) and isinstance(n.target, ast.Name) and n.target.id == "MCP_TOOL_SPECS": - for elt in n.value.elts: - d = ast.literal_eval(elt) - specs.append(d) - -# Print Python source for the registry population -print("""# AUTO-GENERATED by scripts/_migrate_mcp_specs.py on 2026-06-21 -# Source: src/mcp_client.py:1972 MCP_TOOL_SPECS""") -for spec in specs: - params = spec.get("parameters", {}) - props = params.get("properties", {}) - required = set(params.get("required", [])) - tool_params = [] - for pname, pdef in props.items(): - tool_params.append(ToolParameter( - name=pname, - type=pdef.get("type", "string"), - description=pdef.get("description", ""), - required=pname in required, - enum=pdef.get("enum"), - )) - category = "ast" if spec["name"].startswith(("py_", "ts_")) else "file" - if spec["name"] in ("web_search", "fetch_url"): - category = "network" - register(ToolSpec( - name=spec["name"], - description=spec.get("description", ""), - parameters=tuple(tool_params), - category=category, - )) -``` - -Run: `uv run python scripts/_migrate_mcp_specs.py > /tmp/mcp_specs_migration.py` -Then copy the output into `src/mcp_tool_specs.py` (replace the `# migration content` placeholder). - -- [ ] **Step 7: Run tests** - -Run: `uv run pytest tests/test_mcp_tool_specs.py -v` -Expected: PASS (8/8) - -- [ ] **Step 8: Delete migration script + commit** - -```bash -rm scripts/_migrate_mcp_specs.py -git add src/mcp_tool_specs.py tests/test_mcp_tool_specs.py -git commit -m "feat(mcp): ToolSpec + ToolParameter dataclasses; migrate 45 MCP_TOOL_SPECS entries" -``` - -### Task 1.4: Update `src/mcp_client.py` call sites - -- [ ] **Step 1: Find call sites** - -Run: `uv run python scripts/grep.py "MCP_TOOL_SPECS|TOOL_NAMES" src/mcp_client.py` (or use grep tool) -Expected: lines 1944, 1958, 1972 (declaration), 2747 - -- [ ] **Step 2: Update line 1944** (`native_names = {t['name'] for t in MCP_TOOL_SPECS}`) - -Replace with: -```python -from src import mcp_tool_specs -native_names = mcp_tool_specs.tool_names() -``` - -- [ ] **Step 3: Update line 1958** (`res = list(MCP_TOOL_SPECS)`) - -Replace with: -```python -res = mcp_tool_specs.get_tool_schemas() -``` - -- [ ] **Step 4: Delete MCP_TOOL_SPECS declaration** (line 1972) - -The dict literal in `src/mcp_client.py` is now in `src/mcp_tool_specs.py`; delete the declaration. - -- [ ] **Step 5: Update line 2747** (`TOOL_NAMES: set[str] = {t['name'] for t in MCP_TOOL_SPECS}`) - -Replace with: -```python -TOOL_NAMES: set[str] = mcp_tool_specs.tool_names() -``` - -(Keeps backward-compat re-export; can be removed later.) - -- [ ] **Step 6: Run regression tests** - -Run: `uv run pytest tests/test_mcp_client.py -v` -Expected: PASS (all existing tests) - -### Task 1.5: Update `src/ai_client.py` callers - -- [ ] **Step 1: Find call sites** - -Run: `grep -n "mcp_client.TOOL_NAMES" src/ai_client.py` -Expected: lines 560, 582, 1012 - -- [ ] **Step 2: Add import + update 3 sites** - -Add at top of `src/ai_client.py`: -```python -from src import mcp_tool_specs -``` - -Replace all 3 occurrences of `mcp_client.TOOL_NAMES` with `mcp_tool_specs.tool_names()`. - -- [ ] **Step 3: Run regression tests** - -Run: `uv run pytest tests/test_ai_client*.py -v` -Expected: PASS - -### Task 1.6: Phase 1 checkpoint - -- [ ] **Step 1: Full verification** - -Run: -```bash -uv run pytest tests/test_mcp_tool_specs.py tests/test_mcp_client.py tests/test_ai_client*.py --timeout=60 -uv run python scripts/audit_weak_types.py --strict -uv run python scripts/audit_dataclass_coverage.py --strict -``` -Expected: all PASS / exit 0 - -- [ ] **Step 2: Phase 1 checkpoint commit + git note** - -```bash -git add -A -git commit -m "conductor(checkpoint): Phase 1 complete - mcp_tool_specs (P1, 8 sites migrated)" -git notes add -m "Phase 1 checkpoint: src/mcp_tool_specs.py + 45 ToolSpec instances; mcp_client.py + ai_client.py updated; 8 weak sites resolved" HEAD -``` - -Update `conductor/tracks/any_type_componentization_20260621/state.toml` to mark phase_1 status="completed" + checkpointsha. - ---- - -## Phase 2: src/openai_schemas.py (P1, 17 sites) - -Focus: Convert `NormalizedResponse` + `OpenAICompatibleRequest` to use `ChatMessage` + `UsageStats` + `ToolCall` dataclasses. Update `_send_grok` + `_send_minimax` + `_send_llama` in `ai_client.py`. - -### Task 2.1: Write failing tests for ChatMessage + UsageStats + ToolCall - -**Files:** -- Create: `tests/test_openai_schemas.py` - -- [ ] **Step 1: Write tests** - -```python -"""Tests for src/openai_schemas.py.""" -from src.openai_schemas import ToolCall, ToolCallFunction, ChatMessage, UsageStats, NormalizedResponse, OpenAICompatibleRequest - -def test_chat_message_user_role() -> None: - msg = ChatMessage(role="user", content="hello") - assert msg.role == "user" - assert msg.content == "hello" - assert msg.tool_calls is None - assert msg.tool_call_id is None - -def test_chat_message_assistant_with_tool_calls() -> None: - tc = ToolCall(id="1", type="function", function=ToolCallFunction(name="f", arguments="{}")) - msg = ChatMessage(role="assistant", content="", tool_calls=(tc,)) - assert msg.tool_calls == (tc,) - assert len(msg.tool_calls) == 1 - -def test_chat_message_tool_response() -> None: - msg = ChatMessage(role="tool", content="result", tool_call_id="1") - assert msg.tool_call_id == "1" - -def test_usage_stats_field_access() -> None: - usage = UsageStats(input_tokens=100, output_tokens=50) - assert usage.input_tokens == 100 - assert usage.output_tokens == 50 - assert usage.cache_read_tokens == 0 - assert usage.cache_creation_tokens == 0 - -def test_usage_stats_is_frozen() -> None: - usage = UsageStats(input_tokens=100, output_tokens=50) - try: - usage.input_tokens = 200 # type: ignore[misc] - assert False - except Exception: - pass - -def test_tool_call_function_arguments_is_string() -> None: - """arguments is a JSON string, not a dict (matches OpenAI API).""" - tc = ToolCall(id="1", type="function", function=ToolCallFunction(name="f", arguments='{"x":1}')) - assert tc.function.arguments == '{"x":1}' - -def test_normalized_response_uses_usage_stats() -> None: - """NormalizedResponse.usage should be a UsageStats object, not 4 separate int fields.""" - usage = UsageStats(input_tokens=10, output_tokens=5, cache_read_tokens=2) - response = NormalizedResponse(text="hello", tool_calls=(), usage=usage, raw_response=None) - assert response.usage.input_tokens == 10 - assert response.usage.cache_read_tokens == 2 - -def test_normalized_response_raw_response_is_any() -> None: - """Pattern 3: raw_response stays as Any (SDK-specific).""" - import typing - response = NormalizedResponse(text="x", tool_calls=(), usage=UsageStats(input_tokens=0, output_tokens=0), raw_response={"sdk": "data"}) - assert response.raw_response == {"sdk": "data"} - assert "Any" in str(typing.get_type_hints(NormalizedResponse)["raw_response"]) or \ - str(typing.get_type_hints(NormalizedResponse)["raw_response"]) == "typing.Any" - -def test_openai_compatible_request_messages_typed() -> None: - """OpenAICompatibleRequest.messages must be list[ChatMessage].""" - msg = ChatMessage(role="user", content="hi") - req = OpenAICompatibleRequest(messages=[msg], model="gpt-4") - assert req.messages[0].role == "user" - -def test_openai_compatible_request_defaults() -> None: - req = OpenAICompatibleRequest(messages=[], model="gpt-4") - assert req.temperature == 0.0 - assert req.top_p == 1.0 - assert req.max_tokens == 8192 - assert req.stream is False -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `uv run pytest tests/test_openai_schemas.py -v` -Expected: FAIL (ModuleNotFoundError) - -### Task 2.2: Create `src/openai_schemas.py` - -- [ ] **Step 1: Create module** - -Create `src/openai_schemas.py`: -```python -"""OpenAI-compatible API schemas (ChatMessage, UsageStats, ToolCall, NormalizedResponse, OpenAICompatibleRequest). - -Mirrors src/vendor_capabilities.py:64-76 template. -""" -from __future__ import annotations -from dataclasses import dataclass, field -from typing import Any, Callable, Optional -from src.type_aliases import Metadata - -@dataclass(frozen=True) -class ToolCallFunction: - name: str - arguments: str # JSON string (matches OpenAI wire format) - -@dataclass(frozen=True) -class ToolCall: - id: str - type: str = "function" - function: ToolCallFunction = field(default_factory=ToolCallFunction) - -@dataclass(frozen=True) -class ChatMessage: - role: str # "system" | "user" | "assistant" | "tool" - content: str - tool_calls: Optional[tuple[ToolCall, ...]] = None - tool_call_id: Optional[str] = None - name: Optional[str] = None - -@dataclass(frozen=True) -class UsageStats: - input_tokens: int - output_tokens: int - cache_read_tokens: int = 0 - cache_creation_tokens: int = 0 - -@dataclass(frozen=True) -class NormalizedResponse: - text: str - tool_calls: tuple[ToolCall, ...] - usage: UsageStats - raw_response: Any # Pattern 3: SDK-specific (stays Any) - -@dataclass -class OpenAICompatibleRequest: - messages: list[ChatMessage] - model: str - temperature: float = 0.0 - top_p: float = 1.0 - max_tokens: int = 8192 - tools: Optional[list[dict[str, Any]]] = None # TODO(future-track): migrate to list[ToolSpec] - tool_choice: str = "auto" - stream: bool = False - stream_callback: Optional[Callable[[str], None]] = None - extra_body: Optional[dict[str, Any]] = None -``` - -- [ ] **Step 2: Run tests** - -Run: `uv run pytest tests/test_openai_schemas.py -v` -Expected: PASS (10/10) - -- [ ] **Step 3: Commit** - -```bash -git add src/openai_schemas.py tests/test_openai_schemas.py -git commit -m "feat(openai_schemas): ChatMessage + UsageStats + ToolCall + NormalizedResponse + OpenAICompatibleRequest" -``` - -### Task 2.3: Update `src/openai_compatible.py` - -- [ ] **Step 1: Find consumers** - -Run: `grep -n "NormalizedResponse\|OpenAICompatibleRequest" src/openai_compatible.py src/ai_client.py src/api_hook_client.py | head -30` - -- [ ] **Step 2: Update imports + type hints** - -In `src/openai_compatible.py`: -- Add: `from src.openai_schemas import NormalizedResponse, OpenAICompatibleRequest, ChatMessage, UsageStats, ToolCall, ToolCallFunction` -- Remove the local `NormalizedResponse` + `OpenAICompatibleRequest` class definitions (lines ~10-30) - -- [ ] **Step 3: Update internal consumers** (~5 functions) - -Each consumer that constructs or destructures `NormalizedResponse`: -- Replace `usage_input_tokens=..., usage_output_tokens=..., usage_cache_read_tokens=..., usage_cache_creation_tokens=...` with `usage=UsageStats(input_tokens=..., output_tokens=..., cache_read_tokens=..., cache_creation_tokens=...)` -- Replace `tool_calls=[...]` (list) with `tool_calls=(...)` (tuple of ToolCall) -- Replace `messages=[{"role": ..., "content": ...}]` (list of dict) with `messages=[ChatMessage(role=..., content=...)]` - -- [ ] **Step 4: Run regression tests** - -Run: `uv run pytest tests/test_openai_compatible.py -v` -Expected: PASS - -### Task 2.4: Update `src/ai_client.py` _send_grok + _send_minimax + _send_llama - -- [ ] **Step 1: Find the 3 functions** - -Run: `grep -n "def _send_grok\|def _send_minimax\|def _send_llama" src/ai_client.py` - -- [ ] **Step 2: Update each function** (same pattern as Task 2.3) - -For each of `_send_grok`, `_send_minimax`, `_send_llama`: -- Replace dict constructions with dataclass constructions -- Update return type if it returns `dict[str, Any]` for normalized response - -- [ ] **Step 3: Run regression tests** - -Run: `uv run pytest tests/test_ai_client*.py -v` -Expected: PASS - -### Task 2.5: Phase 2 checkpoint - -- [ ] **Step 1: Full verification** - -```bash -uv run pytest tests/test_openai_schemas.py tests/test_openai_compatible.py tests/test_ai_client*.py --timeout=60 -uv run python scripts/audit_weak_types.py --strict -uv run python scripts/audit_dataclass_coverage.py --strict -``` - -- [ ] **Step 2: Checkpoint commit + git note** - -```bash -git add -A -git commit -m "conductor(checkpoint): Phase 2 complete - openai_schemas (P1, 17 sites migrated)" -git notes add -m "Phase 2 checkpoint: src/openai_schemas.py + ChatMessage/UsageStats/ToolCall; 3 send_* functions updated; 17 weak sites resolved" HEAD -``` - ---- - -## Phase 3: src/provider_state.py (P2, 41 sites) [LARGEST PHASE] - -Focus: Replace 14 module globals (7 histories + 7 locks) with `_PROVIDER_HISTORIES` dict. Update ~27 call sites in `ai_client.py`. - -### Task 3.1: Snapshot baseline + write tests - -- [ ] **Step 1: Snapshot pre-Phase-3 baseline** - -Run: `uv run python scripts/audit_dataclass_coverage.py --json > /tmp/pre_phase3.json` -Expected: total_any should be ~baseline minus 25 (Phases 1+2 contributions) - -- [ ] **Step 2: Write tests** - -Create `tests/test_provider_state.py`: -```python -"""Tests for src.provider_state.""" -import threading -from src.provider_state import ProviderHistory, _PROVIDER_HISTORIES, get_history - -def test_all_six_providers_have_history() -> None: - """Each of the 6 providers must have a ProviderHistory instance.""" - assert set(_PROVIDER_HISTORIES.keys()) == {"anthropic", "deepseek", "minimax", "qwen", "grok", "llama"} - -def test_get_history_returns_singleton() -> None: - """get_history(p) must return the same instance across calls.""" - a1 = get_history("anthropic") - a2 = get_history("anthropic") - assert a1 is a2 - -def test_provider_history_append_under_lock() -> None: - """append() must be thread-safe (lock protects mutation).""" - h = ProviderHistory() - results = [] - def worker() -> None: - for i in range(100): - h.append({"role": "user", "content": str(i)}) - results.append(len(h.get_all())) - ts = [threading.Thread(target=worker) for _ in range(10)] - for t in ts: t.start() - for t in ts: t.join() - assert len(h.get_all()) == 1000 - -def test_provider_history_clear_resets_list() -> None: - """clear() must reset the messages list (lock preserved).""" - h = ProviderHistory() - h.append({"role": "user", "content": "a"}) - h.append({"role": "user", "content": "b"}) - assert len(h.get_all()) == 2 - h.clear() - assert h.get_all() == [] - # Lock is preserved (same instance) - assert h.lock is h.lock - -def test_provider_history_replace_all_swaps_list() -> None: - """replace_all(messages) must atomically swap the list.""" - h = ProviderHistory() - h.append({"role": "user", "content": "old"}) - new_msgs = [{"role": "user", "content": "new1"}, {"role": "user", "content": "new2"}] - h.replace_all(new_msgs) - assert h.get_all() == new_msgs - -def test_default_factory_creates_fresh_lock() -> None: - """Each ProviderHistory() must have its own lock (default_factory).""" - h1 = ProviderHistory() - h2 = ProviderHistory() - assert h1.lock is not h2.lock - -def test_global_histories_isolated() -> None: - """Mutating anthropic's history must not affect grok's.""" - a = get_history("anthropic") - g = get_history("grok") - a.append({"role": "user", "content": "a_msg"}) - assert g.get_all() == [] -``` - -- [ ] **Step 3: Run tests to verify they fail** - -Run: `uv run pytest tests/test_provider_state.py -v` -Expected: FAIL (ModuleNotFoundError) - -### Task 3.2: Create `src/provider_state.py` - -- [ ] **Step 1: Create module** - -Create `src/provider_state.py`: -```python -"""Per-provider history state (Phase 3 of any_type_componentization_20260621). - -Replaces 14 module globals in src/ai_client.py (7 __history + 7 __history_lock) -with a single _PROVIDER_HISTORIES dict. - -Mirrors src/vendor_capabilities.py:64-76 template (frozen=False here because ProviderHistory mutates). -""" -from __future__ import annotations -import threading -from dataclasses import dataclass, field -from src.type_aliases import Metadata - -@dataclass -class ProviderHistory: - messages: list[Metadata] = field(default_factory=list) - lock: threading.Lock = field(default_factory=threading.Lock) - - def append(self, message: Metadata) -> None: - with self.lock: - self.messages.append(message) - - def get_all(self) -> list[Metadata]: - with self.lock: - return list(self.messages) - - def replace_all(self, messages: list[Metadata]) -> None: - with self.lock: - self.messages = list(messages) - - def clear(self) -> None: - with self.lock: - self.messages = [] - -_PROVIDER_HISTORIES: dict[str, ProviderHistory] = { - "anthropic": ProviderHistory(), - "deepseek": ProviderHistory(), - "minimax": ProviderHistory(), - "qwen": ProviderHistory(), - "grok": ProviderHistory(), - "llama": ProviderHistory(), -} - -def get_history(provider: str) -> ProviderHistory: - return _PROVIDER_HISTORIES[provider] -``` - -- [ ] **Step 2: Run tests** - -Run: `uv run pytest tests/test_provider_state.py -v` -Expected: PASS (7/7) - -- [ ] **Step 3: Commit** - -```bash -git add src/provider_state.py tests/test_provider_state.py -git commit -m "feat(provider_state): ProviderHistory dataclass + _PROVIDER_HISTORIES dict" -``` - -### Task 3.3: Remove globals from `src/ai_client.py` - -- [ ] **Step 1: Add import + remove 14 globals** (lines 111-133) - -Add at top of `src/ai_client.py` (with other ai_client imports): -```python -from src.provider_state import get_history -``` - -Delete lines 111-133 (the 7 history + 7 lock declarations). - -- [ ] **Step 2: Update cleanup() function** (lines 463-499) - -For each of the 7 providers, replace the lock-guarded reset: -```python -# OLD: -with _anthropic_history_lock: - _anthropic_history = [] - -# NEW: -get_history("anthropic").clear() -``` - -Apply this pattern to all 7 providers in the cleanup() function. - -- [ ] **Step 3: Run tests** - -Run: `uv run pytest tests/test_ai_client_result.py -v` -Expected: PASS (this is the conservative smoke test) - -- [ ] **Step 4: Commit** - -```bash -git add src/ai_client.py -git commit -m "refactor(ai_client): remove 14 module globals; use get_history().clear() in cleanup()" -``` - -### Task 3.4: Update _send_anthropic - -- [ ] **Step 1: Find all references** - -Run: `grep -n "_anthropic_history" src/ai_client.py | head -30` -Expected: ~20 lines (1447, 1457-1460, 1469, 1471, 1475, 1489, 1503, 1506, 1582, etc.) - -- [ ] **Step 2: Mechanical replacement** - -For each `_anthropic_history` reference: -- Direct read (e.g., `for msg in _anthropic_history:`): replace with `for msg in get_history("anthropic").get_all():` -- Direct write (e.g., `_anthropic_history.append({...})`): replace with `get_history("anthropic").append({...})` -- Lock-guarded read (`with _anthropic_history_lock: ... _anthropic_history ...`): replace with `h = get_history("anthropic"); with h.lock: ... h.messages ...` -- The `_repair_anthropic_history(_anthropic_history)` call: pass `get_history("anthropic").get_all()` (the helper takes a list parameter and doesn't need the lock context) - -- [ ] **Step 3: Run tests** - -Run: `uv run pytest tests/test_ai_client*.py -v` -Expected: PASS - -- [ ] **Step 4: Commit** - -```bash -git add src/ai_client.py -git commit -m "refactor(ai_client): _send_anthropic uses get_history('anthropic')" -``` - -### Task 3.5: Update _send_deepseek + _send_grok + _send_minimax + _send_qwen + _send_llama - -Repeat Task 3.4 pattern for each of the remaining 5 send functions. Each function has ~8-10 references. - -- [ ] **Step 1-5: For each function** (deepseek, grok, minimax, qwen, llama): - - Find references via grep - - Replace with `get_history("").X()` pattern - - Run tests after each - - Commit after each - -```bash -# After each function update: -git add src/ai_client.py -git commit -m "refactor(ai_client): _send_ uses get_history('')" -``` - -### Task 3.6: Phase 3 checkpoint - -- [ ] **Step 1: Full verification** - -```bash -uv run pytest tests/test_provider_state.py tests/test_ai_client*.py --timeout=60 -uv run python scripts/audit_weak_types.py --strict -uv run python scripts/audit_dataclass_coverage.py --strict -uv run python -c "from src import ai_client; print('SDK clients preserved:', ai_client._gemini_chat is None or ai_client._gemini_client is not None)" -``` -Expected: all PASS / exit 0; SDK client holders NOT touched - -- [ ] **Step 2: Checkpoint commit + git note** - -```bash -git add -A -git commit -m "conductor(checkpoint): Phase 3 complete - provider_state (P2, 41 sites migrated)" -git notes add -m "Phase 3 checkpoint: src/provider_state.py + ProviderHistory dict; 14 globals removed; ~27 call sites updated; SDK clients (Pattern 3) preserved" HEAD -``` - ---- - -## Phase 4: src/log_registry.py Session (P2, 7 sites) - -Focus: Add `Session` + `SessionMetadata` dataclasses inline; convert `self.data: dict[str, dict[str, Any]]` → `dict[str, Session]`. - -### Task 4.1: Write failing tests - -- [ ] **Step 1: Extend tests/test_log_registry.py** - -Add to existing `tests/test_log_registry.py`: -```python -from src.log_registry import Session, SessionMetadata, LogRegistry - -def test_session_dataclass_has_expected_fields() -> None: - s = Session(session_id="abc", path="/tmp/abc.jsonl", start_time="2026-06-21T00:00:00") - assert s.session_id == "abc" - assert s.path == "/tmp/abc.jsonl" - assert s.whitelisted is False - assert s.metadata is None - -def test_session_metadata_defaults() -> None: - m = SessionMetadata() - assert m.message_count == 0 - assert m.errors == 0 - assert m.size_kb == 0 - assert m.whitelisted is False - -def test_session_is_frozen() -> None: - s = Session(session_id="x", path="/p", start_time="t") - try: - s.session_id = "y" # type: ignore[misc] - assert False - except Exception: - pass - -def test_log_registry_data_is_dict_of_session() -> None: - """self.data must be dict[str, Session], not dict[str, dict[str, Any]].""" - import typing - hints = typing.get_type_hints(LogRegistry) - # Check the 'data' field annotation - data_type = hints.get("data") - assert "Session" in str(data_type) - -def test_session_metadata_is_optional() -> None: - s = Session(session_id="x", path="/p", start_time="t", metadata=SessionMetadata(message_count=5)) - assert s.metadata is not None - assert s.metadata.message_count == 5 -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `uv run pytest tests/test_log_registry.py -v` -Expected: FAIL (ImportError for Session/SessionMetadata) - -### Task 4.2: Add dataclasses to `src/log_registry.py` - -- [ ] **Step 1: Add imports + dataclasses** (top of file) - -Add at top: -```python -from dataclasses import dataclass, field -from typing import Optional - -@dataclass(frozen=True) -class SessionMetadata: - message_count: int = 0 - errors: int = 0 - size_kb: int = 0 - whitelisted: bool = False - reason: str = '' - timestamp: Optional[str] = None - -@dataclass(frozen=True) -class Session: - session_id: str - path: str - start_time: str # ISO format - whitelisted: bool = False - metadata: Optional[SessionMetadata] = None -``` - -- [ ] **Step 2: Update `LogRegistry.data` type annotation** - -Change `self.data: dict[str, dict[str, Any]] = {}` to `self.data: dict[str, Session] = field(default_factory=dict)`. - -Also update the `@dataclass` decorator on `LogRegistry` if not already present. - -- [ ] **Step 3: Run tests** - -Run: `uv run pytest tests/test_log_registry.py -v` -Expected: PASS - -### Task 4.3: Update consumers (session_logger, log_pruner, gui_2) - -- [ ] **Step 1: Find consumer references** - -Run: `grep -n "log_registry\|LogRegistry\|session_log" src/session_logger.py src/log_pruner.py src/gui_2.py | head -30` - -- [ ] **Step 2: Update session_logger.py** (`open_session`, `close_session`) - -Replace dict construction with `Session(...)` dataclass construction. - -- [ ] **Step 3: Update log_pruner.py** (`prune_old_logs`) - -Update iteration over `self.data` to use `Session` field access (`.path`, `.start_time`, `.metadata`). - -- [ ] **Step 4: Update gui_2.py Log Management panel** - -Find the panel rendering code (search for "log_registry" or "session_log" in gui_2.py) and update field access from `data[key]["path"]` to `data[key].path`. - -- [ ] **Step 5: Run regression tests** - -```bash -uv run pytest tests/test_log_registry.py tests/test_session_logger.py tests/test_log_pruner.py --timeout=60 -``` - -### Task 4.4: Phase 4 checkpoint - -```bash -git add -A -git commit -m "conductor(checkpoint): Phase 4 complete - log_registry Session (P2, 7 sites migrated)" -git notes add -m "Phase 4 checkpoint: Session + SessionMetadata dataclasses; 4 files updated; 7 weak sites resolved" HEAD -``` - ---- - -## Phase 5: src/api_hooks.py WebSocketMessage (P3, 16 sites) - -Focus: Add `WebSocketMessage` dataclass; convert `broadcast(channel, payload)` to `broadcast(message: WebSocketMessage)`. - -### Task 5.1: Write failing tests - -- [ ] **Step 1: Extend tests/test_api_hooks.py** - -```python -from src.api_hooks import WebSocketMessage -from src.type_aliases import JsonValue - -def test_websocket_message_is_frozen() -> None: - msg = WebSocketMessage(channel="test", payload={"key": "value"}) - try: - msg.channel = "other" # type: ignore[misc] - assert False - except Exception: - pass - -def test_websocket_message_payload_accepts_json_value() -> None: - """payload must be JsonValue (recursive).""" - msg_str = WebSocketMessage(channel="c", payload="hello") - msg_dict = WebSocketMessage(channel="c", payload={"k": "v"}) - msg_list = WebSocketMessage(channel="c", payload=[1, 2, 3]) - msg_nested = WebSocketMessage(channel="c", payload={"items": [{"id": 1}]}) - assert msg_str.payload == "hello" - assert msg_dict.payload == {"k": "v"} - assert msg_list.payload == [1, 2, 3] - assert msg_nested.payload == {"items": [{"id": 1}]} - -def test_websocket_message_payload_rejects_non_json() -> None: - """payload must reject non-JSON values (type-checker enforces).""" - # This is a static type check, not runtime; just verify the annotation - import typing - hints = typing.get_type_hints(WebSocketMessage) - assert "JsonValue" in str(hints["payload"]) - -def test_serialize_for_api_returns_json_value() -> None: - """_serialize_for_api return type must be JsonValue.""" - import typing - from src.api_hooks import _serialize_for_api - hints = typing.get_type_hints(_serialize_for_api) - assert "JsonValue" in str(hints["return"]) - -def test_get_set_app_attr_unchanged() -> None: - """Pattern 4 preserved: _get_app_attr / _set_app_attr signatures unchanged.""" - from src.api_hooks import _get_app_attr, _set_app_attr - import typing - get_hints = typing.get_type_hints(_get_app_attr) - set_hints = typing.get_type_hints(_set_app_attr) - assert "Any" in str(get_hints.get("return", "")) - assert "Any" in str(get_hints.get("default", "")) -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `uv run pytest tests/test_api_hooks.py -v` -Expected: FAIL (ImportError for WebSocketMessage) - -### Task 5.2: Add dataclass to `src/api_hooks.py` - -- [ ] **Step 1: Add import + dataclass** - -Add to top of `src/api_hooks.py`: -```python -from dataclasses import dataclass -from src.type_aliases import JsonValue - -@dataclass(frozen=True) -class WebSocketMessage: - channel: str - payload: JsonValue -``` - -- [ ] **Step 2: Update `_serialize_for_api` return type** - -Change `def _serialize_for_api(obj: Any) -> Any:` to `def _serialize_for_api(obj: Any) -> JsonValue:`. - -(The body stays the same; the type hint change is the only modification.) - -- [ ] **Step 3: Update `broadcast` signature** - -Change `def broadcast(self, channel: str, payload: dict[str, Any]) -> None:` to `def broadcast(self, message: WebSocketMessage) -> None:` (and update the body to use `message.channel` + `message.payload`). - -- [ ] **Step 4: Update broadcast callers** (~5-10 sites) - -In `src/app_controller.py` and `src/gui_2.py`, change: -```python -broadcast(channel="x", payload={"k": "v"}) -``` -to: -```python -broadcast(WebSocketMessage(channel="x", payload={"k": "v"})) -``` - -- [ ] **Step 5: Run tests** - -```bash -uv run pytest tests/test_api_hooks.py tests/test_app_controller*.py --timeout=60 -``` -Expected: PASS - -### Task 5.3: Phase 5 checkpoint - -```bash -git add -A -git commit -m "conductor(checkpoint): Phase 5 complete - api_hooks WebSocketMessage (P3, 16 sites migrated)" -git notes add -m "Phase 5 checkpoint: WebSocketMessage dataclass + JsonValue usage; _serialize_for_api signature updated; Pattern 4 preserved" HEAD -``` - ---- - -## Phase 6: Verify + Docs + Archive - -Focus: Full audit + 11-tier regression + type registry regeneration + end-of-track report + archive. - -### Task 6.1: Full audit + regression suite - -- [ ] **Step 1: Run all 3 audits** - -```bash -uv run python scripts/audit_weak_types.py --strict -uv run python scripts/audit_dataclass_coverage.py --strict -uv run python scripts/generate_type_registry.py --check -``` -Expected: all exit 0 - -- [ ] **Step 2: Run 11-tier batched regression** - -```bash -uv run python scripts/run_tests_batched.py -``` -Expected: all tiers PASS (or with documented pre-existing skips) - -### Task 6.2: Regenerate type registry - -- [ ] **Step 1: Run the generator** - -```bash -uv run python scripts/generate_type_registry.py -git add docs/type_registry/ -git commit -m "docs(type_registry): regenerate with new modules (mcp_tool_specs, openai_schemas, provider_state)" -``` - -- [ ] **Step 2: Verify --check** - -Run: `uv run python scripts/generate_type_registry.py --check` -Expected: "Registry in sync (N files checked)" - -### Task 6.3: Write end-of-track report - -- [ ] **Step 1: Write the report** - -Create `docs/reports/TRACK_COMPLETION_any_type_componentization_20260621.md` covering: -- Executive summary (89 Any sites resolved; 5 dataclasses added; 3 new modules) -- The 5 candidates (per spec §4) -- Per-phase outcomes (sites migrated, tests added, commits) -- Verification commands + results -- Out of scope (211 Any sites remaining; Pattern 3/4/5 preserved) -- Follow-up tracks (any_type_componentization_phase2, openai_tools_dataclass_bridge) - -- [ ] **Step 2: Commit** - -```bash -git add docs/reports/TRACK_COMPLETION_any_type_componentization_20260621.md -git commit -m "docs(reports): TRACK_COMPLETION_any_type_componentization_20260621" -``` - -### Task 6.4: Archive + tracks.md update - -- [ ] **Step 1: Move track dir to archive** - -```bash -git mv conductor/tracks/any_type_componentization_20260621 conductor/tracks/archive/ -``` - -- [ ] **Step 2: Update tracks.md** - -Find the entry for `any_type_componentization_20260621` (added during track init; mark as `[x]` completed + move to Recently Completed section). - -- [ ] **Step 3: Final state.toml update + commit** - -Update `state.toml` to set all phases `completed` and the track status `completed`. - -```bash -git add -A -git commit -m "conductor(archive): ship any_type_componentization_20260621 to archive" -git notes add -m "TRACK COMPLETE: any_type_componentization_20260621. 5 fat-struct candidates promoted (89 sites); 3 new modules; 1 new audit script; 1 styleguide section. 211 Any sites remain (Patterns 3/4/5)" HEAD -``` - ---- - -## Self-Review (run after writing the plan) - -**1. Spec coverage check:** Each section in `spec.md` maps to a task in this plan. - -| Spec section | Plan coverage | -|---|---| -| §1 Overview + sequencing correction | n/a (background) | -| §2 Goals (A/B/C/D) | Phase 0 (B: JsonValue + styleguide §12) + Phases 1-5 (A: 5 candidates) + Phase 6 (C: registry) | -| §3 Architecture | n/a (design intent, in spec) | -| §4 Per-Phase Plan | Phases 0-6 in plan | -| §5 Audit Script as CI Gate | Phase 0 Task 0.2 | -| §6 Configuration | No new deps (consistent throughout) | -| §7 Testing Strategy | Each phase has its own test file; 48+ tests total | -| §8 Migration Rollout | 7 phases, ~50 commits | -| §9 Risks | Phase 3 Task 3.1 baseline snapshot + Task 3.6 SDK client verification | -| §10 Out of Scope | Pattern 3/4/5 preserved (Phase 3 Task 3.6 verification + Phase 5 Task 5.1 test) | -| §11 Decisions | Documented in spec | -| §12 See Also | n/a (references) | -| §13 Verification Criteria | Phase 6 Task 6.1 + 6.2 | - -**2. Placeholder scan:** Searched the plan for "TBD", "TODO", "fill in details" — none present in actionable steps. - -**3. Type consistency check:** Names used consistently: -- `ToolSpec` (defined Phase 1, used throughout) -- `ProviderHistory` (defined Phase 3, used throughout) -- `ChatMessage`, `UsageStats`, `ToolCall` (defined Phase 2) -- `WebSocketMessage` (defined Phase 5) -- `JsonValue` (defined Phase 0, used Phase 5) - -No naming drift. - -**4. Ambiguity check:** Step descriptions are concrete (exact file:line refs, full code blocks, explicit verification commands). - ---- - -## Execution Handoff - -Plan complete and saved to `conductor/tracks/any_type_componentization_20260621/plan.md`. - -**Two execution options:** - -1. **Subagent-Driven (recommended)** — Dispatch a fresh Tier 3 subagent per task, review between tasks, fast iteration. Best for ~50-commit tracks with clear per-task boundaries. - -2. **Inline Execution** — Execute tasks in this session using executing-plans, batch execution with checkpoints. Best for tight feedback loops when the user is watching. - -**Recommended for this track:** Subagent-Driven (Phase 3 alone has 15 tasks; per-task review prevents the largest ripple from cascading). - -**Blockers before execution can start:** -- `data_structure_strengthening_20260606` must merge to `master` (this track is `blocked_by` it per spec §1 / metadata.json) -- Tier 2 must first apply the data_structure_strengthening polish (5 must-fix items per Tier 1 review) on the tier2 branch - -**Then Tier 2 creates a new branch `tier2/any_type_componentization_20260621` from updated `master` and starts Phase 0.** \ No newline at end of file +# Plan: any_type_componentization_20260621 + +**Track:** Promote `dict[str, Any]` / `list[dict[str, Any]]` to `dataclass(frozen=True)` for the 5 fat-struct candidates identified by `docs/reports/ANY_TYPE_AUDIT_20260621.md` (89 of 300 Any sites; ~30%). + +**Spec:** `spec.md` (the WHY, the architecture, the per-phase intent). + +**State:** `state.toml` (the executable task ledger; 61 tasks across 7 phases; this plan mirrors the per-task breakdown). + +**Reference pattern:** `src/vendor_capabilities.py` (frozen dataclass + module-level `_REGISTRY` dict + factory functions). + +**Styleguides read (per agent mandate):** +- `conductor/code_styleguides/data_oriented_design.md` (canonical DOD; frozen dataclass is a data-first choice) +- `conductor/code_styleguides/error_handling.md` (Result[T] for `from_dict()` returns) +- `conductor/code_styleguides/type_aliases.md` (the 10 existing TypeAliases; this track adds `JsonPrimitive` + `JsonValue`) +- `conductor/code_styleguides/python.md` (1-space indent; frozen=True is preferred for data containers) +- `conductor/code_styleguides/feature_flags.md` (file presence for the new audit script) +- `conductor/code_styleguides/config_state_owner.md` (AppController owns config; audit scripts live in `scripts/`) +- `conductor/code_styleguides/chroma_cache.md` (no RAG work in this track) +- `conductor/code_styleguides/cache_friendly_context.md` (no cache work in this track) +- `conductor/code_styleguides/agent_memory_dimensions.md` (no memory-dim work in this track) +- `conductor/code_styleguides/rag_integration_discipline.md` (no RAG work in this track) +- `conductor/code_styleguides/knowledge_artifacts.md` (no knowledge work in this track) +- `conductor/code_styleguides/test_sandbox.md` (Layer 1 audit hook; tests must write to `tests/artifacts/`) +- `conductor/code_styleguides/workspace_paths.md` (test workspaces under `tests/artifacts/`) + +## Conventions (MUST follow) + +1. **Test runner:** `uv run python scripts/run_tests_batched.py` (NEVER direct `uv run pytest`). +2. **Default branch:** `master` (already on `tier2/any_type_componentization_20260621` branched from `origin/master`). +3. **Line endings:** preserve existing (CRLF stays CRLF, LF stays LF). +4. **Throw-away scripts:** write to `scripts/tier2/artifacts/any_type_componentization_20260621/` (NOT the base dir). +5. **Temp files:** all under `tests/artifacts/tier2_state/` or `tests/artifacts/tier2_failures/`; **NEVER USE APPDATA** (the bash deny rules enforce this). +6. **Tier 3 delegation:** heavy implementation goes to `uv run python scripts/mma_exec.py --role tier3-worker "[PROMPT]"` with exact WHERE/WHAT/HOW/SAFETY fields. Remind the worker: "use exactly 1-space indentation for Python code." + +## TDD protocol (per task) + +1. **RED:** write a failing test; run via `scripts/run_tests_batched.py`; confirm it fails with the expected message (not a typo or import error). +2. **GREEN:** implement the minimum to pass; run; confirm all green. If pass unexpectedly, call `record_red_failure` and check `should_give_up` via `scripts.tier2.failcount`. +3. **COMMIT:** `git add && git commit -m "..."` then `git notes add -m "Task: ..." `. +4. **PLAN:** append commit SHA to `state.toml` task entry; commit the plan update. + +## Phase breakdown (mirrors `state.toml`) + +### Phase 0 - Shared scaffolding (5 tasks; t0_1 to t0_5) +- `scripts/audit_dataclass_coverage.py` (NEW): mirror `audit_weak_types.py` design with `--strict` + `--baseline` + `--json` modes; counts anonymous `dict[str, Any]` and `Any`-typed params outside the promoted sites. +- `src/type_aliases.py` (MODIFIED): add `JsonPrimitive: TypeAlias` + `JsonValue: TypeAlias` (recursive; forward-refs for Python 3.11). +- `conductor/code_styleguides/type_aliases.md` (MODIFIED): add §12 "When to Promote `TypeAlias` to `dataclass`" section per spec §6.5. +- `scripts/audit_dataclass_coverage.baseline.json` (NEW): initial baseline = current count (post-track should be `300 - 89 = 211`). +- `tests/test_audit_dataclass_coverage.py` (NEW): 6+ tests mirroring `tests/test_audit_weak_types.py` structure. + +### Phase 1 - mcp_tool_specs (8 tasks; t1_1 to t1_8) +- `src/mcp_tool_specs.py` (NEW): `ToolParameter` + `ToolSpec` dataclasses + module-level `_REGISTRY` + `register()`/`get_tool_spec()`/`get_tool_schemas()`/`tool_names()` factory functions. +- `src/mcp_client.py` (MODIFIED): `MCP_TOOL_SPECS: list[dict[str, Any]]` → call `mcp_tool_specs.get_tool_schemas()`; `TOOL_NAMES` → `mcp_tool_specs.tool_names()`. +- `src/ai_client.py` (MODIFIED): 3 call sites use `mcp_client.TOOL_NAMES` → `mcp_tool_specs.tool_names()`. +- `tests/test_mcp_tool_specs.py` (NEW): 8+ tests (45 tools registered, dispatch, cross-module invariant vs `models.AGENT_TOOL_NAMES`). + +### Phase 2 - openai_schemas (9 tasks; t2_1 to t2_9) +- `src/openai_schemas.py` (NEW): `ToolCall` + `ToolCallFunction` + `ChatMessage` + `UsageStats` dataclasses. +- `src/openai_compatible.py` (MODIFIED): `NormalizedResponse.tool_calls: list[dict[str, Any]]` → `tuple[ToolCall, ...]`; the 4 `usage_*_tokens` fields → `usage: UsageStats`; `OpenAICompatibleRequest.messages` → `list[ChatMessage]`. +- `src/ai_client.py` (MODIFIED): `_send_grok` + `_send_minimax` + `_send_llama` (3 functions) construct `ChatMessage` and `UsageStats`. +- `tests/test_openai_schemas.py` (NEW): 10+ tests (round-trips for 4 roles, Result[T] error cases, raw_response remains `Any`). + +### Phase 3 - provider_state (15 tasks; t3_1 to t3_15; LARGEST) +- `src/provider_state.py` (NEW): `ProviderHistory` dataclass (messages + lock + append/get_all/replace_all/clear methods) + `_PROVIDER_HISTORIES: dict[str, ProviderHistory]` for 6 providers (anthropic/deepseek/minimax/qwen/grok/llama). +- `src/ai_client.py` (MODIFIED): 14 module globals (`_*_history`, `_*_history_lock`) → `_PROVIDER_HISTORIES` dict; ~27 call sites updated across `_send_()` functions. **SDK client holders (_gemini_chat, etc.) NOT touched (Pattern 3 preserved).** +- `tests/test_provider_state.py` (NEW): 10+ tests (thread safety, singleton, cleanup semantics). + +### Phase 4 - log_registry Session (8 tasks; t4_1 to t4_8) +- `src/log_registry.py` (MODIFIED): add `Session` + `SessionMetadata` dataclasses inline; `LogRegistry.data: dict[str, dict[str, Any]]` → `dict[str, Session]`. +- `src/session_logger.py` (MODIFIED): `open_session()`/`close_session()` consume `Session` dataclass. +- `src/log_pruner.py` (MODIFIED): `prune_old_logs()` consumes `Session` dataclass. +- `src/gui_2.py` (MODIFIED): Log Management panel reads `Session` typed. +- Tests: extend `tests/test_log_registry.py`. + +### Phase 5 - api_hooks WebSocketMessage (8 tasks; t5_1 to t5_8) +- `src/api_hooks.py` (MODIFIED): add `WebSocketMessage` dataclass; `broadcast(channel, payload: dict[str, Any])` → `broadcast(message: WebSocketMessage)`; `_serialize_for_api(obj: Any) -> JsonValue` (uses `JsonValue` TypeAlias from `src/type_aliases.py`). +- `_get_app_attr` / `_set_app_attr` signatures UNCHANGED (Pattern 4 preserved). +- Tests: extend `tests/test_api_hooks.py`. + +### Phase 6 - Verify + docs + archive (8 tasks; t6_1 to t6_8) +- Run `audit_weak_types.py --strict` → exit 0. +- Run `audit_dataclass_coverage.py --strict` → exit 0 (with post-track baseline = 211). +- Run `generate_type_registry.py --check` → exit 0 (5 new .md files appear). +- Run full 11-tier batched regression suite. +- Write `docs/reports/TRACK_COMPLETION_any_type_componentization_20260621.md`. +- `git mv conductor/tracks/any_type_componentization_20260621 conductor/tracks/archive/`. +- Update `conductor/tracks.md`. +- Final state.toml + checkpoint commit + git note. + +## Hard bans (3-layer enforcement) + +| Ban | Why | Replacement | +|---|---|---| +| `git push*` | Sandbox blocks push; user reviews + merges | (user runs merge) | +| `git checkout*` | Banned per Tier 2 conventions (added 2026-06-17) | `git switch -c` for new branches; `git switch` to switch | +| `git restore*` | Banned per AGENTS.md Critical Anti-Patterns | read-only inspection (`git show HEAD:path`) | +| `git reset*` | Banned | n/a | +| AppData writes | OS token + bash deny rules | `tests/artifacts/` only | + +## Failcount contract + +After every task commit, check `should_give_up` via `scripts.tier2.failcount`. Thresholds: +- 3 consecutive red-phase failures +- 3 consecutive green-phase failures +- 30 minutes with no progress (no commit, no green test) + +If `should_give_up` returns True, IMMEDIATELY stop. Call `write_failure_report` from `scripts.tier2.write_report` and print the report path. + +## Verification (per phase) + +Each phase ends with: +1. `uv run python scripts/audit_weak_types.py --strict` exits 0 (the alias convention intact). +2. `uv run python scripts/run_tests_batched.py --tier ` passes. +3. Phase checkpoint commit with git note summarizing per-task results. +4. State.toml update with per-task commit SHAs. +5. State.toml update commit (`conductor(plan): Mark phase N tasks complete`). + +## End-of-track report + +Per Tier 2 conventions: write `docs/reports/TRACK_COMPLETION_any_type_componentization_20260621.md` (follow `TRACK_COMPLETION_tier2_autonomous_sandbox_20260616.md` precedent). Update `conductor/tracks/any_type_componentization_20260621/state.toml` to `status = "completed"`. User reviews + merges. + +## Total scope (per spec §8) + +- **~50 atomic commits** +- **3 new src/ modules** (mcp_tool_specs, openai_schemas, provider_state) +- **6+ src/ files modified** (mcp_client, openai_compatible, ai_client, log_registry, session_logger, log_pruner, gui_2, api_hooks) +- **6 new test files** (one per phase) +- **1 new audit script + baseline** +- **1 styleguide update** (type_aliases.md §12) +- **Type registry auto-regenerated** (5 new .md files) +- **1 end-of-track report** \ No newline at end of file