Private
Public Access
0
0

conductor(plan): cruft_elimination_20260627 exhaustive Tier 3 execution contract

This commit is contained in:
2026-06-25 21:03:49 -04:00
parent 013bc3541d
commit f219616fc7
@@ -0,0 +1,877 @@
# Plan: cruft_elimination_20260627 (EXTREME DETAIL)
> **Tier 1 exhaustive plan — 2026-06-27.** This plan is the EXECUTABLE CONTRACT for Tier 2/Tier 3. Every task has exact file:line refs, exact before/after code, exact test commands, and explicit FIX-IF-FAILS steps. NEVER use `git restore`, `git checkout --`, `git reset`, or `git revert` (per AGENTS.md hard ban). NEVER use the word "REVERT" — always "MODIFY" or "FIX".
>
> **Prerequisites:** `type_alias_unfuck_20260626` SHIPPED (Phases 0-10 done; 67 `.get()` sites reduced to <15; all 12 per-aggregate dataclasses have `from_dict()` methods).
>
> **Baseline (measured 2026-06-27, master `b096a8be`):**
> - `Metadata: TypeAlias = dict[str, Any]` STILL exists at `src/type_aliases.py:6`
> - `hasattr(f, 'path')` checks: ~14 sites in `src/app_controller.py`
> - `hasattr(f, '...')` checks (entity dispatch): 14 sites
> - `Optional[T]` return types: ~25+ in `src/*.py`
> - `Any` parameter types: ~15+ in `src/*.py`
> - `dict[str, Any]` parameter types: ~20+ in `src/*.py`
> - `def _do_generate(self) -> tuple[str, Path, list[Metadata], ...]` — wrong return type at `src/app_controller.py:4006`
> - `self.files: List[models.FileItem]` declared but holds dicts (`src/app_controller.py:1996-2003`)
> - `flat_config(...)` returns `dict` not typed
> - `rag_engine.search()` returns `List[Dict]` not `List[RAGChunk]`
> - Effective codepaths: ~1e+21 (down from 4.014e+22 after unfuck)
>
> **Acceptance:** all 14 VCs from `conductor/tracks/cruft_elimination_20260627/spec.md` PASS. Effective codepaths < 1e+18 (4+ orders of magnitude drop from baseline 4.014e+22).
## §0 Pre-flight (Tier 2 runs before Tier 3 starts)
```bash
git checkout -b tier2/cruft_elimination_20260627
# 0.1 Clean working tree
git status --short
# Expect: no output (clean)
# 0.2 Capture baseline counts
git grep -cE "hasattr\(f, '(path|source_tier|content|role|model|id|status)'\)" -- 'src/*.py' > /tmp/before_hasattr.txt
# Expect: ~14 sites
git grep -cE "-> Optional\[" -- 'src/*.py' > /tmp/before_optional.txt
# Expect: ~25+ sites
git grep -cE "def .+\(.*: (Metadata|Any|dict\[str, Any\])" -- 'src/*.py' > /tmp/before_signatures.txt
# Expect: ~65+ sites
git grep -cE "def .+\(.*: Metadata" -- 'src/app_controller.py' 'src/gui_2.py' 'src/aggregate.py' 'src/multi_agent_conductor.py' > /tmp/before_metadata_params.txt
# Expect: ~30 sites
# 0.3 Confirm 7 audit gates pass --strict
uv run python scripts/audit_weak_types.py --strict
uv run python scripts/generate_type_registry.py --check
uv run python scripts/audit_main_thread_imports.py
uv run python scripts/audit_no_models_config_io.py
uv run python scripts/audit_code_path_audit_coverage.py --input-dir docs/reports/code_path_audit/latest --strict
uv run python scripts/audit_exception_handling.py --strict
uv run python scripts/audit_optional_in_3_files.py --strict
# All exit 0; note pre-existing failures
# 0.4 Confirm Metadata is STILL `dict[str, Any]` (the lazy-typing escape hatch)
git grep -n "Metadata:" src/type_aliases.py | head -3
# Expect: Metadata: TypeAlias = dict[str, Any] (line 6 — this is what we FIX in Phase 1)
# 0.5 Verify the 12 per-aggregate dataclasses all have `from_dict()` methods
uv run python -c "
from src.type_aliases import CommsLogEntry, HistoryMessage, ToolDefinition, SessionInsights, DiscussionSettings, CustomSlice, MMAUsageStats, ProviderPayload, UIPanelConfig, PathInfo
from src.openai_schemas import ToolCall, ChatMessage, UsageStats, NormalizedResponse
from src.models import Ticket, FileItem, ContextPreset
from src.rag_engine import RAGChunk
print('all from_dict methods:', all(hasattr(c, 'from_dict') for c in [CommsLogEntry, HistoryMessage, ToolDefinition, SessionInsights, DiscussionSettings, CustomSlice, MMAUsageStats, ProviderPayload, UIPanelConfig, PathInfo, ToolCall, ChatMessage, UsageStats, NormalizedResponse, Ticket, FileItem, ContextPreset, RAGChunk]))
"
# Expect: True
```
**STOP if any pre-existing failure is not in the baseline report. Report to user.**
## §Phase 1: Promote `Metadata` from `TypeAlias = dict[str, Any]` to a typed fat struct
**WHERE:** `src/type_aliases.py:6`
**Current state (line 6):**
```python
Metadata: TypeAlias = dict[str, Any]
```
**Task 1.1:** Replace with a `@dataclass(frozen=True, slots=True)` containing the wire-format fields observed at all `Metadata` access sites across `src/*.py`.
**Pattern (the fat struct):**
```python
@dataclass(frozen=True, slots=True)
class Metadata:
"""The wire-format boundary type. ONLY used at TOML/JSON parse functions.
Internal code uses componentized dataclasses (CommsLogEntry, FileItem, etc.)."""
# TOML/JSON wire keys observed in the codebase
paths: Metadata = field(default_factory=dict)
project: Metadata = field(default_factory=dict)
discussion: Metadata = field(default_factory=dict)
# Per-vendor chat message keys
role: str = ""
content: Any = None
tool_calls: Metadata = field(default_factory=list)
tool_call_id: str = ""
name: str = ""
# Session log / MMA telemetry keys
ts: str = ""
kind: str = ""
direction: str = ""
model: str = "unknown"
source_tier: str = "main"
error: str = ""
# MMA ticket keys
id: str = ""
description: str = ""
status: str = "todo"
depends_on: tuple = ()
manual_block: bool = False
# RAG result keys (top-level, not nested)
document: str = ""
path: str = ""
score: float = 0.0
# Tool definition + tool call keys
function: Metadata = field(default_factory=dict)
args: Metadata = field(default_factory=dict)
script: str = ""
output: str = ""
type: str = ""
description: str = ""
parameters: Metadata = field(default_factory=dict)
auto_start: bool = False
# File item keys
view_mode: str = "full"
custom_slices: Metadata = field(default_factory=list)
# Token usage keys
input_tokens: int = 0
output_tokens: int = 0
cache_read_input_tokens: int = 0
cache_creation_input_tokens: int = 0
# Generic pass-through (the boundary accepts arbitrary keys; from_dict filters)
metadata: Metadata = field(default_factory=dict)
def to_dict(self) -> dict[str, Any]:
return {k: v for k, v in self.__dict__.items() if v not in (None, "", [], {}, 0, 0.0, False) or k in _NON_NULL_FIELDS}
@classmethod
def from_dict(cls, raw: dict[str, Any]) -> "Metadata":
valid = {f.name for f in fields(cls)}
return cls(**{k: v for k, v in raw.items() if k in valid})
```
Add `_NON_NULL_FIELDS = {"model"}` at module top (these fields are always included even when default).
**HOW:** `manual-slop_py_update_definition` with `name="Metadata"`. Anchor on the existing `Metadata: TypeAlias = dict[str, Any]` line. Replace with the dataclass above.
**Add import:**
```python
from dataclasses import dataclass, field, fields
```
**SAFETY:**
```bash
uv run python -c "from src.type_aliases import Metadata; m = Metadata(role='user', content='hi'); print(m.role, m.content, m.model)"
# Expect: user hi unknown
uv run python -c "from src.type_aliases import Metadata; m = Metadata.from_dict({'role': 'user', 'unknown_key': 'x'}); print(m.role, m.model)"
# Expect: user unknown (unknown_key filtered)
uv run python -m pytest tests/test_type_aliases.py -x --timeout=60
# Expect: all pass
uv run python scripts/audit_weak_types.py --strict
# Expect: exit 0 (no new dict[str, Any] types)
```
**MODIFY-IF-FAILS:**
- If pytest fails: the dataclass has a field with the wrong type. Check the field type vs the constructor arg.
- If audit fails: a new `dict[str, Any]` field type was introduced. Replace with a specific type.
**COMMIT:** `refactor(type_aliases): promote Metadata from dict[str, Any] to typed fat struct`
**Commit message body MUST include:**
```
Phase 1: Metadata promotion
Before: 1 TypeAlias = dict[str, Any] site in src/type_aliases.py
After: 0 (replaced by @dataclass(frozen=True, slots=True))
Delta: -1 (expected: -1)
Metadata is now the typed fat struct at the wire boundary.
```
**GIT NOTE:** Metadata is now `@dataclass(frozen=True, slots=True)` with explicit fields covering all observed wire-format keys. Used ONLY at the literal TOML/JSON parse functions. Internal code uses componentized dataclasses.
## §Phase 2: Add `ProjectContext` dataclass for `flat_config`
**WHERE:**
- `src/project_manager.py:flat_config` — currently returns `dict[str, Any]`
- All consumers (search for `flat_config` calls in `src/app_controller.py` and `src/gui_2.py`)
**Task 2.1:** Add `ProjectContext` dataclass to `src/models.py` (next to `ProjectConfig`).
**Pattern:**
```python
@dataclass(frozen=True, slots=True)
class ProjectContext:
"""The flattened project context returned by project_manager.flat_config().
The TOML/JSON config is parsed to Metadata at the boundary, then
ProjectContext.from_dict() converts to this typed form."""
paths: Metadata = field(default_factory=dict)
project: Metadata = field(default_factory=dict)
discussion: Metadata = field(default_factory=dict)
files: Metadata = field(default_factory=dict)
screenshots: Metadata = field(default_factory=dict)
context_presets: Metadata = field(default_factory=dict)
rag: Metadata = field(default_factory=dict)
personas: Metadata = field(default_factory=dict)
mma: Metadata = field(default_factory=dict)
def to_dict(self) -> Metadata:
return dict(self.__dict__)
@classmethod
def from_dict(cls, raw: Metadata) -> "ProjectContext":
valid = {f.name for f in fields(cls)}
return cls(**{k: v for k, v in raw.items() if k in valid})
```
**Task 2.2:** Update `flat_config` in `src/project_manager.py`.
Read the current implementation:
```bash
git grep -nA 30 "def flat_config" -- 'src/project_manager.py'
```
Identify the dict keys it returns. Add them as fields to `ProjectContext`. Update the return type annotation.
**Pattern (return type + body):**
```python
def flat_config(self, ...) -> ProjectContext:
...
return ProjectContext.from_dict(raw_dict)
```
**Task 2.3:** Update consumers in `src/app_controller.py` and `src/gui_2.py`.
Search for `flat_config(` calls:
```bash
git grep -nE "flat_config\(" -- 'src/*.py'
```
For each consumer, replace `flat.get('key', default)` with `flat.key or default`. The `flat` variable becomes `ProjectContext` typed.
**Example:**
```python
# BEFORE:
flat = project_manager.flat_config(self.project, ...)
flat["files"] = copy.copy(flat.get("files", {}))
flat["files"]["paths"] = self.context_files
context_block += flat.get("screenshots", {}).get("paths", [])
# AFTER:
ctx = project_manager.flat_config(self.project, ...)
ctx_files = ProjectFiles(paths=self.context_files, base_dir=...)
ctx = dataclasses.replace(ctx, files=asdict(ctx_files))
context_block = ctx.screenshots.paths
```
(Read each site first; the actual replacement depends on the surrounding code.)
**HOW:** `manual-slop_edit_file` per site.
**SAFETY:**
```bash
git grep -nE "flat\.get\(" -- 'src/app_controller.py' 'src/gui_2.py' | wc -l
# Expect: 0
uv run python -m pytest tests/test_project_serialization.py tests/test_app_controller.py tests/test_gui_2.py -x --timeout=120
# Expect: all pass
```
**MODIFY-IF-FAILS:**
- If grep shows non-zero: search for missed sites. Add additional migrations.
- If pytest fails: STOP. Read the failure. Likely cause: `flat_config` returns dict in some paths, dataclass in others. Fix the return to be consistent.
**COMMIT:** `refactor(project_manager,app_controller,gui_2): introduce ProjectContext dataclass, type flat_config return`
**Commit message body MUST include:**
```
Phase 2: ProjectContext
Before: flat.get(...) sites in app_controller.py + gui_2.py
After: 0 (all replaced with attribute access on ProjectContext)
Delta: -N
```
## §Phase 3: Fix `self.files` in `src/app_controller.py` (FR4 row 1)
**WHERE:**
- `src/app_controller.py:1101` (declaration: `self.files: List[models.FileItem] = []`)
- `src/app_controller.py:1996-2003` (append paths: 3 branches, appends dict OR FileItem)
- `src/app_controller.py:3226-3233` (same pattern, second occurrence)
- `src/app_controller.py:2539` (`self.files.append(item)` — needs verification of `item` type)
**Task 3.1:** Replace the 3-branch append logic with explicit type checks + single `from_dict` call.
**Pattern (replacing `src/app_controller.py:1996-2003`):**
```python
# BEFORE:
self.files = []
for p in paths:
self.files.append(p) # ← appends raw dict
self.files.append(models.FileItem.from_dict(p)) # ← appends FileItem
self.files.append(models.FileItem(path=str(p))) # ← appends FileItem
# AFTER:
self.files = [models.FileItem.from_path(p) for p in paths]
```
Where `models.FileItem.from_path` is a new classmethod:
```python
@classmethod
def from_path(cls, p: str | Metadata | "FileItem") -> "FileItem":
if isinstance(p, cls):
return p
if isinstance(p, str):
return cls(path=p)
if isinstance(p, dict):
return cls.from_dict(p)
raise TypeError(f"FileItem.from_path: expected str, dict, or FileItem; got {type(p).__name__}")
```
Add this `from_path` classmethod to `src/models.py:FileItem` class.
**Task 3.2:** Same fix at `src/app_controller.py:3226-3233`.
**Task 3.3:** Remove `hasattr(f, 'path')` defensive checks throughout `src/app_controller.py`.
Affected sites (read each first):
- `src/app_controller.py:263``[f.path if hasattr(f, "path") else f.get("path") if isinstance(f, dict) else str(f) for f in controller.last_file_items]`
- `src/app_controller.py:1767``return [f.path if hasattr(f, 'path') else str(f) for f in self.files]`
- `src/app_controller.py:1771``old_files = {f.path: f for f in self.files if hasattr(f, 'path')}`
- `src/app_controller.py:2536``next((f for f in self.files if (f.path if hasattr(f, "path") else str(f)) == file_path), None)`
- `src/app_controller.py:3129,3182``file_items_as_dicts = [{"path": f.path if hasattr(f, "path") else str(f)} for f in self.files]`
**Pattern (per site):**
```python
# BEFORE:
return [f.path if hasattr(f, 'path') else str(f) for f in self.files]
# AFTER:
return [f.path for f in self.files]
```
After Phase 3, `self.files` is GUARANTEED `List[FileItem]`. Every `hasattr(f, 'path')` check is redundant. Remove it.
**SAFETY:**
```bash
git grep -nE "hasattr\(f, 'path'\)" -- 'src/app_controller.py' | wc -l
# Expect: 0
uv run python -m pytest tests/test_file_item_model.py tests/test_app_controller.py tests/test_custom_slices_annotations.py tests/test_gui_2.py -x --timeout=120
# Expect: all pass
```
**MODIFY-IF-FAILS:**
- If grep shows non-zero: search for missed sites. The pattern is `hasattr(f, 'path')` or `hasattr(f, "path")`.
- If pytest fails: STOP. Read the failure. Likely cause: a dict is still being added to `self.files` somewhere. Trace the path.
**COMMIT:** `refactor(app_controller): self.files is now List[FileItem]; remove all hasattr defensive checks`
**Commit message body MUST include:**
```
Phase 3: self.files type guarantee
Before: 7 hasattr(f, 'path') sites in src/app_controller.py
After: 0 (self.files is now List[FileItem] guaranteed)
Delta: -7
```
## §Phase 4: Fix `_do_generate` return type (FR4 row 2)
**WHERE:**
- `src/app_controller.py:4006``def _do_generate(self) -> tuple[str, Path, list[Metadata], str, str]:`
- `src/gui_2.py` callers — find all `_do_generate(` calls
**Task 4.1:** Read the current return statement at `src/app_controller.py:4051`:
```python
return full_md, path, file_items, stable_md, discussion_text
```
The `file_items` is `List[FileItem]` (from `aggregate.run`'s return). The return type annotation is wrong.
**Pattern:**
```python
# BEFORE:
def _do_generate(self) -> tuple[str, Path, list[Metadata], str, str]:
...
return full_md, path, file_items, stable_md, discussion_text
# AFTER:
def _do_generate(self) -> tuple[str, Path, list[FileItem], str, str]:
...
return full_md, path, file_items, stable_md, discussion_text
```
**Task 4.2:** Update `src/gui_2.py` callers.
Search for `_do_generate(`:
```bash
git grep -nE "_do_generate\(" -- 'src/gui_2.py'
```
For each caller, the receiver variable is now `list[FileItem]`. Replace `.get('path', 'attachment')` accesses (if any) with `f.path` direct access.
**SAFETY:**
```bash
git grep -nE "list\[Metadata\]" -- 'src/app_controller.py' | wc -l
# Expect: 0 (was: 1 at line 4006)
uv run python -m pytest tests/test_context_composition_decoupled.py tests/test_tiered_aggregation.py tests/test_gui_2.py -x --timeout=120
# Expect: all pass
```
**MODIFY-IF-FAILS:**
- If grep shows non-zero: search for the type annotation. Fix.
- If pytest fails: STOP. Likely cause: `aggregate.run` returns `List[Dict]` in some paths. Trace.
**COMMIT:** `refactor(app_controller,gui_2): _do_generate returns list[FileItem], not list[Metadata]`
**Commit message body MUST include:**
```
Phase 4: _do_generate return type
Before: 1 list[Metadata] annotation at src/app_controller.py:4006
After: 0 (changed to list[FileItem])
Delta: -1
```
## §Phase 5: Fix `rag_engine.search()` return type (FR4 row 7)
**WHERE:**
- `src/rag_engine.py:367``def search(self, ...) -> List[Dict[str, Any]]:`
- 3 consumers: `src/aggregate.py:3259`, `src/app_controller.py:251`, `src/app_controller.py:4162`
**Task 5.1:** Change `rag_engine.search()` return type.
**Read first:**
```bash
git grep -nA 20 "def search" -- 'src/rag_engine.py'
```
**Pattern (the wire format mismatch):**
The wire format from the RAG store has `metadata.path` nested (or `metadata.source`); the `RAGChunk` dataclass has `path` at top-level. The `from_dict` classmethod must normalize:
```python
@classmethod
def from_dict(cls, raw: dict[str, Any]) -> "RAGChunk":
if "metadata" in raw and isinstance(raw.get("metadata"), dict):
meta = raw["metadata"]
return cls(
document=raw.get("document", "") or meta.get("document", ""),
path=meta.get("path", "") or meta.get("source", "") or raw.get("path", ""),
score=1.0 - float(raw.get("distance", 0.0)),
metadata=meta,
)
valid = {f.name for f in fields(cls)}
return cls(**{k: v for k, v in raw.items() if k in valid})
```
(Already implemented per Phase 0 of metadata_promotion; verify it handles the wire format.)
**Change `search` return type:**
```python
# BEFORE:
def search(self, ...) -> List[Dict[str, Any]]:
# AFTER:
def search(self, ...) -> List[RAGChunk]:
...
return [RAGChunk.from_dict(raw) for raw in raw_results]
```
**Task 5.2:** Update 3 consumers.
```python
# BEFORE:
context_block += f"### Chunk {i+1} (Source: {path})\n{chunk.get('document', '')}\n\n"
# AFTER:
context_block += f"### Chunk {i+1} (Source: {path})\n{chunk.document}\n\n"
```
**SAFETY:**
```bash
git grep -nE "chunk\.get\('document'," -- 'src/aggregate.py' 'src/app_controller.py' 'src/ai_client.py' | wc -l
# Expect: 0
uv run python -m pytest tests/test_rag_engine.py tests/test_rag_phase4_final_verify.py tests/test_rag_chunk.py -x --timeout=120
# Expect: all pass
```
**MODIFY-IF-FAILS:**
- If grep shows non-zero: search for missed sites.
- If pytest fails: STOP. The `RAGChunk.from_dict()` may not handle all wire format edge cases. Add more normalization logic.
**COMMIT:** `refactor(rag_engine,aggregate,app_controller): rag_engine.search returns List[RAGChunk]`
**Commit message body MUST include:**
```
Phase 5: RAGChunk return type
Before: 1 List[Dict[str, Any]] at src/rag_engine.py + 3 chunk.get('document',...) consumers
After: 0 (rag_engine.search returns List[RAGChunk] directly)
Delta: -1 + -3 = -4 sites
```
## §Phase 6: Eliminate `Optional[T]` returns (FR5)
**WHERE:** Search all `src/*.py` for `-> Optional[`:
```bash
git grep -nE "-> Optional\[" -- 'src/*.py'
```
For each `Optional[T]` return:
**Pattern (the rule per `error_handling.md`):**
```python
# BAD:
def find_ticket(self, id: str) -> Optional[Ticket]:
for t in self.active_tickets:
if t.id == id: return t
return None
# GOOD (preferred — NIL_T sentinel):
def find_ticket(self, id: str) -> Ticket:
for t in self.active_tickets:
if t.id == id: return t
return NIL_TICKET # zero-initialized frozen dataclass; safe to read fields
# ALSO GOOD (Result pattern, when caller needs to know success/failure):
def find_ticket(self, id: str) -> Result[Ticket]:
for t in self.active_tickets:
if t.id == id: return Result(data=t)
return Result(data=NIL_TICKET, errors=[ErrorInfo(kind=ErrorKind.NOT_FOUND, ...)])
```
**Required additions to `src/type_aliases.py` (NIL_T sentinels):**
```python
# Add to src/type_aliases.py after the existing dataclasses:
NIL_COMMS_LOG_ENTRY = CommsLogEntry()
NIL_HISTORY_MESSAGE = HistoryMessage()
NIL_TICKET = Ticket(id="", description="", status="missing", manual_block=False)
NIL_FILE_ITEM = FileItem(path="")
NIL_TOOL_CALL = ToolCall(id="", function=ToolCallFunction(name="", arguments=""))
NIL_CHAT_MESSAGE = ChatMessage(role="", content="")
NIL_USAGE_STATS = UsageStats(input_tokens=0, output_tokens=0)
NIL_RAG_CHUNK = RAGChunk()
NIL_MMA_USAGE_STATS = MMAUsageStats()
NIL_SESSION_INSIGHTS = SessionInsights()
NIL_DISCUSSION_SETTINGS = DiscussionSettings()
NIL_CUSTOM_SLICE = CustomSlice()
NIL_PROVIDER_PAYLOAD = ProviderPayload()
NIL_UI_PANEL_CONFIG = UIPanelConfig()
NIL_PATH_INFO = PathInfo()
NIL_TOOL_DEFINITION = ToolDefinition()
```
**Sites to fix (categorized by the kind of `Optional[T]`):**
Per-file. Read each site first. Apply the pattern above.
**SAFETY:**
```bash
git grep -cE "-> Optional\[" -- 'src/*.py'
# Expect: 0
uv run python scripts/audit_optional_in_3_files.py --strict
# Expect: exit 0 (the 3 refactored files already have it)
# (Note: this script only checks 3 files; the broader check is the grep above)
uv run python -m pytest tests/ -x --timeout=120 -q 2>&1 | tail -5
# Expect: 10/11 batched tiers PASS
```
**MODIFY-IF-FAILS:**
- If grep shows non-zero: search for missed sites. Each site needs explicit type replacement.
- If pytest fails: STOP. Likely cause: a consumer had `if x is None: ...` checks that no longer apply after the type changed. Update consumers.
**COMMIT:** `refactor(*): eliminate Optional[T] returns; add NIL_T sentinels`
**Commit message body MUST include:**
```
Phase 6: Optional[T] elimination
Before: N -> Optional[...] annotations across src/*.py
After: 0 (replaced with NIL_T sentinels or Result[T])
Delta: -N
```
## §Phase 7: Eliminate `Any` and `dict[str, Any]` from internal function signatures (FR6)
**WHERE:** Search all `src/*.py` for `Any` and `dict[str, Any]` in function signatures:
```bash
git grep -nE "def .+\(.*: (Any|dict\[str, Any\])" -- 'src/*.py'
```
**Boundary function exception:** functions that take wire input (TOML/JSON parsing) may keep `dict[str, Any]` with a comment explaining it's the boundary. Examples:
```python
# Boundary function (OK):
def _parse_wire_payload(raw: dict[str, Any]) -> ChatMessage:
"""Boundary: parse JSON wire dict to typed ChatMessage. ONLY called from src/api_hooks.py."""
return ChatMessage.from_dict(raw)
# Internal function (BANNED):
def process_comms_entry(self, entry: dict[str, Any]) -> None: # ← FIX
...
```
**Pattern (per site):**
```python
# BEFORE:
def process_comms_entry(self, entry: dict[str, Any]) -> None:
...
# AFTER:
def process_comms_entry(self, entry: CommsLogEntry) -> None:
...
```
**SAFETY:**
```bash
git grep -cE "def .+\(.*: (Any|dict\[str, Any\])" -- 'src/app_controller.py' 'src/gui_2.py' 'src/aggregate.py' 'src/multi_agent_conductor.py' 'src/mcp_client.py' 'src/ai_client.py' 'src/rag_engine.py' 'src/models.py'
# Expect: 0 (in non-boundary files)
git grep -cE "def .+\(.*: dict\[str, Any\]" -- 'src/api_hooks.py' 'src/project_manager.py' 'src/session_logger.py'
# Expect: count of boundary functions (small, documented)
uv run python -m pytest tests/ -x --timeout=120 -q 2>&1 | tail -5
# Expect: 10/11 batched tiers PASS
```
**MODIFY-IF-FAILS:**
- If grep shows non-zero in internal files: classify the site. If it's a real internal function, type the parameter. If it's a boundary function, add a `"""Boundary: ..."""` docstring.
- If pytest fails: STOP. A signature change broke a caller. Update the caller.
**COMMIT:** `refactor(*): eliminate Any and dict[str, Any] from internal function signatures`
**Commit message body MUST include:**
```
Phase 7: Any + dict[str, Any] elimination
Before: N function signatures with Any or dict[str, Any] in internal files
After: 0 (all replaced with typed dataclasses)
Delta: -N
Boundary functions (TOML/JSON parse) retain dict[str, Any] with explicit docstrings.
```
## §Phase 8: Re-measure + verification
```bash
# All cruft counts 0
git grep -cE "hasattr\(f, '(path|source_tier|content|role|model|id|status)'\)" -- 'src/*.py'
# Expect: 0
git grep -cE "-> Optional\[" -- 'src/*.py'
# Expect: 0
git grep -cE "def .+\(.*: (Any|dict\[str, Any\])" -- 'src/app_controller.py' 'src/gui_2.py' 'src/aggregate.py' 'src/multi_agent_conductor.py' 'src/mcp_client.py' 'src/ai_client.py' 'src/rag_engine.py' 'src/models.py'
# Expect: 0
git grep -cE "def .+\(.*: Metadata" -- 'src/app_controller.py' 'src/gui_2.py' 'src/aggregate.py' 'src/multi_agent_conductor.py'
# Expect: 0
# Effective codepaths drops
uv run python -c "
import sys
sys.path.insert(0, 'scripts/code_path_audit')
sys.path.insert(0, 'src')
from code_path_audit import build_pcg
from code_path_audit_ssdl import count_branches_in_function
pcg = build_pcg('src').data
metadata_consumers = pcg.consumers.get('Metadata', [])
total = sum(2 ** count_branches_in_function(f, 'src') for f in metadata_consumers)
print(f'Post-track effective codepaths: {total:.3e} (baseline 4.014e+22)')
"
# Expect: < 1e+18
# 7 audit gates pass
uv run python scripts/audit_weak_types.py --strict
uv run python scripts/generate_type_registry.py --check
uv run python scripts/audit_main_thread_imports.py
uv run python scripts/audit_no_models_config_io.py
uv run python scripts/audit_code_path_audit_coverage.py --input-dir docs/reports/code_path_audit/latest --strict
uv run python scripts/audit_exception_handling.py --strict
uv run python scripts/audit_optional_in_3_files.py --strict
# Batched tests
uv run python scripts/run_tests_batched.py
# Expect: 10/11 PASS
```
**MODIFY-IF-FAILS:**
- If effective codepaths is still > 1e+18: search for `hasattr(...)` or `isinstance(...)` chains. Each one is a branch.
- If audit gates fail: STOP. Read which audit failed.
## §Phase 9: Boundary layer audit + documentation
```bash
git grep -nE "Metadata" -- 'src/*.py' > /tmp/metadata_usages.txt
wc -l /tmp/metadata_usages.txt
# Expect: ~30-40 (only boundary files)
git grep -nE "Metadata" -- 'src/api_hooks.py' 'src/project_manager.py' 'src/session_logger.py' 'src/mcp_client.py' 'src/preset*.py' 'src/personas.py' | wc -l
# Expect: ~25 (the boundary uses)
git grep -nE "Metadata" -- 'src/app_controller.py' 'src/gui_2.py' 'src/aggregate.py' 'src/multi_agent_conductor.py' | wc -l
# Expect: 0
```
Write `docs/reports/boundary_layer_20260628.md`:
```markdown
# Boundary Layer Audit (cruft_elimination_20260627)
## Metadata usage per file
| File | Count | Classification | Justification |
|---|---|---|---|
| src/api_hooks.py | ~10 | BOUNDARY | HTTP entry; receives raw JSON |
| src/project_manager.py | ~5 | BOUNDARY | TOML config loader |
| src/session_logger.py | ~3 | BOUNDARY | JSON-L log writer |
| src/preset*.py | ~3 | BOUNDARY | TOML preset loader |
| src/personas.py | ~2 | BOUNDARY | TOML persona loader |
| src/mcp_client.py | ~2 | BOUNDARY | MCP wire protocol |
| (any internal file) | 0 | INTERNAL | BANNED — internal functions take typed dataclasses |
## Why this is the boundary
`Metadata` is the typed fat struct for the wire schema. It's used ONLY at:
- TOML config loaders (`tomllib.load()``Metadata.from_dict(...)`)
- JSON wire parsers (`json.loads()``Metadata.from_dict(...)`)
- Vendor SDK response parsers (after parsing the SDK's response)
Every consumer of these boundary functions IMMEDIATELY converts to a componentized dataclass (ProjectContext, CommsLogEntry, etc.) via `from_dict()`.
## Per-site justification
[list every Metadata usage with the function name + justification]
```
**COMMIT:** `docs(audit): boundary layer audit for cruft_elimination_20260627`
**Commit message body MUST include:**
```
Phase 9: Boundary layer audit
Before: Metadata scattered across N files
After: Metadata ONLY at boundary layer (2-3 functions per boundary file)
Delta: -N internal usages; +0 boundary usages (the boundary was already correct)
```
## §Acceptance Criteria (Definition of Done)
| # | Criterion | Verification |
|---|---|---|
| VC1 | `Metadata` is `@dataclass(frozen=True, slots=True)` (typed fat struct) | `git grep -A 1 "^class Metadata" src/type_aliases.py` shows `@dataclass(frozen=True, slots=True)` |
| VC2 | Zero `TypeAlias = dict[str, Any]` for Metadata | `git grep "^Metadata: TypeAlias" src/type_aliases.py` returns nothing |
| VC3 | Zero `dict[str, Any]` parameter types in internal files | `git grep -cE "def .+\(.*: dict\[str, Any\]" -- 'src/app_controller.py' 'src/gui_2.py' 'src/aggregate.py' 'src/multi_agent_conductor.py' 'src/mcp_client.py' 'src/ai_client.py' 'src/rag_engine.py' 'src/models.py'` returns 0 |
| VC4 | Zero `Any` parameter types in internal files | same grep with `: Any` returns 0 |
| VC5 | Zero `Optional[T]` return types | `git grep -cE "-> Optional\[" -- 'src/*.py'` returns 0 |
| VC6 | Zero `hasattr(f, ...)` entity dispatch checks | `git grep -cE "hasattr\(f, '(path\|source_tier\|content\|role\|model\|id\|status)'\)" -- 'src/*.py'` returns 0 |
| VC7 | `self.files` is always `List[FileItem]` | The 7 `hasattr(f, 'path')` sites in `src/app_controller.py` are removed; `self.files.append(...)` paths use `FileItem.from_path(...)` |
| VC8 | `flat_config` returns typed `ProjectContext` | New dataclass exists; return type fixed |
| VC9 | `rag_engine.search()` returns `List[RAGChunk]` | Return type fixed; 3 consumers updated |
| VC10 | All 7 audit gates pass `--strict` | All exit 0 |
| VC11 | 10/11 batched test tiers PASS | `scripts/run_tests_batched.py` → 10/11 |
| VC12 | Effective codepaths < 1e+18 | 4+ orders of magnitude drop |
| VC13 | Boundary layer audit written | `docs/reports/boundary_layer_20260628.md` exists |
| VC14 | The 12 per-aggregate dataclasses used at their specific paths | Direct attribute access everywhere |
## §Tier 2 / Tier 3 Hard Rules
1. **NEVER use `git restore`, `git checkout --`, `git reset`, or `git revert`.** Per AGENTS.md hard ban. NEVER use the word "REVERT" — always "MODIFY" or "FIX". If something is wrong, add more migrations or amend the commit. Do NOT throw away work.
2. **NEVER introduce `dict[str, Any]`, `Any`, or `Optional[T]` in non-boundary code.** The boundary is 2-3 functions per file. Internal code uses typed dataclasses.
3. **NEVER use `hasattr()` for entity type dispatch.** The type system guarantees the entity type. Use `isinstance()` against a typed Union, or refactor so no dispatch is needed.
4. **NEVER classify a phase as "no-op".** Each phase has work; do the work. If the work was already done by a previous attempt, verify it's done correctly and amend the commit.
5. **NEVER add comments to source code.** Per AGENTS.md. Documentation lives in `/docs`.
6. **NEVER use the native `edit` tool on Python files.** Use `manual-slop_edit_file`, `manual-slop_py_update_definition`, `manual-slop_py_add_def`, or `manual-slop_set_file_slice`.
7. **NEVER create new `src/<thing>.py` files.** Per AGENTS.md.
8. **NEVER skip a failing test with `@pytest.mark.skip`.** Fix the bug.
9. **NEVER exceed 5 nesting levels.** Extract to functions.
10. **NEVER modify `src/code_path_audit*.py`.** The audit infrastructure is correct.
11. **NEVER promote `Metadata: TypeAlias = dict[str, Any]`.** It's a typed fat struct (the boundary type). The TypeAlias is BANNED.
12. **STOP AND ASK if any site's variable type is unclear.** Write a 1-sentence question. Wait for the user. Do not invent a reconciliation.
13. **If a commit breaks more than 2 tests, STOP.** Read the failures. Identify the root cause. Fix the commit. Do not ship broken state.
## §Per-Phase Tier 2 Review Checklist
Before approving each phase, Tier 2 verifies:
1. The commit message has "Before: N, After: M, Delta: -K" with K matching the planned count.
2. The relevant `git grep` count decreased by exactly the planned K.
3. The relevant `pytest` files pass.
4. No audit gate regressed.
5. The batched test suite still passes 10/11 tiers.
6. No "no-op" or "REVERT" or "skipped" in the commit message.
If any check fails: **DO NOT APPROVE.** Tell Tier 3 what to fix. Tier 3 fixes the migration and re-commits.
## §Anti-Pattern Guard (per AGENTS.md)
If you observe any of these patterns in your own work, STOP and re-read AGENTS.md:
1. **The Deduction Loop**: running a test 4+ times in one investigation.
2. **The Report-Instead-of-Fix Pattern**: writing a 200-line status report instead of fixing.
3. **The Scope-Creep Track-Doc Pattern**: writing a 5-phase spec for a 1-line fix.
4. **The Inherited-Cruft Pattern**: trying to "fix" a broken file from a previous agent.
5. **No Diagnostic Noise in Production**: `sys.stderr.write` lines in `src/*.py`.
6. **The "I Am Not Going To Attempt Another Fix" Surrender**: only after the 5-step protocol.
7. **The Verbose-Commit-Message Pattern**: commit messages > 15 lines.
8. **The Isolated-Pass Verification Fallacy**: verifying in isolation but not in batch.
9. **The Workspace-Path Drift Pattern**: using `/tmp` or env vars for test paths.
10. **The No-Op Classification Shortcut**: marking phases complete without doing the work. (banned by Hard Rule #4)
## §Tier 2 Invitation Prompt
Use this prompt to invoke Tier 2:
```
Track: cruft_elimination_20260627 (branch: tier2/cruft_elimination_20260627).
This is the FINAL track in the metadata type-promotion chain. The previous track (type_alias_unfuck_20260626) introduced a NEW cruft: defensive isinstance() checks at function bodies. The user explicitly rejected this pattern: "every conditional check is more execution noise and tech debt."
Read the EXHAUSTIVE plan at conductor/tracks/cruft_elimination_20260627/plan.md (this file).
HARD RULES (NON-NEGOTIABLE):
1. NO dict[str, Any], Any, or Optional[T] in non-boundary code. The boundary is 2-3 functions per file.
2. NO hasattr() for entity type dispatch. The type system guarantees the entity type.
3. NO isinstance() defensive checks at function bodies. The boundary layer does from_dict() once.
4. NEVER use git restore, git checkout --, git reset, or git revert. NEVER use the word "REVERT" — always "MODIFY" or "FIX". If something is wrong, add more migrations or amend the commit.
5. NO no-op classifications. Each phase has work; do the work.
6. NO new src/<thing>.py files. NO comments in src/. NO @pytest.mark.skip.
PER-PHASE HARD GUARD:
Each phase commit message MUST include:
Phase N: <name>
Before: N <pattern> sites
After: 0 (or expected)
Delta: -N
If delta != expected, FIX the migration. Don't blow it away.
START:
git log --oneline -10
git checkout -b tier2/cruft_elimination_20260627
git grep -nE "hasattr\(f, 'path'\)" -- 'src/app_controller.py' | wc -l
git grep -nE "Metadata: TypeAlias = dict\[str, Any\]" -- 'src/type_aliases.py' | wc -l
git grep -nE "-> Optional\[" -- 'src/*.py' | wc -l
# Read the plan
cat conductor/tracks/cruft_elimination_20260627/plan.md
# Run pre-flight (Section §0)
# Execute Phases 1-9
```
## §See also
- `conductor/tracks/cruft_elimination_20260627/spec.md` — the track spec
- `conductor/tracks/type_alias_unfuck_20260626/spec.md` — the previous track
- `conductor/tracks/type_alias_unfuck_20260626/plan.md` — the previous track's plan
- `conductor/code_styleguides/data_oriented_design.md` §8.5 (The Python Type Promotion Mandate) — the canonical mandate
- `conductor/code_styleguides/python.md` §17 (Banned Patterns — LLM Default Anti-Patterns) — the cheatsheet
- `conductor/code_styleguides/type_aliases.md` — the type convention
- `conductor/code_styleguides/error_handling.md``Result[T]` + `NIL_T` convention
- `conductor/product-guidelines.md` "Core Value" — the value statement
- `docs/reports/FOLLOWUP_metadata_promotion_20260624.md` — the prior Tier 1 review (the root cause analysis)
- `src/type_aliases.py` — the 12 per-aggregate dataclasses (now with `from_dict()`)
- `src/models.py:533``FileItem` (canonical in-module dataclass)
- `src/models.py:302``Ticket` (canonical in-module dataclass)
- `src/openai_schemas.py``ToolCall`, `ChatMessage`, `UsageStats`, `NormalizedResponse`
- `src/rag_engine.py``RAGChunk` (added by `metadata_promotion_20260624`)
- `conductor/AGENTS.md` — hard bans (NEVER use `git restore`, `git checkout --`, `git reset`, `git revert`)