# Track Specification: metadata_promotion_20260624 > **Status:** ACTIVE — corrected 2026-06-25 (Tier 1 audit). The original spec (commit `e50bebdd`, 2026-06-25) proposed a single `@dataclass(frozen=True, slots=True) Metadata` with ~200 fields shared across all 5 sub-aggregates. That proposal was REJECTED on 2026-06-25 (user direction): the 5 sub-aggregates are distinct concepts with distinct field sets; lifting them into one mega-dataclass hides the type information that direct field access is supposed to reveal. The corrected design promotes each sub-aggregate to its OWN dataclass with its OWN fields. See `docs/reports/PLANNING_CORRECTION_metadata_promotion_20260625.md` for the full rationale. ## Overview Promotes the 5 distinct sub-aggregates (`CommsLogEntry`, `HistoryMessage`, `FileItem`, `ToolDefinition`, `ToolCall`) to their own typed `@dataclass(frozen=True, slots=True)` classes (or reuses the existing typed dataclasses where they already exist: `models.FileItem`, `openai_schemas.ToolCall`), then migrates the 107 `.get('key', ...)` + 106 subscript `['key']` access sites on those aggregates to direct field access (`entry.ts`, `t.depends_on`, `chunk.document`). `Metadata: TypeAlias = dict[str, Any]` is preserved as the catch-all for **truly collapsed codepaths** (generic JSON parsing at wire boundaries, `manual_slop.toml` project config, polymorphic containers where the element type is genuinely unknown) and is NOT promoted to a shared mega-dataclass. The combinatoric explosion (`4.01e22` effective codepaths) is addressed by **per-aggregate type promotion**: each known concept gets its own dataclass with its own fields, the `.get()` / `[]` runtime type-dispatch collapses at the source, and the audit's branch count drops per consumer function. ## Current State Audit (master `dc397db7`, measured 2026-06-25) | Metric | Value | Source | |---|---:|---| | `Metadata` consumers in `src/` | **695** | `scripts/code_path_audit.build_pcg` | | Top consumer files | `app_controller.py: 123`, `mcp_client.py: 94`, `ai_client.py: 73`, `gui_2.py: 44`, `models.py: 29` | `Counter` over `pcg.consumers['Metadata']` | | Total branches in Metadata consumers | 3,454 | `scripts/code_path_audit_ssdl.count_branches_in_function` | | **Effective codepaths (the 4.01e22)** | **4.014e+22** | `compute_effective_codepaths` | | `.get('key', ...)` access sites (all sub-aggregates) | 107 | `git grep` in `src/` | | `['key']` subscript access sites | 106 | `git grep` in `src/` | | `is None` / `== None` / `!= None` sites | 106 | `git grep` in `src/` (mostly unrelated to Metadata) | | TypeAlias chain (current state, before this track) | `Metadata: dict[str, Any]`; `CommsLogEntry: Metadata`; `HistoryMessage: Metadata`; `FileItem: "models.FileItem"`; `ToolDefinition: Metadata`; `ToolCall: "openai_schemas.ToolCall"` | `src/type_aliases.py` | | Existing per-aggregate dataclasses | `models.Ticket` (15 fields), `models.FileItem` (10 fields), `models.Track` (3 fields), `openai_schemas.ToolCall` (3 fields), `openai_schemas.ChatMessage` (5 fields), `openai_schemas.UsageStats` (4 fields), `openai_schemas.ToolCallFunction` (2 fields), `openai_schemas.NormalizedResponse` (4 fields), `vendor_capabilities.VendorCapabilities` (22 fields) | `git grep "^class .*(dataclass\|frozen=True)" src/` | | Missing per-aggregate dataclasses | `CommsLogEntry`, `HistoryMessage`, `ToolDefinition`, `RAGChunk`, `SessionInsights`, `DiscussionSettings`, `CustomSlice`, `MMAUsageStats`, `ProviderPayload`, `UIPanelConfig`, `ContextPreset` (full schema), `PathInfo` | actual access patterns from `git grep` on `src/` | ### Why the corrected design (per-aggregate dataclasses) — not one mega-dataclass The 107 `.get('key', default)` and 106 `['key']` access sites in `src/` span **at least 12 distinct aggregates**, not 5. A sampling of the actual access patterns: | Access pattern | Site | Aggregate it actually represents | |---|---|---| | `item.get('custom_slices', [])`, `item.get('content', '')` | `src/aggregate.py:418,421` | **FileItem** (per-file curation) | | `fi.get('path', 'attachment')` | `src/ai_client.py:2565,2807,2898` | **FileItem** | | `chunk.get('document', '')` | `src/aggregate.py:3259`, `src/app_controller.py:251,4162` | **RAGChunk** (RAG retrieval result) | | `entry.get('source_tier', 'main')`, `entry.get('model', 'unknown')` | `src/app_controller.py:2277,2302,2310` | **CommsLogEntry** (AI comms log) | | `u.get('input_tokens', 0)`, `u.get('output_tokens', 0)` | `src/app_controller.py:2304-2309` | **UsageStats** (per-call token usage) | | `t.get('id', '')`, `t.get('depends_on', [])`, `t.get('manual_block', False)`, `t.get('status')` | `src/gui_2.py:1366-1438` | **Ticket** (MMA ticket — already a dataclass) | | `stats.get('model', 'unknown')`, `stats.get('input', 0)`, `stats.get('output', 0)` | `src/gui_2.py:2199-2201,2216` | **MMAUsageStats** (per-tier rollup) | | `insights.get('total_tokens', 0)`, `insights.get('call_count', 0)`, `insights.get('burn_rate', 0)`, `insights.get('session_cost', 0)`, `insights.get('completed_tickets', 0)`, `insights.get('efficiency', 0)` | `src/gui_2.py:4926-4931` | **SessionInsights** (overall session stats) | | `entry.get('temperature', 0.7)`, `entry.get('top_p', 1.0)`, `entry.get('max_output_tokens', 0)` | `src/gui_2.py:3535` | **DiscussionSettings** (per-turn settings) | | `slc.get('tag', '')`, `slc.get('comment', '')` | `src/gui_2.py:4048-4054` | **CustomSlice** (visual slice editor) | | `preset.get('files', [])`, `preset.get('screenshots', [])` | `src/gui_2.py:4184-4185` | **ContextPreset** (file composition) | | `payload.get('script')`, `payload.get('args', {})`, `payload.get('output', '')`, `payload.get('content', '')` | `src/app_controller.py:2274,2287` | **ProviderPayload** (script-execution payload) | | `self.project.get('paths', {})`, `self.project.get('conductor', {})`, `self.project.get('context_presets', {})` | `src/app_controller.py:1972,2016,2033`; `src/gui_2.py:820,4181,4333,4448` | **ProjectConfig** (`manual_slop.toml` — TRUE catch-all dict; uses `Metadata`) | | `gui_cfg.get('separate_message_panel', False)`, `gui_cfg.get('separate_response_panel', False)`, `gui_cfg.get('separate_tool_calls_panel', False)` | `src/app_controller.py:2068-2070` | **UIPanelConfig** | | `self.project.get('discussion', {}).get('discussions', {})` | `src/gui_2.py:5036,5046` | **DiscussionStore** | | `path_info['logs_dir']['path']` | `src/app_controller.py:1984` | **PathInfo** (nested) | **There is no single "Metadata" shape.** The 107 `.get()` sites access ~12 distinct aggregates, each with its own field set. The original spec (commit `e50bebdd`) proposed a single `@dataclass(frozen=True, slots=True) Metadata` with ~200 fields merging all 12 aggregates into one polymorphic mega-struct. That is the wrong direction: - It hides the type distinctions that direct field access is supposed to reveal. - A consumer that has a `Ticket` can read `.source_tier` (a `CommsLogEntry` field) — silently get the empty default — and ship a bug that no type checker will catch. - It is "less defined" than the current `dict[str, Any]`: today, reading `.source_tier` on a `Ticket` raises `AttributeError` immediately; after the mega-dataclass, it silently returns `""`. The corrected design is **per-aggregate dataclasses**: each known concept gets its own typed dataclass with its own fields. `Metadata: TypeAlias = dict[str, Any]` is preserved for the **truly collapsed codepaths** where the shape is genuinely unknown (TOML project config, generic JSON parsing, polymorphic log dumping). ## Goals | ID | Goal | Acceptance | |---|---|---| | G1 | Each known sub-aggregate is its OWN `@dataclass(frozen=True, slots=True)` with its OWN fields (or reuses the existing typed dataclass where one already exists) | `git grep "^@dataclass\|^class .*dataclass" src/` shows `CommsLogEntry`, `HistoryMessage`, `RAGChunk`, `SessionInsights`, `DiscussionSettings`, `CustomSlice`, `MMAUsageStats`, `ProviderPayload`, `UIPanelConfig`, `DiscussionStore`, `ContextPreset` (full), `PathInfo`, `ToolDefinition` each as its own class; the existing `FileItem`, `ToolCall`, `Ticket`, `ChatMessage`, `UsageStats` are reused unchanged | | G2 | `Metadata: TypeAlias = dict[str, Any]` is preserved as the catch-all for collapsed codepaths; NOT promoted to a shared mega-dataclass | `git grep "^Metadata:" src/type_aliases.py` shows `Metadata: TypeAlias = dict[str, Any]` (unchanged); the type is not a dataclass | | G3 | Migrate the 107 `.get('key', ...)` + 106 `['key']` access sites on the KNOWN sub-aggregates to direct field access on the per-aggregate dataclass | `git grep -E "\.get\('[a-z_]+'," HEAD -- 'src/*.py'` returns only legitimate non-aggregate uses (e.g., `.get('mtime', 0)` on file paths, `.get('auto_start', False)` on config dicts); the per-aggregate sites are gone | | G4 | Effective codepaths drops by ≥ 2 orders of magnitude | `compute_effective_codepaths` returns `< 1e+20` (was 4.014e+22) | | G5 | All 7 audit gates pass `--strict` (no regression) | `weak_types`, `type_registry`, `main_thread_imports`, `no_models_config_io`, `code_path_audit_coverage`, `exception_handling`, `optional_in_3_files` all exit 0 | | G6 | All existing tests pass (10/11 batched tiers — RAG flake acceptable) | `scripts/run_tests_batched.py` → 10/11 PASS | | G7 | New regression-guard tests for each new per-aggregate dataclass | `tests/test_metadata_dataclass.py` is split into `tests/test_comms_log_entry.py`, `tests/test_history_message.py`, `tests/test_tool_definition.py`, `tests/test_rag_chunk.py`, `tests/test_session_insights.py`, etc.; each has 5+ tests for: constructor, field access, `to_dict()`/`from_dict()` round-trip, frozen, equality | | G8 | `Metadata` (the catch-all dict) is used ONLY at the genuinely collapsed codepaths — never as a stand-in for a known sub-aggregate | Code review confirms: every `.get('key', default)` site has been classified as either (a) a known sub-aggregate → migrated to direct field access, or (b) a genuinely collapsed codepath (TOML project config, generic JSON parsing, polymorphic log dumping) → keeps `Metadata` | ## Non-Goals - Modifications to `src/code_path_audit*.py` (the audit infrastructure is correct; the migration is on the consumer side) - The 4 NG1 + 7 NG2 audit violations (already addressed in phase 2 + `dc397db7`) - The 4.01e22's nil-check component (per the post-mortem at `docs/reports/SSDL_CAMPAIGN_ABORTED_20260624.md`, this is a minor contributor; the per-aggregate type-dispatch collapse is the dominant cause) - The RAG test pre-existing flake (per the SSDL post-mortem "Out of Scope") - New `src/.py` files (per AGENTS.md hard rule; new dataclasses go in `src/type_aliases.py` for type-system aggregates, or in the existing module for the aggregate — `models.FileItem` stays in `models.py`, `openai_schemas.ToolCall` stays in `openai_schemas.py`, etc.) - Promoting `Metadata: TypeAlias = dict[str, Any]` to a shared mega-dataclass (this is the original spec's bad inference; rejected 2026-06-25) - The collapsed-codepath sites (`self.project.get('paths', {})`, `self.project.get('conductor', {})`, etc.) — these read `manual_slop.toml` and the shape is genuinely unknown at type level; they keep `Metadata` as `dict[str, Any]` ## Functional Requirements ### FR1: Per-aggregate dataclasses (not one mega-dataclass) Each known sub-aggregate becomes its OWN dataclass. The design follows the existing pattern at `src/openai_schemas.py` (`ToolCall`, `ChatMessage`, `UsageStats`, `ToolCallFunction`, `NormalizedResponse` — all separate frozen dataclasses with their own fields). #### Existing dataclasses — REUSED UNCHANGED | Class | Location | Fields | Consumers that need migration | |---|---|---|---| | `Ticket` | `src/models.py:302` | `id, description, target_symbols, context_requirements, depends_on, status, assigned_to, priority, target_file, blocked_reason, step_mode, retry_count, manual_block, model_override, persona_id` (15 fields) | `src/gui_2.py:1366-1438,1682,4810,4820,4868`; `src/conductor_tech_lead.py:125`; `src/app_controller.py:4810-4868` | | `FileItem` | `src/models.py:533` | `path, auto_aggregate, force_full, view_mode, selected, ast_signatures, ast_definitions, ast_mask, custom_slices, injected_at` (10 fields) | `src/aggregate.py:418,421`; `src/ai_client.py:2565,2807,2898`; `src/app_controller.py:3508` | | `ToolCall` | `src/openai_schemas.py:32` | `id, function (ToolCallFunction), type` (3 fields) | `src/mcp_client.py` (tool loop section) | | `ChatMessage` | `src/openai_schemas.py:48` | `role, content, tool_calls, tool_call_id, name` (5 fields) | provider-side history (will replace the per-vendor `_X_history` aliases that were removed in `code_path_audit_phase_3_provider_state_20260624`) | | `UsageStats` | `src/openai_schemas.py:68` | `input_tokens, output_tokens, cache_read_tokens, cache_creation_tokens` (4 fields) | per-call token usage in `src/app_controller.py:2299-2309` | #### NEW dataclasses — to be added | Class | Module | Fields | Consumers that need migration | |---|---|---|---| | `CommsLogEntry` | `src/type_aliases.py` | `ts, role, kind, direction, model, source_tier, content, error` (8 fields) | `src/app_controller.py:2277,2302,2310`; `src/session_logger.py`; `src/multi_agent_conductor.py` | | `HistoryMessage` | `src/type_aliases.py` | `role, content, tool_calls, tool_call_id, name, ts` (6 fields) | UI-layer discussion history (the per-turn editable list, NOT the provider-side `ChatMessage` — these are distinct layers per `data_structure_strengthening_20260606` §3.1) | | `ToolDefinition` | `src/type_aliases.py` | `name, description, parameters, auto_start` (4 fields) | `src/mcp_client.py:_build_anthropic_tools` and equivalent per-vendor tool builders | | `RAGChunk` | `src/rag_engine.py` | `document, path, score, metadata` (4 fields) | `src/aggregate.py:3259`; `src/app_controller.py:251,4162` | | `SessionInsights` | `src/type_aliases.py` | `total_tokens, call_count, burn_rate, session_cost, completed_tickets, efficiency` (6 fields) | `src/gui_2.py:4926-4931` | | `DiscussionSettings` | `src/type_aliases.py` | `temperature, top_p, max_output_tokens` (3 fields) | `src/gui_2.py:3535` | | `CustomSlice` | `src/type_aliases.py` | `tag, comment, start_line, end_line` (4 fields) | `src/gui_2.py:4048-4054,1301-1302` | | `MMAUsageStats` | `src/type_aliases.py` | `model, input, output` (3 fields) | `src/gui_2.py:2199-2201,2216` | | `ProviderPayload` | `src/type_aliases.py` | `script, args, output, source_tier` (4 fields) | `src/app_controller.py:2274,2287` | | `UIPanelConfig` | `src/type_aliases.py` | `separate_message_panel, separate_response_panel, separate_tool_calls_panel` (3 fields) | `src/app_controller.py:2068-2070` | | `PathInfo` | `src/type_aliases.py` | `logs_dir, scripts_dir, project_root` (3 fields, nested) | `src/app_controller.py:1984-1985` | | `ContextPreset` | `src/models.py` (full schema) | `name, files (FileItems), screenshots (list[str])` (3 fields minimum) | `src/gui_2.py:4184-4185,4333,4448` | #### Why per-aggregate dataclasses, not one shared mega-dataclass - **Each aggregate has its own field set.** A `Ticket` has `depends_on: List[str]`, `manual_block: bool`. A `CommsLogEntry` has `source_tier: str`, `model: str`. A `RAGChunk` has `document: str`, `score: float`. They share NO common fields beyond `id`. There is no "common Metadata base" to extract. - **A shared mega-dataclass defeats the type system.** A consumer that has a `Ticket` can read `.source_tier` (a `CommsLogEntry` field) — silently get the empty default — and ship a bug that no type checker will catch. Today, with `dict[str, Any]`, reading `.source_tier` on a `Ticket` raises `AttributeError` immediately. The mega-dataclass is **less defined** than the current state. - **The original convention anticipated per-concept promotion.** Per `data_structure_strengthening_20260606` §3.3: *"Phase 2 can convert `Metadata` to a `TypedDict` (or split into per-concept `TypedDict`s) and the aliases continue to work without breaking changes. The aliases are STABLE NAMES; the underlying type can evolve."* The original 2026-06-06 design intent was per-concept promotion, NOT a mega-dataclass. The original 2026-06-25 metadata_promotion_20260624 spec reversed this direction; the corrected spec restores the original intent. ### FR2: `Metadata` stays as the catch-all for collapsed codepaths `Metadata: TypeAlias = dict[str, Any]` is preserved unchanged. It is used at sites where the shape is genuinely unknown at type level: - `manual_slop.toml` project config loading (`self.project.get('paths', {})`, `self.project.get('conductor', {})`, `self.project.get('context_presets', {})`, `self.project.get('discussion', {})`) — these are top-level TOML keys; the aggregator doesn't know which key it's about to read. - Generic JSON parsing at the wire boundary (REST API payloads, WebSocket messages) — the body shape is defined by the producer, not the consumer. - Polymorphic log dumping — a function that serializes a list of mixed-aggregate entries to JSON without caring about their individual types. These sites keep `Metadata` and `.get('key', default)` because there is no per-aggregate type to promote to. The audit MUST classify every remaining `.get('key', default)` site as one of: (a) "promoted to per-aggregate dataclass → migrated" or (b) "collapsed codepath → keeps Metadata with documented justification in code comment or commit message." ### FR3: Phase-by-phase migration (12+ sub-aggregates, 1 phase per aggregate) The migration is per-aggregate: each aggregate gets its own phase. Phases are ordered to maximize early feedback: | Phase | Sub-aggregate | Est. consumers | Primary files | |---|---|---:|---| | 0 | Design the new dataclasses + add regression-guard test stubs | 0 (design only) | `src/type_aliases.py` (and the existing modules for in-place additions) | | 1 | `Ticket` (already a dataclass; migrate consumers only) | ~30 sites | `src/gui_2.py`, `src/conductor_tech_lead.py`, `src/app_controller.py` | | 2 | `FileItem` (already a dataclass; migrate consumers only) | ~10 sites | `src/aggregate.py`, `src/ai_client.py`, `src/app_controller.py` | | 3 | `CommsLogEntry` (NEW dataclass + migrate consumers) | ~30 sites | `src/type_aliases.py`, `src/session_logger.py`, `src/multi_agent_conductor.py`, `src/app_controller.py` | | 4 | `HistoryMessage` (NEW dataclass + migrate UI-layer consumers) | ~20 sites | `src/type_aliases.py`, `src/gui_2.py` | | 5 | `ChatMessage` (already in `openai_schemas.py`; wire it into the per-vendor send paths) | ~27 sites | `src/ai_client.py` | | 6 | `UsageStats` (already in `openai_schemas.py`; wire into the per-call usage aggregation) | ~10 sites | `src/app_controller.py` | | 7 | `ToolCall` (already in `openai_schemas.py`; wire into the tool loop section) | ~56 sites | `src/ai_client.py`, `src/mcp_client.py` | | 8 | `ToolDefinition` (NEW dataclass + migrate per-vendor tool builders) | ~94 sites | `src/type_aliases.py`, `src/mcp_client.py` | | 9 | `RAGChunk` (NEW dataclass + migrate consumers) | ~5 sites | `src/rag_engine.py`, `src/aggregate.py`, `src/app_controller.py` | | 10 | `SessionInsights`, `DiscussionSettings`, `CustomSlice`, `MMAUsageStats`, `ProviderPayload`, `UIPanelConfig`, `PathInfo`, `ContextPreset` (small aggregates, batched) | ~25 sites | `src/type_aliases.py`, `src/models.py`, `src/gui_2.py`, `src/app_controller.py` | | 11 | `Metadata` collapsed-codepath audit + classification (per FR2) | ~80 sites | every `.get('key', default)` site that is NOT promoted to a per-aggregate dataclass | | 12 | Verification + end-of-track (1 task, 3 commits) | 0 | terminal + `docs/reports/TRACK_COMPLETION_metadata_promotion_20260624.md` (NEW) | Each phase: 1. For NEW dataclasses: define the dataclass in the appropriate module; add regression-guard test 2. For ALL phases: migrate the consumer sites from `.get('key', default)` → `.field_name` (or `.field_name or default` for nullable fields) 3. Per-phase regression-guard test runs 4. Re-measure effective codepaths after the phase ### FR4: Migration patterns (canonical) ```python # BEFORE: x = entry.get('model', 'unknown') y = entry.get('input_tokens', 0) or 0 z = entry.get('source_tier', 'main') if entry.get('manual_block', False): ... role = entry['role'] if 'depends_on' in entry: deps = entry['depends_on'] # AFTER (with per-aggregate dataclass): x = entry.model or 'unknown' # CommsLogEntry y = entry.input_tokens or 0 # UsageStats z = entry.source_tier or 'main' # CommsLogEntry if entry.manual_block: # Ticket ... role = entry.role # HistoryMessage / CommsLogEntry if entry.depends_on: # Ticket deps = entry.depends_on ``` The migration is mechanical but requires care: - For nullable fields: use `entry.field or default_value` - For required fields: use `entry.field` directly - For polymorphic keys (some entries have the key, some don't): the dataclass default handles this (all fields have defaults; `frozen=True, slots=True` ensures immutability) - For `['key']` (subscript) where the key is dynamic: rare; keep as `dict[str, Any]` access (e.g., `entry.to_dict()['dynamic_key']`) — but ONLY if the entry is genuinely a dict, not a dataclass ### FR5: Edge cases **Polymorphic constructors**: many sites do `entry = {'role': 'user', 'content': 'hi'}`. After migration: `entry = HistoryMessage(role='user', content='hi')`. The dataclass has all the fields as `Optional` or with defaults, so this works. **Dynamic dict construction**: `for k, v in raw.items(): entry[k] = v`. After migration: `entry = HistoryMessage(**raw)`. The `**` syntax requires that all keys in `raw` are valid field names; if `raw` has unknown keys, this fails. Solution: use a `from_dict` classmethod that filters out unknown keys (the canonical pattern, already used by `models.FileItem.from_dict` at `src/models.py:600-619` and `openai_schemas.NormalizedResponse.from_dict`): ```python @classmethod def from_dict(cls, raw: dict[str, Any]) -> 'HistoryMessage': valid_fields = {f.name for f in fields(cls)} return cls(**{k: v for k, v in raw.items() if k in valid_fields}) ``` **JSON serialization**: `json.dumps(entry)` fails on dataclass. Solution: `json.dumps(entry.to_dict())` (per the canonical `to_dict()` pattern at `src/models.py:567-579` and `src/openai_schemas.py:36-43`). **Pickle**: `pickle.dumps(entry)` works (dataclass supports pickle natively via `__reduce__`). **Equality**: `entry1 == entry2` now works (dataclass generates `__eq__`); before it was `False` for distinct dict instances even with the same content. **JSON round-trip preservation**: every dataclass in this track has a paired `to_dict()` + `from_dict()` (no information loss). This is enforced by the per-dataclass regression-guard test. ### FR6: `Metadata` collapsed-codepath classification (per FR2) For every remaining `.get('key', default)` site after all phases: 1. The site is classified as either (a) "promoted to per-aggregate dataclass" (migrated) or (b) "collapsed codepath" (keeps `Metadata`). 2. For (b), the justification is documented in the commit message (one line: "this site reads `manual_slop.toml`; the shape is unknown until the TOML is parsed"). 3. The audit `scripts/audit_weak_types.py --strict` continues to flag anonymous dict accesses; the gate is the per-aggregate dataclass promotion, NOT the elimination of all `.get()`. ### FR7: Re-measurement After each phase, re-measure: ```bash uv run python -c " import sys sys.path.insert(0, 'scripts/code_path_audit') sys.path.insert(0, 'src') from code_path_audit import build_pcg from code_path_audit_ssdl import count_branches_in_function pcg = build_pcg('src').data metadata_consumers = pcg.consumers.get('Metadata', []) total = sum(2 ** count_branches_in_function(f, 'src') for f in metadata_consumers) print(f'Effective codepaths: {total:.3e}') print(f'Consumers: {len(metadata_consumers)}') " ``` Expected: drops from 4.014e+22 to < 1e+20 after the aggregate-promotion phases (each phase drops it further as more consumers migrate to direct field access). ## Non-Functional Requirements - NFR1: 1-space indentation (per `conductor/workflow.md`) - NFR2: CRLF line endings on Windows - NFR3: No comments in source code - NFR4: Per-task atomic commits with git notes - NFR5: No new pip dependencies (dataclass is stdlib) - NFR6: `Result[T]` returns for fallible fns (per `error_handling.md`) - NFR7: No new `src/.py` files (per AGENTS.md hard rule; new type-system aggregates go in `src/type_aliases.py`, in-module aggregates stay in their parent module) ## Architecture Reference - `conductor/code_styleguides/data_oriented_design.md` — the canonical DOD reference ("Prefer Fewer Types" — but the types are still distinct) - `conductor/code_styleguides/error_handling.md` — the `Result[T]` convention - `conductor/code_styleguides/type_aliases.md` — the alias convention (preserved; `Metadata: dict[str, Any]` stays as the catch-all) - `src/openai_schemas.py` — the canonical per-aggregate dataclass pattern (`ToolCall`, `ChatMessage`, `UsageStats`); the reference implementation for the NEW dataclasses in this track - `src/models.py:533` — `FileItem` (the canonical in-module dataclass pattern with `to_dict()` / `from_dict()` round-trip) - `src/models.py:302` — `Ticket` (the canonical dataclass with `get()` legacy-compat method, used during migration) - `docs/reports/SSDL_CAMPAIGN_ABORTED_20260624.md` — the post-mortem: the 4.01e22 is from type-dispatch, not nil-checks; the fix is type promotion - `docs/reports/PLANNING_CORRECTION_metadata_promotion_20260625.md` — the corrected-design rationale (this track's correction) - `conductor/tracks/any_type_componentization_20260621/spec.md` — the grandparent track (89 sites promoted to dataclasses across 5 candidates); the per-aggregate pattern this track follows - `conductor/tracks/data_structure_strengthening_20260606/spec.md` §3.3 — the original 2026-06-06 design intent: *"Phase 2 can convert `Metadata` to a `TypedDict` (or split into per-concept `TypedDict`s) and the aliases continue to work without breaking changes. The aliases are STABLE NAMES; the underlying type can evolve."* - `scripts/code_path_audit/code_path_audit.py` — the consumer detection (3-pass AST) - `scripts/code_path_audit/code_path_audit_ssdl.py` — the effective codepaths metric ## Out of Scope - Modifications to `src/code_path_audit*.py` (the audit infrastructure is correct) - The 4 NG1 + 7 NG2 audit violations (already addressed in `dc397db7`) - The 4.01e22's nil-check component (per SSDL post-mortem; minor contributor) - The RAG test pre-existing flake (per SSDL post-mortem) - New `src/.py` files (per AGENTS.md hard rule) - A shared mega-dataclass across the 5+ sub-aggregates (the original spec's bad inference; rejected 2026-06-25) - Promoting `Metadata: TypeAlias = dict[str, Any]` itself to a dataclass (it's the catch-all for collapsed codepaths; not a known sub-aggregate) - Migration of the collapsed-codepath sites (`self.project.get('paths', {})`, etc.) — these read `manual_slop.toml`; the shape is genuinely unknown - Pydantic migration (the canonical pattern in this codebase is stdlib `@dataclass(frozen=True, slots=True)`; Pydantic is for input validation, not for the data structures used internally) ## Verification Criteria (Definition of Done) | # | Criterion | Verification command | |---|---|---| | VC1 | `Metadata: TypeAlias = dict[str, Any]` is UNCHANGED in `src/type_aliases.py` | `git grep "^Metadata:" src/type_aliases.py` shows `Metadata: TypeAlias = dict[str, Any]` | | VC2 | Each new sub-aggregate is its OWN `@dataclass(frozen=True, slots=True)` in the appropriate module | `git grep -A 2 "^class CommsLogEntry\|^class HistoryMessage\|^class ToolDefinition\|^class RAGChunk\|^class SessionInsights\|^class DiscussionSettings\|^class CustomSlice\|^class MMAUsageStats\|^class ProviderPayload\|^class UIPanelConfig\|^class PathInfo" src/` shows each as a separate frozen dataclass | | VC3 | Existing per-aggregate dataclasses (`Ticket`, `FileItem`, `ToolCall`, `ChatMessage`, `UsageStats`) are REUSED unchanged | `git grep "class Ticket\|class FileItem\|class ToolCall\|class ChatMessage\|class UsageStats" src/` shows the existing classes; consumers migrate to direct field access on them | | VC4 | All 107 `.get('key', ...)` access sites on KNOWN sub-aggregates replaced | `git grep -E "\.get\('[a-z_]+'," HEAD -- 'src/*.py'` returns only the FR2 collapsed-codepath sites (documented in the per-site classification) | | VC5 | All 106 `['key']` subscript access sites on KNOWN sub-aggregates replaced | `git grep -E "\[[ ]*'[a-z_]+'[ ]*\]" HEAD -- 'src/*.py'` returns only legitimate non-aggregate uses | | VC6 | Per-aggregate regression-guard tests exist and pass | `uv run pytest tests/test_comms_log_entry.py tests/test_history_message.py tests/test_tool_definition.py tests/test_rag_chunk.py tests/test_session_insights.py -v` → all pass (5+ tests per file) | | VC7 | Effective codepaths drops by ≥ 2 orders of magnitude | `compute_effective_codepaths` returns `< 1e+20` (was 4.014e+22) | | VC8 | All 7 audit gates pass `--strict` (no regression) | `weak_types` ≤ 112; `type_registry` 22 files; `main_thread_imports` 17; `no_models_config_io` 0; `code_path_audit_coverage` 0; `exception_handling` 0; `optional_in_3_files` 0 | | VC9 | 10/11 batched test tiers PASS (RAG flake acceptable) | `scripts/run_tests_batched.py` → 10/11 | | VC10 | End-of-track report written | `docs/reports/TRACK_COMPLETION_metadata_promotion_20260624.md` exists with the new effective-codepaths number and the per-aggregate classification of the remaining `.get()` sites | ## Risks | # | Risk | Likelihood | Mitigation | |---|---|---|---| | R1 | Some sub-aggregate has fields that don't fit cleanly into a frozen dataclass (e.g., mutability needed) | low | The canonical reference is `src/openai_schemas.py`; all 5 existing dataclasses there are `frozen=True`. If a field needs mutability, refactor to use `dataclasses.replace()` instead of mutating in place | | R2 | Some sites mutate `entry` (e.g., `entry['key'] = value`); dataclass is frozen | medium | Audit these sites; if found, replace with `dataclasses.replace(entry, field_name=value)` | | R3 | The dynamic-key subscript sites (`entry[variable_name]`) are not covered by direct field access | low | These sites are rare and already classified as collapsed-codepath per FR2; keep them as `entry.to_dict()[var_name]` if the entry is a dataclass, or `entry[var_name]` if the entry is a dict | | R4 | `to_dict()` round-trip loses information for nested dicts (e.g., `custom_slices: list[dict]` in `FileItem`) | low | `FileItem.to_dict()` already handles this (passes nested dicts through as `dict[str, Any]`); mirror the pattern in the new dataclasses | | R5 | The 695 consumer functions are too many for one track | high | The track is broken into 12 phases (FR3); each phase is independent and per-aggregate; the per-phase regression-guard test catches regressions early | | R6 | A collapsed-codepath site is misclassified as a known sub-aggregate (or vice versa) | medium | The FR6 classification is auditable: every remaining `.get()` site is either (a) "promoted" or (b) "collapsed with documented justification"; the audit `--strict` gate catches drift | | R7 | The dataclass names collide with existing names (e.g., `Metadata` exists in both `src/type_aliases.py` and `src/models.py`) | medium | Use module-qualified imports: `from src.type_aliases import Metadata` for the dict alias; `from src.models import Metadata` for the small dataclass. Document the collision in the per-aggregate test file | ## See also - `docs/reports/SSDL_CAMPAIGN_ABORTED_20260624.md` — the post-mortem: type promotion fixes the 4.01e22, not nil-checks - `docs/reports/PLANNING_CORRECTION_metadata_promotion_20260625.md` — the corrected-design rationale - `conductor/code_styleguides/type_aliases.md` — the alias convention (preserved; `Metadata: dict[str, Any]` stays as the catch-all) - `conductor/code_styleguides/data_oriented_design.md` — the canonical DOD reference - `conductor/tracks/any_type_componentization_20260621/spec.md` — the grandparent track (89 sites already promoted to dataclasses) - `conductor/tracks/data_structure_strengthening_20260606/spec.md` §3.3 — the original 2026-06-06 design intent: per-concept promotion - `src/openai_schemas.py` — the canonical per-aggregate dataclass pattern - `src/models.py:533` — `FileItem` (canonical in-module dataclass with `to_dict()` / `from_dict()`) - `src/models.py:302` — `Ticket` (canonical dataclass with legacy `get()` compat) - `conductor/tracks/code_path_audit_20260607/spec_v2.md` — the audit that established the 4.01e22 baseline - `docs/reports/code_path_audit/2026-06-22/AUDIT_REPORT.md` — the original 6797-line audit report