conductor(campaign): metadata_ssdl_defusing_20260624 - 3-child SSDL defusing campaign

Campaign: address the parent code_path_audit_20260607 Finding 1 (CRITICAL)
Metadata 4.01e22 effective codepaths via 3 SSDL techniques.

3 children, sequential, with budget gates:
1. metadata_nil_sentinel_20260624 (>= 10% drop): introduce
   NIL_METADATA sentinel + migrate 6 nil-check functions.
2. metadata_generational_handle_20260624 (>= 20% drop,
   BLOCKED_BY 1): wrap Metadata in (index, generation) handle;
   collapse lifetime branches to 1 lookup + 1 cmp.
3. metadata_field_cache_20260624 (>= 30% drop, BLOCKED_BY 2):
   MetadataFieldCache keyed by (handle.index, field_name);
   123 string-keyed entry.get('key', default) sites become
   cache lookups.

Each child has its own spec/plan/metadata/state. Budget gate
after each child: re-measure effective codepaths; if drop < threshold,
PAUSE the campaign and report to user.

End-of-campaign TRACK_COMPLETION captures the cumulative reduction
vs the 4.01e22 baseline. Deferred follow-up: apply the same
3 SSDL primitives to the 4 other dict[str, Any] aliases
(FileItem, CommsLogEntry, HistoryMessage, ToolDefinition, ToolCall).

16 files committed: 4 directories x 4 files each (spec, plan,
metadata, state).
This commit is contained in:
ed
2026-06-24 14:53:40 -04:00
parent b4e32a71de
commit 84c0b4ecc4
16 changed files with 1375 additions and 0 deletions
@@ -0,0 +1,147 @@
# Campaign Specification: metadata_ssdl_defusing_20260624
## Overview
3-child campaign to defuse the `Metadata` aggregate's combinatoric explosion (4.01e22 effective codepaths) via Fleury's SSDL techniques. Each child produces one SSDL primitive, is independently shippable, and is gated by a budget check that re-measures effective codepaths after each child.
The parent audit (`code_path_audit_20260607` / `AUDIT_REPORT.md` Finding 1, CRITICAL) identified 3 specific techniques:
1. **Nil Sentinel `[N]`** for the 6 nil-check functions
2. **Generational Handle** wrapping Metadata
3. **Immediate-Mode Cache `[Q:key] -> [I:FetchCached] -> [T]`** for the 123 field-access sites
The campaign executes them in dependency order: child 1 introduces the sentinel fallback that child 2's generation-mismatch path returns to; child 2's stable identity enables child 3's cache keying.
## Current State Audit (as of master @ 7a9261c4, post-fix_test_failures_20260624 merge)
The audit baseline is captured in `docs/reports/code_path_audit/2026-06-22/AUDIT_REPORT.md`:
- **Metadata:** 483 producers / 752 consumers / 123 field-access sites (0 typed)
- **Effective codepaths:** 4.01e22 (sum of 2^branches across 752 consumers)
- **Branch points:** 3466 across consumer functions
- **Nil-check functions:** 6 (the `is None` / `== None` / `!= None` detection sites)
The behavioral SSDL test exists at `tests/test_code_path_audit_ssdl_behavioral.py` and uses a synthetic 5-function × 3-if-statements fixture to assert `compute_effective_codepaths` math (5 × 2^3 = 40). The real-Metadata measurement is captured by `src.code_path_audit_ssdl.compute_effective_codepaths(Metadata_profile)`.
## Goals
| ID | Goal | Acceptance |
|---|---|---|
| G1 | Child 1 (Nil Sentinel) ships: 6 nil-check functions use sentinel-return | 6 functions refactored; behavioral test for sentinel; 4.01e22 number drops measurably |
| G2 | Child 2 (Generational Handle) ships: lifetime branches collapse to 1 lookup + 1 generation comparison | Registry in place; 1 lookup benchmark; further effective-codepath drop |
| G3 | Child 3 (Field Cache) ships: 123 string-keyed `entry.get('key', default)` sites become cache hits | Cache in place; 123 sites benchmarked; further effective-codepath drop |
| G4 | End-of-campaign report: cumulative effective-codepath measurement vs 4.01e22 baseline | `docs/reports/TRACK_COMPLETION_metadata_ssdl_defusing_20260624.md` written |
| G5 | All 4 audit gates remain clean | weak_types ≤ 112, type_registry in sync, main_thread_imports clean, no_models_config_io clean |
## Non-Goals
- Touching any aggregate OTHER than Metadata (the audit's other 9 in-scope aggregates + 3 candidates are out of scope; the SSDL primitives established by this campaign can be applied to them in follow-up campaigns)
- Modifying the audit infrastructure (`src/code_path_audit*.py`); the campaign USES the audit to measure progress but does not change the audit
- Refactoring the 7-file split (NG3 from `code_path_audit_polish_20260622`); that's a separate refactor track
- Runtime profiling (the campaign uses the static `branch_count` heuristic; runtime profiling is Track F from the previous menu)
## Per-Child Specs
### Child 1: `metadata_nil_sentinel_20260624` (Nil Sentinel)
**Scope:** Introduce `NIL_METADATA = Metadata(...)` constant with safe defaults. Migrate the 6 nil-check functions (detected by `src.code_path_audit_ssdl.detect_nil_check_pattern`) to sentinel-return: replace `if entry is None: ...` / `if entry == None: ...` / `if entry != None: ...` patterns with `entry = entry or NIL_METADATA` (or the function's equivalent).
**Acceptance:**
- 6 functions refactored
- 1 behavioral test (`tests/test_metadata_nil_sentinel.py`): asserts the sentinel is used, asserts the 6 functions no longer have the 3-pattern nil-check branches
- Budget gate: re-run `compute_effective_codepaths(Metadata_profile)`; if the number drops by < 10%, pause and report
**Why first:** establishes the sentinel that child 2's generation-mismatch path returns to.
### Child 2: `metadata_generational_handle_20260624` (Generational Handle)
**Scope:** Wrap Metadata in `(index, generation)` resolved through a registry. Validation is one comparison: if `metadata.generation != registry.generations[metadata.index]`, return `NIL_METADATA`. Otherwise, the value is valid.
**Acceptance:**
- `MetadataHandleRegistry` (or equivalent) introduced in a sensible location (likely `src/aggregate.py` or a new sibling module per AGENTS.md §File Naming Convention)
- Migration: the production `Metadata` value is now wrapped in a handle; lifetime-branch code (e.g., the 3466 branch points that include lifetime checks) collapses to 1 lookup + 1 comparison
- 1 behavioral test: assert handle lookup is O(1), assert generation mismatch returns sentinel
- Budget gate: re-run `compute_effective_codepaths(Metadata_profile)`; if the number drops by < 20%, pause and report (the generational handle is expected to produce a larger drop than the sentinel)
**Why second:** builds on child 1's sentinel as the fallback path. Provides a stable identity for child 3's cache keying.
### Child 3: `metadata_field_cache_20260624` (Immediate-Mode Cache)
**Scope:** Introduce `MetadataFieldCache[(handle_id, field_name)] -> value`. Consumers request `(metadata_handle, 'field_name')`, get cached value. The 123 string-keyed `entry.get('key', default)` sites become 123 cache lookups.
**Acceptance:**
- `MetadataFieldCache` introduced
- Migration: the 123 field-access sites in `src/` use the cache
- 1 behavioral test: assert cache hit, assert cache miss with sentinel fallback
- Budget gate: re-run `compute_effective_codepaths(Metadata_profile)`; if the number drops by < 30%, pause and report (the cache is expected to produce the largest drop)
**Why third:** the cache needs the handle's stable identity (child 2) to use as a key.
## Budget Gate Protocol
After each child commits:
1. **Measure:** run `uv run python -c "from src.code_path_audit import AggregateProfile, ...; from src.code_path_audit_ssdl import compute_effective_codepaths; profile = ...; print(compute_effective_codepaths(profile, 'src'))"`
2. **Compare:** diff vs prior measurement (or 4.01e22 baseline for child 1)
3. **Gate:** if drop < expected threshold (10% / 20% / 30% per child), PAUSE the campaign and report to user
4. **Continue:** if drop ≥ threshold, proceed to next child
The measurement is captured in the child track's TRACK_COMPLETION report and rolled up into the campaign's end-of-campaign report.
## Functional Requirements
### FR1: Each child preserves the existing test suite
After each child, all 6 VCs from the parent `fix_test_failures_20260624` track remain green: 14 previously-failing tests still pass; no new failures.
### FR2: Each child is independently shippable
A child can be merged without the other 2 (the campaign gates are forward-looking; a child that meets its budget is shippable on its own).
### FR3: The end-of-campaign report quantifies the cumulative effect
`docs/reports/TRACK_COMPLETION_metadata_ssdl_defusing_20260624.md` shows: pre-campaign baseline 4.01e22, post-child-1 number, post-child-2 number, post-child-3 number, total reduction.
## Non-Functional Requirements
- NFR1: 1-space indentation (project convention)
- NFR2: CRLF line endings on Windows
- NFR3: No comments in source code
- NFR4: No new pip dependencies
- NFR5: Per-task atomic commits with git notes
- NFR6: Each child's `Result[T]` returns for fallible fns (per `conductor/code_styleguides/error_handling.md`)
- NFR7: The new SSDL primitives are exported from a sensible location; no new top-level `src/<thing>.py` files (per AGENTS.md §File Naming Convention) unless the user explicitly approves
## Architecture Reference
- `docs/reports/code_path_audit/2026-06-22/AUDIT_REPORT.md` — Finding 1 (CRITICAL) and the 3 proposed fixes
- `src/code_path_audit_ssdl.py:84-100``detect_nil_check_pattern` (the function that identifies the 6 nil-check sites)
- `src/code_path_audit_ssdl.py:39-55``compute_effective_codepaths` (the measurement function)
- `src/code_path_audit.py:271-296``CANONICAL_MEMORY_DIM` and `MEMORY_DIM_FILE_HEURISTIC` (where to file new primitives)
- `conductor/code_styleguides/data_oriented_design.md` — the canonical DOD reference
- `conductor/code_styleguides/error_handling.md` — the `Result[T]` convention
- `conductor/code_styleguides/agent_memory_dimensions.md` — the 4 memory dimensions (Metadata is `discussion`)
## Out of Scope
- Aggregates other than Metadata (FileItem, CommsLogEntry, HistoryMessage, ToolDefinition, ToolCall, Result, the 3 list-typed, the 3 candidates) — the SSDL primitives are general but the campaign is Metadata-specific
- Modifying `src/code_path_audit*.py` (the audit infrastructure)
- Refactoring the 7-file split
- Runtime profiling (Track F from the previous menu)
- Modifying the campaign structure (3 children are fixed; adding a 4th is out of scope)
## Verification Criteria (Definition of Done)
| # | Criterion | Verification command |
|---|---|---|
| VC1 | All 3 children SHIPPED | Each child track has `status = "completed"`, `current_phase = "complete"`, all phases `completed` |
| VC2 | End-of-campaign report exists | `cat docs/reports/TRACK_COMPLETION_metadata_ssdl_defusing_20260624.md` shows the 3 measurements + cumulative reduction |
| VC3 | Full test suite remains green | `uv run python scripts/run_tests_batched.py` → all 11 tiers PASS |
| VC4 | 4 audit gates remain clean | weak_types ≤ 112, type_registry in sync, main_thread_imports clean, no_models_config_io clean |
| VC5 | No new `src/<thing>.py` files created (per AGENTS.md) | `git diff master..HEAD --stat -- 'src/*.py' \| grep -E 'src/[a-z_]+\.py'` returns only the existing `src/` modules; the new SSDL primitives live in existing files |
| VC6 | Behavioral tests for each child exist and pass | `uv run pytest tests/test_metadata_nil_sentinel.py tests/test_metadata_generational_handle.py tests/test_metadata_field_cache.py -v` → all pass |
## Risks
| # | Risk | Likelihood | Mitigation |
|---|---|---|---|
| R1 | Child 1 doesn't measurably drop the effective-codepaths number | low | The 6 nil-checks are documented in AUDIT_REPORT.md; their removal MUST drop the number. If not, the audit or the SSDL math is wrong (separate investigation). |
| R2 | Child 2 (generational handle) breaks code that expects raw `Metadata` | medium | The handle is a wrapper; consumers can still extract the raw value via `.value` or similar. Behavioral test verifies backwards-compat for the common cases. |
| R3 | Child 3 (field cache) cache invalidation is wrong | medium | The cache is keyed by `(handle_id, field_name)`. When the underlying value changes, the handle's generation bumps, invalidating all cache entries for that handle. The cache is a write-through thin wrapper. |
| R4 | The cumulative drop is less than expected (e.g., 4.01e22 → 1e15 instead of 4.01e22 → 1e5) | low | The campaign's value is in the TECHNIQUE, not the final number. The numbers are heuristic; the actual goal is the structural improvement (sentinel, handle, cache). If the techniques ship, the campaign succeeds regardless of the final heuristic number. |
| R5 | New `src/<thing>.py` files needed for the SSDL primitives | low | Per AGENTS.md, helpers go in the parent module. The new primitives live in `src/aggregate.py` (the parent module for `Metadata`). If the user explicitly approves new top-level files, the campaign can be extended. |