manual_slop/conductor/tracks/metadata_ssdl_defusing_20260624/spec.md

Campaign Specification: metadata_ssdl_defusing_20260624

Overview

3-child campaign to defuse the Metadata aggregate's combinatoric explosion (4.01e22 effective codepaths) via Fleury's SSDL techniques. Each child produces one SSDL primitive, is independently shippable, and is gated by a budget check that re-measures effective codepaths after each child.

The parent audit (code_path_audit_20260607 / AUDIT_REPORT.md Finding 1, CRITICAL) identified 3 specific techniques:

1. Nil Sentinel [N] for the 6 nil-check functions
2. Generational Handle wrapping Metadata
3. Immediate-Mode Cache [Q:key] -> [I:FetchCached] -> [T] for the 123 field-access sites

The campaign executes them in dependency order: child 1 introduces the sentinel fallback that child 2's generation-mismatch path returns to; child 2's stable identity enables child 3's cache keying.

Current State Audit (as of master @ 7a9261c4, post-fix_test_failures_20260624 merge)

The audit baseline is captured in docs/reports/code_path_audit/2026-06-22/AUDIT_REPORT.md:

• Metadata: 483 producers / 752 consumers / 123 field-access sites (0 typed)
• Effective codepaths: 4.01e22 (sum of 2^branches across 752 consumers)
• Branch points: 3466 across consumer functions
• Nil-check functions: 6 (the is None / == None / != None detection sites)

The behavioral SSDL test exists at tests/test_code_path_audit_ssdl_behavioral.py and uses a synthetic 5-function × 3-if-statements fixture to assert compute_effective_codepaths math (5 × 2^3 = 40). The real-Metadata measurement is captured by src.code_path_audit_ssdl.compute_effective_codepaths(Metadata_profile).

Goals

ID	Goal	Acceptance
G1	Child 1 (Nil Sentinel) ships: 6 nil-check functions use sentinel-return	6 functions refactored; behavioral test for sentinel; 4.01e22 number drops measurably
G2	Child 2 (Generational Handle) ships: lifetime branches collapse to 1 lookup + 1 generation comparison	Registry in place; 1 lookup benchmark; further effective-codepath drop
G3	Child 3 (Field Cache) ships: 123 string-keyed entry.get('key', default) sites become cache hits	Cache in place; 123 sites benchmarked; further effective-codepath drop
G4	End-of-campaign report: cumulative effective-codepath measurement vs 4.01e22 baseline	docs/reports/TRACK_COMPLETION_metadata_ssdl_defusing_20260624.md written
G5	All 4 audit gates remain clean	weak_types ≤ 112, type_registry in sync, main_thread_imports clean, no_models_config_io clean

Non-Goals

• Touching any aggregate OTHER than Metadata (the audit's other 9 in-scope aggregates + 3 candidates are out of scope; the SSDL primitives established by this campaign can be applied to them in follow-up campaigns)
• Modifying the audit infrastructure (src/code_path_audit*.py); the campaign USES the audit to measure progress but does not change the audit
• Refactoring the 7-file split (NG3 from code_path_audit_polish_20260622); that's a separate refactor track
• Runtime profiling (the campaign uses the static branch_count heuristic; runtime profiling is Track F from the previous menu)

Per-Child Specs

Child 1: metadata_nil_sentinel_20260624 (Nil Sentinel)

Scope: Introduce NIL_METADATA = Metadata(...) constant with safe defaults. Migrate the 6 nil-check functions (detected by src.code_path_audit_ssdl.detect_nil_check_pattern) to sentinel-return: replace if entry is None: ... / if entry == None: ... / if entry != None: ... patterns with entry = entry or NIL_METADATA (or the function's equivalent).

Acceptance:

• 6 functions refactored
• 1 behavioral test (tests/test_metadata_nil_sentinel.py): asserts the sentinel is used, asserts the 6 functions no longer have the 3-pattern nil-check branches
• Budget gate: re-run compute_effective_codepaths(Metadata_profile); if the number drops by < 10%, pause and report

Why first: establishes the sentinel that child 2's generation-mismatch path returns to.

Child 2: metadata_generational_handle_20260624 (Generational Handle)

Scope: Wrap Metadata in (index, generation) resolved through a registry. Validation is one comparison: if metadata.generation != registry.generations[metadata.index], return NIL_METADATA. Otherwise, the value is valid.

Acceptance:

• MetadataHandleRegistry (or equivalent) introduced in a sensible location (likely src/aggregate.py or a new sibling module per AGENTS.md §File Naming Convention)
• Migration: the production Metadata value is now wrapped in a handle; lifetime-branch code (e.g., the 3466 branch points that include lifetime checks) collapses to 1 lookup + 1 comparison
• 1 behavioral test: assert handle lookup is O(1), assert generation mismatch returns sentinel
• Budget gate: re-run compute_effective_codepaths(Metadata_profile); if the number drops by < 20%, pause and report (the generational handle is expected to produce a larger drop than the sentinel)

Why second: builds on child 1's sentinel as the fallback path. Provides a stable identity for child 3's cache keying.

Child 3: metadata_field_cache_20260624 (Immediate-Mode Cache)

Scope: Introduce MetadataFieldCache[(handle_id, field_name)] -> value. Consumers request (metadata_handle, 'field_name'), get cached value. The 123 string-keyed entry.get('key', default) sites become 123 cache lookups.

Acceptance:

• MetadataFieldCache introduced
• Migration: the 123 field-access sites in src/ use the cache
• 1 behavioral test: assert cache hit, assert cache miss with sentinel fallback
• Budget gate: re-run compute_effective_codepaths(Metadata_profile); if the number drops by < 30%, pause and report (the cache is expected to produce the largest drop)

Why third: the cache needs the handle's stable identity (child 2) to use as a key.

Budget Gate Protocol

REPLACED by Amendment 1 (post-child-1 finding). See amendment_1_budget_gate_metric.md.

The original "X% drop in compute_effective_codepaths(Metadata_profile)" metric is mathematically broken for this codebase: the sum is dominated by the largest 2^N terms, so removing 1 branch from a 10-branch function drops that function 50% but changes the total sum by < 1 part in 4e22. Child 1 measured -0.1% (within rounding error) despite a successful migration.

The new metric is a simple pattern count, testable with git diff:

• Child 1 (Nil Sentinel): count of is None / == None / != None patterns in Metadata-typed code paths eliminated
• Child 2 (Generational Handle): count of lifetime-branch patterns in Metadata-typed code paths eliminated (e.g., if entry.lifetime != current_lifetime: ... replaced with handle.registry_lookup() or NIL_METADATA)
• Child 3 (Field Cache): count of entry.get('key', default) and entry['key'] patterns in Metadata-typed code paths eliminated (replaced with cache.get(handle, 'key'))

The new gate per child: all targeted patterns in the campaign's scope are eliminated (= 0 remaining after the migration). Tier 2 reports: "before N patterns, after 0 patterns, target met."

The measurement is captured in docs/reports/campaign_measurements_20260624.md (existing file, updated per child) and rolled up into the campaign's end-of-campaign report.

Functional Requirements

FR1: Each child preserves the existing test suite

After each child, all 6 VCs from the parent fix_test_failures_20260624 track remain green: 14 previously-failing tests still pass; no new failures.

FR2: Each child is independently shippable

A child can be merged without the other 2 (the campaign gates are forward-looking; a child that meets its budget is shippable on its own).

FR3: The end-of-campaign report quantifies the cumulative effect

docs/reports/TRACK_COMPLETION_metadata_ssdl_defusing_20260624.md shows: pre-campaign baseline 4.01e22, post-child-1 number, post-child-2 number, post-child-3 number, total reduction.

Non-Functional Requirements

• NFR1: 1-space indentation (project convention)
• NFR2: CRLF line endings on Windows
• NFR3: No comments in source code
• NFR4: No new pip dependencies
• NFR5: Per-task atomic commits with git notes
• NFR6: Each child's Result[T] returns for fallible fns (per conductor/code_styleguides/error_handling.md)
• NFR7: The new SSDL primitives are exported from a sensible location; no new top-level src/<thing>.py files (per AGENTS.md §File Naming Convention) unless the user explicitly approves

Architecture Reference

• docs/reports/code_path_audit/2026-06-22/AUDIT_REPORT.md — Finding 1 (CRITICAL) and the 3 proposed fixes
• src/code_path_audit_ssdl.py:84-100 — detect_nil_check_pattern (the function that identifies the 6 nil-check sites)
• src/code_path_audit_ssdl.py:39-55 — compute_effective_codepaths (the measurement function)
• src/code_path_audit.py:271-296 — CANONICAL_MEMORY_DIM and MEMORY_DIM_FILE_HEURISTIC (where to file new primitives)
• conductor/code_styleguides/data_oriented_design.md — the canonical DOD reference
• conductor/code_styleguides/error_handling.md — the Result[T] convention
• conductor/code_styleguides/agent_memory_dimensions.md — the 4 memory dimensions (Metadata is discussion)

Out of Scope

• Aggregates other than Metadata (FileItem, CommsLogEntry, HistoryMessage, ToolDefinition, ToolCall, Result, the 3 list-typed, the 3 candidates) — the SSDL primitives are general but the campaign is Metadata-specific
• Modifying src/code_path_audit*.py (the audit infrastructure)
• Refactoring the 7-file split
• Runtime profiling (Track F from the previous menu)
• Modifying the campaign structure (3 children are fixed; adding a 4th is out of scope)

Verification Criteria (Definition of Done)

#	Criterion	Verification command
VC1	All 3 children SHIPPED	Each child track has status = "completed", current_phase = "complete", all phases completed
VC2	End-of-campaign report exists	cat docs/reports/TRACK_COMPLETION_metadata_ssdl_defusing_20260624.md shows the 3 measurements + cumulative reduction
VC3	Full test suite remains green	uv run python scripts/run_tests_batched.py → all 11 tiers PASS
VC4	4 audit gates remain clean	weak_types ≤ 112, type_registry in sync, main_thread_imports clean, no_models_config_io clean
VC5	No new src/<thing>.py files created (per AGENTS.md)	git diff master..HEAD --stat -- 'src/*.py' | grep -E 'src/[a-z_]+\.py' returns only the existing src/ modules; the new SSDL primitives live in existing files
VC6	Behavioral tests for each child exist and pass	uv run pytest tests/test_metadata_nil_sentinel.py tests/test_metadata_generational_handle.py tests/test_metadata_field_cache.py -v → all pass

Risks

#	Risk	Likelihood	Mitigation
R1	Child 1 doesn't measurably drop the effective-codepaths number	low	The 6 nil-checks are documented in AUDIT_REPORT.md; their removal MUST drop the number. If not, the audit or the SSDL math is wrong (separate investigation).
R2	Child 2 (generational handle) breaks code that expects raw Metadata	medium	The handle is a wrapper; consumers can still extract the raw value via .value or similar. Behavioral test verifies backwards-compat for the common cases.
R3	Child 3 (field cache) cache invalidation is wrong	medium	The cache is keyed by (handle_id, field_name). When the underlying value changes, the handle's generation bumps, invalidating all cache entries for that handle. The cache is a write-through thin wrapper.
R4	The cumulative drop is less than expected (e.g., 4.01e22 → 1e15 instead of 4.01e22 → 1e5)	low	The campaign's value is in the TECHNIQUE, not the final number. The numbers are heuristic; the actual goal is the structural improvement (sentinel, handle, cache). If the techniques ship, the campaign succeeds regardless of the final heuristic number.
R5	New src/<thing>.py files needed for the SSDL primitives	low	Per AGENTS.md, helpers go in the parent module. The new primitives live in src/aggregate.py (the parent module for Metadata). If the user explicitly approves new top-level files, the campaign can be extended.