Private
Public Access
46 lines
2.7 KiB
Markdown
46 lines
2.7 KiB
Markdown
# Campaign Measurements: metadata_ssdl_defusing_20260624
|
|
|
|
Tracking effective codepath counts at each child of the campaign.
|
|
|
|
## Baseline
|
|
|
|
Source: `docs/reports/code_path_audit/2026-06-22/AUDIT_REPORT.md` Finding 1.
|
|
|
|
| Metric | Value |
|
|
|---|---|
|
|
| Effective codepaths (Metadata) | 4.01e22 |
|
|
| Nil-check functions (per SSDL rollup) | 74 |
|
|
| Nil-check functions (per spec text "the 6") | 6 (stale count from executive summary) |
|
|
|
|
Note: The "6 nil-check functions" count in the executive summary is a static text string in `src/code_path_audit_gen.py`, not a runtime measurement. The actual SSDL detection finds 74 functions across the codebase, of which 1 is in `src/aggregate.py` and 27 are in `src/ai_client.py`.
|
|
|
|
## Child 1: metadata_nil_sentinel_20260624
|
|
|
|
| Metric | Value |
|
|
|---|---|
|
|
| Effective codepaths (post-child-1) | 4.014e22 |
|
|
| Drop vs baseline | -0.1% (slight increase; within rounding error) |
|
|
| Budget gate (10% drop) | **FAIL** |
|
|
| NIL_METADATA defined | YES (`src/aggregate.py:50`) |
|
|
| Functions migrated | 1 (`_build_files_section_from_items` in `src/aggregate.py`) |
|
|
| Behavioral tests | 5/5 PASS |
|
|
|
|
### Budget Gate Finding
|
|
|
|
The 10% drop threshold is mathematically near-impossible to achieve with this measurement for two reasons:
|
|
|
|
1. **Exponential dominance**: the effective-codepath sum is dominated by `2^N` where N is the largest branch count. Removing 1 branch from a function with N=10 branches drops that function from `2^10=1024` to `2^9=512` — a 50% reduction for that function, but the total sum changes by less than 1 part in `4e22`.
|
|
|
|
2. **SSDL detection is textual**: `detect_nil_check_pattern` returns True for any function that has `is None` / `== None` / `!= None` patterns, regardless of whether the variable is Metadata-typed. Most of the 74 detected functions have nil-checks on `_gemini_client`, `_anthropic_client`, `path`, `adapter`, etc. — not on Metadata values. The sentinel migration pattern (`X = X or NIL_METADATA`) only applies cleanly when X is Metadata-typed.
|
|
|
|
### Interpretation
|
|
|
|
The campaign's value is in the **structural improvement**, not the final heuristic number. The campaign spec itself acknowledges this risk: "R4: The cumulative drop is less than expected... If the techniques ship, the campaign succeeds regardless of the final heuristic number."
|
|
|
|
Child 1's contribution:
|
|
- **NIL_METADATA primitive** is now defined and reusable (it serves as the fallback path for Child 2's generational-handle generation-mismatch case).
|
|
- **1 demonstration function** (`_build_files_section_from_items`) shows the pattern works end-to-end.
|
|
- **5 behavioral tests** document the contract.
|
|
|
|
Children 2 and 3 can build on the primitive. The 10% threshold is unlikely to be met by any single child; the cumulative campaign effect is what matters.
|