conductor(spec): Add type registry to data_structure_strengthening track

Per user feedback (2026-06-06): instead of a follow-up 'TypedDict
Migration' track, add a NEW deliverable: an auto-generated type registry
in docs/type_registry/ that captures the field information in docs form.

New files:
- scripts/generate_type_registry.py (NEW): AST-based tool that reads
  src/ and writes per-source-file .md files with the fields of every
  @dataclass, NamedTuple, TypeAlias, TypedDict. Has --check (CI mode,
  exits 1 if registry would change) and --diff (dry run) modes.
- docs/type_registry/ (NEW, generated): index.md + per-source-file
  references (type_aliases.md, ai_client.md, models.md, etc.).
- tests/test_generate_type_registry.py (NEW): verify the generator.

Architecture updates:
- Section 3.6 (NEW): Type Registry architecture with example output.
- Section 3.7 (NEW): Why per-source-file docs (locality of reference).
- Section 1.1 (NEW): 'Why docs over TypedDict' analysis (3 reasons:
  lower upfront cost, better fit for AI workflow, auto-maintained).
- Goals table: registry added as a C (innovation) goal.
- Module layout: docs/type_registry/ and scripts/generate_type_registry.py
  added to the new files list.
- Migration: Phase 2 now includes the registry generator + initial docs.
- Out of scope: TypedDict migration REMOVED; 'auto-typing the field
  shape' added with the docs as the chosen approach.
- See Also: TypedDict follow-up REPLACED with 'Registry Maintenance &
  CI Integration' (smaller scope, just wires the generator into CI).

The 'cost we eat' is the LLM reading 200-500 lines of markdown per
query. This is bounded and proportional to actual information need.
The upfront cost of designing TypedDict schemas for every type is
unbounded. Tradeoffs favor the docs approach for v1; TypedDict can
come later as a future track if desired.
This commit is contained in:
ed
2026-06-06 18:06:34 -04:00
parent 61d21c70bb
commit aba35f9f4a
3 changed files with 172 additions and 25 deletions
@@ -10,7 +10,7 @@ last_updated = "2026-06-06"
[phases]
phase_1 = { status = "pending", checkpointsha = "", name = "Aliases + 6-file replacement + audit baseline" }
phase_2 = { status = "pending", checkpointsha = "", name = "NamedTuples + docs + archive" }
phase_2 = { status = "pending", checkpointsha = "", name = "NamedTuples + type registry generator + initial docs + archive" }
[tasks]
# Phase 1: Aliases + 6-file replacement
@@ -28,16 +28,20 @@ t1_11 = { status = "pending", commit_sha = "", description = "Red: tests/test_au
t1_12 = { status = "pending", commit_sha = "", description = "Run full test suite; confirm no regressions in 6 refactored files" }
t1_13 = { status = "pending", commit_sha = "", description = "Run audit; confirm count dropped from 430 to ~60; commit the new baseline" }
t1_14 = { status = "pending", commit_sha = "", description = "Phase 1 checkpoint commit + git note" }
# Phase 2: NamedTuples + docs + archive
# Phase 2: NamedTuples + type registry generator + initial docs + archive
t2_1 = { status = "pending", commit_sha = "", description = "Convert src/ai_client.py:_reread_file_items to return FileItemsDiff NamedTuple (replaces Tuple[List[FileItem], List[FileItem]]); update ~3-4 call sites" }
t2_2 = { status = "pending", commit_sha = "", description = "Opportunistic NamedTuple conversions for 1-2 more tuple returns (screen coords, etc.)" }
t2_3 = { status = "pending", commit_sha = "", description = "Create conductor/code_styleguides/type_aliases.md (canonical reference for the alias convention; 5 patterns + decision tree + examples)" }
t2_4 = { status = "pending", commit_sha = "", description = "Add 'Data Structure Conventions' section to conductor/product-guidelines.md (referencing the new styleguide)" }
t2_5 = { status = "pending", commit_sha = "", description = "Manual smoke test: launch GUI; verify type aliases don't break anything; verify audit --strict mode" }
t2_6 = { status = "pending", commit_sha = "", description = "Phase 2 checkpoint commit + git note (TRACK COMPLETE)" }
t2_7 = { status = "pending", commit_sha = "", description = "git mv conductor/tracks/data_structure_strengthening_20260606 to conductor/tracks/archive/" }
t2_8 = { status = "pending", commit_sha = "", description = "Update conductor/tracks.md: move entry to Recently Completed" }
t2_9 = { status = "pending", commit_sha = "", description = "Final state.toml update: mark all phases completed; add follow-up track typed_dict_migration_20260606 placeholder" }
t2_3 = { status = "pending", commit_sha = "", description = "Red: tests/test_generate_type_registry.py (verify AST extraction of @dataclass, NamedTuple, TypeAlias; verify output markdown structure)" }
t2_4 = { status = "pending", commit_sha = "", description = "Green: implement scripts/generate_type_registry.py (3 modes: default, --check, --diff)" }
t2_5 = { status = "pending", commit_sha = "", description = "Run the generator; commit the initial docs/type_registry/ (index.md + per-source-file .md files)" }
t2_6 = { status = "pending", commit_sha = "", description = "Verify --check mode: introduce a fake change in src/type_aliases.py, run --check, confirm exit 1" }
t2_7 = { status = "pending", commit_sha = "", description = "Create conductor/code_styleguides/type_aliases.md (canonical reference for the alias convention; 5 patterns + decision tree + examples)" }
t2_8 = { status = "pending", commit_sha = "", description = "Add 'Data Structure Conventions' section to conductor/product-guidelines.md (referencing the new styleguide)" }
t2_9 = { status = "pending", commit_sha = "", description = "Manual smoke test: launch GUI; verify type aliases don't break anything; verify audit --strict mode; verify generator --check mode" }
t2_10 = { status = "pending", commit_sha = "", description = "Phase 2 checkpoint commit + git note (TRACK COMPLETE)" }
t2_11 = { status = "pending", commit_sha = "", description = "git mv conductor/tracks/data_structure_strengthening_20260606 to conductor/tracks/archive/" }
t2_12 = { status = "pending", commit_sha = "", description = "Update conductor/tracks.md: move entry to Recently Completed" }
t2_13 = { status = "pending", commit_sha = "", description = "Final state.toml update: mark all phases completed; add follow-up track type_registry_ci_20260606 placeholder" }
[verification]
# Filled as phases complete
@@ -81,10 +85,10 @@ project_manager = { weak_sites_before = 20, weak_sites_after = 0, status = "pend
aggregate = { weak_sites_before = 17, weak_sites_after = 0, status = "pending" }
[typed_dict_migration_followup]
track_id = "typed_dict_migration_20260606"
track_id = "type_registry_ci_20260606"
status = "planned_in_data_structure_strengthening_20260606"
converts = ["CommsLogEntry", "FileItem", "ToolDefinition", "HistoryMessage"]
to = "TypedDict or @dataclass(frozen=True)"
goal = "Promote the type-registry generator from a manual track-completion step to a CI gate. Add --check to CI; wire pre-commit hook; document the per-track commit workflow."
note = "This follow-up REPLACES the earlier 'typed_dict_migration' follow-up. Per user feedback (2026-06-06), the registry approach (docs) is preferred over TypedDict migration (code) for the foreseeable future."
[public_api_migration_followup]
# From the data_oriented_error_handling track