conductor(spec): Add type registry to data_structure_strengthening track

Per user feedback (2026-06-06): instead of a follow-up 'TypedDict
Migration' track, add a NEW deliverable: an auto-generated type registry
in docs/type_registry/ that captures the field information in docs form.

New files:
- scripts/generate_type_registry.py (NEW): AST-based tool that reads
  src/ and writes per-source-file .md files with the fields of every
  @dataclass, NamedTuple, TypeAlias, TypedDict. Has --check (CI mode,
  exits 1 if registry would change) and --diff (dry run) modes.
- docs/type_registry/ (NEW, generated): index.md + per-source-file
  references (type_aliases.md, ai_client.md, models.md, etc.).
- tests/test_generate_type_registry.py (NEW): verify the generator.

Architecture updates:
- Section 3.6 (NEW): Type Registry architecture with example output.
- Section 3.7 (NEW): Why per-source-file docs (locality of reference).
- Section 1.1 (NEW): 'Why docs over TypedDict' analysis (3 reasons:
  lower upfront cost, better fit for AI workflow, auto-maintained).
- Goals table: registry added as a C (innovation) goal.
- Module layout: docs/type_registry/ and scripts/generate_type_registry.py
  added to the new files list.
- Migration: Phase 2 now includes the registry generator + initial docs.
- Out of scope: TypedDict migration REMOVED; 'auto-typing the field
  shape' added with the docs as the chosen approach.
- See Also: TypedDict follow-up REPLACED with 'Registry Maintenance &
  CI Integration' (smaller scope, just wires the generator into CI).

The 'cost we eat' is the LLM reading 200-500 lines of markdown per
query. This is bounded and proportional to actual information need.
The upfront cost of designing TypedDict schemas for every type is
unbounded. Tradeoffs favor the docs approach for v1; TypedDict can
come later as a future track if desired.
This commit is contained in:
ed
2026-06-06 18:06:34 -04:00
parent 61d21c70bb
commit aba35f9f4a
3 changed files with 172 additions and 25 deletions
@@ -11,6 +11,17 @@
"src/type_aliases.py",
"tests/test_type_aliases.py",
"tests/test_audit_weak_types.py",
"tests/test_generate_type_registry.py",
"scripts/generate_type_registry.py",
"docs/type_registry/index.md",
"docs/type_registry/type_aliases.md",
"docs/type_registry/ai_client.md",
"docs/type_registry/app_controller.md",
"docs/type_registry/models.md",
"docs/type_registry/api_hook_client.md",
"docs/type_registry/project_manager.md",
"docs/type_registry/aggregate.md",
"docs/type_registry/result_types.md",
"conductor/code_styleguides/type_aliases.md"
],
"modified_files": [
@@ -25,7 +36,7 @@
]
},
"blocked_by": [],
"blocks": ["typed_dict_migration_20260606" /* not yet created */],
"blocks": ["type_registry_ci_20260606" /* not yet created; the registry-CI-integration follow-up */],
"estimated_phases": 2,
"spec": "spec.md",
"plan": "plan.md",
@@ -89,10 +100,29 @@
"target_reduction": "430 -> ~60 (86% reduction in the 6 high-traffic files)"
},
"ai_performance_analysis": {
"win": "A name is a one-time cost the AI pays to learn, then reuses forever. With 10 aliases covering 370+ usages, the AI's vocabulary cost is bounded while the readability win is unbounded.",
"cost": "10 new names for the AI to learn. Comparable to adding 10 new function names to a module - well within normal Python codebase scale.",
"caveat": "If we add too many aliases (50+), the cognitive cost exceeds the benefit. The proposed 10 is the sweet spot. Phase 2 will convert the most-used aliases to TypedDict, which gives the AI field-level hints, not just a name.",
"honest_assessment": "Net win. The current 0 aliases is the worst case; going to 10 is a strictly better state for AI readability."
"win": "A name is a one-time cost the AI pays to learn, then reuses forever. With 10 aliases covering 370+ usages, the AI's vocabulary cost is bounded while the readability win is unbounded. The auto-generated registry gives the AI field-level information on demand at the cost of a few hundred tokens of context per query.",
"cost": "10 new names for the AI to learn (same as adding 10 new function names to a module - well within normal Python codebase scale). Plus a small token cost when the AI reads a registry file: 200-500 lines of markdown per source file, read once and cached in context.",
"caveat": "If we add too many aliases (50+), the cognitive cost exceeds the benefit. The proposed 10 is the sweet spot. The docs-based registry approach is an alternative to TypedDict migration: docs are advisory but auto-maintained, whereas TypedDict would enforce but cost more upfront.",
"honest_assessment": "Net win. The current 0 aliases is the worst case; going to 10 is a strictly better state for AI readability. Adding auto-generated docs is a further improvement at modest token cost."
},
"type_registry": {
"directory": "docs/type_registry/",
"files": [
"index.md (top-level TOCs)",
"type_aliases.md (the 10 TypeAliases from src/type_aliases.py)",
"result_types.md (the Result/ErrorInfo from data_oriented_error_handling_20260606)",
"<one .md per source file that has structs>"
],
"script": "scripts/generate_type_registry.py",
"script_modes": {
"default": "Generate / regenerate the registry",
"--check": "CI mode; exits 1 if the registry would change",
"--diff": "Dry run; print what would change without writing"
},
"agent_workflow": "The coding agent runs the generator before marking a track complete, and includes the registry diff in the commit. CI runs --check on every PR.",
"ai_token_cost": "200-500 lines of markdown per source file. The LLM reads it once and caches the schema in context. Subsequent references to the same types don't re-fetch.",
"rationale": "Trade upfront cost (TypedDict schema design for every type) for token cost (LLM reads docs at query time). Docs are auto-maintained; TypedDict schemas would need to be hand-maintained. For a codebase where the priority is 'name the shapes first, give them structure later', docs are the right v1 approach."
},
"coexistence_with_data_oriented_track": {
"Result_T": "The data_oriented_error_handling_20260606 track introduces Result[T] as a control-level wrapper. The aliases introduced by THIS track are value-level types (what's inside the T).",