From 62fc04b17222aded25267b7fd67e784d5f76cfe6 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Thu, 2 Jul 2026 21:57:36 -0400 Subject: [PATCH] feat(directives): harvest 3 type/data-structure directives + update boundary_layer_exception --- .../directives/boundary_layer_exception/v1.md | 55 +++++++++++-- .../directives/metadata_boundary_type/v1.md | 80 +++++++++++++++++++ .../directives/typed_dataclass_fields/v1.md | 33 ++++++++ 3 files changed, 163 insertions(+), 5 deletions(-) create mode 100644 conductor/directives/metadata_boundary_type/v1.md create mode 100644 conductor/directives/typed_dataclass_fields/v1.md diff --git a/conductor/directives/boundary_layer_exception/v1.md b/conductor/directives/boundary_layer_exception/v1.md index a528cf2b..001348bb 100644 --- a/conductor/directives/boundary_layer_exception/v1.md +++ b/conductor/directives/boundary_layer_exception/v1.md @@ -1,13 +1,58 @@ # boundary_layer_exception — v1 -**Why this iteration:** Lifted verbatim from `conductor/code_styleguides/python.md` §17.7 (lines 352-354). -This is the baseline encoding — the imperative-ban style currently in production. -Future variants will test alternative encodings (rationale-first, before/after, tabular) against this baseline. +**Why this iteration:** Cross-referenced lift combining `conductor/code_styleguides/python.md` §17.7 + §17.8 (lines 352-359) + `data_oriented_design.md` §8.5 boundary-exception + §8.6 (lines 193-202) + `type_aliases.md` §1 (lines 54-72). This is the boundary-layer rule from three canonical sources, all lifted verbatim. The plan updated the v1 in t1_4 to cross-reference the §17.7 exception, §17.8 enforcement, §8.6 boundary definition, and the type_aliases.md Metadata-as-boundary-type rule. +Future variants will test alternative encodings (rationale-first, tabular, single-source) against this baseline. -**Source:** `conductor/code_styleguides/python.md:352-354` +**Source:** `conductor/code_styleguides/python.md:352-359 + data_oriented_design.md:193-202 + type_aliases.md:54-72` --- ### 17.7 The one exception: the boundary layer -The ONLY place these patterns are allowed is at the literal wire boundary — the function that calls `tomllib.load()`, `json.loads()`, or a vendor SDK's response parser. The boundary is 2-3 functions per file. Every consumer IMMEDIATELY converts to a typed dataclass via `from_dict()`. \ No newline at end of file +The ONLY place these patterns are allowed is at the literal wire boundary — the function that calls `tomllib.load()`, `json.loads()`, or a vendor SDK's response parser. The boundary is 2-3 functions per file. Every consumer IMMEDIATELY converts to a typed dataclass via `from_dict()`. + +### 17.8 Enforcement + +- `scripts/audit_weak_types.py --strict` — flags `dict[str, Any]`, `Any`, anonymous tuple returns +- `scripts/audit_optional_in_3_files.py --strict` — flags `Optional[T]` return types in the baseline 4 files (`src/mcp_client.py`, `src/ai_client.py`, `src/rag_engine.py`, plus one more). An all-`src/*.py` successor (`audit_optional_returns.py`) is referenced in older planning docs but is NOT yet built — the live script today is `audit_optional_in_3_files.py`. + +--- + +**The one exception (the boundary layer) — from `data_oriented_design.md` §8.5:** + +at the literal wire boundary (TOML parsing, JSON parsing, vendor SDK response parsing), the data is open-ended for the 100ns between parsing and `from_dict()` conversion. At that boundary: + +- The function that calls `tomllib.load()` or `json.loads()` may return `Metadata` (the typed fat struct — see §8.6). +- Every consumer of that function IMMEDIATELY calls `SomeTypedDataclass.from_dict(metadata)` and uses the typed result. +- The boundary is 2-3 functions per file (one per wire entry point). + +**No other code uses `Metadata` or `dict[str, Any]` or `Any`.** This is enforced by `scripts/audit_weak_types.py --strict` (existing) + the boundary-layer audit (planned in `conductor/tracks/cruft_elimination_20260627/spec.md`). + +### 8.6 The Boundary Layer (the wire schema) + +The codebase has ONE typed fat struct at the boundary: `Metadata` in `src/type_aliases.py`. It is `@dataclass(frozen=True, slots=True)` with explicit fields covering the TOML/JSON wire schema (paths, project, discussion, role, content, ts, source_tier, model, depends_on, document, script, args, etc.). It is used in exactly 2 places: + +--- + +### 1. Use `Metadata` ONLY at the wire boundary (TOML/JSON parse) — from `type_aliases.md` + +**UPDATED 2026-06-25 (the C11/Odin/Jai-in-Python mandate).** `Metadata` is the typed fat struct at the wire boundary. It is `@dataclass(frozen=True, slots=True)` with explicit fields covering the TOML/JSON wire schema (paths, project, discussion, role, content, ts, source_tier, model, depends_on, document, script, args, etc.). + +```python +# CORRECT — at the literal wire boundary: +def _parse_toml_config(raw: str) -> Metadata: + return Metadata.from_dict(tomllib.loads(raw)) + +# CORRECT — consumer at the boundary, converts immediately: +def _load_project_context(raw_toml: Metadata) -> ProjectContext: + return ProjectContext.from_dict(raw_toml) + +# WRONG — using Metadata as a lazy-typing escape hatch: +def process_event(self, event: Metadata) -> None: + if hasattr(event, 'tool_calls'): + ... # ← BAD: this is the laziest possible typing +``` + +`Metadata` is **NOT** `TypeAlias = dict[str, Any]`. It is a typed fat struct. The boundary is 2-3 functions per file. Every consumer IMMEDIATELY converts to a componentized dataclass via `from_dict()`. + +**Anti-pattern (banned):** `Metadata: TypeAlias = dict[str, Any]` (the lazy-typing escape hatch). LLMs default to this because it's idiomatic Python. This codebase does NOT do idiomatic Python. See `data_oriented_design.md` §8.5. \ No newline at end of file diff --git a/conductor/directives/metadata_boundary_type/v1.md b/conductor/directives/metadata_boundary_type/v1.md new file mode 100644 index 00000000..2e5130ae --- /dev/null +++ b/conductor/directives/metadata_boundary_type/v1.md @@ -0,0 +1,80 @@ +# metadata_boundary_type — v1 + +**Why this iteration:** Lifted verbatim from `conductor/code_styleguides/data_oriented_design.md` §8.6 (lines 200-202) + `conductor/code_styleguides/type_aliases.md` §"The 5 Decision Patterns" #1 + #2.5 (lines 54-132). This is the baseline encoding — the prose-then-code-example style currently in production. +Future variants will test alternative encodings (rule-list, before/after only) against this baseline. + +**Source:** `conductor/code_styleguides/data_oriented_design.md:200-202 + type_aliases.md:54-132` + +--- + +### 8.6 The Boundary Layer (the wire schema) + +The codebase has ONE typed fat struct at the boundary: `Metadata` in `src/type_aliases.py`. It is `@dataclass(frozen=True, slots=True)` with explicit fields covering the TOML/JSON wire schema (paths, project, discussion, role, content, ts, source_tier, model, depends_on, document, script, args, etc.). It is used in exactly 2 places: + +--- + +### 1. Use `Metadata` ONLY at the wire boundary (TOML/JSON parse) + +**UPDATED 2026-06-25 (the C11/Odin/Jai-in-Python mandate).** `Metadata` is the typed fat struct at the wire boundary. It is `@dataclass(frozen=True, slots=True)` with explicit fields covering the TOML/JSON wire schema (paths, project, discussion, role, content, ts, source_tier, model, depends_on, document, script, args, etc.). + +```python +# CORRECT — at the literal wire boundary: +def _parse_toml_config(raw: str) -> Metadata: + return Metadata.from_dict(tomllib.loads(raw)) + +# CORRECT — consumer at the boundary, converts immediately: +def _load_project_context(raw_toml: Metadata) -> ProjectContext: + return ProjectContext.from_dict(raw_toml) + +# WRONG — using Metadata as a lazy-typing escape hatch: +def process_event(self, event: Metadata) -> None: + if hasattr(event, 'tool_calls'): + ... # ← BAD: this is the laziest possible typing +``` + +`Metadata` is **NOT** `TypeAlias = dict[str, Any]`. It is a typed fat struct. The boundary is 2-3 functions per file. Every consumer IMMEDIATELY converts to a componentized dataclass via `from_dict()`. + +**Anti-pattern (banned):** `Metadata: TypeAlias = dict[str, Any]` (the lazy-typing escape hatch). LLMs default to this because it's idiomatic Python. This codebase does NOT do idiomatic Python. See `data_oriented_design.md` §8.5. + +### 2. Use the more specific alias when the role is known + +If the dict is specifically a comms log entry, call it `CommsLogEntry` not `Metadata`. The LLM reader (and the human reviewer) sees the role at the type level. + +```python +def append_comms(entry: CommsLogEntry) -> None: ... + +def get_history() -> History: ... +``` + +**Updated 2026-06-27** — `Metadata` is itself a `@dataclass(frozen=True, slots=True)` with 36 explicit fields covering the wire schema. It is NOT a `TypeAlias = dict[str, Any]` anymore. The aliases below (e.g., `CommsLogEntry`, `HistoryMessage`) point to their own per-aggregate dataclasses, not to `Metadata`. The original "names for shapes" pattern has been promoted to the structural level (per §2.5). + +### 2.5. When the role has stable distinct fields, promote it to its OWN dataclass + +**Added 2026-06-25 (correction to `metadata_promotion_20260624`).** When a sub-aggregate has a known set of stable, distinct fields (e.g., `CommsLogEntry` has `ts, role, kind, direction, model, source_tier, content, error`; `FileItem` has `path, view_mode, custom_slices`; `RAGChunk` has `id, document, path, score, metadata`), promote it to its OWN `@dataclass(frozen=True, slots=True)` with its OWN fields. Do **NOT** share one mega-dataclass across multiple concepts. + +**Why:** the per-aggregate dataclass is the "names for shapes" pattern extended to the structural level. Each concept gets its own type, its own fields, its own `to_dict()` / `from_dict()` round-trip. Consumers use direct field access (`entry.ts`, `t.depends_on`, `chunk.document`) which compiles to a single C-level field read with 0 branches. + +**When NOT to promote:** when the shape is genuinely unknown at type level and the fields are heterogeneous (e.g., log entries from 5 different vendors with mutually-exclusive keys). Use `Metadata: Metadata` (the dataclass) as the catch-all — its 36 explicit fields cover the common wire schema, and its dict-compat methods allow ad-hoc keys for vendor-specific extensions. Do NOT use `dict[str, Any]` directly anywhere; `Metadata` is the typed replacement. + +**Canonical pattern (from `src/openai_schemas.py` and `src/type_aliases.py`):** + +```python +@dataclass(frozen=True, slots=True) +class CommsLogEntry: + ts: str = "" + role: str = "" + kind: str = "" + direction: str = "" + model: str = "unknown" + source_tier: str = "main" + content: Any = None + error: str = "" + + def to_dict(self) -> Metadata: + return asdict(self) + + @classmethod + def from_dict(cls, raw: Metadata) -> "CommsLogEntry": + valid = {f.name for f in fields(cls)} + return cls(**{k: v for k, v in raw.items() if k in valid}) +``` \ No newline at end of file diff --git a/conductor/directives/typed_dataclass_fields/v1.md b/conductor/directives/typed_dataclass_fields/v1.md new file mode 100644 index 00000000..1eec611e --- /dev/null +++ b/conductor/directives/typed_dataclass_fields/v1.md @@ -0,0 +1,33 @@ +# typed_dataclass_fields — v1 + +**Why this iteration:** Lifted verbatim from `conductor/code_styleguides/data_oriented_design.md` §8.5 (lines 175-198). +This is the baseline encoding — the rationale-then-table style currently in production. +Future variants will test alternative encodings (rule-list, before/after only) against this baseline. + +**Source:** `conductor/code_styleguides/data_oriented_design.md:175-198` + +--- + +### 8.5 The Python Type Promotion Mandate (added 2026-06-25) + +**C11/Odin/Jai semantics in a Python runtime.** This codebase is written in Python because of practical constraints (time, dependencies, LLM codegen ability), but the convention is to make Python behave as close to a statically-typed value-typed language as the runtime allows. **LLMs default to opaque types (`dict[str, Any]`, `Any`, `Optional[T]`, `hasattr()` polymorphism) because that's what idiomatic Python training data looks like. That defaults to mediocrity; this rule overrides it.** + +**The 7 banned patterns** (any of these in a non-boundary file is an anti-pattern; the audit scripts flag them): + +| Banned | Why | Use instead | +|---|---|---| +| `dict[str, Any]` (parameter or return) | Open-ended; hides the schema; invites `.get('any_key', default)` defensive checks | A typed dataclass (`@dataclass(frozen=True, slots=True)`) with explicit fields | +| `Any` (parameter, return, or field) | Same problem; LLMs use it to avoid thinking about types | A specific typed dataclass or one of the concrete types in `src/type_aliases.py` | +| `Optional[T]` (return) | `None` requires a runtime check; propagates through call sites | `Result[T]` (with errors as data) or a `NIL_T` sentinel (zero-initialized frozen dataclass) | +| `hasattr(x, 'field')` for entity type dispatch | Runtime type check; defeats the type system | `isinstance(x, TypedDataclass)` against a typed Union, or refactor so the function takes a typed parameter (no dispatch needed) | +| `getattr(x, 'field', default)` on a known-typed value | Same; the type system should guarantee the field exists | `x.field` direct access; if the field is nullable, the dataclass has `Optional[T]` as a field type (and the value is checked at construction, not at every read) | +| `.get('field', default)` on a `dict[str, Any]` for a known field | Runtime type-dispatch branch | Direct attribute access on the typed dataclass | +| `if 'field' in dict` checks | Same | Direct attribute access (the dataclass has a default value) | + +**The one exception (the boundary layer):** at the literal wire boundary (TOML parsing, JSON parsing, vendor SDK response parsing), the data is open-ended for the 100ns between parsing and `from_dict()` conversion. At that boundary: + +- The function that calls `tomllib.load()` or `json.loads()` may return `Metadata` (the typed fat struct — see §8.6). +- Every consumer of that function IMMEDIATELY calls `SomeTypedDataclass.from_dict(metadata)` and uses the typed result. +- The boundary is 2-3 functions per file (one per wire entry point). + +**No other code uses `Metadata` or `dict[str, Any]` or `Any`.** This is enforced by `scripts/audit_weak_types.py --strict` (existing) + the boundary-layer audit (planned in `conductor/tracks/cruft_elimination_20260627/spec.md`). \ No newline at end of file