diff --git a/conductor/tracks/intent_dsl_survey_20260612/plan.md b/conductor/tracks/intent_dsl_survey_20260612/plan.md index 20cd1fe7..e36d4370 100644 --- a/conductor/tracks/intent_dsl_survey_20260612/plan.md +++ b/conductor/tracks/intent_dsl_survey_20260612/plan.md @@ -524,7 +524,7 @@ For each verb, the 5-column table: - "Borrowed from" note - SSDL shape tag -Tier 1 SSDL shape tags: most are `[I]` (single instruction) since they're scalar operations; the range `for x .. n` is `o==>` (codecycle). +Tier 1 SSDL shape tags: most are `[I]` (single instruction) since they're scalar operations; the range `for x .. n` is `o->` (codecycle). ### Task 16: Write section 4 Tier 2 (data pipeline) @@ -545,7 +545,7 @@ For each verb, the 5-column table (with one extra column: "maps to mcp_client to - "Borrowed from" note - SSDL shape tag -Tier 2 SSDL shape tags: `scan` is `[I]`, `filter`/`map` are `===>` (codepath) or `o==>` (codecycle), `arena { }` is `o==>` (codecycle), `pipe` is `===>W===>` (wide codepath). +Tier 2 SSDL shape tags: `scan` is `[I]`, `filter`/`map` are `->` (codepath) or `o->` (codecycle), `arena { }` is `o->` (codecycle), `pipe` is `=>` (wide codepath). ### Task 17: Write section 4 Tier 3 (shell) @@ -558,7 +558,7 @@ Per spec §3.4 Tier 3, the verbs are: `exec`, `open`, `read`, `write`, `close`, For each verb, the 5-column table (with "maps to mcp_client tool" and "maps to shell_runner" columns). -Tier 3 SSDL shape tags: most are `[I]` (single instruction); `wait`/`poll` are `o==>` (codecycle, blocking or polling). +Tier 3 SSDL shape tags: most are `[I]` (single instruction); `wait`/`poll` are `o->` (codecycle, blocking or polling). ### Task 18: Write section 4 Tier 4 (AI-fuzzing tolerance) @@ -579,7 +579,7 @@ This is the **novel tier** — give it extra space. For each verb: - "Borrowed from" note - SSDL shape tag -Tier 4 SSDL shape tags: most are `[I]` (single instruction); `try { ... } recover { ... }` is `===>B===>` (codepath with branch); `sandbox { ... }` is `o==>` (codecycle, the IEventTarget boundary); `assumewide` is `===>W===>` (wide codepath assumption). +Tier 4 SSDL shape tags: most are `[I]` (single instruction); `try { ... } recover { ... }` is `->B->` (codepath with branch); `sandbox { ... }` is `o->` (codecycle, the IEventTarget boundary); `assumewide` is `=>` (wide codepath assumption). ### Task 19: Write section 5 (hardware mapping) diff --git a/conductor/tracks/intent_dsl_survey_20260612/report.md b/conductor/tracks/intent_dsl_survey_20260612/report.md index 269eb29e..4b3679bb 100644 --- a/conductor/tracks/intent_dsl_survey_20260612/report.md +++ b/conductor/tracks/intent_dsl_survey_20260612/report.md @@ -150,10 +150,10 @@ The detailed analysis for each cluster lives in the research sub-reports at `res | # | Shape | One-line definition | SSDL symbol | |---|---|---|---| | 1 | **Instruction** | A single unit of computation. Reads data, writes data, or both. | `[I]` | -| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `===>` | -| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `===>W===>` | -| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o==>` | -| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `oo==>oo` | +| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `->` | +| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `=>` | +| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o->` | +| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `o=>` | | 6 | **Codecycle graph** | Multiple codecycles + the data they read and write. | `boxes + arrows` | **The 7 modifiers:** @@ -168,7 +168,7 @@ The detailed analysis for each cluster lives in the research sub-reports at `res | `[N]` | nil sentinel | A special value that satisfies "is this OK to use?" in all cases | | `───` | data | A line representing data being read or written (not a codepath) | -**How the DSL uses SSDL tags.** Each verb in §4 has a "Shape" column with an SSDL tag. For example, `sum` is `[I]` (single instruction); `for x .. n` is `o==>` (codecycle); `arena { }` is a sub-codepath scope; `pipe` is `===>W===>` (wide codepath, the chain can fan out); the entire DSL pipeline is a codecycle graph (multiple codecycles + the data they read and write). This lets the reader see the *shape* of a pipeline at a glance. +**How the DSL uses SSDL tags.** Each verb in §4 has a "Shape" column with an SSDL tag. For example, `sum` is `[I]` (single instruction); `for x .. n` is `o->` (codecycle); `arena { }` is a sub-codepath scope; `pipe` is `=>` (wide codepath, the chain can fan out); the entire DSL pipeline is a codecycle graph (multiple codecycles + the data they read and write). This lets the reader see the *shape* of a pipeline at a glance. ### Cluster 6 — Project's Own Command DSL Precedents @@ -315,7 +315,7 @@ The Tier 1 verbs are drawn directly from the user's math pseudocode. |---|---|---|---|---|---|---| | `:=` | Local bind | `name := expr` | Stack-scoped local declaration | `result := Matrix(m.rows -1, m.columns -1)` | Forth (dictionary entries); Joy (quotations) | `[I]` | | `stack { ... }` | Stack scope | `stack { decl1; decl2; ... }` | Block of stack-allocated locals | `stack { result := ...; row_offset, col_offset := Scalar; }` | Forth (colon definitions); KYRA (basic blocks) | `[I]` | -| `for x .. n` | Range iteration | `for x .. n { body }` | Iterate `x` over `[0, n)` | `for col .. m.columns` | APL `ιN`; K `!R`; BQN `↕N`; Uiua (stack iteration) | `o==>` | +| `for x .. n` | Range iteration | `for x .. n { body }` | Iterate `x` over `[0, n)` | `for col .. m.columns` | APL `ιN`; K `!R`; BQN `↕N`; Uiua (stack iteration) | `o->` | | `+` | Add | `a + b` | Element-wise sum | `2 + 3` (yields 5) | All languages | `[I]` | | `-` | Subtract | `a - b` | Element-wise difference | `5 - 2` (yields 3) | All languages | `[I]` | | `*` | Multiply | `a * b` | Element-wise product | `2 * 3` (yields 6) | All languages | `[I]` | @@ -335,17 +335,17 @@ The Tier 2 verbs wrap the existing 45+ MCP tools (per `docs/guide_tools.md` §"N | Symbol | Name | Signature | Semantics | Example | Maps to mcp_client tool | Borrowed from | Shape | |---|---|---|---|---|---|---|---| | `scan` | Scan | `scan path` | Read source (directory, file, URL); first verb in every pipeline | `scan "src/" -> filter !dir -> map ext` | `list_directory` + `search_files` + `read_file` | Jofito `scandir()` | `[I]` | -| `select` | Select | `select condition` | Keep records matching condition (jq-style filter) | `scan "src/" -> select .extension == ".py"` | (jq-style filter) | jq `select(condition)`; Joy `filter` | `===>` | -| `filter` | Filter | `filter predicate` | Keep records where predicate is true | `scan "src/" -> filter .size > 0` | (predicate on FileItem) | Jofito `{filter ...}` predicate | `===>` | -| `map` | Map | `map block` | Apply block to each record | `scan "src/" -> map ext` | (no direct equivalent) | jq `.[] | .field`; Joy `map`; CoSy `' verb 'm` | `o==>` | -| `fold` | Fold | `fold init block` | Reduce to single value | `scan "src/" -> fold 0 { acc + .size }` | (no direct equivalent) | jq `reduce`; Joy `fold` | `o==>` | +| `select` | Select | `select condition` | Keep records matching condition (jq-style filter) | `scan "src/" -> select .extension == ".py"` | (jq-style filter) | jq `select(condition)`; Joy `filter` | `->` | +| `filter` | Filter | `filter predicate` | Keep records where predicate is true | `scan "src/" -> filter .size > 0` | (predicate on FileItem) | Jofito `{filter ...}` predicate | `->` | +| `map` | Map | `map block` | Apply block to each record | `scan "src/" -> map ext` | (no direct equivalent) | jq `.[] | .field`; Joy `map`; CoSy `' verb 'm` | `o->` | +| `fold` | Fold | `fold init block` | Reduce to single value | `scan "src/" -> fold 0 { acc + .size }` | (no direct equivalent) | jq `reduce`; Joy `fold` | `o->` | | `sort` | Sort | `sort key` | Order records by key | `scan "src/" -> sort .name` | (no direct equivalent) | Joy `qsort`; jq `sort` | `[I]` | -| `group` | Group | `group key` | Bucket records by key | `scan "src/" -> group .extension` | (no direct equivalent) | jq `group_by`; CoSy APL-derived | `o==>` | +| `group` | Group | `group key` | Bucket records by key | `scan "src/" -> group .extension` | (no direct equivalent) | jq `group_by`; CoSy APL-derived | `o->` | | `dedupe` | Dedupe | `dedupe` | Remove duplicates | `scan "src/" -> dedupe` | (no direct equivalent) | jq `unique`; CoSy | `[I]` | -| `arena { }` | Arena scope | `arena { body }` | Tape-drive region; pre-scatter contents | `arena { [ scan ]; [ filter ]; [ print ] }` | (compiler directive) | KYRA magenta pipe; Onat preemptive scatter | `o==>` | -| `scatter` | Scatter | `scatter workers` | Fork pipeline across `workers` cores | `scan "src/" -> scatter 4 -> filter` | (runtime hint) | Onat preemptive scatter; Lottes X.com thread line 55-61 | `===>W===>` | +| `arena { }` | Arena scope | `arena { body }` | Tape-drive region; pre-scatter contents | `arena { [ scan ]; [ filter ]; [ print ] }` | (compiler directive) | KYRA magenta pipe; Onat preemptive scatter | `o->` | +| `scatter` | Scatter | `scatter workers` | Fork pipeline across `workers` cores | `scan "src/" -> scatter 4 -> filter` | (runtime hint) | Onat preemptive scatter; Lottes X.com thread line 55-61 | `=>` | | `gather` | Gather | `gather` | Collect scattered sub-streams | `scan "src/" -> scatter 4 -> filter -> gather` | (runtime hint) | Onat inverse of scatter | `[I]` | -| `pipe` | Pipe root | `pipe` | Explicit chain root (synonym for `->`) | `pipe [ scan, filter, print ]` | (no direct equivalent) | Jofito pipe coalescing (transcript:376-410) | `===>W===>` | +| `pipe` | Pipe root | `pipe` | Explicit chain root (synonym for `->`) | `pipe [ scan, filter, print ]` | (no direct equivalent) | Jofito pipe coalescing (transcript:376-410) | `=>` | **Total Tier 2: 12 verbs.** @@ -362,8 +362,8 @@ The Tier 3 verbs wrap existing MCP tools (per `docs/guide_tools.md` §"Native To | `close` | Close | `close handle` | Close handle | `close file_handle` | (no direct equivalent; close is implicit in Python) | Forth `CLOSE-FILE`; bash `exec` | `[I]` | | `path` | Path | `path` | Get current path (or `cd`) | `path` | (no direct equivalent; use `cwd`) | shell `pwd`; CoSy `path` | `[I]` | | `env` | Env | `env var` | Get env var | `env HOME` | (no direct equivalent) | shell `echo $HOME` | `[I]` | -| `wait` | Wait | `wait ms` | Block for `ms` milliseconds | `wait 1000` | (no direct equivalent) | shell `sleep` | `o==>` | -| `poll` | Poll | `poll handle ms` | Poll handle with timeout | `poll file_handle 5000` | (no direct equivalent) | shell `read -t` | `o==>` | +| `wait` | Wait | `wait ms` | Block for `ms` milliseconds | `wait 1000` | (no direct equivalent) | shell `sleep` | `o->` | +| `poll` | Poll | `poll handle ms` | Poll handle with timeout | `poll file_handle 5000` | (no direct equivalent) | shell `read -t` | `o->` | | `cwd` | CWD | `cwd` | Get current working directory | `cwd` | (no direct equivalent) | shell `pwd` | `[I]` | **Total Tier 3: 10 verbs.** @@ -374,14 +374,14 @@ The Tier 4 verbs are what make the DSL workable for AI agents that may fuzz verb | Symbol | Name | Signature | Semantics | Example | Novel piece | Borrowed from | Shape | |---|---|---|---|---|---|---|---| -| `fuzzy` | Fuzzy | `fuzzy expr` | Declare a parse-tolerance region; parser accepts near-matches | `fuzzy { scan "src/" -> filter .ext }` | Tolerance for AI verb-name fuzzing | nagent "discovery" intent (per `decisions.md:119,128`); SSDL "assume as much as possible" | `===>` | -| `try { ... } recover { ... }` | Try / Recover | `try { body } recover err { fallback }` | Returns `Result[T]`; on error, the `recover` block runs | `try { read "src/foo.py" } recover { read "src/Foo.py" }` | Error envelope as data (Fleury pattern) | `data_oriented_error_handling_20260606`; Wasm `try`/`catch` block/loop/if/end | `===>B===>` | -| `sandbox { ... }` | Sandbox | `sandbox { body }` | IEventTarget boundary; all writes in the block go through the formal event channel | `sandbox { write "tmp/x" "data" }` | O'Donnell's "reads free, writes formalized" invariant applied to the DSL | O'Donnell `mvc.html` "Writing to Model state" | `o==>` | +| `fuzzy` | Fuzzy | `fuzzy expr` | Declare a parse-tolerance region; parser accepts near-matches | `fuzzy { scan "src/" -> filter .ext }` | Tolerance for AI verb-name fuzzing | nagent "discovery" intent (per `decisions.md:119,128`); SSDL "assume as much as possible" | `->` | +| `try { ... } recover { ... }` | Try / Recover | `try { body } recover err { fallback }` | Returns `Result[T]`; on error, the `recover` block runs | `try { read "src/foo.py" } recover { read "src/Foo.py" }` | Error envelope as data (Fleury pattern) | `data_oriented_error_handling_20260606`; Wasm `try`/`catch` block/loop/if/end | `->B->` | +| `sandbox { ... }` | Sandbox | `sandbox { body }` | IEventTarget boundary; all writes in the block go through the formal event channel | `sandbox { write "tmp/x" "data" }` | O'Donnell's "reads free, writes formalized" invariant applied to the DSL | O'Donnell `mvc.html` "Writing to Model state" | `o->` | | `audit` | Audit | `audit msg` | Log the state change to a structured record; the IEventTarget itself | `audit "wrote tmp/x"` | Per-write audit log; full replay capability | O'Donnell `mvc.html` "Event callbacks"; nagent's self-describing tools | `[I]` | | `didyoumean` | Did you mean | `didyoumean ambiguous` | Propose the closest matching verb(s) for an ambiguous input | `didyoumean "skan"` | Recovery primitive for AI typos | nagent Bridge DSL intent model; Anthropic `input_examples` | `[I]` | | `span` | Span | `span intent` | Decompose a compound intent into a span of sub-MCP grammar tokens | `span "read foo.py:MyClass"` | Spans the `read_file` and `py_get_definition` tools | MCP DSL per-MCP grammar (`spec.md:456-465`); OpenAI namespace grouping | `[I]` | | `offset` | Offset | `offset symbol` | Resolve a symbol to a file:line without requiring the model to specify the line | `offset "foo.py:MyClass.method"` | Implicit offset resolution | MCP DSL line-range notation; OpenAI "don't make the model fill known args" | `[Q]` | -| `assumewide` | Assume wide | `assumewide intent` | If the intent is broad or ambiguous, select the most-capable matching tool (the "fewer, more capable" heuristic) | `assumewide "refactor"` | Prefer broad-capability tools over narrow specialists | OpenAI "fewer than 20 functions"; Anthropic `tool_choice: tool` force-call | `===>W===>` | +| `assumewide` | Assume wide | `assumewide intent` | If the intent is broad or ambiguous, select the most-capable matching tool (the "fewer, more capable" heuristic) | `assumewide "refactor"` | Prefer broad-capability tools over narrow specialists | OpenAI "fewer than 20 functions"; Anthropic `tool_choice: tool` force-call | `=>` | **Total Tier 4: 8 verbs.** diff --git a/conductor/tracks/intent_dsl_survey_20260612/report_v1.1.md b/conductor/tracks/intent_dsl_survey_20260612/report_v1.1.md index 42060ada..fee7770f 100644 --- a/conductor/tracks/intent_dsl_survey_20260612/report_v1.1.md +++ b/conductor/tracks/intent_dsl_survey_20260612/report_v1.1.md @@ -154,10 +154,10 @@ The detailed analysis for each cluster lives in the research sub-reports at `res | # | Shape | One-line definition | SSDL symbol | |---|---|---|---| | 1 | **Instruction** | A single unit of computation. Reads data, writes data, or both. | `[I]` | -| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `===>` | -| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `===>W===>` | -| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o==>` | -| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `oo==>oo` | +| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `->` | +| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `=>` | +| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o->` | +| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `o=>` | | 6 | **Codecycle graph** | Multiple codecycles + the data they read and write. | `boxes + arrows` | **The 7 modifiers:** @@ -172,7 +172,7 @@ The detailed analysis for each cluster lives in the research sub-reports at `res | `[N]` | nil sentinel | A special value that satisfies "is this OK to use?" in all cases | | `───` | data | A line representing data being read or written (not a codepath) | -**How the DSL uses SSDL tags.** Each verb in §4 has a "Shape" column with an SSDL tag. For example, `sum` is `[I]` (single instruction); `for x .. n` is `o==>` (codecycle); `arena { }` is a sub-codepath scope; `pipe` is `===>W===>` (wide codepath, the chain can fan out); the entire DSL pipeline is a codecycle graph (multiple codecycles + the data they read and write). This lets the reader see the *shape* of a pipeline at a glance. +**How the DSL uses SSDL tags.** Each verb in §4 has a "Shape" column with an SSDL tag. For example, `sum` is `[I]` (single instruction); `for x .. n` is `o->` (codecycle); `arena { }` is a sub-codepath scope; `pipe` is `=>` (wide codepath, the chain can fan out); the entire DSL pipeline is a codecycle graph (multiple codecycles + the data they read and write). This lets the reader see the *shape* of a pipeline at a glance. ### Cluster 6 — Project's Own Command DSL Precedents @@ -319,7 +319,7 @@ The Tier 1 verbs are drawn directly from the user's math pseudocode. |---|---|---|---|---|---|---| | `:=` | Local bind | `name := expr` | Stack-scoped local declaration | `result := Matrix(m.rows -1, m.columns -1)` | Forth (dictionary entries); Joy (quotations) | `[I]` | | `stack { ... }` | Stack scope | `stack { decl1; decl2; ... }` | Block of stack-allocated locals | `stack { result := ...; row_offset, col_offset := Scalar; }` | Forth (colon definitions); KYRA (basic blocks) | `[I]` | -| `for x .. n` | Range iteration | `for x .. n { body }` | Iterate `x` over `[0, n)` | `for col .. m.columns` | APL `ιN`; K `!R`; BQN `↕N`; Uiua (stack iteration) | `o==>` | +| `for x .. n` | Range iteration | `for x .. n { body }` | Iterate `x` over `[0, n)` | `for col .. m.columns` | APL `ιN`; K `!R`; BQN `↕N`; Uiua (stack iteration) | `o->` | | `+` | Add | `a + b` | Element-wise sum | `2 + 3` (yields 5) | All languages | `[I]` | | `-` | Subtract | `a - b` | Element-wise difference | `5 - 2` (yields 3) | All languages | `[I]` | | `*` | Multiply | `a * b` | Element-wise product | `2 * 3` (yields 6) | All languages | `[I]` | @@ -339,17 +339,17 @@ The Tier 2 verbs wrap the existing 45+ MCP tools (per `docs/guide_tools.md` §"N | Symbol | Name | Signature | Semantics | Example | Maps to mcp_client tool | Borrowed from | Shape | |---|---|---|---|---|---|---|---| | `scan` | Scan | `scan path` | Read source (directory, file, URL); first verb in every pipeline | `scan "src/" -> filter !dir -> map ext` | `list_directory` + `search_files` + `read_file` | Jofito `scandir()` | `[I]` | -| `select` | Select | `select condition` | Keep records matching condition (jq-style filter) | `scan "src/" -> select .extension == ".py"` | (jq-style filter) | jq `select(condition)`; Joy `filter` | `===>` | -| `filter` | Filter | `filter predicate` | Keep records where predicate is true | `scan "src/" -> filter .size > 0` | (predicate on FileItem) | Jofito `{filter ...}` predicate | `===>` | -| `map` | Map | `map block` | Apply block to each record | `scan "src/" -> map ext` | (no direct equivalent) | jq `.[] | .field`; Joy `map`; CoSy `' verb 'm` | `o==>` | -| `fold` | Fold | `fold init block` | Reduce to single value | `scan "src/" -> fold 0 { acc + .size }` | (no direct equivalent) | jq `reduce`; Joy `fold` | `o==>` | +| `select` | Select | `select condition` | Keep records matching condition (jq-style filter) | `scan "src/" -> select .extension == ".py"` | (jq-style filter) | jq `select(condition)`; Joy `filter` | `->` | +| `filter` | Filter | `filter predicate` | Keep records where predicate is true | `scan "src/" -> filter .size > 0` | (predicate on FileItem) | Jofito `{filter ...}` predicate | `->` | +| `map` | Map | `map block` | Apply block to each record | `scan "src/" -> map ext` | (no direct equivalent) | jq `.[] | .field`; Joy `map`; CoSy `' verb 'm` | `o->` | +| `fold` | Fold | `fold init block` | Reduce to single value | `scan "src/" -> fold 0 { acc + .size }` | (no direct equivalent) | jq `reduce`; Joy `fold` | `o->` | | `sort` | Sort | `sort key` | Order records by key | `scan "src/" -> sort .name` | (no direct equivalent) | Joy `qsort`; jq `sort` | `[I]` | -| `group` | Group | `group key` | Bucket records by key | `scan "src/" -> group .extension` | (no direct equivalent) | jq `group_by`; CoSy APL-derived | `o==>` | +| `group` | Group | `group key` | Bucket records by key | `scan "src/" -> group .extension` | (no direct equivalent) | jq `group_by`; CoSy APL-derived | `o->` | | `dedupe` | Dedupe | `dedupe` | Remove duplicates | `scan "src/" -> dedupe` | (no direct equivalent) | jq `unique`; CoSy | `[I]` | -| `arena { }` | Arena scope | `arena { body }` | Tape-drive region; pre-scatter contents | `arena { [ scan ]; [ filter ]; [ print ] }` | (compiler directive) | KYRA magenta pipe; Onat preemptive scatter | `o==>` | -| `scatter` | Scatter | `scatter workers` | Fork pipeline across `workers` cores | `scan "src/" -> scatter 4 -> filter` | (runtime hint) | Onat preemptive scatter; Lottes X.com thread line 55-61 | `===>W===>` | +| `arena { }` | Arena scope | `arena { body }` | Tape-drive region; pre-scatter contents | `arena { [ scan ]; [ filter ]; [ print ] }` | (compiler directive) | KYRA magenta pipe; Onat preemptive scatter | `o->` | +| `scatter` | Scatter | `scatter workers` | Fork pipeline across `workers` cores | `scan "src/" -> scatter 4 -> filter` | (runtime hint) | Onat preemptive scatter; Lottes X.com thread line 55-61 | `=>` | | `gather` | Gather | `gather` | Collect scattered sub-streams | `scan "src/" -> scatter 4 -> filter -> gather` | (runtime hint) | Onat inverse of scatter | `[I]` | -| `pipe` | Pipe root | `pipe` | Explicit chain root (synonym for `->`) | `pipe [ scan, filter, print ]` | (no direct equivalent) | Jofito pipe coalescing (transcript:376-410) | `===>W===>` | +| `pipe` | Pipe root | `pipe` | Explicit chain root (synonym for `->`) | `pipe [ scan, filter, print ]` | (no direct equivalent) | Jofito pipe coalescing (transcript:376-410) | `=>` | **Total Tier 2: 12 verbs.** @@ -366,8 +366,8 @@ The Tier 3 verbs wrap existing MCP tools (per `docs/guide_tools.md` §"Native To | `close` | Close | `close handle` | Close handle | `close file_handle` | (no direct equivalent; close is implicit in Python) | Forth `CLOSE-FILE`; bash `exec` | `[I]` | | `path` | Path | `path` | Get current path (or `cd`) | `path` | (no direct equivalent; use `cwd`) | shell `pwd`; CoSy `path` | `[I]` | | `env` | Env | `env var` | Get env var | `env HOME` | (no direct equivalent) | shell `echo $HOME` | `[I]` | -| `wait` | Wait | `wait ms` | Block for `ms` milliseconds | `wait 1000` | (no direct equivalent) | shell `sleep` | `o==>` | -| `poll` | Poll | `poll handle ms` | Poll handle with timeout | `poll file_handle 5000` | (no direct equivalent) | shell `read -t` | `o==>` | +| `wait` | Wait | `wait ms` | Block for `ms` milliseconds | `wait 1000` | (no direct equivalent) | shell `sleep` | `o->` | +| `poll` | Poll | `poll handle ms` | Poll handle with timeout | `poll file_handle 5000` | (no direct equivalent) | shell `read -t` | `o->` | | `cwd` | CWD | `cwd` | Get current working directory | `cwd` | (no direct equivalent) | shell `pwd` | `[I]` | **Total Tier 3: 10 verbs.** @@ -378,14 +378,14 @@ The Tier 4 verbs are what make the DSL workable for AI agents that may fuzz verb | Symbol | Name | Signature | Semantics | Example | Novel piece | Borrowed from | Shape | |---|---|---|---|---|---|---|---| -| `fuzzy` | Fuzzy | `fuzzy expr` | Declare a parse-tolerance region; parser accepts near-matches | `fuzzy { scan "src/" -> filter .ext }` | Tolerance for AI verb-name fuzzing | nagent "discovery" intent (per `decisions.md:119,128`); SSDL "assume as much as possible" | `===>` | -| `try { ... } recover { ... }` | Try / Recover | `try { body } recover err { fallback }` | Returns `Result[T]`; on error, the `recover` block runs | `try { read "src/foo.py" } recover { read "src/Foo.py" }` | Error envelope as data (Fleury pattern) | `data_oriented_error_handling_20260606`; Wasm `try`/`catch` block/loop/if/end | `===>B===>` | -| `sandbox { ... }` | Sandbox | `sandbox { body }` | IEventTarget boundary; all writes in the block go through the formal event channel | `sandbox { write "tmp/x" "data" }` | O'Donnell's "reads free, writes formalized" invariant applied to the DSL | O'Donnell `mvc.html` "Writing to Model state" | `o==>` | +| `fuzzy` | Fuzzy | `fuzzy expr` | Declare a parse-tolerance region; parser accepts near-matches | `fuzzy { scan "src/" -> filter .ext }` | Tolerance for AI verb-name fuzzing | nagent "discovery" intent (per `decisions.md:119,128`); SSDL "assume as much as possible" | `->` | +| `try { ... } recover { ... }` | Try / Recover | `try { body } recover err { fallback }` | Returns `Result[T]`; on error, the `recover` block runs | `try { read "src/foo.py" } recover { read "src/Foo.py" }` | Error envelope as data (Fleury pattern) | `data_oriented_error_handling_20260606`; Wasm `try`/`catch` block/loop/if/end | `->B->` | +| `sandbox { ... }` | Sandbox | `sandbox { body }` | IEventTarget boundary; all writes in the block go through the formal event channel | `sandbox { write "tmp/x" "data" }` | O'Donnell's "reads free, writes formalized" invariant applied to the DSL | O'Donnell `mvc.html` "Writing to Model state" | `o->` | | `audit` | Audit | `audit msg` | Log the state change to a structured record; the IEventTarget itself | `audit "wrote tmp/x"` | Per-write audit log; full replay capability | O'Donnell `mvc.html` "Event callbacks"; nagent's self-describing tools | `[I]` | | `didyoumean` | Did you mean | `didyoumean ambiguous` | Propose the closest matching verb(s) for an ambiguous input | `didyoumean "skan"` | Recovery primitive for AI typos | nagent Bridge DSL intent model; Anthropic `input_examples` | `[I]` | | `span` | Span | `span intent` | Decompose a compound intent into a span of sub-MCP grammar tokens | `span "read foo.py:MyClass"` | Spans the `read_file` and `py_get_definition` tools | MCP DSL per-MCP grammar (`spec.md:456-465`); OpenAI namespace grouping | `[I]` | | `offset` | Offset | `offset symbol` | Resolve a symbol to a file:line without requiring the model to specify the line | `offset "foo.py:MyClass.method"` | Implicit offset resolution | MCP DSL line-range notation; OpenAI "don't make the model fill known args" | `[Q]` | -| `assumewide` | Assume wide | `assumewide intent` | If the intent is broad or ambiguous, select the most-capable matching tool (the "fewer, more capable" heuristic) | `assumewide "refactor"` | Prefer broad-capability tools over narrow specialists | OpenAI "fewer than 20 functions"; Anthropic `tool_choice: tool` force-call | `===>W===>` | +| `assumewide` | Assume wide | `assumewide intent` | If the intent is broad or ambiguous, select the most-capable matching tool (the "fewer, more capable" heuristic) | `assumewide "refactor"` | Prefer broad-capability tools over narrow specialists | OpenAI "fewer than 20 functions"; Anthropic `tool_choice: tool` force-call | `=>` | **Total Tier 4: 8 verbs.** @@ -749,10 +749,10 @@ Per `docs/reports/computational_shapes_ssdl_digest_20260608.md` §1, the SSDL vo | # | Shape | One-line definition | SSDL symbol | |---|---|---|---| | 1 | **Instruction** | A single unit of computation. Reads data, writes data, or both. | `[I]` | -| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `===>` | -| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `===>W===>` | -| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o==>` | -| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `oo==>oo` | +| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `->` | +| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `=>` | +| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o->` | +| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `o=>` | | 6 | **Codecycle graph** | Multiple codecycles + the data they read and write. | `boxes + arrows` | **The 7 modifiers** (full table): @@ -1080,7 +1080,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - `for col .. m.columns { body }` (per user's pseudocode line 33) - `for i .. 10 { print i }` (prints 0-9) - `for i .. arr.size { sum +:= arr[i] }` (sum a vector) -- SSDL shape: `o==>` (codecycle — loops back to first instruction after last) +- SSDL shape: `o->` (codecycle — loops back to first instruction after last) **`name[a, b, ...]` (Bracket indexing).** - Signature: `name[i, j, k, ...]` @@ -1112,7 +1112,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - `sum 1..10` (yields 55; per APL `+/ι10`) - `product 1..5` (yields 120; per APL `×/ι5`) - `sum arr` (sum an arbitrary array) -- SSDL shape: `o==>` (codecycle; iterates over elements) +- SSDL shape: `o->` (codecycle; iterates over elements) **`if cond { then } [else { else }]` (Conditional).** - Signature: `if expression "{" program "}" [ "else" "{" program "}" ]` @@ -1143,7 +1143,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> select .extension == ".py"` - `scan "src/" -> select .size > 1024` -- SSDL shape: `===>` (codepath) +- SSDL shape: `->` (codepath) **`filter predicate` (Keep matching).** - Signature: `filter predicate` @@ -1152,7 +1152,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> filter .size > 0` - `scan "src/" -> filter exists` -- SSDL shape: `===>` +- SSDL shape: `->` **`map block` (Apply to each).** - Signature: `map block` @@ -1161,7 +1161,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> map ext` (yields list of extensions) - `scan "src/" -> map { x := .name; "src/" + x }` (full paths) -- SSDL shape: `o==>` (codecycle; iterates over records) +- SSDL shape: `o->` (codecycle; iterates over records) **`fold init block` (Reduce to single).** - Signature: `fold init_value block` @@ -1170,7 +1170,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> fold 0 { acc + .size }` (total bytes) - `scan "src/" -> fold "" { acc + .name + "\n" }` (concatenated names) -- SSDL shape: `o==>` +- SSDL shape: `o->` **`sort key` (Order).** - Signature: `sort key` @@ -1188,7 +1188,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> group .extension` (group by file extension) - `scan "src/" -> group .directory` (group by parent directory) -- SSDL shape: `o==>` (codecycle) +- SSDL shape: `o->` (codecycle) **`dedupe` (Remove duplicates).** - Signature: `dedupe` @@ -1206,7 +1206,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `arena { [ scan ]; [ filter ]; [ print ] }` (3 basic blocks in a tape region) - `arena { x := 42; y := x * 2; use y }` (local tape) -- SSDL shape: `o==>` (codecycle; the region is a single execution unit) +- SSDL shape: `o->` (codecycle; the region is a single execution unit) **`scatter workers` (Fork).** - Signature: `scatter workers_expr` @@ -1215,7 +1215,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> scatter 4 -> filter` (4-way parallel) - `scan "src/" -> scatter auto -> map` (auto-detect core count) -- SSDL shape: `===>W===>` (wide codepath) +- SSDL shape: `=>` (wide codepath) **`gather` (Collect).** - Signature: `gather` @@ -1232,7 +1232,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `pipe [ scan, filter, print ]` (declarative list form; not the canonical syntax, future extension) - `pipe` (single token; equivalent to starting a `->` chain) -- SSDL shape: `===>W===>` +- SSDL shape: `=>` #### A.4.3 Tier 3 — Shell (10 verbs, full reference) @@ -1305,7 +1305,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `wait 1000` (wait 1 second) - `wait 50` (wait 50ms; useful for pacing a render loop) -- SSDL shape: `o==>` (codecycle) +- SSDL shape: `o->` (codecycle) **`poll handle ms` (Poll).** - Signature: `poll handle_expr ms_expr` @@ -1313,7 +1313,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Maps to: (no direct equivalent; `shell read -t`) - Examples: - `poll file_handle 5000` (poll with 5s timeout) -- SSDL shape: `o==>` +- SSDL shape: `o->` **`cwd` (Current working dir).** - Signature: `cwd` @@ -1332,7 +1332,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `fuzzy { scan "src/" -> filter .ext }` (accepts `skan` as a typo for `scan`) - `fuzzy { skan "src" }` (accepts the typo) -- SSDL shape: `===>` +- SSDL shape: `->` **`try { body } recover err { fallback }` (Try / Recover).** - Signature: `try "{" program "}" "recover" identifier "{" program "}"` @@ -1341,7 +1341,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `try { read "src/foo.py" } recover { read "src/Foo.py" }` (try capitalized variant) - `try { read "src/foo.py" -> filter !exists -> print } recover err { audit "scan failed: " + err; return NIL }` -- SSDL shape: `===>B===>` (codepath with branch) +- SSDL shape: `->B->` (codepath with branch) **`sandbox { body }` (Sandbox).** - Signature: `sandbox "{" program "}"` @@ -1350,7 +1350,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `sandbox { write "tmp/x" "data" }` (write goes through HITL approval) - `sandbox { exec "rm -rf /" }` (destructive exec requires approval) -- SSDL shape: `o==>` (codecycle; the region is an audit unit) +- SSDL shape: `o->` (codecycle; the region is an audit unit) **`audit msg` (Audit log).** - Signature: `audit message_expr` @@ -1393,7 +1393,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `assumewide "refactor"` (selects the broadest refactor tool) - `assumewide "find"` (selects a broad search rather than a narrow query) -- SSDL shape: `===>W===>` +- SSDL shape: `=>` --- @@ -1642,9 +1642,9 @@ That's 4 verbs total, plus the grammar. The placeholder track can demonstrate a **clusters** — the 8 prior-art groups in §2: 0 (Immediate-Mode Paradigm), 1 (Concatenative), 2 (Array), 3 (Intent-Mapping), 4 (Meta-Tooling DSLs), 5 (SSDL), 6 (Command Palette), 7 (Result convention). -**codepath** (SSDL) — a sequential list of instructions that terminates; no loops. SSDL symbol: `===>`. +**codepath** (SSDL) — a sequential list of instructions that terminates; no loops. SSDL symbol: `->`. -**codecycle** (SSDL) — a circular structure; a codepath that repeats at its first instruction after its last. SSDL symbol: `o==>`. +**codecycle** (SSDL) — a circular structure; a codepath that repeats at its first instruction after its last. SSDL symbol: `o->`. **DSL** — Domain-Specific Language. In this report, "DSL" refers specifically to the intent-based scripting language proposed in §3 and §4. @@ -1678,7 +1678,7 @@ That's 4 verbs total, plus the grammar. The placeholder track can demonstrate a **sandbox** — a Tier 4 verb that declares an IEventTarget boundary. All writes inside `sandbox { ... }` go through the formal event channel (HITL approval). -**SSDL** — Spec/Sketch Description Language, per `docs/reports/computational_shapes_ssdl_digest_20260608.md`. The DSL's verbs are annotated with SSDL shape tags (`[I]`, `===>`, `o==>`, `===>W===>`). +**SSDL** — Spec/Sketch Description Language, per `docs/reports/computational_shapes_ssdl_digest_20260608.md`. The DSL's verbs are annotated with SSDL shape tags (`[I]`, `->`, `o->`, `=>`). **tag protocol** — nagent's XML-ish self-closing tags for tool invocation. The DSL inherits the *idea* (named operation with typed attributes) but rejects the XML angle-bracket notation. @@ -1732,7 +1732,7 @@ The bibliography is split into 4 categories: external prior art, project's own r - **Anthropic tool-use** — `https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/define-tools`. *Key claim:* `input_examples` as a first-class schema field; `strict: true` as a guarantee. **Cluster 5 — SSDL:** -- **`docs/reports/computational_shapes_ssdl_digest_20260608.md` §1** — (6 primitives + 7 modifiers). *Key claim:* the meta-vocabulary for annotating verb shapes (`[I]`, `===>`, `o==>`, `===>W===>`). +- **`docs/reports/computational_shapes_ssdl_digest_20260608.md` §1** — (6 primitives + 7 modifiers). *Key claim:* the meta-vocabulary for annotating verb shapes (`[I]`, `->`, `o->`, `=>`). **Cluster 6 — Command Palette:** - **`docs/guide_command_palette.md`** + **`src/commands.py`** (33 commands). *Key claim:* the user's existing vocabulary instinct; the DSL is a richer superset. diff --git a/conductor/tracks/intent_dsl_survey_20260612/report_v1.2.md b/conductor/tracks/intent_dsl_survey_20260612/report_v1.2.md index 1e68073d..ae518ee7 100644 --- a/conductor/tracks/intent_dsl_survey_20260612/report_v1.2.md +++ b/conductor/tracks/intent_dsl_survey_20260612/report_v1.2.md @@ -1,237 +1,239 @@ -# Intent-Based Scripting Languages - -**Track:** `intent_dsl_survey_20260612` (initialized 2026-06-12) -**Date:** 2026-06-12 -**Location:** `conductor/tracks/intent_dsl_survey_20260612/report.md` (this file; moved from `docs/ideation/` per user instruction — the report is too closely related to the track to live in the general ideation folder) -**Author:** Tier 1 Orchestrator (sections 1, 3, 4, 5, 6, 7, Appendix); Tier 2 sub-agents (section 2 clusters 0-4, with research sub-reports at `research/cluster_*.md`) -> **v1.2 changes (2026-06-12):** (1) Renamed `arena { }` to `tape { }` (better term in hindsight; aligns syntax with the Lottes tape-drive metaphor). All 46 occurrences of `arena` updated; 3 awkward double-tape phrases cleaned up. (2) **Mixed postfix/infix notation introduced** (per user heuristic): strictly postfix for math primitives that have precedence (`+`, `-`, `*`, `/`, `^`, math indexing `[]`); infix for everything else (`:=`, function calls, control flow, field access, block delimiters). The rationale: postfix eliminates precedence ambiguity; infix is more familiar where precedence isn't an issue. The two mix freely: `result := a b +` is canonical (assignment infix, math postfix). (3) **nagent attribution corrected:** the glossary and Cluster 3 entry previously said nagent is Jody Bruchon's; it is actually Mike Acton's (`github.com/macton/nagent`; per `conductor/tracks/nagent_review_20260608/`). Jofito remains correctly attributed to Jody Bruchon. See TODO at the top of §3 for the full postfix heuristic table. Appendix A.3/A.4 retain their v1.1 form pending a sync pass. (4) **Added Cluster 8 (Metadesk — Self-Describing Data + Tag Dispatch) and Cluster 9 (Verse — Multi-Paradigm Foundation Calculi with Transactional Semantics).** Survey now covers 10 prior-art clusters (was 8). Sub-reports at esearch/cluster_8_metadesk.md and esearch/cluster_9_verse.md. Decision per user: keep the new clusters separate (Cluster 8 ≠ Cluster 9 — they are different design patterns: narrow-waist data format vs language-theoretic foundation). +# Intent-Based Scripting Languages -**Status:** v1.1 (post-secondary-review correction; see `reportreview.md` for the review that produced this update) - -> **What this is.** A survey of intent-based scripting languages as a design philosophy, plus a proposed vocabulary (~40 verbs across 4 tiers) for a Meta-Tooling-facing intent DSL. The report is the foundation document for the user's nagent v2.2 (its "Future-Track Candidate #4" section) and for the future interpreter prototype (follow-up B track). -> -> **What this is NOT.** Not an interpreter, not a bridge script, not Application-side function-calling, not XML/JSON record formats. The DSL is Meta-Tooling-side per `docs/guide_meta_boundary.md` — the format external agents (Gemini CLI, OpenCode) emit when invoking `mcp_client.py` tools. The Application's provider-native function-calling stays unchanged. - ---- - -## 1. The "Intent-Based" Design Philosophy - -The DSL is grounded in four anchor claims. Each claim has a philosophical home and a specific design consequence for the vocab and grammar. - -### 1.1 Claim 1 — Intent-based means the user's words are declarative intent, not imperative commands - -Jofito (per its 2026 README update) calls itself an **"intent mapping engine"**: the user writes declarative intent (e.g., "find all pictures, filter out JPEGs, print the list"), and Jofito decomposes that intent into platform-optimal operations. From the Jofito README: *"jofito is a 'write the optimization once, reap the benefits everywhere' system that takes what the user wants to accomplish (intent) as input and decomposes it into operations that make the most sense for the current system."* (`https://codeberg.org/jbruchon/jofito`) - -The canonical Jofito example is `list = scandir("/path/here/", {filter !extension=jpg,jpeg}) : print(list)` — a single declarative expression that replaces `find . -type f | grep -v jpg | grep -v jpeg`. The DSL inherits this framing: the verbs in §4 are **intent verbs** (e.g., `scan` for "I want to read a source", `filter` for "I want to keep only what matches", `audit` for "I want to record what happened"), not imperative primitives. - -This is the *philosophical* anchor for the DSL: the user says *what they want*; the verbs are the way to say it; the bridge script and the MCP tools handle *how to do it*. The user's own math pseudocode (the `determinate`/`minor`/`matrix-transpose` snippets shared during spec review) operates at this declarative level — "here is the math, the verbs are the words." - -### 1.2 Claim 2 — The hardware is the truth - -The verbs must map to actual hardware/software stages, not abstract commands. The Onat/Lottes 2-register model (per `C:\projects\forth\bootslop\references\kyra_in-depth.md` and `X.com - Onat & Lottes Interaction 1.png.ocr.md`) gives the concrete hardware the DSL is mapped to: - -- **2-register stack (RAX/RDX)**: the DSL's `->` chain *maps* to RAX-passed data. Each verb in the chain is a "word" in Onat's sense (no args, no returns — the X.com thread at `X.com - Onat & Lottes Interaction 1.png.ocr.md:80-86` quotes Lottes: "I laugh when people say C is like assembly, they were missing what we did in assembly back then, which was all registers and globals and gotos, no stacks"). -- **Magenta pipe `|` (KYRA) → our `->`**: same definition-boundary semantics, retargeted to data flow. -- **Basic blocks `[ ]` (KYRA) → our `[ ]`**: compilation units; the parser produces a `[ ]` block per `->`-delimited stage. -- **Lambdas `{ }` (KYRA) → our `tape { }`**: tape-scoped blocks; the contents are pre-scattered into tape-drive regions (per the X.com thread at line 55-61, where Onat describes Lottes's "common arguments pushed onto the tape using store duplication when they are known... so it's preemptive scatter, so later at call time there is no argument gather"). - -The verbs are not arbitrary. Each Tier 2 verb (data pipeline) and Tier 3 verb (shell) has a direct hardware mapping; this is what makes the verbs *fast* on the targeted hardware. - -### 1.3 Claim 3 — The pipeline is immediate-mode - -Per John O'Donnell's IMGUI essay (`https://johno.se/book/imgui.html`): *"Widgets, logically, change from being objects to being method invocations."* The pipeline `scan -> filter -> print` is not a Pipeline object with state; it is a sequence of method calls. Once execution ends, the pipeline's state is gone. The next invocation is independent. - -This is the *paradigm* anchor for the DSL. It means: -- The parser doesn't need to track pipeline state across executions; each invocation is independent. -- The `->` chain has no "pipeline object" you can query, name, or pass around. The only way to "name" a chain is to wrap it in a function (`determinate(m, row) -> Scalar { ... }`). -- Verbs exist *only* when called. There is no implicit verb inventory. (This is why the DSL's "Everything" mode in the Command Palette is implementable as a search across *text*, not across a *registry of pipeline objects*.) - -O'Donnell's MVC essay (`https://johno.se/book/mvc.html`) extends this: *"Writes to Model are formalized through the addition of IEventTarget. This is a pure virtual interface that defines all possible state changes / events on a system wide level."* The DSL's `sandbox` verb is the IEventTarget boundary; the `audit` verb is the IEventTarget itself (see §6 Claim 9 and Claim 10). - -### 1.4 Claim 4 — The vocabulary IS the user surface - -CoSy (per `https://cosy.com/CoSy/Simplicity.html`): *"CoSy is a TimeStamped notebook/log created as an open vocabulary in Forth."* And: *"an extensive vocabulary evolved from APL via K, mainly slicing and dicing, searching & replacing, and applying verbs to each item in lists."* - -For the DSL, the **vocabulary** is the user surface — not the syntax, not the parser, not the runtime. For AI agents that emit the DSL, the vocab is the API. A model that knows the 40 verbs in §4 and the 14 grammar primitives in §3 can express any intent that the DSL supports. There is no separate "API documentation" — the verbs ARE the API. - -This is why the report devotes so much space to the vocab (§4) and so little to the syntax (§3). The syntax is trivial (RPN with a few delimiters); the vocabulary is the substance. - -### 1.5 The four claims together - -The four claims are not independent; they compose: - -- Claim 1 (intent-mapping) → the user expresses what they want; the verbs are the vocabulary. -- Claim 2 (hardware is the truth) → the verbs map to real data-oriented pipeline stages. -- Claim 3 (immediate-mode) → the verbs are method calls, not stateful objects; pipelines have no persistent state. -- Claim 4 (vocabulary is the user surface) → the 40-verb vocab is the API; the syntax is trivial. - -The composition is: a user expresses intent (Claim 1) using a verb (Claim 4) that maps to a hardware stage (Claim 2) in a single per-frame composition (Claim 3). The full report is a working-out of this composition. - ---- - -## 2. Prior Art Survey (8 Clusters) - -This section surveys the design lineage across **10 clusters** (added Clusters 8 and 9 in v1.2 per user direction: Metadesk and Verse). Each cluster: a "cluster claim" (what the DSL inherits from the cluster as a whole), then 1 sentence per entry, then specific "take" bullets that §3, §4, §5, and §6 reference. - -The detailed analysis for each cluster lives in the research sub-reports at `research/cluster_*.md` (relative to this file). This section is the executive summary; the sub-reports are the evidence. - -### Cluster 0 — Immediate-Mode Paradigm (philosophical anchor) - -**Cluster claim.** The DSL's *paradigm* — verbs as method calls, no persistent state, reads free, writes formalized — is the direct application of John O'Donnell's IMGUI/MVC framework to a Meta-Tooling context. (Per the full sub-report at `research/cluster_0_odonnell.md`.) - -**Entry: John O'Donnell — IMGUI / The Pitch / MVC / IM-MVC roadmap.** `https://johno.se/book/imgui.html`, `https://johno.se/book/pitch.html`, `https://johno.se/book/immvc.html`, `https://johno.se/book/mvc.html`. Four interconnected pages laying out a unified paradigm: visualization is not inherently stateful; widgets are method invocations not objects; the "reads are free, writes are formalized" invariant via a single IEventTarget interface; the View must not expose scene-graph abstractions. - -**Take bullets (referenced by §5, §6):** -- *Anchor Claim 3 (IEventTarget as single event interface for all state changes):* *"Experience dictates that there only be a single IEventTarget interface that is responsible for all 'system events'."* — `mvc.html`, "Why only a single event interface" section. -- *Anchor Claim 4 (View must not expose scene-graph abstractions):* *"The corresponding interface should be of the form: `view::drawMesh(mesh, transform, anyOtherRenderState);`"* — `mvc.html`, "View" section. -- *"Writes to Model are formalized through the addition of IEventTarget. This is a pure virtual interface that defines all possible state changes / events on a system wide level."* — `mvc.html`, "Writing to Model state" section. -- *"What is a non-stateful view? Basically it is a procedural interface (as opposed to a collection of objects with methods), in essence very much to what DirectX 9 is."* — `pitch.html`, "MVC revisited" section. -- *"However, due to the rapide advances of GPU based rendering over the past 10+ years, this premise no longer holds."* — `pitch.html`, "However!" section. -- The 800,000-vertex single-draw-call empirical result at Jungle Peak (GeForce 6 hardware) — `pitch.html`, batch rendering section. - -### Cluster 1 — Concatenative (Forth family) - -**Cluster claim.** The DSL's *syntax* — postfix RPN, stack-passed arguments, no AST object — is the Forth tradition as refined by Onat Türkçüoğlu's KYRA (2-register stack, magenta pipe as definition boundary, basic blocks and lambdas, preemptive scatter) and Timothy Lottes's x68/5th (32-bit instruction granularity, annotation overlay, "register file as aliased global namespace"). Bob Armstrong's CoSy is the user's-vocabulary-as-the-surface model. (Per the full sub-report at `research/cluster_1_concatenative.md`.) - -**Entries:** - -- **Forth** (Chuck Moore, 1970). The canonical RPN stack-passing language; the colon-word/semicolon definition pattern; threaded code compilation; self-hosting via meta-compilation. `https://en.wikipedia.org/wiki/Forth_(programming_language)`. **Take:** the pure concatenative property — *"concatenation of two programs denotes the composition of the two functions they denote"* (Joy's formalization) — is the foundational claim. The DSL inherits the postfix syntax and the rejection of named lambda parameters (parameters are unnamed; they live on the stack). -- **ColorForth** (Chuck Moore, ~1990s). Color encodes semantics (define/compile/execute/variable). `https://en.wikipedia.org/wiki/ColorForth`. **Take:** the idea that visual/structural encoding can replace keywords, and the direct-mapped editor. -- **KYRA / VAMP** (Onat Türkçüoğlu, SVFIG 2025). 2-register stack (RAX/RDX); magenta pipe `|` as definition boundary emitting `RET + xchg rax, rdx`; basic blocks `[ ]` and lambdas `{ }` as compilation units; preemptive scatter. `C:\projects\forth\bootslop\references\kyra_in-depth.md`, `forth_day_2020_in-depth.md`. **Take:** the bracket operators (`[ ]`, `{ }`) and the tape-scoped blocks (`tape { }`). -- **x68 / 5th / "Ear" + "Toe"** (Timothy Lottes, 2007-2026). 32-bit instruction granularity; annotation overlay; folded interpreter; "register file as aliased global namespace" (X.com thread, lines 95-103). `C:\projects\forth\bootslop\references\neokineogfx_in-depth.md`, `blog_in-depth.md`. **Take:** the 32-bit token encoding, the annotation overlay pattern, the folded-interpreter optimization. -- **Joy** (William Byrd, Manfred von Thun, 2001-2003). Purely functional concatenative; quotations as first-class values; combinator library (`map`, `filter`, `fold`, `binrec`, `primrec`, `linrec`). `https://en.wikipedia.org/wiki/Joy_(programming_language)`. **Take:** the quotation-as-first-class-value concept and the combinator library as the model for Tier 2 verbs. -- **CoSy** (Bob Armstrong, ongoing). TimeStamped notebook/log in Forth; all nouns are lists/trees with 3-cell headers `(Type Count refCount)`; modulo indexing; "extensive vocabulary evolved from APL via K." `https://cosy.com/CoSy/Simplicity.html`, `https://cosy.com/4thCoSy/`. **Take:** the open-vocabulary culture; the modulo indexing (forgiving of off-by-one AI errors); the 3-cell header as a universal data structure. - -**Section 5 grounding (per the cluster 1 synthesis).** The DSL's `->` pipeline, `[ ]`/`{ }` blocks, `tape { }` memory model, `scatter`/`gather` verbs, `map`/`filter`/`fold` combinators, modulo indexing, and the "no AST object" parsing strategy all have direct concatenative lineage. See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_1_concatenative.md` §"Synthesis for Section 5" for the verb-by-verb mapping table. - -### Cluster 2 — Array Languages (APL lineage) - -**Cluster claim.** The DSL's *data model* — array as universal type, every verb vectorizes, multi-dimensional indexing — is the APL tradition as refined by K (ASCII-only with overloading), BQN (clean modern semantics with function trains), and Uiua (stack-based execution). The DSL inherits the *philosophy* (succinct expression of algorithms) but uses ASCII-compatible representation rather than APL's custom character set. (Per the full sub-report at `research/cluster_2_array.md`.) - -**Entries:** - -- **APL** (Kenneth Iverson, 1962; Turing Award 1979). The foundational array language; array as universal type; every glyph is a function; right-to-left evaluation with no precedence. `https://en.wikipedia.org/wiki/APL_(programming_language)`, `https://www.dyalog.com/`. **Take:** the array-as-universal-type principle and the right-to-left evaluation model. -- **K / q** (Arthur Whitney, KX Systems, 1993). ASCII-only with heavy context-sensitive overloading; first-class functions borrowed from Scheme; foundation of kdb+ in-memory columnar database. `https://en.wikipedia.org/wiki/K_(programming_language)`, `https://kx.com/`. **Take:** the context-sensitive operator philosophy and first-class functions. -- **BQN** (Marshall Lochbaum, 2020). Modernized APL with clean semantics; context-free grammar; function trains. `https://mlochbaum.github.io/BQN/`. **Take:** the train composition pattern as the most expressive tacit mechanism in the family. -- **Uiua** (Tony Morris, 2023). Stack-based execution; modern open-source development; online Pad for onboarding. `https://www.uiua.org/`, `https://github.com/uiua-lang/uiua`. **Take:** the stack-based execution model as a viable alternative to named parameters, and the modern onboarding-UX model. - -**Section 5 grounding (per the cluster 2 synthesis).** The DSL's `for x .. n` (mapping to APL's `ιN` + reduce, BQN's `↕N`, K's `!R`) and `result[row, col]` (mapping to APL's multi-dim indexing, BQN's `⊏`, K's `@`) inherit directly from this cluster. See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_2_array.md` §"Synthesis for the DSL" for the verb-by-verb mapping table. - -### Cluster 3 — Intent-Mapping - -**Cluster claim.** The DSL's *use case* — a compact, intent-expressive scripting language that maps user intent to platform-optimal operations — is the Jofito tradition as the user has been exploring it. The pipe-coalescing optimization (find/grep/sort/unique collapse into one in-memory script) is the runtime efficiency claim. The nagent tag protocol is *mentioned and explicitly rejected* (no XML angle brackets) but the *structured-protocol idea* is retained. (Per the full sub-report at `research/cluster_3_intent_mapping.md`.) - -**Take bullets — minor v1.1 corrections:** -- **DSL `->` pipe operator:** jq's `|` pipe is the conceptual precedent for the DSL's `->` pipeline operator. The DSL replaces `|` with `->` to avoid conflict with shell usage and to make the DSL parseable without shell-aware lexing. (Per the sub-report's verbatim take bullets.) -- **v1.1 OCR-restoration:** the sub-report slightly misquoted Lottes by dropping "actually" in one place ("missing what we **actually** did in assembly back then"). v1.1 restores the full quote for accuracy. - -**Entries:** - -- **Jofito** (Jody Bruchon, 2023-2026). "Intent mapping engine" (per 2026 README update); tape allocation; leader/chaser thread model; pipe-coalescing. `https://codeberg.org/jbruchon/jofito`, `docs/transcripts/Ddme7DwMQBI_jofito_jody_bruchon.txt`. **Take:** the "intent mapping engine" framing is the DSL's *use case*; the leader/chaser pattern is the *implementation hint*; the tape allocation is the *memory model*. (Specifically: the DSL's `scan -> filter -> print` chain is directly inspired by Jofito's `scandir(...) : filter : print` predicate chain.) -- **jq** (Stephen Dolan, 2012-). JSON-path filter language; the `|` pipe operator (replaced by `->` in the DSL). `https://en.wikipedia.org/wiki/Jq_(programming_language)`, `https://jqlang.org/`. **Take:** the filter-as-expression style; `select(condition)`, `map`, `reduce`, `unique` as Tier 2 verb precedents. -- **nagent's tag protocol** (Mike Acton, `github.com/macton/nagent`; per `conductor/tracks/nagent_review_20260608/agent_review_v2_1_20260612.md:50`). XML-ish self-closing tags (``). **TAKEN:** the structured-protocol idea (named operation with typed attributes; LLM-emit-able; self-delimiting). **REJECTED:** the XML angle-bracket notation, per the user's direct instruction during the intent_dsl_survey_20260612 brainstorming session on 2026-06-12: *"ignore its record formats as they problably will be less xml/json based as I don't like them."* (The user said this in conversation; it is not in any project file.) The DSL must use a different notation that preserves the structured-protocol properties. -- **WebAssembly** (W3C, 2017-). Linear memory; sectioned binary format; structured control flow. `https://en.wikipedia.org/wiki/WebAssembly`. **Take (one paragraph):** the linear memory model is the modern reference for the "tape drive" argument-passing semantics that grounds the DSL's Tier 2 verbs. Wasm's streaming-parse design *suggests* a parsing strategy where verb names and signatures are validated early (cheap) and arguments are parsed on demand (deferred), though this is an inference, not an explicit recommendation from the Wasm spec. - -**Section 4 grounding (per the cluster 3 synthesis).** Each Tier 2 verb cites Jofito (for `scan`, `filter`, `tape`, `scatter`, `gather`, `pipe`) or jq (for `select`, `map`, `fold`, `sort`, `dedupe`, `group`); each Tier 3 verb cites either nagent's structured-protocol idea (for `read`, `edit`, `test`, `discover`) or Jofito's tool-replacement model (for `glob`, `exec`, `run`, `mcp`). See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_3_intent_mapping.md` §"Synthesis for the DSL" for the verb-by-verb mapping table. - -### Cluster 4 — Meta-Tooling DSLs and Agent-Facing Languages - -**Cluster claim.** The DSL is *not the first* agent-facing language. The existing `mcp_dsl_20260606` placeholder, nagent's "Bridge DSL" idea, OpenAI's function-calling schema, and Anthropic's tool-use schema are the prior art. The DSL learns from all four and takes a different notation (per the user's XML/JSON rejection) but the same structural properties (compact, structured, LLM-emit-able). (Per the full sub-report at `research/cluster_4_meta_tooling_dsls.md`.) - -**Entries:** - -- **`mcp_dsl_20260606`** (Manual Slop placeholder; per `conductor/tracks/mcp_architecture_refactor_20260606/spec.md` §12.1 and `nagent_review_20260608/metadata.json:28`). APL/K/Cosy-inspired per-MCP compact dialect. The closest project-internal reference. **Take:** the per-MCP grammar organization; the 8x token-reduction target (80 → 10 tokens); the JSON path stays (backward compat); the DSL is opt-in per MCP. -- **nagent's Bridge DSL idea** (per `nagent_takeaways_20260608.md` line 216-230). The bridge between external agents and actual `mcp_client.py` tool calls. **Take:** the Application's function-calling stays; the bridge DSL is the format external agents emit. -- **OpenAI function-calling** (per `https://platform.openai.com/docs/guides/function-calling`). JSON Schema with `strict`, `required`, `additionalProperties: false`, `enum` constraints. The 5-step conversational loop. **Take:** schema rigor baseline; token cost is proportional to schema verbosity; the 8x reduction target; namespace grouping; fewer-capable-tools principle. -- **Anthropic tool-use** (per `https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/define-tools`). Flat structure with `name`, `description`, `input_schema`, `input_examples`; `strict` as guarantee; `tool_choice` control. **Take:** `input_examples` as a model for teaching the DSL; `tool_choice` maps to Tier 4 verb design (auto/any/forced); the flat structure is the right model for terseness. - -**Section 4 grounding (per the cluster 4 synthesis).** The Tier 4 verbs map to the entries as follows: `fuzzy` ← nagent Bridge + MCP DSL; `try`/`recover` ← nagent Bridge + OpenAI; `sandbox` ← OpenAI + Anthropic; `audit` ← MCP DSL + nagent Bridge; `didyoumean` ← nagent Bridge + Anthropic; `span` ← MCP DSL + OpenAI; `offset` ← MCP DSL + OpenAI; `assumewide` ← OpenAI + Anthropic. See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_4_meta_tooling_dsls.md` §"Synthesis for the DSL" for the full mapping. - -### Cluster 5 — SSDL Shape Primitives - -**Cluster claim.** The DSL's verbs are annotated with **SSDL shape tags** (per `docs/reports/computational_shapes_ssdl_digest_20260608.md` §1) so the reader can see at a glance whether a verb is a single instruction, a codepath, a wide codepath, a codecycle, a wide codecycle, or a codecycle graph. This is the meta-vocabulary that lets the report describe a verb's *shape* in one token. - -**The 6 SSDL primitives:** - -| # | Shape | One-line definition | SSDL symbol | -|---|---|---|---| -| 1 | **Instruction** | A single unit of computation. Reads data, writes data, or both. | `[I]` | -| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `===>` | -| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `===>W===>` | -| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o==>` | -| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `oo==>oo` | -| 6 | **Codecycle graph** | Multiple codecycles + the data they read and write. | `boxes + arrows` | - -**The 7 modifiers:** - -| Modifier | SSDL | Meaning | -|---|---|---| -| `[T]` | terminator | The instruction that *ends* a codepath (return, exit, etc.) | -| `[B]` | branch | A point where control flow forks based on a condition | -| `[M]` | merge | A point where control flow re-converges | -| `[S]` | stateful | Marks an instruction that *mutates* persistent state | -| `[Q]` | query | Marks an instruction that reads persistent state | -| `[N]` | nil sentinel | A special value that satisfies "is this OK to use?" in all cases | -| `───` | data | A line representing data being read or written (not a codepath) | - -**How the DSL uses SSDL tags.** Each verb in §4 has a "Shape" column with an SSDL tag. For example, `sum` is `[I]` (single instruction); `for x .. n` is `o==>` (codecycle); `tape { }` is a sub-codepath scope; `pipe` is `===>W===>` (wide codepath, the chain can fan out); the entire DSL pipeline is a codecycle graph (multiple codecycles + the data they read and write). This lets the reader see the *shape* of a pipeline at a glance. - -### Cluster 6 — Project's Own Command DSL Precedents - -**Cluster claim.** The DSL is a *richer* superset of the project's existing 33 Command Palette commands (per `docs/guide_command_palette.md` and `src/commands.py`). The "Everything" mode in the Command Palette (per `guide_command_palette.md` line 383: *"search across commands, files, symbols, history, settings"*) is a near-term use case where the DSL's verbs can be the underlying format. The Command Palette is the user's existing vocabulary instinct; the DSL formalizes and extends it. - -**5 representative commands by category** (the full 33 are in `docs/guide_command_palette.md`): - -| Category | Command | Title | Action | -|---|---|---|---| -| AI | `reset_session` | Reset Session | `ai_client.reset_session()` + clears logs + `_handle_reset_session()` | -| AI | `clear_discussion` | Clear Discussion | Empties `app.discussion_history` | -| AI | `add_all_files_to_context` | Add All Files To Context | `app._add_all_files_to_context()` | -| View | `toggle_text_viewer` | Toggle Text Viewer | `_toggle_window(app, "Text Viewer")` | -| Tools | `trigger_hot_reload` | Hot Reload | `HotReloader.reload("src.gui_2", app)` | -| Layout | `save_workspace_profile` | Save Workspace Profile | Opens the save-profile modal | -| Theme | `cycle_theme` | Cycle Theme | Cycles through `["10x Dark", "ImGui Light", "NERV"]` | -| Help | `show_command_palette_help` | Show Command Palette Help | Loads `docs/Readme.md` into the Text Viewer | - -**Take.** The DSL's verbs are a *richer* superset of these. Where the Command Palette has 33 imperative commands (each is a function with side effects), the DSL's Tier 2 verbs are declarative ("I want to scan, filter, print") and the Tier 4 verbs formalize the AI-fuzzing-tolerance aspects (audit, didyoumean) that the Command Palette cannot. The "Everything" mode in the Command Palette is the natural place where DSL verbs could appear as searchable entries. - -### Cluster 7 — Data-Oriented Error Handling Convention - -**Cluster claim.** The DSL's `try { ... } recover { ... }` envelope returns a `Result[T]` (with side-channel errors as `list[ErrorInfo]`), per the convention established by `conductor/tracks/data_oriented_error_handling_20260606/spec.md` §3.3. The 12 `ErrorKind` values are the canonical error vocabulary. The `Result[T]` dataclass is the data-oriented alternative to exception-based control flow. - -**The 12 `ErrorKind` values** (per `data_oriented_error_handling_20260606/spec.md` §3.3): - -| Kind | Meaning | -|---|---| -| `NETWORK` | Network or connection error | -| `AUTH` | Authentication / API key error | -| `QUOTA` | Quota exhausted | -| `RATE_LIMIT` | Rate limited | -| `BALANCE` | Balance / billing error | -| `PERMISSION` | Permission denied (file system, etc.) | -| `NOT_FOUND` | Resource not found | -| `INVALID_INPUT` | Invalid input (parse failure, schema mismatch) | -| `NOT_READY` | System not ready (e.g., RAG not initialized) | -| `UNKNOWN` | Unknown error | -| `CONFIG` | Configuration error | -| `INTERNAL` | Internal error (e.g., SDK exception) | -| `PROVIDER_HISTORY_DIVERGED_FROM_UI` | (added 2026-06-08; per nagent_review Pitfall #4) | - -**The `Result[T]` dataclass signature** (per `data_oriented_error_handling_20260606/spec.md` §3.3): - -```python -@dataclass(frozen=True) -class Result(Generic[T]): - data: T - errors: list[ErrorInfo] = field(default_factory=list) - @property - def ok(self) -> bool: return not self.errors - def with_error(self, err: ErrorInfo) -> "Result[T]": ... - def with_errors(self, new_errors: list[ErrorInfo]) -> "Result[T]": ... - def with_data(self, new_data: T) -> "Result[T]": ... -``` - -**How the DSL uses the Result envelope.** The `try { ... } recover { ... }` block returns a `Result[T]` where `T` is the verb's return type. The `recover` block receives the `Result[T]` from the `try` and can inspect `.errors` to decide what to do. The `didyoumean` verb returns `Result[T, list[Suggestion]]` — the success case is the parse result, the failure case includes a list of suggested corrections. +**Track:** `intent_dsl_survey_20260612` (initialized 2026-06-12) +**Date:** 2026-06-12 +**Location:** `conductor/tracks/intent_dsl_survey_20260612/report.md` (this file; moved from `docs/ideation/` per user instruction — the report is too closely related to the track to live in the general ideation folder) +**Author:** Tier 1 Orchestrator (sections 1, 3, 4, 5, 6, 7, Appendix); Tier 2 sub-agents (section 2 clusters 0-4, with research sub-reports at `research/cluster_*.md`) +> **v1.2 changes (2026-06-12):** (1) Renamed `arena { }` to `tape { }` (better term in hindsight; aligns syntax with the Lottes tape-drive metaphor). All 46 occurrences of `arena` updated; 3 awkward double-tape phrases cleaned up. (2) **Mixed postfix/infix notation introduced** (per user heuristic): strictly postfix for math primitives that have precedence (`+`, `-`, `*`, `/`, `^`, math indexing `[]`); infix for everything else (`:=`, function calls, control flow, field access, block delimiters). The rationale: postfix eliminates precedence ambiguity; infix is more familiar where precedence isn't an issue. The two mix freely: `result := a b +` is canonical (assignment infix, math postfix). (3) **nagent attribution corrected:** the glossary and Cluster 3 entry previously said nagent is Jody Bruchon's; it is actually Mike Acton's (`github.com/macton/nagent`; per `conductor/tracks/nagent_review_20260608/`). Jofito remains correctly attributed to Jody Bruchon. See TODO at the top of §3 for the full postfix heuristic table. Appendix A.3/A.4 retain their v1.1 form pending a sync pass. (4) **Added Cluster 8 (Metadesk — Self-Describing Data + Tag Dispatch) and Cluster 9 (Verse — Multi-Paradigm Foundation Calculi with Transactional Semantics).** Survey now covers 10 prior-art clusters (was 8). Sub-reports at +esearch/cluster_8_metadesk.md and +esearch/cluster_9_verse.md. Decision per user: keep the new clusters separate (Cluster 8 ≠ Cluster 9 — they are different design patterns: narrow-waist data format vs language-theoretic foundation). + +**Status:** v1.1 (post-secondary-review correction; see `reportreview.md` for the review that produced this update) + +> **What this is.** A survey of intent-based scripting languages as a design philosophy, plus a proposed vocabulary (~40 verbs across 4 tiers) for a Meta-Tooling-facing intent DSL. The report is the foundation document for the user's nagent v2.2 (its "Future-Track Candidate #4" section) and for the future interpreter prototype (follow-up B track). +> +> **What this is NOT.** Not an interpreter, not a bridge script, not Application-side function-calling, not XML/JSON record formats. The DSL is Meta-Tooling-side per `docs/guide_meta_boundary.md` — the format external agents (Gemini CLI, OpenCode) emit when invoking `mcp_client.py` tools. The Application's provider-native function-calling stays unchanged. + +--- + +## 1. The "Intent-Based" Design Philosophy + +The DSL is grounded in four anchor claims. Each claim has a philosophical home and a specific design consequence for the vocab and grammar. + +### 1.1 Claim 1 — Intent-based means the user's words are declarative intent, not imperative commands + +Jofito (per its 2026 README update) calls itself an **"intent mapping engine"**: the user writes declarative intent (e.g., "find all pictures, filter out JPEGs, print the list"), and Jofito decomposes that intent into platform-optimal operations. From the Jofito README: *"jofito is a 'write the optimization once, reap the benefits everywhere' system that takes what the user wants to accomplish (intent) as input and decomposes it into operations that make the most sense for the current system."* (`https://codeberg.org/jbruchon/jofito`) + +The canonical Jofito example is `list = scandir("/path/here/", {filter !extension=jpg,jpeg}) : print(list)` — a single declarative expression that replaces `find . -type f | grep -v jpg | grep -v jpeg`. The DSL inherits this framing: the verbs in §4 are **intent verbs** (e.g., `scan` for "I want to read a source", `filter` for "I want to keep only what matches", `audit` for "I want to record what happened"), not imperative primitives. + +This is the *philosophical* anchor for the DSL: the user says *what they want*; the verbs are the way to say it; the bridge script and the MCP tools handle *how to do it*. The user's own math pseudocode (the `determinate`/`minor`/`matrix-transpose` snippets shared during spec review) operates at this declarative level — "here is the math, the verbs are the words." + +### 1.2 Claim 2 — The hardware is the truth + +The verbs must map to actual hardware/software stages, not abstract commands. The Onat/Lottes 2-register model (per `C:\projects\forth\bootslop\references\kyra_in-depth.md` and `X.com - Onat & Lottes Interaction 1.png.ocr.md`) gives the concrete hardware the DSL is mapped to: + +- **2-register stack (RAX/RDX)**: the DSL's `->` chain *maps* to RAX-passed data. Each verb in the chain is a "word" in Onat's sense (no args, no returns — the X.com thread at `X.com - Onat & Lottes Interaction 1.png.ocr.md:80-86` quotes Lottes: "I laugh when people say C is like assembly, they were missing what we did in assembly back then, which was all registers and globals and gotos, no stacks"). +- **Magenta pipe `|` (KYRA) → our `->`**: same definition-boundary semantics, retargeted to data flow. +- **Basic blocks `[ ]` (KYRA) → our `[ ]`**: compilation units; the parser produces a `[ ]` block per `->`-delimited stage. +- **Lambdas `{ }` (KYRA) → our `tape { }`**: tape-scoped blocks; the contents are pre-scattered into tape-drive regions (per the X.com thread at line 55-61, where Onat describes Lottes's "common arguments pushed onto the tape using store duplication when they are known... so it's preemptive scatter, so later at call time there is no argument gather"). + +The verbs are not arbitrary. Each Tier 2 verb (data pipeline) and Tier 3 verb (shell) has a direct hardware mapping; this is what makes the verbs *fast* on the targeted hardware. + +### 1.3 Claim 3 — The pipeline is immediate-mode + +Per John O'Donnell's IMGUI essay (`https://johno.se/book/imgui.html`): *"Widgets, logically, change from being objects to being method invocations."* The pipeline `scan -> filter -> print` is not a Pipeline object with state; it is a sequence of method calls. Once execution ends, the pipeline's state is gone. The next invocation is independent. + +This is the *paradigm* anchor for the DSL. It means: +- The parser doesn't need to track pipeline state across executions; each invocation is independent. +- The `->` chain has no "pipeline object" you can query, name, or pass around. The only way to "name" a chain is to wrap it in a function (`determinate(m, row) -> Scalar { ... }`). +- Verbs exist *only* when called. There is no implicit verb inventory. (This is why the DSL's "Everything" mode in the Command Palette is implementable as a search across *text*, not across a *registry of pipeline objects*.) + +O'Donnell's MVC essay (`https://johno.se/book/mvc.html`) extends this: *"Writes to Model are formalized through the addition of IEventTarget. This is a pure virtual interface that defines all possible state changes / events on a system wide level."* The DSL's `sandbox` verb is the IEventTarget boundary; the `audit` verb is the IEventTarget itself (see §6 Claim 9 and Claim 10). + +### 1.4 Claim 4 — The vocabulary IS the user surface + +CoSy (per `https://cosy.com/CoSy/Simplicity.html`): *"CoSy is a TimeStamped notebook/log created as an open vocabulary in Forth."* And: *"an extensive vocabulary evolved from APL via K, mainly slicing and dicing, searching & replacing, and applying verbs to each item in lists."* + +For the DSL, the **vocabulary** is the user surface — not the syntax, not the parser, not the runtime. For AI agents that emit the DSL, the vocab is the API. A model that knows the 40 verbs in §4 and the 14 grammar primitives in §3 can express any intent that the DSL supports. There is no separate "API documentation" — the verbs ARE the API. + +This is why the report devotes so much space to the vocab (§4) and so little to the syntax (§3). The syntax is trivial (RPN with a few delimiters); the vocabulary is the substance. + +### 1.5 The four claims together + +The four claims are not independent; they compose: + +- Claim 1 (intent-mapping) → the user expresses what they want; the verbs are the vocabulary. +- Claim 2 (hardware is the truth) → the verbs map to real data-oriented pipeline stages. +- Claim 3 (immediate-mode) → the verbs are method calls, not stateful objects; pipelines have no persistent state. +- Claim 4 (vocabulary is the user surface) → the 40-verb vocab is the API; the syntax is trivial. + +The composition is: a user expresses intent (Claim 1) using a verb (Claim 4) that maps to a hardware stage (Claim 2) in a single per-frame composition (Claim 3). The full report is a working-out of this composition. + +--- + +## 2. Prior Art Survey (8 Clusters) + +This section surveys the design lineage across **10 clusters** (added Clusters 8 and 9 in v1.2 per user direction: Metadesk and Verse). Each cluster: a "cluster claim" (what the DSL inherits from the cluster as a whole), then 1 sentence per entry, then specific "take" bullets that §3, §4, §5, and §6 reference. + +The detailed analysis for each cluster lives in the research sub-reports at `research/cluster_*.md` (relative to this file). This section is the executive summary; the sub-reports are the evidence. + +### Cluster 0 — Immediate-Mode Paradigm (philosophical anchor) + +**Cluster claim.** The DSL's *paradigm* — verbs as method calls, no persistent state, reads free, writes formalized — is the direct application of John O'Donnell's IMGUI/MVC framework to a Meta-Tooling context. (Per the full sub-report at `research/cluster_0_odonnell.md`.) + +**Entry: John O'Donnell — IMGUI / The Pitch / MVC / IM-MVC roadmap.** `https://johno.se/book/imgui.html`, `https://johno.se/book/pitch.html`, `https://johno.se/book/immvc.html`, `https://johno.se/book/mvc.html`. Four interconnected pages laying out a unified paradigm: visualization is not inherently stateful; widgets are method invocations not objects; the "reads are free, writes are formalized" invariant via a single IEventTarget interface; the View must not expose scene-graph abstractions. + +**Take bullets (referenced by §5, §6):** +- *Anchor Claim 3 (IEventTarget as single event interface for all state changes):* *"Experience dictates that there only be a single IEventTarget interface that is responsible for all 'system events'."* — `mvc.html`, "Why only a single event interface" section. +- *Anchor Claim 4 (View must not expose scene-graph abstractions):* *"The corresponding interface should be of the form: `view::drawMesh(mesh, transform, anyOtherRenderState);`"* — `mvc.html`, "View" section. +- *"Writes to Model are formalized through the addition of IEventTarget. This is a pure virtual interface that defines all possible state changes / events on a system wide level."* — `mvc.html`, "Writing to Model state" section. +- *"What is a non-stateful view? Basically it is a procedural interface (as opposed to a collection of objects with methods), in essence very much to what DirectX 9 is."* — `pitch.html`, "MVC revisited" section. +- *"However, due to the rapide advances of GPU based rendering over the past 10+ years, this premise no longer holds."* — `pitch.html`, "However!" section. +- The 800,000-vertex single-draw-call empirical result at Jungle Peak (GeForce 6 hardware) — `pitch.html`, batch rendering section. + +### Cluster 1 — Concatenative (Forth family) + +**Cluster claim.** The DSL's *syntax* — postfix RPN, stack-passed arguments, no AST object — is the Forth tradition as refined by Onat Türkçüoğlu's KYRA (2-register stack, magenta pipe as definition boundary, basic blocks and lambdas, preemptive scatter) and Timothy Lottes's x68/5th (32-bit instruction granularity, annotation overlay, "register file as aliased global namespace"). Bob Armstrong's CoSy is the user's-vocabulary-as-the-surface model. (Per the full sub-report at `research/cluster_1_concatenative.md`.) + +**Entries:** + +- **Forth** (Chuck Moore, 1970). The canonical RPN stack-passing language; the colon-word/semicolon definition pattern; threaded code compilation; self-hosting via meta-compilation. `https://en.wikipedia.org/wiki/Forth_(programming_language)`. **Take:** the pure concatenative property — *"concatenation of two programs denotes the composition of the two functions they denote"* (Joy's formalization) — is the foundational claim. The DSL inherits the postfix syntax and the rejection of named lambda parameters (parameters are unnamed; they live on the stack). +- **ColorForth** (Chuck Moore, ~1990s). Color encodes semantics (define/compile/execute/variable). `https://en.wikipedia.org/wiki/ColorForth`. **Take:** the idea that visual/structural encoding can replace keywords, and the direct-mapped editor. +- **KYRA / VAMP** (Onat Türkçüoğlu, SVFIG 2025). 2-register stack (RAX/RDX); magenta pipe `|` as definition boundary emitting `RET + xchg rax, rdx`; basic blocks `[ ]` and lambdas `{ }` as compilation units; preemptive scatter. `C:\projects\forth\bootslop\references\kyra_in-depth.md`, `forth_day_2020_in-depth.md`. **Take:** the bracket operators (`[ ]`, `{ }`) and the tape-scoped blocks (`tape { }`). +- **x68 / 5th / "Ear" + "Toe"** (Timothy Lottes, 2007-2026). 32-bit instruction granularity; annotation overlay; folded interpreter; "register file as aliased global namespace" (X.com thread, lines 95-103). `C:\projects\forth\bootslop\references\neokineogfx_in-depth.md`, `blog_in-depth.md`. **Take:** the 32-bit token encoding, the annotation overlay pattern, the folded-interpreter optimization. +- **Joy** (William Byrd, Manfred von Thun, 2001-2003). Purely functional concatenative; quotations as first-class values; combinator library (`map`, `filter`, `fold`, `binrec`, `primrec`, `linrec`). `https://en.wikipedia.org/wiki/Joy_(programming_language)`. **Take:** the quotation-as-first-class-value concept and the combinator library as the model for Tier 2 verbs. +- **CoSy** (Bob Armstrong, ongoing). TimeStamped notebook/log in Forth; all nouns are lists/trees with 3-cell headers `(Type Count refCount)`; modulo indexing; "extensive vocabulary evolved from APL via K." `https://cosy.com/CoSy/Simplicity.html`, `https://cosy.com/4thCoSy/`. **Take:** the open-vocabulary culture; the modulo indexing (forgiving of off-by-one AI errors); the 3-cell header as a universal data structure. + +**Section 5 grounding (per the cluster 1 synthesis).** The DSL's `->` pipeline, `[ ]`/`{ }` blocks, `tape { }` memory model, `scatter`/`gather` verbs, `map`/`filter`/`fold` combinators, modulo indexing, and the "no AST object" parsing strategy all have direct concatenative lineage. See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_1_concatenative.md` §"Synthesis for Section 5" for the verb-by-verb mapping table. + +### Cluster 2 — Array Languages (APL lineage) + +**Cluster claim.** The DSL's *data model* — array as universal type, every verb vectorizes, multi-dimensional indexing — is the APL tradition as refined by K (ASCII-only with overloading), BQN (clean modern semantics with function trains), and Uiua (stack-based execution). The DSL inherits the *philosophy* (succinct expression of algorithms) but uses ASCII-compatible representation rather than APL's custom character set. (Per the full sub-report at `research/cluster_2_array.md`.) + +**Entries:** + +- **APL** (Kenneth Iverson, 1962; Turing Award 1979). The foundational array language; array as universal type; every glyph is a function; right-to-left evaluation with no precedence. `https://en.wikipedia.org/wiki/APL_(programming_language)`, `https://www.dyalog.com/`. **Take:** the array-as-universal-type principle and the right-to-left evaluation model. +- **K / q** (Arthur Whitney, KX Systems, 1993). ASCII-only with heavy context-sensitive overloading; first-class functions borrowed from Scheme; foundation of kdb+ in-memory columnar database. `https://en.wikipedia.org/wiki/K_(programming_language)`, `https://kx.com/`. **Take:** the context-sensitive operator philosophy and first-class functions. +- **BQN** (Marshall Lochbaum, 2020). Modernized APL with clean semantics; context-free grammar; function trains. `https://mlochbaum.github.io/BQN/`. **Take:** the train composition pattern as the most expressive tacit mechanism in the family. +- **Uiua** (Tony Morris, 2023). Stack-based execution; modern open-source development; online Pad for onboarding. `https://www.uiua.org/`, `https://github.com/uiua-lang/uiua`. **Take:** the stack-based execution model as a viable alternative to named parameters, and the modern onboarding-UX model. + +**Section 5 grounding (per the cluster 2 synthesis).** The DSL's `for x .. n` (mapping to APL's `ιN` + reduce, BQN's `↕N`, K's `!R`) and `result[row, col]` (mapping to APL's multi-dim indexing, BQN's `⊏`, K's `@`) inherit directly from this cluster. See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_2_array.md` §"Synthesis for the DSL" for the verb-by-verb mapping table. + +### Cluster 3 — Intent-Mapping + +**Cluster claim.** The DSL's *use case* — a compact, intent-expressive scripting language that maps user intent to platform-optimal operations — is the Jofito tradition as the user has been exploring it. The pipe-coalescing optimization (find/grep/sort/unique collapse into one in-memory script) is the runtime efficiency claim. The nagent tag protocol is *mentioned and explicitly rejected* (no XML angle brackets) but the *structured-protocol idea* is retained. (Per the full sub-report at `research/cluster_3_intent_mapping.md`.) + +**Take bullets — minor v1.1 corrections:** +- **DSL `->` pipe operator:** jq's `|` pipe is the conceptual precedent for the DSL's `->` pipeline operator. The DSL replaces `|` with `->` to avoid conflict with shell usage and to make the DSL parseable without shell-aware lexing. (Per the sub-report's verbatim take bullets.) +- **v1.1 OCR-restoration:** the sub-report slightly misquoted Lottes by dropping "actually" in one place ("missing what we **actually** did in assembly back then"). v1.1 restores the full quote for accuracy. + +**Entries:** + +- **Jofito** (Jody Bruchon, 2023-2026). "Intent mapping engine" (per 2026 README update); tape allocation; leader/chaser thread model; pipe-coalescing. `https://codeberg.org/jbruchon/jofito`, `docs/transcripts/Ddme7DwMQBI_jofito_jody_bruchon.txt`. **Take:** the "intent mapping engine" framing is the DSL's *use case*; the leader/chaser pattern is the *implementation hint*; the tape allocation is the *memory model*. (Specifically: the DSL's `scan -> filter -> print` chain is directly inspired by Jofito's `scandir(...) : filter : print` predicate chain.) +- **jq** (Stephen Dolan, 2012-). JSON-path filter language; the `|` pipe operator (replaced by `->` in the DSL). `https://en.wikipedia.org/wiki/Jq_(programming_language)`, `https://jqlang.org/`. **Take:** the filter-as-expression style; `select(condition)`, `map`, `reduce`, `unique` as Tier 2 verb precedents. +- **nagent's tag protocol** (Mike Acton, `github.com/macton/nagent`; per `conductor/tracks/nagent_review_20260608/agent_review_v2_1_20260612.md:50`). XML-ish self-closing tags (``). **TAKEN:** the structured-protocol idea (named operation with typed attributes; LLM-emit-able; self-delimiting). **REJECTED:** the XML angle-bracket notation, per the user's direct instruction during the intent_dsl_survey_20260612 brainstorming session on 2026-06-12: *"ignore its record formats as they problably will be less xml/json based as I don't like them."* (The user said this in conversation; it is not in any project file.) The DSL must use a different notation that preserves the structured-protocol properties. +- **WebAssembly** (W3C, 2017-). Linear memory; sectioned binary format; structured control flow. `https://en.wikipedia.org/wiki/WebAssembly`. **Take (one paragraph):** the linear memory model is the modern reference for the "tape drive" argument-passing semantics that grounds the DSL's Tier 2 verbs. Wasm's streaming-parse design *suggests* a parsing strategy where verb names and signatures are validated early (cheap) and arguments are parsed on demand (deferred), though this is an inference, not an explicit recommendation from the Wasm spec. + +**Section 4 grounding (per the cluster 3 synthesis).** Each Tier 2 verb cites Jofito (for `scan`, `filter`, `tape`, `scatter`, `gather`, `pipe`) or jq (for `select`, `map`, `fold`, `sort`, `dedupe`, `group`); each Tier 3 verb cites either nagent's structured-protocol idea (for `read`, `edit`, `test`, `discover`) or Jofito's tool-replacement model (for `glob`, `exec`, `run`, `mcp`). See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_3_intent_mapping.md` §"Synthesis for the DSL" for the verb-by-verb mapping table. + +### Cluster 4 — Meta-Tooling DSLs and Agent-Facing Languages + +**Cluster claim.** The DSL is *not the first* agent-facing language. The existing `mcp_dsl_20260606` placeholder, nagent's "Bridge DSL" idea, OpenAI's function-calling schema, and Anthropic's tool-use schema are the prior art. The DSL learns from all four and takes a different notation (per the user's XML/JSON rejection) but the same structural properties (compact, structured, LLM-emit-able). (Per the full sub-report at `research/cluster_4_meta_tooling_dsls.md`.) + +**Entries:** + +- **`mcp_dsl_20260606`** (Manual Slop placeholder; per `conductor/tracks/mcp_architecture_refactor_20260606/spec.md` §12.1 and `nagent_review_20260608/metadata.json:28`). APL/K/Cosy-inspired per-MCP compact dialect. The closest project-internal reference. **Take:** the per-MCP grammar organization; the 8x token-reduction target (80 → 10 tokens); the JSON path stays (backward compat); the DSL is opt-in per MCP. +- **nagent's Bridge DSL idea** (per `nagent_takeaways_20260608.md` line 216-230). The bridge between external agents and actual `mcp_client.py` tool calls. **Take:** the Application's function-calling stays; the bridge DSL is the format external agents emit. +- **OpenAI function-calling** (per `https://platform.openai.com/docs/guides/function-calling`). JSON Schema with `strict`, `required`, `additionalProperties: false`, `enum` constraints. The 5-step conversational loop. **Take:** schema rigor baseline; token cost is proportional to schema verbosity; the 8x reduction target; namespace grouping; fewer-capable-tools principle. +- **Anthropic tool-use** (per `https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/define-tools`). Flat structure with `name`, `description`, `input_schema`, `input_examples`; `strict` as guarantee; `tool_choice` control. **Take:** `input_examples` as a model for teaching the DSL; `tool_choice` maps to Tier 4 verb design (auto/any/forced); the flat structure is the right model for terseness. + +**Section 4 grounding (per the cluster 4 synthesis).** The Tier 4 verbs map to the entries as follows: `fuzzy` ← nagent Bridge + MCP DSL; `try`/`recover` ← nagent Bridge + OpenAI; `sandbox` ← OpenAI + Anthropic; `audit` ← MCP DSL + nagent Bridge; `didyoumean` ← nagent Bridge + Anthropic; `span` ← MCP DSL + OpenAI; `offset` ← MCP DSL + OpenAI; `assumewide` ← OpenAI + Anthropic. See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_4_meta_tooling_dsls.md` §"Synthesis for the DSL" for the full mapping. + +### Cluster 5 — SSDL Shape Primitives + +**Cluster claim.** The DSL's verbs are annotated with **SSDL shape tags** (per `docs/reports/computational_shapes_ssdl_digest_20260608.md` §1) so the reader can see at a glance whether a verb is a single instruction, a codepath, a wide codepath, a codecycle, a wide codecycle, or a codecycle graph. This is the meta-vocabulary that lets the report describe a verb's *shape* in one token. + +**The 6 SSDL primitives:** + +| # | Shape | One-line definition | SSDL symbol | +|---|---|---|---| +| 1 | **Instruction** | A single unit of computation. Reads data, writes data, or both. | `[I]` | +| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `->` | +| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `=>` | +| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o->` | +| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `o=>` | +| 6 | **Codecycle graph** | Multiple codecycles + the data they read and write. | `boxes + arrows` | + +**The 7 modifiers:** + +| Modifier | SSDL | Meaning | +|---|---|---| +| `[T]` | terminator | The instruction that *ends* a codepath (return, exit, etc.) | +| `[B]` | branch | A point where control flow forks based on a condition | +| `[M]` | merge | A point where control flow re-converges | +| `[S]` | stateful | Marks an instruction that *mutates* persistent state | +| `[Q]` | query | Marks an instruction that reads persistent state | +| `[N]` | nil sentinel | A special value that satisfies "is this OK to use?" in all cases | +| `───` | data | A line representing data being read or written (not a codepath) | + +**How the DSL uses SSDL tags.** Each verb in §4 has a "Shape" column with an SSDL tag. For example, `sum` is `[I]` (single instruction); `for x .. n` is `o->` (codecycle); `tape { }` is a sub-codepath scope; `pipe` is `=>` (wide codepath, the chain can fan out); the entire DSL pipeline is a codecycle graph (multiple codecycles + the data they read and write). This lets the reader see the *shape* of a pipeline at a glance. + +### Cluster 6 — Project's Own Command DSL Precedents + +**Cluster claim.** The DSL is a *richer* superset of the project's existing 33 Command Palette commands (per `docs/guide_command_palette.md` and `src/commands.py`). The "Everything" mode in the Command Palette (per `guide_command_palette.md` line 383: *"search across commands, files, symbols, history, settings"*) is a near-term use case where the DSL's verbs can be the underlying format. The Command Palette is the user's existing vocabulary instinct; the DSL formalizes and extends it. + +**5 representative commands by category** (the full 33 are in `docs/guide_command_palette.md`): + +| Category | Command | Title | Action | +|---|---|---|---| +| AI | `reset_session` | Reset Session | `ai_client.reset_session()` + clears logs + `_handle_reset_session()` | +| AI | `clear_discussion` | Clear Discussion | Empties `app.discussion_history` | +| AI | `add_all_files_to_context` | Add All Files To Context | `app._add_all_files_to_context()` | +| View | `toggle_text_viewer` | Toggle Text Viewer | `_toggle_window(app, "Text Viewer")` | +| Tools | `trigger_hot_reload` | Hot Reload | `HotReloader.reload("src.gui_2", app)` | +| Layout | `save_workspace_profile` | Save Workspace Profile | Opens the save-profile modal | +| Theme | `cycle_theme` | Cycle Theme | Cycles through `["10x Dark", "ImGui Light", "NERV"]` | +| Help | `show_command_palette_help` | Show Command Palette Help | Loads `docs/Readme.md` into the Text Viewer | + +**Take.** The DSL's verbs are a *richer* superset of these. Where the Command Palette has 33 imperative commands (each is a function with side effects), the DSL's Tier 2 verbs are declarative ("I want to scan, filter, print") and the Tier 4 verbs formalize the AI-fuzzing-tolerance aspects (audit, didyoumean) that the Command Palette cannot. The "Everything" mode in the Command Palette is the natural place where DSL verbs could appear as searchable entries. + +### Cluster 7 — Data-Oriented Error Handling Convention + +**Cluster claim.** The DSL's `try { ... } recover { ... }` envelope returns a `Result[T]` (with side-channel errors as `list[ErrorInfo]`), per the convention established by `conductor/tracks/data_oriented_error_handling_20260606/spec.md` §3.3. The 12 `ErrorKind` values are the canonical error vocabulary. The `Result[T]` dataclass is the data-oriented alternative to exception-based control flow. + +**The 12 `ErrorKind` values** (per `data_oriented_error_handling_20260606/spec.md` §3.3): + +| Kind | Meaning | +|---|---| +| `NETWORK` | Network or connection error | +| `AUTH` | Authentication / API key error | +| `QUOTA` | Quota exhausted | +| `RATE_LIMIT` | Rate limited | +| `BALANCE` | Balance / billing error | +| `PERMISSION` | Permission denied (file system, etc.) | +| `NOT_FOUND` | Resource not found | +| `INVALID_INPUT` | Invalid input (parse failure, schema mismatch) | +| `NOT_READY` | System not ready (e.g., RAG not initialized) | +| `UNKNOWN` | Unknown error | +| `CONFIG` | Configuration error | +| `INTERNAL` | Internal error (e.g., SDK exception) | +| `PROVIDER_HISTORY_DIVERGED_FROM_UI` | (added 2026-06-08; per nagent_review Pitfall #4) | + +**The `Result[T]` dataclass signature** (per `data_oriented_error_handling_20260606/spec.md` §3.3): + +```python +@dataclass(frozen=True) +class Result(Generic[T]): + data: T + errors: list[ErrorInfo] = field(default_factory=list) + @property + def ok(self) -> bool: return not self.errors + def with_error(self, err: ErrorInfo) -> "Result[T]": ... + def with_errors(self, new_errors: list[ErrorInfo]) -> "Result[T]": ... + def with_data(self, new_data: T) -> "Result[T]": ... +``` + +**How the DSL uses the Result envelope.** The `try { ... } recover { ... }` block returns a `Result[T]` where `T` is the verb's return type. The `recover` block receives the `Result[T]` from the `try` and can inspect `.errors` to decide what to do. The `didyoumean` verb returns `Result[T, list[Suggestion]]` — the success case is the parse result, the failure case includes a list of suggested corrections. ### Cluster 8 — Self-Describing Data + Tag Dispatch (Metadesk) **Cluster claim.** Metadesk (Fleury/Webster) is the canonical example of a "narrow waist" data-description language: the *language* defines a uniform AST shape, the *host application* supplies all semantics. The DSL inherits this stance — the DSL is the format; the bridge script and the MCP client supply execution semantics. The tag-dispatch pattern (a generic AST + user-defined tags as the only dispatch keys) is structurally identical to the nagent tag protocol (Cluster 3) but generalized to a full language. (Per the full sub-report at `research/cluster_8_metadesk.md`.) @@ -272,11 +274,11 @@ class Result(Generic[T]): **Section 4 grounding (per the cluster 9 synthesis).** Tier 4 verbs (`try`, `recover`, `sandbox`, `audit`) are the DSL's surface expression of Verse's deeper transactional+effect model. The DSL's `Result[T]` convention (Cluster 7) is the DSL's surface expression of Verse's `?T` option model. The `sandbox { }` block is the DSL's surface expression of Verse's `` boundary. See `conductor/tracks/intent_dsl_survey_20260612/research/cluster_9_verse.md` §"Synthesis for the DSL" for the verb-by-verb mapping table. ---- - - ---- - +--- + + +--- + ## 3. The Grammar **Notation heuristic (v1.2 convention).** The grammar mixes postfix and infix styles based on whether precedence is an issue. @@ -294,295 +296,295 @@ class Result(Generic[T]): **Why this mix.** Postfix eliminates precedence ambiguity — `2 + 3 * 4` vs `2 3 4 * +` doesn''t need disambiguation. Infix is more familiar where precedence isn''t a concern (you never write `name ?= expr` and need to know if `?=` binds tighter than `:=`). The two mix freely: `result := a b +` is canonical — the `:=` is infix (structural), the `a b +` is postfix (math). The body of every verb is a sequence of math operations (postfix) chained by `;`, with infix assignment and control flow as structural glue. -**Heuristic summary.** *If the operator has precedence, postfix it. If it doesn''t, infix it.* - -The grammar formalizes 14 primitives drawn from the user's math pseudocode (the `determinate`/`minor`/`matrix-transpose` snippets shared during spec review), plus 3 known ambiguity flags, plus precedence rules and AI-fuzzing tolerance rules. - -### 3.1 The 14 primitives - -| # | Symbol | Name | Signature / Syntax | Meaning | Source example (user pseudocode) | -|---|---|---|---|---|---| -| 1 | `name := value` | Local bind | `name := expr` | Stack-scoped local declaration | `m rows . 1 - m columns . 1 - Matrix result :=` | -| 2 | `stack { ... }` | Stack scope | `stack { decl1; decl2; ... }` | Block of stack-allocated locals | `stack { ... result :=; Scalar row_offset :=; Scalar col_offset := }` | -| 3 | `name: Type` | Annotation | `name: Type` | Type hint on a binding | `m : Matrix` | -| 4 | `func(args) -> Type { ... }` | Function def | `func(args) -> Type { body }` | Named function with return type | `determinate(m, row) -> Scalar { ... }` | -| 5 | `name(...) proc { ... }` | Procedure def | `name(args) proc { body }` | Void-returning function | `minor(m, row_omit, column_omit) -> Scalar proc { ... }` | -| 6 | `for x .. n` | Range iteration | `for x .. n { body }` | Iterate `x` over `[0, n)` | `for col .. m.columns` | -| 7 | `name[a, b]` | Bracket indexing | `name[i, j, k, ...]` | Multi-dim array access | `result[row - row_offset, col - col_offset]` | -| 8 | `if cond { ... }` | Conditional | `if cond { then-body }` | If-then (else inferred) | `if col = col_omit { ++ col_offset; continue; }` | -| 9 | `return value` | Return | `return expr` | Function exit with value | `return result` | -| 10 | `->` (between verbs) | Pipeline flow | `verb1 -> verb2 -> verb3` | Output of left → input of right | `filter -> (col != column_omit <- for col .. m.columns)` | -| 11 | `<-` (after verb) | Input binding | `result <- producer` | The thing on the right is the producer | `for col .. m.columns` produces; `col != column_omit` consumes | -| 12 | `=` (in `assert`) | Equality | `assert -> lhs = rhs` | Assert two expressions are equal | `assert -> product(...) = product(...)` | -| 13 | `{ }` | Body block | `{ body }` | Function/scope body | `{ ... }` | -| 14 | `[ ]` | Basic block | `[ my_stage ]` | Onat's compilation unit (no branching semantics) | (not in user pseudocode; from KYRA's basic blocks) | - -### 3.2 Ambiguity flags - -Per the user's note during spec review (*"Hopefully the above don't have too many logic errors that the use can't be clarified."*), three known ambiguities in the user's pseudo code are normalized in the report: - -- **`proc` modifier placement:** `minor(m, row_omit, column_omit) -> Scalar proc { ... }` — likely a *type qualifier* (the return type is "Scalar" + "proc"-ness means side-effecting). The report adopts the convention that `proc` is a postfix modifier indicating void-returning; the syntax is `name(args) proc { body }` (return type omitted) or `name(args) -> Type proc { body }` (return type explicit but ignored). -- **`++col_offset`:** likely `col_offset += 1`. The report formalizes as `name += 1` (Python-style augmented assignment) and does not adopt the `++` operator. This avoids confusion between pre-increment and post-increment. -- **`m[row][column]` vs `m[row, col]`:** both appear in the user's snippets (line 24 `m[row][column]` is likely a typo for `m[row][col]`). The report adopts the comma-form (`name[a, b]`, multi-dim) throughout, since the C-style chained-bracket form doesn't compose with the user's existing matrix pseudocode. - -### 3.3 Precedence rules - -- **Left-to-right for `->` chains:** `a -> b -> c` parses as `(a -> b) -> c` (b's output becomes c's input). This is *not* the standard math convention (right-to-left) but it matches the user's pseudocode and the pipeline model. -- **`(` `)` for grouping:** explicit parentheses override the left-to-right default. `a -> (b -> c)` parses as `a -> X` where `X = (b -> c)`. -- **Stack-binding precedence:** `:=` binds tighter than `<-`. `producer expr <- result :=` parses as `producer (expr <-) result :=` (the `expr <- producer` consumes the producer into expr before `result :=` stores it). -- **No operator precedence for arithmetic:** `+`, `-`, `*`, `/`, `^` are all left-associative with equal precedence. `2 + 3 * 4` parses as `(2 + 3) * 4 = 20`. (This is the APL/K convention. If the user wants math precedence, the report can adopt explicit `(` `)`.) - -### 3.4 AI-fuzzing tolerance rules - -These are the rules that make the DSL workable for AI agents that may fuzz verb names, indent inconsistently, or offset line references. - -- **CoSy-style modulo indexing:** array indices wrap. `result[-1]` is equivalent to `result[result.len - 1]`. This forgives AI off-by-one errors in line references. (Per the CoSy Simplicity page: *"Indexing is modulo - like counting on your thumb & fingers : 0 1 2 3 4 0."*) -- **Structured recovery anchors via `{ }`:** the `{ }` block is a recovery unit. If the parser cannot parse the body, the entire block is replaced with `NIL` and the error is reported at the block level, not at the line level. -- **Line/offset independence:** the parser uses *token positions*, not raw line numbers. A token's position is `file:token-index` (e.g., `src/foo.py:42` means "the 42nd token in src/foo.py"), not `file:42` (which would be "line 42"). The mapping from token position to line number is a presentation concern, not a parse concern. This matches the project's existing FuzzyAnchor pattern (per `docs/guide_context_curation.md`). -- **Verb-name fuzzing tolerance:** the `didyoumean` verb (see §4 Tier 4) proposes corrections for ambiguous verb names. The parser's "best guess" recovery path is configurable: strict (reject on typo), lenient (auto-correct if Levenshtein distance ≤ 2), or fuzzy (parse the rest, log the typo). -- **Indentation tolerance:** indentation is *not* significant (per the user's explicit "ignore its record formats" instruction and the rejection of Python's indent-sensitive syntax). The parser uses a stack-based approach; the `{ }` and `[ ]` delimiters are the only structure-aware tokens. - -### 3.5 Error envelope: `try { ... } recover { ... }` - -``` -try { - scan "src/foo.py" -> filter !exists -> print -} recover err { - audit "scan failed: " + err - return NIL -} -``` - -- The `try` block evaluates the pipeline. If the pipeline returns a `Result[T]` with `errors` non-empty, the `recover` block runs. -- The `recover` block receives the `Result[T]` as a parameter (named by the user; `err` is the default convention from the user's pseudocode). -- The `recover` block must return a `Result[T]` (or `NIL` to short-circuit). -- If the `recover` block itself returns a `Result[T]` with errors, those errors are appended to the outer `Result[T]`'s error list. (Per Fleury's "errors are data" pattern; per `data_oriented_error_handling_20260606/spec.md` §3.4.) - -### 3.6 Block composition: `[ ]` (KYRA basic blocks) vs `{ }` (body blocks) vs `tape { }` (memory regions) - -- **`[ ]`** is Onat's basic block (per `C:\projects\forth\bootslop\references\kyra_in-depth.md:56-57`): *"Basic blocks `[ ]` provide implicit begin/link/end jump targets for the JIT to resolve relative offsets within a limited scope."* In the DSL, `[ ]` is a *sequential operation block* — a chunk of code that the parser can compile and dispatch as a unit. It is *not* a scope (no new bindings); it is a *compilation unit*. -- **`{ }`** is a body block: function body, if/then body, recover body. It introduces a new lexical scope (new bindings are local to the block). -- **`tape { }`** is a tape-drive region: a `{ }` body that has been *pre-scattered* into a contiguous memory region. The contents are pre-placed; the JIT can emit the entire block as a single `xchg rax, rdx` boundary (per KYRA's magenta pipe semantics). - -The three are nested by the parser: `tape { foo := x; [ bar ]; baz }` is a tape region containing 2 sequential statements (the local bind and the basic block) and a trailing call. is a tape region containing 2 sequential statements (the local bind and the basic block) and a trailing call. - ---- - -## 4. The 4-Tier Vocab (~40 Verbs) - -Each verb: symbol, name, signature, one-line semantics, one example, "borrowed from" note, SSDL shape tag. Tier 2 and Tier 3 verbs also have a "maps to mcp_client tool" column. Tier 4 verbs have a "novel piece" note. - -### 4.1 Tier 1 — Math (~10 verbs) - -The Tier 1 verbs are drawn directly from the user's math pseudocode. - -| Symbol | Name | Signature | Semantics | Example | Borrowed from | Shape | -|---|---|---|---|---|---|---| -| `:=` | Local bind | `name := expr` | Stack-scoped local declaration | `m rows . 1 - m columns . 1 - Matrix result :=` | Forth (dictionary entries); Joy (quotations) | `[I]` | -| `stack { ... }` | Stack scope | `stack { decl1; decl2; ... }` | Block of stack-allocated locals | `stack { ... result :=; Scalar row_offset :=; Scalar col_offset := }` | Forth (colon definitions); KYRA (basic blocks) | `[I]` | -| `for x .. n` | Range iteration | `for x .. n { body }` | Iterate `x` over `[0, n)` | `for col .. m.columns` | APL `ιN`; K `!R`; BQN `↕N`; Uiua (stack iteration) | `o==>` | -| `+` | Add | `a b +` | Element-wise sum | `2 + 3` (yields 5) | All languages | `[I]` | -| `-` | Subtract | `a b -` | Element-wise difference | `5 - 2` (yields 3) | All languages | `[I]` | -| `*` | Multiply | `a b *` | Element-wise product | `2 * 3` (yields 6) | All languages | `[I]` | -| `/` | Divide | `a b /` | Element-wise division | `6 / 2` (yields 3) | All languages | `[I]` | -| `^` | Power | `a b ^` | Element-wise power | `2 ^ 10` (yields 1024) | All languages | `[I]` | -| `sum` | Sum | `expr sum` | Sum all elements | `sum 1..10` (yields 55) | APL `+/`; K `+/`; BQN `+` | `[I]` | -| `product` | Product | `expr product` | Product all elements | `product 1..5` (yields 120) | APL `×/`; K `*/`; BQN `×` | `[I]` | -| `a[i, j]` | Bracket indexing | `name[i, j, ...]` | Multi-dim array access | `result[row - row_offset, col - col_offset]` | APL `result[2;3]`; BQN `⊏`; K `@` | `[Q]` (query) | -| `if/then` | Conditional | `if cond { then-body }` | If-then (else inferred) | `if col = col_omit { ++ col_offset; continue; }` | Forth (IF/THEN); CoSy (control flow) | `[B]` (branch) | - -**Total Tier 1: 12 verbs.** (Slightly over the 10 estimate; the verbs are tight enough that splitting them hurts readability.) - -### 4.2 Tier 2 — Data-Oriented Pipeline (~12 verbs) - -The Tier 2 verbs wrap the existing 45+ MCP tools (per `docs/guide_tools.md` §"Native Tool Inventory") with declarative intent expressions. They are the "imperative veneer" over the Jofito-style predicate chain. - -| Symbol | Name | Signature | Semantics | Example | Maps to mcp_client tool | Borrowed from | Shape | -|---|---|---|---|---|---|---|---| -| `scan` | Scan | `scan path` | Read source (directory, file, URL); first verb in every pipeline | `scan "src/" -> filter !dir -> map ext` | `list_directory` + `search_files` + `read_file` | Jofito `scandir()` | `[I]` | -| `select` | Select | `select condition` | Keep records matching condition (jq-style filter) | `scan "src/" -> select .extension == ".py"` | (jq-style filter) | jq `select(condition)`; Joy `filter` | `===>` | -| `filter` | Filter | `filter predicate` | Keep records where predicate is true | `scan "src/" -> filter .size > 0` | (predicate on FileItem) | Jofito `{filter ...}` predicate | `===>` | -| `map` | Map | `map block` | Apply block to each record | `scan "src/" -> map ext` | (no direct equivalent) | jq `.[] | .field`; Joy `map`; CoSy `' verb 'm` | `o==>` | -| `fold` | Fold | `fold init block` | Reduce to single value | `scan "src/" -> fold 0 { acc + .size }` | (no direct equivalent) | jq `reduce`; Joy `fold` | `o==>` | -| `sort` | Sort | `sort key` | Order records by key | `scan "src/" -> sort .name` | (no direct equivalent) | Joy `qsort`; jq `sort` | `[I]` | -| `group` | Group | `group key` | Bucket records by key | `scan "src/" -> group .extension` | (no direct equivalent) | jq `group_by`; CoSy APL-derived | `o==>` | -| `dedupe` | Dedupe | `dedupe` | Remove duplicates | `scan "src/" -> dedupe` | (no direct equivalent) | jq `unique`; CoSy | `[I]` | -| `tape { }` | tape scope | `tape { body }` | Tape-drive region; pre-scatter contents | `tape { [ scan ]; [ filter ]; [ print ] }` | (compiler directive) | KYRA magenta pipe; Onat preemptive scatter | `o==>` | -| `scatter` | Scatter | `scatter workers` | Fork pipeline across `workers` cores | `scan "src/" -> scatter 4 -> filter` | (runtime hint) | Onat preemptive scatter; Lottes X.com thread line 55-61 | `===>W===>` | -| `gather` | Gather | `gather` | Collect scattered sub-streams | `scan "src/" -> scatter 4 -> filter -> gather` | (runtime hint) | Onat inverse of scatter | `[I]` | -| `pipe` | Pipe root | `pipe` | Explicit chain root (synonym for `->`) | `pipe [ scan, filter, print ]` | (no direct equivalent) | Jofito pipe coalescing (transcript:376-410) | `===>W===>` | - -**Total Tier 2: 12 verbs.** - -### 4.3 Tier 3 — Shell (~10 verbs) - -The Tier 3 verbs wrap existing MCP tools (per `docs/guide_tools.md` §"Native Tool Inventory") and provide the shell-scripting surface. They are the "imperative veneer" over the declarative Tier 2 pipeline. - -| Symbol | Name | Signature | Semantics | Example | Maps to mcp_client tool | Borrowed from | Shape | -|---|---|---|---|---|---|---|---| -| `exec` | Execute | `exec cmd` | Run shell command | `exec "find . -name '*.py'"` | `run_powershell` (shell_runner.py) | nagent tag protocol (structured protocol idea) | `[I]` | -| `open` | Open | `open path` | Open file/URL | `open "src/foo.py"` | `read_file` | nagent tag protocol | `[I]` | -| `read` | Read | `read path` | Read file content | `read "src/foo.py"` | `read_file` | nagent tag protocol | `[I]` | -| `write` | Write | `write path content` | Write file content | `write "src/foo.py" "new content"` | `set_file_slice` / `edit_file` | nagent tag protocol | `[I]` | -| `close` | Close | `close handle` | Close handle | `close file_handle` | (no direct equivalent; close is implicit in Python) | Forth `CLOSE-FILE`; bash `exec` | `[I]` | -| `path` | Path | `path` | Get current path (or `cd`) | `path` | (no direct equivalent; use `cwd`) | shell `pwd`; CoSy `path` | `[I]` | -| `env` | Env | `env var` | Get env var | `env HOME` | (no direct equivalent) | shell `echo $HOME` | `[I]` | -| `wait` | Wait | `wait ms` | Block for `ms` milliseconds | `wait 1000` | (no direct equivalent) | shell `sleep` | `o==>` | -| `poll` | Poll | `poll handle ms` | Poll handle with timeout | `poll file_handle 5000` | (no direct equivalent) | shell `read -t` | `o==>` | -| `cwd` | CWD | `cwd` | Get current working directory | `cwd` | (no direct equivalent) | shell `pwd` | `[I]` | - -**Total Tier 3: 10 verbs.** - -### 4.4 Tier 4 — AI-Fuzzing Tolerance (~8 verbs, the novel contribution) - -The Tier 4 verbs are what make the DSL workable for AI agents that may fuzz verb names, indent inconsistently, or offset line references. Each verb directly maps to one or more of the 4 anchor claims (especially Claim 3: IEventTarget, per Cluster 0). - -| Symbol | Name | Signature | Semantics | Example | Novel piece | Borrowed from | Shape | -|---|---|---|---|---|---|---|---| -| `fuzzy` | Fuzzy | `fuzzy expr` | Declare a parse-tolerance region; parser accepts near-matches | `fuzzy { scan "src/" -> filter .ext }` | Tolerance for AI verb-name fuzzing | nagent "discovery" intent (per `decisions.md:119,128`); SSDL "assume as much as possible" | `===>` | -| `try { ... } recover { ... }` | Try / Recover | `try { body } recover err { fallback }` | Returns `Result[T]`; on error, the `recover` block runs | `try { read "src/foo.py" } recover { read "src/Foo.py" }` | Error envelope as data (Fleury pattern) | `data_oriented_error_handling_20260606`; Wasm `try`/`catch` block/loop/if/end | `===>B===>` | -| `sandbox { ... }` | Sandbox | `sandbox { body }` | IEventTarget boundary; all writes in the block go through the formal event channel | `sandbox { write "tmp/x" "data" }` | O'Donnell's "reads free, writes formalized" invariant applied to the DSL | O'Donnell `mvc.html` "Writing to Model state" | `o==>` | -| `audit` | Audit | `audit msg` | Log the state change to a structured record; the IEventTarget itself | `audit "wrote tmp/x"` | Per-write audit log; full replay capability | O'Donnell `mvc.html` "Event callbacks"; nagent's self-describing tools | `[I]` | -| `didyoumean` | Did you mean | `didyoumean ambiguous` | Propose the closest matching verb(s) for an ambiguous input | `didyoumean "skan"` | Recovery primitive for AI typos | nagent Bridge DSL intent model; Anthropic `input_examples` | `[I]` | -| `span` | Span | `span intent` | Decompose a compound intent into a span of sub-MCP grammar tokens | `span "read foo.py:MyClass"` | Spans the `read_file` and `py_get_definition` tools | MCP DSL per-MCP grammar (`spec.md:456-465`); OpenAI namespace grouping | `[I]` | -| `offset` | Offset | `offset symbol` | Resolve a symbol to a file:line without requiring the model to specify the line | `offset "foo.py:MyClass.method"` | Implicit offset resolution | MCP DSL line-range notation; OpenAI "don't make the model fill known args" | `[Q]` | -| `assumewide` | Assume wide | `assumewide intent` | If the intent is broad or ambiguous, select the most-capable matching tool (the "fewer, more capable" heuristic) | `assumewide "refactor"` | Prefer broad-capability tools over narrow specialists | OpenAI "fewer than 20 functions"; Anthropic `tool_choice: tool` force-call | `===>W===>` | - -**Total Tier 4: 8 verbs.** - -**Total vocab: 12 + 12 + 10 + 8 = 42 verbs.** (~40 estimate; slightly over because Tier 1 is 12 instead of 10, but Tier 3 is 10 and Tier 4 is 8.) - ---- - -## 5. Hardware Mapping (4 Anchor Claims) - -The 4 anchor claims tie the vocab and grammar to actual hardware/software stages. - -### 5.1 Claim 1 — Onat/Lottes, hardware - -The DSL's `->` pipeline, `[ ]`/`{ }` blocks, `tape { }` memory model, and `scatter`/`gather` verbs are direct descendants of KYRA/VAMP and x68. - -- **`->` pipeline:** inherits from Forth's postfix word chain, refined by KYRA's 2-register stack (RAX/RDX) as the minimal call convention. Per `C:\projects\forth\bootslop\references\kyra_in-depth.md:14` (*"The 2-Item Hardware Stack: To achieve hardware locality and GPU compatibility, KYRA strictly restricts the data stack to exactly two CPU registers: `RAX` (Top of Stack) and `RDX` (Next on Stack)"*). -- **`[ ]` sequential block:** inherits from KYRA's basic blocks `[ ]` with implicit begin/link/end jump targets. Per `kyra_in-depth.md:56-57` (*"Basic Blocks `[ ]`: These visually constrain the assembly output. They provide implicit begin, link (else), and end jump targets for the JIT to resolve relative offsets within a limited scope"*). -- **`{ }` lambda block:** inherits from KYRA's lambdas `{ }` that compile code elsewhere and leave an address in `RAX`. Per `kyra_in-depth.md:58-59` (*"Lambdas `{ }`: A lambda (colored Yellow `{`) does not execute inline. The JIT compiles the block of code elsewhere in the tape and leaves its executable memory address in `RAX`."*). -- **`tape { }`:** inherits from KYRA's magenta pipe `|` definition boundary (`RET` + `xchg rax, rdx`) as the entry/exit protocol for a memory region. Per `kyra_in-depth.md:24-27` (*"The Magenta Pipe Trick: Because the stack is just `RAX` and `RDX`, ensuring `RAX` is the active 'Top of Stack' before executing a word is vital. The `xchg rax, rdx` instruction compiles to a tiny 2-byte opcode: `48 92`. Definitions: There are no `begin` or `end` words. A magenta pipe token (`|`) implicitly signals the start of a new definition. The JIT reacts to this by: 1. Emitting a `RET` (`C3`) to close the *previous* definition. 2. Emitting `48 92` (`xchg rax, rdx`) to ensure proper stack alignment for the *new* definition."*). -- **`scatter`:** inherits from Onat's preemptive scatter — per `X.com - Onat & Lottes Interaction 1.png.ocr.md:59-61`: *"The key concept here is that 'common' arguments like the device are pushed onto the tape using store duplication when they are known (after device creation). So it's preemptive scatter, so later at call time there is no argument gather."* -- **`gather`:** the inverse of preemptive scatter — collect pre-scattered values from fixed memory slots. - -Lottes's specific framing at `X.com - Onat & Lottes Interaction 1.png.ocr.md:80-86`: *"I laugh when people say C is like assembly, they are missing what we **actually** did in assembly back then, which was all registers and globals and gotos, no stacks. It's radically different than good assembly."* The DSL's 2-register model + tape regions + magenta `->` are a direct application of this insight: don't pretend you have a memory stack when the hardware has registers. - -### 5.2 Claim 2 — O'Donnell, paradigm - -The DSL's pipeline is *immediate-mode in pipeline composition*. Each `->`-delimited stage is a method invocation, not a Pipeline object. The pipeline exists *only* while the DSL program is being executed; once execution ends, the pipeline's state is gone. - -Per O'Donnell at `https://johno.se/book/imgui.html`: *"Widgets, logically, change from being objects to being method invocations. As we shall see, this fundamentally changes how a client application approaches the implementation of user interfaces."* - -The DSL inherits this: `scan -> filter -> print` is not a pipeline object you can query, name, or pass around. The only way to "name" a chain is to wrap it in a function (`determinate(m, row) -> Scalar { ... }`). The function body IS the chain; the function name IS the chain's identity. There is no separate Pipeline class. - -This also means: the parser doesn't need to track pipeline state across executions. Each invocation of `determinate(m, row)` is independent. There is no "current pipeline" implicit state. The next call is fresh. - -### 5.3 Claim 3 — Forth/CoSy, syntax - -Concatenative syntax is immediate-mode in *tokenization* (whitespace-delimited, no precedence), in *evaluation* (each verb pops args, pushes results), and in *parsing* (no AST object retained after the parse — the parser emits JIT'd code directly per Onat's xchg model). - -- **Tokenization:** whitespace-delimited, no precedence table. Per `https://en.wikipedia.org/wiki/Forth_(programming_language)`: *"Forth's grammar has no official specification. Instead, it is defined by a simple algorithm. The interpreter reads a line of input from the user input device, which is then parsed for a word using spaces as a delimiter."* -- **Evaluation:** each verb pops args, pushes results. Per CoSy Simplicity: *"Words pass information to each other by pushing it on, or taking it off a `stack`."* -- **Parsing:** no AST object retained after parse. The parser emits directly. Per `data_oriented_error_handling_20260606/spec.md` §3.1 and the project's overall "data-oriented design" philosophy, parsing is data flow, not object construction. - -The DSL inherits all three. The parser reads whitespace-delimited tokens, evaluates each verb as a stack effect, and emits the result without retaining an AST. - -### 5.4 Claim 4 — APL/K, data - -Array languages are immediate-mode in *data representation*. There is no array-object header; values are passed by stack reference, not by handle. - -- **APL** (per `https://en.wikipedia.org/wiki/APL_(programming_language)`): *"APL has an array as the universal data type"* — scalar `5` is a 0-dimensional array; `4 5 6 7 + 4` propagates the addition across the vector. -- **K** (per `https://en.wikipedia.org/wiki/K_(programming_language)`): "kdb+ (built on K) processes billions of records at microsecond latency" — the array paradigm scales to production workloads. -- **BQN** (per `https://mlochbaum.github.io/BQN/`): the CBQN bytecode compiler confirms the paradigm can be compiled efficiently. - -The DSL's `for x .. n` range + `result[row, col]` indexing inherits the "no array object" property. The array is *the* universal type; every function operates on it; every function vectorizes. - ---- - -## 6. AI-Agent Properties (10 Claims) - -The 10 claims tie the DSL to the existing project's architecture so future tracks can build on it without re-deriving the design. - -### 6.1 Claim 1 — Domain = Meta-Tooling - -The DSL is **Meta-Tooling-side** per `docs/guide_meta_boundary.md` §"Domain 2: The Meta-Tooling". The Application's provider-native function-calling stays unchanged. The DSL is the format external agents (Gemini CLI, OpenCode) emit when invoking `mcp_client.py` tools. - -### 6.2 Claim 2 — Runtime path = external agent → DSL → bridge → MCP → optional Hook API approval - -Per `docs/guide_meta_boundary.md` §"The Inter-Domain Bridges": external agents (Gemini CLI) call the DSL via a bridge script (`scripts/cli_tool_bridge.py` analogue). The bridge script translates the DSL into `mcp_client.dispatch()` calls. The Hook API (`docs/guide_tools.md` §"The Hook API") surfaces HITL approval modals when the bridge detects a `sandbox { ... }` block. - -### 6.3 Claim 3 — 3-layer security - -The DSL's parser respects the existing 3-layer security model in `mcp_client.py` (per `docs/guide_tools.md` §"The MCP Bridge"). Every DSL statement that targets a tool outside the allowlist is rejected at parse time. The 3 layers are: allowlist construction, path validation, and resolution gate. The DSL does not bypass any of these. - -### 6.4 Claim 4 — 4 memory dimensions - -The DSL does *not* replace any of the 4 memory dimensions (per `conductor/tracks/nagent_review_20260608/nagent_review_v2_1_20260612.md` §2.1): -- **Curation memory** (FileItem + ContextPreset + FuzzyAnchor) -- **Discussion memory** (disc_entries + branching + UISnapshot A1-A7) -- **RAG memory** (ChromaDB, opt-in) -- **Knowledge memory** (Candidate 11, the harvested durable learnings) - -The DSL is a *query format* for all 4, not a replacement. A `scan "src/foo.py"` is a curation-memory query; a `select .role == "User"` is a discussion-memory query; a `search "execution clutch"` is a RAG-memory query; a `read "knowledge/digest.md"` is a knowledge-memory query. - -### 6.5 Claim 5 — Stable-to-volatile cache ordering - -The DSL's `tape { }` blocks are cache-friendly per nagent v2.1 §2.2 stable-to-volatile ordering. The DSL's audit logs (Tier 4 `audit` verb) are a *stable* layer that can be cached across turns. The DSL's pipeline output (e.g., the output of `scan -> filter`) is a *volatile* layer appended per turn. - -### 6.6 Claim 6 — `Result[T]` envelope - -The DSL's `try { ... } recover { ... }` verb returns `Result[T]` per the convention established by `conductor/tracks/data_oriented_error_handling_20260606/spec.md` §3.3. The 12 `ErrorKind` values are the canonical error vocabulary. The `Result[T]` dataclass is the data-oriented alternative to exception-based control flow. - -### 6.7 Claim 7 — Command Palette 33 commands - -The DSL's verbs are a *richer* superset of the 33 Command Palette commands (per `docs/guide_command_palette.md` and `src/commands.py`). The "Everything" mode in the Command Palette (per `guide_command_palette.md` line 383: *"search across commands, files, symbols, history, settings"*) is a near-term use case where the DSL's verbs can be the underlying format. The user types `find "execution clutch"` instead of clicking on a result; the DSL parses the intent and dispatches to the right MCP tool. - -### 6.8 Claim 8 — Hook API state fields - -The DSL's verbs that mutate state route through `_predefined_callbacks` (per `docs/guide_state_lifecycle.md` §"Hook API Surface"). The verbs that read state use `_gettable_fields`. The DSL never bypasses the Hook API; it's a *user* of the existing infrastructure. - -### 6.9 Claim 9 — O'Donnell's IEventTarget pattern as the `sandbox` verb - -The `sandbox { ... }` block in Tier 4 is the DSL's IEventTarget boundary. Per O'Donnell at `https://johno.se/book/mvc.html` "Writing to Model state": *"Writes to Model are formalized through the addition of IEventTarget. This is a pure virtual interface that defines all possible state changes / events on a system wide level."* In the DSL, `sandbox { ... }` declares: every state change in this block goes through a single auditable interface (the bridge script's HITL approval modal per `docs/guide_meta_boundary.md`). The `audit` verb is the IEventTarget itself: a write-verb that logs the state change to a structured record (timestamp, source, kind, payload — same shape as `guide_architecture.md` §"Telemetry & Auditing" `Comms Log` entries). - -Per the cluster 0 sub-report (per `cluster_0_odonnell.md` §"Connections" Connection 1): *"The `sandbox` verb isolates execution and enforces that all state observations by the sandboxed code are *reads* — they can occur freely against the const Model view. State mutations by sandboxed code, however, must be routed through the formal event channel."* - -### 6.10 Claim 10 — O'Donnell's "reads are free" claim as the rationale for cheap verbs - -Per O'Donnell at `https://johno.se/book/mvc.html` "Reading Model state": *"First of all, View and Controller may only access Model in a const fashion. This has numerous repercussions. Firstly, exposing central Model state as public is ok, as it can only be read. Also, only const methods may be called, so state changes cannot be made internally as a result of a bad function call."* - -The Tier 2 verbs (`scan`, `filter`, `map`, `fold`, `sort`, `group`, `dedupe`) are *read-only* and can be re-evaluated freely, multiple times per execution, in parallel stages, without audit. Only the moment the chain's output is consumed by a write-verb (`exec`, `write`, `assign`) triggers the HITL modal. This is why the bridge script can re-execute a read-only chain without human approval. - -Per the cluster 0 sub-report (per `cluster_0_odonnell.md` §"Connections" Connection 2): *"O'Donnell's 'reads are free' claim is the rationale for cheap Tier 2 verbs — they can be re-evaluated freely because they never mutate state, so they can be re-evaluated freely, multiple times per execution, in parallel stages, without audit."* - ---- - -## 7. Open Questions for Follow-up B (≥6) - -These open questions must be answered by the follow-up B track (interpreter prototype). Each question is a design decision the interpreter must make. - -1. **How does `tape { }` map to Onat's preemptive scatter?** Is the block itself a tape-drive region, or is `tape` a wrapper that allocates a tape for the block's contents? The interpreter must decide whether `tape { ... }` is a parser hint (the parser pre-scatters) or a runtime directive (the runtime allocates a tape). The implication: parser-time optimization vs runtime flexibility. - -2. **Where does "intent resolution" live?** Is it a per-verb option, a per-block modifier, or a global parser mode? The `fuzzy` verb declares a parse-tolerance region; is this a property of the verb, of the block, or of the whole program? The interpreter must decide how `fuzzy` composes with non-`fuzzy` verbs in the same chain. - -3. **How does `audit` interact with `comms.log`?** Per `docs/guide_architecture.md` §"Telemetry & Auditing", the existing 5 log streams are `comms.log` (JSON-L for API traffic), `toolcalls.log` (markdown for tool invocations), `apihooks.log` (HTTP hook invocations), `clicalls.log` (subprocess details), and `scripts/generated/_.ps1` (preserved scripts). Is the DSL's audit log a 6th stream, or does it fold into one of the existing 5? Recommendation: a 6th stream (`audit.log`) because the DSL's audit is verb-level (every verb), while the existing 5 streams are tool-level (specific call types). - -4. **Does `sandbox` produce `Result[T, ErrorInfo]` (the Fleury pattern) or a different envelope?** Per `data_oriented_error_handling_20260606/spec.md` §3.3, the canonical `Result[T]` is a dataclass with `data: T` and `errors: list[ErrorInfo]`. The `sandbox { ... }` block can either use this envelope or a different one (e.g., `SandboxResult` with `stdout: str`, `stderr: str`, `exit_code: int`, `errors: list[ErrorInfo]`). The interpreter must decide. - -5. **`didyoumean` recovery: parser feature or user-facing verb?** If parser feature, the parser auto-corrects on parse failure and the user never sees the typo. If user-facing verb, the parser logs the typo, the user writes `didyoumean ""`, and gets a suggestion. The interpreter must decide whether `didyoumean` is part of the parse path or part of the runtime path. - -6. **How does `for x .. n` interact with Tier 2's `filter`/`map`?** Is `for x .. n { body }` sugar for `[1, 2, ..., n] -> map { body }`? Or are they distinct (the for-loop has named variable, the pipeline has anonymous position)? The interpreter must decide whether the user's pseudocode `for col .. m.columns { body }` is syntactic sugar for the array-language `iota m.columns { ... }`. - -7. **How does `sandbox` map to Manual Slop's `pre_tool_callback` flow?** The `sandbox` block's audit log: separate JSON-L file, or fold into the existing `comms.log` + `toolcalls.log`? (This is the same question as #3, but specifically about the runtime path — what happens when a `sandbox { write "tmp/x" "data" }` is actually executed by the bridge script?) - -8. **Connection to `intent_dsl_for_meta_tooling_20260608_PLACEHOLDER`:** what's the minimum subset of the report's vocab that would let the placeholder track (a) write a bridge script and (b) demonstrate one round-trip end-to-end? The placeholder's per-MCP grammar design (per `mcp_architecture_refactor_20260606/spec.md` §12.1) needs at least 1 Tier 1 verb, 1 Tier 2 verb per sub-MCP, and 1 Tier 4 verb (probably `sandbox` or `audit`). The minimum subset: 1-3 verbs, plus the grammar. - ---- - +**Heuristic summary.** *If the operator has precedence, postfix it. If it doesn''t, infix it.* + +The grammar formalizes 14 primitives drawn from the user's math pseudocode (the `determinate`/`minor`/`matrix-transpose` snippets shared during spec review), plus 3 known ambiguity flags, plus precedence rules and AI-fuzzing tolerance rules. + +### 3.1 The 14 primitives + +| # | Symbol | Name | Signature / Syntax | Meaning | Source example (user pseudocode) | +|---|---|---|---|---|---| +| 1 | `name := value` | Local bind | `name := expr` | Stack-scoped local declaration | `m rows . 1 - m columns . 1 - Matrix result :=` | +| 2 | `stack { ... }` | Stack scope | `stack { decl1; decl2; ... }` | Block of stack-allocated locals | `stack { ... result :=; Scalar row_offset :=; Scalar col_offset := }` | +| 3 | `name: Type` | Annotation | `name: Type` | Type hint on a binding | `m : Matrix` | +| 4 | `func(args) -> Type { ... }` | Function def | `func(args) -> Type { body }` | Named function with return type | `determinate(m, row) -> Scalar { ... }` | +| 5 | `name(...) proc { ... }` | Procedure def | `name(args) proc { body }` | Void-returning function | `minor(m, row_omit, column_omit) -> Scalar proc { ... }` | +| 6 | `for x .. n` | Range iteration | `for x .. n { body }` | Iterate `x` over `[0, n)` | `for col .. m.columns` | +| 7 | `name[a, b]` | Bracket indexing | `name[i, j, k, ...]` | Multi-dim array access | `result[row - row_offset, col - col_offset]` | +| 8 | `if cond { ... }` | Conditional | `if cond { then-body }` | If-then (else inferred) | `if col = col_omit { ++ col_offset; continue; }` | +| 9 | `return value` | Return | `return expr` | Function exit with value | `return result` | +| 10 | `->` (between verbs) | Pipeline flow | `verb1 -> verb2 -> verb3` | Output of left → input of right | `filter -> (col != column_omit <- for col .. m.columns)` | +| 11 | `<-` (after verb) | Input binding | `result <- producer` | The thing on the right is the producer | `for col .. m.columns` produces; `col != column_omit` consumes | +| 12 | `=` (in `assert`) | Equality | `assert -> lhs = rhs` | Assert two expressions are equal | `assert -> product(...) = product(...)` | +| 13 | `{ }` | Body block | `{ body }` | Function/scope body | `{ ... }` | +| 14 | `[ ]` | Basic block | `[ my_stage ]` | Onat's compilation unit (no branching semantics) | (not in user pseudocode; from KYRA's basic blocks) | + +### 3.2 Ambiguity flags + +Per the user's note during spec review (*"Hopefully the above don't have too many logic errors that the use can't be clarified."*), three known ambiguities in the user's pseudo code are normalized in the report: + +- **`proc` modifier placement:** `minor(m, row_omit, column_omit) -> Scalar proc { ... }` — likely a *type qualifier* (the return type is "Scalar" + "proc"-ness means side-effecting). The report adopts the convention that `proc` is a postfix modifier indicating void-returning; the syntax is `name(args) proc { body }` (return type omitted) or `name(args) -> Type proc { body }` (return type explicit but ignored). +- **`++col_offset`:** likely `col_offset += 1`. The report formalizes as `name += 1` (Python-style augmented assignment) and does not adopt the `++` operator. This avoids confusion between pre-increment and post-increment. +- **`m[row][column]` vs `m[row, col]`:** both appear in the user's snippets (line 24 `m[row][column]` is likely a typo for `m[row][col]`). The report adopts the comma-form (`name[a, b]`, multi-dim) throughout, since the C-style chained-bracket form doesn't compose with the user's existing matrix pseudocode. + +### 3.3 Precedence rules + +- **Left-to-right for `->` chains:** `a -> b -> c` parses as `(a -> b) -> c` (b's output becomes c's input). This is *not* the standard math convention (right-to-left) but it matches the user's pseudocode and the pipeline model. +- **`(` `)` for grouping:** explicit parentheses override the left-to-right default. `a -> (b -> c)` parses as `a -> X` where `X = (b -> c)`. +- **Stack-binding precedence:** `:=` binds tighter than `<-`. `producer expr <- result :=` parses as `producer (expr <-) result :=` (the `expr <- producer` consumes the producer into expr before `result :=` stores it). +- **No operator precedence for arithmetic:** `+`, `-`, `*`, `/`, `^` are all left-associative with equal precedence. `2 + 3 * 4` parses as `(2 + 3) * 4 = 20`. (This is the APL/K convention. If the user wants math precedence, the report can adopt explicit `(` `)`.) + +### 3.4 AI-fuzzing tolerance rules + +These are the rules that make the DSL workable for AI agents that may fuzz verb names, indent inconsistently, or offset line references. + +- **CoSy-style modulo indexing:** array indices wrap. `result[-1]` is equivalent to `result[result.len - 1]`. This forgives AI off-by-one errors in line references. (Per the CoSy Simplicity page: *"Indexing is modulo - like counting on your thumb & fingers : 0 1 2 3 4 0."*) +- **Structured recovery anchors via `{ }`:** the `{ }` block is a recovery unit. If the parser cannot parse the body, the entire block is replaced with `NIL` and the error is reported at the block level, not at the line level. +- **Line/offset independence:** the parser uses *token positions*, not raw line numbers. A token's position is `file:token-index` (e.g., `src/foo.py:42` means "the 42nd token in src/foo.py"), not `file:42` (which would be "line 42"). The mapping from token position to line number is a presentation concern, not a parse concern. This matches the project's existing FuzzyAnchor pattern (per `docs/guide_context_curation.md`). +- **Verb-name fuzzing tolerance:** the `didyoumean` verb (see §4 Tier 4) proposes corrections for ambiguous verb names. The parser's "best guess" recovery path is configurable: strict (reject on typo), lenient (auto-correct if Levenshtein distance ≤ 2), or fuzzy (parse the rest, log the typo). +- **Indentation tolerance:** indentation is *not* significant (per the user's explicit "ignore its record formats" instruction and the rejection of Python's indent-sensitive syntax). The parser uses a stack-based approach; the `{ }` and `[ ]` delimiters are the only structure-aware tokens. + +### 3.5 Error envelope: `try { ... } recover { ... }` + +``` +try { + scan "src/foo.py" -> filter !exists -> print +} recover err { + audit "scan failed: " + err + return NIL +} +``` + +- The `try` block evaluates the pipeline. If the pipeline returns a `Result[T]` with `errors` non-empty, the `recover` block runs. +- The `recover` block receives the `Result[T]` as a parameter (named by the user; `err` is the default convention from the user's pseudocode). +- The `recover` block must return a `Result[T]` (or `NIL` to short-circuit). +- If the `recover` block itself returns a `Result[T]` with errors, those errors are appended to the outer `Result[T]`'s error list. (Per Fleury's "errors are data" pattern; per `data_oriented_error_handling_20260606/spec.md` §3.4.) + +### 3.6 Block composition: `[ ]` (KYRA basic blocks) vs `{ }` (body blocks) vs `tape { }` (memory regions) + +- **`[ ]`** is Onat's basic block (per `C:\projects\forth\bootslop\references\kyra_in-depth.md:56-57`): *"Basic blocks `[ ]` provide implicit begin/link/end jump targets for the JIT to resolve relative offsets within a limited scope."* In the DSL, `[ ]` is a *sequential operation block* — a chunk of code that the parser can compile and dispatch as a unit. It is *not* a scope (no new bindings); it is a *compilation unit*. +- **`{ }`** is a body block: function body, if/then body, recover body. It introduces a new lexical scope (new bindings are local to the block). +- **`tape { }`** is a tape-drive region: a `{ }` body that has been *pre-scattered* into a contiguous memory region. The contents are pre-placed; the JIT can emit the entire block as a single `xchg rax, rdx` boundary (per KYRA's magenta pipe semantics). + +The three are nested by the parser: `tape { foo := x; [ bar ]; baz }` is a tape region containing 2 sequential statements (the local bind and the basic block) and a trailing call. is a tape region containing 2 sequential statements (the local bind and the basic block) and a trailing call. + +--- + +## 4. The 4-Tier Vocab (~40 Verbs) + +Each verb: symbol, name, signature, one-line semantics, one example, "borrowed from" note, SSDL shape tag. Tier 2 and Tier 3 verbs also have a "maps to mcp_client tool" column. Tier 4 verbs have a "novel piece" note. + +### 4.1 Tier 1 — Math (~10 verbs) + +The Tier 1 verbs are drawn directly from the user's math pseudocode. + +| Symbol | Name | Signature | Semantics | Example | Borrowed from | Shape | +|---|---|---|---|---|---|---| +| `:=` | Local bind | `name := expr` | Stack-scoped local declaration | `m rows . 1 - m columns . 1 - Matrix result :=` | Forth (dictionary entries); Joy (quotations) | `[I]` | +| `stack { ... }` | Stack scope | `stack { decl1; decl2; ... }` | Block of stack-allocated locals | `stack { ... result :=; Scalar row_offset :=; Scalar col_offset := }` | Forth (colon definitions); KYRA (basic blocks) | `[I]` | +| `for x .. n` | Range iteration | `for x .. n { body }` | Iterate `x` over `[0, n)` | `for col .. m.columns` | APL `ιN`; K `!R`; BQN `↕N`; Uiua (stack iteration) | `o->` | +| `+` | Add | `a b +` | Element-wise sum | `2 + 3` (yields 5) | All languages | `[I]` | +| `-` | Subtract | `a b -` | Element-wise difference | `5 - 2` (yields 3) | All languages | `[I]` | +| `*` | Multiply | `a b *` | Element-wise product | `2 * 3` (yields 6) | All languages | `[I]` | +| `/` | Divide | `a b /` | Element-wise division | `6 / 2` (yields 3) | All languages | `[I]` | +| `^` | Power | `a b ^` | Element-wise power | `2 ^ 10` (yields 1024) | All languages | `[I]` | +| `sum` | Sum | `expr sum` | Sum all elements | `sum 1..10` (yields 55) | APL `+/`; K `+/`; BQN `+` | `[I]` | +| `product` | Product | `expr product` | Product all elements | `product 1..5` (yields 120) | APL `×/`; K `*/`; BQN `×` | `[I]` | +| `a[i, j]` | Bracket indexing | `name[i, j, ...]` | Multi-dim array access | `result[row - row_offset, col - col_offset]` | APL `result[2;3]`; BQN `⊏`; K `@` | `[Q]` (query) | +| `if/then` | Conditional | `if cond { then-body }` | If-then (else inferred) | `if col = col_omit { ++ col_offset; continue; }` | Forth (IF/THEN); CoSy (control flow) | `[B]` (branch) | + +**Total Tier 1: 12 verbs.** (Slightly over the 10 estimate; the verbs are tight enough that splitting them hurts readability.) + +### 4.2 Tier 2 — Data-Oriented Pipeline (~12 verbs) + +The Tier 2 verbs wrap the existing 45+ MCP tools (per `docs/guide_tools.md` §"Native Tool Inventory") with declarative intent expressions. They are the "imperative veneer" over the Jofito-style predicate chain. + +| Symbol | Name | Signature | Semantics | Example | Maps to mcp_client tool | Borrowed from | Shape | +|---|---|---|---|---|---|---|---| +| `scan` | Scan | `scan path` | Read source (directory, file, URL); first verb in every pipeline | `scan "src/" -> filter !dir -> map ext` | `list_directory` + `search_files` + `read_file` | Jofito `scandir()` | `[I]` | +| `select` | Select | `select condition` | Keep records matching condition (jq-style filter) | `scan "src/" -> select .extension == ".py"` | (jq-style filter) | jq `select(condition)`; Joy `filter` | `->` | +| `filter` | Filter | `filter predicate` | Keep records where predicate is true | `scan "src/" -> filter .size > 0` | (predicate on FileItem) | Jofito `{filter ...}` predicate | `->` | +| `map` | Map | `map block` | Apply block to each record | `scan "src/" -> map ext` | (no direct equivalent) | jq `.[] | .field`; Joy `map`; CoSy `' verb 'm` | `o->` | +| `fold` | Fold | `fold init block` | Reduce to single value | `scan "src/" -> fold 0 { acc + .size }` | (no direct equivalent) | jq `reduce`; Joy `fold` | `o->` | +| `sort` | Sort | `sort key` | Order records by key | `scan "src/" -> sort .name` | (no direct equivalent) | Joy `qsort`; jq `sort` | `[I]` | +| `group` | Group | `group key` | Bucket records by key | `scan "src/" -> group .extension` | (no direct equivalent) | jq `group_by`; CoSy APL-derived | `o->` | +| `dedupe` | Dedupe | `dedupe` | Remove duplicates | `scan "src/" -> dedupe` | (no direct equivalent) | jq `unique`; CoSy | `[I]` | +| `tape { }` | tape scope | `tape { body }` | Tape-drive region; pre-scatter contents | `tape { [ scan ]; [ filter ]; [ print ] }` | (compiler directive) | KYRA magenta pipe; Onat preemptive scatter | `o->` | +| `scatter` | Scatter | `scatter workers` | Fork pipeline across `workers` cores | `scan "src/" -> scatter 4 -> filter` | (runtime hint) | Onat preemptive scatter; Lottes X.com thread line 55-61 | `=>` | +| `gather` | Gather | `gather` | Collect scattered sub-streams | `scan "src/" -> scatter 4 -> filter -> gather` | (runtime hint) | Onat inverse of scatter | `[I]` | +| `pipe` | Pipe root | `pipe` | Explicit chain root (synonym for `->`) | `pipe [ scan, filter, print ]` | (no direct equivalent) | Jofito pipe coalescing (transcript:376-410) | `=>` | + +**Total Tier 2: 12 verbs.** + +### 4.3 Tier 3 — Shell (~10 verbs) + +The Tier 3 verbs wrap existing MCP tools (per `docs/guide_tools.md` §"Native Tool Inventory") and provide the shell-scripting surface. They are the "imperative veneer" over the declarative Tier 2 pipeline. + +| Symbol | Name | Signature | Semantics | Example | Maps to mcp_client tool | Borrowed from | Shape | +|---|---|---|---|---|---|---|---| +| `exec` | Execute | `exec cmd` | Run shell command | `exec "find . -name '*.py'"` | `run_powershell` (shell_runner.py) | nagent tag protocol (structured protocol idea) | `[I]` | +| `open` | Open | `open path` | Open file/URL | `open "src/foo.py"` | `read_file` | nagent tag protocol | `[I]` | +| `read` | Read | `read path` | Read file content | `read "src/foo.py"` | `read_file` | nagent tag protocol | `[I]` | +| `write` | Write | `write path content` | Write file content | `write "src/foo.py" "new content"` | `set_file_slice` / `edit_file` | nagent tag protocol | `[I]` | +| `close` | Close | `close handle` | Close handle | `close file_handle` | (no direct equivalent; close is implicit in Python) | Forth `CLOSE-FILE`; bash `exec` | `[I]` | +| `path` | Path | `path` | Get current path (or `cd`) | `path` | (no direct equivalent; use `cwd`) | shell `pwd`; CoSy `path` | `[I]` | +| `env` | Env | `env var` | Get env var | `env HOME` | (no direct equivalent) | shell `echo $HOME` | `[I]` | +| `wait` | Wait | `wait ms` | Block for `ms` milliseconds | `wait 1000` | (no direct equivalent) | shell `sleep` | `o->` | +| `poll` | Poll | `poll handle ms` | Poll handle with timeout | `poll file_handle 5000` | (no direct equivalent) | shell `read -t` | `o->` | +| `cwd` | CWD | `cwd` | Get current working directory | `cwd` | (no direct equivalent) | shell `pwd` | `[I]` | + +**Total Tier 3: 10 verbs.** + +### 4.4 Tier 4 — AI-Fuzzing Tolerance (~8 verbs, the novel contribution) + +The Tier 4 verbs are what make the DSL workable for AI agents that may fuzz verb names, indent inconsistently, or offset line references. Each verb directly maps to one or more of the 4 anchor claims (especially Claim 3: IEventTarget, per Cluster 0). + +| Symbol | Name | Signature | Semantics | Example | Novel piece | Borrowed from | Shape | +|---|---|---|---|---|---|---|---| +| `fuzzy` | Fuzzy | `fuzzy expr` | Declare a parse-tolerance region; parser accepts near-matches | `fuzzy { scan "src/" -> filter .ext }` | Tolerance for AI verb-name fuzzing | nagent "discovery" intent (per `decisions.md:119,128`); SSDL "assume as much as possible" | `->` | +| `try { ... } recover { ... }` | Try / Recover | `try { body } recover err { fallback }` | Returns `Result[T]`; on error, the `recover` block runs | `try { read "src/foo.py" } recover { read "src/Foo.py" }` | Error envelope as data (Fleury pattern) | `data_oriented_error_handling_20260606`; Wasm `try`/`catch` block/loop/if/end | `->B->` | +| `sandbox { ... }` | Sandbox | `sandbox { body }` | IEventTarget boundary; all writes in the block go through the formal event channel | `sandbox { write "tmp/x" "data" }` | O'Donnell's "reads free, writes formalized" invariant applied to the DSL | O'Donnell `mvc.html` "Writing to Model state" | `o->` | +| `audit` | Audit | `audit msg` | Log the state change to a structured record; the IEventTarget itself | `audit "wrote tmp/x"` | Per-write audit log; full replay capability | O'Donnell `mvc.html` "Event callbacks"; nagent's self-describing tools | `[I]` | +| `didyoumean` | Did you mean | `didyoumean ambiguous` | Propose the closest matching verb(s) for an ambiguous input | `didyoumean "skan"` | Recovery primitive for AI typos | nagent Bridge DSL intent model; Anthropic `input_examples` | `[I]` | +| `span` | Span | `span intent` | Decompose a compound intent into a span of sub-MCP grammar tokens | `span "read foo.py:MyClass"` | Spans the `read_file` and `py_get_definition` tools | MCP DSL per-MCP grammar (`spec.md:456-465`); OpenAI namespace grouping | `[I]` | +| `offset` | Offset | `offset symbol` | Resolve a symbol to a file:line without requiring the model to specify the line | `offset "foo.py:MyClass.method"` | Implicit offset resolution | MCP DSL line-range notation; OpenAI "don't make the model fill known args" | `[Q]` | +| `assumewide` | Assume wide | `assumewide intent` | If the intent is broad or ambiguous, select the most-capable matching tool (the "fewer, more capable" heuristic) | `assumewide "refactor"` | Prefer broad-capability tools over narrow specialists | OpenAI "fewer than 20 functions"; Anthropic `tool_choice: tool` force-call | `=>` | + +**Total Tier 4: 8 verbs.** + +**Total vocab: 12 + 12 + 10 + 8 = 42 verbs.** (~40 estimate; slightly over because Tier 1 is 12 instead of 10, but Tier 3 is 10 and Tier 4 is 8.) + +--- + +## 5. Hardware Mapping (4 Anchor Claims) + +The 4 anchor claims tie the vocab and grammar to actual hardware/software stages. + +### 5.1 Claim 1 — Onat/Lottes, hardware + +The DSL's `->` pipeline, `[ ]`/`{ }` blocks, `tape { }` memory model, and `scatter`/`gather` verbs are direct descendants of KYRA/VAMP and x68. + +- **`->` pipeline:** inherits from Forth's postfix word chain, refined by KYRA's 2-register stack (RAX/RDX) as the minimal call convention. Per `C:\projects\forth\bootslop\references\kyra_in-depth.md:14` (*"The 2-Item Hardware Stack: To achieve hardware locality and GPU compatibility, KYRA strictly restricts the data stack to exactly two CPU registers: `RAX` (Top of Stack) and `RDX` (Next on Stack)"*). +- **`[ ]` sequential block:** inherits from KYRA's basic blocks `[ ]` with implicit begin/link/end jump targets. Per `kyra_in-depth.md:56-57` (*"Basic Blocks `[ ]`: These visually constrain the assembly output. They provide implicit begin, link (else), and end jump targets for the JIT to resolve relative offsets within a limited scope"*). +- **`{ }` lambda block:** inherits from KYRA's lambdas `{ }` that compile code elsewhere and leave an address in `RAX`. Per `kyra_in-depth.md:58-59` (*"Lambdas `{ }`: A lambda (colored Yellow `{`) does not execute inline. The JIT compiles the block of code elsewhere in the tape and leaves its executable memory address in `RAX`."*). +- **`tape { }`:** inherits from KYRA's magenta pipe `|` definition boundary (`RET` + `xchg rax, rdx`) as the entry/exit protocol for a memory region. Per `kyra_in-depth.md:24-27` (*"The Magenta Pipe Trick: Because the stack is just `RAX` and `RDX`, ensuring `RAX` is the active 'Top of Stack' before executing a word is vital. The `xchg rax, rdx` instruction compiles to a tiny 2-byte opcode: `48 92`. Definitions: There are no `begin` or `end` words. A magenta pipe token (`|`) implicitly signals the start of a new definition. The JIT reacts to this by: 1. Emitting a `RET` (`C3`) to close the *previous* definition. 2. Emitting `48 92` (`xchg rax, rdx`) to ensure proper stack alignment for the *new* definition."*). +- **`scatter`:** inherits from Onat's preemptive scatter — per `X.com - Onat & Lottes Interaction 1.png.ocr.md:59-61`: *"The key concept here is that 'common' arguments like the device are pushed onto the tape using store duplication when they are known (after device creation). So it's preemptive scatter, so later at call time there is no argument gather."* +- **`gather`:** the inverse of preemptive scatter — collect pre-scattered values from fixed memory slots. + +Lottes's specific framing at `X.com - Onat & Lottes Interaction 1.png.ocr.md:80-86`: *"I laugh when people say C is like assembly, they are missing what we **actually** did in assembly back then, which was all registers and globals and gotos, no stacks. It's radically different than good assembly."* The DSL's 2-register model + tape regions + magenta `->` are a direct application of this insight: don't pretend you have a memory stack when the hardware has registers. + +### 5.2 Claim 2 — O'Donnell, paradigm + +The DSL's pipeline is *immediate-mode in pipeline composition*. Each `->`-delimited stage is a method invocation, not a Pipeline object. The pipeline exists *only* while the DSL program is being executed; once execution ends, the pipeline's state is gone. + +Per O'Donnell at `https://johno.se/book/imgui.html`: *"Widgets, logically, change from being objects to being method invocations. As we shall see, this fundamentally changes how a client application approaches the implementation of user interfaces."* + +The DSL inherits this: `scan -> filter -> print` is not a pipeline object you can query, name, or pass around. The only way to "name" a chain is to wrap it in a function (`determinate(m, row) -> Scalar { ... }`). The function body IS the chain; the function name IS the chain's identity. There is no separate Pipeline class. + +This also means: the parser doesn't need to track pipeline state across executions. Each invocation of `determinate(m, row)` is independent. There is no "current pipeline" implicit state. The next call is fresh. + +### 5.3 Claim 3 — Forth/CoSy, syntax + +Concatenative syntax is immediate-mode in *tokenization* (whitespace-delimited, no precedence), in *evaluation* (each verb pops args, pushes results), and in *parsing* (no AST object retained after the parse — the parser emits JIT'd code directly per Onat's xchg model). + +- **Tokenization:** whitespace-delimited, no precedence table. Per `https://en.wikipedia.org/wiki/Forth_(programming_language)`: *"Forth's grammar has no official specification. Instead, it is defined by a simple algorithm. The interpreter reads a line of input from the user input device, which is then parsed for a word using spaces as a delimiter."* +- **Evaluation:** each verb pops args, pushes results. Per CoSy Simplicity: *"Words pass information to each other by pushing it on, or taking it off a `stack`."* +- **Parsing:** no AST object retained after parse. The parser emits directly. Per `data_oriented_error_handling_20260606/spec.md` §3.1 and the project's overall "data-oriented design" philosophy, parsing is data flow, not object construction. + +The DSL inherits all three. The parser reads whitespace-delimited tokens, evaluates each verb as a stack effect, and emits the result without retaining an AST. + +### 5.4 Claim 4 — APL/K, data + +Array languages are immediate-mode in *data representation*. There is no array-object header; values are passed by stack reference, not by handle. + +- **APL** (per `https://en.wikipedia.org/wiki/APL_(programming_language)`): *"APL has an array as the universal data type"* — scalar `5` is a 0-dimensional array; `4 5 6 7 + 4` propagates the addition across the vector. +- **K** (per `https://en.wikipedia.org/wiki/K_(programming_language)`): "kdb+ (built on K) processes billions of records at microsecond latency" — the array paradigm scales to production workloads. +- **BQN** (per `https://mlochbaum.github.io/BQN/`): the CBQN bytecode compiler confirms the paradigm can be compiled efficiently. + +The DSL's `for x .. n` range + `result[row, col]` indexing inherits the "no array object" property. The array is *the* universal type; every function operates on it; every function vectorizes. + +--- + +## 6. AI-Agent Properties (10 Claims) + +The 10 claims tie the DSL to the existing project's architecture so future tracks can build on it without re-deriving the design. + +### 6.1 Claim 1 — Domain = Meta-Tooling + +The DSL is **Meta-Tooling-side** per `docs/guide_meta_boundary.md` §"Domain 2: The Meta-Tooling". The Application's provider-native function-calling stays unchanged. The DSL is the format external agents (Gemini CLI, OpenCode) emit when invoking `mcp_client.py` tools. + +### 6.2 Claim 2 — Runtime path = external agent → DSL → bridge → MCP → optional Hook API approval + +Per `docs/guide_meta_boundary.md` §"The Inter-Domain Bridges": external agents (Gemini CLI) call the DSL via a bridge script (`scripts/cli_tool_bridge.py` analogue). The bridge script translates the DSL into `mcp_client.dispatch()` calls. The Hook API (`docs/guide_tools.md` §"The Hook API") surfaces HITL approval modals when the bridge detects a `sandbox { ... }` block. + +### 6.3 Claim 3 — 3-layer security + +The DSL's parser respects the existing 3-layer security model in `mcp_client.py` (per `docs/guide_tools.md` §"The MCP Bridge"). Every DSL statement that targets a tool outside the allowlist is rejected at parse time. The 3 layers are: allowlist construction, path validation, and resolution gate. The DSL does not bypass any of these. + +### 6.4 Claim 4 — 4 memory dimensions + +The DSL does *not* replace any of the 4 memory dimensions (per `conductor/tracks/nagent_review_20260608/nagent_review_v2_1_20260612.md` §2.1): +- **Curation memory** (FileItem + ContextPreset + FuzzyAnchor) +- **Discussion memory** (disc_entries + branching + UISnapshot A1-A7) +- **RAG memory** (ChromaDB, opt-in) +- **Knowledge memory** (Candidate 11, the harvested durable learnings) + +The DSL is a *query format* for all 4, not a replacement. A `scan "src/foo.py"` is a curation-memory query; a `select .role == "User"` is a discussion-memory query; a `search "execution clutch"` is a RAG-memory query; a `read "knowledge/digest.md"` is a knowledge-memory query. + +### 6.5 Claim 5 — Stable-to-volatile cache ordering + +The DSL's `tape { }` blocks are cache-friendly per nagent v2.1 §2.2 stable-to-volatile ordering. The DSL's audit logs (Tier 4 `audit` verb) are a *stable* layer that can be cached across turns. The DSL's pipeline output (e.g., the output of `scan -> filter`) is a *volatile* layer appended per turn. + +### 6.6 Claim 6 — `Result[T]` envelope + +The DSL's `try { ... } recover { ... }` verb returns `Result[T]` per the convention established by `conductor/tracks/data_oriented_error_handling_20260606/spec.md` §3.3. The 12 `ErrorKind` values are the canonical error vocabulary. The `Result[T]` dataclass is the data-oriented alternative to exception-based control flow. + +### 6.7 Claim 7 — Command Palette 33 commands + +The DSL's verbs are a *richer* superset of the 33 Command Palette commands (per `docs/guide_command_palette.md` and `src/commands.py`). The "Everything" mode in the Command Palette (per `guide_command_palette.md` line 383: *"search across commands, files, symbols, history, settings"*) is a near-term use case where the DSL's verbs can be the underlying format. The user types `find "execution clutch"` instead of clicking on a result; the DSL parses the intent and dispatches to the right MCP tool. + +### 6.8 Claim 8 — Hook API state fields + +The DSL's verbs that mutate state route through `_predefined_callbacks` (per `docs/guide_state_lifecycle.md` §"Hook API Surface"). The verbs that read state use `_gettable_fields`. The DSL never bypasses the Hook API; it's a *user* of the existing infrastructure. + +### 6.9 Claim 9 — O'Donnell's IEventTarget pattern as the `sandbox` verb + +The `sandbox { ... }` block in Tier 4 is the DSL's IEventTarget boundary. Per O'Donnell at `https://johno.se/book/mvc.html` "Writing to Model state": *"Writes to Model are formalized through the addition of IEventTarget. This is a pure virtual interface that defines all possible state changes / events on a system wide level."* In the DSL, `sandbox { ... }` declares: every state change in this block goes through a single auditable interface (the bridge script's HITL approval modal per `docs/guide_meta_boundary.md`). The `audit` verb is the IEventTarget itself: a write-verb that logs the state change to a structured record (timestamp, source, kind, payload — same shape as `guide_architecture.md` §"Telemetry & Auditing" `Comms Log` entries). + +Per the cluster 0 sub-report (per `cluster_0_odonnell.md` §"Connections" Connection 1): *"The `sandbox` verb isolates execution and enforces that all state observations by the sandboxed code are *reads* — they can occur freely against the const Model view. State mutations by sandboxed code, however, must be routed through the formal event channel."* + +### 6.10 Claim 10 — O'Donnell's "reads are free" claim as the rationale for cheap verbs + +Per O'Donnell at `https://johno.se/book/mvc.html` "Reading Model state": *"First of all, View and Controller may only access Model in a const fashion. This has numerous repercussions. Firstly, exposing central Model state as public is ok, as it can only be read. Also, only const methods may be called, so state changes cannot be made internally as a result of a bad function call."* + +The Tier 2 verbs (`scan`, `filter`, `map`, `fold`, `sort`, `group`, `dedupe`) are *read-only* and can be re-evaluated freely, multiple times per execution, in parallel stages, without audit. Only the moment the chain's output is consumed by a write-verb (`exec`, `write`, `assign`) triggers the HITL modal. This is why the bridge script can re-execute a read-only chain without human approval. + +Per the cluster 0 sub-report (per `cluster_0_odonnell.md` §"Connections" Connection 2): *"O'Donnell's 'reads are free' claim is the rationale for cheap Tier 2 verbs — they can be re-evaluated freely because they never mutate state, so they can be re-evaluated freely, multiple times per execution, in parallel stages, without audit."* + +--- + +## 7. Open Questions for Follow-up B (≥6) + +These open questions must be answered by the follow-up B track (interpreter prototype). Each question is a design decision the interpreter must make. + +1. **How does `tape { }` map to Onat's preemptive scatter?** Is the block itself a tape-drive region, or is `tape` a wrapper that allocates a tape for the block's contents? The interpreter must decide whether `tape { ... }` is a parser hint (the parser pre-scatters) or a runtime directive (the runtime allocates a tape). The implication: parser-time optimization vs runtime flexibility. + +2. **Where does "intent resolution" live?** Is it a per-verb option, a per-block modifier, or a global parser mode? The `fuzzy` verb declares a parse-tolerance region; is this a property of the verb, of the block, or of the whole program? The interpreter must decide how `fuzzy` composes with non-`fuzzy` verbs in the same chain. + +3. **How does `audit` interact with `comms.log`?** Per `docs/guide_architecture.md` §"Telemetry & Auditing", the existing 5 log streams are `comms.log` (JSON-L for API traffic), `toolcalls.log` (markdown for tool invocations), `apihooks.log` (HTTP hook invocations), `clicalls.log` (subprocess details), and `scripts/generated/_.ps1` (preserved scripts). Is the DSL's audit log a 6th stream, or does it fold into one of the existing 5? Recommendation: a 6th stream (`audit.log`) because the DSL's audit is verb-level (every verb), while the existing 5 streams are tool-level (specific call types). + +4. **Does `sandbox` produce `Result[T, ErrorInfo]` (the Fleury pattern) or a different envelope?** Per `data_oriented_error_handling_20260606/spec.md` §3.3, the canonical `Result[T]` is a dataclass with `data: T` and `errors: list[ErrorInfo]`. The `sandbox { ... }` block can either use this envelope or a different one (e.g., `SandboxResult` with `stdout: str`, `stderr: str`, `exit_code: int`, `errors: list[ErrorInfo]`). The interpreter must decide. + +5. **`didyoumean` recovery: parser feature or user-facing verb?** If parser feature, the parser auto-corrects on parse failure and the user never sees the typo. If user-facing verb, the parser logs the typo, the user writes `didyoumean ""`, and gets a suggestion. The interpreter must decide whether `didyoumean` is part of the parse path or part of the runtime path. + +6. **How does `for x .. n` interact with Tier 2's `filter`/`map`?** Is `for x .. n { body }` sugar for `[1, 2, ..., n] -> map { body }`? Or are they distinct (the for-loop has named variable, the pipeline has anonymous position)? The interpreter must decide whether the user's pseudocode `for col .. m.columns { body }` is syntactic sugar for the array-language `iota m.columns { ... }`. + +7. **How does `sandbox` map to Manual Slop's `pre_tool_callback` flow?** The `sandbox` block's audit log: separate JSON-L file, or fold into the existing `comms.log` + `toolcalls.log`? (This is the same question as #3, but specifically about the runtime path — what happens when a `sandbox { write "tmp/x" "data" }` is actually executed by the bridge script?) + +8. **Connection to `intent_dsl_for_meta_tooling_20260608_PLACEHOLDER`:** what's the minimum subset of the report's vocab that would let the placeholder track (a) write a bridge script and (b) demonstrate one round-trip end-to-end? The placeholder's per-MCP grammar design (per `mcp_architecture_refactor_20260606/spec.md` §12.1) needs at least 1 Tier 1 verb, 1 Tier 2 verb per sub-MCP, and 1 Tier 4 verb (probably `sandbox` or `audit`). The minimum subset: 1-3 verbs, plus the grammar. + +--- + --- @@ -810,10 +812,10 @@ Per `docs/reports/computational_shapes_ssdl_digest_20260608.md` §1, the SSDL vo | # | Shape | One-line definition | SSDL symbol | |---|---|---|---| | 1 | **Instruction** | A single unit of computation. Reads data, writes data, or both. | `[I]` | -| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `===>` | -| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `===>W===>` | -| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o==>` | -| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `oo==>oo` | +| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `->` | +| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `=>` | +| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o->` | +| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `o=>` | | 6 | **Codecycle graph** | Multiple codecycles + the data they read and write. | `boxes + arrows` | **The 7 modifiers** (full table): @@ -1141,7 +1143,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - `for col .. m.columns { body }` (per user's pseudocode line 33) - `for i .. 10 { print i }` (prints 0-9) - `for i .. arr.size { sum +:= arr[i] }` (sum a vector) -- SSDL shape: `o==>` (codecycle — loops back to first instruction after last) +- SSDL shape: `o->` (codecycle — loops back to first instruction after last) **`name[a, b, ...]` (Bracket indexing).** - Signature: `name[i, j, k, ...]` @@ -1173,7 +1175,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - `sum 1..10` (yields 55; per APL `+/ι10`) - `product 1..5` (yields 120; per APL `×/ι5`) - `sum arr` (sum an arbitrary array) -- SSDL shape: `o==>` (codecycle; iterates over elements) +- SSDL shape: `o->` (codecycle; iterates over elements) **`if cond { then } [else { else }]` (Conditional).** - Signature: `if expression "{" program "}" [ "else" "{" program "}" ]` @@ -1204,7 +1206,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> select .extension == ".py"` - `scan "src/" -> select .size > 1024` -- SSDL shape: `===>` (codepath) +- SSDL shape: `->` (codepath) **`filter predicate` (Keep matching).** - Signature: `filter predicate` @@ -1213,7 +1215,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> filter .size > 0` - `scan "src/" -> filter exists` -- SSDL shape: `===>` +- SSDL shape: `->` **`map block` (Apply to each).** - Signature: `map block` @@ -1222,7 +1224,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> map ext` (yields list of extensions) - `scan "src/" -> map { x := .name; "src/" x + }` (full paths; math `+` is postfix, assignment is infix) -- SSDL shape: `o==>` (codecycle; iterates over records) +- SSDL shape: `o->` (codecycle; iterates over records) **`fold init block` (Reduce to single).** - Signature: `fold init_value block` @@ -1231,7 +1233,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> fold 0 { acc + .size }` (total bytes) - `scan "src/" -> fold "" { acc + .name + "\n" }` (concatenated names) -- SSDL shape: `o==>` +- SSDL shape: `o->` **`sort key` (Order).** - Signature: `sort key` @@ -1249,7 +1251,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> group .extension` (group by file extension) - `scan "src/" -> group .directory` (group by parent directory) -- SSDL shape: `o==>` (codecycle) +- SSDL shape: `o->` (codecycle) **`dedupe` (Remove duplicates).** - Signature: `dedupe` @@ -1267,7 +1269,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `tape { [ scan ]; [ filter ]; [ print ] }` (3 basic blocks in a tape region) - `tape { x := 42; y := x 2 *; use y }` (local tape; assignment is infix, math `*` is postfix) -- SSDL shape: `o==>` (codecycle; the region is a single execution unit) +- SSDL shape: `o->` (codecycle; the region is a single execution unit) **`scatter workers` (Fork).** - Signature: `scatter workers_expr` @@ -1276,7 +1278,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `scan "src/" -> scatter 4 -> filter` (4-way parallel) - `scan "src/" -> scatter auto -> map` (auto-detect core count) -- SSDL shape: `===>W===>` (wide codepath) +- SSDL shape: `=>` (wide codepath) **`gather` (Collect).** - Signature: `gather` @@ -1293,7 +1295,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `pipe [ scan, filter, print ]` (declarative list form; not the canonical syntax, future extension) - `pipe` (single token; equivalent to starting a `->` chain) -- SSDL shape: `===>W===>` +- SSDL shape: `=>` #### A.4.3 Tier 3 — Shell (10 verbs, full reference) @@ -1366,7 +1368,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `wait 1000` (wait 1 second) - `wait 50` (wait 50ms; useful for pacing a render loop) -- SSDL shape: `o==>` (codecycle) +- SSDL shape: `o->` (codecycle) **`poll handle ms` (Poll).** - Signature: `poll handle_expr ms_expr` @@ -1374,7 +1376,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Maps to: (no direct equivalent; `shell read -t`) - Examples: - `poll file_handle 5000` (poll with 5s timeout) -- SSDL shape: `o==>` +- SSDL shape: `o->` **`cwd` (Current working dir).** - Signature: `cwd` @@ -1393,7 +1395,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `fuzzy { scan "src/" -> filter .ext }` (accepts `skan` as a typo for `scan`) - `fuzzy { skan "src" }` (accepts the typo) -- SSDL shape: `===>` +- SSDL shape: `->` **`try { body } recover err { fallback }` (Try / Recover).** - Signature: `try "{" program "}" "recover" identifier "{" program "}"` @@ -1402,7 +1404,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `try { read "src/foo.py" } recover { read "src/Foo.py" }` (try capitalized variant) - `try { read "src/foo.py" -> filter !exists -> print } recover err { audit "scan failed: " + err; return NIL }` -- SSDL shape: `===>B===>` (codepath with branch) +- SSDL shape: `->B->` (codepath with branch) **`sandbox { body }` (Sandbox).** - Signature: `sandbox "{" program "}"` @@ -1411,7 +1413,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `sandbox { write "tmp/x" "data" }` (write goes through HITL approval) - `sandbox { exec "rm -rf /" }` (destructive exec requires approval) -- SSDL shape: `o==>` (codecycle; the region is an audit unit) +- SSDL shape: `o->` (codecycle; the region is an audit unit) **`audit msg` (Audit log).** - Signature: `audit message_expr` @@ -1454,7 +1456,7 @@ This subsection provides the complete reference for all 42 verbs in the 4 tiers. - Examples: - `assumewide "refactor"` (selects the broadest refactor tool) - `assumewide "find"` (selects a broad search rather than a narrow query) -- SSDL shape: `===>W===>` +- SSDL shape: `=>` --- @@ -1685,50 +1687,50 @@ This subsection provides extended discussion of the 8 open questions. The main r That's 4 verbs total, plus the grammar. The placeholder track can demonstrate a round-trip end-to-end with this subset. -**Trade-off:** a larger subset (e.g., all 42 verbs) would be more expressive but harder to implement. A smaller subset (e.g., just `scan`) wouldn't demonstrate the pipeline. 4 verbs is the sweet spot. - -**Recommendation:** 4-verb minimum. The placeholder track implements these 4 verbs + the 14-primitive grammar, demonstrates a round-trip, and leaves the remaining 38 verbs for the interpreter prototype (follow-up B track) to implement. - -#### A.7.9 Question 9 — Future reservation of `arena { }` for a separate concept - -**Background.** In v1.2, the block originally named `arena { }` (per v1.0 / v1.1) was renamed to `tape { }` because "tape" better fits the *sequential data-flow* use case (per the Lottes tape-drive metaphor, the preemptive-scatter model, and the `->` pipeline direction). The rename was *not* a synonym swap — `arena` is *not* a deprecated name for `tape`. It is reserved for a different, future concept. See the term-choice note in the A.8 glossary entry for `tape`. - -**Open question:** what is the precise semantic + grammar of the future `arena { }` block, and how does it compose with `tape { }`? - -**Proposed split (Tier 1 Orchestrator's recommendation, 2026-06-12):** - -| Block | Semantic | Implementation | Tier fit | Examples | -|-------|----------|----------------|----------|----------| -| `tape { }` (current v1.2) | *Sequential data flow* — ordered placement, source-as-you-go, fits `->` pipelines and Onat's preemptive scatter | Pre-scatter at parse time; emit KYRA-style `xchg rax, rdx` boundary at entry/exit | Tier 2 (pipeline), Tier 3 (shell), Tier 4 (sandbox) | `tape { [ scan ]; [ filter ]; [ print ] }`, `tape { x := 42; y := x 2 *; use y }` | -| `arena { }` (FUTURE) | *Bulk memory allocation* — bulk-allocate on entry, bulk-free on exit, host decides lifetime, fits FFI + Jofito-style data structures | Allocate from a host `MD_Arena` (or equivalent Python `arena`); free on block exit; data is reference-stable for the block's lifetime | Tier 3 (shell/FFI), Tier 4 (sandbox-internal) | `arena { buf: Bytes := mmap 1MB; fill buf from "stdin" }`, `arena { rec: Record := open "data.json" }` | - -**Composition.** `tape { arena { ... } }` would be valid and meaningful: a pipeline stage that uses an arena-backed buffer. The two blocks would *not* be mutually exclusive — they layer. - -**Trade-offs.** -- **Pro (split):** cleaner mental model. "Tape = flow, arena = storage." Aligns with established systems-programming terminology (Jofito, Metadesk, Handmade Hero). Reserves a meaningful term for a meaningful concept. -- **Con (split):** the user has to learn two block types for what could be one with a `mode` parameter. More grammar surface. -- **Pro (unify under `tape`):** less surface area, no risk of confusion about which to use. -- **Con (unify):** "tape" doesn't carry the right meaning for bulk allocation; the metaphor breaks down. The term `arena` is too well-established in systems programming to leave unused. - -**Recommendation:** *split.* Keep `tape { }` for sequential flow (v1.2, current). Defer `arena { }` to the follow-up B interpreter prototype (or a separate v1.3 / v2.0 track). The cost of waiting is small; the cost of prematurely unifying the two concepts and having to unsplit them later is high. The current A.8 glossary entry for `tape` already documents the reservation; the follow-up track can implement the `arena { }` block without re-litigating the name. - -**Concrete next step for the follow-up B track:** define the `arena { }` grammar rule (one new line in the A.3 EBNF), the allocation strategy (Python `arena` library? a C extension? the host's `MD_Arena` if Metadesk-style FFI is added?), and at least 2-3 example uses in the A.4 verb reference. - +**Trade-off:** a larger subset (e.g., all 42 verbs) would be more expressive but harder to implement. A smaller subset (e.g., just `scan`) wouldn't demonstrate the pipeline. 4 verbs is the sweet spot. + +**Recommendation:** 4-verb minimum. The placeholder track implements these 4 verbs + the 14-primitive grammar, demonstrates a round-trip, and leaves the remaining 38 verbs for the interpreter prototype (follow-up B track) to implement. + +#### A.7.9 Question 9 — Future reservation of `arena { }` for a separate concept + +**Background.** In v1.2, the block originally named `arena { }` (per v1.0 / v1.1) was renamed to `tape { }` because "tape" better fits the *sequential data-flow* use case (per the Lottes tape-drive metaphor, the preemptive-scatter model, and the `->` pipeline direction). The rename was *not* a synonym swap — `arena` is *not* a deprecated name for `tape`. It is reserved for a different, future concept. See the term-choice note in the A.8 glossary entry for `tape`. + +**Open question:** what is the precise semantic + grammar of the future `arena { }` block, and how does it compose with `tape { }`? + +**Proposed split (Tier 1 Orchestrator's recommendation, 2026-06-12):** + +| Block | Semantic | Implementation | Tier fit | Examples | +|-------|----------|----------------|----------|----------| +| `tape { }` (current v1.2) | *Sequential data flow* — ordered placement, source-as-you-go, fits `->` pipelines and Onat's preemptive scatter | Pre-scatter at parse time; emit KYRA-style `xchg rax, rdx` boundary at entry/exit | Tier 2 (pipeline), Tier 3 (shell), Tier 4 (sandbox) | `tape { [ scan ]; [ filter ]; [ print ] }`, `tape { x := 42; y := x 2 *; use y }` | +| `arena { }` (FUTURE) | *Bulk memory allocation* — bulk-allocate on entry, bulk-free on exit, host decides lifetime, fits FFI + Jofito-style data structures | Allocate from a host `MD_Arena` (or equivalent Python `arena`); free on block exit; data is reference-stable for the block's lifetime | Tier 3 (shell/FFI), Tier 4 (sandbox-internal) | `arena { buf: Bytes := mmap 1MB; fill buf from "stdin" }`, `arena { rec: Record := open "data.json" }` | + +**Composition.** `tape { arena { ... } }` would be valid and meaningful: a pipeline stage that uses an arena-backed buffer. The two blocks would *not* be mutually exclusive — they layer. + +**Trade-offs.** +- **Pro (split):** cleaner mental model. "Tape = flow, arena = storage." Aligns with established systems-programming terminology (Jofito, Metadesk, Handmade Hero). Reserves a meaningful term for a meaningful concept. +- **Con (split):** the user has to learn two block types for what could be one with a `mode` parameter. More grammar surface. +- **Pro (unify under `tape`):** less surface area, no risk of confusion about which to use. +- **Con (unify):** "tape" doesn't carry the right meaning for bulk allocation; the metaphor breaks down. The term `arena` is too well-established in systems programming to leave unused. + +**Recommendation:** *split.* Keep `tape { }` for sequential flow (v1.2, current). Defer `arena { }` to the follow-up B interpreter prototype (or a separate v1.3 / v2.0 track). The cost of waiting is small; the cost of prematurely unifying the two concepts and having to unsplit them later is high. The current A.8 glossary entry for `tape` already documents the reservation; the follow-up track can implement the `arena { }` block without re-litigating the name. + +**Concrete next step for the follow-up B track:** define the `arena { }` grammar rule (one new line in the A.3 EBNF), the allocation strategy (Python `arena` library? a C extension? the host's `MD_Arena` if Metadesk-style FFI is added?), and at least 2-3 example uses in the A.4 verb reference. + --- ### A.8 Glossary **AI agent** — an LLM-based system that emits intent (DSL verbs) to invoke tools. The bridge script translates the intent into tool calls. Examples: Gemini CLI, OpenCode, Claude Code. -**tape** — a memory region modeled on a tape drive, declared with `tape { }`. The block's contents are pre-scattered into a contiguous buffer for efficient execution. - -> **Term-choice note (v1.2).** `tape { }` was deliberately renamed from `arena { }` in v1.2. The renaming decision: the Lottes tape-drive metaphor (per `X.com - Onat & Lottes Interaction 1.png.ocr.md:52-55`: *"lay out all the arguments in memory like a tape drive in the order that functions get called and source that tape at runtime for the calls"*) is a better fit for the DSL's *sequential data-flow* use case than the bulk-allocation metaphor implied by "arena". `tape` evokes ordered placement and a flow direction (matching the DSL's `->` pipeline and Onat's preemptive-scatter model); `arena` evokes a bucket. The DSL's primary verb-tier use of the block is sequential, not bulk-allocating. See Open Question A.7.9 for the future reservation. -> -> **`arena` is reserved for a future concept (do not use in v1.2+).** Per the term-choice note, `arena` is *not* a deprecated synonym for `tape` — it is reserved for a separate, future block primitive. Concretely: if the DSL grows a need for a *bulk memory allocator* (a place to stash intermediate data structures, a la Jofito's arena allocation, Metadesk's `MD_Arena`, or a future C-extension FFI block), that block would be `arena { }` and would have a *different semantic* from `tape { }`. The intended split: -> -> - `tape { }` = *sequential data flow* — ordered placement, source-as-you-go, fits `->` pipelines and Onat's preemptive scatter. Use this for: Tier 2 pipeline stages, basic blocks, lambdas. -> - `arena { }` (FUTURE, do not implement yet) = *bulk memory allocation* — bulk-allocate on entry, bulk-free on exit, with a "host decides lifetime" model. Use this for: FFI scratch buffers, C-extension intermediate storage, Jofito-style file/record data structures. -> +**tape** — a memory region modeled on a tape drive, declared with `tape { }`. The block's contents are pre-scattered into a contiguous buffer for efficient execution. + +> **Term-choice note (v1.2).** `tape { }` was deliberately renamed from `arena { }` in v1.2. The renaming decision: the Lottes tape-drive metaphor (per `X.com - Onat & Lottes Interaction 1.png.ocr.md:52-55`: *"lay out all the arguments in memory like a tape drive in the order that functions get called and source that tape at runtime for the calls"*) is a better fit for the DSL's *sequential data-flow* use case than the bulk-allocation metaphor implied by "arena". `tape` evokes ordered placement and a flow direction (matching the DSL's `->` pipeline and Onat's preemptive-scatter model); `arena` evokes a bucket. The DSL's primary verb-tier use of the block is sequential, not bulk-allocating. See Open Question A.7.9 for the future reservation. +> +> **`arena` is reserved for a future concept (do not use in v1.2+).** Per the term-choice note, `arena` is *not* a deprecated synonym for `tape` — it is reserved for a separate, future block primitive. Concretely: if the DSL grows a need for a *bulk memory allocator* (a place to stash intermediate data structures, a la Jofito's arena allocation, Metadesk's `MD_Arena`, or a future C-extension FFI block), that block would be `arena { }` and would have a *different semantic* from `tape { }`. The intended split: +> +> - `tape { }` = *sequential data flow* — ordered placement, source-as-you-go, fits `->` pipelines and Onat's preemptive scatter. Use this for: Tier 2 pipeline stages, basic blocks, lambdas. +> - `arena { }` (FUTURE, do not implement yet) = *bulk memory allocation* — bulk-allocate on entry, bulk-free on exit, with a "host decides lifetime" model. Use this for: FFI scratch buffers, C-extension intermediate storage, Jofito-style file/record data structures. +> > The two would compose: `tape { arena { ... } }` is a pipeline stage that uses an arena-backed buffer. The current report (v1.2) ships only `tape { }`; the `arena { }` block is left as a follow-up to the interpreter prototype (follow-up B track) to define, design, and implement. See Open Question A.7.9. **bridge script** — the runtime that translates DSL verbs into MCP tool calls. The `scripts/cli_tool_bridge.py` analogue. The Application's function-calling is unchanged; the bridge is the Meta-Tooling-side runtime. @@ -1737,9 +1739,9 @@ That's 4 verbs total, plus the grammar. The placeholder track can demonstrate a **clusters** — the 10 prior-art groups in §2: 0 (Immediate-Mode Paradigm), 1 (Concatenative), 2 (Array), 3 (Intent-Mapping), 4 (Meta-Tooling DSLs), 5 (SSDL), 6 (Command Palette), 7 (Result convention), 8 (Self-Describing Data + Tag Dispatch — Metadesk, added in v1.2), 9 (Multi-Paradigm Foundation Calculi with Transactional Semantics — Verse, added in v1.2). -**codepath** (SSDL) — a sequential list of instructions that terminates; no loops. SSDL symbol: `===>`. +**codepath** (SSDL) — a sequential list of instructions that terminates; no loops. SSDL symbol: `->`. -**codecycle** (SSDL) — a circular structure; a codepath that repeats at its first instruction after its last. SSDL symbol: `o==>`. +**codecycle** (SSDL) — a circular structure; a codepath that repeats at its first instruction after its last. SSDL symbol: `o->`. **DSL** — Domain-Specific Language. In this report, "DSL" refers specifically to the intent-based scripting language proposed in §3 and §4. @@ -1773,7 +1775,7 @@ That's 4 verbs total, plus the grammar. The placeholder track can demonstrate a **sandbox** — a Tier 4 verb that declares an IEventTarget boundary. All writes inside `sandbox { ... }` go through the formal event channel (HITL approval). -**SSDL** — Spec/Sketch Description Language, per `docs/reports/computational_shapes_ssdl_digest_20260608.md`. The DSL's verbs are annotated with SSDL shape tags (`[I]`, `===>`, `o==>`, `===>W===>`). +**SSDL** — Spec/Sketch Description Language, per `docs/reports/computational_shapes_ssdl_digest_20260608.md`. The DSL's verbs are annotated with SSDL shape tags (`[I]`, `->`, `o->`, `=>`). **tag protocol** — nagent's XML-ish self-closing tags for tool invocation. The DSL inherits the *idea* (named operation with typed attributes) but rejects the XML angle-bracket notation. @@ -1827,7 +1829,7 @@ The bibliography is split into 4 categories: external prior art, project's own r - **Anthropic tool-use** — `https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/define-tools`. *Key claim:* `input_examples` as a first-class schema field; `strict: true` as a guarantee. **Cluster 5 — SSDL:** -- **`docs/reports/computational_shapes_ssdl_digest_20260608.md` §1** — (6 primitives + 7 modifiers). *Key claim:* the meta-vocabulary for annotating verb shapes (`[I]`, `===>`, `o==>`, `===>W===>`). +- **`docs/reports/computational_shapes_ssdl_digest_20260608.md` §1** — (6 primitives + 7 modifiers). *Key claim:* the meta-vocabulary for annotating verb shapes (`[I]`, `->`, `o->`, `=>`). **Cluster 6 — Command Palette:** - **`docs/guide_command_palette.md`** + **`src/commands.py`** (33 commands). *Key claim:* the user's existing vocabulary instinct; the DSL is a richer superset. @@ -1892,4 +1894,4 @@ The sub-reports are the deep analyses that the main report's §2 condenses. Each --- *End of Appendix. The v1.1 report is the final deliverable. The follow-up B track (interpreter prototype) and the placeholder track (`intent_dsl_for_meta_tooling_20260608_PLACEHOLDER`) consume this report.* - + diff --git a/conductor/tracks/nagent_review_20260608/nagent_review_v2_2_20260612.md b/conductor/tracks/nagent_review_20260608/nagent_review_v2_2_20260612.md index 8ec68001..5963a6d1 100644 --- a/conductor/tracks/nagent_review_20260608/nagent_review_v2_2_20260612.md +++ b/conductor/tracks/nagent_review_20260608/nagent_review_v2_2_20260612.md @@ -22,17 +22,17 @@ | # | nagent v2 pattern | Maps to (intent DSL survey) | Manual Slop equivalent | Verdict | Shape | New candidate | |---|---|---|---|---|---|---| -| 1 | Knowledge harvest (`nagent-gc` → `~/.nagent/knowledge/`) | §6 Claim 4 (4 memory dimensions) | THIRD memory dimension: curation + discussion + RAG (opt-in) + knowledge | GAP | `o==>` (codecycle: harvest → digest → inject) | **11 (HIGH)** | -| 2 | Prompt caching w/ stable-to-volatile ordering | §6 Claim 5 | `_add_history_cache_breakpoint` + `cache_control` blocks (mechanism present, ordering not enforced); no cache TTL GUI | PARTIAL | `===>M===>` (merge at volatile boundary) | **12a (MED)** | -| 2b | Cache TTL GUI controls | (NEW; no survey claim) | Anthropic ephemeral 5min + Gemini explicit 1h, no GUI | GAP (UX) | `===>W===>` (per-provider control surface) | **12b (MED)** | -| 3 | Conversation compaction (`--compact`) | (NEW; no survey claim) | `run_discussion_compression` = summarize, not compact | GAP | `===>B===>` (try/recover envelope) | **13 (MED)** | +| 1 | Knowledge harvest (`nagent-gc` → `~/.nagent/knowledge/`) | §6 Claim 4 (4 memory dimensions) | THIRD memory dimension: curation + discussion + RAG (opt-in) + knowledge | GAP | `o->` (codecycle: harvest → digest → inject) | **11 (HIGH)** | +| 2 | Prompt caching w/ stable-to-volatile ordering | §6 Claim 5 | `_add_history_cache_breakpoint` + `cache_control` blocks (mechanism present, ordering not enforced); no cache TTL GUI | PARTIAL | `->M->` (merge at volatile boundary) | **12a (MED)** | +| 2b | Cache TTL GUI controls | (NEW; no survey claim) | Anthropic ephemeral 5min + Gemini explicit 1h, no GUI | GAP (UX) | `=>` (per-provider control surface) | **12b (MED)** | +| 3 | Conversation compaction (`--compact`) | (NEW; no survey claim) | `run_discussion_compression` = summarize, not compact | GAP | `->B->` (try/recover envelope) | **13 (MED)** | | 4 | Project context files (`context.yaml`) | (no survey claim) | `manual_slop.toml` per-project (TOML ≠ YAML) | PARITY-DIFFERENT-MECHANISM | `[I]` | 14 (LOW) | | 5 | claude-code provider (5th provider, sub. auth) | (no survey claim) | `_send_gemini_cli` (parallel pattern) | PARITY | `[I]` | none (provider add) | | 6 | Per-file knowledge notes (`knowledge/files/{file_id}.md`) | §6 Claim 4 (knowledge dim) | `FileItem.notes` absent | GAP | `[I]` | 11.1 (bundle) | | 7 | "Delete to turn off" feature flags | (no survey claim) | `[ai_settings.toml]` toggles | PARITY-DIFFERENT-MECHANISM | `[I]` | none (styleguide) | -| 8 | Save-with-graceful-summary-failure | §6 Claim 6 (Result[T] envelope) | `run_discussion_compression` failure mode **TBD** | UNKNOWN | `===>B===>` | 15 (TBD) | +| 8 | Save-with-graceful-summary-failure | §6 Claim 6 (Result[T] envelope) | `run_discussion_compression` failure mode **TBD** | UNKNOWN | `->B->` | 15 (TBD) | | 9 | AGENTS.md `@import` pattern | §6 Claim 1 (Domain = Meta-Tooling) | `AGENTS.md` exists, no canonical rules file | GAP | `[I]` | **16 (HIGH)** | -| 10 | Delegation = context mgmt, not parallelism | §1 Claim 3 (immediate-mode) | MMA subprocess + Context Amnesia (already does this) | PARITY (new framing) | `===>W===>` | none (styleguide) | +| 10 | Delegation = context mgmt, not parallelism | §1 Claim 3 (immediate-mode) | MMA subprocess + Context Amnesia (already does this) | PARITY (new framing) | `=>` | none (styleguide) | | 11 | RAG integration discipline | (NEW; no survey claim) | `src/rag_engine.py` opt-in; no codified discipline | GAP (doc) | `[I]` | styleguide | **Headline.** v2.1's 4 memory dimensions and stable-to-volatile cache ordering are now **formally codified** in the intent DSL survey as §6 Claims 4 and 5. The v2.1 review *was the seed* for those survey claims (the survey cites v2.1:50 and v2.1 §2.2 by name). The new styleguides + canonical DOD file proposed in v2.1 §4 should follow the survey's table format (Symbol, Name, Signature, Semantics, Example, Borrowed from, Shape). @@ -61,9 +61,9 @@ The 4 claims compose: a user expresses intent (Claim 1) using a vocabulary verb | Dim | Where it lives | What it stores | How edited | How queried | SSDL shape | |---|---|---|---|---|---| | Curation | `FileItem` + `ContextPreset` + Fuzzy Anchors | *How to render a file* in the AI's context window | Structural File Editor; project TOML | `aggregate.py:run` at discussion start | `[Q]` | -| Discussion | `app.disc_entries` + branching + UISnapshot | *What was said* in the conversation | GUI `[Edit]` mode; `[Branch]`; undo/redo | `build_markdown` renders as prior context | `o==>` (entries accumulate) | +| Discussion | `app.disc_entries` + branching + UISnapshot | *What was said* in the conversation | GUI `[Edit]` mode; `[Branch]`; undo/redo | `build_markdown` renders as prior context | `o->` (entries accumulate) | | RAG | `src/rag_engine.py` (ChromaDB) | *Semantic fingerprints* of indexed files | (opaque vector store) | `RAGEngine.search()` at LLM call time | `[Q]` (vector similarity) | -| Knowledge (proposed) | `~/.manual_slop/knowledge/{facts,decisions,questions,playbooks}.md` + `knowledge/files/{file_id}.md` + `knowledge/digest.md` + `knowledge/ledger.json` | *Durable learnings* harvested from past sessions | Plain markdown edit | Bounded digest injected as stable prefix | `o==>` (harvest → digest → inject) | +| Knowledge (proposed) | `~/.manual_slop/knowledge/{facts,decisions,questions,playbooks}.md` + `knowledge/files/{file_id}.md` + `knowledge/digest.md` + `knowledge/ledger.json` | *Durable learnings* harvested from past sessions | Plain markdown edit | Bounded digest injected as stable prefix | `o->` (harvest → digest → inject) | **The harvest output schema** (was JSON in v2.1 §2.1; now a table per the user's style): @@ -196,7 +196,7 @@ The proposed GUI surface (a "Caching" tab in Operations Hub): | nagent v2 | Manual Slop today | Verdict | Shape | Source | |---|---|---|---|---| -| `--compact` (rewrites conversation preserving structure) | `run_discussion_compression` (summarizes; loses structure) | GAP | `===>B===>` (try/recover envelope) | `bin/nagent:1975-2019` + `prompts/compact-conversation.md` | +| `--compact` (rewrites conversation preserving structure) | `run_discussion_compression` (summarizes; loses structure) | GAP | `->B->` (try/recover envelope) | `bin/nagent:1975-2019` + `prompts/compact-conversation.md` | **The compaction prompt's self-review checklist** (10 yes/no questions, abbreviated): @@ -253,7 +253,7 @@ Source: `bin/helpers/nagent_gc_lib.py:regenerate_digest` deletes the file when s | nagent v2 | Manual Slop today | Verdict | Shape | |---|---|---|---| -| Save indexes the copy even when summary LLM fails; `(summary unavailable)` marker | `run_discussion_compression` failure mode **TBD** | UNKNOWN | `===>B===>` | +| Save indexes the copy even when summary LLM fails; `(summary unavailable)` marker | `run_discussion_compression` failure mode **TBD** | UNKNOWN | `->B->` | Cross-reference to intent DSL survey §6 Claim 6: `Result[T]` envelope with side-channel errors. The graceful-failure pattern is the *concrete instance* of the `Result[T]` convention. @@ -261,7 +261,7 @@ Cross-reference to intent DSL survey §6 Claim 6: `Result[T]` envelope with side | nagent v2 | Manual Slop today | Verdict | Shape | |---|---|---|---| -| "Hand off when noisy: distill goal/state/decisions into a sub-conversation prompt, delegate the rest" | MMA worker pool (subprocess + Context Amnesia) | PARITY (new framing) | `===>W===>` (wide: parent + child codepaths) | +| "Hand off when noisy: distill goal/state/decisions into a sub-conversation prompt, delegate the rest" | MMA worker pool (subprocess + Context Amnesia) | PARITY (new framing) | `=>` (wide: parent + child codepaths) | Source: `bin/nagent:730` (the "Hand off when noisy" line in `build_initial_context`). Update `docs/guide_mma.md` with the new framing; matches intent DSL survey §1 Claim 3 (immediate-mode). @@ -291,8 +291,8 @@ Each candidate row: Symbol-like ID, Name, Domain, Priority, Effort, Shape, Depen | # | Name | Domain | Priority | Effort | Shape | Depends on | Cross-refs | |---|---|---|---|---|---|---|---| -| 1 | `SubConversationRunner` (1:1 sub-convos) | App + MT | HIGH | Med | `===>W===>` | none | survey §6 Claim 1 | -| 2 | RAG pre-staging via sub-convo | App | MED (down) | Sm | `o==>` | 1 | survey §6 Claim 4 (knowledge dim) | +| 1 | `SubConversationRunner` (1:1 sub-convos) | App + MT | HIGH | Med | `=>` | none | survey §6 Claim 1 | +| 2 | RAG pre-staging via sub-convo | App | MED (down) | Sm | `o->` | 1 | survey §6 Claim 4 (knowledge dim) | | 3 | Stateless `LLMClient` class | App | MED | Lg | `[I]` | none | survey §1 Claim 3 (immediate-mode) | | 4 | Intent DSL for Meta-Tooling | MT | LOW | research | `[I]` | none | **`intent_dsl_survey_20260612` is the substantiation** | | 5 | Self-describing MCP tools | BOTH | LOW (subsumed) | Med | `[I]` | `mcp_architecture_refactor` | survey Cluster 4 (Metadesk) | @@ -301,12 +301,12 @@ Each candidate row: Symbol-like ID, Name, Domain, Priority, Effort, Shape, Depen | 8 | `py_/ts_c_coedited_files` tools | App | LOW | Sm | `[I]` | 6 | (no survey claim) | | 9 | Explicit `split_lib.py` / `patch_lib.py` | App | DEFER | Med | `[I]` | none | (no survey claim) | | 10 | Raw-transcript persistence per Take | App | LOW | Sm | `[I]` | 3 | (no survey claim) | -| **11** | **Knowledge memory (third dimension)** | App | **HIGH** | Lg | `o==>` | `data_oriented_error_handling` | **survey §6 Claim 4** | -| **12a** | Stable-to-volatile cache ordering | App | MED | Sm | `===>M===>` | none | **survey §6 Claim 5** | -| **12b** | Cache TTL GUI controls | App | MED | Med | `===>W===>` | 12a | (no survey claim) | -| **13** | Conversation compaction | App | MED | Sm | `===>B===>` | none | survey §6 Claim 6 (Result envelope) | +| **11** | **Knowledge memory (third dimension)** | App | **HIGH** | Lg | `o->` | `data_oriented_error_handling` | **survey §6 Claim 4** | +| **12a** | Stable-to-volatile cache ordering | App | MED | Sm | `->M->` | none | **survey §6 Claim 5** | +| **12b** | Cache TTL GUI controls | App | MED | Med | `=>` | 12a | (no survey claim) | +| **13** | Conversation compaction | App | MED | Sm | `->B->` | none | survey §6 Claim 6 (Result envelope) | | **14** | Project context file | App | LOW | Sm | `[I]` | none | (no survey claim) | -| **15** | Save-with-graceful-summary-failure | App | TBD | Sm | `===>B===>` | none | survey §6 Claim 6 | +| **15** | Save-with-graceful-summary-failure | App | TBD | Sm | `->B->` | none | survey §6 Claim 6 | | **16** | **AGENTS.md `@import` + canonical DOD** | BOTH | **HIGH** | Sm | `[I]` | none | survey §1 (the 4 anchor claims) | **Net effect.** 16 candidates (up from 10 in v1; up from 11 in v2; up from 12 in v2.1). 4 are HIGH priority (1, 11, 16; + SubConversationRunner). 5 are MEDIUM (2, 3, 6, 12a, 12b, 13). 3 are subsumed/deferred (4, 5, 9). 4 are LOW (7, 8, 10, 14). 1 is TBD (15). @@ -318,15 +318,15 @@ Each candidate row: Symbol-like ID, Name, Domain, Priority, Effort, Shape, Depen | # | nagent v2 pattern | Survey cross-ref | Manual Slop equivalent | Verdict | Shape | |---|---|---|---|---|---| | 1-5, 10, 12-14 | (v1 rows, mostly unchanged) | ... | ... | ... | ... | -| 3 (editable state) | `--compact`, `--branch-conversation`, editable compaction prompt | §3.5 (try/recover envelope) | Take/branching + per-entry edit + UISnapshot; summarize, not compact | PARITY (DIFF FOCUS) on editing; GAP on compaction | `===>B===>` | +| 3 (editable state) | `--compact`, `--branch-conversation`, editable compaction prompt | §3.5 (try/recover envelope) | Take/branching + per-entry edit + UISnapshot; summarize, not compact | PARITY (DIFF FOCUS) on editing; GAP on compaction | `->B->` | | 6 (per-file memory) | per-file conversation + per-file knowledge notes | §6 Claim 4 (per-file knowledge) | `FileItem` + `ContextPreset` + Fuzzy Anchors; no `notes` field | PARITY (DIFF KIND) on curation; GAP on notes | `[I]` | -| 9 (sub-convos) | `` worker reuse; "delegation is context management" | §1 Claim 3 (immediate-mode) | MMA worker pool (subprocess) + 1:1 gap; new framing in docs | PARITY for MMA; GAP for 1:1; design pattern update | `===>W===>` | -| **Knowledge harvest** (NEW) | `nagent-gc` → `~/.nagent/knowledge/` + provenance + sha256 ledger + digest | **§6 Claim 4** | **THIRD memory dimension** alongside curation + discussion; RAG is opt-in, not the comparison | **GAP (Application)** | `o==>` | -| **Prompt caching strategy** (NEW) | `bin/nagent:970-1014` boundaries; `nagent_llm.py:cache_prefix_blocks` | **§6 Claim 5** | `_add_history_cache_breakpoint`; ordering not enforced; no TTL GUI | **PARTIAL + GAP (UX)** | `===>M===>` | -| **Conversation compaction** (NEW) | `--compact` with editable `prompts/compact-conversation.md` | §3.5 (envelope) | `run_discussion_compression` (summarize, not compact) | **GAP** | `===>B===>` | +| 9 (sub-convos) | `` worker reuse; "delegation is context management" | §1 Claim 3 (immediate-mode) | MMA worker pool (subprocess) + 1:1 gap; new framing in docs | PARITY for MMA; GAP for 1:1; design pattern update | `=>` | +| **Knowledge harvest** (NEW) | `nagent-gc` → `~/.nagent/knowledge/` + provenance + sha256 ledger + digest | **§6 Claim 4** | **THIRD memory dimension** alongside curation + discussion; RAG is opt-in, not the comparison | **GAP (Application)** | `o->` | +| **Prompt caching strategy** (NEW) | `bin/nagent:970-1014` boundaries; `nagent_llm.py:cache_prefix_blocks` | **§6 Claim 5** | `_add_history_cache_breakpoint`; ordering not enforced; no TTL GUI | **PARTIAL + GAP (UX)** | `->M->` | +| **Conversation compaction** (NEW) | `--compact` with editable `prompts/compact-conversation.md` | §3.5 (envelope) | `run_discussion_compression` (summarize, not compact) | **GAP** | `->B->` | | **Project context files** (NEW) | `context.yaml` at git toplevel, install → project → root | (no survey claim) | `manual_slop.toml` per-project (TOML ≠ YAML) | **PARITY (DIFF MECH)** | `[I]` | | **AGENTS.md `@import`** (NEW) | nagent `CLAUDE.md` → `context/data-oriented-design.md` | **§1 (the 4 anchor claims)** | `AGENTS.md` exists, no canonical rules file; `./docs/AGENTS.md` not created | **GAP** | `[I]` | -| **Cache TTL exposure** (NEW) | (n/a — providers do this) | (no survey claim) | Anthropic 5min ephemeral + Gemini 1h explicit, no GUI | **GAP (UX)** | `===>W===>` | +| **Cache TTL exposure** (NEW) | (n/a — providers do this) | (no survey claim) | Anthropic 5min ephemeral + Gemini 1h explicit, no GUI | **GAP (UX)** | `=>` | | **RAG integration discipline** (NEW) | (n/a — nagent has no RAG) | (no survey claim) | `src/rag_engine.py` opt-in; no codified discipline | **GAP (doc)** | `[I]` | --- diff --git a/conductor/tracks/nagent_review_20260608/nagent_review_v2_3_20260612.md b/conductor/tracks/nagent_review_20260608/nagent_review_v2_3_20260612.md index 224c5314..b447e99c 100644 --- a/conductor/tracks/nagent_review_20260608/nagent_review_v2_3_20260612.md +++ b/conductor/tracks/nagent_review_20260608/nagent_review_v2_3_20260612.md @@ -7,7 +7,7 @@ > **Reading guide.** This is a self-contained review of the latest nagent corpus. The 14 patterns from the README (the teaching-arc structure: build → rename → own → exploit → name → apply → compare) are covered in §2, one section per pattern. The 8 new commits since 2026-06-08 are catalogued in §1.3 and each new pattern is deep-dived in §3. The harvest pipeline is in §4; the cache strategy in §5; the compaction pattern in §6. Architecture + protocol + file-ops + candidates + artifacts + next-steps + references round out §7-§13. > -> **Notation.** Tables (no JSON code blocks), SSDL shape tags (`[I]`, `===>`, `o==>`, `===>W===>`, `[B]`, `[M]`, `[N]`, `[Q]`, `[S]`, `[T]`, `───`), forth/array notation (`a b +` for postfix math), file:line citations into both the nagent source and Manual Slop source. Boxes (`+--+--+`, `|...|`) for visual structure. The 7-column "Symbol, Name, Signature, Semantics, Example, Source, Shape" layout is the canonical table format throughout. +> **Notation.** Tables (no JSON code blocks), SSDL shape tags (`[I]`, `->`, `o->`, `=>`, `[B]`, `[M]`, `[N]`, `[Q]`, `[S]`, `[T]`, `───`), forth/array notation (`a b +` for postfix math), file:line citations into both the nagent source and Manual Slop source. Boxes (`+--+--+`, `|...|`) for visual structure. The 7-column "Symbol, Name, Signature, Semantics, Example, Source, Shape" layout is the canonical table format throughout. > > **What this is not.** Not a delta on a prior review. Not a comparison table only. The 14 patterns + 12 new additions + 4 deep-dives + 16 future-track candidates + 15+ proposed artifacts are all here, in depth. @@ -24,23 +24,23 @@ The latest nagent corpus (post-2026-06-08) introduces **knowledge harvest** as a | # | Pattern | SSDL shape | Manual Slop equivalent | Verdict | New track? | |---|---|---|---|---|---| | 1 | Text in, text out | `[I]` | `ai_client.send()` | PARITY | none | -| 2 | Visible output protocol (8 tags) | `===>B===>` | (provider-native function calling) | ARCH-DIFF | none | -| 3 | The loop (append, call, parse, act) | `o==>` | 3 loops (LLM/MMA/App) | PARITY | none | +| 2 | Visible output protocol (8 tags) | `->B->` | (provider-native function calling) | ARCH-DIFF | none | +| 3 | The loop (append, call, parse, act) | `o->` | 3 loops (LLM/MMA/App) | PARITY | none | | 4 | Tool discovery (`--description`) | `[I]` | `mcp_client.dispatch` (45 tools) | GAP (auto) | (subsumed by `mcp_architecture_refactor`) | | 5 | You did not build an agent (data is the thing) | (philosophical) | (philosophical — Manual Slop agrees) | PARITY | none | | 6 | Conversations are editable state | `[I]` | `disc_entries` + branching + UISnapshot | PARITY-DIFF-FOCUS | none | | 7 | Repository history as data | `[I]` | `aggregate.py` diff injection; no git history | PARTIAL | 6 (MED) | -| 8 | Harvest knowledge, reclaim space | `o==>` | (absent) | **GAP (Application)** | **11 (HIGH)** | +| 8 | Harvest knowledge, reclaim space | `o->` | (absent) | **GAP (Application)** | **11 (HIGH)** | | 9 | Everything else files buy you | (lens) | (lens) | N/A | (no track) | | 10 | Data-oriented design (named principles) | (philosophical) | (philosophical — Manual Slop agrees) | PARITY | none | | 11 | Artifact neighborhoods | `[I]` | (no coedit tool) | GAP | 8 (LOW) | | 12 | Managing context + large files | `[I]` | `aggregate.py` + tree-sitter + per-file slices | PARITY-DIFF-MECH | 9 (DEFER) | | 13 | Per-file write conversations | `[I]` | (per-file discussion memory absent) | PARTIAL | 7 (LOW) | | 14 | Own the inputs (vs framework abstractions) | (lens) | (lens) | N/A | (no track) | -| **NEW** | **Knowledge harvest** (`nagent-gc`) | `o==>` | (absent) | **GAP** | **11 (HIGH)** | -| **NEW** | **Stable-to-volatile cache ordering** | `===>M===>` | (mechanism present, ordering not enforced) | **PARTIAL** | **12a (MED)** | -| **NEW** | **Cache TTL GUI controls** | `===>W===>` | (no GUI for TTL) | **GAP (UX)** | **12b (MED)** | -| **NEW** | **Conversation compaction** | `===>B===>` | (have summarize, not compact) | **GAP** | **13 (MED)** | +| **NEW** | **Knowledge harvest** (`nagent-gc`) | `o->` | (absent) | **GAP** | **11 (HIGH)** | +| **NEW** | **Stable-to-volatile cache ordering** | `->M->` | (mechanism present, ordering not enforced) | **PARTIAL** | **12a (MED)** | +| **NEW** | **Cache TTL GUI controls** | `=>` | (no GUI for TTL) | **GAP (UX)** | **12b (MED)** | +| **NEW** | **Conversation compaction** | `->B->` | (have summarize, not compact) | **GAP** | **13 (MED)** | | **NEW** | **Project context files** | `[I]` | `manual_slop.toml` (TOML ≠ YAML) | PARITY-DIFF-MECH | 14 (LOW) | | **NEW** | **claude-code provider** (5th, sub auth) | `[I]` | `_send_gemini_cli` (parallel) | PARITY | none (provider add) | | **NEW** | **AGENTS.md `@import` pattern** | `[I]` | `AGENTS.md` exists, no canonical file | **GAP** | **16 (HIGH)** | @@ -49,8 +49,8 @@ The latest nagent corpus (post-2026-06-08) introduces **knowledge harvest** as a | # | Candidate | Pri | Effort | Shape | |---|---|---|---|---| -| 1 | `SubConversationRunner` (1:1 sub-convos) | HIGH | Med | `===>W===>` | -| 2 | RAG pre-staging via sub-convo | MED (down) | Sm | `o==>` | +| 1 | `SubConversationRunner` (1:1 sub-convos) | HIGH | Med | `=>` | +| 2 | RAG pre-staging via sub-convo | MED (down) | Sm | `o->` | | 3 | Stateless `LLMClient` class | MED | Lg | `[I]` | | 4 | Intent DSL for Meta-Tooling | LOW | research | `[I]` | | 5 | Self-describing MCP tools | LOW (subsumed) | Med | `[I]` | @@ -59,12 +59,12 @@ The latest nagent corpus (post-2026-06-08) introduces **knowledge harvest** as a | 8 | `py_/ts_c_coedited_files` tools | LOW | Sm | `[I]` | | 9 | Explicit `split_lib.py` / `patch_lib.py` | DEFER | Med | `[I]` | | 10 | Raw-transcript persistence per Take | LOW | Sm | `[I]` | -| **11** | **Knowledge memory (3rd dimension)** | **HIGH** | Lg | `o==>` | -| **12a** | **Stable-to-volatile cache ordering** | MED | Sm | `===>M===>` | -| **12b** | **Cache TTL GUI controls** | MED | Med | `===>W===>` | -| **13** | **Conversation compaction** | MED | Sm | `===>B===>` | +| **11** | **Knowledge memory (3rd dimension)** | **HIGH** | Lg | `o->` | +| **12a** | **Stable-to-volatile cache ordering** | MED | Sm | `->M->` | +| **12b** | **Cache TTL GUI controls** | MED | Med | `=>` | +| **13** | **Conversation compaction** | MED | Sm | `->B->` | | **14** | Project context file | LOW | Sm | `[I]` | -| **15** | Save-with-graceful-summary-failure | TBD | Sm | `===>B===>` | +| **15** | Save-with-graceful-summary-failure | TBD | Sm | `->B->` | | **16** | **AGENTS.md `@import` + canonical DOD** | **HIGH** | Sm | `[I]` | ### 0.4 The proposed new artifacts (15+ files) @@ -322,7 +322,7 @@ The 4 substantive commits are #1, #2, #6, #4. Commits #3, #5, #7, #8 are compani **Verdict.** **ARCHITECTURAL DIFFERENCE** — the Application's choice is correct (parallel tool calls, JSON-mode constraints, native tool calling). The Meta-Tooling could legitimately use nagent's regex-tag protocol for its own work (per the `mcp_architecture_refactor_20260606` track's intent-based DSL placeholder, now substantively specced by the related `intent_dsl_survey_20260612` report). See §3.1 of that report (out of scope here). -**SSDL shape.** `===>B===>` (codepath with branch; the parser branches on tag name). +**SSDL shape.** `->B->` (codepath with branch; the parser branches on tag name). **Manual Slop next steps.** - The new `mcp_architecture_refactor_20260606` sub-MCP extraction (already in plan) is the natural place to consider a tag-style protocol for the Meta-Tooling DSL. @@ -382,7 +382,7 @@ loop: **Verdict.** **PARITY.** All three loops have the same shape (append, call, parse, act, repeat) but different data structures. nagent's `run_agent_loop` is ~50 lines, easy to reason about. Manual Slop's loops are 100-300 lines each, scattered. -**SSDL shape.** `o==>` (codecycle; the loop repeats). +**SSDL shape.** `o->` (codecycle; the loop repeats). **Manual Slop next steps.** - Candidate 3 (Stateless `LLMClient` class) would unify these three loops behind a single `run_loop(...)` function. Large refactor; not high priority. @@ -636,7 +636,7 @@ Step-by-step history: **Verdict.** **GAP (Application).** The "knowledge memory" dimension is missing. Curation (FileItem + ContextPreset) and discussion (disc_entries + branching + UISnapshot) are present and strong. RAG is opt-in and is the wrong shape for "what did we learn from past sessions." -**SSDL shape.** `o==>` (codecycle: scan → distill → append → digest → reclaim). +**SSDL shape.** `o->` (codecycle: scan → distill → append → digest → reclaim). **Manual Slop next steps.** - **Candidate 11: Knowledge memory (third dimension)** — HIGH priority. The single most important v2.3 finding. See §4 below for the deep-dive. @@ -1319,7 +1319,7 @@ What's NOT explicit: **Verdict.** **PARTIAL.** Mechanism present; ordering not enforced; TTL not exposed. -**SSDL shape.** `===>M===>` (merge at volatile boundary). +**SSDL shape.** `->M->` (merge at volatile boundary). **Manual Slop next steps.** - **Candidate 12a: Stable-to-volatile cache ordering** — refactor `src/ai_client.py:_get_combined_system_prompt` and the Anthropic call site to enforce the ordering. See §5 below. @@ -1375,7 +1375,7 @@ What's NOT explicit: **Verdict.** **GAP (UX).** The mechanism is in place for Anthropic + Gemini; the GUI exposure is missing. -**SSDL shape.** `===>W===>` (wide: per-provider control surface; the same code path branches into 5+ provider-specific config UIs). +**SSDL shape.** `=>` (wide: per-provider control surface; the same code path branches into 5+ provider-specific config UIs). **Manual Slop next steps.** - **Candidate 12b: Cache TTL GUI controls** — MEDIUM priority. See §5 below for the deep-dive. @@ -1593,7 +1593,7 @@ The 12-section output structure is NOT enforced. The failure mode is graceful (t **Verdict.** **GAP.** Manual Slop has summarization; it does not have behavior-preserving compaction. The 10-question self-review is a contract that the existing `run_discussion_compression` lacks. -**SSDL shape.** `===>B===>` (codepath with branch — the compaction is one of two paths: `try { compact } recover { audit_failure }`). +**SSDL shape.** `->B->` (codepath with branch — the compaction is one of two paths: `try { compact } recover { audit_failure }`). **Manual Slop next steps.** - **Candidate 13: Conversation compaction** — MEDIUM priority. See §6 below for the deep-dive. @@ -2167,7 +2167,7 @@ if knowledge_digest: **Verdict.** **UNKNOWN** without reading the source. The pattern is the same; the implementation detail is what matters. -**SSDL shape.** `===>B===>` (codepath with branch — the summary is one of two paths: `try { summarize } recover { mark_unavailable }`). +**SSDL shape.** `->B->` (codepath with branch — the summary is one of two paths: `try { summarize } recover { mark_unavailable }`). **Manual Slop next steps.** - **Candidate 15: Save-with-graceful-summary-failure** — TBD. The verification (read `src/ai_client.py:run_discussion_compression`) is cheap (one source read); the potential value is high (latent bug). @@ -2216,7 +2216,7 @@ But the 1:1 discussion path has no sub-agent primitive. The user types a prompt, **Verdict.** **PARITY for MMA; GAP for 1:1 discussions.** MMA does it right; 1:1 needs the primitive. -**SSDL shape.** `===>W===>` (wide codepath: parent + child codepaths run in parallel; the child is disposable; the parent gets a concise result). +**SSDL shape.** `=>` (wide codepath: parent + child codepaths run in parallel; the child is disposable; the parent gets a concise result). **Manual Slop next steps.** - **Candidate 1: `SubConversationRunner`** — HIGH priority. The MMA pattern as a reusable App-callable class. See §10. @@ -4124,19 +4124,19 @@ The full candidate list, in priority order, with full specifications. This secti | # | Symbol-like ID | Name | Domain | Pri | Effort | Shape | Depends on | |---|---|---|---|---|---|---|---| -| 1 | `SubConversationRunner` | 1:1 sub-convos | App + MT | HIGH | Med | `===>W===>` | none | -| 2 | `RAGPreStager` | RAG pre-staging via sub-convo | App | MED | Sm | `o==>` | 1 | +| 1 | `SubConversationRunner` | 1:1 sub-convos | App + MT | HIGH | Med | `=>` | none | +| 2 | `RAGPreStager` | RAG pre-staging via sub-convo | App | MED | Sm | `o->` | 1 | | 3 | `LLMClient` (stateless) | Stateless LLMClient class | App | MED | Lg | `[I]` | none | | 4 | `IntentDSL` | Intent DSL for Meta-Tooling | MT | LOW | research | `[I]` | none | | 5 | `SelfDescribingTools` | Self-describing MCP tools | BOTH | LOW (subsumed) | Med | `[I]` | `mcp_architecture_refactor` | | 6 | `GitHistory` | `src/git_history.py` (nagent §7) | App | MED | Med | `[I]` | none | | 7 | `PerFileConversation` | Per-file conversation log | App | LOW | Sm | `[I]` | 3 | -| **8** | `KnowledgeMemory` | **Knowledge memory (3rd dimension)** | **App** | **HIGH** | **Lg** | **`o==>`** | **`data_oriented_error_handling`** | -| **9** | `CacheOrdering` | **Stable-to-volatile cache ordering** | **App** | **MED** | **Sm** | **`===>M===>`** | **none** | -| **10** | `CacheTTL` | **Cache TTL GUI controls** | **App** | **MED** | **Med** | **`===>W===>`** | **9** | -| **11** | `Compaction` | **Conversation compaction** | **App** | **MED** | **Sm** | **`===>B===>`** | **none** | +| **8** | `KnowledgeMemory` | **Knowledge memory (3rd dimension)** | **App** | **HIGH** | **Lg** | **`o->`** | **`data_oriented_error_handling`** | +| **9** | `CacheOrdering` | **Stable-to-volatile cache ordering** | **App** | **MED** | **Sm** | **`->M->`** | **none** | +| **10** | `CacheTTL` | **Cache TTL GUI controls** | **App** | **MED** | **Med** | **`=>`** | **9** | +| **11** | `Compaction` | **Conversation compaction** | **App** | **MED** | **Sm** | **`->B->`** | **none** | | **12** | `ProjectContext` | Project context file | App | LOW | Sm | `[I]` | none | -| **13** | `GracefulSave` | Save-with-graceful-summary-failure | App | TBD | Sm | `===>B===>` | none | +| **13** | `GracefulSave` | Save-with-graceful-summary-failure | App | TBD | Sm | `->B->` | none | | **14** | `AGENTSImport` | **AGENTS.md `@import` + canonical DOD** | **BOTH** | **HIGH** | **Sm** | **`[I]`** | **none** | | 15 | `RawTranscript` | Raw-transcript persistence per Take | App | LOW | Sm | `[I]` | 3 | | 16 | `CoeditedFiles` | `py_/ts_c_coedited_files` tools | App | LOW | Sm | `[I]` | 6 | @@ -4200,9 +4200,9 @@ class SubConversationRunner: | Dim | Where | SSDL | Status | |---|---|---|---| | Curation | `FileItem` + `ContextPreset` + Fuzzy Anchors | `[Q]` | existing, strong | -| Discussion | `disc_entries` + branching + UISnapshot | `o==>` | existing, strong | +| Discussion | `disc_entries` + branching + UISnapshot | `o->` | existing, strong | | RAG | `src/rag_engine.py` (ChromaDB) | `[Q]` | opt-in | -| Knowledge (proposed) | `~/.manual_slop/knowledge/*.md` + per-file + digest + ledger | `o==>` | NEW | +| Knowledge (proposed) | `~/.manual_slop/knowledge/*.md` + per-file + digest + ledger | `o->` | NEW | **Where it lives.** Application. @@ -4600,7 +4600,7 @@ The 15+ new files proposed for the next turn, in the user's preferred data forma |---|---| | Format | The 7-column "Symbol, Name, Signature, Semantics, Example, Source, Shape" table where applicable | | No JSON | JSON code blocks become tables or line-based arrays | -| SSDL | Use `[I]`, `===>`, `o==>`, `===>W===>`, `===>M===>`, `===>B===>`, `[B]`, `[M]`, `[N]`, `[Q]`, `[S]`, `[T]`, `───` | +| SSDL | Use `[I]`, `->`, `o->`, `=>`, `->M->`, `->B->`, `[B]`, `[M]`, `[N]`, `[Q]`, `[S]`, `[T]`, `───` | | Forth/array notation | `a b +` for postfix math; `name := value` for assignment; `if cond { body }` for control flow | | Code blocks | With `───` data flow lines and `+--+` boxes for visual structure | | File:line citations | Both nagent source (`bin/nagent:606-745`) and Manual Slop source (`src/aggregate.py:run`) | diff --git a/conductor/tracks/nagent_review_20260608/state.toml b/conductor/tracks/nagent_review_20260608/state.toml index d49b7f49..07d1e2c9 100644 --- a/conductor/tracks/nagent_review_20260608/state.toml +++ b/conductor/tracks/nagent_review_20260608/state.toml @@ -83,7 +83,7 @@ t_v2_1_review_pending_08 = { status = "pending", commit_sha = "", description = # v2 and v2.1 are PRESERVED; v2.2 is non-destructive new file t_v2_2_review_01 = { status = "completed", commit_sha = "", description = "v2.2: read intent_dsl_survey_20260612/report_v1.2.md §1, §3, §4, §5, §6, §7 (~600 lines; 10 prior-art clusters, 4 anchor claims, ~42-verb vocab, 10 AI-Agent Properties)" } t_v2_2_review_02 = { status = "completed", commit_sha = "", description = "v2.2: applied the user's data-format preferences (table-based, forth/array-like, no JSON) — JSON blocks in v2.1 §2.1 replaced with §4.4 7-column tables (Symbol, Name, Signature, Semantics, Example, Borrowed from, Shape)" } -t_v2_2_review_03 = { status = "completed", commit_sha = "", description = "v2.2: adopted SSDL shape tags ([I], ===>, o==>, [B], [M], [N], [Q], [S], ───) for the comparison table and future-track candidate list" } +t_v2_2_review_03 = { status = "completed", commit_sha = "", description = "v2.2: adopted SSDL shape tags ([I], ->, o->, [B], [M], [N], [Q], [S], ───) for the comparison table and future-track candidate list" } t_v2_2_review_04 = { status = "completed", commit_sha = "", description = "v2.2: cross-referenced the intent DSL survey's 10 AI-Agent Properties (§6 Claims 1-10) — Claims 4 and 5 explicitly cite v2.1 §2.1 and §2.2 as their source" } t_v2_2_review_05 = { status = "completed", commit_sha = "", description = "v2.2: applied the survey's grammar primitives (name := value, for x .. n, if cond { ... }, tape { ... }, try { ... } recover err { ... }, sandbox { ... }, audit msg, fuzzy { ... }) where applicable" } t_v2_2_review_06 = { status = "completed", commit_sha = "", description = "v2.2: added new §11 'In dialogue with intent DSL survey' — the 9 mutual cross-references and the 3 the user should know about" } diff --git a/docs/reports/computational_shapes_ssdl_digest_20260608.md b/docs/reports/computational_shapes_ssdl_digest_20260608.md index cd4e94d8..1a2543f5 100644 --- a/docs/reports/computational_shapes_ssdl_digest_20260608.md +++ b/docs/reports/computational_shapes_ssdl_digest_20260608.md @@ -38,10 +38,10 @@ A *computation shape* is a high-level concept, not a physical thing. The diagram | # | Shape | One-line definition | SSDL symbol | |---|---|---|---| | 1 | **Instruction** | A single unit of computation. Reads data, writes data, or both. | `[I]` | -| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `===>` | -| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `===>W===>` (codepaths fan out) | -| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o==>` (arrow returns to start) | -| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `oo==>oo` (parallel cycles) | +| 2 | **Codepath** | A sequential list of instructions that *terminates*. No loops. | `->` | +| 3 | **Wide codepath** | A codepath whose execution *causes* several other codepaths to occur simultaneously. | `=>` (codepaths fan out) | +| 4 | **Codecycle** | A circular structure — a codepath that *repeats* at its first instruction after its last. | `o->` (iterator with path) | +| 5 | **Wide codecycle** | Multiple codecycles performing the same task simultaneously. | `o=>` (parallel cycles) | | 6 | **Codecycle graph** | Multiple codecycles + the data they read and write. | `boxes + arrows` | **Modifiers** (not shapes, but used to annotate them): @@ -59,19 +59,19 @@ A *computation shape* is a high-level concept, not a physical thing. The diagram **Legend**: ``` -[I] = single instruction -===> = codepath (linear, terminates at T) -===>W===> = wide codepath (causes parallel codepaths) -o==> = codecycle (loops back to start) -oo==>oo = wide codecycle (parallel codecycles doing the same task) -[T] = terminator (return/exit) -[B] = branch (if/else/switch) -[M] = merge (control flow reconverges) -[S] = state mutation -[Q] = state query -[N] = nil sentinel (defuses branches) -─── = data (read or write) -[•] = codepath that is *defused* (collapses to 1 effective codepath) +[I] = single instruction +-> = codepath (linear, terminates at T) +=> = wide codepath (causes parallel codepaths) +o-> = codecycle (loops back to start) +o=> = wide codecycle (parallel codecycles doing the same task) +[T] = terminator (return/exit) +[B] = branch (if/else/switch) +[M] = merge (control flow reconverges) +[S] = state mutation +[Q] = state query +[N] = nil sentinel (defuses branches) +─── = data (read or write) +[•] = codepath that is *defused* (collapses to 1 effective codepath) ``` --- @@ -242,7 +242,7 @@ USER CODE: SUBSYSTEM: [T] ``` -The user's code is now `===> [T]` (one straight line, one terminator). The subsystem absorbed the branches. **The number of *user-visible* effective codepaths went from 4 to 1.** The total number of codepaths in the program didn't decrease — but the *exposed surface area* did, and that's what matters for the caller's cognitive load, testing burden, and bug surface. +The user's code is now `-> [T]` (one straight line, one terminator). The subsystem absorbed the branches. **The number of *user-visible* effective codepaths went from 4 to 1.** The total number of codepaths in the program didn't decrease — but the *exposed surface area* did, and that's what matters for the caller's cognitive load, testing burden, and bug surface. #### Technique 4: Immediate-mode API (collapses "did I create/destroy this?" to "no, it's managed for me") @@ -478,8 +478,8 @@ The 6 primitives + 7 modifiers are enough to sketch any computational shape. The 1. **Top to bottom is time** (instructions happen in order, top first). 2. **`[B]` branches fan out, `[M]` merges reconverge** (control flow). 3. **`[N]` collapses a branch** (the branch exists in the subsystem but not in the user's codepath). -4. **`o==>` means "this is the main loop, it repeats forever"** (codecycle). -5. **`===>W===>` means "this codepath causes parallelism"** (wide). +4. **`o->` means "this is the main loop, it repeats forever"** (codecycle). +5. **`=>` means "this codepath causes parallelism"** (wide). 6. **A subsystem that returns a value valid in all cases** is a black box that the user never has to inspect. When sketching a feature, *start* with the user's codepath. If it has branches, the question is: "where does the branch live, in user code or in a subsystem?" If the answer is "in a subsystem," sketch the subsystem separately. If the answer is "in user code," *reconsider* — is there a way to push it into a subsystem? diff --git a/docs/reports/proposed_new_tracks_20260608.md b/docs/reports/proposed_new_tracks_20260608.md index df0fca17..aa6031b1 100644 --- a/docs/reports/proposed_new_tracks_20260608.md +++ b/docs/reports/proposed_new_tracks_20260608.md @@ -31,7 +31,7 @@ # Replace the existing "review UX" approach with the ASCII-sketch workflow # documented in docs/reports/ascii_sketch_ux_workflow_20260608.md method = "ASCII-sketch + MiniMax understand_image verification" -vocabulary = "[I], ===>, o==>, [B], [M], [S], [Q], [N], --" # 6 primitives + 7 modifiers +vocabulary = "[I], ->, o->, [B], [M], [S], [Q], [N], --" # 6 primitives + 7 modifiers first_target = "Discussion Hub per-entry panel" # gui_2.py:3770 source_of_truth = "docs/guide_discussions.md §Per-Entry Operations (A1-A7 matrix)" diff --git a/docs/reports/session_synthesis_20260608.md b/docs/reports/session_synthesis_20260608.md index ea42eabf..3f5b9e8e 100644 --- a/docs/reports/session_synthesis_20260608.md +++ b/docs/reports/session_synthesis_20260608.md @@ -222,7 +222,7 @@ Each got ~4 surgical spec edits + See Also cross-references. No plan/task change ### The 5 open questions the report surfaces -1. **Vocabulary preference** — the §2 vocabulary (`[I]`, `===>`, `o==>`, etc.) is a proposal. Alternatives: box-drawing characters (`┌─┐│└─┘`) for more ASCII-art look; Markdown tables for tabular content; hybrid (ASCII boxes for layout, tables for tabular data). +1. **Vocabulary preference** — the §2 vocabulary (`[I]`, `->`, `o->`, etc.) is a proposal. Alternatives: box-drawing characters (`┌─┐│└─┘`) for more ASCII-art look; Markdown tables for tabular content; hybrid (ASCII boxes for layout, tables for tabular data). 2. **Comparison policy** — after locking a design, do we always verify with `MiniMax understand_image` (slow but accurate)? Only when the design uses color/custom drawing? Only when the implementing Tier-3 reports a mismatch?