ed/pikuma_ps1
1
0
Fork 0
mirror of https://github.com/Ed94/pikuma_ps1.git synced 2026-07-20 15:36:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

First pass review

Browse Source
This commit is contained in:
Edward R. Gonzalez
2026-07-14 22:55:08 -04:00
parent 7d5b13aadb
commit 137549b1c8
17 changed files with 666 additions and 993 deletions
Download Patch File Download Diff File Expand all files Collapse all files
code/duffle/atom_dsl.h
+36 -129
View File
@@ -3,36 +3,20 @@
* ============================================================================
*
* ATOM DSL: Annotation layer for tape atoms (lottes_tape.h).
* The metaprogram (scripts/passes/annotation.lua) reads source-as-written and validates:
* - atom_info(...) shape: up to three sub-calls (atom_bind(Binds_X), atom_reads(...), atom_writes(...)) in any order and are optional.
* - rbind atoms (atom_info(..., atom_bind(Binds_X), ...)) reference a real Binds_* struct declaration.
* - wave-context positions only reference the 4-register set: R_PrimCursor / R_FaceCursor / R_VertBase / R_OtBase.
* Note(Ed): Not sure if I'll generlaize this later.
* - atom word-counts in word_counts.metadata.h match the body's actual .word count.
*
* WHAT THIS HEADER IS
* -------------------
* The metaprogram (scripts/passes/annotation.lua) reads source-as-written
* and validates:
* - atom_info(...) shape: up to three sub-calls (atom_bind(Binds_X),
* atom_reads(...), atom_writes(...)) in any order. All optional.
* (No phase token for now; phases may be reintroduced later.)
* - rbind atoms (atom_info(..., atom_bind(Binds_X), ...)) reference a
* real Binds_* struct declaration.
* - wave-context positions only reference the canonical 4-register
* set: R_PrimCursor / R_FaceCursor / R_VertBase / R_OtBase.
* - atom word-counts in word_counts.metadata.h agree with the body's
* actual .word count.
*
* WHY PURE MACROS
* ---------------
* atom_info, atom_bind, atom_reads, atom_writes, atom_label,
* atom_dbg_skip_over each expand to a C comment or to nothing. The C
* comment or to nothing. The C preprocessor strips them to whitespace.
* The metaprogram reads the literal tokens from source-as-written, NOT from
* the preprocessed output. This means:
* - the C compiler does no work for them (no __attribute__, no
* _Pragma, no asm side-effects)
* - they can never silently drift from the metaprogram's view
* (the metaprogram re-reads the source on every build)
* - the annotation is invisible to the linker, debugger, and IDE
* Pure macro anntation.
* ---------------
* Don't want to constraint the macro usage to some attribute placment constraint, etc, don't want ot dela with the compiler.
* atom_info, atom_bind, atom_reads, atom_writes, atom_label, atom_dbg_skip_over each expand to a C comment or to nothing
* (C preprocessor strips them to whitespace).
*
* ============================================================================
*
* Usage:
* MipsAtom_(cube_tri) atom_info(
* atom_reads (R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
@@ -62,30 +46,15 @@
*
* Annotation rules
* ----------------
* 1. atom_info(...) is OPTIONAL. Most atoms have no annotation.
* Atoms without atom_info are silently skipped by the metaprogram.
*
* 2. If present, atom_info takes up to three sub-calls, all
* order-independent within the arg list:
* - atom_bind(Binds_X) (optional; only for rbind atoms)
* - atom_reads(...) (optional; wave-context registers)
* - atom_writes(...) (optional; wave-context registers)
*
* 3. atom_bind(Binds_X) pins the ABI-struct shape -- the metaprogram
* cross-references Binds_X against the
* `typedef struct Binds_X { ... } Binds_X;` declaration.
*
* 4. atom_reads(...) and atom_writes(...) args are wave-context
* registers: R_PrimCursor / R_FaceCursor / R_VertBase / R_OtBase.
* Closed set. GTE / SP / DMA / I/O state is declared in source
* comments, not in atom_reads/atom_writes.
*
* 5. atom_label(name) is an anchor -- the macro is empty in C; the
* metaprogram records the marker at the current pos for offset
* calculation.
*
* 6. atom_offset(F, T) is resolved by gen/atom_offsets.h, generated
* from the atom_label markers.
* 1. atom_info(...) is OPTIONAL. Atoms without atom_info are silently skipped by the metaprogram.
* 2. If present, atom_info takes up to three sub-calls, all order-independent within the arg list:
* - atom_bind(Binds_X) (binds a struct context to an atom).
* - atom_reads(...) (context registers)
* - atom_writes(...) (context registers)
* 3. atom_bind(Binds_X): metaprogram cross-references Binds_X against the `typedef struct Binds_X { ... } Binds_X;` declaration.
* 4. atom_reads(...) and atom_writes(...) used to to check if registers are used correctly in macros: R_PrimCursor / R_FaceCursor / R_VertBase / R_OtBase.
* 5. atom_label(name) utilize with atom_offset as a target location.
* 6. atom_offset(F, T) is resolved by gen/atom_offsets.h, generated from the atom_label markers. Calculated during the offset pass of the lua metaprogram.
*/
#ifdef INTELLISENSE_DIRECTIVES
@@ -94,75 +63,32 @@
#endif
/* ============================================================================
* WAVE-CONTEXT REGISTERS -- canonical register set for the tape wave model.
*
* R_PrimCursor output pointer into the prim arena (next OT entry to write)
* R_FaceCursor input pointer into the face array (next face to consume)
* R_VertBase base pointer into the vertex arena (this wave's vertices)
* R_OtBase base pointer into the ordering table (this wave's OT slot)
*
* Closed set. If your atom needs to touch GTE / SP / DMA / other side state,
* declare it at the source level as you normally would -- but DO NOT put
* those registers in atom_reads/atom_writes.
*
* ============================================================================*/
/* ============================================================================
* atom_reads(...) / atom_writes(...) -- wave-context register list
*
* atom_reads(R_PrimCursor, R_FaceCursor)
* -> (R_PrimCursor, R_FaceCursor) // comma-evaluated, discarded
*
* The macro produces a comma-evaluated expression that the C compiler
* silently discards (it sits in an unused arg position -- the result is
* never bound). The Lua tool pattern-matches the "atom_reads(...)" /
* "atom_writes(...)" token to extract the list.
*
* You can have at most one atom_reads(...) and at most one atom_writes(...)
* in an atom_info(...) call. To declare multiple disjoint sets (rare), just
* declare the union -- the metaprogram doesn't track which reads need which
* writes at this granularity.
* atom_reads(...) / atom_writes(...)
*
* Used during the static analysis pass of the metaprogram to do
* ============================================================================*/
#define atom_reads(...) (__VA_ARGS__)
#define atom_writes(...) (__VA_ARGS__)
/* ============================================================================
* ATOM ANNOTATION MACROS
*
* atom_info -- single unified annotation. OPTIONAL. Most atoms have none.
*
* atom_info :
* MipsAtom_(cube_tri) atom_info(
* atom_reads (R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
* , atom_writes(R_PrimCursor, R_FaceCursor)
* ){ ... };
*
* Shape (sub-args order-independent; all optional):
* - atom_bind(Binds_X): at most one; pins the ABI-struct shape
* - atom_reads(...): at most one; comma-list of wave-context registers
* - atom_writes(...): at most one; comma-list of wave-context registers
*
* No phase token for now. The metaprogram doesn't check ordering across
* atoms -- phases (init / bind / setup / work / commit / terminate) will
* be reintroduced when ordering checks are added.
*
* The macro expands to a C comment (or to nothing). The C compiler does
* no work. The metaprogram reads the source-as-written directly.
*
*
* - atom_bind(Binds_X): metaprogram cross-references Binds_X against the `typedef struct Binds_X { ... } Binds_X;` declaration.
* - atom_reads(...): comma-list of registers
* - atom_writes(...): comma-list of registers
* ============================================================================*/
#define atom_info(...) /* atom_info(__VA_ARGS__) */
/* ----------------------------------------------------------------------------
* DEBUG SOURCE-STEP MARKERS
*
* Place atom_dbg_skip_over() before a MipsAtom_, MipsAtomComp_, or
* MipsAtomComp_Proc_ declaration. The following declaration kind determines
* whether the marker selects a whole atom or a component inline view. The
* source scanner associates the marker with that declaration; placement
* diagnostics are handled by the annotation pass.
*
* The macro expands to a comment only. It emits no C data or MIPS words and
* therefore cannot affect runtime output.
* Place atom_dbg_skip_over() before a MipsAtom_, MipsAtomComp_, or MipsAtomComp_Proc_.
* The following declaration kind determines whether the marker selects a whole atom or a component inline view.
* The source scanner associates the marker with that declaration; placement diagnostics are handled by the annotation pass.
* ----------------------------------------------------------------------------*/
#define atom_dbg_skip_over() /* atom_dbg_skip_over: skip the following atom or component source view */
@@ -174,13 +100,7 @@
* , atom_writes(R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
* ){ ... };
*
* The Binds_X MUST be a typedef'd type (declared via
* `typedef struct Binds_X { ... } Binds_X;` somewhere in the source).
* The Lua tool cross-references this. Missing struct = error.
*
* atom_bind is a SUB-CALL of atom_info, not a standalone annotation macro.
*
* The macro expands to a C comment. The metaprogram reads source-as-written.
* The Binds_X MUST be a typedef'd type (declared via `typedef struct Binds_X { ... } Binds_X;` somewhere in the source).
* ----------------------------------------------------------------------------*/
#define atom_bind(binds_struct) /* atom_bind(binds_struct) */
@@ -193,25 +113,12 @@
*
* atom_offset(culling, bounds_chk) ← resolved by gen/.offsets.h
*
* The metaprogram generates gen/atom_offsets.h with one
* #define atom_offset__culling__bounds_chk ((target - branch_pos - 1))
* per atom_offset(F, T) call. The preprocessor then expands your call to
* the right immediate value.
*
* If gen/atom_offsets.h is stale (or atom_label(name) is undefined),
* `atom_offset__F__T` becomes an undefined macro and the C build fails.
* This catches:
* - typo in atom_label (no anchor → metaprogram doesn't emit the macro)
* - .offsets.h not regenerated after body edits
* - body edit that broke the offset math (recompile + retest picks it up
* in CPU emulator)
* The metaprogram generates gen/atom_offsets.h with one #define with the offset value per atom_offset(F, T) call.
* The preprocessor then expands the call to the right immediate value.
*
* If gen/atom_offsets.h is stale (or atom_label(name) is undefined), `atom_offset_F_T`
* becomes an undefined macro and the C build fails.
* ============================================================================*/
#define atom_offset(F, T) atom_offset_ ## F ## _ ## T
/* atom_label is a pure annotation for the metaprogram's offset calculations.
* The macro expands to a C comment, so the C preprocessor strips it to
* whitespace — NO instruction word is emitted in the asm. The metaprogram
* still recognises the literal `atom_label(name)` token in source and
* records the marker at the current pos. */
// atom_label is a pure annotation for the metaprogram's offset calculations.
#define atom_label(name) /* atom_label anchor: name */
code/duffle/lottes_tape.h
+1 -2
View File
@@ -168,8 +168,7 @@ MipsAtomComp_Proc_(ac_pack_color_word, {
})
/* Words: 3; Emits the F3 command+color word (cmd byte | BLUE | GREEN | RED)
* Args: _r, _g, _b are 8-bit RGB byte values (not raw 16-bit fields).
* Migrated from hello_gte_tape.c; takes RGB form per the Phase 3 convention. */
* Args: _r, _g, _b are 8-bit RGB byte values (not raw 16-bit fields). */
FI_ MipsAtom ac_format_f3_color(U1 r, U1 g, U1 b)
MipsAtomComp_Proc_(ac_format_f3_color, { mac_pack_color_word(O_(Poly_F3,color), gp0_cmd_poly_f3, r, g, b) })
scripts/build_psyq.ps1
+3 -1
View File
@@ -485,7 +485,9 @@ function build-gte_hello {
& $Objcopy --add-section ".debug_loc=$dwarfLocBin" $injectElf
if ($LASTEXITCODE -ne 0) {
Write-Warning "[build] objcopy .debug_loc add-section failed (exit $LASTEXITCODE)"
} else {
}
else
{
# .debug_loclists doesn't exist in the source ELF; --add-section creates it.
& $Objcopy --add-section ".debug_loclists=$dwarfLoclistsBin" $injectElf
if ($LASTEXITCODE -ne 0) {
scripts/duffle.lua
+15 -26
View File
@@ -261,11 +261,8 @@ function M.write_file_lf(path, content)
f:write(content); f:close()
end
-- Return `{path, ...}` for files in `out_root` whose basename matches
-- `pattern` (Lua pattern, NOT regex — `%.` not `\.`). Empty list if
-- `out_root` doesn't exist or matches nothing.
--
-- **Cost:** ~2ms native (lfs.dir) vs ~56ms subprocess (`dir /b`).
-- Return `{path, ...}` for files in `out_root` whose basename matches `pattern` (Lua pattern, NOT regex — `%.` not `\.`).
-- Empty list if `out_root` doesn't exist or matches nothing.
-- @param out_root Path
-- @param pattern string -- Lua pattern matched against basename only
-- @return string[]
@@ -309,9 +306,6 @@ function M.to_absolute_path(path)
end
-- Cache of directories already verified to exist in this process.
-- Each ensure_dir() call may otherwise spawn a `cmd.exe mkdir` (50-100ms per call on Windows) — calling it inside per-source loops added 1.5+
-- seconds to the report pass. Cache makes ensure_dir idempotent within the process lifetime.
-- (safe across passes; the dir state doesn't change).
local _ensured_dirs = {}
function M.ensure_dir(path)
@@ -445,7 +439,7 @@ end
--
-- FIX (2026-07-09): split at top-level NEWLINES and SEMICOLONS too, AND emit a token break after a top-level comment/string.
-- Previous behavior glued the macro call after a comment into the same token, so `word_count_of_token` only saw the
-- leading ident (often nil after stripping the comment), undercounting the body. See Phase 1 of the branch-offset regression investigation.
-- leading ident (often nil after stripping the comment), undercounting the body.
-- Pure-comment / pure-string chunks (which now appear between real statements) are filtered out so they contribute 0 words instead of 1.
function M.split_top_level_commas(body)
local tokens = {}
@@ -537,9 +531,6 @@ end
-- Section 4b: tokenize_body + build_body_line_index (shared, memoized)
-- ════════════════════════════════════════════════════════════════════════════
-- Moved here from passes/static_analysis.lua so all passes can share the memoized
-- per-body tokenization. The memoization key is the body string (immutable per pass).
local _tokenize_body_cache = {}
local _tokenize_body_simple_cache = {}
local _body_line_index_cache = {}
@@ -562,20 +553,17 @@ function M.tokenize_body(body)
local scan = rel
while scan <= len do
local c = body:byte(scan)
-- Terminator bytes (delimit a token at the top level): ',' = 0x2C,
-- '\n' = 0x0A, ';' = 0x3B. These also appear as separators between
-- argument lists inside the parens/braces/brackets, so we stop the
-- scan when we hit any of them.
-- Terminator bytes (delimit a token at the top level): ',' = 0x2C, '\n' = 0x0A, ';' = 0x3B.
-- These also appear as separators between argument lists inside the parens/braces/brackets,
-- so we stop the scan when we hit any of them.
if c == BYTE_COMMA then break end
if c == BYTE_NEWLINE then break end
if c == BYTE_SEMI then break end
-- Group opener bytes (consume the balanced group via the matching reader):
-- '(' = 0x28, '{' = 0x7B, '[' = 0x5B.
-- Group opener bytes (consume the balanced group via the matching reader): '(' = 0x28, '{' = 0x7B, '[' = 0x5B.
if c == BYTE_OPEN_PAREN then local _, a = M.read_parens (body, scan); scan = a
elseif c == BYTE_OPEN_BRACE then local _, a = M.read_braces (body, scan); scan = a
elseif c == BYTE_OPEN_BRACK then local _, a = M.read_brackets (body, scan); scan = a
-- String-literal byte ('"' = 0x22 or '\'' = 0x27): skip past the
-- quoted region in one shot.
-- String-literal byte ('"' = 0x22 or '\'' = 0x27): skip past the quoted region in one shot.
elseif c == BYTE_DQUOTE or c == BYTE_SQUOTE then
scan = M.skip_str_or_cmt(body, scan) + 1
else
@@ -731,12 +719,13 @@ M.TAPE_ATOM_MACROS = {
-- The check (`scripts/passes/static_analysis.lua :: check_gte_pipeline_fill`) walks each atom body,
-- counts the consecutive nop words before every `gte_cmdw_*` invocation, and reports a finding if the count is below this minimum.
--
-- PRE-FILL vs POST-FILL: this table models PRE-cmdw nops (retiring preceding C2 writes), NOT the post-cmdw input-latch
-- window. The PSX-SPX pipeline timings doc (`docs/psx-spx/docs/gtepipelinetimings.md`) measures a DIFFERENT number:
-- the smallest N nops between `cop2` and `mtc2` to a specific input register at which the write no longer affects
-- the output. For nearly all instructions, inputs latch in the first 0-4 cycles — the GTE snapshots its input
-- register file early and works from internal pipeline storage afterward. The documented total cycle count is
-- NOT the "do not touch inputs" window; the actual read window is much shorter.
-- PRE-FILL vs POST-FILL: this table models PRE-cmdw nops (retiring preceding C2 writes),
-- NOT the post-cmdw input-latch window.
-- The PSX-SPX pipeline timings doc (`docs/psx-spx/docs/gtepipelinetimings.md`) measures a DIFFERENT number:
-- the smallest N nops between `cop2` and `mtc2` to a specific input register at which the write no longer affects the output.
-- For nearly all instructions, inputs latch in the first 0-4 cycles — the GTE snapshots its input register file early and works
-- from internal pipeline storage afterward. The documented total cycle count is NOT the "do not touch inputs" window;
-- the actual read window is much shorter.
--
-- The `gte_rtpt()` / `gte_nclip()` wrapper macros in gte.h emit the pre-cmd nops internally (asm_words(nop, nop, ...)),
-- but THOSE WRAPPERS ARE NOT USED INSIDE ATOM BODIES in this codebase.
scripts/elf_dwarf.lua
+101 -133
View File
@@ -14,9 +14,8 @@
-- Native dependencies
-- ════════════════════════════════════════════════════════════════════════════
-- lfs is wired into package.cpath by `duffle_paths.lua` (vendored under
-- `toolchain/lfs/lfs.dll`). Required here for native directory ops
-- (replaces the ~56ms `dir /b` subprocess with ~2ms native).
-- lfs is wired into package.cpath by `duffle_paths.lua` (vendored under `toolchain/lfs/lfs.dll`).
-- Required here for native directory ops (replaces the ~56ms `dir /b` subprocess with ~2ms native).
local lfs = require("lfs")
local M = {}
@@ -24,8 +23,7 @@ local M = {}
-- ════════════════════════════════════════════════════════════════════════════
-- DWARF tag + form constants
-- ════════════════════════════════════════════
-- (DWARF5 §7.5.5 "Tag Encodings" + Table 7.1; gcc emits these exact values for
-- the DWARF3-extension and DWARF5 line units.)
-- (DWARF5 §7.5.5 "Tag Encodings" + Table 7.1; gcc emits these exact values for the DWARF3-extension and DWARF5 line units.)
M.DW_TAG = {
compile_unit = 0x11,
@@ -80,13 +78,13 @@ M.DW_FORM = {
}
M.DW_ATE = {
address = 0x01,
boolean = 0x02,
address = 0x01,
boolean = 0x02,
complex_float = 0x03,
float = 0x04,
signed = 0x05,
signed_char = 0x06,
unsigned = 0x07,
float = 0x04,
signed = 0x05,
signed_char = 0x06,
unsigned = 0x07,
unsigned_char = 0x08,
}
@@ -107,13 +105,12 @@ M.MIPS_BYTES_PER_WORD = 0x04
-- ----------------------------------------------------------------------------
-- ELF32 (System V ABI gABI v1.2)
-- ----------------------------------------------------------------------------
--
-- All offsets are 1-INDEXED (matching Lua string.sub convention),
-- expressed in hex so they map directly to the wire-format byte positions in the binary file.
-- To compute the 0-indexed file offset, subtract 1.
--
-- Example: e_shoff_offset = 0x21 means the 4-byte e_shoff field
-- starts at string.sub byte 0x21 (= 33 in 1-indexed), i.e. file offset 0x20 (= 32).
-- Example: e_shoff_offset = 0x21 means the 4-byte e_shoff field starts at
-- string.sub byte 0x21 (= 33 in 1-indexed), i.e. file offset 0x20 (= 32).
--- spec: System V ABI gABI v1.2 §"ELF Header" (Table 1) + §"Section Header Table"
M.ELF32 = {
@@ -139,7 +136,6 @@ M.ELF32 = {
-- ----------------------------------------------------------------------------
-- DWARF4 .debug_aranges (per DWARF5 spec §7.4 — Address Range Table)
-- ----------------------------------------------------------------------------
--
-- All offsets are 1-INDEXED (matching Lua string.sub convention), in hex.
--- spec: DWARF5 spec §7.4 (Address Range Table) — 32-bit DWARF form
@@ -159,7 +155,6 @@ M.DWARF4_ARANGES = {
-- ----------------------------------------------------------------------------
-- DWARF5 .debug_rnglists (per DWARF5 spec §2.17 + §7.21)
-- ----------------------------------------------------------------------------
--
-- All offsets are 1-INDEXED (matching Lua string.sub convention), in hex.
--- spec: DWARF5 spec §2.17 + §7.21 (Range List Table) — 32-bit DWARF form
@@ -181,7 +176,6 @@ M.DWARF5_RNGLISTS = {
-- ----------------------------------------------------------------------------
-- DWARF line-program opcodes (per DWARF5 spec §6.2.5)
-- ----------------------------------------------------------------------------
--
-- Opcode VALUES stay in decimal — they're identifiers (DW_LNS_copy = 1), not binary positions.
-- Compare to the *_offset fields above which are hex.
@@ -275,28 +269,26 @@ local function read_sleb128_at(buf, pos)
return nil, pos
end
-- Find the 0-based offset of the table-terminator byte (a single 0) for the
-- abbrev table starting at `table_start`. Returns nil on truncated input.
-- Walks declaration headers (code, tag, has_children, attr/form pairs,
-- DW_FORM_implicit_const constant) until it finds a 0 byte that follows a
-- complete declaration. Mirrors dwarf_injection.lua :: find_abbrev_table_end.
-- Find the 0-based offset of the table-terminator byte (a single 0) for the abbrev table starting at `table_start`.
-- Returns nil on truncated input. Walks declaration headers
-- (code, tag, has_children, attr/form pairs, DW_FORM_implicit_const constant) until it finds a 0 byte that follows a complete declaration. Mirrors dwarf_injection.lua :: find_abbrev_table_end.
local function find_abbrev_table_end(table_bytes, table_start)
local pos, len = table_start, #table_bytes
if pos >= len or table_bytes:byte(pos + 1) == 0 then return pos end
while pos < len do
local _code, code_end = read_uleb128_at(table_bytes, pos)
local _code, code_end = read_uleb128_at(table_bytes, pos)
if not _code then return nil end
pos = code_end
local _tag, tag_end = read_uleb128_at(table_bytes, pos)
local _tag, tag_end = read_uleb128_at(table_bytes, pos)
if not _tag then return nil end
pos = tag_end
if pos >= len then return nil end
pos = pos + 1 -- has_children byte
while pos < len do
local attr, attr_end = read_uleb128_at(table_bytes, pos)
local attr, attr_end = read_uleb128_at(table_bytes, pos)
if not attr then return nil end
pos = attr_end
local form, form_end = read_uleb128_at(table_bytes, pos)
local form, form_end = read_uleb128_at(table_bytes, pos)
if not form then return nil end
pos = form_end
if attr == 0 and form == 0 then break end
@@ -321,8 +313,8 @@ local function read_c_string_at(buf, off)
return buf:sub(start + 1, off)
end
-- Walk the .debug_abbrev table starting at 0-based offset `table_start` and
-- return a list of declarations: {code, tag, has_children, attrs={ {name, form}, ... }}.
-- Walk the .debug_abbrev table starting at 0-based offset `table_start` and return a list of declarations:
-- {code, tag, has_children, attrs={ {name, form}, ... }}.
-- Stops at the table terminator.
local function parse_abbrev_table(table_bytes, table_start)
local table_end = find_abbrev_table_end(table_bytes, table_start)
@@ -330,17 +322,17 @@ local function parse_abbrev_table(table_bytes, table_start)
local decls = {}
local pos = table_start
while pos < table_end do
local code, code_end = read_uleb128_at(table_bytes, pos)
local code, code_end = read_uleb128_at(table_bytes, pos)
if not code then return nil, "truncated code" end
pos = code_end
local tag, tag_end = read_uleb128_at(table_bytes, pos)
local tag, tag_end = read_uleb128_at(table_bytes, pos)
if not tag then return nil, "truncated tag" end
pos = tag_end
local has_children = table_bytes:byte(pos + 1)
pos = pos + 1
local attrs = {}
while true do
local attr, attr_end = read_uleb128_at(table_bytes, pos)
local attr, attr_end = read_uleb128_at(table_bytes, pos)
if not attr then return nil, "truncated attr" end
pos = attr_end
local form, form_end = read_uleb128_at(table_bytes, pos)
@@ -349,7 +341,7 @@ local function parse_abbrev_table(table_bytes, table_start)
if attr == 0 and form == 0 then break end
attrs[#attrs + 1] = { name = attr, form = form }
if form == DW_FORM_implicit_const then
local _c, ce = read_sleb128_at(table_bytes, pos)
local _c, ce = read_sleb128_at(table_bytes, pos)
if not _c then return nil, "truncated const" end
pos = ce
end
@@ -362,8 +354,8 @@ end
-- Read a ULEB attribute value at 0-based offset `pos` for the given `form`.
-- Returns (value, next_pos). For DW_FORM_string we return the inline string.
-- For DW_FORM_strp we return the inline string resolved from `str_buf`.
-- For DW_FORM_ref4 we return the absolute CU-relative offset. The caller
-- decides whether to interpret that as a section offset.
-- For DW_FORM_ref4 we return the absolute CU-relative offset.
-- The caller decides whether to interpret that as a section offset.
local function read_form_value(buf, str_buf, pos, form)
if form == M.DW_FORM.addr then
return M.read_u32_le(buf, pos + 1), pos + 4
@@ -374,16 +366,11 @@ local function read_form_value(buf, str_buf, pos, form)
-- DW_FORM_strp: 4-byte offset into .debug_str.
local strp_off = M.read_u32_le(buf, pos + 1)
return read_c_string_at(str_buf, strp_off), pos + 4
elseif form == M.DW_FORM.udata then
return read_uleb128_at(buf, pos)
elseif form == M.DW_FORM.data1 then
return buf:byte(pos + 1), pos + 1
elseif form == M.DW_FORM.data2 then
return M.read_u16_le(buf, pos + 1), pos + 2
elseif form == M.DW_FORM.data4 then
return M.read_u32_le(buf, pos + 1), pos + 4
elseif form == M.DW_FORM.ref4 then
return M.read_u32_le(buf, pos + 1), pos + 4
elseif form == M.DW_FORM.udata then return read_uleb128_at(buf, pos)
elseif form == M.DW_FORM.data1 then return buf:byte(pos + 1), pos + 1
elseif form == M.DW_FORM.data2 then return M.read_u16_le(buf, pos + 1), pos + 2
elseif form == M.DW_FORM.data4 then return M.read_u32_le(buf, pos + 1), pos + 4
elseif form == M.DW_FORM.ref4 then return M.read_u32_le(buf, pos + 1), pos + 4
elseif form == M.DW_FORM.sec_offset then
-- DW_FORM_sec_offset: 4-byte offset (size depends on DWARF version;
-- on DWARF5 32-bit it's always 4 bytes).
@@ -392,7 +379,7 @@ local function read_form_value(buf, str_buf, pos, form)
return 1, pos
elseif form == M.DW_FORM.exprloc then
-- DW_FORM_exprloc: ULEB byte count + that many bytes of DW_OP_*.
local len, ne = read_uleb128_at(buf, pos)
local len, ne = read_uleb128_at(buf, pos)
if not len then return nil, pos end
return nil, ne + len
elseif form == DW_FORM_implicit_const then
@@ -405,53 +392,48 @@ local function read_form_value(buf, str_buf, pos, form)
end
end
-- Index the .debug_info + .debug_abbrev sections of an existing ELF and
-- collect one entry per "interesting" type DIE in the FIRST compilation
-- unit. The index supports typed-view resolution for Phase 5:
--
-- Index the .debug_info + .debug_abbrev sections of an existing ELF and collect one entry
-- per "interesting" type DIE in the FIRST compilation unit.
-- The index supports typed-views:
-- index = {
-- by_name = { ["V4_S2"] = {kind="structure_type", die_offset, byte_size, fields={...}},
-- ["U4"] = {kind="base_type", die_offset, byte_size, encoding="unsigned"},
-- ["MipsCode"]= {kind="typedef", die_offset, target_kind=..., target_die_offset=...} },
-- by_name = { ["V4_S2"] = {kind="structure_type", die_offset, byte_size, fields={...}},
-- ["U4"] = {kind="base_type", die_offset, byte_size, encoding="unsigned"},
-- ["MipsCode"] = {kind="typedef", die_offset, target_kind=..., target_die_offset=...} },
-- by_offset = { [die_offset] = {kind, name, ...} }, -- reverse lookup
-- }
--
-- Only the main CU is indexed. We do not walk nested CUs (this matches the
-- Phase 2-4 scope: the main CU is the only one in this build).
-- @param info string -- .debug_info section bytes
-- @param abbrev string -- .debug_abbrev section bytes
-- @param str_buf string -- .debug_str section bytes (for DW_FORM_strp name resolution)
-- @param abbrev_offset integer -- 0-based offset of the main CU's abbrev table
-- @param cu_start integer|nil -- 0-based offset of the main CU (caller-known)
-- @param info string -- .debug_info section bytes
-- @param abbrev string -- .debug_abbrev section bytes
-- @param str_buf string -- .debug_str section bytes (for DW_FORM_strp name resolution)
-- @param abbrev_offset integer -- 0-based offset of the main CU's abbrev table
-- @param cu_start integer|nil -- 0-based offset of the main CU (caller-known)
-- @return table|nil, string|nil -- (index, error)
function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
if not info or #info < 12 or not abbrev or not abbrev_offset then return nil, "missing input" end
str_buf = str_buf or ""
-- Index the main table at abbrev_offset. The F' pass writes a trailing
-- 0 byte after the main table; the G' pass may append additional codes
-- (100-108) + a 0 terminator. The walker may encounter a code that
-- wasn't in the main table but exists later in the same .debug_abbrev;
-- on the first miss, walk the rest of the section to add any new
-- abbrevs we encounter.
local abbrev_decls, err = parse_abbrev_table(abbrev, abbrev_offset)
-- Index the main table at abbrev_offset.
-- The F' pass writes a trailing 0 byte after the main table; the G' pass may append additional codes (100-108) + a 0 terminator.
-- The walker may encounter a code that wasn't in the main table but exists later in the same .debug_abbrev;
-- on the first miss, walk the rest of the section to add any new abbrevs we encounter.
local abbrev_decls, err = parse_abbrev_table(abbrev, abbrev_offset)
if not abbrev_decls then return nil, err end
local abbrev_by_code = {}
local abbrev_by_code = {}
for _, d in ipairs(abbrev_decls) do abbrev_by_code[d.code] = d end
-- Resolve cu_start + cu_end_excl.
local cu_end_excl
if cu_start then
local ul = M.read_u32_le(info, cu_start + 1)
if ul == 0xFFFFFFFF then return nil, "DWARF64 not supported" end
if ul == 0xFFFFFFFF then return nil, "DWARF64 not supported" end
cu_end_excl = cu_start + 4 + ul
else
local pos = 0
cu_start = nil
while pos < #info do
local ul = M.read_u32_le(info, pos + 1)
if ul == 0xFFFFFFFF then break end
if ul == 0xFFFFFFFF then break end
local unit_end = pos + 4 + ul
if unit_end > #info then break end
if unit_end > #info then break end
local unit_abbrev = M.read_u32_le(info, pos + 9)
if unit_abbrev == abbrev_offset then
cu_start = pos
@@ -464,7 +446,7 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
end
-- Walk the CU's DIE tree at 0-based offset cu_start + 12.
-- Emit a flat list of type-bearing DEIs; recursion handles nested children
-- Emit a flat list of type-bearing DEIs; recursion handles nested children
-- (members inside structure_type, types inside subprograms, etc.).
-- A non-type DIE's subtree is SKIPPED by walking until the matching null.
local by_name = {}
@@ -475,7 +457,7 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
local root_decl_code
root_decl_code, pos_cursor = read_uleb128_at(info, pos_cursor)
if not root_decl_code then return nil, "truncated root DIE code" end
local root_decl = abbrev_by_code[root_decl_code]
local root_decl = abbrev_by_code[root_decl_code]
if not root_decl then return nil, "unknown root DIE abbrev" end
-- Skip root DIE attributes.
for _, attr in ipairs(root_decl.attrs) do
@@ -488,11 +470,10 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
return { by_name = by_name, by_offset = by_offset }
end
-- Helper: skip a subtree rooted at the current DIE (pos_cursor is
-- positioned at the first child). Walks down and right until the
-- matching null terminator is consumed. Returns the new pos_cursor.
-- For DIE trees that contain only the closed type + member shapes,
-- the depth never exceeds 2 (type DIE -> member DIE -> null).
-- Helper: skip a subtree rooted at the current DIE (pos_cursor is positioned at the first child).
-- Walks down and right until the matching null terminator is consumed.
-- Returns the new pos_cursor.
-- For DIE trees that contain only the closed type + member shapes, the depth never exceeds 2 (type DIE -> member DIE -> null).
local function skip_subtree(pos)
local depth = 1
while pos < cu_end_excl and depth > 0 do
@@ -513,8 +494,8 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
-- Pre-declare n_visited so the closure helpers can read it.
local n_visited = 0
-- Helper: read one DIE's attributes. Returns (name, byte_size, encoding,
-- type_ref, new_pos_cursor) on success; (nil, error_string) on truncation.
-- Helper: read one DIE's attributes.
-- Returns (name, byte_size, encoding, type_ref, new_pos_cursor) on success; (nil, error_string) on truncation.
local function read_die_attributes(decl, pos)
local die_name = nil
local die_byte_size = nil
@@ -567,24 +548,23 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
return fields, pos
end
-- Top-level walker: iterate siblings. For each DIE, decide whether to
-- record (if type), descend (if structure_type with members), or skip
-- its subtree.
-- Top-level walker: iterate siblings.
-- For each DIE, decide whether to record (if type), descend (if structure_type with members), or skip its subtree.
while pos_cursor < cu_end_excl do
local code = info:byte(pos_cursor + 1)
if code == 0 then pos_cursor = pos_cursor + 1; break end
if code == 0 then pos_cursor = pos_cursor + 1; break end
local decl = abbrev_by_code[code]
if not decl then
-- Lazily scan the rest of the section for the missing code.
-- The G' pass appends new abbrevs (100-108) after the main table's
-- terminator. On the first miss, walk the rest of the section.
-- The G' pass appends new abbrevs (100-108) after the main table's terminator.
-- On the first miss, walk the rest of the section.
local scan_pos = 0
while scan_pos < #abbrev do
if abbrev:byte(scan_pos + 1) == 0 then
scan_pos = scan_pos + 1
goto continue
end
local new_table, e3 = parse_abbrev_table(abbrev, scan_pos)
local new_table, e3 = parse_abbrev_table(abbrev, scan_pos)
if not new_table then
scan_pos = scan_pos + 1
goto continue
@@ -595,7 +575,7 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
end
end
-- Find the terminator (0 byte) of this table.
local term = find_abbrev_table_end(abbrev, scan_pos)
local term = find_abbrev_table_end(abbrev, scan_pos)
if not term then break end
scan_pos = term + 1
::continue::
@@ -608,11 +588,11 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
local die_offset = pos_cursor
pos_cursor = pos_cursor + 1
local read_result = { read_die_attributes(decl, pos_cursor) }
if #read_result == 2 then
if #read_result == 2 then
return nil, read_result[2]
end
local die_name, die_byte_size, die_encoding, die_type_ref, pos_after_attrs
= read_result[1], read_result[2], read_result[3], read_result[4], read_result[5]
= read_result[1], read_result[2], read_result[3], read_result[4], read_result[5]
n_visited = n_visited + 1
local is_type = (decl.tag == M.DW_TAG.base_type
or decl.tag == M.DW_TAG.structure_type
@@ -623,10 +603,10 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
local member_fields = nil
pos_cursor = pos_after_attrs
if decl.tag == M.DW_TAG.structure_type and decl.has_children ~= 0 then
local f, np = read_member_fields(decl, pos_cursor)
local f, np = read_member_fields(decl, pos_cursor)
if not f then return nil, np end
member_fields = f
pos_cursor = np
pos_cursor = np
elseif decl.has_children ~= 0 then
-- Skip the subtree (e.g., DW_TAG_subprogram, DW_TAG_variable, etc.).
pos_cursor = skip_subtree(pos_cursor)
@@ -634,35 +614,34 @@ function M.index_main_cu_types(info, abbrev, str_buf, abbrev_offset, cu_start)
if is_type and die_name then
local kind
if decl.tag == M.DW_TAG.base_type then kind = "base_type"
if decl.tag == M.DW_TAG.base_type then kind = "base_type"
elseif decl.tag == M.DW_TAG.structure_type then kind = "structure_type"
elseif decl.tag == M.DW_TAG.typedef then kind = "typedef"
elseif decl.tag == M.DW_TAG.pointer_type then kind = "pointer_type"
elseif decl.tag == M.DW_TAG.const_type then kind = "const_type" end
elseif decl.tag == M.DW_TAG.typedef then kind = "typedef"
elseif decl.tag == M.DW_TAG.pointer_type then kind = "pointer_type"
elseif decl.tag == M.DW_TAG.const_type then kind = "const_type" end
local entry = {
kind = kind,
name = die_name,
die_offset = die_offset,
byte_size = die_byte_size,
encoding = die_encoding,
type_ref = die_type_ref,
fields = member_fields,
kind = kind,
name = die_name,
die_offset = die_offset,
byte_size = die_byte_size,
encoding = die_encoding,
type_ref = die_type_ref,
fields = member_fields,
}
by_name[die_name] = entry
by_name[die_name] = entry
by_offset[die_offset] = entry
end
end
return { by_name = by_name, by_offset = by_offset }
end
-- Resolve a chain of pointer + const + typedef + structure_type down to a
-- canonical struct or base type. The returned entry has kind, name, byte_size,
-- and (for structure_type) fields with {name, offset, type_name, pointer_depth}.
-- Resolve a chain of pointer + const + typedef + structure_type down to a canonical struct or base type.
-- The returned entry has kind, name, byte_size, and (for structure_type) fields with {name, offset, type_name, pointer_depth}.
-- Returns nil if the chain cannot be resolved (e.g., missing DIE).
-- @param index table -- M.index_main_cu_types result
-- @param start_offset integer -- CU-relative DW_FORM_ref4 offset of the start
-- @param max_depth integer -- cycle protection
-- @return table|nil -- {kind, name, byte_size, fields?, pointer_depth}
-- @param index table -- M.index_main_cu_types result
-- @param start_offset integer -- CU-relative DW_FORM_ref4 offset of the start
-- @param max_depth integer -- cycle protection
-- @return table|nil -- {kind, name, byte_size, fields?, pointer_depth}
function M.resolve_type_chain(index, start_offset, max_depth)
max_depth = max_depth or 16
if not index or not start_offset then return nil end
@@ -673,12 +652,9 @@ function M.resolve_type_chain(index, start_offset, max_depth)
local entry = index.by_offset[cur_offset]
if not entry then return nil end
chain[#chain + 1] = entry
if entry.kind == "pointer_type" then
cur_offset = entry.type_ref
elseif entry.kind == "const_type" then
cur_offset = entry.type_ref
elseif entry.kind == "typedef" then
cur_offset = entry.type_ref
if entry.kind == "pointer_type" then cur_offset = entry.type_ref
elseif entry.kind == "const_type" then cur_offset = entry.type_ref
elseif entry.kind == "typedef" then cur_offset = entry.type_ref
else
break
end
@@ -767,13 +743,13 @@ end
--- (no subprocess; lfs only for the existence check). Returns `{[name] = bytes_or_empty_string, ...}`.
---
--- **Convention:** offsets from `M.ELF32` (1-indexed for string.sub).
--- Every requested name has an entry in the returned dict; missing sections have an empty string (NOT nil)
--- so callers can do `sections[".debug_x"] or ""` for the missing case.
--- Every requested name has an entry in the returned dict;
--- missing sections have an empty string (NOT nil) so callers can do `sections[".debug_x"] or ""` for the missing case.
---
--- **Cost:** one file open + one `f:seek` + one `f:read` per section header
--- (we walk all `e_shnum` headers regardless of how many names are requested, to find the .shstrtab first).
--- For frequent callers, pass the union of all needed sections in one call.
-- can add `.debug_info` + `.debug_loc` + `.debug_str_offsets` to the list without writing a 2nd ELF walker.
-- Can add `.debug_info` + `.debug_loc` + `.debug_str_offsets` to the list without writing a 2nd ELF walker.
--- @param elf_path Path
--- @param section_names string[] -- list of section names to read
--- @return table<string, string>
@@ -897,8 +873,7 @@ function M.read_nm(elf_path)
end
-- Iterate the 16-byte ELF32 symtab entries.
-- Each entry (1-indexed): st_name at 1, st_value at 5, st_size at 9,
-- st_info at 13, st_other at 14, st_shndx at 15.
-- Each entry (1-indexed): st_name at 1, st_value at 5, st_size at 9, st_info at 13, st_other at 14, st_shndx at 15.
local SYM_ENTRY_BYTES = 0x10
local SYM_ST_NAME = 0x01
local SYM_ST_VALUE = 0x05
@@ -940,7 +915,6 @@ end
-- Both encoders pack 7 bits of data per byte + 1 bit of "more bytes follow" signaling.
--
-- Per-byte layout:
--
-- bit: 7 6 5 4 3 2 1 0
-- │ └───── 7-bit data ─────┘
-- └─ continuation flag (LEB_CONT_BIT = 0x80)
@@ -966,13 +940,11 @@ local LEB_DATA_MASK = 0x7F
local SLEB_SIGN_BIT = 0x40
--- ULEB128 (Unsigned Little-Endian Base 128) encoder. Returns the byte string for the non-negative integer `n`.
---
--- Algorithm:
--- - Extract the low 7 bits of `n` (LEB_DATA_MASK = 0x7F).
--- - Shift `n` right by 7 bits.
--- - If more bytes remain, OR in the continuation flag (LEB_CONT_BIT).
--- - Repeat until `n` is fully consumed.
---
--- @param n integer -- non-negative
--- @return string
function M.uleb128(n)
@@ -992,9 +964,7 @@ function M.uleb128(n)
return table.concat(bytes)
end
--- SLEB128 (Signed Little-Endian Base 128) encoder. Returns the byte
--- string for the integer `n` (may be negative).
---
--- SLEB128 (Signed Little-Endian Base 128) encoder. Returns the byte string for the integer `n` (may be negative).
--- Algorithm differs from ULEB128 by the termination condition: stop when
--- the remaining bits can be inferred from the sign bit in the last byte's
--- 7-bit data payload.
@@ -1003,15 +973,14 @@ end
---
--- Without these checks, the decoder would round-trip to a different value
--- (e.g. encoding `0` as `0x80 0x00` decodes to `0` correctly but is 2 bytes long; the termination check picks the 1-byte `0x00` form).
---
--- @param n integer -- any integer (negative allowed)
--- @return string
function M.sleb128(n)
local bytes = {}
local more = true
while more do
local b = n % (LEB_DATA_MASK + 1) -- extract low 7 bits
n = (n - b) / (LEB_DATA_MASK + 1) -- arithmetic shift right by 7
local b = n % (LEB_DATA_MASK + 1) -- extract low 7 bits
n = (n - b) / (LEB_DATA_MASK + 1) -- arithmetic shift right by 7
-- Termination: remaining value bits fit in the sign bit of the last byte.
if n == 0 and b < SLEB_SIGN_BIT then more = false end -- positive terminator
if n == -1 and b >= SLEB_SIGN_BIT then more = false end -- negative terminator
@@ -1087,7 +1056,7 @@ end
--- Returns `{name -> {total = N, words = {{pos, call_file, call_line, comp_name, comp_file, comp_line}, ...}}}`.
--- Returns `{}` on format-version mismatch (and logs to stderr).
---
--- **Wire format** (emitted by `passes/atoms_source_map.lua` since the Phase 3 debug_ux work):
--- **Wire format** (emitted by `passes/atoms_source_map.lua`):
--- ```
--- # FORMAT_VERSION <n>
--- ATOM <name> "<abs-source-path>" <total>
@@ -1097,11 +1066,10 @@ end
--- ENDATOM
--- ```
---
--- **Used by** `passes/dwarf_injection.lua` (Phase 3 — debug_ux) to:
--- **Used by** `passes/dwarf_injection.lua` to:
--- - group consecutive MACRO rows into component invocations (one `DW_TAG_inlined_subroutine` each)
--- - emit abstract `DW_TAG_subprogram` per unique component name
--- - extend `.debug_line` so stepping into a `mac_X(...)` lands on the component's source line.
---
--- @param prov_path string -- path to *.atoms.provenance.txt
--- @param expected_version integer -- expected FORMAT_VERSION line
--- @return table<string, table>
scripts/gdb/gdb_tape_atoms.gdb
+15 -29
View File
@@ -1,29 +1,15 @@
# scripts/gdb/gdb_tape_atoms.gdb
#
# Wrapper for the tape-atom step-debug helpers. The 9 user commands are defined
# here as STUBS (degraded-state messages). The real implementations + the
# per-atom data tables are emitted by `passes/atoms_source_map.lua` (post-link
# invocation: `ps1_meta.lua --atoms-source-map --gdb-runtime --elf <elf>`) into
# `build/gen/gdb_tape_atoms_runtime.gdb`. Sourcing that file RE-DEFINES the
# commands with real implementations.
# Wrapper for the tape-atom step-debug helpers.
# The 9 user commands are defined here as STUBS (degraded-state messages).
# The real implementations + the per-atom data tables are emitted by `passes/atoms_source_map.lua`
# (post-link invocation: `ps1_meta.lua --atoms-source-map --gdb-runtime --elf <elf>`) into `build/gen/gdb_tape_atoms_runtime.gdb`.
# Sourcing that file RE-DEFINES the commands with real implementations.
#
# If `build/gen/gdb_tape_atoms_runtime.gdb` is missing or stale, the stubs
# remain (E1: no source map). The user just needs to re-run `build_psyq.ps1`
# to regenerate. No exceptions; no crashes.
#
# Why a wrapper + separate runtime file?
# - The runtime file is auto-generated per-build; not in git.
# - The wrapper is checked into git; always works.
# - This split keeps the script trivial and the data plumbing out of git.
#
# Compatible with every gdb build (no Python, no Tcl, no Guile required) —
# pure gdb command scripting + `set $var = val` + `define ... end`.
#
# Generated by track gdb_tape_atom_debugging_20260711 — see
# C:\projects\Pikuma\ps1-ai\docs\gdb_tape_atom_debugging.md for the manual.
# If `build/gen/gdb_tape_atoms_runtime.gdb` is missing or stale, the stubs remain (E1: no source map).
# The user just needs to re-run `build_psyq.ps1` to regenerate.
# ── Stub commands (defined here so they're always present, even if the
# runtime file is missing). The runtime file overrides these if sourced. ──
# ── Stub commands (defined here so they're always present, even if the runtime file is missing). The runtime file overrides these if sourced. ──
define tape_atoms
echo "[gdb_tape_atoms] STUB: runtime file build/gen/gdb_tape_atoms_runtime.gdb not found."
@@ -77,7 +63,7 @@ define show_c2
printf "C2[14] 0x%08x [sxy2]\n", $c2_data[14]
printf "C2[24] 0x%08x [mac0]\n", $c2_data[24]
printf "...\n"
echo "(STUB state: only 7 representative regs shown. Run build_psyq.ps1 for full dump.)"
echo "(STUB state: only 7 representative regs shown. Run build_psyq.ps1 for full dump.)"
end
document show_c2
Pretty-print all 32 C2 data registers as hex + named alias. STUB state (7 reg subset).
@@ -107,13 +93,13 @@ end
# Try to source from project-root-relative path first (the typical case).
# If the user is in a different CWD, the source will fail and stubs remain.
# The runtime file path is computed relative to the ELF's source map convention
# (build/gen/gdb_tape_atoms_runtime.gdb).
# The runtime file path is computed relative to the ELF's source map convention (build/gen/gdb_tape_atoms_runtime.gdb).
echo [gdb_tape_atoms] Wrapper loaded. Sourcing runtime file...
# Suppress the "Redefine command" prompts that would otherwise appear when the
# runtime file overrides the 9 stub commands defined above. The runtime's
# `define` blocks are intended to overwrite — there's no ambiguity to confirm.
# Suppress the "Redefine command" prompts that would otherwise appear when the runtime file overrides the 9 stub commands defined above.
# The runtime's `define` blocks are intended to overwrite — there's no ambiguity to confirm.
set confirm off
# Source the runtime file (re-defines commands with real impls + data).
source build/gen/gdb_tape_atoms_runtime.gdb
set confirm on
echo [gdb_tape_atoms] Runtime sourced successfully (9 commands now have real implementations).
echo [gdb_tape_atoms] Runtime sourced successfully (9 commands now have real implementations).
scripts/passes/annotation.lua
+52 -62
View File
@@ -13,8 +13,7 @@
-- Bootstrap: same as entry scripts. See `ps1_meta.lua` for the rationale.
-- Bootstrap: load `scripts/duffle_paths.lua` (sets package.path + package.cpath).
-- Uses `debug.getinfo` to find this file's own directory, so it works
-- both standalone and when require'd from the orchestrator.
-- Uses `debug.getinfo` to find this file's own directory, so it works both standalone and when require'd from the orchestrator.
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source` (works both standalone + when require'd).
-- duffle_paths.lua sets package.path then returns `require("duffle")` at the bottom, so the dofile value IS the duffle module.
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
@@ -27,8 +26,8 @@ local WAVE_CONTEXT_REGS = duffle.WAVE_CONTEXT_REGS
local function is_wave_context_reg(n) return WAVE_CONTEXT_REGS[n] ~= nil end
-- Phase 5 closed sets. atom_dbg_reg_default and atom_reg_types MUST target
-- one of these registers; atom_view MUST reference a real Binds_* struct.
-- Closed sets. atom_dbg_reg_default and atom_reg_types MUST target one of these registers;
-- atom_view MUST reference a real Binds_* struct.
local SEMANTIC_DEFAULT_REGS = {
["R_TapePtr"] = true, ["R_AtomJmp"] = true,
["R_PrimCursor"] = true, ["R_FaceCursor"] = true,
@@ -39,9 +38,8 @@ local COMPUTE_REG_PREFIX = "R_"
local COMPUTE_REG_RE = -- permits R_T0..R_T3 + caller-trash (not wave-context)
"^R_T[0-3]$"
-- Compute-register types are restricted to byte-width primitives (no struct
-- identity in this phase). Pointer depth is also bounded — typed pointer
-- chains live in the Binds_* struct, not in a per-atom override.
-- Compute-register types are restricted to byte-width primitives.
-- Pointer depth is also bounded typed pointer chains live in the Binds_* struct, not in a per-atom override.
local ALLOWED_COMPUTE_TYPES = {
["U4"] = true, ["S4"] = true,
["U2"] = true, ["S2"] = true,
@@ -68,7 +66,7 @@ local ALLOWED_COMPUTE_TYPES = {
--- @field project_root string
--- @field upstream table<string, table>
--- @field flags table
--- @field flags._annot_results table[] -- stashed by annotation pass; consumed by report.lua
--- @field flags._annot_results table[] -- stashed by annotation pass; consumed by report.lua
--- @field dry_run boolean
--- @field verbose boolean
@@ -88,18 +86,18 @@ local ALLOWED_COMPUTE_TYPES = {
--- @field errors string[]|nil -- parse-time errors from scan_source (atom_info body malformed)
--- @class SkipOverMarker -- sub-shape of scan_source.lua's @class SkipOverMarker
--- @field marker_kind string -- exact marker ident (always "atom_dbg_skip_over")
--- @field marker_kind string -- exact marker ident (always "atom_dbg_skip_over")
--- @field marker_line integer
--- @field args string|nil -- trimmed text inside the parens (nil when has_parens is false)
--- @field args string|nil -- trimmed text inside the parens (nil when has_parens is false)
--- @field has_parens boolean
--- @field pending boolean -- true while awaiting the following declaration
--- @field superseded_by_marker_line integer|nil -- set on a marker that was bumped out of the pending slot
--- @field target_kind string|nil -- "atom" | "comp_bare" | "comp_proc" | "unrelated" once observed
--- @field pending boolean -- true while awaiting the following declaration
--- @field superseded_by_marker_line integer|nil -- set on a marker that was bumped out of the pending slot
--- @field target_kind string|nil -- "atom" | "comp_bare" | "comp_proc" | "unrelated" once observed
--- @field declaration_line integer|nil
--- @class Finding
--- @field line integer -- source line (or 0 for pass-level)
--- @field msg string -- finding message
--- @field line integer -- source line (or 0 for pass-level)
--- @field msg string -- finding message
--- @class Findings
--- @field errors Finding[]
@@ -107,14 +105,14 @@ local ALLOWED_COMPUTE_TYPES = {
--- @field info Finding[]
--- @class PipeCtx
--- @field atom_index table<string, AtomAnnotation> -- name -> AtomAnnotation (only kind=="atom")
--- @field binds_index table<string, BindsStruct> -- name -> BindsStruct
--- @field annot_counts table<string, integer> -- name -> annotation count (for unique_annotation check)
--- @field types table<string, RegTypeDefault> -- from scan_source
--- @field atom_views table<string, AtomViewEntry> -- from scan_source
--- @field seen_defaults table<string, integer> -- duplicate atom_dbg_reg_default detection
--- @field seen_field table<string, integer> -- Binds_* -> count of fields (set/checked by check_binds_no_duplicate_fields)
--- @field _scan SourceScan -- full scan payload (typed-view sub-calls live here)
--- @field atom_index table<string, AtomAnnotation> -- name -> AtomAnnotation (only kind=="atom")
--- @field binds_index table<string, BindsStruct> -- name -> BindsStruct
--- @field annot_counts table<string, integer> -- name -> annotation count (for unique_annotation check)
--- @field types table<string, RegTypeDefault> -- from scan_source
--- @field atom_views table<string, AtomViewEntry> -- from scan_source
--- @field seen_defaults table<string, integer> -- duplicate atom_dbg_reg_default detection
--- @field seen_field table<string, integer> -- Binds_* -> count of fields (set/checked by check_binds_no_duplicate_fields)
--- @field _scan SourceScan -- full scan payload (typed-view sub-calls live here)
--- @class AnnotatedResult
--- @field atoms AtomEntry[]
@@ -132,8 +130,8 @@ local ALLOWED_COMPUTE_TYPES = {
-- Each check has a uniform `append_to_findings` shape (errors[] / warnings[] / info[]).
-- The dispatcher in `validate()` decides which findings list each check writes to — by convention,
-- "existence" checks (declaration must exist, struct must exist) write errors[]; "shape" checks
-- (writes/reads must be wave-context) write warnings[]. The `macro_word_drift` check writes
-- both errors[] (missing/mismatch) and info[] (match).
-- (writes/reads must be wave-context) write warnings[].
-- The `macro_word_drift` check writes both errors[] (missing/mismatch) and info[] (match).
--- Check: every annotated atom must have a matching MipsAtom_(name) declaration.
--- @param a AtomAnnotation
@@ -164,9 +162,8 @@ local function check_unique_annotation(pipe_ctx, findings)
end
--- Check: BIND atoms must reference a real Binds_* struct.
--- Demoted from error to warning (2026-07-10): the same condition is now caught by passes/static_analysis.lua's
--- check_abi_handoff() as an error. Emitting a warning here keeps the annotation pass from being stop-on-error
--- for the common test-fixture case, while still surfacing the issue in the report.
--- Emitting a warning here keeps the annotation pass from being stop-on-error for the common test-fixture case,
--- while still surfacing the issue in the report.
--- The static-analysis report remains the source of truth for build-stopping errors.
--- @param a AtomAnnotation
--- @param pipe_ctx PipeCtx
@@ -229,7 +226,7 @@ end
--- Check: TAPE_WORDS(mac_X, N) ↔ WORD_COUNT(mac_X, N) drift.
--- Three outcomes: missing (error), mismatch (error), match (info).
--- @param m MacroEntry
--- @param wc table<string, integer> -- the shared word-count table (from ctx.shared.word_counts)
--- @param wc table<string, integer> -- the shared word-count table (from ctx.shared.word_counts)
--- @param findings Findings
local function check_macro_word_drift(m, wc, findings)
local declared = wc[m.name]
@@ -253,16 +250,13 @@ local function check_macro_word_drift(m, wc, findings)
}
end
---- Check: atom_dbg_reg_default(R_X, <type>) must target a wave-context register
--- and the type name must be a recognized C type (we currently allow the
--- types actually used in the codebase; pointer depth must be 0 or 1).
---- Check: atom_dbg_reg_default(R_X, <type>) must target a wave-context register and the type name must be a recognized C type
--- (we currently allow the types actually used in the codebase; pointer depth must be 0 or 1).
--- Also detects duplicate defaults for the same register.
--- @param _src SourceFile -- unused (kept for the per_source shape)
--- @param pipe_ctx PipeCtx
--- @param findings Findings
-- Known type names accepted in atom_dbg_reg_default and the Binds_*-via-atom_view
-- resolution path. The Phase 5 list mirrors the canonical duffle math.h
-- typedefs plus the byte-width primitives used as compute-register overrides.
-- Known type names accepted in atom_dbg_reg_default and the Binds_*-via-atom_view resolution path.
local KNOWN_REG_DEFAULT_TYPES = {
["U4"] = true, ["S4"] = true,
["U2"] = true, ["S2"] = true,
@@ -349,8 +343,7 @@ local function check_atom_reg_types(_src, pipe_ctx, findings)
end
end
--- Check: atom_view(Binds_X) entries must reference a real Binds_* struct
--- and that struct must declare at least one field.
--- Check: atom_view(Binds_X) entries must reference a real Binds_* struct and that struct must declare at least one field.
--- @param _src SourceFile
--- @param pipe_ctx PipeCtx
--- @param findings Findings
@@ -379,8 +372,8 @@ local function check_atom_view_layout(_src, pipe_ctx, findings)
end
end
--- Check: Binds_* structs may not have duplicate field names (they would
--- defeat the typed-field name lookup that atom_view exposes in gdb).
--- Check: Binds_* structs may not have duplicate field names
--- (they would defeat the typed-field name lookup that atom_view exposes in gdb).
--- @param _src SourceFile
--- @param pipe_ctx PipeCtx
--- @param findings Findings
@@ -413,7 +406,7 @@ end
--- 4. pending + no target_kind -> dangling (no following declaration)
--- 5. unsupported target_kind -> marker precedes an unrelated declaration
--- Valid markers before whole-atom / bare-component / proc-component declarations
--- emit no error and remain in src.scan.skip_over.atoms / .components for Task 15.
--- emit no error and remain in src.scan.skip_over.atoms / .components.
--- @param marker SkipOverMarker
--- @param _pipe_ctx PipeCtx -- unused today; kept for plex-shape consistency with per_annot
--- @param findings Findings
@@ -440,8 +433,8 @@ local function check_skip_marker(marker, _pipe_ctx, findings)
if marker.superseded_by_marker_line then
findings.errors[#findings.errors + 1] = {
line = line,
msg = string.format("duplicate %s marker at line %d; superseded by another %s marker at line %d",
kind, line, kind, marker.superseded_by_marker_line),
msg = string.format("duplicate %s marker at line %d; superseded by another %s marker at line %d"
, kind, line, kind, marker.superseded_by_marker_line),
}
return
end
@@ -449,8 +442,8 @@ local function check_skip_marker(marker, _pipe_ctx, findings)
if marker.pending and not marker.target_kind then
findings.errors[#findings.errors + 1] = {
line = line,
msg = string.format("dangling %s marker at line %d: no following MipsAtom_/MipsAtomComp_/MipsAtomComp_Proc_ declaration",
kind, line),
msg = string.format("dangling %s marker at line %d: no following MipsAtom_/MipsAtomComp_/MipsAtomComp_Proc_ declaration"
, kind, line),
}
return
end
@@ -461,8 +454,8 @@ local function check_skip_marker(marker, _pipe_ctx, findings)
and marker.target_kind ~= "comp_proc" then
findings.errors[#findings.errors + 1] = {
line = line,
msg = string.format("%s marker at line %d must precede MipsAtom_, MipsAtomComp_, or MipsAtomComp_Proc_; found an unrelated declaration",
kind, line),
msg = string.format("%s marker at line %d must precede MipsAtom_, MipsAtomComp_, or MipsAtomComp_Proc_; found an unrelated declaration"
, kind, line),
}
end
end
@@ -472,10 +465,10 @@ end
-- ════════════════════════════════════════════════════════════════════════════
--
-- Each rule entry picks one of four "shapes" of dispatch:
-- per_annot(annot, pipe_ctx, findings) — runs once per AtomAnnotation
-- post(pipe_ctx, findings) — runs once after all per_annot calls complete (full-corpus aggregation)
-- per_macro(macro, wc, findings) — runs once per TAPE_WORDS / _Pragma macro declaration
-- per_skip_marker(marker, pipe_ctx, findings) — runs once per src.scan.skip_over.markers entry (Task 14)
-- per_annot(annot, pipe_ctx, findings) — runs once per AtomAnnotation
-- post(pipe_ctx, findings) — runs once after all per_annot calls complete (full-corpus aggregation)
-- per_macro(macro, wc, findings) — runs once per TAPE_WORDS / _Pragma macro declaration
-- per_skip_marker(marker, pipe_ctx, findings) — runs once per src.scan.skip_over.markers entry
--
-- Adding a new check = 1 row here + 1 function above. The `validate()` dispatch loop never needs editing.
@@ -532,8 +525,7 @@ local function validate(ctx, src)
-- Build pipe_ctx (Fleury: expose structure). Pre-compute everything the per-check functions need.
-- Single source of truth for atom / binds / annotation-count lookups.
-- Phase 5 typed views: pipe_ctx.types / pipe_ctx.atom_views / pipe_ctx.seen_defaults
-- are projected from the scan payload so per_source check rules can iterate.
-- pipe_ctx.types / pipe_ctx.atom_views / pipe_ctx.seen_defaults are projected from the scan payload so per_source check rules can iterate.
local seen_defaults = {}
for reg, _ in pairs(scan.types or {}) do
seen_defaults[reg] = (seen_defaults[reg] or 0) + 1
@@ -592,10 +584,10 @@ local function validate(ctx, src)
if rule.post then rule.post(pipe_ctx, findings) end
end
-- Per-skip-marker rules (Task 14). Each raw marker recorded by scan_source
-- (in scan.skip_over.markers) is validated independently; the check emits at
-- most one error per marker. Valid markers stay attached to scan.skip_over.atoms /
-- .components for Task 15's dwarf_injection.lua consumer.
-- Per-skip-marker rules.
-- Each raw marker recorded by scan_source (in scan.skip_over.markers) is validated independently;
-- the check emits at most one error per marker.
-- Valid markers stay attached to scan.skip_over.atoms /.components for dwarf_injection.lua consumer.
local skip_markers = scan.skip_over and scan.skip_over.markers or {}
for _, marker in ipairs(skip_markers) do
for _, rule in ipairs(CHECK_RULES) do
@@ -611,9 +603,8 @@ local function validate(ctx, src)
end
end
-- Per-source rules (Phase 5 typed views: reg defaults, atom_view layout,
-- compute-register type overrides, Binds_* field uniqueness). Each
-- per_source rule sees the full scan payload via pipe_ctx.
-- Per-source rules (reg defaults, atom_view layout, compute-register type overrides, Binds_* field uniqueness).
-- Each per_source rule sees the full scan payload via pipe_ctx.
for _, rule in ipairs(CHECK_RULES) do
if rule.per_source then rule.per_source(src, pipe_ctx, findings) end
end
@@ -707,7 +698,6 @@ function M.run(ctx)
for dir, dir_sources in pairs(by_dir) do
local dir_basename = dir:match("([^/\\]+)$") or dir
local dir_atoms = 0
local dir_errors = {}
local dir_warnings = {}
@@ -716,8 +706,8 @@ function M.run(ctx)
ctx.flags._annot_source_results = ctx.flags._annot_source_results or {}
for _, src in ipairs(dir_sources) do
local result = validate(ctx, src)
result.source = src.path -- tag for downstream rendering
ctx.flags._annot_source_results[src.path] = result -- stash so report.lua reads from cache instead of re-running validate()
result.source = src.path -- tag for downstream rendering
ctx.flags._annot_source_results[src.path] = result -- stash so report.lua reads from cache instead of re-running validate()
dir_atoms = dir_atoms + #result.atoms
for _, e in ipairs(result.errors) do
dir_errors[#dir_errors + 1] = { line = e.line, msg = e.msg, source = src.path }
scripts/passes/atoms_source_map.lua
+64 -101
View File
@@ -10,14 +10,14 @@
--- **Two output forms** (per the workspace's per-emission-form pattern from
--- `guide_metaprogram_ssdl.md`):
--- 1. **Canonical text form** — `<out_root>/<basename>.atoms.sourcemap.txt`.
--- Always emitted. Format-version-tagged for forward-compat.
--- Lives in `<out_root>/` (build/gen) NOT `<source_dir>/gen/`. This file is a **build report**, not a compile artifact.
--- Format-version-tagged for forward-compat.
--- Lives in `<out_root>/` (build/gen).
--- Matches the convention used by `annotation.lua` (`<out_root>/<basename>.errors.h`) + `static_analysis.lua` (`<out_root>/<basename>.static_analysis.txt`).
--- Compile artifacts (`*.macs.h`, `*.offsets.h`) stay in `<source_dir>/gen/`.
--- 2. **gdb-runtime form** — `<ctx.out_root>/gdb_tape_atoms_runtime.gdb`
--- (pure gdb command script; addresses pre-computed via `nm`; the 9 user commands defined as `define ... end` blocks).
--- Emitted ONLY when `ctx.flags.gdb_runtime` is true AND `ctx.flags.elf_path` points to an existing ELF.
--- The gdb runtime form lets `gdb-multiarch --without-python` users (the common case on Windows MinGW builds)
--- The gdb runtime form lets `gdb-multiarch --without-python` users (the common case on Windows MinGW builds)
--- load the source-map data via `source <path>` — no Python/Tcl/Guile required.
---
--- **Output format** (canonical text form):
@@ -47,9 +47,9 @@
-- Module-scope requires + package.path setup
-- ════════════════════════════════════════════════════════════════════════════
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source` (works
-- both standalone + when require'd). `duffle_paths.lua` sets package.path then
-- returns `require("duffle")` at the bottom, so the dofile value IS the duffle module.
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source`
-- (works both standalone + when require'd). `duffle_paths.lua` sets package.path then returns `require("duffle")`
-- at the bottom, so the dofile value IS the duffle module.
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
local elf_dwarf = require("elf_dwarf")
@@ -60,8 +60,8 @@ local count_token_words = word_count_eval.count_token_words
-- Constants
-- ════════════════════════════════════════════════════════════════════════════
-- Format version emitted as the first line. Bump + add a migration test if the
-- format changes; the gdb runtime loader rejects mismatches (E2).
-- Format version emitted as the first line. Bump + add a migration test if the format changes;
-- the gdb runtime loader rejects mismatches (E2).
local FORMAT_VERSION = 1
-- Marker-call identifiers (mirrors offsets.lua:33-34).
@@ -73,12 +73,12 @@ local OFFSET_MARKER = "atom_offset"
-- ════════════════════════════════════════════════════════════════════════════
--- @class AtomSourceMapCtx
--- @field sources table[] -- SourceScan payload per source (from `ctx.sources`)
--- @field shared table -- `ctx.shared`
--- @field shared.word_counts table -- macro name -> word count (populated by word-counts + components passes)
--- @field out_root string -- output root (e.g. "build/gen")
--- @field dry_run boolean -- if true, compute but don't write
--- @field flags table -- `ctx.flags`; reads `flags.gdb_runtime` + `flags.elf_path`
--- @field sources table[] -- SourceScan payload per source (from `ctx.sources`)
--- @field shared table -- `ctx.shared`
--- @field shared.word_counts table -- macro name -> word count (populated by word-counts + components passes)
--- @field out_root string -- output root (e.g. "build/gen")
--- @field dry_run boolean -- if true, compute but don't write
--- @field flags table -- `ctx.flags`; reads `flags.gdb_runtime` + `flags.elf_path`
-- ════════════════════════════════════════════════════════════════════════════
-- Helpers
@@ -108,8 +108,7 @@ local function count_marker_rest(tok, wc)
end
--- Compute per-word entries for an atom.
--- Shared between the canonical text form (per-source `.atoms.sourcemap.txt`)
--- and the gdb-runtime form (`gdb_tape_atoms_runtime.gdb`).
--- Shared between the canonical text form and the gdb-runtime form.
---
--- Returns a list of `{pos, line, text}` entries + the total word count.
--- Markers contribute 0 entries (the marker call emits 0 `.word`s).
@@ -135,9 +134,8 @@ local function compute_word_entries(atom, src, wc)
-- Source line for THIS token = line containing byte offset `atom.body_off + rel`.
-- `src.scan.line_of(...)` is O(log N) via LineIndex.
local line = src.scan.line_of(atom.body_off + rel)
-- Flatten newlines + tabs in TEXT to spaces so each WORD entry fits on
-- one physical line. The gdb Python parser (or our pure-gdb parser)
-- does line-based splits; multi-line TEXT would break it.
-- Flatten newlines + tabs in TEXT to spaces so each WORD entry fits on one physical line.
-- The gdb Python parser (or our pure-gdb parser) does line-based splits; multi-line TEXT would break it.
local text = duffle.trim(tok):gsub("[\t\r\n]+", " ")
for _ = 1, words do
entries[#entries + 1] = { pos = pos, line = line, text = text }
@@ -149,16 +147,16 @@ local function compute_word_entries(atom, src, wc)
end
-- ════════════════════════════════════════════════════════════════════════════
-- Provenance emission (Phase 3 — debug_ux)
-- Provenance emission
-- ════════════════════════════════════════════════════════════════════════════
-- Component-macro invocation prefix (mirrors components.lua's MAC_PREFIX).
local MAC_PREFIX = "mac_"
local MAC_PREFIX_LEN = 4
--- Strip the `mac_` prefix from a token's leading identifier. Returns nil
--- if the identifier doesn't start with `mac_` (so non-component tokens
--- like `load_half_u`, `nop2`, `gte_cmdw_*` fall through cleanly).
--- Strip the `mac_` prefix from a token's leading identifier.
--- Returns nil if the identifier doesn't start with `mac_`
--- (so non-component tokens like `load_half_u`, `nop2`, `gte_cmdw_*` fall through cleanly).
--- @param tok string
--- @return string|nil
local function strip_mac_prefix_from_token(tok)
@@ -170,14 +168,13 @@ local function strip_mac_prefix_from_token(tok)
return nil
end
--- Compute per-word provenance entries for an atom. Mirrors `compute_word_entries`
--- but additionally classifies each emitted `.word` as either:
--- Compute per-word provenance entries for an atom. Mirrors `compute_word_entries` but additionally classifies each emitted `.word` as either:
--- - `RAW` — emitted by a direct instruction token (no component provenance)
--- - `MACRO X` — emitted by a `mac_X(...)` component invocation, with the
--- component's definition file:line resolved from `ctx.shared.components`.
--- - `MACRO X` — emitted by a `mac_X(...)` component invocation,
--- with the component's definition file:line resolved from `ctx.shared.components`.
---
--- Returns a list of `{pos, line, text, comp_name, comp_line, comp_path}` entries + the
--- total word count. `comp_name` is nil for RAW rows.
--- Returns a list of `{pos, line, text, comp_name, comp_line, comp_path}` entries + the total word count.
--- `comp_name` is nil for RAW rows.
--- @param atom table -- one entry of scan.atoms / scan.raw_atoms
--- @param src table -- SourceFile (has .scan with .line_of(), .path)
--- @param wc table -- shared.word_counts
@@ -232,7 +229,7 @@ end
--- Render one atom's provenance stanza. Format:
--- `WORD N CALL <src-path>:<src-line> MACRO <name> "<def-path>:<def-line>"` (for component words)
--- `WORD N CALL <src-path>:<src-line> RAW` (for direct instructions)
--- `WORD N CALL <src-path>:<src-line> RAW` (for direct instructions)
--- Returns (lines, total_words).
--- @param src table
--- @param atom table
@@ -249,13 +246,10 @@ local function emit_provenance_stanza(src, atom, wc, comp)
for _, pe in ipairs(entries) do
if pe.comp_name then
lines[#lines + 1] = string.format(
'WORD %d CALL %s:%d MACRO %s "%s:%d"',
lines[#lines + 1] = string.format('WORD %d CALL %s:%d MACRO %s "%s:%d"',
pe.pos, rel_path, pe.line, pe.comp_name, pe.comp_path, pe.comp_line)
else
lines[#lines + 1] = string.format(
"WORD %d CALL %s:%d RAW",
pe.pos, rel_path, pe.line)
lines[#lines + 1] = string.format("WORD %d CALL %s:%d RAW", pe.pos, rel_path, pe.line)
end
end
@@ -276,27 +270,23 @@ local function render_provenance(src, wc, comp)
lines[#lines + 1] = "# auto-generated by ps1_meta.lua (passes/atoms_source_map.lua) — DO NOT EDIT"
lines[#lines + 1] = "# Per-.word provenance: maps each emitted .word to its call site (atom body"
lines[#lines + 1] = "# file:line) and, when the word was emitted by a `mac_X(...)` component invocation,"
lines[#lines + 1] = "# the component's definition file:line. Used by dwarf_injection (Phase 3) to synthesize"
lines[#lines + 1] = "# the component's definition file:line. Used by dwarf_injection to synthesize"
lines[#lines + 1] = "# DW_TAG_inlined_subroutine instances for component step-into."
for _, atom in ipairs(src.scan.atoms or {}) do
local stanza = emit_provenance_stanza(src, atom, wc, comp)
for _, line in ipairs(stanza) do
lines[#lines + 1] = line
end
for _, line in ipairs(stanza) do lines[#lines + 1] = line end
end
for _, atom in ipairs(src.scan.raw_atoms or {}) do
local stanza = emit_provenance_stanza(src, atom, wc, comp)
for _, line in ipairs(stanza) do
lines[#lines + 1] = line
end
for _, line in ipairs(stanza) do lines[#lines + 1] = line end
end
return table.concat(lines, "\n") .. "\n"
end
--- Render one atom's stanza for the canonical text form
--- (ATOM header line, N WORD lines, ENDATOM marker). Returns (lines, total_words).
--- Render one atom's stanza for the canonical text form (ATOM header line, N WORD lines, ENDATOM marker).
--- Returns (lines, total_words).
--- @param src table
--- @param atom table
--- @param wc table
@@ -308,7 +298,6 @@ local function emit_atom_stanza(src, atom, wc)
-- ATOM header line with placeholder total (patched after we know it).
lines[#lines + 1] = string.format('ATOM %s "%s" 0', atom.raw_name or atom.name, rel_path)
for _, we in ipairs(entries) do
lines[#lines + 1] = string.format("WORD %d LINE %d TEXT %s",
we.pos, we.line, we.text)
@@ -330,17 +319,13 @@ local function render_source_map(src, wc)
lines[#lines + 1] = "# FORMAT_VERSION " .. FORMAT_VERSION
lines[#lines + 1] = "# auto-generated by ps1_meta.lua (passes/atoms_source_map.lua) — DO NOT EDIT"
for _, atom in ipairs(src.scan.atoms or {}) do
for _, atom in ipairs(src.scan.atoms or {}) do
local stanza = emit_atom_stanza(src, atom, wc)
for _, line in ipairs(stanza) do
lines[#lines + 1] = line
end
for _, line in ipairs(stanza) do lines[#lines + 1] = line end
end
for _, atom in ipairs(src.scan.raw_atoms or {}) do
for _, atom in ipairs(src.scan.raw_atoms or {}) do
local stanza = emit_atom_stanza(src, atom, wc)
for _, line in ipairs(stanza) do
lines[#lines + 1] = line
end
for _, line in ipairs(stanza) do lines[#lines + 1] = line end
end
return table.concat(lines, "\n") .. "\n"
@@ -363,7 +348,7 @@ end
--- @param ctx PassCtx
--- @return table[] -- list of {idx, name, src_path, file_base, addr, size_bytes, words, entries}
local function build_atom_table(ctx)
local wc = (ctx.shared and ctx.shared.word_counts) or {}
local wc = (ctx.shared and ctx.shared.word_counts) or {}
local addrs = elf_dwarf.read_nm(ctx.flags.elf_path)
local matched = {}
@@ -431,8 +416,7 @@ local function append_gdb_commands(lines, matched)
for _, a in ipairs(matched) do
-- gdb 12.1 quirk: literals in printf args require an attached target.
-- Use the per-atom convenience vars set above as printf args.
lines[#lines + 1] = string.format(
' printf " code_%%-32s @ 0x%%08x %%4d words\\n", $__atom_name_%d, $__atom_addr_%d, $__atom_words_%d',
lines[#lines + 1] = string.format(' printf " code_%%-32s @ 0x%%08x %%4d words\\n", $__atom_name_%d, $__atom_addr_%d, $__atom_words_%d',
a.idx, a.idx, a.idx)
end
lines[#lines + 1] = "end"
@@ -445,8 +429,7 @@ local function append_gdb_commands(lines, matched)
lines[#lines + 1] = "define break_atom"
lines[#lines + 1] = ' echo "Usage: break_atom_<exact_name> (pick from the list below)"'
for _, a in ipairs(matched) do
lines[#lines + 1] = string.format(
' printf " break_atom_%%-32s\\n", $__atom_name_%d', a.idx)
lines[#lines + 1] = string.format(' printf " break_atom_%%-32s\\n", $__atom_name_%d', a.idx)
end
lines[#lines + 1] = "end"
lines[#lines + 1] = "document break_atom"
@@ -457,8 +440,7 @@ local function append_gdb_commands(lines, matched)
for _, a in ipairs(matched) do
lines[#lines + 1] = string.format("define break_atom_%s", a.name)
lines[#lines + 1] = string.format(" break *$__atom_addr_%d", a.idx)
lines[#lines + 1] = string.format(
' printf " Breakpoint set at code_%s (0x%%08x)\\n", $__atom_addr_%d', a.name, a.idx)
lines[#lines + 1] = string.format(' printf " Breakpoint set at code_%s (0x%%08x)\\n", $__atom_addr_%d', a.name, a.idx)
lines[#lines + 1] = "end"
lines[#lines + 1] = string.format("document break_atom_%s", a.name)
lines[#lines + 1] = string.format(" Set a breakpoint at code_%s.", a.name)
@@ -494,33 +476,24 @@ local function append_gdb_commands(lines, matched)
lines[#lines + 1] = " set $__matched = 0"
for _, a in ipairs(matched) do
-- Precompute end_addr (gdb 12.1's expression evaluator chokes on `addr + words*4`).
lines[#lines + 1] = string.format(
" set $__end_%d = $__atom_addr_%d + $__atom_words_%d * 4", a.idx, a.idx, a.idx)
lines[#lines + 1] = string.format(
" if $__pc >= $__atom_addr_%d && $__pc < $__end_%d", a.idx, a.idx)
lines[#lines + 1] = string.format(
' printf "atom: code_%%s\\n", $__atom_name_%d', a.idx)
lines[#lines + 1] = string.format(" set $__end_%d = $__atom_addr_%d + $__atom_words_%d * 4", a.idx, a.idx, a.idx)
lines[#lines + 1] = string.format(" if $__pc >= $__atom_addr_%d && $__pc < $__end_%d", a.idx, a.idx)
lines[#lines + 1] = string.format(' printf "atom: code_%%s\\n", $__atom_name_%d', a.idx)
lines[#lines + 1] = ' printf "addr: 0x%08x\\n", $__pc'
lines[#lines + 1] = string.format(
" set $__word = ($__pc - $__atom_addr_%d) / 4", a.idx)
lines[#lines + 1] = string.format(
' printf "word: %%d/%%d\\n", $__word, $__atom_words_%d', a.idx)
lines[#lines + 1] = string.format(" set $__word = ($__pc - $__atom_addr_%d) / 4", a.idx)
lines[#lines + 1] = string.format(' printf "word: %%d/%%d\\n", $__word, $__atom_words_%d', a.idx)
-- One inner-if per WORD entry. Each word's line + text hardcoded.
for _, we in ipairs(a.entries) do
lines[#lines + 1] = string.format(
" if $__word == %d", we.pos)
lines[#lines + 1] = string.format(" if $__word == %d", we.pos)
-- Escape TEXT for printf format string.
local escaped_text = we.text:gsub("%%", "%%%%"):gsub('"', '\\"')
lines[#lines + 1] = string.format(
' printf "source: %%s:%%d %%s\\n", $__atom_file_%d, %d, "%s"',
a.idx, we.line, escaped_text)
lines[#lines + 1] = string.format(' printf "source: %%s:%%d %%s\\n", $__atom_file_%d, %d, "%s"', a.idx, we.line, escaped_text)
lines[#lines + 1] = " end"
end
-- Fallback for words beyond the source map (shouldn't happen if nm matches).
local max_word = 0
if #a.entries > 0 then max_word = a.entries[#a.entries].pos end
lines[#lines + 1] = string.format(
' if $__word > %d', max_word)
lines[#lines + 1] = string.format(' if $__word > %d', max_word)
lines[#lines + 1] = ' printf "source: (no source-map entry for word %%d; map may be stale)\\n", $__word'
lines[#lines + 1] = " end"
lines[#lines + 1] = " set $__matched = 1"
@@ -537,19 +510,16 @@ local function append_gdb_commands(lines, matched)
-- ── stepi_inside_atom ──
-- Hardcoded one if-containment-check per atom (no loop).
-- Precompute end_addr in Lua so we don't ask gdb to evaluate `addr + words*4`
-- inside the if condition (gdb 12.1's expression evaluator chokes on the
-- `*` and emits a misleading 'function malloc' error in some gdb builds).
-- Precompute end_addr in Lua so we don't ask gdb to evaluate `addr + words*4` inside the if condition
-- (gdb 12.1's expression evaluator chokes on the `*` and emits a misleading 'function malloc' error in some gdb builds).
lines[#lines + 1] = "define stepi_inside_atom"
lines[#lines + 1] = " set $__in_atom = 0"
lines[#lines + 1] = " set $__did_step = 0"
lines[#lines + 1] = " set $__pc = (unsigned int)$pc"
for _, a in ipairs(matched) do
-- Precompute end_addr in the convenience var (single expression gdb handles).
lines[#lines + 1] = string.format(
" set $__end_%d = $__atom_addr_%d + $__atom_words_%d * 4", a.idx, a.idx, a.idx)
lines[#lines + 1] = string.format(
" if $__pc >= $__atom_addr_%d && $__pc < $__end_%d", a.idx, a.idx)
lines[#lines + 1] = string.format(" set $__end_%d = $__atom_addr_%d + $__atom_words_%d * 4", a.idx, a.idx, a.idx)
lines[#lines + 1] = string.format(" if $__pc >= $__atom_addr_%d && $__pc < $__end_%d", a.idx, a.idx)
lines[#lines + 1] = " set $__in_atom = 1"
lines[#lines + 1] = " stepi"
lines[#lines + 1] = " set $__did_step = 1"
@@ -609,7 +579,7 @@ end
--- @param ctx PassCtx
local function emit_gdb_runtime(ctx)
if not (ctx.flags and ctx.flags.gdb_runtime) then return end
local elf_path = ctx.flags.elf_path
local elf_path = ctx.flags.elf_path
if not elf_path or elf_path == "" then
io.stderr:write("[atoms_source_map] --gdb-runtime requires --elf <elf>\n")
return
@@ -678,13 +648,10 @@ end
local M = {}
--- Pass entry: emit one `<out_root>/<basename>.atoms.sourcemap.txt` per source file
--- that contains at least one `MipsAtom_(name)` / `MipsCode code_<name>` declaration.
--- Also emits `<out_root>/<basename>.atoms.provenance.txt` (Phase 3 — debug_ux):
--- per-.word provenance with `mac_X(...)` component resolution back to the
--- component's definition file:line.
--- Optionally also emit `<ctx.out_root>/gdb_tape_atoms_runtime.gdb` when
--- `ctx.flags.gdb_runtime` is true.
--- Pass entry: emit one `<out_root>/<basename>.atoms.sourcemap.txt` per source file that contains at least one `MipsAtom_(name)` / `MipsCode code_<name>` declaration.
--- Also emits `<out_root>/<basename>.atoms.provenance.txt`:
--- per-.word provenance with `mac_X(...)` component resolution back to the component's definition file:line.
--- Optionally also emit `<ctx.out_root>/gdb_tape_atoms_runtime.gdb` when `ctx.flags.gdb_runtime` is true.
--- @param ctx PassCtx
--- @return PassResult
function M.run(ctx)
@@ -702,10 +669,9 @@ function M.run(ctx)
}
end
-- Phase 3 (debug_ux): shared.components map is populated by `passes/components.lua`.
-- Used to attribute each emitted `.word` to either a component macro or the
-- enclosing atom body. If absent, all words fall through as RAW (correct
-- behavior — provenance is additive).
-- shared.components map is populated by `passes/components.lua`.
-- Used to attribute each emitted `.word` to either a component macro or the enclosing atom body.
-- If absent, all words fall through as RAW (correct behavior — provenance is additive).
local comp = (ctx.shared and ctx.shared.components) or {}
-- Always emit the canonical text form (per-source).
@@ -720,11 +686,8 @@ function M.run(ctx)
local sourcemap_path = ctx.out_root .. "/" .. basename .. ".atoms.sourcemap.txt"
local sourcemap_body = render_source_map(src, wc)
-- (2) atoms.provenance.txt — Phase 3 (debug_ux): per-.word provenance
-- with `mac_X(...)` component resolution back to the component's
-- definition file:line. Consumed by `passes/dwarf_injection.lua`
-- to synthesize `DW_TAG_inlined_subroutine` instances for
-- source-level Step Into on component invocations.
-- (2) atoms.provenance.txt — per-.word provenance with `mac_X(...)` component resolution back to the component's definition file:line.
-- Consumed by `passes/dwarf_injection.lua` to synthesize `DW_TAG_inlined_subroutine` instances for source-level Step Into on component invocations.
local prov_path = ctx.out_root .. "/" .. basename .. ".atoms.provenance.txt"
local prov_body = render_provenance(src, wc, comp)
scripts/passes/components.lua
+29 -36
View File
@@ -1,12 +1,12 @@
--- passes/components.lua — Component-macro header generator.
---
--- Reads the pre-scanned SourceScan payload (produced once upstream by `duffle.scan_source`)
--- for `MipsAtomComp_(ac_X)` and `MipsAtomComp_Proc_(ac_X, { body })` declarations, then does
--- per-source backward lookups for the function-args string (from the preceding `FI_ MipsAtom ac_X(...)`
--- function declaration) and the preceding comment block (for LSP/IntelliSense signature docs).
--- for `MipsAtomComp_(ac_X)` and `MipsAtomComp_Proc_(ac_X, { body })` declarations, then does per-source backward lookups
--- for the function-args string (from the preceding `FI_ MipsAtom ac_X(...)` function declaration)
--- and the preceding comment block (for LSP/IntelliSense signature docs).
---
--- Emits a per-directory `<dir_basename>.macs.h` containing one `#define mac_X(sig) \` macro per component
--- + `WORD_COUNT(mac_X, N)` entries for downstream offset computation.
--- Emits a per-directory `<dir_basename>.macs.h` containing one `#define mac_X(sig) \` macro per component + `WORD_COUNT(mac_X, N)`
--- entries for downstream offset computation.
---
--- **Conventions**: tabs (1/level), EmmyLua annotations, no regex,
--- Lua 5.3 compatible.
@@ -26,8 +26,7 @@
-- Bootstrap: same as entry scripts. See `ps1_meta.lua` for the rationale.
-- Bootstrap: load `scripts/duffle_paths.lua` (sets package.path + package.cpath).
-- Uses `debug.getinfo` to find this file's own directory, so it works
-- both standalone and when require'd from the orchestrator.
-- Uses `debug.getinfo` to find this file's own directory, so it works both standalone and when require'd from the orchestrator.
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source` (works both standalone + when require'd).
-- duffle_paths.lua sets package.path then returns `require("duffle")` at the bottom, so the dofile value IS the duffle module.
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
@@ -43,9 +42,9 @@ local ATOM_COMP_PROC = "MipsAtomComp_Proc_"
local MIPS_ATOM = "MipsAtom" -- prefix on the function declaration that wraps an AtomComp_Proc_
-- Component-name prefixes.
local AC_PREFIX = "ac_" -- arg to MipsAtomComp_(ac_X); the X is the atom name
local AC_PREFIX = "ac_" -- arg to MipsAtomComp_(ac_X); the X is the atom name
local AC_PREFIX_LEN = 3
local MAC_PREFIX = "mac_" -- prefix on generated macros; the rest is the atom name
local MAC_PREFIX = "mac_" -- prefix on generated macros; the rest is the atom name
local MAC_PREFIX_LEN = 4
-- ASCII byte values used in tokenization.
@@ -84,11 +83,11 @@ local GEN_SUBDIR = "gen"
--- @field warnings table[] -- {line=, msg=} entries; build-succeeds
--- @class Component
--- @field name string -- atom name (without `ac_` prefix)
--- @field body string -- brace-delimited body (without the braces)
--- @field args string|nil -- function-args string (function form only)
--- @field line integer -- source line of the declaration
--- @field comment string|nil -- preceding `/* */` or `//` comment block (signature doc)
--- @field name string -- atom name (without `ac_` prefix)
--- @field body string -- brace-delimited body (without the braces)
--- @field args string|nil -- function-args string (function form only)
--- @field line integer -- source line of the declaration
--- @field comment string|nil -- preceding `/* */` or `//` comment block (signature doc)
-- ════════════════════════════════════════════════════════════════════════════
-- Local helpers (file I/O + path normalization)
@@ -348,8 +347,7 @@ end
-- Project pre-scanned MipsAtomComp_ / MipsAtomComp_Proc_ entries into Component shape.
-- Does per-source backward lookups for args (preceding function decl) and comment (preceding comment block).
-- Carries `body_tokens` forward from scan-source so word_count_rec reads from the precomputed table
-- instead of calling duffle.tokenize_body again.
-- Carries `body_tokens` forward from scan-source so word_count_rec reads from the precomputed table instead of calling duffle.tokenize_body again.
-- @param source string -- the full source text (needed for backward lookups)
-- @param scan table -- SourceScan from duffle.scan_source
-- @return Component[]
@@ -366,7 +364,7 @@ local function project_components(source, scan)
body_tokens = a.body_tokens,
args = args,
comment = comment,
kind = a.kind, -- "comp_bare" | "comp_proc"; Phase 3 provenance emitter reads this.
kind = a.kind, -- "comp_bare" | "comp_proc"; provenance emitter reads this.
}
end
end
@@ -474,10 +472,9 @@ end
--- Compute word counts for every component in `components` in a single pass.
--- The name-lookup table + memoization cache are built ONCE (per source) instead of per-component,
--- so the cache survives across siblings and a component's recursive `mac_Y(...)` references hit memoized values
--- instead of re-walking the body.
--- so the cache survives across siblings and a component's recursive `mac_Y(...)`
--- references hit memoized values instead of re-walking the body.
--- Cycle detection (A -> B -> A) is preserved via the in-progress `-1` sentinel in `cache`.
---
--- @param components Component[]
--- @param wc table<string, integer>
--- @return table<string, integer> -- map of component name (without `mac_`) -> word count
@@ -608,8 +605,8 @@ local function header_boilerplate(src)
"// Component atoms (MipsAtomComp_(ac_*)) -> macro variants (mac_*)",
"",
-- Self-contained: define WORD_COUNT if not already defined.
-- We use the same definition here so the auto-generated entries below expand to compile-time constants whether
-- the metadata file is included first or not.
-- We use the same definition here so the auto-generated entries below expand
-- to compile-time constants whether the metadata file is included first or not.
"#ifndef WORD_COUNT",
"#define WORD_COUNT(name, count) enum { words_##name = (count) };",
"#endif",
@@ -633,7 +630,6 @@ end
--- Emit a per-source `.macs.h` header with the `mac_X` macros + `WORD_COUNT` entries.
--- Writes in BINARY mode so LF line endings are preserved (the git blob is LF; Windows text-mode would emit CRLF and break the byte-identical diff).
--- Honors `ctx.dry_run`: prints the intended path but does not write the file.
---
--- @param ctx PassCtx
--- @param src SourceFile
--- @param components Component[]
@@ -666,8 +662,7 @@ end
-- Pass entry
-- ════════════════════════════════════════════════════════════════════════════
-- (internal) Extend `ctx.shared.word_counts` with this source's component macros
-- so offsets sees them without re-reading the file.
-- (internal) Extend `ctx.shared.word_counts` with this source's component macros so offsets sees them without re-reading the file.
-- @param ctx PassCtx
-- @param components Component[]
-- @param counts table<string, integer> -- precomputed word counts (from count_all_components)
@@ -684,11 +679,9 @@ end
--- @field path string -- absolute source path of the definition
--- @field kind string -- "comp_bare" | "comp_proc"
--- (internal) Extend `ctx.shared.components` with this source's components-by-name
-- map so downstream passes (atoms_source_map, dwarf_injection) can resolve
-- `mac_X(...)` invocations back to their component definition file:line.
--- Phase 3 (debug_ux) provenance emission uses this to attribute each emitted
-- `.word` to either a component macro or the enclosing atom body.
--- (internal) Extend `ctx.shared.components` with this source's components-by-name map so downstream passes
--- (atoms_source_map, dwarf_injection) can resolve `mac_X(...)` invocations back to their component definition file:line.
--- provenance emission uses this to attribute each emitted `.word` to either a component macro or the enclosing atom body.
-- @param ctx PassCtx
-- @param src SourceFile
-- @param components Component[]
@@ -696,8 +689,8 @@ local function update_shared_components(ctx, src, components)
ctx.shared.components = ctx.shared.components or {}
local rel_path = src.path:gsub("\\", "/")
for _, c in ipairs(components) do
-- Keyed by bare name (e.g. `yield`, `load_tri_indices`). The atoms_source_map
-- pass strips the `mac_` prefix from the call site identifier before lookup.
-- Keyed by bare name (e.g. `yield`, `load_tri_indices`).
-- The atoms_source_map pass strips the `mac_` prefix from the call site identifier before lookup.
ctx.shared.components[c.name] = {
name = c.name,
line = c.line,
@@ -714,9 +707,9 @@ function M.run(ctx)
local errors = {}
local warnings = {}
-- Initialize shared component map (Phase 3 — debug_ux). The atoms_source_map
-- and dwarf_injection passes consume `ctx.shared.components` to resolve
-- `mac_X(...)` invocations back to the component's definition file:line.
-- Initialize shared component map.
-- The atoms_source_map and dwarf_injection passes consume `ctx.shared.components` to resolve `mac_X(...)`
-- invocations back to the component's definition file:line.
ctx.shared.components = ctx.shared.components or {}
for _, src in ipairs(ctx.sources) do
@@ -729,7 +722,7 @@ function M.run(ctx)
if macs_path then
outputs[#outputs + 1] = { macs_h = macs_path }
update_shared_word_counts(ctx, components, counts)
-- Phase 3 (debug_ux): share component definitions with downstream passes.
-- share component definitions with downstream passes.
-- `mac_X(...)` invocations in atom bodies resolve back to (path, line) via this map.
update_shared_components(ctx, src, components)
end
scripts/passes/dwarf_injection.lua
+223 -310
View File
File diff suppressed because it is too large Load Diff
scripts/passes/offsets.lua
+1 -2
View File
@@ -231,8 +231,7 @@ end
-- Offset computation + header generation
-- ════════════════════════════════════════════════════════════════════════════
-- Compute branch offsets as `target_word - branch_word - 1`
-- (the standard MIPS branch-immediate encoding).
-- Compute branch offsets as `target_word - branch_word - 1` (the standard MIPS branch-immediate encoding).
-- @param labels table<string, integer>
-- @param branches table[]
-- @return BranchOffset[]
scripts/passes/report.lua
+13 -13
View File
@@ -16,7 +16,7 @@
-- ════════════════════════════════════════════════════════════════════════════
-- Resolve `arg[0]` to an absolute-ish script directory so that `require("duffle")` resolves against `scripts/` regardless of CWD.
-- Note: this boilerplate is duplicated in 6 other entry scripts; a Phase-6 extraction target (`duffle.setup_package_path()`).
-- Note: this boilerplate is duplicated in 6 other entry scripts; extraction target (`duffle.setup_package_path()`).
-- Bootstrap: see `ps1_meta.lua` for the rationale.
-- Bootstrap: load `scripts/duffle_paths.lua` (sets package.path + package.cpath).
-- Uses `debug.getinfo` to find this file's own directory, so it works
@@ -61,16 +61,16 @@ local PASS_NAME = "report"
--- @field basename string -- filename without extension
--- @class PassCtx
--- @field sources SourceFile[] -- all source files in the build
--- @field metadata_path string -- path to word_count.metadata.h
--- @field shared table -- cross-pass shared state
--- @field out_root string -- output root (e.g. "build/gen")
--- @field project_root string -- project root (e.g. "code/")
--- @field upstream table<string, table> -- per-pass upstream outputs
--- @field flags table -- CLI flags + per-pass stash
--- @field flags._annot_results ModuleEntry[] -- stashed by annotation pass
--- @field dry_run boolean -- if true, compute but don't write
--- @field verbose boolean -- if true, log diagnostic info
--- @field sources SourceFile[] -- all source files in the build
--- @field metadata_path string -- path to word_count.metadata.h
--- @field shared table -- cross-pass shared state
--- @field out_root string -- output root (e.g. "build/gen")
--- @field project_root string -- project root (e.g. "code/")
--- @field upstream table<string, table> -- per-pass upstream outputs
--- @field flags table -- CLI flags + per-pass stash
--- @field flags._annot_results ModuleEntry[] -- stashed by annotation pass
--- @field dry_run boolean -- if true, compute but don't write
--- @field verbose boolean -- if true, log diagnostic info
--- @class PassResult
--- @field outputs table[] -- {kind=, path=} entries describing emit files
@@ -312,8 +312,8 @@ local function render_module_report(dir, sources, results)
macros = total_macros, errors = total_errors, warnings = total_warnings,
}
-- THE per-section dispatch. ONE loop over SECTION_RENDERERS. Each renderer writes its
-- header + content via the `add` closure (pre-bound above).
-- THE per-section dispatch. ONE loop over SECTION_RENDERERS.
-- Each renderer writes its header + content via the `add` closure (pre-bound above).
-- Adding a new section = 1 row here + 1 render_<thing>_section function.
for _, section in ipairs(SECTION_RENDERERS) do
add(section.header)
scripts/passes/scan_source.lua
+28 -31
View File
@@ -38,8 +38,8 @@ local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
--- @field macros MacroEntry[] -- #pragma mac_X tape_atom words=N + _Pragma("...")
--- @field skip_over SkipOverScan -- atom/component debug-step markers + resolved declaration associations
--- @field types table<string, RegTypeDefault> -- atom_dbg_reg_default(R_X, <type>) declarations
--- @field atom_views table<string, AtomViewEntry> -- MipsAtom_(name) -> {binds_name, reg_type_overrides, info_line}
--- @field line_of fun(pos: integer): integer -- shared LineIndex closure
--- @field atom_views table<string, AtomViewEntry> -- MipsAtom_(name) -> {binds_name, reg_type_overrides, info_line}
--- @field line_of fun(pos: integer): integer -- shared LineIndex closure
--- @class SkipOverScan
--- @field atoms table<string, SkipOverAssociation>
@@ -51,7 +51,7 @@ local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
--- @field marker_line integer
--- @field marker_pos integer
--- @field after_paren integer
--- @field args string|nil -- trimmed marker args; valid Task 13 markers use ""
--- @field args string|nil -- trimmed marker args""
--- @field has_parens boolean
--- @field pending boolean
--- @field superseded_by_marker_line integer|nil
@@ -110,15 +110,15 @@ local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
--- @class AtomEntry
--- @field line integer
--- @field name string -- atom name (for components: without ac_ prefix)
--- @field body string -- brace-delimited body (without the braces)
--- @field body_off integer -- char offset of body[1] in source
--- @field kind string -- "atom" | "comp_bare" | "comp_proc" | "raw_atom"
--- @field raw_name string -- un-stripped name (for components: with ac_ prefix)
--- @field ident_pos integer -- position of the MipsAtom_/MipsAtomComp_ ident start
--- @field after_paren integer -- position past the closing paren
--- @field args string|nil -- populated by components pass (backward lookup)
--- @field comment string|nil -- populated by components pass (backward lookup)
--- @field name string -- atom name (for components: without ac_ prefix)
--- @field body string -- brace-delimited body (without the braces)
--- @field body_off integer -- char offset of body[1] in source
--- @field kind string -- "atom" | "comp_bare" | "comp_proc" | "raw_atom"
--- @field raw_name string -- un-stripped name (for components: with ac_ prefix)
--- @field ident_pos integer -- position of the MipsAtom_/MipsAtomComp_ ident start
--- @field after_paren integer -- position past the closing paren
--- @field args string|nil -- populated by components pass (backward lookup)
--- @field comment string|nil -- populated by components pass (backward lookup)
-- ════════════════════════════════════════════════════════════════════════════
-- Local helpers (shared by per-form parsers)
@@ -133,13 +133,13 @@ local QUALIFIER_KEYWORDS = {
["internal"] = true, ["LP_"] = true, ["global"] = true, ["gkknown"] = true,
}
-- "ac_" prefix length on component names (e.g., `MipsAtomComp_(ac_X, ...)`). The components pass strips
-- this prefix to derive the macro name (e.g., `mac_X`). Single source of truth — was duplicated in two
-- branches of the pre-refactor scan_source.
-- "ac_" prefix length on component names (e.g., `MipsAtomComp_(ac_X, ...)`).
-- The components pass strips this prefix to derive the macro name (e.g., `mac_X`).
local AC_PREFIX = "ac_"
local AC_PREFIX_LEN = 3
-- Strip the "ac_" prefix from a component name. Returns the input unchanged if it doesn't start with the prefix.
-- Strip the "ac_" prefix from a component name.
-- Returns the input unchanged if it doesn't start with the prefix.
-- @param raw_name string
-- @return string
local function strip_ac_prefix(raw_name)
@@ -150,8 +150,6 @@ local function strip_ac_prefix(raw_name)
end
-- Preserve a source marker until the following declaration parser observes it.
-- Task 13 records evidence only; Task 14 diagnoses non-empty args,
-- duplicates, dangling markers, and unrelated declaration placement.
local function push_skip_over_marker(out, marker)
local markers = out.skip_over.markers
local prior = markers[#markers]
@@ -163,9 +161,9 @@ local function push_skip_over_marker(out, marker)
markers[#markers + 1] = marker
end
-- Attach the pending marker to the next declaration. The declaration form
-- disambiguates whole atoms from components; unsupported declarations retain
-- placement evidence for annotation.lua and populate neither lookup table.
-- Attach the pending marker to the next declaration.
-- The declaration form disambiguates whole atoms from components;
-- unsupported declarations retain placement evidence for annotation.lua and populate neither lookup table.
local function associate_skip_over_marker(out, target_name, target_raw_name, target_kind, declaration_line, declaration_pos)
local markers = out.skip_over.markers
local marker = markers[#markers]
@@ -193,8 +191,8 @@ local function associate_skip_over_marker(out, target_name, target_raw_name, tar
end
end
-- Read a top-level comma-delimited argument list. Mirrors split_top_level_commas
-- in shape; kept inline so scan_source can remain dependency-free.
-- Read a top-level comma-delimited argument list. Mirrors split_top_level_commas in shape;
-- kept inline so scan_source can remain dependency-free.
local function read_top_level_args(text, pos)
local args = {}
pos = duffle.skip_ws_and_cmt(text, pos)
@@ -219,8 +217,8 @@ local function read_top_level_args(text, pos)
return args
end
-- Parse a `Type*` chain (zero or more `*` separated by optional whitespace)
-- followed by the type ident. Returns (type_name, pointer_depth) or nil.
-- Parse a `Type*` chain (zero or more `*` separated by optional whitespace) followed by the type ident.
-- Returns (type_name, pointer_depth) or nil.
local function parse_type_chain(text, pos)
if pos > #text then return nil end
-- Skip leading whitespace before the type ident.
@@ -419,8 +417,8 @@ local function parse_atom_dbg_skip_over(source, pos, ident_end, line_of, out)
return parse_skip_over_marker(source, pos, ident_end, line_of, out, "atom_dbg_skip_over")
end
-- Parse `atom_dbg_reg_default(R_X, <type>...)`; the second argument may be
-- a `Type` or `Type*`/`Type**` chain. Records in `out.types[R_X]`.
-- Parse `atom_dbg_reg_default(R_X, <type>...)`;
-- the second argument may be a `Type` or `Type*`/`Type**` chain. Records in `out.types[R_X]`.
local function parse_atom_dbg_reg_default(source, pos, ident_end, line_of, out)
local open_paren = duffle.skip_ws_and_cmt(source, ident_end)
if source:sub(open_paren, open_paren) ~= "(" then return open_paren + 1 end
@@ -488,15 +486,14 @@ local function parse_mips_atom(source, pos, ident_end, line_of, out)
}
elseif raw_name and ai_overrides then
-- Record per-atom overrides even without atom_view.
out.atom_views[raw_name] = out.atom_views[raw_name]
or { atom_name = raw_name, binds_name = nil, reg_type_overrides = nil, info_line = line_of(lookahead) }
out.atom_views[raw_name] = out.atom_views[raw_name] or { atom_name = raw_name, binds_name = nil, reg_type_overrides = nil, info_line = line_of(lookahead) }
out.atom_views[raw_name].reg_type_overrides = ai_overrides
end
brace_search_pos = info_after
end
end
local brace = duffle.scan_to_char(source, "{", brace_search_pos)
local brace = duffle.scan_to_char(source, "{", brace_search_pos)
if not brace then return open_paren + 1 end
local body, after_brace = duffle.read_braces(source, brace)
@@ -525,7 +522,7 @@ local function parse_mips_atom_comp(source, pos, ident_end, line_of, out)
if source:sub(open_paren, open_paren) ~= "(" then return open_paren + 1 end
local inner, after_paren = duffle.read_parens(source, open_paren)
local raw_name = duffle.read_ident(inner, 1)
local raw_name = duffle.read_ident(inner, 1)
if not raw_name then return open_paren + 1 end
local brace = duffle.scan_to_char(source, "{", after_paren)
scripts/passes/static_analysis.lua
+58 -72
View File
@@ -9,8 +9,7 @@
--- `mac_insert_ot_tag_X` words must equal the GP0 cmd's expected packet size.
--- 5. **per-atom cycle budget** — sum each atom body's instruction latencies (per `duffle.INSTRUCTION_LATENCY`); report total.
---
--- The orchestrator (`ps1_meta.lua`) wires this module in via the
--- PASSES table:
--- The orchestrator (`ps1_meta.lua`) wires this module in via the PASSES table:
--- `["static-analysis"] = { module = "passes.static_analysis", kind = "validation", deps = {"word-counts", "components"},
--- out = { { kind = "report", path_template = "<out_root>/<basename>.static_analysis.txt" } } }`
---
@@ -37,22 +36,22 @@ local ATOM_COMP = "MipsAtomComp_"
local ATOM_COMP_PROC = "MipsAtomComp_Proc_"
-- Marker-call identifiers inside atom bodies.
local ATOM_LABEL = "atom_label"
local ATOM_OFFSET = "atom_offset"
local ATOM_INFO = "atom_info"
local ATOM_BIND = "atom_bind"
local ATOM_READS = "atom_reads"
local ATOM_WRITES = "atom_writes"
local ATOM_YIELD = "mac_yield"
local ATOM_LABEL = "atom_label"
local ATOM_OFFSET = "atom_offset"
local ATOM_INFO = "atom_info"
local ATOM_BIND = "atom_bind"
local ATOM_READS = "atom_reads"
local ATOM_WRITES = "atom_writes"
local ATOM_YIELD = "mac_yield"
local WORD_COUNT_PRAGMA = "WORD_COUNT("
-- ASCII byte values used in tokenization.
local BYTE_NEWLINE = 10
local BYTE_HASH = 35 -- '#'
local BYTE_OPEN_PAREN = 40
local BYTE_OPEN_BRACE = 123
local BYTE_OPEN_BRACK = 91
local BYTE_SEMI = 59
local BYTE_NEWLINE = 10
local BYTE_HASH = 35 -- '#'
local BYTE_OPEN_PAREN = 40
local BYTE_OPEN_BRACE = 123
local BYTE_OPEN_BRACK = 91
local BYTE_SEMI = 59
-- Per-check output paths (relative to ctx.out_root).
local OUTPUT_EXTENSION = ".static_analysis.txt"
@@ -96,23 +95,23 @@ local OUTPUT_EXTENSION = ".static_analysis.txt"
--- @field kind string -- "atom" | "comp_bare" | "comp_proc"
--- @class Token
--- @field tok string -- the raw token text (trimmed)
--- @field line integer -- source line of the token's start
--- @field tok string -- the raw token text (trimmed)
--- @field line integer -- source line of the token's start
--- @field ident string|nil -- the leading ident of the token (if any)
--- @field kind string -- "n_words" | "mac_yield" | "gte_cmdw" | "mac_format" | "mac_gte_store" | "mac_insert_ot_tag" | "atom_label" | "atom_offset" | "other"
--- @field kind string -- "n_words" | "mac_yield" | "gte_cmdw" | "mac_format" | "mac_gte_store" | "mac_insert_ot_tag" | "atom_label" | "atom_offset" | "other"
--- @class Finding
--- @field line integer -- source line of the finding
--- @field atom AtomName -- the atom this finding is for (or "")
--- @class Finding
--- @field line integer -- source line of the finding
--- @field atom AtomName -- the atom this finding is for (or "")
--- @field check CheckName -- the check identifier
--- @field kind string -- "error" | "warning" | "info"
--- @field msg string -- the finding message
--- @field kind string -- "error" | "warning" | "info"
--- @field msg string -- the finding message
--- @class AtomAnalysis
--- @field atom AtomBody
--- @field tokens Token[] -- the tokens in the atom body, annotated
--- @field findings Finding[] -- findings for this atom
--- @field total_cycles integer -- sum of token cycle costs
--- @field tokens Token[] -- the tokens in the atom body, annotated
--- @field findings Finding[] -- findings for this atom
--- @field total_cycles integer -- sum of token cycle costs
-- ════════════════════════════════════════════════════════════════════════════
-- classify_tokens — per-token classification (the plex's pre-computed data layer)
@@ -156,8 +155,8 @@ local OUTPUT_EXTENSION = ".static_analysis.txt"
--- @field o_arg2 string|nil -- second arg of O_(<a>, <b>) captures
--- @field s_arg1 string|nil -- arg of S_(<a>) captures; nil for non-S_ tokens
-- Patterns for O_(<arg1>, <arg2>) and S_(<arg>) captures. UNANCHORED — the substring can appear
-- anywhere in the token (e.g., `load_word(R_T0, R_TapePtr, O_(Binds_X, field))` matches at position ~24).
-- Patterns for O_(<arg1>, <arg2>) and S_(<arg>) captures.
-- UNANCHORED, the substring can appea anywhere in the token (e.g., `load_word(R_T0, R_TapePtr, O_(Binds_X, field))` matches at position ~24).
-- The binds_name match is deferred to check_abi_handoff (which compares tc.o_arg1 == atom.info.binds).
local O_PATTERN = "O_%(([%w_]+),%s*([%w_]+)%s*%)"
local S_PATTERN = "S_%(([%w_]+)%s*%)"
@@ -181,15 +180,15 @@ local function classify_tokens(tokens)
local is_load_word = ident == "load_word"
local is_store_word = ident == "store_word"
-- Per-check pre-computes (R3 lift). Each pre-compute eliminates one per-token regex/string-find
-- call from check_abi_handoff / check_gpu_portstore_shape.
local mac_format_shape = nil
local is_gte_store = false
local is_ot_tag = false
-- Per-check pre-computes (R3 lift).
-- Each pre-compute eliminates one per-token regex/string-find call from check_abi_handoff / check_gpu_portstore_shape.
local mac_format_shape = nil
local is_gte_store = false
local is_ot_tag = false
local writes_r_prim_cursor = false
local reads_r_tape_ptr = false
local o_arg1, o_arg2 = nil, nil
local s_arg1 = nil
local reads_r_tape_ptr = false
local o_arg1, o_arg2 = nil, nil
local s_arg1 = nil
if ident == "atom_label" then
is_atom_label = true
@@ -235,10 +234,8 @@ local function classify_tokens(tokens)
s_arg1 = s_arg1,
}
-- Advance the nop run for the NEXT token.
if nop_words > 0 then
nop_run = nop_run + nop_words
else
nop_run = 0
if nop_words > 0 then nop_run = nop_run + nop_words
else nop_run = 0
end
end
return tc
@@ -268,9 +265,9 @@ local function check_one_gte_cmdw(atom, tc_entry, ti, line_in_body, findings)
"%s at line %d uses `gte_cmdw_%s` but that macro is not in duffle.GTE_PIPELINE_LATENCY -- add a min_nops entry",
atom.name, line, variant),
}
elseif need > 0 then
elseif need > 0 then
local have = tc_entry.nop_prefix
if have < need then
if have < need then
findings[#findings + 1] = {
atom = atom.name,
line = line,
@@ -435,8 +432,8 @@ local function check_abi_handoff(atom, pipe_ctx, findings)
local found_field_set = {}
local found_advance = false
-- Reads from tc_entry fields pre-computed by classify_tokens (R3 lift). Eliminates 3 per-token
-- string-find/match calls (R_TapePtr + O_(binds_name,...) + bind_re) → 3 O(1) field reads.
-- Reads from tc_entry fields pre-computed by classify_tokens (R3 lift).
-- Eliminates 3 per-token string-find/match calls (R_TapePtr + O_(binds_name,...) + bind_re) → 3 O(1) field reads.
for tok_idx = 1, #tokens do
local tc_entry = tc[tok_idx]
-- scan: load_word(R_*, R_TapePtr, O_(<Binds_X>, <field>))
@@ -510,7 +507,7 @@ local function check_gpu_portstore_shape(atom, pipe_ctx, findings)
-- string matches (mac_format_X_color + mac_gte_store_<shape> + mac_insert_ot_tag_<shape> + R_PrimCursor)
for tok_idx = 1, #tokens do
local tc_entry = tc[tok_idx]
local shape = tc_entry.mac_format_shape
local shape = tc_entry.mac_format_shape
if shape and duffle.GP0_CMD_BY_SHAPE[shape] then
if not cmd_byte then
cmd_byte = duffle.GP0_CMD_BY_SHAPE[shape]
@@ -607,14 +604,10 @@ local function analyze_atom_paths(atom)
end
-- A token is a terminator if it's `mac_yield`.
local function is_terminator(tok_idx)
return tc[tok_idx].is_yield
end
local function is_terminator(tok_idx) return tc[tok_idx].is_yield end
-- A token is a "branch" if the classification says so.
local function is_branch(tok_idx)
return tc[tok_idx].is_branch
end
local function is_branch(tok_idx) return tc[tok_idx].is_branch end
local function successors(tok_idx)
local tok = tokens[tok_idx].tok
if is_terminator(tok_idx) then
@@ -639,9 +632,7 @@ local function analyze_atom_paths(atom)
return succ, nil
end
-- Normal token: just the next one
if tok_idx + 1 <= n then
return { tok_idx + 1 }, nil
end
if tok_idx + 1 <= n then return { tok_idx + 1 }, nil end
return {}, nil
end
@@ -696,7 +687,7 @@ local function analyze_atom_paths(atom)
-- If no paths were recorded (e.g. atom body is empty), cycles_min/max default to 0 (atom costs nothing).
if cycles_min == math.huge then cycles_min = 0 end
if cycles_max == -1 then cycles_max = 0 end
if cycles_max == -1 then cycles_max = 0 end
local unknown_list = {}
for macro_name in pairs(unknown_set) do unknown_list[#unknown_list + 1] = macro_name end
@@ -722,14 +713,9 @@ end
--- Per-source check that emits one finding per unknown macro seen
--- (deduplicated across atoms so the warning section doesn't get spammed with N copies of "macro X not in duffle.INSTRUCTION_LATENCY").
--- Reuses `analyze_atom_paths`'s per-atom unknown_macros discovery (it's the canonical place that walks tokens
--- and computes per-token cycle costs). We just sort + emit.
--- Per-atom: emit one finding per unknown macro seen, deduplicated across atoms (so the warning
--- section doesn't get spammed with N copies of "macro X not in duffle.INSTRUCTION_LATENCY").
--- Reuses `analyze_atom_paths`'s per-atom unknown_macros discovery (it's the canonical place that walks tokens
--- and computes per-token cycle costs). We just sort + emit.
--- Signature changed in Stage 1B: `(atom, pipe_ctx, findings)` — the `unknown_seen` dedup table lives on
--- `pipe_ctx` so it persists across the per-atom loop in validate().
--- Per-atom: emit one finding per unknown macro seen, deduplicated across atoms
--- (so the warning section doesn't get spammed with N copies of "macro X not in duffle.INSTRUCTION_LATENCY").
--- Reuses `analyze_atom_paths`'s per-atom unknown_macros discovery (it's the canonical place that walks tokens and computes per-token cycle costs).
local function check_per_atom_cycle_budget(atom, pipe_ctx, findings)
local p = atom.paths or {}
for _, name in ipairs(p.unknown_macros or {}) do
@@ -756,10 +742,10 @@ end
-- This is the plex pattern: the iteration is in ONE place (validate), the variation is in DATA (this table).
local CHECK_RULES = {
{ name = "gte_pipeline_fill", per_atom = check_gte_pipeline_fill },
{ name = "mac_yield_uniformity", per_atom = check_mac_yield_uniformity },
{ name = "abi_handoff", per_atom = check_abi_handoff },
{ name = "gpu_portstore_shape", per_atom = check_gpu_portstore_shape },
{ name = "gte_pipeline_fill", per_atom = check_gte_pipeline_fill },
{ name = "mac_yield_uniformity", per_atom = check_mac_yield_uniformity },
{ name = "abi_handoff", per_atom = check_abi_handoff },
{ name = "gpu_portstore_shape", per_atom = check_gpu_portstore_shape },
{ name = "per_atom_cycle_budget", per_atom = check_per_atom_cycle_budget },
}
@@ -782,8 +768,8 @@ local function validate(ctx, src)
end
-- pipe_ctx: the cross-atom shared state for the per-atom pipeline (Fleury "expose structure").
-- Pre-allocated here, mutated by each per-atom check call below. Replaces the per-check
-- local tables that used to live inside each check_* function body.
-- Pre-allocated here, mutated by each per-atom check call below.
-- Replaces the per-check local tables that used to live inside each check_* function body.
-- info_by_atom — atom_name -> atom_info (built once; check_abi_handoff reads it)
-- binds_index — Binds_X -> binds struct (built once; check_abi_handoff reads it)
-- unknown_seen — macro_name -> first atom line (accumulated across atoms; check_per_atom_cycle_budget dedups)
@@ -919,7 +905,7 @@ local function emit_module_static_analysis_txt(ctx, dir, dir_sources, atoms, fin
-- Group findings by atom (with source prefix when multi-source module)
local multi_source = #dir_sources > 1
local by_atom = {}
local by_atom = {}
for _, f in ipairs(findings) do
by_atom[f.atom] = by_atom[f.atom] or {}
by_atom[f.atom][#by_atom[f.atom] + 1] = f
@@ -1066,9 +1052,9 @@ function M.run(ctx)
local errors = {}
local warnings = {}
-- Aggregate per-DIRECTORY (per-module). One static_analysis.txt per source-directory, emitted only if the directory contains at least one atom.
-- Aggregate per-DIRECTORY (per-module).
-- One static_analysis.txt per source-directory, emitted only if the directory contains at least one atom.
-- Empty-source directories (e.g. duffle headers with no atoms) produce no report.
--
-- Group sources by `src.dir`. The first component of `dir` is the module name (e.g. "code/duffle" -> "duffle", "code/gte_hello" -> "gte_hello").
-- Output path is `<out_root>/<module_basename>.static_analysis.txt`.
local by_dir = ctx.by_dir or duffle.group_sources_by_dir(ctx.sources)
scripts/passes/word_count_eval.lua
+1 -12
View File
@@ -15,7 +15,7 @@
-- ════════════════════════════════════════════════════════════════════════════
-- Resolve `arg[0]` to an absolute-ish script directory so that `require("duffle")` resolves against `scripts/` regardless of CWD.
-- Note: this boilerplate is duplicated in 6 other entry scripts; a Phase-6 extraction target (`duffle.setup_package_path()`).
-- Note: this boilerplate is duplicated in 6 other entry scripts; extraction target (`duffle.setup_package_path()`).
-- Bootstrap: see `ps1_meta.lua` for the rationale.
-- Bootstrap: load `scripts/duffle_paths.lua` (sets package.path + package.cpath).
-- Uses `debug.getinfo` to find this file's own directory, so it works both standalone and when require'd from the orchestrator.
@@ -80,7 +80,6 @@ local M = {}
--- For most tokens (regular MIPS instructions) this returns 1.
--- For `mac_X(...)` calls, this returns the resolved word count from `wc` (recursively if needed). For `nop2` etc., returns wc[name].
--- For unknown macros, returns 1 and (optionally) warns.
---
--- @param token string -- a single token from split_top_level_commas
--- @param wc WordCounts -- the shared word-count table
--- @return integer
@@ -101,14 +100,6 @@ end
-- │ Shared utility: scan_dir │
-- └────────────────────────────────────────────────────────────────────┘
--- Recursively scan a directory for files matching a glob suffix.
---
--- The `.macs.h` files produced by the components pass always live at `<project_root>/<module>/gen/`.
--- Native walk via lfs.attributes + lfs.dir: ~2ms vs ~56ms for the prior `dir /b /s` subprocess.
---
--- @param dir string -- directory to scan (absolute or relative)
--- @param suffix string -- file pattern, e.g. "*.macs.h"
--- @return string[]
-- Cache the scan_dir result per (dir, suffix) in package.loaded.
-- The cache persists for the lifetime of the Lua process (cleared when ps1_meta.lua exits).
-- If a build removes/creates .macs.h files mid-process, the caller can invalidate by calling `M._invalidate_scan_cache()`.
@@ -116,7 +107,6 @@ local SCAN_CACHE_KEY = "__word_count_eval_scan_cache__"
--- Scan `code/` for files matching `suffix` (e.g. `*.macs.h`).
--- Native directory enumeration via lfs (~2ms). Zero subprocess spawns.
---
--- @param dir string -- project root directory
--- @param suffix string -- file pattern, e.g. "*.macs.h"
--- @return string[]
@@ -160,7 +150,6 @@ function M._invalidate_scan_cache() package.loaded[SCAN_CACHE_KEY] = nil end
--- Load metadata.h + scan for existing *.macs.h files into ctx.shared.word_counts.
--- Loading the .macs.h files is idempotent: entries from later (current-build) .macs.h files override metadata.h entries of the same name.
---
--- @param ctx PassCtx
--- @return PassResult
function M.run(ctx)
scripts/pcsx_debug_helper/autoexec.lua
+7 -11
View File
@@ -1,17 +1,15 @@
-- autoexec.lua - pcsx_debug_helper plugin entry point.
-- Packaged in scripts/pcsx_debug_helper.zip. Loaded by pcsx-redux via
-- the -archive CLI flag (see scripts/launch_pcsx_debug.ps1).
-- Packaged in scripts/pcsx_debug_helper.zip. Loaded by pcsx-redux via the -archive CLI flag (see scripts/launch_pcsx_debug.ps1).
--
-- Registers two web handlers for external CLI tools:
-- /api/v1/lua/gte - full GTE state (32 data + 32 control regs + PC)
-- /api/v1/lua/gp - GP state summary (screenshot endpoint + VRAM endpoint refs)
--
-- The GTE handler reads COP2 regs via PCSX.getRegisters().CP2D/CP2C. The
-- pcsx-redux gdb stub doesn't expose COP2, so this is the only way for
-- external tools to see GTE state.
-- The GTE handler reads COP2 regs via PCSX.getRegisters().CP2D/CP2C.
-- The pcsx-redux gdb stub doesn't expose COP2, so this is the only way for external tools to see GTE state.
--
-- The GP handler is a thin pointer: pcsx-redux's Lua API exposes only PCSX.GPU.takeScreenShot()
-- (no GPUSTAT, no GP0/GP1 command log, no display state). For richer GP state, the existing web endpoints are the practical path:
-- The GP handler is a thin pointer:
-- pcsx-redux's Lua API exposes only PCSX.GPU.takeScreenShot() (no GPUSTAT, no GP0/GP1 command log, no display state). For richer GP state, the existing web endpoints are the practical path:
-- /api/v1/state/still - PNG screenshot
-- /api/v1/gpu/vram/raw - VRAM raw bytes (1MB)
--
@@ -48,8 +46,6 @@ local function register_handlers()
end
local ok, err = pcall(register_handlers)
if ok then
print("[pcsx_debug_helper] handlers registered: gte, gp")
else
print("[pcsx_debug_helper] registration failed: " .. tostring(err))
if ok then print("[pcsx_debug_helper] handlers registered: gte, gp")
else print("[pcsx_debug_helper] registration failed: " .. tostring(err))
end
scripts/ps1_meta.lua
+19 -23
View File
@@ -1,16 +1,14 @@
--- ps1_meta.lua — Orchestrator entry point for the tape-atom metaprogram pipeline.
--- ps1_meta.lua — Orchestrator entry point for the tape-atom metaprogram.
---
--- Dispatches to pass modules under `scripts/passes/`, resolving
--- dependencies topologically (Kahn's algorithm + cycle detection).
--- Single CLI surface (`--<pass>` flags + auto-dep expansion + --dry-run).
--- Dispatches to pass modules under `scripts/passes/`, resolving dependencies topologically (Kahn's algorithm + cycle detection).
---
--- **Architecture**:
--- - **PASSES table** — declarative dep graph (data, not code).
--- - **FLAG_HANDLERS table** — per-flag CLI dispatchers (handler-map pattern; replaces an 8-way if/elseif chain).
--- - **parse_args** → **build_ctx** (just opens + reads source files; no inline scanning) → **topo_sort** → **dispatch_passes**.
--- - The first pass in the dep graph is `scan-source` (see `passes/scan_source.lua`).
--- - The first pass in the dep graph is `scan-source` (see `passes/scan_source.lua`).
--- It calls `duffle.scan_source` once per source to produce the fat `SourceScan` payload, which is attached to each `src.scan`.
--- Every other pass that reads source structure depends on `scan-source` and consumes `src.scan` as a read-only payload.
--- Every other pass that reads source structure depends on `scan-source` and consumes `src.scan` as a read-only.
---
--- **Conventions**: tabs (1/level), EmmyLua annotations, no regex,
--- Lua 5.3 compatible.
@@ -152,7 +150,7 @@ local PASSES = {
module = "passes.atoms_source_map",
kind = "header-output",
deps = {"word-counts", "components"},
desc = "Emit gen/<basename>.atoms.sourcemap.txt (per-.word C source line map for gdb debugging) AND gen/<basename>.atoms.provenance.txt (per-.word provenance; each word tagged with its call-site file:line and, when emitted by a mac_X(...) component invocation, the component's definition file:line). Consumed by passes/dwarf_injection.lua to synthesize DW_TAG_inlined_subroutine instances for source-level Step Into on component invocations (Phase 3 — debug_ux).",
desc = "Emit gen/<basename>.atoms.sourcemap.txt (per-.word C source line map for gdb debugging) AND gen/<basename>.atoms.provenance.txt (per-.word provenance; each word tagged with its call-site file:line and, when emitted by a mac_X(...) component invocation, the component's definition file:line). Consumed by passes/dwarf_injection.lua to synthesize DW_TAG_inlined_subroutine instances for source-level Step Into on component invocations.",
out = {
{ kind = "report", path_template = "<out_root>/<basename>.atoms.sourcemap.txt" },
{ kind = "report", path_template = "<out_root>/<basename>.atoms.provenance.txt" },
@@ -206,8 +204,15 @@ local PASS_FLAG_TO_NAME = {
}
local ALL_PASS_NAMES = {
"scan-source", "word-counts", "components", "annotation",
"offsets", "static-analysis", "atoms-source-map", "dwarf-injection", "report",
"scan-source",
"word-counts",
"components",
"annotation",
"offsets",
"static-analysis",
"atoms-source-map",
"dwarf-injection",
"report",
}
--- Append every pass name to args.requested_set. Used by --all and by the "default to --all if no pass flags were given" fallback.
@@ -286,8 +291,8 @@ FLAG_HANDLERS["--metadata"] = function(args, argv, arg_idx) args.metadata
FLAG_HANDLERS["--out-root"] = function(args, argv, arg_idx) args.out_root = argv[arg_idx + 1]; return arg_idx + 1 end
FLAG_HANDLERS["--project-root"] = function(args, argv, arg_idx) args.project_root = argv[arg_idx + 1]; return arg_idx + 1 end
-- Per-pass stash flags. Read by `passes/atoms_source_map.lua` to opt into the
-- post-link gdb-runtime emission. Same shape as the existing per-flag handlers:
-- Per-pass stash flags. Read by `passes/atoms_source_map.lua` to opt into the post-link gdb-runtime emission.
-- Same shape as the existing per-flag handlers:
-- mutates `args.flags` (which propagates into `ctx.flags`).
FLAG_HANDLERS["--gdb-runtime"] = function(args) args.flags = args.flags or {}; args.flags.gdb_runtime = true end
FLAG_HANDLERS["--elf"] = function(args, argv, arg_idx) args.flags = args.flags or {}; args.flags.elf_path = argv[arg_idx + 1]; return arg_idx + 1 end
@@ -300,7 +305,7 @@ end
-- G' (atom locals) is now consolidated into --dwarf-injection; no separate flag.
-- Pass-flag handler. Reads the closed-set table, expands --all, appends to requested_set. Single-statement, no nesting.
-- Pass-flag handler. Reads the closed-set table, expands --all, appends to requested_set.
FLAG_HANDLERS[PASS_FLAG_DISPATCH_KEY] = function(args, a)
local name = PASS_FLAG_TO_NAME[a]
if name == ALL_PASSES_SENTINEL then
@@ -311,7 +316,6 @@ FLAG_HANDLERS[PASS_FLAG_DISPATCH_KEY] = function(args, a)
end
--- Parse argv into a structured table. Validates against a closed enum.
---
--- @param argv string[]
--- @return ParsedArgs
local function parse_args(argv)
@@ -371,7 +375,6 @@ end
--- Build the PassCtx from parsed args. Reads each source file once at startup;
--- passes consume `src.text`, not the path (path is preserved for error reporting).
---
--- @param args ParsedArgs
--- @return PassCtx
local function build_ctx(args)
@@ -425,7 +428,6 @@ end
-- ════════════════════════════════════════════════════════════════════════════
--- Compute the dep-closure of `requested_set`: include every pass name transitively required by the requested set.
---
--- @param passes table<string, PassDescriptor>
--- @param requested_set string[]
--- @return table<string, boolean> -- set of pass names needed (including transitive deps)
@@ -461,7 +463,6 @@ local function count_entries(t)
end
--- Compute in-degrees for the Kahn sort: for each pass in `needed`, the number of its deps that are also in `needed`.
---
--- @param passes table<string, PassDescriptor>
--- @param needed table<string, boolean>
--- @return table<string, integer>
@@ -479,8 +480,7 @@ local function compute_in_degrees(passes, needed)
end
--- Seed the Kahn ready queue with passes whose in-degree is 0, sorted alphabetically for deterministic execution order.
---
--- @param in_degree table<string, integer>
-- @param in_degree table<string, integer>
--- @return string[]
local function seed_ready_queue(in_degree)
local ready = {}
@@ -518,7 +518,6 @@ end
--- Topologically sort the requested pass set, augmented with all transitive deps.
--- Detects cycles and errors out with details.
---
--- @param passes table<string, PassDescriptor>
--- @param requested_set string[]
--- @return string[] -- execution order
@@ -552,7 +551,6 @@ end
--- Render the dep graph as ASCII art. Output width capped at 78 columns.
--- Falls back to the simpler "Resolved dependency order" list only if graph width exceeds terminal width.
---
--- @param passes table<string, PassDescriptor>
--- @param requested string[] -- originally-requested passes (subset of closed)
--- @param closed string[] -- dep-closed execution order
@@ -572,8 +570,7 @@ local function render_dep_graph(passes, requested, closed)
add("")
-- Data-driven ASCII graph built from the actual PASSES table.
-- Shows the source -> scan_source -> pass chain. Each pass is
-- shown once; edges are "feeds into" arrows based on deps.
-- Shows the source -> scan_source -> pass chain. Each pass is shown once; edges are "feeds into" arrows based on deps.
add("[ps1_meta] Pass graph (read top-to-bottom; edges = 'feeds into'):")
add("")
@@ -658,7 +655,6 @@ local function report_validation_errors(pass_name, pass, result)
end
-- (internal) Run each pass in `order` in topological sequence.
--
-- @param ctx PassCtx
-- @param order string[]
-- @return boolean -- true if any validation errors were reported