Author SHA1 Message Date
ed 27a9038e0d req c11, 2026-07-26 17:36:46 -04:00
ed 8c8d2e54aa remove cruft 2026-07-26 14:40:57 -04:00
ed 80a35aa23a WIP: Better step debug on atom components, better db_skip annotation, lots of curation passes on lua.
Still don't have this thing in its final state for  the curse but its close.
2026-07-26 13:55:47 -04:00
ed f247d56c32 Debug vis ergonomics 2026-07-25 13:19:35 -04:00
ed 590ff1e2ec Curation pass: reduce nested conditional branching in some defnitions. 2026-07-25 13:00:36 -04:00
ed 653e18ee28 remove code related to dry run and dep graph rendering (ps1 meta) 2026-07-25 11:59:41 -04:00
ed ebb876fe89 report.lua: Remove redudnant section formatting/header 2026-07-25 11:25:12 -04:00
ed 1b40b16c0e Review pass. 2026-07-25 11:20:53 -04:00
ed 9ffd6592bc Better static analysis for C0 <-> C2 data race hazards. 2026-07-25 04:09:48 -04:00
ed d56adab38f branch delay slot better support.
Still reviewing. Need to see if gte is handled properly.
2026-07-23 18:35:02 -04:00
ed 08af73d0d2 Lua Metaprogram: Improvements to static analysis + others. 2026-07-23 10:18:30 -04:00
ed 67d54debfa offset corections (dwarf) 2026-07-22 18:00:09 -04:00
ed 3c25306070 fixes 2026-07-22 09:47:01 -04:00
ed c3cf05950e good enough for now 2026-07-21 22:29:22 -04:00
ed f6b4d9895e Adjustments to offset convention (don't want 1s based addresssing to mess with the spec defined encoding) 2026-07-21 20:52:13 -04:00
ed e70361b548 curation: first pass 2026-07-21 19:20:30 -04:00
ed ed3eb45b1d Fixes atom component gdb stepping. New phase/ctx annotations for atoms. Attempt at type views on registers (gdb pretty print failures).
Needs heavy curation and problably simplicication.
2026-07-18 10:29:04 -04:00
ed d7770b6e1d review pass on c code. 2026-07-15 08:56:37 -04:00
ed 137549b1c8 First pass review 2026-07-14 22:55:16 -04:00
ed 7d5b13aadb TODO: need to review snapshot 2026-07-14 12:16:00 -04:00
34 changed files with 9239 additions and 3572 deletions
+5 -4
View File
@@ -24,7 +24,7 @@
"osx": {
"gdbpath": "gdb"
},
"executable": "${workspaceRoot}/build/hello_psyq.elf",
"executable": "${workspaceRoot}/build/hello_gte.elf",
"setupCommands": [
{ "text": "set mi-async off" },
{ "text": "set remotetimeout 0" },
@@ -33,7 +33,7 @@
],
"autorun": [
"monitor reset shellhalt",
"load hello_psyq.elf",
"load hello_gte.elf",
"source scripts/gdb/gdb_tape_atoms.gdb",
"tbreak main",
"continue"
@@ -59,7 +59,7 @@
"osx": {
"gdbpath": "gdb"
},
"executable": "${workspaceRoot}/build/hello_gpu.elf",
"executable": "${workspaceRoot}/build/hello_gte.elf",
"setupCommands": [
{ "text": "set mi-async off" },
{ "text": "set remotetimeout 0" },
@@ -68,7 +68,7 @@
],
"autorun": [
"monitor reset shellhalt",
"load hello_gpu.elf",
"load hello_gte.elf",
"tbreak main",
"continue"
]
@@ -138,6 +138,7 @@
"monitor reset shellhalt",
"load build/hello_gte.dwarf-injected.elf",
"source scripts/gdb/gdb_tape_atoms.gdb",
"source build/gen/hello_gte.gdbinit",
"tbreak main",
"continue"
]
+78 -119
View File
@@ -3,34 +3,18 @@
* ============================================================================
*
* ATOM DSL: Annotation layer for tape atoms (lottes_tape.h).
* The metaprogram (scripts/passes/annotation.lua) reads source-as-written and validates:
* - atom_info(...) shape: up to three sub-calls (atom_bind(Binds_X), atom_reads(...), atom_writes(...)) in any order and are optional.
* - rbind atoms (atom_info(..., atom_bind(Binds_X), ...)) reference a real Binds_* struct declaration.
* - atom word-counts in word_counts.metadata.h match the body's actual .word count.
*
* WHAT THIS HEADER IS
* -------------------
* The metaprogram (scripts/passes/annotation.lua) reads source-as-written
* and validates:
* - atom_info(...) shape: up to three sub-calls (atom_bind(Binds_X),
* atom_reads(...), atom_writes(...)) in any order. All optional.
* (No phase token for now; phases may be reintroduced later.)
* - rbind atoms (atom_info(..., atom_bind(Binds_X), ...)) reference a
* real Binds_* struct declaration.
* - wave-context positions only reference the canonical 4-register
* set: R_PrimCursor / R_FaceCursor / R_VertBase / R_OtBase.
* - atom word-counts in word_counts.metadata.h agree with the body's
* actual .word count.
*
* WHY A PURE MACRO (atom_info, atom_bind, atom_reads, atom_writes, atom_label)
* -----------------------------------------------------------------
* Each of these expands to a C comment or to nothing. The C preprocessor
* strips them to whitespace. The metaprogram reads the literal token from
* source-as-written, NOT from the preprocessed output. This means:
* - the C compiler does no work for them (no __attribute__, no
* _Pragma, no asm side-effects)
* - they can never silently drift from the metaprogram's view
* (the metaprogram re-reads the source on every build)
* - the annotation is invisible to the linker, debugger, and IDE
* Pure macro anntation.
* ---------------
* Don't want to constraint the macro usage to some attribute placment constraint, etc, don't want ot dela with the compiler.
* atom_info, atom_bind, atom_reads, atom_writes, atom_label, atom_dbg_skip each expand to a C comment or to nothing
* (C preprocessor strips them to whitespace).
*
* ============================================================================
*
* Usage:
* MipsAtom_(cube_tri) atom_info(
* atom_reads (R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
@@ -60,30 +44,15 @@
*
* Annotation rules
* ----------------
* 1. atom_info(...) is OPTIONAL. Most atoms have no annotation.
* Atoms without atom_info are silently skipped by the metaprogram.
*
* 2. If present, atom_info takes up to three sub-calls, all
* order-independent within the arg list:
* - atom_bind(Binds_X) (optional; only for rbind atoms)
* - atom_reads(...) (optional; wave-context registers)
* - atom_writes(...) (optional; wave-context registers)
*
* 3. atom_bind(Binds_X) pins the ABI-struct shape -- the metaprogram
* cross-references Binds_X against the
* `typedef struct Binds_X { ... } Binds_X;` declaration.
*
* 4. atom_reads(...) and atom_writes(...) args are wave-context
* registers: R_PrimCursor / R_FaceCursor / R_VertBase / R_OtBase.
* Closed set. GTE / SP / DMA / I/O state is declared in source
* comments, not in atom_reads/atom_writes.
*
* 5. atom_label(name) is an anchor -- the macro is empty in C; the
* metaprogram records the marker at the current pos for offset
* calculation.
*
* 6. atom_offset(F, T) is resolved by gen/atom_offsets.h, generated
* from the atom_label markers.
* 1. atom_info(...) is OPTIONAL. Atoms without atom_info are silently skipped by the metaprogram.
* 2. If present, atom_info takes up to three sub-calls, all order-independent within the arg list:
* - atom_bind(Binds_X)
* - atom_reads(...)
* - atom_writes(...)
* 3. atom_bind(Binds_X): metaprogram cross-references Binds_X against the `typedef struct Binds_X { ... } Binds_X;` declaration.
* 4. atom_reads(...) and atom_writes(...): Used to to check if registers are used correctly in macros: R_PrimCursor / R_FaceCursor / R_VertBase / R_OtBase.
* 5. atom_label(name: Utilize with atom_offset as a target location.
* 6. atom_offset(F, T): Resolved by gen/atom_offsets.h, generated from the atom_label markers. Calculated during the offset pass of the lua metaprogram.
*/
#ifdef INTELLISENSE_DIRECTIVES
@@ -92,64 +61,74 @@
#endif
/* ============================================================================
* WAVE-CONTEXT REGISTERS -- canonical register set for the tape wave model.
*
* R_PrimCursor output pointer into the prim arena (next OT entry to write)
* R_FaceCursor input pointer into the face array (next face to consume)
* R_VertBase base pointer into the vertex arena (this wave's vertices)
* R_OtBase base pointer into the ordering table (this wave's OT slot)
*
* Closed set. If your atom needs to touch GTE / SP / DMA / other side state,
* declare it at the source level as you normally would -- but DO NOT put
* those registers in atom_reads/atom_writes.
*
* ============================================================================*/
/* ============================================================================
* atom_reads(...) / atom_writes(...) -- wave-context register list
*
* atom_reads(R_PrimCursor, R_FaceCursor)
* -> (R_PrimCursor, R_FaceCursor) // comma-evaluated, discarded
*
* The macro produces a comma-evaluated expression that the C compiler
* silently discards (it sits in an unused arg position -- the result is
* never bound). The Lua tool pattern-matches the "atom_reads(...)" /
* "atom_writes(...)" token to extract the list.
*
* You can have at most one atom_reads(...) and at most one atom_writes(...)
* in an atom_info(...) call. To declare multiple disjoint sets (rare), just
* declare the union -- the metaprogram doesn't track which reads need which
* writes at this granularity.
* atom_reads(...) / atom_writes(...)
*
* Used during the static analysis pass of the metaprogram to do
* ============================================================================*/
#define atom_reads(...) (__VA_ARGS__)
#define atom_writes(...) (__VA_ARGS__)
/* ----------------------------------------------------------------------------
* atom_reg (per-enum opt-in marker for the DWARF register-alias registry)
*
* The bare `atom_reg` token adjacent to an enum entry in mips.h / lottes_tape.h flags that alias as debug-visible for scan_source's register_alias_registry.
* The C preprocessor strips it to a comment so no runtime symbol is created; the Lua scanner reads the bare token.
* ----------------------------------------------------------------------------*/
#define atom_reg /* atom_reg: opt the preceding enum entry into the DWARF registry */
/* ============================================================================
* ATOM ANNOTATION MACROS
*
* atom_info -- single unified annotation. OPTIONAL. Most atoms have none.
*
* atom_info :
* MipsAtom_(cube_tri) atom_info(
* atom_reads (R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
* , atom_writes(R_PrimCursor, R_FaceCursor)
* ){ ... };
*
* Shape (sub-args order-independent; all optional):
* - atom_bind(Binds_X): at most one; pins the ABI-struct shape
* - atom_reads(...): at most one; comma-list of wave-context registers
* - atom_writes(...): at most one; comma-list of wave-context registers
*
* No phase token for now. The metaprogram doesn't check ordering across
* atoms -- phases (init / bind / setup / work / commit / terminate) will
* be reintroduced when ordering checks are added.
*
* The macro expands to a C comment (or to nothing). The C compiler does
* no work. The metaprogram reads the source-as-written directly.
*
*
* - atom_bind(Binds_X): metaprogram cross-references Binds_X against the `typedef struct Binds_X { ... } Binds_X;` declaration.
* - atom_reads(...): comma-list of registers
* - atom_writes(...): comma-list of registers
* ============================================================================*/
#define atom_info(...) /* atom_info(__VA_ARGS__) */
/* ----------------------------------------------------------------------------
* DEBUG SOURCE-STEP MARKER
*
* Place `atom_dbg_skip` (BARE) before a MipsAtom_, MipsAtomComp_, or MipsAtomComp_Proc_.
* The following declaration kind determines whether the marker selects a whole atom or a component inline view.
* The source scanner associates the marker with that declaration; placement diagnostics are handled by the annotation pass.
*
* Example:
* atom_dbg_skip MipsAtom_(tape_exit) { jump_reg(rret_addr), nop };
* atom_dbg_skip MipsAtomComp_(ac_yield) { ... };
* atom_dbg_skip MipsAtomComp_Proc_(ac_format_f3_color, { ... });
* ----------------------------------------------------------------------------*/
#define atom_dbg_skip /* atom_dbg_skip: skip the following atom or component source view */
/* ----------------------------------------------------------------------------
* Typed-view annotations (Registry for DWARF RR_<R_X> chain resolution)
* atom_type(<T>) -- overloaded:
* (a) enum-site default: `R_Foo = R_Tn, atom_reg atom_type(T)`
* Sets the per-alias default typed view in the register_alias_registry.
* Consumed by the DWARF chain step (e) when no per-atom atom_ctx / atom_phase / atom_type callsite provides a stronger resolution.
* (b) callsite override: `atom_reads(R_Foo atom_type(T), ...)` Overrides the per-alias default for THIS atom only.
* Last-write-wins per R_Name; conflict -> error.
* atom_ctx(<atom_name>) -- atom-info sub-call:
* Propagate another atom's atom.rbind.fields (its Binds_* typed fields) into THIS atom's typed-view resolution.
* The named atom must be an rbind atom (have `atom_bind(Binds_X)` in its `atom_info`).
* Used as the escape hatch when atom_phase is not the natural correlation.
* atom_phase(<label>) -- atom-info sub-call:
* Free-form C-identifier label for grouping atoms.
* Within a phase, the FIRST atom in source-order that owns its own atom.rbind provides
* the Binds_* field types used by all other atoms in the same phase.
* The preferred correlation mechanism; atom_ctx is the escape hatch for non-natural cases.
*
* All three expand to C comments
* (the bare-token convention matching `atom_reg` and `atom_dbg_skip`).
* The Lua scanner reads the bare tokens in source-as-written; the C preprocessor strips them.
* ----------------------------------------------------------------------------*/
#define atom_type(T) /* atom_type: associate <T> with the preceding enum entry (enum site) or this register (atom-info site) */
#define atom_ctx(atom_name) /* atom_ctx: propagate <atom_name>'s Binds_* field types into this atom's typed views */
#define atom_phase(label) /* atom_phase: tag this atom with <label> for grouped typed-view resolution */
/* ----------------------------------------------------------------------------
* atom_bind(Binds_X) -- rbind sub-call of atom_info
*
@@ -158,13 +137,7 @@
* , atom_writes(R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
* ){ ... };
*
* The Binds_X MUST be a typedef'd type (declared via
* `typedef struct Binds_X { ... } Binds_X;` somewhere in the source).
* The Lua tool cross-references this. Missing struct = error.
*
* atom_bind is a SUB-CALL of atom_info, not a standalone annotation macro.
*
* The macro expands to a C comment. The metaprogram reads source-as-written.
* The Binds_X MUST be a typedef'd type (declared via `typedef struct Binds_X { ... } Binds_X;` somewhere in the source).
* ----------------------------------------------------------------------------*/
#define atom_bind(binds_struct) /* atom_bind(binds_struct) */
@@ -177,25 +150,11 @@
*
* atom_offset(culling, bounds_chk) ← resolved by gen/.offsets.h
*
* The metaprogram generates gen/atom_offsets.h with one
* #define atom_offset__culling__bounds_chk ((target - branch_pos - 1))
* per atom_offset(F, T) call. The preprocessor then expands your call to
* the right immediate value.
*
* If gen/atom_offsets.h is stale (or atom_label(name) is undefined),
* `atom_offset__F__T` becomes an undefined macro and the C build fails.
* This catches:
* - typo in atom_label (no anchor → metaprogram doesn't emit the macro)
* - .offsets.h not regenerated after body edits
* - body edit that broke the offset math (recompile + retest picks it up
* in CPU emulator)
* The metaprogram generates gen/atom_offsets.h with one #define with the offset value per atom_offset(F, T) call.
* The preprocessor then expands the call to the right immediate value.
*
* If gen/atom_offsets.h is stale (or atom_label(name) is undefined), `atom_offset_F_T` becomes an undefined macro and the C build fails.
* ============================================================================*/
#define atom_offset(F, T) atom_offset_ ## F ## _ ## T
/* atom_label is a pure annotation for the metaprogram's offset calculations.
* The macro expands to a C comment, so the C preprocessor strips it to
* whitespace — NO instruction word is emitted in the asm. The metaprogram
* still recognises the literal `atom_label(name)` token in source and
* records the marker at the current pos. */
// atom_label is a pure annotation for the metaprogram's offset calculations.
#define atom_label(name) /* atom_label anchor: name */
+3
View File
@@ -218,3 +218,6 @@ IA_ void assert(U8 cond) { if(cond){return;} else{debug_trap(); ms_exit_process(
#endif
#pragma endregion Debug
#endif
#define GCC_OPTIMIZATION_DISABLE _Pragma("GCC push_options") _Pragma("GCC optimize(\"O0\")")
#define GCC_OPTIMIZATION_ENABLE _Pragma("GCC pop_options")
+14 -1
View File
@@ -9,6 +9,12 @@
#define WORD_COUNT(name, count) enum { words_##name = (count) };
#endif
/* atom_dbg_skip */
/* ---------------------------------------------------------------------------
* MACRO ATOM Components (Reusable Assembly Components)
* These do NOT yield. They are expanded inline inside Tape Atoms.
* ---------------------------------------------------------------------------*/
// The 'Yield' sequence for Tape Atoms (mac_yield).
#define mac_yield(...) \
load_word(R_AtomJmp, R_TapePtr, 0) \
, add_ui_self( R_TapePtr, S_(MipsCode)) \
@@ -16,6 +22,7 @@
, nop
WORD_COUNT(mac_yield, 4)
/* atom_dbg_skip */
/* Words: 3; Loads 3 S2 indices from the face array */
#define mac_load_tri_indices(...) \
load_half_u(R_T0, R_FaceCursor, 0 * S_(S2)) \
@@ -23,6 +30,7 @@ WORD_COUNT(mac_yield, 4)
, load_half_u(R_T2, R_FaceCursor, 2 * S_(S2))
WORD_COUNT(mac_load_tri_indices, 3)
/* atom_dbg_skip */
/* Words: 18; Translates indices to vertex addresses and pushes them to GTE */
#define mac_gte_load_tri_verts(...) \
shift_lleft(R_AT, R_T0, v3s2_byteoff) \
@@ -75,16 +83,19 @@ WORD_COUNT(mac_insert_ot_tag_f3, 11)
, store_word( R_AT, R_T1, O_(PolyTag,code)) /* OrderingTable[OTZ] = PrimCursor */
WORD_COUNT(mac_insert_ot_tag_g4, 11)
/* atom_dbg_skip */
#define mac_pack_color_word(off, cmd, r, g, b) \
load_upper_i(R_AT, (cmd) << 8 | (b)) \
, or_i_self( R_AT, ((g) << 8) | (r)) \
, or_i_self( R_AT, ((g) << 8) | (r)) \
, store_word( R_AT, R_PrimCursor, (off))
WORD_COUNT(mac_pack_color_word, 3)
/* atom_dbg_skip */
#define mac_format_f3_color(r, g, b) \
mac_pack_color_word(O_(Poly_F3,color), gp0_cmd_poly_f3, r, g, b)
WORD_COUNT(mac_format_f3_color, 3)
/* atom_dbg_skip */
/* Words: 3; Stores the 3 transformed (V2_S2 screen) vertices to the F3.
* PIPELINE: post-RTPT (SXY0=v0.screen, SXY1=v1.screen, SXY2=v2.screen). */
#define mac_gte_store_f3_post_rtpt(...) \
@@ -100,6 +111,7 @@ WORD_COUNT(mac_gte_store_f3_post_rtpt, 3)
, mac_pack_color_word(O_(Poly_G4,c3), 0, r3,g3,b3)
WORD_COUNT(mac_format_g4_color, 12)
/* atom_dbg_skip */
/* Words: 3; Stores the 3 transformed (V2_S2 screen) vertices of the
* G4 triangle portion to p0/p1/p2.
* PIPELINE: post-RTPT, pre-RTPS (SXY0=v0.screen, SXY1=v1.screen, SXY2=v2.screen).
@@ -114,6 +126,7 @@ WORD_COUNT(mac_format_g4_color, 12)
, gte_sw(C2_SXY2, R_PrimCursor, O_(Poly_G4,p2))
WORD_COUNT(mac_gte_store_g4_p012_post_rtpt_pre_rtps, 3)
/* atom_dbg_skip */
/* Words: 1; Stores the V3 screen coord to the G4's p3 slot.
* PIPELINE: post-RTPS (SXY2 holds v3.screen because RTPS writes its
* single-vertex result to SXY2; SXY0 still holds v0.screen from the
+37 -64
View File
@@ -1,7 +1,6 @@
/* ============================================================================
* duffle DSL Suffix Conventions
* ============================================================================
*
* Every mnemonic in this header follows the same suffix grammar:
*
* Primitive commands: gp0_cmd_poly_f3 = 0x20 (byte opcode)
@@ -26,8 +25,7 @@
* 0. Opcode IDs gp0_cmd_poly_f3 = 0x20
*
* Vendor mnemonics (gte_mtc2, gte_mfc2, etc.) are NOT in this header.
* They live in the opt-in `gp_vendor_sym.h` for users who prefer the
* PSYQ-style names.
* They live in the opt-in `gp_vendor_sym.h` for users who prefer the PSYQ-style names.
* ============================================================================ */
#ifdef INTELLISENSE_DIRECTIVES
@@ -41,15 +39,14 @@
/* ============================================================================
* Hardware MMIO Addresses
* ============================================================================
*
* PSX GPU has two 32-bit ports in the I/O register region at KSEG2
* 0x1F800000+. GP0 (offset 0x10) is the data port (commands + params).
* GP1 (offset 0x14) is the control port (status, ctrl writes).
* ============================================================================ */
/* IO base address (KSEG2 0x1F800000+ for the I/O register region).
* The 16-bit upper half `IO_BASE_ADDR_HI16` is the form used by
* tape-side macros that pin a register to hold the IO base and access
* ports via offsets — `lui $reg, 0x1F80` (1 word) then `sw $data, GPIO_PORT*_OFFSET($reg)` (1 word).
* The 16-bit upper half `IO_BASE_ADDR_HI16` is the form used by tape-side macros that pin a register
* to hold the IO base and access ports via offsets:
* `lui $reg, 0x1F80` (1 word) then `sw $data, GPIO_PORT*_OFFSET($reg)` (1 word).
* Mirrors the `IO_BASE_ADDR equ 0x1F80` + `gpio_port0 equ 0x1810` pattern from graphics_hello/gp.s. */
enum {
IO_BASE_ADDR = 0x1F800000, /* full 32-bit I/O region base */
@@ -75,12 +72,10 @@ enum {
/* ============================================================================
* GP0 command byte constants + Layer 1 (GPU bitfield shifts)
* ============================================================================
*
* 8-bit GP0 opcodes (the upper byte of a primitive's first word). These are the BYTE only.
* The layer-1 bitfield-layout constants live in the same enum block
* so the encoder can reference them by name.
* NO macro body past this point uses a raw shift or raw mask.
* Every shift/width/mask is named here, named once.
* The layer-1 bitfield-layout constants live in the same enum block so the encoder can reference them by name.
* NO macro body past this point uses a raw shift or raw mask.
* Every shift/width/mask is named here, named once.
* Mirrors the OPCODE_SHIFT / RS_SHIFT / REG_MASK convention from mips.h.
* ============================================================================ */
enum {
@@ -143,9 +138,7 @@ enum {
/* ============================================================================
* Layer 1.5 (per-field encoders) + Layer 2 (composite) + Layer 3 (semantic GP0 word builders)
* ============================================================================
*
* Layer 1.5 encoders take one field's value, mask it to its own width,
* and shift it to its own position.
* Layer 1.5 encoders take one field's value, mask it to its own width, and shift it to its own position.
* Mirrors `enc_op` / `enc_rs` / `enc_rt` in mips.h and `enc_gte_sf` / `enc_gte_mx` in gte.h.
* Layer-2 composite encoders OR the per-field encoders together; layer-3 semantic macros delegate to the composites.
* No raw shifts or magic numbers in any macro body below this point.
@@ -186,10 +179,9 @@ enum {
/* ============================================================================
* GP1 command byte constants + Layer 1 (display-mode + range + draw-area bitfield shifts)
* ============================================================================
*
* GP1 status bits are read from HW_GP1; ctrl writes use GP1 commands
* packed into 32-bit words (cmd byte in the upper 8 bits via
* `enc_gp0_cmd(cmd)` — never a raw shift).
* GP1 status bits are read from HW_GP1;
* ctrl writes use GP1 commands packed into 32-bit words
* (cmd byte in the upper 8 bits via `enc_gp0_cmd(cmd)`).
* ============================================================================ */
enum {
gp1_cmd_Reset = 0x00,
@@ -202,10 +194,9 @@ enum {
gp1_cmd_VerticalDisplayRange = 0x07,
gp1_cmd_DisplayMode = 0x08,
/* Note: GP1 only has commands 0x00..0x08.
* The state-setter commands (SetTextureWindow, * SetDrawArea*,
* SetDrawOffset, SetMaskBit) live in the GP0 enum as * 0xE1..0xE6.
* DrawArea word builders are below as GP0s * macros
* (since they emit GP0 commands). */
* The state-setter commands (SetTextureWindow, * SetDrawArea*, SetDrawOffset, SetMaskBit)
* live in the GP0 enum as * 0xE1..0xE6.
* DrawArea word builders are below as GP0s * macros (since they emit GP0 commands). */
/* ---- Display-mode payload flags (per PSX-SPX §"GP1 Display Mode").
* Bit positions match the encoder shifts below; values are the
@@ -259,8 +250,7 @@ enum {
#define enc_gp1_vrange_word(y1, y2) (enc_gp0_cmd(gp1_cmd_VerticalDisplayRange) | enc_gp1_vrange_y1(y1) | enc_gp1_vrange_y2(y2))
/* ---- Layer 2: GP0 state-setter composite encoders ----
* GP0(0xE3) SetDrawArea top-left and GP0(0xE4) SetDrawArea bottom-right
* both use the same X/Y 10-bit signed payload as GP1 DisplayRange. */
* GP0(0xE3) SetDrawArea top-left and GP0(0xE4) SetDrawArea bottom-right both use the same X/Y 10-bit signed payload as GP1 DisplayRange. */
#define enc_gp0_draw_area_tl_word(x, y) (enc_gp0_cmd(gp0_cmd_SetDrawArea_TopLeft) | enc_gp1_draw_x(x) | enc_gp1_draw_y(y))
#define enc_gp0_draw_area_br_word(x, y) (enc_gp0_cmd(gp0_cmd_SetDrawArea_BotRight) | enc_gp1_draw_x(x) | enc_gp1_draw_y(y))
@@ -282,7 +272,6 @@ enum {
/* ============================================================================
* Pre-baked GPU state words
* ============================================================================
*
* Common command words for boot-time GPU init and standard display configurations.
* ============================================================================ */
@@ -356,7 +345,6 @@ enum {
/* ============================================================================
* Primitive structs (8 polygon variants + tag)
* ============================================================================
*
* Each struct follows the GPU-documented memory layout for the corresponding primitive command.
* The PolyTag is the OT-link header; the rest of the struct is the primitive's body.
*
@@ -390,9 +378,9 @@ typedef Struct_(PolyTag) {
* No raw C-style casts. RHS values are assumed to be `U4` — caller passes a `U4` directly. */
#define set_len(tag,v) (C_(PolyTag_R,tag)->len = u4_(v))
#define set_addr(tag,v) (C_(PolyTag_R,tag)->addr = u4_(v))
/* `set_code` is no longer in the new PolyTag design — the code byte lives
* in the primitive body (e.g. `((Poly_F3*)(p))->code`), not in the tag.
* Use the typed primitive structs (Poly_F3, Poly_G4, etc.) and the `set_poly_*` setters,
/* `set_code` is no longer in the new PolyTag design — the code byte lives in the primitive body
* (e.g. `((Poly_F3*)(p))->code`), not in the tag.
* Use the typed primitive structs (Poly_F3, Poly_G4, etc.) and the `set_poly_*` setters,
* which set both the tag's length and the code. */
#define get_len(tag) C_(U4,C_(PolyTag_R,tag)->len)
#define get_addr(tag) C_(U4,C_(PolyTag_R,tag)->addr)
@@ -511,7 +499,6 @@ typedef Struct_(Poly_GT4) {
/* ============================================================================
* Texture Page (TPage) bit layout
* ============================================================================
*
* The TPage data word sent via GP0(0x2X) has:
* bits 0..3 = texture page X (4 bits, 64-px units, 0..16)
* bit 4 = texture page Y (1 bit, 64-px units, 0/1)
@@ -575,7 +562,6 @@ typedef Struct_(TexturePage) { U4 raw; };
/* ============================================================================
* CLUT (Color Look-Up Table) semantics
* ============================================================================
*
* CLUT is loaded into VRAM by sending a GP0 command whose payload is:
* bits 0..5 = Y in 16-px units (palette row)
* bits 6..14 = X in 16-px units (palette column)
@@ -608,7 +594,6 @@ enum {
/* ============================================================================
* TIM file format constants and headers
* ============================================================================
*
* TIM (Sony .TIM texture image) file structure:
* +0x00 U4 file_id (always 0x10 = TIM magic)
* +0x04 U4 version (always 0x00 for v1)
@@ -626,9 +611,8 @@ enum {
* +0x06 U2 px_height
* +0x08 ... pixel data
*
* Future?: add `tim_load_to_vram(tim_ptr, vram_addr)` that
* emits the necessary GP0 commands. Stoppped for now at the
* struct + enum level.
* Future?: add `tim_load_to_vram(tim_ptr, vram_addr)` that emits the necessary GP0 commands.
* Stoppped for now at the struct + enum level.
* ============================================================================ */
enum {
tim_file_id_magic = 0x10,
@@ -659,35 +643,24 @@ typedef Struct_(TIM_SectionHeader) {
* Tape-side GPU operations (NOT in this header)
* ============================================================================
*
* No `mac_gp0_send` or related macros live in gp.h. Rationale: the
* Lottes tape model uses OT-DMA for primitive submission, so atom bodies
* write to main RAM (the OT/primitive buffer) and to GTE state — never
* directly to the GPU ports at 0x1F801810 / 0x1F801814. See
* `mac_format_f3_color`, `mac_insert_ot_tag`, `mac_gte_store_f3` in
* lottes_tape.h for the patterns atom bodies actually use.
* No `mac_gp0_send` or related macros live in gp.h.
* Rationale: the Lottes tape model uses OT-DMA for primitive submission, so atom bodies write to main RAM (the OT/primitive buffer)
* and to GTE state — never directly to the GPU ports at 0x1F801810 / 0x1F801814.
* See `mac_format_f3_color`, `mac_insert_ot_tag`, `mac_gte_store_f3` in lottes_tape.h for the patterns atom bodies actually use.
*
* If a feature need arises requires tape-side GPU port writes (e.g. DMA-kick to
* start GPU consumption of the OT, VBlank sync via GP1 status poll),
* the right home is `lottes_tape.h` alongside the rest of the `mac_*`
* family — the encoder infrastructure is already in place:
* If a feature need arises requires tape-side GPU port writes
* (e.g. DMA-kick to start GPU consumption of the OT, VBlank sync via GP1 status poll),
* the right home is `lottes_tape.h` alongside the rest of the `mac_*` family:
* 1. The caller pins a register to hold the IO base, e.g. register U4 r_io rgcc(R_T4) = IO_BASE_ADDR;
* The compiler emits `lui R_T4, IO_BASE_ADDR_HI16` outside the atom body (in the C prologue before tape_run).
* 2. The atom body uses `store_word(R_data, R_T4, GPIO_PORT0_OFFSET)` to write to GP0, and `store_word(R_data, R_T4, GPIO_PORT1_OFFSET)`
* to write to GP1. Both are preprocessor-encodable because R_T4 is a fixed register and the GPIO_PORT*_OFFSET constants
* fit in the `sw`'s 16-bit signed offset field. No placeholder-pun, no asm constraints, no hidden register choice.
* Same pattern as the old graphics_hello/hello_gp_routines.s `reg_io_offset`/`gcmd_push` convention.
*
* 1. The caller pins a register to hold the IO base, e.g.
* register U4 r_io rgcc(R_T4) = IO_BASE_ADDR;
* The compiler emits `lui R_T4, IO_BASE_ADDR_HI16` outside the
* atom body (in the C prologue before tape_run).
*
* 2. The atom body uses `store_word(R_data, R_T4, GPIO_PORT0_OFFSET)`
* to write to GP0, and `store_word(R_data, R_T4, GPIO_PORT1_OFFSET)`
* to write to GP1. Both are preprocessor-encodable because R_T4 is
* a fixed register and the GPIO_PORT*_OFFSET constants fit in the
* `sw`'s 16-bit signed offset field. No placeholder-pun, no asm
* constraints, no hidden register choice. Same pattern as the
* old graphics_hello/hello_gp_routines.s `reg_io_offset`/`gcmd_push`
* convention.
*
* This mirrors the existing tape-side wave-context discipline: the
* caller binds the IO-base register via `rgcc()`, the macro assumes
* the binding is in effect, and the encoding falls out at preprocessor
* time. No additional GPU-domain macro layer required.
* This mirrors the existing tape-side wave-context discipline:
* the caller binds the IO-base register via `rgcc()`, the macro assumes the binding is in effect,
* and the encoding falls out at preprocessor time.
* No additional GPU-domain macro layer required.
* ============================================================================ */
#pragma endregion Tape-Side Macros
+6 -12
View File
@@ -2,10 +2,8 @@
* duffle DSL — GPU Vendor Mnemonics (opt-in)
* ============================================================================
*
* Provides the PSYQ-style CamelCase aliases for the canonical duffle GPU
* primitive setters and OT operations. The duffle snake_case names are
* primary; this header is for users who prefer the PSYQ SDK function
* names from the legacy C API.
* Provides the PSYQ-style CamelCase aliases for the duffle GPU primitive setters and OT operations.
* The duffle snake_case names are primary; this header is for users who prefer the PSYQ SDK function names from the legacy C API.
*
* USAGE: #include "duffle/gp_vendor_sym.h" // after gp.h
*
@@ -23,15 +21,11 @@
* OT operations:
* AddPrim(ot, p) -> orderingtbl_add_primitive(ot, p)
*
* The gp0_cmd_* / gp1_cmd_* byte constants are already short and
* descriptive; no vendor alias is provided for them.
*
* The vendor mnemonics are NOT registered with the duffle word-count
* metadata (word_counts.metadata.h). They expand to the duffle canonical
* macros which DO have word-count entries (the ones emitted by
* mac_format_f3_color / mac_gte_store_f3 / etc.). Verification: V13
* (objdump byte-identical) holds.
* The gp0_cmd_* / gp1_cmd_* byte constants are already short and descriptive; no vendor alias is provided for them.
*
* The vendor mnemonics are NOT registered with the duffle word-count metadata (word_counts.metadata.h).
* They expand to the duffle macros which DO have word-count entries
* (the ones emitted by mac_format_f3_color / mac_gte_store_f3 / etc.). Verification: V13 (objdump byte-identical) holds.
* ============================================================================ */
#ifdef INTELLISENSE_DIRECTIVES
+31 -44
View File
@@ -353,41 +353,34 @@ enum { _C2_TX_SUBS_ = 0
/* GTE command words for the common cases.
*
* These are pure compile-time integer constants — the C compiler
* constant-folds them into `.word` directives in .rodata. Use them
* inside `asm_inline(...)` blocks (see `gte_rtpt` below for the
* canonical idiom).
* These are pure compile-time integer constants — the C compiler constant-folds them into `.word` directives in .rodata.
* Use them inside `asm_inline(...)` blocks (see `gte_rtpt` below for the idiom).
*
* Decomposition (per the `enc_gte_<field>` definitions above):
* gte_cmdw_<name> = gte_cmd_base | enc_gte_cmd(<cmd>)
* The SF/MX/V/CV/LM fields are all zero in the common cases (standard
* rotation-matrix, no scaling factor, V0 vector, translation vector,
* no clamp), so the only varying bits are the `cmd` field.
* The SF/MX/V/CV/LM fields are all zero in the common cases
* (standard rotation-matrix, no scaling factor, V0 vector, translation vector, no clamp),
* so the only varying bits are the `cmd` field.
*
* Naming follows the file's convention: `gte_cmd_*` is the raw
* 6-bit `cmd` field id, `gte_cmdw_*` is the fully-encoded 32-bit
* instruction word ready to drop into a `.word` directive.
* Naming follows the file's convention: `gte_cmd_*` is the raw 6-bit `cmd` field id, `gte_cmdw_*`
* is the fully-encoded 32-bit instruction word ready to drop into a `.word` directive.
*
* --------------------------------------------------------------------------
* PsyQ-compatibility note (RTPS/RTPT):
* The original Sony PsyQ `inline_n.h` ships RTPT as `cop2 0x0280030` and
* RTPS as `cop2 0x0180001`. Both have `0x20` set in the upper-reserved
* region (bit 21) AND `sf=1` (bit 19) — i.e. the "no division" flag.
* Per psx-spec these bits are reserved/must-be-zero, but the real GTE
* hardware and PCSX-Redux's GTE model both IGNORE them on these two
* commands (the perspective divide happens regardless of `sf`).
* The original Sony PsyQ `inline_n.h` ships RTPT as `cop2 0x0280030` and RTPS as `cop2 0x0180001`.
* Both have `0x20` set in the upper-reserved region (bit 21) AND `sf=1` (bit 19) — i.e. the "no division" flag.
* Per psx-spec these bits are reserved/must-be-zero,
* but the real GTE hardware and PCSX-Redux's GTE model both IGNORE them on these two commands
* (the perspective divide happens regardless of `sf`).
*
* If we emit a strictly-spec-compliant word (`sf=0`, reserved bits
* clear), PCSX-Redux's GTE checks those bits more strictly than the
* silicon does and RTPT silently no-ops — the floor's screen
* coordinates come out as raw projection-of-rotation (Z never
* divided), `nclip` ends up wrong, and the triangle is culled.
* If we emit a strictly-spec-compliant word (`sf=0`, reserved bits clear),
* PCSX-Redux's GTE checks those bits more strictly than the silicon does and RTPT silently no-ops —
* the floor's screen coordinates come out as raw projection-of-rotation (Z never divided),
* `nclip` ends up wrong, and the triangle is culled.
*
* So for RTPS and RTPT we OR-in the `0x28` "PsyQ compat" pattern to
* match the working bit pattern everyone has shipped for 25 years.
* NCLIP/OP/MVMVA stay spec-clean — their reserved bits really are
* zero in the original PsyQ source.
* So for RTPS and RTPT we OR-in the `0x28` "PsyQ compat" pattern to match the working bit pattern everyone has shipped for 25 years.
* NCLIP/OP/MVMVA stay spec-clean — their reserved bits really are zero in the original PsyQ source.
* --------------------------------------------------------------------------
*/
#define gte_cmdw_psyq_compat (1u << 21 | enc_gte_sf(gte_sf_integer))
@@ -424,9 +417,9 @@ enum { _C2_TX_SUBS_ = 0
* @details Loads values from an SVECTOR struct to GTE data registers C2_VXY0
* (XY at offset 0) and C2_VZ0 (Z at offset 4) using `lwc2`.
*
* Uses string-style GCC inline asm with `%0` substitution because the
* base register `r0` is a runtime GPR chosen by the compiler — it cannot
* be encoded into a static `.word` constant.
* Uses string-style GCC inline asm with `%0` substitution because the
* base register `r0` is a runtime GPR chosen by the compiler.
* It cannot be encoded into a static `.word` constant.
*
* Usage:
* asm_gte_load_v0(svector_ptr);
@@ -458,26 +451,21 @@ enum {
/* gte_load_vN(r_ptr, base) — placeholder-punned lwc2 loaders
*
* Emits `.word` constants encoding `lwc2 $N, off(<base>)` for the chosen
* GTE vector register, where `<base>` is the GPR number you pass in
* Emits `.word` constants encoding `lwc2 $N, off(<base>)` for the chosen GTE vector register, where `<base>` is the GPR number you pass in
* (typically one of R_T4..R_T9 for the standard "3-pointer" pattern).
*
* The caller MUST bind `r_ptr` to that same GPR via a register variable:
* register V3_S2* p_in_12 __asm__("$12") = my_ptr;
* gte_load_v0(p_in_12, R_T4); // R_T4 = 12, base is $12
*
* Then `"r"(r_ptr)` inside the asm binds to $12 (the only register
* `p_in_12` can live in), which is exactly the register the .word
* constants expect. A `"$12"` clobber would conflict with the
* register-variable binding ("asm specifier for variable conflicts
* with asm clobber list"), so we omit it. The other ABI-clobbers
* ($2/$8/$9/$31) stay because the GTE instructions don't touch
* caller-saved GPRs but the kernel does treat them as volatile.
* Then `"r"(r_ptr)` inside the asm binds to $12 (the only register `p_in_12` can live in),
* which is exactly the register the .word constants expect. A `"$12"` clobber would conflict with the register-variable binding
* ("asm specifier for variable conflicts with asm clobber list"), so we omit it.
* The other ABI-clobbers ($2/$8/$9/$31) stay because the GTE instructions don't touch caller-saved GPRs but the kernel does treat them as volatile.
*
* WHICH REGISTER TO PICK
* ----------------------
* Any caller-saved GPR is safe. Recommended default for an RTPT-style
* 3-pointer pipeline:
* Any caller-saved GPR is safe. Recommended default for an RTPT-style 3-pointer pipeline:
* gte_load_v0(p0, R_T4); // $12
* gte_load_v1(p1, R_T5); // $13
* gte_load_v2(p2, R_T6); // $14
@@ -490,8 +478,7 @@ enum {
* clobbers section : "$2", "$8", ..., "memory" (from asm_clobber)
* 3 colons total, GCC-legal. No string-syntax mnemonics in the .word body.
*
* The `asm_clobber(...)` helper from gcc_asm.h prepends the colon that
* starts the clobbers section. */
* The `asm_clobber(...)` helper from gcc_asm.h prepends the colon that starts the clobbers section. */
#define gte_load_v0(r_ptr, base) asm volatile( \
asm_words( gte_lw_v0_xy(base), gte_lw_v0_z(base) ) \
asm_rpins, r_use(r_ptr) \
@@ -510,11 +497,11 @@ enum {
asm_clobber: rlit(R_V0), rlit(R_T0), rlit(R_T1), rlit(R_RA), clb_mem_drain \
)
/* gte_load_v0v1v2(p0, p1, p2, b0, b1, b2) — the canonical prelude to gte_cmd_rtpt.
/* gte_load_v0v1v2(p0, p1, p2, b0, b1, b2) — prelude to gte_cmd_rtpt.
*
* Loads all three GTE input vectors (6 words) from three separate pointers,
* one per GTE vector register, each loaded from its own base GPR. Caller
* must bind each `pN` to `bN` via a register variable.
* one per GTE vector register, each loaded from its own base GPR.
* Caller must bind each `pN` to `bN` via a register variable.
*
* register V3_S2* p0 rgcc(R_T4) = verts[0].ptr; // → __asm__("$12")
* register V3_S2* p1 rgcc(R_T5) = verts[1].ptr; // → __asm__("$13")
+2 -4
View File
@@ -2,10 +2,8 @@
* duffle DSL — GTE Vendor Mnemonics (opt-in)
* ============================================================================
*
* Provides the textbook MIPS assembly mnemonics for the GTE/COP2
* instructions as thin aliases to the canonical duffle macros in gte.h.
* The duffle names are primary; this header is for users who prefer
* the textbook mnemonics.
* Provides the textbook MIPS assembly mnemonics for the GTE/COP2 instructions as thin aliases to the duffle macros in gte.h.
* The duffle names are primary; this header is for users who prefer the textbook mnemonics.
*
* USAGE: #include "duffle/gte_vendor_sym.h" // after gte.h
*
+24 -31
View File
@@ -16,14 +16,12 @@ typedef Slice_MipsCode MipsAtom;
#define MipsAtom_(sym) MipsCode sym [] align_(4) =
// Bare form: file-scope declaration with hardcoded body.
// Used for components with no args (e.g., ac_load_tri_indices) or identifier-args (hardcoded register names).
// MipsAtomComp_(ac_X) { body }
// expands to:
// MipsCode ac_X[] align_(4) = { body };
#define MipsAtomComp_(sym) MipsCode sym [] align_(4) =
// Function form: function-body block that returns a MipsAtom slice.
// Used for components with value-args (e.g., ac_format_f3_color).
// FI_ MipsAtom ac_X(args) MipsAtomComp_Proc_(ac_X, { body })
// expands to:
@@ -34,16 +32,16 @@ typedef Slice_MipsCode MipsAtom;
/* Register aliases */
enum {
R_AtomJmp = R_T9,
R_TapePtr = R_T8, /* The Instruction Stream Pointer */
R_InCursor = R_T4, /* Input data cursor */
R_AtomJmp = R_T9 atom_reg, /* debug-visible; tape yield handshake scratch */
R_TapePtr = R_T8 atom_reg, /* The Instruction Stream Pointer */
R_InCursor = R_T4,
R_PrimCursor = R_T7, /* VRAM output cursor (primitive buffer) */
R_FaceCursor = R_T4, /* Input data cursor (indices/faces) */
R_VertBase = R_T5, /* Base address of the vertex array */
R_OtBase = R_T6, /* Base address of the Ordering Table */
R_PrimCursor = R_T7 atom_reg atom_type(U4 *), /* VRAM output cursor (primitive buffer) */
R_FaceCursor = R_T4 atom_reg atom_type(V4_S2 *), /* Cube face-index cursor (V4_S2*); floor context switches to V3_S2* via atom_phase */
R_VertBase = R_T5 atom_reg atom_type(V3_S2 *), /* Base address of the vertex array */
R_OtBase = R_T6 atom_reg atom_type(U4 *), /* Base address of the Ordering Table */
/* Stringification codes for the GCC inline assembler clobber lists */
/* Stringification codes for the GCC inline assembler clobber lists. */
#define R_TapePtr_Code R_T8_Code
#define R_InCursor_Code R_T4_Code
@@ -59,10 +57,10 @@ enum {
* ---------------------------------------------------------------------------*/
/* The 'Exit' Atom */
MipsAtom_(tape_exit) { jump_reg(rret_addr), nop };
atom_dbg_skip MipsAtom_(tape_exit) { jump_reg(rret_addr), nop };
/* Generalized Tape Engine Runner */
FI_ void tape_run(Slice_U4 tape) { register U4* tp rgcc(R_TapePtr) = tape.ptr; asm volatile(
NI_ void tape_run(Slice_MipsCode tape) { register U4* tp rgcc(R_TapePtr) = u4_r(tape.ptr); asm volatile(
asm_words(
add_ui( R_SP, R_SP, -MipsStackAlignment) /* Allocate stack space */
, store_word( R_RA, R_SP, 0) /* Safely backup $ra to the stack */
@@ -93,8 +91,8 @@ FI_ TapeBuilder tb_make(Slice mem) { return (TapeBuilder){ mem.ptr, mem.len, 0 }
FI_ void tb_emit(TapeBuilder* tb, MipsCode* atom) { u4_r(tb->ptr)[tb->used] = u4_(atom); ++ tb->used; }
FI_ void tb_data(TapeBuilder* tb, U4 data) { u4_r(tb->ptr)[tb->used] = u4_(data); ++ tb->used; }
FI_ Slice_U4 tb_end (TapeBuilder* tb) { tb_emit(tb,tape_exit); return (Slice_U4){ C_(U4*,tb->ptr), tb->used }; }
FI_ Slice_U4 tb_slice(TapeBuilder tb) { return (Slice_U4){ C_(U4*,tb.ptr), tb.used }; }
FI_ Slice_MipsCode tb_end (TapeBuilder* tb) { tb_emit(tb,tape_exit); return (Slice_MipsCode){ C_(U4*,tb->ptr), tb->used }; }
FI_ Slice_MipsCode tb_slice(TapeBuilder tb) { return (Slice_MipsCode){ C_(U4*,tb.ptr), tb.used }; }
#define tb_scope(tb) for(U4 tbs_once=0;tbs_once==0;++tbs_once,tb_emit(tb,tape_exit))
#pragma endregion Tape Drive
@@ -106,22 +104,21 @@ FI_ Slice_U4 tb_slice(TapeBuilder tb) { return (Sli
* ---------------------------------------------------------------------------*/
// The 'Yield' sequence for Tape Atoms (mac_yield).
MipsAtomComp_(ac_yield) {
atom_dbg_skip MipsAtomComp_(ac_yield) {
load_word(R_AtomJmp, R_TapePtr, 0),
add_ui_self( R_TapePtr, S_(MipsCode)),
jump_reg( R_AtomJmp),
nop,
jump_reg( R_AtomJmp), nop,
};
/* Words: 3; Loads 3 S2 indices from the face array */
MipsAtomComp_(ac_load_tri_indices) {
atom_dbg_skip MipsAtomComp_(ac_load_tri_indices) {
load_half_u(R_T0, R_FaceCursor, 0 * S_(S2)),
load_half_u(R_T1, R_FaceCursor, 1 * S_(S2)),
load_half_u(R_T2, R_FaceCursor, 2 * S_(S2)),
};
/* Words: 18; Translates indices to vertex addresses and pushes them to GTE */
MipsAtomComp_(ac_gte_load_tri_verts) {
atom_dbg_skip MipsAtomComp_(ac_gte_load_tri_verts) {
shift_lleft(R_AT, R_T0, v3s2_byteoff), add_u_self(R_AT, R_VertBase), load_word(R_V0, R_AT, O_(V3_S2,x)), load_word(R_V1, R_AT, O_(V3_S2,z)), gte_mv_to_data_r(R_V0, C2_VXY0), gte_mv_to_data_r(R_V1, C2_VZ0),
shift_lleft(R_AT, R_T1, v3s2_byteoff), add_u_self(R_AT, R_VertBase), load_word(R_V0, R_AT, O_(V3_S2,x)), load_word(R_V1, R_AT, O_(V3_S2,z)), gte_mv_to_data_r(R_V0, C2_VXY1), gte_mv_to_data_r(R_V1, C2_VZ1),
shift_lleft(R_AT, R_T2, v3s2_byteoff), add_u_self(R_AT, R_VertBase), load_word(R_V0, R_AT, O_(V3_S2,x)), load_word(R_V1, R_AT, O_(V3_S2,z)), gte_mv_to_data_r(R_V0, C2_VXY2), gte_mv_to_data_r(R_V1, C2_VZ2),
@@ -160,21 +157,20 @@ MipsAtomComp_(ac_insert_ot_tag_g4) {
/* Words: 3; Emits one (cmd|color) word to R_PrimCursor at the given
* byte offset. Internal helper used by the *_format_*_color macros. */
FI_ MipsAtom ac_pack_color_word(U4 off, U4 cmd, U1 r, U1 g, U1 b)
MipsAtomComp_Proc_(ac_pack_color_word, {
atom_dbg_skip MipsAtomComp_Proc_(ac_pack_color_word, {
load_upper_i(R_AT, (cmd) << 8 | (b)),
or_i_self( R_AT, ((g) << 8) | (r)),
or_i_self( R_AT, ((g) << 8) | (r)),
store_word( R_AT, R_PrimCursor, (off)),
})
/* Words: 3; Emits the F3 command+color word (cmd byte | BLUE | GREEN | RED)
* Args: _r, _g, _b are 8-bit RGB byte values (not raw 16-bit fields).
* Migrated from hello_gte_tape.c; takes RGB form per the Phase 3 convention. */
* Args: _r, _g, _b are 8-bit RGB byte values (not raw 16-bit fields). */
FI_ MipsAtom ac_format_f3_color(U1 r, U1 g, U1 b)
MipsAtomComp_Proc_(ac_format_f3_color, { mac_pack_color_word(O_(Poly_F3,color), gp0_cmd_poly_f3, r, g, b) })
atom_dbg_skip MipsAtomComp_Proc_(ac_format_f3_color, { mac_pack_color_word(O_(Poly_F3,color), gp0_cmd_poly_f3, r, g, b) })
/* Words: 3; Stores the 3 transformed (V2_S2 screen) vertices to the F3.
* PIPELINE: post-RTPT (SXY0=v0.screen, SXY1=v1.screen, SXY2=v2.screen). */
MipsAtomComp_(ac_gte_store_f3_post_rtpt) {
atom_dbg_skip MipsAtomComp_(ac_gte_store_f3_post_rtpt) {
gte_sw(C2_SXY0, R_PrimCursor, O_(Poly_F3,p0)),
gte_sw(C2_SXY1, R_PrimCursor, O_(Poly_F3,p1)),
gte_sw(C2_SXY2, R_PrimCursor, O_(Poly_F3,p2)),
@@ -202,7 +198,7 @@ MipsAtomComp_Proc_(ac_format_g4_color, {
* three registers aligned with v0/v1/v2 you must store before RTPS).
* The macro name declares the pipeline position; check #6 (GTE state-
* machine validation) verifies the call site matches the declaration. */
MipsAtomComp_(ac_gte_store_g4_p012_post_rtpt_pre_rtps) {
atom_dbg_skip MipsAtomComp_(ac_gte_store_g4_p012_post_rtpt_pre_rtps) {
gte_sw(C2_SXY0, R_PrimCursor, O_(Poly_G4,p0)),
gte_sw(C2_SXY1, R_PrimCursor, O_(Poly_G4,p1)),
gte_sw(C2_SXY2, R_PrimCursor, O_(Poly_G4,p2)),
@@ -214,7 +210,7 @@ MipsAtomComp_(ac_gte_store_g4_p012_post_rtpt_pre_rtps) {
* earlier RTPT — DO NOT read SXY0 here, that's the bug this name
* prevents).
*/
MipsAtomComp_(ac_gte_store_g4_p3_post_rtps) { gte_sw(C2_SXY2, R_PrimCursor, O_(Poly_G4,p3)) };
atom_dbg_skip MipsAtomComp_(ac_gte_store_g4_p3_post_rtps) { gte_sw(C2_SXY2, R_PrimCursor, O_(Poly_G4,p3)) };
#pragma endregion Macro Atom Components
@@ -268,8 +264,7 @@ internal MipsAtom_(mips_flush_icache) {
store_word(rret_addr, rstack_ptr, S_(U4)), // sw $ra, 4($sp)
add_ui(rret_0, rdiscard, bios_flushcache), // addiu $a0, $0, 0x44
add_ui(rtmp_0, rdiscard, bios_table_addr), // addiu $t0, $0, 0xA0
jump_link(rtmp_0, rret_addr), // jalr $t0, $ra
nop, // BD slot
jump_link(rtmp_0, rret_addr), nop, // jalr $t0, $ra, BD slot
load_word(rret_addr, rstack_ptr, S_(U4)), // lw $ra, 4($sp)
jump_reg(rret_addr), // jr $ra
add_ui(rstack_ptr, rstack_ptr, MipsStackAlignment), // sp += 8 (BD)
@@ -299,7 +294,6 @@ internal MipsAtom_(set_gte_world) atom_info(
/* DIAGNOSTIC 1: Pure tape loop test */
internal MipsAtom_(diag_yield) { mac_yield() };
// TODO(Ed): Reduce magic numbers/offsets
/* DIAGNOSTIC 2: Pure memory test (No GTE). Draws a fixed cyan triangle. */
internal MipsAtom_(diag_color) {
store_word( R_0, R_T7, 0),
@@ -328,7 +322,6 @@ internal MipsAtom_(diag_color) {
mac_yield()
};
// TODO(Ed): Reduce magic numbers/offsets
/* DIAGNOSTIC 3: Pure GTE test (No Memory Writes) */
internal MipsAtom_(diag_gte) {
/* Load 3 indices */
+1 -1
View File
@@ -103,7 +103,7 @@ FI_ void farena_init(FArena_R arena, Slice mem) { assert(arena != nullptr);
arena->used = 0;
}
FI_ FArena farena_make(Slice mem) { FArena a; farena_init(& a, mem); return a; }
I_ Slice farena_push(FArena_R arena, U4 amount, Opt_farena o) {
I_ Slice farena_push(FArena_R arena, U4 amount, Opt_farena o) {
if (amount == 0) { return (Slice){}; }
U4 desired = amount * (o.type_width == 0 ? 1 : o.type_width);
U4 to_commit = align_pow2(desired, o.alignment ? o.alignment : MEM_ALIGNMENT_DEFAULT);
+73 -112
View File
@@ -1,38 +1,28 @@
/* ============================================================================
* duffle DSL Suffix Conventions
* ============================================================================
*
* Every mnemonic in this header follows the same suffix grammar:
*
* _i Immediate value (16-bit constant operand). Combine with
* _u or _s (single-letter modifier + type combined): add_ui,
* add_si. Examples: add_ui, add_si, and_i, or_i, xor_i,
* load_upper_i. and_i is sign-agnostic (andi zero-extends).
* load_upper_i is a unique verb; _i is the immediate marker,
* not a modifier+type combination.
*
* _u Unsigned (no-overflow, no-sign-extension). R-type
* arithmetic examples: add_u, sub_u, mult_u, div_u. I-type
* (combined with _i): add_ui.
*
* _s Signed (overflow-traps, sign-extends). R-type: add_s,
* sub_s, mult_s, div_s, set_lt_s. I-type (combined with _i):
* add_si.
* _i: Immediate value (16-bit constant operand).
* Combine with _u or _s (single-letter modifier + type combined): add_ui, add_si.
* Examples: add_ui, add_si, and_i, or_i, xor_i, load_upper_i. and_i is sign-agnostic (andi zero-extends).
* load_upper_i is a unique verb; _i is the immediate marker, not a modifier+type combination.
* _u: Unsigned (no-overflow, no-sign-extension).
* R-type arithmetic examples: add_u, sub_u, mult_u, div_u. I-type (combined with _i): add_ui.
* _s: Signed (overflow-traps, sign-extends).
* R-type: add_s, sub_s, mult_s, div_s, set_lt_s. I-type (combined with _i): add_si.
*
* --- Shift family (R-type): verb-modifier-direction ---
* The shift macros use `shift_<modifier><direction>`. Modifier is
* the single letter `l` (logical) or `a` (arithmetic). Direction
* is the word `left` or `right`. Combined: `_lleft`, `_lright`,
* `_aright`. Examples: shift_lleft( rd, rt, shamt) (= sll)
* shift_lright(rd, rt, shamt) (= srl)
* shift_aright(rd, rt, shamt) (= sra)
* (no `_aleft`; MIPS has no `sla` — arithmetic-left is bit-identical
* to logical-left, so use shift_lleft for that case)
* The shift macros use `shift_<modifier><direction>`.
* Modifier is the single letter `l` (logical) or `a` (arithmetic).
* Direction is the word `left` or `right`. Combined: `_lleft`, `_lright`, `_aright`.
* Examples: shift_lleft( rd, rt, shamt) (= sll)
* shift_lright(rd, rt, shamt) (= srl)
* shift_aright(rd, rt, shamt) (= sra)
* (no `_aleft`; MIPS has no `sla` — arithmetic-left is bit-identical to logical-left, so use shift_lleft for that case)
*
* --- Jump/Call family ---
* Simple jumps keep the original short names: jump (j), jump_reg
* (jr), jump_link (jalr rs, rd). The jump-and-link-to variants
* (jal, jalr rs with default $ra) get the `call_` verb instead:
* Simple jumps keep the original short names: jump (j), jump_reg (jr), jump_link (jalr rs, rd).
* The jump-and-link-to variants (jal, jalr rs with default $ra) get the `call_` verb instead:
* call_addr (jal), call_reg (jalr rs, default $ra).
* Examples: jump(off) (= j)
* jump_reg(rs) (= jr)
@@ -40,32 +30,22 @@
* call_reg(rs) (= jalr rs, default $ra)
* call_addr(off) (= jal)
*
* _r Register marker — used only when the register type needs
* disambiguation (e.g., GTE data register vs control
* register). NOT used in plain R-type arithmetic (the
* R-type is implicit). Examples: gte_mv_to_data_r,
* gte_mv_to_ctrl_r.
* _r: Register marker — used only when the register type needs disambiguation (e.g., GTE data register vs control register).
* NOT used in plain R-type arithmetic (the R-type is implicit). Examples: gte_mv_to_data_r, gte_mv_to_ctrl_r.
* _self: Destination equals one source operand.
* Examples: add_ui_self (I-type, to self), add_u_self (R-type, to self).
* _mv_to_: Direction: data flows into X.
* Example: gte_mv_to_data_r, gte_mv_to_ctrl_r.
* _mv_from_: Direction: data flows out of X.
* Example: gte_mv_from_data_r, gte_mv_from_ctrl_r.
* _str: String-form — emits inline-asm string instead of `.word`.
* Example: gte_rtpt_asm_str.
* _2w / _1w: Word count of the emitted sequence.
* Example: load_imm_2w.
*
* _self Destination equals one source operand.
* Examples: add_ui_self (I-type, to self),
* add_u_self (R-type, to self).
*
* _mv_to_ Direction: data flows into X.
* Example: gte_mv_to_data_r, gte_mv_to_ctrl_r.
*
* _mv_from_ Direction: data flows out of X.
* Example: gte_mv_from_data_r, gte_mv_from_ctrl_r.
*
* _str String-form — emits inline-asm string instead of `.word`.
* Example: gte_rtpt_asm_str.
*
* _2w / _1w Word count of the emitted sequence.
* Example: load_imm_2w.
*
* _cop2 RESERVED — DO NOT USE in macro names. The `gte_` namespace
* prefix already implies coprocessor 2. Use `c2` only in:
* (a) integer opcode enums (op_lwc2 = 0x32, op_swc2 = 0x3A)
* (b) vendor-mnemonic macro aliases (gte_mtc2, gte_mfc2)
* _cop2: RESERVED — DO NOT USE in macro names. The `gte_` namespace prefix already implies coprocessor 2. Use `c2` only in:
* (a) integer opcode enums (op_lwc2 = 0x32, op_swc2 = 0x3A)
* (b) vendor-mnemonic macro aliases (gte_mtc2, gte_mfc2)
*
* Primitive commands: gp0_cmd_poly_f3 = 0x20 (byte opcode)
* Packed 32-bit cmd: gp0_word_poly_f3(r, g, b) (32-bit, shifted)
@@ -80,9 +60,8 @@
* gte_lw_v0_xy(base) (gte + lw + v0 + xy)
* load_upper_i (load-upper + immediate, unique verb)
*
* Vendor mnemonics (sll, srl, sra, jr, j, jal, jalr) are NOT in this
* header. They live in the opt-in `mips_vendor_sym.h` for users who
* prefer the textbook MIPS assembly mnemonics.
* Vendor mnemonics (sll, srl, sra, jr, j, jal, jalr) are NOT in this header.
* They live in the opt-in `mips_vendor_sym.h` for users who prefer the textbook MIPS assembly mnemonics.
* ============================================================================ */
#ifdef INTELLISENSE_DIRECTIVES
@@ -98,19 +77,17 @@ enum {
/* ============================================================================
* REGISTER INTEGER IDS (preprocessor-visible)
* ============================================================================
* Every R_* enum below has a parallel R_*_Code `#define` so that the
* preprocessor can stringify the integer (e.g. for asm clobber lists and
* register-variable declarations via `rgcc(R_X)`). The enum value is
* bound to the `#define` so the two forms cannot drift apart.
* Every R_* enum below has a parallel R_*_Code `#define` so that the preprocessor can stringify the integer
* (e.g. for asm clobber lists and register-variable declarations via `rgcc(R_X)`).
* The enum value is bound to the `#define` so the two forms cannot drift apart.
*
* Only registers that get stringified need a `_Code` form; the rest are
* plain enum values. If you need to add a new one, follow the pattern:
* Only registers that get stringified need a `_Code` form; the rest are plain enum values.
* If you need to add a new one, follow the pattern:
* #define R_T7_Code 15
* R_T7 = R_T7_Code, // in the enum
* R_T7 = R_T7_Code, // in the enum
*
* User code should always reference the enum form (`R_T4`) at arithmetic
* sites and let `rlit(R_T4_Code)` / `rgcc(R_T4)` handle the stringify
* cases — never write the bare number `12`.
* User code should always reference the enum form (`R_T4`) at arithmetic sites and let
* `rlit(R_T4_Code)` / `rgcc(R_T4)` handle the stringify cases — never write the bare number `12`.
* ============================================================================ */
#define R_0_Code 0
#define R_AT_Code 1
@@ -225,7 +202,6 @@ enum {
/* 2F: N/A */
// , op_lwc0
// , op_load_addr = op_la
// , op_load_imm = op_li
, op_jump = op_j
@@ -327,15 +303,15 @@ enum { _BitOffsets = 0
* Argument order matches the MIPS assembly syntax:
* dest-first, then source operands, then immediate last.
*
* load_word(rt, base, off) → lw rt, off(base)
* store_word(rt, base, off) → sw rt, off(base)
* add_ui(rt, rs, imm) → addiu rt, rs, imm
* shift_lleft(rd, rt, shamt) → sll rd, rt, shamt
* shift_lright(rd, rt, shamt) → srl rd, rt, shamt
* shift_aright(rd, rt, shamt) → sra rd, rt, shamt
* jump_reg(rs) → jr rs
* jump_link(rs, rd) → jalr rs (link in rd, default $ra)
* nop → sll $0, $0, 0
* load_word(rt, base, off) → lw rt, off(base)
* store_word(rt, base, off) → sw rt, off(base)
* add_ui(rt, rs, imm) → addiu rt, rs, imm
* shift_lleft(rd, rt, shamt) → sll rd, rt, shamt
* shift_lright(rd, rt, shamt) → srl rd, rt, shamt
* shift_aright(rd, rt, shamt) → sra rd, rt, shamt
* jump_reg(rs) → jr rs
* jump_link(rs, rd) → jalr rs (link in rd, default $ra)
* nop → sll $0, $0, 0
*/
#define load_word(rt, base, off) enc_i(op_lw, (base), (rt), (off))
#define load_byte(rt, base, off) enc_i(op_lb, (base), (rt), (off))
@@ -404,12 +380,9 @@ enum { _BitOffsets = 0
* mult_s / mult_u → mult / multu (writes HI/LO; result in LO)
* div_s / div_u → div / divu (LO = quot, HI = rem)
*
* NOTE: dsl.h defines `add_s`/`sub_s`/`mut_s`/`gt_s`/etc. as
* _Generic-based signed integer-arithmetic helpers for U1/U2/U4. Those
* live in a different conceptual layer (generic arithmetic on DSL
* types) and would collide with the instruction encoders here. The
* `#undef` below lets the gas-style names below win; if a file needs
* both, the dsl.h versions can be reached via their long forms
* NOTE: dsl.h defines `add_s`/`sub_s`/`mut_s`/`gt_s`/etc. as _Generic-based signed integer-arithmetic helpers for U1/U2/U4.
* Those live in a different conceptual layer (generic arithmetic on DSL types) and would collide with the instruction encoders here.
* The `#undef` below lets the gas-style names below win; if a file needs both, the dsl.h versions can be reached via their long forms
* (e.g. `def_signed_op`-style or the underlying `add_s1/s2/s4`). */
#undef add_s
#undef sub_s
@@ -441,7 +414,7 @@ enum { _BitOffsets = 0
#define mov_to_low(rs) enc_r(op_special, (rs), R_0, R_0, 0, fc_mtlo)
/* --- Atomic branches (no pseudos like bgt/bge; compose with slt_* + branch_ne) ---
* branch_equal rs, rt, off → beq rs, rt, off
* branch_equal rs, rt, off → beq rs, rt, off
* branch_ne rs, rt, off → bne rs, rt, off
* branch_lt_zero rs, off → bltz rs, off
* branch_gt_zero rs, off → bgtz rs, off
@@ -463,7 +436,7 @@ enum { _BitOffsets = 0
/* --- Shift-amount alias (matches the gas convention `\p3 = shamt`) --- */
#define shift_amount(rd, rt, n) shift_lleft(rd, rt, n)
/* nop — canonical sll $0, $0, 0 */
/* nop — sll $0, $0, 0 */
#define nop shift_lleft(rdiscard, rdiscard, 0)
#define nop2 nop, nop
@@ -472,22 +445,18 @@ enum { _BitOffsets = 0
/* load_imm_2w — unconditional 2-word `li` form: `lui` + (ori | addi).
*
* Granular companion to `load_imm`: skips the compile-time range checks
* and always emits 2 .words. Use this when:
* Granular companion to `load_imm`: skips the compile-time range checks and always emits 2 .words. Use this when:
* - you know `imm` is > 0xFFFF (otherwise you're wasting a word), OR
* - `imm` is not a compile-time constant and you want predictable
* 2-word emission without the `__builtin_constant_p` branches.
*
* The lo16 strategy is still chosen at expansion time on the lo half:
* lo16 in 0x0000..0x7FFF → addi (sign-ext is harmless, the lui
* already cleared bits 15..0)
* lo16 in 0x8000..0xFFFF → ori (zero-extends to preserve the
* intended bit pattern)
* lo16 in 0x0000..0x7FFF → addi (sign-ext is harmless, the lui already cleared bits 15..0)
* lo16 in 0x8000..0xFFFF ori (zero-extends to preserve the intended bit pattern)
*
* For situations where you need to bypass even this choice (e.g. to
* force a specific encoding for a known discontiguous high/low pair),
* For situations where you need to bypass even this choice
* (e.g. to force a specific encoding for a known discontiguous high/low pair),
* see `load_imm_2w_ori_forced` and `load_imm_2w_addi_forced` below.
*
* Statement-level (not expression-level): emits its own `asm volatile(...)`.
*/
#define load_imm_2w(rt, imm) do { \
@@ -518,9 +487,8 @@ enum { _BitOffsets = 0
} while (0)
/* load_imm_2w_addi_forced — force the `lui` + `addi` form regardless of lo16 sign.
* Use when you know sign-extension is fine (e.g. lo16 is treated as
* signed downstream) and you want a smaller effective instruction
* (the assembler/MIPS hardware will sign-extend the imm16). */
* Use when you know sign-extension is fine (e.g. lo16 is treated as signed downstream)
* and you want a smaller effective instruction (the assembler/MIPS hardware will sign-extend the imm16). */
#define load_imm_2w_addi_forced(rt, imm) do { \
/*U4 _li2a_imm_ = (U4)(imm);*/ \
asm volatile(asm_words( \
@@ -532,23 +500,17 @@ enum { _BitOffsets = 0
/* load_imm rt, imm — true `li` semantics (assembler `li` pseudo)
*
* Dispatches at compile time on the immediate's range, picking the
* smallest single-instruction form when possible:
*
* imm in 0 .. 0x7FFF addi rt, $0, imm (1 word)
* imm in 0x8000 .. 0xFFFF → ori rt, $0, imm (1 word; sign-bit must be zeroed)
* imm in 0x10000 .. 0xFFFFFFFF → lui + (ori | addi) (2 words)
*
* Statement-level (not expression-level): the macro emits its own
* `asm volatile(...)` block with 1 or 2 .word constants. Callers can
* group multiple `load_imm` calls in a single volatile by using the
* lower-level encoders directly:
* Dispatches at compile time on the immediate's range, picking the smallest single-instruction form when possible:
* imm in 0 .. 0x7FFF → addi rt, $0, imm (1 word)
* imm in 0x8000 .. 0xFFFF → ori rt, $0, imm (1 word; sign-bit must be zeroed)
* imm in 0x10000 .. 0xFFFFFFFF → lui + (ori | addi) (2 words)
*
* Statement-level (not expression-level): the macro emits its own `asm volatile(...)` block with 1 or 2 .word constants.
* Callers can group multiple `load_imm` calls in a single volatile by using the lower-level encoders directly:
* load_imm(R_T4, 0x12345678); // emits 2 .words
*
* Falls back to a 2-word form if `imm` is not a compile-time constant,
* but that path is unusual (load_imm is most useful with literal
* addresses and magic numbers). */
* Falls back to a 2-word form if `imm` is not a compile-time constant, but that path is unusual
* (load_imm is most useful with literal addresses and magic numbers). */
#define load_imm(rt, imm) do { \
if (cexpr_(imm) && ((imm) <= 0x7FFFU)) { \
/* Small positive: addi rt, $0, imm */ \
@@ -588,9 +550,8 @@ enum { _BitOffsets = 0
/* Standard clobber list for pure-MIPS asm volatile blocks: caller-saved
* GPRs that the kernel treats as volatile (v0/v1/t0/t1/ra) plus the
* "memory" barrier. The register ids are passed through `rlit` so
* the R_*_Code `#define`s are stringified into "$N" at expansion time. */
* GPRs that the kernel treats as volatile (v0/v1/t0/t1/ra) plus the "memory" barrier.
* The register ids are passed through `rlit` so the R_*_Code `#define`s are stringified into "$N" at expansion time. */
#define clbr_volatile_gprs rlit(R_V0), rlit(R_T0), rlit(R_T1), rlit(R_RA), clb_mem_drain
#define asm_mips_flush_icache() asm volatile( asm_words( \
+2 -3
View File
@@ -2,9 +2,8 @@
* duffle DSL — MIPS Vendor Mnemonics (opt-in)
* ============================================================================
*
* Provides the textbook MIPS assembly mnemonics as thin aliases to the
* canonical duffle macros in mips.h. The duffle names are primary; this
* header is for users who prefer the textbook mnemonics.
* Provides the textbook MIPS assembly mnemonics as thin aliases to the duffle macros in mips.h.
* The duffle names are primary; this header is for users who prefer the textbook mnemonics.
*
* USAGE: #include "duffle/mips_vendor_sym.h" // after mips.h
*
+4 -5
View File
@@ -5,11 +5,10 @@
// Format: WORD_COUNT(MACRO_NAME, COUNT)
// One line per macro that appears in your atom sources.
//
// This file is encoding-macros-only. The auto-generated component
// macros (mac_X) live in duffle/gen/<dir>.macs.h (included separately
// by the unity build). The unity build should include THIS file and
// the .macs.h file in the same TU, with both wrapped (or the
// include guard order handled) to avoid WORD_COUNT redeclaration.
// This file is encoding-macros-only.
// The auto-generated component macros (mac_X) live in duffle/gen/<dir>.macs.h (included separately by the unity build).
// The unity build should include THIS file and the .macs.h file in the same TU, with both wrapped
// (or the include guard order handled) to avoid WORD_COUNT redeclaration.
//
// To regenerate: hand-count the instructions in each macro definition.
// (You'll only need to do this once per macro — they don't change often.)
+2 -2
View File
@@ -15,9 +15,9 @@ enum {
atom_offset_bounds_chk_cube_g4_face_exit = _atom_offset_bounds_chk_cube_g4_face_exit,
};
// --- atom: floor_f3_face (66 words) ---
// --- atom: floor_f3_face (58 words) ---
#define _atom_offset_culling_floor_f3_face_exit 29
#define _atom_offset_culling_floor_f3_face_exit 25
#define _atom_offset_bounds_chk_floor_f3_face_exit 13
enum {
+16 -17
View File
@@ -1,6 +1,6 @@
#include "stdio.h"
#include <stdio.h>
#include <stdlib.h>
#include "assert.h"
#include <assert.h>
// #include "libgpu.h"
// #include "libetc.h"
// #include "libgte.h"
@@ -99,8 +99,13 @@ typedef Struct_(Ent_Floor) {
A2_V3_S2 faces;
};
enum { scratchpad_size = 1024, };
enum {
Scratchpad_Len = 1024,
MemTape_Len = 512,
};
typedef Struct_(SMemory) {
U4 MemTape[MemTape_Len];
DoubleBuffer screen_buf;
A2_OrderingTable_Buffer ordering_tbl;
PrimitiveArena primitives;
@@ -182,6 +187,7 @@ void gp_display_frame(DoubleBuffer* screen_buf, S4* active_buf_id, U4* ordering_
void render(void) {
}
GCC_OPTIMIZATION_DISABLE
void update(PrimitiveArena* pa, U4* ordering_buf)
{
orderingtbl_clear_reverse(ordering_buf, OrderingTbl_Len);
@@ -207,6 +213,8 @@ void update(PrimitiveArena* pa, U4* ordering_buf)
A2_S2 p; //???
S4 flag; //????
TapeBuilder tb = tb_make(slice_ut_arr(smem.MemTape));
// Draw Cube
if (0)
{
@@ -259,8 +267,7 @@ void update(PrimitiveArena* pa, U4* ordering_buf)
U4 prim_base = u4_(pa->buf[smem.active_buf_id]);
U4 prim_cursor = prim_base + pa->used;
LP_ U4 mem_temp_tape[512]; FArena tape_arena; farena_init(& tape_arena, slice_ut_arr(mem_temp_tape));
TapeBuilder tb = tb_make_old(&tape_arena); tb_scope(& tb) {
tb.used = 0; tb_scope(& tb) {
tb_emit(& tb, rbind_cube_g4_face);
tb_data(& tb, prim_cursor);
tb_data(& tb, u4_(smem.cube.faces));
@@ -344,12 +351,11 @@ void update(PrimitiveArena* pa, U4* ordering_buf)
U4 prim_base = u4_(pa->buf[smem.active_buf_id]);
U4 prim_cursor = prim_base + pa->used;
// TODO(Ed): We should do a bounds check beforehand to confirm pa can hold all tris.
// TODO(Ed): We should do a bounds check beforehand to confirm pa can hold all tris?
// The tape atoms in-flight should not need to care.
// Prepare the tape. (Push protocol to tape)
LP_ U4 mem_temp_tape[512];
TapeBuilder tb = tb_make(slice_ut_arr(mem_temp_tape)); tb_scope(& tb) {
tb.used = 0; tb_scope(& tb) {
tb_emit(& tb, set_gte_world);
tb_data(& tb, u4_(& smem.tform_world));
@@ -367,25 +373,18 @@ void update(PrimitiveArena* pa, U4* ordering_buf)
tb_data(& tb, u4_(& pa->used));
tb_data(& tb, prim_base);
}
tape_run(tb_slice(tb));// Fire off the tape.
// C-side state (pa->used) has already been updated by the tape!
smem.floor.rot.y += 5;
}
// --- TAPE DIAGNOSTICS ---
if (1)
if (0)
{
LP_ U4 mem_temp_tape[512]; FArena tape_arena; farena_init(& tape_arena, slice_ut_arr(mem_temp_tape));
TapeBuilder tb = tb_make_old(& tape_arena); tb_scope(& tb) {
// Skip set_gte_world atom for diagnostics to isolate the triangle loop
for (U4 i = 0; i < Floor_num_faces; i++) {
// =======================================================
// SWAP EMIT TO TEST DIFFERENT PARTS OF THE PIPELINE:
// =======================================================
// 1. code_diag_yield -> Tests Tape Engine jump logic
// 2. code_diag_color -> Tests OT and Prim Arena memory
// 3. code_diag_gte -> Tests Vertex arrays and GTE Math
// tb_emit(& tb, code_diag_yield);
// tb_emit(& tb, code_diag_color);
// tb_emit(& tb, code_diag_gte);
@@ -394,9 +393,9 @@ void update(PrimitiveArena* pa, U4* ordering_buf)
B1* prim_cursor = (B1*)r_(pa->buf)[smem.active_buf_id] + pa->used;
tape_run(tb_slice(tb));
pa->used = (U4)prim_cursor - (U4)r_(pa->buf)[smem.active_buf_id];
smem.floor.rot.y += 5;
}
}
GCC_OPTIMIZATION_ENABLE
int main(void)
{
+13 -18
View File
@@ -22,7 +22,7 @@ typedef Struct_(Binds_CubeTri) {
V3_S2* VertBase;
U4* OtBase;
};
internal MipsAtom_(rbind_cube_g4_face) atom_info(atom_bind(Binds_CubeTri)
internal MipsAtom_(rbind_cube_g4_face) atom_info(atom_bind(Binds_CubeTri), atom_phase(cube_g4)
, atom_reads(R_TapePtr)
, atom_writes(R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
){
@@ -35,14 +35,9 @@ internal MipsAtom_(rbind_cube_g4_face) atom_info(atom_bind(Binds_CubeTri)
mac_yield()
};
/* ============================================================================
* cube_g4_face — Draw one cube face (Gouraud-shaded quad) via the GTE tape pipeline
* ============================================================================
* Reads 4 indices from R_FaceCur (V4_S2 = 8 bytes), loads 4 vertices into
* the GTE, runs the PsyQ RotAverageNclip4 sequence, and renders a Poly_G4.
*/
// cube_g4_face — Draw one cube face (Gouraud-shaded quad) via the GTE tape pipeline
internal
MipsAtom_(cube_g4_face) atom_info(
MipsAtom_(cube_g4_face) atom_info(atom_phase(cube_g4),
atom_reads( R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase),
atom_writes(R_PrimCursor, R_FaceCursor)
){
@@ -95,7 +90,7 @@ typedef Struct_(Binds_FloorTri) {
U4* OtBase;
};
internal
MipsAtom_(rbind_floor_f3_face) atom_info(atom_bind(Binds_FloorTri)
MipsAtom_(rbind_floor_f3_face) atom_info(atom_bind(Binds_FloorTri), atom_phase(floor_f3)
, atom_reads(R_TapePtr)
, atom_writes(R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
){
@@ -108,27 +103,27 @@ MipsAtom_(rbind_floor_f3_face) atom_info(atom_bind(Binds_FloorTri)
mac_yield()
};
atom_dbg_skip
internal
MipsAtom_(floor_f3_face) atom_info(
MipsAtom_(floor_f3_face) atom_info(atom_phase(floor_f3)
, atom_reads( R_PrimCursor, R_FaceCursor, R_VertBase, R_OtBase)
, atom_writes(R_PrimCursor, R_FaceCursor)
, atom_writes(R_PrimCursor, R_FaceCursr)
) {
mac_load_tri_indices( R_T0, R_T1, R_T2),
mac_gte_load_tri_verts(R_T0, R_T1, R_T2),
nop2, gte_cmdw_rotate_translate_perspective_triple,
nop2, gte_cmdw_nclip,
nop2, gte_cmdw_rotate_translate_perspective_triple, // 2 nops retire the final cpu -> gte writes before RTPT
gte_cmdw_nclip,
/* Culling (Branch forward if Backface) */
nop2, gte_mv_from_data_r(R_T0, C2_MAC0),
nop,
branch_le_zero(R_T0, atom_offset(culling, floor_f3_face_exit)), nop,
gte_mv_from_data_r(R_T0, C2_MAC0),
nop, branch_le_zero(R_T0, atom_offset(culling, floor_f3_face_exit)), nop, // required gte -> cpu load-delay slot.
/* Format Primitive */
mac_format_f3_color(0xFF, 0xFF, 0xFF), // RGB-form (R=FF, G=FF, B=FF = white)
mac_gte_store_f3_post_rtpt(),
/* Calculate Depth */
nop2, gte_avg_sort_z3,
nop2, gte_mv_from_data_r(R_T1, C2_OTZ),
gte_avg_sort_z3,
gte_mv_from_data_r(R_T1, C2_OTZ),
/* Bounds Check OTZ < 2048 (Branch forward to skip insertion) */
add_ui( R_AT, R_0, OrderingTbl_Len),
set_lt_u( R_AT, R_T1, R_AT),
+95 -127
View File
@@ -81,19 +81,6 @@ $path_psyq = join-path $path_toolchain 'psyq-4_7'
$path_psyq_iwyu = join-path $path_toolchain 'psyq_iwyu'
$path_psyq_imyu_inc = join-path $path_psyq_iwyu 'include'
function Get-SourceFiles { param([Parameter(Mandatory=$true)] [string[]]$paths, [Parameter(Mandatory=$true)] [string[]]$extensions)
$files = @()
foreach ($p in $paths) {
if (-not (test-path $p)) { continue }
foreach ($ext in $extensions) {
Get-ChildItem -Path $p -File -Recurse -Filter "*$ext" -ErrorAction SilentlyContinue | ForEach-Object {
$files += $_.FullName
}
}
}
return ($files | Sort-Object -Unique)
}
function assemble-unit { param(
[string] $unit,
[string] $link_module,
@@ -153,7 +140,7 @@ function compile-unit { param(
$f_arch_no_shared,
$f_arch_no_stack_prot
)
# $compile_args += $f_std_c23
$compile_args += $f_std_c11
$compile_args += ($f_include + $path_psyq_imyu_inc)
$compile_args += ($f_include + $path_nugget)
@@ -243,6 +230,52 @@ function make-binary { param([string]$elf, [string]$exe)
if ($LASTEXITCODE -ne 0) { Write-Error "Objcopy failed. Aborting."; exit 1 }
}
function ps1-meta { param(
[string]$unity_root,
[string[]]$sources,
[Parameter(Mandatory=$true)][string]$metadata,
[string]$out_root = (join-path $path_build 'gen'),
[string[]]$passes = @('--pre-link'),
[string[]]$extra_args = @()
)
# `--unity-root` and `--source` are
# mutually exclusive. Exactly one of `$unity_root` / `$sources` must
# be supplied; the other must be absent.
if ($null -ne $unity_root -and $unity_root -ne '')
{
if ($null -ne $sources -and $sources.Count -gt 0) {
write-error 'ps1-meta: -unity_root and -sources are mutually exclusive'
exit 2
}
}
elseif ($null -eq $sources -or $sources.Count -eq 0) {
write-error 'ps1-meta: either -unity_root <file> or -sources <file...> is required'
exit 2
}
$script = join-path $path_scripts 'ps1_meta.lua'
$input_summary = if ($null -ne $unity_root -and $unity_root -ne '') {
"unity=$unity_root"
}
else {
"$($sources.Count) source(s)"
}
write-host "ps1-meta $input_summary, passes=$($passes -join ',')" ` -ForegroundColor Magenta
$arg_list = @($passes) + @('--metadata', $metadata) + @('--out-root', $out_root) + @($extra_args)
if ($null -ne $unity_root -and $unity_root -ne '') {
$arg_list += @('--unity-root', $unity_root)
}
else {
foreach ($s in $sources) { $arg_list += @('--source', $s) }
}
& luajit $script @arg_list
if ($LASTEXITCODE -ne 0) {
write-error "ps1-meta failed (exit $LASTEXITCODE). Aborting."
exit $LASTEXITCODE
}
}
function build-hello_psyqo {
$includes += @()
@@ -317,34 +350,16 @@ function build-graphis_hello {
}
# build-graphis_hello
function ps1-meta { param(
[Parameter(Mandatory=$true)][string[]]$sources,
[Parameter(Mandatory=$true)][string]$metadata,
[string]$out_root = (join-path $path_build 'gen'),
[string[]]$passes = @('--all'),
[string[]]$extra_args = @()
)
$script = join-path $path_scripts 'ps1_meta.lua'
write-host "ps1-meta $($sources.Count) source(s), passes=$($passes -join ',')" ` -ForegroundColor Magenta
$arg_list = @($passes) + @('--metadata', $metadata) + @('--out-root', $out_root) + @($extra_args)
foreach ($s in $sources) { $arg_list += @('--source', $s) }
& luajit $script @arg_list
if ($LASTEXITCODE -ne 0) {
write-error "ps1-meta failed (exit $LASTEXITCODE). Aborting."
exit $LASTEXITCODE
}
}
function build-gte_hello {
$includes += @()
$path_module = join-path $path_code 'gte_hello'
$path_duffle = join-path $path_code 'duffle'
$path_atom_metadata = join-path $path_duffle 'word_count.metadata.h'
$path_build_gen = join-path $path_build 'gen'
$source_dirs = @($path_duffle, $path_module)
$atom_sources = Get-SourceFiles -paths $source_dirs -extensions @('.h', '.c')
ps1-meta -sources $atom_sources -metadata $path_atom_metadata -out_root (join-path $path_build 'gen')
$src_c = join-path $path_module 'hello_gte.c'
ps1-meta -unity_root $src_c -metadata $path_atom_metadata -out_root $path_build_gen
$assemble_args = @()
$assemble_args += $f_debug
@@ -360,7 +375,6 @@ function build-gte_hello {
# assemble-unit $src_asm $module_asm $includes $assemble_args
$src_c = join-path $path_module 'hello_gte.c'
$module_c = join-path $path_build 'hello_gte_c.o'
$compile_args = @()
@@ -379,114 +393,68 @@ function build-gte_hello {
$link_args += $f_debug
# $link_args += $f_optimize_size
$link_modules = @(
$module_asm_crt,
$module_asm_crt,
$module_c
)
link-modules $link_modules $elf $link_args
make-binary $elf $exe
# TODO(Ed): Do both -gdb-runtime and dwarf-injection passes in a single ps1-meta call.
# Post-link: gdb-runtime + dwarf-injection in a single Lua invocation (one luajit cold start).
ps1-meta -unity_root $src_c -metadata $path_atom_metadata -out_root $path_build_gen -passes @('--post-link') ` -extra_args @('--elf', $elf)
# Post-link: emit ONLY build/gen/gdb_tape_atoms_runtime.gdb.
# The per-source *.atoms.sourcemap.txt was already generated by the pre-link --all call,
# so we skip --atoms-source-map here to avoid re-doing the work.
# The gdb-runtime emission requires --elf (for nm-based address lookup) so it MUST happen post-link.
ps1-meta -sources $atom_sources -metadata $path_atom_metadata `
-out_root (join-path $path_build 'gen') `
-passes @('--gdb-runtime') `
-extra_args @('--elf', $elf)
# F' + G' consolidated: --dwarf-injection now emits 7 .bin blobs
# (.debug_line, .debug_aranges, .debug_rnglists, .debug_info, .debug_abbrev, .debug_str, .debug_loc) all in one pass.
ps1-meta -sources $atom_sources -metadata $path_atom_metadata `
-out_root (join-path $path_build 'gen') `
-passes @('--dwarf-injection') `
-extra_args @('--elf', $elf)
#TODO(Ed): Move the below into ps-1 meta pass to reduce syscall latency?
# F' track: post-link DWARF injection. The new Lua pass writes build/gen/<basename>.dwarf_*.bin blobs;
# we splice them into a COPY of the ELF via objcopy --update-section (works fine from PowerShell).
# The un-injected $elf + $exe are unchanged (shipping binary).
$dwarfLineBin = Join-Path (Join-Path $path_build 'gen') 'hello_gte.dwarf_line.bin'
$dwarfArangesBin = Join-Path (Join-Path $path_build 'gen') 'hello_gte.dwarf_aranges.bin'
$dwarfRnglistsBin = Join-Path (Join-Path $path_build 'gen') 'hello_gte.dwarf_rnglists.bin'
$injectElf = Join-Path $path_build 'hello_gte.dwarf-injected.elf'
if ((Test-Path $dwarfLineBin) -and (Test-Path $dwarfArangesBin) -and (Test-Path $dwarfRnglistsBin))
$dwarfLineBin = join-path $path_build_gen 'hello_gte.dwarf_line.bin'
$dwarfArangesBin = join-path $path_build_gen 'hello_gte.dwarf_aranges.bin'
$dwarfRnglistsBin = join-path $path_build_gen 'hello_gte.dwarf_rnglists.bin'
$injectElf = join-path $path_build 'hello_gte.dwarf-injected.elf'
if ((Test-Path $dwarfLineBin) -and (Test-Path $dwarfArangesBin) -and (Test-Path $dwarfRnglistsBin))
{
Write-Host "[build] DWARF-injecting $elf -> $injectElf"
Copy-Item -LiteralPath $elf -Destination $injectElf
& $Objcopy --update-section ".debug_line=$dwarfLineBin" $injectElf
$last_exit_code_error = $LASTEXITCODE -ne 0
if ($last_exit_code_error) {
Write-Warning "[build] objcopy .debug_line update failed (exit $LASTEXITCODE); removing $injectElf"
Remove-Item -LiteralPath $injectElf -ErrorAction SilentlyContinue
return;
}
& $Objcopy --update-section ".debug_aranges=$dwarfArangesBin" $injectElf
$last_exit_code_error = $LASTEXITCODE -ne 0
Copy-Item -LiteralPath $elf -Destination $injectElf -Force
# Objcopy call: 3x --update-section for (line, aranges, rnglists).
$f_args = @(
"--update-section=.debug_line=$dwarfLineBin",
"--update-section=.debug_aranges=$dwarfArangesBin",
"--update-section=.debug_rnglists=$dwarfRnglistsBin"
)
& $Objcopy @f_args $injectElf 2>&1 | Out-Null
if ($LASTEXITCODE -ne 0) {
Write-Warning "[build] objcopy .debug_aranges update failed (exit $LASTEXITCODE); removing $injectElf"
Write-Warning "[build] objcopy F' splice failed (exit $LASTEXITCODE); removing $injectElf"
Remove-Item -LiteralPath $injectElf -ErrorAction SilentlyContinue
return;
}
& $Objcopy --update-section ".debug_rnglists=$dwarfRnglistsBin" $injectElf
if ($LASTEXITCODE -ne 0) {
Write-Warning "[build] objcopy .debug_rnglists update failed (exit $LASTEXITCODE); removing $injectElf"
Remove-Item -LiteralPath $injectElf -ErrorAction SilentlyContinue
}
else
{
# Baked atoms execute from RAM but are emitted as C data arrays, so their ELF sections lack SHF_EXECINSTR.
# GDB discards line rows for non-code sections.
# Mark only the debug-copy sections executable; the shipping ELF and PS-EXE remain byte/flag unchanged.
& $Objcopy `
--set-section-flags ".rodata=alloc,load,readonly,code,contents" `
--set-section-flags ".data=alloc,load,data,code,contents" `
$injectElf
if ($LASTEXITCODE -ne 0) {
Write-Warning "[build] objcopy atom-section flag update failed (exit $LASTEXITCODE); removing $injectElf"
Remove-Item -LiteralPath $injectElf -ErrorAction SilentlyContinue
} else {
Write-Host "[build] DWARF-injected ELF: $injectElf"
}
}
}
# G' (atom locals) is now part of --dwarf-injection.
# The F' splice block above already covered .debug_line / .debug_aranges / .debug_rnglists;
# we extend the same Copy-Item + objcopy chain to splice the G' 4 sections
# (.debug_info, .debug_abbrev, .debug_str via --update-section; .debug_loc via --add-section since it doesn't exist in the source ELF).
$dwarfInfoBin = Join-Path (Join-Path $path_build 'gen') 'hello_gte.dwarf_info.bin'
$dwarfAbbrevBin = Join-Path (Join-Path $path_build 'gen') 'hello_gte.dwarf_abbrev.bin'
$dwarfStrBin = Join-Path (Join-Path $path_build 'gen') 'hello_gte.dwarf_str.bin'
$dwarfLocBin = Join-Path (Join-Path $path_build 'gen') 'hello_gte.dwarf_loc.bin'
if ((Test-Path $dwarfInfoBin) -and (Test-Path $dwarfAbbrevBin) -and (Test-Path $dwarfStrBin) -and (Test-Path $dwarfLocBin))
{
Write-Host "[build] G' atom-locals: splicing .debug_info/.debug_abbrev/.debug_str/.debug_loc into $injectElf"
& $Objcopy --update-section ".debug_info=$dwarfInfoBin" $injectElf
$last_exit_code_error = ($LASTEXITCODE -ne 0)
if ($last_exit_code_error) {
Write-Warning "[build] objcopy .debug_info update failed (exit $LASTEXITCODE)"
return;
}
& $Objcopy --update-section ".debug_abbrev=$dwarfAbbrevBin" $injectElf
$dwarfInfoBin = join-path $path_build_gen 'hello_gte.dwarf_info.bin'
$dwarfAbbrevBin = join-path $path_build_gen 'hello_gte.dwarf_abbrev.bin'
$dwarfStrBin = join-path $path_build_gen 'hello_gte.dwarf_str.bin'
$dwarfLocBin = join-path $path_build_gen 'hello_gte.dwarf_loc.bin'
$dwarfLoclistsBin = join-path $path_build_gen 'hello_gte.dwarf_loclists.bin'
$g_args = @(
"--update-section=.debug_info=$dwarfInfoBin",
"--update-section=.debug_abbrev=$dwarfAbbrevBin",
"--update-section=.debug_str=$dwarfStrBin",
"--add-section=.debug_loc=$dwarfLocBin",
"--add-section=.debug_loclists=$dwarfLoclistsBin"
)
& $Objcopy @g_args $injectElf 2>&1 | Out-Null
if ($LASTEXITCODE -ne 0) {
Write-Warning "[build] objcopy .debug_abbrev update failed (exit $LASTEXITCODE)"
Write-Warning "[build] objcopy G' splice failed (exit $LASTEXITCODE); removing $injectElf"
Remove-Item -LiteralPath $injectElf -ErrorAction SilentlyContinue
return;
}
& $Objcopy --update-section ".debug_str=$dwarfStrBin" $injectElf
}
# Baked atoms execute from RAM but are emitted as C data arrays, so their ELF sections lack SHF_EXECINSTR.
# GDB discards line rows for non-code sections. Mark only the debug-copy sections executable.
# The original ELF and PS-EXE remain byte/flag unchanged.
& $Objcopy `
--set-section-flags ".rodata=alloc,load,readonly,code,contents" `
--set-section-flags ".data=alloc,load,data,code,contents" `
$injectElf 2>&1 | Out-Null
if ($LASTEXITCODE -ne 0) {
Write-Warning "[build] objcopy .debug_str update failed (exit $LASTEXITCODE)"
Write-Warning "[build] atom-section flag update failed (exit $LASTEXITCODE); removing $injectElf"
Remove-Item -LiteralPath $injectElf -ErrorAction SilentlyContinue
}
else
{
# .debug_loc doesn't exist in the source ELF; --add-section creates it.
& $Objcopy --add-section ".debug_loc=$dwarfLocBin" $injectElf
if ($LASTEXITCODE -ne 0) {
Write-Warning "[build] objcopy .debug_loc add-section failed (exit $LASTEXITCODE)"
} else {
Write-Host "[build] G' atom-locals-injected: $injectElf"
}
else {
Write-Host "[build] DWARF-injected ELF: $injectElf"
}
}
}
+1873 -277
View File
File diff suppressed because it is too large Load Diff
+7 -6
View File
@@ -11,11 +11,10 @@
--- ```
---
--- That small bootstrap: (a) locates this helper via `arg[0]` / `debug.getinfo`,
--- (b) loads it (which sets `package.path` + `package.cpath` via cached `git rev-parse`),
--- (b) loads it (which sets `package.path` + `package.cpath`),
--- (c) at the bottom calls `require("duffle")` (now resolvable since `package.path` was just set) and returns the duffle M.
--- Net effect: the caller gets the duffle module in one statement; no separate `dofile(...)` + `require("duffle")` dance.
---
--- Replaces the prior 2-line (entry) or 4-line (pass) pattern that had the call site do its own path resolution + duplicated setup.
local M = {}
@@ -27,9 +26,6 @@ local CACHE_KEY = "__duffle_repo_root__"
--- parent of the directory containing this script. We derive it directly from `debug.getinfo(1, "S").source`
--- (returns `@<path>` for the currently-running chunk).
---
--- Replaces the prior `io.popen("git rev-parse --show-toplevel")` approach, which cost ~100-180ms per
--- LuaJIT process on Windows due to git's CLI startup. The path-derive approach costs <1ms.
---
--- If `debug.getinfo` can't parse this script's path (shouldn't happen — dofile always populates source),
--- return nil and let `M.setup()` fail loud.
--- @return string|nil
@@ -61,7 +57,12 @@ end
function M.setup()
local repo_root = find_repo_root()
if not repo_root then
io.stderr:write("[duffle_paths] git rev-parse failed -- not in a git repo?\n")
-- Unreachable in practice: find_repo_root() derives the repo root from this script's
-- own source path via debug.getinfo(1, "S").source (no subprocess, no git CLI, <1ms).
-- A nil return means the source path did not match the expected
-- <repo>/scripts/duffle_paths.lua layout — a packaging bug, not a "missing git repo"
-- condition. os.exit(2) is retained so a real failure surfaces loud rather than
-- silently producing an unconfigured module table.
os.exit(2)
end
+433 -153
View File
@@ -1,10 +1,9 @@
--- elf_dwarf.lua — ELF32 + DWARF + atoms source-map utilities for the F'' track.
---
--- elf_dwarf.lua — ELF32 + DWARF + atoms source-map utilities.
--- All ELF32 + DWARF-specific code lives here.
---
--- **What this module contains:**
--- - **Format-constant tables** (the byte-offset / opcode / size encyclopedias for ELF32, DWARF4 aranges, DWARF5 rnglists, DWARF line-program, MIPS).
--- Every constant carries a spec:` comment naming the spec section that defines it (convention established by F'').
--- Every constant carries a spec:` comment naming the spec section that defines it.
--- - **I/O helpers**: little-endian byte read/write, ELF32 section walker, nm symbol reader, source-map parser, native directory glob.
---
--- **Conventions:** tabs (1/level), EmmyLua annotations, no regex,
@@ -14,13 +13,82 @@
-- Native dependencies
-- ════════════════════════════════════════════════════════════════════════════
-- lfs is wired into package.cpath by `duffle_paths.lua` (vendored under
-- `toolchain/lfs/lfs.dll`). Required here for native directory ops
-- (replaces the ~56ms `dir /b` subprocess with ~2ms native).
-- lfs is wired into package.cpath by `duffle_paths.lua` (vendored under `toolchain/lfs/lfs.dll`).
local lfs = require("lfs")
local M = {}
-- ════════════════════════════════════════════════════════════════════════════
-- DWARF tag + form constants
-- ════════════════════════════════════════════
-- (DWARF5 §7.5.5 "Tag Encodings" + Table 7.1; gcc emits these exact values for the DWARF3-extension and DWARF5 line units.)
M.DW_TAG = {
compile_unit = 0x11,
subprogram = 0x2E,
variable = 0x34,
structure_type = 0x13,
member = 0x0D,
base_type = 0x24,
typedef = 0x2A,
pointer_type = 0x0F,
const_type = 0x26,
volatile_type = 0x27,
inlined_subroutine = 0x1D,
-- We index the canonical gcc-emitted tags. Anything else falls through.
}
M.DW_AT = {
name = 0x03,
low_pc = 0x11,
high_pc = 0x12,
language = 0x13,
location = 0x02,
comp_dir = 0x1B,
byte_size = 0x0B,
encoding = 0x3E,
data_member_location = 0x38,
type = 0x49,
linkage_name = 0x6E,
external = 0x3F,
abstract_origin = 0x31,
call_file = 0x58,
call_line = 0x59,
inline = 0x20,
decl_file = 0x3A,
decl_line = 0x3B,
}
M.DW_FORM = {
addr = 0x01,
data1 = 0x0B,
data2 = 0x05,
data4 = 0x06,
string = 0x08,
strp = 0x0E,
exprloc = 0x18,
ref4 = 0x13,
udata = 0x0F,
ref_sig8 = 0x20,
implicit_const = 0x21,
flag_present = 0x19,
sec_offset = 0x17,
}
M.DW_ATE = {
address = 0x01,
boolean = 0x02,
complex_float = 0x03,
float = 0x04,
signed = 0x05,
signed_char = 0x06,
unsigned = 0x07,
unsigned_char = 0x08,
}
-- DWARF5 §7.5.6 DW_FORM_implicit_const
local DW_FORM_implicit_const = 0x21
-- ════════════════════════════════════════════════════════════════════════════
-- Format-constant tables
-- ════════════════════════════════════════════════════════════════════════════
@@ -32,51 +100,51 @@ local M = {}
--- spec: MIPS o32 ABI §"Register Usage" — 32-bit general-purpose registers
M.MIPS_BYTES_PER_WORD = 0x04
-- ----------------------------------------------------------------------------
-- ELF32 (System V ABI gABI v1.2)
-- ----------------------------------------------------------------------------
--
-- All offsets are 1-INDEXED (matching Lua string.sub convention),
-- expressed in hex so they map directly to the wire-format byte positions in the binary file.
-- To compute the 0-indexed file offset, subtract 1.
--
-- Example: e_shoff_offset = 0x21 means the 4-byte e_shoff field
-- starts at string.sub byte 0x21 (= 33 in 1-indexed), i.e. file offset 0x20 (= 32).
--- **Wire-offset contract:** format offsets, fixed-width reader offsets, LEB/parser cursors,
--- and section-relative values are zero-based wire offsets. Only Lua string APIs receive
--- a `+ 1` conversion at their boundary (`byte`, `sub`, and `find`).
---
--- ELF/DWARF field offsets are expressed in hex so they map directly to the
--- zero-based byte positions in the binary file.
--- spec: System V ABI gABI v1.2 §"ELF Header" (Table 1) + §"Section Header Table"
M.ELF32 = {
magic_offset = 0x01, -- 4-byte magic "\127ELF" at file offset 0x00
magic_offset = 0x00, -- 4-byte magic "\127ELF" at file offset 0x00
magic = "\127ELF",
class_offset = 0x05, -- 1-byte; 1 = ELF32, 2 = ELF64
class_offset = 0x04, -- 1-byte; 1 = ELF32, 2 = ELF64
class_elf32 = 1,
endian_offset = 0x06, -- 1-byte; 1 = little-endian, 2 = big-endian
endian_offset = 0x05, -- 1-byte; 1 = little-endian, 2 = big-endian
endian_little = 1,
header_bytes = 0x34, -- spec: gABI v1.2 §"ELF Header" — ELF32 header is 52 bytes total
e_shoff_offset = 0x21, -- 4-byte LE; section-header table file offset
e_shentsize_offset = 0x2F, -- 2-byte LE; section-header entry size in bytes
e_shnum_offset = 0x31, -- 2-byte LE; number of section headers
e_shstrndx_offset = 0x33, -- 2-byte LE; index of section-name string table
e_shoff_offset = 0x20, -- 4-byte LE; section-header table file offset
e_shentsize_offset = 0x2E, -- 2-byte LE; section-header entry size in bytes
e_shnum_offset = 0x30, -- 2-byte LE; number of section headers
e_shstrndx_offset = 0x32, -- 2-byte LE; index of section-name string table
sh_size_bytes = 0x28, -- spec: gABI v1.2 §"Section Header Table" — each entry is 40 bytes
sh_name_offset = 0x01, -- 4-byte LE; offset into .shstrtab
sh_type_offset = 0x05, -- 4-byte LE; section type (SHT_*)
sh_offset_offset = 0x11, -- 4-byte LE; section's file offset
sh_size_offset = 0x15, -- 4-byte LE; section's size in bytes
sh_name_offset = 0x00, -- 4-byte LE; offset into .shstrtab
sh_type_offset = 0x04, -- 4-byte LE; section type (SHT_*)
sh_offset_offset = 0x10, -- 4-byte LE; section's file offset
sh_size_offset = 0x14, -- 4-byte LE; section's size in bytes
dw_dwarf32_terminator = 0xFFFFFFFF, -- spec: DWARF4 spec §7.4 — 32-bit DWARF initial-length terminator
}
-- ----------------------------------------------------------------------------
-- DWARF4 .debug_aranges (per DWARF5 spec §7.4 — Address Range Table)
-- ----------------------------------------------------------------------------
--
-- All offsets are 1-INDEXED (matching Lua string.sub convention), in hex.
-- All offsets are zero-based wire offsets.
--- spec: DWARF5 spec §7.4 (Address Range Table) — 32-bit DWARF form
M.DWARF4_ARANGES = {
unit_length_offset = 0x01, -- 4-byte LE; length of unit body (excludes these 4 bytes)
version_offset = 0x05, -- 2-byte LE; expected = 2
cu_offset_offset = 0x07, -- 4-byte LE; CU DIE offset in .debug_info
addr_size_offset = 0x0B, -- 1-byte; expected = 4 (32-bit MIPS)
seg_size_offset = 0x0C, -- 1-byte; expected = 0
unit_length_offset = 0x00, -- 4-byte LE; length of unit body (excludes these 4 bytes)
version_offset = 0x04, -- 2-byte LE; expected = 2
cu_offset_offset = 0x06, -- 4-byte LE; CU DIE offset in .debug_info
addr_size_offset = 0x0A, -- 1-byte; expected = 4 (32-bit MIPS)
seg_size_offset = 0x0B, -- 1-byte; expected = 0
entry_size = 0x08, -- 4-byte addr + 4-byte length (per §7.4)
terminator_size = 0x08, -- 8 zero bytes (per §7.4 end-of-list marker)
version_expected = 2,
@@ -87,17 +155,16 @@ M.DWARF4_ARANGES = {
-- ----------------------------------------------------------------------------
-- DWARF5 .debug_rnglists (per DWARF5 spec §2.17 + §7.21)
-- ----------------------------------------------------------------------------
--
-- All offsets are 1-INDEXED (matching Lua string.sub convention), in hex.
-- All offsets are zero-based wire offsets.
--- spec: DWARF5 spec §2.17 + §7.21 (Range List Table) — 32-bit DWARF form
M.DWARF5_RNGLISTS = {
unit_length_offset = 0x01, -- 4-byte LE
version_offset = 0x05, -- 2-byte LE; expected = 5
addr_size_offset = 0x07, -- 1-byte; expected = 4
seg_size_offset = 0x08, -- 1-byte; expected = 0
offset_count_offset = 0x09, -- 4-byte LE; expected = 0
first_entry_offset = 0x0D,
unit_length_offset = 0x00, -- 4-byte LE
version_offset = 0x04, -- 2-byte LE; expected = 5
addr_size_offset = 0x06, -- 1-byte; expected = 4
seg_size_offset = 0x07, -- 1-byte; expected = 0
offset_count_offset = 0x08, -- 4-byte LE; expected = 0
first_entry_offset = 0x0C,
end_of_list = 0x00, -- spec: DWARF5 §7.7 — DW_RLE_end_of_list byte value
start_length = 0x07, -- spec: DWARF5 §7.7 — DW_RLE_start_length byte value
version_expected = 5,
@@ -109,7 +176,6 @@ M.DWARF5_RNGLISTS = {
-- ----------------------------------------------------------------------------
-- DWARF line-program opcodes (per DWARF5 spec §6.2.5)
-- ----------------------------------------------------------------------------
--
-- Opcode VALUES stay in decimal — they're identifiers (DW_LNS_copy = 1), not binary positions.
-- Compare to the *_offset fields above which are hex.
@@ -121,6 +187,7 @@ M.DWARF_LINE_OPS = {
DW_LNS_advance_pc = 2,
DW_LNS_advance_line = 3,
DW_LNS_set_file = 4,
DW_LNS_negate_stmt = 6, -- spec: §6.2.5.2 — toggle the line-state is_stmt register
-- Extended sub-opcodes (§6.2.5.3)
DW_LNE_end_sequence = 1, -- spec: §6.2.5.3
DW_LNE_set_address = 2, -- spec: §6.2.5.3
@@ -141,39 +208,279 @@ M.DWARF_LINE_OPS = {
-- I/O helpers: little-endian byte read/write
-- ════════════════════════════════════════════════════════════════════════════
--- Read a 4-byte little-endian unsigned integer from `buf` at 1-indexed offset `off`.
--- Equivalent to `string.unpack("<I4", buf, off)` but avoids the table-return shape + works under LuaJIT 2.1
--- Read a 4-byte little-endian unsigned integer from `buf` at zero-based wire offset `off`.
--- Equivalent to `string.unpack("<I4", buf, off + 1)` but avoids the table-return shape + works under LuaJIT 2.1
--- (which has partial `string.unpack` coverage).
---
--- **Convention:** offsets are 1-indexed (matching Lua `string.sub`).
--- **Convention:** `off` is a zero-based wire offset; `+ 1` is applied only at the `string.byte` boundary.
---
--- **Byte weights** are written as `0x100`, `0x10000`, `0x1000000` (i.e.
--- 2^8, 2^16, 2^24) so the LE byte positions are visually explicit:
--- **Byte weights** are written as `0x100`, `0x10000`, `0x1000000` (i.e. 2^8, 2^16, 2^24) so the LE byte positions are visually explicit:
--- byte 0 contributes its value directly; byte 1 is shifted left by 8
--- (= 0x100); byte 2 by 16 (= 0x10000); byte 3 by 24 (= 0x1000000).
---
--- @param buf string
--- @param off integer -- 1-indexed
--- @param off integer -- zero-based wire offset
--- @return integer
function M.read_u32_le(buf, off)
return buf:byte(off)
+ buf:byte(off + 0x01) * 0x00000100
+ buf:byte(off + 0x02) * 0x00010000
+ buf:byte(off + 0x03) * 0x01000000
local byte_off = off + 1
return buf:byte(byte_off)
+ buf:byte(byte_off + 0x01) * 0x00000100
+ buf:byte(byte_off + 0x02) * 0x00010000
+ buf:byte(byte_off + 0x03) * 0x01000000
end
--- Read a 2-byte little-endian unsigned integer from `buf` at 1-indexed offset `off`.
--- (1-indexed convention; matches `M.read_u32_le`.)
--- Read a 2-byte little-endian unsigned integer from `buf` at zero-based wire offset `off`.
--- (`off` is zero-based; `+ 1` is applied only at the `string.byte` boundary.)
--- @param buf string
--- @param off integer -- 1-indexed
--- @param off integer -- zero-based wire offset
--- @return integer
function M.read_u16_le(buf, off)
return buf:byte(off) + buf:byte(off + 0x01) * 0x00000100
local byte_off = off + 1
return buf:byte(byte_off) + buf:byte(byte_off + 0x01) * 0x00000100
end
-- Pure-Lua 5.3 LEB128 readers (no `bit` library). `2^shift` arithmetic matches the existing parser.
-- Offsets are 0-based; returns (value, next_pos).
-- Promoted from `local function` to M.* exports so passes/dwarf_injection.lua
-- can import them as file-scope locals per the 2nd-caller lift precedent
-- (the uleb128 + sleb128 encoders were promoted the same way).
function M.read_uleb128_at(buf, pos)
local value, shift = 0, 0
local len = #buf
while pos < len do
local b = buf:byte(pos + 1)
value = value + (b % 0x80) * (2 ^ shift)
shift = shift + 7
pos = pos + 1
if b < 0x80 then return value, pos end
end
return nil, pos
end
function M.read_sleb128_at(buf, pos)
local value, shift = 0, 0
local len = #buf
while pos < len do
local b = buf:byte(pos + 1)
value = value + (b % 0x80) * (2 ^ shift)
shift = shift + 7
pos = pos + 1
if b < 0x80 then
if b >= 0x40 then value = value - (2 ^ shift) end
return value, pos
end
end
return nil, pos
end
-- Find the 0-based offset of the table-terminator byte (a single 0) for the abbrev table starting at `table_start`.
-- Returns nil on truncated input. Walks declaration headers
-- (code, tag, has_children, attr/form pairs, DW_FORM_implicit_const constant) until it finds a 0 byte that follows a complete declaration.
function M.find_abbrev_table_end(table_bytes, table_start)
local pos, len = table_start, #table_bytes
if pos >= len or table_bytes:byte(pos + 1) == 0 then return pos end
while pos < len do
local _code, code_end = M.read_uleb128_at(table_bytes, pos)
if not _code then return nil end
pos = code_end
local _tag, tag_end = M.read_uleb128_at(table_bytes, pos)
if not _tag then return nil end
pos = tag_end
if pos >= len then return nil end
pos = pos + 1 -- has_children byte
while pos < len do
local attr, attr_end = M.read_uleb128_at(table_bytes, pos)
if not attr then return nil end
pos = attr_end
local form, form_end = M.read_uleb128_at(table_bytes, pos)
if not form then return nil end
pos = form_end
if attr == 0 and form == 0 then break end
if form == DW_FORM_implicit_const then
local _c, ce = M.read_sleb128_at(table_bytes, pos)
if not _c then return nil end
pos = ce
end
end
if pos >= len then return nil end
if table_bytes:byte(pos + 1) == 0 then return pos end
end
return nil
end
-- Read the null-terminated C string at 0-based offset `off` in `buf`.
-- Stops at the first 0 byte or end of buffer.
local function read_c_string_at(buf, off)
local len = #buf
local start = off
while off < len and buf:byte(off + 1) ~= 0 do off = off + 1 end
return buf:sub(start + 1, off)
end
-- Walk the .debug_abbrev table starting at 0-based offset `table_start` and return a list of declarations:
-- {code, tag, has_children, attrs={ {name, form}, ... }}.
-- Stops at the table terminator.
local function parse_abbrev_table(table_bytes, table_start)
local table_end = M.find_abbrev_table_end(table_bytes, table_start)
if not table_end then return nil, "no terminator" end
local decls = {}
local pos = table_start
while pos < table_end do
local code, code_end = M.read_uleb128_at(table_bytes, pos)
if not code then return nil, "truncated code" end
pos = code_end
local tag, tag_end = M.read_uleb128_at(table_bytes, pos)
if not tag then return nil, "truncated tag" end
pos = tag_end
local has_children = table_bytes:byte(pos + 1)
pos = pos + 1
local attrs = {}
while true do
local attr, attr_end = M.read_uleb128_at(table_bytes, pos)
if not attr then return nil, "truncated attr" end
pos = attr_end
local form, form_end = M.read_uleb128_at(table_bytes, pos)
if not form then return nil, "truncated form" end
pos = form_end
if attr == 0 and form == 0 then break end
attrs[#attrs + 1] = { name = attr, form = form }
if form == DW_FORM_implicit_const then
local _c, ce = M.read_sleb128_at(table_bytes, pos)
if not _c then return nil, "truncated const" end
pos = ce
end
end
decls[#decls + 1] = { code = code, tag = tag, has_children = has_children, attrs = attrs }
end
return decls
end
-- Read a ULEB attribute value at 0-based offset `pos` for the given `form`.
-- Returns (value, next_pos). For DW_FORM_string we return the inline string.
-- For DW_FORM_strp we return the inline string resolved from `str_buf`.
-- For DW_FORM_ref4 we return the absolute CU-relative offset.
-- The caller decides whether to interpret that as a section offset.
local function read_form_value(buf, str_buf, pos, form)
if form == M.DW_FORM.addr then
return M.read_u32_le(buf, pos), pos + 4
elseif form == M.DW_FORM.string then
local s = read_c_string_at(buf, pos)
return s, pos + #s + 1
elseif form == M.DW_FORM.strp then
-- DW_FORM_strp: 4-byte offset into .debug_str.
local strp_off = M.read_u32_le(buf, pos)
return read_c_string_at(str_buf, strp_off), pos + 4
elseif form == M.DW_FORM.udata then return M.read_uleb128_at(buf, pos)
elseif form == M.DW_FORM.data1 then return buf:byte(pos + 1), pos + 1
elseif form == M.DW_FORM.data2 then return M.read_u16_le(buf, pos), pos + 2
elseif form == M.DW_FORM.data4 then return M.read_u32_le(buf, pos), pos + 4
elseif form == M.DW_FORM.ref4 then return M.read_u32_le(buf, pos), pos + 4
elseif form == M.DW_FORM.sec_offset then
-- DW_FORM_sec_offset: 4-byte offset (size depends on DWARF version;
-- on DWARF5 32-bit it's always 4 bytes).
return M.read_u32_le(buf, pos), pos + 4
elseif form == M.DW_FORM.flag_present then
return 1, pos
elseif form == M.DW_FORM.exprloc then
-- DW_FORM_exprloc: ULEB byte count + that many bytes of DW_OP_*.
local len, ne = M.read_uleb128_at(buf, pos)
if not len then return nil, pos end
return nil, ne + len
elseif form == DW_FORM_implicit_const then
-- The constant is declared in the abbrev; no value bytes in the DIE.
return nil, pos
elseif form == M.DW_FORM.ref_sig8 then
-- DW_FORM_ref_sig8 (DWARF5 §7.4.2): an 8-byte value identifying a type
-- by signature. The low 4 bytes (LE) are the type signature (content hash);
-- the high 4 bytes (LE) are a CU-relative offset into the matching type unit.
-- Consumers use the low 4 to look up the type unit (see M.find_type_unit_by_signature)
-- then the high 4 to resolve the specific type within it.
-- Return the low 4 as the primary value to preserve the (value, next_pos) shape;
-- the high 4 is exposed via M.read_ref_sig8 (which returns both halves).
local _, _, next_pos = M.read_ref_sig8(buf, pos)
return M.read_u32_le(buf, pos), next_pos
else
return nil, pos
end
end
--- Read a `DW_FORM_ref_sig8` value at 0-based offset `pos` from `buf`.
--- Returns the low 4 bytes (LE) as `low`, the high 4 bytes (LE) as `high`, and the cursor position after the 8-byte value as `next_pos`.
--- Callers that need the full type-unit + type-offset pair (e.g. to resolve a type identifier embedded as a signature)
--- should use this directly rather than going through `read_form_value`,
--- which only exposes the low 4 bytes to preserve its existing (value, next_pos) return shape.
--- @param buf string
--- @param pos integer -- zero-based wire offset
--- @return integer -- low 4 bytes (LE), the type signature
--- @return integer -- high 4 bytes (LE), the offset within the matching type unit
--- @return integer -- cursor after the 8-byte value
function M.read_ref_sig8(buf, pos)
return M.read_u32_le(buf, pos), M.read_u32_le(buf, pos + 4), pos + 8
end
-- DWARF5 §7.5.6 (Type Entries).
-- Walk all units in `info` and return the 0-based offset of the first unit
-- whose `DW_AT_type_signature` (8-byte value at the end of the unit header) equals `target_sig`.
-- The signature is interpreted as two 32-bit halves (low/high) per the read_ref_sig8 contract;
-- we match both halves (i.e. the 8-byte value as a whole). Returns nil if no matching unit exists.
--
-- Unit header layout (from pos 0):
-- unit_length(4) + version(2) + unit_type(1) + address_size(1) + debug_abbrev_offset(4)
-- followed by type_unit_specific fields: type_signature(8) + type_offset(4)
-- The type_signature is at byte offset 8 of the body (right after debug_abbrev_offset).
-- @param info string -- the .debug_info section bytes
-- @param target_sig_lo integer -- low 4 bytes (LE) of the desired signature
-- @param target_sig_hi integer -- high 4 bytes (LE) of the desired signature
-- @return integer|nil, integer|nil -- unit offset, type_offset within the unit
function M.find_type_unit_by_signature(info, target_sig_lo, target_sig_hi)
local pos = 0
local section_len = #info
while pos + 4 < section_len do
local unit_length = M.read_u32_le(info, pos)
if unit_length == 0xFFFFFFFF then
return nil, nil -- DWARF64 not supported
end
-- unit_length is the body size, NOT including the 4-byte unit_length field itself.
local body_start = pos + 4
local body_end = body_start + unit_length
if body_end > section_len then
return nil, nil -- malformed
end
-- Per DWARF5 §7.5.6, the type_unit (DW_UT_type = 0x02) body layout is:
-- 0: version (2)
-- 2: unit_type (1) -- DW_UT_type = 0x02
-- 3: address_size (1)
-- 4: debug_abbrev_offset (4)
-- 8: type_signature (8)
-- 16: type_offset (4)
-- 20: <children>
if body_end - body_start >= 20 then
-- read_ref_sig8 / write_u32_le / etc. are 1-indexed (string:byte);
-- pos / body_start / body_end are 0-based wire offsets, so the
-- 1-indexed byte at 0-based wire offset X is string:byte(X + 1).
-- Per DWARF5 §7.5.6, the type_unit body is laid out as:
-- byte 0-1: version (2)
-- byte 2: unit_type (1) -- DW_UT_type = 0x02
-- byte 3: address_size (1)
-- byte 4-7: debug_abbrev_offset (4)
-- byte 8-15: type_signature (8)
-- byte 16-19: type_offset (4)
local unit_type = info:byte(body_start + 2 + 1) -- 0-based +2 = unit_type in 1-indexed
if unit_type == 0x02 then -- DW_UT_type
local sig_lo, sig_hi, _ = M.read_ref_sig8(info, body_start + 8) -- 0-based +8 = type_signature in 1-indexed
if sig_lo == target_sig_lo and sig_hi == target_sig_hi then
local type_offset = M.read_u32_le(info, body_start + 16) -- 0-based +16 = type_offset in 1-indexed
return pos, type_offset
end
end
end
-- Advance to the next unit (the 4-byte unit_length + the body).
pos = body_end
end
return nil, nil
end
--- Return a 4-byte little-endian byte string for `value`.
--- Caller concatenates with `..` if composing multi-word blobs.
---
--- **Byte weights** written as `0x100` etc. (see `M.read_u32_le` for rationale).
--- @param value integer -- 0 ≤ value ≤ 0xFFFFFFFF
--- @return string
@@ -199,14 +506,14 @@ end
--- Read the named sections from a post-link ELF32 by walking the ELF32 section-header table directly
--- (no subprocess; lfs only for the existence check). Returns `{[name] = bytes_or_empty_string, ...}`.
---
--- **Convention:** offsets from `M.ELF32` (1-indexed for string.sub).
--- Every requested name has an entry in the returned dict; missing sections have an empty string (NOT nil)
--- so callers can do `sections[".debug_x"] or ""` for the missing case.
--- **Convention:** ELF/DWARF offsets are zero-based wire offsets. Direct Lua string APIs add `+ 1` at the boundary.
--- Every requested name has an entry in the returned dict;
--- missing sections have an empty string (NOT nil) so callers can do `sections[".debug_x"] or ""` for the missing case.
---
--- **Cost:** one file open + one `f:seek` + one `f:read` per section header
--- (we walk all `e_shnum` headers regardless of how many names are requested, to find the .shstrtab first).
--- For frequent callers, pass the union of all needed sections in one call.
-- can add `.debug_info` + `.debug_loc` + `.debug_str_offsets` to the list without writing a 2nd ELF walker.
-- Can add `.debug_info` + `.debug_loc` + `.debug_str_offsets` to the list without writing a 2nd ELF walker.
--- @param elf_path Path
--- @param section_names string[] -- list of section names to read
--- @return table<string, string>
@@ -241,17 +548,17 @@ function M.read_elf_sections(elf_path, section_names)
end
-- Sanity-check magic + class + endianness.
if header:sub(M.ELF32.magic_offset, M.ELF32.magic_offset + 0x03) ~= M.ELF32.magic then
if header:sub(M.ELF32.magic_offset + 1, M.ELF32.magic_offset + 0x04) ~= M.ELF32.magic then
io.stderr:write("[elf_dwarf.read_elf_sections] not an ELF file\n")
f:close()
return result
end
if header:byte(M.ELF32.class_offset) ~= M.ELF32.class_elf32 then
io.stderr:write(string.format("[elf_dwarf.read_elf_sections] not ELF32 (class=%d)\n", header:byte(M.ELF32.class_offset)))
if header:byte(M.ELF32.class_offset + 1) ~= M.ELF32.class_elf32 then
io.stderr:write(string.format("[elf_dwarf.read_elf_sections] not ELF32 (class=%d)\n", header:byte(M.ELF32.class_offset + 1)))
f:close()
return result
end
if header:byte(M.ELF32.endian_offset) ~= M.ELF32.endian_little then
if header:byte(M.ELF32.endian_offset + 1) ~= M.ELF32.endian_little then
io.stderr:write("[elf_dwarf.read_elf_sections] not little-endian; unsupported\n")
f:close()
return result
@@ -305,16 +612,12 @@ end
--- Read ELF symbol addresses by walking the `.symtab` + `.strtab` sections directly (no `nm` subprocess).
--- Returns a map `{name -> {addr, size_bytes}}` for every `code_<name>` symbol.
---
--- **Why direct parsing instead of `mipsel-none-elf-nm -S`?**
--- The `nm` subprocess costs ~50ms per spawn on Windows (cmd.exe + mipsel-none-elf-nm.exe). Parsing `.symtab` ourselves is ~0ms.
--- Same return shape, same `code_` prefix filter.
---
--- **Conventions:**
--- - ELF32 symtab entry = 16 bytes (`st_name:4 + st_value:4 + st_size:4 + st_info:1 + st_other:1 + st_shndx:2`). 1-indexed for Lua string.sub.
--- - ELF32 symtab entry = 16 bytes (`st_name:4 + st_value:4 + st_size:4 + st_info:1 + st_other:1 + st_shndx:2`); offsets within each entry are zero-based wire offsets.
--- - Direct Lua `string.byte`/`string.sub`/`string.find` boundaries receive `+ 1`.
--- - We filter on STB_GLOBAL (high nibble of st_info = 1) to match `nm`'s default (external symbols only). STB_WEAK excluded.
--- - We strip the `code_` prefix to match the previous `read_nm` output.
--- - The `code_` prefix is stripped (MipsAtom_ macros emit bare atom names, no `code_` prefix).
--- - `st_size > 0` filter excludes undefined/imported symbols.
---
--- @param elf_path Path
--- @return table<string, {integer, integer}>
function M.read_nm(elf_path)
@@ -330,32 +633,31 @@ function M.read_nm(elf_path)
end
-- Iterate the 16-byte ELF32 symtab entries.
-- Each entry (1-indexed): st_name at 1, st_value at 5, st_size at 9,
-- st_info at 13, st_other at 14, st_shndx at 15.
-- Each entry (zero-based): st_name at 0, st_value at 4, st_size at 8, st_info at 12, st_other at 13, st_shndx at 14.
local SYM_ENTRY_BYTES = 0x10
local SYM_ST_NAME = 0x01
local SYM_ST_VALUE = 0x05
local SYM_ST_SIZE = 0x09
local SYM_ST_INFO = 0x0D
local SYM_ST_NAME = 0x00
local SYM_ST_VALUE = 0x04
local SYM_ST_SIZE = 0x08
local SYM_ST_INFO = 0x0C
local n_syms = #symtab / SYM_ENTRY_BYTES
for i = 0, n_syms - 1 do
local entry_off = i * SYM_ENTRY_BYTES + 1 -- 1-indexed
local st_info = symtab:byte(entry_off + SYM_ST_INFO - 1)
local entry_off = i * SYM_ENTRY_BYTES
local st_info = symtab:byte(entry_off + SYM_ST_INFO + 1)
-- High nibble = binding (STB_LOCAL=0, STB_GLOBAL=1, STB_WEAK=2).
-- Use math.floor(/16) instead of bit.rshift for LuaJIT 2.1 compat
-- (LuaJIT's `>>` is 5.3+, but math.floor(x/16) works on all versions).
local binding = math.floor(st_info / 16)
if binding == 0 or binding == 1 then -- STB_LOCAL or STB_GLOBAL
local st_size = M.read_u32_le(symtab, entry_off + SYM_ST_SIZE - 1)
local st_size = M.read_u32_le(symtab, entry_off + SYM_ST_SIZE)
if st_size > 0 then
local st_name_off = M.read_u32_le(symtab, entry_off + SYM_ST_NAME - 1)
local st_name_off = M.read_u32_le(symtab, entry_off + SYM_ST_NAME)
-- Extract the name from .strtab (null-terminated C string).
local name_end = strtab:find("\0", st_name_off + 1, true) or (st_name_off + 1)
local name = strtab:sub(st_name_off + 1, name_end - 1)
-- Filter: keep all symbol-table symbols (atoms emit their name as the bare `<name>` since the `code_` prefix was removed from the MipsAtom_ macro).
-- Filter: keep all symbol-table symbols (atoms emit their name as the bare `<name>` — MipsAtom_ macros strip the `code_` prefix).
-- The atoms_source_map pass already filters out non-atom symbols via the source-map.txt cross-ref.
if name and #name > 0 then
local st_value = M.read_u32_le(symtab, entry_off + SYM_ST_VALUE - 1)
local st_value = M.read_u32_le(symtab, entry_off + SYM_ST_VALUE)
addrs[name] = { st_value, st_size }
end
end
@@ -373,7 +675,6 @@ end
-- Both encoders pack 7 bits of data per byte + 1 bit of "more bytes follow" signaling.
--
-- Per-byte layout:
--
-- bit: 7 6 5 4 3 2 1 0
-- │ └───── 7-bit data ─────┘
-- └─ continuation flag (LEB_CONT_BIT = 0x80)
@@ -399,16 +700,19 @@ local LEB_DATA_MASK = 0x7F
local SLEB_SIGN_BIT = 0x40
--- ULEB128 (Unsigned Little-Endian Base 128) encoder. Returns the byte string for the non-negative integer `n`.
---
--- Algorithm:
--- - Extract the low 7 bits of `n` (LEB_DATA_MASK = 0x7F).
--- - Shift `n` right by 7 bits.
--- - If more bytes remain, OR in the continuation flag (LEB_CONT_BIT).
--- - Repeat until `n` is fully consumed.
---
--- @param n integer -- non-negative
--- @return string
function M.uleb128(n)
if n == nil or type(n) ~= "number" then
io.stderr:write("[elf_dwarf.uleb128] got " .. type(n) .. ": " .. tostring(n) .. "\n")
io.stderr:write(debug.traceback() .. "\n")
error("uleb128 requires non-negative number")
end
assert(n >= 0, "uleb128 requires non-negative input")
local bytes = {}
repeat
@@ -420,26 +724,22 @@ function M.uleb128(n)
return table.concat(bytes)
end
--- SLEB128 (Signed Little-Endian Base 128) encoder. Returns the byte
--- string for the integer `n` (may be negative).
---
--- Algorithm differs from ULEB128 by the termination condition: stop when
--- the remaining bits can be inferred from the sign bit in the last byte's
--- 7-bit data payload.
--- SLEB128 (Signed Little-Endian Base 128) encoder. Returns the byte string for the integer `n` (may be negative).
--- Algorithm differs from ULEB128 by the termination condition:
--- stop when the remaining bits can be inferred from the sign bit in the last byte's 7-bit data payload.
--- - If `n == 0` (no more value bits) AND bit 6 of the data = 0 → positive terminator (sign bit says "zero-extend").
--- - If `n == -1` (sign-extended all-1s) AND bit 6 of the data = 1 → negative terminator (sign bit says "one-extend").
---
--- Without these checks, the decoder would round-trip to a different value
--- (e.g. encoding `0` as `0x80 0x00` decodes to `0` correctly but is 2 bytes long; the termination check picks the 1-byte `0x00` form).
---
--- @param n integer -- any integer (negative allowed)
--- @return string
function M.sleb128(n)
local bytes = {}
local more = true
while more do
local b = n % (LEB_DATA_MASK + 1) -- extract low 7 bits
n = (n - b) / (LEB_DATA_MASK + 1) -- arithmetic shift right by 7
local b = n % (LEB_DATA_MASK + 1) -- extract low 7 bits
n = (n - b) / (LEB_DATA_MASK + 1) -- arithmetic shift right by 7
-- Termination: remaining value bits fit in the sign bit of the last byte.
if n == 0 and b < SLEB_SIGN_BIT then more = false end -- positive terminator
if n == -1 and b >= SLEB_SIGN_BIT then more = false end -- negative terminator
@@ -449,66 +749,46 @@ function M.sleb128(n)
return table.concat(bytes)
end
--- ULEB128 byte-length: number of bytes the encoder M.uleb128 would produce for `n`.
--- Used by callers that need to size a buffer before encoding (e.g. compute_loclists_offsets
--- needs the encoded length of an `uleb128(4)` for a `DW_OP_piece + uleb128(U4_BYTE_SIZE)` tail).
--- @param n integer -- non-negative
--- @return integer -- 1..5 for n in [0, 2^32)
function M.uleb128_size(n)
assert(n >= 0, "uleb128_size requires non-negative input")
if n == 0 then return 1 end
local bytes = 1
while n >= 0x80 do
n = (n - (n % (LEB_DATA_MASK + 1))) / (LEB_DATA_MASK + 1) -- arithmetic shift right by 7
bytes = bytes + 1
end
return bytes
end
--- SLEB128 byte-length: number of bytes the encoder M.sleb128 would produce for `n`.
--- Used by callers that need to size a buffer before encoding.
--- (e.g. compute_loclists_offsets needs the encoded length of an `sleb128(field.offset)` in a tape piece).
--- Handles the signed DWARF5 termination: positive terminator if (n == 0) and bit 6 of last byte is unset;
--- negative terminator if (n == -1) and bit 6 of last byte is set.
--- @param n integer -- any integer (negative allowed)
--- @return integer
function M.sleb128_size(n)
local more = true
local bytes = 0
local v = n
while more do
local b = v % (LEB_DATA_MASK + 1) -- extract low 7 bits
v = (v - b) / (LEB_DATA_MASK + 1) -- arithmetic shift right by 7
if v == 0 and b < SLEB_SIGN_BIT then more = false end -- positive terminator
if v == -1 and b >= SLEB_SIGN_BIT then more = false end -- negative terminator
if more then b = b + LEB_CONT_BIT end
bytes = bytes + 1
end
return bytes
end
-- ════════════════════════════════════════════════════════════════════════════
-- I/O helpers: atoms source-map + native directory glob
-- ════════════════════════════════════════════════════════════════════════════
--- Parse a FORMAT_VERSION <expected_version> `*.atoms.sourcemap.txt` file.
--- Returns `{name -> {total = N, words = {{pos, line}, ...}}}`.
--- Returns `{}` on format-version mismatch (and logs to stderr).
---
--- **Wire format** (emitted by `passes/atoms_source_map.lua`):
--- ```
--- # FORMAT_VERSION <n>
--- ATOM <name> "<abs-source-path>" <total>
--- WORD <n> LINE <line> TEXT <text...>
--- ...
--- ENDATOM
--- ```
---
--- **Conventions:** the in-memory shape uses `{pos, line, text}`
--- (`atoms_source_map.lua:142`); the `.txt` file uses `WORD <n>` so the parser maps `n` → `pos` field name.
--- @param sm_path Path
--- @param expected_version integer -- expected FORMAT_VERSION line
--- @return table<string, table>
function M.parse_source_map_file(sm_path, expected_version)
local out = {}
local cur_name, cur_words = nil, {}
for raw in io.lines(sm_path) do
local line = raw
if line:match("^#") then
local ver = line:match("^# FORMAT_VERSION%s+(%d+)")
if ver and tonumber(ver) ~= expected_version then
io.stderr:write(string.format(
"[elf_dwarf.parse_source_map_file] source-map version mismatch (got %s, expected %d) in %s\n",
ver, expected_version, sm_path))
return {}
end
-- skip other comments
elseif line:sub(1, 4) == "ATOM" then
-- ATOM <name> "<abs-source-path>" <total>
local _, _, name = line:find("ATOM%s+(%S+)%s+\"[^\"]*\"%s+(%d+)")
if name then
cur_name = name
cur_words = {}
out[name] = { total = 0, words = cur_words }
end
elseif line == "ENDATOM" then
-- Update the recorded total from the entries count
-- (matches the `lines[1] = lines[1]:gsub(" 0$", " " .. total)` patch in atoms_source_map.lua:170).
if cur_name and out[cur_name] then
out[cur_name].total = #cur_words
end
cur_name, cur_words = nil, {}
elseif line:sub(1, 4) == "WORD" and cur_name then
-- WORD <n> LINE <line> TEXT <text...>
local _, n, _, src_line = line:find("WORD%s+(%d+)%s+LINE%s+(%d+)")
if n and src_line then
cur_words[#cur_words + 1] = { pos = tonumber(n), line = tonumber(src_line) }
end
end
end
return out
end
return M
+15 -29
View File
@@ -1,29 +1,15 @@
# scripts/gdb/gdb_tape_atoms.gdb
#
# Wrapper for the tape-atom step-debug helpers. The 9 user commands are defined
# here as STUBS (degraded-state messages). The real implementations + the
# per-atom data tables are emitted by `passes/atoms_source_map.lua` (post-link
# invocation: `ps1_meta.lua --atoms-source-map --gdb-runtime --elf <elf>`) into
# `build/gen/gdb_tape_atoms_runtime.gdb`. Sourcing that file RE-DEFINES the
# commands with real implementations.
# Wrapper for the tape-atom step-debug helpers.
# The 9 user commands are defined here as STUBS (degraded-state messages).
# The real implementations + the per-atom data tables are emitted by `passes/atoms_source_map.lua`
# (post-link invocation: `ps1_meta.lua --atoms-source-map --gdb-runtime --elf <elf>`) into `build/gen/gdb_tape_atoms_runtime.gdb`.
# Sourcing that file RE-DEFINES the commands with real implementations.
#
# If `build/gen/gdb_tape_atoms_runtime.gdb` is missing or stale, the stubs
# remain (E1: no source map). The user just needs to re-run `build_psyq.ps1`
# to regenerate. No exceptions; no crashes.
#
# Why a wrapper + separate runtime file?
# - The runtime file is auto-generated per-build; not in git.
# - The wrapper is checked into git; always works.
# - This split keeps the script trivial and the data plumbing out of git.
#
# Compatible with every gdb build (no Python, no Tcl, no Guile required) —
# pure gdb command scripting + `set $var = val` + `define ... end`.
#
# Generated by track gdb_tape_atom_debugging_20260711 — see
# C:\projects\Pikuma\ps1-ai\docs\gdb_tape_atom_debugging.md for the manual.
# If `build/gen/gdb_tape_atoms_runtime.gdb` is missing or stale, the stubs remain (E1: no source map).
# The user just needs to re-run `build_psyq.ps1` to regenerate.
# ── Stub commands (defined here so they're always present, even if the
# runtime file is missing). The runtime file overrides these if sourced. ──
# ── Stub commands (defined here so they're always present, even if the runtime file is missing). The runtime file overrides these if sourced. ──
define tape_atoms
echo "[gdb_tape_atoms] STUB: runtime file build/gen/gdb_tape_atoms_runtime.gdb not found."
@@ -77,7 +63,7 @@ define show_c2
printf "C2[14] 0x%08x [sxy2]\n", $c2_data[14]
printf "C2[24] 0x%08x [mac0]\n", $c2_data[24]
printf "...\n"
echo "(STUB state: only 7 representative regs shown. Run build_psyq.ps1 for full dump.)"
echo "(STUB state: only 7 representative regs shown. Run build_psyq.ps1 for full dump.)"
end
document show_c2
Pretty-print all 32 C2 data registers as hex + named alias. STUB state (7 reg subset).
@@ -107,13 +93,13 @@ end
# Try to source from project-root-relative path first (the typical case).
# If the user is in a different CWD, the source will fail and stubs remain.
# The runtime file path is computed relative to the ELF's source map convention
# (build/gen/gdb_tape_atoms_runtime.gdb).
# The runtime file path is computed relative to the ELF's source map convention (build/gen/gdb_tape_atoms_runtime.gdb).
echo [gdb_tape_atoms] Wrapper loaded. Sourcing runtime file...
# Suppress the "Redefine command" prompts that would otherwise appear when the
# runtime file overrides the 9 stub commands defined above. The runtime's
# `define` blocks are intended to overwrite — there's no ambiguity to confirm.
# Suppress the "Redefine command" prompts that would otherwise appear when the runtime file overrides the 9 stub commands defined above.
# The runtime's `define` blocks are intended to overwrite — there's no ambiguity to confirm.
set confirm off
# Source the runtime file (re-defines commands with real impls + data).
source build/gen/gdb_tape_atoms_runtime.gdb
set confirm on
echo [gdb_tape_atoms] Runtime sourced successfully (9 commands now have real implementations).
echo [gdb_tape_atoms] Runtime sourced successfully (9 commands now have real implementations).
+382 -138
View File
@@ -1,31 +1,28 @@
--- passes/annotation.lua — Atom-annotation DSL validator.
---
--- Validates `MipsAtom_(name) atom_info(atom_bind(Binds_X), atom_reads(...), atom_writes(...)) { ... }` declarations in source files.
--- Also reads: `Binds_*` struct declarations (`typedef Struct_(Binds_X) { ... };`)
--- Also reads `Binds_*` struct declarations (`typedef Struct_(Binds_X) { ... };`).
---
--- Source scanning: done ONCE upstream by `duffle.scan_source()` (ps1_meta.lua pre-scans each source and stashes the result in `src.scan`).
--- `duffle.scan_source()` scans each source once upstream; `ps1_meta.lua` stores that result in `src.scan`.
---
--- Writes:
--- - `<ctx.out_root>/<dir_basename>.errors.h` — one per module, with `#error` directives on findings (the C compile will surface the error)
--- - The annotations.txt report is rendered by `passes/report.lua` from the per-module results stashed in `ctx.flags._annot_results`
--- Ownership: the canonical `ctx.shared.corpus` supplies cross-source registries, while each `src.scan` supplies its source's declarations and bodies.
--- A context without `ctx.shared.corpus` is rejected with an explicit canonical-corpus message.
---
--- **Conventions**: tabs (1/level), EmmyLua annotations, no regex, Lua 5.3 compatible
--- Writes `<ctx.out_root>/<dir_basename>.errors.h` once per module, with `#error` directives for findings that the C compile surfaces.
--- `passes/report.lua` renders annotations.txt from `corpus.sources_by_dir`, re-validating each source through `M.validate()`.
---
--- **Conventions**: tabs (1/level), EmmyLua annotations, no regex, Lua 5.3 compatible.
-- Bootstrap: same as entry scripts. See `ps1_meta.lua` for the rationale.
-- Bootstrap: load `scripts/duffle_paths.lua` (sets package.path + package.cpath).
-- Uses `debug.getinfo` to find this file's own directory, so it works
-- both standalone and when require'd from the orchestrator.
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source` (works both standalone + when require'd).
-- duffle_paths.lua sets package.path then returns `require("duffle")` at the bottom, so the dofile value IS the duffle module.
-- Bootstrap follows the entry scripts; `scripts/duffle_paths.lua` sets package.path and package.cpath. See `ps1_meta.lua` for the rationale.
-- `debug.getinfo(1, "S").source` locates this file for standalone and orchestrated runs, then `duffle_paths.lua` returns the loaded `duffle` module.
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
local write_file = duffle.write_file
local ensure_dir = duffle.ensure_dir
-- Domain tables (single source of truth in duffle.lua).
local WAVE_CONTEXT_REGS = duffle.WAVE_CONTEXT_REGS
local function is_wave_context_reg(n) return WAVE_CONTEXT_REGS[n] ~= nil end
-- The annotation pass reads the source-derived registries from scan_source:
-- * pipe_ctx.register_alias_registry — for atom_dbg_reg_default(R_X, ...) and atom_reg_types(R_X, ...) member-identity checks
-- * pipe_ctx.type_name_registry — for atom_dbg_reg_default(<T>, ...) and atom_reg_types(<T>, ...) type-identity checks
-- ════════════════════════════════════════════════════════════════════════════
-- Type declarations
@@ -47,8 +44,6 @@ local function is_wave_context_reg(n) return WAVE_CONTEXT_REGS[n] ~= nil end
--- @field project_root string
--- @field upstream table<string, table>
--- @field flags table
--- @field flags._annot_results table[] -- stashed by annotation pass; consumed by report.lua
--- @field dry_run boolean
--- @field verbose boolean
--- @class PassResult
@@ -66,9 +61,19 @@ local function is_wave_context_reg(n) return WAVE_CONTEXT_REGS[n] ~= nil end
--- @field writes string[] -- R_* names (write targets)
--- @field errors string[]|nil -- parse-time errors from scan_source (atom_info body malformed)
--- @class DebugSkipMarker -- sub-shape of scan_source.lua's @class DebugSkipMarker
--- @field marker_kind string -- exact marker ident read from source. Only "atom_dbg_skip" (bare) is positive.
--- @field marker_line integer
--- @field args string|nil -- trimmed text inside the parens (nil when has_parens is false)
--- @field has_parens boolean
--- @field is_bare boolean -- true iff marker_kind == "atom_dbg_skip" AND has_parens == false (the only positive form)
--- @field pending boolean -- true while awaiting the following declaration
--- @field superseded_by_marker_line integer|nil -- set on a marker that was bumped out of the pending slot
--- @field target_kind string|nil -- "atom" | "comp_bare" | "comp_proc" | "unrelated" once observed
--- @class Finding
--- @field line integer -- source line (or 0 for pass-level)
--- @field msg string -- finding message
--- @field line integer -- source line (or 0 for pass-level)
--- @field msg string -- finding message
--- @class Findings
--- @field errors Finding[]
@@ -76,9 +81,14 @@ local function is_wave_context_reg(n) return WAVE_CONTEXT_REGS[n] ~= nil end
--- @field info Finding[]
--- @class PipeCtx
--- @field atom_index table<string, AtomAnnotation> -- name -> AtomAnnotation (only kind=="atom")
--- @field binds_index table<string, BindsStruct> -- name -> BindsStruct
--- @field annot_counts table<string, integer> -- name -> annotation count (for unique_annotation check)
--- @field atom_index table<string, AtomAnnotation> -- name -> AtomAnnotation (only kind=="atom")
--- @field binds_index table<string, BindsStruct> -- name -> BindsStruct
--- @field annot_counts table<string, integer> -- name -> annotation count (for unique_annotation check)
--- @field types table<string, RegTypeDefault> -- from scan_source
--- @field atom_views table<string, AtomViewEntry> -- from scan_source
--- @field seen_defaults table<string, integer> -- duplicate atom_dbg_reg_default detection
--- @field seen_field table<string, integer> -- Binds_* -> count of fields (set/checked by check_binds_no_duplicate_fields)
--- @field _scan SourceScan -- full scan payload (typed-view sub-calls live here)
--- @class AnnotatedResult
--- @field atoms AtomEntry[]
@@ -93,14 +103,11 @@ local function is_wave_context_reg(n) return WAVE_CONTEXT_REGS[n] ~= nil end
-- Per-check functions (the CHECK_RULES table's payload)
-- ════════════════════════════════════════════════════════════════════════════
--
-- Each check has a uniform `append_to_findings` shape (errors[] / warnings[] / info[]).
-- The dispatcher in `validate()` decides which findings list each check writes to — by convention,
-- "existence" checks (declaration must exist, struct must exist) write errors[]; "shape" checks
-- (writes/reads must be wave-context) write warnings[]. The `macro_word_drift` check writes
-- both errors[] (missing/mismatch) and info[] (match).
--- The dispatcher in `validate()` routes each result by convention: existence checks write errors[] and shape checks write warnings[].
--- `macro_word_drift` writes errors[] for missing or mismatched metadata and info[] for a match.
--- Check: every annotated atom must have a matching MipsAtom_(name) declaration.
--- @param a AtomAnnotation
--- @param a AtomAnnotation
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_atom_decl_exists(a, pipe_ctx, findings)
@@ -128,15 +135,12 @@ local function check_unique_annotation(pipe_ctx, findings)
end
--- Check: BIND atoms must reference a real Binds_* struct.
--- Demoted from error to warning (2026-07-10): the same condition is now caught by passes/static_analysis.lua's
--- check_abi_handoff() as an error. Emitting a warning here keeps the annotation pass from being stop-on-error
--- for the common test-fixture case, while still surfacing the issue in the report.
--- The static-analysis report remains the source of truth for build-stopping errors.
--- @param a AtomAnnotation
--- I keep this as a warning so the annotation pass can report the common test-fixture case; `check_abi_handoff` in static analysis supplies the build-stopping error.
--- @param a AtomAnnotation
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_binds_struct_exists(a, pipe_ctx, findings)
if not a.binds then return end
if not a.binds then return end
if pipe_ctx.binds_index[a.binds] then return end
findings.warnings[#findings.warnings + 1] = {
line = a.line,
@@ -146,57 +150,13 @@ local function check_binds_struct_exists(a, pipe_ctx, findings)
}
end
--- Check: Binds_* struct fields must correspond to known wave-context registers.
--- Also checks that all `atom_writes(...)` entries are wave-context registers.
--- @param a AtomAnnotation
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_binds_field_wave_context(a, pipe_ctx, findings)
if not (a.binds and pipe_ctx.binds_index[a.binds]) then return end
local bs = pipe_ctx.binds_index[a.binds]
for _, f in ipairs(bs.fields) do
local candidate = "R_" .. f.name
if not is_wave_context_reg(candidate) then
findings.warnings[#findings.warnings + 1] = {
line = bs.line,
msg = string.format("%s field '%s' doesn't match a known wave-context register (candidate '%s')", a.binds, f.name, candidate),
}
end
end
for _, w in ipairs(a.writes) do
if not is_wave_context_reg(w) then
findings.warnings[#findings.warnings + 1] = {
line = a.line,
msg = string.format("%s writes '%s' which is not a known wave-context register", a.name, w),
}
end
end
end
--- Check: atom_reads(...) entries should be wave-context registers (or R_TapePtr for rbind).
--- @param a AtomAnnotation
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_reads_wave_context(a, pipe_ctx, findings)
for _, r in ipairs(a.reads) do
if not is_wave_context_reg(r) and r ~= "R_TapePtr" then
findings.warnings[#findings.warnings + 1] = {
line = a.line,
msg = string.format("atom '%s' reads '%s' which is not a known wave-context register", a.name, r),
}
end
end
end
--- Check: TAPE_WORDS(mac_X, N) ↔ WORD_COUNT(mac_X, N) drift.
--- Three outcomes: missing (error), mismatch (error), match (info).
--- @param m MacroEntry
--- @param wc table<string, integer> -- the shared word-count table (from ctx.shared.word_counts)
--- @param m MacroEntry
--- @param wc table<string, integer> -- the shared word-count table (from ctx.shared.word_counts)
--- @param findings Findings
local function check_macro_word_drift(m, wc, findings)
local declared = wc[m.name]
local declared = wc[m.name]
if not declared then
findings.errors[#findings.errors + 1] = {
line = m.line,
@@ -217,38 +177,313 @@ local function check_macro_word_drift(m, wc, findings)
}
end
--- Check: atom_dbg_reg_default(R_X, <type>) targets an alias in `pipe_ctx.register_alias_registry` and a type in `pipe_ctx.type_name_registry`.
--- Pointer depth remains bounded to 0 or 1, and duplicate defaults remain errors.
--- @param _src SourceFile -- unused (kept for the per_source shape)
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_semantic_reg_defaults(_src, pipe_ctx, findings)
-- Detect duplicate defaults using the ordered occurrence list (the out.types hash only retains the last declaration).
local seen_first_line = {}
for _, occ in ipairs(pipe_ctx.type_occurrences or {}) do
if seen_first_line[occ.reg] == nil then
seen_first_line[occ.reg] = occ.source_line
else
findings.errors[#findings.errors + 1] = {
line = occ.source_line,
msg = string.format(
"duplicate atom_dbg_reg_default for %q at line %d (first declared at line %d); one default per register",
occ.reg, occ.source_line, seen_first_line[occ.reg]),
}
end
end
local reg_registry = pipe_ctx.register_alias_registry or {}
local type_registry = pipe_ctx.type_name_registry or {}
for reg, def in pairs(pipe_ctx.types or {}) do
if not reg_registry[reg] then
findings.errors[#findings.errors + 1] = {
line = def.source_line,
msg = string.format(
"atom_dbg_reg_default at line %d references unknown register %q (not in register_alias_registry)",
def.source_line, reg),
}
end
if def.pointer_depth == nil or def.pointer_depth < 0 or def.pointer_depth > 1 then
findings.errors[#findings.errors + 1] = {
line = def.source_line,
msg = string.format(
"atom_dbg_reg_default at line %d for %q has unsupported pointer depth %d (expected 0 or 1)",
def.source_line, reg, def.pointer_depth or -1),
}
end
if not def.type_name or not type_registry[def.type_name] then
findings.errors[#findings.errors + 1] = {
line = def.source_line,
msg = string.format(
"atom_dbg_reg_default at line %d for %q uses unknown type %q (not in type_name_registry)",
def.source_line, reg, tostring(def.type_name)),
}
end
end
end
--- Check: atom_reg_types(R_X, <type>) entries target an alias in `pipe_ctx.register_alias_registry` and a type in `pipe_ctx.type_name_registry`.
--- A bare `atom_reg` marker opts the `R_<n>` alias into GPR identity; references to R_T0..R_T3 require the same explicit marker.
--- @param _src SourceFile
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_atom_reg_types(_src, pipe_ctx, findings)
local reg_registry = pipe_ctx.register_alias_registry or {}
local type_registry = pipe_ctx.type_name_registry or {}
for _, ai in ipairs(pipe_ctx.atom_infos_list or {}) do
if ai.reg_type_overrides then
for reg, ov in pairs(ai.reg_type_overrides) do
if not reg_registry[reg] then
findings.errors[#findings.errors + 1] = {
line = ai.info_line,
msg = string.format(
"atom '%s' has atom_reg_types for %q; compute-register types are restricted to opt-in aliases (%q not in register_alias_registry)",
ai.atom_name, reg, reg),
}
end
if not ov.type_name or not type_registry[ov.type_name] then
findings.errors[#findings.errors + 1] = {
line = ai.info_line,
msg = string.format(
"atom '%s' atom_reg_types for %q uses unknown compute type %q (not in type_name_registry)",
ai.atom_name, reg, tostring(ov.type_name)),
}
end
end
end
end
end
--- Check: atom_view(Binds_X) entries reference a Binds_* struct with at least one field.
--- @param _src SourceFile
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_atom_view_layout(_src, pipe_ctx, findings)
for atom_name, view in pairs(pipe_ctx.atom_views or {}) do
if not view.binds_name then
-- The atom had atom_reg_types but no atom_view; no layout check needed.
else
local bs = pipe_ctx.binds_index[view.binds_name]
if not bs then
findings.errors[#findings.errors + 1] = {
line = view.info_line,
msg = string.format(
"atom '%s' has atom_view(%s) but no Struct_(%s) { ... } declaration was found",
atom_name, view.binds_name, view.binds_name),
}
elseif not bs.fields or #bs.fields == 0 then
findings.errors[#findings.errors + 1] = {
line = bs.line,
msg = string.format(
"atom '%s' has atom_view(%s) but that struct declares zero typed fields",
atom_name, view.binds_name),
}
end
end
end
end
--- Check: Binds_* structs require unique field names because atom_view uses those names for typed-field lookup in gdb.
--- @param _src SourceFile
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_binds_no_duplicate_fields(_src, pipe_ctx, findings)
for _, bs in ipairs(pipe_ctx.binds_list or {}) do
local seen = {}
for _, f in ipairs(bs.fields or {}) do
seen[f.name] = (seen[f.name] or 0) + 1
end
for name, count in pairs(seen) do
if count > 1 then
findings.errors[#findings.errors + 1] = {
line = bs.line,
msg = string.format(
"%s has duplicate field name %q (count %d); the typed-view contract requires unique field names",
bs.name, name, count),
}
end
end
end
end
-- Check: debug-skip markers must satisfy shape + placement constraints.
--- Walks the priority list once; each marker produces at most one error, so one source defect yields one finding.
--- Priority order (first defect wins):
--- 1. marker_kind ~= "atom_dbg_skip" -> legacy/renamed spelling (use `atom_dbg_skip`)
--- 2. marker_kind == "atom_dbg_skip" AND has_parens -> parenthesized form (the marker is bare-only)
--- 3. args ~= "" -> takes no arguments
--- 4. superseded_by_marker_line -> duplicate marker (cite superseding line)
--- 5. pending + no target_kind -> dangling (no following declaration)
--- 6. unsupported target_kind -> marker precedes an unrelated declaration
--- Valid markers stamp `debug_skip` on whole-atom, bare-component, and proc-component declaration records in scan_source.lua.
--- @param marker DebugSkipMarker
--- @param _pipe_ctx PipeCtx -- unused today; kept for plex-shape consistency with per_annot
--- @param findings Findings
local function check_skip_marker(marker, _pipe_ctx, findings)
local kind = marker.marker_kind
local line = marker.marker_line
-- Left `scan.debug_skip_markers` with production records for `atom_dbg_skip` only; other identifiers take the walker's unrelated branch.
if marker.has_parens then
findings.errors[#findings.errors + 1] = {
line = line,
msg = string.format("%s marker at line %d must be bare; the parenthesized form is no longer accepted (use `atom_dbg_skip MipsAtom_(name) { ... }`)",
kind, line),
}
return
end
if marker.args ~= nil and marker.args ~= "" then
findings.errors[#findings.errors + 1] = {
line = line,
msg = string.format("%s marker at line %d takes no arguments; found %q", kind, line, marker.args),
}
return
end
if marker.superseded_by_marker_line then
findings.errors[#findings.errors + 1] = {
line = line,
msg = string.format("duplicate %s marker at line %d; superseded by another %s marker at line %d"
, kind, line, kind, marker.superseded_by_marker_line),
}
return
end
if marker.pending and not marker.target_kind then
findings.errors[#findings.errors + 1] = {
line = line,
msg = string.format("dangling %s marker at line %d: no following MipsAtom_/MipsAtomComp_/MipsAtomComp_Proc_ declaration"
, kind, line),
}
return
end
if marker.target_kind
and marker.target_kind ~= "atom"
and marker.target_kind ~= "comp_bare"
and marker.target_kind ~= "comp_proc" then
findings.errors[#findings.errors + 1] = {
line = line,
msg = string.format("%s marker at line %d must precede MipsAtom_, MipsAtomComp_, or MipsAtomComp_Proc_; found an unrelated declaration"
, kind, line),
}
end
end
--- Warn when a source references an unregistered alias.
---
--- R_TapePtr, R_AtomJmp, R_PrimCursor, R_FaceCursor, R_VertBase, and R_OtBase opt in through `#define atom_reg` in lottes_tape.h.
--- When a source uses an unregistered R_X, this check emits one pass-level info entry for that source and directs C-ABI register names to explicit alias registration.
--- @param _src SourceFile
--- @param pipe_ctx PipeCtx
--- @param findings Findings
local function check_wave_context_migration(_src, pipe_ctx, findings)
if not (pipe_ctx.types and next(pipe_ctx.types)) then return end
if not (pipe_ctx.atom_infos_list) then return end
local reg_registry = pipe_ctx.register_alias_registry or {}
for _, ai in ipairs(pipe_ctx.atom_infos_list) do
if ai.reg_type_overrides then
for reg, _ in pairs(ai.reg_type_overrides) do
if not reg_registry[reg] then
findings.warnings[#findings.warnings + 1] = {
line = 0,
msg = "wave-context removed; opt in via #define atom_reg in mips.h "
.. "(every R_<alias> that should be visible to the annotation pass "
.. "must be enum-declared with the bare atom_reg marker)",
}
return
end
end
end
end
end
-- ════════════════════════════════════════════════════════════════════════════
-- CHECK_RULES — data-driven check dispatch (the plex pattern)
-- ════════════════════════════════════════════════════════════════════════════
--
-- Each rule entry picks one of three "shapes" of dispatch:
-- per_annot(annot, pipe_ctx, findings) runs once per AtomAnnotation
-- post(pipe_ctx, findings) runs once after all per_annot calls complete (full-corpus aggregation)
-- per_macro(macro, wc, findings) runs once per TAPE_WORDS / _Pragma macro declaration
-- Each rule entry picks one of four "shapes" of dispatch:
-- per_annot(annot, pipe_ctx, findings) -- runs once per AtomAnnotation
-- post(pipe_ctx, findings) -- runs once after all per_annot calls complete (full-corpus aggregation)
-- per_macro(macro, wc, findings) -- runs once per TAPE_WORDS / _Pragma macro declaration
-- per_skip_marker(marker, pipe_ctx, findings) -- runs once per src.scan.debug_skip_markers entry
--
-- Adding a new check = 1 row here + 1 function above. The `validate()` dispatch loop never needs editing.
local CHECK_RULES = {
{ name = "atom_decl_exists", per_annot = check_atom_decl_exists },
{ name = "binds_struct_exists", per_annot = check_binds_struct_exists },
{ name = "binds_field_wave_context", per_annot = check_binds_field_wave_context },
{ name = "reads_wave_context", per_annot = check_reads_wave_context },
{ name = "unique_annotation", post = check_unique_annotation },
{ name = "macro_word_drift", per_macro = check_macro_word_drift },
{ name = "atom_decl_exists", per_annot = check_atom_decl_exists },
{ name = "binds_struct_exists", per_annot = check_binds_struct_exists },
{ name = "unique_annotation", post = check_unique_annotation },
{ name = "macro_word_drift", per_macro = check_macro_word_drift },
{ name = "skip_marker_validation", per_skip_marker = check_skip_marker },
{ name = "semantic_reg_defaults", per_source = check_semantic_reg_defaults },
{ name = "atom_reg_types", per_source = check_atom_reg_types },
{ name = "atom_view_layout", per_source = check_atom_view_layout },
{ name = "binds_no_duplicate_fields", per_source = check_binds_no_duplicate_fields },
{ name = "wave_context_migration", per_source = check_wave_context_migration },
}
-- ════════════════════════════════════════════════════════════════════════════
-- Validation
-- ════════════════════════════════════════════════════════════════════════════
--
-- Pure check: read from src.scan, run validations, emit findings.
-- No source walking; no parsing. The scan was done once upstream.
-- Pure check: read from src.scan, run validations, emit findings. The scan was done once upstream.
--- Validate one source against its pre-scanned SourceScan payload.
--- Builds one pass-wide pipe_ctx from the merged `corpus.*` registries and source-ordered `corpus.atom_infos`; per-source declarations and bodies remain in `src.scan`.
--- The module ownership contract above requires callers to construct `ctx.shared.corpus` through `build_ctx`; the error message below enforces that gate.
--- @param ctx PassCtx
--- @param src SourceFile
--- @return PipeCtx
local function build_corpus_pipe_ctx(ctx)
local corpus = ctx.shared and ctx.shared.corpus
if not corpus then
error("annotation requires ctx.shared.corpus "
.. "(the canonical corpus is the source of truth; "
.. "no per-source fallback is supported)", 0)
end
-- `corpus.atom_infos` preserves source order and duplicates; I precompute counts here for `check_unique_annotation` and the per-source checks.
local annot_counts = {}
for _, info in ipairs(corpus.atom_infos or {}) do
if info and info.atom_name then
annot_counts[info.atom_name] = (annot_counts[info.atom_name] or 0) + 1
end
end
-- Every consumer of these fields observes mutations via the canonical corpus without independently mutable registry construction.
return {
-- Cross-source lookup tables from corpus.
register_alias_registry = corpus.register_alias_registry or {},
type_name_registry = corpus.type_name_registry or {},
atom_views = corpus.atom_views or {},
atom_ctxs = corpus.atom_ctxs or {},
atom_phases = corpus.atom_phases or {},
binds_by_name = corpus.binds_by_name or {},
atoms_by_name = corpus.atoms_by_name or {},
-- Corpus-wide ordered list of atom_info records (source-order + duplicates).
atom_infos_list = corpus.atom_infos or {},
-- Corpus-wide annotation count aggregation (post-rule consumes this).
annot_counts = annot_counts,
-- Corpus-wide collisions (recorded by scan_source.merge_corpus_registries).
collisions = corpus.collisions or {},
-- `check_macro_word_drift` reads `corpus.word_counts`, populated by word_count_eval.run.
word_counts = corpus.word_counts or {},
}
end
--- Validate one source against its pre-scanned SourceScan payload + the corpus-wide pipe_ctx.
--- @param ctx PassCtx
--- @param src SourceFile
--- @param corpus_pipe_ctx PipeCtx|nil -- built once per pass from corpus registries; nil builds the same projection here.
--- @return AnnotatedResult
local function validate(ctx, src)
local function validate(ctx, src, corpus_pipe_ctx)
corpus_pipe_ctx = corpus_pipe_ctx or build_corpus_pipe_ctx(ctx)
local scan = src.scan
-- Project the pre-scanned atoms to the AtomEntry shape this pass needs.
@@ -274,28 +509,38 @@ local function validate(ctx, src)
}
end
-- Build pipe_ctx (Fleury: expose structure). Pre-compute everything the per-check functions need.
-- Single source of truth for atom / binds / annotation-count lookups.
-- Build a per-source pipe_ctx: shared lookups come from `corpus_pipe_ctx`, while declarations, bodies, types, views, defaults, and occurrences come from `src.scan`.
local seen_defaults = {}
for reg, _ in pairs(scan.types or {}) do
seen_defaults[reg] = (seen_defaults[reg] or 0) + 1
end
local atom_infos_list = {}
for _, ai in ipairs(scan.atom_infos or {}) do
atom_infos_list[#atom_infos_list + 1] = ai
end
local pipe_ctx = {
atom_index = {},
binds_index = {},
annot_counts = {},
atom_index = {},
binds_index = {},
annot_counts = corpus_pipe_ctx.annot_counts,
types = scan.types or {},
type_occurrences = scan.type_occurrences or {},
atom_views = scan.atom_views or {},
seen_defaults = seen_defaults,
atom_infos_list = atom_infos_list,
binds_list = scan.binds or {},
-- See the module ownership contract; these shared lookup tables come from corpus_pipe_ctx.
register_alias_registry = corpus_pipe_ctx.register_alias_registry,
type_name_registry = corpus_pipe_ctx.type_name_registry,
}
for _, a in ipairs(atoms) do pipe_ctx.atom_index [a.name] = a end
for _, b in ipairs(scan.binds) do pipe_ctx.binds_index[b.name] = b end
for _, a in ipairs(annots) do
if a.name then
pipe_ctx.annot_counts[a.name] = (pipe_ctx.annot_counts[a.name] or 0) + 1
end
end
-- Findings live in a single struct with three lists (errors / warnings / info).
-- Each check writes to the list appropriate for its severity.
local findings = { errors = {}, warnings = {}, info = {} }
-- Propagate parse-time errors from scan_source's atom_info parsing.
-- These are errors found in the atom_info(...) body itself (e.g., malformed args).
-- They are pre-existing in the scan payload — we just lift them into our findings list.
-- Lift parse-time errors already recorded in scan_source's atom_info payload into this pass's findings list.
for _, a in ipairs(annots) do
if a.errors then
for _, msg in ipairs(a.errors) do
@@ -319,19 +564,34 @@ local function validate(ctx, src)
if rule.post then rule.post(pipe_ctx, findings) end
end
-- scan_source records each marker in scan.debug_skip_markers; this loop validates each record independently and emits at most one error per marker.
-- Valid markers stamp `debug_skip = true` on the following atom or component declaration, which downstream consumers read directly.
local skip_markers = scan.debug_skip_markers or {}
for _, marker in ipairs(skip_markers) do
for _, rule in ipairs(CHECK_RULES) do
if rule.per_skip_marker then rule.per_skip_marker(marker, pipe_ctx, findings) end
end
end
-- Per-macro rules (TAPE_WORDS vs WORD_COUNT drift).
local wc = ctx.shared.word_counts
local wc = corpus_pipe_ctx.word_counts
for _, m in ipairs(scan.macros) do
for _, rule in ipairs(CHECK_RULES) do
if rule.per_macro then rule.per_macro(m, wc, findings) end
end
end
-- Per-source rules (reg defaults, atom_view layout, compute-register type overrides, Binds_* field uniqueness).
-- Each per_source rule sees the full scan payload via pipe_ctx.
for _, rule in ipairs(CHECK_RULES) do
if rule.per_source then rule.per_source(src, pipe_ctx, findings) end
end
-- Information summary (always emitted).
findings.info[#findings.info + 1] = {
line = 0,
msg = string.format("scanned: %d atom(s), %d annotation(s), %d macro-word-decl(s), %d binds struct(s)",
#atoms, #annots, #scan.macros, #scan.binds),
msg = string.format("scanned: %d atom(s), %d annotation(s), %d macro-word-decl(s), %d binds struct(s)"
, #atoms, #annots, #scan.macros, #scan.binds),
}
return {
@@ -352,7 +612,6 @@ end
--- Render `<dir_basename>.errors.h` with `#error` directives for every error found across all sources in the directory.
--- Empty directories (no errors, no atoms) produce no file.
local function emit_module_errors_h(ctx, dir_basename, atoms_count, errors, sources)
if ctx.dry_run then return nil end
if atoms_count == 0 and #errors == 0 then
return nil
end
@@ -380,17 +639,6 @@ local function emit_module_errors_h(ctx, dir_basename, atoms_count, errors, sour
return out_path
end
--- Stash aggregated per-module results for the report pass to consume.
local function emit_module_annotations_stub(ctx, dir, dir_basename, atoms_count)
ctx.flags = ctx.flags or {}
ctx.flags._annot_results = ctx.flags._annot_results or {}
ctx.flags._annot_results[#ctx.flags._annot_results + 1] = {
dir = dir,
dir_basename = dir_basename,
atoms_count = atoms_count,
}
end
-- ════════════════════════════════════════════════════════════════════════════
-- M.run — orchestrator entry
-- ════════════════════════════════════════════════════════════════════════════
@@ -409,24 +657,22 @@ function M.run(ctx)
local errors = {}
local warnings = {}
-- Per-DIRECTORY (per-module) aggregation. Group sources by `src.dir`,
-- validate every source in the dir, then emit ONE errors.h per dir.
-- `ctx.by_dir` is pre-computed in build_ctx (shared across all passes).
local by_dir = ctx.by_dir or duffle.group_sources_by_dir(ctx.sources)
-- Build the shared pipe_ctx once for this run; every validate() call sees the same cross-source registries.
-- The corpus owns the canonical cross-source registries; per-source scans retain body / declaration ownership.
local corpus_pipe_ctx = build_corpus_pipe_ctx(ctx)
local corpus = ctx.shared.corpus
-- Group `corpus.sources_by_dir` by module, validate every source in each bucket, and emit one errors.h per directory.
local by_dir = (corpus and corpus.sources_by_dir) or {}
for dir, dir_sources in pairs(by_dir) do
local dir_basename = dir:match("([^/\\]+)$") or dir
local dir_atoms = 0
local dir_errors = {}
local dir_warnings = {}
-- Per-source validate() results, cached for the report pass (it reads from this instead of re-validating each source).
ctx.flags = ctx.flags or {}
ctx.flags._annot_source_results = ctx.flags._annot_source_results or {}
for _, src in ipairs(dir_sources) do
local result = validate(ctx, src)
result.source = src.path -- tag for downstream rendering
ctx.flags._annot_source_results[src.path] = result -- stash so report.lua reads from cache instead of re-running validate()
local result = validate(ctx, src, corpus_pipe_ctx)
result.source = src.path -- tag for downstream rendering
dir_atoms = dir_atoms + #result.atoms
for _, e in ipairs(result.errors) do
dir_errors[#dir_errors + 1] = { line = e.line, msg = e.msg, source = src.path }
@@ -442,8 +688,6 @@ function M.run(ctx)
if err_path then
table.insert(outputs, { errors_h = err_path })
end
emit_module_annotations_stub(ctx, dir, dir_basename, dir_atoms)
end
return { outputs = outputs, errors = errors, warnings = warnings }
+223 -274
View File
@@ -1,26 +1,23 @@
--- passes/atoms_source_map.lua — Per-.word source-line map emitter for tape atoms.
---
--- Reads the pre-scanned SourceScan payload (produced once upstream by `duffle.scan_source`)
--- for `MipsAtom_(name)` (kind="atom"), `MipsAtomComp_` / `MipsAtomComp_Proc_` (kind="comp_*"),
--- and `MipsCode code_<name>` (kind="raw_atom") declarations.
--- Walks each atom's pre-tokenized body (`{{tok=string, rel=integer}, ...}` from `duffle.tokenize_body`),
--- counts per-token word contributions via `ctx.shared.word_counts`, and emits one
--- `WORD N LINE L TEXT T` line per `.word` to `<out_root>/<basename>.atoms.sourcemap.txt`.
--- Writer: this pass, given `atom.paths` (the per-atom mutable surface owned by `emission_model`). Readers:
--- `passes/dwarf_injection.lua` (synthesizes DW_TAG_inlined_subroutine + per-word line program rows) and the gdb-runtime
--- wrapper at `scripts/gdb/gdb_tape_atoms.gdb` (loads the source map via `source <path>`).
---
--- **Two output forms** (per the workspace's per-emission-form pattern from
--- `guide_metaprogram_ssdl.md`):
--- 1. **Canonical text form** — `<out_root>/<basename>.atoms.sourcemap.txt`.
--- Always emitted. Format-version-tagged for forward-compat.
--- Lives in `<out_root>/` (build/gen) NOT `<source_dir>/gen/`. This file is a **build report**, not a compile artifact.
--- Matches the convention used by `annotation.lua` (`<out_root>/<basename>.errors.h`) + `static_analysis.lua` (`<out_root>/<basename>.static_analysis.txt`).
--- Inputs from `atom.paths`: the ordered `items` stream, dense `word_events`, `invocations` views. Outputs: one
--- `WORD N LINE L TEXT T` line per emitted `.word`, plus the per-word provenance form that DWARF synthesis consumes.
---
--- **Two output forms** (per the workspace's per-emission-form pattern from `guide_metaprogram_ssdl.md`):
--- 1. **Sourcemap.txt form** — `<out_root>/<basename>.atoms.sourcemap.txt`. Format-version-tagged for forward-compat.
--- Lives in `<out_root>/` (build/gen). Mirrors the convention used by `annotation.lua`
--- (`<out_root>/<basename>.errors.h`) and `static_analysis.lua` (`<out_root>/<basename>.static_analysis.txt`).
--- Compile artifacts (`*.macs.h`, `*.offsets.h`) stay in `<source_dir>/gen/`.
--- 2. **gdb-runtime form** — `<ctx.out_root>/gdb_tape_atoms_runtime.gdb`
--- (pure gdb command script; addresses pre-computed via `nm`; the 9 user commands defined as `define ... end` blocks).
--- Emitted ONLY when `ctx.flags.gdb_runtime` is true AND `ctx.flags.elf_path` points to an existing ELF.
--- The gdb runtime form lets `gdb-multiarch --without-python` users (the common case on Windows MinGW builds)
--- load the source-map data via `source <path>` — no Python/Tcl/Guile required.
--- 2. **gdb-runtime form** — `<ctx.out_root>/gdb_tape_atoms_runtime.gdb`. A pure gdb command script — addresses come
--- from `nm`, the 9 user commands are static `define ... end` blocks. Emitted when `ctx.flags.gdb_runtime` is true
--- AND `ctx.flags.elf_path` points to an existing ELF. Useful for `gdb-multiarch --without-python` users
--- (the common case on Windows MinGW builds) — `source <path>` loads it with no Python / Tcl / Guile required.
---
--- **Output format** (canonical text form):
--- **Output format** (sourcemap.txt form):
--- ```
--- # FORMAT_VERSION 1
--- # auto-generated by ps1_meta.lua (passes/atoms_source_map.lua) — DO NOT EDIT
@@ -34,166 +31,174 @@
--- ENDATOM
--- ```
---
--- Marker calls (`atom_label(...)`, `atom_offset(...)`) emit 0 `.word`s.
--- They share the same walking convention as `passes/offsets.lua :: scan_atom_body`:
--- markers do NOT advance the word-offset counter, but if a marker is bundled on the same token with a
--- trailing instruction (e.g. `atom_label(foo) load_half_u(...)`),
--- the trailing instruction's word count is added. This matches `offsets.lua :: count_marker_rest`.
--- Marker records are zero-width in `atom.paths.items`, so they emit no WORD rows in the dense word view.
---
--- **Conventions:** tabs (1/level), EmmyLua annotations, no regex,
--- Lua 5.3 compatible.
--- **Conventions:** tabs (1/level), EmmyLua annotations, no regex, Lua 5.3 compatible.
-- ════════════════════════════════════════════════════════════════════════════
-- Module-scope requires + package.path setup
-- ════════════════════════════════════════════════════════════════════════════
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source` (works
-- both standalone + when require'd). `duffle_paths.lua` sets package.path then
-- returns `require("duffle")` at the bottom, so the dofile value IS the duffle module.
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source`
-- (works both standalone + when require'd). `duffle_paths.lua` sets package.path then returns `require("duffle")`
-- at the bottom, so the dofile value IS the duffle module.
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
local elf_dwarf = require("elf_dwarf")
local word_count_eval = require("word_count_eval")
local count_token_words = word_count_eval.count_token_words
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
local elf_dwarf = require("elf_dwarf")
-- ════════════════════════════════════════════════════════════════════════════
-- Constants
-- ════════════════════════════════════════════════════════════════════════════
-- Format version emitted as the first line. Bump + add a migration test if the
-- format changes; the gdb runtime loader rejects mismatches (E2).
-- Format version emitted as the first line. Bump + add a migration test if the format changes;
-- the gdb runtime loader rejects mismatches (E2).
local FORMAT_VERSION = 1
-- Marker-call identifiers (mirrors offsets.lua:33-34).
local LABEL_MARKER = "atom_label"
local OFFSET_MARKER = "atom_offset"
-- ════════════════════════════════════════════════════════════════════════════
-- Type declarations
-- ════════════════════════════════════════════════════════════════════════════
--- @class AtomSourceMapCtx
--- @field sources table[] -- SourceScan payload per source (from `ctx.sources`)
--- @field shared table -- `ctx.shared`
--- @field shared.word_counts table -- macro name -> word count (populated by word-counts + components passes)
--- @field out_root string -- output root (e.g. "build/gen")
--- @field dry_run boolean -- if true, compute but don't write
--- @field flags table -- `ctx.flags`; reads `flags.gdb_runtime` + `flags.elf_path`
--- @field shared table -- `ctx.shared`
--- @field shared.corpus table -- source-order registry; single writer is build_ctx
--- @field shared.word_counts table
--- @field out_root string -- output root (e.g. "build/gen")
--- @field flags table -- `ctx.flags`; reads `flags.gdb_runtime` + `flags.elf_path`
-- ════════════════════════════════════════════════════════════════════════════
-- Helpers
-- Atom-path renderers
-- ════════════════════════════════════════════════════════════════════════════
--- True iff the leading identifier of `tok` is a marker call (`atom_label` / `atom_offset`).
--- Mirrors `passes/offsets.lua :: is_marker_token` (which is file-local there).
--- @param tok string
--- @return boolean
local function is_marker_token(tok)
local leading = duffle.read_ident(tok, 1)
return leading == LABEL_MARKER or leading == OFFSET_MARKER
end
--- Count words contributed by the non-marker portion of `tok` (after the marker's closing `)`).
--- Mirrors offsets.lua:182 `count_marker_rest`.
--- Returns 0 if there's no trailing content after the marker call.
--- @param tok string
--- @param wc table
--- @return integer
local function count_marker_rest(tok, wc)
local marker_end = duffle.find_marker_call_end(tok)
if not marker_end or marker_end >= #tok then return 0 end
local rest = duffle.trim(tok:sub(marker_end))
if rest == "" then return 0 end
return count_token_words(rest, wc)
end
--- Compute per-word entries for an atom.
--- Shared between the canonical text form (per-source `.atoms.sourcemap.txt`)
--- and the gdb-runtime form (`gdb_tape_atoms_runtime.gdb`).
---
--- Returns a list of `{pos, line, text}` entries + the total word count.
--- Markers contribute 0 entries (the marker call emits 0 `.word`s).
--- @param atom table -- one entry of scan.atoms / scan.raw_atoms
--- @param src table -- SourceFile (has .scan with .line_of(), .path)
--- @param wc table -- shared.word_counts
--- Join word boundaries (from `items`) to per-word call text + source lines (from `word_events`).
--- @param atom table
--- @return table[], integer
local function compute_word_entries(atom, src, wc)
local entries = {}
local pos = 0
for _, t in ipairs(atom.body_tokens) do
local tok = t.tok
local rel = t.rel
local words
if is_marker_token(tok) then
words = count_marker_rest(tok, wc)
else
words = count_token_words(tok, wc)
end
if words > 0 then
-- Source line for THIS token = line containing byte offset `atom.body_off + rel`.
-- `src.scan.line_of(...)` is O(log N) via LineIndex.
local line = src.scan.line_of(atom.body_off + rel)
-- Flatten newlines + tabs in TEXT to spaces so each WORD entry fits on
-- one physical line. The gdb Python parser (or our pure-gdb parser)
-- does line-based splits; multi-line TEXT would break it.
local text = duffle.trim(tok):gsub("[\t\r\n]+", " ")
for _ = 1, words do
entries[#entries + 1] = { pos = pos, line = line, text = text }
pos = pos + 1
end
end
local function canonical_word_entries(atom)
local paths = atom.paths or {}
local events = paths.word_events or {}
local word_items = {}
for _, item in ipairs(paths.items or {}) do
if item.kind == "word" then word_items[#word_items + 1] = item end
end
return entries, pos
local entries = {}
for index, event in ipairs(events) do
local item = word_items[index] or {}
entries[#entries + 1] = {
pos = event.i or (index - 1),
line = event.call_line or item.line or 0,
text = event.call_text or item.call_text or "",
body_line = event.body_line or item.body_line or item.line or 0,
invocation = (event.outermost_invocation_id
and paths.invocations
and paths.invocations[event.outermost_invocation_id]) or nil,
}
end
return entries, #events
end
--- Render one atom's stanza for the canonical text form
--- (ATOM header line, N WORD lines, ENDATOM marker). Returns (lines, total_words).
--- @param src table
--- @param atom table
--- @param wc table
--- Render one atom's provenance stanza. Format 1 line shapes:
--- `WORD N CALL <src-path>:<src-line> MACRO <name> "<def-path>:<def-line>" BODY <line>` (component invocation)
--- `WORD N CALL <src-path>:<src-line> RAW` (raw `.word` outside any mac_* component)
--- Component identity comes from the outermost invocation record; the count-table lookup confirms the component was
--- declared in `corpus.word_counts` (populated by word_count_eval + components passes).
--- @param src table
--- @param atom table
--- @param wc table -- identity alias of corpus.word_counts
--- @return string[], integer
local function emit_atom_stanza(src, atom, wc)
local lines = {}
local rel_path = src.path:gsub("\\", "/")
local entries, total = compute_word_entries(atom, src, wc)
local function emit_provenance_stanza(src, atom, wc)
local lines = {}
local rel_path = src.path:gsub("\\\\", "/")
local entries, total = canonical_word_entries(atom)
-- ATOM header line with placeholder total (patched after we know it).
lines[#lines + 1] = string.format('ATOM %s "%s" 0', atom.raw_name or atom.name, rel_path)
for _, we in ipairs(entries) do
lines[#lines + 1] = string.format("WORD %d LINE %d TEXT %s",
we.pos, we.line, we.text)
for _, entry in ipairs(entries) do
local inv = entry.invocation
local macro_count = inv and wc["mac_" .. inv.component_name]
if inv and macro_count ~= nil then
lines[#lines + 1] = string.format(
'WORD %d CALL %s:%d MACRO %s "%s:%d" BODY %d',
entry.pos, rel_path, entry.line, inv.component_name,
inv.def_path or "", inv.def_line or 0, entry.body_line)
else
lines[#lines + 1] = string.format(
"WORD %d CALL %s:%d RAW", entry.pos, rel_path, entry.line)
end
end
-- Patch the placeholder total in the ATOM header line.
lines[1] = lines[1]:gsub(" 0$", " " .. tostring(total))
lines[#lines + 1] = "ENDATOM"
return lines, total
end
--- Render the full source map file content for one source (one .atoms.sourcemap.txt per source).
--- Mirrors offsets.lua's `project_atoms` shape: scan.atoms + scan.raw_atoms, no kind filter.
--- Render the full provenance file content for one source.
--- @param src table
--- @param wc table
--- @return string
local function render_source_map(src, wc)
local function render_provenance(src, wc)
local lines = {}
lines[#lines + 1] = "# FORMAT_VERSION 1"
lines[#lines + 1] = "# auto-generated by ps1_meta.lua (passes/atoms_source_map.lua) — DO NOT EDIT"
lines[#lines + 1] = "# Per-.word provenance: maps each emitted .word to its call site (atom body"
lines[#lines + 1] = "# file:line) and, when the word was emitted by a `mac_X(...)` component invocation,"
lines[#lines + 1] = "# the component's definition file:line + the per-word BODY line. Used by"
lines[#lines + 1] = "# dwarf_injection to synthesize DW_TAG_inlined_subroutine instances + per-word"
lines[#lines + 1] = "# line program rows for native source-level step into component bodies."
local function append(atom)
local stanza = emit_provenance_stanza(src, atom, wc)
for _, line in ipairs(stanza) do lines[#lines + 1] = line end
end
for _, atom in ipairs(src.scan.atoms or {}) do
if atom.paths then append(atom) end
end
for _, atom in ipairs(src.scan.raw_atoms or {}) do
if atom.paths then append(atom) end
end
return table.concat(lines, "\n") .. "\n"
end
--- Render one atom's stanza for the sourcemap.txt form (ATOM header line, N WORD lines, ENDATOM marker).
--- Returns (lines, total_words).
--- @param src table
--- @param atom table
--- @param wc table
--- @return string[], integer
local function emit_atom_stanza(src, atom)
local lines = {}
local rel_path = src.path:gsub("\\\\", "/")
local entries, total = canonical_word_entries(atom)
lines[#lines + 1] = string.format('ATOM %s "%s" 0', atom.raw_name or atom.name, rel_path)
for _, entry in ipairs(entries) do
lines[#lines + 1] = string.format("WORD %d LINE %d TEXT %s",
entry.pos, entry.line, entry.text)
end
lines[1] = lines[1]:gsub(" 0$", " " .. tostring(total))
lines[#lines + 1] = "ENDATOM"
return lines, total
end
--- Render the full source map file content for one source (one .atoms.sourcemap.txt per source). Mirrors offsets.lua's
--- `project_atoms` shape: scan.atoms + scan.raw_atoms, no kind filter.
--- @param src table
--- @param wc table
--- @return string
local function render_source_map(src)
local lines = {}
lines[#lines + 1] = "# FORMAT_VERSION " .. FORMAT_VERSION
lines[#lines + 1] = "# auto-generated by ps1_meta.lua (passes/atoms_source_map.lua) — DO NOT EDIT"
local function append(atom)
local stanza = emit_atom_stanza(src, atom)
for _, line in ipairs(stanza) do lines[#lines + 1] = line end
end
for _, atom in ipairs(src.scan.atoms or {}) do
local stanza = emit_atom_stanza(src, atom, wc)
for _, line in ipairs(stanza) do
lines[#lines + 1] = line
end
if atom.paths then append(atom) end
end
for _, atom in ipairs(src.scan.raw_atoms or {}) do
local stanza = emit_atom_stanza(src, atom, wc)
for _, line in ipairs(stanza) do
lines[#lines + 1] = line
end
if atom.paths then append(atom) end
end
return table.concat(lines, "\n") .. "\n"
@@ -211,70 +216,48 @@ local function gdb_escape(s)
return (s:gsub("\\", "\\\\"):gsub('"', '\\"'))
end
--- Build the list of atoms with addresses + word entries.
--- Shared helper for the gdb-runtime file emission.
--- Build the list of atoms with addresses + word entries. Shared helper for the gdb-runtime file emission.
--- @param ctx PassCtx
--- @return table[] -- list of {idx, name, src_path, file_base, addr, size_bytes, words, entries}
local function build_atom_table(ctx)
local wc = (ctx.shared and ctx.shared.word_counts) or {}
local addrs = elf_dwarf.read_nm(ctx.flags.elf_path)
local corpus = ctx.shared and ctx.shared.corpus
local matched = {}
for _, src in ipairs(ctx.sources) do
if src.scan then
local file_base = src.path:match("([^/\\]+)$") or src.path
for _, atom in ipairs(src.scan.atoms or {}) do
if atom.kind == nil or atom.kind == "atom" then
local name = atom.raw_name or atom.name
local info = addrs[name]
if info then
local entries, total = compute_word_entries(atom, src, wc)
matched[#matched + 1] = {
name = name,
src_path = src.path,
file_base = file_base,
addr = info[1],
size_bytes = info[2],
words = total,
entries = entries,
}
end
end
end
for _, atom in ipairs(src.scan.raw_atoms or {}) do
local name = atom.name
local info = addrs[name]
if info then
local entries, total = compute_word_entries(atom, src, wc)
matched[#matched + 1] = {
name = name,
src_path = src.path,
file_base = file_base,
addr = info[1],
size_bytes = info[2],
words = total,
entries = entries,
}
end
end
for _, src in ipairs(corpus.source_order or {}) do
local file_base = src.path:match("([^/\\\\]+)$") or src.path
local function append(atom)
if not atom.paths then return end
local name = atom.raw_name or atom.name
local info = addrs[name]
if not info then return end
local entries, total = canonical_word_entries(atom)
matched[#matched + 1] = {
name = name,
src_path = src.path,
file_base = file_base,
addr = info[1],
size_bytes = info[2],
words = total,
entries = entries,
}
end
for _, atom in ipairs((src.scan or {}).atoms or {}) do append(atom) end
for _, atom in ipairs((src.scan or {}).raw_atoms or {}) do append(atom) end
end
-- Deterministic order: sort by address (matches `nm` output ordering).
table.sort(matched, function(a, b) return a.addr < b.addr end)
for i, a in ipairs(matched) do
a.idx = i - 1
end
for i, a in ipairs(matched) do a.idx = i - 1 end
return matched
end
--- Append the 9 gdb command definitions to `lines`. Pure gdb scripting no Python, no Tcl, no Guile required.
--- **Fully hardcoded per-atom** because gdb doesn't do nested `$` substitution in var names
--- `$__atom_name_$__i` inside a `while` loop is treated as one literal identifier, not a concat.
--- Append the 9 gdb command definitions to `lines`. Pure gdb scripting — addresses come from `nm`, the convenience
--- vars set in `emit_gdb_runtime` provide printf args, and each command is a static sequence of `printf` / `tbreak` /
--- `if ... end` blocks. The Lua pass emits N atoms' worth of lines; runtime iteration is gdb's job.
---
--- Each command is a static sequence of `printf` / `tbreak` / `if ... end` blocks.
--- The Lua pass emits N atoms' worth of lines — no runtime iteration.
--- With 7 atoms + ~200 word entries, the runtime file is ~2000 lines, all auto-generated, no human edit ever.
--- Why hardcoded per-atom: gdb's `$` substitution doesn't concat inside var names — `$__atom_name_$__i` in a `while`
--- loop resolves to one literal identifier, not `name_i`. Compile-time emission is the only path.
--- @param lines table -- output line buffer (mutated in place)
--- @param matched table -- list of atom records from `build_atom_table`
local function append_gdb_commands(lines, matched)
@@ -284,8 +267,7 @@ local function append_gdb_commands(lines, matched)
for _, a in ipairs(matched) do
-- gdb 12.1 quirk: literals in printf args require an attached target.
-- Use the per-atom convenience vars set above as printf args.
lines[#lines + 1] = string.format(
' printf " code_%%-32s @ 0x%%08x %%4d words\\n", $__atom_name_%d, $__atom_addr_%d, $__atom_words_%d',
lines[#lines + 1] = string.format(' printf " code_%%-32s @ 0x%%08x %%4d words\\n", $__atom_name_%d, $__atom_addr_%d, $__atom_words_%d',
a.idx, a.idx, a.idx)
end
lines[#lines + 1] = "end"
@@ -298,8 +280,7 @@ local function append_gdb_commands(lines, matched)
lines[#lines + 1] = "define break_atom"
lines[#lines + 1] = ' echo "Usage: break_atom_<exact_name> (pick from the list below)"'
for _, a in ipairs(matched) do
lines[#lines + 1] = string.format(
' printf " break_atom_%%-32s\\n", $__atom_name_%d', a.idx)
lines[#lines + 1] = string.format(' printf " break_atom_%%-32s\\n", $__atom_name_%d', a.idx)
end
lines[#lines + 1] = "end"
lines[#lines + 1] = "document break_atom"
@@ -310,8 +291,7 @@ local function append_gdb_commands(lines, matched)
for _, a in ipairs(matched) do
lines[#lines + 1] = string.format("define break_atom_%s", a.name)
lines[#lines + 1] = string.format(" break *$__atom_addr_%d", a.idx)
lines[#lines + 1] = string.format(
' printf " Breakpoint set at code_%s (0x%%08x)\\n", $__atom_addr_%d', a.name, a.idx)
lines[#lines + 1] = string.format(' printf " Breakpoint set at code_%s (0x%%08x)\\n", $__atom_addr_%d', a.name, a.idx)
lines[#lines + 1] = "end"
lines[#lines + 1] = string.format("document break_atom_%s", a.name)
lines[#lines + 1] = string.format(" Set a breakpoint at code_%s.", a.name)
@@ -347,33 +327,24 @@ local function append_gdb_commands(lines, matched)
lines[#lines + 1] = " set $__matched = 0"
for _, a in ipairs(matched) do
-- Precompute end_addr (gdb 12.1's expression evaluator chokes on `addr + words*4`).
lines[#lines + 1] = string.format(
" set $__end_%d = $__atom_addr_%d + $__atom_words_%d * 4", a.idx, a.idx, a.idx)
lines[#lines + 1] = string.format(
" if $__pc >= $__atom_addr_%d && $__pc < $__end_%d", a.idx, a.idx)
lines[#lines + 1] = string.format(
' printf "atom: code_%%s\\n", $__atom_name_%d', a.idx)
lines[#lines + 1] = string.format(" set $__end_%d = $__atom_addr_%d + $__atom_words_%d * 4", a.idx, a.idx, a.idx)
lines[#lines + 1] = string.format(" if $__pc >= $__atom_addr_%d && $__pc < $__end_%d", a.idx, a.idx)
lines[#lines + 1] = string.format(' printf "atom: code_%%s\\n", $__atom_name_%d', a.idx)
lines[#lines + 1] = ' printf "addr: 0x%08x\\n", $__pc'
lines[#lines + 1] = string.format(
" set $__word = ($__pc - $__atom_addr_%d) / 4", a.idx)
lines[#lines + 1] = string.format(
' printf "word: %%d/%%d\\n", $__word, $__atom_words_%d', a.idx)
lines[#lines + 1] = string.format(" set $__word = ($__pc - $__atom_addr_%d) / 4", a.idx)
lines[#lines + 1] = string.format(' printf "word: %%d/%%d\\n", $__word, $__atom_words_%d', a.idx)
-- One inner-if per WORD entry. Each word's line + text hardcoded.
for _, we in ipairs(a.entries) do
lines[#lines + 1] = string.format(
" if $__word == %d", we.pos)
lines[#lines + 1] = string.format(" if $__word == %d", we.pos)
-- Escape TEXT for printf format string.
local escaped_text = we.text:gsub("%%", "%%%%"):gsub('"', '\\"')
lines[#lines + 1] = string.format(
' printf "source: %%s:%%d %%s\\n", $__atom_file_%d, %d, "%s"',
a.idx, we.line, escaped_text)
lines[#lines + 1] = string.format(' printf "source: %%s:%%d %%s\\n", $__atom_file_%d, %d, "%s"', a.idx, we.line, escaped_text)
lines[#lines + 1] = " end"
end
-- Fallback for words beyond the source map (shouldn't happen if nm matches).
local max_word = 0
if #a.entries > 0 then max_word = a.entries[#a.entries].pos end
lines[#lines + 1] = string.format(
' if $__word > %d', max_word)
lines[#lines + 1] = string.format(' if $__word > %d', max_word)
lines[#lines + 1] = ' printf "source: (no source-map entry for word %%d; map may be stale)\\n", $__word'
lines[#lines + 1] = " end"
lines[#lines + 1] = " set $__matched = 1"
@@ -390,19 +361,16 @@ local function append_gdb_commands(lines, matched)
-- ── stepi_inside_atom ──
-- Hardcoded one if-containment-check per atom (no loop).
-- Precompute end_addr in Lua so we don't ask gdb to evaluate `addr + words*4`
-- inside the if condition (gdb 12.1's expression evaluator chokes on the
-- `*` and emits a misleading 'function malloc' error in some gdb builds).
-- Precompute end_addr in Lua so we don't ask gdb to evaluate `addr + words*4` inside the if condition
-- (gdb 12.1's expression evaluator chokes on the `*` and emits a misleading 'function malloc' error in some gdb builds).
lines[#lines + 1] = "define stepi_inside_atom"
lines[#lines + 1] = " set $__in_atom = 0"
lines[#lines + 1] = " set $__did_step = 0"
lines[#lines + 1] = " set $__pc = (unsigned int)$pc"
for _, a in ipairs(matched) do
-- Precompute end_addr in the convenience var (single expression gdb handles).
lines[#lines + 1] = string.format(
" set $__end_%d = $__atom_addr_%d + $__atom_words_%d * 4", a.idx, a.idx, a.idx)
lines[#lines + 1] = string.format(
" if $__pc >= $__atom_addr_%d && $__pc < $__end_%d", a.idx, a.idx)
lines[#lines + 1] = string.format(" set $__end_%d = $__atom_addr_%d + $__atom_words_%d * 4", a.idx, a.idx, a.idx)
lines[#lines + 1] = string.format(" if $__pc >= $__atom_addr_%d && $__pc < $__end_%d", a.idx, a.idx)
lines[#lines + 1] = " set $__in_atom = 1"
lines[#lines + 1] = " stepi"
lines[#lines + 1] = " set $__did_step = 1"
@@ -418,32 +386,6 @@ local function append_gdb_commands(lines, matched)
lines[#lines + 1] = "end"
lines[#lines + 1] = ""
-- ── show_c2 ──
-- GTE data regs (COP2). pcsx-redux's gdb stub doesn't expose COP2 (only
-- 72 regs: 32 GPR + COP0 + FPR).
-- curl http://localhost:8080/api/v1/lua/gte
-- We keep the command definition as a stub that points the user at the plugin.
lines[#lines + 1] = "define show_c2"
lines[#lines + 1] = ' echo "[gdb_tape_atoms] show_c2: gdb stub does not expose COP2 in this build."'
lines[#lines + 1] = ' echo "[gdb_tape_atoms] Use scripts/pcsx_debug_helper.zip + curl http://localhost:8080/api/v1/lua/gte"'
lines[#lines + 1] = ' echo "[gdb_tape_atoms] (or pcsx-redux Debug > Registers window for a native view)"'
lines[#lines + 1] = "end"
lines[#lines + 1] = "document show_c2"
lines[#lines + 1] = " Stub. The gdb stub in this pcsx-redux build does not expose COP2 regs."
lines[#lines + 1] = " For GTE data + control state, use the pcsx_debug_helper Lua plugin or the"
lines[#lines + 1] = " pcsx-redux Debug > Registers window."
lines[#lines + 1] = "end"
lines[#lines + 1] = ""
-- ── show_c2ctl ──
lines[#lines + 1] = "define show_c2ctl"
lines[#lines + 1] = ' echo "[gdb_tape_atoms] show_c2ctl: see show_c2 for the same workaround."'
lines[#lines + 1] = "end"
lines[#lines + 1] = "document show_c2ctl"
lines[#lines + 1] = " Stub. Same workaround as show_c2."
lines[#lines + 1] = "end"
lines[#lines + 1] = ""
-- ── wave_ctx ──
lines[#lines + 1] = "define wave_ctx"
lines[#lines + 1] = ' printf "$t4 = R_FaceCursor 0x%08x\\n", $t4'
@@ -456,13 +398,13 @@ local function append_gdb_commands(lines, matched)
lines[#lines + 1] = "end"
end
--- Emit the gdb-runtime file (post-link). Pure gdb scripting — no Python.
--- Reads ELF addresses via `mipsel-none-elf-nm -S`, embeds them in `<ctx.out_root>/gdb_tape_atoms_runtime.gdb`
--- so gdb loads the data via `set $var = ...` + `define ... end` blocks at source-time.
--- Emit the gdb-runtime file (post-link). Pure gdb scripting — addresses come from `mipsel-none-elf-nm -S`, get embedded
--- in `<ctx.out_root>/gdb_tape_atoms_runtime.gdb`, and load via `set $var = ...` + `define ... end` blocks at gdb
--- source-time.
--- @param ctx PassCtx
local function emit_gdb_runtime(ctx)
if not (ctx.flags and ctx.flags.gdb_runtime) then return end
local elf_path = ctx.flags.elf_path
local elf_path = ctx.flags.elf_path
if not elf_path or elf_path == "" then
io.stderr:write("[atoms_source_map] --gdb-runtime requires --elf <elf>\n")
return
@@ -517,12 +459,9 @@ local function emit_gdb_runtime(ctx)
lines[#lines + 1] = 'printf "[gdb_tape_atoms] runtime loaded %d atoms from %s\\n", $__atom_count, $__elf_path'
local out_path = ctx.out_root .. "/gdb_tape_atoms_runtime.gdb"
if not ctx.dry_run then
duffle.ensure_dir(duffle.dirname(out_path))
duffle.write_file_lf(out_path, table.concat(lines, "\n") .. "\n")
end
io.stderr:write(string.format(
"[atoms_source_map] wrote %s (%d atoms)\n", out_path, #matched))
duffle.ensure_dir(duffle.dirname(out_path))
duffle.write_file_lf(out_path, table.concat(lines, "\n") .. "\n")
-- io.stderr:write(string.format("[atoms_source_map] wrote %s (%d atoms)\n", out_path, #matched))
end
-- ════════════════════════════════════════════════════════════════════════════
@@ -531,10 +470,10 @@ end
local M = {}
--- Pass entry: emit one `<out_root>/<basename>.atoms.sourcemap.txt` per source file
--- that contains at least one `MipsAtom_(name)` / `MipsCode code_<name>` declaration.
--- Optionally also emit `<ctx.out_root>/gdb_tape_atoms_runtime.gdb` when
--- `ctx.flags.gdb_runtime` is true.
--- Pass entry. For each source that declares at least one `MipsAtom_(name)` / `MipsCode code_<name>`, emit two files
--- in `<out_root>/`: `<basename>.atoms.sourcemap.txt` (per-word call-site map) and `<basename>.atoms.provenance.txt`
--- (per-word definition + body line, resolved via the outermost `mac_X(...)` invocation). When `ctx.flags.gdb_runtime`
--- is true and `ctx.flags.elf_path` exists, also emit the post-link gdb script `<ctx.out_root>/gdb_tape_atoms_runtime.gdb`.
--- @param ctx PassCtx
--- @return PassResult
function M.run(ctx)
@@ -542,37 +481,47 @@ function M.run(ctx)
local errors = {}
local warnings = {}
-- word-counts + components passes must have populated shared.word_counts.
-- If absent, the orchestrator wired the deps wrong — fail loud.
local wc = (ctx.shared and ctx.shared.word_counts) or {}
if not wc or not next(wc) then
local corpus = ctx.shared and ctx.shared.corpus
if type(corpus) ~= "table" or type(corpus.source_order) ~= "table" then
error("atoms_source_map.run requires ctx.shared.corpus.source_order (canonical corpus).", 0)
end
-- Word counts come from `corpus.word_counts` (populated by word_count_eval + components passes).
local wc = corpus.word_counts or {}
if not next(wc) then
warnings[#warnings + 1] = {
line = 0,
msg = "atoms_source_map: ctx.shared.word_counts is empty; the word-counts + components passes may not have populated it. Check the PASSES dep edges.",
msg = "atoms_source_map: corpus.word_counts is empty; the word-counts + components passes may not have populated it. Check the PASSES dep edges.",
}
end
-- Always emit the canonical text form (per-source).
for _, src in ipairs(ctx.sources) do
if src.scan then
local n_atoms = src.scan.atoms and #src.scan.atoms or 0
local n_raw_atoms = src.scan.raw_atoms and #src.scan.raw_atoms or 0
if n_atoms + n_raw_atoms > 0 then
local basename = duffle.basename_no_ext(src.path)
-- Build report, NOT compile artifact: live in <out_root> alongside the other reports
-- (annotation.lua's *.errors.h, static_analysis.lua's *.static_analysis.txt, gdb_tape_atoms_runtime.gdb).
-- The per-source <source_dir>/gen/ is reserved for headers actually #included by C.
local out_path = ctx.out_root .. "/" .. basename .. ".atoms.sourcemap.txt"
local content = render_source_map(src, wc)
if not ctx.dry_run then
duffle.ensure_dir(duffle.dirname(out_path))
duffle.write_file_lf(out_path, content)
end
outputs[#outputs + 1] = { kind = "report", path = out_path }
-- Always emit the text form (per-source).
for _, src in ipairs(corpus.source_order) do
local has_projection = false
for _, atom in ipairs((src.scan or {}).atoms or {}) do
if (atom.kind == "atom" or atom.kind == "raw_atom") and atom.paths then
has_projection = true; break
end
end
if not has_projection then
for _, atom in ipairs((src.scan or {}).raw_atoms or {}) do
if atom.paths then has_projection = true; break end
end
end
if has_projection then
local basename = duffle.basename_no_ext(src.path)
-- (1) atoms.sourcemap.txt — format-1 per-word call-site map.
local sourcemap_path = ctx.out_root .. "/" .. basename .. ".atoms.sourcemap.txt"
local sourcemap_body = render_source_map(src)
-- (2) atoms.provenance.txt — format-1 per-word definition/body map.
local prov_path = ctx.out_root .. "/" .. basename .. ".atoms.provenance.txt"
local prov_body = render_provenance(src, wc)
duffle.ensure_dir(duffle.dirname(sourcemap_path))
duffle.write_file_lf(sourcemap_path, sourcemap_body)
duffle.write_file_lf(prov_path, prov_body)
outputs[#outputs + 1] = { kind = "report", path = sourcemap_path }
outputs[#outputs + 1] = { kind = "report", path = prov_path }
end
end
-- Optionally emit the gdb-runtime form (post-link, one file per build).
+232 -287
View File
@@ -1,38 +1,27 @@
--- passes/components.lua — Component-macro header generator.
---
--- Reads the pre-scanned SourceScan payload (produced once upstream by `duffle.scan_source`)
--- for `MipsAtomComp_(ac_X)` and `MipsAtomComp_Proc_(ac_X, { body })` declarations, then does
--- per-source backward lookups for the function-args string (from the preceding `FI_ MipsAtom ac_X(...)`
--- function declaration) and the preceding comment block (for LSP/IntelliSense signature docs).
--- Ownership: `corpus.word_counts`, `corpus.components`, and `corpus.component_body_index`.
--- Scanner owns `declaration_comment` and `debug_skip` on each declaration record; this pass projects both forward.
---
--- Emits a per-directory `<dir_basename>.macs.h` containing one `#define mac_X(sig) \` macro per component
--- + `WORD_COUNT(mac_X, N)` entries for downstream offset computation.
--- Reads the pre-scanned SourceScan payload from `duffle.scan_source` for `MipsAtomComp_(ac_X)` and `MipsAtomComp_Proc_(ac_X, { body })` declarations,
--- then resolves the function-args string from the preceding `FI_ MipsAtom ac_X(...)` declaration via a backward walk.
---
--- Emits one `<dir_basename>.macs.h` per source with `#define mac_X(sig) \` macros plus `WORD_COUNT(mac_X, N)` entries for downstream offset computation.
---
--- **Conventions**: tabs (1/level), EmmyLua annotations, no regex,
--- Lua 5.3 compatible.
--- @class Component
--- @field name string
--- @field body string
--- @field args string|nil
--- @field line integer
--- @field comment string|nil
--- @class M
-- ════════════════════════════════════════════════════════════════════════════
-- Module-scope requires + package.path setup
-- ════════════════════════════════════════════════════════════════════════════
-- Bootstrap: same as entry scripts. See `ps1_meta.lua` for the rationale.
-- Bootstrap: load `scripts/duffle_paths.lua` (sets package.path + package.cpath).
-- Uses `debug.getinfo` to find this file's own directory, so it works
-- both standalone and when require'd from the orchestrator.
-- Uses `debug.getinfo` to find this file's own directory, so it works both standalone and when require'd from the orchestrator.
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source` (works both standalone + when require'd).
-- duffle_paths.lua sets package.path then returns `require("duffle")` at the bottom, so the dofile value IS the duffle module.
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
local word_count_eval = require("word_count_eval")
-- ════════════════════════════════════════════════════════════════════════════
-- Constants
@@ -43,9 +32,9 @@ local ATOM_COMP_PROC = "MipsAtomComp_Proc_"
local MIPS_ATOM = "MipsAtom" -- prefix on the function declaration that wraps an AtomComp_Proc_
-- Component-name prefixes.
local AC_PREFIX = "ac_" -- arg to MipsAtomComp_(ac_X); the X is the atom name
local AC_PREFIX = "ac_" -- arg to MipsAtomComp_(ac_X); the X is the atom name
local AC_PREFIX_LEN = 3
local MAC_PREFIX = "mac_" -- prefix on generated macros; the rest is the atom name
local MAC_PREFIX = "mac_" -- prefix on generated macros; the rest is the atom name
local MAC_PREFIX_LEN = 4
-- ASCII byte values used in tokenization.
@@ -70,12 +59,10 @@ local GEN_SUBDIR = "gen"
--- @field sources SourceFile[] -- all source files in the build
--- @field metadata_path string -- path to word_count.metadata.h
--- @field shared table -- cross-pass shared state
--- @field shared.word_counts table<string, integer> -- populated by word-counts + components
--- @field out_root string -- output root (e.g. "build/gen")
--- @field project_root string -- project root (e.g. "code/")
--- @field upstream table<string, table> -- per-pass upstream outputs
--- @field flags table -- CLI flags
--- @field dry_run boolean -- if true, compute but don't write
--- @field verbose boolean -- log diagnostic info
--- @class PassResult
@@ -84,11 +71,13 @@ local GEN_SUBDIR = "gen"
--- @field warnings table[] -- {line=, msg=} entries; build-succeeds
--- @class Component
--- @field name string -- atom name (without `ac_` prefix)
--- @field body string -- brace-delimited body (without the braces)
--- @field args string|nil -- function-args string (function form only)
--- @field line integer -- source line of the declaration
--- @field comment string|nil -- preceding `/* */` or `//` comment block (signature doc)
--- @field name string -- atom name (without `ac_` prefix)
--- @field body string -- brace-delimited body (without the braces)
--- @field args string|nil -- function-args string (function form only)
--- @field line integer -- source line of the declaration
--- @field comment string|nil -- scanner-owned `declaration_comment`; the components pass reads it from the scanner record
--- @field kind string -- "comp_bare" | "comp_proc"
--- @field debug_skip boolean -- mirror of `a.debug_skip` (scanner-owned); true iff a bare `atom_dbg_skip` marker immediately preceded the declaration
-- ════════════════════════════════════════════════════════════════════════════
-- Local helpers (file I/O + path normalization)
@@ -97,16 +86,28 @@ local GEN_SUBDIR = "gen"
local M = {}
-- ════════════════════════════════════════════════════════════════════════════
-- Function-args extraction (precedes MipsAtomComp_Proc_ invocations)
-- Back-walk helpers (composed into the entry point below: find_function_args_for)
--
-- Only the function-args lookup for proc components occurs here.
-- The preceding-comment walk occur in `scan_source.lua` — `a.declaration_comment` carries the resolved comment,
-- so this file reads it forward rather than re-walking the source.
-- ════════════════════════════════════════════════════════════════════════════
-- Find the LAST occurrence of `name + "("` in `source[1..before_pos]`.
-- Returns the position of the open paren, or nil if not found.
-- @param source string
-- @param name string
-- @param before_pos integer
-- @return integer|nil
local function find_last_name_open_paren(source, name, before_pos)
--- Find the args of the function declaration that immediately precedes a `MipsAtomComp_Proc_` invocation of the given name.
--- Returns the args string (e.g., `"U4 off, U4 code, U1 r, U1 g, U1 b"`) or nil if no function declaration is found.
---
--- Convention: function form is
--- `FI_ MipsAtom ac_X(args) MipsAtomComp_Proc_(ac_X, { body })`
--- We find the LAST occurrence of `"ac_X("` before `before_pos` and extract the args from inside the parens.
--- We then verify the preceding context ends with `MipsAtom`
--- (the function-decl keyword with possible qualifiers between).
---
--- @param source string
--- @param name string
--- @param before_pos integer
--- @return string|nil
local function find_function_args_for(source, name, before_pos)
-- Find the LAST occurrence of `name + "("` in `source[1..before_pos]`.
local name_open = name .. "("
local last_idx = nil
local scan_pos = 1
@@ -118,24 +119,6 @@ local function find_last_name_open_paren(source, name, before_pos)
last_idx = found
scan_pos = found + #name_open
end
return last_idx
end
--- Find the args of the function declaration that immediately precedes a `MipsAtomComp_Proc_` invocation of the given name.
--- Returns the args string (e.g., `"U4 off, U4 code, U1 r, U1 g, U1 b"`) or nil if no function declaration is found.
---
--- Convention: function form is
--- `FI_ MipsAtom ac_X(args) MipsAtomComp_Proc_(ac_X, { body })`
--- We find the LAST occurrence of `"ac_X("` before `before_pos` and extract the args from inside the parens.
--- We then verify the preceding context ends with `MipsAtom`
--- (the function-decl keyword with possible qualifiers between).
---
--- @param source string
--- @param name string
--- @param before_pos integer
--- @return string|nil
local function find_function_args_for(source, name, before_pos)
local last_idx = find_last_name_open_paren(source, name, before_pos)
if not last_idx then return nil end
-- Verify the preceding context ends with "MipsAtom" (with possible qualifiers between).
@@ -154,171 +137,10 @@ local function find_function_args_for(source, name, before_pos)
return inner
end
-- ════════════════════════════════════════════════════════════════════════════
-- Preceding-comment-block extraction
-- ════════════════════════════════════════════════════════════════════════════
-- Skip whitespace (space/tab/newline/CR) backward from `pos`, returning the position of the first non-whitespace char.
-- @param source string
-- @param pos integer
-- @return integer
local function skip_ws_backward(source, pos)
local back = pos - 1
while back > 0 do
local ch = source:sub(back, back)
if ch == " " or ch == "\t" or ch == "\n" or ch == "\r" then
back = back - 1
else
break
end
end
return back
end
-- Find the opening `/*` for a block comment whose `*/` ends at `close_pos`.
-- Returns the position of `/`, or nil if not found.
-- @param source string
-- @param close_pos integer -- position of the closing `*` of `*/`
-- @return integer|nil
local function find_block_comment_open(source, close_pos)
local prefix = source:sub(1, close_pos - 1)
local open_at = nil
for scan = #prefix - 1, 1, -1 do
if prefix:sub(scan, scan + 1) == "/*" then
open_at = scan
break
end
end
return open_at
end
-- Walk back from `open_at` over leading spaces + tabs to include the indentation before the `/*` in the captured comment.
-- @param source string
-- @param open_at integer
-- @return integer
local function extend_left_over_indent(source, open_at)
local start = open_at
while start > 1 do
local ch = source:sub(start - 1, start - 1)
if ch == " " or ch == "\t" then
start = start - 1
else
break
end
end
return start
end
-- Walk back from `line_end` to the start of the source line (the most recent `\n` or position 1).
-- @param source string
-- @param line_end integer
-- @return integer
local function find_line_start(source, line_end)
local start = line_end
while start > 1 and source:sub(start - 1, start - 1) ~= "\n" do
start = start - 1
end
return start
end
-- (internal) Capture one `/* ... */` block comment whose closing `*/`
-- ends at `close_end_pos`. Returns (block_text, new_scan_pos) where `new_scan_pos`
-- is where to continue scanning for more comments, or nil if no block comment was found.
local function capture_block_comment(source, close_end_pos)
local open_at = find_block_comment_open(source, close_end_pos)
if not open_at then return nil end
local block_start = extend_left_over_indent(source, open_at)
return source:sub(block_start, close_end_pos), block_start
end
-- (internal) Capture one `// ...` line comment ending at `line_end_pos`.
-- Returns (comment_text, new_scan_pos) or nil if the line is not a `//` comment.
local function capture_line_comment(source, line_end_pos)
local line_start = find_line_start(source, line_end_pos)
local line = source:sub(line_start, line_end_pos)
if line:sub(1, 2) == "//" then
return line, line_start - 1
end
return nil
end
--- Find the contiguous comment block immediately preceding `pos` in `source`.
--- Returns the comment text (with the `/* */` or `//` markers preserved) or an empty string if no comment is adjacent.
---
--- Used to copy signature comments from the source declaration (`MipsAtomComp_` / `MipsAtomComp_Proc_` / function decl)
--- over to the generated `mac_X` macro, so LSP/IntelliSense displays the args doc.
---
--- @param source string
--- @param pos integer
--- @return string
local function preceding_comment_block(source, pos)
local scan_pos = pos
local pieces = {}
while true do
local non_ws = skip_ws_backward(source, scan_pos)
if non_ws == 0 then break end
local is_block_close = non_ws >= 2 and source:sub(non_ws - 1, non_ws) == "*/"
local is_line_end = source:sub(non_ws, non_ws) == "\n" or source:sub(non_ws, non_ws) == "\r"
if is_block_close then
local block_text, new_scan_pos = capture_block_comment(source, non_ws)
if not block_text then break end
table.insert(pieces, 1, block_text)
scan_pos = new_scan_pos
elseif is_line_end then
local line_text, new_scan_pos = capture_line_comment(source, non_ws)
if not line_text then break end
table.insert(pieces, 1, line_text)
scan_pos = new_scan_pos
else
break
end
end
if #pieces == 0 then return "" end
return table.concat(pieces, "\n")
end
-- ════════════════════════════════════════════════════════════════════════════
-- Argument-name extraction
-- ════════════════════════════════════════════════════════════════════════════
-- Walk `trimmed` backward from `pos` over trailing whitespace / asterisks / brackets,
-- returning the position of the first non-trailer character (i.e. the end of the identifier).
-- @param trimmed string
-- @param pos integer
-- @return integer
local function trim_trailer_back(trimmed, pos)
local back = pos
while back > 0 do
local ch = trimmed:sub(back, back)
if ch == " " or ch == "\t" or ch == "*" or ch == "]" or ch == "[" then
back = back - 1
else
break
end
end
return back
end
-- Walk `trimmed` backward from `pos` over identifier chars (alnum + `_`),
-- returning the position just before the identifier starts.
-- @param trimmed string
-- @param pos integer
-- @return integer
local function trim_ident_back(trimmed, pos)
local back = pos
while back > 0 do
local ch = trimmed:sub(back, back)
if duffle.is_alnum(ch) or ch == "_" then
back = back - 1
else
break
end
end
return back
end
--- Extract just the parameter NAMES from a function-args string (stripping type annotations). E.g.,
--- `"U4 off, U4 code, U1 r, U1 g, U1 b"` -> `{"off", "code", "r", "g", "b"}`
--- `"U4 *ptr"` -> `{"ptr"}`
@@ -332,9 +154,28 @@ local function extract_arg_names(args_str)
for _, tok in ipairs(tokens) do
local trimmed = duffle.trim(tok)
if trimmed ~= "" then
local ident_end = trim_trailer_back(trimmed, #trimmed)
local ident_start = trim_ident_back(trimmed, ident_end) + 1
local name = trimmed:sub(ident_start, ident_end)
-- Find the identifier at the end: walk back over trailers (whitespace + `*` + `[]`),
-- then walk back over the identifier chars (alnum + `_`).
local ident_end = #trimmed
while ident_end > 0 do
local ch = trimmed:sub(ident_end, ident_end)
if ch == " " or ch == "\t" or ch == "*" or ch == "]" or ch == "[" then
ident_end = ident_end - 1
else
break
end
end
local ident_start = ident_end
while ident_start > 0 do
local ch = trimmed:sub(ident_start, ident_start)
if duffle.is_alnum_byte(string.byte(ch)) or ch == "_" then
ident_start = ident_start - 1
else
break
end
end
ident_start = ident_start + 1
local name = trimmed:sub(ident_start, ident_end)
if name ~= "" then names[#names + 1] = name end
end
end
@@ -346,26 +187,34 @@ end
-- Component projection (read from pre-scanned SourceScan)
-- ════════════════════════════════════════════════════════════════════════════
-- Project pre-scanned MipsAtomComp_ / MipsAtomComp_Proc_ entries into Component shape.
-- Does per-source backward lookups for args (preceding function decl) and comment (preceding comment block).
-- Carries `body_tokens` forward from scan-source so word_count_rec reads from the precomputed table
-- instead of calling duffle.tokenize_body again.
-- @param source string -- the full source text (needed for backward lookups)
-- @param scan table -- SourceScan from duffle.scan_source
-- @return Component[]
--- Project pre-scanned MipsAtomComp_ / MipsAtomComp_Proc_ entries into Component shape.
--- Reads the scanner-owned `declaration_comment` (resolved by scan_source.lua, skipping backward across an associated bare `atom_dbg_skip` marker when present).
--- Per-source backward lookups remain in place only for the function `args` of proc components.
--- That lookup is unique to components.lua and stays separate from the declaration-comment walk.
--- Carries `body_tokens` forward from scan-source so word_count_rec reads from the precomputed table instead of calling duffle.tokenize_body again.
--- Carries the scanner-owned `debug_skip` flag forward so the generated projection can emit `/* atom_dbg_skip */`
--- before the authored comment and so `update_canonical_components` can mirror the same field onto `corpus.components[name]`.
--- @param source string -- the full source text (needed for backward lookups)
--- @param scan table -- SourceScan from duffle.scan_source
--- @return Component[]
local function project_components(source, scan)
local out = {}
for _, a in ipairs(scan.atoms) do
if a.kind == "comp_bare" or a.kind == "comp_proc" then
local args = find_function_args_for(source, a.raw_name, a.ident_pos)
local comment = preceding_comment_block(source, a.ident_pos)
local args = find_function_args_for(source, a.raw_name, a.ident_pos)
-- Comment ownership: scan_source.lua stamps `declaration_comment` on the record by walking backward past any associated bare marker.
-- The pass reads `declaration_comment` directly.
local comment = a.declaration_comment or ""
out[#out + 1] = {
line = a.line,
name = a.name,
body = a.body,
body_off = a.body_off,
body_tokens = a.body_tokens,
args = args,
comment = comment,
kind = a.kind, -- "comp_bare" | "comp_proc"; provenance emitter reads this.
debug_skip = a.debug_skip == true,
}
end
end
@@ -419,11 +268,11 @@ end
-- Word-count computation (memoized recursive lookup)
-- ════════════════════════════════════════════════════════════════════════════
-- Strip the `mac_` prefix from a component-call ident so we can look it up against the components-by-name table.
-- Returns the ident unchanged if it doesn't start with the prefix
-- (so a non-component ident like `mask_upper` falls through to the wc-table branch).
-- @param ident string|nil
-- @return string|nil
--- Strip the `mac_` prefix from a component-call ident so we can look it up against the components-by-name table.
--- Returns the ident unchanged if it doesn't start with the prefix
--- (so a non-component ident like `mask_upper` falls through to the wc-table branch).
--- @param ident string|nil
--- @return string|nil
local function strip_mac_prefix(ident)
if not ident then return nil end
if ident:sub(1, MAC_PREFIX_LEN) == MAC_PREFIX then
@@ -432,13 +281,13 @@ local function strip_mac_prefix(ident)
return ident
end
-- (internal) Recursive word-count lookup. `cache` is the memoization table shared across all components
-- in a single source's `count_all_components` pass; the in-progress -1 sentinel detects cycles (A -> B -> A).
-- @param name string -- the component name (without `mac_`)
-- @param comp_by_name table<string, Component>
-- @param wc table<string, integer>
-- @param cache table<string, integer>
-- @return integer
--- (internal) Recursive word-count lookup. `cache` is the memoization table shared across all components
--- in a single source's `count_all_components` pass; the in-progress -1 sentinel detects cycles (A -> B -> A).
--- @param name string -- the component name (without `mac_`)
--- @param comp_by_name table<string, Component>
--- @param wc table<string, integer>
--- @param cache table<string, integer>
--- @return integer
local function word_count_rec(name, comp_by_name, wc, cache)
if cache[name] ~= nil then return cache[name] end
cache[name] = -1 -- mark in-progress (cycle detection)
@@ -473,12 +322,11 @@ end
--- Compute word counts for every component in `components` in a single pass.
--- The name-lookup table + memoization cache are built ONCE (per source) instead of per-component,
--- so the cache survives across siblings and a component's recursive `mac_Y(...)` references hit memoized values
--- instead of re-walking the body.
--- so the cache survives across siblings and a component's recursive `mac_Y(...)`
--- references hit memoized values instead of re-walking the body.
--- Cycle detection (A -> B -> A) is preserved via the in-progress `-1` sentinel in `cache`.
---
--- @param components Component[]
--- @param wc table<string, integer>
--- @param wc table<string, integer>
--- @return table<string, integer> -- map of component name (without `mac_`) -> word count
local function count_all_components(components, wc)
local comp_by_name = {}
@@ -515,13 +363,6 @@ local function split_comment_lines(s)
return out
end
--- Split an atom body by top-level commas; drop empty tokens.
--- @param body string
--- @return string[]
local function tokens_from_body(body)
return duffle.tokenize_body_simple(body)
end
--- Determine the macro signature: function-args list (function form) or variadic-ignored (bare form).
--- @param args_str string|nil
--- @return string
@@ -558,20 +399,31 @@ end
--- Build the list of lines for one component
--- (signature comment, `#define mac_X(...)` line with backslash-continued tokens, then `WORD_COUNT(mac_X, N)` entry).
--- @param c Component
--- For skipped components, a `/* atom_dbg_skip */` marker comment is emitted immediately before the authored comment block.
--- The marker is a single line, the comment comes next, and the `#define` line follows. The `debug_skip` stamp is scanner-owned
--- (`a.debug_skip == true` on the declaration record); the components pass projects it directly.
--- @param c Component
--- @param components Component[]
--- @param wc table<string, integer>
--- @param wc table<string, integer>
--- @return string[] -- list of lines for this component
local function build_component_lines(c, counts)
local lines = {}
-- Marker comment: emitted once for every skipped component.
-- The marker is scanner-owned (declared by `atom_dbg_skip` immediately before the declaration in the source);
-- the components pass projects `c.debug_skip` and emits the marker as a generated comment.
if c.debug_skip then
lines[#lines + 1] = "/* atom_dbg_skip */"
end
if c.comment and c.comment ~= "" then
for _, line in ipairs(split_comment_lines(c.comment)) do
lines[#lines + 1] = line
end
end
local tokens = tokens_from_body(c.body)
local tokens = duffle.split_top_level_commas(c.body)
for i = 1, #tokens do tokens[i] = duffle.trim(tokens[i]) end
local sig = signature_from_args(c.args)
-- Direct lookup against the per-source precomputed `counts` table (built once by count_all_components).
local n = counts[c.name]
@@ -591,10 +443,10 @@ end
-- Per-source emit logic
-- ════════════════════════════════════════════════════════════════════════════
-- Build the boilerplate header lines (the `#ifdef INTELLISENSE_DIRECTIVES` block,
-- the `// Auto-generated` comment, the `// Source:` line, and the self-contained `WORD_COUNT` macro definition).
-- @param src SourceFile
-- @return string[]
--- Build the boilerplate header lines (the `#ifdef INTELLISENSE_DIRECTIVES` block,
--- the `// Auto-generated` comment, the `// Source:` line, and the self-contained `WORD_COUNT` macro definition).
--- @param src SourceFile
--- @return string[]
local function header_boilerplate(src)
return {
-- #pragma once wrapped in #ifdef INTELLISENSE_DIRECTIVES, matching the convention in lottes_tape.h.
@@ -607,8 +459,8 @@ local function header_boilerplate(src)
"// Component atoms (MipsAtomComp_(ac_*)) -> macro variants (mac_*)",
"",
-- Self-contained: define WORD_COUNT if not already defined.
-- We use the same definition here so the auto-generated entries below expand to compile-time constants whether
-- the metadata file is included first or not.
-- We use the same definition here so the auto-generated entries below expand
-- to compile-time constants whether the metadata file is included first or not.
"#ifndef WORD_COUNT",
"#define WORD_COUNT(name, count) enum { words_##name = (count) };",
"#endif",
@@ -616,13 +468,13 @@ local function header_boilerplate(src)
}
end
-- Compute the output path for one source's `.macs.h` file.
-- The pre-rework convention uses the *directory* basename
-- (not the source file basename) — e.g. `code/duffle/lottes_tape.h` produces `code/duffle/gen/duffle.macs.h`.
-- This matches what the C codebase #includes.
-- @param src SourceFile
-- @return string -- the output directory
-- @return string -- the full output path
--- Compute the output path for one source's `.macs.h` file.
--- The pre-rework convention uses the *directory* basename (not the source file basename)
--- e.g. `code/duffle/lottes_tape.h` produces `code/duffle/gen/duffle.macs.h`.
--- This matches what the C codebase #includes.
--- @param src SourceFile
--- @return string -- the output directory
--- @return string -- the full output path
local function compute_macs_h_path(src)
local out_dir = src.dir .. "/" .. GEN_SUBDIR
local out_path = out_dir .. "/" .. duffle.basename_no_ext(src.dir) .. ".macs.h"
@@ -631,12 +483,10 @@ end
--- Emit a per-source `.macs.h` header with the `mac_X` macros + `WORD_COUNT` entries.
--- Writes in BINARY mode so LF line endings are preserved (the git blob is LF; Windows text-mode would emit CRLF and break the byte-identical diff).
--- Honors `ctx.dry_run`: prints the intended path but does not write the file.
---
--- @param ctx PassCtx
--- @param src SourceFile
--- @param ctx PassCtx
--- @param src SourceFile
--- @param components Component[]
--- @param counts table<string, integer> -- precomputed word counts (from count_all_components)
--- @param counts table<string, integer> -- precomputed word counts (from count_all_components)
--- @return string|nil -- path to the written file (nil if no components)
local function emit_component_macros_h(ctx, src, components, counts)
if #components == 0 then return nil end
@@ -650,11 +500,6 @@ local function emit_component_macros_h(ctx, src, components, counts)
end
local content = table.concat(lines, "\n") .. "\n"
if ctx.dry_run then
print(string.format(" -> %s (dry-run)", out_path))
return out_path
end
duffle.ensure_dir(out_dir)
duffle.write_file_lf(out_path, content)
print(string.format(" -> %s", out_path))
@@ -665,15 +510,90 @@ end
-- Pass entry
-- ════════════════════════════════════════════════════════════════════════════
-- (internal) Extend `ctx.shared.word_counts` with this source's component macros
-- so offsets sees them without re-reading the file.
-- @param ctx PassCtx
-- @param components Component[]
-- @param counts table<string, integer> -- precomputed word counts (from count_all_components)
local function update_shared_word_counts(ctx, components, counts)
local wc = ctx.shared.word_counts
--- (internal) Extend `corpus.word_counts` with this source's component macros so offsets sees them without re-reading the file.
--- First declaration wins: a later caller's count is dropped (the existing entry from the first source is preserved).
--- @param corpus table -- the corpus
--- @param components Component[]
--- @param counts table<string, integer> -- precomputed word counts (from count_all_components)
local function update_canonical_word_counts(corpus, components, counts)
local wc = corpus.word_counts
for _, c in ipairs(components) do
wc["mac_" .. c.name] = counts[c.name]
local key = "mac_" .. c.name
if wc[key] == nil then
wc[key] = counts[c.name]
end
end
end
--- @class ComponentDef
--- @field name string -- bare name (without ac_/mac_ prefix)
--- @field line integer -- definition source line (line of `MipsAtomComp_(ac_X)` / `MipsAtomComp_Proc_(ac_X, ...)`)
--- @field path string -- absolute source path of the definition
--- @field kind string -- "comp_bare" | "comp_proc"
--- @field debug_skip boolean -- mirror of the scanner-owned `a.debug_skip`; consumers read this directly
--- (internal) Populate `corpus.components` with this source's components-by-name map.
--- First declaration wins; later declarations of the same bare name are dropped and recorded as a collision via `corpus.collisions` (kind = "component").
--- The pass does NOT write to `ctx.shared.components` (ownership follows the canonical contract).
--- The `debug_skip` field mirrors the scanner-owned declaration record (`c.debug_skip`).
--- No parallel skip map is built here; consumers that need the per-component skip state read `corpus.components[name].debug_skip` directly.
--- @param corpus table -- the corpus
--- @param src SourceFile
--- @param components Component[]
local function update_canonical_components(corpus, src, components)
local rel_path = src.path:gsub("\\", "/")
for _, c in ipairs(components) do
-- Keyed by bare name (e.g. `yield`, `load_tri_indices`).
-- The atoms_source_map pass looks up components by bare name from the corpus;
-- `mac_` prefix lives at the call-site identifier and is stripped before lookup.
if corpus.components[c.name] == nil then
corpus.components[c.name] = {
name = c.name,
line = c.line,
path = rel_path,
kind = c.kind or "comp_bare",
debug_skip = c.debug_skip == true,
}
else
-- A second declaration of the same bare name: record a typed collision so static-analysis + the report can surface it.
-- Identical-shape declarations (same path + line) reuse the first-wins entry without a collision record.
local existing = corpus.components[c.name]
if existing.path ~= rel_path or existing.line ~= c.line then
local kind = c.kind or "comp_bare"
local first_kind = existing.kind or "comp_bare"
corpus.collisions[#corpus.collisions + 1] = {
kind = "component",
name = c.name,
first_site = { path = existing.path, line = existing.line },
conflicting_site = { path = rel_path, line = c.line },
first_shape = "kind=" .. first_kind,
conflicting_shape = "kind=" .. kind,
}
end
end
end
end
--- (internal) Populate `corpus.component_body_index` with this source's body index entries.
--- First declaration wins; later declarations are dropped (no separate collision record: the components collision is already surfaced by `update_canonical_components`).
--- The pass writes to `corpus.component_body_index` only (the corpus owns this projection).
--- @param corpus table -- the corpus
--- @param src SourceFile
--- @param components Component[]
--- @param scan table -- the SourceScan payload (for line_of)
local function update_canonical_component_body_index(corpus, src, components, scan)
local line_of = scan and scan.line_of
for _, c in ipairs(components) do
if corpus.component_body_index[c.name] == nil then
corpus.component_body_index[c.name] = {
body_tokens = c.body_tokens,
body_off = c.body_off,
line_of = line_of,
source = src.path,
declaration = c.line,
kind = c.kind,
}
end
end
end
@@ -684,16 +604,41 @@ function M.run(ctx)
local errors = {}
local warnings = {}
for _, src in ipairs(ctx.sources) do
-- Corpus ownership gate.
local corpus = ctx.shared and ctx.shared.corpus
if type(corpus) ~= "table" then
error("components.run requires ctx.shared.corpus.", 0)
end
if type(corpus.source_order) ~= "table" then
error("components.run requires ctx.shared.corpus.source_order.", 0)
end
if type(corpus.word_counts) ~= "table" then
error("components.run requires ctx.shared.corpus.word_counts; "
.. "word_count_eval.run must run before components.run "
.. "(see PASSES deps).", 0)
end
-- Projection ownership:
-- * `corpus.word_counts["mac_"..name]` — current component count
-- * `corpus.components[name]` — bare-name component definition
-- * `corpus.component_body_index[name]` — body / line_of / source index
-- The pass writes to the corpus only; consumers read from the corpus directly.
for _, src in ipairs(corpus.source_order) do
-- project_components reads from src.scan + does backward lookups on src.text
local components = project_components(src.text, src.scan)
if #components > 0 then
-- Compute word counts for ALL components once (was: rebuilt per call inside the helpers).
local counts = count_all_components(components, ctx.shared.word_counts)
-- Compute all component word counts once per source.
-- Use `corpus.word_counts` so the recursive lookup sees both authored-metadata entries
-- (loaded by word_count_eval.run) AND same-source component entries (populated earlier in this loop by `update_canonical_word_counts`).
local counts = count_all_components(components, corpus.word_counts)
local macs_path = emit_component_macros_h(ctx, src, components, counts)
if macs_path then
outputs[#outputs + 1] = { macs_h = macs_path }
update_shared_word_counts(ctx, components, counts)
-- Populate the projections AFTER disk emission (so the byte-identical `.macs.h` contract is preserved before any current-count mutation).
update_canonical_word_counts(corpus, components, counts)
update_canonical_components(corpus, src, components)
update_canonical_component_body_index(corpus, src, components, src.scan)
end
end
end
File diff suppressed because it is too large Load Diff
+239
View File
@@ -0,0 +1,239 @@
--- passes/emission_model.lua: Per-atom emission projection.
---
--- The `emission-model` pass owns `atom.paths`, the canonical per-atom mutable surface for atoms and raw atoms with bodies in `ctx.shared.corpus.source_order`.
--- For each atom, the pass invokes `duffle.project_emission(body_text, component_index, word_counts, components)`.
--- It stores the ordered `items` stream plus the dense `word_events` / `markers` / `invocations` views on `atom.paths`.
---
--- Public boundary:
--- * `M.run(ctx)` is the only entry point.
--- * The pass returns `{outputs = {}, errors = ..., warnings = ...}`.
--- Pass kind = `validation` → `PASS_KIND_STOP_ON_ERROR.validation` preserves the existing build-stopping policy.
---
--- Source-order discipline:
--- * `corpus.source_order` sets the source-record order.
--- * Within each source, the pass visits `src.scan.atoms` and `src.scan.raw_atoms` in declaration order.
---
--- Per-atom projection fields on `atom.paths`:
--- `tokens`, `line_in_body`, `items`, `word_events`, `markers`, `invocations`, `errors`, `warnings`.
--- The construction walk appends `items` and derives each dense view from that ordered stream.
---
--- Component expansion and construction validation:
--- * known `mac_X(...)` calls recursively expand component bodies;
--- * invocation records retain monotonic IDs, parent IDs, immediate call text, and the immutable outermost root call text;
--- * invocation construction stamps `debug_skip` from `corpus.components[name].debug_skip` at the construction site (no second pass, no source parse, no parallel lookup);
--- * component cycles close balanced invocation boundaries and emit a `cycle` construction error at the recursive edge;
--- * declared-vs-measured component word counts emit `count_mismatch` construction errors; opaque uncounted macros emit warnings.
---
--- `passes.scan_source` strips its private `_code_macros` / `_code_macro_bodies` tables before this pass runs.
local M = {}
-- ─────────────────────────────────────────────────────────────────────────
-- Bootstrap: load `duffle_paths.lua` via debug.getinfo so the module works standalone (run as `luajit passes/emission_model.lua`) and when require'd from the orchestrator.
-- ─────────────────────────────────────────────────────────────────────────
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
-- ─────────────────────────────────────────────────────────────────────────
-- Helpers
-- ─────────────────────────────────────────────────────────────────────────
-- Convert the recursive walk's body-relative line numbers into physical source lines once.
-- The walker builds `line_of` from `body_text` and stamps body-relative line numbers (1..N) into `item.line` and `invocation.call_line`.
-- This function converts those values to physical source lines at the close site with the forwarded source `line_of` closure.
--
-- `call_line` discipline:
-- * ROOT invocations (`inv.parent_id == 0`) receive body-relative `call_line` values directly from `M.LineIndex(body_text)` in the walker.
-- The source `line_of` closure supplies physical lines at the close site, so this function converts each root value exactly once.
-- * INNER invocations (`inv.parent_id ~= 0`) receive physical `call_line` values directly from the COMPONENT's `line_of` in the walker.
-- Recursive descent forwards that closure through `corpus.component_body_index[name].line_of`; those values arrive physical and remain unchanged.
--
-- After this function, every `inv.call_line` is physical. DWARF and provenance output read it directly.
-- The word-event loop forwards the already-physical `outer_inv.call_line` into `we.call_line` for words inside an invocation.
local function stamp_root_provenance(projection, atom_record, src, corpus)
local root_line_of = src.scan and src.scan.line_of
assert(type(root_line_of) == "function"
, "emission_model: src.scan.line_of is required (canonical LineIndex closure over the source text) to stamp physical provenance")
assert(type(atom_record.body_off) == "number"
, "emission_model: atom_record.body_off (byte offset of the body's first byte in source) is required to derive `root_body_line`. The scanner must populate body_off for every atom record.")
-- `root_body_line` is the physical source line of the ATOM HEADER byte containing the opening `{`; that byte is one byte BEFORE `atom_record.body_off`.
-- The walker assigns line 2 to the body's first content line because line 1 is the trailing `\n` after `{`. Body-text line k therefore maps to `root_body_line + (k - 1)`.
-- `body_off - 1` points at the opening `{`, whose line index identifies the header line. `body_off` points after `{` and would shift every word row forward by one line.
local root_body_line = root_line_of(atom_record.body_off - 1)
or atom_record.line or 0
local component_index = corpus.component_body_index or {}
local word_items = {}
for _, item in ipairs(projection.items) do
if item.kind == "word" then word_items[#word_items + 1] = item end
end
-- Resolve one word's physical body line, where the byte containing that word appears in source.
-- * Component expansions carry `invocation_ids`; the component's full-file `line_of` leaves `item.line` physical.
-- * Raw tokens in the root atom body carry an empty `invocation_ids` list and a body-relative `item.line`; convert them here.
local function body_line_for(event, item)
local ids = event.invocation_ids or {}
-- The innermost open invocation identifies which line index the walker used.
-- A component `line_of` makes `item.line` physical; the atom's `body_text` line index makes it body-relative.
if ids and #ids > 0 then
local inner_id = ids[#ids]
local inner_inv = inner_id and projection.invocations[inner_id]
if inner_inv then
local component = component_index[inner_inv.component_name]
if component and component.line_of then
-- Walker used `comp.line_of`, which is the source's physical LineIndex. item.line is already physical.
return item.line or 0
end
end
end
-- RAW root-body word: item.line is body-text's 1-based line number (the first content line is line 2 because line 1 is the trailing `\n` after `{`).
-- Convert body-text-relative → physical using `root_body_line + (item.line - 1)`.
return (root_body_line or 0) + (item.line or 1) - 1
end
-- Stamp the root source path onto invocation records whose `call_path` the walker left empty.
-- The walker passes `body_entry.source` to `emit_invoke_begin`; `M.project_emission` creates the root `body_entry` with source `""`, leaving its `call_path` empty.
-- This stamp gives every invocation a physical `call_path` matching `passes/atoms_source_map.lua`'s in-memory provenance projection.
local root_path = src.path or ""
for _, inv in ipairs(projection.invocations) do
if inv.call_path == nil or inv.call_path == "" then
inv.call_path = root_path
end
end
-- Normalize `inv.call_line` to a physical source line.
-- * ROOT invocations (`parent_id == 0`) carry body-relative `call_line` values from `M.LineIndex(body_text)`; convert them once with `root_body_line`.
-- * INNER invocations (`parent_id ~= 0`) carry physical `call_line` values from the component's `line_of`; retain them unchanged.
for _, inv in ipairs(projection.invocations) do
if inv.parent_id == 0 then
inv.call_line = (root_body_line or 0) + (inv.call_line or 1) - 1
end
end
-- Build `body_lines` for each invocation.
-- `atoms_source_map` and `dwarf_injection` read `inv.body_lines[k]` directly from the invocation record created here.
-- Component words already carry physical `item.line` values from the walker's COMPONENT line index, so `body_line_for` returns them unchanged.
for _, inv in ipairs(projection.invocations) do
local sw = inv.start_word
local ew = inv.end_word
local bls = {}
for i = sw, ew do
local it = projection.items and projection.items[i]
if it and it.kind == "word" then
local fake_event = { invocation_ids = { inv.id } }
bls[#bls + 1] = body_line_for(fake_event, it) or 0
end
end
inv.body_lines = bls
end
-- Resolve each `word_event`'s physical `body_line` and `call_line`.
-- For words inside an invocation, `we.call_line` identifies the OUTER atom source line containing the `mac_X(...)` token that triggered expansion.
-- The root-invocation conversion above makes every `inv.call_line` physical; forward it directly and use each raw word's `body_line` as the fallback.
for index, we in ipairs(projection.word_events) do
local item = word_items[index] or {}
local body_line = body_line_for(we, item)
item.line = body_line
we.body_line = body_line
local call_line = body_line
local outer_id = we.outermost_invocation_id or 0
local outer_inv = projection.invocations[outer_id]
if outer_inv then
-- `outer_inv.call_line` is physical after the conversion loop above, so use it directly.
call_line = outer_inv.call_line
end
we.call_line = call_line
if we.def_path == nil or we.def_path == "" then we.def_path = src.path or "" end
if we.def_line == nil or we.def_line == 0 then we.def_line = atom_record.line or 0 end
if we.call_path == nil or we.call_path == "" then we.call_path = src.path or "" end
end
end
-- Project one atom record into `atom.paths`.
-- Mutates the atom record in-place and returns the projection (for pass-level error/warning accumulation).
local function project_atom(atom_record, src, corpus)
local body = atom_record.body or ""
local wc = corpus.word_counts or {}
local cbi = corpus.component_body_index or {}
-- That construction site stamps `invocation.debug_skip` while appending each record to `proj.invocations`.
local proj = duffle.project_emission(body, cbi, wc, corpus.components)
local paths = {
tokens = atom_record.body_tokens or {},
line_in_body = duffle.build_body_line_index(body),
items = proj.items,
word_events = proj.word_events,
markers = proj.markers,
invocations = proj.invocations,
errors = proj.errors,
warnings = proj.warnings,
}
stamp_root_provenance(proj, atom_record, src, corpus)
atom_record.paths = paths
return proj
end
-- ─────────────────────────────────────────────────────────────────────────
-- Run the emission-model pass.
-- ─────────────────────────────────────────────────────────────────────────
--- @param ctx PassCtx -- { shared = { corpus = ... }, out_root, ... }
--- @return PassResult
function M.run(ctx)
local outputs = {}
local errors = {}
local warnings = {}
local corpus = ctx and ctx.shared and ctx.shared.corpus
if type(corpus) ~= "table" then error("emission_model: ctx.shared.corpus is required (canonical projection)", 0) end
if type(corpus.source_order) ~= "table" then error("emission_model: ctx.shared.corpus.source_order is required", 0) end
-- Project once, collect errors + warnings for one atom.
-- Kind must be one of: atom | raw_atom | comp_bare | comp_proc.
local function process_atom(atom, src)
if not (atom and atom.body) then return end
local kind = atom.kind
if kind ~= "atom" and kind ~= "raw_atom" and kind ~= "comp_bare" and kind ~= "comp_proc" then
return
end
local proj = project_atom(atom, src, corpus)
for _, e in ipairs(proj.errors) do
-- Preserve `kind` (cycle / count_mismatch / unbalanced) so readers dispatch on the diagnostic class and leave the message string as display text.
errors[#errors + 1] = {
kind = e.kind,
line = e.line,
msg = e.msg,
source = e.source or src.path,
}
end
for _, w in ipairs(proj.warnings) do
warnings[#warnings + 1] = {
kind = w.kind,
line = w.line,
msg = w.msg,
}
end
end
-- Walk `corpus.source_order`; within each source, visit atoms followed by raw_atoms.
-- Recognized kinds (atom | raw_atom | comp_bare | comp_proc) each receive the atom.paths projection via duffle.project_emission.
-- Components are macros inlined into atom bodies; focused tests and isolated component analyses consume atom.paths directly.
for _, src in ipairs(corpus.source_order) do
local scan = src.scan or {}
for _, atom in ipairs(scan.atoms or {}) do
process_atom(atom, src)
end
for _, atom in ipairs(scan.raw_atoms or {}) do
process_atom(atom, src)
end
end
return {
outputs = outputs,
errors = errors,
warnings = warnings,
}
end
return M
+88 -224
View File
@@ -21,18 +21,12 @@
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source` (works both standalone + when require'd).
-- duffle_paths.lua sets package.path then returns `require("duffle")` at the bottom, so the dofile value IS the duffle module.
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
local word_count_eval = require("word_count_eval")
local count_token_words = word_count_eval.count_token_words
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
-- ════════════════════════════════════════════════════════════════════════════
-- Constants
-- ════════════════════════════════════════════════════════════════════════════
-- Marker-call identifiers inside atom bodies.
local LABEL_MARKER = "atom_label"
local OFFSET_MARKER = "atom_offset"
-- Offset macro/enum naming prefixes (the emitted header uses these).
local OFFSET_MACRO_PREFIX = "_atom_offset_"
local OFFSET_ENUM_PREFIX = "atom_offset_"
@@ -52,16 +46,10 @@ local OFFSET_MACRO_COL = 44
--- @field scan table -- pre-scanned SourceScan payload (from duffle.scan_source)
--- @class PassCtx
--- @field sources SourceFile[] -- all source files in the build
--- @field metadata_path string -- path to word_count.metadata.h
--- @field shared table -- cross-pass shared state
--- @field shared.word_counts table -- macro name -> word count
--- @field shared.corpus table -- canonical corpus projection
--- @field shared.word_counts table
--- @field out_root string -- output root (e.g. "build/gen")
--- @field project_root string -- project root (e.g. "code/")
--- @field upstream table<string, table> -- per-pass upstream outputs
--- @field flags table -- CLI flags
--- @field dry_run boolean -- if true, compute but don't write
--- @field verbose boolean -- log diagnostic info
--- @class PassResult
--- @field outputs table[] -- {kind=, path=} entries describing emit files
@@ -69,10 +57,10 @@ local OFFSET_MACRO_COL = 44
--- @field warnings table[] -- {line=, msg=} entries; build-succeeds
--- @class BranchOffset
--- @field tag string -- the marker tag (e.g. "F" in `atom_offset(F, T)`)
--- @field target string -- the target label name (e.g. "T" in `atom_offset(F, T)`)
--- @field pos integer -- the branch's word position within the atom body
--- @field offset integer -- computed `target_word - branch_word - 1`
--- @field tag string -- the marker tag (e.g. "F" in `atom_offset(F, T)`)
--- @field target string -- the target label name (e.g. "T" in `atom_offset(F, T)`)
--- @field branch_word integer -- branch word position within the atom body
--- @field offset integer -- computed `target_word - branch_word - 1`
--- @class AtomData
--- @field name string -- atom name
@@ -80,196 +68,85 @@ local OFFSET_MACRO_COL = 44
--- @field offsets BranchOffset[] -- per-branch offset list
-- ════════════════════════════════════════════════════════════════════════════
-- Per-token marker-call helpers (atom_label / atom_offset inside bodies)
-- Canonical marker projection
-- ════════════════════════════════════════════════════════════════════════════
-- Extract comma-separated identifier args from a parenthesized group after a function-like macro call.
-- Returns (args, after_paren) where `after_paren` is the position just past the closing `)`, or nil if `token` did not start with `(`.
-- @param token string
-- @param after_ident integer
-- @return string[], integer|nil
local function extract_ident_args(token, after_ident)
local arg_start = duffle.skip_ws_and_cmt(token, after_ident)
if token:sub(arg_start, arg_start) ~= "(" then return {}, nil end
local inner, after_paren = duffle.read_parens(token, arg_start)
-- scan: <marker>(<args>)
local args = {}
local pos = 1
local inner_len = #inner
while pos <= inner_len do
pos = duffle.skip_ws_and_cmt(inner, pos)
if pos > inner_len then break end
local ident, after = duffle.read_ident(inner, pos)
if ident and ident ~= "" then
table.insert(args, ident)
pos = after
else
pos = pos + 1
end
pos = duffle.skip_ws_and_cmt(inner, pos)
if pos <= inner_len and inner:sub(pos, pos) == "," then pos = pos + 1 end
end
return args, after_paren
end
-- (internal) Record a `atom_label(name)` marker — `at_pos` is the branch-free word position within the atom body.
-- @param labels table<string, integer>
-- @param args string[]
-- @param at_pos integer
local function record_label_marker(labels, args, at_pos)
if #args >= 1 then labels[args[1]] = at_pos end
end
-- (internal) Record a `atom_offset(tag, target)` marker.
-- @param branches table[] -- list of {pos=, target=, tag=}
-- @param args string[]
-- @param at_pos integer
local function record_offset_marker(branches, args, at_pos)
if #args >= 2 then
table.insert(branches, { pos = at_pos, target = args[2], tag = args[1] })
end
end
-- MARKER_TO_HANDLER — data-driven marker dispatch (the plex pattern).
-- Maps the marker ident to its recorder function. Each handler takes (out_table, args, at_pos).
-- Adding a new marker type = 1 row + 1 recorder function.
local MARKER_TO_HANDLER = {
[LABEL_MARKER] = record_label_marker,
[OFFSET_MARKER] = record_offset_marker,
-- MARKER_PROJECTORS is the marker-kind data table.
-- The emission-model pass already records marker word positions;
-- this pass only projects those records into the label/branch lookup shape needed by offset computation.
local MARKER_PROJECTORS = {
label = function(state, marker)
state.labels[marker.name] = marker.word_index
end,
offset = function(state, marker)
state.branches[#state.branches + 1] = {
tag = marker.name,
target = marker.target,
branch_word = marker.word_index,
}
end,
}
--- Scan a single token for atom_label/atom_offset markers, walking through balanced groups transparently (so nested calls are found).
--- @param token string
--- @param at_pos integer -- the branch-free word position of this token in the body
--- @param labels table<string, integer>
--- @param branches table[]
local function scan_for_atom_markers(token, at_pos, labels, branches)
local pos = 1
local tok_len = #token
while pos <= tok_len do
pos = duffle.skip_ws_and_cmt(token, pos)
if pos > tok_len then break end
local ch = token:sub(pos, pos)
if duffle.is_alpha(ch) then
local ident, after = duffle.read_ident(token, pos)
local handler = MARKER_TO_HANDLER[ident]
if handler then
local args, after_paren = extract_ident_args(token, after)
-- Marker found — dispatch to its recorder. markers share labels and branches as
-- out-tables; the recorder picks which one(s) to write to based on its semantics.
-- (record_label_marker writes to labels; record_offset_marker writes to branches.)
handler(ident == LABEL_MARKER and labels or branches, args, at_pos)
pos = after_paren or after
else
pos = after
end
else
local nx = duffle.skip_str_or_cmt(token, pos)
pos = (nx > pos) and nx or (pos + 1)
end
--- Project canonical marker records into the two lookup tables used by the offset renderer.
--- No source text, body text, or body token is inspected.
--- @param markers table[] -- atom.paths.markers
--- @return table<string, integer>, table[]
local function project_markers(markers)
local state = { labels = {}, branches = {} }
for _, marker in ipairs(markers or {}) do
local project = MARKER_PROJECTORS[marker.kind]
if project then project(state, marker) end
end
end
-- (internal) Count words emitted by the rest of `tok` after a marker call
-- (the marker call itself emits 0 words, but the source pattern may bundle the marker with the next instruction on the same line,
-- separated by no top-level comma).
-- Returns the word count contributed by that rest.
-- @param tok string
-- @param word_counts table
-- @return integer
local function count_marker_rest(tok, word_counts)
-- duffle.find_marker_call_end returns the position PAST the closing `)` of the marker call
-- (or nil if `tok` isn't a marker call). Canonical impl in duffle.lua is faster than the
-- file-local copy that used to live here (byte-indexed, no `tok:sub` per char).
local marker_end = duffle.find_marker_call_end(tok)
if not marker_end or marker_end >= #tok then return 0 end
local rest = duffle.trim(tok:sub(marker_end))
if rest == "" then return 0 end
return count_token_words(rest, word_counts)
end
-- (internal) Is this token a marker call (`atom_label` or `atom_offset`)?
-- @param tok string
-- @return boolean
local function is_marker_token(tok)
local leading_ident = duffle.read_ident(tok, 1)
return leading_ident == LABEL_MARKER or leading_ident == OFFSET_MARKER
end
--- Scan an atom body for labels + branches, count total words.
--- Returns (labels, branches, total_words).
--- @param body string
--- @param word_counts table
--- @return table<string, integer>, table[], integer
-- scan_atom_body: walk pre-tokenized body for atom_label/atom_offset markers + word counts.
-- Uses `atom.body_tokens` from the SourceScan payload (pre-tokenized by scan-source pass).
-- @param body_tokens table[] -- {{tok=string, rel=integer}, ...} from duffle.tokenize_body
-- @param word_counts table
-- @return table, table, integer -- labels, branches, total_words
local function scan_atom_body(body_tokens, word_counts)
local pos = 0
local labels = {}
local branches = {}
for _, t in ipairs(body_tokens) do
local tok = t.tok
if is_marker_token(tok) then
-- Marker call: record at the current pos, do NOT advance pos.
scan_for_atom_markers(tok, pos, labels, branches)
pos = pos + count_marker_rest(tok, word_counts)
else
local words = count_token_words(tok, word_counts)
scan_for_atom_markers(tok, pos, labels, branches)
pos = pos + words
end
end
return labels, branches, pos
return state.labels, state.branches
end
-- ════════════════════════════════════════════════════════════════════════════
-- Offset computation + header generation
-- ════════════════════════════════════════════════════════════════════════════
-- Compute branch offsets as `target_word - branch_word - 1`
-- (the standard MIPS branch-immediate encoding).
-- @param labels table<string, integer>
-- @param branches table[]
-- @return BranchOffset[]
--- Compute branch offsets as `target_word - branch_word - 1` (the standard MIPS branch-immediate encoding).
--- @param labels table<string, integer>
--- @param branches table[]
--- @return BranchOffset[]
local function compute_offsets(labels, branches)
local results = {}
for _, br in ipairs(branches) do
local target = labels[br.target]
if not target then
error("Branch target '" .. br.target .. "' has no atom_label (at word " .. br.pos .. ")")
error("Branch target '" .. br.target .. "' has no atom_label (at word " .. br.branch_word .. ")")
end
results[#results + 1] = { target = br.target, tag = br.tag, offset = target - br.pos - 1 }
results[#results + 1] = {
target = br.target,
tag = br.tag,
branch_word = br.branch_word,
offset = target - br.branch_word - 1,
}
end
return results
end
-- Right-pad `s` with spaces to width `w`. If `s` is already `w` or wider, no padding is added.
-- @param s string
-- @param w integer
-- @return string
--- Right-pad `s` with spaces to width `w`. If `s` is already `w` or wider, no padding is added.
--- @param s string
--- @param w integer
--- @return string
local function pad_right(s, w)
return s .. string.rep(" ", math.max(0, w - #s))
end
-- (internal) Build a constant-table entry `{macro_name, enum_name, value}` from a BranchOffset.
-- @param r BranchOffset
-- @return table
local function make_offset_const(r)
--- (internal) Build a constant-table entry `{macro_name, enum_name, value}` from a BranchOffset.
--- @param bo BranchOffset
--- @return table
local function make_offset_const(bo)
return {
macro_name = OFFSET_MACRO_PREFIX .. r.tag .. "_" .. r.target,
enum_name = OFFSET_ENUM_PREFIX .. r.tag .. "_" .. r.target,
value = r.offset,
macro_name = OFFSET_MACRO_PREFIX .. bo.tag .. "_" .. bo.target,
enum_name = OFFSET_ENUM_PREFIX .. bo.tag .. "_" .. bo.target,
value = bo.offset,
}
end
-- (internal) Emit one atom's offset constants + enum into the lines buffer.
-- @param add fun(s: string)
-- @param atom AtomData
--- (internal) Emit one atom's offset constants + enum into the lines buffer.
--- @param add fun(s: string)
--- @param atom AtomData
local function emit_atom_offsets(add, atom)
if #atom.offsets == 0 then return end
add("// --- atom: " .. atom.name .. " (" .. atom.total_words .. " words) ---")
@@ -290,10 +167,10 @@ local function emit_atom_offsets(add, atom)
add("")
end
-- Generate the per-source .offsets.h header.
-- @param source_path string
-- @param atoms_data AtomData[]
-- @return string
--- Generate the per-source .offsets.h header.
--- @param source_path string
--- @param atoms_data AtomData[]
--- @return string
local function generate_header(source_path, atoms_data)
local basename = duffle.basename_no_ext(source_path)
@@ -315,59 +192,41 @@ local function generate_header(source_path, atoms_data)
return table.concat(lines, "\n") .. "\n"
end
-- ════════════════════════════════════════════════════════════════════════════
-- M — module exports
-- ════════════════════════════════════════════════════════════════════════════
local M = {}
-- Project the pre-scanned SourceScan entries into the {name, body, body_tokens} shape this pass needs.
-- MipsAtom_ entries have kind="atom"; MipsCode code_<name> entries have kind="raw_atom".
-- `body_tokens` is set by scan-source on every `scan.atoms[i]` / `scan.raw_atoms[i]`; we carry it forward
-- so `scan_atom_body` reads from the precomputed table directly (no per-atom tokenize_body fallback).
-- @param scan table -- SourceScan from duffle.scan_source
-- @return table[] -- list of {name=, body=, body_tokens=}
local function project_atoms(scan)
local out = {}
for _, a in ipairs(scan.atoms) do
out[#out + 1] = { name = a.raw_name, body = a.body, body_tokens = a.body_tokens }
end
for _, a in ipairs(scan.raw_atoms) do
out[#out + 1] = { name = a.name, body = a.body, body_tokens = a.body_tokens }
end
return out
end
-- (internal) Process one source: project atoms from scan, scan bodies, write header.
-- Returns the offsets_h path if a header was written, or nil.
-- @param ctx PassCtx
-- @param src SourceFile
-- @return string|nil -- the offsets_h path
--- (internal) Process one source: render offsets from canonical atom paths.
--- Returns the offsets_h path if a header was written, or nil.
--- @param ctx PassCtx
--- @param src SourceFile
--- @return string|nil -- the offsets_h path
local function process_source(ctx, src)
local atoms = project_atoms(src.scan)
if #atoms == 0 then return nil end
local atoms_data = {}
for _, atom in ipairs(atoms) do
local labels, branches, total = scan_atom_body(atom.body_tokens, ctx.shared.word_counts)
local scan = src.scan or {}
local function append_atom(atom)
local paths = atom and atom.paths
if not paths then return end
local labels, branches = project_markers(paths.markers)
atoms_data[#atoms_data + 1] = {
name = atom.name,
total_words = total,
name = atom.raw_name or atom.name,
total_words = #(paths.word_events or {}),
offsets = compute_offsets(labels, branches),
}
end
for _, atom in ipairs(scan.atoms or {}) do append_atom(atom) end
for _, atom in ipairs(scan.raw_atoms or {}) do append_atom(atom) end
if #atoms_data == 0 then return nil end
local out_path = src.dir .. "/gen/" .. duffle.basename_no_ext(src.dir) .. ".offsets.h"
if not ctx.dry_run then
duffle.ensure_dir(duffle.dirname(out_path))
duffle.write_file(out_path, generate_header(src.path, atoms_data))
end
duffle.ensure_dir(duffle.dirname(out_path))
duffle.write_file(out_path, generate_header(src.path:gsub("/", "\\"), atoms_data))
return out_path
end
--- Run the offsets pass.
--- For each source, emits a per-module `<dir_basename>.offsets.h` containing `#define _atom_offset_F_T = N` constants
--- for every `atom_offset(F, T)` reference in the source's atoms.
--- For each canonical source, emits a per-module `<dir_basename>.offsets.h`
--- containing constants for every marker recorded in atom.paths.
--- @param ctx PassCtx
--- @return PassResult
function M.run(ctx)
@@ -375,7 +234,12 @@ function M.run(ctx)
local errors = {}
local warnings = {}
for _, src in ipairs(ctx.sources) do
local corpus = ctx.shared and ctx.shared.corpus
if type(corpus) ~= "table" or type(corpus.source_order) ~= "table" then
error("offsets.run requires ctx.shared.corpus.source_order (canonical corpus).", 0)
end
for _, src in ipairs(corpus.source_order) do
local out_path = process_source(ctx, src)
if out_path then
outputs[#outputs + 1] = { offsets_h = out_path }
+57 -58
View File
@@ -5,8 +5,8 @@
--- - `build/gen/<dir_basename>.annotations.txt` — one per source-directory containing atoms; aggregates across all sources in the directory.
--- - `build/gen/annotation_validation.txt` — the project summary.
---
--- The annotation pass stashes per-MODULE summary entries in `ctx.flags._annot_results` (set by `passes/annotation.lua`).
--- This pass re-validates each source via `annotation.validate()` to get the detailed per-source results needed for the report.
--- The annotation pass emits `errors.h` files per module and the canonical `corpus.sources_by_dir` projection groups sources by directory.
--- This pass iterates the canonical dir projection directly and re-validates each source via `annotation.validate()` to get the detailed per-source results.
---
--- **Conventions**: tabs (1/level), EmmyLua annotations, no regex,
--- Lua 5.3 compatible.
@@ -16,7 +16,6 @@
-- ════════════════════════════════════════════════════════════════════════════
-- Resolve `arg[0]` to an absolute-ish script directory so that `require("duffle")` resolves against `scripts/` regardless of CWD.
-- Note: this boilerplate is duplicated in 6 other entry scripts; a Phase-6 extraction target (`duffle.setup_package_path()`).
-- Bootstrap: see `ps1_meta.lua` for the rationale.
-- Bootstrap: load `scripts/duffle_paths.lua` (sets package.path + package.cpath).
-- Uses `debug.getinfo` to find this file's own directory, so it works
@@ -26,6 +25,11 @@
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
-- Load the annotation pass so we can re-validate each source against the canonical corpus projection.
-- The annotation pass exposes `M.validate`, which returns the per-source AnnotationResult (atoms / annots / macros / binds / errors / warnings)
-- that the report pass renders into the per-module `<dir_basename>.annotations.txt` output.
local annotation = dofile(_bootstrap_dir .. "annotation.lua")
-- ════════════════════════════════════════════════════════════════════════════
-- Constants
-- ════════════════════════════════════════════════════════════════════════════
@@ -61,16 +65,14 @@ local PASS_NAME = "report"
--- @field basename string -- filename without extension
--- @class PassCtx
--- @field sources SourceFile[] -- all source files in the build
--- @field metadata_path string -- path to word_count.metadata.h
--- @field shared table -- cross-pass shared state
--- @field out_root string -- output root (e.g. "build/gen")
--- @field project_root string -- project root (e.g. "code/")
--- @field upstream table<string, table> -- per-pass upstream outputs
--- @field flags table -- CLI flags + per-pass stash
--- @field flags._annot_results ModuleEntry[] -- stashed by annotation pass
--- @field dry_run boolean -- if true, compute but don't write
--- @field verbose boolean -- if true, log diagnostic info
--- @field sources SourceFile[] -- all source files in the build
--- @field metadata_path string -- path to word_count.metadata.h
--- @field shared table -- cross-pass shared state
--- @field out_root string -- output root (e.g. "build/gen")
--- @field project_root string -- project root (e.g. "code/")
--- @field upstream table<string, table> -- per-pass upstream outputs
--- @field flags table -- CLI flags + per-pass stash
--- @field verbose boolean -- if true, log diagnostic info
--- @class PassResult
--- @field outputs table[] -- {kind=, path=} entries describing emit files
@@ -139,31 +141,31 @@ local PASS_NAME = "report"
-- Per-MODULE annotation report (aggregated across all sources in a dir)
-- ════════════════════════════════════════════════════════════════════════════
-- Extract the basename (last path segment) of a forward- or back-slash separated path. Returns the input unchanged if no separator is found.
-- @param path string
-- @return string
--- Extract the basename (last path segment) of a forward- or back-slash separated path. Returns the input unchanged if no separator is found.
--- @param path string
--- @return string
local function source_basename(path)
return path:match(BASENAME_PATTERN) or path
end
-- (internal) Format a single annotation entry as one rendered line.
-- @param a AnnotEntry
-- @param src_name string
-- @return string
--- (internal) Format a single annotation entry as one rendered line.
--- @param a AnnotEntry
--- @param src_name string
--- @return string
local function format_annot_line(a, src_name)
if a.error then
return string.format(" ✗ line %d %s [ERROR: %s] [%s]", a.line, a.macro or "?", a.error, src_name)
end
local line = string.format(" ● line %d %s [%s]", a.line, a.name, src_name)
if a.binds then line = line .. " binds=" .. a.binds end
if #a.reads > 0 then line = line .. " reads={" .. table.concat(a.reads, ",") .. "}" end
if #a.writes > 0 then line = line .. " writes={" .. table.concat(a.writes, ",") .. "}" end
if a.binds then line = line .. " binds=" .. a.binds end
if #a.reads > 0 then line = line .. " reads={" .. table.concat(a.reads, ",") .. "}" end
if #a.writes > 0 then line = line .. " writes={" .. table.concat(a.writes, ",") .. "}" end
return line
end
-- (internal) Tally totals across all results in a module.
-- @param results AnnotationResult[]
-- @return integer, integer, integer, integer, integer, integer
--- (internal) Tally totals across all results in a module.
--- @param results AnnotationResult[]
--- @return integer, integer, integer, integer, integer, integer
local function tally_module_totals(results)
local total_atoms, total_annots, total_binds, total_macros = 0, 0, 0, 0
local total_errors, total_warnings = 0, 0
@@ -312,13 +314,11 @@ local function render_module_report(dir, sources, results)
macros = total_macros, errors = total_errors, warnings = total_warnings,
}
-- THE per-section dispatch. ONE loop over SECTION_RENDERERS. Each renderer writes its
-- header + content via the `add` closure (pre-bound above).
-- THE per-section dispatch. ONE loop over SECTION_RENDERERS.
-- Each renderer writes its header + content via the `add` closure (pre-bound above).
-- Adding a new section = 1 row here + 1 render_<thing>_section function.
for _, section in ipairs(SECTION_RENDERERS) do
add(section.header)
section.render(add, results, totals)
add("")
end
return table.concat(lines, "\n") .. "\n"
@@ -378,21 +378,20 @@ end
-- Orchestration helpers
-- ════════════════════════════════════════════════════════════════════════════
-- (internal) Pull per-source validate() results from the annotation pass's stash.
-- The annotation pass runs first in the dep chain and caches results in `ctx.flags._annot_source_results`;
-- we read from there instead of re-validating each source.
-- Returns the list of module results + the flat list of all results (for the project-wide summary).
-- @param ctx PassCtx
-- @param dir_sources SourceFile[]
-- @return AnnotationResult[], AnnotationResult[]
--- (internal) Re-validate every source in a directory against the canonical corpus projection.
--- Calls `annotation.validate()` per source to produce the per-source AnnotationResult (atoms / annots / macros / binds / errors / warnings)
--- that the report renderer consumes. Eeach report pass run is reproducible from the corpus.
--- Returns the list of module results + the flat list of all results (for the project-wide summary).
--- @param ctx PassCtx
--- @param dir_sources SourceFile[]
--- @return AnnotationResult[], AnnotationResult[]
local function lookup_module_results(ctx, dir_sources)
local src_cache = (ctx.flags and ctx.flags._annot_source_results) or {}
local module_results = {}
local all_results = {}
for _, src in ipairs(dir_sources) do
local result = src_cache[src.path]
if result then
result.source = src.path -- defensive (annotation tags it too; this guards against cache misses from earlier iterations)
if src.scan then
local result = annotation.validate(ctx, src, nil)
result.source = src.path -- tag for downstream rendering
module_results[#module_results + 1] = result
all_results[#all_results + 1] = result
end
@@ -400,9 +399,9 @@ local function lookup_module_results(ctx, dir_sources)
return module_results, all_results
end
-- (internal) Does this module's results contain anything worth emitting?
-- @param module_results AnnotationResult[]
-- @return boolean
--- (internal) Does this module's results contain anything worth emitting?
--- @param module_results AnnotationResult[]
--- @return boolean
local function module_has_content(module_results)
for _, r in ipairs(module_results) do
if #r.atoms > 0 or #r.annots > 0 or #r.binds > 0
@@ -413,8 +412,8 @@ local function module_has_content(module_results)
return false
end
-- (internal) Log a debug message if `_G[DEBUG_FLAG]` is truthy.
-- @param fmt string
--- (internal) Log a debug message if `_G[DEBUG_FLAG]` is truthy.
--- @param fmt string
local function debug_log(fmt, ...)
if _G[DEBUG_FLAG] then
io.stderr:write(string.format("[%s] " .. fmt, PASS_NAME, ...))
@@ -436,27 +435,27 @@ function M.run(ctx)
local errors = {}
local warnings = {}
local module_entries = (ctx.flags and ctx.flags._annot_results) or {}
local by_dir = ctx.by_dir or duffle.group_sources_by_dir(ctx.sources)
-- Module grouping comes from `corpus.sources_by_dir` (the canonical projection).
-- Iterate it directly; no private cache, no per-pass stash.
local corpus = ctx.shared and ctx.shared.corpus
local by_dir = (corpus and corpus.sources_by_dir) or {}
if not ctx.dry_run then duffle.ensure_dir(ctx.out_root) end
duffle.ensure_dir(ctx.out_root)
local all_results_for_summary = {}
for _, entry in ipairs(module_entries) do
debug_log("entry: dir=%s basename=%s atoms_count=%d dir_sources=%d\n", entry.dir, entry.dir_basename, entry.atoms_count, #(by_dir[entry.dir] or {}))
for dir, dir_sources in pairs(by_dir) do
local dir_basename = dir:match("([^/\\]+)$") or dir
debug_log("dir=%s basename=%s sources=%d\n", dir, dir_basename, #dir_sources)
if entry.atoms_count > 0 or #(by_dir[entry.dir] or {}) > 0 then
local dir_sources = by_dir[entry.dir] or {}
if #dir_sources > 0 then
local module_results, all_results = lookup_module_results(ctx, dir_sources)
for _, r in ipairs(all_results) do
all_results_for_summary[#all_results_for_summary + 1] = r
end
if module_has_content(module_results) then
local out_path = ctx.out_root .. "/" .. entry.dir_basename .. ".annotations.txt"
if not ctx.dry_run then
duffle.write_file(out_path, render_module_report(entry.dir, dir_sources, module_results))
end
local out_path = ctx.out_root .. "/" .. dir_basename .. ".annotations.txt"
duffle.write_file(out_path, render_module_report(dir, dir_sources, module_results))
outputs[#outputs + 1] = { annotations_txt = out_path }
else
debug_log(" -> no content; skipping\n")
@@ -464,7 +463,7 @@ function M.run(ctx)
end
end
if not ctx.dry_run and #all_results_for_summary > 0 then
if #all_results_for_summary > 0 then
local summary_path = ctx.out_root .. "/annotation_validation.txt"
duffle.write_file(summary_path, render_project_report(all_results_for_summary))
outputs[#outputs + 1] = { summary_txt = summary_path }
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+43 -105
View File
@@ -1,11 +1,17 @@
--- word_count_eval.lua — Word-counting logic for the tape-atom metaprogram pipeline.
---
--- Three responsibilities:
--- 1. **Public utilities** (used by `passes/components.lua`, `passes/offsets.lua`, `passes/annotation.lua`):
--- - `M.count_token_words(token, wc)` — words emitted by one token
--- - `M.scan_dir(dir, suffix)` — glob walk for *.macs.h
--- 2. **Pass entry** `M.run(ctx)` — loads metadata.h + *.macs.h into `ctx.shared.word_counts` for downstream passes.
--- 3. **Internal helpers** for the body scanner.
--- Two responsibilities:
--- 1. **Public utility** `M.count_token_words(token, wc)`: Used by `passes/offsets.lua`, `passes/annotation.lua`, and other passes.
--- 2. **Pass entry** `M.run(ctx)`: Loads the authored `word_count.metadata.h` into `ctx.shared.corpus.word_counts` for downstream passes.
--- The generated `.macs.h` files are OUTPUT artifacts and are NOT inputs to this pass;
--- Current component counts are owned by `passes/components.lua` (which populates `corpus.word_counts` and `corpus.component_body_index`
--- AFTER computing each current count from the just-built body + `corpus.word_counts`).
---
--- **Canonical contract**:
--- * `ctx.shared.corpus.word_counts` is the count table.
--- * `corpus.word_counts` is the sole count table. Consumers read `corpus.word_counts` directly.
--- * `ctx.shared.components` and `ctx.shared.component_body_index` are NOT created by this pass (projections only).
--- * No `.macs.h` recursive discovery (no `scan_dir`, no scan cache, no `_invalidate_scan_cache`).
---
--- **Conventions**: tabs (1/level), EmmyLua annotations, no regex,
--- Lua 5.3 compatible.
@@ -14,27 +20,11 @@
-- Module-scope requires + package.path setup
-- ════════════════════════════════════════════════════════════════════════════
-- Resolve `arg[0]` to an absolute-ish script directory so that `require("duffle")` resolves against `scripts/` regardless of CWD.
-- Note: this boilerplate is duplicated in 6 other entry scripts; a Phase-6 extraction target (`duffle.setup_package_path()`).
-- Bootstrap: see `ps1_meta.lua` for the rationale.
-- Bootstrap: load `scripts/duffle_paths.lua` (sets package.path + package.cpath).
-- Uses `debug.getinfo` to find this file's own directory, so it works both standalone and when require'd from the orchestrator.
-- Bootstrap: load `duffle_paths.lua` via `debug.getinfo(1, "S").source` (works both standalone + when require'd).
-- duffle_paths.lua sets package.path then returns `require("duffle")` at the bottom, so the dofile value IS the duffle module.
local _bootstrap_dir = debug.getinfo(1, "S").source:match("^@?(.*[/\\])") or "./"
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
-- ════════════════════════════════════════════════════════════════════════════
-- Constants
-- ════════════════════════════════════════════════════════════════════════════
-- Windows separator char — used by `fname:match` to recognize `.macs.h` files.
local PATH_SEP_BACKSLASH = "\\"
-- Required native extension: lfs (LuaFileSystem). Built by `update_deps.ps1` to
-- `toolchain/lfs/lfs.dll` and wired into package.cpath by `scripts/duffle_paths.lua`.
-- If lfs is missing, `require` throws — fail loud per the build-tool convention.
local lfs = require("lfs")
local duffle = dofile(_bootstrap_dir .. "../duffle_paths.lua")
-- ════════════════════════════════════════════════════════════════════════════
-- Type declarations
@@ -53,12 +43,12 @@ local lfs = require("lfs")
--- @field sources SourceFile[] -- all source files in the build
--- @field metadata_path string -- path to word_count.metadata.h
--- @field shared table -- cross-pass shared state
--- @field shared.word_counts WordCounts -- populated by this pass
--- @field shared.corpus table -- canonical corpus (required)
--- @field shared.corpus.word_counts WordCounts -- canonical count table (populated by this pass)
--- @field out_root string -- output root (e.g. "build/gen")
--- @field project_root string -- project root (e.g. "code/")
--- @field upstream table<string, table> -- per-pass upstream outputs
--- @field flags table -- CLI flags
--- @field dry_run boolean -- if true, compute but don't write
--- @field verbose boolean -- if true, log diagnostic info
--- @class PassResult
@@ -80,9 +70,8 @@ local M = {}
--- For most tokens (regular MIPS instructions) this returns 1.
--- For `mac_X(...)` calls, this returns the resolved word count from `wc` (recursively if needed). For `nop2` etc., returns wc[name].
--- For unknown macros, returns 1 and (optionally) warns.
---
--- @param token string -- a single token from split_top_level_commas
--- @param wc WordCounts -- the shared word-count table
--- @param token string -- a single token from split_top_level_commas
--- @param wc WordCounts -- the shared word-count table
--- @return integer
function M.count_token_words(token, wc)
local s = duffle.trim(token)
@@ -97,93 +86,42 @@ function M.count_token_words(token, wc)
return 1
end
-- ┌────────────────────────────────────────────────────────────────────┐
-- │ Shared utility: scan_dir │
-- └────────────────────────────────────────────────────────────────────┘
--- Recursively scan a directory for files matching a glob suffix.
---
--- The `.macs.h` files produced by the components pass always live at `<project_root>/<module>/gen/`.
--- Native walk via lfs.attributes + lfs.dir: ~2ms vs ~56ms for the prior `dir /b /s` subprocess.
---
--- @param dir string -- directory to scan (absolute or relative)
--- @param suffix string -- file pattern, e.g. "*.macs.h"
--- @return string[]
-- Cache the scan_dir result per (dir, suffix) in package.loaded.
-- The cache persists for the lifetime of the Lua process (cleared when ps1_meta.lua exits).
-- If a build removes/creates .macs.h files mid-process, the caller can invalidate by calling `M._invalidate_scan_cache()`.
local SCAN_CACHE_KEY = "__word_count_eval_scan_cache__"
--- Scan `code/` for files matching `suffix` (e.g. `*.macs.h`).
--- Native directory enumeration via lfs (~2ms). Zero subprocess spawns.
---
--- @param dir string -- project root directory
--- @param suffix string -- file pattern, e.g. "*.macs.h"
--- @return string[]
function M.scan_dir(dir, suffix)
local key = dir .. "\0" .. suffix
local cache = package.loaded[SCAN_CACHE_KEY]
if cache and cache[key] then return cache[key] end
local results = {}
local code_dir = dir .. "/code"
if lfs.attributes(code_dir, "mode") == "directory" then
for mod_name in lfs.dir(code_dir) do
if mod_name ~= "." and mod_name ~= ".." then
local gen_path = code_dir .. "/" .. mod_name .. "/gen"
if lfs.attributes(gen_path, "mode") == "directory" then
for fname in lfs.dir(gen_path) do
if fname:match("%.macs%.h$") then
results[#results + 1] = gen_path .. "/" .. fname
end
end
end
end
end
end
-- Cache the result (including empty results).
cache = cache or {}
cache[key] = results
package.loaded[SCAN_CACHE_KEY] = cache
return results
end
--- Invalidate the scan cache (call after creating new .macs.h files in the same Lua process — usually not needed).
function M._invalidate_scan_cache() package.loaded[SCAN_CACHE_KEY] = nil end
-- ┌────────────────────────────────────────────────────────────────────┐
-- │ Pass entry: M.run(ctx) — "word-counts" pass │
-- └────────────────────────────────────────────────────────────────────┘
--- Load metadata.h + scan for existing *.macs.h files into ctx.shared.word_counts.
--- Loading the .macs.h files is idempotent: entries from later (current-build) .macs.h files override metadata.h entries of the same name.
--- Load the authored `word_count.metadata.h` into `ctx.shared.corpus.word_counts`.
--- Generated `.macs.h` files are OUTPUT artifacts and are NOT scanned as inputs.
--- Current component counts are computed and inserted by `passes/components.lua`
--- after the components pass iterates `corpus.source_order` and writes each source's `<dir_basename>.macs.h` file.
---
--- Contract:
--- * `ctx.shared.corpus` MUST exist (canonical corpus ownership).
--- * `ctx.metadata_path` MUST be a readable file path to the authored `word_count.metadata.h`.
--- * The pass assigns exactly one table to `corpus.word_counts`.
--- Consumers read the corpus-owned table directly.
--- Consumers must read `corpus.word_counts` directly.
--- @param ctx PassCtx
--- @return PassResult
function M.run(ctx)
local wc = {}
-- 1. Load metadata.h (the encoding-macro source of truth).
local meta_counts = duffle.load_word_counts(ctx.metadata_path)
for name, count in pairs(meta_counts) do wc[name] = count end
-- 2. Scan project_root recursively for *.macs.h files (component-macro source).
local macs_files = M.scan_dir(ctx.project_root, "*.macs.h")
for _, macs_path in ipairs(macs_files) do
local ok, mc = pcall(duffle.load_word_counts, macs_path)
if not ok then
io.stderr:write(string.format("[word_count_eval] parse error in '%s': %s\n", macs_path, tostring(mc)))
elseif type(mc) ~= "table" then
io.stderr:write(string.format("[word_count_eval] '%s' did not return a table (got %s)\n", macs_path, type(mc)))
else
for name, count in pairs(mc) do wc[name] = count end
end
-- 1. Canonical-corpus ownership gate.
local corpus = ctx.shared and ctx.shared.corpus
if type(corpus) ~= "table" then
error("word_count_eval.run requires ctx.shared.corpus (canonical corpus). The fixture must install the corpus before running this pass.", 0)
end
ctx.shared.word_counts = wc
-- 2. metadata_path gate.
if type(ctx.metadata_path) ~= "string" or ctx.metadata_path == "" then
error("word_count_eval.run requires ctx.metadata_path (path to the authored word_count.metadata.h).", 0)
end
-- 3. Load authored metadata. Generated .macs.h files are NOT scanned
-- (the pass computes their counts from the just-built bodies after disk emission; see passes/components.lua).
local wc = duffle.load_word_counts(ctx.metadata_path)
-- 4. Assign the count table. ONE assignment, no copy. The assignment creates no secondary alias.
corpus.word_counts = wc
return { outputs = {}, errors = {}, warnings = {} }
end
+7 -11
View File
@@ -1,17 +1,15 @@
-- autoexec.lua - pcsx_debug_helper plugin entry point.
-- Packaged in scripts/pcsx_debug_helper.zip. Loaded by pcsx-redux via
-- the -archive CLI flag (see scripts/launch_pcsx_debug.ps1).
-- Packaged in scripts/pcsx_debug_helper.zip. Loaded by pcsx-redux via the -archive CLI flag (see scripts/launch_pcsx_debug.ps1).
--
-- Registers two web handlers for external CLI tools:
-- /api/v1/lua/gte - full GTE state (32 data + 32 control regs + PC)
-- /api/v1/lua/gp - GP state summary (screenshot endpoint + VRAM endpoint refs)
--
-- The GTE handler reads COP2 regs via PCSX.getRegisters().CP2D/CP2C. The
-- pcsx-redux gdb stub doesn't expose COP2, so this is the only way for
-- external tools to see GTE state.
-- The GTE handler reads COP2 regs via PCSX.getRegisters().CP2D/CP2C.
-- The pcsx-redux gdb stub doesn't expose COP2, so this is the only way for external tools to see GTE state.
--
-- The GP handler is a thin pointer: pcsx-redux's Lua API exposes only PCSX.GPU.takeScreenShot()
-- (no GPUSTAT, no GP0/GP1 command log, no display state). For richer GP state, the existing web endpoints are the practical path:
-- The GP handler is a thin pointer:
-- pcsx-redux's Lua API exposes only PCSX.GPU.takeScreenShot() (no GPUSTAT, no GP0/GP1 command log, no display state). For richer GP state, the existing web endpoints are the practical path:
-- /api/v1/state/still - PNG screenshot
-- /api/v1/gpu/vram/raw - VRAM raw bytes (1MB)
--
@@ -48,8 +46,6 @@ local function register_handlers()
end
local ok, err = pcall(register_handlers)
if ok then
print("[pcsx_debug_helper] handlers registered: gte, gp")
else
print("[pcsx_debug_helper] registration failed: " .. tostring(err))
if ok then print("[pcsx_debug_helper] handlers registered: gte, gp")
else print("[pcsx_debug_helper] registration failed: " .. tostring(err))
end
+416 -350
View File
File diff suppressed because it is too large Load Diff
+1 -16
View File
@@ -39,13 +39,7 @@ pop-location
# ════════════════════════════════════════════════════════════════════════════
# PCSX-Redux — built via MSBuild (VS2022)
#
# Requires: Visual Studio 2022 with the C++ desktop workload.
# The .vcxproj files target platform toolset v145, but VS2022 ships v143;
# we pass /p:PlatformToolset=v143 to retarget at build time (no file edits).
# NuGet packages (glfw, luajit.native, libFFmpeg-lite, x64sentry) are
# restored automatically by MSBuild on first build.
#
# Output: toolchain\pcsx-redux\vsprojects\x64\Debug\pcsx-redux.exe
# ════════════════════════════════════════════════════════════════════════════
@@ -65,8 +59,7 @@ $path_pcsx_sln = join-path $path_pcsx_redux 'vsprojects\pcsx-redux.sln'
& $msbuild_exe $path_pcsx_sln /p:Configuration=Release /p:Platform=x64 /p:PlatformToolset=v143 /m /v:minimal
# Locate luajit via scoop. `luajit.exe` is on PATH via scoop's shim;
# we use `scoop prefix` to find the install root for the include dir
# (needed to compile lpeg against luajit's headers).
# we use `scoop prefix` to find the install root for the include dir (needed to compile lpeg against luajit's headers).
# If scoop or luajit is missing, fail fast with an actionable message.
$luajit_prefix = & scoop prefix luajit 2>$null
if (-not $luajit_prefix -or -not (Test-Path (Join-Path $luajit_prefix 'bin/luajit.exe'))) {
@@ -87,7 +80,6 @@ if (-not $lua_inc_dir) {
# Generate lpeg.dll by compiling the 6 source files directly.
# `gcc` is on PATH (scoop's shim puts it there).
# The source files: lpcap.c lpcode.c lpcset.c lpprint.c lptree.c lpvm.c
# (per the lpeg makefile — no `make.lua` template generator in this version).
# Link against luajit's import library (`libluajit-5.1.a`) for the Lua C API symbols (lua_*, luaL_*).
$luajit_lib_dir = Join-Path $luajit_prefix 'lib'
$lpeg_sources = @('lpcap.c', 'lpcode.c', 'lpcset.c', 'lpprint.c', 'lptree.c', 'lpvm.c')
@@ -103,8 +95,6 @@ pop-location
# ════════════════════════════════════════════════════════════════════════════
# lfs (LuaFileSystem) — compiled from pcsx-redux's vendored luafilesystem source.
# Used by word_count_eval.lua :: scan_dir for native directory enumeration (~2ms)
# instead of spawning `dir /b /s` as a subprocess (~56ms).
# Source: toolchain/pcsx-redux/third_party/luafilesystem/src/lfs.c
# Output: toolchain/lfs/lfs.dll
# ════════════════════════════════════════════════════════════════════════════
@@ -118,11 +108,6 @@ $lfs_dll_import = join-path $luajit_lib_dir 'libluajit-5.1.dll.a'
# ════════════════════════════════════════════════════════════════════════════
# OpenBIOS — built from the PCSX-Redux source tree via make + mipsel-none-elf
#
# OpenBIOS is an open-source PS1 BIOS implementation (no retail BIOS dump needed).
# It builds with the MIPS cross-toolchain (`mipsel-none-elf-gcc`, on PATH via the `mips` toolchain installer)
# + `make` (on PATH via scoop).
#
# Output: toolchain\pcsx-redux\src\mips\openbios\openbios.bin
# ════════════════════════════════════════════════════════════════════════════