notes

CONVENTIONS.md

@@ -10,6 +10,7 @@ This document outlines the strict C style and architectural conventions expected
     *   Float: `F4`, `F8`
     *   Boolean: `B1`, `B2`, `B4`, `B8` (use `true`/`false` primitives)
     *   Strings/Chars: `UTF8` (for characters), `Str8` (for string slices)
+*   **Fundamental Type Casts:** Strictly use the provided casting macros (e.g., `u8_(val)`, `u4_r(ptr)`, `s4_(val)`) instead of standard C-style cast syntax like `(U8)val`. Standard casts should only be used for complex types or when an appropriate macro isn't available.
 *   **WinAPI Structs:** Only use `MS_` prefixed fundamental types (e.g., `MS_LONG`, `MS_DWORD`) *inside* WinAPI struct definitions (`MS_WNDCLASSA`, etc.) to maintain FFI compatibility. Do not use them in general application logic.
 
 ## 2. Declaration Wrappers & X-Macros
@@ -18,8 +19,8 @@ This document outlines the strict C style and architectural conventions expected
     *   `typedef Enum_(UnderlyingType, Name) { ... };`
 *   **X-Macros:** Use X-Macros to tightly couple Enums with their corresponding string representations or metadata.
     ```c
-    #define My_Tag_Entries() 
+    #define My_Tag_Entries() \
-            X(Define, "Define") 
+            X(Define, "Define") \
             X(Call,   "Call")
     ```
 
@@ -32,8 +33,27 @@ This document outlines the strict C style and architectural conventions expected
 
 ## 4. Formatting & Layout
 *   **Vertical Alignment:** Align related variable declarations, struct fields, and function prototypes into columns to create a "sheet-like" layout. This improves visual parsing.
-*   **Brace Style:** Use Allman style (braces on a new line) for function bodies control block (`if`, `for`, `switch`, etc.) that spans more than 50 lines or contains nested logic.
+    *   Example Struct:
-*   **Conditionals:** Always place `else if` and `else` statements on a new line, un-nested from the previous closing brace.
+        ```c
+        typedef struct MS_WNDCLASSA {
+            U4          style;
+            S8          (*lpfnWndProc)(void*, U4, U8, S8);
+            S4          cbClsExtra;
+            // ...
+            char const* lpszClassName;
+        } MS_WNDCLASSA;
+        ```
+*   **Multi-line Argument Alignment:** For long function signatures, place one argument per line with a single 4-space indent.
+*   **WinAPI Grouping:** Group foreign procedure declarations by their originating OS library (e.g., Kernel32, User32, GDI32) using comment headers.
+*   **Brace Style:** Use Allman style (braces on a new line) for function bodies or control blocks (`if`, `for`, `switch`, etc.) that are large or complex. Smaller blocks may use K&R style.
+*   **Conditionals & Control Flow:** Always place `else if` and `else` statements on a new line. Align control flow parentheses (e.g., between consecutive `while` and `if` blocks) vertically when possible for aesthetic uniformity:
+    ```c
+    while (len < 8) len ++;
+    if    (len > 0) { ... }
+    ```
+*   **Address-Of Operator:** Do insert a space between the address-of operator (`&`) and the variable name.
+    *   **Correct:** `& my_var`
+    *   **Incorrect:** `&my_var`
 
 ## 5. Memory Management
 *   **Standard Library:** The C standard library is linked, but headers like `<stdlib.h>` or `<string.h>` should not be included directly. Required functions should be declared manually if needed, or accessed via compiler builtins.
@@ -45,3 +65,14 @@ This document outlines the strict C style and architectural conventions expected
     *   **Correct:** `char const* my_ptr;` (Pointer to a constant character)
     *   **Correct:** `U4 const* const my_ptr;` (Constant pointer to a constant U4)
     *   **Incorrect:** `const char* my_ptr;`
+
+## 7. Metadata Coupling (X-Macros)
+*   **Metadata Enums:** Use X-Macros to define Enums that are tightly coupled with static metadata (colors, prefixes, names).
+    *   Example:
+        ```c
+        #define Tag_Entries() \
+                X(Define,  "Define",  0x0018AEFF, ":") \
+                X(Call,    "Call",    0x00D6A454, "~")
+        ```
+*   **Naming Conventions:** When using X-Macros for Tags, entry names should be PascalCase, and the Enum symbols should be prefixed with the Enum type name (e.g., `tmpl(STag, Define)` -> `STag_Define`).
+

GEMINI.md

@@ -5,8 +5,10 @@
 DO NOT EVER make a shell script unless told to. DO NOT EVER make a readme or a file describing your changes unless your are told to. If you have commands I should be entering into the command line or if you have something to explain to me, please just use code blocks or normal text output. DO NOT DO ANYTHING OTHER THAN WHAT YOU WERE TOLD TODO. DO NOT EVER, EVER DO ANYTHING OTHER THAN WHAT YOU WERE TOLD TO DO. IF YOU WANT TO DO OTHER THINGS, SIMPLY SUGGEST THEM, AND THEN I WILL REVIEW YOUR CHANGES, AND MAKE THE DECISION ON HOW TO PROCEED. WHEN WRITING SCRIPTS USE A 120-160 character limit per line. I don't want to see scrunched code.
 The user will often screenshot various aspects of the development with ShareX, which will be available in the current months directory: 'C:\Users\Ed\scoop\apps\sharex\current\ShareX\Screenshots\2026-02'
 You may read fromt his and the user will let you know (by last modified) which of the last screenshots are the most relevant. Otherwise they manually paste relevant content in the './gallery' directory.
+Do not use the .gitignore as a reference for WHAT YOU SHOULD IGNORE. THAT IS STRICT FOR THE GIT REPO, NOT FOR INFERENCING FILE RELEVANCE.
 
 ## Coding Conventions
+
 Before writing any C code in this workspace, you MUST review the strict stylistic and architectural guidelines defined in [CONVENTIONS.md](./CONVENTIONS.md). These dictate the usage of byte-width types, X-Macros, WinAPI FFI mapping, and memory arenas.
 
 ## Necessary Background for Goal
@@ -38,3 +40,31 @@ Based on the curation in `./references/`, the resulting system MUST adhere to th
 4. **Preemptive Scatter ("Tape Drive"):** Function arguments are not pushed to a stack before a call. They are "scattered" into pre-allocated, contiguous global memory slots during compilation/initialization. The function simply reads from these known offsets, eliminating argument gathering overhead.
 5. **No `if/then` branches:** Rely on hardware-level flags like conditional returns (`ret-if-signed`) combined with factored calls to avoid writing complex AST parsers.
 6. **No Dependencies:** C implementation must be minimal (`-nostdlib`), ideally running directly against OS APIs (e.g., WinAPI `VirtualAlloc`, `ExitProcess`, `GDI32` for rendering).
+
+## Current Development Roadmap (attempt_1)
+
+Here's a breakdown of the next steps to advance the `attempt_1` implementation towards a ColorForth derivative:
+
+1.  **Enhance Lexer/Parser/Compiler (JIT) in `main.c`:**
+    *   **Token Interpretation:** Refine the interpretation of the 28-bit payload based on the 4-bit color tag (e.g., differentiate between immediate values, dictionary IDs, and data addresses).
+    *   **Dictionary Lookup:** Improve the efficiency and scalability of dictionary lookups for custom words beyond the current linear search.
+    *   **New Word Definition:** Implement mechanisms for defining new Forth words directly within the editor, compiling them into the `code_arena`.
+
+2.  **Refine Visual Editor (`win_proc` in `main.c`):**
+    *   **Dynamic Colorization:** Ensure all rendered tokens accurately reflect their 4-bit color tags, updating dynamically with changes.
+    *   **Annotation Handling:** Implement more sophisticated display for token annotations, supporting up to 8 characters clearly without truncation or visual artifacts.
+    *   **Input Handling:** Improve text input for `STag_Data` (e.g., supporting full hexadecimal input, backspace functionality).
+    *   **Cursor Behavior:** Ensure the cursor accurately reflects the current editing position within the token stream.
+
+3.  **Expand Register-Only Stack Operations:**
+    *   Implement core Forth stack manipulation words (e.g., `DUP`, `DROP`, `OVER`, `ROT`) by generating appropriate x86-64 assembly instructions that operate solely on `RAX` and `RDX`.
+
+4.  **Develop `Tape Drive` Memory Management:**
+    *   Ensure all memory access (read/write) for Forth variables and data structures correctly utilize the `vm_globals` array and the "preemptive scatter" approach.
+
+5.  **Implement Control Flow without Branches:**
+    *   Leverage conditional returns and factored calls to create more complex control flow structures (e.g., `IF`/`ELSE`/`THEN` equivalents) without introducing explicit `jmp` instructions where not architecturally intended.
+
+6.  **Continuous Validation & Debugging:**
+    *   Enhance debugging output within the UI to provide clearer insight into VM state (RAX, RDX, global memory, log buffer) during execution.
+    *   Consider adding simple "tests" as Forth sequences within `tape_arena` to verify new features.

attempt_1/duffle.amd64.win32.h

@@ -696,4 +696,15 @@ WinAPI S4    ms_delete_object(void* ho)                                    asm("
 #define MS_VK_DOWN             0x28
 
 #define MS_PAGE_EXECUTE_READWRITE 0x40
+
+#define MS_WM_CHAR   0x0102
+#define MS_VK_RETURN 0x0D
+#define MS_VK_BACK   0x08
+#define MS_VK_TAB    0x09
+#define MS_VK_SPACE  0x20
+#define MS_VK_F5     0x74
+#define MS_VK_PRIOR  0x21
+#define MS_VK_NEXT   0x22
+
+#define MS_VK_SHIFT 0x10
 #pragma endregion OS_GDI_And_Minimal

attempt_1/main.c

@@ -71,15 +71,15 @@ global U4 log_count      = 0;
 global S4 scroll_y_offset = 0;
 
 void ms_builtin_print(U8 val) {
-		if (log_count < 16) {
+	if (log_count < 16) {
-				log_buffer[log_count++] = val;
+		log_buffer[log_count++] = val;
-		}
+	}
 }
 
 // Dictionary
 typedef struct {
-		U4 val;
+	U4 val;
-		U4 offset;
+	U4 offset;
 } DictEntry;
 global DictEntry dict[256];
 global U8        dict_count = 0;
@@ -216,11 +216,11 @@ IA_ void compile_and_run_tape(void)
 				in_def = false;
 			}
 		} 
-		else if (tag == tmpl(STag, Data)) {
+		else if (tag == STag_Data) {
 			emit8(0x48); emit8(0x89); emit8(0xC2); // mov rdx, rax
 			emit8(0x48); emit8(0xC7); emit8(0xC0); emit32(val); // mov rax, imm32
 		} 
-		else if (tag == tmpl(STag, Imm)) 
+		else if (tag == STag_Imm) 
 		{
 			if (in_def) {
 				// If we execute something, we jump out of def block first
@@ -253,17 +253,6 @@ IA_ void compile_and_run_tape(void)
 	vm_rdx = vm_globals[15];
 }
 
-#define MS_WM_CHAR 0x0102
-#define MS_VK_RETURN 0x0D
-#define MS_VK_BACK 0x08
-#define MS_VK_TAB 0x09
-#define MS_VK_SPACE 0x20
-#define MS_VK_F5 0x74
-#define MS_VK_PRIOR 0x21
-#define MS_VK_NEXT 0x22
-
-#define MS_VK_SHIFT 0x10
-
 // --- Window Procedure ---
 S8 win_proc(void* hwnd, U4 msg, U8 wparam, S8 lparam) 
 {

references/Video_Breakdowns.md

@@ -0,0 +1,92 @@
+# In-Depth Chronological Breakdown of Source-Less Programming Reference Videos
+
+This document provides an exhaustive, highly detailed chronological paraphrase of the technical specifics, screen visuals, and mechanical explanations provided by Timothy Lottes and Onat Türkçüoğlu.
+
+---
+
+## 1. "Forth Day 2020 - Preview of x64 & ColorForth & SPIR V" (Onat, 2020)
+
+**0:00 - 3:00 | Introduction & The Editor Visuals**
+Onat introduces his 1-month-old iteration of Forth, inspired by ColorForth. 
+*   **Screen Details:** A custom 3-pane UI rendered in C and Vulkan. Left/center panes show block-based colored tokens; the right pane displays live x64 assembly output that updates instantly as he edits.
+*   The editor treats code blocks as tracked state objects, supporting native undo/redo.
+
+**3:00 - 6:00 | O(1) Dictionary Lookup & Execution Tracing**
+*   To avoid hashing, his compiler allocates an extra 4 bytes per character strictly to store the *source memory location* of the currently compiled word.
+*   **Visual Feature:** "Jump to Definition" and an "Execution Trace" overlay. He demonstrates invoking a command that instantly numbers every occurrence of a word across the codebase in the exact chronological order of execution, providing a "compile-time call graph" without running the program.
+
+**6:00 - 11:00 | The High-Level x64 Macro Assembler & SPIR-V**
+*   **Screen Details:** Syntax like `AX to BX` or `CX + offset`. Toggling a "direction register" macro changes `from AX to BX register, let's move an unsigned` into a 32-bit `mov ebx, eax`. Modifiers like `long` emit 64-bit `mov rbx, rax`.
+*   He uses this same macro-assembler to generate SPIR-V. He notes x64 was actually less complicated than SPIR-V because x64 is a flat instruction stream, whereas SPIR-V requires strict sections, type declarations, and capabilities, forcing him to introduce "sections" into his JIT.
+
+---
+
+## 2. "4th And Beyond" (Timothy Lottes, NeoKineoGfx, 2026)
+
+**0:00 - 8:00 | HP48 Evolution & ColorForth Mechanics**
+*   Lottes advocates removing compilers, linkers, and debuggers. He starts with HP48's RPN as the baseline.
+*   **Screen Details:** He defines a red word `4K` pointing to the next item on the data stack. Typing `1024 4 *` computes `4096`. `4K` acts as a variable.
+*   He defines `DROP` pointing to `add esi, -4` and `ret`. `4K 1 2 + DROP` yields 4096.
+*   He reviews ColorForth: Code compiles onto the data stack. Yellow = Execute, Red = Define, Green = Compile, Magenta = Variable. A Yellow-to-Green transition pops the stack and emits a `push` instruction.
+*   **Screen Details:** Disassembly of Block 24/26 shows `168B 2 , C28B0689 ,`. This pushes bytes onto the stack, disassembling to `mov edx, dword ptr esi` and `mov dword ptr esi, eax` (literally byte-banging machine code).
+
+**8:00 - 20:00 | Branch Misprediction, Folded Interpreter, & x68**
+*   Standard Forth causes 16-clock branch misprediction stalls due to tag branching.
+*   **The Folded Interpreter:** Lottes fixes this by folding a 5-byte interpreter into the end of every word: `LODSD`, lookup, `JMP RAX`. Every transition gets its own branch predictor slot.
+*   **x68 Architecture:** Forces all instructions to 32-bit boundaries. `RET` (`C3`) is padded with three `NOP`s (`90 90 90`). `MOV ESI, imm32` is padded with a `3E` ignored DS prefix.
+*   This makes relative offsets (`CALL`, `JMP`) align perfectly. The editor auto-relinks offsets as tokens are inserted/deleted.
+*   **Assembly Shorthand:** Editor maps `add rcx, qword ptr [rdx + 0x8]` to visual `h + at i08`.
+
+**20:00 - End | Live Execution (SteamOS/Linux)**
+*   Lottes targets a mix of high-level JIT and raw x68 sourceless.
+*   **Cartridge execution:** The binary copies itself to `cart.back`, maps into memory at a fixed address (bypassing ASLR), and provides a zero-fill space. 32-bit tokens act as direct absolute memory pointers, removing lookup overhead.
+
+---
+
+## 3. "Metaprogramming VAMP in KYRA" (Onat, SVFIG, 2025-04-26)
+
+This presentation contains the most explicit, hardcore low-level details regarding Onat's binary-encoded compiler (VAMP).
+
+**0:00 - 10:00 | The Binary Editor, Compilation Speed, & The 2-Item Stack**
+*   VAMP compiles the entire program (Vulkan renderers, UI) in **8.24 milliseconds** on Windows/Linux. His previous text-based Forth took 16-17.8ms just to compile the editor.
+*   **Hardware Locality & The Stack:** Traditional Forth is "runtime opinionated" with a memory data stack, making GPU compute shaders difficult. Onat strictly restricts the stack to two CPU registers: **`RAX` and `RDX`**.
+*   **Screen Details:** The stack state is constantly visualized in the top left corner.
+*   **Magenta Pipe `|`:** There are no `begin` or `end` definition words. A magenta pipe token implicitly signals the end of the previous definition (compiling a `ret`) and starts the new one. Spaces between words imply execution.
+
+**10:00 - 18:00 | Dictionary Management, UX, & Indexing**
+*   **Dictionary Encoding:** Words are stored as 24-bit indices pointing to 8-byte cells, packed with an 8-bit color tag. (He notes the next iteration will use 32-bit indices + a separate 1-byte tag block for faster skipping of empty blocks).
+*   This pure index mapping eliminates hashing and string parsing. It allows IP-protection: you can ship the source indices without the symbols/dictionary. Core language is just 2 to 4 KB.
+*   **Screen Details:** Words are organized explicitly into 16-word horizontal "scrolls" (e.g., "Vulkan API", "FFMPEG", "x64 Assembly"). He presses `Ctrl+Space` to manually redefine a word in a specific scroll.
+*   **Comments:** A comment (Blue tag) is encoded as a string directly inside the 24-bit payload (3 characters).
+
+**18:00 - 28:00 | Data Flow Visualization & Global Memory**
+*   **Free Printf:** Hovering over a word injects code to record `RAX` and `RDX`. Pressing Previous/Next steps through the execution flow visually.
+*   **Global Variables vs. Stacks:** To pass complex state (since the stack only holds two items), he relies entirely on global memory. He explicitly critiques Rust's "safe programming" for forcing developers to pass state through 30 layers of call stacks.
+*   **Single-Register Memory Access:** He dedicates a single CPU register to act as the base pointer for all program memory, giving instant access to "gigabytes of state".
+
+**28:00 - 45:00 | Syntax, Tags, and JIT Assembly Mechanics**
+*   He demonstrates compiling Vulcan commands. Instead of typing `vkGetSwapchainImagesKHR`, he defines a word `get swap chain images` in the `vk device` scroll.
+*   **The `xchg` Trick (`48 92`):** Because the stack is just `RAX` and `RDX`, keeping `RAX` as the Top of Stack is vital. He explicitly notes that `xchg rax, rdx` compiles to just two bytes: `48 92` (REX.W + xchg eax, edx). Before starting a definition or making a call, the JIT emits `48 92` to ensure `RAX` is correctly aligned as the top.
+*   **Color Semantics:**
+    *   **White (Call):** Emits a `CALL` or `JMP RAX` (e.g., `FFE0`).
+    *   **Green (Load):** Emits `mov rax, [global_offset]`.
+    *   **Red (Store):** Emits `mov [global_offset], rax`.
+    *   **Yellow (Immediate/Execute):** Used heavily. For a number, emits `mov rax, imm`. Also used to invoke a lambda block.
+    *   **Blue (Comment):** Ignored.
+    *   **Cyan (Number):** Data literal.
+
+**45:00 - 55:00 | Lambdas `{ }` & Basic Blocks `[ ]`**
+*   He explicitly eliminates `if/else` ASTs.
+*   **Lambdas `{ }`:** Defining a lambda block (Yellow `{`) does not execute it. It compiles the block elsewhere and leaves its executable memory address in `RAX`.
+*   **Basic Blocks `[ ]`:** These define a constrained range of assembly with implicit begin, link, and end jump targets.
+*   **Conditionals in Blocks:** He shows checking `if luma > 0.6`. He explicitly creates a `condition` variable (e.g., `26E`). The `>` operator consumes the values and writes the boolean to `condition`. The conditional word then reads `condition` and consumes the lambda address from `RAX`, emitting a `cmp condition, 0` and `jz lambda_address`.
+
+**55:00 - 1:10:00 | FFI, Stack Pointers, and OS Interop**
+*   **`RSP` Alignment:** The hardware stack pointer (`RSP`) is exclusively used for the call stack, eliminating buffer overflows. When calling OS APIs (like FFMPEG), he explicitly reads `RSP` into a variable to align it to 16 bytes (required by C ABI), makes the call, and restores it.
+*   **Filling Structs:** For `VkImageCreateInfo`, he uses a temporary variable `$` (Dollar sign). He doesn't use C headers. He knows `14` is the Type ID, manually pushing offsets into the contiguous memory space (e.g., `info + offset`).
+
+**1:10:00 - End | SPIR-V, Bug Triage, and Implicit Registers**
+*   **SPIR-V Generation:** VAMP directly emits SPIR-V. He shows the spec (Opcode 194 is Shift Right Logical) and demonstrates a 4-line definition that writes exactly `194` and its operands into a binary vector, replacing a 100MB `glslang` compiler with ~256KB of VAMP code.
+*   **Bug Triage:** He does not use tests or asserts. He triages bugs by commenting out blocks of code (disabling them) and hitting compile (8ms) until the crash stops.
+*   **Implicit Register Passing:** He shows UI hover logic where the `slot ID` is implicitly passed in register `R12D` across functions, completely avoiding pushing it to the data stack.
+*   **Lock Prefix:** Writing concurrent code is handled by the macro assembler. Placing the word `lock` before an `inc` token simply emits the `F0` prefix byte.