LICENSE

@@ -26,19 +26,3 @@ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-Source URL: https://github.com/Ed94/gencpp
-
-Acknowledgements
-
-* The dependencies for gencpp source are derived from the zpl library: https://github.com/zpl-c/zpl
-
-Special thanks to:
-
-* The Handmade Community.
-
-* Casey Muratori, Ginger Bill (Bill Hall), Mr. 4th (Allen Webster), Ryan Fluery: Influnced conceptually how to handle staged metaprograming.
-
-* Jonathan Blow: Jai's metaprogramming influenced the design of this library.
-
-* My friends for putting up with discord spam on this library.

base/Readme.md

@@ -14,7 +14,7 @@ If using the library's provided build scripts:
 .\build.ps1 <compiler> <debug or omit> base
 ```
 
-## Content Overview
+Standard formats:
 
 * **base**: Files are in granular pieces separated into four directories:
   * **dependencies**: Originally from the c-zpl library and modified thereafter.
@@ -123,15 +123,15 @@ The vast majority of macros should be single-line subsitutions that either add:
 
 There are ***five*** header files which are automatically generated using [base_codegen.hpp](./helpers/base_codegen.hpp) by [base.cpp](./base.cpp). They are all located in [components/gen](./components/gen/).
 
-* [ecodetypes.hpp](./components/gen/ecode.hpp): `CodeType` enum definition and related implementaiton. Generation is based off of [ECodeType.csv](./enums/ECodeTypes.csv).
+* [`ecodetypes.hpp`](./components/gen/ecode.hpp): `CodeType` enum definition and related implementaiton. Generation is based off of [`ECodeType.csv](./enums/ECodeTypes.csv).
-* [especifier.hpp](./components/gen/especifier.hpp): `Specifier` enum definition, etc. Generated using [ESpecifier.csv](./enums/ESpecifier.csv).
+* [`especifier.hpp`](./components/gen/especifier.hpp): `Specifier` enum definition, etc. Generated using [`ESpecifier.csv`](./enums/ESpecifier.csv).
-* [eoperator.hpp](./components/gen/eoperator.hpp): `Operator` enum definition, etc. Generated using [EOperator.hpp](./enums/EOperator.csv).
+* [`eoperator.hpp`](./components/gen/eoperator.hpp): `Operator` enum definition, etc. Generated using [`EOperator.hpp`](./enums/EOperator.csv).
-* [etoktype.cpp](./components/gen/etoktype.cpp): `TokType` enum defininition, etc. Used by the lexer and parser backend. Uses two csvs:
+* [`etoktype.cpp`](./components/gen/etoktype.cpp): `TokType` enum defininition, etc. Used by the lexer and parser backend. Uses two csvs:
-  * [ETokType.csv](./enums/ETokType.csv): Provides the enum entries and their strinng ids.
+  * [`ETokType.csv`](./enums/ETokType.csv): Provides the enum entries and their strinng ids.
-  * [AttributeTokens.csv](./enums/AttributeTokens.csv): Provides tokens entries that should be considered as attributes  by the lexer and parser. Sspecfiically macro attributes such as those use for exporting symbols.
+  * [`AttributeTokens.csv`](./enums/AttributeTokens.csv): Provides tokens entries that should be considered as attributes  by the lexer and parser. Sspecfiically macro attributes such as those use for exporting symbols.
-* [ast_inlines.hpp](./components/gen/ast_inlines.hpp): Member trivial `operator` definitions for C++ code types. Does not use a csv.
+* [`ast_inlines.hpp`](./components/gen/ast_inlines.hpp): Member trivial `operator` definitions for C++ code types. Does not use a csv.
 
-[misc.hpp](./helpers/misc.hpp): Has shared functions used by the library generation meta-programs throughout this codebase.
+[`misc.hpp`](./helpers/misc.hpp): Has shared functions used by the library generation meta-programs throughout this codebase.
 
 If using the library's provided build scripts:
 
@@ -160,53 +160,9 @@ Names or Content fields are interned strings and thus showed be cached using `ca
 
 `def_operator` is the most sophisticated upfront constructor as it has multiple permutations of definitions that could be created that are not trivial to determine if valid.
 
-The parser is documented under [`docs/Parsing.md`](../docs/Parsing.md) and [`docs/Parser_Algo.md`](../docs/Parser_Algo.md). Read that and the entire library if you want to extend it.
+The parser is documented under [`docs/Parsing.md`](../docs/Parsing.md) and [`docs/Parser_Algo.md`](../docs/Parser_Algo.md).
-
-### Attributes
-
-To add additional macro attributes, all that has to be done is modifying [`AttributeTokens.csv`](./enums/AttributeTokens.csv).
-
-### Specifiers
-
-To add additional macro specifiers, the following needs to be done:
-
-1. Adjust [especifier.hpp](./components/gen/especifier.hpp)
-2. Adjust [etoktype.cpp](./components/gen/etoktype.cpp)
-3. Adjust [parser_case_macros.cpp](./components/parser_case_macros.cpp)
-
-If the specifier is a new trailing specifier on function definitions:
-
-Head into [base_codegen.hpp](./helpers/base_codegen.hpp): `gen_especifier`. There will be an `is_trailing` function that needs to be adjusted with an additional case for the user's new trailing specifier.
-
-### Code Types
-
-These require the following to be handled to the equivalent extent as the other types:
-
-1. Adjust [ECodeTypes.csv](./enums/ECodeTypes.csv) with the new types
-2. Define a new `AST_<Name>` and `Code<Name>`. See
-    * [ast.hpp](./components/ast.hpp): Initial forwards and user defined conversion for Code.
-    * [ast_types.hpp](./components/ast_types.hpp): Define the `AST_<Name>` struct.
-    * [code_types.hpp](./components/code_types.hpp): Defne the `CodeType` struct. If its needs an iterator see: `struct CodeBody` & `struct CodeParams`.
-3. [ast_case_macros.cpp](./components/ast_case_macros.cpp): Review cases here if the new code type needs to be considered.
-4. [ast.cpp](./components/ast.cpp): Need to review
-    * `code_debug_str`
-    * `code_is_equal`
-    * `code_to_strbuilder_ptr`
-    * `code_validate_body`
-5. [code_serialization.cpp](./components/code_serialization.cpp): Define serialization here.
-6. [inlines.hpp](./components/inlines.hpp): Any inline definitions for the `struct Code<Name>` are defined here.
-7. [interface.cpp](./components/interface.hpp): Define the `Code<Name>` upfront and parsing interface.
-8. [interface.upfront.cpp](./components/interface.upfront.cpp): Define the upfront constructor implementation.
-9. [interface.parsing.cpp](./components/interface.parsing.cpp): Define the parsing interface implementation.
-10. [lexer.cpp](./components/lexer.cpp): Adjust the lexer as needed.
-11. [parser.cpp](./components/parser.cpp): Adjust the parser as needed.
 
 ## A note on compilation and runtime generation speed
 
-The library is designed to be fast to compile and generate code at runtime as fast as possible on a debug build.
+The library is designed to be fast to compile and generate code at runtime as fast as resonable possible on a debug build.
 Its recommended that your metaprogam be compiled using a single translation unit (unity build).
-
-## Whats with the expression / executions support #ifd and enums?
-
-The library is a *work in progress* and those are unfinished hypotheticals for adding the ability to manage or parse the AST of expresions or execution scope code.
-They are entirely untested and not meant to be used yet, futher there is no parsing support or an upfront interface for what CodeTypes are defined so far.

base/base.cpp

@@ -1,5 +1,6 @@
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
+#define GEN_EXPOSE_BACKEND
 #define GEN_C_LIKE_CPP 1
 #include "gen.cpp"
 #include "helpers/push_ignores.inline.hpp"

base/components/ast.hpp

@@ -43,7 +43,7 @@ struct AST_Pragma;
 struct AST_PreprocessCond;
 struct AST_Specifiers;
 
-#ifdef GEN_EXECUTION_EXPRESSION_SUPPORT
+#if GEN_EXECUTION_EXPRESSION_SUPPORT
 struct AST_Expr;
 struct AST_Expr_Assign;
 struct AST_Expr_Alignof;
@@ -140,7 +140,7 @@ struct CodePragma;
 struct CodeSpecifiers;
 #endif
 
-#ifdef GEN_EXECUTION_EXPRESSION_SUPPORT
+#if GEN_EXECUTION_EXPRESSION_SUPPORT
 
 #if GEN_COMPILER_C
 typedef AST_Expr*                CodeExpr;

base/components/ast_types.hpp

@@ -29,16 +29,16 @@
 struct AST_Body
 {
 	union {
-		char  _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
+		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached Name;
+	StrCached      Name;
-	Code      Front;
+	Code              Front;
-	Code      Back;
+	Code              Back;
-	Token*    Tok;
+	Token*            Tok;
-	Code      Parent;
+	Code              Parent;
-	CodeType  Type;
+	CodeType          Type;
-	char      _PAD_UNUSED_[ sizeof(ModuleFlag) ];
+	char              _PAD_UNUSED_[ sizeof(ModuleFlag) ];
-	s32       NumEntries;
+	s32 			  NumEntries;
 };
 static_assert( sizeof(AST_Body) == sizeof(AST), "ERROR: AST_Body is not the same size as AST");
 
@@ -47,7 +47,7 @@ struct AST_Attributes
 {
 	union {
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
-		StrCached     Content;
+		StrCached  Content;
 	};
 	StrCached         Name;
 	Code              Prev;
@@ -65,7 +65,7 @@ struct AST_BaseClass
 	union {
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached         Name;
+	StrCached      Name;
 	Code              Prev;
 	Code              Next;
 	Token*            Tok;
@@ -82,7 +82,7 @@ struct AST_Comment
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 		StrCached  Content;
 	};
-	StrCached         Name;
+	StrCached      Name;
 	Code              Prev;
 	Code              Next;
 	Token*            Tok;
@@ -100,14 +100,14 @@ struct AST_Class
 		{
 			CodeComment     InlineCmt; // Only supported by forward declarations
 			CodeAttributes  Attributes;
-			char            _PAD_SPECS_ [ sizeof(AST*) ];
+			char 	        _PAD_SPECS_ [ sizeof(AST*) ];
 			CodeTypename    ParentType;
-			char            _PAD_PARAMS_[ sizeof(AST*) ];
+			char 	        _PAD_PARAMS_[ sizeof(AST*) ];
 			CodeBody        Body;
-			char            _PAD_PROPERTIES_2_[ sizeof(AST*) ];
+			char 	        _PAD_PROPERTIES_2_[ sizeof(AST*) ];
 		};
 	};
-	StrCached               Name;
+	StrCached            Name;
 	CodeTypename            Prev;
 	CodeTypename            Next;
 	Token*                  Tok;
@@ -130,10 +130,10 @@ struct AST_Constructor
 			Code           InitializerList;
 			CodeParams     Params;
 			Code           Body;
-			char           _PAD_PROPERTIES_2_ [ sizeof(AST*) * 2 ];
+			char 		   _PAD_PROPERTIES_2_ [ sizeof(AST*) * 2 ];
 		};
 	};
-	StrCached         Name;
+	StrCached      Name;
 	Code              Prev;
 	Code              Next;
 	Token*            Tok;
@@ -155,13 +155,13 @@ struct AST_Define
 			char              _PAD_PROPERTIES_2_ [ sizeof(AST*) * 1 ];
 		};
 	};
-	StrCached Name;
+	StrCached         Name;
-	Code      Prev;
+	Code              Prev;
-	Code      Next;
+	Code              Next;
-	Token*    Tok;
+	Token*            Tok;
-	Code      Parent;
+	Code              Parent;
-	CodeType  Type;
+	CodeType          Type;
-	char      _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
 };
 static_assert( sizeof(AST_Define) == sizeof(AST), "ERROR: AST_Define is not the same size as AST");
 
@@ -176,7 +176,7 @@ struct AST_DefineParams
 	Token*            Tok;
 	Code              Parent;
 	CodeType          Type;
-	char              _PAD_UNUSED_[ sizeof(ModuleFlag) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) ];
 	s32               NumEntries;
 };
 static_assert( sizeof(AST_DefineParams) == sizeof(AST), "ERROR: AST_DefineParams is not the same size as AST");
@@ -192,10 +192,10 @@ struct AST_Destructor
 			CodeSpecifiers Specs;
 			char           _PAD_PROPERTIES_2_ [ sizeof(AST*) * 2 ];
 			Code           Body;
-			char           _PAD_PROPERTIES_3_ [ sizeof(AST*) ];
+			char 		   _PAD_PROPERTIES_3_ [ sizeof(AST*) ];
 		};
 	};
-	StrCached              Name;
+	StrCached           Name;
 	Code                   Prev;
 	Code                   Next;
 	Token*                 Tok;
@@ -217,10 +217,10 @@ struct AST_Enum
 			CodeTypename   UnderlyingType;
 			Code           UnderlyingTypeMacro;
 			CodeBody       Body;
-			char           _PAD_PROPERTIES_2_[ sizeof(AST*) ];
+			char 	       _PAD_PROPERTIES_2_[ sizeof(AST*) ];
 		};
 	};
-	StrCached              Name;
+	StrCached           Name;
 	Code                   Prev;
 	Code                   Next;
 	Token*                 Tok;
@@ -237,7 +237,7 @@ struct AST_Exec
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 		StrCached  Content;
 	};
-	StrCached         Name;
+	StrCached      Name;
 	Code              Prev;
 	Code              Next;
 	Token*            Tok;
@@ -247,7 +247,7 @@ struct AST_Exec
 };
 static_assert( sizeof(AST_Exec) == sizeof(AST), "ERROR: AST_Exec is not the same size as AST");
 
-#ifdef GEN_EXECUTION_EXPRESSION_SUPPORT
+#if GEN_EXECUTION_EXPRESSION_SUPPORT
 struct AST_Expr
 {
 	union {
@@ -515,13 +515,13 @@ struct AST_Extern
 			char      _PAD_PROPERTIES_2_[ sizeof(AST*) ];
 		};
 	};
-	StrCached         Name;
+	StrCached      Name;
 	Code              Prev;
 	Code              Next;
 	Token*            Tok;
 	Code              Parent;
 	CodeType          Type;
-	char              _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
 };
 static_assert( sizeof(AST_Extern) == sizeof(AST), "ERROR: AST_Extern is not the same size as AST");
 
@@ -531,13 +531,13 @@ struct AST_Include
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 		StrCached  Content;
 	};
-	StrCached Name;
+	StrCached      Name;
-	Code      Prev;
+	Code              Prev;
-	Code      Next;
+	Code              Next;
-	Token*    Tok;
+	Token*            Tok;
-	Code      Parent;
+	Code              Parent;
-	CodeType  Type;
+	CodeType          Type;
-	char      _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
 };
 static_assert( sizeof(AST_Include) == sizeof(AST), "ERROR: AST_Include is not the same size as AST");
 
@@ -550,16 +550,16 @@ struct AST_Friend
 			CodeComment InlineCmt;
 			char        _PAD_PROPERTIES_[ sizeof(AST*) * 4 ];
 			Code        Declaration;
-			char        _PAD_PROPERTIES_2_[ sizeof(AST*) ];
+			char 	    _PAD_PROPERTIES_2_[ sizeof(AST*) ];
 		};
 	};
-	StrCached Name;
+	StrCached      Name;
-	Code      Prev;
+	Code              Prev;
-	Code      Next;
+	Code              Next;
-	Token*    Tok;
+	Token*            Tok;
-	Code      Parent;
+	Code              Parent;
-	CodeType  Type;
+	CodeType          Type;
-	char      _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
 };
 static_assert( sizeof(AST_Friend) == sizeof(AST), "ERROR: AST_Friend is not the same size as AST");
 
@@ -573,19 +573,19 @@ struct AST_Fn
 			CodeAttributes  Attributes;
 			CodeSpecifiers  Specs;
 			CodeTypename    ReturnType;
-			CodeParams      Params;
+			CodeParams 	    Params;
 			CodeBody        Body;
-			char            _PAD_PROPERTIES_ [ sizeof(AST*) ];
+			char 	        _PAD_PROPERTIES_ [ sizeof(AST*) ];
 		};
 	};
-	StrCached  Name;
+	StrCached            Name;
-	Code       Prev;
+	Code                    Prev;
-	Code       Next;
+	Code                    Next;
-	Token*     Tok;
+	Token*                  Tok;
-	Code       Parent;
+	Code                    Parent;
-	CodeType   Type;
+	CodeType                Type;
-	ModuleFlag ModuleFlags;
+	ModuleFlag              ModuleFlags;
-	char       _PAD_UNUSED_[ sizeof(u32) ];
+	char 			        _PAD_UNUSED_[ sizeof(u32) ];
 };
 static_assert( sizeof(AST_Fn) == sizeof(AST), "ERROR: AST_Fn is not the same size as AST");
 
@@ -594,14 +594,14 @@ struct AST_Module
 	union {
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached         Name;
+	StrCached      Name;
 	Code              Prev;
 	Code              Next;
 	Token*            Tok;
 	Code              Parent;
 	CodeType          Type;
 	ModuleFlag        ModuleFlags;
-	char             _PAD_UNUSED_[ sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(u32) ];
 };
 static_assert( sizeof(AST_Module) == sizeof(AST), "ERROR: AST_Module is not the same size as AST");
 
@@ -610,45 +610,45 @@ struct AST_NS
 	union {
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 		struct {
-			char      _PAD_PROPERTIES_[ sizeof(AST*) * 5 ];
+			char 	  _PAD_PROPERTIES_[ sizeof(AST*) * 5 ];
 			CodeBody  Body;
-			char      _PAD_PROPERTIES_2_[ sizeof(AST*) ];
+			char 	  _PAD_PROPERTIES_2_[ sizeof(AST*) ];
 		};
 	};
-	StrCached  Name;
+	StrCached      Name;
-	Code       Prev;
+	Code              Prev;
-	Code       Next;
+	Code              Next;
-	Token*     Tok;
+	Token*            Tok;
-	Code       Parent;
+	Code              Parent;
-	CodeType   Type;
+	CodeType          Type;
-	ModuleFlag ModuleFlags;
+	ModuleFlag        ModuleFlags;
-	char       _PAD_UNUSED_[ sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(u32) ];
 };
 static_assert( sizeof(AST_NS) == sizeof(AST), "ERROR: AST_NS is not the same size as AST");
 
 struct AST_Operator
 {
 	union {
-		char               _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
+		char                _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 		struct
 		{
-			CodeComment    InlineCmt;
+			CodeComment     InlineCmt;
-			CodeAttributes Attributes;
+			CodeAttributes  Attributes;
-			CodeSpecifiers Specs;
+			CodeSpecifiers  Specs;
-			CodeTypename   ReturnType;
+			CodeTypename    ReturnType;
-			CodeParams 	   Params;
+			CodeParams 	    Params;
-			CodeBody       Body;
+			CodeBody        Body;
-			char           _PAD_PROPERTIES_ [ sizeof(AST*) ];
+			char 	        _PAD_PROPERTIES_ [ sizeof(AST*) ];
 		};
 	};
-	StrCached  Name;
+	StrCached   Name;
-	Code       Prev;
+	Code           Prev;
-	Code       Next;
+	Code           Next;
-	Token*     Tok;
+	Token*         Tok;
-	Code       Parent;
+	Code           Parent;
-	CodeType   Type;
+	CodeType       Type;
-	ModuleFlag ModuleFlags;
+	ModuleFlag     ModuleFlags;
-	Operator   Op;
+	Operator       Op;
 };
 static_assert( sizeof(AST_Operator) == sizeof(AST), "ERROR: AST_Operator is not the same size as AST");
 
@@ -658,22 +658,22 @@ struct AST_OpCast
 		char               _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 		struct
 		{
-			CodeComment    InlineCmt;
+			CodeComment     InlineCmt;
-			char           _PAD_PROPERTIES_[ sizeof(AST*)  ];
+			char 	        _PAD_PROPERTIES_[ sizeof(AST*)  ];
-			CodeSpecifiers Specs;
+			CodeSpecifiers  Specs;
-			CodeTypename   ValueType;
+			CodeTypename    ValueType;
-			char           _PAD_PROPERTIES_2_[ sizeof(AST*) ];
+			char 	        _PAD_PROPERTIES_2_[ sizeof(AST*) ];
-			CodeBody       Body;
+			CodeBody        Body;
-			char           _PAD_PROPERTIES_3_[ sizeof(AST*) ];
+			char 	        _PAD_PROPERTIES_3_[ sizeof(AST*) ];
 		};
 	};
-	StrCached Name;
+	StrCached      Name;
-	Code      Prev;
+	Code              Prev;
-	Code      Next;
+	Code              Next;
-	Token*    Tok;
+	Token*            Tok;
-	Code      Parent;
+	Code              Parent;
-	CodeType  Type;
+	CodeType          Type;
-	char      _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
 };
 static_assert( sizeof(AST_OpCast) == sizeof(AST), "ERROR: AST_OpCast is not the same size as AST");
 
@@ -684,7 +684,7 @@ struct AST_Params
 		struct
 		{
 			// TODO(Ed): Support attributes for parameters (Some prefix macros can be converted to that...)
-			char         _PAD_PROPERTIES_2_[ sizeof(AST*) * 3 ];
+			char 	     _PAD_PROPERTIES_2_[ sizeof(AST*) * 3 ];
 			CodeTypename ValueType;
 			Code         Macro;
 			Code         Value;
@@ -692,14 +692,14 @@ struct AST_Params
 			// char     _PAD_PROPERTIES_3_[sizeof( AST* )];
 		};
 	};
-	StrCached  Name;
+	StrCached         Name;
-	CodeParams Last;
+	CodeParams        Last;
-	CodeParams Next;
+	CodeParams        Next;
-	Token*     Tok;
+	Token*            Tok;
-	Code       Parent;
+	Code              Parent;
-	CodeType   Type;
+	CodeType          Type;
-	char       _PAD_UNUSED_[ sizeof(ModuleFlag) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) ];
-	s32        NumEntries;
+	s32               NumEntries;
 };
 static_assert( sizeof(AST_Params) == sizeof(AST), "ERROR: AST_Params is not the same size as AST");
 
@@ -709,13 +709,13 @@ struct AST_Pragma
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 		StrCached  Content;
 	};
-	StrCached Name;
+	StrCached      Name;
-	Code      Prev;
+	Code              Prev;
-	Code      Next;
+	Code              Next;
-	Token*    Tok;
+	Token*            Tok;
-	Code      Parent;
+	Code              Parent;
-	CodeType  Type;
+	CodeType          Type;
-	char      _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
 };
 static_assert( sizeof(AST_Pragma) == sizeof(AST), "ERROR: AST_Pragma is not the same size as AST");
 
@@ -723,40 +723,40 @@ struct AST_PreprocessCond
 {
 	union {
 		char          _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
-		StrCached     Content;
+		StrCached  Content;
 	};
-	StrCached Name;
+	StrCached      Name;
-	Code      Prev;
+	Code              Prev;
-	Code      Next;
+	Code              Next;
-	Token*    Tok;
+	Token*            Tok;
-	Code      Parent;
+	Code              Parent;
-	CodeType  Type;
+	CodeType          Type;
-	char      _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
 };
 static_assert( sizeof(AST_PreprocessCond) == sizeof(AST), "ERROR: AST_PreprocessCond is not the same size as AST");
 
 struct AST_Specifiers
 {
-	Specifier      ArrSpecs[ AST_ArrSpecs_Cap ];
+	Specifier         ArrSpecs[ AST_ArrSpecs_Cap ];
 	StrCached      Name;
-	CodeSpecifiers NextSpecs;
+	CodeSpecifiers    NextSpecs;
-	Code           Prev;
+	Code              Prev;
-	Code           Next;
+	Code              Next;
-	Token*         Tok;
+	Token*            Tok;
-	Code           Parent;
+	Code              Parent;
-	CodeType       Type;
+	CodeType          Type;
-	char           _PAD_UNUSED_[ sizeof(ModuleFlag) ];
+	char 			  _PAD_UNUSED_[ sizeof(ModuleFlag) ];
-	s32            NumEntries;
+	s32               NumEntries;
 };
 static_assert( sizeof(AST_Specifiers) == sizeof(AST), "ERROR: AST_Specifier is not the same size as AST");
 
-#ifdef GEN_EXECUTION_EXPRESSION_SUPPORT
+#if GEN_EXECUTION_EXPRESSION_SUPPORT
 struct AST_Stmt
 {
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -771,7 +771,7 @@ struct AST_Stmt_Break
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -786,7 +786,7 @@ struct AST_Stmt_Case
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -801,7 +801,7 @@ struct AST_Stmt_Continue
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -816,7 +816,7 @@ struct AST_Stmt_Decl
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -831,7 +831,7 @@ struct AST_Stmt_Do
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -846,7 +846,7 @@ struct AST_Stmt_Expr
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -861,7 +861,7 @@ struct AST_Stmt_Else
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -876,7 +876,7 @@ struct AST_Stmt_If
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -891,7 +891,7 @@ struct AST_Stmt_For
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -906,7 +906,7 @@ struct AST_Stmt_Goto
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -921,7 +921,7 @@ struct AST_Stmt_Label
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -936,7 +936,7 @@ struct AST_Stmt_Switch
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -951,7 +951,7 @@ struct AST_Stmt_While
 	union {
 		char _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 	};
-	StrCached      Name;
+	StrCached   Name;
 	CodeExpr       Prev;
 	CodeExpr       Next;
 	Token*         Tok;
@@ -970,14 +970,14 @@ struct AST_Struct
 		{
 			CodeComment    InlineCmt;
 			CodeAttributes Attributes;
-			char           _PAD_SPECS_ [ sizeof(AST*) ];
+			char 	       _PAD_SPECS_ [ sizeof(AST*) ];
 			CodeTypename   ParentType;
-			char           _PAD_PARAMS_[ sizeof(AST*) ];
+			char 	       _PAD_PARAMS_[ sizeof(AST*) ];
 			CodeBody       Body;
-			char          _PAD_PROPERTIES_2_[ sizeof(AST*) ];
+			char 	       _PAD_PROPERTIES_2_[ sizeof(AST*) ];
 		};
 	};
-	StrCached              Name;
+	StrCached           Name;
 	CodeTypename           Prev;
 	CodeTypename           Next;
 	Token*                 Tok;
@@ -994,20 +994,20 @@ struct AST_Template
 		char               _PAD_[ sizeof(Specifier) * AST_ArrSpecs_Cap + sizeof(AST*) ];
 		struct
 		{
-			char           _PAD_PROPERTIES_[ sizeof(AST*) * 4 ];
+			char 	       _PAD_PROPERTIES_[ sizeof(AST*) * 4 ];
 			CodeParams 	   Params;
 			Code           Declaration;
-			char           _PAD_PROPERTIES_2_[ sizeof(AST*) ];
+			char 	       _PAD_PROPERTIES_2_[ sizeof(AST*) ];
 		};
 	};
-	StrCached  Name;
+	StrCached           Name;
-	Code       Prev;
+	Code                   Prev;
-	Code       Next;
+	Code                   Next;
-	Token*     Tok;
+	Token*                 Tok;
-	Code       Parent;
+	Code                   Parent;
-	CodeType   Type;
+	CodeType               Type;
-	ModuleFlag ModuleFlags;
+	ModuleFlag             ModuleFlags;
-	char       _PAD_UNUSED_[ sizeof(u32) ];
+	char 			       _PAD_UNUSED_[ sizeof(u32) ];
 };
 static_assert( sizeof(AST_Template) == sizeof(AST), "ERROR: AST_Template is not the same size as AST");
 
@@ -1029,13 +1029,13 @@ struct AST_Type
 			// CodeSpecifiers SpecsFuncSuffix; // Only used for function signatures
 		};
 	};
-	StrCached              Name;
+	StrCached           Name;
 	Code                   Prev;
 	Code                   Next;
-	Token*                 Tok;
+	Token*                  Tok;
 	Code                   Parent;
 	CodeType               Type;
-	char                   _PAD_UNUSED_[ sizeof(ModuleFlag) ];
+	char 			       _PAD_UNUSED_[ sizeof(ModuleFlag) ];
 	b32                    IsParamPack;
 };
 static_assert( sizeof(AST_Type) == sizeof(AST), "ERROR: AST_Type is not the same size as AST");
@@ -1062,7 +1062,7 @@ struct AST_Typename
 	Token*                 Tok;
 	Code                   Parent;
 	CodeType               Type;
-	char                   _PAD_UNUSED_[ sizeof(ModuleFlag) ];
+	char 			       _PAD_UNUSED_[ sizeof(ModuleFlag) ];
 	struct {
 		b16                IsParamPack;   // Used by typename to know if type should be considered a parameter pack.
 		ETypenameTag       TypeTag;       // Used by typename to keep track of explicitly declared tags for the identifier (enum, struct, union)
@@ -1077,9 +1077,9 @@ struct AST_Typedef
 		struct
 		{
 			CodeComment    InlineCmt;
-			char           _PAD_PROPERTIES_[ sizeof(AST*) * 2 ];
+			char 	       _PAD_PROPERTIES_[ sizeof(AST*) * 2 ];
 			Code           UnderlyingType;
-			char           _PAD_PROPERTIES_2_[ sizeof(AST*) * 3 ];
+			char 	       _PAD_PROPERTIES_2_[ sizeof(AST*) * 3 ];
 		};
 	};
 	StrCached           Name;
@@ -1101,19 +1101,19 @@ struct AST_Union
 		{
 			char           _PAD_INLINE_CMT_[ sizeof(AST*) ];
 			CodeAttributes Attributes;
-			char           _PAD_PROPERTIES_[ sizeof(AST*) * 3 ];
+			char 	       _PAD_PROPERTIES_[ sizeof(AST*) * 3 ];
 			CodeBody       Body;
-			char           _PAD_PROPERTIES_2_[ sizeof(AST*) ];
+			char 	       _PAD_PROPERTIES_2_[ sizeof(AST*) ];
 		};
 	};
-	StrCached  Name;
+	StrCached           Name;
-	Code       Prev;
+	Code                   Prev;
-	Code       Next;
+	Code                   Next;
-	Token*     Tok;
+	Token*                 Tok;
-	Code       Parent;
+	Code                   Parent;
-	CodeType   Type;
+	CodeType               Type;
-	ModuleFlag ModuleFlags;
+	ModuleFlag             ModuleFlags;
-	char       _PAD_UNUSED_[ sizeof(u32) ];
+	char 			       _PAD_UNUSED_[ sizeof(u32) ];
 };
 static_assert( sizeof(AST_Union) == sizeof(AST), "ERROR: AST_Union is not the same size as AST");
 
@@ -1125,19 +1125,19 @@ struct AST_Using
 		{
 			CodeComment     InlineCmt;
 			CodeAttributes  Attributes;
-			char            _PAD_SPECS_     [ sizeof(AST*) ];
+			char 	        _PAD_SPECS_     [ sizeof(AST*) ];
 			CodeTypename    UnderlyingType;
-			char            _PAD_PROPERTIES_[ sizeof(AST*) * 3 ];
+			char 	        _PAD_PROPERTIES_[ sizeof(AST*) * 3 ];
 		};
 	};
-	StrCached  Name;
+	StrCached            Name;
-	Code       Prev;
+	Code                    Prev;
-	Code       Next;
+	Code                    Next;
-	Token*     Tok;
+	Token*                  Tok;
-	Code       Parent;
+	Code                    Parent;
-	CodeType   Type;
+	CodeType                Type;
-	ModuleFlag ModuleFlags;
+	ModuleFlag              ModuleFlags;
-	char       _PAD_UNUSED_[ sizeof(u32) ];
+	char 			        _PAD_UNUSED_[ sizeof(u32) ];
 };
 static_assert( sizeof(AST_Using) == sizeof(AST), "ERROR: AST_Using is not the same size as AST");
 
@@ -1153,17 +1153,17 @@ struct AST_Var
 			CodeTypename   ValueType;
 			Code           BitfieldSize;
 			Code           Value;
-			CodeVar        NextVar;
+			CodeVar		   NextVar;
 		};
 	};
-	StrCached  Name;
+	StrCached           Name;
-	Code       Prev;
+	Code                   Prev;
-	Code       Next;
+	Code                   Next;
-	Token*     Tok;
+	Token*                 Tok;
-	Code       Parent;
+	Code                   Parent;
-	CodeType   Type;
+	CodeType               Type;
-	ModuleFlag ModuleFlags;
+	ModuleFlag             ModuleFlags;
-	s32        VarParenthesizedInit;
+	s32                    VarParenthesizedInit;
 };
 static_assert( sizeof(AST_Var) == sizeof(AST), "ERROR: AST_Var is not the same size as AST");
 

base/components/code_types.hpp

@@ -362,7 +362,7 @@ struct CodeExec
 	AST_Exec *ast;
 };
 
-#ifdef GEN_EXECUTION_EXPRESSION_SUPPORT
+#if GEN_EXECUTION_EXPRESSION_SUPPORT
 struct CodeExpr
 {
 #if ! GEN_C_LIKE_CPP
@@ -689,7 +689,7 @@ struct CodePreprocessCond
 	AST_PreprocessCond* ast;
 };
 
-#ifdef GEN_EXECUTION_EXPRESSION_SUPPORT
+#if GEN_EXECUTION_EXPRESSION_SUPPORT
 struct CodeStmt
 {
 #if ! GEN_C_LIKE_CPP

base/components/gen/especifier.hpp

@@ -72,18 +72,7 @@ inline Str spec_to_str( Specifier type )
 
 inline bool spec_is_trailing( Specifier specifier )
 {
-	switch (specifier) {
+	return specifier > Spec_Virtual;
-		case Spec_Const:
-		case Spec_Final:
-		case Spec_NoExceptions:
-		case Spec_Override:
-		case Spec_Pure:
-		case Spec_Volatile:
-			return true;
-
-		default:
-			return false;
-	}
 }
 
 inline Specifier str_to_specifier( Str str )

base/components/parser.cpp

@@ -987,7 +987,7 @@ CodeBody parse_class_struct_body( TokType which, Token name )
 				// <Attributes>
 			}
 			//! Fallthrough intended
-			GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_TOK_SPECIFIER_CASES:
+			GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_TOK_SPECIFIERS_CASES:
 			{
 				Specifier specs_found[16] = { Spec_NumSpecifiers };
 				s32        NumSpecifiers = 0;
@@ -1000,7 +1000,7 @@ CodeBody parse_class_struct_body( TokType which, Token name )
 
 					switch ( spec )
 					{
-						GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_SPECIFIER_CASES:
+						GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_SPECIFIERS_CASES:
 						break;
 
 						case Spec_Consteval:
@@ -3993,7 +3993,7 @@ CodeFriend parser_parse_friend()
 
 			switch ( spec )
 			{
-				GEN_PARSER_FRIEND_ALLOWED_SPECIFIER_CASES:
+				GEN_PARSER_FRIEND_ALLOWED_SPECIFIERS_CASES:
 				break;
 
 				default :
@@ -4111,7 +4111,7 @@ CodeFn parser_parse_function()
 
 		switch ( spec )
 		{
-			GEN_PARSER_FUNCTION_ALLOWED_SPECIFIER_CASES:
+			GEN_PARSER_FUNCTION_ALLOWED_SPECIFIERS_CASES:
 			break;
 
 			default:
@@ -4133,6 +4133,13 @@ CodeFn parser_parse_function()
 	}
 	// <export> <Attributes> <Specifiers>
 
+	// Note(Ed): We're enforcing that using this codepath requires non-macro jank.
+	// Code macro_stmt = parse_macro_as_definiton(attributes, specifiers);
+	// if (macro_stmt) {
+	// 	parser_pop(& _ctx->parser);
+	// 	return macro_stmt;
+	// }
+
 	CodeTypename ret_type = parser_parse_type(parser_not_from_template, nullptr);
 	if ( cast(Code, ret_type) == Code_Invalid ) {
 		parser_pop(& _ctx->parser);
@@ -4212,7 +4219,7 @@ CodeOperator parser_parse_operator()
 
 		switch ( spec )
 		{
-			GEN_PARSER_OPERATOR_ALLOWED_SPECIFIER_CASES:
+			GEN_PARSER_OPERATOR_ALLOWED_SPECIFIERS_CASES:
 			break;
 
 			default:
@@ -4445,7 +4452,7 @@ CodeTemplate parser_parse_template()
 
 				switch ( spec )
 				{
-					GEN_PARSER_TEMPLATE_ALLOWED_SPECIFIER_CASES:
+					GEN_PARSER_TEMPLATE_ALLOWED_SPECIFIERS_CASES:
 					break;
 
 					case Spec_Consteval :

base/components/parser_case_macros.cpp

@@ -1,6 +1,6 @@
 // These macros are used in the swtich cases within parser.cpp
 
-#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_TOK_SPECIFIER_CASES \
+#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_TOK_SPECIFIERS_CASES \
 case Tok_Spec_Consteval:   \
 case Tok_Spec_Constexpr:   \
 case Tok_Spec_Constinit:   \
@@ -13,7 +13,7 @@ case Tok_Spec_Static:      \
 case Tok_Spec_Volatile:    \
 case Tok_Spec_Virtual
 
-#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_SPECIFIER_CASES \
+#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_SPECIFIERS_CASES \
 case Spec_Constexpr:   \
 case Spec_Constinit:   \
 case Spec_Explicit:    \
@@ -50,12 +50,12 @@ case Spec_NeverInline:      \
 case Spec_Static:           \
 case Spec_Volatile
 
-#define GEN_PARSER_FRIEND_ALLOWED_SPECIFIER_CASES \
+#define GEN_PARSER_FRIEND_ALLOWED_SPECIFIERS_CASES \
 case Spec_Const:       \
 case Spec_Inline:      \
 case Spec_ForceInline
 
-#define GEN_PARSER_FUNCTION_ALLOWED_SPECIFIER_CASES \
+#define GEN_PARSER_FUNCTION_ALLOWED_SPECIFIERS_CASES \
 case Spec_Const:            \
 case Spec_Consteval:        \
 case Spec_Constexpr:        \
@@ -66,7 +66,7 @@ case Spec_Inline:           \
 case Spec_NeverInline:      \
 case Spec_Static
 
-#define GEN_PARSER_OPERATOR_ALLOWED_SPECIFIER_CASES \
+#define GEN_PARSER_OPERATOR_ALLOWED_SPECIFIERS_CASES \
 case Spec_Const:       \
 case Spec_Constexpr:   \
 case Spec_ForceInline: \
@@ -74,7 +74,7 @@ case Spec_Inline:      \
 case Spec_NeverInline: \
 case Spec_Static
 
-#define GEN_PARSER_TEMPLATE_ALLOWED_SPECIFIER_CASES \
+#define GEN_PARSER_TEMPLATE_ALLOWED_SPECIFIERS_CASES \
 case Spec_Const:            \
 case Spec_Constexpr:        \
 case Spec_Constinit:        \

base/components/parser_types.hpp

@@ -132,6 +132,8 @@ enum MacroType : u16
 	MT_Expression,     // A macro is assumed to be a expression if not resolved.
 	MT_Statement,      
 	MT_Typename,
+	MT_Attribute,      // More of a note to the parser than anythign else (attributes should be defined in the user attribues def).
+	MT_Specifier,      // More of a note to the parser than anythign else (specifiers should be defined in the user attribues def).
 	MT_Block_Start,    // Not Supported yet
 	MT_Block_End,      // Not Supported yet
 	MT_Case_Statement, // Not Supported yet
@@ -158,6 +160,8 @@ Str macrotype_to_str( MacroType type )
 		{ "Statement",        sizeof("Statement")        - 1 },
 		{ "Expression",       sizeof("Expression")       - 1 },
 		{ "Typename",         sizeof("Typename")         - 1 },
+		{ "Attribute(Macro)", sizeof("Attribute(Macro)") - 1 },
+		{ "Specifier(Macro)", sizeof("Specifier(Macro)") - 1 },
 		{ "Block_Start",      sizeof("Block_Start")      - 1 },
 		{ "Block_End",        sizeof("Block_End")        - 1 },
 		{ "Case_Statement",   sizeof("Case_Statement")   - 1 },

base/helpers/base_codegen.hpp

@@ -214,18 +214,7 @@ CodeBody gen_especifier( char const* path, bool use_c_definition = false )
 		inline
 		bool spec_is_trailing( Specifier specifier )
 		{
-			switch (specifier) {
+			return specifier > Spec_Virtual;
-				case Spec_Const:
-				case Spec_Final:
-				case Spec_NoExceptions:
-				case Spec_Override:
-				case Spec_Pure:
-				case Spec_Volatile:
-					return true;
-		
-				default:
-					return false;
-			}
 		}
 	)));
 

docs/AST_Design.md

@@ -53,7 +53,6 @@ StrBuilder <prefix>_to_strbuilder(Code code);
 
 Where the first generates strings allocated using Allocator_StringArena and the other appends an existing strings with their backed allocator.
 
-Serialization of for the AST is defined for `Code` in [`ast.cpp`](../base/components/ast.cpp) with `code_to_strbuilder_ptr` & `code_to_strbuilder`.  
+Serialization of for the AST is defined for `Code` in [`ast.chpp`](../base/components/ast.cpp) with `code_to_strbuilder_ptr` & `code_to_strbuilder`.  
 Serializtion for the rest of the code types is within [`code_serialization.cpp`](../base/components/code_serialization.cpp).  
-
 Gencpp's serialization does not provide coherent formatting of the code. The user should use a formatter after serializing.

docs/AST_Types.md

@@ -25,13 +25,13 @@ These are containers representing a scope body of a definition that can be of th
 Fields:
 
 ```cpp
-StrCached Name;
+StrCached   Name;
-Code      Front;
+Code           Front;
-Code      Back;
+Code           Back;
-Token*    Tok;
+Token*         Tok;
-Code      Parent;
+Code           Parent;
-CodeType  Type;
+CodeT          Type;
-s32       NumEntries;
+s32            NumEntries;
 ```
 
 The `Front` member represents the start of the link list and `Back` the end.
@@ -56,13 +56,13 @@ Represent standard or vendor specific C/C++ attributes.
 Fields:
 
 ```cpp
-StrCached Content;
+StrCached   Content;
-StrCached Name;
+StrCached   Name;
-Code      Prev;
+Code           Prev;
-Code      Next;
+Code           Next;
-Token*    Tok;
+Token*         Tok;
-Code      Parent;
+Code           Parent;
-CodeType  Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -80,13 +80,13 @@ Stores a comment.
 Fields:
 
 ```cpp
-StrCached Content;
+StrCached   Content;
-StrCached Name;
+StrCached   Name;
-Code      Prev;
+Code           Prev;
-Code      Next;
+Code           Next;
-Token*    Tok;
+Token*         Tok;
-Code      Parent;
+Code           Parent;
-CodeType  Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -109,12 +109,12 @@ CodeComment    InlineCmt;  // Only supported by forward declarations
 CodeAttributes Attributes;
 CodeType       ParentType;
 CodeBody       Body;
-StrCached      Name;
+StrCached   Name;
 CodeType       Prev;
 CodeType       Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 ModuleFlag     ModuleFlags;
 AccessSpec     ParentAccess;
 ```
@@ -139,16 +139,16 @@ You'll notice that only one parent type is supported only with parent access. Th
 Fields:
 
 ```cpp
-CodeComment InlineCmt;  // Only supported by forward declarations
+CodeComment    InlineCmt;  // Only supported by forward declarations
-Code        InitializerList;
+Code           InitializerList;
-CodeParams  Params;
+CodeParams     Params;
-Code        Body;
+Code           Body;
 StrCached   Name;
-Code        Prev;
+Code           Prev;
-Code        Next;
+Code           Next;
-Token*      Tok;
+Token*         Tok;
-Code        Parent;
+Code           Parent;
-CodeType    Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -178,14 +178,13 @@ Represents a preprocessor define
 Fields:
 
 ```cpp
-CodeDefineParams Params;
+StrCached   Content;
-Code             Body;
+StrCached   Name;
-StrCached        Name;
+Code           Prev;
-Code             Prev;
+Code           Next;
-Code             Next;
+Token*         Tok;
-Token*           Tok;
+Code           Parent;
-Code             Parent;
+CodeT          Type;
-CodeType         Type;
 ```
 
 Serialization:
@@ -194,28 +193,6 @@ Serialization:
 #define <Name> <Content>
 ```
 
-## DefineParams
-
-Preprocessor define's parameters.
-
-Fields:
-
-```cpp
-StrCached Name;
-Code      Last;
-Code      Next;
-Token*    Tok;
-Code      Parent;
-CodeType  Type;
-s32       NumEntries;
-```
-
-Serialization:
-
-```cpp
-<Name>, <Next> ...
-```
-
 ## Destructor
 
 Fields:
@@ -224,12 +201,12 @@ Fields:
 CodeComment    InlineCmt;
 CodeSpecifiers Specs;
 Code           Body;
-StrCached      Name;
+StrCached   Name;
 Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -265,8 +242,8 @@ Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-StrCached      Name;
+StrCached   Name;
-CodeType       Type;
+CodeT          Type;
 ModuleFlag     ModuleFlags;
 ```
 
@@ -294,13 +271,13 @@ Will be obsolute when function body parsing is implemented.
 Fields:
 
 ```cpp
-StrCached Content;
+StrCached   Content;
-StrCached Name;
+StrCached   Name;
-Code      Prev;
+Code           Prev;
-Code      Next;
+Code           Next;
-Token*    Tok;
+Token*         Tok;
-Code      Parent;
+Code           Parent;
-CodeType  Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -309,18 +286,18 @@ Serialization:
 <Content>
 ```
 
-## External Linkage (Extern)
+## External Linkage
 
 Fields:
 
 ```cpp
-CodeBody  Body;
+CodeBody       Body;
-StrCached Name;
+StrCached   Name;
-Code      Prev;
+Code           Prev;
-Code      Next;
+Code           Next;
-Token*    Tok;
+Token*         Tok;
-Code      Parent;
+Code           Parent;
-CodeType  Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -337,13 +314,13 @@ extern "<Name>"
 Fields:
 
 ```cpp
-StrCached Content;
+StrCached   Content;
-StrCached Name;
+StrCached   Name;
-Code      Prev;
+Code           Prev;
-Code      Next;
+Code           Next;
-Code      Parent;
+Code           Parent;
-Token*    Tok;
+Token*         Tok;
-CodeType  Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -359,14 +336,14 @@ This library (until its necessary become some third-party library to do otherwis
 Fields:
 
 ```cpp
-CodeComment InlineCmt;
+CodeComment    InlineCmt;
-Code        Declaration;
+Code           Declaration;
 StrCached   Name;
-Code        Prev;
+Code           Prev;
-Code        Next;
+Code           Next;
-Token*      Tok;
+Token*         Tok;
-Code        Parent;
+Code           Parent;
-CodeType    Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -386,12 +363,12 @@ CodeSpecifiers Specs;
 CodeType       ReturnType;
 CodeParams     Params;
 CodeBody       Body;
-StrCached      Name;
+StrCached   Name;
 Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 ModuleFlag     ModuleFlags;
 ```
 
@@ -413,13 +390,13 @@ Serialization:
 Fields:
 
 ```cpp
-StrCached  Name;
+StrCached   Name;
-Code       Prev;
+Code           Prev;
-Code       Next;
+Code           Next;
-Token*     Tok;
+Token*         Tok;
-Code       Parent;
+Code           Parent;
-CodeType   Type;
+CodeT          Type;
-ModuleFlag ModuleFlags;
+ModuleFlag     ModuleFlags;
 ```
 
 Serialization:
@@ -433,14 +410,14 @@ Serialization:
 Fields:
 
 ```cpp
-CodeBody   Body;
+CodeBody       Body;
-StrCached  Name;
+StrCached   Name;
-Code       Prev;
+Code           Prev;
-Code       Next;
+Code           Next;
-Token*     Tok;
+Token*         Tok;
-Code       Parent;
+Code           Parent;
-CodeType   Type;
+CodeT          Type;
-ModuleFlag ModuleFlags;
+ModuleFlag     ModuleFlags;
 ```
 
 Serialization:
@@ -463,12 +440,12 @@ CodeSpecifiers Specs;
 CodeType       ReturnType;
 CodeParams     Params;
 CodeBody       Body;
-StrCached      Name;
+StrCached   Name;
 Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 ModuleFlag     ModuleFlags;
 OperatorT      Op;
 ```
@@ -495,12 +472,12 @@ CodeComment    InlineCmt;
 CodeSpecifiers Specs;
 CodeType       ValueType;
 CodeBody       Body;
-StrCached      Name;
+StrCached   Name;
 Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -521,17 +498,17 @@ Serialization:
 Fields:
 
 ```cpp
-CodeType   ValueType;
+CodeType       ValueType;
-Code       Macro;
+Code           Macro;
-Code       Value;
+Code           Value;
-Code       PostNameMacro;
+Code           PostNameMacro;
-StrCached  Name;
+StrCached   Name;
-CodeParams Last;
+CodeParams     Last;
-CodeParams Next;
+CodeParams     Next;
-Token*     Tok;
+Token*         Tok;
-Code       Parent;
+Code           Parent;
-CodeType   Type;
+CodeT          Type;
-s32        NumEntries;
+s32            NumEntries;
 ```
 
 Serialization:
@@ -547,13 +524,13 @@ Serialization:
 Fields:
 
 ```cpp
-StrCached Content;
+StrCached   Content;
-StrCached Name;
+StrCached   Name;
-Code      Prev;
+Code           Prev;
-Code      Next;
+Code           Next;
-Token*    Tok;
+Token*         Tok;
-Code      Parent;
+Code           Parent;
-CodeType  Type;
+CodeT          Type;
 ```
 
 Serialization:
@@ -567,13 +544,13 @@ Serialization:
 Fields:
 
 ```cpp
-StrCached Content;
+StrCached  Content;
-StrCached Name;
+StrCached  Name;
-Code      Prev;
+Code          Prev;
-Code      Next;
+Code          Next;
-Token*    Tok;
+Token*        Tok;
-Code      Parent;
+Code          Parent;
-CodeType  Type;
+CodeT         Type;
 ```
 
 Serialization:
@@ -589,12 +566,12 @@ Fields:
 ```cpp
 SpecifierT     ArrSpecs[ AST_ArrSpecs_Cap ];
 CodeSpecifiers NextSpecs;
-StrCached      Name;
+StrCached   Name;
 Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 s32            NumEntries;
 ```
 
@@ -609,15 +586,15 @@ Serialization:
 Fields:
 
 ```cpp
-CodeParams  Params;
+CodeParams     Params;
-Code        Declaration;
+Code           Declaration;
 StrCached   Name;
-Code        Prev;
+Code           Prev;
-Code        Next;
+Code           Next;
-Token*      Tok;
+Token*         Tok;
-Code        Parent;
+Code           Parent;
-CodeType    Type;
+CodeT          Type;
-ModuleFlag  ModuleFlags;
+ModuleFlag     ModuleFlags;
 ```
 
 Serialization:
@@ -644,8 +621,8 @@ Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-StrCached      Name;
+StrCached   Name;
-CodeType       Type;
+CodeT          Type;
 b32            IsParamPack;
 ETypenameTag   TypeTag;
 ```
@@ -670,16 +647,16 @@ Those (macros) don't use the underlying type field as everything was serialized
 Fields:
 
 ```cpp
-CodeComment InlineCmt;
+CodeComment   InlineCmt;
-Code        UnderlyingType;
+Code          UnderlyingType;
-StrCached   Name;
+StrCached  Name;
-Code        Prev;
+Code          Prev;
-Code        Next;
+Code          Next;
-Token*      Tok
+Token*        Tok
-Code        Parent;
+Code          Parent;
-CodeType    Type;
+CodeT         Type;
-ModuleFlag  ModuleFlags;
+ModuleFlag    ModuleFlags;
-b32         IsFunction;
+b32           IsFunction;
 ```
 
 Serialization:
@@ -705,12 +682,12 @@ Fields:
 ```cpp
 CodeAttributes Attributes;
 CodeBody       Body;
-StrCached      Name;
+StrCached   Name;
 Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 ModuleFlag     ModuleFlags;
 ```
 
@@ -731,12 +708,12 @@ Fields:
 CodeComment    InlineCmt;
 CodeAttributes Attributes;
 CodeType       UnderlyingType;
-StrCached      Name;
+StrCached   Name;
 Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 ModuleFlag     ModuleFlags;
 ```
 
@@ -763,13 +740,13 @@ CodeSpecifiers Specs;
 CodeType       ValueType;
 Code           BitfieldSize;
 Code           Value;
-StrCached      Name;
+StrCached   Name;
 CodeVar        NextVar;
 Code           Prev;
 Code           Next;
 Token*         Tok;
 Code           Parent;
-CodeType       Type;
+CodeT          Type;
 ModuleFlag     ModuleFlags;
 s32            VarParenthesizedInit;
 ```

docs/Parser_Algo.md

@@ -12,9 +12,7 @@ gencpp uses a hand-written recursive descent parser. Both the lexer and parser c
 
 ### Lexer
 
-File: [lexer.cpp](../base/components/lexer.cpp)
+The lex procedure does the lexical pass of content provided as a `Str` type.  
-
-The `lex` procedure does the lexical pass of content provided as a `Str` type.  
 The tokens are stored (for now) in `Lexer_Tokens`.
 
 Fields:
@@ -26,15 +24,17 @@ s32          Idx;
 
 What token types are supported can be found in [ETokType.csv](../base/enums/ETokType.csv) you can also find the token types in [ETokType.h](../base/components/gen/etoktype.cpp) , which is the generated enum from the csv file.
 
+Tokens are defined with the struct `gen::parser::Token`:
+
+Fields:
+
 ```cpp
-struct Token
+char const* Text;
-{
+sptr        Length;
-    Str     Text;
+TokType     Type;
-    TokType Type;
+s32         Line;
-    s32     Line;
+s32         Column;
-    s32     Column;
+u32         Flags;
-    u32     Flags;
-}
 ```
 
 Flags is a bitfield made up of TokFlags (Token Flags):
@@ -52,17 +52,25 @@ Flags is a bitfield made up of TokFlags (Token Flags):
 * `TF_EndDefinition` : Can be interpreted as an end definition for a scope.
 * `TF_Formatting` : Considered a part of the formatting
 * `TF_Literal` : Anything considered a literal by C++.
-* `TF_Macro_Functional` : Used to indicate macro token is flagged as `MF_Functional`.
+
-* `TF_Macro_Expects_Body` : Used to indicate macro token is flagged as `MF_Expects_Body`.
+I plan to replace IsAssign with a general flags field and properly keep track of all operator types instead of abstracting it away to `ETokType::Operator`.
+
+Traversing the tokens is done with the following interface macros:
+
+| Macro | Description |
+| --- | --- |
+| `currtok_noskip` | Get the current token without skipping whitespace |
+| `currtok` | Get the current token, skip any whitespace tokens |
+| `prevtok` | Get the previous token (does not skip whitespace) |
+| `nexttok` | Get the next token (does not skip whitespace) |
+| `eat( Token Type )` | Check to see if the current token is of the given type, if so, advance Token's index to the next token |
+| `left` | Get the number of tokens left in the token array |
+| `check_noskip` | Check to see if the current token is of the given type, without skipping whitespace |
+| `check` | Check to see if the current token is of the given type, skip any whitespace tokens |
 
 ### Parser
 
-Files:
+The parser has a limited user interface, only specific types of definitions or statements are expected to be provided by the user directly when using to construct an AST dynamically (See SOA for example). It however does attempt to provide capability to parse a full C/C++ from production codebases.
-
-* [interface.parsering.cpp](../base/components/interface.parsing.cpp)
-* [parser.cpp](../base/components/parser.cpp)
-
-The parser has a limited user interface, only specific types of definitions or statements are expected to be provided by the user directly when using to construct an AST dynamically. It however does attempt to provide capability to parse a full C/C++ from production codebases.
 
 Each public user interface procedure has the following format:
 
@@ -81,7 +89,8 @@ Each public user interface procedure has the following format:
 }
 ```
 
-The most top-level parsing procedure used for C/C++ file parsing is `parse_global_body`.  
+The most top-level parsing procedure used for C/C++ file parsing is `parse_global_body`:
+
 It uses a helper procedure called `parse_global_nspace`.
 
 Each internal procedure will have the following format:
@@ -102,300 +111,126 @@ internal
 }
 ```
 
-The parsing implementation contains throughut the codeapths to indicate how far their contextual AST node has been resolved.
-
-Example:
-
-```c
-internal
-CodeFn parser_parse_function()
-{
-    push_scope();
-
-    Specifier specs_found[16] = { Spec_NumSpecifiers };
-    s32        NumSpecifiers = 0;
-
-    CodeAttributes attributes = { nullptr };
-    CodeSpecifiers specifiers = { nullptr };
-    ModuleFlag     mflags     = ModuleFlag_None;
-
-    if ( check(Tok_Module_Export) ) {
-        mflags = ModuleFlag_Export;
-        eat( Tok_Module_Export );
-    }
-    // <export>
-
-    attributes = parse_attributes();
-    // <export> <Attributes>
-
-    while ( left && tok_is_specifier(currtok) )
-    {
-        Specifier spec = str_to_specifier( tok_to_str(currtok) );
-
-        switch ( spec )
-        {
-            GEN_PARSER_FUNCTION_ALLOWED_SPECIFIER_CASES:
-            break;
-
-            default:
-                log_failure( "Invalid specifier %S for functon\n%SB", spec_to_str(spec), parser_to_strbuilder(_ctx->parser) );
-                parser_pop(& _ctx->parser);
-                return InvalidCode;
-        }
-
-        if ( spec == Spec_Const )
-            continue;
-
-        specs_found[NumSpecifiers] = spec;
-        NumSpecifiers++;
-        eat( currtok.Type );
-    }
-
-    if ( NumSpecifiers ) {
-        specifiers = def_specifiers_arr( NumSpecifiers, specs_found );
-    }
-    // <export> <Attributes> <Specifiers>
-
-    CodeTypename ret_type = parser_parse_type(parser_not_from_template, nullptr);
-    if ( cast(Code, ret_type) == Code_Invalid ) {
-        parser_pop(& _ctx->parser);
-        return InvalidCode;
-    }
-    // <export> <Attributes> <Specifiers> <ReturnType>
-
-    Token name = parse_identifier(nullptr);
-    _ctx->parser.Scope->Name = name.Text;
-    if ( ! tok_is_valid(name) ) {
-        parser_pop(& _ctx->parser);
-        return InvalidCode;
-    }
-    // <export> <Attributes> <Specifiers> <ReturnType> <Name>
-
-    CodeFn result = parse_function_after_name( mflags, attributes, specifiers, ret_type, name );
-    // <export> <Attributes> <Specifiers> <ReturnType> <Name> ...
-
-    parser_pop(& _ctx->parser);
-    return result;
-}
-```
-
-In the above `parse_function` implementation:
-
-`// <intutive expression of AST component> ...`
-
-Will be conventionlly used where by that point in time for the codepath: `<intutive expression of AST component>` should be resolved for the AST.
-
-## Outline of parsing codepaths
-
 Below is an outline of the general alogirithim used for these internal procedures. The intention is to provide a basic briefing to aid the user in traversing the actual code definitions. These appear in the same order as they are in the `parser.cpp` file
 
 ***NOTE: This is still heavily in an alpha state. A large swaph of this can change, make sure these docs are up to date before considering them 1:1 with the repo commit your considering.***
 
 ## `parse_array_decl`
 
-1. Push parser scope
+1. Check if its an array declaration with no expression.
-2. Check for empty array `[]`
+    1. Consume and return empty array declaration
-   1. Return untyped string with single space if found
+2. Opening square bracket
-3. Check for opening bracket `[`
+3. Consume expression
-   1. Validate parser not at EOF
+4. Closing square bracket
-   2. Reject empty array expression
+5. If adjacent opening bracket
-   3. Capture expression tokens until closing bracket
+    1. Repeat array declaration parse until no brackets remain
-   4. Calculate expression span length
+
-   5. Convert to untyped string
+## `parse_assignment_expression`
-4. Validate and consume closing bracket `]`
+
-5. Handle multi-dimensional case
+1. Eat the assignment operator
-   1. If adjacent `[` detected, recursively parse
+2. Make sure there is content or at least an end statement after.
-   2. Link array expressions via Next pointer
+3. Flatten the assignment expression to an untyped Code string.
-6. Pop parser scope
-7. Return array expression or NullCode on failure
 
 ## `parse_attributes`
 
-1. Push parser scope and initialize tracking
+1. Check for standard attribute
-   1. Store initial token position
+2. Check for GNU attribute
-   2. Initialize length counter
+3. Check for MSVC attribute
-2. Process attributes while available 
+4. Check for a token registered as an attribute
-   1. Handle C++ style attributes `[[...]]`
+  a. Check and grab the arguments of a token registered of an attribute if it has any.
-       1. Consume opening `[[`
+5. Repeat for chained attributes. Flatten them to a single attribute AST node.
-       2. Capture content until closing `]]`
-       3. Calculate attribute span length
-   2. Handle GNU style `__attribute__((...))`
-       1. Consume `__attribute__` keyword and opening `((`
-       2. Capture content until closing `))`
-       3. Calculate attribute span length
-   3. Handle MSVC style `__declspec(...)`
-       1. Consume `__declspec` and opening `(`
-       2. Capture content until closing `)`
-       3. Calculate attribute span length
-   4. Handle macro-style attributes
-       1. Consume attribute token
-       2. If followed by parentheses
-           1. Handle nested parentheses tracking
-           2. Capture content maintaining paren balance
-       3. Calculate attribute span length
-3. Generate attribute code if content captured
-   1. Create attribute text span from start position and length
-   2. Strip formatting from attribute text
-   3. Construct Code node
-       1. Set type to `CT_PlatformAttributes`
-       2. Cache and set Name and Content fields
-   4. Return as CodeAttributes
-4. Pop parser scope
-5. Return NullCode if no attributes found
 
 ## `parse_class_struct`
 
-1. Validate token type is class or struct
+1. Check for export module specifier
-   1. Return InvalidCode if validation fails
+2. class or struct keyword
-2. Initialize class/struct metadata
+3. `parse_attributes`
-   1. Access specifier (default)
+4. If identifier : `parse_identifier`
-   2. Parent class/struct reference
+5. Parse inherited parent or interfaces
-   3. Class/struct body
+6. If opening curly brace : `parse_class_struct_body`
-   4. Attributes
+7. If not an inplace definition
-   5. Module flags
+    1. End statement
-3. Parse module export flag if present
+    2. Check for inline comment
-   1. Set ModuleFlag_Export
-   2. Consume export token
-4. Consume class/struct token
-5. Parse attributes via `parse_attributes()`
-6. Parse class/struct identifier
-   1. Update parser scope name
-7. Initialize interface array (4KB arena)
-8. Parse inheritance/implementation
-   1. If classifier token (`:`) present
-       1. Parse access specifier if exists
-       2. Parse parent class/struct name
-       3. Parse additional interfaces
-           1. Separated by commas
-           2. Optional access specifiers
-           3. Store in interface array
-9. Parse class body if present
-   1. Triggered by opening brace
-   2. Parse via `parse_class_struct_body`
-10. Handle statement termination
-    1. Skip for inplace definitions
-    2. Consume semicolon
-    3. Parse inline comment if present
-11. Construct result node
-    1. Create class definition if Tok_Decl_Class
-    2. Create struct definition if Tok_Decl_Struct
-    3. Attach inline comment if exists
-12. Cleanup interface array and return result
 
 ## `parse_class_struct_body`
 
-1. Initialize scope and body structure
+1. Opening curly brace
-   1. Push parser scope
+2. Parse the body (Possible options):
-   2. Consume opening brace
+    1. Ignore dangling end statements
-   3. Create code node with `CT_Class_Body` or `CT_Struct_Body` type
+    2. Newline : ast constant
-2. Parse body members while not at closing brace
+    3. Comment : `parse_comment`
-   1. Initialize member parsing state
+    4. Access_Public : ast constant
-       1. Code member (InvalidCode)
+    5. Access_Protected : ast constant
-       2. Attributes (null)
+    6. Access_Private : ast constant
-       3. Specifiers (null)
+    7. Decl_Class : `parse_complicated_definition`
-       4. Function expectation flag
+    8. Decl_Enum : `parse_complicated_definition`
-   2. Handle preprocessor hash if present
+    9. Decl_Friend : `parse_friend`
-   3. Process member by token type in switch statement
+    10. Decl_Operator : `parse_operator_cast`
-       1. Statement termination - warn and skip
+    11. Decl_Struct : `parse_complicated_definition`
-       2. Newline - format member
+    12. Decl_Template : `parse_template`
-       3. Comments - parse comment
+    13. Decl_Typedef : `parse_typedef`
-       4. Access specifiers - `public/protected/private`
+    14. Decl_Union : `parse_complicated_definition`
-       5. Declarations - `class/enum/struct/union/typedef/using`
+    15. Decl_Using : `parse_using`
-       6. Operators - `destructors/casts`
+    16. Operator == '~'
-       7. Preprocessor directives - `define/include/conditionals/pragmas`
+        1. `parse_destructor`
-       8. Preprocessor statement macros
+    17. Preprocess_Define : `parse_define`
-       9. Report naked preprocossor expression macros detected as an error.
+    18. Preprocess_Include : `parse_include`
-       10. Static assertions
+    19. Preprocess_Conditional (if, ifdef, ifndef, elif, else, endif) : `parse_preprocess_cond` or else/endif ast constant
-       11. Attributes and specifiers
+    20. Preprocess_Macro : `parse_simple_preprocess`
-           1. Parse attributes
+    21. Preprocess_Pragma : `parse_pragma`
-           2. Parse valid member specifiers
+    22. Preprocess_Unsupported : `parse_simple_preprocess`
-           3. Handle `attribute-specifier-attribute` case
+    23. StaticAssert : `parse_static_assert`
-       12. Identifiers and types
+    24. The following compound into a resolved definition or declaration:
-           1. Check for constructors
+        1. Attributes (Standard, GNU, MSVC) : `parse_attributes`
-           2. Parse `operator/function/variable`
+        2. Specifiers (consteval, constexpr, constinit, explicit, forceinline, inline, mutable, neverinline, static, volatile, virtual)
-       13. Default - capture unknown content until closing brace
+        3. Possible Destructor : `parse_destructor`
-   4. Validate member parsing
+        4. Possible User defined operator cast : `parse_operator_cast`
-       1. Return InvalidCode if member invalid
+        5. Possible Constructor : `parse_constructor`
-       2. Append valid member to body
+        6. Something that has the following: (identifier, const, unsigned, signed, short, long, bool, char, int, double)
-3. Finalize body
+            1. Possible Constructor `parse_constructor`
-   1. Consume closing brace
+            2. Possible Operator, Function, or varaible : `parse_operator_function_or_variable`
-   2. Pop parser scope
+    25. Something completely unknown (will just make untyped...) : `parse_untyped`
-   3. Return completed CodeBody
 
 ## `parse_comment`
 
 1. Just wrap the token into a cached string ( the lexer did the processing )
 
-## `parse_complicated_definition`
+## `parse_compilcated_definition`
 
-1. Initialize parsing context
+This is a helper function used by the following functions to help resolve a declaration or definition:
-   1. Push scope
-   2. Set inplace flag false
-   3. Get token array reference
-2. Scan ahead for statement termination
-   1. Track brace nesting level
-   2. Find first semicolon at level 0
-3. Handle declaration variants
-   1. Forward declaration case
-       1. Check if only 2 tokens before semicolon
-       2. Parse via `parse_forward_or_definition`
-   2. Function with trailing specifiers
-       1. Identify trailing specifiers
-       2. Check for function pattern
-       3. Parse as `operator/function/variable`
-       4. Return `InvalidCode` if pattern invalid
-   3. Identifier-based declarations
-       1. Check identifier patterns
-           1. Inplace definition `{...} id;`
-           2. Namespace type variable `which id id;`
-           3. Enum with class qualifier
-           4. `Pointer/reference` types
-       2. Parse as `operator/function/variable` if valid
-       3. Return `InvalidCode` if pattern invalid
-   4. Basic type declarations
-       1. Validate enum class pattern
-       2. Parse via `parser_parse_enum`
-       3. Return `InvalidCode` if invalid
-   5. Direct definitions
-       1. Handle closing brace - `parse_forward_or_definition`
-       2. Handle array definitions - `parse_operator_function_or_variable`
-       3. Return InvalidCode for unknown patterns
 
-## `parse_assignment_expression`
+* `parse_class_struct_body`
+* `parse_global_nspace`
+* `parse_union`
 
-1. Initialize expression parsing
+A portion of the code in `parse_typedef` is very similar to this as both have to resolve a similar issue.
-   1. Null expression pointer
+
-   2. Consume assignment operator token
+1. Look ahead to the termination token (End statement)
-   3. Capture initial expression token
+2. Check to see if it fits the pattern for a forward declare
-2. Validate expression presence
+3. If the previous token was an identifier ( `token[-1]` ):
-   1. Check for immediate termination
+    1. Look back one more token : `[-2]`
-   2. Return `InvalidCode` if missing expression
+    2. If the token has a closing brace its an inplace definition
-3. Parse balanced expression
+    3. If the `token[-2]` is an identifier & `token[-3]` is the declaration type, its a variable using a namespaced type.
-   1. Track nesting level for
+    4. If the `token[-2]` is an indirection, then its a variable using a namespaced/forwarded type.
-       1. Curly braces
+    5. If the `token[-2]` is an assign classifier, and the starting tokens were the which type with possible `class` token after, its an enum forward declaration.
-       2. Parentheses
+    6. If any of the above is the case, `parse_operator_function_or_variable`
-   2. Continue until
+4. If the `token[2]` is a vendor fundamental type (builtin) then it is an enum forward declaration.
-       1. End of input, or
+5. If the previous token was a closing curly brace, its a definition : `parse_forward_or_definition`
-       2. Statement terminator, or
+6. If the previous token was a closing square brace, its an array definition : `parse_operator_function_or_variable`
-       3. Unnested comma
+
-   3. Consume tokens sequentially
+## `parse_define`
-4. Generate expression code
+
-   1. Calculate expression span length
+1. Define directive
-   2. Convert to untyped string
+2. Get identifier
-   3. Return expression node
+3. Get Content (Optional)
 
 ## `parse_forward_or_definition`
 
-1. Declaration type routing
+* Parse any of the following for either a forward declaration or definition:
-   1. Class (`Tok_Decl_Class`) -> `parser_parse_class`
+    1. Decl_Class : `parse_class`
-   2. Enum (`Tok_Decl_Enum`) -> `parser_parse_enum` 
+    2. Decl_Enum : `parse_enum`
-   3. Struct (`Tok_Decl_Struct`) -> `parser_parse_struct`
+    3. Decl_Struct : `parse_struct`
-   4. Union (`Tok_Decl_Union`) -> `parser_parse_union`
+    4. Decl_Union : `parse_union`
-2. Error handling
-   1. Return `InvalidCode` for unsupported token types
-   2. Log failure with parser context
-
-`is_inplace` flag propagates to specialized codepaths to maintain parsing context.
 
 ## `parse_function_after_name`
 
@@ -404,191 +239,80 @@ after its been made ceratin that the type of declaration or definition is indeed
 
 By the point this function is called the following are known : export module flag, attributes, specifiers, return type, & name
 
-1. Parameter parsing
+1. `parse_parameters`
-   1. Push scope
+2. parse postfix specifiers (we do not check if the specifier here is correct or not to be here... yet)
-   2. Parse parameter list with parentheses
+3. If there is a body : `parse_body`
-2. Post-parameter specifier processing
+4. Otherwise :
-   1. Collect trailing specifiers
+     1. Statment end
-   2. Initialize or append to existing specifiers
+     2. Check for inline comment
-3. Parse function termination
-   1. Function body case
-       1. Parse body if open brace found
-       2. Validate body type (`CT_Function_Body` or `CT_Untyped`)
-   2. Pure virtual case
-       1. Handle "`= 0`" syntax
-       2. Append pure specifier
-   3. Forward declaration case
-       1. Consume statement terminator
-   4. Handle inline comments for all cases
-4. Construct function node
-   1. Strip whitespace from name
-   2. Initialize `CodeFn` with base properties
-       1. Name (cached, stripped)
-       2. Module flags
-   3. Set node type
-       1. `CT_Function` if body present
-       2. `CT_Function_Fwd` if declaration only
-   4. Attach components
-       1. Attributes if present
-       2. Specifiers if present
-       3. Return type
-       4. Parameters if present
-       5. Inline comment if present
-5. Cleanup and return
-   1. Pop scope
-   2. Return completed function node
 
 ## `parse_function_body`
 
 Currently there is no actual parsing of the function body. Any content with the braces is shoved into an execution AST node.
 In the future statements and expressions will be parsed.
 
-1. Initialize body parsing
+1. Open curly brace
-   1. Push scope
+2. Grab all tokens between the brace and the closing brace, shove them in a execution AST node.
-   2. Consume opening brace
+3. Closing curly brace
-   3. Create CodeBody with CT_Function_Body type
-2. Capture function content
-   1. Record start token position
-   2. Track brace nesting level
-   3. Consume tokens while
-       1. Input remains AND
-       2. Not at unmatched closing brace
-   4. Update level counters
-       1. Increment on open brace
-       2. Decrement on closing brace when level > 0
-3. Process captured content
-   1. Calculate content length via pointer arithmetic
-   2. Create execution block if content exists
-       1. Construct string span from start position and length
-       2. Wrap in execution node
-       3. Append to body
-4. Finalize
-   1. Consume closing brace
-   2. Pop scope
-   3. Return cast body node
 
 ## `parse_global_nspace`
 
-1. State initialization
+1. Make sure this is being called for a valid type (namespace, global body, export body, linkage body)
-   1. Push parser scope
+2. If its not a global body, consume the opening curly brace
-   2. Validate namespace type (Global, Namespace, Export, Extern Linkage)
+3. Parse the body (Possible options):
-   3. Consume opening brace for non-global scopes
+    1. Ignore dangling end statements
-   4. Initialize `CodeBody` with specified body type: `which`
+    2. NewLine : ast constant
-2. Member parsing loop (while not at closing brace)
+    3. Comment : `parse_comment`
-   1. Reset parse state
+    4. Decl_Cass : `parse_complicated_definition`
-       * Member code
+    5. Decl_Enum : `parse_complicated_definition`
-       * Attributes
+    6. Decl_Extern_Linkage : `parse_extern_link`
-       * Specifiers
+    7. Decl_Namespace : `parse_namespace`
-       * Function expectation flag
+    8. Decl_Struct : `parse_complicated_definition`
-   2. Member type handling
+    9. Decl_Template : `parse_template`
-       1. Declarations
+    10. Decl_Typedef : `parse_typedef`
-           * `Class/Struct/Union/Enum` via `parse_complicated_definition`
+    11. Decl_Union : `parse_complicated_definition`
-           * `Template/Typedef/Using` via dedicated parsers
+    12. Decl_Using : `parse_using`
-           * `Namespace/Export/Extern` declarations
+    13. Preprocess_Define : `parse_define`
-       2. Preprocessor directivess
+    14. Preprocess_Include : `parse_include`
-           * Include/Define
+    15. Preprocess_If, IfDef, IfNotDef, Elif : `parse_preprocess_cond`
-           * Conditionals `(if / ifdef / ifndef / elif / else / endif)`
+    16. Preprocess_Else : ast constant
-           * Pragmas
+    17. Preprocess_Endif : ast constant
-           * Preprocessor statement macros
+    18. Preprocess_Macro : `parse_simple_preprocess`
-           * Report naked preprocossor expression macros detected as an error.
+    19. Preprocess_Pragma : `parse_pragma`
-       3. Comments/Formatting
+    20. Preprocess_Unsupported : `parse_simple_preprocess`
-           * Newlines
+    21. StaticAssert : `parse_static_assert`
-           * Comments
+    22. Module_Export : `parse_export_body`
-       4. Static assertions
+    23. Module_Import : NOT_IMPLEMENTED
-   3. Attributes and specifiers
+    24. The following compound into a resolved definition or declaration:
-       1. Parse attributes if present
+        1. Attributes ( Standard, GNU, MSVC, Macro ) : `parse_attributes`
-       2. Collect valid specifiers (max 16)
+        2. Specifiers ( consteval, constexpr, constinit, extern, forceinline, global, inline, internal_linkage, neverinline, static )
-       3. Handle `consteval` for function expectation
+        3. Is either ( identifier, const specifier, long, short, signed, unsigned, bool, char, double, int)
-   4. Identifier resolution
+            1. Attempt to parse as construtor or destructor : `parse_global_nspace_constructor_destructor`
-       1. Check `constructor/destructor` implementation
+            2. If its an operator cast (definition outside class) : `parse_operator_cast`
-       2. Look ahead for user defined operator implementation outside of class
+            3. Its an operator, function, or varaible : `parse_operator_function_or_varaible`
-       3. Default to `operator/function/variable` parse
+4. If its not a global body, consume the closing curly brace
-3. Member validation/storage
-   1. Validate parsed member
-   2. Append to body if valid
-   3. Return `InvalidCode` on parse failure
-4. Scope finalization
-   1. Consume closing brace for non-global scopes
-   2. Pop parser scope
-   3. Return completed body
 
 ## `parse_global_nspace_constructor_destructor`
 
-1. Forward Token Analysis
+1. Look ahead for the start of the arguments for a possible constructor/destructor
-   1. Scan for parameter list opening parenthesis
+2. Go back past the identifier
-   2. Template Expression Handling
+3. Check to see if its a destructor by checking for the `~`
-      * Track template nesting depth
+4. Continue the next token should be a `::`
-      * Account for nested parentheses within templates
+5. Determine if the next valid identifier (ignoring possible template parameters) is the same as the first identifier of the function.
-      * Skip until template closure or parameter start
+6. If it is we have either a constructor or destructor so parse using their respective functions (`parse_constructor`, `parse_destructor`).
-
-```cpp
-      // Valid patterns:
-      ClassName          ::  ClassName(...)
-      ClassName          :: ~ClassName(...)
-      ClassName< T ... > ::  ClassName(...)
-```
-
-2. Constructor/Destructor Identification
-   1. Token Validation Sequence
-      * Verify identifier preceding parameters
-      * Check for destructor indicator (`~`)
-      * Validate scope resolution operator (`::`)
-   2. Left-side Token Analysis
-      * Process nested template expressions
-      * Maintain template/capture level tracking
-      * Locate matching identifier token
-3. Parser Resolution
-   1. Name Pattern Validation
-      * Compare identifier tokens for exact match
-   2. Specialized Parsing
-      * Route to `parser_parse_destructor` for '~' prefix
-      * Route to `parser_parse_constructor` for direct match
-   3. Apply specifiers to resulting node
-4. Return result (`NullCode` on pattern mismatch)
-
-### Implementation Constraints
-
-* Cannot definitively distinguish nested namespaces with identical names
-* Return type detection requires parser enhancement
-* Template parameter validation is syntax-based only
-* Future enhancement: Implement type parsing with rollback capability
 
 ## `parse_identifier`
 
 This is going to get heavily changed down the line to have a more broken down "identifier expression" so that the qualifier, template args, etc, can be distinguished between the targeted identifier.  
 The function can parse all of them, however the AST node compresses them all into a string.
 
-1. Initialize identifier context
+1. Consume first identifier
-   1. Push parser scope
+2. `parse_template_args`
-   2. Capture initial token as name
+3. While there is a static symbol accessor ( `::` )
-   3. Set scope name from token text
+    1. Consume `::`
-2. Process initial identifier component
+    2. Consume member identifier
-   1. Consume identifier token
+    3. `parse_template args` (for member identifier)
-   2. Parse template arguments if present
+    4. If a `~` is encounted and the scope is for a destructor's identifier, do not consume it and return with what parsed.
-3. Handle qualified identifiers (loop while `::` found)
-   1. Consume static access operator
-   2. Validate token sequence:
-       1. Handle destructor operator (`~`)
-           * Validate destructor parsing context
-           * Update name span if valid
-           * Return invalid on context mismatch
-       2. Process member function pointer (`*`)
-           * Set possible_member_function flag if context allows
-           * Return invalid if pointer unexpected
-       3. Verify identifier token follows
-   3. Update identifier span
-       1. Extend name length to include new qualifier
-       2. Consume identifier token
-       3. Parse additional template arguments
-4. Return completed identifier token
-
-Technical notes:
-
-* Current implementation treats identifier as single token span
-* TODO: Refactor to AST-based identifier representation for:
-  * Distinct qualifier/symbol tracking
-  * Improved semantic analysis capabilities
-  * Better support for nested symbol resolution
 
 ## `parse_include`
 
@@ -599,45 +323,16 @@ Technical notes:
 
 This is needed as a operator defintion is not easily resolvable early on, as such this function handles resolving a operator after its been made ceratin that the type of declaration or definition is indeed for a operator signature.
 
-By the point this function is called the following are known : export module flag, attributes, specifiers, and return type
+By the point this function is called the following are known : export module flag, attributes, specifiers, return type
 
-1. Initialize operator context
+1. If there is any qualifiers for the operator, parse them
-   1. Push scope
+2. Consume operator keyword
-   2. Parse qualified namespace identifier
+3. Determine the operator type (This will be offloaded to the lexer moreso than how it is now) & consume
-   3. Consume `operator` keyword
+4. `parse_params`
-2. Operator identification
+5. If there is no parameters this is operator is a member of pointer if its symbols is a *.
-   1. Validate operator token presence
+6. Parse postfix specifiers
-   2. Set scope name from operator token
+7. If there is a opening curly brace, `parse function_body`
-   3. Map operator token to internal operator enum:
+8. Otherwise: consume end statement, check for inline comment.
-       * Arithmetic: `+, -, *, /, %`
-       * Assignment: `+=, -=, *=, /=, %=, =`
-       * Bitwise: `&, |, ^, ~, >>`
-       * Logical: `&&, ||, !, ==`
-       * Comparison: `<, >, <=, >=`
-       * Member access: `->, ->*`
-       * Special: `(), [], new, delete`
-   4. Handle array variants for new/delete
-3. Parameter and specifier processing
-   1. Parse parameter list
-   2. Handle multiply/member-pointer ambiguity
-   3. Collect trailing specifiers
-   4. Merge with existing specifiers
-4. Function body handling
-   1. Parse implementation if present
-   2. Otherwise consume statement terminator
-   3. Capture inline comments
-5. Result construction
-   1. Create operator node with:
-       * Operator type
-       * Namespace qualification
-       * Parameters
-       * Return type
-       * Implementation body
-       * Specifiers
-       * Attributes
-       * Module flags
-   2. Attach inline comments
-6. Pop scope
 
 ## `parse_operator_function_or_variable`
 

docs/Parsing.md

@@ -6,9 +6,9 @@
 
 # Parsing
 
-The library features a naive single-pass parser, tailored for only what the library needs; for construction of C++ code into gencpp's AST for *"front-end"* meta-programming purposes.
+The library features a naive single-pass parser tailored for only what the library needs to construct the supported syntax of C++ into its AST for *"front-end"* meta-programming purposes.
 
-This parser does not, and should not do the compiler's job. By only supporting this minimal set of features, the parser is kept (so far) around ~7000 loc. I hope to keep it under 10-15k loc worst case.
+This parser does not, and should not do the compiler's job. By only supporting this minimal set of features, the parser is kept (so far) around ~7000 loc. I hope to keep it under 10k loc worst case.
 
 You can think of this parser as *frontend parser* vs a *semantic parser*. Its intuitively similar to WYSIWYG. What you ***precerive*** as the syntax from the user-side before the compiler gets a hold of it, is what you get.
 
@@ -17,7 +17,6 @@ User exposed interface:
 ```cpp
 CodeClass       parse_class        ( Str class_def       );
 CodeConstructor parse_constructor  ( Str constructor_def );
-CodeDefine      parse_define       ( Str define_def      );
 CodeDestructor  parse_destructor   ( Str destructor_def  );
 CodeEnum        parse_enum         ( Str enum_def        );
 CodeBody        parse_export_body  ( Str export_def      );
@@ -54,98 +53,38 @@ The keywords supported for the preprocessor are:
 * endif
 * pragma
 
-Each directive `#` line is considered one preproecessor unit, and will be treated as one Preprocessor AST node.
+Each directive `#` line is considered one preproecessor unit, and will be treated as one Preprocessor AST. 
 If a directive is used with an unsupported keyword its will be processed as an untyped AST.
 
-The preprocessor lines are stored as members of their associated scope they are parsed within ( Global, Namespace, Class/Struct ).
+The preprocessor lines are stored as members of their associated scope they are parsed within. ( Global, Namespace, Class/Struct )  
-***Again: These ASTs will be considered members or entries of braced scope they reside within***
+***Again (Its not standard): These ASTs will be considered members or entries of braced scope they reside within***
 
 Any preprocessor definition abuse that changes the syntax of the core language is unsupported and will fail to parse if not kept within an execution scope (function body, or expression assignment).  
 Exceptions:
 
-* varaible definitions are allowed for a preprocessed macro `extern MACRO();`
+* function signatures are allowed for a preprocessed macro: `neverinline MACRO() { ... }`
-* function definitions are allowed for a preprocessed macro: `neverinline MACRO() { ... }`
   * Disable with: `#define GEN_PARSER_DISABLE_MACRO_FUNCTION_SIGNATURES`
 * typedefs allow for a preprocessed macro: `typedef MACRO();`
   * Disable with: `#define GEN_PARSER_DISABLE_MACRO_TYPEDEF`
 * Macros can behave as typenames
-* There is some macro support in parameters for functions or templates *(Specifically added to support parsing Unreal Engine source)*.
+* There is some macro support in paramters for functions or templates *(Specifically added to support parsing Unreal Engine source)*.
 
 *(Exceptions are added on an on-demand basis)*
 *(See functions `parse_operator_function_or_variable` and `parse_typedef` )*
 
 Adding your own exceptions is possible by simply modifying the parser to allow for the syntax you need.
 
-*Note: You could interpret this strictness as a feature. This would allow the user to see if their codebase or a third-party's codebase contains some egregious preprocessor abuse.*
+*Note: You could interpret this strictness as a feature. This would allow the user to see if their codebase or a third-party's codebase some some egregious preprocessor abuse.*
 
-Macros used within a file should be registered by the user before parsing. This can be done two ways:
+If a macro is not defined withint e scope of parsing a set of files, it can be defined beforehand by:
 
-1. The register macro interface within [interface.hpp](../base/components/interface.hpp).
+* Appending the [`PreprocessorDefines`](https://github.com/Ed94/gencpp/blob/a18b5b97aa5cfd20242065cbf53462a623cd18fa/base/components/header_end.hpp#L137) array.
-2. Using `def_define` to create a CodeDefine and making sure to not set `opts.dont_register_to_preprocess_macros` to `true`.
+  * For functional macros a "(" just needs to be added after the name like: `<name>(` so that it will tokenize its arguments as part of the token during lexing.
+* Defining a CodeDefine using `def_define`. The definition will be processed by the interface for user into `PreprocessorDefines`.
+  * This can be prevented by setting the optional prameter `dont_append_preprocess_defines`.
 
-## Registering macros
+The lexing and parsing takes shortcuts from whats expected in the standard.
 
-While the registeration of macros in the meta-program's side for parsing can be considered tedius, its necessary for the parser to accurately resolve the macros intent in one pass (and it provides some hygenics by verifying that they are used as intended).
-
-The following can be used to register a macro:
-
-```c
-GEN_API void register_macro( Macro macro );
-GEN_API void register_macros( s32 num, ... );
-GEN_API void register_macros_arr( s32 num, Macro* macros );
-```
-
-The Macro typename is defined with the following in [parser_types.hpp](../base/components/parser_types.hpp):
-
-```c
-struct Macro
-{
-    StrCached  Name;
-    MacroType  Type;
-    MacroFlags Flags;
-};
-```
-
-The macro can be designated one of the following types:
-
-* `MT_Expression`: Intended to resolve to an expression expansion.
-* `MT_Statement`: Intended to resolve an statement expansion.
-* `MT_Typename`: Intended to resolve to a typename.
-
-Additioonally tthe following flags may be set:
-
-* `MF_Functional`: The macro intended to be passed arguments are at least have the calling `()` as part of its usage.
-* `MF_Expects_Body`: The parser should expect a braced-body `{ ... }` after the macro signature `<name> <params>`
-* `MF_Allow_As_Identifier`: Will allow the macro to be an acceptable token/s when an `Tok_Identifier` is expected.
-* `MF_Allow_As_Attribute`: Will allow the macro to be an acceptable token/s when an attribute token/s is expected.
-* `MF_Allow_As_Definition`: Will allow the macro be an acceptable token/s when the parser expects a declartion or definition to resolve after attributes or specifiers have been identified beforehand.
-  * This flag requires that the macro is of type `MT_Statement` to make any sense of usage.
-
-If a macro is not defined the following warning will be issued if `GEN_BUILD_DEBUG=1` during lexing within [lexer.cpp](../base/components/lexer.cpp) - `lex_preprocessor_define`:
-
-```c
-log_fmt("Warning: '%S' was not registered before the lexer processed its #define directive, it will be registered as a expression macro\n"
-    , name.Text 
-);
-```
-
-Further within the same scope, the lexer will issue a warning if it detects a macro was not flagged as function but has an open parenthesis `(` token right after is name with no whitespace:
-
-```c
-log_fmt("Warning: %S registered macro is not flagged as functional yet the definition detects opening parenthesis '(' for arguments\n"
-    , name.Text
-);
-```
-
-Macros are tracked using a `MacroTable Macros;` defined as a member of the library's `Context`.
-
-```c
-typedef HashTable(Macro) MacroTable;
-```
-
-## Notes
-
-* Empty lines used throughout the file are preserved for formatting purposes during ast serialization (they have a dedicated Token: `Tok_NewLine`).
 * Numeric literals are not checked for validity.
 * The parse API treats any execution scope definitions with no validation and are turned into untyped Code ASTs. (There is a [todo](https://github.com/Ed94/gencpp/issues/49) to add support)
   * *This includes the assignment of variables.*
@@ -156,4 +95,4 @@ typedef HashTable(Macro) MacroTable;
 * Parsing attributes can be extended to support user defined macros by defining `GEN_DEFINE_ATTRIBUTE_TOKENS` (see `gen.hpp` for the formatting)
   * This is useful for example: parsing Unreal `Module_API` macros.
 
-**The lexer & parser do not gracefully attempt to continue when it comes across incorrect code, and doesn't properly track errors into a listing (yet).**
+Empty lines used throughout the file are preserved for formatting purposes during ast serialization.

docs/Readme.md

@@ -26,7 +26,7 @@ This means that the typename entry for the parameter AST would be either:
 ***Concepts and Constraints are not supported***  
 Its a [todo](https://github.com/Ed94/gencpp/issues/21)
 
-### Feature Macros
+### Feature Macros:
 
 * `GEN_DEFINE_ATTRIBUTE_TOKENS` : Allows user to define their own attribute macros for use in parsing.
   * This can be generated using base.cpp.
@@ -36,41 +36,10 @@ Its a [todo](https://github.com/Ed94/gencpp/issues/21)
 * `GEN_EXPOSE_BACKEND` : Will expose symbols meant for internal use only.
 * `GEN_ROLL_OWN_DEPENDENCIES` : Optional override so that user may define the dependencies themselves.
 * `GEN_DONT_ALLOW_INVALID_CODE` (Not implemented yet) : Will fail when an invalid code is constructed, parsed, or serialized.
-* `GEN_C_LIKE_CPP` : Setting to `<true or 1>` Will prevent usage of function defnitions using references and structs with member functions. Structs will still have user-defined operator conversions, for-range support, and other operator overloads
+* `GEN_C_LIKE_PP` : Setting to `<true or 1>` Will prevent usage of function defnitions using references and structs with member functions. Structs will still have user-defined operator conversions, for-range support, and other operator overloads
 
 ### The Data & Interface
 
-The library's persistent state is managed tracked by a context struct: `global Context* _ctx;` defined within [static_data.cpp](../base/components/static_data.cpp)
-
-https://github.com/Ed94/gencpp/blob/967a044637f1615c709cb723dc61118fcc08dcdb/base/components/interface.hpp#L39-L97
-
-The interface for the context:
-
-* `init`: Initializtion
-* `deinit`: De-initialization.
-* `reset`: Clears the allocations, but doesn't free the memoery, then calls `init()` on `_ctx` again.
-* `get_context`: Retreive the currently tracked context.
-* `set_context`: Swap out the current tracked context.
-
-
-#### Allocato usage
-
-* `Allocator_DyanmicContainers`: Growing arrays, hash tables. (Unbounded sized containers)
-* `Allocator_Pool`: Fixed-sized object allocations (ASTs, etc)
-* `Allocator_StrCache`: StrCached allocations
-* `Allocator_Temp`: Temporary alloations mostly intended for StrBuilder usage. Manually cleared by the user by their own discretion.
-
-The allocator definitions used are exposed to the user incase they want to dictate memory usage
-
-* Allocators are defined with the `AllocatorInfo` structure found in [`memory.hpp`](../base/dependencies/memory.hpp)
-* Most of the work is just defining the allocation procedure:
-
-```cpp
-    void* ( void* allocator_data, AllocType type, ssize size, ssize alignment, void* old_memory, ssize old_size, u64 flags );
-```
-
-For any allocator above that the user does not define before `init`, a fallback allocator will be assigned that utiizes the `fallback_allocator_proc` wtihin [interface.cpp](../base/components/interface.cpp).
-
 As mentioned in root readme, the user is provided Code objects by calling the constructor's functions to generate them or find existing matches.
 
 The AST is managed by the library and provided to the user via its interface.  
@@ -78,14 +47,14 @@ However, the user may specifiy memory configuration.
 
 [Data layout of AST struct (Subject to heavily change with upcoming todos)](../base/components/ast.hpp#L396-461)  
 
-https://github.com/Ed94/gencpp/blob/967a044637f1615c709cb723dc61118fcc08dcdb/base/components/ast.hpp#L369-L435
+https://github.com/Ed94/gencpp/blob/eea4ebf5c40d5d87baa465abfb1be30845b2377e/base/components/ast.hpp#L396-L461
 
 *`StringCahced` is a typedef for `Str` (a string slice), to denote it is an interned string*  
 *`CodeType` is enum taggin the type of code. Has an underlying type of `u32`*  
 *`OperatorT` is a typedef for `EOperator::Type` which has an underlying type of `u32`*  
-*`StrBuilder` is the dynamically allocating string builder type for the library*  
+*`StrBuilder` is the dynamically allocated string type for the library*  
 
-AST widths are setup to be AST_POD_Size (128 bytes by default).
+AST widths are setup to be AST_POD_Size.  
 The width dictates how much the static array can hold before it must give way to using an allocated array:
 
 ```cpp
@@ -104,16 +73,41 @@ int AST_ArrSpecs_Cap =
 )
 / sizeof(Specifier) - 1;
 ```
+
+*Ex: If the AST_POD_Size is 128 the capacity of the static array is 20.*
+
 Data Notes:
 
+* The allocator definitions used are exposed to the user incase they want to dictate memory usage
+  * You'll find the memory handling in `init`, `deinit`, `reset`, `gen_strbuilder_allocator`, `cache_str`, `make_code`.
+  * Allocators are defined with the `AllocatorInfo` structure found in [`memory.hpp`](../base/dependencies/memory.hpp)
+  * Most of the work is just defining the allocation procedure:
+
+```cpp
+    void* ( void* allocator_data, AllocType type, ssize size, ssize alignment, void* old_memory, ssize old_size, u64 flags );
+```
+
 * ASTs are wrapped for the user in a Code struct which is a wrapper for a AST* type.
 * Code types have member symbols but their data layout is enforced to be POD types.
 * This library treats memory failures as fatal.
 * Cached Strings are stored in their own set of arenas. AST constructors use cached strings for names, and content.
+  * `StringArenas`, `StringCache`, `Allocator_StringArena`, and `Allocator_StringTable` are the associated containers or allocators.
 * Strings used for serialization and file buffers are not contained by those used for cached strings.
-  * `_ctx->Allocator_Temp` is used.
+  * They are currently using `FallbackAllocator`, which are tracked array of arenas that grows as needed (adds buckets when one runs out).
+  * Memory within the buckets is not reused, so its inherently wasteful.
+  * I will be augmenting the default allocator with virtual memory & a slab allocator in the [future](https://github.com/Ed94/gencpp/issues/12)
+* Intrusive linked lists used children nodes on bodies, and parameters.
 * Its intended to generate the AST in one go and serialize after. The constructors and serializer are designed to be a "one pass, front to back" setup.
-  * Any modifcations to an existing AST should be to just construct another with the modifications done on-demand while traversing the AST (non-destructive).
+* Allocations can be tuned by defining the folloiwng macros (will be moved to runtime configuration in the future):
+  * `GEN_GLOBAL_BUCKET_SIZE` : Size of each bucket area for the global allocator
+  * `GEN_CODEPOOL_NUM_BLOCKS` : Number of blocks per code pool in the code allocator
+  * `GEN_SIZE_PER_STRING_ARENA` : Size per arena used with string caching.
+  * `GEN_MAX_COMMENT_LINE_LENGTH` : Longest length a comment can have per line.
+  * `GEN_MAX_NAME_LENGTH` : Max length of any identifier.
+  * `GEN_MAX_UNTYPED_STR_LENGTH` : Max content length for any untyped code.
+  * `TokenMap_FixedArena` : token_fmt_va uses local_persit memory of this arena type for the hashtable.
+  * `GEN_LEX_ALLOCATOR_SIZE`
+  * `GEN_BUILDER_STR_BUFFER_RESERVE`
 
 The following CodeTypes are used which the user may optionally use strong typing with if they enable: `GEN_ENFORCE_STRONG_CODE_TYPES`
 
@@ -123,7 +117,6 @@ The following CodeTypes are used which the user may optionally use strong typing
 * CodeClass
 * CodeConstructor
 * CodeDefine
-* CodeDefineParams
 * CodeDestructor
 * CodeEnum
 * CodeExec
@@ -134,7 +127,7 @@ The following CodeTypes are used which the user may optionally use strong typing
 * CodeModule
 * CodeNS
 * CodeOperator
-* CodeOpCast : User defined member operator conversion
+* CodeOpCast
 * CodeParams : Has support for `for : range` iterating across parameters.
 * CodePreprocessCond
 * CodePragma
@@ -147,15 +140,11 @@ The following CodeTypes are used which the user may optionally use strong typing
 * CodeUsing
 * CodeVar
 
-Each `struct Code<Name>` has an associated "filtered AST" with the naming convention: `AST_<CodeName>`
+Each Code boy has an associated "filtered AST" with the naming convention: `AST_<CodeName>`
 Unrelated fields of the AST for that node type are omitted and only necessary padding members are defined otherwise.
+Retrieving a raw version of the ast can be done using the `raw()` function defined in each AST.
 
-For the interface related to these code types see:
+## There are three sets of interfaces for Code AST generation the library provides
-
-* [ast.hpp](../base/components/ast.hpp): Under the region pragma `Code C-Interface`
-* [code_types.hpp](../base/components/code_types.hpp): Under the region pragma `Code C-Interface`. Additional functionlity for c++ will be within the struct definitions or at the end of the file.
-
-## There are three categories of interfaces for Code AST generation & reflection
 
 * Upfront
 * Parsing
@@ -175,7 +164,6 @@ Interface :``
 * def_class
 * def_constructor
 * def_define
-* def_define_params
 * def_destructor
 * def_enum
 * def_execution
@@ -230,27 +218,6 @@ Code <name>
 
 ```
 
-All optional parmeters are defined within `struct Opts_def_<functon name>`. This was done to setup a [macro trick](https://x.com/vkrajacic/status/1749816169736073295) for default optional parameers in the C library:
-
-```cpp
-struct gen_Opts_def_struct
-{
-	gen_CodeBody       body;
-	gen_CodeTypename   parent;
-	gen_AccessSpec     parent_access;
-	gen_CodeAttributes attributes;
-	gen_CodeTypename*  interfaces;
-	gen_s32            num_interfaces;
-	gen_ModuleFlag     mflags;
-};
-typedef struct gen_Opts_def_struct gen_Opts_def_struct;
-
-GEN_API gen_CodeClass gen_def__struct( gen_Str name, gen_Opts_def_struct opts GEN_PARAM_DEFAULT );
-#define gen_def_struct( name, ... ) gen_def__struct( name, ( gen_Opts_def_struct ) { __VA_ARGS__ } )
-```
-
-In the C++ library, the `def_<funtion name>` is not wrapped in a macro.
-
 When using the body functions, its recommended to use the args macro to auto determine the number of arguments for the varadic:
 
 ```cpp
@@ -261,7 +228,7 @@ def_global_body( 3, ht_entry, array_ht_entry, hashtable );
 ```
 
 If a more incremental approach is desired for the body ASTs, `Code def_body( CodeT type )` can be used to create an empty body.
-When the members have been populated use: `code_validate_body` to verify that the members are valid entires for that type.
+When the members have been populated use: `AST::validate_body` to verify that the members are valid entires for that type.
 
 ### Parse construction
 
@@ -277,6 +244,7 @@ Interface :
 * parse_export_body
 * parse_extern_link
 * parse_friend
+  * Purposefully are only support forward declares with this constructor.
 * parse_function
 * parse_global_body
 * parse_namespace

gen_c_library/Readme.md

@@ -91,7 +91,7 @@ C_Expression _Generic( selector_arg, a_type_expr_pair, ... ) {
 The first `arg` of _Generic behaves as the "controlling expression" or the expression that resolves to a type which will dictate which of the following expressions provided after to `_Generic` will be resolved as the one used inline for the implemenation.
 
 
-For this library's purposes we'll be using the functional macro equivalent *(if there is an exception I'll link it at the end of this section)*:
+For this library's purposes we'll be using the functional macro equivalent *(if there is an excpetion I'll link it at the end fo the section)*:
 
 ```c
 #define macro_that_uses_selector_arg_for_resolving_a_fucntion( selecting_exp) \
@@ -142,4 +142,4 @@ So for any given templated container interface. Expect the follwoing (taken stra
 
 `GEN_RESOLVED_FUNCTION_CALL` is an empty define, its just to indicate that its intended to expand to a function call.
 
-To see the the actual macro definitions used - see: [generic_macros.h](./components/generic_macros.h). They'll be injected right after the usual macros in the generated header file.
+To see the thea actual macro definitions used: [generic_macros.h](./components/generic_macros.h) has them. They'll be injected right after the usual macros are positioned in the header file.

gen_c_library/c_library.cpp

@@ -2,6 +2,7 @@
 
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
+#define GEN_EXPOSE_BACKEND
 #include "gen.cpp"
 #include "helpers/push_ignores.inline.hpp"
 
@@ -800,9 +801,6 @@ do                          \
 			b32 found = ignore_preprocess_cond_block(txt("GEN_INTELLISENSE_DIRECTIVES"), entry, parsed_ast, ast );
 			if (found) break;
 
-			found = ignore_preprocess_cond_block(txt("GEN_EXECUTION_EXPRESSION_SUPPORT"), entry, parsed_ast, ast );
-			if (found) break;
-
 			ast.append(entry);
 		}
 		break;
@@ -981,9 +979,6 @@ R"(#define AST_ArrSpecs_Cap \
 			found = ignore_preprocess_cond_block(txt("GEN_INTELLISENSE_DIRECTIVES"), entry, parsed_code_types, code_types );
 			if (found) break;
 
-			found = ignore_preprocess_cond_block(txt("GEN_EXECUTION_EXPRESSION_SUPPORT"), entry, parsed_code_types, code_types);
-			if (found) break;
-
 			code_types.append(entry);
 		}
 		break;
@@ -1078,10 +1073,7 @@ R"(#define <interface_name>( code ) _Generic( (code), \
 		case CT_Preprocess_If:
 		case CT_Preprocess_IfDef:
 		{
-			b32 found = ignore_preprocess_cond_block(txt("GEN_INTELLISENSE_DIRECTIVES"), entry, parsed_ast_types, ast_types );
+			b32 found = ignore_preprocess_cond_block(txt("GEN_INTELLISENSE_DIRECTIVES"), entry, parsed_code_types, code_types );
-			if (found) break;
-
-			found = ignore_preprocess_cond_block(txt("GEN_EXECUTION_EXPRESSION_SUPPORT"), entry, parsed_ast_types, ast_types);
 			if (found) break;
 
 			ast_types.append(entry);
@@ -1137,9 +1129,6 @@ R"(#define <interface_name>( code ) _Generic( (code), \
 			found = ignore_preprocess_cond_block(txt("GEN_COMPILER_CPP"), entry, parsed_interface, interface);
 			if (found) break;
 
-			found = ignore_preprocess_cond_block(txt("0"), entry, parsed_interface, interface);
-			if (found) break;
-
 			interface.append(entry);
 		}
 		break;

gen_c_library/c_library.refactor

@@ -420,8 +420,6 @@ word make_code, gen_make_code
 
 namespace set_allocator_, gen_set_allocator_
 
-namespace Opts_, gen_Opts_
-
 namespace def_,     gen_def_
 namespace parse_,   gen_parse_
 namespace token_,   gen_token_

gen_segmented/Readme.md

@@ -8,13 +8,11 @@
 
 # Segemented Library Generation
 
-Create a segemented library using `segemented.cpp`
-
 The principal (user) files are `gen.hpp` and `gen.cpp`.
 They contain includes for its various components: `components/<component_name>.<hpp/cpp>`
 
 Dependencies are bundled into `gen.dep.<hpp/cpp>`. They are included in `gen.<hpp/cpp>` before component includes.
-Just like the `gen.<hpp/cpp>` they include their components: `dependencies/<dependency_name>.<hpp/cpp>`. The auxillary content (builder & scanner) is given their own files.
+Just like the `gen.<hpp/cpp>` they include their components: `dependencies/<dependency_name>.<hpp/cpp>`
 
 If using the library's provided build scripts:
 

gen_segmented/segmented.cpp

@@ -2,6 +2,7 @@
 
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
+#define GEN_EXPOSE_BACKEND
 #define GEN_C_LIKE_CPP 1
 #include "gen.cpp"
 #include "helpers/push_ignores.inline.hpp"

gen_singleheader/singleheader.cpp

@@ -1,5 +1,6 @@
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
+#define GEN_EXPOSE_BACKEND
 #include "gen.cpp"
 #include "helpers/push_ignores.inline.hpp"
 

gen_unreal_engine/Readme.md

@@ -15,20 +15,3 @@ If using the library's provided build scripts:
 ```ps1
 .\build.ps1 <compiler> <debug or omit> unreal
 ```
-
-## Notables
-
-For the most part this follows the same conventions as `gen_segmented`.
-
-This generator uses a separate enumeration definitions for the following:
-
-* [AttributeTokens.csv](./enums/AttributeTokens.csv) : Add your own <MODULE>_API attributes, etc here that are encountered within the Engine.
-* [ESpecifier.csv](./enums/ESpecifier.csv) : Adds the `FORCEINLINE` & `FORCEINLINE_DEBUGGABLE` specfiers (additions are made as they are encountered)
-* [ETokType.csv](./enums/ETokType.csv) : Same modifications as ESpecifier.csv.
-
-A separate [parser_case_macros.cpp](./components/parser_case_macros.cpp) is used to accomodate for the new forceinline specifiers.
-
-The `global` macro the library uses is redefined as an empty substiution.
-
-
-The expected usage of this library is to put into into a third-party plugin module to then use either in editor modules or in shell script done in some stage of hot-reloading or building the Unreal Engine or Project.

gen_unreal_engine/components/parser_case_macros.cpp

@@ -1,6 +1,6 @@
 // These macros are used in the swtich cases within parser.cpp
 
-#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_TOK_SPECIFIER_CASES \
+#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_TOK_SPECIFIERS_CASES \
 case Tok_Spec_Consteval:              \
 case Tok_Spec_Constexpr:              \
 case Tok_Spec_Constinit:              \
@@ -14,7 +14,7 @@ case Tok_Spec_Static:                 \
 case Tok_Spec_Volatile:               \
 case Tok_Spec_Virtual
 
-#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_SPECIFIER_CASES \
+#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_SPECIFIERS_CASES \
 case Spec_Constexpr:              \
 case Spec_Constinit:              \
 case Spec_Explicit:               \
@@ -54,12 +54,12 @@ case Spec_NeverInline:            \
 case Spec_Static:                 \
 case Spec_Volatile
 
-#define GEN_PARSER_FRIEND_ALLOWED_SPECIFIER_CASES \
+#define GEN_PARSER_FRIEND_ALLOWED_SPECIFIERS_CASES \
 case Spec_Const:       \
 case Spec_Inline:      \
 case Spec_ForceInline
 
-#define GEN_PARSER_FUNCTION_ALLOWED_SPECIFIER_CASES \
+#define GEN_PARSER_FUNCTION_ALLOWED_SPECIFIERS_CASES \
 case Spec_Const:                  \
 case Spec_Consteval:              \
 case Spec_Constexpr:              \
@@ -71,7 +71,7 @@ case Spec_Inline:                 \
 case Spec_NeverInline:            \
 case Spec_Static
 
-#define GEN_PARSER_OPERATOR_ALLOWED_SPECIFIER_CASES \
+#define GEN_PARSER_OPERATOR_ALLOWED_SPECIFIERS_CASES \
 case Spec_Const:       \
 case Spec_Constexpr:   \
 case Spec_ForceInline: \
@@ -79,7 +79,7 @@ case Spec_Inline:      \
 case Spec_NeverInline: \
 case Spec_Static
 
-#define GEN_PARSER_TEMPLATE_ALLOWED_SPECIFIER_CASES \
+#define GEN_PARSER_TEMPLATE_ALLOWED_SPECIFIERS_CASES \
 case Spec_Const:                  \
 case Spec_Constexpr:              \
 case Spec_Constinit:              \

gen_unreal_engine/enums/ETokType.csv

@@ -13,8 +13,8 @@ BraceCurly_Open,             "{"
 BraceCurly_Close,            "}"
 BraceSquare_Open,            "["
 BraceSquare_Close,           "]"
-Paren_Open,                  "("
+Capture_Start,               "("
-Paren_Close,                 ")"
+Capture_End,                 ")"
 Comment,                     "__comment__"
 Comment_End,                 "__comment_end__"
 Comment_Start,               "__comment_start__"

gen_unreal_engine/unreal.cpp

@@ -1,5 +1,6 @@
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
+#define GEN_EXPOSE_BACKEND
 #include "gen.cpp"
 #include "helpers/push_ignores.inline.hpp"