Fixed swap fn

The entire project uses the namespace and it felt redundant.

There is a fix for array append_at. Finally got csv parsing working with it.

.vscode/c_cpp_properties.json

@@ -9,7 +9,7 @@
                 "_DEBUG",
                 "UNICODE",
                 "_UNICODE",
-                "gen_time"
+                "GEN_TIME"
             ],
             "windowsSdkVersion": "10.0.19041.0",
             "compilerPath": "C:/Users/Ed/scoop/apps/llvm/current/bin/clang++.exe",

.vscode/settings.json

@@ -23,5 +23,6 @@
 	"C_Cpp.intelliSenseEngineFallback": "disabled",
 	"mesonbuild.configureOnOpen": true,
 	"C_Cpp.errorSquiggles": "enabled",
-	"godot_tools.scene_file_config": ""
+	"godot_tools.scene_file_config": "",
+	"C_Cpp.default.compilerPath": "cl.exe"
 }

Readme.md

@@ -2,7 +2,7 @@
 
 An attempt at simple staged metaprogramming for c/c++.
 
-The library API is a compositon of code element constructors.  
+The library API is a composition of code element constructors.  
 These build up a code AST to then serialize with a file builder.
 
 ### TOC
@@ -18,14 +18,12 @@ These build up a code AST to then serialize with a file builder.
 * [On multithreading](#on-multi-threading)
 * [Extending the library](#extending-the-library)
 * [TODO](#todo)
+* [Thoughts](#thoughts)
 
 ## Notes
 
 The project has reached an *alpha* state, all the current functionality works for the test cases but it will most likely break in many other cases.
 
-Note: Do not trying to do any large generations with this (at least not without changing the serialization implementation).
-It does not resue any memory yet for dynamic strings and thus any signficant size memory will result in massive consumption. 
-
 The project has no external dependencies beyond:
 
 * `stdarg.h`
@@ -47,7 +45,7 @@ A C variant is hosted [here](https://github.com/Ed94/genc); I haven't gotten hea
 
 ## Usage
 
-A metaprogram is built to generate files before the main program is built. We'll term runtime for this program as `gen_time`. The metaprogram's core implementation are within `gen.hpp` and `gen.cpp` in the project directory.
+A metaprogram is built to generate files before the main program is built. We'll term runtime for this program as `GEN_TIME`. The metaprogram's core implementation are within `gen.hpp` and `gen.cpp` in the project directory.
 
 `gen.cpp` \`s  `main()` is defined as `gen_main()` which the user will have to define once for their program. There they will dictate everything that should be generated.
 
@@ -56,10 +54,9 @@ In order to keep the locality of this code within the same files the following p
 Within `program.cpp` :
 
 ```cpp
+#ifdef GEN_TIME
 #include "gen.hpp"
 
-#ifdef gen_time
-
 ...
 
 u32 gen_main()
@@ -68,8 +65,7 @@ u32 gen_main()
 }
 #endif
 
-
-#ifndef gen_time
+#ifndef GEN_TIME
 #include "program.gen.cpp"
 
     // Regular runtime dependent on the generated code here.
@@ -128,7 +124,7 @@ Code header = untyped_str( code(
 ));
 ```
 
-`name` is a helper macro for providing a string literal with its size, intended for the name paraemter of functions.  
+`name` is a helper macro for providing a string literal with its size, intended for the name parameter of functions.  
 `code` is a helper macro for providing a string literal with its size, but intended for code string parameters.  
 `args` is a helper macro for providing the number of arguments to varadic constructors.
 
@@ -152,7 +148,7 @@ An example of building is provided in the test directory.
 There are two meson build files the one within test is the program's build specification.  
 The other one in the gen directory within test is the metaprogram's build specification.
 
-Both of them build the same source file: `test.cpp`. The only differences are that gen needs a different relative path to the include directories and defines the macro definition: `gen_time`.
+Both of them build the same source file: `test.cpp`. The only differences are that gen needs a different relative path to the include directories and defines the macro definition: `GEN_TIME`.
 
 This method is setup where all the metaprogram's code are the within the same files as the regular program's code.  
 
@@ -187,7 +183,7 @@ Especially when the priority is to keep this library small and easy to grasp for
 
 When it comes to templates:
 
-Only trivial template support is provided. the intention is for only simple, non-recursive subsitution.  
+Only trivial template support is provided. the intention is for only simple, non-recursive substitution.  
 The parameters of the template are treated like regular parameter AST entries.  
 This means that the typename entry for the parameter AST would be either:
 
@@ -198,7 +194,7 @@ This means that the typename entry for the parameter AST would be either:
 Anything beyond this usage is not supported by parse_template for arguments (at least not intentionally).  
 Use at your own mental peril...
 
-*Concepts and Constraints are not supported, its usage is non-tirival substiution.*
+*Concepts and Constraints are not supported, its usage is non-trivial substitution.*
 
 ### The Data & Interface
 
@@ -280,21 +276,22 @@ Data Notes:
 
 * The allocator definitions used are exposed to the user incase they want to dictate memory usage
   * You'll find the memory handling in `init`, `gen_string_allocator`, `get_cached_string`, `make_code`.
-* ASTs are wrapped for the user in a Code struct which is a warpper for a AST* type.
+* ASTs are wrapped for the user in a Code struct which is a wrapper for a AST* type.
 * Both AST and Code have member symbols but their data layout is enforced to be POD types.
 * This library treats memory failures as fatal.
-* Strings are stored in their own set of arenas. AST constructors use cached strings for names, and content.
+* Cached Strings are stored in their own set of arenas. AST constructors use cached strings for names, and content.
   * `StringArenas`, `StringCache`, `Allocator_StringArena`, and `Allocator_StringTable` are the associated containers or allocators.
-* Strings used for seralization and file buffers are not contained by those used for cached strings.
-  * They are currently using `Memory::GlobalAllocator`, which are tracked array of arenas that grows as needed (adds buckets when one runs out).
-  * Memory within the buckets is not resused, so its inherently wasteful (most likely will give non-cached strings their own tailored allocator later)
+* Strings used for serialization and file buffers are not contained by those used for cached strings.
+  * They are currently using `GlobalAllocator`, which are tracked array of arenas that grows as needed (adds buckets when one runs out).
+  * Memory within the buckets is not reused, so its inherently wasteful.
+  * I will be augmenting the single arena with a simple slag allocator.
 * Linked lists used children nodes on bodies, and parameters.
-* Its intended to generate the AST in one go and serialize after. The contructors and serializer are designed to be a "one pass, front to back" setup.
+* Its intended to generate the AST in one go and serialize after. The constructors and serializer are designed to be a "one pass, front to back" setup.
 * When benchmarking, the three most significant var to tune are:
-  * `Memory::Global_BlockSize` (found gen_dep.hpp) : Used by the GlobalAllocator for the size of each global arena.
+  * `Global_BlockSize` (found gen_dep.hpp) : Used by the GlobalAllocator for the size of each global arena.
   * `SizePer_StringArena` (found in gen.hpp under the constants region) : Used by the string cache to store strings.
   * `CodePool_NumBlocks` (found in gen.hpp under constants region) : Used by code pool to store ASTs.
-  * The default values can handled generating for a stirng up to a size of ~650 kbs (bottleneck is serialization).
+  * The default values can handled generating for a string up to a size of ~650 kbs (bottleneck is serialization).
   * Increasing the values can generate files upwards of over a million lines without issue (the formatter will most likely run slower than it)
 
 Two generic templated containers are used throughout the library:
@@ -302,11 +299,11 @@ Two generic templated containers are used throughout the library:
 * `template< class Type> struct Array`
 * `template< class Type> struct HashTable`
 
-Both Code and AST definitions have a `template< class Type> Code/AST cast()`. Its just an alternative way to explictly cast to each other.
+Both Code and AST definitions have a `template< class Type> Code/AST cast()`. Its just an alternative way to explicitly cast to each other.
 
 Otherwise the library is free of any templates.
 
-The following CodeTypes are used which the user may optionally use strong typeing with if they enable: `GEN_ENFORCE_STRONG_CODE_TYPES`
+The following CodeTypes are used which the user may optionally use strong typing with if they enable: `GEN_ENFORCE_STRONG_CODE_TYPES`
 
 * CodeBody : Has support for `for-range` iterating across Code objects.
 * CodeAttributes
@@ -322,7 +319,7 @@ The following CodeTypes are used which the user may optionally use strong typein
 * CodeNamespace
 * CodeOperator
 * CodeOpCast
-* CodeParam : Has suppor for `for-range` iterating across parameters.
+* CodeParam : Has support for `for-range` iterating across parameters.
 * CodeSpecifier : Has support for `for-range` iterating across specifiers.
 * CodeStruct
 * CodeTemplate
@@ -333,9 +330,9 @@ The following CodeTypes are used which the user may optionally use strong typein
 * CodeUsingNamespace
 * CodeVar
 
-Each Code boy has an assoicated "filtered AST" with the naming convention: `AST_<CodeName>`
-Unrelated fields of the AST for that node type are omitted and only necesary padding members are defined otherwise.
-Retreiving a raw version of the ast can be done using the `raw()` function defined in each AST.
+Each Code boy has an associated "filtered AST" with the naming convention: `AST_<CodeName>`
+Unrelated fields of the AST for that node type are omitted and only necessary padding members are defined otherwise.
+Retrieving a raw version of the ast can be done using the `raw()` function defined in each AST.
 
 ## There are three sets of interfaces for Code AST generation the library provides
 
@@ -351,7 +348,7 @@ The construction will fail and return Code::Invalid otherwise.
 Interface :``
 
 * def_attributes
-  * *This is preappened right before the function symbol, or placed after the class or struct keyword for any flavor of attributes used.*
+  * *This is pre-appended right before the function symbol, or placed after the class or struct keyword for any flavor of attributes used.*
   * *Its up to the user to use the desired attribute formatting: `[[]]` (standard), `__declspec` (Microsoft), or `__attribute__` (GNU).*
 * def_comment
 * def_class
@@ -451,7 +448,7 @@ The lexing and parsing takes shortcuts from whats expected in the standard.
   * Assumed to *come before specifiers* (`const`, `constexpr`, `extern`, `static`, etc) for a function
   * Or in the usual spot for class, structs, (*right after the declaration keyword*)
   * typedefs have attributes with the type (`parse_type`)
-* As a general rule; if its not available from the upfront contructors, its not available in the parsing constructors.
+* As a general rule; if its not available from the upfront constructors, its not available in the parsing constructors.
   * *Upfront constructors are not necessarily used in the parsing constructors, this is just a good metric to know what can be parsed.*
 
 Usage:
@@ -494,9 +491,18 @@ These restrictions help prevent abuse of untyped code to some extent.
 Usage Conventions:
 
 ```cpp
-Code <name> = def_varaible( <type>, <name>, untyped_<function name>(
+Code <name> = def_variable( <type>, <name>, untyped_<function name>(
     <string with code>
 ));
+
+Code <name> = untyped_str( code(
+    <some code without "" quotes>
+));
+```
+
+Optionally, `code_str`, and `code_fmt` macros can be used so that the code macro doesn't have to be used:
+```cpp
+Code <name> = code_str( <some code without "" quotes > )
 ```
 
 Template metaprogramming in the traditional sense becomes possible with the use of `token_fmt` and parse constructors:
@@ -592,9 +598,9 @@ Editor and Scanner are disabled by default, use `GEN_FEATURE_EDITOR` and `GEN_FE
 ### Builder is a similar object to the jai language's string_builder
 
 * The purpose of it is to generate a file.
-* A file is specified and opened for writting using the open( file_path) ) function.
-* The code is provided via print( code ) function  will be seralized to its buffer.
-* When all seralization is finished, use the write() command to write the buffer to the file.
+* A file is specified and opened for writing using the open( file_path) ) function.
+* The code is provided via print( code ) function  will be serialized to its buffer.
+* When all serialization is finished, use the write() command to write the buffer to the file.
 
 ### Editor is for editing a series of files based on a set of requests provided to it
 
@@ -627,7 +633,7 @@ It will on call add a request to the queue to run the refactor script on the fil
 * The purpose is to grab definitions to generate metadata or generate new code from these definitions.
 * Requests are populated using the add( SymbolInfo, Policy ) function. The symbol info is the same as the one used for the editor. So is the case with Policy.
 
-The file will only be read from, no writting supported.
+The file will only be read from, no writing supported.
 
 One great use case is for example: generating the single-header library for gencpp!
 
@@ -645,10 +651,10 @@ Request queue in both Editor and Scanner are cleared once process_requests compl
 
 Currently unsupported. The following changes would have to be made:
 
-* Setup static data accesss with fences if more than one thread will generate ASTs ( or keep a different set for each thread)
-* Make sure local peristent data of functions are also thread local.
+* Setup static data access with fences if more than one thread will generate ASTs ( or keep a different set for each thread)
+* Make sure local persistent data of functions are also thread local.
 * The builder should be done on a per-thread basis.
-* Due to the design of the editor and scanner, it will most likely be best to make each file a job to process request entries on. Receipts should have an an array to store per thread. They can be combined to the final reciepts array when all files have been processed.
+* Due to the design of the editor and scanner, it will most likely be best to make each file a job to process request entries on. Receipts should have an an array to store per thread. They can be combined to the final receipts array when all files have been processed.
 
 ## Extending the library
 
@@ -669,21 +675,39 @@ The convention you'll see used throughout the API of the library is as follows:
 1. Check name or parameters to make sure they are valid for the construction requested
 2. Create a code object using `make_code`.
 3. Populate immediate fields (Name, Type, ModuleFlags, etc)
-4. Populate sub-entires using `add_entry`. If using the default seralization function `to_string`, follow the order at which entires are expected to appear (there is a strong ordering expected).
+4. Populate sub-entires using `add_entry`. If using the default serialization function `to_string`, follow the order at which entires are expected to appear (there is a strong ordering expected).
 
 Names or Content fields are interned strings and thus showed be cached using `get_cached_string` if its desired to preserve that behavior.
 
-`def_operator` is the most sophisitacated constructor as it has multiple permutations of definitions that could be created that are not trivial to determine if valid.
+`def_operator` is the most sophisticated constructor as it has multiple permutations of definitions that could be created that are not trivial to determine if valid.
 
 # TODO
 
-* Implement a context stack for the parsing, allows for accurate scope validation for the AST types.
-* Make a more robust test suite.
-* Generate a single-header library.
-* Improve the allocation strategy for strings in `Builder`, `AST::to_string`, `Parser::lex`, all three can use some form of slab allocation strategy...
-  * Can most likely use a simple slag allocator.
-* May be in need of a better name, I found a few repos with this same one...
+* Support defining & parsing full definitions inside a typedef. (For C patterns)
 * Support module and attribute parsing (Marked with TODOs for now..)
-* Suffix specifiers for functions (const, override, final)
+* Implement a context stack for the parsing, allows for accurate scope validation for the AST types.
+* Trailing specifiers ( postfix ) for functions (const, override, final)
+* Make a more robust test suite.
+* Generate a single-header library
+  * Componetize the library, make a metaprogram using gencpp to bootstrap itself.
 * Implement the Scanner
 * Implement the Editor
+* Should the builder be an "extension" header?
+  * Technically the library doesn't require it and the user can use their own filesystem library.
+  * It would allow me to remove the filesystem dependencies and related functions outside of gen base headers. ( At least 1k loc reduced )
+  * ADT and the CSV parser depend on it as well. The `str_fmt_file` related functions do as well but they can be ommited.
+  * Scanner and editor will also depend on it so they would need to include w/e the depency header for all three file-interacting interfaces.
+    * `gen.dep.files.hpp`
+* Convert GlobalAllocator to a slab allocator:
+
+```md
+Slab classes (for 10 mb arena)
+
+1KB   slab: 5MB  ( 5MB  / 1KB   ~ 5,000 blocks )
+4KB   slab: 10MB ( 10MB / 4KB   ~ 2,500 blocks )
+16KB  slab: 15MB ( 15MB / 16KB  ~ 960 blocks   )
+64KB  slab: 15MB ( 15MB / 64KB  ~ 240 blocks   )
+256KB slab: 20MB ( 20MB / 256KB ~ 80 blocks    )
+512KB slab: 20MB ( 20MB / 512KB ~ 40 blocks    )
+1MB   slab: 15MB ( 15MB / 1MB   ~ 15 blocks    )
+```

gencpp.vcxproj

@@ -79,7 +79,7 @@
     <NMakeBuildCommandLine>powershell.exe -ExecutionPolicy Unrestricted -File "$(ProjectDir)scripts\build.ps1"</NMakeBuildCommandLine>
     <NMakeReBuildCommandLine>powershell.exe -ExecutionPolicy Unrestricted -File "$(ProjectDir)scripts\build.ps1"</NMakeReBuildCommandLine>
     <NMakeCleanCommandLine>powershell.exe -ExecutionPolicy Unrestricted -File "$(ProjectDir)scripts\clean.ps1"</NMakeCleanCommandLine>
-    <NMakePreprocessorDefinitions>gen_time;$(NMakePreprocessorDefinitions)</NMakePreprocessorDefinitions>
+    <NMakePreprocessorDefinitions>GEN_TIME;$(NMakePreprocessorDefinitions)</NMakePreprocessorDefinitions>
     <IncludePath>$(ProjectDir)thirdparty;$(ProjectDir)project;$(ProjectDir)test;$(IncludePath)</IncludePath>
     <SourcePath>$(ProjectDir)project;$(ProjectDir)test;$(SourcePath)</SourcePath>
   </PropertyGroup>
@@ -87,7 +87,7 @@
     <NMakeBuildCommandLine>powershell.exe -ExecutionPolicy Unrestricted -File "$(ProjectDir)scripts\build.ps1"</NMakeBuildCommandLine>
     <NMakeReBuildCommandLine>powershell.exe -ExecutionPolicy Unrestricted -File "$(ProjectDir)scripts\build.ps1"</NMakeReBuildCommandLine>
     <NMakeCleanCommandLine>powershell.exe -ExecutionPolicy Unrestricted -File "$(ProjectDir)scripts\clean.ps1"</NMakeCleanCommandLine>
-    <NMakePreprocessorDefinitions>gen_time;$(NMakePreprocessorDefinitions)</NMakePreprocessorDefinitions>
+    <NMakePreprocessorDefinitions>GEN_TIME;$(NMakePreprocessorDefinitions)</NMakePreprocessorDefinitions>
     <IncludePath>$(ProjectDir)thirdparty;$(ProjectDir)project;$(ProjectDir)test;$(IncludePath)</IncludePath>
     <SourcePath>$(ProjectDir)project;$(ProjectDir)test;$(SourcePath)</SourcePath>
   </PropertyGroup>

project/Readme.md

@@ -1,22 +1,24 @@
 # Documentation
 
-All the library code is contained in two files: `gen.hpp` and `gen.cpp`
+The core library is contained within `gen.hpp` and `gen.cpp`.
+Things related to the editor and scanner are in their own respective files. (Ex: `gen.scanner.<hpp/cpp>` )
+
+
 
 ## gen.hpp
 
 Feature Macros:
 
+* `GEN_DONT_ENFORCE_GEN_TIME_GUARD` : By default, the library ( gen.hpp/ gen.cpp ) expects the macro `GEN_TIME` to be defined, this disables that.
 * `GEN_ROLL_OWN_DEPENDENCIES` : Optional override so that user may define the dependencies themselves.
 * `GEN_DEFINE_LIBRARY_CORE_CONSTANTS` : Optional typename codes as they are non-standard to C/C++ and not necessary to library usage
 * `GEN_ENFORCE_STRONG_CODE_TYPES` : Enforces casts to filtered code types.
-* `GEN_FEATURE_PARSING` : Defines the parse constructors
-* `GEN_FEATURE_EDITOR` : Defines the file editing features for changing definitions based on ASTs
-* `GEN_FEATURE_SCANNER` : Defines the file scanning features for generating ASTs
 
 `GEN_USE_RECURSIVE_AST_DUPLICATION` is available but its not well tested and should not need to be used.  
 If constructing ASTs properly. There should be no modification of ASTs, and thus this would never become an issue.  
+(I will probably remove down the line...)
 
-Due to the design of `gen.hpp` to support being written alongside runtime intended code (in the same file), all the code is wrapped in a `gen_time` `#ifdef` and then wrapped further in a `gen` namespace to avoid pollution of the global scope.
+Due to the design of `gen.hpp` to support being written alongside runtime intended code (in the same file), all the code is wrapped in a `GEN_TIME` `#ifdef` and then wrapped further in a `gen` namespace to avoid pollution of the global scope.
 
 *Note: Its possible with the scanner feature to support parsing runtime files that use "generic" macros or identifiers with certain patterns.  
 This can be used to auto-queue generation of dependent definitions for the symbols used.*
@@ -30,27 +32,27 @@ log_failure definition : based on whether to always use fatal on all errors
 Major enum definitions and their associated functions used with the AST data
 
 * `ECode` : Used to tag ASTs by their type
-* `EOperator` : Used to tag operator overloads with thier op type
+* `EOperator` : Used to tag operator overloads with their op type
 * `ESpecifier` : Used with specifier ASTs for all specifiers the user may tag an associated
 AST with.
 * `AccessSpec` : Used with class and struct ASTs to denote the public, protected, or private fields.
 * `EnumT` : Used with def_enum to determine if constructing a regular enum or an enum class.
-* `ModuleFlag` : Used with any valid definition that can have export or import related keywords assoicated with it.
+* `ModuleFlag` : Used with any valid definition that can have export or import related keywords associated with it.
 
 #### Data Structures
 
 `StringCache` : Hash table for cached strings. (`StringCached` typedef used to denote strings managed by it)
 
 `Code` : Wrapper for `AST` with functionality for handling it appropriately.  
-`AST` : The node data strucuture for the code.  
+`AST` : The node data structure for the code.  
 `Code Types` : Codes with typed ASTs. Body, Param, and Specifier have unique implementation, the rest use `Define_CodeType`  
 `AST Types` : Filtered AST definitions.  
 
 #### Gen Interface
 
-First set of fowards are either backend functions used for various aspects of AST generation or configurating allocators used for different containers.
+First set of forwards are either backend functions used for various aspects of AST generation or configurations allocators used for different containers.
 
-Interface fowards defined in order of: Upfront, Parsing, Untyped.
+Interface forwards defined in order of: Upfront, Parsing, Untyped.
 
 From there forwards for the File handlers are defined: Builder, Editor, Scanner.
 

project/gen.cpp

@@ -1,25 +1,31 @@
 // ReSharper disable CppClangTidyClangDiagnosticSwitchEnum
 
-#ifdef gen_time
+#if ! defined(GEN_DONT_ENFORCE_GEN_TIME_GUARD) && ! defined(GEN_TIME)
+#error Gen.hpp : GEN_TIME not defined
+#endif
+
+#include "gen.push_ignores.inline.hpp"
+
 //! If its desired to roll your own dependencies, define GEN_ROLL_OWN_DEPENDENCIES before including this file.
 //! Dependencies are derived from the c-zpl library: https://github.com/zpl-c/zpl
 #ifndef GEN_ROLL_OWN_DEPENDENCIES
-#	include "gen_dep.cpp"
+#	include "gen.dep.cpp"
 #endif
 
 #include "gen.hpp"
 
-namespace gen
-{
-	namespace StaticData
-	{
+namespace gen {
+
+#pragma region StaticData
+// TODO : Convert global allocation strategy to use the dual-scratch allocator for a contextual scope.
+global AllocatorInfo  GlobalAllocator;
+global Array<Arena>   Global_AllocatorBuckets;
+
 global Array< Pool >  CodePools         = { nullptr };
 global Array< Arena > StringArenas      = { nullptr };
 
 global StringTable StringCache;
 
-		// TODO : Need to implement String memory management for seriaization intermediates.
-
 global Arena LexArena;
 
 global AllocatorInfo Allocator_DataArrays       = heap();
@@ -28,7 +34,7 @@ namespace gen
 global AllocatorInfo Allocator_StringArena      = heap();
 global AllocatorInfo Allocator_StringTable      = heap();
 global AllocatorInfo Allocator_TypeTable        = heap();
-	}
+#pragma endregion StaticData
 
 #pragma region Constants
 global CodeType t_auto;
@@ -169,6 +175,7 @@ namespace gen
 	mem_copy( result, this, sizeof( AST ) );
 	result->Parent = nullptr;
 #else
+	// TODO : Stress test this...
 	switch ( Type )
 	{
 		case Invalid:
@@ -391,7 +398,7 @@ namespace gen
 #endif
 
 	// TODO : Need to refactor so that intermeidate strings are freed conviently.
-		String result = String::make( Memory::GlobalAllocator, "" );
+	String result = String::make( GlobalAllocator, "" );
 
 	switch ( Type )
 	{
@@ -582,7 +589,7 @@ namespace gen
 		{
 			result.append_fmt( "export\n{\n" );
 
-				Code curr = cast<Code>();
+			Code curr = { this };
 			s32  left = NumEntries;
 			while ( left-- )
 			{
@@ -751,7 +758,7 @@ namespace gen
 
 			if ( NumEntries - 1 > 0)
 			{
-					for ( CodeParam param : Next->cast<CodeParam>() )
+				for ( CodeParam param :  CodeParam { (AST_Param*) Next } )
 				{
 					result.append_fmt( ", %s", param.to_string() );
 				}
@@ -849,7 +856,7 @@ namespace gen
 
 			result.append_fmt( "%s %s", UnderlyingType->to_string(), Name );
 
-				if ( UnderlyingType->ArrExpr )
+			if ( UnderlyingType->Type == Typename && UnderlyingType->ArrExpr )
 			{
 				result.append_fmt( "[%s];", UnderlyingType->ArrExpr->to_string() );
 			}
@@ -1087,54 +1094,70 @@ namespace gen
 #pragma endregion AST
 
 #pragma region Gen Interface
-	void init()
+internal void* Global_Allocator_Proc( void* allocator_data, AllocType type, sw size, sw alignment, void* old_memory, sw old_size, u64 flags )
 {
-		using namespace StaticData;
+	Arena* last = & Global_AllocatorBuckets.back();
 
-		Memory::setup();
-
-		// Setup the arrays
+	switch ( type )
 	{
-			CodePools = Array<Pool>::init_reserve( Allocator_DataArrays, InitSize_DataArrays );
+		case EAllocation_ALLOC:
+		{
+			if ( ( last->TotalUsed + size ) > last->TotalSize )
+			{
+				Arena bucket = Arena::init_from_allocator( heap(), Global_BucketSize );
 
-			if ( CodePools == nullptr )
-				fatal( "gen::init: Failed to initialize the CodePools array" );
+				if ( bucket.PhysicalStart == nullptr )
+					fatal( "Failed to create bucket for Global_AllocatorBuckets");
 
-			StringArenas = Array<Arena>::init_reserve( Allocator_DataArrays, InitSize_DataArrays );
+				if ( ! Global_AllocatorBuckets.append( bucket ) )
+					fatal( "Failed to append bucket to Global_AllocatorBuckets");
 
-			if ( StringArenas == nullptr )
-				fatal( "gen::init: Failed to initialize the StringArenas array" );
+				last = & Global_AllocatorBuckets.back();
 			}
 
-		// Setup the code pool and code entries arena.
+			return alloc_align( * last, size, alignment );
+		}
+		case EAllocation_FREE:
 		{
-			Pool code_pool = Pool::init( Allocator_CodePool, CodePool_NumBlocks, sizeof(AST) );
+			// Doesn't recycle.
+		}
+		break;
+		case EAllocation_FREE_ALL:
+		{
+			// Memory::cleanup instead.
+		}
+		break;
+		case EAllocation_RESIZE:
+		{
+			if ( last->TotalUsed + size > last->TotalSize )
+			{
+				Arena bucket = Arena::init_from_allocator( heap(), Global_BucketSize );
 
-			if ( code_pool.PhysicalStart == nullptr )
-				fatal( "gen::init: Failed to initialize the code pool" );
+				if ( bucket.PhysicalStart == nullptr )
+					fatal( "Failed to create bucket for Global_AllocatorBuckets");
 
-			CodePools.append( code_pool );
+				if ( ! Global_AllocatorBuckets.append( bucket ) )
+					fatal( "Failed to append bucket to Global_AllocatorBuckets");
 
-		#ifdef GEN_FEATURE_PARSING
-			LexArena = Arena::init_from_allocator( Allocator_Lexer, LexAllocator_Size );
-		#endif
-
-			Arena string_arena = Arena::init_from_allocator( Allocator_StringArena, SizePer_StringArena );
-
-			if ( string_arena.PhysicalStart == nullptr )
-				fatal( "gen::init: Failed to initialize the string arena" );
-
-			StringArenas.append( string_arena );
+				last = & Global_AllocatorBuckets.back();
 			}
 
-		// Setup the hash tables
-		{
-			StringCache = StringTable::init( Allocator_StringTable );
+			void* result = alloc_align( last->Backing, size, alignment );
 
-			if ( StringCache.Entries == nullptr )
-				fatal( "gen::init: Failed to initialize the StringCache");
+			if ( result != nullptr && old_memory != nullptr )
+			{
+				mem_copy( result, old_memory, old_size );
 			}
 
+			return result;
+		}
+	}
+
+	return nullptr;
+}
+
+internal void define_constants()
+{
 	Code::Global          = make_code();
 	Code::Global->Name    = get_cached_string( txt_StrC("Global Code") );
 	Code::Global->Content = Code::Global->Name;
@@ -1212,9 +1235,9 @@ namespace gen
 #	pragma push_macro( "global" )
 #	pragma push_macro( "internal" )
 #	pragma push_macro( "local_persist" )
-	#	define global        global
-	#	define internal      internal
-	#	define local_persist local_persist
+#	undef global
+#	undef internal
+#	undef local_persist
 
 #	define def_constant_spec( Type_, ... )                                    \
 	spec_##Type_ = def_specifiers( num_args(__VA_ARGS__), __VA_ARGS__); \
@@ -1248,12 +1271,73 @@ namespace gen
 #	undef def_constant_spec
 }
 
+void init()
+{
+	// Setup global allocator
+	{
+		GlobalAllocator = AllocatorInfo { & Global_Allocator_Proc, nullptr };
+
+		Global_AllocatorBuckets = Array<Arena>::init_reserve( heap(), 128 );
+
+		if ( Global_AllocatorBuckets == nullptr )
+			fatal( "Failed to reserve memory for Global_AllocatorBuckets");
+
+		Arena bucket = Arena::init_from_allocator( heap(), Global_BucketSize );
+
+		if ( bucket.PhysicalStart == nullptr )
+			fatal( "Failed to create first bucket for Global_AllocatorBuckets");
+
+		Global_AllocatorBuckets.append( bucket );
+
+	}
+
+	// Setup the arrays
+	{
+		CodePools = Array<Pool>::init_reserve( Allocator_DataArrays, InitSize_DataArrays );
+
+		if ( CodePools == nullptr )
+			fatal( "gen::init: Failed to initialize the CodePools array" );
+
+		StringArenas = Array<Arena>::init_reserve( Allocator_DataArrays, InitSize_DataArrays );
+
+		if ( StringArenas == nullptr )
+			fatal( "gen::init: Failed to initialize the StringArenas array" );
+	}
+
+	// Setup the code pool and code entries arena.
+	{
+		Pool code_pool = Pool::init( Allocator_CodePool, CodePool_NumBlocks, sizeof(AST) );
+
+		if ( code_pool.PhysicalStart == nullptr )
+			fatal( "gen::init: Failed to initialize the code pool" );
+
+		CodePools.append( code_pool );
+
+		LexArena = Arena::init_from_allocator( Allocator_Lexer, LexAllocator_Size );
+
+		Arena string_arena = Arena::init_from_allocator( Allocator_StringArena, SizePer_StringArena );
+
+		if ( string_arena.PhysicalStart == nullptr )
+			fatal( "gen::init: Failed to initialize the string arena" );
+
+		StringArenas.append( string_arena );
+	}
+
+	// Setup the hash tables
+	{
+		StringCache = StringTable::init( Allocator_StringTable );
+
+		if ( StringCache.Entries == nullptr )
+			fatal( "gen::init: Failed to initialize the StringCache");
+	}
+
+	define_constants();
+}
+
 void deinit()
 {
-		using namespace StaticData;
-
-		s32 index = 0;
-		s32 left  = CodePools.num();
+	uw index = 0;
+	uw left  = CodePools.num();
 	do
 	{
 		Pool* code_pool = & CodePools[index];
@@ -1277,17 +1361,50 @@ namespace gen
 	CodePools.free();
 	StringArenas.free();
 
-	#ifdef GEN_FEATURE_PARSING
 	LexArena.free();
-	#endif
 
-		Memory::cleanup();
+	index = 0;
+	left  = Global_AllocatorBuckets.num();
+	do
+	{
+		Arena* bucket = & Global_AllocatorBuckets[ index ];
+		bucket->free();
+		index++;
+	}
+	while ( left--, left );
+
+	Global_AllocatorBuckets.free();
+}
+
+void reset()
+{
+	s32 index = 0;
+	s32 left  = CodePools.num();
+	do
+	{
+		Pool* code_pool = & CodePools[index];
+		code_pool->clear();
+		index++;
+	}
+	while ( left--, left );
+
+	index = 0;
+	left  = StringArenas.num();
+	do
+	{
+		Arena* string_arena = & StringArenas[index];
+		string_arena->TotalUsed = 0;;
+		index++;
+	}
+	while ( left--, left );
+
+	StringCache.clear();
+
+	define_constants();
 }
 
 AllocatorInfo get_string_allocator( s32 str_length )
 {
-		using namespace StaticData;
-
 	Arena* last = & StringArenas.back();
 
 	uw size_req = str_length + sizeof(String::Header) + sizeof(char*);
@@ -1308,8 +1425,6 @@ namespace gen
 // Will either make or retrive a code string.
 StringCached get_cached_string( StrC str )
 {
-		using namespace StaticData;
-
 	s32 hash_length = str.Len > kilobytes(1) ? kilobytes(1) : str.Len;
 	u64 key         = crc32( str.Ptr, hash_length );
 	{
@@ -1331,8 +1446,6 @@ namespace gen
 */
 Code make_code()
 {
-		using namespace StaticData;
-
 	Pool* allocator = & CodePools.back();
 	if ( allocator->FreeList == nullptr )
 	{
@@ -1693,30 +1806,30 @@ namespace gen
 
 void set_allocator_data_arrays( AllocatorInfo allocator )
 {
-		StaticData::Allocator_DataArrays = allocator;
+	Allocator_DataArrays = allocator;
 }
 
 void set_allocator_code_pool( AllocatorInfo allocator )
 {
-		StaticData::Allocator_CodePool = allocator;
+	Allocator_CodePool = allocator;
 }
 
 void set_allocator_lexer( AllocatorInfo allocator )
 {
-		StaticData::Allocator_Lexer = allocator;
+	Allocator_Lexer = allocator;
 }
 
 void set_allocator_string_arena( AllocatorInfo allocator )
 {
-		StaticData::Allocator_StringArena = allocator;
+	Allocator_StringArena = allocator;
 }
 
 void set_allocator_string_table( AllocatorInfo allocator )
 {
-		StaticData::Allocator_StringArena = allocator;
+	Allocator_StringArena = allocator;
 }
 
-#pragma region Helper Marcos
+#pragma region Helper Marcojs
 // This snippet is used in nearly all the functions.
 #define name_check( Context_, Name_ )                                                                 \
 {                                                                                                     \
@@ -2420,7 +2533,7 @@ namespace gen
 	return result;
 }
 
-	CodeTypedef def_typedef( StrC name, CodeType type, CodeAttributes attributes, ModuleFlag mflags )
+CodeTypedef def_typedef( StrC name, Code type, CodeAttributes attributes, ModuleFlag mflags )
 {
 	name_check( def_typedef, name );
 	null_check( def_typedef, type );
@@ -3183,17 +3296,17 @@ namespace gen
 #pragma endregion Upfront Constructors
 
 #pragma region Parsing Constructors
-#ifdef GEN_FEATURE_PARSING
 /*
-	These constructors are the most implementation intensive other than the edtior or scanner.
+These constructors are the most implementation intensive other than the editor or scanner.
 */
 
 namespace Parser
 {
-
 /*
 	This is a simple lexer that focuses on tokenizing only tokens relevant to the library.
 	It will not be capable of lexing C++ code with unsupported features.
+
+	For the sake of scanning files, it can scan preprocessor directives
 */
 
 #	define Define_TokType \
@@ -3226,11 +3339,13 @@ namespace gen
 	Entry( Decl_Typedef,           "typedef" )         \
 	Entry( Decl_Using,             "using" )           \
 	Entry( Decl_Union,             "union" )           \
-		Entry( Identifier,            "__SymID__" )       \
+	Entry( Identifier,             "__identifier__" )  \
 	Entry( Module_Import,          "import" )          \
 	Entry( Module_Export,          "export" )          \
 	Entry( Number,                 "number" )          \
 	Entry( Operator,               "operator" )        \
+	Entry( Preprocessor_Directive, "#")                \
+	Entry( Preprocessor_Include,   "include" )         \
 	Entry( Spec_Alignas,           "alignas" )         \
 	Entry( Spec_Const,             "const" )           \
 	Entry( Spec_Consteval,         "consteval" )       \
@@ -3247,7 +3362,7 @@ namespace gen
 	Entry( Spec_Volatile,          "volatile")         \
 	Entry( Star,                   "*" )               \
 	Entry( Statement_End,          ";" )               \
-		Entry( String,                "__String__" )      \
+	Entry( String,                 "__string__" )      \
 	Entry( Type_Unsigned, 	       "unsigned" )        \
 	Entry( Type_Signed,            "signed" )          \
 	Entry( Type_Short,             "short" )           \
@@ -3368,7 +3483,7 @@ namespace gen
 
 			if ( Arr[Idx].Type != type )
 			{
-					String token_str = String::make( Memory::GlobalAllocator, { Arr[Idx].Length, Arr[Idx].Text } );
+				String token_str = String::make( GlobalAllocator, { Arr[Idx].Length, Arr[Idx].Text } );
 
 				log_failure( "gen::%s: expected %s, got %s", context, str_tok_type(type), str_tok_type(Arr[Idx].Type) );
 
@@ -3395,7 +3510,7 @@ namespace gen
 		}
 	};
 
-		TokArray lex( StrC content )
+	TokArray lex( StrC content, bool keep_preprocess_directives = false )
 	{
 	#	define current ( * scanner )
 
@@ -3441,7 +3556,7 @@ namespace gen
 			Tokens.free();
 		}
 
-			Tokens = Array<Token>::init_reserve( StaticData::LexArena, content.Len / 6 );
+		Tokens = Array<Token>::init_reserve( LexArena, content.Len / 6 );
 
 		while (left )
 		{
@@ -3453,6 +3568,29 @@ namespace gen
 
 			switch ( current )
 			{
+				case '#':
+					token.Text   = scanner;
+					token.Length = 1;
+					token.Type   = TokType::Preprocessor_Directive;
+					move_forward();
+
+					while (left && current != '\n' )
+					{
+						if ( current == '\\'  )
+						{
+							move_forward();
+
+							if ( current != '\n' && keep_preprocess_directives )
+							{
+								log_failure( "gen::lex: invalid preprocessor directive, will still grab but will not compile %s", token.Text );
+							}
+						}
+
+						move_forward();
+						token.Length++;
+					}
+					goto FoundToken;
+
 				case '.':
 					token.Text   = scanner;
 					token.Length = 1;
@@ -3826,7 +3964,7 @@ namespace gen
 			}
 			else
 			{
-					String context_str = String::fmt_buf( Memory::GlobalAllocator, "%s", scanner, min( 100, left ) );
+				String context_str = String::fmt_buf( GlobalAllocator, "%s", scanner, min( 100, left ) );
 
 				log_failure( "Failed to lex token %s", context_str );
 
@@ -3841,6 +3979,9 @@ namespace gen
 
 			if ( token.Type != TokType::Invalid )
 			{
+				if ( token.Type == TokType::Preprocessor_Directive && keep_preprocess_directives == false )
+					continue;
+
 				Tokens.append( token );
 				continue;
 			}
@@ -3848,10 +3989,7 @@ namespace gen
 			TokType type = get_tok_type( token.Text, token.Length );
 
 			if ( type == TokType::Invalid)
-				{
-					// Its most likely an identifier...
 				type = TokType::Identifier;
-				}
 
 			token.Type = type;
 			Tokens.append( token );
@@ -3893,6 +4031,12 @@ namespace gen
 #	define check( Type_ ) ( left && currtok.Type == Type_ )
 #pragma endregion Helper Macros
 
+struct ParseContext
+{
+	ParseContext* Parent;
+	char const*   Fn;
+};
+
 internal Code parse_function_body    ( Parser::TokArray& toks, char const* context );
 internal Code parse_global_nspace    ( Parser::TokArray& toks, char const* context );
 
@@ -4968,13 +5112,13 @@ namespace gen
 
 			case TokType::Module_Import:
 			{
-					not_implemented();
+				not_implemented( context );
 			}
 			//! Fallthrough intentional
 			case TokType::BraceSquare_Open:
 			case TokType::Attr_Keyword:
 			{
-					not_implemented();
+				not_implemented( context );
 
 				// Standard attribute
 				if ( currtok.Type == TokType::BraceSquare_Open  )
@@ -5754,6 +5898,7 @@ namespace gen
 	}
 
 	if (   currtok.Type == TokType::Decl_Class
+		|| currtok.Type == TokType::Decl_Enum
 		|| currtok.Type == TokType::Decl_Struct )
 	{
 		name = currtok;
@@ -5909,11 +6054,21 @@ namespace gen
 
 	Token name       = { nullptr, 0, TokType::Invalid };
 	Code  array_expr = { nullptr };
-		CodeType type       = { nullptr };
+	Code  type       = { nullptr };
 
 	eat( TokType::Decl_Typedef );
 
-		type = parse_type( toks, stringize(parse_typedef) );
+	if ( check( TokType::Decl_Enum ) )
+		type = parse_enum( toks, context );
+
+	else if ( check(TokType::Decl_Struct ) )
+		type = parse_enum( toks, context );
+
+	else if ( check(TokType::Decl_Union) )
+		type = parse_union( toks, context );
+
+	else
+		type = parse_type( toks, context );
 
 	if ( ! check( TokType::Identifier ) )
 	{
@@ -5924,7 +6079,7 @@ namespace gen
 	name = currtok;
 	eat( TokType::Identifier );
 
-		array_expr = parse_array_decl( toks, stringize(parse_typedef) );
+	array_expr = parse_array_decl( toks, context );
 
 	eat( TokType::Statement_End );
 
@@ -5937,8 +6092,8 @@ namespace gen
 
 	result->UnderlyingType = type;
 
-		if ( array_expr && array_expr->Type != Invalid )
-			type->ArrExpr = array_expr;
+	if ( type->Type == Typename && array_expr && array_expr->Type != Invalid )
+		type.cast<CodeType>()->ArrExpr = array_expr;
 
 	return result;
 }
@@ -5984,7 +6139,7 @@ namespace gen
 
 	while ( ! check( TokType::BraceCurly_Close ) )
 	{
-			Code entry = parse_variable( toks, stringize(parse_union) );
+		Code entry = parse_variable( toks, context );
 
 		if ( entry )
 			body.append( entry );
@@ -6054,10 +6209,10 @@ namespace gen
 
 		eat( TokType::Operator );
 
-			type = parse_type( toks, stringize(parse_typedef) );
+		type = parse_type( toks, context );
 	}
 
-		array_expr = parse_array_decl( toks, stringize(parse_typedef) );
+	array_expr = parse_array_decl( toks, context );
 
 	eat( TokType::Statement_End );
 
@@ -6151,7 +6306,7 @@ namespace gen
 		specifiers = def_specifiers( num_specifiers, specs_found );
 	}
 
-		CodeType type = parse_type( toks, stringize(parse_variable) );
+	CodeType type = parse_type( toks, context );
 
 	if ( type == Code::Invalid )
 		return CodeInvalid;
@@ -6159,7 +6314,7 @@ namespace gen
 	name = currtok;
 	eat( TokType::Identifier );
 
-		CodeVar result = parse_variable_after_name( ModuleFlag::None, attributes, specifiers, type, name, toks, stringize(parse_variable) );
+	CodeVar result = parse_variable_after_name( ModuleFlag::None, attributes, specifiers, type, name, toks, context );
 
 	return result;
 }
@@ -6181,8 +6336,6 @@ namespace gen
 #	undef curr_tok
 #	undef eat
 #	undef left
-//	End GEN_FEATURE_PARSING
-#endif
 #pragma endregion Parsing Constructors
 
 #pragma region Untyped Constructors
@@ -6364,7 +6517,7 @@ namespace gen
 		return false;
 	}
 
-		Buffer = String::make_reserve( Memory::GlobalAllocator, Builder_StrBufferReserve );
+	Buffer = String::make_reserve( GlobalAllocator, Builder_StrBufferReserve );
 
 	return true;
 }
@@ -6381,16 +6534,7 @@ namespace gen
 }
 #pragma endregion Builder
 
-#pragma region Editor
-#ifdef GEN_FEATURE_EDITOR
-#endif
-#pragma endregion Editor
-
-#pragma region Scanner
-#ifdef GEN_FEATURE_SCANNER
-#endif
-#pragma endregion Scanner
+// namespace gen
 }
-// End: gen_time
-#endif
 
+#include "gen.pop_ignores.inline.hpp"

project/gen.editor.hpp

@@ -0,0 +1,73 @@
+#pragma once
+
+#include "gen.scanner.hpp"
+
+namespace gen {
+
+struct Policy
+{
+	// Nothing for now.
+};
+
+enum class SymbolType : u32
+{
+	Code,
+	Line,
+	Marker
+};
+
+struct Editor
+{
+	enum RequestType : u32
+	{
+		Add,
+		Replace,
+		Remove
+	};
+
+	struct SymbolData
+	{
+		Policy      Policy;
+		SymbolInfo  Info;
+	};
+
+	struct RequestEntry
+	{
+		union {
+			SymbolData Symbol;
+			String     Specification;
+		};
+		RequestType    Type;
+	};
+
+	struct Receipt
+	{
+		StringCached File;
+		Code         Found;
+		Code         Written;
+		bool         Result;
+	};
+
+	static AllocatorInfo Allocator;
+
+	static void set_allocator( AllocatorInfo  allocator );
+
+	Array<FileInfo>     Files;
+	String              Buffer;
+	Array<RequestEntry> Requests;
+
+	void add_files( s32 num, char const** files );
+
+	void add    ( SymbolInfo definition,  Policy policy, Code to_inject );
+	void remove ( SymbolInfo definition,  Policy policy );
+	void replace( SymbolInfo definition,  Policy policy, Code to_replace);
+
+#	ifdef GEN_FEATURE_EDITOR_REFACTOR
+	void refactor( char const* file_path, char const* specification_path );
+#	endif
+
+	bool process_requests( Array<Receipt> out_receipts );
+};
+
+// namespace gen
+};

project/gen.hpp

@@ -1,5 +1,5 @@
 /*
-	gencpp: An attempt at simple staged metaprogramming for c/c++.
+	gencpp: An attempt at "simple" staged metaprogramming for c/c++.
 
 	See Readme.md for more information from the project repository.
 
@@ -8,15 +8,21 @@
 */
 #pragma once
 
-#ifdef gen_time
-//! If its desired to roll your own dependencies, define GENCPP_PROVIDE_DEPENDENCIES before including this file.
-// Dependencies are derived from the c-zpl library: https://github.com/zpl-c/zpl
-#ifndef GEN_ROLL_OWN_DEPENDENCIES
-#	include "gen_dep.hpp"
+#if ! defined(GEN_DONT_ENFORCE_GEN_TIME_GUARD) && ! defined(GEN_TIME)
+#error Gen.hpp : GEN_TIME not defined
 #endif
 
-namespace gen
-{
+#include "gen.push_ignores.inline.hpp"
+
+//! If its desired to roll your own dependencies, define GEN_ROLL_OWN_DEPENDENCIES before including this file.
+// Dependencies are derived from the c-zpl library: https://github.com/zpl-c/zpl
+#ifndef GEN_ROLL_OWN_DEPENDENCIES
+#	include "gen.dep.hpp"
+#endif
+
+namespace gen {
+
+#pragma region Types
 using LogFailType = sw(*)(char const*, ...);
 
 // By default this library will either crash or exit if an error is detected while generating codes.
@@ -234,9 +240,9 @@ namespace gen
 		#	pragma push_macro( "global" )
 		#	pragma push_macro( "internal" )
 		#	pragma push_macro( "local_persist" )
-			#	define global        global
-			#	define internal      internal
-			#	define local_persist local_persist
+		#	undef global
+		#	undef internal
+		#	undef local_persist
 
 		#	define Entry( Spec_, Code_ ) { sizeof(stringize(Code_)), stringize(Code_) },
 			Define_Specifiers
@@ -342,7 +348,7 @@ namespace gen
 	constexpr char const* API_Import = stringize( GEN_API_Import_Code  );
 	constexpr char const* Keyword    = stringize( GEN_Attribute_Keyword);
 
-	#elif GEN_HAS_ATTRIBUTE( visibility ) || GEN_GCC_VERSION_CHECK( 3, 3, 0 ) || GEN_INTEL_VERSION_CHECK( 13, 0, 0 )
+#elif GEN_HAS_ATTRIBUTE( visibility ) || GEN_GCC_VERSION_CHECK( 3, 3, 0 )
 #	define GEN_API_Export_Code   __attribute__ ((visibility ("default")))
 #	define GEN_API_Import_Code   __attribute__ ((visibility ("default")))
 #	define GEN_Attribute_Keyword __attribute__
@@ -361,6 +367,7 @@ namespace gen
 	constexpr char const* Keyword    = "";
 #endif
 }
+#pragma endregion Types
 
 #pragma region Data Structures
 // Implements basic string interning. Data structure is based off the ZPL Hashtable.
@@ -533,8 +540,7 @@ namespace gen
 	template< class Type >
 	Type cast()
 	{
-			AST* ast = this;
-			return * rcast( Type*, & ast );
+		return * this;
 	}
 
 	operator Code();
@@ -669,7 +675,7 @@ namespace gen
 
 // Used when the its desired when omission is allowed in a definition.
 #define NoCode      { nullptr }
-	#define CodeInvalid (* Code::Invalid.ast)
+#define CodeInvalid (* Code::Invalid.ast) // Uses an implicitly overloaded cast from the AST to the desired code type.
 
 #pragma region Code Types
 #define Define_CodeType( Typename )                           \
@@ -704,6 +710,13 @@ namespace gen
 	{
 		raw()->append( other.ast );
 	}
+	void append( CodeBody body )
+	{
+		for ( Code entry : body )
+		{
+			append( entry );
+		}
+	}
 	bool has_entries()
 	{
 		return rcast( AST*, ast )->has_entries();
@@ -1243,7 +1256,7 @@ namespace gen
 		{
 			CodeAttributes Attributes;
 			char 	       _PAD_SPECS_     [ sizeof(AST*) ];
-				CodeType       UnderlyingType;
+			Code           UnderlyingType;
 			char 	       _PAD_PROPERTIES_[ sizeof(AST*) * 2 ];
 		};
 	};
@@ -1335,6 +1348,10 @@ namespace gen
 // However on Windows at least, it doesn't need to occur as the OS will clean up after the process.
 void deinit();
 
+// Clears the allocations, but doesn't return to the heap, the calls init() again.
+// Ease of use.
+void reset();
+
 // Used internally to retrive or make string allocations.
 // Strings are stored in a series of string arenas of fixed size (SizePer_StringArena)
 StringCached get_cached_string( StrC str );
@@ -1403,7 +1420,7 @@ namespace gen
 CodeTemplate def_template( CodeParam params, Code definition, ModuleFlag mflags = ModuleFlag::None );
 
 CodeType    def_type   ( StrC name, Code arrayexpr = NoCode, CodeSpecifier specifiers = NoCode, CodeAttributes attributes = NoCode );
-	CodeTypedef def_typedef( StrC name, CodeType type, CodeAttributes attributes = NoCode, ModuleFlag mflags = ModuleFlag::None );
+CodeTypedef def_typedef( StrC name, Code type, CodeAttributes attributes = NoCode, ModuleFlag mflags = ModuleFlag::None );
 
 CodeUnion def_union( StrC name, Code body, CodeAttributes attributes = NoCode, ModuleFlag mflags = ModuleFlag::None );
 
@@ -1448,7 +1465,6 @@ namespace gen
 #pragma endregion Upfront
 
 #pragma region Parsing
-#	ifdef GEN_FEATURE_PARSING
 CodeClass      parse_class        ( StrC class_def     );
 CodeEnum       parse_enum         ( StrC enum_def      );
 CodeBody       parse_export_body  ( StrC export_def    );
@@ -1466,7 +1482,6 @@ namespace gen
 CodeUnion      parse_union        ( StrC union_def     );
 CodeUsing      parse_using        ( StrC using_def     );
 CodeVar        parse_variable     ( StrC var_def       );
-#	endif
 #pragma endregion Parsing
 
 #pragma region Untyped text
@@ -1478,6 +1493,8 @@ namespace gen
 Code untyped_token_fmt( char const* fmt, s32 num_tokens, ... );
 #pragma endregion Untyped text
 
+#pragma endregion Gen Interaface
+
 struct Builder
 {
 	FileInfo File;
@@ -1490,223 +1507,7 @@ namespace gen
 	void write();
 };
 
-#if defined(GEN_FEATURE_EDITOR) || defined(GEN_FEATURE_SCANNER)
-	struct SymbolInfo
-	{
-		StringCached File;
-		char const*  Marker;
-		Code         Signature;
-	};
-#endif
-
-#ifdef GEN_FEATURE_EDITOR
-	struct Policy
-	{
-		// Nothing for now.
-	};
-
-	enum class SymbolType : u32
-	{
-		Code,
-		Line,
-		Marker
-	};
-
-	struct Editor
-	{
-		enum RequestType : u32
-		{
-			Add,
-			Replace,
-			Remove
-		};
-
-		struct SymbolData
-		{
-			Policy      Policy;
-			SymbolInfo  Info;
-		};
-
-		struct RequestEntry
-		{
-			union {
-				SymbolData Symbol;
-				String     Specification;
-			};
-			RequestType    Type;
-		};
-
-		struct Receipt
-		{
-			StringCached File;
-			Code         Found;
-			Code         Written;
-			bool         Result;
-		};
-
-		static AllocatorInfo Allocator;
-
-		static void set_allocator( AllocatorInfo  allocator );
-
-		Array(FileInfo)     Files;
-		String              Buffer;
-		Array(RequestEntry) Requests;
-
-		void add_files( s32 num, char const** files );
-
-		void add    ( SymbolInfo definition,  Policy policy, Code to_inject );
-		void remove ( SymbolInfo definition,  Policy policy );
-		void replace( SymbolInfo definition,  Policy policy, Code to_replace);
-
-#	ifdef GEN_FEATURE_EDITOR_REFACTOR
-		void refactor( char const* file_path, char const* specification_path );
-#	endif
-
-		bool process_requests( Array(Receipt) out_receipts );
-	};
-#endif
-
-#ifdef GEN_FEATURE_SCANNER
-	struct Scanner
-	{
-		struct RequestEntry
-		{
-			SymbolInfo Info;
-		};
-
-		struct Receipt
-		{
-			StringCached File;
-			Code         Defintion;
-			bool         Result;
-		};
-
-		AllocatorInfo MemAlloc;
-
-		static void set_allocator( AllocatorInfo allocator );
-
-		Array(FileInfo)     Files;
-		String              Buffer;
-		Array(RequestEntry) Requests;
-
-		void add_files( s32 num, char const** files );
-
-		void add( SymbolInfo signature, Policy policy );
-
-		bool process_requests( Array(Receipt) out_receipts );
-	};
-#endif
-#pragma endregion Gen Interface
-}
-
-#pragma region Macros
-#	define gen_main main
-
-#	define __ NoCode
-
-//	Convienence for defining any name used with the gen api.
-//  Lets you provide the length and string literal to the functions without the need for the DSL.
-#	define name( Id_ )   { sizeof(stringize( Id_ )) - 1, stringize(Id_) }
-
-//  Same as name just used to indicate intention of literal for code instead of names.
-#	define code( ... ) { sizeof(stringize(__VA_ARGS__)) - 1, stringize( __VA_ARGS__ ) }
-
-#	define args( ... ) num_args( __VA_ARGS__ ), __VA_ARGS__
-
-#	define code_str( ... ) gen::untyped_str( code( __VA_ARGS__ ) )
-#	define code_fmt( ... ) gen::untyped_str( token_fmt( __VA_ARGS__ ) )
-
-// Takes a format string (char const*) and a list of tokens (StrC) and returns a StrC of the formatted string.
-#	define token_fmt( ... ) gen::token_fmt_impl( (num_args( __VA_ARGS__ ) + 1) / 2, __VA_ARGS__ )
-#pragma endregion Macros
-
-#pragma region Constants
-#ifdef GEN_DEFINE_LIBRARY_CODE_CONSTANTS
-namespace gen
-{
-	// Predefined typename codes. Are set to readonly and are setup during gen::init()
-
-	extern CodeType t_b32;
-
-	extern CodeType t_s8;
-	extern CodeType t_s16;
-	extern CodeType t_s32;
-	extern CodeType t_s64;
-
-	extern CodeType t_u8;
-	extern CodeType t_u16;
-	extern CodeType t_u32;
-	extern CodeType t_u64;
-
-	extern CodeType t_sw;
-	extern CodeType t_uw;
-
-	extern CodeType t_f32;
-	extern CodeType t_f64;
-}
-#endif
-
-namespace gen
-{
-	// These constexprs are used for allocation behavior of data structures
-	// or string handling while constructing or serializing.
-	// Change them to suit your needs.
-
-	constexpr s32 InitSize_DataArrays  = 16;
-	constexpr s32 InitSize_StringTable = megabytes(4);
-
-	constexpr s32 CodePool_NumBlocks        = kilobytes(4);
-	constexpr s32 SizePer_StringArena       = megabytes(1);
-
-	constexpr s32 MaxCommentLineLength      = 1024;
-	constexpr s32 MaxNameLength             = 128;
-	constexpr s32 MaxUntypedStrLength       = kilobytes(640);
-	constexpr s32 StringTable_MaxHashLength = kilobytes(1);
-	constexpr s32 TokenFmt_TokenMap_MemSize	= kilobytes(4);
-	constexpr s32 LexAllocator_Size         = megabytes(10);
-	constexpr s32 Builder_StrBufferReserve  = megabytes(1);
-
-	extern CodeType t_auto;
-	extern CodeType t_void;
-	extern CodeType t_int;
-	extern CodeType t_bool;
-	extern CodeType t_char;
-	extern CodeType t_wchar_t;
-	extern CodeType t_class;
-	extern CodeType t_typename;
-
-	extern Code access_public;
-	extern Code access_protected;
-	extern Code access_private;
-
-	extern Code module_global_fragment;
-	extern Code module_private_fragment;
-
-	extern Code pragma_once;
-
-	extern CodeSpecifier spec_const;
-	extern CodeSpecifier spec_consteval;
-	extern CodeSpecifier spec_constexpr;
-	extern CodeSpecifier spec_constinit;
-	extern CodeSpecifier spec_extern_linkage;
-	extern CodeSpecifier spec_global;
-	extern CodeSpecifier spec_inline;
-	extern CodeSpecifier spec_internal_linkage;
-	extern CodeSpecifier spec_local_persist;
-	extern CodeSpecifier spec_mutable;
-	extern CodeSpecifier spec_ptr;
-	extern CodeSpecifier spec_ref;
-	extern CodeSpecifier spec_register;
-	extern CodeSpecifier spec_rvalue;
-	extern CodeSpecifier spec_static_member;
-	extern CodeSpecifier spec_thread_local;
-	extern CodeSpecifier spec_volatile;
-}
-#pragma endregion Constants
-
 #pragma region Inlines
-namespace gen
-{
 void AST::append( AST* other )
 {
 	if ( other->Parent )
@@ -1920,7 +1721,6 @@ namespace gen
 Define_AST_Cast( Var );
 #undef Define_AST_Cast
 
-
 #define Define_CodeCast( type )     \
 Code::operator Code ## type() const \
 {                                   \
@@ -2044,22 +1844,125 @@ namespace gen
 
 	return { result, buf };
 }
-}
 #pragma endregion Inlines
 
-// end: gen_time
+// namespace gen
+}
+
+#pragma region Constants
+#ifdef GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+namespace gen
+{
+	// Predefined typename codes. Are set to readonly and are setup during gen::init()
+
+	extern CodeType t_b32;
+
+	extern CodeType t_s8;
+	extern CodeType t_s16;
+	extern CodeType t_s32;
+	extern CodeType t_s64;
+
+	extern CodeType t_u8;
+	extern CodeType t_u16;
+	extern CodeType t_u32;
+	extern CodeType t_u64;
+
+	extern CodeType t_sw;
+	extern CodeType t_uw;
+
+	extern CodeType t_f32;
+	extern CodeType t_f64;
+}
 #endif
 
+namespace gen
+{
+	// These constexprs are used for allocation behavior of data structures
+	// or string handling while constructing or serializing.
+	// Change them to suit your needs.
+
+	constexpr s32 InitSize_DataArrays  = 16;
+	constexpr s32 InitSize_StringTable = megabytes(4);
+
+	// NOTE: This limits the maximum size of an allocation
+	// If you are generating a string larger than this, increase the size of the bucket here.
+	constexpr uw Global_BucketSize          = megabytes(10);
+	constexpr s32 CodePool_NumBlocks        = kilobytes(64);
+	constexpr s32 SizePer_StringArena       = megabytes(1);
+
+	constexpr s32 MaxCommentLineLength      = 1024;
+	constexpr s32 MaxNameLength             = 128;
+	constexpr s32 MaxUntypedStrLength       = kilobytes(640);
+	constexpr s32 StringTable_MaxHashLength = kilobytes(1);
+	constexpr s32 TokenFmt_TokenMap_MemSize	= kilobytes(4);
+	constexpr s32 LexAllocator_Size         = megabytes(10);
+	constexpr s32 Builder_StrBufferReserve  = megabytes(1);
+
+	extern CodeType t_auto;
+	extern CodeType t_void;
+	extern CodeType t_int;
+	extern CodeType t_bool;
+	extern CodeType t_char;
+	extern CodeType t_wchar_t;
+	extern CodeType t_class;
+	extern CodeType t_typename;
+
+	extern Code access_public;
+	extern Code access_protected;
+	extern Code access_private;
+
+	extern Code module_global_fragment;
+	extern Code module_private_fragment;
+
+	extern Code pragma_once;
+
+	extern CodeSpecifier spec_const;
+	extern CodeSpecifier spec_consteval;
+	extern CodeSpecifier spec_constexpr;
+	extern CodeSpecifier spec_constinit;
+	extern CodeSpecifier spec_extern_linkage;
+	extern CodeSpecifier spec_global;
+	extern CodeSpecifier spec_inline;
+	extern CodeSpecifier spec_internal_linkage;
+	extern CodeSpecifier spec_local_persist;
+	extern CodeSpecifier spec_mutable;
+	extern CodeSpecifier spec_ptr;
+	extern CodeSpecifier spec_ref;
+	extern CodeSpecifier spec_register;
+	extern CodeSpecifier spec_rvalue;
+	extern CodeSpecifier spec_static_member;
+	extern CodeSpecifier spec_thread_local;
+	extern CodeSpecifier spec_volatile;
+}
+#pragma endregion Constants
+
+#pragma region Macros
+#	define gen_main main
+
+#	define __ NoCode
+
+//	Convienence for defining any name used with the gen api.
+//  Lets you provide the length and string literal to the functions without the need for the DSL.
+#	define name( Id_ )   { sizeof(stringize( Id_ )) - 1, stringize(Id_) }
+
+//  Same as name just used to indicate intention of literal for code instead of names.
+#	define code( ... ) { sizeof(stringize(__VA_ARGS__)) - 1, stringize( __VA_ARGS__ ) }
+
+#	define args( ... ) num_args( __VA_ARGS__ ), __VA_ARGS__
+
+#	define code_str( ... ) gen::untyped_str( code( __VA_ARGS__ ) )
+#	define code_fmt( ... ) gen::untyped_str( token_fmt( __VA_ARGS__ ) )
+
+// Takes a format string (char const*) and a list of tokens (StrC) and returns a StrC of the formatted string.
+#	define token_fmt( ... ) gen::token_fmt_impl( (num_args( __VA_ARGS__ ) + 1) / 2, __VA_ARGS__ )
+#pragma endregion Macros
+
 #ifdef GEN_EXPOSE_BACKEND
 namespace gen
 {
-	namespace Memory
-	{
+	// Global allocator used for data with process lifetime.
+	extern AllocatorInfo  GlobalAllocator;
 	extern Array< Arena > Global_AllocatorBuckets;
-	}
-
-	namespace StaticData
-	{
 	extern Array< Pool >  CodePools;
 	extern Array< Arena > StringArenas;
 
@@ -2074,6 +1977,6 @@ namespace gen
 	extern AllocatorInfo Allocator_StringTable;
 	extern AllocatorInfo Allocator_TypeTable;
 }
-}
-
 #endif
+
+#include "gen.pop_ignores.inline.hpp"

project/gen.pop_ignores.inline.hpp

@@ -0,0 +1,7 @@
+#if __clang__
+#	pragma clang diagnostic pop
+#endif
+
+#if __GNUC__
+#	pragma GCC diagnostic pop
+#endif

project/gen.push_ignores.inline.hpp

@@ -0,0 +1,17 @@
+#if __clang__
+#	pragma clang diagnostic push
+#	pragma clang diagnostic ignored "-Wunused-const-variable"
+#	pragma clang diagnostic ignored "-Wswitch"
+#	pragma clang diagnostic ignored "-Wunused-variable"
+#   pragma clang diagnostic ignored "-Wunknown-pragmas"
+#	pragma clang diagnostic ignored "-Wvarargs"
+#	pragma clang diagnostic ignored "-Wunused-function"
+#endif
+
+#if __GNUC__
+#	pragma GCC diagnostic push
+#   pragma GCC diagnostic ignored "-Wunknown-pragmas"
+#	pragma GCC diagnostic ignored "-Wcomment"
+#	pragma GCC diagnostic ignored "-Wswitch"
+#	pragma GCC diagnostic ignored "-Wunused-variable"
+#endif

project/gen.scanner.hpp

@@ -0,0 +1,44 @@
+#pragma once
+#include "gen.hpp"
+
+namespace gen {
+
+struct SymbolInfo
+{
+	StringCached File;
+	char const*  Marker;
+	Code         Signature;
+};
+
+
+struct Scanner
+{
+	struct RequestEntry
+	{
+		SymbolInfo Info;
+	};
+
+	struct Receipt
+	{
+		StringCached File;
+		Code         Defintion;
+		bool         Result;
+	};
+
+	AllocatorInfo MemAlloc;
+
+	static void set_allocator( AllocatorInfo allocator );
+
+	Array<FileInfo>     Files;
+	String              Buffer;
+	Array<RequestEntry> Requests;
+
+	void add_files( s32 num, char const** files );
+
+	void add( SymbolInfo signature, Policy policy );
+
+	bool process_requests( Array<Receipt> out_receipts );
+};
+
+// namespace gen
+}

project/gen.undef.macros.hpp

@@ -1,4 +1,4 @@
-#if gen_time
+#if GEN_TIME
 // This undefines the macros used by the gen library but are not necessary for the user.
 
 #undef GEN_ARCH_64_BIT
@@ -66,11 +66,15 @@
 #undef stringize_va
 #undef txt_StrC
 
-#undef __
-#undef args
+#undef GEN_TIME
 #undef gen_main
-#undef gen_time
+#undef __
+#undef name
+#undef code
+#undef args
+#undef code_str
+#undef code_fmt
 #undef token_fmt
 
-// gen_time
+// GEN_TIME
 #endif

scripts/Readme.md

@@ -1,6 +1,6 @@
 # Scripts
 
-Build and cleanup scripts for the test deirectory are found here along with `natvis` and `natstepfilter` files for debugging.
+Build and cleanup scripts for the test directory are found here along with `natvis` and `natstepfilter` files for debugging.
 
 The build works as follows:
 
@@ -9,4 +9,4 @@ The build works as follows:
 * Build a program that uses some the generated definitions. (Have not done yet)
 
 The `test/gen` directory has the meson.build config for the meta-program
-The `test` directory has the one for the depdendent-program.
+The `test` directory has the one for the dependent-program.

scripts/build.ci.ps1

@@ -43,53 +43,3 @@ Push-Location $path_root
 
 	& ninja $args_ninja
 Pop-Location
-
-Push-location $path_gen
-# Run meta-program
-	$gencpp = Join-Path $path_gen_build gencpp.exe
-
-	Write-Host `nRunning tests...
-	& $gencpp
-
-# Format generated files
-	Write-Host `nBeginning format...
-	$formatParams = @(
-		'-i'          # In-place
-		'-style=file' # Search for a .clang-format file in the parent directory of the source file.
-		'-verbose'
-	)
-
-	$include = @('*.gen.hpp', '*.gen.cpp')
-	$exclude = $null
-
-	$targetFiles = @(Get-ChildItem -Recurse -Path $path_gen -Include $include -Exclude $exclude | Select-Object -ExpandProperty FullName)
-
-	clang-format $formatParams $targetFiles
-	Write-Host "`nFormatting complete"
-Pop-Location
-
-# Build the program depending on generated files.
-if ( -not( Test-Path $path_test_build ) )
-{
-Push-Location $path_test
-	$args_meson = @()
-	$args_meson += "setup"
-	$args_meson += $path_test_build
-
-# 	& meson $args_meson
-Pop-Location
-}
-
-Push-Location $path_root
-	$args_ninja = @()
-	$args_ninja += "-C"
-	$args_ninja += $path_test_build
-
-	# ninja $args_ninja
-Pop-Location
-
-Push-Location $path_test
-	$testcpp = Join-Path $path_test_build testcpp.exe
-
-	# & $testcpp
-Pop-Location

scripts/gen.ps1

@@ -0,0 +1,95 @@
+[string] $type = $null
+[string] $test = $false
+
+foreach ( $arg in $args )
+{
+	if ( $arg -eq "test" )
+	{
+		$test = $true
+	}
+	else
+	{
+		$type = $arg
+	}
+}
+
+$path_root       = git rev-parse --show-toplevel
+$path_build      = Join-Path $path_root build
+$path_scripts    = Join-Path $path_root scripts
+$path_test       = Join-Path $path_root test
+$path_gen        = Join-Path $path_test gen
+$path_test_build = Join-Path $path_test build
+$path_gen_build  = Join-Path $path_gen  build
+
+write-host "`n`nBuilding Test`n"
+
+if ( -not( Test-Path $path_gen_build ) )
+{
+# Generate build files for meta-program
+Push-Location $path_gen
+	$args_meson  = @()
+	$args_meson += "setup"
+	$args_meson += $path_gen_build
+
+	& meson $args_meson
+Pop-Location
+}
+
+# Compile meta-program
+Push-Location $path_root
+	$args_ninja = @()
+	$args_ninja += "-C"
+	$args_ninja += $path_gen_build
+
+	& ninja $args_ninja
+Pop-Location
+
+Push-location $path_gen
+# Run meta-program
+	$gencpp = Join-Path $path_gen_build gencpp.exe
+
+	Write-Host `nRunning tests...
+	& $gencpp
+
+# Format generated files
+	Write-Host `nBeginning format...
+	$formatParams = @(
+		'-i'          # In-place
+		'-style=file' # Search for a .clang-format file in the parent directory of the source file.
+		'-verbose'
+	)
+
+	$include = @('*.gen.hpp', '*.gen.cpp')
+	$exclude = $null
+
+	$targetFiles = @(Get-ChildItem -Recurse -Path $path_gen -Include $include -Exclude $exclude | Select-Object -ExpandProperty FullName)
+
+	clang-format $formatParams $targetFiles
+	Write-Host "`nFormatting complete"
+Pop-Location
+
+# Build the program depending on generated files.
+if ( -not( Test-Path $path_test_build ) )
+{
+Push-Location $path_test
+	$args_meson = @()
+	$args_meson += "setup"
+	$args_meson += $path_test_build
+
+# 	& meson $args_meson
+Pop-Location
+}
+
+Push-Location $path_root
+	$args_ninja = @()
+	$args_ninja += "-C"
+	$args_ninja += $path_test_build
+
+	# ninja $args_ninja
+Pop-Location
+
+Push-Location $path_test
+	$testcpp = Join-Path $path_test_build testcpp.exe
+
+	# & $testcpp
+Pop-Location

scripts/gencpp.refactor

@@ -352,7 +352,7 @@
 // gencpp macros
 
 // word gen_main     new_name
-// word gen_time     new_name
+// word GEN_TIME     new_name
 
 // word __           new_name
 // word code         new_name

test/DummyInclude.hpp

@@ -1 +0,0 @@
-// Only for gen testing.

test/Parsed/Array.Parsed.hpp

@@ -1,6 +1,6 @@
 #pragma once
 
-#if gen_time
+#if GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES

test/Parsed/Buffer.Parsed.hpp

@@ -1,6 +1,6 @@
 #pragma once
 
-#if gen_time
+#if GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
@@ -203,4 +203,4 @@ u32 gen_buffer_file()
 	return 0;
 }
 
-#endif // gen_time
+#endif // GEN_TIME

test/Parsed/HashTable.Parsed.hpp

@@ -1,6 +1,6 @@
 #pragma once
 
-#if gen_time
+#if GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
@@ -357,4 +357,4 @@ u32 gen_hashtable_file()
 	return 0;
 }
 
-#endif // gen_time
+#endif // GEN_TIME

test/Parsed/Ring.Parsed.hpp

@@ -1,6 +1,6 @@
 #pragma once
 
-#if gen_time
+#if GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
@@ -173,4 +173,4 @@ u32 gen_ring_file()
 	return 0;
 }
 
-#endif // gen_time
+#endif // GEN_TIME

test/Parsed/Sanity.Parsed.hpp

@@ -1,5 +1,5 @@
 #pragma once
-#ifdef gen_time
+#ifdef GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES

test/Readme.md

@@ -2,10 +2,10 @@
 
 The following tests focus on attempting to generate some math, containers, and the memory module of zpl.
 
-Not all the files are written how I would practically use the librarry, the containers for example would
+Not all the files are written how I would practically use the library, the containers for example would
 be better on in c++ as templates, since the templates they generate are trivial symbols to inspect or debug.
 
-An exmaple of a non-trival generation is a container for elements with SOA or AOS policy for layout.
+An example of a non-trivial generation is a container for elements with SOA or AOS policy for layout.
 (If a unified element syntax is desired)
 
 The test is divided between two major sets of tests: Parsed and Upfront.

test/SOA.cpp

@@ -1,9 +1,9 @@
-#pragma once
-
-#ifdef gen_time
+#if GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
+#define GEN_EXPOSE_BACKEND
+#define GEN_BENCHMARK
 #include "gen.hpp"
 using namespace gen;
 
@@ -91,7 +91,7 @@ Code gen_SOA( CodeStruct struct_def, s32 num_entries = 0 )
 			}
 		));
 
-		String content = String::make( Memory::GlobalAllocator, "return\n{\n" );
+		String content = String::make( GlobalAllocator, "return\n{\n" );
 
 		for ( CodeVar member : vars )
 		{
@@ -114,4 +114,33 @@ Code gen_SOA( CodeStruct struct_def, s32 num_entries = 0 )
 
 	return soa;
 }
+
+void check_SOA()
+{
+	log_fmt("\ncheck_SOA:");
+	gen::init();
+	Builder soa_test; soa_test.open( "SOA.gen.hpp" );
+
+	soa_test.print( parse_using( code(
+		using u16 = unsigned short;
+	)));
+	soa_test.print( def_include( txt_StrC("gen.hpp")));
+	soa_test.print( def_using_namespace( name(gen) ) );
+
+	soa_test.print( gen_SOA(
+		parse_struct( code(
+			struct TestStruct
+			{
+				u8  A;
+				u16 B;
+				u32 C;
+				u64 D;
+			};
+		))
+	));
+
+	soa_test.write();
+	gen::deinit();
+	log_fmt(" passed!\n");
+}
 #endif

test/Upfront/Array.Upfront.hpp

@@ -1,6 +1,6 @@
 #pragma once
 
-#if gen_time
+#if GEN_TIME
 #include "gen.hpp"
 
 using namespace gen;

test/Upfront/Buffer.Upfront.hpp

@@ -1,6 +1,6 @@
 #pragma once
 
-#if gen_time
+#if GEN_TIME
 #include "gen.hpp"
 
 using namespace gen;
@@ -271,4 +271,4 @@ u32 gen_buffer_file()
 	return 0;
 }
 
-#endif // gen_time
+#endif // GEN_TIME

test/Upfront/HashTable.Upfront.hpp

@@ -1,6 +1,6 @@
 #pragma once
 
-#if gen_time
+#if GEN_TIME
 #include "gen.hpp"
 #include "Array.Upfront.hpp"
 
@@ -483,4 +483,4 @@ u32 gen_hashtable_file()
 	return 0;
 }
 
-#endif // gen_time
+#endif // GEN_TIME

test/Upfront/Ring.Upfront.hpp

@@ -1,6 +1,6 @@
 #pragma once
 
-#if gen_time
+#if GEN_TIME
 #include "gen.hpp"
 #include "Buffer.Upfront.hpp"
 
@@ -225,4 +225,4 @@ u32 gen_ring_file()
 	return 0;
 }
 
-#endif // gen_time
+#endif // GEN_TIME

test/Upfront/Sanity.Upfront.hpp

@@ -1,4 +1,4 @@
-#ifdef gen_time
+#ifdef GEN_TIME
 #include "gen.hpp"
 
 using namespace gen;

test/gen/meson.build

@@ -22,6 +22,6 @@ endif
   # add_project_arguments('-E', language : ['c', 'cpp'])
   # add_global_arguments( '-E', language : ['cpp'])
 
-add_project_arguments('-Dgen_time', language : ['c', 'cpp'])
+add_project_arguments('-DGEN_TIME', language : ['c', 'cpp'])
 
 executable( 'gencpp', sources, include_directories : includes )

test/sanity.cpp

@@ -1,6 +1,6 @@
 // Testing to make sure backend of library is operating properly.
 
-#ifdef gen_time
+#ifdef GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
@@ -41,14 +41,14 @@ void check_sanity()
 	constexpr
 	s32 num_iterations = 650000;
 
-	Array<CodeTypedef> typedefs = Array<CodeTypedef>::init_reserve( Memory::GlobalAllocator, num_iterations * 2 );
+	Array<CodeTypedef> typedefs = Array<CodeTypedef>::init_reserve( GlobalAllocator, num_iterations * 2 );
 
 	s32 idx = num_iterations;
 	while( --idx )
 	{
 		// Stress testing string allocation
-		String type_name = String::fmt_buf( Memory::GlobalAllocator, "type_%d", idx );
-		String typedef_name = String::fmt_buf( Memory::GlobalAllocator, "typedef_%d", idx );
+		String type_name = String::fmt_buf( GlobalAllocator, "type_%ld", idx );
+		String typedef_name = String::fmt_buf(GlobalAllocator, "typedef_%ld", idx );
 
 		CodeTypedef type_as_int = def_typedef( type_name, t_int );
 		CodeType    type        = def_type( type_name );
@@ -59,15 +59,15 @@ void check_sanity()
 	}
 
 	log_fmt("\nMemory before builder:\n");
-	log_fmt("Num Global Arenas       : %llu TotalSize: %llu !\n", Memory::Global_AllocatorBuckets.num(), Memory::Global_AllocatorBuckets.num() * Memory::Global_BucketSize);
-	log_fmt("Num Code Pools          : %llu TotalSize: %llu !\n", StaticData::CodePools.num(), StaticData::CodePools.num() * CodePool_NumBlocks * StaticData::CodePools.back().BlockSize);
-	log_fmt("Num String Cache Arenas : %llu TotalSize: %llu !\n", StaticData::StringArenas.num(), StaticData::StringArenas.num() * SizePer_StringArena);
-	log_fmt("Num String Cache        : %llu\n", StaticData::StringCache.Entries.num(), StaticData::StringCache);
+	log_fmt("Num Global Arenas       : %llu TotalSize: %llu !\n", Global_AllocatorBuckets.num(), Global_AllocatorBuckets.num() * Global_BucketSize);
+	log_fmt("Num Code Pools          : %llu TotalSize: %llu !\n", CodePools.num(), CodePools.num() * CodePool_NumBlocks * CodePools.back().BlockSize);
+	log_fmt("Num String Cache Arenas : %llu TotalSize: %llu !\n", StringArenas.num(), StringArenas.num() * SizePer_StringArena);
+	log_fmt("Num String Cache        : %llu\n", StringCache.Entries.num(), StringCache);
 
 	Builder builder;
 	builder.open( "sanity.gen.hpp" );
 
-	idx = num_iterations;
+	idx = typedefs.num();
 #ifdef GEN_BENCHMARK
 	u64 time_start = time_rel_ms();
 #endif
@@ -82,12 +82,12 @@ void check_sanity()
 #endif
 
 	log_fmt("\nMemory after builder:\n");
-	log_fmt("Num Global Arenas       : %llu TotalSize: %llu !\n", Memory::Global_AllocatorBuckets.num(), Memory::Global_AllocatorBuckets.num() * Memory::Global_BucketSize);
-	log_fmt("Num Code Pools          : %llu TotalSize: %llu !\n", StaticData::CodePools.num(), StaticData::CodePools.num() * CodePool_NumBlocks * StaticData::CodePools.back().BlockSize);
-	log_fmt("Num String Cache Arenas : %llu TotalSize: %llu !\n", StaticData::StringArenas.num(), StaticData::StringArenas.num() * SizePer_StringArena);
-	log_fmt("Num String Cache        : %llu\n", StaticData::StringCache.Entries.num(), StaticData::StringCache);
+	log_fmt("Num Global Arenas       : %llu TotalSize: %llu !\n", Global_AllocatorBuckets.num(), Global_AllocatorBuckets.num() * Global_BucketSize);
+	log_fmt("Num Code Pools          : %llu TotalSize: %llu !\n", CodePools.num(), CodePools.num() * CodePool_NumBlocks * CodePools.back().BlockSize);
+	log_fmt("Num String Cache Arenas : %llu TotalSize: %llu !\n", StringArenas.num(), StringArenas.num() * SizePer_StringArena);
+	log_fmt("Num String Cache        : %llu\n", StringCache.Entries.num(), StringCache);
 
-	log_fmt("\nSanity passed!\n");
 	gen::deinit();
+	log_fmt("\nSanity passed!\n");
 }
 #endif

test/test.Upfront.cpp

@@ -1,4 +1,4 @@
-#ifdef gen_time
+#ifdef GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES

test/test.cpp

@@ -1,7 +1,13 @@
+#define GEN_FEATURE_PARSING
+#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+#define GEN_ENFORCE_STRONG_CODE_TYPES
+#define GEN_EXPOSE_BACKEND
+#define GEN_BENCHMARK
 #include "gen.cpp"
 #include "sanity.cpp"
+#include "SOA.cpp"
 
-#if gen_time
+#if GEN_TIME
 int gen_main()
 {
 	using namespace gen;
@@ -9,13 +15,15 @@ int gen_main()
 
 	check_sanity();
 
+	check_SOA();
+
 	return 0;
 }
 #endif
 
 
 // This only has to be done if symbol conflicts occur.
-#ifndef gen_time
+#ifndef GEN_TIME
 int main()
 {
 

test/test.parsing.cpp

@@ -1,4 +1,4 @@
-#ifdef gen_time
+#ifdef GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES
@@ -9,7 +9,7 @@
 #include "Parsed\HashTable.Parsed.hpp"
 #include "Parsed\Ring.Parsed.hpp"
 #include "Parsed\Sanity.Parsed.hpp"
-#include "SOA.hpp"
+#include "SOA.cpp"
 #include "gen.cpp"
 
 using namespace gen;

test/upfront.hpp

@@ -1,4 +1,4 @@
-#ifdef gen_time
+#ifdef GEN_TIME
 #define GEN_FEATURE_PARSING
 #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_ENFORCE_STRONG_CODE_TYPES