Readme update

- Added support for anonymous structs.
- Gave Token_Fmt::token_map its own static memory.
- Minor natvis fix for CodeBody
- Renamed ESpecifier::Static_Member to just Static (acts as a general use case) specifier option
- Setup the lex token array with a configurable arena allocator.

Two major things left before V0.3-4:
- Attribute and Module parisng support with base case test
- AST serializtaion strings get a dedicated slag allocator.

Readme.md

@@ -21,7 +21,7 @@ These build up a code AST to then serialize with a file builder.
 
 ## Notes
 
-The project has reached a sort of *alpha* state, all the current functionality works for the test cases but it will most likely break in many other cases.
+The project has reached an *alpha* state, all the current functionality works for the test cases but it will most likely break in many other cases.
 
 The project has no external dependencies beyond:
 
@@ -36,13 +36,12 @@ The project has no external dependencies beyond:
 Dependencies for the project are wrapped within `GENCPP_ROLL_OWN_DEPENDENCIES` (Defining it will disable them).  
 The majority of the dependency's implementation was derived from the [c-zpl library](https://github.com/zpl-c/zpl).
 
-When gencpp is in a stable state, I will make a C variant with the same feature set.  
-A single-header version will also be generated for both.
-
 A `natvis` and `natstepfilter` are provided in the scripts directory.
 
 ***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.***
 
+A C variant is hosted [here](https://github.com/Ed94/genc); I haven't gotten headwind on it, should be easier to make than this...
+
 ## Usage
 
 A metaprogram is built to generate files before the main program is built. We'll term runtime for this program as `gen_time`. The metaprogram's core implementation are within `gen.hpp` and `gen.cpp` in the project directory.
@@ -159,10 +158,12 @@ This method is setup where all the metaprogram's code are the within the same fi
 ### *WHAT IS NOT PROVIDED*
 
 * Lambdas
-* Vendor provided dynamic dispatch (virtuals) : `override` and `final` specifiers complicate the specifier parsing and serialization. (I'll problably end up adding in later)
+* Lang provided dynamic dispatch (virtuals) : `override` and `final` specifiers complicate the specifier parsing and serialization. (Its a todo)
+* Suffix specifiers for functions (Ex: void() const ). Same reason as virtual/override/final missing for now.
 * RTTI
 * Exceptions
 * Execution statement validation              : Execution expressions are defined using the untyped API.
+* Parsing support for module specifiers and attributes. (Its a todo)
 
 Keywords kept from "Modern C++":
 
@@ -588,6 +589,8 @@ Editor and Scanner are disabled by default, use `GEN_FEATURE_EDITOR` and `GEN_FE
 
 ### Editor is for editing a series of files based on a set of requests provided to it
 
+**Note: Not implemented yet**
+
 * The purpose is to overrite a specific file, it places its contents in a buffer to scan.
 * Requests are populated using the following interface:
   * add : Add code.
@@ -610,6 +613,8 @@ It will on call add a request to the queue to run the refactor script on the fil
 
 ### Scanner allows the user to generate Code ASTs by reading files
 
+**Note: Not implemented yet**
+
 * The purpose is to grab definitions to generate metadata or generate new code from these definitions.
 * Requests are populated using the add( SymbolInfo, Policy ) function. The symbol info is the same as the one used for the editor. So is the case with Policy.
 
@@ -664,8 +669,12 @@ Names or Content fields are interned strings and thus showed be cached using `ge
 # TODO
 
 * Implement a context stack for the parsing, allows for accurate scope validation for the AST types.
-* Make a test suite thats covers some base cases and zpl containers (+ anything else suitable)
-* Finish support for module specifiers and standard/platform attributes.
+* Make a more robust test suite.
 * Generate a single-header library.
-* Improve the allocation strategy for strings in `AST::to_string`, `Parser::lex`, and `token_fmt_va`
+* Improve the allocation strategy for strings in `Builder`, `AST::to_string`, `Parser::lex`, all three can use some form of slab allocation strategy...
+  * Can most likely use a simple slag allocator.
 * May be in need of a better name, I found a few repos with this same one...
+* Support module and attribute parsing (Marked with TODOs for now..)
+* Suffix specifiers for functions (const, override, final)
+* Implement the Scanner
+* Implement the Editor

project/Readme.md

@@ -13,6 +13,9 @@ Feature Macros:
 * `GEN_FEATURE_EDITOR` : Defines the file editing features for changing definitions based on ASTs
 * `GEN_FEATURE_SCANNER` : Defines the file scanning features for generating ASTs
 
+`GEN_USE_RECURSIVE_AST_DUPLICATION` is available but its not well tested and should not need to be used.
+If constructing ASTs properly. There should be no modification of ASTs, and thus this would never become an issue.
+
 Due to the design of `gen.hpp` to support being written alongside runtime intended code (in the same file), all the code is wrapped in a `gen_time` `#ifdef` and then wrapped further in a `gen` namespace to avoid pollution of the global scope.
 
 *Note: Its possible with the scanner feature to support parsing runtime files that use "generic" macros or identifiers with certain patterns.  

project/gen.cpp

@@ -1712,8 +1712,13 @@ namespace gen
 
 		global StringTable StringCache;
 
+		// TODO : Need to implement String memory management for seriaization intermediates.
+
+		global Arena LexArena;
+
 		global AllocatorInfo Allocator_DataArrays       = heap();
 		global AllocatorInfo Allocator_CodePool         = heap();
+		global AllocatorInfo Allocator_Lexer            = heap();
 		global AllocatorInfo Allocator_StringArena      = heap();
 		global AllocatorInfo Allocator_StringTable      = heap();
 		global AllocatorInfo Allocator_TypeTable        = heap();
@@ -1852,12 +1857,215 @@ namespace gen
 	{
 		using namespace ECode;
 
-		Code result = make_code();
+		AST*
+		result = make_code().ast;
+	#ifndef GEN_USE_RECURSIVE_AST_DUPLICATION
+		mem_copy( result, this, sizeof( AST ) );
+		result->Parent = nullptr;
+	#else
+		switch ( Type )
+		{
+			case Invalid:
+				log_failure("Attempted to duplicate invalid code! - \n%s", Parent ? Parent->debug_str() : Name );
+				return nullptr
+			case Untyped:
+			case Comment:
+			case Execution:
+			case Access_Private:
+			case Access_Protected:
+			case Access_Public:
+			case PlatformAttributes:
+			case Preprocessor_Include:
+			case Module:
+			case Specifiers:
+			case Using_Namespace:
+				mem_copy( result, this, sizeof( AST ) );
+			break;
 
-		// TODO : Bring back some of the old way, we need to recursively duplicate the children.
-		mem_copy( result.ast, this, sizeof( AST ) );
+			case Extern_Linkage:
+			case Friend:
+				mem_copy( result, this, sizeof( AST ) );
 
-		return result.ast;
+				if (Value)
+					result->Value = Value->duplicate();
+			break;
+
+			case Class:
+			case Struct:
+			case Enum:
+				mem_copy( result, this, sizeof( AST ) );
+
+				if ( Attributes)
+					result->Attributes = Attributes->duplicate();
+
+				if ( ParentType )
+					result->ParentType = ParentType->duplicate();
+
+				result->Body = Body->duplicate();
+			break;
+
+			case Enum_Fwd:
+			case Class_Fwd:
+			case Struct_Fwd:
+				mem_copy( result, this, sizeof( AST ) );
+
+				if ( Attributes)
+					result->Attributes = Attributes->duplicate();
+
+				if ( ParentType )
+					result->ParentType = ParentType->duplicate();
+			break;
+
+			case Function:
+			case Operator:
+			case Operator_Member:
+				mem_copy( result, this, sizeof( AST ) );
+
+				if ( Attributes)
+					result->Attributes = Attributes->duplicate();
+
+				if ( Specs )
+					result->ParentType = ParentType->duplicate();
+
+				if ( ReturnType )
+					result->ReturnType = ReturnType->duplicate();
+
+				if ( Params )
+					result->Params = Params->duplicate();
+
+				result->Body = Body->duplicate();
+			break;
+
+			case Function_Fwd:
+			case Operator_Fwd:
+			case Operator_Member_Fwd:
+				mem_copy( result, this, sizeof( AST ) );
+
+				if ( Attributes)
+					result->Attributes = Attributes->duplicate();
+
+				if ( Specs )
+					result->ParentType = ParentType->duplicate();
+
+				if ( ReturnType )
+					result->ReturnType = ReturnType->duplicate();
+
+				if ( Params )
+					result->Params = Params->duplicate();
+			break;
+
+			case Namespace:
+				mem_copy( result, this, sizeof( AST ) );
+
+				result->Body = Body->duplicate();
+			break;
+
+			case Operator_Cast:
+				mem_copy( result, this, sizeof( AST ) );
+
+				result->ValueType = ValueType->duplicate();
+				result->Body      = Body->duplicate();
+			break;
+			case Operator_Cast_Fwd:
+				mem_copy( result, this, sizeof( AST ) );
+
+				result->ValueType = ValueType->duplicate();
+			break;
+
+			case Parameters:
+				mem_copy( result, this, sizeof( AST ) );
+
+				result->NumEntries = 0;
+				result->Last = nullptr;
+				result->Next = nullptr;
+
+				if ( NumEntries - 1 > 0 )
+				{
+					CodeParam parent = result->cast<CodeParam>();
+					for ( CodeParam param : Next->cast<CodeParam>() )
+					{
+						parent.append( param );
+					}
+				}
+			break;
+
+			case Template:
+				mem_copy( result, this, sizeof( AST ) );
+
+				result->Params      = Params->duplicate();
+				result->Declaration = Declaration->duplicate();
+			break;
+
+			case Typename:
+				mem_copy( result, this, sizeof( AST ) );
+
+				if (Attributes)
+					result->Attributes = Attributes->duplicate();
+
+				if ( Specs )
+					result->Specs = Specs->duplicate();
+
+				if ( ArrExpr )
+					result->ArrExpr = ArrExpr->duplicate();
+			break;
+
+			case Typedef:
+			case Using:
+				mem_copy( result, this, sizeof( AST ) );
+
+				if (Attributes)
+					result->Attributes = Attributes->duplicate();
+
+				if ( UnderlyingType )
+					result->UnderlyingType = UnderlyingType->duplicate();
+			break;
+
+			case Union:
+				mem_copy( result, this, sizeof( AST ) );
+
+				if ( Attributes)
+					result->Attributes = Attributes->duplicate();
+
+				result->Body = Body->duplicate();
+			break;
+
+			case Variable:
+				mem_copy( result, this, sizeof( AST ) );
+
+				if (Attributes)
+					result->Attributes = Attributes->duplicate();
+
+				if ( Specs )
+					result->Specs = Specs->duplicate();
+
+				result->ValueType = UnderlyingType->duplicate();
+
+				if ( Value )
+					result->Value = Value->duplicate();
+			break;
+
+			case Class_Body:
+			case Enum_Body:
+			case Export_Body:
+			case Extern_Linkage_Body:
+			case Function_Body:
+			case Global_Body:
+			case Namespace_Body:
+			case Struct_Body:
+			case Union_Body:
+				CodeBody
+				body         = cast<CodeBody>();
+				body->Name   = Name;
+				body->Type   = Type;
+				for ( Code entry : cast<CodeBody>() )
+				{
+					result->append( entry.ast );
+				}
+			break;
+		}
+	#endif
+
+		return result;
 	}
 
 	String AST::to_string()
@@ -1869,6 +2077,7 @@ namespace gen
 		if ( bitfield_is_equal( u32, ModuleFlags, ModuleFlag::Import ))  \
 			result.append( "import " );             \
 
+		// TODO : Need to refactor so that intermeidate strings are freed conviently.
 		String result = String::make( Memory::GlobalAllocator, "" );
 
 		switch ( Type )
@@ -1876,7 +2085,7 @@ namespace gen
 			using namespace ECode;
 
 			case Invalid:
-				log_failure("Attempted to serialize invalid code! - %s", Name);
+				log_failure("Attempted to serialize invalid code! - %s", Parent ? Parent->debug_str() : Name );
 			break;
 
 			case Untyped:
@@ -1926,7 +2135,9 @@ namespace gen
 					result.append( "class " );
 
 					if ( Attributes )
+					{
 						result.append_fmt( "%s ", Attributes->to_string() );
+					}
 
 					if ( ParentType )
 					{
@@ -2216,14 +2427,15 @@ namespace gen
 
 			case Parameters:
 			{
-				// TODO : Parameters can have default values.
-
 				if ( Name )
 					result.append_fmt( "%s %s", ValueType->to_string(), Name );
 
 				else
 					result.append_fmt( "%s", ValueType->to_string() );
 
+				if ( Value )
+					result.append_fmt( "= %s", Value->to_string() );
+
 				if ( NumEntries - 1 > 0)
 				{
 					for ( CodeParam param : Next->cast<CodeParam>() )
@@ -2254,15 +2466,19 @@ namespace gen
 			{
 				ProcessModuleFlags();
 
+				if ( Name == nullptr)
+				{
+					result.append( "struct\n{\n%s\n};", Body->to_string() );
+					break;
+				}
+
 				if ( Attributes || ParentType )
 				{
 					result.append( "struct " );
 
-					if ( ParentType )
+					if ( Attributes )
 						result.append_fmt( "%s ", Attributes->to_string() );
 
-					// TODO : Structs can be anonymous, so we need to check for that
-
 					if ( ParentType )
 					{
 						char const* access_level = to_str( ParentAccess );
@@ -2276,6 +2492,8 @@ namespace gen
 					}
 					else
 					{
+						if ( Name )
+
 						result.append_fmt( "%s \n{\n%s\n};", Name, Body->to_string() );
 					}
 				}
@@ -2465,7 +2683,6 @@ namespace gen
 
 		switch ( Type )
 		{
-			// TODO : Finish this...
 			case ECode::Typedef:
 			case ECode::Typename:
 			{
@@ -2559,6 +2776,8 @@ namespace gen
 	{
 		using namespace StaticData;
 
+		Memory::setup();
+
 		// Setup the arrays
 		{
 			CodePools = Array<Pool>::init_reserve( Allocator_DataArrays, InitSize_DataArrays );
@@ -2581,6 +2800,10 @@ namespace gen
 
 			CodePools.append( code_pool );
 
+		#ifdef GEN_FEATURE_PARSING
+			LexArena = Arena::init_from_allocator( Allocator_Lexer, LexAllocator_Size );
+		#endif
+
 			Arena string_arena = Arena::init_from_allocator( Allocator_StringArena, SizePer_StringArena );
 
 			if ( string_arena.PhysicalStart == nullptr )
@@ -2696,7 +2919,7 @@ namespace gen
 		def_constant_spec( ref,              ESpecifier::Ref );
 		def_constant_spec( register,         ESpecifier::Register );
 		def_constant_spec( rvalue,           ESpecifier::RValue );
-		def_constant_spec( static_member,    ESpecifier::Static_Member );
+		def_constant_spec( static_member,    ESpecifier::Static );
 		def_constant_spec( thread_local,     ESpecifier::Thread_Local );
 		def_constant_spec( volatile, 	     ESpecifier::Volatile)
 
@@ -2738,6 +2961,12 @@ namespace gen
 
 		CodePools.free();
 		StringArenas.free();
+
+	#ifdef GEN_FEATURE_PARSING
+		LexArena.free();
+	#endif
+
+		Memory::cleanup();
 	}
 
 	AllocatorInfo get_string_allocator( s32 str_length )
@@ -3169,6 +3398,11 @@ namespace gen
 		StaticData::Allocator_CodePool = allocator;
 	}
 
+	void set_allocator_lexer( AllocatorInfo allocator )
+	{
+		StaticData::Allocator_Lexer = allocator;
+	}
+
 	void set_allocator_string_arena( AllocatorInfo allocator )
 	{
 		StaticData::Allocator_StringArena = allocator;
@@ -3764,8 +3998,6 @@ namespace gen
 	{
 		using namespace ECode;
 
-		name_check( def_struct, name );
-
 		if ( attributes && attributes->Type != PlatformAttributes )
 		{
 			log_failure( "gen::def_struct: attributes was not a `PlatformAttributes` type - %s", attributes.debug_str() );
@@ -3786,9 +4018,11 @@ namespace gen
 
 		CodeStruct
 		result              = (CodeStruct) make_code();
-		result->Name        = get_cached_string( name );
 		result->ModuleFlags = mflags;
 
+		if ( name )
+			result->Name = get_cached_string( name );
+
 		if ( body )
 		{
 			result->Type = Struct;
@@ -4816,7 +5050,6 @@ namespace gen
 			return scast(AccessSpec, tok.Type);
 		}
 
-		Arena LexAllocator;
 
 		struct TokArray
 		{
@@ -4885,16 +5118,15 @@ namespace gen
 				return { 0, nullptr };                                       \
 			}
 
-			do_once_start
-				// TODO : Use the global memory allocator for this...
-				LexAllocator = Arena::init_from_allocator( heap(), megabytes(10) );
+			// do_once_start
+			// 	LexAllocator = Arena::init_from_allocator( heap(), megabytes(10) );
 
-				if ( LexAllocator.PhysicalStart == nullptr )
-				{
-					log_failure( "gen::lex: failed to allocate memory for parsing constructor's lexer");
-					return { { nullptr }, 0  };
-				}
-			do_once_end
+			// 	if ( LexAllocator.PhysicalStart == nullptr )
+			// 	{
+			// 		log_failure( "gen::lex: failed to allocate memory for parsing constructor's lexer");
+			// 		return { { nullptr }, 0  };
+			// 	}
+			// do_once_end
 
 			local_persist thread_local
 			Array<Token> Tokens = { nullptr };
@@ -4913,9 +5145,11 @@ namespace gen
 			}
 
 			if ( Tokens )
-				Tokens.clear();
+			{
+				Tokens.free();
+			}
 
-			Tokens = Array<Token>::init_reserve( LexAllocator, content.Len / 8 );
+			Tokens = Array<Token>::init_reserve( StaticData::LexArena, content.Len / 6 );
 
 			while (left )
 			{
@@ -6192,7 +6426,7 @@ namespace gen
 							case ESpecifier::Constinit:
 							case ESpecifier::Inline:
 							case ESpecifier::Mutable:
-							case ESpecifier::Static_Member:
+							case ESpecifier::Static:
 							case ESpecifier::Volatile:
 							break;
 
@@ -6279,7 +6513,8 @@ namespace gen
 
 		// TODO : Parse attributes
 
-		name = parse_identifier( toks, context );
+		if ( check( TokType::Identifier ) )
+			name = parse_identifier( toks, context );
 
 		AccessSpec access     = AccessSpec::Default;
 		Token      parent_tok = { nullptr, 0, TokType::Invalid };
@@ -6500,7 +6735,7 @@ namespace gen
 							case ESpecifier::Constinit:
 							case ESpecifier::Inline:
 							case ESpecifier::Mutable:
-							case ESpecifier::Static_Member:
+							case ESpecifier::Static:
 							case ESpecifier::Volatile:
 							break;
 
@@ -6839,7 +7074,7 @@ namespace gen
 				case ESpecifier::Constexpr:
 				case ESpecifier::External_Linkage:
 				case ESpecifier::Inline:
-				case ESpecifier::Static_Member:
+				case ESpecifier::Static:
 				break;
 
 				default:
@@ -6951,7 +7186,7 @@ namespace gen
 				case ESpecifier::Const:
 				case ESpecifier::Constexpr:
 				case ESpecifier::Inline:
-				case ESpecifier::Static_Member:
+				case ESpecifier::Static:
 				break;
 
 				default:
@@ -7138,7 +7373,7 @@ namespace gen
 					case ESpecifier::Inline:
 					case ESpecifier::Local_Persist:
 					case ESpecifier::Mutable:
-					case ESpecifier::Static_Member:
+					case ESpecifier::Static:
 					case ESpecifier::Thread_Local:
 					case ESpecifier::Volatile:
 					break;
@@ -7602,7 +7837,7 @@ namespace gen
 				case ESpecifier::Inline:
 				case ESpecifier::Local_Persist:
 				case ESpecifier::Mutable:
-				case ESpecifier::Static_Member:
+				case ESpecifier::Static:
 				case ESpecifier::Thread_Local:
 				case ESpecifier::Volatile:
 				break;
@@ -7666,10 +7901,15 @@ namespace gen
 		char const* buf_begin = buf;
 		sw          remaining = buf_size;
 
+		static Arena tok_map_arena;
+
 		HashTable<StrC> tok_map;
 		{
-			// TODO : Switch this to use an arena that makes use of the stack (cap the size of the token table to around 4096 bytes)
-			tok_map = HashTable<StrC>::init( Memory::GlobalAllocator );
+			static char tok_map_mem[ TokenFmt_TokenMap_MemSize ];
+
+			tok_map_arena = Arena::init_from_memory( tok_map_mem, sizeof(tok_map_mem) );
+
+			tok_map = HashTable<StrC>::init( tok_map_arena );
 
 			s32 left = num_tokens - 1;
 
@@ -7747,6 +7987,7 @@ namespace gen
 		}
 
 		tok_map.clear();
+		tok_map_arena.free();
 
 		sw result = buf_size - remaining;
 

project/gen.hpp

@@ -8,10 +8,6 @@
 */
 #pragma once
 
-#define GEN_FEATURE_PARSING
-#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
-#define GEN_ENFORCE_STRONG_CODE_TYPES
-
 #ifdef gen_time
 #pragma region GENCPP DEPENDENCIES
 //! If its desired to roll your own dependencies, define GENCPP_PROVIDE_DEPENDENCIES before including this file.
@@ -1587,6 +1583,9 @@ namespace gen
 			return { str_len( str ), str };
 		}
 
+		// Currently sed with strings as a parameter to indicate to free after append.
+		constexpr sw FreeAfter = 0xF4EEAF7E4;
+
 		// Dynamic String
 		// This is directly based off the ZPL string api.
 		// They used a header pattern
@@ -1771,7 +1770,7 @@ namespace gen
 
 			bool append( const String other )
 			{
-				return append( other.Data, other.length() );
+				return append( other.Data, other.length() );;
 			}
 
 			bool append_fmt( char const* fmt, ... );
@@ -2442,7 +2441,7 @@ namespace gen
 		Entry( Ref,              & )                 \
 		Entry( Register,         register )          \
 		Entry( RValue,           && )                \
-		Entry( Static_Member,    static  )           \
+		Entry( Static,           static  )           \
 		Entry( Thread_Local,     thread_local )      \
 		Entry( Volatile,         volatile )
 
@@ -2757,6 +2756,7 @@ namespace gen
 		bool        has_entries();
 		bool        is_equal   ( AST* other );
 		String      to_string  ();
+		// String      to_string  ( BatchFreeStrings& defer ); // TODO : Implement BatchFree sub version.
 		char const* type_str();
 		bool        validate_body();
 
@@ -2848,17 +2848,6 @@ namespace gen
 		};
 	};
 
-	void assign( AST* field, AST* other )
-	{
-		if ( other->Parent )
-		{
-			field = other->duplicate();
-			return;
-		}
-
-		field = other;
-	}
-
 	struct AST_POD
 	{
 		union {
@@ -2908,16 +2897,9 @@ namespace gen
 	static_assert( sizeof(AST)     == sizeof(AST_POD), "ERROR: AST IS NOT POD" );
 	static_assert( sizeof(AST_POD) == AST_POD_Size,    "ERROR: AST POD is not size of AST_POD_Size" );
 
-	#ifdef GEN_ENFORCE_STRONG_CODE_TYPES
-		// Used when the its desired when omission is allowed in a definition.
-		#define NoCode      { nullptr }
-		#define CodeInvalid (* Code::Invalid.ast)
-	#else
-		// Used when the its desired when omission is allowed in a definition.
-		constexpr Code NoCode      = { nullptr };
-		constexpr Code CodeInvalid = Code::Invalid;
-	#endif
-
+	// Used when the its desired when omission is allowed in a definition.
+	#define NoCode      { nullptr }
+	#define CodeInvalid (* Code::Invalid.ast)
 
 #pragma region Code Types
 	#define Define_CodeType( Typename )                           \
@@ -3220,8 +3202,7 @@ namespace gen
 			char          _PAD_[ sizeof(SpecifierT) * AST::ArrSpecs_Cap ];
 			struct
 			{
-				char      _PAD_PROPERTIES_[ sizeof(AST*) * 4 ];
-				Code      Value;
+				char      _PAD_PROPERTIES_[ sizeof(AST*) * 5 ];
 			};
 		};
 		Code              Prev;
@@ -3613,6 +3594,7 @@ namespace gen
 
 	void set_allocator_data_arrays       ( AllocatorInfo data_array_allocator );
 	void set_allocator_code_pool         ( AllocatorInfo pool_allocator );
+	void set_allocator_lexer             ( AllocatorInfo lex_allocator );
 	void set_allocator_string_arena      ( AllocatorInfo string_allocator );
 	void set_allocator_string_table      ( AllocatorInfo string_allocator );
 	void set_allocator_type_table        ( AllocatorInfo type_reg_allocator );
@@ -3915,7 +3897,6 @@ namespace gen
 
 	constexpr s32 InitSize_DataArrays  = 16;
 	constexpr s32 InitSize_StringTable = megabytes(4);
-	constexpr s32 InitSize_TypeTable   = megabytes(4);
 
 	constexpr s32 CodePool_NumBlocks        = 4096;
 	constexpr s32 SizePer_StringArena       = megabytes(32);
@@ -3924,6 +3905,8 @@ namespace gen
 	constexpr s32 MaxNameLength             = 128;
 	constexpr s32 MaxUntypedStrLength       = kilobytes(640);
 	constexpr s32 StringTable_MaxHashLength = kilobytes(1);
+	constexpr s32 TokenFmt_TokenMap_MemSize	= kilobytes(4);
+	constexpr s32 LexAllocator_Size         = megabytes(10);
 
 	extern CodeType t_auto;
 	extern CodeType t_void;
@@ -4105,10 +4088,12 @@ namespace gen
 	Typename& Typename::operator =( Code other )                                         \
 	{                                                                                    \
 		if ( other.ast && other->Parent )                                                \
+		{                                                                                \
 			ast = rcast( decltype(ast), other.ast->duplicate() );                        \
+			rcast( AST*, ast)->Parent = nullptr;                                         \
+		}                                                                                \
                                                                                          \
 		ast = rcast( decltype(ast), other.ast );                                         \
-                                                                                         \
 		return *this;                                                                    \
 	}                                                                                    \
 	bool Typename::operator ==( Code other )                                             \

project/gen.undef.macros.hpp

@@ -1,6 +1,5 @@
 #if gen_time
 // This undefines the macros used by the gen library but are not necessary for the user.
-// TODO : This is incomplete until all dependencies are brough in from ZPL into bloat.
 
 #undef GEN_ARCH_64_BIT
 #undef GEN_ARCH_32_BIT

scripts/gencpp.natvis

@@ -365,8 +365,8 @@
 		<DisplayString Condition="ast != nullptr">{ast->Name} {ast->Type}</DisplayString>
 		<Expand>
 			<Item Name="Parent">ast->Parent</Item>
-			<Item Name="Prev">ast->Front</Item>
-			<Item Name="Next">ast->Back</Item>
+			<Item Name="Front">ast->Front</Item>
+			<Item Name="Back">ast->Back</Item>
 			<Item Name="NumEntries">ast->NumEntries</Item>
 		</Expand>
 	</Type>

test/Parsed/Array.Parsed.hpp

@@ -1,6 +1,9 @@
 #pragma once
 
 #if gen_time
+#define GEN_FEATURE_PARSING
+#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+#define GEN_ENFORCE_STRONG_CODE_TYPES
 #include "gen.hpp"
 
 using namespace gen;

test/Parsed/Buffer.Parsed.hpp

@@ -1,6 +1,9 @@
 #pragma once
 
 #if gen_time
+#define GEN_FEATURE_PARSING
+#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+#define GEN_ENFORCE_STRONG_CODE_TYPES
 #include "gen.hpp"
 
 using namespace gen;

test/Parsed/HashTable.Parsed.hpp

@@ -1,6 +1,9 @@
 #pragma once
 
 #if gen_time
+#define GEN_FEATURE_PARSING
+#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+#define GEN_ENFORCE_STRONG_CODE_TYPES
 #include "gen.hpp"
 #include "Array.Parsed.hpp"
 

test/Parsed/Ring.Parsed.hpp

@@ -1,6 +1,9 @@
 #pragma once
 
 #if gen_time
+#define GEN_FEATURE_PARSING
+#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+#define GEN_ENFORCE_STRONG_CODE_TYPES
 #include "gen.hpp"
 #include "Buffer.Parsed.hpp"
 

test/Parsed/Sanity.Parsed.hpp

@@ -1,5 +1,8 @@
 #pragma once
 #ifdef gen_time
+#define GEN_FEATURE_PARSING
+#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+#define GEN_ENFORCE_STRONG_CODE_TYPES
 #include "gen.hpp"
 
 using namespace gen;

test/SOA.hpp

@@ -1,6 +1,9 @@
 #pragma once
 
 #ifdef gen_time
+#define GEN_FEATURE_PARSING
+#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+#define GEN_ENFORCE_STRONG_CODE_TYPES
 #include "gen.hpp"
 using namespace gen;
 
@@ -11,7 +14,7 @@ Code gen_SOA( CodeStruct struct_def, s32 num_entries = 0 )
 	));
 
 	Code
-	soa_entry = { struct_def.raw()->duplicate() };
+	soa_entry = { struct_def.duplicate() };
 	soa_entry->Name = get_cached_string( name(Entry) );
 
 	constexpr s32 Num_Vars_Cap = 128;

test/Upfront/Array.Upfront.hpp

@@ -18,7 +18,7 @@ Code gen__array_base()
 
 	CodeFn grow_formula = def_function( name(array_grow_formula), def_param( t_uw, name(value)), t_uw
 		, def_execution( code( return 2 * value * 8; ) )
-		, def_specifiers( args( ESpecifier::Static_Member, ESpecifier::Inline ) )
+		, def_specifiers( args( ESpecifier::Static, ESpecifier::Inline ) )
 	);
 
 	return def_global_body( args( header, grow_formula ) );
@@ -29,9 +29,9 @@ Code gen__array( StrC type )
 	static CodeType t_allocator_info = def_type( name(AllocatorInfo) );
 	static Code     v_nullptr        = code_str(nullptr);
 
-	static CodeSpecifier spec_ct_member     = def_specifiers( 2, ESpecifier::Constexpr, ESpecifier::Static_Member );
-	static CodeSpecifier spec_static_inline = def_specifiers( 2, ESpecifier::Static_Member, ESpecifier::Inline );
-	static CodeSpecifier spec_static        = def_specifier( ESpecifier::Static_Member );
+	static CodeSpecifier spec_ct_member     = def_specifiers( 2, ESpecifier::Constexpr, ESpecifier::Static );
+	static CodeSpecifier spec_static_inline = def_specifiers( 2, ESpecifier::Static, ESpecifier::Inline );
+	static CodeSpecifier spec_static        = def_specifier( ESpecifier::Static );
 
 	static CodeUsing using_header    = def_using( name(Header), def_type( name(ArrayHeader) ) );
 	static CodeVar ct_grow_formula = def_variable( t_auto, name(grow_formula), untyped_str( code( & array_grow_formula )), spec_ct_member );

test/test.Upfront.cpp

@@ -15,7 +15,6 @@ using namespace gen;
 
 int gen_main()
 {
-	Memory::setup();
 	gen::init();
 
 	gen_sanity_upfront();
@@ -35,7 +34,6 @@ int gen_main()
 	gen_ring_file();
 
 	gen::deinit();
-	Memory::cleanup();
 	return 0;
 }
 #endif

test/test.cpp

@@ -1,6 +1,7 @@
 #ifdef gen_time
-#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
 #define GEN_FEATURE_PARSING
+#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
+#define GEN_ENFORCE_STRONG_CODE_TYPES
 #include "Parsed\Array.Parsed.hpp"
 #include "Parsed\Buffer.Parsed.hpp"
 #include "Parsed\HashTable.Parsed.hpp"
@@ -11,11 +12,10 @@
 
 using namespace gen;
 
-// TODO : Rewrite this to include both upfront and parsed testing.
+// TODO : Need to make a more robust test suite
 
 int gen_main()
 {
-	Memory::setup();
 	gen::init();
 
 	gen_sanity();
@@ -60,7 +60,6 @@ int gen_main()
 	soa_test.write();
 
 	gen::deinit();
-	Memory::cleanup();
 	return 0;
 }
 #endif