Minor refactor, added optional recursive dups for ast, ...

- Added support for anonymous structs.
- Gave Token_Fmt::token_map its own static memory.
- Minor natvis fix for CodeBody
- Renamed ESpecifier::Static_Member to just Static (acts as a general use case) specifier option
- Setup the lex token array with a configurable arena allocator.

Two major things left before V0.3-4:
- Attribute and Module parisng support with base case test
- AST serializtaion strings get a dedicated slag allocator.
This commit is contained in:
Edward R. Gonzalez 2023-07-16 02:26:55 -04:00
parent 79c3459f08
commit 1f77e39694
14 changed files with 331 additions and 89 deletions

View File

@ -21,7 +21,7 @@ These build up a code AST to then serialize with a file builder.
## Notes ## Notes
The project has reached a sort of *alpha* state, all the current functionality works for the test cases but it will most likely break in many other cases. The project has reached an *alpha* state, all the current functionality works for the test cases but it will most likely break in many other cases.
The project has no external dependencies beyond: The project has no external dependencies beyond:
@ -36,13 +36,12 @@ The project has no external dependencies beyond:
Dependencies for the project are wrapped within `GENCPP_ROLL_OWN_DEPENDENCIES` (Defining it will disable them). Dependencies for the project are wrapped within `GENCPP_ROLL_OWN_DEPENDENCIES` (Defining it will disable them).
The majority of the dependency's implementation was derived from the [c-zpl library](https://github.com/zpl-c/zpl). The majority of the dependency's implementation was derived from the [c-zpl library](https://github.com/zpl-c/zpl).
When gencpp is in a stable state, I will make a C variant with the same feature set.
A single-header version will also be generated for both.
A `natvis` and `natstepfilter` are provided in the scripts directory. A `natvis` and `natstepfilter` are provided in the scripts directory.
***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.*** ***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.***
A C variant is hosted [here](https://github.com/Ed94/genc); I haven't gotten headwind on it, should be easier to make than this...
## Usage ## Usage
A metaprogram is built to generate files before the main program is built. We'll term runtime for this program as `gen_time`. The metaprogram's core implementation are within `gen.hpp` and `gen.cpp` in the project directory. A metaprogram is built to generate files before the main program is built. We'll term runtime for this program as `gen_time`. The metaprogram's core implementation are within `gen.hpp` and `gen.cpp` in the project directory.
@ -159,10 +158,12 @@ This method is setup where all the metaprogram's code are the within the same fi
### *WHAT IS NOT PROVIDED* ### *WHAT IS NOT PROVIDED*
* Lambdas * Lambdas
* Vendor provided dynamic dispatch (virtuals) : `override` and `final` specifiers complicate the specifier parsing and serialization. (I'll problably end up adding in later) * Lang provided dynamic dispatch (virtuals) : `override` and `final` specifiers complicate the specifier parsing and serialization. (Its a todo)
* Suffix specifiers for functions (Ex: void() const ). Same reason as virtual/override/final missing for now.
* RTTI * RTTI
* Exceptions * Exceptions
* Execution statement validation : Execution expressions are defined using the untyped API. * Execution statement validation : Execution expressions are defined using the untyped API.
* Parsing support for module specifiers and attributes. (Its a todo)
Keywords kept from "Modern C++": Keywords kept from "Modern C++":
@ -664,8 +665,9 @@ Names or Content fields are interned strings and thus showed be cached using `ge
# TODO # TODO
* Implement a context stack for the parsing, allows for accurate scope validation for the AST types. * Implement a context stack for the parsing, allows for accurate scope validation for the AST types.
* Make a test suite thats covers some base cases and zpl containers (+ anything else suitable) * Make a more robust test suite.
* Finish support for module specifiers and standard/platform attributes.
* Generate a single-header library. * Generate a single-header library.
* Improve the allocation strategy for strings in `AST::to_string`, `Parser::lex`, and `token_fmt_va` * Improve the allocation strategy for strings in `Builder`, `AST::to_string`, `Parser::lex`, all three can use some form of slab allocation strategy...
* May be in need of a better name, I found a few repos with this same one... * May be in need of a better name, I found a few repos with this same one...
* Support module and attribute parsing (Marked with TODOs for now..)
* Suffix specifiers for functions (const, override, final)

View File

@ -1712,8 +1712,13 @@ namespace gen
global StringTable StringCache; global StringTable StringCache;
// TODO : Need to implement String memory management for seriaization intermediates.
global Arena LexArena;
global AllocatorInfo Allocator_DataArrays = heap(); global AllocatorInfo Allocator_DataArrays = heap();
global AllocatorInfo Allocator_CodePool = heap(); global AllocatorInfo Allocator_CodePool = heap();
global AllocatorInfo Allocator_Lexer = heap();
global AllocatorInfo Allocator_StringArena = heap(); global AllocatorInfo Allocator_StringArena = heap();
global AllocatorInfo Allocator_StringTable = heap(); global AllocatorInfo Allocator_StringTable = heap();
global AllocatorInfo Allocator_TypeTable = heap(); global AllocatorInfo Allocator_TypeTable = heap();
@ -1852,12 +1857,215 @@ namespace gen
{ {
using namespace ECode; using namespace ECode;
Code result = make_code(); AST*
result = make_code().ast;
#ifndef GEN_USE_RECURSIVE_AST_DUPLICATION
mem_copy( result, this, sizeof( AST ) );
result->Parent = nullptr;
#else
switch ( Type )
{
case Invalid:
log_failure("Attempted to duplicate invalid code! - \n%s", Parent ? Parent->debug_str() : Name );
return nullptr
case Untyped:
case Comment:
case Execution:
case Access_Private:
case Access_Protected:
case Access_Public:
case PlatformAttributes:
case Preprocessor_Include:
case Module:
case Specifiers:
case Using_Namespace:
mem_copy( result, this, sizeof( AST ) );
break;
// TODO : Bring back some of the old way, we need to recursively duplicate the children. case Extern_Linkage:
mem_copy( result.ast, this, sizeof( AST ) ); case Friend:
mem_copy( result, this, sizeof( AST ) );
return result.ast; if (Value)
result->Value = Value->duplicate();
break;
case Class:
case Struct:
case Enum:
mem_copy( result, this, sizeof( AST ) );
if ( Attributes)
result->Attributes = Attributes->duplicate();
if ( ParentType )
result->ParentType = ParentType->duplicate();
result->Body = Body->duplicate();
break;
case Enum_Fwd:
case Class_Fwd:
case Struct_Fwd:
mem_copy( result, this, sizeof( AST ) );
if ( Attributes)
result->Attributes = Attributes->duplicate();
if ( ParentType )
result->ParentType = ParentType->duplicate();
break;
case Function:
case Operator:
case Operator_Member:
mem_copy( result, this, sizeof( AST ) );
if ( Attributes)
result->Attributes = Attributes->duplicate();
if ( Specs )
result->ParentType = ParentType->duplicate();
if ( ReturnType )
result->ReturnType = ReturnType->duplicate();
if ( Params )
result->Params = Params->duplicate();
result->Body = Body->duplicate();
break;
case Function_Fwd:
case Operator_Fwd:
case Operator_Member_Fwd:
mem_copy( result, this, sizeof( AST ) );
if ( Attributes)
result->Attributes = Attributes->duplicate();
if ( Specs )
result->ParentType = ParentType->duplicate();
if ( ReturnType )
result->ReturnType = ReturnType->duplicate();
if ( Params )
result->Params = Params->duplicate();
break;
case Namespace:
mem_copy( result, this, sizeof( AST ) );
result->Body = Body->duplicate();
break;
case Operator_Cast:
mem_copy( result, this, sizeof( AST ) );
result->ValueType = ValueType->duplicate();
result->Body = Body->duplicate();
break;
case Operator_Cast_Fwd:
mem_copy( result, this, sizeof( AST ) );
result->ValueType = ValueType->duplicate();
break;
case Parameters:
mem_copy( result, this, sizeof( AST ) );
result->NumEntries = 0;
result->Last = nullptr;
result->Next = nullptr;
if ( NumEntries - 1 > 0 )
{
CodeParam parent = result->cast<CodeParam>();
for ( CodeParam param : Next->cast<CodeParam>() )
{
parent.append( param );
}
}
break;
case Template:
mem_copy( result, this, sizeof( AST ) );
result->Params = Params->duplicate();
result->Declaration = Declaration->duplicate();
break;
case Typename:
mem_copy( result, this, sizeof( AST ) );
if (Attributes)
result->Attributes = Attributes->duplicate();
if ( Specs )
result->Specs = Specs->duplicate();
if ( ArrExpr )
result->ArrExpr = ArrExpr->duplicate();
break;
case Typedef:
case Using:
mem_copy( result, this, sizeof( AST ) );
if (Attributes)
result->Attributes = Attributes->duplicate();
if ( UnderlyingType )
result->UnderlyingType = UnderlyingType->duplicate();
break;
case Union:
mem_copy( result, this, sizeof( AST ) );
if ( Attributes)
result->Attributes = Attributes->duplicate();
result->Body = Body->duplicate();
break;
case Variable:
mem_copy( result, this, sizeof( AST ) );
if (Attributes)
result->Attributes = Attributes->duplicate();
if ( Specs )
result->Specs = Specs->duplicate();
result->ValueType = UnderlyingType->duplicate();
if ( Value )
result->Value = Value->duplicate();
break;
case Class_Body:
case Enum_Body:
case Export_Body:
case Extern_Linkage_Body:
case Function_Body:
case Global_Body:
case Namespace_Body:
case Struct_Body:
case Union_Body:
CodeBody
body = cast<CodeBody>();
body->Name = Name;
body->Type = Type;
for ( Code entry : cast<CodeBody>() )
{
result->append( entry.ast );
}
break;
}
#endif
return result;
} }
String AST::to_string() String AST::to_string()
@ -1869,6 +2077,7 @@ namespace gen
if ( bitfield_is_equal( u32, ModuleFlags, ModuleFlag::Import )) \ if ( bitfield_is_equal( u32, ModuleFlags, ModuleFlag::Import )) \
result.append( "import " ); \ result.append( "import " ); \
// TODO : Need to refactor so that intermeidate strings are freed conviently.
String result = String::make( Memory::GlobalAllocator, "" ); String result = String::make( Memory::GlobalAllocator, "" );
switch ( Type ) switch ( Type )
@ -1876,7 +2085,7 @@ namespace gen
using namespace ECode; using namespace ECode;
case Invalid: case Invalid:
log_failure("Attempted to serialize invalid code! - %s", Name); log_failure("Attempted to serialize invalid code! - %s", Parent ? Parent->debug_str() : Name );
break; break;
case Untyped: case Untyped:
@ -1926,7 +2135,9 @@ namespace gen
result.append( "class " ); result.append( "class " );
if ( Attributes ) if ( Attributes )
{
result.append_fmt( "%s ", Attributes->to_string() ); result.append_fmt( "%s ", Attributes->to_string() );
}
if ( ParentType ) if ( ParentType )
{ {
@ -2216,14 +2427,15 @@ namespace gen
case Parameters: case Parameters:
{ {
// TODO : Parameters can have default values.
if ( Name ) if ( Name )
result.append_fmt( "%s %s", ValueType->to_string(), Name ); result.append_fmt( "%s %s", ValueType->to_string(), Name );
else else
result.append_fmt( "%s", ValueType->to_string() ); result.append_fmt( "%s", ValueType->to_string() );
if ( Value )
result.append_fmt( "= %s", Value->to_string() );
if ( NumEntries - 1 > 0) if ( NumEntries - 1 > 0)
{ {
for ( CodeParam param : Next->cast<CodeParam>() ) for ( CodeParam param : Next->cast<CodeParam>() )
@ -2254,15 +2466,19 @@ namespace gen
{ {
ProcessModuleFlags(); ProcessModuleFlags();
if ( Name == nullptr)
{
result.append( "struct\n{\n%s\n};", Body->to_string() );
break;
}
if ( Attributes || ParentType ) if ( Attributes || ParentType )
{ {
result.append( "struct " ); result.append( "struct " );
if ( ParentType ) if ( Attributes )
result.append_fmt( "%s ", Attributes->to_string() ); result.append_fmt( "%s ", Attributes->to_string() );
// TODO : Structs can be anonymous, so we need to check for that
if ( ParentType ) if ( ParentType )
{ {
char const* access_level = to_str( ParentAccess ); char const* access_level = to_str( ParentAccess );
@ -2276,6 +2492,8 @@ namespace gen
} }
else else
{ {
if ( Name )
result.append_fmt( "%s \n{\n%s\n};", Name, Body->to_string() ); result.append_fmt( "%s \n{\n%s\n};", Name, Body->to_string() );
} }
} }
@ -2465,7 +2683,6 @@ namespace gen
switch ( Type ) switch ( Type )
{ {
// TODO : Finish this...
case ECode::Typedef: case ECode::Typedef:
case ECode::Typename: case ECode::Typename:
{ {
@ -2559,6 +2776,8 @@ namespace gen
{ {
using namespace StaticData; using namespace StaticData;
Memory::setup();
// Setup the arrays // Setup the arrays
{ {
CodePools = Array<Pool>::init_reserve( Allocator_DataArrays, InitSize_DataArrays ); CodePools = Array<Pool>::init_reserve( Allocator_DataArrays, InitSize_DataArrays );
@ -2581,6 +2800,10 @@ namespace gen
CodePools.append( code_pool ); CodePools.append( code_pool );
#ifdef GEN_FEATURE_PARSING
LexArena = Arena::init_from_allocator( Allocator_Lexer, LexAllocator_Size );
#endif
Arena string_arena = Arena::init_from_allocator( Allocator_StringArena, SizePer_StringArena ); Arena string_arena = Arena::init_from_allocator( Allocator_StringArena, SizePer_StringArena );
if ( string_arena.PhysicalStart == nullptr ) if ( string_arena.PhysicalStart == nullptr )
@ -2696,7 +2919,7 @@ namespace gen
def_constant_spec( ref, ESpecifier::Ref ); def_constant_spec( ref, ESpecifier::Ref );
def_constant_spec( register, ESpecifier::Register ); def_constant_spec( register, ESpecifier::Register );
def_constant_spec( rvalue, ESpecifier::RValue ); def_constant_spec( rvalue, ESpecifier::RValue );
def_constant_spec( static_member, ESpecifier::Static_Member ); def_constant_spec( static_member, ESpecifier::Static );
def_constant_spec( thread_local, ESpecifier::Thread_Local ); def_constant_spec( thread_local, ESpecifier::Thread_Local );
def_constant_spec( volatile, ESpecifier::Volatile) def_constant_spec( volatile, ESpecifier::Volatile)
@ -2738,6 +2961,12 @@ namespace gen
CodePools.free(); CodePools.free();
StringArenas.free(); StringArenas.free();
#ifdef GEN_FEATURE_PARSING
LexArena.free();
#endif
Memory::cleanup();
} }
AllocatorInfo get_string_allocator( s32 str_length ) AllocatorInfo get_string_allocator( s32 str_length )
@ -3169,6 +3398,11 @@ namespace gen
StaticData::Allocator_CodePool = allocator; StaticData::Allocator_CodePool = allocator;
} }
void set_allocator_lexer( AllocatorInfo allocator )
{
StaticData::Allocator_Lexer = allocator;
}
void set_allocator_string_arena( AllocatorInfo allocator ) void set_allocator_string_arena( AllocatorInfo allocator )
{ {
StaticData::Allocator_StringArena = allocator; StaticData::Allocator_StringArena = allocator;
@ -3764,8 +3998,6 @@ namespace gen
{ {
using namespace ECode; using namespace ECode;
name_check( def_struct, name );
if ( attributes && attributes->Type != PlatformAttributes ) if ( attributes && attributes->Type != PlatformAttributes )
{ {
log_failure( "gen::def_struct: attributes was not a `PlatformAttributes` type - %s", attributes.debug_str() ); log_failure( "gen::def_struct: attributes was not a `PlatformAttributes` type - %s", attributes.debug_str() );
@ -3786,9 +4018,11 @@ namespace gen
CodeStruct CodeStruct
result = (CodeStruct) make_code(); result = (CodeStruct) make_code();
result->Name = get_cached_string( name );
result->ModuleFlags = mflags; result->ModuleFlags = mflags;
if ( name )
result->Name = get_cached_string( name );
if ( body ) if ( body )
{ {
result->Type = Struct; result->Type = Struct;
@ -4816,7 +5050,6 @@ namespace gen
return scast(AccessSpec, tok.Type); return scast(AccessSpec, tok.Type);
} }
Arena LexAllocator;
struct TokArray struct TokArray
{ {
@ -4885,16 +5118,15 @@ namespace gen
return { 0, nullptr }; \ return { 0, nullptr }; \
} }
do_once_start // do_once_start
// TODO : Use the global memory allocator for this... // LexAllocator = Arena::init_from_allocator( heap(), megabytes(10) );
LexAllocator = Arena::init_from_allocator( heap(), megabytes(10) );
if ( LexAllocator.PhysicalStart == nullptr ) // if ( LexAllocator.PhysicalStart == nullptr )
{ // {
log_failure( "gen::lex: failed to allocate memory for parsing constructor's lexer"); // log_failure( "gen::lex: failed to allocate memory for parsing constructor's lexer");
return { { nullptr }, 0 }; // return { { nullptr }, 0 };
} // }
do_once_end // do_once_end
local_persist thread_local local_persist thread_local
Array<Token> Tokens = { nullptr }; Array<Token> Tokens = { nullptr };
@ -4913,9 +5145,11 @@ namespace gen
} }
if ( Tokens ) if ( Tokens )
Tokens.clear(); {
Tokens.free();
}
Tokens = Array<Token>::init_reserve( LexAllocator, content.Len / 8 ); Tokens = Array<Token>::init_reserve( StaticData::LexArena, content.Len / 6 );
while (left ) while (left )
{ {
@ -6192,7 +6426,7 @@ namespace gen
case ESpecifier::Constinit: case ESpecifier::Constinit:
case ESpecifier::Inline: case ESpecifier::Inline:
case ESpecifier::Mutable: case ESpecifier::Mutable:
case ESpecifier::Static_Member: case ESpecifier::Static:
case ESpecifier::Volatile: case ESpecifier::Volatile:
break; break;
@ -6279,6 +6513,7 @@ namespace gen
// TODO : Parse attributes // TODO : Parse attributes
if ( check( TokType::Identifier ) )
name = parse_identifier( toks, context ); name = parse_identifier( toks, context );
AccessSpec access = AccessSpec::Default; AccessSpec access = AccessSpec::Default;
@ -6500,7 +6735,7 @@ namespace gen
case ESpecifier::Constinit: case ESpecifier::Constinit:
case ESpecifier::Inline: case ESpecifier::Inline:
case ESpecifier::Mutable: case ESpecifier::Mutable:
case ESpecifier::Static_Member: case ESpecifier::Static:
case ESpecifier::Volatile: case ESpecifier::Volatile:
break; break;
@ -6839,7 +7074,7 @@ namespace gen
case ESpecifier::Constexpr: case ESpecifier::Constexpr:
case ESpecifier::External_Linkage: case ESpecifier::External_Linkage:
case ESpecifier::Inline: case ESpecifier::Inline:
case ESpecifier::Static_Member: case ESpecifier::Static:
break; break;
default: default:
@ -6951,7 +7186,7 @@ namespace gen
case ESpecifier::Const: case ESpecifier::Const:
case ESpecifier::Constexpr: case ESpecifier::Constexpr:
case ESpecifier::Inline: case ESpecifier::Inline:
case ESpecifier::Static_Member: case ESpecifier::Static:
break; break;
default: default:
@ -7138,7 +7373,7 @@ namespace gen
case ESpecifier::Inline: case ESpecifier::Inline:
case ESpecifier::Local_Persist: case ESpecifier::Local_Persist:
case ESpecifier::Mutable: case ESpecifier::Mutable:
case ESpecifier::Static_Member: case ESpecifier::Static:
case ESpecifier::Thread_Local: case ESpecifier::Thread_Local:
case ESpecifier::Volatile: case ESpecifier::Volatile:
break; break;
@ -7602,7 +7837,7 @@ namespace gen
case ESpecifier::Inline: case ESpecifier::Inline:
case ESpecifier::Local_Persist: case ESpecifier::Local_Persist:
case ESpecifier::Mutable: case ESpecifier::Mutable:
case ESpecifier::Static_Member: case ESpecifier::Static:
case ESpecifier::Thread_Local: case ESpecifier::Thread_Local:
case ESpecifier::Volatile: case ESpecifier::Volatile:
break; break;
@ -7666,10 +7901,15 @@ namespace gen
char const* buf_begin = buf; char const* buf_begin = buf;
sw remaining = buf_size; sw remaining = buf_size;
static Arena tok_map_arena;
HashTable<StrC> tok_map; HashTable<StrC> tok_map;
{ {
// TODO : Switch this to use an arena that makes use of the stack (cap the size of the token table to around 4096 bytes) static char tok_map_mem[ TokenFmt_TokenMap_MemSize ];
tok_map = HashTable<StrC>::init( Memory::GlobalAllocator );
tok_map_arena = Arena::init_from_memory( tok_map_mem, sizeof(tok_map_mem) );
tok_map = HashTable<StrC>::init( tok_map_arena );
s32 left = num_tokens - 1; s32 left = num_tokens - 1;
@ -7747,6 +7987,7 @@ namespace gen
} }
tok_map.clear(); tok_map.clear();
tok_map_arena.free();
sw result = buf_size - remaining; sw result = buf_size - remaining;

View File

@ -8,10 +8,6 @@
*/ */
#pragma once #pragma once
#define GEN_FEATURE_PARSING
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#ifdef gen_time #ifdef gen_time
#pragma region GENCPP DEPENDENCIES #pragma region GENCPP DEPENDENCIES
//! If its desired to roll your own dependencies, define GENCPP_PROVIDE_DEPENDENCIES before including this file. //! If its desired to roll your own dependencies, define GENCPP_PROVIDE_DEPENDENCIES before including this file.
@ -1587,6 +1583,9 @@ namespace gen
return { str_len( str ), str }; return { str_len( str ), str };
} }
// Currently sed with strings as a parameter to indicate to free after append.
constexpr sw FreeAfter = 0xF4EEAF7E4;
// Dynamic String // Dynamic String
// This is directly based off the ZPL string api. // This is directly based off the ZPL string api.
// They used a header pattern // They used a header pattern
@ -1771,7 +1770,7 @@ namespace gen
bool append( const String other ) bool append( const String other )
{ {
return append( other.Data, other.length() ); return append( other.Data, other.length() );;
} }
bool append_fmt( char const* fmt, ... ); bool append_fmt( char const* fmt, ... );
@ -2442,7 +2441,7 @@ namespace gen
Entry( Ref, & ) \ Entry( Ref, & ) \
Entry( Register, register ) \ Entry( Register, register ) \
Entry( RValue, && ) \ Entry( RValue, && ) \
Entry( Static_Member, static ) \ Entry( Static, static ) \
Entry( Thread_Local, thread_local ) \ Entry( Thread_Local, thread_local ) \
Entry( Volatile, volatile ) Entry( Volatile, volatile )
@ -2757,6 +2756,7 @@ namespace gen
bool has_entries(); bool has_entries();
bool is_equal ( AST* other ); bool is_equal ( AST* other );
String to_string (); String to_string ();
// String to_string ( BatchFreeStrings& defer ); // TODO : Implement BatchFree sub version.
char const* type_str(); char const* type_str();
bool validate_body(); bool validate_body();
@ -2848,17 +2848,6 @@ namespace gen
}; };
}; };
void assign( AST* field, AST* other )
{
if ( other->Parent )
{
field = other->duplicate();
return;
}
field = other;
}
struct AST_POD struct AST_POD
{ {
union { union {
@ -2908,16 +2897,9 @@ namespace gen
static_assert( sizeof(AST) == sizeof(AST_POD), "ERROR: AST IS NOT POD" ); static_assert( sizeof(AST) == sizeof(AST_POD), "ERROR: AST IS NOT POD" );
static_assert( sizeof(AST_POD) == AST_POD_Size, "ERROR: AST POD is not size of AST_POD_Size" ); static_assert( sizeof(AST_POD) == AST_POD_Size, "ERROR: AST POD is not size of AST_POD_Size" );
#ifdef GEN_ENFORCE_STRONG_CODE_TYPES
// Used when the its desired when omission is allowed in a definition. // Used when the its desired when omission is allowed in a definition.
#define NoCode { nullptr } #define NoCode { nullptr }
#define CodeInvalid (* Code::Invalid.ast) #define CodeInvalid (* Code::Invalid.ast)
#else
// Used when the its desired when omission is allowed in a definition.
constexpr Code NoCode = { nullptr };
constexpr Code CodeInvalid = Code::Invalid;
#endif
#pragma region Code Types #pragma region Code Types
#define Define_CodeType( Typename ) \ #define Define_CodeType( Typename ) \
@ -3220,8 +3202,7 @@ namespace gen
char _PAD_[ sizeof(SpecifierT) * AST::ArrSpecs_Cap ]; char _PAD_[ sizeof(SpecifierT) * AST::ArrSpecs_Cap ];
struct struct
{ {
char _PAD_PROPERTIES_[ sizeof(AST*) * 4 ]; char _PAD_PROPERTIES_[ sizeof(AST*) * 5 ];
Code Value;
}; };
}; };
Code Prev; Code Prev;
@ -3613,6 +3594,7 @@ namespace gen
void set_allocator_data_arrays ( AllocatorInfo data_array_allocator ); void set_allocator_data_arrays ( AllocatorInfo data_array_allocator );
void set_allocator_code_pool ( AllocatorInfo pool_allocator ); void set_allocator_code_pool ( AllocatorInfo pool_allocator );
void set_allocator_lexer ( AllocatorInfo lex_allocator );
void set_allocator_string_arena ( AllocatorInfo string_allocator ); void set_allocator_string_arena ( AllocatorInfo string_allocator );
void set_allocator_string_table ( AllocatorInfo string_allocator ); void set_allocator_string_table ( AllocatorInfo string_allocator );
void set_allocator_type_table ( AllocatorInfo type_reg_allocator ); void set_allocator_type_table ( AllocatorInfo type_reg_allocator );
@ -3915,7 +3897,6 @@ namespace gen
constexpr s32 InitSize_DataArrays = 16; constexpr s32 InitSize_DataArrays = 16;
constexpr s32 InitSize_StringTable = megabytes(4); constexpr s32 InitSize_StringTable = megabytes(4);
constexpr s32 InitSize_TypeTable = megabytes(4);
constexpr s32 CodePool_NumBlocks = 4096; constexpr s32 CodePool_NumBlocks = 4096;
constexpr s32 SizePer_StringArena = megabytes(32); constexpr s32 SizePer_StringArena = megabytes(32);
@ -3924,6 +3905,8 @@ namespace gen
constexpr s32 MaxNameLength = 128; constexpr s32 MaxNameLength = 128;
constexpr s32 MaxUntypedStrLength = kilobytes(640); constexpr s32 MaxUntypedStrLength = kilobytes(640);
constexpr s32 StringTable_MaxHashLength = kilobytes(1); constexpr s32 StringTable_MaxHashLength = kilobytes(1);
constexpr s32 TokenFmt_TokenMap_MemSize = kilobytes(4);
constexpr s32 LexAllocator_Size = megabytes(10);
extern CodeType t_auto; extern CodeType t_auto;
extern CodeType t_void; extern CodeType t_void;
@ -4105,10 +4088,12 @@ namespace gen
Typename& Typename::operator =( Code other ) \ Typename& Typename::operator =( Code other ) \
{ \ { \
if ( other.ast && other->Parent ) \ if ( other.ast && other->Parent ) \
{ \
ast = rcast( decltype(ast), other.ast->duplicate() ); \ ast = rcast( decltype(ast), other.ast->duplicate() ); \
rcast( AST*, ast)->Parent = nullptr; \
} \
\ \
ast = rcast( decltype(ast), other.ast ); \ ast = rcast( decltype(ast), other.ast ); \
\
return *this; \ return *this; \
} \ } \
bool Typename::operator ==( Code other ) \ bool Typename::operator ==( Code other ) \

View File

@ -1,6 +1,5 @@
#if gen_time #if gen_time
// This undefines the macros used by the gen library but are not necessary for the user. // This undefines the macros used by the gen library but are not necessary for the user.
// TODO : This is incomplete until all dependencies are brough in from ZPL into bloat.
#undef GEN_ARCH_64_BIT #undef GEN_ARCH_64_BIT
#undef GEN_ARCH_32_BIT #undef GEN_ARCH_32_BIT

View File

@ -365,8 +365,8 @@
<DisplayString Condition="ast != nullptr">{ast->Name} {ast->Type}</DisplayString> <DisplayString Condition="ast != nullptr">{ast->Name} {ast->Type}</DisplayString>
<Expand> <Expand>
<Item Name="Parent">ast->Parent</Item> <Item Name="Parent">ast->Parent</Item>
<Item Name="Prev">ast->Front</Item> <Item Name="Front">ast->Front</Item>
<Item Name="Next">ast->Back</Item> <Item Name="Back">ast->Back</Item>
<Item Name="NumEntries">ast->NumEntries</Item> <Item Name="NumEntries">ast->NumEntries</Item>
</Expand> </Expand>
</Type> </Type>

View File

@ -1,6 +1,9 @@
#pragma once #pragma once
#if gen_time #if gen_time
#define GEN_FEATURE_PARSING
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.hpp" #include "gen.hpp"
using namespace gen; using namespace gen;

View File

@ -1,6 +1,9 @@
#pragma once #pragma once
#if gen_time #if gen_time
#define GEN_FEATURE_PARSING
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.hpp" #include "gen.hpp"
using namespace gen; using namespace gen;

View File

@ -1,6 +1,9 @@
#pragma once #pragma once
#if gen_time #if gen_time
#define GEN_FEATURE_PARSING
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.hpp" #include "gen.hpp"
#include "Array.Parsed.hpp" #include "Array.Parsed.hpp"

View File

@ -1,6 +1,9 @@
#pragma once #pragma once
#if gen_time #if gen_time
#define GEN_FEATURE_PARSING
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.hpp" #include "gen.hpp"
#include "Buffer.Parsed.hpp" #include "Buffer.Parsed.hpp"

View File

@ -1,5 +1,8 @@
#pragma once #pragma once
#ifdef gen_time #ifdef gen_time
#define GEN_FEATURE_PARSING
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.hpp" #include "gen.hpp"
using namespace gen; using namespace gen;

View File

@ -1,6 +1,9 @@
#pragma once #pragma once
#ifdef gen_time #ifdef gen_time
#define GEN_FEATURE_PARSING
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.hpp" #include "gen.hpp"
using namespace gen; using namespace gen;
@ -11,7 +14,7 @@ Code gen_SOA( CodeStruct struct_def, s32 num_entries = 0 )
)); ));
Code Code
soa_entry = { struct_def.raw()->duplicate() }; soa_entry = { struct_def.duplicate() };
soa_entry->Name = get_cached_string( name(Entry) ); soa_entry->Name = get_cached_string( name(Entry) );
constexpr s32 Num_Vars_Cap = 128; constexpr s32 Num_Vars_Cap = 128;

View File

@ -18,7 +18,7 @@ Code gen__array_base()
CodeFn grow_formula = def_function( name(array_grow_formula), def_param( t_uw, name(value)), t_uw CodeFn grow_formula = def_function( name(array_grow_formula), def_param( t_uw, name(value)), t_uw
, def_execution( code( return 2 * value * 8; ) ) , def_execution( code( return 2 * value * 8; ) )
, def_specifiers( args( ESpecifier::Static_Member, ESpecifier::Inline ) ) , def_specifiers( args( ESpecifier::Static, ESpecifier::Inline ) )
); );
return def_global_body( args( header, grow_formula ) ); return def_global_body( args( header, grow_formula ) );
@ -29,9 +29,9 @@ Code gen__array( StrC type )
static CodeType t_allocator_info = def_type( name(AllocatorInfo) ); static CodeType t_allocator_info = def_type( name(AllocatorInfo) );
static Code v_nullptr = code_str(nullptr); static Code v_nullptr = code_str(nullptr);
static CodeSpecifier spec_ct_member = def_specifiers( 2, ESpecifier::Constexpr, ESpecifier::Static_Member ); static CodeSpecifier spec_ct_member = def_specifiers( 2, ESpecifier::Constexpr, ESpecifier::Static );
static CodeSpecifier spec_static_inline = def_specifiers( 2, ESpecifier::Static_Member, ESpecifier::Inline ); static CodeSpecifier spec_static_inline = def_specifiers( 2, ESpecifier::Static, ESpecifier::Inline );
static CodeSpecifier spec_static = def_specifier( ESpecifier::Static_Member ); static CodeSpecifier spec_static = def_specifier( ESpecifier::Static );
static CodeUsing using_header = def_using( name(Header), def_type( name(ArrayHeader) ) ); static CodeUsing using_header = def_using( name(Header), def_type( name(ArrayHeader) ) );
static CodeVar ct_grow_formula = def_variable( t_auto, name(grow_formula), untyped_str( code( & array_grow_formula )), spec_ct_member ); static CodeVar ct_grow_formula = def_variable( t_auto, name(grow_formula), untyped_str( code( & array_grow_formula )), spec_ct_member );

View File

@ -15,7 +15,6 @@ using namespace gen;
int gen_main() int gen_main()
{ {
Memory::setup();
gen::init(); gen::init();
gen_sanity_upfront(); gen_sanity_upfront();
@ -35,7 +34,6 @@ int gen_main()
gen_ring_file(); gen_ring_file();
gen::deinit(); gen::deinit();
Memory::cleanup();
return 0; return 0;
} }
#endif #endif

View File

@ -1,6 +1,7 @@
#ifdef gen_time #ifdef gen_time
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_FEATURE_PARSING #define GEN_FEATURE_PARSING
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "Parsed\Array.Parsed.hpp" #include "Parsed\Array.Parsed.hpp"
#include "Parsed\Buffer.Parsed.hpp" #include "Parsed\Buffer.Parsed.hpp"
#include "Parsed\HashTable.Parsed.hpp" #include "Parsed\HashTable.Parsed.hpp"
@ -11,11 +12,10 @@
using namespace gen; using namespace gen;
// TODO : Rewrite this to include both upfront and parsed testing. // TODO : Need to make a more robust test suite
int gen_main() int gen_main()
{ {
Memory::setup();
gen::init(); gen::init();
gen_sanity(); gen_sanity();
@ -60,7 +60,6 @@ int gen_main()
soa_test.write(); soa_test.write();
gen::deinit(); gen::deinit();
Memory::cleanup();
return 0; return 0;
} }
#endif #endif