diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/AST_Design.md b/Project/Source/GasaEditor/GasaGen/gencpp/AST_Design.md index 8b123e9..2f3175a 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/AST_Design.md +++ b/Project/Source/GasaEditor/GasaGen/gencpp/AST_Design.md @@ -1,44 +1,58 @@ +## Navigation + +[Top](../Readme.md) + +<- [docs - General](Readme.md) + ## Current Design `AST` is the actual managed node object for the library. Its raw and really not meant to be used directly. All user interaction must be with its pointer so the type they deal with is `AST*`. -For user-facing code, they should never be giveen a nullptr. Instead, they should be given a designated `Invalid` AST node. +In order to abstract away constant use of `AST*` its wrapped in a Code type which can be either: -In order to abstract away constant use of `AST*`, I wanted to provide a wrapper for it. - -The simpliest being just a type alias. - -```cpp -using Code = AST*; +When its the [C generated variant of the library](../gen_c_library/) +```c +typedef AST* Code; +tyepdef AST_* Code; +... ``` -This is what the genc library would have to use due to its constraints of a langauge. -The actual content per type of AST is covered within [AST_Types.md](AST_Types.md). - -These are pure PODS that just have the lay members relevant to the type of AST node they represent. -Each of them has a Code type alias specific to it. - -Again, the simpliest case for these would be a type alias. +**or** +For C++: ```cpp -using struct AST_Typedef CodeTypedef; +struct Code { + AST* ast; +}; +struct Code { + ... + + AST_* ast; +}; ``` -As of November 21st, 2023, the AST has had a strict layout for how its content is laid out. -This will be abandoned during its redesign that will occur starting with support for statments & expressions for either execution and type declarations. -Having a strict layout is too resctrictive vs allowing each AST type to have maximum control over the layout. +The full definitions of all asts are within: -The redesign will occur after the following todos are addressed: +* [`ast.hpp`](../base/components/ast.hpp) +* [`ast_types.hpp`](../base/components/ast_types.hpp) +* [`code_types.hpp`](../base/components/code_types.hpp) -* [Improvements Lexer & Token struct#27](https://github.com/Ed94/gencpp/issues/27) -* [Generalize AST Flags to a single 4-byte flag#42](https://github.com/Ed94/gencpp/issues/42) -* [AST-Code Object Redesign.#38](https://github.com/Ed94/gencpp/issues/38) -* [Code-AST Documentation#40](https://github.com/Ed94/gencpp/issues/40) -* [AST::debug_str() improvements#33](https://github.com/Ed94/gencpp/issues/33) -* [AST::is_equal implemented and works with singleheader-test#31](https://github.com/Ed94/gencpp/issues/31) -* [Parser : Add ability to have a parse failure and continue with errors recorded.#35](https://github.com/Ed94/gencpp/issues/35) -* [Scanner : Add CodeFile#29](https://github.com/Ed94/gencpp/issues/29) -* [Auxiliary : AST visual debugger#36](https://github.com/Ed94/gencpp/issues/36) +The C/C++ interface procedures are located with `ast.hpp` (for the Code type), and `code_types.hpp` for all others. +## Serialization + +All code types can either serialize using a function of the pattern: + +```c +StrBuilder _to_strbuilder(Code code); +// or +_to_strbuilder(Code code, StrBuilder& result); +``` + +Where the first generates strings allocated using Allocator_StringArena and the other appends an existing strings with their backed allocator. + +Serialization of for the AST is defined for `Code` in [`ast.chpp`](../base/components/ast.cpp) with `code_to_strbuilder_ptr` & `code_to_strbuilder`. +Serializtion for the rest of the code types is within [`code_serialization.cpp`](../base/components/code_serialization.cpp). +Gencpp's serialization does not provide coherent formatting of the code. The user should use a formatter after serializing. diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/AST_Types.md b/Project/Source/GasaEditor/GasaGen/gencpp/AST_Types.md index 3f96ceb..2541162 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/AST_Types.md +++ b/Project/Source/GasaEditor/GasaGen/gencpp/AST_Types.md @@ -1,10 +1,16 @@ +## Navigation + +[Top](../Readme.md) + +<- [docs - General](Readme.md) + # AST Types Documentation While the Readme for docs covers the data layout per AST, this will focus on the AST types avaialble, and their nuances. ## Body -These are containers representing a scope body of a definition that can be of the following `ECode` type: +These are containers representing a scope body of a definition that can be of the following `CodeType` type: * Class_Body * Enum_Body @@ -19,11 +25,11 @@ These are containers representing a scope body of a definition that can be of th Fields: ```cpp +StrCached Name; Code Front; Code Back; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; s32 NumEntries; ``` @@ -31,14 +37,15 @@ s32 NumEntries; The `Front` member represents the start of the link list and `Back` the end. NumEntries is the number of entries in the body. -Parent should have a compatible ECode type for the type of defintion used. +Parent should have a compatible CodeType type for the type of defintion used. Serialization: Will output only the entries, the braces are handled by the parent. ```cpp -... + +... ``` @@ -49,12 +56,12 @@ Represent standard or vendor specific C/C++ attributes. Fields: ```cpp -StringCached Content; +StrCached Content; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -73,12 +80,12 @@ Stores a comment. Fields: ```cpp -StringCached Content; +StrCached Content; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -102,11 +109,11 @@ CodeComment InlineCmt; // Only supported by forward declarations CodeAttributes Attributes; CodeType ParentType; CodeBody Body; -CodeType Prev; // Used to store references to interfaces -CodeType Next; // Used to store references to interfaces -parser::Token* Tok; +StrCached Name; +CodeType Prev; +CodeType Next; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; AccessSpec ParentAccess; @@ -125,7 +132,7 @@ Serialization: }; ``` -You'll notice that only one parent type is supported only with parent access. This library only supports single inheritance, the rest must be done through interfaces. +You'll notice that only one parent type is supported only with parent access. This library only supports single inheritance, the rest are assumed to be interfaces and are given public acess specifiers. ## Constructor @@ -134,13 +141,13 @@ Fields: ```cpp CodeComment InlineCmt; // Only supported by forward declarations Code InitializerList; -CodeParam Params; +CodeParams Params; Code Body; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -171,12 +178,12 @@ Represents a preprocessor define Fields: ```cpp -StringCached Content; +StrCached Content; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -194,11 +201,11 @@ Fields: CodeComment InlineCmt; CodeSpecifiers Specs; Code Body; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -229,24 +236,28 @@ Fields: CodeComment InlineCmt; CodeAttributes Attributes; CodeType UnderlyingType; +Code UnderlyingTypeMacro; CodeBody Body; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; +StrCached Name; CodeT Type; ModuleFlag ModuleFlags; ``` +UnderlyingTypeMacro is a macro the library natively supports: `enum_underlying(type)` that is meant to behave as a wrapper for underlying type assignment. +The `enum_underlying_sig` is a `Str` global var that can be set which will be defined within `PreprocessorDefines` and used in `parser_parse_enum` to identify a valid macro. + Serialization: ```cpp // Enum_Fwd - enum class : ; + enum class : or ; // Enum - : + : or { }; @@ -260,12 +271,12 @@ Will be obsolute when function body parsing is implemented. Fields: ```cpp -StringCached Content; +StrCached Content; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -281,11 +292,11 @@ Fields: ```cpp CodeBody Body; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -303,12 +314,12 @@ extern "" Fields: ```cpp -StringCached Content; +StrCached Content; +StrCached Name; Code Prev; Code Next; Code Parent; -parser::Token* Tok; -StringCached Name; +Token* Tok; CodeT Type; ``` @@ -327,11 +338,11 @@ Fields: ```cpp CodeComment InlineCmt; Code Declaration; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -350,13 +361,13 @@ CodeComment InlineCmt; CodeAttributes Attributes; CodeSpecifiers Specs; CodeType ReturnType; -CodeParam Params; +CodeParams Params; CodeBody Body; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; ``` @@ -379,11 +390,11 @@ Serialization: Fields: ```cpp +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; ``` @@ -400,11 +411,11 @@ Fields: ```cpp CodeBody Body; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; ``` @@ -418,7 +429,7 @@ Serialization: } ``` -## Operator Overload +## Operator Overload (Operator) Fields: @@ -427,13 +438,13 @@ CodeComment InlineCmt; CodeAttributes Attributes; CodeSpecifiers Specs; CodeType ReturnType; -CodeParam Params; +CodeParams Params; CodeBody Body; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; OperatorT Op; @@ -452,7 +463,7 @@ Serialization: } ``` -## Operator Cast Overload ( User-Defined Type Conversion ) +## Operator Cast Overload ( User-Defined Type Conversion, OpCast ) Fields: @@ -461,11 +472,11 @@ CodeComment InlineCmt; CodeSpecifiers Specs; CodeType ValueType; CodeBody Body; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -482,7 +493,7 @@ Serialization: } ``` -## Parameters (AST_Param) +## Parameters (AST_Params) Fields: @@ -490,11 +501,12 @@ Fields: CodeType ValueType; Code Macro; Code Value; -CodeParam Last; -CodeParam Next; -parser::Token* Tok; +Code PostNameMacro; +StrCached Name; +CodeParams Last; +CodeParams Next; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; s32 NumEntries; ``` @@ -504,7 +516,7 @@ Serialization: ```cpp , ... - , ... + = , ... ``` ## Pragma @@ -512,12 +524,12 @@ Serialization: Fields: ```cpp -StringCached Content; +StrCached Content; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -532,12 +544,12 @@ Serialization: Fields: ```cpp -StringCached Content; +StrCached Content; +StrCached Name; Code Prev; Code Next; -paser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ``` @@ -552,13 +564,13 @@ Serialization: Fields: ```cpp -SpecifierT ArrSpecs[ AST::ArrSpecs_Cap ]; +SpecifierT ArrSpecs[ AST_ArrSpecs_Cap ]; CodeSpecifiers NextSpecs; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; s32 NumEntries; ``` @@ -574,13 +586,13 @@ Serialization: Fields: ```cpp -CodeParam Params; +CodeParams Params; Code Declaration; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; ``` @@ -603,23 +615,30 @@ Fields: CodeAttributes Attributes; CodeSpecifiers Specs; CodeReturnType ReturnType; -CodeParam Params; +CodeParams Params; Code ArrExpr; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; +StrCached Name; CodeT Type; b32 IsParamPack; +ETypenameTag TypeTag; ``` Serialization: ```cpp - + +// Function + ``` +`` currently has the full serialization of anything with + +*Note: ArrExpr is not used in serialization by `typename_to_strbuilder_ref` its instead handled by a parent AST's serailization (variable, typedef, using).* + ## Typedef Behave as usual except function or macro typedefs. @@ -630,11 +649,11 @@ Fields: ```cpp CodeComment InlineCmt; Code UnderlyingType; +StrCached Name; Code Prev; Code Next; -parse::Token* Tok +Token* Tok Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; b32 IsFunction; @@ -644,11 +663,16 @@ Serialization: ```cpp // Regular - typedef ; + typedef ; // Functions - typedef ( ); - typedef ( )( ); + +// Currently: + typedef ; + +// Desired: Not handled yet + typedef ReturnType> UnderlyingType->Name> ( Parameters> ); + typedef ReturnType> ( Namespace> forhas(Spec_Ptr) ?: *> Name> ) ( Parameters> ); ``` ## Union @@ -658,11 +682,11 @@ Fields: ```cpp CodeAttributes Attributes; CodeBody Body; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; ``` @@ -684,11 +708,11 @@ Fields: CodeComment InlineCmt; CodeAttributes Attributes; CodeType UnderlyingType; +StrCached Name; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; ``` @@ -716,14 +740,15 @@ CodeSpecifiers Specs; CodeType ValueType; Code BitfieldSize; Code Value; +StrCached Name; CodeVar NextVar; Code Prev; Code Next; -parser::Token* Tok; +Token* Tok; Code Parent; -StringCached Name; CodeT Type; ModuleFlag ModuleFlags; +s32 VarParenthesizedInit; ``` Serialization: @@ -734,4 +759,7 @@ Serialization: // Bitfield : = , NextVar ...; + +// VarParenthesizedInit + ( , NextVar ... ); ``` diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/Parser_Algo.md b/Project/Source/GasaEditor/GasaGen/gencpp/Parser_Algo.md index d085f3b..8180168 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/Parser_Algo.md +++ b/Project/Source/GasaEditor/GasaGen/gencpp/Parser_Algo.md @@ -1,3 +1,9 @@ +## Navigation + +[Top](../Readme.md) + +<- [docs - General](Readme.md) + # Parser's Algorithim gencpp uses a hand-written recursive descent parser. Both the lexer and parser currently handle a full C/C++ file in a single pass. @@ -6,8 +12,8 @@ gencpp uses a hand-written recursive descent parser. Both the lexer and parser c ### Lexer -The lex procedure does the lexical pass of content provided as a `StrC` type. -The tokens are stored (for now) in `gen::parser::Tokens`. +The lex procedure does the lexical pass of content provided as a `Str` type. +The tokens are stored (for now) in `Lexer_Tokens`. Fields: @@ -16,7 +22,7 @@ Array Arr; s32 Idx; ``` -What token types are supported can be found in [ETokType.csv](../project/enums/ETokType.csv) you can also find the token types in [ETokType.h](../project/components/gen/etoktype.cpp) , which is the generated enum from the csv file. +What token types are supported can be found in [ETokType.csv](../base/enums/ETokType.csv) you can also find the token types in [ETokType.h](../base/components/gen/etoktype.cpp) , which is the generated enum from the csv file. Tokens are defined with the struct `gen::parser::Token`: @@ -69,7 +75,7 @@ The parser has a limited user interface, only specific types of definitions or s Each public user interface procedure has the following format: ```cpp - parse_( StrC def ) + parse_( Str def ) { check_parse_args( def ); using namespace Parser; @@ -281,7 +287,7 @@ In the future statements and expressions will be parsed. 1. Attributes ( Standard, GNU, MSVC, Macro ) : `parse_attributes` 2. Specifiers ( consteval, constexpr, constinit, extern, forceinline, global, inline, internal_linkage, neverinline, static ) 3. Is either ( identifier, const specifier, long, short, signed, unsigned, bool, char, double, int) - 1. Attempt to parse as constrcutor or destructor : `parse_global_nspace_constructor_destructor` + 1. Attempt to parse as construtor or destructor : `parse_global_nspace_constructor_destructor` 2. If its an operator cast (definition outside class) : `parse_operator_cast` 3. Its an operator, function, or varaible : `parse_operator_function_or_varaible` 4. If its not a global body, consume the closing curly brace diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/Parsing.md b/Project/Source/GasaEditor/GasaGen/gencpp/Parsing.md index 3c2348a..c29fc17 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/Parsing.md +++ b/Project/Source/GasaEditor/GasaGen/gencpp/Parsing.md @@ -1,33 +1,39 @@ +## Navigation + +[Top](../Readme.md) + +<- [docs - General](Readme.md) + # Parsing -The library features a naive parser tailored for only what the library needs to construct the supported syntax of C++ into its AST. +The library features a naive single-pass parser tailored for only what the library needs to construct the supported syntax of C++ into its AST for *"front-end"* meta-programming purposes. -This parser does not, and should not do the compiler's job. By only supporting this minimal set of features, the parser is kept (so far) around ~5600 loc. I hope to keep it under 10k loc worst case. +This parser does not, and should not do the compiler's job. By only supporting this minimal set of features, the parser is kept (so far) around ~7000 loc. I hope to keep it under 10k loc worst case. -You can think of this parser of a frontend parser vs a semantic parser. Its intuitively similar to WYSIWYG. What you precerive as the syntax from the user-side before the compiler gets a hold of it, is what you get. +You can think of this parser as *frontend parser* vs a *semantic parser*. Its intuitively similar to WYSIWYG. What you ***precerive*** as the syntax from the user-side before the compiler gets a hold of it, is what you get. User exposed interface: ```cpp -CodeClass parse_class ( StrC class_def ); -CodeConstructor parse_constructor ( StrC constructor_def ); -CodeDestructor parse_destructor ( StrC destructor_def ); -CodeEnum parse_enum ( StrC enum_def ); -CodeBody parse_export_body ( StrC export_def ); -CodeExtern parse_extern_link ( StrC exten_link_def ); -CodeFriend parse_friend ( StrC friend_def ); -CodeFn parse_function ( StrC fn_def ); -CodeBody parse_global_body ( StrC body_def ); -CodeNS parse_namespace ( StrC namespace_def ); -CodeOperator parse_operator ( StrC operator_def ); -CodeOpCast parse_operator_cast( StrC operator_def ); -CodeStruct parse_struct ( StrC struct_def ); -CodeTemplate parse_template ( StrC template_def ); -CodeType parse_type ( StrC type_def ); -CodeTypedef parse_typedef ( StrC typedef_def ); -CodeUnion parse_union ( StrC union_def ); -CodeUsing parse_using ( StrC using_def ); -CodeVar parse_variable ( StrC var_def ); +CodeClass parse_class ( Str class_def ); +CodeConstructor parse_constructor ( Str constructor_def ); +CodeDestructor parse_destructor ( Str destructor_def ); +CodeEnum parse_enum ( Str enum_def ); +CodeBody parse_export_body ( Str export_def ); +CodeExtern parse_extern_link ( Str exten_link_def ); +CodeFriend parse_friend ( Str friend_def ); +CodeFn parse_function ( Str fn_def ); +CodeBody parse_global_body ( Str body_def ); +CodeNS parse_namespace ( Str namespace_def ); +CodeOperator parse_operator ( Str operator_def ); +CodeOpCast parse_operator_cast( Str operator_def ); +CodeStruct parse_struct ( Str struct_def ); +CodeTemplate parse_template ( Str template_def ); +CodeType parse_type ( Str type_def ); +CodeTypedef parse_typedef ( Str typedef_def ); +CodeUnion parse_union ( Str union_def ); +CodeUsing parse_using ( Str using_def ); +CodeVar parse_variable ( Str var_def ); ``` To parse file buffers, use the `parse_global_body` function. @@ -47,10 +53,11 @@ The keywords supported for the preprocessor are: * endif * pragma -Each directive `#` line is considered one preproecessor unit, and will be treated as one Preprocessor AST. *These ASTs will be considered members or entries of braced scope they reside within*. +Each directive `#` line is considered one preproecessor unit, and will be treated as one Preprocessor AST. If a directive is used with an unsupported keyword its will be processed as an untyped AST. -The preprocessor lines are stored as members of their associated scope they are parsed within. ( Global, Namespace, Class/Struct ) +The preprocessor lines are stored as members of their associated scope they are parsed within. ( Global, Namespace, Class/Struct ) +***Again (Its not standard): These ASTs will be considered members or entries of braced scope they reside within*** Any preprocessor definition abuse that changes the syntax of the core language is unsupported and will fail to parse if not kept within an execution scope (function body, or expression assignment). Exceptions: @@ -59,6 +66,8 @@ Exceptions: * Disable with: `#define GEN_PARSER_DISABLE_MACRO_FUNCTION_SIGNATURES` * typedefs allow for a preprocessed macro: `typedef MACRO();` * Disable with: `#define GEN_PARSER_DISABLE_MACRO_TYPEDEF` +* Macros can behave as typenames +* There is some macro support in paramters for functions or templates *(Specifically added to support parsing Unreal Engine source)*. *(Exceptions are added on an on-demand basis)* *(See functions `parse_operator_function_or_variable` and `parse_typedef` )* @@ -67,15 +76,23 @@ Adding your own exceptions is possible by simply modifying the parser to allow f *Note: You could interpret this strictness as a feature. This would allow the user to see if their codebase or a third-party's codebase some some egregious preprocessor abuse.* +If a macro is not defined withint e scope of parsing a set of files, it can be defined beforehand by: + +* Appending the [`PreprocessorDefines`](https://github.com/Ed94/gencpp/blob/a18b5b97aa5cfd20242065cbf53462a623cd18fa/base/components/header_end.hpp#L137) array. + * For functional macros a "(" just needs to be added after the name like: `(` so that it will tokenize its arguments as part of the token during lexing. +* Defining a CodeDefine using `def_define`. The definition will be processed by the interface for user into `PreprocessorDefines`. + * This can be prevented by setting the optional prameter `dont_append_preprocess_defines`. + The lexing and parsing takes shortcuts from whats expected in the standard. * Numeric literals are not checked for validity. -* The parse API treats any execution scope definitions with no validation and are turned into untyped Code ASTs. +* The parse API treats any execution scope definitions with no validation and are turned into untyped Code ASTs. (There is a [todo](https://github.com/Ed94/gencpp/issues/49) to add support) * *This includes the assignment of variables.* * Attributes ( `[[]]` (standard), `__declspec` (Microsoft), or `__attribute__` (GNU) ) * Assumed to *come before specifiers* (`const`, `constexpr`, `extern`, `static`, etc) for a function or right afterthe return type. * Or in the usual spot for class, structs, (*right after the declaration keyword*) * typedefs have attributes with the type (`parse_type`) * Parsing attributes can be extended to support user defined macros by defining `GEN_DEFINE_ATTRIBUTE_TOKENS` (see `gen.hpp` for the formatting) + * This is useful for example: parsing Unreal `Module_API` macros. Empty lines used throughout the file are preserved for formatting purposes during ast serialization. diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/Readme.md b/Project/Source/GasaEditor/GasaGen/gencpp/Readme.md index 20d84fd..7be3fd6 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/Readme.md +++ b/Project/Source/GasaEditor/GasaGen/gencpp/Readme.md @@ -1,137 +1,17 @@ -# gencpp +## Navigation -An attempt at simple staged metaprogramming for c/c++. +# base -The library API is a composition of code element constructors. -These build up a code AST to then serialize with a file builder. +[Top](../Readme.md) -This code base attempts follow the [handmade philosophy](https://handmade.network/manifesto). -Its not meant to be a black box metaprogramming utility, it should be easy to intergrate into a user's project domain. +* [docs](../docs/Readme.md) -## Notes +# Unreal Engine Version Generator -**On Partial Hiatus: Life has got me tackling other issues..** -I will be passively updating the library with bug fixes and minor improvements as I use it for my personal projects. -There won't be any major reworks or features to this thing for a while. +This generates a variant of gencpp thats compatiable with use as a thirdparty module within a plugin or module of an Unreal Project or the Engine itself. -This project is still in development (very much an alpha state), so expect bugs and missing features. -See [issues](https://github.com/Ed94/gencpp/issues) for a list of known bugs or todos. - -The library can already be used to generate code just fine, but the parser is where the most work is needed. If your C++ isn't "down to earth" expect issues. - -A `natvis` and `natstepfilter` are provided in the scripts directory (its outdated, I'll update this readme when its not). - -***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.*** - -A C variant is hosted [here](https://github.com/Ed94/genc); I will complete it when this library is feature complete, it should be easier to make than this... - -## Usage - -A metaprogram is built to generate files before the main program is built. We'll term runtime for this program as `GEN_TIME`. The metaprogram's core implementation are within `gen.hpp` and `gen.cpp` in the project directory. - -`gen.cpp` \`s `main()` is defined as `gen_main()` which the user will have to define once for their program. There they will dictate everything that should be generated. - -In order to keep the locality of this code within the same files the following pattern may be used (although this pattern isn't required at all): - -Within `program.cpp` : - -```cpp -#ifdef GEN_TIME -#include "gen.hpp" - -... - -u32 gen_main() -{ - ... -} -#endif - -// "Stage" agnostic code. - -#ifndef GEN_TIME -#include "program.gen.cpp" - - // Regular runtime dependent on the generated code here. -#endif +If using the library's provided build scripts: +```ps1 +.\build.ps1 unreal ``` - -The design uses a constructive builder API for the code to generate. -The user is provided `Code` objects that are used to build up the AST. - -Example using each construction interface: - -### Upfront - -Validation and construction through a functional interface. - -```cpp -Code t_uw = def_type( name(usize) ); -Code t_allocator = def_type( name(allocator) ); -Code t_string_const = def_type( name(char), def_specifiers( args( ESpecifier::Const, ESpecifier::Ptr ) )); - -Code header; -{ - Code num = def_variable( t_uw, name(Num) ); - Code cap = def_variable( t_uw, name(Capacity) ); - Code mem_alloc = def_variable( t_allocator, name(Allocator) ); - Code body = def_struct_body( args( num, cap, mem_alloc ) ); - - header = def_struct( name(ArrayHeader), __, __, body ); -} -``` - -### Parse - -Validation through ast construction. - -```cpp -Code header = parse_struct( code( - struct ArrayHeader - { - usize Num; - usize Capacity; - allocator Allocator; - }; -)); - -``` - -### Untyped - -No validation, just glorified text injection. - -```cpp -Code header = code_str( - struct ArrayHeader - { - usize Num; - usize Capacity; - allocator Allocator; - }; -); -``` - -`name` is a helper macro for providing a string literal with its size, intended for the name parameter of functions. -`code` is a helper macro for providing a string literal with its size, but intended for code string parameters. -`args` is a helper macro for providing the number of arguments to varadic constructors. -`code_str` is a helper macro for writting `untyped_str( code( ))` - -All three constrcuton interfaces will generate the following C code: - -```cpp -struct ArrayHeader -{ - usize Num; - usize Capacity; - allocator Allocator; -}; -``` - -**Note: The formatting shown here is not how it will look. For your desired formatting its recommended to run a pass through the files with an auto-formatter.** -*(The library currently uses clang-format for formatting, beware its pretty slow...)* - -## Building - -See the [scripts directory](scripts/). diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/Readme_Docs.md b/Project/Source/GasaEditor/GasaGen/gencpp/Readme_Docs.md index 7f2da37..4655f0a 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/Readme_Docs.md +++ b/Project/Source/GasaEditor/GasaGen/gencpp/Readme_Docs.md @@ -1,62 +1,42 @@ -## Documentation +# General Docs -The project has no external dependencies beyond: +[Top](../Readme.md) -* `errno.h` -* `stat.h` -* `stdarg.h` -* `stddef.h` -* `stdio.h` -* `copyfile.h` (Mac) -* `types.h` (Linux) -* `unistd.h` (Linux/Mac) -* `intrin.h` (Windows) -* `io.h` (Windows with gcc) -* `windows.h` (Windows) +Contains: -Dependencies for the project are wrapped within `GENCPP_ROLL_OWN_DEPENDENCIES` (Defining it will disable them). -The majority of the dependency's implementation was derived from the [c-zpl library](https://github.com/zpl-c/zpl). +* [AST_Design](./AST_Design.md): Overview of ASTs +* [AST Types](./AST_Types.md): Listing of all AST types along with their Code type interface. +* [Parsing](./Parsing.md): Overview of the parsing interface. +* [Parser Algo](./Parser_Algo.md): In-depth breakdown of the parser's implementation. -This library was written in a subset of C++ where the following are not used at all: - -* RAII (Constructors/Destructors), lifetimes are managed using named static or regular functions. -* Language provide dynamic dispatch, RTTI -* Object-Oriented Inheritance -* Exceptions - -Polymorphic & Member-functions are used as an ergonomic choice, along with a conserative use of operator overloads. -There are only 4 template definitions in the entire library. (`Array`, `Hashtable`, `swap`, and `AST/Code::cast`) - -Two generic templated containers are used throughout the library: - -* `template< class Type> struct Array` -* `template< class Type> struct HashTable` - -Both Code and AST definitions have a `template< class Type> Code/AST :: cast()`. Its just an alternative way to explicitly cast to each other. - -`template< class Type> swap( Type& a, Type& b)` is used over a macro. - -Otherwise the library is free of any templates. - -### *WHAT IS NOT PROVIDED* +### *CURRENTLY UNSUPPORTED* **There is no support for validating expressions.** -Its difficult to parse without enough benefits (At the metaprogramming level). -I plan to add this only at the tail of the project parsing milestone. +Its a [todo](https://github.com/Ed94/gencpp/issues/49) -**Only trivial template support is provided.** -The intention is for only simple, non-recursive substitution. -The parameters of the template are treated like regular parameter AST entries. +**Only trivial template support is provided.** +The intention is for only simple, non-recursive substitution. +The parameters of the template are treated like regular parameter AST entries. This means that the typename entry for the parameter AST would be either: * `class` * `typename` * A fundamental type, function, or pointer type. -Anything beyond this usage is not supported by parse_template for arguments (at least not intentionally). -Use at your own mental peril. +***Concepts and Constraints are not supported*** +Its a [todo](https://github.com/Ed94/gencpp/issues/21) -*Concepts and Constraints are not supported, its usage is non-trivial substitution.* +### Feature Macros: + +* `GEN_DEFINE_ATTRIBUTE_TOKENS` : Allows user to define their own attribute macros for use in parsing. + * This can be generated using base.cpp. +* `GEN_DEFINE_LIBRARY_CORE_CONSTANTS` : Optional typename codes as they are non-standard to C/C++ and not necessary to library usage +* `GEN_DONT_ENFORCE_GEN_TIME_GUARD` : By default, the library ( gen.hpp/ gen.cpp ) expects the macro `GEN_TIME` to be defined, this disables that. +* `GEN_ENFORCE_STRONG_CODE_TYPES` : Enforces casts to filtered code types. +* `GEN_EXPOSE_BACKEND` : Will expose symbols meant for internal use only. +* `GEN_ROLL_OWN_DEPENDENCIES` : Optional override so that user may define the dependencies themselves. +* `GEN_DONT_ALLOW_INVALID_CODE` (Not implemented yet) : Will fail when an invalid code is constructed, parsed, or serialized. +* `GEN_C_LIKE_PP` : Setting to `` Will prevent usage of function defnitions using references and structs with member functions. Structs will still have user-defined operator conversions, for-range support, and other operator overloads ### The Data & Interface @@ -65,87 +45,33 @@ As mentioned in root readme, the user is provided Code objects by calling the co The AST is managed by the library and provided to the user via its interface. However, the user may specifiy memory configuration. -Data layout of AST struct (Subject to heavily change with upcoming redesign): +[Data layout of AST struct (Subject to heavily change with upcoming todos)](../base/components/ast.hpp#L396-461) -```cpp -union { - struct - { - AST* InlineCmt; // Class, Constructor, Destructor, Enum, Friend, Functon, Operator, OpCast, Struct, Typedef, Using, Variable - AST* Attributes; // Class, Enum, Function, Struct, Typedef, Union, Using, Variable - AST* Specs; // Destructor, Function, Operator, Typename, Variable - union { - AST* InitializerList; // Constructor - AST* ParentType; // Class, Struct, ParentType->Next has a possible list of interfaces. - AST* ReturnType; // Function, Operator, Typename - AST* UnderlyingType; // Enum, Typedef - AST* ValueType; // Parameter, Variable - }; - union { - AST* Macro; // Parameters - AST* BitfieldSize; // Variable (Class/Struct Data Member) - AST* Params; // Constructor, Function, Operator, Template, Typename - }; - union { - AST* ArrExpr; // Typename - AST* Body; // Class, Constructr, Destructor, Enum, Function, Namespace, Struct, Union - AST* Declaration; // Friend, Template - AST* Value; // Parameter, Variable - }; - union { - AST* NextVar; // Variable; Possible way to handle comma separated variables declarations. ( , NextVar->Specs NextVar->Name NextVar->ArrExpr = NextVar->Value ) - AST* SpecsFuncSuffix; // Only used with typenames, to store the function suffix if typename is function signature. - }; - }; - StringCached Content; // Attributes, Comment, Execution, Include - struct { - SpecifierT ArrSpecs[AST::ArrSpecs_Cap]; // Specifiers - AST* NextSpecs; // Specifiers - }; -}; -union { - AST* Prev; - AST* Front; - AST* Last; -}; -union { - AST* Next; - AST* Back; -}; -AST* Parent; -StringCached Name; -CodeT Type; -ModuleFlag ModuleFlags; -union { - b32 IsFunction; // Used by typedef to not serialize the name field. - b32 IsParamPack; // Used by typename to know if type should be considered a parameter pack. - OperatorT Op; - AccessSpec ParentAccess; - s32 NumEntries; -}; -s32 Token; // Handle to the token, stored in the CodeFile (Otherwise unretrivable) -``` +https://github.com/Ed94/gencpp/blob/eea4ebf5c40d5d87baa465abfb1be30845b2377e/base/components/ast.hpp#L396-L461 -*`CodeT` is a typedef for `ECode::Type` which has an underlying type of `u32`* +*`StringCahced` is a typedef for `Str` (a string slice), to denote it is an interned string* +*`CodeType` is enum taggin the type of code. Has an underlying type of `u32`* *`OperatorT` is a typedef for `EOperator::Type` which has an underlying type of `u32`* -*`StringCahced` is a typedef for `String const`, to denote it is an interned string* -*`String` is the dynamically allocated string type for the library* +*`StrBuilder` is the dynamically allocated string type for the library* -AST widths are setup to be AST_POD_Size. +AST widths are setup to be AST_POD_Size. The width dictates how much the static array can hold before it must give way to using an allocated array: ```cpp constexpr static -usize ArrSpecs_Cap = +int AST_ArrSpecs_Cap = ( AST_POD_Size - - sizeof(AST*) * 3 - - sizeof(StringCached) - - sizeof(CodeT) + - sizeof(Code) + - sizeof(StrCached) + - sizeof(Code) * 2 + - sizeof(Token*) + - sizeof(Code) + - sizeof(CodeType) - sizeof(ModuleFlag) - sizeof(u32) ) -/ sizeof(SpecifierT) -1; // -1 for 4 extra bytes (Odd num of AST*) +/ sizeof(Specifier) - 1; ``` *Ex: If the AST_POD_Size is 128 the capacity of the static array is 20.* @@ -153,8 +79,8 @@ usize ArrSpecs_Cap = Data Notes: * The allocator definitions used are exposed to the user incase they want to dictate memory usage - * You'll find the memory handling in `init`, `deinit`, `reset`, `gen_string_allocator`, `get_cached_string`, `make_code`. - * Allocators are defined with the `AllocatorInfo` structure found in `dependencies\memory.hpp` + * You'll find the memory handling in `init`, `deinit`, `reset`, `gen_strbuilder_allocator`, `cache_str`, `make_code`. + * Allocators are defined with the `AllocatorInfo` structure found in [`memory.hpp`](../base/dependencies/memory.hpp) * Most of the work is just defining the allocation procedure: ```cpp @@ -162,30 +88,30 @@ Data Notes: ``` * ASTs are wrapped for the user in a Code struct which is a wrapper for a AST* type. -* Both AST and Code have member symbols but their data layout is enforced to be POD types. +* Code types have member symbols but their data layout is enforced to be POD types. * This library treats memory failures as fatal. * Cached Strings are stored in their own set of arenas. AST constructors use cached strings for names, and content. * `StringArenas`, `StringCache`, `Allocator_StringArena`, and `Allocator_StringTable` are the associated containers or allocators. * Strings used for serialization and file buffers are not contained by those used for cached strings. - * They are currently using `GlobalAllocator`, which are tracked array of arenas that grows as needed (adds buckets when one runs out). + * They are currently using `FallbackAllocator`, which are tracked array of arenas that grows as needed (adds buckets when one runs out). * Memory within the buckets is not reused, so its inherently wasteful. - * I will be augmenting the single arena with a simple slag allocator. -* Linked lists used children nodes on bodies, and parameters. + * I will be augmenting the default allocator with virtual memory & a slab allocator in the [future](https://github.com/Ed94/gencpp/issues/12) +* Intrusive linked lists used children nodes on bodies, and parameters. * Its intended to generate the AST in one go and serialize after. The constructors and serializer are designed to be a "one pass, front to back" setup. -* Allocations can be tuned by defining the folloiwng macros: +* Allocations can be tuned by defining the folloiwng macros (will be moved to runtime configuration in the future): * `GEN_GLOBAL_BUCKET_SIZE` : Size of each bucket area for the global allocator * `GEN_CODEPOOL_NUM_BLOCKS` : Number of blocks per code pool in the code allocator * `GEN_SIZE_PER_STRING_ARENA` : Size per arena used with string caching. * `GEN_MAX_COMMENT_LINE_LENGTH` : Longest length a comment can have per line. * `GEN_MAX_NAME_LENGTH` : Max length of any identifier. * `GEN_MAX_UNTYPED_STR_LENGTH` : Max content length for any untyped code. - * `GEN_TOKEN_FMT_TOKEN_MAP_MEM_SIZE` : token_fmt_va uses local_persit memory of this size for the hashtable. + * `TokenMap_FixedArena` : token_fmt_va uses local_persit memory of this arena type for the hashtable. * `GEN_LEX_ALLOCATOR_SIZE` * `GEN_BUILDER_STR_BUFFER_RESERVE` The following CodeTypes are used which the user may optionally use strong typing with if they enable: `GEN_ENFORCE_STRONG_CODE_TYPES` -* CodeBody : Has support for `for-range` iterating across Code objects. +* CodeBody : Has support for `for : range` iterating across Code objects. * CodeAttributes * CodeComment * CodeClass @@ -202,13 +128,13 @@ The following CodeTypes are used which the user may optionally use strong typing * CodeNS * CodeOperator * CodeOpCast -* CodeParam : Has support for `for-range` iterating across parameters. +* CodeParams : Has support for `for : range` iterating across parameters. * CodePreprocessCond * CodePragma -* CodeSpecifiers : Has support for `for-range` iterating across specifiers. +* CodeSpecifiers : Has support for `for : range` iterating across specifiers. * CodeStruct * CodeTemplate -* CodeType +* CodeTypename * CodeTypedef * CodeUnion * CodeUsing @@ -293,6 +219,7 @@ Code ``` When using the body functions, its recommended to use the args macro to auto determine the number of arguments for the varadic: + ```cpp def_global_body( args( ht_entry, array_ht_entry, hashtable )); @@ -300,7 +227,7 @@ def_global_body( args( ht_entry, array_ht_entry, hashtable )); def_global_body( 3, ht_entry, array_ht_entry, hashtable ); ``` -If a more incremental approach is desired for the body ASTs, `Code def_body( CodeT type )` can be used to create an empty body. +If a more incremental approach is desired for the body ASTs, `Code def_body( CodeT type )` can be used to create an empty body. When the members have been populated use: `AST::validate_body` to verify that the members are valid entires for that type. ### Parse construction @@ -352,7 +279,7 @@ Interface : * untyped_fmt * untyped_token_fmt -During serialization any untyped Code AST has its string value directly injected inline of whatever context the content existed as an entry within. +During serialization any untyped Code AST has its string value directly injected inline of whatever context the content existed as an entry within. Even though these are not validated from somewhat correct c/c++ syntax or components, it doesn't mean that Untyped code can be added as any component of a Code AST: * Untyped code cannot have children, thus there cannot be recursive injection this way. @@ -373,6 +300,7 @@ Code = untyped_str( code( ``` Optionally, `code_str`, and `code_fmt` macros can be used so that the code macro doesn't have to be used: + ```cpp Code = code_str( ) ``` @@ -380,7 +308,7 @@ Code = code_str( ) Template metaprogramming in the traditional sense becomes possible with the use of `token_fmt` and parse constructors: ```cpp -StrC value = txt("Something"); +Str value = txt("Something"); char const* template_str = txt( Code with to replace with token_values @@ -402,8 +330,8 @@ The following are provided predefined by the library as they are commonly used: * `module_global_fragment` * `module_private_fragment` * `fmt_newline` -* `param_varaidc` (Used for varadic definitions) * `pragma_once` +* `param_varaidc` (Used for varadic definitions) * `preprocess_else` * `preprocess_endif` * `spec_const` @@ -419,6 +347,7 @@ The following are provided predefined by the library as they are commonly used: * `spec_local_persist` (local_persist macro) * `spec_mutable` * `spec_neverinline` +* `spec_noexcept` * `spec_override` * `spec_ptr` * `spec_pure` @@ -450,8 +379,8 @@ Optionally the following may be defined if `GEN_DEFINE_LIBRARY_CODE_CONSTANTS` i * `t_u16` * `t_u32` * `t_u64` -* `t_sw` (ssize_t) -* `t_uw` (size_t) +* `t_ssize` (ssize_t) +* `t_usize` (size_t) * `t_f32` * `t_f64` @@ -469,15 +398,12 @@ and have the desired specifiers assigned to them beforehand. ## Code generation and modification -There are three provided auxillary interfaces: +There are two provided auxillary interfaces: * Builder -* Editor * Scanner -Editor and Scanner are disabled by default, use `GEN_FEATURE_EDITOR` and `GEN_FEATURE_SCANNER` to enable them. - -### Builder is a similar object to the jai language's string_builder +### Builder is a similar object to the jai language's strbuilder_builder * The purpose of it is to generate a file. * A file is specified and opened for writing using the open( file_path) function. @@ -486,4 +412,8 @@ Editor and Scanner are disabled by default, use `GEN_FEATURE_EDITOR` and `GEN_FE ### Scanner Auxillary Interface -Provides *(eventually)* `scan_file` to automatically populate a CodeFile which contains a parsed AST (`Code`) of the file, with any contextual failures that are reported from the parser. +* The purpose is to scan or parse files +* Some with two basic functions to convert a fil to code: `scan_file` and `parse_file` + * `scan_file`: Merely grabs the file and stores it in an untyped Code. + * `parse_file`: Will parse the file using `parse_global_body` and return a `CodeBody`. +* Two basic functions for grabbing columns from a CSV: `parse_csv_one_column` and `parse_csv_two_columns` diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/gen.builder.cpp b/Project/Source/GasaEditor/GasaGen/gencpp/gen.builder.cpp index 8e1d35e..1cdcac5 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/gen.builder.cpp +++ b/Project/Source/GasaEditor/GasaGen/gencpp/gen.builder.cpp @@ -9,6 +9,9 @@ # pragma clang diagnostic ignored "-Wunknown-pragmas" # pragma clang diagnostic ignored "-Wvarargs" # pragma clang diagnostic ignored "-Wunused-function" +# pragma clang diagnostic ignored "-Wbraced-scalar-init" +# pragma clang diagnostic ignored "-W#pragma-messages" +# pragma clang diagnostic ignored "-Wstatic-in-inline" #endif #ifdef __GNUC__ @@ -23,7 +26,9 @@ GEN_NS_BEGIN -Builder Builder::open( char const* path ) +#pragma region Builder + +Builder builder_open( char const* path ) { Builder result; @@ -34,51 +39,49 @@ Builder Builder::open( char const* path ) return result; } - result.Buffer = String::make_reserve( GlobalAllocator, Builder_StrBufferReserve ); + result.Buffer = strbuilder_make_reserve( _ctx->Allocator_Temp, _ctx->InitSize_BuilderBuffer ); // log_fmt("$Builder - Opened file: %s\n", result.File.filename ); return result; } -void Builder::pad_lines( s32 num ) +void builder_pad_lines( Builder* builder, s32 num ) { - Buffer.append( "\n" ); + strbuilder_append_str( & builder->Buffer, txt("\n") ); } -void Builder::print( Code code ) +void builder_print( Builder* builder, Code code ) { - String str = code->to_string(); + StrBuilder str = code_to_strbuilder(code); // const ssize len = str.length(); // log_fmt( "%s - print: %.*s\n", File.filename, len > 80 ? 80 : len, str.Data ); - Buffer.append( str ); + strbuilder_append_string( & builder->Buffer, str ); } -void Builder::print_fmt( char const* fmt, ... ) +void builder_print_fmt_va( Builder* builder, char const* fmt, va_list va ) { ssize res; char buf[ GEN_PRINTF_MAXLEN ] = { 0 }; - va_list va; - va_start( va, fmt ); - res = str_fmt_va( buf, count_of( buf ) - 1, fmt, va ) - 1; - va_end( va ); + res = c_str_fmt_va( buf, count_of( buf ) - 1, fmt, va ) - 1; - // log_fmt( "$%s - print_fmt: %.*s\n", File.filename, res > 80 ? 80 : res, buf ); - Buffer.append( buf, res ); + strbuilder_append_c_str_len( (StrBuilder*) & (builder->Buffer), (char const*)buf, res); } -void Builder::write() +void builder_write(Builder* builder) { - b32 result = file_write( & File, Buffer, Buffer.length() ); + b32 result = file_write( & builder->File, builder->Buffer, strbuilder_length(builder->Buffer) ); if ( result == false ) - log_failure("gen::File::write - Failed to write to file: %s\n", file_name( & File ) ); + log_failure("gen::File::write - Failed to write to file: %s\n", file_name( & builder->File ) ); - log_fmt( "Generated: %s\n", File.filename ); - file_close( & File ); - Buffer.free(); + log_fmt( "Generated: %s\n", builder->File.filename ); + file_close( & builder->File ); + strbuilder_free(& builder->Buffer); } +#pragma endregion Builder + GEN_NS_END #ifdef __clang__ diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/gen.builder.hpp b/Project/Source/GasaEditor/GasaGen/gencpp/gen.builder.hpp index 66f5f01..60d8a08 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/gen.builder.hpp +++ b/Project/Source/GasaEditor/GasaGen/gencpp/gen.builder.hpp @@ -9,6 +9,9 @@ # pragma clang diagnostic ignored "-Wunknown-pragmas" # pragma clang diagnostic ignored "-Wvarargs" # pragma clang diagnostic ignored "-Wunused-function" +# pragma clang diagnostic ignored "-Wbraced-scalar-init" +# pragma clang diagnostic ignored "-W#pragma-messages" +# pragma clang diagnostic ignored "-Wstatic-in-inline" #endif #ifdef __GNUC__ @@ -25,21 +28,59 @@ GEN_NS_BEGIN +#pragma region Builder + +struct Builder; +typedef struct Builder Builder; + +Builder builder_open ( char const* path ); +void builder_pad_lines ( Builder* builder, s32 num ); +void builder_print ( Builder* builder, Code code ); +void builder_print_fmt_va( Builder* builder, char const* fmt, va_list va ); +void builder_print_fmt ( Builder* builder, char const* fmt, ... ) { + va_list va; + va_start( va, fmt ); + builder_print_fmt_va( builder, fmt, va ); + va_end( va ); +} +void builder_write( Builder* builder ); + struct Builder { FileInfo File; - String Buffer; + StrBuilder Buffer; - static Builder open( char const* path ); +#if GEN_COMPILER_CPP && ! GEN_C_LIKE_CPP + FORCEINLINE static Builder open( char const* path ) { return builder_open(path); } - void pad_lines( s32 num ); + FORCEINLINE void pad_lines( s32 num ) { return builder_pad_lines(this, num); } - void print( Code ); - void print_fmt( char const* fmt, ... ); + FORCEINLINE void print( Code code ) { return builder_print(this, code); } + FORCEINLINE void print_fmt( char const* fmt, ... ) { + va_list va; + va_start( va, fmt ); + builder_print_fmt_va( this, fmt, va ); + va_end( va ); + } - void write(); + FORCEINLINE void write() { return builder_write(this); } +#endif }; +#if GEN_COMPILER_CPP && ! GEN_C_LIKE_CPP +void builder_pad_lines( Builder& builder, s32 num ) { return builder_pad_lines(& builder, num); } +void builder_print ( Builder& builder, Code code ) { return builder_print(& builder, code); } +void builder_write ( Builder& builder ) { return builder_write(& builder ); } +void builder_print_fmt( Builder& builder, char const* fmt, ...) { + va_list va; + va_start( va, fmt ); + builder_print_fmt_va( & builder, fmt, va ); + va_end( va ); +} +#endif + +#pragma endregion Builder + GEN_NS_END #ifdef __clang__ diff --git a/Project/Source/GasaEditor/GasaGen/gencpp/gen.cpp b/Project/Source/GasaEditor/GasaGen/gencpp/gen.cpp index 666f214..b5e9341 100644 --- a/Project/Source/GasaEditor/GasaGen/gencpp/gen.cpp +++ b/Project/Source/GasaEditor/GasaGen/gencpp/gen.cpp @@ -9,6 +9,9 @@ # pragma clang diagnostic ignored "-Wunknown-pragmas" # pragma clang diagnostic ignored "-Wvarargs" # pragma clang diagnostic ignored "-Wunused-function" +# pragma clang diagnostic ignored "-Wbraced-scalar-init" +# pragma clang diagnostic ignored "-W#pragma-messages" +# pragma clang diagnostic ignored "-Wstatic-in-inline" #endif #ifdef __GNUC__ @@ -28,29 +31,15 @@ GEN_NS_BEGIN #pragma region StaticData - -// TODO : Convert global allocation strategy to use a slab allocation strategy. -global AllocatorInfo GlobalAllocator; -global Array Global_AllocatorBuckets; - -// TODO(Ed) : Make the code pool a dynamic arena -global Array< Pool > CodePools = { nullptr }; -global Array< Arena > StringArenas = { nullptr }; - -global StringTable StringCache; - -global Arena LexArena; - -global AllocatorInfo Allocator_DataArrays = heap(); -global AllocatorInfo Allocator_CodePool = heap(); -global AllocatorInfo Allocator_Lexer = heap(); -global AllocatorInfo Allocator_StringArena = heap(); -global AllocatorInfo Allocator_StringTable = heap(); -global AllocatorInfo Allocator_TypeTable = heap(); - -#pragma endregion StaticData +global Context* _ctx; #pragma region Constants +global u32 context_counter; + +global Macro enum_underlying_macro; + +global Code Code_Global; +global Code Code_Invalid; global Code access_public; global Code access_protected; @@ -64,7 +53,7 @@ global Code module_private_fragment; global Code fmt_newline; -global CodeParam param_varadic; +global CodeParams param_varadic; global CodePragma pragma_once; @@ -96,709 +85,724 @@ global CodeSpecifiers spec_thread_local; global CodeSpecifiers spec_virtual; global CodeSpecifiers spec_volatile; -global CodeType t_empty; -global CodeType t_auto; -global CodeType t_void; -global CodeType t_int; -global CodeType t_bool; -global CodeType t_char; -global CodeType t_wchar_t; -global CodeType t_class; -global CodeType t_typename; - -global Array< StringCached > PreprocessorDefines; +global CodeTypename t_empty; +global CodeTypename t_auto; +global CodeTypename t_void; +global CodeTypename t_int; +global CodeTypename t_bool; +global CodeTypename t_char; +global CodeTypename t_wchar_t; +global CodeTypename t_class; +global CodeTypename t_typename; #ifdef GEN_DEFINE_LIBRARY_CODE_CONSTANTS -global CodeType t_b32; +global CodeTypename t_b32; -global CodeType t_s8; -global CodeType t_s16; -global CodeType t_s32; -global CodeType t_s64; +global CodeTypename t_s8; +global CodeTypename t_s16; +global CodeTypename t_s32; +global CodeTypename t_s64; -global CodeType t_u8; -global CodeType t_u16; -global CodeType t_u32; -global CodeType t_u64; +global CodeTypename t_u8; +global CodeTypename t_u16; +global CodeTypename t_u32; +global CodeTypename t_u64; -global CodeType t_ssize; -global CodeType t_usize; +global CodeTypename t_ssize; +global CodeTypename t_usize; -global CodeType t_f32; -global CodeType t_f64; +global CodeTypename t_f32; +global CodeTypename t_f64; #endif #pragma endregion Constants +#pragma endregion StaticData + #pragma region AST -# define GEN_AST_BODY_CLASS_UNALLOWED_TYPES \ - case PlatformAttributes: \ - case Class_Body: \ - case Enum_Body: \ - case Extern_Linkage: \ - case Function_Body: \ - case Function_Fwd: \ - case Global_Body: \ - case Namespace: \ - case Namespace_Body: \ - case Operator: \ - case Operator_Fwd: \ - case Parameters: \ - case Specifiers: \ - case Struct_Body: \ - case Typename: -# define GEN_AST_BODY_STRUCT_UNALLOWED_TYPES GEN_AST_BODY_CLASS_UNALLOWED_TYPES +// These macros are used in the swtich cases within ast.cpp, inteface.upfront.cpp, parser.cpp -# define GEN_AST_BODY_FUNCTION_UNALLOWED_TYPES \ - case Access_Public: \ - case Access_Protected: \ - case Access_Private: \ - case PlatformAttributes: \ - case Class_Body: \ - case Enum_Body: \ - case Extern_Linkage: \ - case Friend: \ - case Function_Body: \ - case Function_Fwd: \ - case Global_Body: \ - case Namespace: \ - case Namespace_Body: \ - case Operator: \ - case Operator_Fwd: \ - case Operator_Member: \ - case Operator_Member_Fwd: \ - case Parameters: \ - case Specifiers: \ - case Struct_Body: \ - case Typename: +# define GEN_AST_BODY_CLASS_UNALLOWED_TYPES_CASES \ + case CT_PlatformAttributes: \ + case CT_Class_Body: \ + case CT_Enum_Body: \ + case CT_Extern_Linkage: \ + case CT_Function_Body: \ + case CT_Function_Fwd: \ + case CT_Global_Body: \ + case CT_Namespace: \ + case CT_Namespace_Body: \ + case CT_Operator: \ + case CT_Operator_Fwd: \ + case CT_Parameters: \ + case CT_Specifiers: \ + case CT_Struct_Body: \ + case CT_Typename +# define GEN_AST_BODY_STRUCT_UNALLOWED_TYPES_CASES GEN_AST_BODY_CLASS_UNALLOWED_TYPES_CASES -# define GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES \ - case Access_Public: \ - case Access_Protected: \ - case Access_Private: \ - case PlatformAttributes: \ - case Class_Body: \ - case Enum_Body: \ - case Execution: \ - case Friend: \ - case Function_Body: \ - case Namespace_Body: \ - case Operator_Member: \ - case Operator_Member_Fwd: \ - case Parameters: \ - case Specifiers: \ - case Struct_Body: \ - case Typename: -# define GEN_AST_BODY_EXPORT_UNALLOWED_TYPES GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES -# define GEN_AST_BODY_EXTERN_LINKAGE_UNALLOWED_TYPES GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES +# define GEN_AST_BODY_FUNCTION_UNALLOWED_TYPES_CASES \ + case CT_Access_Public: \ + case CT_Access_Protected: \ + case CT_Access_Private: \ + case CT_PlatformAttributes: \ + case CT_Class_Body: \ + case CT_Enum_Body: \ + case CT_Extern_Linkage: \ + case CT_Friend: \ + case CT_Function_Body: \ + case CT_Function_Fwd: \ + case CT_Global_Body: \ + case CT_Namespace: \ + case CT_Namespace_Body: \ + case CT_Operator: \ + case CT_Operator_Fwd: \ + case CT_Operator_Member: \ + case CT_Operator_Member_Fwd: \ + case CT_Parameters: \ + case CT_Specifiers: \ + case CT_Struct_Body: \ + case CT_Typename -# define GEN_AST_BODY_NAMESPACE_UNALLOWED_TYPES \ - case Access_Public: \ - case Access_Protected: \ - case Access_Private: \ - case PlatformAttributes: \ - case Class_Body: \ - case Enum_Body: \ - case Execution: \ - case Friend: \ - case Function_Body: \ - case Namespace_Body: \ - case Operator_Member: \ - case Operator_Member_Fwd: \ - case Parameters: \ - case Specifiers: \ - case Struct_Body: \ - case Typename: +# define GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES_CASES \ + case CT_Access_Public: \ + case CT_Access_Protected: \ + case CT_Access_Private: \ + case CT_PlatformAttributes: \ + case CT_Class_Body: \ + case CT_Enum_Body: \ + case CT_Execution: \ + case CT_Friend: \ + case CT_Function_Body: \ + case CT_Namespace_Body: \ + case CT_Operator_Member: \ + case CT_Operator_Member_Fwd: \ + case CT_Parameters: \ + case CT_Specifiers: \ + case CT_Struct_Body: \ + case CT_Typename +# define GEN_AST_BODY_EXPORT_UNALLOWED_TYPES_CASES GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES_CASES +# define GEN_AST_BODY_EXTERN_LINKAGE_UNALLOWED_TYPES_CASES GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES_CASES -Code Code::Global; -Code Code::Invalid; +# define GEN_AST_BODY_NAMESPACE_UNALLOWED_TYPES_CASES \ + case CT_Access_Public: \ + case CT_Access_Protected: \ + case CT_Access_Private: \ + case CT_PlatformAttributes: \ + case CT_Class_Body: \ + case CT_Enum_Body: \ + case CT_Execution: \ + case CT_Friend: \ + case CT_Function_Body: \ + case CT_Namespace_Body: \ + case CT_Operator_Member: \ + case CT_Operator_Member_Fwd: \ + case CT_Parameters: \ + case CT_Specifiers: \ + case CT_Struct_Body: \ + case CT_Typename // This serializes all the data-members in a "debug" format, where each member is printed with its associated value. -char const* AST::debug_str() +Str code_debug_str(Code self) { - String result = String::make_reserve( GlobalAllocator, kilobytes(1) ); + GEN_ASSERT(self != nullptr); + StrBuilder result_stack = strbuilder_make_reserve( _ctx->Allocator_Temp, kilobytes(1) ); + StrBuilder* result = & result_stack; - if ( Parent ) - result.append_fmt( "\n\tParent : %S %S", Parent->type_str(), Name ? Name : "" ); + if ( self->Parent ) + strbuilder_append_fmt( result, "\n\tParent : %S %S", code_type_str(self->Parent), self->Name.Len ? self->Name : txt("Null") ); else - result.append_fmt( "\n\tParent : %S", "Null" ); + strbuilder_append_fmt( result, "\n\tParent : %S", txt("Null") ); - result.append_fmt( "\n\tName : %S", Name ? Name : "Null" ); - result.append_fmt( "\n\tType : %S", type_str() ); - result.append_fmt( "\n\tModule Flags : %S", to_str( ModuleFlags ) ); + strbuilder_append_fmt( result, "\n\tName : %S", self->Name.Len ? self->Name : txt("Null") ); + strbuilder_append_fmt( result, "\n\tType : %S", code_type_str(self) ); + strbuilder_append_fmt( result, "\n\tModule Flags : %S", module_flag_to_str( self->ModuleFlags ) ); - switch ( Type ) + switch ( self->Type ) { - using namespace ECode; - - case Invalid: - case NewLine: - case Access_Private: - case Access_Protected: - case Access_Public: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Invalid: + case CT_NewLine: + case CT_Access_Private: + case CT_Access_Protected: + case CT_Access_Public: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); break; - case Untyped: - case Execution: - case Comment: - case PlatformAttributes: - case Preprocess_Define: - case Preprocess_Include: - case Preprocess_Pragma: - case Preprocess_If: - case Preprocess_ElIf: - case Preprocess_Else: - case Preprocess_IfDef: - case Preprocess_IfNotDef: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Untyped: + case CT_Execution: + case CT_Comment: + case CT_PlatformAttributes: + case CT_Preprocess_Include: + case CT_Preprocess_Pragma: + case CT_Preprocess_If: + case CT_Preprocess_ElIf: + case CT_Preprocess_Else: + case CT_Preprocess_IfDef: + case CT_Preprocess_IfNotDef: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tContent: %S", Content ); + strbuilder_append_fmt( result, "\n\tContent: %S", self->Content ); break; - case Class: - case Struct: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - - result.append_fmt( "\n\tInlineCmd : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes : %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tParentAccess: %s", ParentType ? to_str( ParentAccess ) : "No Parent" ); - result.append_fmt( "\n\tParentType : %s", ParentType ? ParentType->type_str() : "Null" ); - result.append_fmt( "\n\tBody : %S", Body ? Body->debug_str() : "Null" ); + case CT_Preprocess_Define: + // TODO(ED): Needs implementaton + log_failure("code_debug_str: NOT IMPLEMENTED for CT_Preprocess_Define"); break; - case Class_Fwd: - case Struct_Fwd: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Class: + case CT_Struct: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmd : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes : %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tParentAccess: %s", ParentType ? to_str( ParentAccess ) : "No Parent" ); - result.append_fmt( "\n\tParentType : %s", ParentType ? ParentType->type_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes : %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParentAccess: %S", self->ParentType ? access_spec_to_str( self->ParentAccess ) : txt("No Parent") ); + strbuilder_append_fmt( result, "\n\tParentType : %S", self->ParentType ? code_type_str(self->ParentType) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBody : %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); break; - case Constructor: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Class_Fwd: + case CT_Struct_Fwd: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tInitializerList: %S", InitializerList ? InitializerList->to_string() : "Null" ); - result.append_fmt( "\n\tParams : %S", Params ? Params->to_string() : "Null" ); - result.append_fmt( "\n\tBody : %S", Body ? Body->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes : %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParentAccess: %S", self->ParentType ? access_spec_to_str( self->ParentAccess ) : txt("No Parent") ); + strbuilder_append_fmt( result, "\n\tParentType : %S", self->ParentType ? code_type_str(self->ParentType) : txt("Null") ); break; - case Constructor_Fwd: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Constructor: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tInitializerList: %S", InitializerList ? InitializerList->to_string() : "Null" ); - result.append_fmt( "\n\tParams : %S", Params ? Params->to_string() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tInitializerList: %S", self->InitializerList ? strbuilder_to_str( code_to_strbuilder(self->InitializerList) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParams : %S", self->Params ? strbuilder_to_str( code_to_strbuilder(self->Params) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBody : %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); break; - case Destructor: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Constructor_Fwd: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tBody : %S", Body ? Body->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tInitializerList: %S", self->InitializerList ? strbuilder_to_str( code_to_strbuilder(self->InitializerList) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParams : %S", self->Params ? strbuilder_to_str( code_to_strbuilder(self->Params) ) : txt("Null") ); break; - case Destructor_Fwd: + case CT_Destructor: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBody : %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); break; - case Enum: - case Enum_Class: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes : %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tUnderlying Type : %S", UnderlyingType ? UnderlyingType->to_string() : "Null" ); - result.append_fmt( "\n\tBody : %S", Body ? Body->debug_str() : "Null" ); + case CT_Destructor_Fwd: break; - case Enum_Fwd: - case Enum_Class_Fwd: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Enum: + case CT_Enum_Class: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes : %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tUnderlying Type : %S", UnderlyingType ? UnderlyingType->to_string() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes : %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tUnderlying Type : %S", self->UnderlyingType ? strbuilder_to_str( code_to_strbuilder(self->UnderlyingType)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBody : %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); break; - case Extern_Linkage: - case Namespace: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Enum_Fwd: + case CT_Enum_Class_Fwd: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tBody: %S", Body ? Body->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes : %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tUnderlying Type : %S", self->UnderlyingType ? strbuilder_to_str( code_to_strbuilder(self->UnderlyingType)) : txt("Null") ); break; - case Friend: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Extern_Linkage: + case CT_Namespace: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tDeclaration: %S", Declaration ? Declaration->to_string() : "Null" ); + strbuilder_append_fmt( result, "\n\tBody: %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); break; - case Function: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Friend: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes: %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tReturnType: %S", ReturnType ? ReturnType->to_string() : "Null" ); - result.append_fmt( "\n\tParams : %S", Params ? Params->to_string() : "Null" ); - result.append_fmt( "\n\tBody : %S", Body ? Body->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tDeclaration: %S", self->Declaration ? strbuilder_to_str( code_to_strbuilder(self->Declaration)) : txt("Null") ); break; - case Function_Fwd: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Function: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes: %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tReturnType: %S", ReturnType ? ReturnType->to_string() : "Null" ); - result.append_fmt( "\n\tParams : %S", Params ? Params->to_string() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes: %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tReturnType: %S", self->ReturnType ? strbuilder_to_str( code_to_strbuilder(self->ReturnType)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParams : %S", self->Params ? strbuilder_to_str( code_to_strbuilder(self->Params)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBody : %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); break; - case Module: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Function_Fwd: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes: %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tReturnType: %S", self->ReturnType ? strbuilder_to_str( code_to_strbuilder(self->ReturnType)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParams : %S", self->Params ? strbuilder_to_str( code_to_strbuilder(self->Params)) : txt("Null") ); break; - case Operator: - case Operator_Member: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes: %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tReturnType: %S", ReturnType ? ReturnType->to_string() : "Null" ); - result.append_fmt( "\n\tParams : %S", Params ? Params->to_string() : "Null" ); - result.append_fmt( "\n\tBody : %S", Body ? Body->debug_str() : "Null" ); - result.append_fmt( "\n\tOp : %S", to_str( Op ) ); + case CT_Module: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); break; - case Operator_Fwd: - case Operator_Member_Fwd: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Operator: + case CT_Operator_Member: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes: %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tReturnType: %S", ReturnType ? ReturnType->to_string() : "Null" ); - result.append_fmt( "\n\tParams : %S", Params ? Params->to_string() : "Null" ); - result.append_fmt( "\n\tOp : %S", to_str( Op ) ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes: %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tReturnType: %S", self->ReturnType ? strbuilder_to_str( code_to_strbuilder(self->ReturnType)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParams : %S", self->Params ? strbuilder_to_str( code_to_strbuilder(self->Params)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBody : %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tOp : %S", operator_to_str( self->Op ) ); break; - case Operator_Cast: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Operator_Fwd: + case CT_Operator_Member_Fwd: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tValueType : %S", ValueType ? ValueType->to_string() : "Null" ); - result.append_fmt( "\n\tBody : %S", Body ? Body->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes: %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tReturnType: %S", self->ReturnType ? strbuilder_to_str( code_to_strbuilder(self->ReturnType) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParams : %S", self->Params ? strbuilder_to_str( code_to_strbuilder(self->Params) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tOp : %S", operator_to_str( self->Op ) ); break; - case Operator_Cast_Fwd: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Operator_Cast: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tValueType : %S", ValueType ? ValueType->to_string() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tValueType : %S", self->ValueType ? strbuilder_to_str( code_to_strbuilder(self->ValueType)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBody : %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); break; - case Parameters: - result.append_fmt( "\n\tNumEntries: %d", NumEntries ); - result.append_fmt( "\n\tLast : %S", Last->Name ); - result.append_fmt( "\n\tNext : %S", Next->Name ); - result.append_fmt( "\n\tValueType : %S", ValueType ? ValueType->to_string() : "Null" ); - result.append_fmt( "\n\tValue : %S", Value ? Value->to_string() : "Null" ); + case CT_Operator_Cast_Fwd: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tValueType : %S", self->ValueType ? strbuilder_to_str( code_to_strbuilder(self->ValueType)) : txt("Null") ); break; - case Specifiers: + case CT_Parameters: + strbuilder_append_fmt( result, "\n\tNumEntries: %d", self->NumEntries ); + strbuilder_append_fmt( result, "\n\tLast : %S", self->Last->Name ); + strbuilder_append_fmt( result, "\n\tNext : %S", self->Next->Name ); + strbuilder_append_fmt( result, "\n\tValueType : %S", self->ValueType ? strbuilder_to_str( code_to_strbuilder(self->ValueType)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tValue : %S", self->Value ? strbuilder_to_str( code_to_strbuilder(self->Value)) : txt("Null") ); + break; + + case CT_Parameters_Define: + // TODO(ED): Needs implementaton + log_failure("code_debug_str: NOT IMPLEMENTED for CT_Parameters_Define"); + break; + + case CT_Specifiers: { - result.append_fmt( "\n\tNumEntries: %d", NumEntries ); - result.append( "\n\tArrSpecs: " ); + strbuilder_append_fmt( result, "\n\tNumEntries: %d", self->NumEntries ); + strbuilder_append_str( result, txt("\n\tArrSpecs: ") ); s32 idx = 0; - s32 left = NumEntries; + s32 left = self->NumEntries; while ( left-- ) { - StrC spec = ESpecifier::to_str( ArrSpecs[idx] ); - result.append_fmt( "%.*s, ", spec.Len, spec.Ptr ); + Str spec = spec_to_str( self->ArrSpecs[idx] ); + strbuilder_append_fmt( result, "%.*s, ", spec.Len, spec.Ptr ); idx++; } - result.append_fmt( "\n\tNextSpecs: %S", NextSpecs ? NextSpecs->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tNextSpecs: %S", self->NextSpecs ? code_debug_str(self->NextSpecs) : txt("Null") ); } break; - case Template: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Template: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tParams : %S", Params ? Params->to_string() : "Null" ); - result.append_fmt( "\n\tDeclaration: %S", Declaration ? Declaration->to_string() : "Null" ); + strbuilder_append_fmt( result, "\n\tParams : %S", self->Params ? strbuilder_to_str( code_to_strbuilder(self->Params)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tDeclaration: %S", self->Declaration ? strbuilder_to_str( code_to_strbuilder(self->Declaration)) : txt("Null") ); break; - case Typedef: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Typedef: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tUnderlyingType: %S", UnderlyingType ? UnderlyingType->to_string() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tUnderlyingType: %S", self->UnderlyingType ? strbuilder_to_str( code_to_strbuilder(self->UnderlyingType)) : txt("Null") ); break; - case Typename: - result.append_fmt( "\n\tAttributes : %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tReturnType : %S", ReturnType ? ReturnType->to_string() : "Null" ); - result.append_fmt( "\n\tParams : %S", Params ? Params->to_string() : "Null" ); - result.append_fmt( "\n\tArrExpr : %S", ArrExpr ? ArrExpr->to_string() : "Null" ); + case CT_Typename: + strbuilder_append_fmt( result, "\n\tAttributes : %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tReturnType : %S", self->ReturnType ? strbuilder_to_str( code_to_strbuilder(self->ReturnType)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tParams : %S", self->Params ? strbuilder_to_str( code_to_strbuilder(self->Params)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tArrExpr : %S", self->ArrExpr ? strbuilder_to_str( code_to_strbuilder(self->ArrExpr)) : txt("Null") ); break; - case Union: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Union: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tAttributes: %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tBody : %S", Body ? Body->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tAttributes: %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBody : %S", self->Body ? code_debug_str(self->Body) : txt("Null") ); break; - case Using: - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + case CT_Using: + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes : %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tUnderlyingType: %S", UnderlyingType ? UnderlyingType->to_string() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes : %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tUnderlyingType: %S", self->UnderlyingType ? strbuilder_to_str( code_to_strbuilder(self->UnderlyingType)) : txt("Null") ); break; - case Variable: + case CT_Variable: - if ( Parent && Parent->Type == Variable ) + if ( self->Parent && self->Parent->Type == CT_Variable ) { // Its a NextVar - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tValue : %S", Value ? Value->to_string() : "Null" ); - result.append_fmt( "\n\tBitfieldSize: %S", BitfieldSize ? BitfieldSize->to_string() : "Null" ); - result.append_fmt( "\n\tNextVar : %S", NextVar ? NextVar->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tValue : %S", self->Value ? strbuilder_to_str( code_to_strbuilder(self->Value)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBitfieldSize: %S", self->BitfieldSize ? strbuilder_to_str( code_to_strbuilder(self->BitfieldSize)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tNextVar : %S", self->NextVar ? code_debug_str(self->NextVar) : txt("Null") ); break; } - if ( Prev ) - result.append_fmt( "\n\tPrev: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); - if ( Next ) - result.append_fmt( "\n\tNext: %S %S", Prev->type_str(), Prev->Name ? Prev->Name : "Null" ); + if ( self->Prev ) + strbuilder_append_fmt( result, "\n\tPrev: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); + if ( self->Next ) + strbuilder_append_fmt( result, "\n\tNext: %S %S", code_type_str(self->Prev), self->Prev->Name.Len ? self->Prev->Name : txt("Null") ); - result.append_fmt( "\n\tInlineCmt : %S", InlineCmt ? InlineCmt->Content : "Null" ); - result.append_fmt( "\n\tAttributes : %S", Attributes ? Attributes->to_string() : "Null" ); - result.append_fmt( "\n\tSpecs : %S", Specs ? Specs->to_string() : "Null" ); - result.append_fmt( "\n\tValueType : %S", ValueType ? ValueType->to_string() : "Null" ); - result.append_fmt( "\n\tBitfieldSize: %S", BitfieldSize ? BitfieldSize->to_string() : "Null" ); - result.append_fmt( "\n\tValue : %S", Value ? Value->to_string() : "Null" ); - result.append_fmt( "\n\tNextVar : %S", NextVar ? NextVar->debug_str() : "Null" ); + strbuilder_append_fmt( result, "\n\tInlineCmt : %S", self->InlineCmt ? self->InlineCmt->Content : txt("Null") ); + strbuilder_append_fmt( result, "\n\tAttributes : %S", self->Attributes ? strbuilder_to_str( code_to_strbuilder(self->Attributes) ) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tSpecs : %S", self->Specs ? strbuilder_to_str( code_to_strbuilder(self->Specs)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tValueType : %S", self->ValueType ? strbuilder_to_str( code_to_strbuilder(self->ValueType)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tBitfieldSize: %S", self->BitfieldSize ? strbuilder_to_str( code_to_strbuilder(self->BitfieldSize)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tValue : %S", self->Value ? strbuilder_to_str( code_to_strbuilder(self->Value)) : txt("Null") ); + strbuilder_append_fmt( result, "\n\tNextVar : %S", self->NextVar ? code_debug_str(self->NextVar) : txt("Null") ); break; } + return strbuilder_to_str( * result ); +} + +Code code_duplicate(Code self) +{ + Code result = make_code(); + + void* mem_result = rcast(void*, cast(AST*, result)); + void* mem_self = rcast(void*, cast(AST*, self)); + mem_copy( mem_result, mem_self, sizeof( AST ) ); + + result->Parent = NullCode; return result; } -AST* AST::duplicate() +StrBuilder code_to_strbuilder(Code self) { - using namespace ECode; - - AST* result = make_code().ast; - - mem_copy( result, this, sizeof( AST ) ); - - result->Parent = nullptr; + StrBuilder result = strbuilder_make_str( _ctx->Allocator_Temp, txt("") ); + code_to_strbuilder_ptr( self, & result ); return result; } -String AST::to_string() -{ - String result = String::make( GlobalAllocator, "" ); - to_string( result ); - return result; -} - -void AST::to_string( String& result ) +void code_to_strbuilder_ptr( Code self, StrBuilder* result ) { + GEN_ASSERT(self != nullptr); local_persist thread_local char SerializationLevel = 0; - switch ( Type ) + switch ( self->Type ) { - using namespace ECode; - - case Invalid: + case CT_Invalid: #ifdef GEN_DONT_ALLOW_INVALID_CODE - log_failure("Attempted to serialize invalid code! - %S", Parent ? Parent->debug_str() : Name ); + log_failure("Attempted to serialize invalid code! - %S", Parent ? Parent->code_debug_str() : Name ); #else - result.append_fmt( "Invalid Code!" ); + strbuilder_append_fmt( result, "Invalid Code!" ); #endif break; - case NewLine: - result.append("\n"); + case CT_NewLine: + strbuilder_append_str( result, txt("\n")); break; - case Untyped: - case Execution: - case Comment: - case PlatformAttributes: - result.append( Content ); + case CT_Untyped: + case CT_Execution: + case CT_Comment: + case CT_PlatformAttributes: + strbuilder_append_str( result, self->Content ); break; - case Access_Private: - case Access_Protected: - case Access_Public: - result.append( Name ); + case CT_Access_Private: + case CT_Access_Protected: + case CT_Access_Public: + strbuilder_append_str( result, self->Name ); break; - case Class: - cast().to_string_def( result ); + case CT_Class: + class_to_strbuilder_def(cast(CodeClass, self), result ); break; - case Class_Fwd: - cast().to_string_fwd( result ); + case CT_Class_Fwd: + class_to_strbuilder_fwd(cast(CodeClass, self), result ); break; - case Constructor: - cast().to_string_def( result ); + case CT_Constructor: + constructor_to_strbuilder_def(cast(CodeConstructor, self), result ); break; - case Constructor_Fwd: - cast().to_string_fwd( result ); + case CT_Constructor_Fwd: + constructor_to_strbuilder_fwd(cast(CodeConstructor, self), result ); break; - case Destructor: - cast().to_string_def( result ); + case CT_Destructor: + destructor_to_strbuilder_def(cast(CodeDestructor, self), result ); break; - case Destructor_Fwd: - cast().to_string_fwd( result ); + case CT_Destructor_Fwd: + destructor_to_strbuilder_fwd(cast(CodeDestructor, self), result ); break; - case Enum: - cast().to_string_def( result ); + case CT_Enum: + enum_to_strbuilder_def(cast(CodeEnum, self), result ); break; - case Enum_Fwd: - cast().to_string_fwd( result ); + case CT_Enum_Fwd: + enum_to_strbuilder_fwd(cast(CodeEnum, self), result ); break; - case Enum_Class: - cast().to_string_class_def( result ); + case CT_Enum_Class: + enum_to_strbuilder_class_def(cast(CodeEnum, self), result ); break; - case Enum_Class_Fwd: - cast().to_string_class_fwd( result ); + case CT_Enum_Class_Fwd: + enum_to_strbuilder_class_fwd(cast(CodeEnum, self), result ); break; - case Export_Body: - cast().to_string_export( result ); + case CT_Export_Body: + body_to_strbuilder_export(cast(CodeBody, self), result ); break; - case Extern_Linkage: - cast().to_string( result ); + case CT_Extern_Linkage: + extern_to_strbuilder(cast(CodeExtern, self), result ); break; - case Friend: - cast().to_string( result ); + case CT_Friend: + friend_to_strbuilder_ref(cast(CodeFriend, self), result ); break; - case Function: - cast().to_string_def( result ); + case CT_Function: + fn_to_strbuilder_def(cast(CodeFn, self), result ); break; - case Function_Fwd: - cast().to_string_fwd( result ); + case CT_Function_Fwd: + fn_to_strbuilder_fwd(cast(CodeFn, self), result ); break; - case Module: - cast().to_string( result ); + case CT_Module: + module_to_strbuilder_ref(cast(CodeModule, self), result ); break; - case Namespace: - cast().to_string( result ); + case CT_Namespace: + namespace_to_strbuilder_ref(cast(CodeNS, self), result ); break; - case Operator: - case Operator_Member: - cast().to_string_def( result ); + case CT_Operator: + case CT_Operator_Member: + code_op_to_strbuilder_def(cast(CodeOperator, self), result ); break; - case Operator_Fwd: - case Operator_Member_Fwd: - cast().to_string_fwd( result ); + case CT_Operator_Fwd: + case CT_Operator_Member_Fwd: + code_op_to_strbuilder_fwd(cast(CodeOperator, self), result ); break; - case Operator_Cast: - cast().to_string_def( result ); + case CT_Operator_Cast: + opcast_to_strbuilder_def(cast(CodeOpCast, self), result ); break; - case Operator_Cast_Fwd: - cast().to_string_fwd( result ); + case CT_Operator_Cast_Fwd: + opcast_to_strbuilder_fwd(cast(CodeOpCast, self), result ); break; - case Parameters: - cast().to_string( result ); + case CT_Parameters: + params_to_strbuilder_ref(cast(CodeParams, self), result ); break; - case Preprocess_Define: - cast().to_string( result ); + case CT_Parameters_Define: + define_params_to_strbuilder_ref(cast(CodeDefineParams, self), result); break; - case Preprocess_If: - cast().to_string_if( result ); + case CT_Preprocess_Define: + define_to_strbuilder_ref(cast(CodeDefine, self), result ); break; - case Preprocess_IfDef: - cast().to_string_ifdef( result ); + case CT_Preprocess_If: + preprocess_to_strbuilder_if(cast(CodePreprocessCond, self), result ); break; - case Preprocess_IfNotDef: - cast().to_string_ifndef( result ); + case CT_Preprocess_IfDef: + preprocess_to_strbuilder_ifdef(cast(CodePreprocessCond, self), result ); break; - case Preprocess_Include: - cast().to_string( result ); + case CT_Preprocess_IfNotDef: + preprocess_to_strbuilder_ifndef(cast(CodePreprocessCond, self), result ); break; - case Preprocess_ElIf: - cast().to_string_elif( result ); + case CT_Preprocess_Include: + include_to_strbuilder_ref(cast(CodeInclude, self), result ); break; - case Preprocess_Else: - cast().to_string_else( result ); + case CT_Preprocess_ElIf: + preprocess_to_strbuilder_elif(cast(CodePreprocessCond, self), result ); break; - case Preprocess_EndIf: - cast().to_string_endif( result ); + case CT_Preprocess_Else: + preprocess_to_strbuilder_else(cast(CodePreprocessCond, self), result ); break; - case Preprocess_Pragma: - cast().to_string( result ); + case CT_Preprocess_EndIf: + preprocess_to_strbuilder_endif(cast(CodePreprocessCond, self), result ); break; - case Specifiers: - cast().to_string( result ); + case CT_Preprocess_Pragma: + pragma_to_strbuilder_ref(cast(CodePragma, self), result ); break; - case Struct: - cast().to_string_def( result ); + case CT_Specifiers: + specifiers_to_strbuilder_ref(cast(CodeSpecifiers, self), result ); break; - case Struct_Fwd: - cast().to_string_fwd( result ); + case CT_Struct: + struct_to_strbuilder_def(cast(CodeStruct, self), result ); break; - case Template: - cast().to_string( result ); + case CT_Struct_Fwd: + struct_to_strbuilder_fwd(cast(CodeStruct, self), result ); break; - case Typedef: - cast().to_string( result ); + case CT_Template: + template_to_strbuilder_ref(cast(CodeTemplate, self), result ); break; - case Typename: - cast().to_string( result ); + case CT_Typedef: + typedef_to_strbuilder_ref(cast(CodeTypedef, self), result ); break; - case Union: - cast().to_string( result ); + case CT_Typename: + typename_to_strbuilder_ref(cast(CodeTypename, self), result ); break; - case Using: - cast().to_string( result ); + case CT_Union: + union_to_strbuilder_def( cast(CodeUnion, self), result ); break; - case Using_Namespace: - cast().to_string_ns( result ); + case CT_Union_Fwd: + union_to_strbuilder_fwd( cast(CodeUnion, self), result ); break; - case Variable: - cast().to_string( result ); + case CT_Using: + using_to_strbuilder_ref(cast(CodeUsing, self), result ); break; - case Enum_Body: - case Class_Body: - case Extern_Linkage_Body: - case Function_Body: - case Global_Body: - case Namespace_Body: - case Struct_Body: - case Union_Body: - cast().to_string( result ); + case CT_Using_Namespace: + using_to_strbuilder_ns(cast(CodeUsing, self), result ); + break; + + case CT_Variable: + var_to_strbuilder_ref(cast(CodeVar, self), result ); + break; + + case CT_Enum_Body: + case CT_Class_Body: + case CT_Extern_Linkage_Body: + case CT_Function_Body: + case CT_Global_Body: + case CT_Namespace_Body: + case CT_Struct_Body: + case CT_Union_Body: + body_to_strbuilder_ref( cast(CodeBody, self), result ); break; } } -bool AST::is_equal( AST* other ) +bool code_is_equal( Code self, Code other ) { /* AST values are either some u32 value, a cached string, or a pointer to another AST. @@ -809,127 +813,124 @@ bool AST::is_equal( AST* other ) */ if ( other == nullptr ) { - log_fmt( "AST::is_equal: other is null\nAST: %S", debug_str() ); + log_fmt( "AST::is_equal: other is null\nAST: %S", code_debug_str(self) ); return false; } - if ( Type != other->Type ) + if ( self->Type != other->Type ) { log_fmt("AST::is_equal: Type check failure with other\nAST: %S\nOther: %S" - , debug_str() - , other->debug_str() + , code_debug_str(self) + , code_debug_str(other) ); return false; } - switch ( Type ) + switch ( self->Type ) { - using namespace ECode; - #define check_member_val( val ) \ - if ( val != other->val ) \ + if ( self->val != other->val ) \ { \ log_fmt("\nAST::is_equal: Member - " #val " failed\n" \ "AST : %S\n" \ "Other: %S\n" \ - , debug_str() \ - , other->debug_str() \ + , code_debug_str(self) \ + ,code_debug_str(other) \ ); \ - \ + \ return false; \ } #define check_member_str( str ) \ - if ( str != other->str ) \ + if ( ! str_are_equal( self->str, other->str ) ) \ { \ log_fmt("\nAST::is_equal: Member string - "#str " failed\n" \ "AST : %S\n" \ "Other: %S\n" \ - , debug_str() \ - , other->debug_str() \ + , code_debug_str(self) \ + ,code_debug_str(other) \ ); \ \ return false; \ } - #define check_member_content( content ) \ - if ( content != other->content ) \ - { \ - log_fmt("\nAST::is_equal: Member content - "#content " failed\n" \ - "AST : %S\n" \ - "Other: %S\n" \ - , debug_str() \ - , other->debug_str() \ - ); \ - \ - log_fmt("Content cannot be trusted to be unique with this check " \ - "so it must be verified by eye for now\n" \ - "AST Content:\n%S\n" \ - "Other Content:\n%S\n" \ - , content.visualize_whitespace() \ - , other->content.visualize_whitespace() \ - ); \ + #define check_member_content( content ) \ + if ( ! str_are_equal( self->content, other->content )) \ + { \ + log_fmt("\nAST::is_equal: Member content - "#content " failed\n" \ + "AST : %S\n" \ + "Other: %S\n" \ + , code_debug_str(self) \ + , code_debug_str(other) \ + ); \ + \ + log_fmt("Content cannot be trusted to be unique with this check " \ + "so it must be verified by eye for now\n" \ + "AST Content:\n%S\n" \ + "Other Content:\n%S\n" \ + , str_visualize_whitespace(self->content, _ctx->Allocator_Temp) \ + , str_visualize_whitespace(other->content, _ctx->Allocator_Temp) \ + ); \ } #define check_member_ast( ast ) \ - if ( ast ) \ + if ( self->ast ) \ { \ if ( other->ast == nullptr ) \ { \ log_fmt("\nAST::is_equal: Failed for member " #ast " other equivalent param is null\n" \ - "AST : %s\n" \ - "Other: %s\n" \ - "For ast member: %s\n" \ - , debug_str() \ - , other->debug_str() \ - , ast->debug_str() \ + "AST : %S\n" \ + "Other: %S\n" \ + "For ast member: %S\n" \ + , code_debug_str(self) \ + , code_debug_str(other) \ + , code_debug_str(self->ast) \ ); \ \ return false; \ } \ \ - if ( ! ast->is_equal( other->ast ) ) \ + if ( ! code_is_equal(self->ast, other->ast ) ) \ { \ log_fmt( "\nAST::is_equal: Failed for " #ast"\n" \ - "AST : %S\n" \ - "Other: %S\n" \ - "For ast member: %S\n" \ - "other's ast member: %S\n" \ - , debug_str() \ - , other->debug_str() \ - , ast->debug_str() \ - , other->ast->debug_str() \ + "AST : %S\n" \ + "Other: %S\n" \ + "For ast member: %S\n" \ + "other's ast member: %S\n" \ + , code_debug_str(self) \ + , code_debug_str(other) \ + , code_debug_str(self->ast) \ + , code_debug_str(other->ast) \ ); \ \ return false; \ } \ } - case NewLine: - case Access_Public: - case Access_Protected: - case Access_Private: - case Preprocess_Else: - case Preprocess_EndIf: + case CT_NewLine: + case CT_Access_Public: + case CT_Access_Protected: + case CT_Access_Private: + case CT_Preprocess_Else: + case CT_Preprocess_EndIf: return true; // Comments are not validated. - case Comment: + case CT_Comment: return true; - case Execution: - case PlatformAttributes: - case Untyped: + case CT_Execution: + case CT_PlatformAttributes: + case CT_Untyped: { check_member_content( Content ); - return true; } - case Class_Fwd: - case Struct_Fwd: + case CT_Class_Fwd: + case CT_Struct_Fwd: { check_member_str( Name ); check_member_ast( ParentType ); @@ -939,8 +940,8 @@ bool AST::is_equal( AST* other ) return true; } - case Class: - case Struct: + case CT_Class: + case CT_Struct: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -952,7 +953,7 @@ bool AST::is_equal( AST* other ) return true; } - case Constructor: + case CT_Constructor: { check_member_ast( InitializerList ); check_member_ast( Params ); @@ -961,7 +962,7 @@ bool AST::is_equal( AST* other ) return true; } - case Constructor_Fwd: + case CT_Constructor_Fwd: { check_member_ast( InitializerList ); check_member_ast( Params ); @@ -969,7 +970,7 @@ bool AST::is_equal( AST* other ) return true; } - case Destructor: + case CT_Destructor: { check_member_ast( Specs ); check_member_ast( Body ); @@ -977,37 +978,39 @@ bool AST::is_equal( AST* other ) return true; } - case Destructor_Fwd: + case CT_Destructor_Fwd: { check_member_ast( Specs ); return true; } - case Enum: - case Enum_Class: + case CT_Enum: + case CT_Enum_Class: { check_member_val( ModuleFlags ); check_member_str( Name ); check_member_ast( Attributes ); check_member_ast( UnderlyingType ); check_member_ast( Body ); + check_member_ast( UnderlyingTypeMacro ); return true; } - case Enum_Fwd: - case Enum_Class_Fwd: + case CT_Enum_Fwd: + case CT_Enum_Class_Fwd: { check_member_val( ModuleFlags ); check_member_str( Name ); check_member_ast( Attributes ); check_member_ast( UnderlyingType ); + check_member_ast( UnderlyingTypeMacro ); return true; } - case Extern_Linkage: + case CT_Extern_Linkage: { check_member_str( Name ); check_member_ast( Body ); @@ -1015,7 +1018,7 @@ bool AST::is_equal( AST* other ) return true; } - case Friend: + case CT_Friend: { check_member_str( Name ); check_member_ast( Declaration ); @@ -1023,7 +1026,7 @@ bool AST::is_equal( AST* other ) return true; } - case Function: + case CT_Function: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1036,7 +1039,7 @@ bool AST::is_equal( AST* other ) return true; } - case Function_Fwd: + case CT_Function_Fwd: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1048,7 +1051,7 @@ bool AST::is_equal( AST* other ) return true; } - case Module: + case CT_Module: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1056,7 +1059,7 @@ bool AST::is_equal( AST* other ) return true; } - case Namespace: + case CT_Namespace: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1065,8 +1068,8 @@ bool AST::is_equal( AST* other ) return true; } - case Operator: - case Operator_Member: + case CT_Operator: + case CT_Operator_Member: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1079,8 +1082,8 @@ bool AST::is_equal( AST* other ) return true; } - case Operator_Fwd: - case Operator_Member_Fwd: + case CT_Operator_Fwd: + case CT_Operator_Member_Fwd: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1092,7 +1095,7 @@ bool AST::is_equal( AST* other ) return true; } - case Operator_Cast: + case CT_Operator_Cast: { check_member_str( Name ); check_member_ast( Specs ); @@ -1102,7 +1105,7 @@ bool AST::is_equal( AST* other ) return true; } - case Operator_Cast_Fwd: + case CT_Operator_Cast_Fwd: { check_member_str( Name ); check_member_ast( Specs ); @@ -1111,12 +1114,12 @@ bool AST::is_equal( AST* other ) return true; } - case Parameters: + case CT_Parameters: { - if ( NumEntries > 1 ) + if ( self->NumEntries > 1 ) { - AST* curr = this; - AST* curr_other = other; + Code curr = self; + Code curr_other = other; while ( curr != nullptr ) { if ( curr ) @@ -1124,56 +1127,56 @@ bool AST::is_equal( AST* other ) if ( curr_other == nullptr ) { log_fmt("\nAST::is_equal: Failed for parameter, other equivalent param is null\n" - "AST : %S\n" - "Other: %S\n" - "For ast member: %S\n" - , curr->debug_str() + "AST : %S\n" + "Other: %S\n" + "For ast member: %S\n" + , code_debug_str(curr) ); return false; } - if ( curr->Name != curr_other->Name ) + if ( str_are_equal(curr->Name, curr_other->Name) ) { log_fmt( "\nAST::is_equal: Failed for parameter name check\n" "AST : %S\n" "Other: %S\n" "For ast member: %S\n" "other's ast member: %S\n" - , debug_str() - , other->debug_str() - , curr->debug_str() - , curr_other->debug_str() + , code_debug_str(self) + , code_debug_str(other) + , code_debug_str(curr) + , code_debug_str(curr_other) ); return false; } - if ( curr->ValueType && ! curr->ValueType->is_equal(curr_other->ValueType) ) + if ( curr->ValueType && ! code_is_equal(curr->ValueType, curr_other->ValueType) ) { log_fmt( "\nAST::is_equal: Failed for parameter value type check\n" "AST : %S\n" "Other: %S\n" "For ast member: %S\n" "other's ast member: %S\n" - , debug_str() - , other->debug_str() - , curr->debug_str() - , curr_other->debug_str() + , code_debug_str(self) + , code_debug_str(other) + , code_debug_str(curr) + , code_debug_str(curr_other) ); return false; } - if ( curr->Value && ! curr->Value->is_equal(curr_other->Value) ) + if ( curr->Value && ! code_is_equal(curr->Value, curr_other->Value) ) { log_fmt( "\nAST::is_equal: Failed for parameter value check\n" "AST : %S\n" "Other: %S\n" "For ast member: %S\n" "other's ast member: %S\n" - , debug_str() - , other->debug_str() - , curr->debug_str() - , curr_other->debug_str() + , code_debug_str(self) + , code_debug_str(other) + , code_debug_str(curr) + , code_debug_str(curr_other) ); return false; } @@ -1196,44 +1199,50 @@ bool AST::is_equal( AST* other ) return true; } - case Preprocess_Define: + case CT_Parameters_Define: + { + // TODO(ED): Needs implementaton + log_failure("code_is_equal: NOT IMPLEMENTED for CT_Parameters_Define"); + return false; + } + + case CT_Preprocess_Define: { check_member_str( Name ); - check_member_content( Content ); - + check_member_content( Body->Content ); return true; } - case Preprocess_If: - case Preprocess_IfDef: - case Preprocess_IfNotDef: - case Preprocess_ElIf: + case CT_Preprocess_If: + case CT_Preprocess_IfDef: + case CT_Preprocess_IfNotDef: + case CT_Preprocess_ElIf: { check_member_content( Content ); return true; } - case Preprocess_Include: - case Preprocess_Pragma: + case CT_Preprocess_Include: + case CT_Preprocess_Pragma: { check_member_content( Content ); return true; } - case Specifiers: + case CT_Specifiers: { check_member_val( NumEntries ); check_member_str( Name ); - for ( s32 idx = 0; idx < NumEntries; ++idx ) + for ( s32 idx = 0; idx < self->NumEntries; ++idx ) { check_member_val( ArrSpecs[ idx ] ); } return true; } - case Template: + case CT_Template: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1243,7 +1252,7 @@ bool AST::is_equal( AST* other ) return true; } - case Typedef: + case CT_Typedef: { check_member_val( IsFunction ); check_member_val( ModuleFlags ); @@ -1253,7 +1262,7 @@ bool AST::is_equal( AST* other ) return true; } - case Typename: + case CT_Typename: { check_member_val( IsParamPack ); check_member_str( Name ); @@ -1263,7 +1272,7 @@ bool AST::is_equal( AST* other ) return true; } - case Union: + case CT_Union: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1273,8 +1282,15 @@ bool AST::is_equal( AST* other ) return true; } - case Using: - case Using_Namespace: + case CT_Union_Fwd: + { + check_member_val( ModuleFlags ); + check_member_str( Name ); + check_member_ast( Attributes ); + } + + case CT_Using: + case CT_Using_Namespace: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1284,7 +1300,7 @@ bool AST::is_equal( AST* other ) return true; } - case Variable: + case CT_Variable: { check_member_val( ModuleFlags ); check_member_str( Name ); @@ -1298,44 +1314,44 @@ bool AST::is_equal( AST* other ) return true; } - case Class_Body: - case Enum_Body: - case Export_Body: - case Global_Body: - case Namespace_Body: - case Struct_Body: - case Union_Body: + case CT_Class_Body: + case CT_Enum_Body: + case CT_Export_Body: + case CT_Global_Body: + case CT_Namespace_Body: + case CT_Struct_Body: + case CT_Union_Body: { check_member_ast( Front ); check_member_ast( Back ); - AST* curr = Front; - AST* curr_other = other->Front; + Code curr = self->Front; + Code curr_other = other->Front; while ( curr != nullptr ) { if ( curr_other == nullptr ) { log_fmt("\nAST::is_equal: Failed for body, other equivalent param is null\n" - "AST : %S\n" - "Other: %S\n" - "For ast member: %S\n" - , curr->debug_str() + "AST : %S\n" + "Other: %S\n" + , code_debug_str(curr) + , code_debug_str(other) ); return false; } - if ( ! curr->is_equal( curr_other ) ) + if ( ! code_is_equal( curr, curr_other ) ) { log_fmt( "\nAST::is_equal: Failed for body\n" "AST : %S\n" "Other: %S\n" "For ast member: %S\n" "other's ast member: %S\n" - , debug_str() - , other->debug_str() - , curr->debug_str() - , curr_other->debug_str() + , code_debug_str(self) + , code_debug_str(other) + , code_debug_str(curr) + , code_debug_str(curr_other) ); return false; @@ -1358,1505 +1374,1669 @@ bool AST::is_equal( AST* other ) return true; } -bool AST::validate_body() +bool code_validate_body(Code self) { - using namespace ECode; - -#define CheckEntries( Unallowed_Types ) \ - do \ - { \ - for ( Code entry : cast() ) \ - { \ - switch ( entry->Type ) \ - { \ - Unallowed_Types \ - log_failure( "AST::validate_body: Invalid entry in body %s", entry.debug_str() ); \ - return false; \ - } \ - } \ - } \ - while (0); - - switch ( Type ) + switch ( self->Type ) { - case Class_Body: - CheckEntries( GEN_AST_BODY_CLASS_UNALLOWED_TYPES ); - break; - case Enum_Body: - for ( Code entry : cast() ) + case CT_Class_Body: + { + CodeBody body = cast(CodeBody, self); + for (Code code_entry = begin_CodeBody(body); code_entry != end_CodeBody(body); next_CodeBody(body, code_entry)) switch (code_entry->Type) { - if ( entry->Type != Untyped ) + GEN_AST_BODY_CLASS_UNALLOWED_TYPES_CASES: + log_failure("AST::validate_body: Invalid entry in body %S", code_debug_str(code_entry)); + return false; + + default: + continue; + } + } + break; + case CT_Enum_Body: + { + CodeBody body = cast(CodeBody, self); + for ( Code entry = begin_CodeBody(body); entry != end_CodeBody(body); next_CodeBody(body, entry) ) + { + if ( entry->Type != CT_Untyped ) { - log_failure( "AST::validate_body: Invalid entry in enum body (needs to be untyped or comment) %s", entry.debug_str() ); + log_failure( "AST::validate_body: Invalid entry in enum body (needs to be untyped or comment) %S", code_debug_str(entry) ); return false; } } + } break; - case Export_Body: - CheckEntries( GEN_AST_BODY_CLASS_UNALLOWED_TYPES ); - break; - case Extern_Linkage: - CheckEntries( GEN_AST_BODY_EXTERN_LINKAGE_UNALLOWED_TYPES ); - break; - case Function_Body: - CheckEntries( GEN_AST_BODY_FUNCTION_UNALLOWED_TYPES ); - break; - case Global_Body: - for (Code entry : cast()) + case CT_Export_Body: + { + CodeBody body = cast(CodeBody, self); + for (Code code_entry = begin_CodeBody(body); code_entry != end_CodeBody(body); next_CodeBody(body, code_entry)) switch (code_entry->Type) { - switch (entry->Type) + GEN_AST_BODY_EXPORT_UNALLOWED_TYPES_CASES: + log_failure("AST::validate_body: Invalid entry in body %S", code_debug_str(code_entry)); + return false; + + default: + continue; + } + } + break; + case CT_Extern_Linkage: + { + CodeBody body = cast(CodeBody, self); + for (Code code_entry = begin_CodeBody(body); code_entry != end_CodeBody(body); next_CodeBody(body, code_entry)) switch (code_entry->Type) + { + GEN_AST_BODY_EXTERN_LINKAGE_UNALLOWED_TYPES_CASES: + log_failure("AST::validate_body: Invalid entry in body %S", code_debug_str(code_entry)); + return false; + + default: + continue; + } + } + break; + case CT_Function_Body: + { + CodeBody body = cast(CodeBody, self); + for (Code code_entry = begin_CodeBody(body); code_entry != end_CodeBody(body); next_CodeBody(body, code_entry)) switch (code_entry->Type) + { + GEN_AST_BODY_FUNCTION_UNALLOWED_TYPES_CASES: + log_failure("AST::validate_body: Invalid entry in body %S", code_debug_str(code_entry)); + return false; + + default: + continue; + } + } + break; + case CT_Global_Body: + { + CodeBody body = cast(CodeBody, self); + for ( Code entry = begin_CodeBody(body); entry != end_CodeBody(body); next_CodeBody(body, entry) )switch (entry->Type) + { + GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES_CASES: + log_failure("AST::validate_body: Invalid entry in body %S", code_debug_str(entry)); + return false; + } + } + break; + case CT_Namespace_Body: + { + CodeBody body = cast(CodeBody, self); + for ( Code entry = begin_CodeBody(body); entry != end_CodeBody(body); next_CodeBody(body, entry) ) switch (entry->Type) + { + GEN_AST_BODY_NAMESPACE_UNALLOWED_TYPES_CASES: + log_failure("AST::validate_body: Invalid entry in body %S", code_debug_str(entry)); + return false; + } + } + break; + case CT_Struct_Body: + { + CodeBody body = cast(CodeBody, self); + for ( Code entry = begin_CodeBody(body); entry != end_CodeBody(body); next_CodeBody(body, entry) ) switch (entry->Type) + { + GEN_AST_BODY_STRUCT_UNALLOWED_TYPES_CASES: + log_failure("AST::validate_body: Invalid entry in body %S", code_debug_str(entry)); + return false; + } + } + break; + case CT_Union_Body: + { + CodeBody body = cast(CodeBody, self); + for ( Code entry = begin_CodeBody(body); entry != end_CodeBody(body); next_CodeBody(body, entry) ) + { + if ( entry->Type != CT_Untyped ) { - case Access_Public: - case Access_Protected: - case Access_Private: - case PlatformAttributes: - case Class_Body: - case Enum_Body: - case Execution: - case Friend: - case Function_Body: - case Global_Body: - case Namespace_Body: - case Operator_Member: - case Operator_Member_Fwd: - case Parameters: - case Specifiers: - case Struct_Body: - case Typename: - log_failure("AST::validate_body: Invalid entry in body %s", entry.debug_str()); - return false; - } - } - break; - case Namespace_Body: - CheckEntries( GEN_AST_BODY_NAMESPACE_UNALLOWED_TYPES ); - break; - case Struct_Body: - CheckEntries( GEN_AST_BODY_STRUCT_UNALLOWED_TYPES ); - break; - case Union_Body: - for ( Code entry : Body->cast() ) - { - if ( entry->Type != Untyped ) - { - log_failure( "AST::validate_body: Invalid entry in union body (needs to be untyped or comment) %s", entry.debug_str() ); + log_failure( "AST::validate_body: Invalid entry in union body (needs to be untyped or comment) %S", code_debug_str(entry) ); return false; } } + } break; default: - log_failure( "AST::validate_body: Invalid this AST does not have a body %s", debug_str() ); + log_failure( "AST::validate_body: Invalid this AST does not have a body %S", code_debug_str(self) ); return false; } - return false; - -#undef CheckEntries } -String Code::to_string() +inline +StrBuilder attributes_to_strbuilder(CodeAttributes attributes) { + GEN_ASSERT(attributes); + char* raw = ccast(char*, str_duplicate( attributes->Content, _ctx->Allocator_Temp ).Ptr); + StrBuilder result = { raw }; + return result; +} + +inline +void attributes_to_strbuilder_ref(CodeAttributes attributes, StrBuilder* result) { + GEN_ASSERT(attributes); + GEN_ASSERT(result); + strbuilder_append_str(result, attributes->Content); +} + +StrBuilder body_to_strbuilder(CodeBody body) { - if ( ast == nullptr ) + GEN_ASSERT(body); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 128 ); + switch ( body->Type ) { - log_failure( "Code::to_string: Cannot convert code to string, AST is null!" ); - return { nullptr }; - } - return rcast( AST*, ast )->to_string(); -} - -String CodeAttributes::to_string() -{ - return ast->Content.duplicate( GlobalAllocator ); -} - -String CodeBody::to_string() -{ - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) - { - using namespace ECode; - case Untyped: - case Execution: - result.append( raw()->Content ); + case CT_Untyped: + case CT_Execution: + strbuilder_append_str( & result, cast(Code, body)->Content ); break; - case Enum_Body: - case Class_Body: - case Extern_Linkage_Body: - case Function_Body: - case Global_Body: - case Namespace_Body: - case Struct_Body: - case Union_Body: - to_string( result ); + case CT_Enum_Body: + case CT_Class_Body: + case CT_Extern_Linkage_Body: + case CT_Function_Body: + case CT_Global_Body: + case CT_Namespace_Body: + case CT_Struct_Body: + case CT_Union_Body: + body_to_strbuilder_ref( body, & result ); break; - case Export_Body: - to_string_export( result ); + case CT_Export_Body: + body_to_strbuilder_export( body, & result ); break; } return result; } -void CodeBody::to_string( String& result ) +void body_to_strbuilder_ref( CodeBody body, StrBuilder* result ) { - Code curr = ast->Front; - s32 left = ast->NumEntries; + GEN_ASSERT(body != nullptr); + GEN_ASSERT(result != nullptr); + Code curr = body->Front; + s32 left = body->NumEntries; while ( left -- ) { - result.append_fmt( "%S", curr.to_string() ); + code_to_strbuilder_ptr(curr, result); + // strbuilder_append_fmt( result, "%SB", code_to_strbuilder(curr) ); ++curr; } } -void CodeBody::to_string_export( String& result ) +void body_to_strbuilder_export( CodeBody body, StrBuilder* result ) { - result.append_fmt( "export\n{\n" ); + GEN_ASSERT(body != nullptr); + GEN_ASSERT(result != nullptr); + strbuilder_append_fmt( result, "export\n{\n" ); - Code curr = *this; - s32 left = ast->NumEntries; + Code curr = cast(Code, body); + s32 left = body->NumEntries; while ( left-- ) { - result.append_fmt( "%S", curr.to_string() ); + code_to_strbuilder_ptr(curr, result); + // strbuilder_append_fmt( result, "%SB", code_to_strbuilder(curr) ); ++curr; } - result.append_fmt( "};\n" ); + strbuilder_append_fmt( result, "};\n" ); } -String CodeComment::to_string() -{ - return ast->Content.duplicate( GlobalAllocator ); +inline +StrBuilder comment_to_strbuilder(CodeComment comment) { + GEN_ASSERT(comment); + char* raw = ccast(char*, str_duplicate( comment->Content, _ctx->Allocator_Temp ).Ptr); + StrBuilder result = { raw }; + return result; } -String CodeConstructor::to_string() +inline +void comment_to_strbuilder_ref(CodeComment comment, StrBuilder* result) { + GEN_ASSERT(comment); + GEN_ASSERT(result); + strbuilder_append_str(result, comment->Content); +} + +StrBuilder constructor_to_strbuilder(CodeConstructor self) { - String result = String::make( GlobalAllocator, "" ); - switch (ast->Type) + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 128 ); + switch (self->Type) { - using namespace ECode; - case Constructor: - to_string_def( result ); + case CT_Constructor: + constructor_to_strbuilder_def( self, & result ); break; - case Constructor_Fwd: - to_string_fwd( result ); + case CT_Constructor_Fwd: + constructor_to_strbuilder_fwd( self, & result ); break; } return result; } -void CodeConstructor::to_string_def( String& result ) +void constructor_to_strbuilder_def(CodeConstructor self, StrBuilder* result ) { - AST* ClassStructParent = ast->Parent->Parent; + Code ClassStructParent = self->Parent->Parent; if (ClassStructParent) { - result.append( ClassStructParent->Name ); + strbuilder_append_str( result, ClassStructParent->Name ); } else { - result.append( ast->Name ); + strbuilder_append_str( result, self->Name ); } - if ( ast->Params ) - result.append_fmt( "( %S )", ast->Params.to_string() ); + if ( self->Params ) + strbuilder_append_fmt( result, "( %SB )", params_to_strbuilder(self->Params) ); else - result.append( "()" ); + strbuilder_append_str( result, txt("()") ); - if ( ast->InitializerList ) - result.append_fmt( " : %S", ast->InitializerList.to_string() ); + if ( self->InitializerList ) + strbuilder_append_fmt( result, " : %SB", code_to_strbuilder(self->InitializerList) ); - if ( ast->InlineCmt ) - result.append_fmt( " // %S", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, " // %S", self->InlineCmt->Content ); - result.append_fmt( "\n{\n%S\n}\n", ast->Body.to_string() ); + strbuilder_append_fmt( result, "\n{\n%SB\n}\n", code_to_strbuilder(self->Body) ); } -void CodeConstructor::to_string_fwd( String& result ) +void constructor_to_strbuilder_fwd(CodeConstructor self, StrBuilder* result ) { - AST* ClassStructParent = ast->Parent->Parent; + Code ClassStructParent = self->Parent->Parent; if (ClassStructParent) { - result.append( ClassStructParent->Name ); + strbuilder_append_str( result, ClassStructParent->Name ); } else { - result.append( ast->Name ); + strbuilder_append_str( result, self->Name ); } - if ( ast->Params ) - result.append_fmt( "( %S )", ast->Params.to_string() ); + if ( self->Params ) + strbuilder_append_fmt( result, "( %SB )", params_to_strbuilder(self->Params) ); else - result.append_fmt("()"); + strbuilder_append_fmt( result, "()"); - if (ast->Body) - result.append_fmt( " = %S", ast->Body.to_string() ); + if (self->Body) + strbuilder_append_fmt( result, " = %SB", code_to_strbuilder(self->Body) ); - if ( ast->InlineCmt ) - result.append_fmt( "; // %S\n", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; // %S\n", self->InlineCmt->Content ); else - result.append( ";\n" ); + strbuilder_append_str( result, txt(";\n") ); } -String CodeClass::to_string() +StrBuilder class_to_strbuilder( CodeClass self ) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 512 ); + switch ( self->Type ) { - using namespace ECode; - case Class: - to_string_def( result ); + case CT_Class: + class_to_strbuilder_def(self, & result ); break; - case Class_Fwd: - to_string_fwd( result ); + case CT_Class_Fwd: + class_to_strbuilder_fwd(self, & result ); break; } return result; } -void CodeClass::to_string_def( String& result ) +void class_to_strbuilder_def( CodeClass self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + GEN_ASSERT(self); - result.append( "class " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes ) + strbuilder_append_str( result, txt("class ") ); + + if ( self->Attributes ) { - result.append_fmt( "%S ", ast->Attributes.to_string() ); + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); } - if ( ast->ParentType ) + if ( self->ParentType ) { - char const* access_level = to_str( ast->ParentAccess ); + Str access_level = access_spec_to_str( self->ParentAccess ); + strbuilder_append_fmt( result, "%S : %S %SB", self->Name, access_level, typename_to_strbuilder(self->ParentType) ); - result.append_fmt( "%S : %s %S", ast->Name, access_level, ast->ParentType.to_string() ); - - CodeType interface = ast->ParentType->Next->cast< CodeType >(); + CodeTypename interface = cast(CodeTypename, self->ParentType->Next); if ( interface ) - result.append( "\n" ); + strbuilder_append_str( result, txt("\n") ); while ( interface ) { - result.append_fmt( ", %S", interface.to_string() ); - interface = interface->Next ? interface->Next->cast< CodeType >() : CodeType { nullptr }; + strbuilder_append_fmt( result, ", public %SB", typename_to_strbuilder(interface) ); + interface = interface->Next ? cast(CodeTypename, interface->Next) : NullCode; } } - else if ( ast->Name ) + else if ( self->Name.Len ) { - result.append( ast->Name ); + strbuilder_append_str( result, self->Name ); } - if ( ast->InlineCmt ) + if ( self->InlineCmt ) { - result.append_fmt( " // %S", ast->InlineCmt->Content ); + strbuilder_append_fmt( result, " // %S", self->InlineCmt->Content ); } - result.append_fmt( "\n{\n%S\n}", ast->Body.to_string() ); + strbuilder_append_fmt( result, "\n{\n%SB\n}", body_to_strbuilder(self->Body) ); - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) - result.append(";\n"); + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) + strbuilder_append_str( result, txt(";\n") ); } -void CodeClass::to_string_fwd( String& result ) +void class_to_strbuilder_fwd( CodeClass self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + GEN_ASSERT(self); - if ( ast->Attributes ) - result.append_fmt( "class %S %S", ast->Attributes.to_string(), ast->Name ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - else result.append_fmt( "class %S", ast->Name ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "class %SB %S", attributes_to_strbuilder(self->Attributes), self->Name ); + + else strbuilder_append_fmt( result, "class %S", self->Name ); // Check if it can have an end-statement - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) { - if ( ast->InlineCmt ) - result.append_fmt( "; // %S\n", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; // %S\n", self->InlineCmt->Content ); else - result.append(";\n"); + strbuilder_append_str( result, txt(";\n") ); } } -String CodeDefine::to_string() +StrBuilder define_to_strbuilder(CodeDefine define) { - return String::fmt_buf( GlobalAllocator, "#define %S %S\n", ast->Name, ast->Content ); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 512 ); + define_to_strbuilder_ref(define, & result); + return result; } -void CodeDefine::to_string( String& result ) +void define_to_strbuilder_ref(CodeDefine define, StrBuilder* result ) { - result.append_fmt( "#define %S %S\n", ast->Name, ast->Content ); + GEN_ASSERT(define); + GEN_ASSERT(define->Body); + GEN_ASSERT(define->Body->Content.Ptr && define->Body->Content.Len > 0); + if (define->Params) { + StrBuilder params_builder = define_params_to_strbuilder(define->Params); + strbuilder_append_fmt( result, "#define %S(%S) %S", define->Name, strbuilder_to_str(params_builder), define->Body->Content ); + } + else { + strbuilder_append_fmt( result, "#define %S %S", define->Name, define->Body->Content ); + } } -String CodeDestructor::to_string() +StrBuilder define_params_to_strbuilder(CodeDefineParams params) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + GEN_ASSERT(params); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 128 ); + define_params_to_strbuilder_ref( params, & result ); + return result; +} + +void define_params_to_strbuilder_ref(CodeDefineParams self, StrBuilder* result) +{ + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( self->Name.Ptr && self->Name.Len ) { - using namespace ECode; - case Destructor: - to_string_def( result ); + strbuilder_append_fmt( result, " %S", self->Name ); + } + if ( self->NumEntries - 1 > 0 ) + { + for ( CodeDefineParams param = begin_CodeDefineParams(self->Next); param != end_CodeDefineParams(self->Next); param = next_CodeDefineParams(self->Next, param) ) + { + strbuilder_append_fmt( result, ", %SB", define_params_to_strbuilder(param) ); + } + } +} + +StrBuilder destructor_to_strbuilder(CodeDestructor self) +{ + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 128 ); + switch ( self->Type ) + { + case CT_Destructor: + destructor_to_strbuilder_def( self, & result ); break; - case Destructor_Fwd: - to_string_fwd( result ); + case CT_Destructor_Fwd: + destructor_to_strbuilder_fwd( self, & result ); break; } return result; } -void CodeDestructor::to_string_def( String& result ) +void destructor_to_strbuilder_def(CodeDestructor self, StrBuilder* result ) { - if ( ast->Name ) + if ( self->Name.Len ) { - result.append_fmt( "%S()", ast->Name ); + strbuilder_append_fmt( result, "%S()", self->Name ); } - else if ( ast->Specs ) + else if ( self->Specs ) { - if ( ast->Specs.has( ESpecifier::Virtual ) ) - result.append_fmt( "virtual ~%S()", ast->Parent->Name ); + if ( specifiers_has(self->Specs, Spec_Virtual ) ) + strbuilder_append_fmt( result, "virtual ~%S()", self->Parent->Name ); else - result.append_fmt( "~%S()", ast->Parent->Name ); + strbuilder_append_fmt( result, "~%S()", self->Parent->Name ); } else - result.append_fmt( "~%S()", ast->Parent->Name ); + strbuilder_append_fmt( result, "~%S()", self->Parent->Name ); - result.append_fmt( "\n{\n%S\n}\n", ast->Body.to_string() ); + strbuilder_append_fmt( result, "\n{\n%SB\n}\n", code_to_strbuilder(self->Body) ); } -void CodeDestructor::to_string_fwd( String& result ) +void destructor_to_strbuilder_fwd(CodeDestructor self, StrBuilder* result ) { - if ( ast->Specs ) + if ( self->Specs ) { - if ( ast->Specs.has( ESpecifier::Virtual ) ) - result.append_fmt( "virtual ~%S();\n", ast->Parent->Name ); + if ( specifiers_has(self->Specs, Spec_Virtual ) ) + strbuilder_append_fmt( result, "virtual ~%S();\n", self->Parent->Name ); else - result.append_fmt( "~%S()", ast->Parent->Name ); + strbuilder_append_fmt( result, "~%S()", self->Parent->Name ); - if ( ast->Specs.has( ESpecifier::Pure ) ) - result.append( " = 0;" ); - else if (ast->Body) - result.append_fmt( " = %S;", ast->Body.to_string() ); + if ( specifiers_has(self->Specs, Spec_Pure ) ) + strbuilder_append_str( result, txt(" = 0;") ); + else if (self->Body) + strbuilder_append_fmt( result, " = %SB;", code_to_strbuilder(self->Body) ); } else - result.append_fmt( "~%S();", ast->Parent->Name ); + strbuilder_append_fmt( result, "~%S();", self->Parent->Name ); - if ( ast->InlineCmt ) - result.append_fmt( " %S", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, " %S", self->InlineCmt->Content ); else - result.append("\n"); + strbuilder_append_str( result, txt("\n")); } -String CodeEnum::to_string() +StrBuilder enum_to_strbuilder(CodeEnum self) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 512 ); + switch ( self->Type ) { - using namespace ECode; - case Enum: - to_string_def( result ); + case CT_Enum: + enum_to_strbuilder_def(self, & result ); break; - case Enum_Fwd: - to_string_fwd( result ); + case CT_Enum_Fwd: + enum_to_strbuilder_fwd(self, & result ); break; - case Enum_Class: - to_string_class_def( result ); + case CT_Enum_Class: + enum_to_strbuilder_class_def(self, & result ); break; - case Enum_Class_Fwd: - to_string_class_fwd( result ); + case CT_Enum_Class_Fwd: + enum_to_strbuilder_class_fwd(self, & result ); break; } return result; } -void CodeEnum::to_string_def( String& result ) +void enum_to_strbuilder_def(CodeEnum self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes || ast->UnderlyingType ) + if ( self->Attributes || self->UnderlyingType || self->UnderlyingTypeMacro ) { - result.append( "enum " ); + strbuilder_append_str( result, txt("enum ") ); - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - if ( ast->UnderlyingType ) - result.append_fmt( "%S : %S\n{\n%S\n}" - , ast->Name - , ast->UnderlyingType.to_string() - , ast->Body.to_string() + if ( self->UnderlyingType ) + strbuilder_append_fmt( result, "%S : %SB\n{\n%SB\n}" + , self->Name + , typename_to_strbuilder(self->UnderlyingType) + , body_to_strbuilder(self->Body) + ); + else if ( self->UnderlyingTypeMacro ) + strbuilder_append_fmt( result, "%S %SB\n{\n%SB\n}" + , self->Name + , code_to_strbuilder(self->UnderlyingTypeMacro) + , body_to_strbuilder(self->Body) ); - else result.append_fmt( "%S\n{\n%S\n}", ast->Name, ast->Body.to_string() ); + else strbuilder_append_fmt( result, "%S\n{\n%SB\n}", self->Name, body_to_strbuilder(self->Body) ); } - else result.append_fmt( "enum %S\n{\n%S\n}", ast->Name, ast->Body.to_string() ); + else strbuilder_append_fmt( result, "enum %S\n{\n%SB\n}", self->Name, body_to_strbuilder(self->Body) ); - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) - result.append(";\n"); + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) + strbuilder_append_str( result, txt(";\n")); } -void CodeEnum::to_string_fwd( String& result ) +void enum_to_strbuilder_fwd(CodeEnum self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - result.append_fmt( "enum %S : %S", ast->Name, ast->UnderlyingType.to_string() ); - - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) + if ( self->UnderlyingType ) + strbuilder_append_fmt( result, "enum %S : %SB", self->Name, typename_to_strbuilder(self->UnderlyingType) ); + else if (self->UnderlyingTypeMacro) { - if ( ast->InlineCmt ) - result.append_fmt("; %S", ast->InlineCmt->Content ); + log_fmt("IDENTIFIED A UNDERLYING ENUM MACRO"); + strbuilder_append_fmt( result, "enum %S %SB", self->Name, code_to_strbuilder(self->UnderlyingTypeMacro) ); + } + else + strbuilder_append_fmt( result, "enum %S", self->Name ); + + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) + { + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; %S", self->InlineCmt->Content ); else - result.append(";\n"); + strbuilder_append_str( result, txt(";\n")); } } -void CodeEnum::to_string_class_def( String& result ) +void enum_to_strbuilder_class_def(CodeEnum self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes || ast->UnderlyingType ) + if ( self->Attributes || self->UnderlyingType ) { - result.append( "enum class " ); + strbuilder_append_str( result, txt("enum class ") ); - if ( ast->Attributes ) + if ( self->Attributes ) { - result.append_fmt( "%S ", ast->Attributes.to_string() ); + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); } - if ( ast->UnderlyingType ) + if ( self->UnderlyingType ) { - result.append_fmt( "%S : %S\n{\n%S\n}", ast->Name, ast->UnderlyingType.to_string(), ast->Body.to_string() ); + strbuilder_append_fmt( result, "%S : %SB\n{\n%SB\n}", self->Name, typename_to_strbuilder(self->UnderlyingType), body_to_strbuilder(self->Body) ); } else { - result.append_fmt( "%S\n{\n%S\n}", ast->Name, ast->Body.to_string() ); + strbuilder_append_fmt( result, "%S\n{\n%SB\n}", self->Name, body_to_strbuilder(self->Body) ); } } else { - result.append_fmt( "enum class %S\n{\n%S\n}", ast->Body.to_string() ); + strbuilder_append_fmt( result, "enum %S\n{\n%SB\n}", self->Name, body_to_strbuilder(self->Body) ); } - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) - result.append(";\n"); + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) + strbuilder_append_str( result, txt(";\n")); } -void CodeEnum::to_string_class_fwd( String& result ) +void enum_to_strbuilder_class_fwd(CodeEnum self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - result.append( "enum class " ); + strbuilder_append_str( result, txt("enum class ") ); - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - result.append_fmt( "%S : %S", ast->Name, ast->UnderlyingType.to_string() ); + strbuilder_append_fmt( result, "%S : %SB", self->Name, typename_to_strbuilder(self->UnderlyingType) ); - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) { - if ( ast->InlineCmt ) - result.append_fmt("; %S", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; %S", self->InlineCmt->Content ); else - result.append(";\n"); + strbuilder_append_str( result, txt(";\n")); } } -String CodeExec::to_string() +StrBuilder exec_to_strbuilder(CodeExec exec) { - return ast->Content.duplicate( GlobalAllocator ); -} - -void CodeExtern::to_string( String& result ) -{ - if ( ast->Body ) - result.append_fmt( "extern \"%S\"\n{\n%S\n}\n", ast->Name, ast->Body.to_string() ); - else - result.append_fmt( "extern \"%S\"\n{}\n", ast->Name ); -} - -String CodeInclude::to_string() -{ - return String::fmt_buf( GlobalAllocator, "#include %S\n", ast->Content ); -} - -void CodeInclude::to_string( String& result ) -{ - result.append_fmt( "#include %S\n", ast->Content ); -} - -String CodeFriend::to_string() -{ - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + GEN_ASSERT(exec); + char* raw = ccast(char*, str_duplicate( exec->Content, _ctx->Allocator_Temp ).Ptr); + StrBuilder result = { raw }; return result; } -void CodeFriend::to_string( String& result ) +void extern_to_strbuilder(CodeExtern self, StrBuilder* result ) { - result.append_fmt( "friend %S", ast->Declaration->to_string() ); - - if ( ast->Declaration->Type != ECode::Function && result[ result.length() - 1 ] != ';' ) - { - result.append( ";" ); - } - - if ( ast->InlineCmt ) - result.append_fmt(" %S", ast->InlineCmt->Content ); + if ( self->Body ) + strbuilder_append_fmt( result, "extern \"%S\"\n{\n%SB\n}\n", self->Name, body_to_strbuilder(self->Body) ); else - result.append("\n"); + strbuilder_append_fmt( result, "extern \"%S\"\n{}\n", self->Name ); } -String CodeFn::to_string() +StrBuilder include_to_strbuilder(CodeInclude include) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + return strbuilder_fmt_buf( _ctx->Allocator_Temp, "#include %S\n", include->Content ); +} + +void include_to_strbuilder_ref( CodeInclude include, StrBuilder* result ) +{ + strbuilder_append_fmt( result, "#include %S\n", include->Content ); +} + +StrBuilder friend_to_strbuilder(CodeFriend self) +{ + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 256 ); + friend_to_strbuilder_ref( self, & result ); + return result; +} + +void friend_to_strbuilder_ref(CodeFriend self, StrBuilder* result ) +{ + strbuilder_append_fmt( result, "friend %SB", code_to_strbuilder(self->Declaration) ); + + if ( self->Declaration->Type != CT_Function && self->Declaration->Type != CT_Operator && (* result)[ strbuilder_length(* result) - 1 ] != ';' ) { - using namespace ECode; - case Function: - to_string_def( result ); + strbuilder_append_str( result, txt(";") ); + } + + if ( self->InlineCmt ) + strbuilder_append_fmt( result, " %S", self->InlineCmt->Content ); + else + strbuilder_append_str( result, txt("\n")); +} + +StrBuilder fn_to_strbuilder(CodeFn self) +{ + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 512 ); + switch ( self->Type ) + { + case CT_Function: + fn_to_strbuilder_def(self, & result ); break; - case Function_Fwd: - to_string_fwd( result ); + case CT_Function_Fwd: + fn_to_strbuilder_fwd(self, & result ); break; } return result; } -void CodeFn::to_string_def( String& result ) +void fn_to_strbuilder_def(CodeFn self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export" ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export") ); - if ( ast->Attributes ) - result.append_fmt( " %S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, " %SB ", attributes_to_strbuilder(self->Attributes) ); bool prefix_specs = false; - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ! ESpecifier::is_trailing( spec ) ) + if ( ! spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); prefix_specs = true; } } } - if ( ast->Attributes || prefix_specs ) - result.append( "\n" ); + if ( self->Attributes || prefix_specs ) + strbuilder_append_str( result, txt("\n") ); - if ( ast->ReturnType ) - result.append_fmt( "%S %S(", ast->ReturnType.to_string(), ast->Name ); + if ( self->ReturnType ) + strbuilder_append_fmt( result, "%SB %S(", typename_to_strbuilder(self->ReturnType), self->Name ); else - result.append_fmt( "%S(", ast->Name ); + strbuilder_append_fmt( result, "%S(", self->Name ); - if ( ast->Params ) - result.append_fmt( "%S)", ast->Params.to_string() ); + if ( self->Params ) + strbuilder_append_fmt( result, "%SB)", params_to_strbuilder(self->Params) ); else - result.append( ")" ); + strbuilder_append_str( result, txt(")") ); - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ESpecifier::is_trailing( spec ) ) + if ( spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); } } } - result.append_fmt( "\n{\n%S\n}\n", ast->Body.to_string() ); + strbuilder_append_fmt( result, "\n{\n%SB\n}\n", body_to_strbuilder(self->Body) ); } -void CodeFn::to_string_fwd( String& result ) +void fn_to_strbuilder_fwd(CodeFn self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); b32 prefix_specs = false; - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ! ESpecifier::is_trailing( spec ) || ! (spec != ESpecifier::Pure) ) + if ( ! spec_is_trailing( * spec ) || ! ( * spec != Spec_Pure) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); prefix_specs = true; } } } - if ( ast->Attributes || prefix_specs ) + if ( self->Attributes || prefix_specs ) { - result.append("\n" ); + strbuilder_append_str( result, txt("\n") ); } - if ( ast->ReturnType ) - result.append_fmt( "%S %S(", ast->ReturnType.to_string(), ast->Name ); + if ( self->ReturnType ) + strbuilder_append_fmt( result, "%SB %S(", typename_to_strbuilder(self->ReturnType), self->Name ); else - result.append_fmt( "%S(", ast->Name ); + strbuilder_append_fmt( result, "%S(", self->Name ); - if ( ast->Params ) - result.append_fmt( "%S)", ast->Params.to_string() ); + if ( self->Params ) + strbuilder_append_fmt( result, "%SB)", params_to_strbuilder(self->Params) ); else - result.append( ")" ); + strbuilder_append_str( result, txt(")") ); - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ESpecifier::is_trailing( spec ) ) + if ( spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); } } } - if ( ast->Specs && ast->Specs.has( ESpecifier::Pure ) >= 0 ) - result.append( " = 0;" ); - else if (ast->Body) - result.append_fmt( " = %S;", ast->Body.to_string() ); + if ( self->Specs && specifiers_has(self->Specs, Spec_Pure ) >= 0 ) + strbuilder_append_str( result, txt(" = 0;") ); + else if (self->Body) + strbuilder_append_fmt( result, " = %SB;", body_to_strbuilder(self->Body) ); - if ( ast->InlineCmt ) - result.append_fmt( "; %S", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; %S", self->InlineCmt->Content ); else - result.append( ";\n" ); + strbuilder_append_str( result, txt(";\n") ); } -String CodeModule::to_string() +StrBuilder module_to_strbuilder(CodeModule self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 64 ); + module_to_strbuilder_ref( self, & result ); return result; } -void CodeModule::to_string( String& result ) +void module_to_strbuilder_ref(CodeModule self, StrBuilder* result ) { - if (((u32(ModuleFlag::Export) & u32(ast->ModuleFlags)) == u32(ModuleFlag::Export))) - result.append("export "); + if (((scast(u32, ModuleFlag_Export) & scast(u32, self->ModuleFlags)) == scast(u32, ModuleFlag_Export))) + strbuilder_append_str( result, txt("export ")); - if (((u32(ModuleFlag::Import) & u32(ast->ModuleFlags)) == u32(ModuleFlag::Import))) - result.append("import "); + if (((scast(u32, ModuleFlag_Import) & scast(u32, self->ModuleFlags)) == scast(u32, ModuleFlag_Import))) + strbuilder_append_str( result, txt("import ")); - result.append_fmt( "%S;\n", ast->Name ); + strbuilder_append_fmt( result, "%S;\n", self->Name ); } -String CodeNS::to_string() +StrBuilder namespace_to_strbuilder(CodeNS self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 512 ); + namespace_to_strbuilder_ref( self, & result ); return result; } -void CodeNS::to_string( String& result ) +void namespace_to_strbuilder_ref(CodeNS self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - result.append_fmt( "namespace %S\n{\n%S\n}\n", ast->Name , ast->Body.to_string() ); + strbuilder_append_fmt( result, "namespace %S\n{\n%SB\n}\n", self->Name, body_to_strbuilder(self->Body) ); } -String CodeOperator::to_string() +StrBuilder code_op_to_strbuilder(CodeOperator self) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 512 ); + switch ( self->Type ) { - using namespace ECode; - case Operator: - case Operator_Member: - to_string_def( result ); + case CT_Operator: + case CT_Operator_Member: + code_op_to_strbuilder_def( self, & result ); break; - case Operator_Fwd: - case Operator_Member_Fwd: - to_string_fwd( result ); + case CT_Operator_Fwd: + case CT_Operator_Member_Fwd: + code_op_to_strbuilder_fwd( self, & result ); break; } return result; } -void CodeOperator::to_string_def( String& result ) +void code_op_to_strbuilder_def(CodeOperator self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ! ESpecifier::is_trailing( spec ) ) + if ( ! spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); } } } - if ( ast->Attributes || ast->Specs ) + if ( self->Attributes || self->Specs ) { - result.append("\n" ); + strbuilder_append_str( result, txt("\n") ); } - if ( ast->ReturnType ) - result.append_fmt( "%S %S (", ast->ReturnType.to_string(), ast->Name ); + if ( self->ReturnType ) + strbuilder_append_fmt( result, "%SB %S (", typename_to_strbuilder(self->ReturnType), self->Name ); - if ( ast->Params ) - result.append_fmt( "%S)", ast->Params.to_string() ); + if ( self->Params ) + strbuilder_append_fmt( result, "%SB)", params_to_strbuilder(self->Params) ); else - result.append( ")" ); + strbuilder_append_str( result, txt(")") ); - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ESpecifier::is_trailing( spec ) ) + if ( spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); } } } - result.append_fmt( "\n{\n%S\n}\n" - , ast->Body.to_string() + strbuilder_append_fmt( result, "\n{\n%SB\n}\n" + , body_to_strbuilder(self->Body) ); } -void CodeOperator::to_string_fwd( String& result ) +void code_op_to_strbuilder_fwd(CodeOperator self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes ) - result.append_fmt( "%S\n", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB\n", attributes_to_strbuilder(self->Attributes) ); - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ! ESpecifier::is_trailing( spec ) ) + if ( ! spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); } } } - if ( ast->Attributes || ast->Specs ) + if ( self->Attributes || self->Specs ) { - result.append("\n" ); + strbuilder_append_str( result, txt("\n") ); } - result.append_fmt( "%S %S (", ast->ReturnType.to_string(), ast->Name ); + strbuilder_append_fmt( result, "%SB %S (", typename_to_strbuilder(self->ReturnType), self->Name ); - if ( ast->Params ) - result.append_fmt( "%S)", ast->Params.to_string() ); + if ( self->Params ) + strbuilder_append_fmt( result, "%SB)", params_to_strbuilder(self->Params) ); else - result.append_fmt( ")" ); + strbuilder_append_fmt( result, ")" ); - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ESpecifier::is_trailing( spec ) ) + if ( spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); } } } - if ( ast->InlineCmt ) - result.append_fmt( "; %S", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; %S", self->InlineCmt->Content ); else - result.append( ";\n" ); + strbuilder_append_str( result, txt(";\n") ); } -String CodeOpCast::to_string() +StrBuilder opcast_to_strbuilder(CodeOpCast self) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 128 ); + switch ( self->Type ) { - using namespace ECode; - case Operator_Cast: - to_string_def( result ); + case CT_Operator_Cast: + opcast_to_strbuilder_def(self, & result ); break; - case Operator_Cast_Fwd: - to_string_fwd( result ); + case CT_Operator_Cast_Fwd: + opcast_to_strbuilder_fwd(self, & result ); break; } return result; } -void CodeOpCast::to_string_def( String& result ) +void opcast_to_strbuilder_def(CodeOpCast self, StrBuilder* result ) { - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ! ESpecifier::is_trailing( spec ) ) + if ( ! spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( "%*s ", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, "%*s ", spec_str.Len, spec_str.Ptr ); } } - if ( ast->Name && ast->Name.length() ) - result.append_fmt( "%Soperator %S()", ast->Name, ast->ValueType.to_string() ); + if ( self->Name.Ptr && self->Name.Len ) + strbuilder_append_fmt( result, "%S operator %SB()", self->Name, typename_to_strbuilder(self->ValueType) ); else - result.append_fmt( "operator %S()", ast->ValueType.to_string() ); + strbuilder_append_fmt( result, "operator %SB()", typename_to_strbuilder(self->ValueType) ); - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ESpecifier::is_trailing( spec ) ) + if ( spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %.*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %.*s", spec_str.Len, spec_str.Ptr ); } } - result.append_fmt( "\n{\n%S\n}\n", ast->Body.to_string() ); + strbuilder_append_fmt( result, "\n{\n%SB\n}\n", body_to_strbuilder(self->Body) ); return; } - if ( ast->Name && ast->Name.length() ) - result.append_fmt("%Soperator %S()\n{\n%S\n}\n", ast->Name, ast->ValueType.to_string(), ast->Body.to_string() ); + if ( self->Name.Ptr && self->Name.Len ) + strbuilder_append_fmt( result, "%S operator %SB()\n{\n%SB\n}\n", self->Name, typename_to_strbuilder(self->ValueType), body_to_strbuilder(self->Body) ); else - result.append_fmt("operator %S()\n{\n%S\n}\n", ast->ValueType.to_string(), ast->Body.to_string() ); + strbuilder_append_fmt( result, "operator %SB()\n{\n%SB\n}\n", typename_to_strbuilder(self->ValueType), body_to_strbuilder(self->Body) ); } -void CodeOpCast::to_string_fwd( String& result ) +void opcast_to_strbuilder_fwd(CodeOpCast self, StrBuilder* result ) { - if ( ast->Specs ) + if ( self->Specs ) { - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ! ESpecifier::is_trailing( spec ) ) + if ( ! spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( "%*s ", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, "%*s ", spec_str.Len, spec_str.Ptr ); } } - result.append_fmt( "operator %S()", ast->ValueType.to_string() ); + strbuilder_append_fmt( result, "operator %SB()", typename_to_strbuilder(self->ValueType) ); - for ( SpecifierT spec : ast->Specs ) + for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) ) { - if ( ESpecifier::is_trailing( spec ) ) + if ( spec_is_trailing( * spec ) ) { - StrC spec_str = ESpecifier::to_str( spec ); - result.append_fmt( " %*s", spec_str.Len, spec_str.Ptr ); + Str spec_str = spec_to_str( * spec ); + strbuilder_append_fmt( result, " %*s", spec_str.Len, spec_str.Ptr ); } } - if ( ast->InlineCmt ) - result.append_fmt( "; %S", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; %S", self->InlineCmt->Content ); else - result.append( ";\n" ); + strbuilder_append_str( result, txt(";\n") ); return; } - if ( ast->InlineCmt ) - result.append_fmt("operator %S(); %S", ast->ValueType.to_string() ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "operator %SB(); %SB", typename_to_strbuilder(self->ValueType) ); else - result.append_fmt("operator %S();\n", ast->ValueType.to_string() ); + strbuilder_append_fmt( result, "operator %SB();\n", typename_to_strbuilder(self->ValueType) ); } -String CodeParam::to_string() +StrBuilder params_to_strbuilder(CodeParams self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + GEN_ASSERT(self); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 128 ); + params_to_strbuilder_ref( self, & result ); return result; } -void CodeParam::to_string( String& result ) +void params_to_strbuilder_ref( CodeParams self, StrBuilder* result ) { - if ( ast->Macro ) + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( self->Macro ) { // Related to parsing: ( , ... ) - result.append( ast->Macro.ast->Content ); + strbuilder_append_str( result, self->Macro->Content ); // Could also be: ( , ... ) } - if ( ast->Name ) + if ( self->Name.Ptr && self->Name.Len ) { - if ( ast->ValueType.ast == nullptr ) - result.append_fmt( " %S", ast->Name ); + if ( self->ValueType == nullptr ) + strbuilder_append_fmt( result, " %S", self->Name ); else - result.append_fmt( " %S %S", ast->ValueType.to_string(), ast->Name ); + strbuilder_append_fmt( result, " %SB %S", typename_to_strbuilder(self->ValueType), self->Name ); } - else if ( ast->ValueType ) - result.append_fmt( " %S", ast->ValueType.to_string() ); + else if ( self->ValueType ) + strbuilder_append_fmt( result, " %SB", typename_to_strbuilder(self->ValueType) ); - if ( ast->PostNameMacro ) + if ( self->PostNameMacro ) { - result.append_fmt(" %S", ast->PostNameMacro.to_string() ); + strbuilder_append_fmt( result, " %SB", code_to_strbuilder(self->PostNameMacro) ); } - if ( ast->Value ) - result.append_fmt( " = %S", ast->Value.to_string() ); + if ( self->Value ) + strbuilder_append_fmt( result, " = %SB", code_to_strbuilder(self->Value) ); - if ( ast->NumEntries - 1 > 0 ) + if ( self->NumEntries - 1 > 0 ) { - for ( CodeParam param : ast->Next ) + for ( CodeParams param = begin_CodeParams(self->Next); param != end_CodeParams(self->Next); param = next_CodeParams(self->Next, param) ) { - result.append_fmt( ", %S", param.to_string() ); + strbuilder_append_fmt( result, ", %SB", params_to_strbuilder(param) ); } } } -String CodePreprocessCond::to_string() +StrBuilder preprocess_to_strbuilder(CodePreprocessCond self) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + GEN_ASSERT(self); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 256 ); + switch ( self->Type ) { - using namespace ECode; - case Preprocess_If: - to_string_if( result ); + case CT_Preprocess_If: + preprocess_to_strbuilder_if( self, & result ); break; - case Preprocess_IfDef: - to_string_ifdef( result ); + case CT_Preprocess_IfDef: + preprocess_to_strbuilder_ifdef( self, & result ); break; - case Preprocess_IfNotDef: - to_string_ifndef( result ); + case CT_Preprocess_IfNotDef: + preprocess_to_strbuilder_ifndef( self, & result ); break; - case Preprocess_ElIf: - to_string_elif( result ); + case CT_Preprocess_ElIf: + preprocess_to_strbuilder_elif( self, & result ); break; - case Preprocess_Else: - to_string_else( result ); + case CT_Preprocess_Else: + preprocess_to_strbuilder_else( self, & result ); break; - case Preprocess_EndIf: - to_string_endif( result ); + case CT_Preprocess_EndIf: + preprocess_to_strbuilder_endif( self, & result ); break; } return result; } -void CodePreprocessCond::to_string_if( String& result ) +void preprocess_to_strbuilder_if(CodePreprocessCond cond, StrBuilder* result ) { - result.append_fmt( "#if %S\n", ast->Content ); + GEN_ASSERT(cond); + strbuilder_append_fmt( result, "#if %S", cond->Content ); } -void CodePreprocessCond::to_string_ifdef( String& result ) +void preprocess_to_strbuilder_ifdef(CodePreprocessCond cond, StrBuilder* result ) { - result.append_fmt( "#ifdef %S\n", ast->Content ); + GEN_ASSERT(cond); + strbuilder_append_fmt( result, "#ifdef %S\n", cond->Content ); } -void CodePreprocessCond::to_string_ifndef( String& result ) +void preprocess_to_strbuilder_ifndef(CodePreprocessCond cond, StrBuilder* result ) { - result.append_fmt( "#ifndef %S\n", ast->Content ); + GEN_ASSERT(cond); + strbuilder_append_fmt( result, "#ifndef %S", cond->Content ); } -void CodePreprocessCond::to_string_elif( String& result ) +void preprocess_to_strbuilder_elif(CodePreprocessCond cond, StrBuilder* result ) { - result.append_fmt( "#elif %S\n", ast->Content ); + GEN_ASSERT(cond); + strbuilder_append_fmt( result, "#elif %S\n", cond->Content ); } -void CodePreprocessCond::to_string_else( String& result ) +void preprocess_to_strbuilder_else(CodePreprocessCond cond, StrBuilder* result ) { - result.append_fmt( "#else\n" ); + GEN_ASSERT(cond); + strbuilder_append_str( result, txt("#else\n") ); } -void CodePreprocessCond::to_string_endif( String& result ) +void preprocess_to_strbuilder_endif(CodePreprocessCond cond, StrBuilder* result ) { - result.append_fmt( "#endif\n" ); + GEN_ASSERT(cond); + strbuilder_append_str( result, txt("#endif\n") ); } -String CodePragma::to_string() +StrBuilder pragma_to_strbuilder(CodePragma self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + GEN_ASSERT(self); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 256 ); + pragma_to_strbuilder_ref( self, & result ); return result; } -void CodePragma::to_string( String& result ) +void pragma_to_strbuilder_ref(CodePragma self, StrBuilder* result ) { - result.append_fmt( "#pragma %S\n", ast->Content ); + strbuilder_append_fmt( result, "#pragma %S\n", self->Content ); } -String CodeSpecifiers::to_string() +StrBuilder specifiers_to_strbuilder(CodeSpecifiers self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 64 ); + specifiers_to_strbuilder_ref( self, & result ); return result; } -void CodeSpecifiers::to_string( String& result ) +void specifiers_to_strbuilder_ref( CodeSpecifiers self, StrBuilder* result ) { + GEN_ASSERT(self); + GEN_ASSERT(result); s32 idx = 0; - s32 left = ast->NumEntries; + s32 left = self->NumEntries; while ( left-- ) { - StrC spec = ESpecifier::to_str( ast->ArrSpecs[idx] ); - result.append_fmt( "%.*s ", spec.Len, spec.Ptr ); + Str spec = spec_to_str( self->ArrSpecs[idx] ); + strbuilder_append_fmt( result, "%.*s ", spec.Len, spec.Ptr ); idx++; } } -String CodeStruct::to_string() +StrBuilder struct_to_strbuilder(CodeStruct self) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + GEN_ASSERT(self); + GEN_ASSERT(self); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 512 ); + switch ( self->Type ) { - using namespace ECode; - case Struct: - to_string_def( result ); + case CT_Struct: + struct_to_strbuilder_def( self, & result ); break; - case Struct_Fwd: - to_string_fwd( result ); + case CT_Struct_Fwd: + struct_to_strbuilder_fwd( self, & result ); break; } return result; } -void CodeStruct::to_string_def( String& result ) +void struct_to_strbuilder_def( CodeStruct self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - result.append( "struct " ); + strbuilder_append_str( result, txt("struct ") ); - if ( ast->Attributes ) + if ( self->Attributes ) { - result.append_fmt( "%S ", ast->Attributes.to_string() ); + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); } - if ( ast->ParentType ) + if ( self->ParentType ) { - char const* access_level = to_str( ast->ParentAccess ); + Str access_level = access_spec_to_str( self->ParentAccess ); - result.append_fmt( "%S : %s %S", ast->Name, access_level, ast->ParentType.to_string() ); + strbuilder_append_fmt( result, "%S : %S %SB", self->Name, access_level, typename_to_strbuilder(self->ParentType) ); - CodeType interface = ast->ParentType->Next->cast< CodeType >(); + CodeTypename interface = cast(CodeTypename, self->ParentType->Next); if ( interface ) - result.append( "\n" ); + strbuilder_append_str( result, txt("\n") ); while ( interface ) { - result.append_fmt( ", %S", interface.to_string() ); - interface = interface->Next ? interface->Next->cast< CodeType >() : CodeType { nullptr }; + strbuilder_append_fmt( result, ", %SB", typename_to_strbuilder(interface) ); + interface = interface->Next ? cast( CodeTypename, interface->Next) : NullCode; } } - else if ( ast->Name ) + else if ( self->Name.Len ) { - result.append( ast->Name ); + strbuilder_append_str( result, self->Name ); } - if ( ast->InlineCmt ) + if ( self->InlineCmt ) { - result.append_fmt( " // %S", ast->InlineCmt->Content ); + strbuilder_append_fmt( result, " // %S", self->InlineCmt->Content ); } - result.append_fmt( "\n{\n%S\n}", ast->Body.to_string() ); + strbuilder_append_fmt( result, "\n{\n%SB\n}", body_to_strbuilder(self->Body) ); - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) - result.append(";\n"); + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) + strbuilder_append_str( result, txt(";\n")); } -void CodeStruct::to_string_fwd( String& result ) +void struct_to_strbuilder_fwd( CodeStruct self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes ) - result.append_fmt( "struct %S %S", ast->Attributes.to_string(), ast->Name ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "struct %SB %S", attributes_to_strbuilder(self->Attributes), self->Name ); - else result.append_fmt( "struct %S", ast->Name ); + else strbuilder_append_fmt( result, "struct %S", self->Name ); - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) { - if ( ast->InlineCmt ) - result.append_fmt("; %S", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; %S", self->InlineCmt->Content ); else - result.append(";\n"); + strbuilder_append_str( result, txt( ";\n") ); } } -String CodeTemplate::to_string() +StrBuilder template_to_strbuilder(CodeTemplate self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + GEN_ASSERT(self); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 1024 ); + template_to_strbuilder_ref( self, & result ); return result; } -void CodeTemplate::to_string( String& result ) +void template_to_strbuilder_ref(CodeTemplate self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Params ) - result.append_fmt( "template< %S >\n%S", ast->Params.to_string(), ast->Declaration.to_string() ); + if ( self->Params ) + strbuilder_append_fmt( result, "template< %SB >\n%SB", params_to_strbuilder(self->Params), code_to_strbuilder(self->Declaration) ); else - result.append_fmt( "template<>\n%S", ast->Declaration.to_string() ); + strbuilder_append_fmt( result, "template<>\n%SB", code_to_strbuilder(self->Declaration) ); } -String CodeTypedef::to_string() +StrBuilder typedef_to_strbuilder(CodeTypedef self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 128 ); + typedef_to_strbuilder_ref( self, & result ); return result; } -void CodeTypedef::to_string( String& result ) +void typedef_to_strbuilder_ref(CodeTypedef self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - result.append( "typedef "); + strbuilder_append_str( result, txt("typedef ")); // Determines if the typedef is a function typename - if ( ast->UnderlyingType->ReturnType ) - result.append( ast->UnderlyingType.to_string() ); + if ( self->UnderlyingType->ReturnType ) + strbuilder_append_string( result, code_to_strbuilder(self->UnderlyingType) ); else - result.append_fmt( "%S %S", ast->UnderlyingType.to_string(), ast->Name ); + strbuilder_append_fmt( result, "%SB %S", code_to_strbuilder(self->UnderlyingType), self->Name ); - if ( ast->UnderlyingType->Type == ECode::Typename && ast->UnderlyingType->ArrExpr ) + if ( self->UnderlyingType->Type == CT_Typename && self->UnderlyingType->ArrExpr ) { - result.append_fmt( "[ %S ];", ast->UnderlyingType->ArrExpr->to_string() ); + strbuilder_append_fmt( result, "[ %SB ];", code_to_strbuilder(self->UnderlyingType->ArrExpr) ); - AST* next_arr_expr = ast->UnderlyingType->ArrExpr->Next; + Code next_arr_expr = self->UnderlyingType->ArrExpr->Next; while ( next_arr_expr ) { - result.append_fmt( "[ %S ];", next_arr_expr->to_string() ); + strbuilder_append_fmt( result, "[ %SB ];", code_to_strbuilder(next_arr_expr) ); next_arr_expr = next_arr_expr->Next; } } else { - result.append( ";" ); + strbuilder_append_str( result, txt(";") ); } - if ( ast->InlineCmt ) - result.append_fmt(" %S", ast->InlineCmt->Content); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, " %S", self->InlineCmt->Content); else - result.append("\n"); + strbuilder_append_str( result, txt("\n")); } -String CodeType::to_string() +StrBuilder typename_to_strbuilder(CodeTypename self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + StrBuilder result = strbuilder_make_str( _ctx->Allocator_Temp, txt("") ); + typename_to_strbuilder_ref( self, & result ); return result; } -void CodeType::to_string( String& result ) +void typename_to_strbuilder_ref(CodeTypename self, StrBuilder* result ) { #if defined(GEN_USE_NEW_TYPENAME_PARSING) - if ( ast->ReturnType && ast->Params ) + if ( self->ReturnType && self->Params ) { - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); else { - if ( ast->Specs ) - result.append_fmt( "%S ( %S ) ( %S ) %S", ast->ReturnType.to_string(), ast->Name, ast->Params.to_string(), ast->Specs.to_string() ); + if ( self->Specs ) + strbuilder_append_fmt( result, "%SB ( %S ) ( %SB ) %SB", typename_to_strbuilder(self->ReturnType), self->Name, params_to_strbuilder(self->Params), specifiers_to_strbuilder(self->Specs) ); else - result.append_fmt( "%S ( %S ) ( %S )", ast->ReturnType.to_string(), ast->Name, ast->Params.to_string() ); + strbuilder_append_fmt( result, "%SB ( %S ) ( %SB )", typename_to_strbuilder(self->ReturnType), self->Name, params_to_strbuilder(self->Params) ); } break; } #else - if ( ast->ReturnType && ast->Params ) + if ( self->ReturnType && self->Params ) { - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); else { - if ( ast->Specs ) - result.append_fmt( "%S %S ( %S ) %S", ast->ReturnType.to_string(), ast->Name, ast->Params.to_string(), ast->Specs.to_string() ); + if ( self->Specs ) + strbuilder_append_fmt( result, "%SB %S ( %SB ) %SB", typename_to_strbuilder(self->ReturnType), self->Name, params_to_strbuilder(self->Params), specifiers_to_strbuilder(self->Specs) ); else - result.append_fmt( "%S %S ( %S )", ast->ReturnType.to_string(), ast->Name, ast->Params.to_string() ); + strbuilder_append_fmt( result, "%SB %S ( %SB )", typename_to_strbuilder(self->ReturnType), self->Name, params_to_strbuilder(self->Params) ); } return; } #endif - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - if ( ast->Specs ) - result.append_fmt( "%S %S", ast->Name, ast->Specs.to_string() ); + switch ( self->TypeTag ) + { + case Tag_Class : strbuilder_append_str( result, txt("class ")); break; + case Tag_Enum : strbuilder_append_str( result, txt("enum ")); break; + case Tag_Struct : strbuilder_append_str( result, txt("struct ")); break; + case Tag_Union : strbuilder_append_str( result, txt("union ")); break; + default: + break; + } + + if ( self->Specs ) + strbuilder_append_fmt( result, "%S %SB", self->Name, specifiers_to_strbuilder(self->Specs) ); else - result.append_fmt( "%S", ast->Name ); + strbuilder_append_fmt( result, "%S", self->Name ); - if ( ast->IsParamPack ) - result.append("..."); + if ( self->IsParamPack ) + strbuilder_append_str( result, txt("...")); } -String CodeUnion::to_string() +StrBuilder union_to_strbuilder(CodeUnion self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 512 ); + switch ( self->Type ) + { + case CT_Union: + union_to_strbuilder_def( self, & result ); + break; + case CT_Union_Fwd: + union_to_strbuilder_fwd( self, & result ); + break; + } return result; } -void CodeUnion::to_string( String& result ) +void union_to_strbuilder_def(CodeUnion self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - result.append( "union " ); + strbuilder_append_str( result, txt("union ") ); - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - if ( ast->Name ) + if ( self->Name.Len ) { - result.append_fmt( "%S\n{\n%S\n}" - , ast->Name - , ast->Body.to_string() + strbuilder_append_fmt( result, "%S\n{\n%SB\n}" + , self->Name + , body_to_strbuilder(self->Body) ); } else { // Anonymous union - result.append_fmt( "\n{\n%S\n}" - , ast->Body.to_string() + strbuilder_append_fmt( result, "\n{\n%SB\n}" + , body_to_strbuilder(self->Body) ); } - if ( ast->Parent.ast == nullptr || ( ast->Parent->Type != ECode::Typedef && ast->Parent->Type != ECode::Variable ) ) - result.append(";\n"); + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) + strbuilder_append_str( result, txt(";\n")); } -String CodeUsing::to_string() +void union_to_strbuilder_fwd(CodeUnion self, StrBuilder* result ) { - String result = String::make( GlobalAllocator, "" ); - switch ( ast->Type ) + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); + + strbuilder_append_str( result, txt("union ") ); + + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); + + if ( self->Name.Len ) { - using namespace ECode; - case Using: - to_string( result ); + strbuilder_append_fmt( result, "%S", self->Name); + } + + if ( self->Parent == nullptr || ( self->Parent->Type != CT_Typedef && self->Parent->Type != CT_Variable ) ) + strbuilder_append_str( result, txt(";\n")); +} + +StrBuilder using_to_strbuilder(CodeUsing self) +{ + GEN_ASSERT(self); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 128 ); + switch ( self->Type ) + { + case CT_Using: + using_to_strbuilder_ref( self, & result ); break; - case Using_Namespace: - to_string_ns( result ); + case CT_Using_Namespace: + using_to_strbuilder_ns( self, & result ); break; } return result; } -void CodeUsing::to_string( String& result ) +void using_to_strbuilder_ref(CodeUsing self, StrBuilder* result ) { - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Attributes.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - if ( ast->UnderlyingType ) + if ( self->UnderlyingType ) { - result.append_fmt( "using %S = %S", ast->Name, ast->UnderlyingType.to_string() ); + strbuilder_append_fmt( result, "using %S = %SB", self->Name, typename_to_strbuilder(self->UnderlyingType) ); - if ( ast->UnderlyingType->ArrExpr ) + if ( self->UnderlyingType->ArrExpr ) { - result.append_fmt( "[ %S ]", ast->UnderlyingType->ArrExpr.to_string() ); + strbuilder_append_fmt( result, "[ %SB ]", code_to_strbuilder(self->UnderlyingType->ArrExpr) ); - AST* next_arr_expr = ast->UnderlyingType->ArrExpr->Next; + Code next_arr_expr = self->UnderlyingType->ArrExpr->Next; while ( next_arr_expr ) { - result.append_fmt( "[ %S ]", next_arr_expr->to_string() ); + strbuilder_append_fmt( result, "[ %SB ]", code_to_strbuilder(next_arr_expr) ); next_arr_expr = next_arr_expr->Next; } } - result.append( ";" ); + strbuilder_append_str( result, txt(";") ); } else - result.append_fmt( "using %S;", ast->Name ); + strbuilder_append_fmt( result, "using %S;", self->Name ); - if ( ast->InlineCmt ) - result.append_fmt(" %S\n", ast->InlineCmt->Content ); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, " %S\n", self->InlineCmt->Content ); else - result.append("\n"); + strbuilder_append_str( result, txt("\n")); } -void CodeUsing::to_string_ns( String& result ) +inline +void using_to_strbuilder_ns(CodeUsing self, StrBuilder* result ) { - if ( ast->InlineCmt ) - result.append_fmt( "using namespace $S; %S", ast->Name, ast->InlineCmt->Content ); + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "using namespace $SC; %S", self->Name, self->InlineCmt->Content ); else - result.append_fmt( "using namespace %s;\n", ast->Name ); + strbuilder_append_fmt( result, "using namespace %S;\n", self->Name ); } -String CodeVar::to_string() +inline +StrBuilder var_to_strbuilder(CodeVar self) { - String result = String::make( GlobalAllocator, "" ); - to_string( result ); + GEN_ASSERT(self); + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, 256 ); + var_to_strbuilder_ref( self, & result ); return result; } -void CodeVar::to_string( String& result ) +neverinline +void var_to_strbuilder_ref(CodeVar self, StrBuilder* result ) { - if ( ast->Parent && ast->Parent->Type == ECode::Variable ) + GEN_ASSERT(self); + GEN_ASSERT(result); + if ( self->Parent && self->Parent->Type == CT_Variable ) { // Its a comma-separated variable ( a NextVar ) - if ( ast->Specs ) - result.append_fmt( "%S ", ast->Specs.to_string() ); + if ( self->Specs ) + strbuilder_append_fmt( result, "%SB ", specifiers_to_strbuilder(self->Specs) ); - result.append( ast->Name ); + strbuilder_append_str( result, self->Name ); - if ( ast->ValueType->ArrExpr ) + if ( self->ValueType && self->ValueType->ArrExpr ) { - result.append_fmt( "[ %S ]", ast->ValueType->ArrExpr.to_string() ); + strbuilder_append_fmt( result, "[ %SB ]", code_to_strbuilder(self->ValueType->ArrExpr) ); - AST* next_arr_expr = ast->ValueType->ArrExpr->Next; + Code next_arr_expr = self->ValueType->ArrExpr->Next; while ( next_arr_expr ) { - result.append_fmt( "[ %S ]", next_arr_expr->to_string() ); + strbuilder_append_fmt( result, "[ %SB ]", code_to_strbuilder(next_arr_expr) ); next_arr_expr = next_arr_expr->Next; } } - if ( ast->Value ) + if ( self->Value ) { - if ( ast->VarConstructorInit ) - result.append_fmt( "( %S ", ast->Value.to_string() ); + if ( self->VarParenthesizedInit ) + strbuilder_append_fmt( result, "( %SB ", code_to_strbuilder(self->Value) ); else - result.append_fmt( " = %S", ast->Value.to_string() ); + strbuilder_append_fmt( result, " = %SB", code_to_strbuilder(self->Value) ); } // Keep the chain going... - if ( ast->NextVar ) - result.append_fmt( ", %S", ast->NextVar.to_string() ); + if ( self->NextVar ) + strbuilder_append_fmt( result, ", %SB", var_to_strbuilder(self->NextVar) ); - if ( ast->VarConstructorInit ) - result.append( " )"); + if ( self->VarParenthesizedInit ) + strbuilder_append_str( result, txt(" )")); return; } - if ( bitfield_is_equal( u32, ast->ModuleFlags, ModuleFlag::Export )) - result.append( "export " ); + if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export )) + strbuilder_append_str( result, txt("export ") ); - if ( ast->Attributes || ast->Specs ) + if ( self->Attributes || self->Specs ) { - if ( ast->Attributes ) - result.append_fmt( "%S ", ast->Specs.to_string() ); + if ( self->Attributes ) + strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) ); - if ( ast->Specs ) - result.append_fmt( "%S\n", ast->Specs.to_string() ); + if ( self->Specs ) + strbuilder_append_fmt( result, "%SB\n", specifiers_to_strbuilder(self->Specs) ); - result.append_fmt( "%S %S", ast->ValueType.to_string(), ast->Name ); + strbuilder_append_fmt( result, "%SB %S", typename_to_strbuilder(self->ValueType), self->Name ); - if ( ast->ValueType->ArrExpr ) + if ( self->ValueType && self->ValueType->ArrExpr ) { - result.append_fmt( "[ %S ]", ast->ValueType->ArrExpr.to_string() ); + strbuilder_append_fmt( result, "[ %SB ]", code_to_strbuilder(self->ValueType->ArrExpr) ); - AST* next_arr_expr = ast->ValueType->ArrExpr->Next; + Code next_arr_expr = self->ValueType->ArrExpr->Next; while ( next_arr_expr ) { - result.append_fmt( "[ %S ]", next_arr_expr->to_string() ); + strbuilder_append_fmt( result, "[ %SB ]", code_to_strbuilder(next_arr_expr) ); next_arr_expr = next_arr_expr->Next; } } - if ( ast->BitfieldSize ) - result.append_fmt( " : %S", ast->BitfieldSize.to_string() ); + if ( self->BitfieldSize ) + strbuilder_append_fmt( result, " : %SB", code_to_strbuilder(self->BitfieldSize) ); - if ( ast->Value ) + if ( self->Value ) { - if ( ast->VarConstructorInit ) - result.append_fmt( "( %S ", ast->Value.to_string() ); + if ( self->VarParenthesizedInit ) + strbuilder_append_fmt( result, "( %SB ", code_to_strbuilder(self->Value) ); else - result.append_fmt( " = %S", ast->Value.to_string() ); + strbuilder_append_fmt( result, " = %SB", code_to_strbuilder(self->Value) ); } - if ( ast->NextVar ) - result.append_fmt( ", %S", ast->NextVar.to_string() ); + if ( self->NextVar ) + strbuilder_append_fmt( result, ", %SB", var_to_strbuilder(self->NextVar) ); - if ( ast->VarConstructorInit ) - result.append( " )"); + if ( self->VarParenthesizedInit ) + strbuilder_append_str( result, txt(" )")); - if ( ast->InlineCmt ) - result.append_fmt("; %S", ast->InlineCmt->Content); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, "; %S", self->InlineCmt->Content); else - result.append( ";\n" ); + strbuilder_append_str( result, txt(";\n") ); return; } - if ( ast->BitfieldSize ) - result.append_fmt( "%S %S : %S", ast->ValueType.to_string(), ast->Name, ast->BitfieldSize.to_string() ); + if ( self->BitfieldSize ) + strbuilder_append_fmt( result, "%SB %S : %SB", typename_to_strbuilder(self->ValueType), self->Name, code_to_strbuilder(self->BitfieldSize) ); - else if ( ast->ValueType->ArrExpr ) + else if ( self->ValueType && self->ValueType->ArrExpr ) { - result.append_fmt( "%S %S[ %S ]", ast->ValueType.to_string(), ast->Name, ast->ValueType->ArrExpr.to_string() ); + strbuilder_append_fmt( result, "%SB %S[ %SB ]", typename_to_strbuilder(self->ValueType), self->Name, code_to_strbuilder(self->ValueType->ArrExpr) ); - AST* next_arr_expr = ast->ValueType->ArrExpr->Next; + Code next_arr_expr = self->ValueType->ArrExpr->Next; while ( next_arr_expr ) { - result.append_fmt( "[ %S ]", next_arr_expr->to_string() ); + strbuilder_append_fmt( result, "[ %SB ]", code_to_strbuilder(next_arr_expr) ); next_arr_expr = next_arr_expr->Next; } } else - result.append_fmt( "%S %S", ast->ValueType.to_string(), ast->Name ); + strbuilder_append_fmt( result, "%SB %S", typename_to_strbuilder(self->ValueType), self->Name ); - if ( ast->Value ) + if ( self->Value ) { - if ( ast->VarConstructorInit ) - result.append_fmt( "( %S ", ast->Value.to_string() ); + if ( self->VarParenthesizedInit ) + strbuilder_append_fmt( result, "( %SB ", code_to_strbuilder(self->Value) ); else - result.append_fmt( " = %S", ast->Value.to_string() ); + strbuilder_append_fmt( result, " = %SB", code_to_strbuilder(self->Value) ); } - if ( ast->NextVar ) - result.append_fmt( ", %S", ast->NextVar.to_string() ); + if ( self->NextVar ) + strbuilder_append_fmt( result, ", %SB", var_to_strbuilder( self->NextVar) ); - if ( ast->VarConstructorInit ) - result.append( " )"); + if ( self->VarParenthesizedInit ) + strbuilder_append_str( result, txt(" )")); - result.append( ";" ); + strbuilder_append_str( result, txt(";") ); - if ( ast->InlineCmt ) - result.append_fmt(" %S", ast->InlineCmt->Content); + if ( self->InlineCmt ) + strbuilder_append_fmt( result, " %S", self->InlineCmt->Content); else - result.append("\n"); + strbuilder_append_str( result, txt("\n")); } #pragma endregion AST #pragma region Interface -namespace parser { -internal void init(); -internal void deinit(); -} +internal void parser_init(); +internal void parser_deinit(); internal -void* Global_Allocator_Proc( void* allocator_data, AllocType type, ssize size, ssize alignment, void* old_memory, ssize old_size, u64 flags ) +void* fallback_allocator_proc( void* allocator_data, AllocType type, ssize size, ssize alignment, void* old_memory, ssize old_size, u64 flags ) { - Arena* last = & Global_AllocatorBuckets.back(); + GEN_ASSERT(_ctx); + GEN_ASSERT(_ctx->Fallback_AllocatorBuckets); + Arena* last = array_back(_ctx->Fallback_AllocatorBuckets); switch ( type ) { @@ -2864,18 +3044,18 @@ void* Global_Allocator_Proc( void* allocator_data, AllocType type, ssize size, s { if ( ( last->TotalUsed + size ) > last->TotalSize ) { - Arena bucket = Arena::init_from_allocator( heap(), Global_BucketSize ); + Arena bucket = arena_init_from_allocator( heap(), _ctx->InitSize_Fallback_Allocator_Bucket_Size ); if ( bucket.PhysicalStart == nullptr ) - GEN_FATAL( "Failed to create bucket for Global_AllocatorBuckets"); + GEN_FATAL( "Failed to create bucket for Fallback_AllocatorBuckets"); - if ( ! Global_AllocatorBuckets.append( bucket ) ) - GEN_FATAL( "Failed to append bucket to Global_AllocatorBuckets"); + if ( ! array_append( _ctx->Fallback_AllocatorBuckets, bucket ) ) + GEN_FATAL( "Failed to append bucket to Fallback_AllocatorBuckets"); - last = & Global_AllocatorBuckets.back(); + last = array_back(_ctx->Fallback_AllocatorBuckets); } - return alloc_align( * last, size, alignment ); + return alloc_align( arena_allocator_info(last), size, alignment ); } case EAllocation_FREE: { @@ -2891,15 +3071,15 @@ void* Global_Allocator_Proc( void* allocator_data, AllocType type, ssize size, s { if ( last->TotalUsed + size > last->TotalSize ) { - Arena bucket = Arena::init_from_allocator( heap(), Global_BucketSize ); + Arena bucket = arena_init_from_allocator( heap(), _ctx->InitSize_Fallback_Allocator_Bucket_Size ); if ( bucket.PhysicalStart == nullptr ) - GEN_FATAL( "Failed to create bucket for Global_AllocatorBuckets"); + GEN_FATAL( "Failed to create bucket for Fallback_AllocatorBuckets"); - if ( ! Global_AllocatorBuckets.append( bucket ) ) - GEN_FATAL( "Failed to append bucket to Global_AllocatorBuckets"); + if ( ! array_append( _ctx->Fallback_AllocatorBuckets, bucket ) ) + GEN_FATAL( "Failed to append bucket to Fallback_AllocatorBuckets"); - last = & Global_AllocatorBuckets.back(); + last = array_back( _ctx->Fallback_AllocatorBuckets); } void* result = alloc_align( last->Backing, size, alignment ); @@ -2919,451 +3099,502 @@ void* Global_Allocator_Proc( void* allocator_data, AllocType type, ssize size, s internal void define_constants() { - Code::Global = make_code(); - Code::Global->Name = get_cached_string( txt("Global Code") ); - Code::Global->Content = Code::Global->Name; + // We only initalize these if there is no base context. + if ( context_counter > 0 ) + return; - Code::Invalid = make_code(); - Code::Invalid.set_global(); + Code_Global = make_code(); + Code_Global->Name = cache_str( txt("Global Code") ); + Code_Global->Content = Code_Global->Name; - t_empty = (CodeType) make_code(); - t_empty->Type = ECode::Typename; - t_empty->Name = get_cached_string( txt("") ); - t_empty.set_global(); + Code_Invalid = make_code(); + code_set_global(Code_Invalid); + + t_empty = (CodeTypename) make_code(); + t_empty->Type = CT_Typename; + t_empty->Name = cache_str( txt("") ); + code_set_global(cast(Code, t_empty)); access_private = make_code(); - access_private->Type = ECode::Access_Private; - access_private->Name = get_cached_string( txt("private:\n") ); - access_private.set_global(); + access_private->Type = CT_Access_Private; + access_private->Name = cache_str( txt("private:\n") ); + code_set_global(cast(Code, access_private)); access_protected = make_code(); - access_protected->Type = ECode::Access_Protected; - access_protected->Name = get_cached_string( txt("protected:\n") ); - access_protected.set_global(); + access_protected->Type = CT_Access_Protected; + access_protected->Name = cache_str( txt("protected:\n") ); + code_set_global(access_protected); access_public = make_code(); - access_public->Type = ECode::Access_Public; - access_public->Name = get_cached_string( txt("public:\n") ); - access_public.set_global(); + access_public->Type = CT_Access_Public; + access_public->Name = cache_str( txt("public:\n") ); + code_set_global(access_public); - attrib_api_export = def_attributes( code(GEN_API_Export_Code)); - attrib_api_export.set_global(); + Str api_export_str = code(GEN_API_Export_Code); + attrib_api_export = def_attributes( api_export_str ); + code_set_global(cast(Code, attrib_api_export)); - attrib_api_import = def_attributes( code(GEN_API_Import_Code)); - attrib_api_import.set_global(); + Str api_import_str = code(GEN_API_Import_Code); + attrib_api_import = def_attributes( api_import_str ); + code_set_global(cast(Code, attrib_api_import)); module_global_fragment = make_code(); - module_global_fragment->Type = ECode::Untyped; - module_global_fragment->Name = get_cached_string( txt("module;") ); + module_global_fragment->Type = CT_Untyped; + module_global_fragment->Name = cache_str( txt("module;") ); module_global_fragment->Content = module_global_fragment->Name; - module_global_fragment.set_global(); + code_set_global(cast(Code, module_global_fragment)); module_private_fragment = make_code(); - module_private_fragment->Type = ECode::Untyped; - module_private_fragment->Name = get_cached_string( txt("module : private;") ); + module_private_fragment->Type = CT_Untyped; + module_private_fragment->Name = cache_str( txt("module : private;") ); module_private_fragment->Content = module_private_fragment->Name; - module_private_fragment.set_global(); + code_set_global(cast(Code, module_private_fragment)); fmt_newline = make_code(); - fmt_newline->Type = ECode::NewLine; - fmt_newline.set_global(); + fmt_newline->Type = CT_NewLine; + code_set_global((Code)fmt_newline); pragma_once = (CodePragma) make_code(); - pragma_once->Type = ECode::Preprocess_Pragma; - pragma_once->Name = get_cached_string( txt("once") ); + pragma_once->Type = CT_Preprocess_Pragma; + pragma_once->Name = cache_str( txt("once") ); pragma_once->Content = pragma_once->Name; - pragma_once.set_global(); + code_set_global((Code)pragma_once); - param_varadic = (CodeType) make_code(); - param_varadic->Type = ECode::Parameters; - param_varadic->Name = get_cached_string( txt("...") ); + param_varadic = (CodeParams) make_code(); + param_varadic->Type = CT_Parameters; + param_varadic->Name = cache_str( txt("...") ); param_varadic->ValueType = t_empty; - param_varadic.set_global(); + code_set_global((Code)param_varadic); preprocess_else = (CodePreprocessCond) make_code(); - preprocess_else->Type = ECode::Preprocess_Else; - preprocess_else.set_global(); + preprocess_else->Type = CT_Preprocess_Else; + code_set_global((Code)preprocess_else); preprocess_endif = (CodePreprocessCond) make_code(); - preprocess_endif->Type = ECode::Preprocess_EndIf; - preprocess_endif.set_global(); + preprocess_endif->Type = CT_Preprocess_EndIf; + code_set_global((Code)preprocess_endif); -# define def_constant_code_type( Type_ ) \ - t_##Type_ = def_type( name(Type_) ); \ - t_##Type_.set_global(); - - def_constant_code_type( auto ); - def_constant_code_type( void ); - def_constant_code_type( int ); - def_constant_code_type( bool ); - def_constant_code_type( char ); - def_constant_code_type( wchar_t ); - def_constant_code_type( class ); - def_constant_code_type( typename ); + Str auto_str = txt("auto"); t_auto = def_type( auto_str ); code_set_global( t_auto ); + Str void_str = txt("void"); t_void = def_type( void_str ); code_set_global( t_void ); + Str int_str = txt("int"); t_int = def_type( int_str ); code_set_global( t_int ); + Str bool_str = txt("bool"); t_bool = def_type( bool_str ); code_set_global( t_bool ); + Str char_str = txt("char"); t_char = def_type( char_str ); code_set_global( t_char ); + Str wchar_str = txt("wchar_t"); t_wchar_t = def_type( wchar_str ); code_set_global( t_wchar_t ); + Str class_str = txt("class"); t_class = def_type( class_str ); code_set_global( t_class ); + Str typename_str = txt("typename"); t_typename = def_type( typename_str ); code_set_global( t_typename ); #ifdef GEN_DEFINE_LIBRARY_CODE_CONSTANTS - t_b32 = def_type( name(b32) ); + t_b32 = def_type( name(b32) ); code_set_global( t_b32 ); - def_constant_code_type( s8 ); - def_constant_code_type( s16 ); - def_constant_code_type( s32 ); - def_constant_code_type( s64 ); + Str s8_str = txt("s8"); t_s8 = def_type( s8_str ); code_set_global( t_s8 ); + Str s16_str = txt("s16"); t_s16 = def_type( s16_str ); code_set_global( t_s16 ); + Str s32_str = txt("s32"); t_s32 = def_type( s32_str ); code_set_global( t_s32 ); + Str s64_str = txt("s64"); t_s64 = def_type( s64_str ); code_set_global( t_s64 ); - def_constant_code_type( u8 ); - def_constant_code_type( u16 ); - def_constant_code_type( u32 ); - def_constant_code_type( u64 ); + Str u8_str = txt("u8"); t_u8 = def_type( u8_str ); code_set_global( t_u8 ); + Str u16_str = txt("u16"); t_u16 = def_type( u16_str ); code_set_global( t_u16 ); + Str u32_str = txt("u32"); t_u32 = def_type( u32_str ); code_set_global( t_u32 ); + Str u64_str = txt("u64"); t_u64 = def_type( u64_str ); code_set_global( t_u64 ); - def_constant_code_type( ssize ); - def_constant_code_type( usize ); + Str ssize_str = txt("ssize"); t_ssize = def_type( ssize_str ); code_set_global( t_ssize ); + Str usize_str = txt("usize"); t_usize = def_type( usize_str ); code_set_global( t_usize ); - def_constant_code_type( f32 ); - def_constant_code_type( f64 ); + Str f32_str = txt("f32"); t_f32 = def_type( f32_str ); code_set_global( t_f32 ); + Str f64_str = txt("f64"); t_f64 = def_type( f64_str ); code_set_global( t_f64 ); #endif -# undef def_constant_code_type + spec_const = def_specifier( Spec_Const); code_set_global( cast(Code, spec_const )); + spec_consteval = def_specifier( Spec_Consteval); code_set_global( cast(Code, spec_consteval )); + spec_constexpr = def_specifier( Spec_Constexpr); code_set_global( cast(Code, spec_constexpr )); + spec_constinit = def_specifier( Spec_Constinit); code_set_global( cast(Code, spec_constinit )); + spec_extern_linkage = def_specifier( Spec_External_Linkage); code_set_global( cast(Code, spec_extern_linkage )); + spec_final = def_specifier( Spec_Final); code_set_global( cast(Code, spec_final )); + spec_FORCEINLINE = def_specifier( Spec_ForceInline); code_set_global( cast(Code, spec_FORCEINLINE )); + spec_global = def_specifier( Spec_Global); code_set_global( cast(Code, spec_global )); + spec_inline = def_specifier( Spec_Inline); code_set_global( cast(Code, spec_inline )); + spec_internal_linkage = def_specifier( Spec_Internal_Linkage); code_set_global( cast(Code, spec_internal_linkage )); + spec_local_persist = def_specifier( Spec_Local_Persist); code_set_global( cast(Code, spec_local_persist )); + spec_mutable = def_specifier( Spec_Mutable); code_set_global( cast(Code, spec_mutable )); + spec_neverinline = def_specifier( Spec_NeverInline); code_set_global( cast(Code, spec_neverinline )); + spec_noexcept = def_specifier( Spec_NoExceptions); code_set_global( cast(Code, spec_noexcept )); + spec_override = def_specifier( Spec_Override); code_set_global( cast(Code, spec_override )); + spec_ptr = def_specifier( Spec_Ptr); code_set_global( cast(Code, spec_ptr )); + spec_pure = def_specifier( Spec_Pure); code_set_global( cast(Code, spec_pure )); + spec_ref = def_specifier( Spec_Ref); code_set_global( cast(Code, spec_ref )); + spec_register = def_specifier( Spec_Register); code_set_global( cast(Code, spec_register )); + spec_rvalue = def_specifier( Spec_RValue); code_set_global( cast(Code, spec_rvalue )); + spec_static_member = def_specifier( Spec_Static); code_set_global( cast(Code, spec_static_member )); + spec_thread_local = def_specifier( Spec_Thread_Local); code_set_global( cast(Code, spec_thread_local )); + spec_virtual = def_specifier( Spec_Virtual); code_set_global( cast(Code, spec_virtual )); + spec_volatile = def_specifier( Spec_Volatile); code_set_global( cast(Code, spec_volatile )); -# define def_constant_spec( Type_, ... ) \ - spec_##Type_ = def_specifiers( num_args(__VA_ARGS__), __VA_ARGS__); \ - spec_##Type_.set_global(); + spec_local_persist = def_specifiers( 1, Spec_Local_Persist ); + code_set_global(cast(Code, spec_local_persist)); -# pragma push_macro("FORCEINLINE") -# pragma push_macro("global") -# pragma push_macro("internal") -# pragma push_macro("local_persist") -# pragma push_macro("neverinline") -# undef FORCEINLINE -# undef global -# undef internal -# undef local_persist -# undef neverinline - def_constant_spec( const, ESpecifier::Const ); - def_constant_spec( consteval, ESpecifier::Consteval ); - def_constant_spec( constexpr, ESpecifier::Constexpr ); - def_constant_spec( constinit, ESpecifier::Constinit ); - def_constant_spec( extern_linkage, ESpecifier::External_Linkage ); - def_constant_spec( final, ESpecifier::Final ); - def_constant_spec( FORCEINLINE, ESpecifier::ForceInline ); - def_constant_spec( global, ESpecifier::Global ); - def_constant_spec( inline, ESpecifier::Inline ); - def_constant_spec( internal_linkage, ESpecifier::Internal_Linkage ); - def_constant_spec( local_persist, ESpecifier::Local_Persist ); - def_constant_spec( mutable, ESpecifier::Mutable ); - def_constant_spec( neverinline, ESpecifier::NeverInline ); - def_constant_spec( noexcept, ESpecifier::NoExceptions ); - def_constant_spec( override, ESpecifier::Override ); - def_constant_spec( ptr, ESpecifier::Ptr ); - def_constant_spec( pure, ESpecifier::Pure ) - def_constant_spec( ref, ESpecifier::Ref ); - def_constant_spec( register, ESpecifier::Register ); - def_constant_spec( rvalue, ESpecifier::RValue ); - def_constant_spec( static_member, ESpecifier::Static ); - def_constant_spec( thread_local, ESpecifier::Thread_Local ); - def_constant_spec( virtual, ESpecifier::Virtual ); - def_constant_spec( volatile, ESpecifier::Volatile) - - spec_local_persist = def_specifiers( 1, ESpecifier::Local_Persist ); - spec_local_persist.set_global(); - -# pragma pop_macro("FORCEINLINE") -# pragma pop_macro("global") -# pragma pop_macro("internal") -# pragma pop_macro("local_persist") -# pragma pop_macro("neverinline") - -# undef def_constant_spec + if (enum_underlying_macro.Name.Len == 0) { + enum_underlying_macro.Name = txt("enum_underlying"); + enum_underlying_macro.Type = MT_Expression; + enum_underlying_macro.Flags = MF_Functional; + } + register_macro(enum_underlying_macro); } -void init() +void init(Context* ctx) { - // Setup global allocator - { - GlobalAllocator = AllocatorInfo { & Global_Allocator_Proc, nullptr }; - - Global_AllocatorBuckets = Array::init_reserve( heap(), 128 ); - - if ( Global_AllocatorBuckets == nullptr ) - GEN_FATAL( "Failed to reserve memory for Global_AllocatorBuckets"); - - Arena bucket = Arena::init_from_allocator( heap(), Global_BucketSize ); - - if ( bucket.PhysicalStart == nullptr ) - GEN_FATAL( "Failed to create first bucket for Global_AllocatorBuckets"); - - Global_AllocatorBuckets.append( bucket ); - + do_once() { + context_counter = 0; } + AllocatorInfo fallback_allocator = { & fallback_allocator_proc, nullptr }; + + b32 using_fallback_allocator = false; + if (ctx->Allocator_DyanmicContainers.Proc == nullptr) { + ctx->Allocator_DyanmicContainers = fallback_allocator; + using_fallback_allocator = true; + } + if (ctx->Allocator_Pool.Proc == nullptr ) { + ctx->Allocator_Pool = fallback_allocator; + using_fallback_allocator = true; + } + if (ctx->Allocator_StrCache.Proc == nullptr) { + ctx->Allocator_StrCache = fallback_allocator; + using_fallback_allocator = true; + } + if (ctx->Allocator_Temp.Proc == nullptr) { + ctx->Allocator_Temp = fallback_allocator; + using_fallback_allocator = true; + } + // Setup fallback allocator + if (using_fallback_allocator) + { + ctx->Fallback_AllocatorBuckets = array_init_reserve(Arena, heap(), 128 ); + if ( ctx->Fallback_AllocatorBuckets == nullptr ) + GEN_FATAL( "Failed to reserve memory for Fallback_AllocatorBuckets"); + + Arena bucket = arena_init_from_allocator( heap(), ctx->InitSize_Fallback_Allocator_Bucket_Size ); + if ( bucket.PhysicalStart == nullptr ) + GEN_FATAL( "Failed to create first bucket for Fallback_AllocatorBuckets"); + + array_append( ctx->Fallback_AllocatorBuckets, bucket ); + } + + if (ctx->Max_CommentLineLength == 0) { + ctx->Max_CommentLineLength = 1024; + } + if (ctx->Max_StrCacheLength == 0) { + ctx->Max_StrCacheLength = kilobytes(512); + } + + if (ctx->InitSize_BuilderBuffer == 0) { + ctx->InitSize_BuilderBuffer = megabytes(2); + } + if (ctx->InitSize_CodePoolsArray == 0) { + ctx->InitSize_CodePoolsArray = 16; + } + if (ctx->InitSize_StringArenasArray == 0) { + ctx->InitSize_StringArenasArray = 16; + } + if (ctx->CodePool_NumBlocks == 0) { + ctx->CodePool_NumBlocks = kilobytes(16); + } + + if (ctx->InitSize_LexerTokens == 0 ) { + ctx->InitSize_LexerTokens = kilobytes(64); + } + if (ctx->SizePer_StringArena == 0) { + ctx->SizePer_StringArena = megabytes(1); + } + + if (ctx->InitSize_Fallback_Allocator_Bucket_Size == 0) { + ctx->InitSize_Fallback_Allocator_Bucket_Size = megabytes(8); + } + + // Override the current context (user has to put it back if unwanted). + _ctx = ctx; // Setup the arrays { - CodePools = Array::init_reserve( Allocator_DataArrays, InitSize_DataArrays ); - - if ( CodePools == nullptr ) + ctx->CodePools = array_init_reserve(Pool, ctx->Allocator_DyanmicContainers, ctx->InitSize_CodePoolsArray ); + if ( ctx->CodePools == nullptr ) GEN_FATAL( "gen::init: Failed to initialize the CodePools array" ); - StringArenas = Array::init_reserve( Allocator_DataArrays, InitSize_DataArrays ); - - if ( StringArenas == nullptr ) + ctx->StringArenas = array_init_reserve(Arena, ctx->Allocator_DyanmicContainers, ctx->InitSize_StringArenasArray ); + if ( ctx->StringArenas == nullptr ) GEN_FATAL( "gen::init: Failed to initialize the StringArenas array" ); } - // Setup the code pool and code entries arena. { - Pool code_pool = Pool::init( Allocator_CodePool, CodePool_NumBlocks, sizeof(AST) ); - + Pool code_pool = pool_init( ctx->Allocator_Pool, ctx->CodePool_NumBlocks, sizeof(AST) ); if ( code_pool.PhysicalStart == nullptr ) GEN_FATAL( "gen::init: Failed to initialize the code pool" ); + array_append( ctx->CodePools, code_pool ); - CodePools.append( code_pool ); - - LexArena = Arena::init_from_allocator( Allocator_Lexer, LexAllocator_Size ); - - Arena string_arena = Arena::init_from_allocator( Allocator_StringArena, SizePer_StringArena ); - - if ( string_arena.PhysicalStart == nullptr ) + // TODO(Ed): Eventually the string arenas needs to be phased out for a dedicated string slab allocator + Arena strbuilder_arena = arena_init_from_allocator( ctx->Allocator_StrCache, ctx->SizePer_StringArena ); + if ( strbuilder_arena.PhysicalStart == nullptr ) GEN_FATAL( "gen::init: Failed to initialize the string arena" ); - - StringArenas.append( string_arena ); + array_append( ctx->StringArenas, strbuilder_arena ); } - // Setup the hash tables { - StringCache = StringTable::init( Allocator_StringTable ); - - if ( StringCache.Entries == nullptr ) + ctx->StrCache = hashtable_init(StrCached, ctx->Allocator_DyanmicContainers); + if ( ctx->StrCache.Entries == nullptr ) GEN_FATAL( "gen::init: Failed to initialize the StringCache"); - } - // Preprocessor Defines - PreprocessorDefines = Array::init_reserve( GlobalAllocator, kilobytes(1) ); + ctx->Macros = hashtable_init(Macro, ctx->Allocator_DyanmicContainers); + if (ctx->Macros.Hashes == nullptr || ctx->Macros.Entries == nullptr) { + GEN_FATAL( "gen::init: Failed to initialize the PreprocessMacros table" ); + } + } define_constants(); - parser::init(); + parser_init(); + + ++ context_counter; } -void deinit() +void deinit(Context* ctx) { + GEN_ASSERT(context_counter); + GEN_ASSERT_MSG(context_counter > 0, "Attempted to deinit a context that for some reason wan't accounted for!"); usize index = 0; - usize left = CodePools.num(); + usize left = array_num(ctx->CodePools); do { - Pool* code_pool = & CodePools[index]; - code_pool->free(); + Pool* code_pool = & ctx->CodePools[index]; + pool_free(code_pool); index++; } while ( left--, left ); index = 0; - left = StringArenas.num(); + left = array_num(ctx->StringArenas); do { - Arena* string_arena = & StringArenas[index]; - string_arena->free(); + Arena* strbuilder_arena = & ctx->StringArenas[index]; + arena_free(strbuilder_arena); index++; } while ( left--, left ); - StringCache.destroy(); + hashtable_destroy(ctx->StrCache); - CodePools.free(); - StringArenas.free(); + array_free( ctx->CodePools); + array_free( ctx->StringArenas); - LexArena.free(); + hashtable_destroy(ctx->Macros); - PreprocessorDefines.free(); - - index = 0; - left = Global_AllocatorBuckets.num(); - do + left = array_num( ctx->Fallback_AllocatorBuckets); + if (left) { - Arena* bucket = & Global_AllocatorBuckets[ index ]; - bucket->free(); - index++; + index = 0; + do + { + Arena* bucket = & ctx->Fallback_AllocatorBuckets[ index ]; + arena_free(bucket); + index++; + } + while ( left--, left ); + array_free( ctx->Fallback_AllocatorBuckets); } - while ( left--, left ); + parser_deinit(); - Global_AllocatorBuckets.free(); - parser::deinit(); + if (_ctx == ctx) + _ctx = nullptr; + -- context_counter; } -void reset() +void reset(Context* ctx) { s32 index = 0; - s32 left = CodePools.num(); + s32 left = array_num(ctx->CodePools); do { - Pool* code_pool = & CodePools[index]; - code_pool->clear(); + Pool* code_pool = & ctx->CodePools[index]; + pool_clear(code_pool); index++; } while ( left--, left ); index = 0; - left = StringArenas.num(); + left = array_num(ctx->StringArenas); do { - Arena* string_arena = & StringArenas[index]; - string_arena->TotalUsed = 0;; + Arena* strbuilder_arena = & ctx->StringArenas[index]; + strbuilder_arena->TotalUsed = 0;; index++; } while ( left--, left ); - StringCache.clear(); - + hashtable_clear(ctx->StrCache); + hashtable_clear(ctx->Macros); define_constants(); } -AllocatorInfo get_string_allocator( s32 str_length ) +void set_context(Context* new_ctx) { + GEN_ASSERT(new_ctx); + _ctx = new_ctx; +} + +AllocatorInfo get_cached_str_allocator( s32 str_length ) { - Arena* last = & StringArenas.back(); - - usize size_req = str_length + sizeof(String::Header) + sizeof(char*); - - if ( last->TotalUsed + ssize(size_req) > last->TotalSize ) + Arena* last = array_back(_ctx->StringArenas); + usize size_req = str_length + sizeof(StrBuilderHeader) + sizeof(char*); + if ( last->TotalUsed + scast(ssize, size_req) > last->TotalSize ) { - Arena new_arena = Arena::init_from_allocator( Allocator_StringArena, SizePer_StringArena ); + Arena new_arena = arena_init_from_allocator( _ctx->Allocator_StrCache, _ctx->SizePer_StringArena ); + if ( ! array_append( _ctx->StringArenas, new_arena ) ) + GEN_FATAL( "gen::get_cached_str_allocator: Failed to allocate a new string arena" ); - if ( ! StringArenas.append( new_arena ) ) - GEN_FATAL( "gen::get_string_allocator: Failed to allocate a new string arena" ); - - last = & StringArenas.back(); + last = array_back( _ctx->StringArenas); } - - return * last; + return arena_allocator_info(last); } // Will either make or retrive a code string. -StringCached get_cached_string( StrC str ) +StrCached cache_str( Str str ) { - s32 hash_length = str.Len > kilobytes(1) ? kilobytes(1) : str.Len; - u64 key = crc32( str.Ptr, hash_length ); - { - StringCached* result = StringCache.get( key ); - + if (str.Len > _ctx->Max_StrCacheLength) { + // Do not cache the string, just shove into the arena and and return it. + Str result = strbuilder_to_str( strbuilder_make_str( get_cached_str_allocator( str.Len ), str )); + return result; + } + u64 key = crc32( str.Ptr, str.Len ); { + StrCached* result = hashtable_get( _ctx->StrCache, key ); if ( result ) return * result; } - - String result = String::make( get_string_allocator( str.Len ), str ); - StringCache.set( key, result ); - + Str result = strbuilder_to_str( strbuilder_make_str( get_cached_str_allocator( str.Len ), str )); + hashtable_set( _ctx->StrCache, key, result ); return result; } // Used internally to retireve a Code object form the CodePool. Code make_code() { - Pool* allocator = & CodePools.back(); + Pool* allocator = array_back( _ctx->CodePools); if ( allocator->FreeList == nullptr ) { - Pool code_pool = Pool::init( Allocator_CodePool, CodePool_NumBlocks, sizeof(AST) ); + Pool code_pool = pool_init( _ctx->Allocator_Pool, _ctx->CodePool_NumBlocks, sizeof(AST) ); if ( code_pool.PhysicalStart == nullptr ) GEN_FATAL( "gen::make_code: Failed to allocate a new code pool - CodePool allcoator returned nullptr." ); - if ( ! CodePools.append( code_pool ) ) + if ( ! array_append( _ctx->CodePools, code_pool ) ) GEN_FATAL( "gen::make_code: Failed to allocate a new code pool - CodePools failed to append new pool." ); - allocator = & CodePools.back(); + allocator = array_back( _ctx->CodePools); } - - Code result { rcast( AST*, alloc( * allocator, sizeof(AST) )) }; - mem_set( result.ast, 0, sizeof(AST) ); - // result->Type = ECode::Invalid; - - // result->Content = { nullptr }; - // result->Prev = { nullptr }; - // result->Next = { nullptr }; - // result->Token = nullptr; - // result->Parent = { nullptr }; - // result->Name = { nullptr }; - // result->Type = ECode::Invalid; - // result->ModuleFlags = ModuleFlag::Invalid; - // result->NumEntries = 0; - + Code result = { rcast( AST*, alloc( pool_allocator_info(allocator), sizeof(AST) )) }; + mem_set( rcast(void*, cast(AST*, result)), 0, sizeof(AST) ); return result; } -void set_allocator_data_arrays( AllocatorInfo allocator ) -{ - Allocator_DataArrays = allocator; +Macro* lookup_macro( Str name ) { + u32 key = crc32( name.Ptr, name.Len ); + return hashtable_get( _ctx->Macros, key ); } -void set_allocator_code_pool( AllocatorInfo allocator ) -{ - Allocator_CodePool = allocator; +void register_macro( Macro macro ) { + GEN_ASSERT_NOT_NULL(macro.Name.Ptr); + GEN_ASSERT(macro.Name.Len > 0); + u32 key = crc32( macro.Name.Ptr, macro.Name.Len ); + macro.Name = cache_str(macro.Name); + hashtable_set( _ctx->Macros, key, macro ); } -void set_allocator_lexer( AllocatorInfo allocator ) +void register_macros( s32 num, ... ) { - Allocator_Lexer = allocator; + GEN_ASSERT(num > 0); + va_list va; + va_start(va, num); + do + { + Macro macro = va_arg(va, Macro); + GEN_ASSERT_NOT_NULL(macro.Name.Ptr); + GEN_ASSERT(macro.Name.Len > 0); + macro.Name = cache_str(macro.Name); + + u32 key = crc32( macro.Name.Ptr, macro.Name.Len ); + hashtable_set( _ctx->Macros, key, macro ); + } + while (num--, num > 0); + va_end(va); } -void set_allocator_string_arena( AllocatorInfo allocator ) +void register_macros( s32 num, Macro* macros ) { - Allocator_StringArena = allocator; -} + GEN_ASSERT(num > 0); + do + { + Macro macro = * macros; + GEN_ASSERT_NOT_NULL(macro.Name.Ptr); + GEN_ASSERT(macro.Name.Len > 0); + macro.Name = cache_str(macro.Name); -void set_allocator_string_table( AllocatorInfo allocator ) -{ - Allocator_StringArena = allocator; + u32 key = crc32( macro.Name.Ptr, macro.Name.Len ); + hashtable_set( _ctx->Macros, key, macro ); + ++ macros; + } + while (num--, num > 0); } #pragma region Upfront -enum class OpValidateResult : u32 +enum OpValidateResult : u32 { - Fail, - Global, - Member + OpValResult_Fail, + OpValResult_Global, + OpValResult_Member }; -OpValidateResult operator__validate( OperatorT op, CodeParam params_code, CodeType ret_type, CodeSpecifiers specifier ) +internal neverinline +OpValidateResult operator__validate( Operator op, CodeParams params_code, CodeTypename ret_type, CodeSpecifiers specifier ) { - using namespace EOperator; - - if ( op == EOperator::Invalid ) + if ( op == Op_Invalid ) { log_failure("gen::def_operator: op cannot be invalid"); - return OpValidateResult::Fail; + return OpValResult_Fail; } #pragma region Helper Macros -# define check_params() \ - if ( ! params_code ) \ - { \ - log_failure("gen::def_operator: params is null and operator%s requires it", to_str(op)); \ - return OpValidateResult::Fail; \ - } \ - if ( params_code->Type != ECode::Parameters ) \ - { \ - log_failure("gen::def_operator: params is not of Parameters type - %s", params_code.debug_str()); \ - return OpValidateResult::Fail; \ +# define check_params() \ + if ( ! params_code ) \ + { \ + log_failure("gen::def_operator: params is null and operator %S requires it", operator_to_str(op)); \ + return OpValResult_Fail; \ + } \ + if ( params_code->Type != CT_Parameters ) \ + { \ + log_failure("gen::def_operator: params is not of Parameters type - %S", code_debug_str( cast(Code, params_code))); \ + return OpValResult_Fail; \ } -# define check_param_eq_ret() \ - if ( ! is_member_symbol && ! params_code->ValueType.is_equal( ret_type) ) \ - { \ - log_failure("gen::def_operator: operator%s requires first parameter to equal return type\n" \ - "param types: %s\n" \ - "return type: %s", \ - to_str(op).Ptr, \ - params_code.debug_str(), \ - ret_type.debug_str() \ - ); \ - return OpValidateResult::Fail; \ +# define check_param_eq_ret() \ + if ( ! is_member_symbol && ! code_is_equal(cast(Code, params_code->ValueType), cast(Code, ret_type)) ) \ + { \ + log_failure("gen::def_operator: operator %S requires first parameter to equal return type\n" \ + "param types: %S\n" \ + "return type: %S", \ + operator_to_str(op), \ + code_debug_str(cast(Code, params_code)), \ + code_debug_str(cast(Code, ret_type)) \ + ); \ + return OpValResult_Fail; \ } #pragma endregion Helper Macros if ( ! ret_type ) { - log_failure("gen::def_operator: ret_type is null but is required by operator%s", to_str(op)); + log_failure("gen::def_operator: ret_type is null but is required by operator %S", operator_to_str(op)); } - if ( ret_type->Type != ECode::Typename ) + if ( ret_type->Type != CT_Typename ) { - log_failure("gen::def_operator: ret_type is not of typename type - %s", ret_type.debug_str()); - return OpValidateResult::Fail; + log_failure("gen::def_operator: operator %S - ret_type is not of typename type - %S", + operator_to_str(op), + code_debug_str(cast(Code, ret_type)) + ); + return OpValResult_Fail; } bool is_member_symbol = false; @@ -3371,32 +3602,32 @@ OpValidateResult operator__validate( OperatorT op, CodeParam params_code, CodeTy switch ( op ) { # define specs( ... ) num_args( __VA_ARGS__ ), __VA_ARGS__ - case Assign: + case Op_Assign: check_params(); if ( params_code->NumEntries > 1 ) { log_failure("gen::def_operator: " - "operator%s does not support non-member definition (more than one parameter provided) - %s", - to_str(op), - params_code.debug_str() + "operator %S does not support non-member definition (more than one parameter provided) - %S", + operator_to_str(op), + code_debug_str(cast(Code, params_code)) ); - return OpValidateResult::Fail; + return OpValResult_Fail; } is_member_symbol = true; break; - case Assign_Add: - case Assign_Subtract: - case Assign_Multiply: - case Assign_Divide: - case Assign_Modulo: - case Assign_BAnd: - case Assign_BOr: - case Assign_BXOr: - case Assign_LShift: - case Assign_RShift: + case Op_Assign_Add: + case Op_Assign_Subtract: + case Op_Assign_Multiply: + case Op_Assign_Divide: + case Op_Assign_Modulo: + case Op_Assign_BAnd: + case Op_Assign_BOr: + case Op_Assign_BXOr: + case Op_Assign_LShift: + case Op_Assign_RShift: check_params(); if ( params_code->NumEntries == 1 ) @@ -3407,33 +3638,33 @@ OpValidateResult operator__validate( OperatorT op, CodeParam params_code, CodeTy if (params_code->NumEntries > 2 ) { - log_failure("gen::def_operator: operator%s may not be defined with more than two parametes - param count; %d\n%s" - , to_str(op) + log_failure("gen::def_operator: operator %S may not be defined with more than two parametes - param count; %d\n%S" + , operator_to_str(op) , params_code->NumEntries - , params_code.debug_str() + , code_debug_str(cast(Code, params_code)) ); - return OpValidateResult::Fail; + return OpValResult_Fail; } break; - case Increment: - case Decrement: + case Op_Increment: + case Op_Decrement: // If its not set, it just means its a prefix member op. if ( params_code ) { - if ( params_code->Type != ECode::Parameters ) + if ( params_code->Type != CT_Parameters ) { - log_failure("gen::def_operator: operator%s params code provided is not of Parameters type - %s" - , to_str(op) - , params_code.debug_str() + log_failure("gen::def_operator: operator %S params code provided is not of Parameters type - %S" + , operator_to_str(op) + , code_debug_str(cast(Code, params_code)) ); - return OpValidateResult::Fail; + return OpValResult_Fail; } switch ( params_code->NumEntries ) { case 1: - if ( params_code->ValueType.is_equal( t_int ) ) + if ( code_is_equal((Code)params_code->ValueType, (Code)t_int ) ) is_member_symbol = true; else @@ -3443,69 +3674,69 @@ OpValidateResult operator__validate( OperatorT op, CodeParam params_code, CodeTy case 2: check_param_eq_ret(); - if ( ! params_code.get(1).is_equal( t_int ) ) + if ( ! code_is_equal((Code)params_get(params_code, 1), (Code)t_int ) ) { log_failure("gen::def_operator: " - "operator%s requires second parameter of non-member definition to be int for post-decrement", - to_str(op) + "operator %S requires second parameter of non-member definition to be int for post-decrement", + operator_to_str(op) ); - return OpValidateResult::Fail; + return OpValResult_Fail; } break; default: - log_failure("gen::def_operator: operator%s recieved unexpected number of parameters recived %d instead of 0-2" - , to_str(op) + log_failure("gen::def_operator: operator %S recieved unexpected number of parameters recived %d instead of 0-2" + , operator_to_str(op) , params_code->NumEntries ); - return OpValidateResult::Fail; + return OpValResult_Fail; } } break; - case Unary_Plus: - case Unary_Minus: + case Op_Unary_Plus: + case Op_Unary_Minus: if ( ! params_code ) is_member_symbol = true; else { - if ( params_code->Type != ECode::Parameters ) + if ( params_code->Type != CT_Parameters ) { - log_failure("gen::def_operator: params is not of Parameters type - %s", params_code.debug_str()); - return OpValidateResult::Fail; + log_failure("gen::def_operator: params is not of Parameters type - %S", code_debug_str((Code)params_code)); + return OpValResult_Fail; } - if ( params_code->ValueType.is_equal( ret_type ) ) + if ( code_is_equal((Code)params_code->ValueType, (Code)ret_type ) ) { log_failure("gen::def_operator: " - "operator%s is non-member symbol yet first paramter does not equal return type\n" - "param type: %s\n" - "return type: %s\n" - , params_code.debug_str() - , ret_type.debug_str() + "operator %S is non-member symbol yet first paramter does not equal return type\n" + "param type: %S\n" + "return type: %S\n" + , code_debug_str((Code)params_code) + , code_debug_str((Code)ret_type) ); - return OpValidateResult::Fail; + return OpValResult_Fail; } if ( params_code->NumEntries > 1 ) { - log_failure("gen::def_operator: operator%s may not have more than one parameter - param count: %d" - , to_str(op) + log_failure("gen::def_operator: operator %S may not have more than one parameter - param count: %d" + , operator_to_str(op) , params_code->NumEntries ); - return OpValidateResult::Fail; + return OpValResult_Fail; } } break; - case BNot: + case Op_BNot: { // Some compilers let you do this... #if 0 if ( ! ret_type.is_equal( t_bool) ) { - log_failure( "gen::def_operator: return type is not a boolean - %s", params_code.debug_str() ); + log_failure( "gen::def_operator: operator %S return type is not a boolean - %S", operator_to_str(op) code_debug_str(params_code) ); return OpValidateResult::Fail; } #endif @@ -3515,35 +3746,35 @@ OpValidateResult operator__validate( OperatorT op, CodeParam params_code, CodeTy else { - if ( params_code->Type != ECode::Parameters ) + if ( params_code->Type != CT_Parameters ) { - log_failure( "gen::def_operator: params is not of Parameters type - %s", params_code.debug_str() ); - return OpValidateResult::Fail; + log_failure( "gen::def_operator: operator %S - params is not of Parameters type - %S", operator_to_str(op), code_debug_str((Code)params_code) ); + return OpValResult_Fail; } if ( params_code->NumEntries > 1 ) { log_failure( - "gen::def_operator: operator%s may not have more than one parameter - param count: %d", - to_str( op ), - params_code->NumEntries + "gen::def_operator: operator %S may not have more than one parameter - param count: %d", + operator_to_str( op ), + params_code->NumEntries ); - return OpValidateResult::Fail; + return OpValResult_Fail; } } break; } - case Add: - case Subtract: - case Multiply: - case Divide: - case Modulo: - case BAnd: - case BOr: - case BXOr: - case LShift: - case RShift: + case Op_Add: + case Op_Subtract: + case Op_Multiply: + case Op_Divide: + case Op_Modulo: + case Op_BAnd: + case Op_BOr: + case Op_BXOr: + case Op_LShift: + case Op_RShift: check_params(); switch ( params_code->NumEntries ) @@ -3553,68 +3784,69 @@ OpValidateResult operator__validate( OperatorT op, CodeParam params_code, CodeTy break; case 2: - if ( ! params_code->ValueType.is_equal( ret_type ) ) - { - log_failure("gen::def_operator: " - "operator%s is non-member symbol yet first paramter does not equal return type\n" - "param type: %s\n" - "return type: %s\n" - , params_code.debug_str() - , ret_type.debug_str() - ); - return OpValidateResult::Fail; - } + // This is allowed for arithemtic operators + // if ( ! code_is_equal((Code)params_code->ValueType, (Code)ret_type ) ) + // { + // log_failure("gen::def_operator: " + // "operator %S is non-member symbol yet first paramter does not equal return type\n" + // "param type: %S\n" + // "return type: %S\n" + // , code_debug_str((Code)params_code) + // , code_debug_str((Code)ret_type) + // ); + // return OpValResult_Fail; + // } break; default: - log_failure("gen::def_operator: operator%s recieved unexpected number of paramters recived %d instead of 0-2" - , to_str(op) + log_failure("gen::def_operator: operator %S recieved unexpected number of paramters recived %d instead of 0-2" + , operator_to_str(op) , params_code->NumEntries ); - return OpValidateResult::Fail; + return OpValResult_Fail; } break; - case UnaryNot: + case Op_UnaryNot: if ( ! params_code ) is_member_symbol = true; else { - if ( params_code->Type != ECode::Parameters ) + if ( params_code->Type != CT_Parameters ) { - log_failure("gen::def_operator: params is not of Parameters type - %s", params_code.debug_str()); - return OpValidateResult::Fail; + log_failure("gen::def_operator: operator %S - params is not of Parameters type - %S", operator_to_str(op), code_debug_str((Code)params_code)); + return OpValResult_Fail; } if ( params_code->NumEntries != 1 ) { - log_failure("gen::def_operator: operator%s recieved unexpected number of paramters recived %d instead of 0-1" - , to_str(op) + log_failure("gen::def_operator: operator %S recieved unexpected number of paramters recived %d instead of 0-1" + , operator_to_str(op) , params_code->NumEntries ); - return OpValidateResult::Fail; + return OpValResult_Fail; } } - if ( ! ret_type.is_equal( t_bool )) + if ( ! code_is_equal((Code)ret_type, (Code)t_bool )) { - log_failure("gen::def_operator: operator%s return type must be of type bool - %s" - , to_str(op) - , ret_type.debug_str() + log_failure("gen::def_operator: operator %S return type must be of type bool - %S" + , operator_to_str(op) + , code_debug_str((Code)ret_type) ); - return OpValidateResult::Fail; + return OpValResult_Fail; } break; - case LAnd: - case LOr: - case LEqual: - case LNot: - case Lesser: - case Greater: - case LesserEqual: - case GreaterEqual: + case Op_LAnd: + case Op_LOr: + case Op_LEqual: + case Op_LNot: + case Op_Lesser: + case Op_Greater: + case Op_LesserEqual: + case Op_GreaterEqual: check_params(); switch ( params_code->NumEntries ) @@ -3627,24 +3859,24 @@ OpValidateResult operator__validate( OperatorT op, CodeParam params_code, CodeTy break; default: - log_failure("gen::def_operator: operator%s recieved unexpected number of paramters recived %d instead of 1-2" - , to_str(op) + log_failure("gen::def_operator: operator %S recieved unexpected number of paramters recived %d instead of 1-2" + , operator_to_str(op) , params_code->NumEntries ); - return OpValidateResult::Fail; + return OpValResult_Fail; } break; - case Indirection: - case AddressOf: - case MemberOfPointer: + case Op_Indirection: + case Op_AddressOf: + case Op_MemberOfPointer: if ( params_code && params_code->NumEntries > 1) { - log_failure("gen::def_operator: operator%s recieved unexpected number of paramters recived %d instead of 0-1" - , to_str(op) + log_failure("gen::def_operator: operator %S recieved unexpected number of paramters recived %d instead of 0-1" + , operator_to_str(op) , params_code->NumEntries ); - return OpValidateResult::Fail; + return OpValResult_Fail; } else { @@ -3652,78 +3884,57 @@ OpValidateResult operator__validate( OperatorT op, CodeParam params_code, CodeTy } break; - case PtrToMemOfPtr: + case Op_PtrToMemOfPtr: if ( params_code ) { - log_failure("gen::def_operator: operator%s expects no paramters - %s", to_str(op), params_code.debug_str()); - return OpValidateResult::Fail; + log_failure("gen::def_operator: operator %S expects no paramters - %S", operator_to_str(op), code_debug_str((Code)params_code)); + return OpValResult_Fail; } break; - case Subscript: - case FunctionCall: - case Comma: + case Op_Subscript: + case Op_FunctionCall: + case Op_Comma: check_params(); break; - case New: - case Delete: + case Op_New: + case Op_Delete: // This library doesn't support validating new and delete yet. break; # undef specs } - return is_member_symbol ? OpValidateResult::Member : OpValidateResult::Global; + return is_member_symbol ? OpValResult_Member : OpValResult_Global; # undef check_params # undef check_ret_type # undef check_param_eq_ret } - -#pragma region Helper Marcos -// This snippet is used in nearly all the functions. -#define name_check( Context_, Name_ ) \ -{ \ - if ( Name_.Len <= 0 ) \ - { \ - log_failure( "gen::" stringize(Context_) ": Invalid name length provided - %d", Name_.Len ); \ - return CodeInvalid; \ - } \ - \ - if ( Name_.Ptr == nullptr ) \ - { \ - log_failure( "gen::" stringize(Context_) ": name is null" ); \ - return CodeInvalid; \ - } \ -} - -#define null_check( Context_, Code_ ) \ - if ( ! Code_ ) \ - { \ - log_failure( "gen::" stringize(Context_) ": " stringize(Code_) " provided is null" ); \ - return CodeInvalid; \ +FORCEINLINE +bool name__check( char const* context, Str name ) +{ + if ( name.Len <= 0 ) { + log_failure( "gen::%s: Invalid name length provided - %d", name.Len ); + return false; } - -#define null_or_invalid_check( Context_, Code_ ) \ -{ \ - if ( ! Code_ ) \ - { \ - log_failure( "gen::" stringize(Context_) ": " stringize(Code_) " provided is null" ); \ - return CodeInvalid; \ - } \ - \ - if ( Code_->is_invalid() ) \ - { \ - log_failure("gen::" stringize(Context_) ": " stringize(Code_) " provided is invalid" ); \ - return CodeInvalid; \ - } \ + if ( name.Ptr == nullptr ) { + log_failure( "gen::%s: name is null" ); + return false; + } + return true; } +#define name_check( context, name ) name__check( #context, name ) -#define not_implemented( Context_ ) \ - log_failure( "gen::%s: This function is not implemented" ); \ - return CodeInvalid; -#pragma endregion Helper Marcos - +FORCEINLINE +bool null__check( char const* context, char const* code_id, Code code ) { + if ( code == nullptr ) { + log_failure( "gen::%s: %s provided is null", context, code_id ); + return false; + } + return true; +} +#define null_check( context, code ) null__check( #context, #code, cast(Code, code) ) /* The implementaiton of the upfront constructors involves doing three things: @@ -3737,34 +3948,31 @@ identify the issue without having to debug too much (at least they can debug tho The largest of the functions is related to operator overload definitions. The library validates a good protion of their form and thus the argument processing for is quite a bit. */ -CodeAttributes def_attributes( StrC content ) +CodeAttributes def_attributes( Str content ) { - if ( content.Len <= 0 || content.Ptr == nullptr ) - { + if ( content.Len <= 0 || content.Ptr == nullptr ) { log_failure( "gen::def_attributes: Invalid attributes provided" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } - Code result = make_code(); - result->Type = ECode::PlatformAttributes; - result->Name = get_cached_string( content ); + result->Type = CT_PlatformAttributes; + result->Name = cache_str( content ); result->Content = result->Name; - return (CodeAttributes) result; } -CodeComment def_comment( StrC content ) +CodeComment def_comment( Str content ) { if ( content.Len <= 0 || content.Ptr == nullptr ) { log_failure( "gen::def_comment: Invalid comment provided:" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } - static char line[ MaxCommentLineLength ]; - - String cmt_formatted = String::make_reserve( GlobalAllocator, kilobytes(1) ); + StrBuilder cmt_formatted = strbuilder_make_reserve( _ctx->Allocator_Temp, kilobytes(1) ); char const* end = content.Ptr + content.Len; char const* scanner = content.Ptr; s32 curr = 0; @@ -3779,1062 +3987,883 @@ CodeComment def_comment( StrC content ) } length++; - str_copy( line, scanner, length ); - cmt_formatted.append_fmt( "//%.*s", length, line ); - mem_set( line, 0, MaxCommentLineLength ); - + strbuilder_append_fmt(& cmt_formatted, "//%.*s", length, scanner ); scanner += length; } while ( scanner <= end ); - if ( cmt_formatted.back() != '\n' ) - cmt_formatted.append( "\n" ); + if ( * strbuilder_back(cmt_formatted) != '\n' ) + strbuilder_append_str( & cmt_formatted, txt("\n") ); + + Str name = strbuilder_to_str(cmt_formatted); Code result = make_code(); - result->Type = ECode::Comment; - result->Name = get_cached_string( cmt_formatted ); + result->Type = CT_Comment; + result->Name = cache_str( name ); result->Content = result->Name; - cmt_formatted.free(); + strbuilder_free(& cmt_formatted); return (CodeComment) result; } -CodeConstructor def_constructor( CodeParam params, Code initializer_list, Code body ) +CodeConstructor def_constructor( Opts_def_constructor p ) { - using namespace ECode; - - if ( params && params->Type != Parameters ) - { - log_failure("gen::def_constructor: params must be of Parameters type - %s", params.debug_str()); - return CodeInvalid; + if ( p.params && p.params->Type != CT_Parameters ) { + log_failure("gen::def_constructor: params must be of Parameters type - %s", code_debug_str((Code)p.params)); + GEN_DEBUG_TRAP(); + return InvalidCode; } - CodeConstructor - result = (CodeConstructor) make_code(); - - if ( params ) - { - result->Params = params; + CodeConstructor result = (CodeConstructor) make_code(); + if ( p.params ) { + result->Params = p.params; } - - if ( initializer_list ) - { - result->InitializerList = initializer_list; + if ( p.initializer_list ) { + result->InitializerList = p.initializer_list; } - - if ( body ) + if ( p.body ) { - switch ( body->Type ) - { - case Function_Body: - case Untyped: + switch ( p.body->Type ) { + case CT_Function_Body: + case CT_Untyped: break; default: - log_failure("gen::def_constructor: body must be either of Function_Body or Untyped type - %s", body.debug_str()); - return CodeInvalid; + log_failure("gen::def_constructor: body must be either of Function_Body or Untyped type - %s", code_debug_str(p.body)); + return InvalidCode; } - result->Type = Constructor; - result->Body = body; + result->Type = CT_Constructor; + result->Body = p.body; } else { - result->Type = Constructor_Fwd; + result->Type = CT_Constructor_Fwd; } - return result; } -CodeClass def_class( StrC name - , Code body - , CodeType parent, AccessSpec parent_access - , CodeAttributes attributes - , ModuleFlag mflags - , CodeType* interfaces, s32 num_interfaces ) +CodeClass def_class( Str name, Opts_def_struct p ) { - using namespace ECode; - - name_check( def_class, name ); - - if ( attributes && attributes->Type != PlatformAttributes ) - { - log_failure( "gen::def_class: attributes was not a 'PlatformAttributes' type: %s", attributes.debug_str() ); - return CodeInvalid; + if ( ! name_check( def_class, name ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( parent && ( parent->Type != Class && parent->Type != Struct && parent->Type != Typename && parent->Type != Untyped ) ) - { - log_failure( "gen::def_class: parent provided is not type 'Class', 'Struct', 'Typeanme', or 'Untyped': %s", parent.debug_str() ); - return CodeInvalid; + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_class: attributes was not a 'PlatformAttributes' type: %s", code_debug_str(p.attributes) ); + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( p.parent && ( p.parent->Type != CT_Class && p.parent->Type != CT_Struct && p.parent->Type != CT_Typename && p.parent->Type != CT_Untyped ) ) { + log_failure( "gen::def_class: parent provided is not type 'Class', 'Struct', 'Typeanme', or 'Untyped': %s", code_debug_str(p.parent) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } CodeClass - result = (CodeClass) make_code(); - result->Name = get_cached_string( name ); - result->ModuleFlags = mflags; - - if ( body ) + result = (CodeClass) make_code(); + result->Name = cache_str( name ); + result->ModuleFlags = p.mflags; + result->Attributes = p.attributes; + result->ParentAccess = p.parent_access; + result->ParentType = p.parent; + if ( p.body ) { - switch ( body->Type ) + switch ( p.body->Type ) { - case Class_Body: - case Untyped: + case CT_Class_Body: + case CT_Untyped: break; default: - log_failure("gen::def_class: body must be either of Class_Body or Untyped type - %s", body.debug_str()); - return CodeInvalid; + log_failure("gen::def_class: body must be either of Class_Body or Untyped type - %s", code_debug_str(p.body)); + return InvalidCode; } - result->Type = Class; - result->Body = body; - result->Body->Parent = result; // TODO(Ed): Review this? + result->Type = CT_Class; + result->Body = p.body; + result->Body->Parent = cast(Code, result); } - else - { - result->Type = Class_Fwd; + else { + result->Type = CT_Class_Fwd; } - - if ( attributes ) - result->Attributes = attributes; - - if ( parent ) - { - result->ParentAccess = parent_access; - result->ParentType = parent; + for (s32 idx = 0; idx < p.num_interfaces; idx++ ) { + class_add_interface(result, p.interfaces[idx] ); } - - if ( interfaces ) - { - for (s32 idx = 0; idx < num_interfaces; idx++ ) - { - result.add_interface( interfaces[idx] ); - } - } - return result; } -CodeDefine def_define( StrC name, StrC content ) +CodeDefine def_define( Str name, MacroType type, Opts_def_define p ) { - using namespace ECode; - - name_check( def_define, name ); - - // Defines can be empty definitions -#if 0 - if ( content.Len <= 0 || content.Ptr == nullptr ) - { - log_failure( "gen::def_define: Invalid value provided" ); - return CodeInvalid; + if ( ! name_check( def_define, name ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; } -#endif - CodeDefine result = (CodeDefine) make_code(); - result->Type = Preprocess_Define; - result->Name = get_cached_string( name ); - if ( content.Len <= 0 || content.Ptr == nullptr ) - { - result->Content = get_cached_string( txt("") ); - } + result->Type = CT_Preprocess_Define; + result->Name = cache_str( name ); + result->Params = p.params; + if ( p.content.Len <= 0 || p.content.Ptr == nullptr ) + result->Body = untyped_str( txt("\n") ); else - result->Content = get_cached_string( content ); + result->Body = untyped_str( strbuilder_to_str(strbuilder_fmt_buf(_ctx->Allocator_Temp, "%S\n", p.content)) ); + b32 register_define = ! p.dont_register_to_preprocess_macros; + if ( register_define ) { + Macro macro_entry = { result->Name, type, p.flags }; + register_macro(macro_entry); + } return result; } -CodeDestructor def_destructor( Code body, CodeSpecifiers specifiers ) +CodeDestructor def_destructor( Opts_def_destructor p ) { - using namespace ECode; - - if ( specifiers && specifiers->Type != Specifiers ) - { - log_failure( "gen::def_destructor: specifiers was not a 'Specifiers' type: %s", specifiers.debug_str() ); - return CodeInvalid; + if ( p.specifiers && p.specifiers->Type != CT_Specifiers ) { + log_failure( "gen::def_destructor: specifiers was not a 'Specifiers' type: %s", code_debug_str(p.specifiers) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - CodeDestructor result = (CodeDestructor) make_code(); - - if ( specifiers ) - result->Specs = specifiers; - - if ( body ) + CodeDestructor + result = (CodeDestructor) make_code(); + result->Specs = p.specifiers; + if ( p.body ) { - switch ( body->Type ) + switch ( p.body->Type ) { - case Function_Body: - case Untyped: + case CT_Function_Body: + case CT_Untyped: break; default: - log_failure("gen::def_destructor: body must be either of Function_Body or Untyped type - %s", body.debug_str()); - return CodeInvalid; + log_failure("gen::def_destructor: body must be either of Function_Body or Untyped type - %s", code_debug_str(p.body)); + return InvalidCode; } - result->Type = Destructor; - result->Body = body; + result->Type = CT_Destructor; + result->Body = p.body; } else { - result->Type = Destructor_Fwd; + result->Type = CT_Destructor_Fwd; } - return result; } -CodeEnum def_enum( StrC name - , Code body, CodeType type - , EnumT specifier, CodeAttributes attributes - , ModuleFlag mflags ) +CodeEnum def_enum( Str name, Opts_def_enum p ) { - using namespace ECode; - - name_check( def_enum, name ); - - if ( type && type->Type != Typename ) - { - log_failure( "gen::def_enum: enum underlying type provided was not of type Typename: %s", type.debug_str() ); - return CodeInvalid; + if ( ! name_check( def_enum, name ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( attributes && attributes->Type != PlatformAttributes ) - { - log_failure( "gen::def_enum: attributes was not a 'PlatformAttributes' type: %s", attributes.debug_str() ); - return CodeInvalid; + if ( p.type && p.type->Type != CT_Typename ) { + log_failure( "gen::def_enum: enum underlying type provided was not of type Typename: %s", code_debug_str(p.type) ); + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_enum: attributes was not a 'PlatformAttributes' type: %s", code_debug_str(p.attributes) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } CodeEnum result = (CodeEnum) make_code(); - result->Name = get_cached_string( name ); - result->ModuleFlags = mflags; - - if ( body ) + result->Name = cache_str( name ); + result->ModuleFlags = p.mflags; + if ( p.body ) { - switch ( body->Type ) + switch ( p.body->Type ) { - case Enum_Body: - case Untyped: + case CT_Enum_Body: + case CT_Untyped: break; default: - log_failure( "gen::def_enum: body must be of Enum_Body or Untyped type %s", body.debug_str()); - return CodeInvalid; + log_failure( "gen::def_enum: body must be of Enum_Body or Untyped type %s", code_debug_str(p.body)); + return InvalidCode; } - result->Type = specifier == EnumClass ? - Enum_Class : Enum; + result->Type = p.specifier == EnumDecl_Class ? + CT_Enum_Class : CT_Enum; - result->Body = body; + result->Body = p.body; } else { - result->Type = specifier == EnumClass ? - Enum_Class_Fwd : Enum_Fwd; + result->Type = p.specifier == EnumDecl_Class ? + CT_Enum_Class_Fwd : CT_Enum_Fwd; } + result->Attributes = p.attributes; - if ( attributes ) - result->Attributes = attributes; - - if ( type ) - { - result->UnderlyingType = type; + if ( p.type ) { + result->UnderlyingType = p.type; } - else if ( result->Type != Enum_Class_Fwd && result->Type != Enum_Fwd ) + else if ( p.type_macro ) { + result->UnderlyingTypeMacro = p.type_macro; + } + else if ( result->Type != CT_Enum_Class_Fwd && result->Type != CT_Enum_Fwd ) { log_failure( "gen::def_enum: enum forward declaration must have an underlying type" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } - return result; } -CodeExec def_execution( StrC content ) +CodeExec def_execution( Str content ) { - if ( content.Len <= 0 || content.Ptr == nullptr ) - { + if ( content.Len <= 0 || content.Ptr == nullptr ) { log_failure( "gen::def_execution: Invalid execution provided" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } - - Code - result = make_code(); - result->Type = ECode::Execution; - result->Name = get_cached_string( content ); - result->Content = result->Name; - - return (CodeExec) result; + CodeExec + result = (CodeExec) make_code(); + result->Type = CT_Execution; + result->Content = cache_str( content ); + return result; } -CodeExtern def_extern_link( StrC name, Code body ) +CodeExtern def_extern_link( Str name, CodeBody body ) { - using namespace ECode; - - name_check( def_extern_linkage, name ); - null_check( def_extern_linkage, body ); - - if ( body->Type != Extern_Linkage_Body && body->Type != Untyped ) - { - log_failure("gen::def_extern_linkage: body is not of extern_linkage or untyped type %s", body->debug_str()); - return CodeInvalid; + if ( ! name_check(def_extern_link, name) || ! null_check(def_extern_link, body) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( body->Type != CT_Extern_Linkage_Body && body->Type != CT_Untyped ) { + log_failure("gen::def_extern_linkage: body is not of extern_linkage or untyped type %s", code_debug_str(body)); + GEN_DEBUG_TRAP(); + return InvalidCode; } - CodeExtern result = (CodeExtern)make_code(); - result->Type = Extern_Linkage; - result->Name = get_cached_string( name ); + result->Type = CT_Extern_Linkage; + result->Name = cache_str( name ); result->Body = body; - - return (CodeExtern) result; + return result; } CodeFriend def_friend( Code declaration ) { - using namespace ECode; - - null_check( def_friend, declaration ); - + if ( ! null_check( def_friend, declaration ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } switch ( declaration->Type ) { - case Class_Fwd: - case Function_Fwd: - case Operator_Fwd: - case Struct_Fwd: - case Class: - case Function: - case Operator: - case Struct: + case CT_Class_Fwd: + case CT_Function_Fwd: + case CT_Operator_Fwd: + case CT_Struct_Fwd: + case CT_Class: + case CT_Function: + case CT_Operator: + case CT_Struct: break; default: - log_failure("gen::def_friend: requires declartion to have class, function, operator, or struct - %s", declaration->debug_str()); - return CodeInvalid; + log_failure("gen::def_friend: requires declartion to have class, function, operator, or struct - %s", code_debug_str(declaration)); + return InvalidCode; } - CodeFriend - result = (CodeFriend) make_code(); - result->Type = Friend; - + result = (CodeFriend) make_code(); + result->Type = CT_Friend; result->Declaration = declaration; - return result; } -CodeFn def_function( StrC name - , CodeParam params , CodeType ret_type, Code body - , CodeSpecifiers specifiers, CodeAttributes attributes - , ModuleFlag mflags ) +CodeFn def_function( Str name, Opts_def_function p ) { - using namespace ECode; - - name_check( def_function, name ); - - if ( params && params->Type != Parameters ) - { - log_failure( "gen::def_function: params was not a `Parameters` type: %s", params.debug_str() ); - return CodeInvalid; + if ( ! name_check( def_function, name )) { + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( ret_type && ret_type->Type != Typename ) - { - log_failure( "gen::def_function: ret_type was not a Typename: %s", ret_type.debug_str() ); - return CodeInvalid; + if ( p.params && p.params->Type != CT_Parameters ) { + log_failure( "gen::def_function: params was not a `Parameters` type: %s", code_debug_str(p.params) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( specifiers && specifiers->Type != Specifiers ) - { - log_failure( "gen::def_function: specifiers was not a `Specifiers` type: %s", specifiers.debug_str() ); - return CodeInvalid; + if ( p.ret_type && p.ret_type->Type != CT_Typename ) { + log_failure( "gen::def_function: ret_type was not a Typename: %s", code_debug_str(p.ret_type) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( attributes && attributes->Type != PlatformAttributes ) - { - log_failure( "gen::def_function: attributes was not a `PlatformAttributes` type: %s", attributes.debug_str() ); - return CodeInvalid; + if ( p.specs && p.specs-> Type != CT_Specifiers ) { + log_failure( "gen::def_function: specifiers was not a `Specifiers` type: %s", code_debug_str(p.specs) ); + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( p.attrs && p.attrs->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_function: attributes was not a `PlatformAttributes` type: %s", code_debug_str(p.attrs) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } CodeFn result = (CodeFn) make_code(); - result->Name = get_cached_string( name ); - result->ModuleFlags = mflags; - - if ( body ) + result->Name = cache_str( name ); + result->ModuleFlags = p.mflags; + if ( p.body ) { - switch ( body->Type ) + switch ( p.body->Type ) { - case Function_Body: - case Execution: - case Untyped: + case CT_Function_Body: + case CT_Execution: + case CT_Untyped: break; default: { - log_failure("gen::def_function: body must be either of Function_Body, Execution, or Untyped type. %s", body->debug_str()); - return CodeInvalid; + log_failure("gen::def_function: body must be either of Function_Body, Execution, or Untyped type. %s", code_debug_str(p.body)); + return InvalidCode; } } - - result->Type = Function; - result->Body = body; + result->Type = CT_Function; + result->Body = p.body; } else { - result->Type = Function_Fwd; + result->Type = CT_Function_Fwd; } - - if ( attributes ) - result->Attributes = attributes; - - if ( specifiers ) - result->Specs = specifiers; - - if ( ret_type ) - { - result->ReturnType = ret_type; - } - else - { - result->ReturnType = t_void; - } - - if ( params ) - result->Params = params; - + result->Attributes = p.attrs; + result->Specs = p.specs; + result->Params = p.params; + result->ReturnType = p.ret_type ? p.ret_type : t_void; return result; } -CodeInclude def_include( StrC path, bool foreign ) +CodeInclude def_include( Str path, Opts_def_include p ) { - if ( path.Len <= 0 || path.Ptr == nullptr ) - { + if ( path.Len <= 0 || path.Ptr == nullptr ) { log_failure( "gen::def_include: Invalid path provided - %d" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } + StrBuilder content = p.foreign ? + strbuilder_fmt_buf( _ctx->Allocator_Temp, "<%.*s>", path.Len, path.Ptr ) + : strbuilder_fmt_buf( _ctx->Allocator_Temp, "\"%.*s\"", path.Len, path.Ptr ); - StrC content = foreign ? - to_str( str_fmt_buf( "<%.*s>", path.Len, path.Ptr )) - : to_str( str_fmt_buf( "\"%.*s\"", path.Len, path.Ptr )); - - Code - result = make_code(); - result->Type = ECode::Preprocess_Include; - result->Name = get_cached_string( content ); + CodeInclude + result = (CodeInclude) make_code(); + result->Type = CT_Preprocess_Include; + result->Name = cache_str( strbuilder_to_str(content) ); result->Content = result->Name; - - return (CodeInclude) result; + return result; } -CodeModule def_module( StrC name, ModuleFlag mflags ) +CodeModule def_module( Str name, Opts_def_module p ) { - name_check( def_module, name ); - - Code - result = make_code(); - result->Type = ECode::Module; - result->Name = get_cached_string( name ); - result->Content = result->Name; - result->ModuleFlags = mflags; - - return (CodeModule) result; -} - -CodeNS def_namespace( StrC name, Code body, ModuleFlag mflags ) -{ - using namespace ECode; - - name_check( def_namespace, name ); - null_check( def_namespace, body ); - - if ( body->Type != Namespace_Body && body->Type != Untyped ) - { - log_failure("gen::def_namespace: body is not of namespace or untyped type %s", body.debug_str()); - return CodeInvalid; + if ( ! name_check( def_module, name )) { + GEN_DEBUG_TRAP(); + return InvalidCode; } + CodeModule + result = (CodeModule) make_code(); + result->Type = CT_Module; + result->Name = cache_str( name ); + result->ModuleFlags = p.mflags; + return result; +} +CodeNS def_namespace( Str name, CodeBody body, Opts_def_namespace p ) +{ + if ( ! name_check( def_namespace, name )) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( ! null_check( def_namespace, body)) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( body && body->Type != CT_Namespace_Body && body->Type != CT_Untyped ) { + log_failure("gen::def_namespace: body is not of namespace or untyped type %s", code_debug_str(body)); + GEN_DEBUG_TRAP(); + return InvalidCode; + } CodeNS result = (CodeNS) make_code(); - result->Type = Namespace; - result->Name = get_cached_string( name ); - result->ModuleFlags = mflags; + result->Type = CT_Namespace; + result->Name = cache_str( name ); + result->ModuleFlags = p.mflags; result->Body = body; - return result; } -CodeOperator def_operator( OperatorT op, StrC nspace - , CodeParam params_code, CodeType ret_type, Code body - , CodeSpecifiers specifiers, CodeAttributes attributes - , ModuleFlag mflags ) +CodeOperator def_operator( Operator op, Str nspace, Opts_def_operator p ) { - using namespace ECode; - - if ( attributes && attributes->Type != PlatformAttributes ) - { - log_failure( "gen::def_operator: PlatformAttributes was provided but its not of attributes type: %s", attributes.debug_str() ); - return CodeInvalid; + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_operator: PlatformAttributes was provided but its not of attributes type: %s", code_debug_str(p.attributes) ); + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( p.specifiers && p.specifiers->Type != CT_Specifiers ) { + log_failure( "gen::def_operator: Specifiers was provided but its not of specifiers type: %s", code_debug_str(p.specifiers) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - if ( specifiers && specifiers->Type != Specifiers ) - { - log_failure( "gen::def_operator: Specifiers was provided but its not of specifiers type: %s", specifiers.debug_str() ); - return CodeInvalid; - } - - OpValidateResult check_result = operator__validate( op, params_code, ret_type, specifiers ); - - if ( check_result == OpValidateResult::Fail ) - { - return CodeInvalid; + OpValidateResult check_result = operator__validate( op, p.params, p.ret_type, p.specifiers ); + if ( check_result == OpValResult_Fail ) { + return InvalidCode; } char const* name = nullptr; - StrC op_str = to_str( op ); + Str op_str = operator_to_str( op ); if ( nspace.Len > 0 ) - name = str_fmt_buf( "%.*soperator %.*s", nspace.Len, nspace.Ptr, op_str.Len, op_str.Ptr ); + name = c_str_fmt_buf( "%.*soperator %.*s", nspace.Len, nspace.Ptr, op_str.Len, op_str.Ptr ); else - name = str_fmt_buf( "operator %.*s", op_str.Len, op_str.Ptr ); + name = c_str_fmt_buf( "operator %.*s", op_str.Len, op_str.Ptr ); + + Str name_resolved = { name, c_str_len(name) }; + CodeOperator result = (CodeOperator) make_code(); - result->Name = get_cached_string( { str_len(name), name } ); - result->ModuleFlags = mflags; + result->Name = cache_str( name_resolved ); + result->ModuleFlags = p.mflags; result->Op = op; - - if ( body ) + if ( p.body ) { - switch ( body->Type ) + switch ( p.body->Type ) { - case Function_Body: - case Execution: - case Untyped: + case CT_Function_Body: + case CT_Execution: + case CT_Untyped: break; default: { - log_failure("gen::def_operator: body must be either of Function_Body, Execution, or Untyped type. %s", body->debug_str()); - return CodeInvalid; + log_failure("gen::def_operator: body must be either of Function_Body, Execution, or Untyped type. %s", code_debug_str(p.body)); + GEN_DEBUG_TRAP(); + return InvalidCode; } } - result->Type = check_result == OpValidateResult::Global ? - Operator : Operator_Member; + result->Type = check_result == OpValResult_Global ? + CT_Operator : CT_Operator_Member; - result->Body = body; + result->Body = p.body; } else { - result->Type = check_result == OpValidateResult::Global ? - Operator_Fwd : Operator_Member_Fwd; + result->Type = check_result == OpValResult_Global ? + CT_Operator_Fwd : CT_Operator_Member_Fwd; } - - if ( attributes ) - result->Attributes = attributes; - - if ( specifiers ) - result->Specs = specifiers; - - result->ReturnType = ret_type; - - if (params_code) - result->Params = params_code; - + result->Attributes = p.attributes; + result->Specs = p.specifiers; + result->ReturnType = p.ret_type; + result->Params = p.params; return result; } -CodeOpCast def_operator_cast( CodeType type, Code body, CodeSpecifiers const_spec ) +CodeOpCast def_operator_cast( CodeTypename type, Opts_def_operator_cast p ) { - using namespace ECode; - null_check( def_operator_cast, type ); - - if ( type->Type != Typename ) - { - log_failure( "gen::def_operator_cast: type is not a typename - %s", type.debug_str() ); - return CodeInvalid; + if ( ! null_check( def_operator_cast, type )) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( type->Type != CT_Typename ) { + log_failure( "gen::def_operator_cast: type is not a typename - %s", code_debug_str(type) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } CodeOpCast result = (CodeOpCast) make_code(); - - if (body) + if (p.body) { - result->Type = Operator_Cast; + result->Type = CT_Operator_Cast; - if ( body->Type != Function_Body && body->Type != Execution ) - { - log_failure( "gen::def_operator_cast: body is not of function body or execution type - %s", body.debug_str() ); - return CodeInvalid; + if ( p.body->Type != CT_Function_Body && p.body->Type != CT_Execution ) { + log_failure( "gen::def_operator_cast: body is not of function body or execution type - %s", code_debug_str(p.body) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - result->Body = body; + result->Body = p.body; } else { - result->Type = Operator_Cast_Fwd; + result->Type = CT_Operator_Cast_Fwd; } - - if ( const_spec ) - { - result->Specs = const_spec; - } - + result->Specs = p.specs; result->ValueType = type; return result; } -CodeParam def_param( CodeType type, StrC name, Code value ) +CodeParams def_param( CodeTypename type, Str name, Opts_def_param p ) { - using namespace ECode; - - name_check( def_param, name ); - null_check( def_param, type ); - - if ( type->Type != Typename ) - { - log_failure( "gen::def_param: type is not a typename - %s", type.debug_str() ); - return CodeInvalid; + if ( ! name_check( def_param, name ) || ! null_check( def_param, type ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( value && value->Type != Untyped ) - { - log_failure( "gen::def_param: value is not untyped - %s", value.debug_str() ); - return CodeInvalid; + if ( type->Type != CT_Typename ) { + log_failure( "gen::def_param: type is not a typename - %s", code_debug_str(type) ); + return InvalidCode; } - - CodeParam - result = (CodeParam) make_code(); - result->Type = Parameters; - result->Name = get_cached_string( name ); - + if ( p.value && p.value->Type != CT_Untyped ) { + log_failure( "gen::def_param: value is not untyped - %s", code_debug_str(p.value) ); + return InvalidCode; + } + CodeParams + result = (CodeParams) make_code(); + result->Type = CT_Parameters; + result->Name = cache_str( name ); result->ValueType = type; - - if ( value ) - result->Value = value; - + result->Value = p.value; result->NumEntries++; - return result; } -CodePragma def_pragma( StrC directive ) +CodePragma def_pragma( Str directive ) { - using namespace ECode; - - if ( directive.Len <= 0 || directive.Ptr == nullptr ) - { + if ( directive.Len <= 0 || directive.Ptr == nullptr ) { log_failure( "gen::def_comment: Invalid comment provided:" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } - CodePragma result = (CodePragma) make_code(); - result->Type = Preprocess_Pragma; - result->Content = get_cached_string( directive ); - + result->Type = CT_Preprocess_Pragma; + result->Content = cache_str( directive ); return result; } -CodePreprocessCond def_preprocess_cond( EPreprocessCond type, StrC expr ) +CodePreprocessCond def_preprocess_cond( EPreprocessCond type, Str expr ) { - using namespace ECode; - - if ( expr.Len <= 0 || expr.Ptr == nullptr ) - { + if ( expr.Len <= 0 || expr.Ptr == nullptr ) { log_failure( "gen::def_comment: Invalid comment provided:" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } - CodePreprocessCond result = (CodePreprocessCond) make_code(); - result->Content = get_cached_string( expr ); - + result->Content = cache_str( expr ); switch (type) { - case EPreprocessCond::If: - result->Type = Preprocess_If; + case PreprocessCond_If: + result->Type = CT_Preprocess_If; break; - case EPreprocessCond::IfDef: - result->Type = Preprocess_IfDef; + case PreprocessCond_IfDef: + result->Type = CT_Preprocess_IfDef; break; - case EPreprocessCond::IfNotDef: - result->Type = Preprocess_IfNotDef; + case PreprocessCond_IfNotDef: + result->Type = CT_Preprocess_IfNotDef; break; - case EPreprocessCond::ElIf: - result->Type = Preprocess_ElIf; + case PreprocessCond_ElIf: + result->Type = CT_Preprocess_ElIf; break; } - return result; } -CodeSpecifiers def_specifier( SpecifierT spec ) +CodeSpecifiers def_specifier( Specifier spec ) { CodeSpecifiers result = (CodeSpecifiers) make_code(); - result->Type = ECode::Specifiers; - result.append( spec ); - + result->Type = CT_Specifiers; + specifiers_append(result, spec ); return result; } -CodeStruct def_struct( StrC name - , Code body - , CodeType parent, AccessSpec parent_access - , CodeAttributes attributes - , ModuleFlag mflags - , CodeType* interfaces, s32 num_interfaces ) +CodeStruct def_struct( Str name, Opts_def_struct p ) { - using namespace ECode; - - if ( attributes && attributes->Type != PlatformAttributes ) - { - log_failure( "gen::def_struct: attributes was not a `PlatformAttributes` type - %s", attributes.debug_str() ); - return CodeInvalid; + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_struct: attributes was not a `PlatformAttributes` type - %s", code_debug_str(cast(Code, p.attributes)) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( parent && parent->Type != Typename ) - { - log_failure( "gen::def_struct: parent was not a `Struct` type - %s", parent.debug_str() ); - return CodeInvalid; + if ( p.parent && p.parent->Type != CT_Typename ) { + log_failure( "gen::def_struct: parent was not a `Struct` type - %s", code_debug_str(p.parent) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( body && body->Type != Struct_Body ) - { - log_failure( "gen::def_struct: body was not a Struct_Body type - %s", body.debug_str() ); - return CodeInvalid; + if ( p.body && p.body->Type != CT_Struct_Body ) { + log_failure( "gen::def_struct: body was not a Struct_Body type - %s", code_debug_str(p.body) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } CodeStruct result = (CodeStruct) make_code(); - result->ModuleFlags = mflags; + result->ModuleFlags = p.mflags; + if ( name.Len ) + result->Name = cache_str( name ); - if ( name ) - result->Name = get_cached_string( name ); - - if ( body ) - { - result->Type = Struct; - result->Body = body; + if ( p.body ) { + result->Type = CT_Struct; + result->Body = p.body; } - else - { - result->Type = Struct_Fwd; + else { + result->Type = CT_Struct_Fwd; } + result->Attributes = p.attributes; + result->ParentAccess = p.parent_access; + result->ParentType = p.parent; - if ( attributes ) - result->Attributes = attributes; - - if ( parent ) - { - result->ParentAccess = parent_access; - result->ParentType = parent; + for (s32 idx = 0; idx < p.num_interfaces; idx++ ) { + struct_add_interface(result, p.interfaces[idx] ); } - - if ( interfaces ) - { - for (s32 idx = 0; idx < num_interfaces; idx++ ) - { - result.add_interface( interfaces[idx] ); - } - } - return result; } -CodeTemplate def_template( CodeParam params, Code declaration, ModuleFlag mflags ) +CodeTemplate def_template( CodeParams params, Code declaration, Opts_def_template p ) { - null_check( def_template, declaration ); - - if ( params && params->Type != ECode::Parameters ) - { - log_failure( "gen::def_template: params is not of parameters type - %s", params.debug_str() ); - return CodeInvalid; + if ( ! null_check( def_template, declaration ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( params && params->Type != CT_Parameters ){ + log_failure( "gen::def_template: params is not of parameters type - %s", code_debug_str(params) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - switch (declaration->Type ) { - case ECode::Class: - case ECode::Function: - case ECode::Struct: - case ECode::Variable: - case ECode::Using: + case CT_Class: + case CT_Function: + case CT_Struct: + case CT_Variable: + case CT_Using: break; default: - log_failure( "gen::def_template: declaration is not of class, function, struct, variable, or using type - %s", declaration.debug_str() ); + log_failure( "gen::def_template: declaration is not of class, function, struct, variable, or using type - %s", code_debug_str(declaration) ); } - CodeTemplate result = (CodeTemplate) make_code(); - result->Type = ECode::Template; - result->ModuleFlags = mflags; + result->Type = CT_Template; + result->ModuleFlags = p.mflags; result->Params = params; result->Declaration = declaration; - return result; } -CodeType def_type( StrC name, Code arrayexpr, CodeSpecifiers specifiers, CodeAttributes attributes ) +CodeTypename def_type( Str name, Opts_def_type p ) { - name_check( def_type, name ); - - if ( attributes && attributes->Type != ECode::PlatformAttributes ) - { - log_failure( "gen::def_type: attributes is not of attributes type - %s", attributes.debug_str() ); - return CodeInvalid; + if ( ! name_check( def_type, name )) { + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( specifiers && specifiers->Type != ECode::Specifiers ) - { - log_failure( "gen::def_type: specifiers is not of specifiers type - %s", specifiers.debug_str() ); - return CodeInvalid; + Code arrayexpr = p.arrayexpr; + CodeSpecifiers specifiers = p.specifiers; + CodeAttributes attributes = p.attributes; + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_type: attributes is not of attributes type - %s", code_debug_str((Code)p.attributes) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( arrayexpr && arrayexpr->Type != ECode::Untyped ) - { - log_failure( "gen::def_type: arrayexpr is not of untyped type - %s", arrayexpr->debug_str() ); - return CodeInvalid; + if ( p.specifiers && p.specifiers->Type != CT_Specifiers ) { + log_failure( "gen::def_type: specifiers is not of specifiers type - %s", code_debug_str((Code)p.specifiers) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - CodeType - result = (CodeType) make_code(); - result->Name = get_cached_string( name ); - result->Type = ECode::Typename; - - if ( attributes ) - result->Attributes = attributes; - - if ( specifiers ) - result->Specs = specifiers; - - if ( arrayexpr ) - result->ArrExpr = arrayexpr; - + if ( p.arrayexpr && p.arrayexpr->Type != CT_Untyped ) { + log_failure( "gen::def_type: arrayexpr is not of untyped type - %s", code_debug_str((Code)p.arrayexpr) ); + GEN_DEBUG_TRAP(); + return InvalidCode; + } + CodeTypename + result = (CodeTypename) make_code(); + result->Name = cache_str( name ); + result->Type = CT_Typename; + result->Attributes = p.attributes; + result->Specs = p.specifiers; + result->ArrExpr = p.arrayexpr; + result->TypeTag = p.type_tag; return result; } -CodeTypedef def_typedef( StrC name, Code type, CodeAttributes attributes, ModuleFlag mflags ) +CodeTypedef def_typedef( Str name, Code type, Opts_def_typedef p ) { - using namespace ECode; - - null_check( def_typedef, type ); - + if ( ! null_check( def_typedef, type ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } switch ( type->Type ) { - case Class: - case Class_Fwd: - case Enum: - case Enum_Fwd: - case Enum_Class: - case Enum_Class_Fwd: - case Function_Fwd: - case Struct: - case Struct_Fwd: - case Union: - case Typename: + case CT_Class: + case CT_Class_Fwd: + case CT_Enum: + case CT_Enum_Fwd: + case CT_Enum_Class: + case CT_Enum_Class_Fwd: + case CT_Function_Fwd: + case CT_Struct: + case CT_Struct_Fwd: + case CT_Union: + case CT_Typename: break; default: - log_failure( "gen::def_typedef: type was not a Class, Enum, Function Forward, Struct, Typename, or Union - %s", type.debug_str() ); - return CodeInvalid; + log_failure( "gen::def_typedef: type was not a Class, Enum, Function Forward, Struct, Typename, or Union - %s", code_debug_str((Code)type) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( attributes && attributes->Type != ECode::PlatformAttributes ) - { - log_failure( "gen::def_typedef: attributes was not a PlatformAttributes - %s", attributes.debug_str() ); - return CodeInvalid; + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_typedef: attributes was not a PlatformAttributes - %s", code_debug_str((Code)p.attributes) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } // Registering the type. - Code registered_type = def_type( name ); - - if ( ! registered_type ) - { + CodeTypename registered_type = def_type( name ); + if ( ! registered_type ) { log_failure( "gen::def_typedef: failed to register type" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } CodeTypedef - result = (CodeTypedef) make_code(); - result->Type = ECode::Typedef; - result->ModuleFlags = mflags; - + result = (CodeTypedef) make_code(); + result->Type = CT_Typedef; + result->ModuleFlags = p.mflags; result->UnderlyingType = type; if ( name.Len <= 0 ) { - if (type->Type != Untyped) - { - log_failure( "gen::def_typedef: name was empty and type was not untyped (indicating its a function typedef) - %s", type.debug_str() ); - return CodeInvalid; + if (type->Type != CT_Untyped) { + log_failure( "gen::def_typedef: name was empty and type was not untyped (indicating its a function typedef) - %s", code_debug_str(type) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - - result->Name = get_cached_string( type->Name ); + result->Name = cache_str( type->Name ); result->IsFunction = true; } else { - result->Name = get_cached_string( name ); + result->Name = cache_str( name ); result->IsFunction = false; } - return result; } -CodeUnion def_union( StrC name, Code body, CodeAttributes attributes, ModuleFlag mflags ) +CodeUnion def_union( Str name, CodeBody body, Opts_def_union p ) { - null_check( def_union, body ); - - if ( body->Type != ECode::Union_Body ) - { - log_failure( "gen::def_union: body was not a Union_Body type - %s", body.debug_str() ); - return CodeInvalid; + if ( ! null_check( def_union, body ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( attributes && attributes->Type != ECode::PlatformAttributes ) - { - log_failure( "gen::def_union: attributes was not a PlatformAttributes type - %s", attributes.debug_str() ); - return CodeInvalid; + if ( body->Type != CT_Union_Body ) { + log_failure( "gen::def_union: body was not a Union_Body type - %s", code_debug_str(body) ); + GEN_DEBUG_TRAP(); + return InvalidCode; + } + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_union: attributes was not a PlatformAttributes type - %s", code_debug_str(p.attributes) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - CodeUnion result = (CodeUnion) make_code(); - result->ModuleFlags = mflags; - result->Type = ECode::Union; - + result->ModuleFlags = p.mflags; + result->Type = CT_Union; + result->Body = body; + result->Attributes = p.attributes; if ( name.Ptr ) - result->Name = get_cached_string( name ); - - result->Body = body; - - if ( attributes ) - result->Attributes = attributes; - + result->Name = cache_str( name ); return result; } -CodeUsing def_using( StrC name, CodeType type - , CodeAttributes attributes - , ModuleFlag mflags ) +CodeUsing def_using( Str name, CodeTypename type, Opts_def_using p ) { - name_check( def_using, name ); - null_check( def_using, type ); + if ( ! name_check( def_using, name ) || null_check( def_using, type ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } - Code register_type = def_type( name ); - - if ( ! register_type ) - { + CodeTypename register_type = def_type( name ); + if ( ! register_type ) { log_failure( "gen::def_using: failed to register type" ); - return CodeInvalid; + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( attributes && attributes->Type != ECode::PlatformAttributes ) - { - log_failure( "gen::def_using: attributes was not a PlatformAttributes type - %s", attributes.debug_str() ); - return CodeInvalid; + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { + log_failure( "gen::def_using: attributes was not a PlatformAttributes type - %s", code_debug_str(p.attributes) ); + GEN_DEBUG_TRAP(); + return InvalidCode; } - CodeUsing - result = (CodeUsing) make_code(); - result->Name = get_cached_string( name ); - result->ModuleFlags = mflags; - result->Type = ECode::Using; - + result = (CodeUsing) make_code(); + result->Name = cache_str( name ); + result->ModuleFlags = p.mflags; + result->Type = CT_Using; result->UnderlyingType = type; - - if ( attributes ) - result->Attributes = attributes; - + result->Attributes = p.attributes; return result; } -CodeUsing def_using_namespace( StrC name ) +CodeUsing def_using_namespace( Str name ) { - name_check( def_using_namespace, name ); - - Code - result = make_code(); - result->Name = get_cached_string( name ); - result->Content = result->Name; - result->Type = ECode::Using_Namespace; - - return (CodeUsing) result; + if ( ! name_check( def_using_namespace, name ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; + } + CodeUsing + result = (CodeUsing) make_code(); + result->Name = cache_str( name ); + result->Type = CT_Using_Namespace; + return result; } -CodeVar def_variable( CodeType type, StrC name, Code value - , CodeSpecifiers specifiers, CodeAttributes attributes - , ModuleFlag mflags ) +CodeVar def_variable( CodeTypename type, Str name, Opts_def_variable p ) { - name_check( def_variable, name ); - null_check( def_variable, type ); - - if ( attributes && attributes->Type != ECode::PlatformAttributes ) - { - log_failure( "gen::def_variable: attributes was not a `PlatformAttributes` type - %s", attributes.debug_str() ); - return CodeInvalid; + if ( ! name_check( def_variable, name ) || null_check( def_variable, type ) ) { + GEN_DEBUG_TRAP(); + return InvalidCode; } - - if ( specifiers && specifiers->Type != ECode::Specifiers ) + if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) { - log_failure( "gen::def_variable: specifiers was not a `Specifiers` type - %s", specifiers.debug_str() ); - return CodeInvalid; + log_failure( "gen::def_variable: attributes was not a `PlatformAttributes` type - %s", code_debug_str(p.attributes) ); + return InvalidCode; } - - if ( type->Type != ECode::Typename ) + if ( p.specifiers && p.specifiers->Type != CT_Specifiers ) { - log_failure( "gen::def_variable: type was not a Typename - %s", type.debug_str() ); - return CodeInvalid; + log_failure( "gen::def_variable: specifiers was not a `Specifiers` type - %s", code_debug_str(p.specifiers) ); + return InvalidCode; } - - if ( value && value->Type != ECode::Untyped ) + if ( type->Type != CT_Typename ) { - log_failure( "gen::def_variable: value was not a `Untyped` type - %s", value.debug_str() ); - return CodeInvalid; + log_failure( "gen::def_variable: type was not a Typename - %s", code_debug_str(type) ); + return InvalidCode; + } + if ( p.value && p.value->Type != CT_Untyped ) + { + log_failure( "gen::def_variable: value was not a `Untyped` type - %s", code_debug_str(p.value) ); + return InvalidCode; } - CodeVar result = (CodeVar) make_code(); - result->Name = get_cached_string( name ); - result->Type = ECode::Variable; - result->ModuleFlags = mflags; - - result->ValueType = type; - - if ( attributes ) - result->Attributes = attributes; - - if ( specifiers ) - result->Specs = specifiers; - - if ( value ) - result->Value = value; - + result->Name = cache_str( name ); + result->Type = CT_Variable; + result->ModuleFlags = p.mflags; + result->ValueType = type; + result->Attributes = p.attributes; + result->Specs = p.specifiers; + result->Value = p.value; return result; } #pragma region Helper Macros for def_**_body functions #define def_body_start( Name_ ) \ -using namespace ECode; \ - \ if ( num <= 0 ) \ { \ log_failure("gen::" stringize(Name_) ": num cannot be zero or negative"); \ - return CodeInvalid; \ + return InvalidCode; \ } #define def_body_code_array_start( Name_ ) \ -using namespace ECode; \ - \ if ( num <= 0 ) \ { \ log_failure("gen::" stringize(Name_) ": num cannot be zero or negative"); \ - return CodeInvalid; \ + return InvalidCode; \ } \ \ if ( codes == nullptr ) \ { \ log_failure("gen::" stringize(Name_)" : Provided a null array of codes"); \ - return CodeInvalid; \ + return InvalidCode; \ } #pragma endregion Helper Macros for def_**_body functions @@ -4844,7 +4873,7 @@ CodeBody def_class_body( s32 num, ... ) def_body_start( def_class_body ); CodeBody result = ( CodeBody )make_code(); - result->Type = Class_Body; + result->Type = CT_Class_Body; va_list va; va_start( va, num ); @@ -4852,26 +4881,22 @@ CodeBody def_class_body( s32 num, ... ) { Code_POD pod = va_arg(va, Code_POD); Code entry = pcast(Code, pod); - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_class_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_CLASS_UNALLOWED_TYPES - log_failure("gen::" "def_class_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_CLASS_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_class_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); va_end(va); @@ -4885,43 +4910,99 @@ CodeBody def_class_body( s32 num, Code* codes ) CodeBody result = (CodeBody) make_code(); - result->Type = Function_Body; - + result->Type = CT_Function_Body; do { Code entry = *codes; codes++; - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_class_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_CLASS_UNALLOWED_TYPES - log_failure("gen::" "def_class_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_CLASS_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_class_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); return result; } +CodeDefineParams def_define_params( s32 num, ... ) +{ + def_body_start( def_define_params ); + + va_list va; + va_start(va, num); + + Code_POD pod = va_arg(va, Code_POD); + CodeDefineParams param = pcast( CodeDefineParams, pod ); + + null_check( def_define_params, param ); + if ( param->Type != CT_Parameters_Define ) { + log_failure( "gen::def_define_params: param %d is not a parameter for a preprocessor define", num - num + 1 ); + return InvalidCode; + } + + CodeDefineParams result = (CodeDefineParams) code_duplicate(param); + while ( -- num ) + { + pod = va_arg(va, Code_POD); + param = pcast( CodeDefineParams, pod ); + if ( param->Type != CT_Parameters_Define ) { + log_failure( "gen::def_define_params: param %d is not a parameter for a preprocessor define", num - num + 1 ); + return InvalidCode; + } + define_params_append(result, param ); + } + va_end(va); + + return result; +} + +CodeDefineParams def_define_params( s32 num, CodeDefineParams* codes ) +{ + def_body_code_array_start( def_define_params ); + +# define check_current(current) \ + if ( current == nullptr ) { \ + log_failure("gen::def_define_params: Provide a null code in codes array"); \ + return InvalidCode; \ + } \ + if (current->Type != CT_Parameters_Define ) { \ + log_failure("gen::def_define_params: Code in coes array is not of paramter for preprocessor define type - %s", code_debug_str(current) ); \ + return InvalidCode; \ + } + CodeDefineParams current = (CodeDefineParams)code_duplicate(* codes); + check_current(current); + + CodeDefineParams + result = (CodeDefineParams) make_code(); + result->Name = current->Name; + result->Type = current->Type; + while( codes++, current = * codes, num--, num > 0 ) { + check_current(current); + define_params_append(result, current ); + } +# undef check_current + + return result; +} + CodeBody def_enum_body( s32 num, ... ) { def_body_start( def_enum_body ); CodeBody result = (CodeBody) make_code(); - result->Type = Enum_Body; + result->Type = CT_Enum_Body; va_list va; va_start(va, num); @@ -4929,20 +5010,15 @@ CodeBody def_enum_body( s32 num, ... ) { Code_POD pod = va_arg(va, Code_POD); Code entry = pcast(Code, pod); - - if ( ! entry ) - { + if ( ! entry ) { log_failure("gen::def_enum_body: Provided a null entry"); - return CodeInvalid; + return InvalidCode; } - - if ( entry->Type != Untyped && entry->Type != Comment ) - { - log_failure("gen::def_enum_body: Entry type is not allowed - %s. Must be of untyped or comment type.", entry.debug_str() ); - return CodeInvalid; + if ( entry->Type != CT_Untyped && entry->Type != CT_Comment ) { + log_failure("gen::def_enum_body: Entry type is not allowed - %s. Must be of untyped or comment type.", code_debug_str(entry) ); + return InvalidCode; } - - result.append( entry ); + body_append(result, entry ); } while ( num--, num > 0 ); va_end(va); @@ -4956,25 +5032,19 @@ CodeBody def_enum_body( s32 num, Code* codes ) CodeBody result = (CodeBody) make_code(); - result->Type = Enum_Body; - + result->Type = CT_Enum_Body; do { Code entry = *codes; - - if ( ! entry ) - { + if ( ! entry ) { log_failure("gen::def_enum_body: Provided a null entry"); - return CodeInvalid; + return InvalidCode; } - - if ( entry->Type != Untyped && entry->Type != Comment ) - { - log_failure("gen::def_enum_body: Entry type is not allowed: %s", entry.debug_str() ); - return CodeInvalid; + if ( entry->Type != CT_Untyped && entry->Type != CT_Comment ) { + log_failure("gen::def_enum_body: Entry type is not allowed: %s", code_debug_str(entry) ); + return InvalidCode; } - - result.append( entry ); + body_append(result, entry ); } while ( codes++, num--, num > 0 ); @@ -4987,7 +5057,7 @@ CodeBody def_export_body( s32 num, ... ) CodeBody result = (CodeBody) make_code(); - result->Type = Export_Body; + result->Type = CT_Export_Body; va_list va; va_start(va, num); @@ -4995,24 +5065,21 @@ CodeBody def_export_body( s32 num, ... ) { Code_POD pod = va_arg(va, Code_POD); Code entry = pcast(Code, pod); - - if (!entry) + if ( ! entry) { log_failure("gen::" "def_export_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_EXPORT_UNALLOWED_TYPES - log_failure("gen::" "def_export_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_EXPORT_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_export_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); va_end(va); @@ -5026,30 +5093,25 @@ CodeBody def_export_body( s32 num, Code* codes ) CodeBody result = (CodeBody) make_code(); - result->Type = Export_Body; - + result->Type = CT_Export_Body; do { Code entry = *codes; codes++; - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_export_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_EXPORT_UNALLOWED_TYPES - log_failure("gen::" "def_export_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_EXPORT_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_export_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); @@ -5062,32 +5124,28 @@ CodeBody def_extern_link_body( s32 num, ... ) CodeBody result = (CodeBody) make_code(); - result->Type = Extern_Linkage_Body; + result->Type = CT_Extern_Linkage_Body; va_list va; va_start(va, num); do { - Code_POD pod = va_arg(va, Code_POD); - Code entry = pcast(Code, pod); - - if (!entry) - { + Code_POD pod = va_arg(va, Code_POD); + Code entry = pcast(Code, pod); + if ( ! entry) { log_failure("gen::" "def_extern_linkage_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_EXTERN_LINKAGE_UNALLOWED_TYPES - log_failure("gen::" "def_extern_linkage_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_EXTERN_LINKAGE_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_extern_linkage_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); va_end(va); @@ -5101,31 +5159,26 @@ CodeBody def_extern_link_body( s32 num, Code* codes ) CodeBody result = (CodeBody) make_code(); - result->Type = Extern_Linkage_Body; - + result->Type = CT_Extern_Linkage_Body; do { Code entry = *codes; codes++; - if (!entry) { log_failure("gen::" "def_extern_linkage_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_EXTERN_LINKAGE_UNALLOWED_TYPES - log_failure("gen::" "def_extern_linkage_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_EXTERN_LINKAGE_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_extern_linkage_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); - + body_append(result, entry); } while (num--, num > 0); @@ -5138,7 +5191,7 @@ CodeBody def_function_body( s32 num, ... ) CodeBody result = (CodeBody) make_code(); - result->Type = Function_Body; + result->Type = CT_Function_Body; va_list va; va_start(va, num); @@ -5146,25 +5199,20 @@ CodeBody def_function_body( s32 num, ... ) { Code_POD pod = va_arg(va, Code_POD); Code entry = pcast(Code, pod); - - if (!entry) - { + if ( ! entry) { log_failure("gen::" stringize(def_function_body) ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - - GEN_AST_BODY_FUNCTION_UNALLOWED_TYPES - log_failure("gen::" stringize(def_function_body) ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_FUNCTION_UNALLOWED_TYPES_CASES: + log_failure("gen::" stringize(def_function_body) ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); va_end(va); @@ -5178,29 +5226,25 @@ CodeBody def_function_body( s32 num, Code* codes ) CodeBody result = (CodeBody) make_code(); - result->Type = Function_Body; - + result->Type = CT_Function_Body; do { Code entry = *codes; codes++; - - if (!entry) - { + if (!entry) { log_failure("gen::" "def_function_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_FUNCTION_UNALLOWED_TYPES - log_failure("gen::" "def_function_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_FUNCTION_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_function_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - result.append(entry); + body_append(result, entry); } while (num--, num > 0); @@ -5213,7 +5257,7 @@ CodeBody def_global_body( s32 num, ... ) CodeBody result = (CodeBody) make_code(); - result->Type = Global_Body; + result->Type = CT_Global_Body; va_list va; va_start(va, num); @@ -5221,28 +5265,25 @@ CodeBody def_global_body( s32 num, ... ) { Code_POD pod = va_arg(va, Code_POD); Code entry = pcast(Code, pod); - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_global_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - case Global_Body: - result.append( entry.cast() ) ; + case CT_Global_Body: + // result.body_append( entry.code_cast() ) ; + body_append_body( result, cast(CodeBody, entry) ); continue; - GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES - log_failure("gen::" "def_global_body" ": Entry type is not allowed: %s", entry.debug_str()); - return (*Code::Invalid.ast); + GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_global_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); va_end(va); @@ -5256,34 +5297,30 @@ CodeBody def_global_body( s32 num, Code* codes ) CodeBody result = (CodeBody) make_code(); - result->Type = Global_Body; - + result->Type = CT_Global_Body; do { Code entry = *codes; codes++; - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_global_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - case Global_Body: - result.append( entry.cast() ) ; + case CT_Global_Body: + body_append_body(result, cast(CodeBody, entry) ); continue; - GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES - log_failure("gen::" "def_global_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_GLOBAL_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_global_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - result.append(entry); + body_append(result, entry); } while (num--, num > 0); @@ -5296,7 +5333,7 @@ CodeBody def_namespace_body( s32 num, ... ) CodeBody result = (CodeBody) make_code(); - result->Type = Namespace_Body; + result->Type = CT_Namespace_Body; va_list va; va_start(va, num); @@ -5304,24 +5341,20 @@ CodeBody def_namespace_body( s32 num, ... ) { Code_POD pod = va_arg(va, Code_POD); Code entry = pcast(Code, pod); - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_namespace_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_NAMESPACE_UNALLOWED_TYPES - log_failure("gen::" "def_namespace_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_NAMESPACE_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_namespace_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); va_end(va); @@ -5335,36 +5368,31 @@ CodeBody def_namespace_body( s32 num, Code* codes ) CodeBody result = (CodeBody) make_code(); - result->Type = Global_Body; - + result->Type = CT_Global_Body; do { Code entry = *codes; codes++; - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_namespace_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_NAMESPACE_UNALLOWED_TYPES - log_failure("gen::" "def_namespace_body" ": Entry type is not allowed: %s", entry.debug_str() ); - return CodeInvalid; + GEN_AST_BODY_NAMESPACE_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_namespace_body" ": Entry type is not allowed: %s", code_debug_str(entry) ); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); return result; } -CodeParam def_params( s32 num, ... ) +CodeParams def_params( s32 num, ... ) { def_body_start( def_params ); @@ -5372,66 +5400,54 @@ CodeParam def_params( s32 num, ... ) va_start(va, num); Code_POD pod = va_arg(va, Code_POD); - CodeParam param = pcast( CodeParam, pod ); + CodeParams param = pcast( CodeParams, pod ); null_check( def_params, param ); - - if ( param->Type != Parameters ) - { + if ( param->Type != CT_Parameters ) { log_failure( "gen::def_params: param %d is not a Parameters", num - num + 1 ); - return CodeInvalid; + return InvalidCode; } - CodeParam result = (CodeParam) param.duplicate(); - + CodeParams result = (CodeParams) code_duplicate(param); while ( -- num ) { pod = va_arg(va, Code_POD); - param = pcast( CodeParam, pod ); - - if ( param->Type != Parameters ) - { + param = pcast( CodeParams, pod ); + if ( param->Type != CT_Parameters ) { log_failure( "gen::def_params: param %d is not a Parameters", num - num + 1 ); - return CodeInvalid; + return InvalidCode; } - - result.append( param ); + params_append(result, param ); } va_end(va); return result; } -CodeParam def_params( s32 num, CodeParam* codes ) +CodeParams def_params( s32 num, CodeParams* codes ) { def_body_code_array_start( def_params ); -# define check_current() \ - if ( current.ast == nullptr ) \ - { \ - log_failure("gen::def_params: Provide a null code in codes array"); \ - return CodeInvalid; \ - } \ - \ - if (current->Type != Parameters ) \ - { \ - log_failure("gen::def_params: Code in coes array is not of paramter type - %s", current.debug_str() ); \ - return CodeInvalid; \ +# define check_current(current) \ + if ( current == nullptr ) { \ + log_failure("gen::def_params: Provide a null code in codes array"); \ + return InvalidCode; \ + } \ + if (current->Type != CT_Parameters ) { \ + log_failure("gen::def_params: Code in coes array is not of paramter type - %s", code_debug_str(current) ); \ + return InvalidCode; \ } + CodeParams current = (CodeParams)code_duplicate(* codes); + check_current(current); - CodeParam current = (CodeParam) codes->duplicate(); - check_current(); - - CodeParam - result = (CodeParam) make_code(); + CodeParams + result = (CodeParams) make_code(); result->Name = current->Name; result->Type = current->Type; result->ValueType = current->ValueType; - - while( codes++, current = * codes, num--, num > 0 ) - { - check_current(); - result.append( current ); + while( codes++, current = * codes, num--, num > 0 ) { + check_current(current); + params_append(result, current ); } # undef check_current @@ -5440,29 +5456,23 @@ CodeParam def_params( s32 num, CodeParam* codes ) CodeSpecifiers def_specifiers( s32 num, ... ) { - if ( num <= 0 ) - { + if ( num <= 0 ) { log_failure("gen::def_specifiers: num cannot be zero or less"); - return CodeInvalid; + return InvalidCode; } - - if ( num > AST::ArrSpecs_Cap ) - { + if ( num > AST_ArrSpecs_Cap ) { log_failure("gen::def_specifiers: num of speciifers to define AST larger than AST specicifier capacity - %d", num); - return CodeInvalid; + return InvalidCode; } - CodeSpecifiers result = (CodeSpecifiers) make_code(); - result->Type = ECode::Specifiers; + result->Type = CT_Specifiers; va_list va; va_start(va, num); - do - { - SpecifierT type = (SpecifierT)va_arg(va, int); - - result.append( type ); + do { + Specifier type = (Specifier)va_arg(va, int); + specifiers_append(result, type ); } while ( --num, num ); va_end(va); @@ -5470,28 +5480,23 @@ CodeSpecifiers def_specifiers( s32 num, ... ) return result; } -CodeSpecifiers def_specifiers( s32 num, SpecifierT* specs ) +CodeSpecifiers def_specifiers( s32 num, Specifier* specs ) { - if ( num <= 0 ) - { + if ( num <= 0 ) { log_failure("gen::def_specifiers: num cannot be zero or less"); - return CodeInvalid; + return InvalidCode; } - - if ( num > AST::ArrSpecs_Cap ) - { + if ( num > AST_ArrSpecs_Cap ) { log_failure("gen::def_specifiers: num of speciifers to define AST larger than AST specicifier capacity - %d", num); - return CodeInvalid; + return InvalidCode; } - CodeSpecifiers result = (CodeSpecifiers) make_code(); - result->Type = ECode::Specifiers; + result->Type = CT_Specifiers; s32 idx = 0; - do - { - result.append( specs[idx] ); + do { + specifiers_append(result, specs[idx] ); idx++; } while ( --num, num ); @@ -5505,7 +5510,7 @@ CodeBody def_struct_body( s32 num, ... ) CodeBody result = (CodeBody) make_code(); - result->Type = Struct_Body; + result->Type = CT_Struct_Body; va_list va; va_start(va, num); @@ -5513,24 +5518,20 @@ CodeBody def_struct_body( s32 num, ... ) { Code_POD pod = va_arg(va, Code_POD); Code entry = pcast(Code, pod); - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_struct_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_STRUCT_UNALLOWED_TYPES - log_failure("gen::" "def_struct_body" ": Entry type is not allowed: %s", entry.debug_str()); - return CodeInvalid; + GEN_AST_BODY_STRUCT_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_struct_body" ": Entry type is not allowed: %s", code_debug_str(entry)); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); va_end(va); @@ -5544,30 +5545,25 @@ CodeBody def_struct_body( s32 num, Code* codes ) CodeBody result = (CodeBody) make_code(); - result->Type = Struct_Body; - + result->Type = CT_Struct_Body; do { Code entry = *codes; codes++; - - if (!entry) - { + if ( ! entry) { log_failure("gen::" "def_struct_body" ": Provided an null entry"); - return CodeInvalid; + return InvalidCode; } - switch (entry->Type) { - GEN_AST_BODY_STRUCT_UNALLOWED_TYPES - log_failure("gen::" "def_struct_body" ": Entry type is not allowed: %s", entry.debug_str() ); - return CodeInvalid; + GEN_AST_BODY_STRUCT_UNALLOWED_TYPES_CASES: + log_failure("gen::" "def_struct_body" ": Entry type is not allowed: %s", code_debug_str(entry) ); + return InvalidCode; default: break; } - - result.append(entry); + body_append(result, entry); } while (num--, num > 0); @@ -5580,7 +5576,7 @@ CodeBody def_union_body( s32 num, ... ) CodeBody result = (CodeBody) make_code(); - result->Type = Union_Body; + result->Type = CT_Union_Body; va_list va; va_start(va, num); @@ -5588,20 +5584,15 @@ CodeBody def_union_body( s32 num, ... ) { Code_POD pod = va_arg(va, Code_POD); Code entry = pcast( Code, pod ); - - if ( ! entry ) - { + if ( ! entry ) { log_failure("gen::def_union_body: Provided a null entry"); - return CodeInvalid; + return InvalidCode; } - - if ( entry->Type != Untyped && entry->Type != Comment ) - { - log_failure("gen::def_union_body: Entry type is not allowed - %s. Must be of untyped or comment type.", entry.debug_str() ); - return CodeInvalid; + if ( entry->Type != CT_Untyped && entry->Type != CT_Comment ) { + log_failure("gen::def_union_body: Entry type is not allowed - %s. Must be of untyped or comment type.", code_debug_str(entry) ); + return InvalidCode; } - - result.append( entry ); + body_append(result, entry ); } while ( num--, num > 0 ); va_end(va); @@ -5609,31 +5600,25 @@ CodeBody def_union_body( s32 num, ... ) return result; } -CodeBody def_union_body( s32 num, CodeUnion* codes ) +CodeBody def_union_body( s32 num, Code* codes ) { def_body_code_array_start( def_union_body ); CodeBody result = (CodeBody) make_code(); - result->Type = Union_Body; - + result->Type = CT_Union_Body; do { Code entry = *codes; - - if ( ! entry ) - { + if ( ! entry ) { log_failure("gen::def_union_body: Provided a null entry"); - return CodeInvalid; + return InvalidCode; } - - if ( entry->Type != Untyped && entry->Type != Comment ) - { - log_failure("gen::def_union_body: Entry type is not allowed: %s", entry.debug_str() ); - return CodeInvalid; + if ( entry->Type != CT_Untyped && entry->Type != CT_Comment ) { + log_failure("gen::def_union_body: Entry type is not allowed: %s", code_debug_str(entry) ); + return InvalidCode; } - - result.append( entry ); + body_append(result, entry ); } while ( codes++, num--, num > 0 ); @@ -5642,7 +5627,6 @@ CodeBody def_union_body( s32 num, CodeUnion* codes ) # undef name_check # undef null_check -# undef null_or_invalid_check # undef def_body_start # undef def_body_code_array_start @@ -5650,455 +5634,70 @@ CodeBody def_union_body( s32 num, CodeUnion* codes ) #pragma region Parsing -namespace parser + +StrBuilder tok_to_strbuilder(Token tok) { - namespace ETokType - { -#define GEN_DEFINE_ATTRIBUTE_TOKENS \ - Entry( Attribute_API_Export, "GEN_API_Export_Code" ) Entry( Attribute_API_Import, "GEN_API_Import_Code" ) \ - Entry( Attribute_COREUOBJECT_API, "COREUOBJECT_API" ) Entry( Attribute_ENGINE_API, "ENGINE_API" ) \ - Entry( Attribute_GAMEPLAYABILITIES_API, "GAMEPLAYABILITIES_API" ) Entry( Attribute_UMG_API, "UMG_API" ) \ - Entry( Attribute_UE_DEPRECATED, "UE_DEPRECATED" ) + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, kilobytes(4) ); + Str type_str = toktype_to_str( tok.Type ); - enum Type : u32 - { - Invalid, - Access_Private, - Access_Protected, - Access_Public, - Access_MemberSymbol, - Access_StaticSymbol, - Ampersand, - Ampersand_DBL, - Assign_Classifer, - Attribute_Open, - Attribute_Close, - BraceCurly_Open, - BraceCurly_Close, - BraceSquare_Open, - BraceSquare_Close, - Capture_Start, - Capture_End, - Comment, - Comment_End, - Comment_Start, - Char, - Comma, - Decl_Class, - Decl_GNU_Attribute, - Decl_MSVC_Attribute, - Decl_Enum, - Decl_Extern_Linkage, - Decl_Friend, - Decl_Module, - Decl_Namespace, - Decl_Operator, - Decl_Struct, - Decl_Template, - Decl_Typedef, - Decl_Using, - Decl_Union, - Identifier, - Module_Import, - Module_Export, - NewLine, - Number, - Operator, - Preprocess_Hash, - Preprocess_Define, - Preprocess_If, - Preprocess_IfDef, - Preprocess_IfNotDef, - Preprocess_ElIf, - Preprocess_Else, - Preprocess_EndIf, - Preprocess_Include, - Preprocess_Pragma, - Preprocess_Content, - Preprocess_Macro, - Preprocess_Unsupported, - Spec_Alignas, - Spec_Const, - Spec_Consteval, - Spec_Constexpr, - Spec_Constinit, - Spec_Explicit, - Spec_Extern, - Spec_Final, - Spec_ForceInline, - Spec_Global, - Spec_Inline, - Spec_Internal_Linkage, - Spec_LocalPersist, - Spec_Mutable, - Spec_NeverInline, - Spec_Override, - Spec_Static, - Spec_ThreadLocal, - Spec_Volatile, - Spec_Virtual, - Star, - Statement_End, - StaticAssert, - String, - Type_Typename, - Type_Unsigned, - Type_Signed, - Type_Short, - Type_Long, - Type_bool, - Type_char, - Type_int, - Type_double, - Type_MS_int8, - Type_MS_int16, - Type_MS_int32, - Type_MS_int64, - Type_MS_W64, - Varadic_Argument, - __Attributes_Start, - Attribute_API_Export, - Attribute_API_Import, - Attribute_COREUOBJECT_API, - Attribute_ENGINE_API, - Attribute_GAMEPLAYABILITIES_API, - Attribute_UMG_API, - Attribute_UE_DEPRECATED, - NumTokens - }; + strbuilder_append_fmt( & result, "Line: %d Column: %d, Type: %.*s Content: %.*s" + , tok.Line, tok.Column + , type_str.Len, type_str.Ptr + , tok.Text.Len, tok.Text.Ptr + ); + return result; +} - inline StrC to_str( Type type ) - { - local_persist StrC lookup[] { - { sizeof( "__invalid__" ), "__invalid__" }, - { sizeof( "private" ), "private" }, - { sizeof( "protected" ), "protected" }, - { sizeof( "public" ), "public" }, - { sizeof( "." ), "." }, - { sizeof( "::" ), "::" }, - { sizeof( "&" ), "&" }, - { sizeof( "&&" ), "&&" }, - { sizeof( ":" ), ":" }, - { sizeof( "[[" ), "[[" }, - { sizeof( "]]" ), "]]" }, - { sizeof( "{" ), "{" }, - { sizeof( "}" ), "}" }, - { sizeof( "[" ), "[" }, - { sizeof( "]" ), "]" }, - { sizeof( "(" ), "(" }, - { sizeof( ")" ), ")" }, - { sizeof( "__comment__" ), "__comment__" }, - { sizeof( "__comment_end__" ), "__comment_end__" }, - { sizeof( "__comment_start__" ), "__comment_start__" }, - { sizeof( "__character__" ), "__character__" }, - { sizeof( "," ), "," }, - { sizeof( "class" ), "class" }, - { sizeof( "__attribute__" ), "__attribute__" }, - { sizeof( "__declspec" ), "__declspec" }, - { sizeof( "enum" ), "enum" }, - { sizeof( "extern" ), "extern" }, - { sizeof( "friend" ), "friend" }, - { sizeof( "module" ), "module" }, - { sizeof( "namespace" ), "namespace" }, - { sizeof( "operator" ), "operator" }, - { sizeof( "struct" ), "struct" }, - { sizeof( "template" ), "template" }, - { sizeof( "typedef" ), "typedef" }, - { sizeof( "using" ), "using" }, - { sizeof( "union" ), "union" }, - { sizeof( "__identifier__" ), "__identifier__" }, - { sizeof( "import" ), "import" }, - { sizeof( "export" ), "export" }, - { sizeof( "__new_line__" ), "__new_line__" }, - { sizeof( "__number__" ), "__number__" }, - { sizeof( "__operator__" ), "__operator__" }, - { sizeof( "#" ), "#" }, - { sizeof( "define" ), "define" }, - { sizeof( "if" ), "if" }, - { sizeof( "ifdef" ), "ifdef" }, - { sizeof( "ifndef" ), "ifndef" }, - { sizeof( "elif" ), "elif" }, - { sizeof( "else" ), "else" }, - { sizeof( "endif" ), "endif" }, - { sizeof( "include" ), "include" }, - { sizeof( "pragma" ), "pragma" }, - { sizeof( "__macro_content__" ), "__macro_content__" }, - { sizeof( "__macro__" ), "__macro__" }, - { sizeof( "__unsupported__" ), "__unsupported__" }, - { sizeof( "alignas" ), "alignas" }, - { sizeof( "const" ), "const" }, - { sizeof( "consteval" ), "consteval" }, - { sizeof( "constexpr" ), "constexpr" }, - { sizeof( "constinit" ), "constinit" }, - { sizeof( "explicit" ), "explicit" }, - { sizeof( "extern" ), "extern" }, - { sizeof( "final" ), "final" }, - { sizeof( "FORCEINLINE" ), "FORCEINLINE" }, - { sizeof( "global" ), "global" }, - { sizeof( "inline" ), "inline" }, - { sizeof( "internal" ), "internal" }, - { sizeof( "local_persist" ), "local_persist" }, - { sizeof( "mutable" ), "mutable" }, - { sizeof( "neverinline" ), "neverinline" }, - { sizeof( "override" ), "override" }, - { sizeof( "static" ), "static" }, - { sizeof( "thread_local" ), "thread_local" }, - { sizeof( "volatile" ), "volatile" }, - { sizeof( "virtual" ), "virtual" }, - { sizeof( "*" ), "*" }, - { sizeof( ";" ), ";" }, - { sizeof( "static_assert" ), "static_assert" }, - { sizeof( "__string__" ), "__string__" }, - { sizeof( "typename" ), "typename" }, - { sizeof( "unsigned" ), "unsigned" }, - { sizeof( "signed" ), "signed" }, - { sizeof( "short" ), "short" }, - { sizeof( "long" ), "long" }, - { sizeof( "bool" ), "bool" }, - { sizeof( "char" ), "char" }, - { sizeof( "int" ), "int" }, - { sizeof( "double" ), "double" }, - { sizeof( "__int8" ), "__int8" }, - { sizeof( "__int16" ), "__int16" }, - { sizeof( "__int32" ), "__int32" }, - { sizeof( "__int64" ), "__int64" }, - { sizeof( "_W64" ), "_W64" }, - { sizeof( "..." ), "..." }, - { sizeof( "__attrib_start__" ), "__attrib_start__" }, - { sizeof( "GEN_API_Export_Code" ), "GEN_API_Export_Code" }, - { sizeof( "GEN_API_Import_Code" ), "GEN_API_Import_Code" }, - { sizeof( "COREUOBJECT_API" ), "COREUOBJECT_API" }, - { sizeof( "ENGINE_API" ), "ENGINE_API" }, - { sizeof( "GAMEPLAYABILITIES_API" ), "GAMEPLAYABILITIES_API" }, - { sizeof( "UMG_API" ), "UMG_API" }, - { sizeof( "UE_DEPRECATED" ), "UE_DEPRECATED" }, - }; - return lookup[type]; - } +bool lex__eat( TokArray* self, TokType type ); - inline Type to_type( StrC str ) - { - local_persist u32 keymap[NumTokens]; - do_once_start for ( u32 index = 0; index < NumTokens; index++ ) - { - StrC enum_str = to_str( (Type)index ); - keymap[index] = crc32( enum_str.Ptr, enum_str.Len - 1 ); - } - do_once_end u32 hash = crc32( str.Ptr, str.Len ); - for ( u32 index = 0; index < NumTokens; index++ ) - { - if ( keymap[index] == hash ) - return (Type)index; - } - return Invalid; - } - - } // namespace ETokType - - using TokType = ETokType::Type; - -} // namespace parser - -namespace parser { - -enum TokFlags : u32 +Token* lex_current(TokArray* self, bool skip_formatting ) { - TF_Operator = bit(0), - TF_Assign = bit(1), - TF_Preprocess = bit(2), - TF_Preprocess_Cond = bit(3), - TF_Attribute = bit(6), - TF_AccessOperator = bit( 7 ), - TF_AccessSpecifier = bit( 8 ), - TF_Specifier = bit( 9 ), - TF_EndDefinition = bit( 10 ), // Either ; or } - TF_Formatting = bit( 11 ), - TF_Literal = bit( 12 ), + if ( skip_formatting ) + { + while ( self->Arr[self->Idx].Type == Tok_NewLine || self->Arr[self->Idx].Type == Tok_Comment ) + self->Idx++; + } + return & self->Arr[self->Idx]; +} - TF_Null = 0, -}; - -struct Token +Token* lex_peek(TokArray self, bool skip_formatting) { - char const* Text; - sptr Length; - TokType Type; - s32 Line; - s32 Column; - u32 Flags; - - operator bool() + s32 idx = self.Idx; + if ( skip_formatting ) { - return Text && Length && Type != TokType::Invalid; + while ( self.Arr[idx].Type == Tok_NewLine ) + idx++; + + return & self.Arr[idx]; } + return & self.Arr[idx]; +} - operator StrC() - { - return { Length, Text }; - } - - bool is_access_operator() - { - return bitfield_is_equal( u32, Flags, TF_AccessOperator ); - } - - bool is_access_specifier() - { - return bitfield_is_equal( u32, Flags, TF_AccessSpecifier ); - } - - bool is_attribute() - { - return bitfield_is_equal( u32, Flags, TF_Attribute ); - } - - bool is_operator() - { - return bitfield_is_equal( u32, Flags, TF_Operator ); - } - - bool is_preprocessor() - { - return bitfield_is_equal( u32, Flags, TF_Preprocess ); - } - - bool is_preprocess_cond() - { - return bitfield_is_equal( u32, Flags, TF_Preprocess_Cond ); - } - - bool is_specifier() - { - return bitfield_is_equal( u32, Flags, TF_Specifier ); - } - - bool is_end_definition() - { - return bitfield_is_equal( u32, Flags, TF_EndDefinition ); - } - - AccessSpec to_access_specifier() - { - return scast(AccessSpec, Type); - } - - String to_string() - { - String result = String::make_reserve( GlobalAllocator, kilobytes(4) ); - - StrC type_str = ETokType::to_str( Type ); - - result.append_fmt( "Line: %d Column: %d, Type: %.*s Content: %.*s" - , Line, Column - , type_str.Len, type_str.Ptr - , Length, Text - ); - - return result; - } -}; - -constexpr Token NullToken { nullptr, 0, TokType::Invalid, false, 0, TF_Null }; - -struct TokArray +Token* lex_previous(TokArray self, bool skip_formatting) { - Array Arr; - s32 Idx; - - bool __eat( TokType type ); - - Token& current( bool skip_formatting = true ) + s32 idx = self.Idx; + if ( skip_formatting ) { - if ( skip_formatting ) - { - while ( Arr[Idx].Type == TokType::NewLine || Arr[Idx].Type == TokType::Comment ) - Idx++; - } + while ( self.Arr[idx].Type == Tok_NewLine ) + idx --; - return Arr[Idx]; + return & self.Arr[idx]; } + return & self.Arr[idx - 1]; +} - Token& previous( bool skip_formatting = false ) +Token* lex_next(TokArray self, bool skip_formatting) +{ + s32 idx = self.Idx; + if ( skip_formatting ) { - s32 idx = this->Idx; + while ( self.Arr[idx].Type == Tok_NewLine ) + idx++; - if ( skip_formatting ) - { - while ( Arr[idx].Type == TokType::NewLine ) - idx--; - - return Arr[idx]; - } - - return Arr[idx - 1]; + return & self.Arr[idx + 1]; } - - Token& next( bool skip_formatting = false ) - { - s32 idx = this->Idx; - - if ( skip_formatting ) - { - while ( Arr[idx].Type == TokType::NewLine ) - idx++; - - return Arr[idx + 1]; - } - - return Arr[idx + 1]; - } - - Token& operator []( s32 idx ) - { - return Arr[idx]; - } -}; - -global Arena_256KB defines_map_arena; -global HashTable defines; -global Array Tokens; - -#define current ( * scanner ) - -#define move_forward() \ - { \ - if ( current == '\n' ) \ - { \ - line++; \ - column = 1; \ - } \ - else \ - { \ - column++; \ - } \ - left--; \ - scanner++; \ - } - -#define SkipWhitespace() \ - while ( left && char_is_space( current ) ) \ - { \ - move_forward(); \ - } - -#define end_line() \ - do \ - { \ - while ( left && current == ' ' ) \ - { \ - move_forward(); \ - } \ - if ( left && current == '\r' ) \ - { \ - move_forward(); \ - move_forward(); \ - } \ - else if ( left && current == '\n' ) \ - { \ - move_forward(); \ - } \ - } \ - while (0) + return & self.Arr[idx + 1]; +} enum { @@ -6107,475 +5706,627 @@ enum }; FORCEINLINE -s32 lex_preprocessor_directive( - StrC& content - , s32& left - , char const*& scanner - , s32& line - , s32& column - , HashTable& defines - , Token& token ) +void lexer_move_forward( LexContext* ctx ) { - char const* hash = scanner; - Tokens.append( { hash, 1, TokType::Preprocess_Hash, line, column, TF_Preprocess } ); + if ( * ctx->scanner == '\n' ) { + ctx->line += 1; + ctx->column = 1; + } + else { + ++ ctx->column; + } + -- ctx->left; + ++ ctx->scanner; +} +#define move_forward() lexer_move_forward(ctx) - move_forward(); - SkipWhitespace(); - - token.Text = scanner; - while (left && ! char_is_space(current) ) - { +FORCEINLINE +void lexer_skip_whitespace( LexContext* ctx ) +{ + while ( ctx->left && char_is_space( * ctx->scanner ) ) move_forward(); - token.Length++; +} +#define skip_whitespace() lexer_skip_whitespace(ctx) + +FORCEINLINE +void lexer_end_line( LexContext* ctx ) +{ + while ( ctx->left && (* ctx->scanner) == ' ' ) + move_forward(); + + if ( ctx->left && (* ctx->scanner) == '\r' ) { + move_forward(); + move_forward(); + } + else if ( ctx->left && (* ctx->scanner) == '\n' ) + move_forward(); +} +#define end_line() lexer_end_line(ctx) + +// TODO(Ed): We need to to attempt to recover from a lex failure? +s32 lex_preprocessor_define( LexContext* ctx ) +{ + Token name = { { ctx->scanner, 1 }, Tok_Identifier, ctx->line, ctx->column, TF_Preprocess }; + move_forward(); + + while ( ctx->left && ( char_is_alphanumeric((* ctx->scanner)) || (* ctx->scanner) == '_' ) ) { + move_forward(); + name.Text.Len++; } - token.Type = ETokType::to_type( token ); + Specifier spec = str_to_specifier( name.Text ); + TokType attrib = str_to_toktype( name.Text ); + b32 not_specifier = spec == Spec_Invalid; + b32 not_attribute = attrib <= Tok___Attributes_Start; - bool is_preprocessor = token.Type >= TokType::Preprocess_Define && token.Type <= TokType::Preprocess_Pragma; + Macro macro = { name.Text, MT_Expression, (MacroFlags)0 }; + Macro* registered_macro = lookup_macro(name.Text); + + if ( registered_macro == nullptr && not_specifier && not_attribute ) { + log_fmt("Warning: '%S' was not registered before the lexer processed its #define directive, it will be registered as a expression macro\n" + , name.Text + ); + // GEN_DEBUG_TRAP(); + } + array_append( _ctx->Lexer_Tokens, name ); + + if ( ctx->left && (* ctx->scanner) == '(' ) + { + if (registered_macro && ! macro_is_functional(* registered_macro)) { + log_fmt("Warning: %S registered macro is not flagged as functional yet the definition detects opening parenthesis '(' for arguments\n" + , name.Text + ); + // GEN_DEBUG_TRAP(); + } + else { + macro.Flags |= MF_Functional; + } + + Token opening_paren = { { ctx->scanner, 1 }, Tok_Capture_Start, ctx->line, ctx->column, TF_Preprocess }; + array_append( _ctx->Lexer_Tokens, opening_paren ); + move_forward(); + + Token last_parameter; + // We need to tokenize the define's arguments now: + while( ctx->left && * ctx->scanner != ')') + { + skip_whitespace(); + + Str possible_varadic = { ctx->scanner, 3 }; + if ( ctx->left > 3 && str_are_equal( txt("..."), possible_varadic ) ) { + Token parameter = { { ctx->scanner, 3 }, Tok_Preprocess_Define_Param, ctx->line, ctx->column, TF_Preprocess }; + move_forward(); + move_forward(); + move_forward(); + + array_append(_ctx->Lexer_Tokens, parameter); + skip_whitespace(); + last_parameter = parameter; + + while ( (* ctx->scanner) == '\\' ) { + move_forward(); + skip_whitespace(); + } + if (* ctx->scanner != ')' ) + { + log_failure("lex_preprocessor_define(%d, %d): Expected a ')' after '...' (varaidc macro param) %S\n" + , ctx->line + , ctx->column + , name.Text + ); + return Lex_ReturnNull; + } + break; + } + else if ( (* ctx->scanner) == '\\' ) { + move_forward(); + skip_whitespace(); + continue; + } + else if ( char_is_alpha( (* ctx->scanner) ) || (* ctx->scanner) == '_' ) + { + Token parameter = { { ctx->scanner, 1 }, Tok_Preprocess_Define_Param, ctx->line, ctx->column, TF_Preprocess }; + move_forward(); + + while ( ctx->left && ( char_is_alphanumeric((* ctx->scanner)) || (* ctx->scanner) == '_' ) ) + { + move_forward(); + parameter.Text.Len++; + } + array_append(_ctx->Lexer_Tokens, parameter); + skip_whitespace(); + last_parameter = parameter; + } + else { + log_failure("lex_preprocessor_define(%d, %d): Expected a '_' or alpha character for a parameter name for %S\n" + , ctx->line + , ctx->column + , name.Text + ); + return Lex_ReturnNull; + } + + if (* ctx->scanner == ')' ) + break; + + // There should be a comma + if ( * ctx->scanner != ',' ) { + log_failure("lex_preprocessor_define(%d, %d): Expected a comma after parameter %S for %S\n" + , ctx->line + , ctx->column + , last_parameter.Text + , name.Text + ); + return Lex_ReturnNull; + } + Token comma = { { ctx->scanner, 1 }, Tok_Comma, ctx->line, ctx->column, TF_Preprocess }; + array_append(_ctx->Lexer_Tokens, comma); + move_forward(); + } + + if ( * ctx->scanner != ')' ) { + log_failure("lex_preprocessor_define(%d, %d): Expected a ')' after last_parameter %S for %S (ran out of characters...)\n" + , ctx->line + , ctx->column + , last_parameter.Text + , name.Text + ); + return Lex_ReturnNull; + } + Token closing_paren = { { ctx->scanner, 1 }, Tok_Capture_End, ctx->line, ctx->column, TF_Preprocess }; + array_append(_ctx->Lexer_Tokens, closing_paren); + move_forward(); + } + else if ( registered_macro && macro_is_functional( * registered_macro) ) { + if (registered_macro && ! macro_is_functional(* registered_macro)) { + log_fmt("Warning: %S registered macro is flagged as functional yet the definition detects no opening parenthesis '(' for arguments\n" + , name.Text + ); + GEN_DEBUG_TRAP(); + } + } + + if ( registered_macro == nullptr ) { + register_macro(macro); + } + + // Define's content handled by lex_preprocessor_directive (the original caller of this) + return Lex_Continue; +} + +// TODO(Ed): We need to to attempt to recover from a lex failure? +FORCEINLINE +s32 lex_preprocessor_directive( LexContext* ctx ) +{ + char const* hash = ctx->scanner; + Token hash_tok = { { hash, 1 }, Tok_Preprocess_Hash, ctx->line, ctx->column, TF_Preprocess }; + array_append( _ctx->Lexer_Tokens, hash_tok ); + + move_forward(); + skip_whitespace(); + + ctx->token.Text.Ptr = ctx->scanner; + while (ctx->left && ! char_is_space((* ctx->scanner)) ) + { + move_forward(); + ctx->token.Text.Len++; + } + + ctx->token.Type = str_to_toktype( tok_to_str(ctx->token) ); + + bool is_preprocessor = ctx->token.Type >= Tok_Preprocess_Define && ctx->token.Type <= Tok_Preprocess_Pragma; if ( ! is_preprocessor ) { - token.Type = TokType::Preprocess_Unsupported; + ctx->token.Type = Tok_Preprocess_Unsupported; // Its an unsupported directive, skip it s32 within_string = false; s32 within_char = false; - while ( left ) + while ( ctx->left ) { - if ( current == '"' && ! within_char ) + if ( * ctx->scanner == '"' && ! within_char ) within_string ^= true; - if ( current == '\'' && ! within_string ) + if ( * ctx->scanner == '\'' && ! within_string ) within_char ^= true; - if ( current == '\\' && ! within_string && ! within_char ) + if ( * ctx->scanner == '\\' && ! within_string && ! within_char ) { move_forward(); - token.Length++; + ctx->token.Text.Len++; - if ( current == '\r' ) + if ( (* ctx->scanner) == '\r' ) { move_forward(); - token.Length++; + ctx->token.Text.Len++; } - if ( current == '\n' ) + if ( (* ctx->scanner) == '\n' ) { move_forward(); - token.Length++; + ctx->token.Text.Len++; continue; } else { log_failure( "gen::Parser::lex: Invalid escape sequence '\\%c' (%d, %d)" " in preprocessor directive (%d, %d)\n%.100s" - , current, line, column - , token.Line, token.Column, token.Text ); + , (* ctx->scanner), ctx->line, ctx->column + , ctx->token.Line, ctx->token.Column, ctx->token.Text ); break; } } - if ( current == '\r' ) + if ( (* ctx->scanner) == '\r' ) { move_forward(); - token.Length++; + ctx->token.Text.Len++; } - if ( current == '\n' ) + if ( (* ctx->scanner) == '\n' ) { move_forward(); - token.Length++; + ctx->token.Text.Len++; break; } move_forward(); - token.Length++; + ctx->token.Text.Len++; } - token.Length = token.Length + token.Text - hash; - token.Text = hash; - Tokens.append( token ); + ctx->token.Text.Len = ctx->token.Text.Len + ctx->token.Text.Ptr - hash; + ctx->token.Text.Ptr = hash; + array_append( _ctx->Lexer_Tokens, ctx->token ); return Lex_Continue; // Skip found token, its all handled here. } - if ( token.Type == TokType::Preprocess_Else || token.Type == TokType::Preprocess_EndIf ) + if ( ctx->token.Type == Tok_Preprocess_Else || ctx->token.Type == Tok_Preprocess_EndIf ) { - token.Flags |= TF_Preprocess_Cond; - Tokens.append( token ); + ctx->token.Flags |= TF_Preprocess_Cond; + array_append( _ctx->Lexer_Tokens, ctx->token ); end_line(); return Lex_Continue; } - else if ( token.Type >= TokType::Preprocess_If && token.Type <= TokType::Preprocess_ElIf ) + else if ( ctx->token.Type >= Tok_Preprocess_If && ctx->token.Type <= Tok_Preprocess_ElIf ) { - token.Flags |= TF_Preprocess_Cond; + ctx->token.Flags |= TF_Preprocess_Cond; } - Tokens.append( token ); + array_append( _ctx->Lexer_Tokens, ctx->token ); - SkipWhitespace(); + skip_whitespace(); - if ( token.Type == TokType::Preprocess_Define ) + if ( ctx->token.Type == Tok_Preprocess_Define ) { - Token name = { scanner, 0, TokType::Identifier, line, column, TF_Preprocess }; - - name.Text = scanner; - name.Length = 1; - move_forward(); - - while ( left && ( char_is_alphanumeric(current) || current == '_' ) ) - { - move_forward(); - name.Length++; - } - - if ( left && current == '(' ) - { - move_forward(); - name.Length++; - } - - Tokens.append( name ); - - u64 key = crc32( name.Text, name.Length ); - defines.set( key, name ); + u32 result = lex_preprocessor_define(ctx); // handles: #define ( ) - define's content handled later on within this scope. + if (result != Lex_Continue) + return Lex_ReturnNull; } - Token preprocess_content = { scanner, 0, TokType::Preprocess_Content, line, column, TF_Preprocess }; + Token preprocess_content = { { ctx->scanner, 0 }, Tok_Preprocess_Content, ctx->line, ctx->column, TF_Preprocess }; - if ( token.Type == TokType::Preprocess_Include ) + if ( ctx->token.Type == Tok_Preprocess_Include ) { - preprocess_content.Type = TokType::String; + preprocess_content.Type = Tok_String; - if ( current != '"' && current != '<' ) + if ( (* ctx->scanner) != '"' && (* ctx->scanner) != '<' ) { - String directive_str = String::fmt_buf( GlobalAllocator, "%.*s", min( 80, left + preprocess_content.Length ), token.Text ); + StrBuilder directive_str = strbuilder_fmt_buf( _ctx->Allocator_Temp, "%.*s", min( 80, ctx->left + preprocess_content.Text.Len ), ctx->token.Text.Ptr ); log_failure( "gen::Parser::lex: Expected '\"' or '<' after #include, not '%c' (%d, %d)\n%s" - , current + , (* ctx->scanner) , preprocess_content.Line , preprocess_content.Column - , directive_str.Data + , (char*) directive_str ); return Lex_ReturnNull; } move_forward(); - preprocess_content.Length++; + preprocess_content.Text.Len++; - while ( left && current != '"' && current != '>' ) + while ( ctx->left && (* ctx->scanner) != '"' && (* ctx->scanner) != '>' ) { move_forward(); - preprocess_content.Length++; + preprocess_content.Text.Len++; } move_forward(); - preprocess_content.Length++; + preprocess_content.Text.Len++; - if ( current == '\r' && scanner[1] == '\n' ) + if ( (* ctx->scanner) == '\r' && ctx->scanner[1] == '\n' ) { move_forward(); move_forward(); } - else if ( current == '\n' ) + else if ( (* ctx->scanner) == '\n' ) { move_forward(); } - Tokens.append( preprocess_content ); + array_append( _ctx->Lexer_Tokens, preprocess_content ); return Lex_Continue; // Skip found token, its all handled here. } s32 within_string = false; s32 within_char = false; - // SkipWhitespace(); - while ( left ) + // Consume preprocess content + while ( ctx->left ) { - if ( current == '"' && ! within_char ) + if ( (* ctx->scanner) == '"' && ! within_char ) within_string ^= true; - if ( current == '\'' && ! within_string ) + if ( (* ctx->scanner) == '\'' && ! within_string ) within_char ^= true; - if ( current == '\\' && ! within_string && ! within_char ) + if ( (* ctx->scanner) == '\\' && ! within_string && ! within_char ) { move_forward(); - preprocess_content.Length++; + preprocess_content.Text.Len++; - if ( current == '\r' ) + if ( (* ctx->scanner) == '\r' ) { move_forward(); - preprocess_content.Length++; + preprocess_content.Text.Len++; } - if ( current == '\n' ) + if ( (* ctx->scanner) == '\n' ) { move_forward(); - preprocess_content.Length++; + preprocess_content.Text.Len++; continue; } else { - String directive_str = String::make_length( GlobalAllocator, token.Text, token.Length ); - String content_str = String::fmt_buf( GlobalAllocator, "%.*s", min( 400, left + preprocess_content.Length ), preprocess_content.Text ); + StrBuilder directive_str = strbuilder_make_length( _ctx->Allocator_Temp, ctx->token.Text.Ptr, ctx->token.Text.Len ); + StrBuilder content_str = strbuilder_fmt_buf( _ctx->Allocator_Temp, "%.*s", min( 400, ctx->left + preprocess_content.Text.Len ), preprocess_content.Text.Ptr ); log_failure( "gen::Parser::lex: Invalid escape sequence '\\%c' (%d, %d)" " in preprocessor directive '%s' (%d, %d)\n%s" - , current, line, column + , (* ctx->scanner), ctx->line, ctx->column , directive_str, preprocess_content.Line, preprocess_content.Column , content_str ); + return Lex_ReturnNull; break; } } - if ( current == '\r' ) + if ( (* ctx->scanner) == '\r' ) { - move_forward(); + break; + //move_forward(); } - if ( current == '\n' ) + if ( (* ctx->scanner) == '\n' ) { - move_forward(); + //move_forward(); break; } move_forward(); - preprocess_content.Length++; + preprocess_content.Text.Len++; } - Tokens.append( preprocess_content ); + array_append( _ctx->Lexer_Tokens, preprocess_content ); return Lex_Continue; // Skip found token, its all handled here. } FORCEINLINE -void lex_found_token( StrC& content - , s32& left - , char const*& scanner - , s32& line - , s32& column - , HashTable& defines - , Token& token ) +void lex_found_token( LexContext* ctx ) { - if ( token.Type != TokType::Invalid ) - { - Tokens.append( token ); + if ( ctx->token.Type != Tok_Invalid ) { + array_append( _ctx->Lexer_Tokens, ctx->token ); return; } - TokType type = ETokType::to_type( token ); + TokType type = str_to_toktype( tok_to_str(ctx->token) ); - if (type <= TokType::Access_Public && type >= TokType::Access_Private ) - { - token.Flags |= TF_AccessSpecifier; + if (type <= Tok_Access_Public && type >= Tok_Access_Private ) { + ctx->token.Flags |= TF_AccessSpecifier; } - - if ( type > TokType::__Attributes_Start ) - { - token.Flags |= TF_Attribute; + if ( type > Tok___Attributes_Start ) { + ctx->token.Flags |= TF_Attribute; } - - if ( type == ETokType::Decl_Extern_Linkage ) + if ( type == Tok_Decl_Extern_Linkage ) { - SkipWhitespace(); + skip_whitespace(); - if ( current != '"' ) - { - type = ETokType::Spec_Extern; - token.Flags |= TF_Specifier; + if ( (* ctx->scanner) != '"' ) { + type = Tok_Spec_Extern; + ctx->token.Flags |= TF_Specifier; } - token.Type = type; - Tokens.append( token ); + ctx->token.Type = type; + array_append( _ctx->Lexer_Tokens, ctx->token ); + return; + } + if ( ( type <= Tok_Star && type >= Tok_Spec_Alignas) + || type == Tok_Ampersand + || type == Tok_Ampersand_DBL ) + { + ctx->token.Type = type; + ctx->token.Flags |= TF_Specifier; + array_append( _ctx->Lexer_Tokens, ctx->token ); + return; + } + if ( type != Tok_Invalid ) + { + ctx->token.Type = type; + array_append( _ctx->Lexer_Tokens, ctx->token ); return; } - if ( ( type <= TokType::Star && type >= TokType::Spec_Alignas) - || type == TokType::Ampersand - || type == TokType::Ampersand_DBL ) - { - token.Type = type; - token.Flags |= TF_Specifier; - Tokens.append( token ); - return; + Macro* macro = lookup_macro( ctx->token.Text ); + b32 has_args = ctx->left && (* ctx->scanner) == '('; + b32 resolved_to_macro = false; + if (macro) { + ctx->token.Type = macrotype_to_toktype(macro->Type); + b32 is_functional = macro_is_functional(* macro); + resolved_to_macro = has_args ? is_functional : ! is_functional; + if ( ! resolved_to_macro ) { + log_fmt("Info(%d, %d): %S identified as a macro but usage here does not resolve to one (interpreting as identifier)\n" + , ctx->token.Line + , ctx->token.Line + , macro->Name + ); + } } - - - if ( type != TokType::Invalid ) + if ( resolved_to_macro ) { - token.Type = type; - Tokens.append( token ); - return; - } - - u64 key = 0; - if ( current == '(') - key = crc32( token.Text, token.Length + 1 ); - else - key = crc32( token.Text, token.Length ); - - StrC* define = defines.get( key ); - if ( define ) - { - token.Type = TokType::Preprocess_Macro; - + // TODO(Ed): When we introduce a macro AST (and expression support), we'll properly lex this section. // Want to ignore any arguments the define may have as they can be execution expressions. - if ( left && current == '(' ) - { - move_forward(); - token.Length++; - - s32 level = 0; - while ( left && (current != ')' || level > 0) ) - { - if ( current == '(' ) - level++; - - else if ( current == ')' && level > 0 ) - level--; - - move_forward(); - token.Length++; - } - - move_forward(); - token.Length++; + if ( has_args ) { + ctx->token.Flags |= TF_Macro_Functional; } - - if ( current == '\r' && scanner[1] == '\n' ) - { - move_forward(); - } - else if ( current == '\n' ) - { - move_forward(); + if ( bitfield_is_set(MacroFlags, macro->Flags, MF_Allow_As_Attribute) ) { + ctx->token.Flags |= TF_Attribute; } } else { - token.Type = TokType::Identifier; + ctx->token.Type = Tok_Identifier; } - Tokens.append( token ); + array_append( _ctx->Lexer_Tokens, ctx->token ); } - +// TODO(Ed): We need to to attempt to recover from a lex failure? neverinline -// TokArray lex( Array tokens, StrC content ) -TokArray lex( StrC content ) +// TokArray lex( Array tokens, Str content ) +TokArray lex( Str content ) { - s32 left = content.Len; - char const* scanner = content.Ptr; + LexContext c; LexContext* ctx = & c; + c.content = content; + c.left = content.Len; + c.scanner = content.Ptr; - char const* word = scanner; + char const* word = c.scanner; s32 word_length = 0; - s32 line = 1; - s32 column = 1; + c.line = 1; + c.column = 1; - SkipWhitespace(); - if ( left <= 0 ) + skip_whitespace(); + if ( c.left <= 0 ) { log_failure( "gen::lex: no tokens found (only whitespace provided)" ); - return { { nullptr }, 0 }; + TokArray null_array = {}; + return null_array; } - for ( StringCached entry : PreprocessorDefines ) - { - s32 length = 0; - char const* scanner = entry.Data; - while ( entry.length() > length && (char_is_alphanumeric( *scanner ) || *scanner == '_') ) - { - scanner++; - length ++; - } - if ( scanner[0] == '(' ) - { - length++; - } + array_clear(_ctx->Lexer_Tokens); - u64 key = crc32( entry.Data, length ); - defines.set( key, entry ); - } + b32 preprocess_args = true; - Tokens.clear(); - - while (left ) + while (c.left ) { #if 0 if (Tokens.num()) { - log_fmt("\nLastTok: %S", Tokens.back().to_string()); + log_fmt("\nLastTok: %SB", Tokens.back().to_strbuilder()); } #endif - Token token = { scanner, 0, TokType::Invalid, line, column, TF_Null }; + { + Token thanks_c = { { c.scanner, 0 }, Tok_Invalid, c.line, c.column, TF_Null }; + c.token = thanks_c; + } bool is_define = false; - if ( column == 1 ) + if ( c.column == 1 ) { - if ( current == '\r') + if ( (* ctx->scanner) == '\r') { move_forward(); - token.Length = 1; + c.token.Text.Len = 1; } - if ( current == '\n' ) + if ( (* ctx->scanner) == '\n' ) { move_forward(); - token.Type = TokType::NewLine; - token.Length++; + c.token.Type = Tok_NewLine; + c.token.Text.Len++; - Tokens.append( token ); + array_append( _ctx->Lexer_Tokens, c.token ); continue; } } - token.Length = 0; + c.token.Text.Len = 0; - SkipWhitespace(); - if ( left <= 0 ) + skip_whitespace(); + if ( c.left <= 0 ) break; - switch ( current ) + switch ( (* ctx->scanner) ) { case '#': { - s32 result = lex_preprocessor_directive( content, left, scanner, line, column, defines, token ); + s32 result = lex_preprocessor_directive( ctx ); switch ( result ) { case Lex_Continue: + { + //TokType last_type = Tokens[array_get_header(Tokens)->Num - 2].Type; + //if ( last_type == Tok_Preprocess_Pragma ) + { + { + Token thanks_c = { { c.scanner, 0 }, Tok_Invalid, c.line, c.column, TF_Null }; + c.token = thanks_c; + } + if ( (* ctx->scanner) == '\r') + { + move_forward(); + c.token.Text.Len = 1; + } + + if ( (* ctx->scanner) == '\n' ) + { + c.token.Type = Tok_NewLine; + c.token.Text.Len++; + move_forward(); + + array_append( _ctx->Lexer_Tokens, c.token ); + } + } continue; + } case Lex_ReturnNull: - return { { nullptr }, 0 }; + { + TokArray tok_array = {}; + return tok_array; + } } } case '.': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Access_MemberSymbol; - token.Flags = TF_AccessOperator; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Access_MemberSymbol; + c.token.Flags = TF_AccessOperator; - if (left) { + if (c.left) { move_forward(); } - if ( current == '.' ) + if ( (* ctx->scanner) == '.' ) { move_forward(); - if( current == '.' ) + if( (* ctx->scanner) == '.' ) { - token.Length = 3; - token.Type = TokType::Varadic_Argument; - token.Flags = TF_Null; + c.token.Text.Len = 3; + c.token.Type = Tok_Varadic_Argument; + c.token.Flags = TF_Null; move_forward(); } else { - String context_str = String::fmt_buf( GlobalAllocator, "%s", scanner, min( 100, left ) ); + StrBuilder context_str = strbuilder_fmt_buf( _ctx->Allocator_Temp, "%s", c.scanner, min( 100, c.left ) ); - log_failure( "gen::lex: invalid varadic argument, expected '...' got '..%c' (%d, %d)\n%s", current, line, column, context_str ); + log_failure( "gen::lex: invalid varadic argument, expected '...' got '..%c' (%d, %d)\n%s", (* ctx->scanner), c.line, c.column, context_str ); } } @@ -6583,21 +6334,21 @@ TokArray lex( StrC content ) } case '&' : { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Ampersand; - token.Flags |= TF_Operator; - token.Flags |= TF_Specifier; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Ampersand; + c.token.Flags |= TF_Operator; + c.token.Flags |= TF_Specifier; - if (left) + if (c.left) move_forward(); - if ( current == '&' ) // && + if ( (* ctx->scanner) == '&' ) // && { - token.Length = 2; - token.Type = TokType::Ampersand_DBL; + c.token.Text.Len = 2; + c.token.Type = Tok_Ampersand_DBL; - if (left) + if (c.left) move_forward(); } @@ -6605,41 +6356,41 @@ TokArray lex( StrC content ) } case ':': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Assign_Classifer; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Assign_Classifer; // Can be either a classifier (ParentType, Bitfield width), or ternary else - // token.Type = TokType::Colon; + // token.Type = Tok_Colon; - if (left) + if (c.left) move_forward(); - if ( current == ':' ) + if ( (* ctx->scanner) == ':' ) { move_forward(); - token.Type = TokType::Access_StaticSymbol; - token.Length++; + c.token.Type = Tok_Access_StaticSymbol; + c.token.Text.Len++; } goto FoundToken; } case '{': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::BraceCurly_Open; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_BraceCurly_Open; - if (left) + if (c.left) move_forward(); goto FoundToken; } case '}': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::BraceCurly_Close; - token.Flags = TF_EndDefinition; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_BraceCurly_Close; + c.token.Flags = TF_EndDefinition; - if (left) + if (c.left) move_forward(); end_line(); @@ -6647,17 +6398,17 @@ TokArray lex( StrC content ) } case '[': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::BraceSquare_Open; - if ( left ) + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_BraceSquare_Open; + if ( c.left ) { move_forward(); - if ( current == ']' ) + if ( (* ctx->scanner) == ']' ) { - token.Length = 2; - token.Type = TokType::Operator; + c.token.Text.Len = 2; + c.token.Type = Tok_Operator; move_forward(); } } @@ -6665,97 +6416,97 @@ TokArray lex( StrC content ) } case ']': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::BraceSquare_Close; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_BraceSquare_Close; - if (left) + if (c.left) move_forward(); goto FoundToken; } case '(': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Capture_Start; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Capture_Start; - if (left) + if (c.left) move_forward(); goto FoundToken; } case ')': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Capture_End; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Capture_End; - if (left) + if (c.left) move_forward(); goto FoundToken; } case '\'': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Char; - token.Flags = TF_Literal; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Char; + c.token.Flags = TF_Literal; move_forward(); - if ( left && current == '\\' ) + if ( c.left && (* ctx->scanner) == '\\' ) { move_forward(); - token.Length++; + c.token.Text.Len++; - if ( current == '\'' ) + if ( (* ctx->scanner) == '\'' ) { move_forward(); - token.Length++; + c.token.Text.Len++; } } - while ( left && current != '\'' ) + while ( c.left && (* ctx->scanner) != '\'' ) { move_forward(); - token.Length++; + c.token.Text.Len++; } - if ( left ) + if ( c.left ) { move_forward(); - token.Length++; + c.token.Text.Len++; } goto FoundToken; } case ',': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Comma; - token.Flags = TF_Operator; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Comma; + c.token.Flags = TF_Operator; - if (left) + if (c.left) move_forward(); goto FoundToken; } case '*': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Star; - token.Flags |= TF_Specifier; - token.Flags |= TF_Operator; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Star; + c.token.Flags |= TF_Specifier; + c.token.Flags |= TF_Operator; - if (left) + if (c.left) move_forward(); - if ( current == '=' ) + if ( (* ctx->scanner) == '=' ) { - token.Length++; - token.Flags |= TF_Assign; - // token.Type = TokType::Assign_Multiply; + c.token.Text.Len++; + c.token.Flags |= TF_Assign; + // c.token.Type = Tok_Assign_Multiply; - if ( left ) + if ( c.left ) move_forward(); } @@ -6763,12 +6514,12 @@ TokArray lex( StrC content ) } case ';': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Statement_End; - token.Flags = TF_EndDefinition; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Statement_End; + c.token.Flags = TF_EndDefinition; - if (left) + if (c.left) move_forward(); end_line(); @@ -6776,69 +6527,69 @@ TokArray lex( StrC content ) } case '"': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::String; - token.Flags |= TF_Literal; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_String; + c.token.Flags |= TF_Literal; move_forward(); - while ( left ) + while ( c.left ) { - if ( current == '"' ) + if ( (* ctx->scanner) == '"' ) { move_forward(); break; } - if ( current == '\\' ) + if ( (* ctx->scanner) == '\\' ) { move_forward(); - token.Length++; + c.token.Text.Len++; - if ( left ) + if ( c.left ) { move_forward(); - token.Length++; + c.token.Text.Len++; } continue; } move_forward(); - token.Length++; + c.token.Text.Len++; } goto FoundToken; } case '?': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Operator; - // token.Type = TokType::Ternary; - token.Flags = TF_Operator; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Operator; + // c.token.Type = Tok_Ternary; + c.token.Flags = TF_Operator; - if (left) + if (c.left) move_forward(); goto FoundToken; } case '=': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Operator; - // token.Type = TokType::Assign; - token.Flags = TF_Operator; - token.Flags |= TF_Assign; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Operator; + // c.token.Type = Tok_Assign; + c.token.Flags = TF_Operator; + c.token.Flags |= TF_Assign; - if (left) + if (c.left) move_forward(); - if ( current == '=' ) + if ( (* ctx->scanner) == '=' ) { - token.Length++; - token.Flags = TF_Operator; + c.token.Text.Len++; + c.token.Flags = TF_Operator; - if (left) + if (c.left) move_forward(); } @@ -6846,108 +6597,108 @@ TokArray lex( StrC content ) } case '+': { - // token.Type = TokType::Add + // c.token.Type = Tok_Add } case '%': { - // token.Type = TokType::Modulo; + // c.token.Type = Tok_Modulo; } case '^': { - // token.Type = TokType::B_XOr; + // c.token.Type = Tok_B_XOr; } case '~': { - // token.Type = TokType::Unary_Not; + // c.token.Type = Tok_Unary_Not; } case '!': { - // token.Type = TokType::L_Not; + // c.token.Type = Tok_L_Not; } case '<': { - // token.Type = TokType::Lesser; + // c.token.Type = Tok_Lesser; } case '>': { - // token.Type = TokType::Greater; + // c.token.Type = Tok_Greater; } case '|': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Operator; - token.Flags = TF_Operator; - // token.Type = TokType::L_Or; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Operator; + c.token.Flags = TF_Operator; + // token.Type = Tok_L_Or; - if (left) + if (c.left) move_forward(); - if ( current == '=' ) + if ( (* ctx->scanner) == '=' ) { - token.Length++; - token.Flags |= TF_Assign; + c.token.Text.Len++; + c.token.Flags |= TF_Assign; // token.Flags |= TokFlags::Assignment; - // token.Type = TokType::Assign_L_Or; + // token.Type = Tok_Assign_L_Or; - if (left) + if (c.left) move_forward(); } - else while ( left && current == *(scanner - 1) && token.Length < 3 ) + else while ( c.left && (* ctx->scanner) == *(c.scanner - 1) && c.token.Text.Len < 3 ) { - token.Length++; + c.token.Text.Len++; - if (left) + if (c.left) move_forward(); } goto FoundToken; } - // Dash is unfortunatlly a bit more complicated... + // Dash is unfortunately a bit more complicated... case '-': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Operator; - // token.Type = TokType::Subtract; - token.Flags = TF_Operator; - if ( left ) + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Operator; + // token.Type = Tok_Subtract; + c.token.Flags = TF_Operator; + if ( c.left ) { move_forward(); - if ( current == '>' ) + if ( (* ctx->scanner) == '>' ) { - token.Length++; -// token.Type = TokType::Access_PointerToMemberSymbol; - token.Flags |= TF_AccessOperator; + c.token.Text.Len++; +// token.Type = Tok_Access_PointerToMemberSymbol; + c.token.Flags |= TF_AccessOperator; move_forward(); - if ( current == '*' ) + if ( (* ctx->scanner) == '*' ) { -// token.Type = TokType::Access_PointerToMemberOfPointerSymbol; - token.Length++; +// token.Type = Tok_Access_PointerToMemberOfPointerSymbol; + c.token.Text.Len++; move_forward(); } } - else if ( current == '=' ) + else if ( (* ctx->scanner) == '=' ) { - token.Length++; - // token.Type = TokType::Assign_Subtract; - token.Flags |= TF_Assign; + c.token.Text.Len++; + // token.Type = Tok_Assign_Subtract; + c.token.Flags |= TF_Assign; - if (left) + if (c.left) move_forward(); } - else while ( left && current == *(scanner - 1) && token.Length < 3 ) + else while ( c.left && (* ctx->scanner) == *(c.scanner - 1) && c.token.Text.Len < 3 ) { - token.Length++; + c.token.Text.Len++; - if (left) + if (c.left) move_forward(); } } @@ -6955,82 +6706,82 @@ TokArray lex( StrC content ) } case '/': { - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Operator; - // token.Type = TokType::Divide; - token.Flags = TF_Operator; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Operator; + // token.Type = Tok_Divide; + c.token.Flags = TF_Operator; move_forward(); - if ( left ) + if ( c.left ) { - if ( current == '=' ) + if ( (* ctx->scanner) == '=' ) { // token.Type = TokeType::Assign_Divide; move_forward(); - token.Length++; - token.Flags = TF_Assign; + c.token.Text.Len++; + c.token.Flags = TF_Assign; } - else if ( current == '/' ) + else if ( (* ctx->scanner) == '/' ) { - token.Type = TokType::Comment; - token.Length = 2; - token.Flags = TF_Null; + c.token.Type = Tok_Comment; + c.token.Text.Len = 2; + c.token.Flags = TF_Null; move_forward(); - while ( left && current != '\n' && current != '\r' ) + while ( c.left && (* ctx->scanner) != '\n' && (* ctx->scanner) != '\r' ) { move_forward(); - token.Length++; + c.token.Text.Len++; } - if ( current == '\r' ) + if ( (* ctx->scanner) == '\r' ) { move_forward(); - token.Length++; + c.token.Text.Len++; } - if ( current == '\n' ) + if ( (* ctx->scanner) == '\n' ) { move_forward(); - token.Length++; + c.token.Text.Len++; } - Tokens.append( token ); + array_append( _ctx->Lexer_Tokens, c.token ); continue; } - else if ( current == '*' ) + else if ( (* ctx->scanner) == '*' ) { - token.Type = TokType::Comment; - token.Length = 2; - token.Flags = TF_Null; + c.token.Type = Tok_Comment; + c.token.Text.Len = 2; + c.token.Flags = TF_Null; move_forward(); - bool star = current == '*'; - bool slash = scanner[1] == '/'; + bool star = (* ctx->scanner) == '*'; + bool slash = c.scanner[1] == '/'; bool at_end = star && slash; - while ( left && ! at_end ) + while ( c.left && ! at_end ) { move_forward(); - token.Length++; + c.token.Text.Len++; - star = current == '*'; - slash = scanner[1] == '/'; + star = (* ctx->scanner) == '*'; + slash = c.scanner[1] == '/'; at_end = star && slash; } - token.Length += 2; + c.token.Text.Len += 2; move_forward(); move_forward(); - if ( current == '\r' ) + if ( (* ctx->scanner) == '\r' ) { move_forward(); - token.Length++; + c.token.Text.Len++; } - if ( current == '\n' ) + if ( (* ctx->scanner) == '\n' ) { move_forward(); - token.Length++; + c.token.Text.Len++; } - Tokens.append( token ); + array_append( _ctx->Lexer_Tokens, c.token ); // end_line(); continue; } @@ -7039,82 +6790,77 @@ TokArray lex( StrC content ) } } - if ( char_is_alpha( current ) || current == '_' ) + if ( char_is_alpha( (* ctx->scanner) ) || (* ctx->scanner) == '_' ) { - token.Text = scanner; - token.Length = 1; + Str text = { c.scanner, 1 }; + c.token.Text = text; move_forward(); - while ( left && ( char_is_alphanumeric(current) || current == '_' ) ) - { + while ( c.left && ( char_is_alphanumeric((* ctx->scanner)) || (* ctx->scanner) == '_' ) ) { move_forward(); - token.Length++; + c.token.Text.Len++; } goto FoundToken; } - else if ( char_is_digit(current) ) + else if ( char_is_digit((* ctx->scanner)) ) { // This is a very brute force lex, no checks are done for validity of literal. - token.Text = scanner; - token.Length = 1; - token.Type = TokType::Number; - token.Flags = TF_Literal; + Str text = { c.scanner, 1 }; + c.token.Text = text; + c.token.Type = Tok_Number; + c.token.Flags = TF_Literal; move_forward(); - if (left - && ( current == 'x' || current == 'X' - || current == 'b' || current == 'B' - || current == 'o' || current == 'O' ) + if (c.left + && ( (* ctx->scanner) == 'x' || (* ctx->scanner) == 'X' + || (* ctx->scanner) == 'b' || (* ctx->scanner) == 'B' + || (* ctx->scanner) == 'o' || (* ctx->scanner) == 'O' ) ) { move_forward(); - token.Length++; + c.token.Text.Len++; - while ( left && char_is_hex_digit(current) ) - { + while ( c.left && char_is_hex_digit((* ctx->scanner)) ) { move_forward(); - token.Length++; + c.token.Text.Len++; } goto FoundToken; } - while ( left && char_is_digit(current) ) - { + while ( c.left && char_is_digit((* ctx->scanner)) ) { move_forward(); - token.Length++; + c.token.Text.Len++; } - if ( left && current == '.' ) + if ( c.left && (* ctx->scanner) == '.' ) { move_forward(); - token.Length++; + c.token.Text.Len++; - while ( left && char_is_digit(current) ) - { + while ( c.left && char_is_digit((* ctx->scanner)) ) { move_forward(); - token.Length++; + c.token.Text.Len++; } // Handle number literal suffixes in a botched way - if (left && ( - current == 'l' || current == 'L' || // long/long long - current == 'u' || current == 'U' || // unsigned - current == 'f' || current == 'F' || // float - current == 'i' || current == 'I' || // imaginary - current == 'z' || current == 'Z')) // complex + if (c.left && ( + (* ctx->scanner) == 'l' || (* ctx->scanner) == 'L' || // long/long long + (* ctx->scanner) == 'u' || (* ctx->scanner) == 'U' || // unsigned + (* ctx->scanner) == 'f' || (* ctx->scanner) == 'F' || // float + (* ctx->scanner) == 'i' || (* ctx->scanner) == 'I' || // imaginary + (* ctx->scanner) == 'z' || (* ctx->scanner) == 'Z')) // complex { - char prev = current; + char prev = (* ctx->scanner); move_forward(); - token.Length++; + c.token.Text.Len++; // Handle 'll'/'LL' as a special case when we just processed an 'l'/'L' - if (left && (prev == 'l' || prev == 'L') && (current == 'l' || current == 'L')) - { + if (c.left && (prev == 'l' || prev == 'L') && ((* ctx->scanner) == 'l' || (* ctx->scanner) == 'L')) { move_forward(); - token.Length++; + c.token.Text.Len++; } } } @@ -7123,211 +6869,338 @@ TokArray lex( StrC content ) } else { - s32 start = max( 0, Tokens.num() - 100 ); + s32 start = max( 0, array_num(_ctx->Lexer_Tokens) - 100 ); log_fmt("\n%d\n", start); - for ( s32 idx = start; idx < Tokens.num(); idx++ ) + for ( s32 idx = start; idx < array_num(_ctx->Lexer_Tokens); idx++ ) { log_fmt( "Token %d Type: %s : %.*s\n" , idx - , ETokType::to_str( Tokens[ idx ].Type ).Ptr - , Tokens[ idx ].Length, Tokens[ idx ].Text + , toktype_to_str( _ctx->Lexer_Tokens[ idx ].Type ).Ptr + , _ctx->Lexer_Tokens[ idx ].Text.Len, _ctx->Lexer_Tokens[ idx ].Text.Ptr ); } - String context_str = String::fmt_buf( GlobalAllocator, "%.*s", min( 100, left ), scanner ); - log_failure( "Failed to lex token '%c' (%d, %d)\n%s", current, line, column, context_str ); + StrBuilder context_str = strbuilder_fmt_buf( _ctx->Allocator_Temp, "%.*s", min( 100, c.left ), c.scanner ); + log_failure( "Failed to lex token '%c' (%d, %d)\n%s", (* ctx->scanner), c.line, c.column, context_str ); // Skip to next whitespace since we can't know if anything else is valid until then. - while ( left && ! char_is_space( current ) ) - { + while ( c.left && ! char_is_space( (* ctx->scanner) ) ) { move_forward(); } } - FoundToken: - lex_found_token( content, left, scanner, line, column, defines, token ); + FoundToken: + { + lex_found_token( ctx ); + TokType last_type = array_back(_ctx->Lexer_Tokens)->Type; + if ( last_type == Tok_Preprocess_Macro_Stmt || last_type == Tok_Preprocess_Macro_Expr ) + { + Token thanks_c = { { c.scanner, 0 }, Tok_Invalid, c.line, c.column, TF_Null }; + c.token = thanks_c; + if ( (* ctx->scanner) == '\r') { + move_forward(); + c.token.Text.Len = 1; + } + if ( (* ctx->scanner) == '\n' ) + { + c.token.Type = Tok_NewLine; + c.token.Text.Len++; + move_forward(); + + array_append( _ctx->Lexer_Tokens, c.token ); + continue; + } + } + } } - if ( Tokens.num() == 0 ) - { + if ( array_num(_ctx->Lexer_Tokens) == 0 ) { log_failure( "Failed to lex any tokens" ); - return { { nullptr }, 0 }; + TokArray tok_array = {}; + return tok_array; } - defines.clear(); - // defines_map_arena.free(); - return { Tokens, 0 }; + TokArray result = { _ctx->Lexer_Tokens, 0 }; + return result; } -#undef current + #undef move_forward -#undef SkipWhitespace +#undef skip_whitespace +#undef end_line +// These macros are used in the swtich cases within parser.cpp -// namespace parser +#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_TOK_SPECIFIERS_CASES \ +case Tok_Spec_Consteval: \ +case Tok_Spec_Constexpr: \ +case Tok_Spec_Constinit: \ +case Tok_Spec_Explicit: \ +case Tok_Spec_ForceInline: \ +case Tok_Spec_ForceInline_Debuggable: \ +case Tok_Spec_Inline: \ +case Tok_Spec_Mutable: \ +case Tok_Spec_NeverInline: \ +case Tok_Spec_Static: \ +case Tok_Spec_Volatile: \ +case Tok_Spec_Virtual + +#define GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_SPECIFIERS_CASES \ +case Spec_Constexpr: \ +case Spec_Constinit: \ +case Spec_Explicit: \ +case Spec_Inline: \ +case Spec_ForceInline: \ +case Spec_ForceInline_Debuggable: \ +case Spec_Mutable: \ +case Spec_NeverInline: \ +case Spec_Static: \ +case Spec_Volatile: \ +case Spec_Virtual + +#define GEN_PARSER_CLASS_GLOBAL_NSPACE_ALLOWED_MEMBER_TOK_SPECIFIER_CASES \ +case Tok_Spec_Consteval: \ +case Tok_Spec_Constexpr: \ +case Tok_Spec_Constinit: \ +case Tok_Spec_Extern: \ +case Tok_Spec_ForceInline: \ +case Tok_Spec_ForceInline_Debuggable: \ +case Tok_Spec_Global: \ +case Tok_Spec_Inline: \ +case Tok_Spec_Internal_Linkage: \ +case Tok_Spec_NeverInline: \ +case Tok_Spec_Static + +#define GEN_PARSER_CLASS_GLOBAL_NSPACE_ALLOWED_MEMBER_SPECIFIER_CASES \ +case Spec_Constexpr: \ +case Spec_Constinit: \ +case Spec_ForceInline: \ +case Spec_ForceInline_Debuggable: \ +case Spec_Global: \ +case Spec_External_Linkage: \ +case Spec_Internal_Linkage: \ +case Spec_Inline: \ +case Spec_Mutable: \ +case Spec_NeverInline: \ +case Spec_Static: \ +case Spec_Volatile + +#define GEN_PARSER_FRIEND_ALLOWED_SPECIFIERS_CASES \ +case Spec_Const: \ +case Spec_Inline: \ +case Spec_ForceInline + +#define GEN_PARSER_FUNCTION_ALLOWED_SPECIFIERS_CASES \ +case Spec_Const: \ +case Spec_Consteval: \ +case Spec_Constexpr: \ +case Spec_External_Linkage: \ +case Spec_Internal_Linkage: \ +case Spec_ForceInline: \ +case Spec_ForceInline_Debuggable: \ +case Spec_Inline: \ +case Spec_NeverInline: \ +case Spec_Static + +#define GEN_PARSER_OPERATOR_ALLOWED_SPECIFIERS_CASES \ +case Spec_Const: \ +case Spec_Constexpr: \ +case Spec_ForceInline: \ +case Spec_Inline: \ +case Spec_NeverInline: \ +case Spec_Static + +#define GEN_PARSER_TEMPLATE_ALLOWED_SPECIFIERS_CASES \ +case Spec_Const: \ +case Spec_Constexpr: \ +case Spec_Constinit: \ +case Spec_External_Linkage: \ +case Spec_Global: \ +case Spec_Inline: \ +case Spec_ForceInline: \ +case Spec_ForceInline_Debuggable: \ +case Spec_Local_Persist: \ +case Spec_Mutable: \ +case Spec_Static: \ +case Spec_Thread_Local: \ +case Spec_Volatile + +#define GEN_PARSER_VARIABLE_ALLOWED_SPECIFIER_CASES \ +case Spec_Const: \ +case Spec_Constexpr: \ +case Spec_Constinit: \ +case Spec_External_Linkage: \ +case Spec_Global: \ +case Spec_Inline: \ +case Spec_Local_Persist: \ +case Spec_Mutable: \ +case Spec_Static: \ +case Spec_Thread_Local: \ +case Spec_Volatile + +// TODO(Ed) : Rename ETok_Capture_Start, ETok_Capture_End to Open_Parenthesis adn Close_Parenthesis + +constexpr bool lex_dont_skip_formatting = false; +constexpr bool lex_skip_formatting = true; + +void parser_push( ParseContext* ctx, StackNode* node ) +{ + node->Prev = ctx->Scope; + ctx->Scope = node; + +#if 0 && GEN_BUILD_DEBUG + log_fmt("\tEntering _ctx->parser: %.*s\n", Scope->ProcName.Len, Scope->ProcName.Ptr ); +#endif } -namespace parser { - -// TODO(Ed) : Rename ETokType::Capture_Start, ETokType::Capture_End to Open_Parenthesis adn Close_Parenthesis - -constexpr bool dont_skip_formatting = false; - -struct StackNode +void parser_pop(ParseContext* ctx) { - StackNode* Prev; +#if 0 && GEN_BUILD_DEBUG + log_fmt("\tPopping _ctx->parser: %.*s\n", Scope->ProcName.Len, Scope->ProcName.Ptr ); +#endif + ctx->Scope = ctx->Scope->Prev; +} - Token Start; - Token Name; // The name of the AST node (if parsed) - StrC ProcName; // The name of the procedure -}; - -struct ParseContext +StrBuilder parser_to_strbuilder(ParseContext ctx) { - TokArray Tokens; - StackNode* Scope; + StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, kilobytes(4) ); - void push( StackNode* node ) + Token scope_start = * ctx.Scope->Start; + Token last_valid = ctx.Tokens.Idx >= array_num(ctx.Tokens.Arr) ? ctx.Tokens.Arr[array_num(ctx.Tokens.Arr) -1] : (* lex_current(& ctx.Tokens, true)); + + sptr length = scope_start.Text.Len; + char const* current = scope_start.Text.Ptr + length; + while ( current <= array_back( ctx.Tokens.Arr)->Text.Ptr && (* current) != '\n' && length < 74 ) { - node->Prev = Scope; - Scope = node; - - #if 0 && Build_Debug - log_fmt("\tEntering Context: %.*s\n", Scope->ProcName.Len, Scope->ProcName.Ptr ); - #endif + current++; + length++; } - void pop() + Str scope_str = { scope_start.Text.Ptr, length }; + StrBuilder line = strbuilder_make_str( _ctx->Allocator_Temp, scope_str ); + strbuilder_append_fmt( & result, "\tScope : %s\n", line ); + strbuilder_free(& line); + + sptr dist = (sptr)last_valid.Text.Ptr - (sptr)scope_start.Text.Ptr + 2; + sptr length_from_err = dist; + + Str err_str = { last_valid.Text.Ptr, length_from_err }; + StrBuilder line_from_err = strbuilder_make_str( _ctx->Allocator_Temp, err_str ); + + if ( length_from_err < 100 ) + strbuilder_append_fmt(& result, "\t(%d, %d):%*c\n", last_valid.Line, last_valid.Column, length_from_err, '^' ); + else + strbuilder_append_fmt(& result, "\t(%d, %d)\n", last_valid.Line, last_valid.Column ); + + StackNode* curr_scope = ctx.Scope; + s32 level = 0; + do { - #if 0 && Build_Debug - log_fmt("\tPopping Context: %.*s\n", Scope->ProcName.Len, Scope->ProcName.Ptr ); - #endif - Scope = Scope->Prev; - } - - String to_string() - { - String result = String::make_reserve( GlobalAllocator, kilobytes(4) ); - - Token scope_start = Scope->Start; - Token last_valid = Tokens.Idx >= Tokens.Arr.num() ? Tokens.Arr[Tokens.Arr.num() -1] : Tokens.current(); - - sptr length = scope_start.Length; - char const* current = scope_start.Text + length; - while ( current <= Tokens.Arr.back().Text && *current != '\n' && length < 74 ) - { - current++; - length++; + if ( curr_scope->Name.Ptr ) { + strbuilder_append_fmt(& result, "\t%d: %s, AST Name: %.*s\n", level, curr_scope->ProcName.Ptr, curr_scope->Name.Len, curr_scope->Name.Ptr ); + } + else { + strbuilder_append_fmt(& result, "\t%d: %s\n", level, curr_scope->ProcName.Ptr ); } - String line = String::make( GlobalAllocator, { length, scope_start.Text } ); - result.append_fmt("\tScope : %s\n", line ); - line.free(); - - sptr dist = (sptr)last_valid.Text - (sptr)scope_start.Text + 2; - sptr length_from_err = dist; - String line_from_err = String::make( GlobalAllocator, { length_from_err, last_valid.Text } ); - - if ( length_from_err < 100 ) - result.append_fmt("\t(%d, %d):%*c\n", last_valid.Line, last_valid.Column, length_from_err, '^' ); - else - result.append_fmt("\t(%d, %d)\n", last_valid.Line, last_valid.Column ); - - StackNode* curr_scope = Scope; - s32 level = 0; - do - { - if ( curr_scope->Name ) - { - result.append_fmt("\t%d: %s, AST Name: %.*s\n", level, curr_scope->ProcName.Ptr, curr_scope->Name.Length, curr_scope->Name.Text ); - } - else - { - result.append_fmt("\t%d: %s\n", level, curr_scope->ProcName.Ptr ); - } - - curr_scope = curr_scope->Prev; - level++; - } - while ( curr_scope ); - return result; + curr_scope = curr_scope->Prev; + level++; } -}; + while ( curr_scope ); + return result; +} -global ParseContext Context; - -bool TokArray::__eat( TokType type ) +bool lex__eat(TokArray* self, TokType type ) { - if ( Arr.num() - Idx <= 0 ) - { - log_failure( "No tokens left.\n%s", Context.to_string() ); + if ( array_num(self->Arr) - self->Idx <= 0 ) { + log_failure( "No tokens left.\n%s", parser_to_strbuilder(_ctx->parser) ); return false; } - if ( ( Arr[ Idx ].Type == TokType::NewLine && type != TokType::NewLine ) - || ( Arr[ Idx ].Type == TokType::Comment && type != TokType::Comment ) ) + Token at_idx = self->Arr[ self->Idx ]; + + if ( ( at_idx.Type == Tok_NewLine && type != Tok_NewLine ) + || ( at_idx.Type == Tok_Comment && type != Tok_Comment ) ) { - Idx++; + self->Idx ++; } - if ( Arr[Idx].Type != type ) + b32 not_accepted = at_idx.Type != type; + b32 is_identifier = at_idx.Type == Tok_Identifier; + if ( not_accepted ) { + Macro* macro = lookup_macro(at_idx.Text); + b32 accept_as_identifier = macro && bitfield_is_set(MacroFlags, macro->Flags, MF_Allow_As_Identifier ); + not_accepted = type == Tok_Identifier && accept_as_identifier ? false : true; + } + if ( not_accepted ) + { + Token tok = * lex_current( self, lex_skip_formatting ); log_failure( "Parse Error, TokArray::eat, Expected: ' %s ' not ' %.*s ' (%d, %d)`\n%s" - , ETokType::to_str(type).Ptr - , Arr[Idx].Length, Arr[Idx].Text - , current().Line - , current().Column - , Context.to_string() + , toktype_to_str(type).Ptr + , at_idx.Text.Len, at_idx.Text.Ptr + , tok.Line + , tok.Column + , parser_to_strbuilder(_ctx->parser) ); - + GEN_DEBUG_TRAP(); return false; } -#if 0 && Build_Debug - log_fmt("Ate: %S\n", Arr[Idx].to_string() ); +#if 0 && GEN_BUILD_DEBUG + log_fmt("Ate: %SB\n", self->Arr[Idx].to_strbuilder() ); #endif - Idx++; + self->Idx ++; return true; } internal -void init() +void parser_init() { - Tokens = Array::init_reserve( LexArena - , ( LexAllocator_Size - sizeof( Array::Header ) ) / sizeof(Token) - ); - - defines_map_arena = Arena_256KB::init(); - defines = HashTable::init_reserve( defines_map_arena, 256 ); + _ctx->Lexer_Tokens = array_init_reserve(Token, _ctx->Allocator_DyanmicContainers, _ctx->InitSize_LexerTokens ); } internal -void deinit() +void parser_deinit() { - parser::Tokens = { nullptr }; + Array(Token) null_array = { nullptr }; + _ctx->Lexer_Tokens = null_array; } #pragma region Helper Macros -# define check_parse_args( def ) \ -if ( def.Len <= 0 ) \ -{ \ - log_failure( "gen::" stringize(__func__) ": length must greater than 0" ); \ - parser::Context.pop(); \ - return CodeInvalid; \ -} \ -if ( def.Ptr == nullptr ) \ -{ \ - log_failure( "gen::" stringize(__func__) ": def was null" ); \ - parser::Context.pop(); \ - return CodeInvalid; \ +#define check_parse_args( def ) _check_parse_args(def, stringize(_func_) ) +bool _check_parse_args( Str def, char const* func_name ) +{ + if ( def.Len <= 0 ) + { + log_failure( c_str_fmt_buf("gen::%s: length must greater than 0", func_name) ); + parser_pop(& _ctx->parser); + return false; + } + if ( def.Ptr == nullptr ) + { + log_failure( c_str_fmt_buf("gen::%s: def was null", func_name) ); + parser_pop(& _ctx->parser); + return false; + } + return true; } -# define currtok_noskip Context.Tokens.current( dont_skip_formatting ) -# define currtok Context.Tokens.current() -# define prevtok Context.Tokens.previous() -# define nexttok Context.Tokens.next() -# define eat( Type_ ) Context.Tokens.__eat( Type_ ) -# define left ( Context.Tokens.Arr.num() - Context.Tokens.Idx ) +# define currtok_noskip (* lex_current( & _ctx->parser.Tokens, lex_dont_skip_formatting )) +# define currtok (* lex_current( & _ctx->parser.Tokens, lex_skip_formatting )) +# define peektok (* lex_peek(_ctx->parser.Tokens, lex_skip_formatting)) +# define prevtok (* lex_previous( _ctx->parser.Tokens, lex_dont_skip_formatting)) +# define nexttok (* lex_next( _ctx->parser.Tokens, lex_skip_formatting )) +# define nexttok_noskip (* lex_next( _ctx->parser.Tokens, lex_dont_skip_formatting)) +# define eat( Type_ ) lex__eat( & _ctx->parser.Tokens, Type_ ) +# define left ( array_num(_ctx->parser.Tokens.Arr) - _ctx->parser.Tokens.Idx ) + +#if GEN_COMPILER_CPP +# define def_assign( ... ) { __VA_ARGS__ } +#else +# define def_assign( ... ) __VA_ARGS__ +#endif + #ifdef check #define CHECK_WAS_DEFINED @@ -7336,11 +7209,12 @@ if ( def.Ptr == nullptr ) \ #endif # define check_noskip( Type_ ) ( left && currtok_noskip.Type == Type_ ) -# define check( Type_ ) ( left && currtok.Type == Type_ ) +# define check( Type_ ) ( left && currtok.Type == Type_ ) -# define push_scope() \ - StackNode scope { nullptr, currtok_noskip, NullToken, txt( __func__ ) }; \ - Context.push( & scope ) +# define push_scope() \ + Str null_name = {}; \ + StackNode scope = { nullptr, lex_current( & _ctx->parser.Tokens, lex_dont_skip_formatting ), null_name, txt( __func__ ) }; \ + parser_push( & _ctx->parser, & scope ) #pragma endregion Helper Macros @@ -7350,75 +7224,81 @@ internal Code parse_array_decl (); internal CodeAttributes parse_attributes (); internal CodeComment parse_comment (); internal Code parse_complicated_definition ( TokType which ); -internal CodeBody parse_class_struct_body ( TokType which, Token name = NullToken ); +internal CodeBody parse_class_struct_body ( TokType which, Token name ); internal Code parse_class_struct ( TokType which, bool inplace_def ); -internal CodeDefine parse_define (); internal Code parse_expression (); internal Code parse_forward_or_definition ( TokType which, bool is_inplace ); -internal CodeFn parse_function_after_name ( ModuleFlag mflags, CodeAttributes attributes, CodeSpecifiers specifiers, CodeType ret_type, Token name ); +internal CodeFn parse_function_after_name ( ModuleFlag mflags, CodeAttributes attributes, CodeSpecifiers specifiers, CodeTypename ret_type, Token name ); internal Code parse_function_body (); -internal Code parse_global_nspace (); +internal CodeBody parse_global_nspace ( CodeType which ); internal Code parse_global_nspace_constructor_destructor( CodeSpecifiers specifiers ); -internal Token parse_identifier ( bool* possible_member_function = nullptr ); +internal Token parse_identifier ( bool* possible_member_function ); internal CodeInclude parse_include (); -internal CodeOperator parse_operator_after_ret_type ( ModuleFlag mflags, CodeAttributes attributes, CodeSpecifiers specifiers, CodeType ret_type ); +internal Code parse_macro_as_definiton ( CodeAttributes attributes, CodeSpecifiers specifiers ); +internal CodeOperator parse_operator_after_ret_type ( ModuleFlag mflags, CodeAttributes attributes, CodeSpecifiers specifiers, CodeTypename ret_type ); internal Code parse_operator_function_or_variable( bool expects_function, CodeAttributes attributes, CodeSpecifiers specifiers ); internal CodePragma parse_pragma (); -internal CodeParam parse_params ( bool use_template_capture = false ); +internal CodeParams parse_params ( bool use_template_capture ); internal CodePreprocessCond parse_preprocess_cond (); -internal Code parse_simple_preprocess ( TokType which ); +internal Code parse_simple_preprocess ( TokType which ); internal Code parse_static_assert (); -internal void parse_template_args ( Token& token ); -internal CodeVar parse_variable_after_name ( ModuleFlag mflags, CodeAttributes attributes, CodeSpecifiers specifiers, CodeType type, StrC name ); +internal void parse_template_args ( Token* token ); +internal CodeVar parse_variable_after_name ( ModuleFlag mflags, CodeAttributes attributes, CodeSpecifiers specifiers, CodeTypename type, Str name ); internal CodeVar parse_variable_declaration_list (); -internal CodeClass parse_class ( bool inplace_def = false ); -internal CodeConstructor parse_constructor ( CodeSpecifiers specifiers ); -internal CodeDestructor parse_destructor ( CodeSpecifiers specifiers = NoCode ); -internal CodeEnum parse_enum ( bool inplace_def = false ); -internal CodeBody parse_export_body (); -internal CodeBody parse_extern_link_body(); -internal CodeExtern parse_extern_link (); -internal CodeFriend parse_friend (); -internal CodeFn parse_function (); -internal CodeNS parse_namespace (); -internal CodeOpCast parse_operator_cast ( CodeSpecifiers specifiers = NoCode ); -internal CodeStruct parse_struct ( bool inplace_def = false ); -internal CodeVar parse_variable (); -internal CodeTemplate parse_template (); -internal CodeType parse_type ( bool from_template = false, bool* is_function = nullptr ); -internal CodeTypedef parse_typedef (); -internal CodeUnion parse_union ( bool inplace_def = false ); -internal CodeUsing parse_using (); +internal CodeClass parser_parse_class ( bool inplace_def ); +internal CodeConstructor parser_parse_constructor ( CodeSpecifiers specifiers ); +internal CodeDefine parser_parse_define (); +internal CodeDestructor parser_parse_destructor ( CodeSpecifiers specifiers ); +internal CodeEnum parser_parse_enum ( bool inplace_def ); +internal CodeBody parser_parse_export_body (); +internal CodeBody parser_parse_extern_link_body(); +internal CodeExtern parser_parse_extern_link (); +internal CodeFriend parser_parse_friend (); +internal CodeFn parser_parse_function (); +internal CodeNS parser_parse_namespace (); +internal CodeOpCast parser_parse_operator_cast ( CodeSpecifiers specifiers ); +internal CodeStruct parser_parse_struct ( bool inplace_def ); +internal CodeVar parser_parse_variable (); +internal CodeTemplate parser_parse_template (); +internal CodeTypename parser_parse_type ( bool from_template, bool* is_function ); +internal CodeTypedef parser_parse_typedef (); +internal CodeUnion parser_parse_union ( bool inplace_def ); +internal CodeUsing parser_parse_using (); -constexpr bool inplace_def = true; +constexpr bool parser_inplace_def = true; +constexpr bool parser_not_inplace_def = false; +constexpr bool parser_dont_consume_braces = true; +constexpr bool parser_consume_braces = false; +constexpr bool parser_not_from_template = false; + +constexpr bool parser_use_parenthesis = false; // Internal parsing functions -constexpr bool strip_formatting_dont_preserve_newlines = false; +constexpr bool parser_strip_formatting_dont_preserve_newlines = false; /* This function was an attempt at stripping formatting from any c++ code. It has edge case failures that prevent it from being used in function bodies. */ internal -String strip_formatting( StrC raw_text, bool preserve_newlines = true ) +StrBuilder parser_strip_formatting( Str raw_text, bool preserve_newlines ) { - String content = String::make_reserve( GlobalAllocator, raw_text.Len ); + StrBuilder content = strbuilder_make_reserve( _ctx->Allocator_Temp, raw_text.Len ); if ( raw_text.Len == 0 ) return content; #define cut_length ( scanner - raw_text.Ptr - last_cut ) #define cut_ptr ( raw_text.Ptr + last_cut ) -#define pos ( sptr( scanner ) - sptr( raw_text.Ptr ) ) +#define pos ( rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ) ) #define move_fwd() do { scanner++; tokleft--; } while(0) s32 tokleft = raw_text.Len; sptr last_cut = 0; char const* scanner = raw_text.Ptr; - if ( scanner[0] == ' ' ) - { + if ( scanner[0] == ' ' ) { move_fwd(); last_cut = 1; } @@ -7450,8 +7330,8 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) if ( tokleft ) move_fwd(); - content.append( cut_ptr, cut_length ); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); + last_cut = rcast(sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } @@ -7472,8 +7352,8 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) if ( tokleft ) move_fwd(); - content.append( cut_ptr, cut_length ); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } @@ -7486,8 +7366,8 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) scanner += 2; tokleft -= 2; - content.append( cut_ptr, cut_length ); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } @@ -7505,8 +7385,8 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) if (tokleft) move_fwd(); - content.append( cut_ptr, cut_length ); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } @@ -7514,13 +7394,13 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) if (scanner[0] == '\t') { if (pos > last_cut) - content.append(cut_ptr, cut_length); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length); - if ( content.back() != ' ' ) - content.append(' '); + if ( * strbuilder_back( content ) != ' ' ) + strbuilder_append_char( & content, ' ' ); move_fwd(); - last_cut = sptr(scanner) - sptr(raw_text.Ptr); + last_cut = rcast( sptr, scanner) - rcast( sptr, raw_text.Ptr); continue; } @@ -7533,22 +7413,22 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) scanner += 2; tokleft -= 2; - content.append( cut_ptr, cut_length ); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } if ( pos > last_cut ) - content.append( cut_ptr, cut_length ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); // Replace with a space - if ( content.back() != ' ' ) - content.append( ' ' ); + if ( * strbuilder_back( content ) != ' ' ) + strbuilder_append_char( & content, ' ' ); scanner += 2; tokleft -= 2; - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } @@ -7560,28 +7440,28 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) move_fwd(); - content.append( cut_ptr, cut_length ); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } if ( pos > last_cut ) - content.append( cut_ptr, cut_length ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); // Replace with a space - if ( content.back() != ' ' ) - content.append( ' ' ); + if ( * strbuilder_back( content ) != ' ' ) + strbuilder_append_char( & content, ' ' ); move_fwd(); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } // Escaped newlines if ( scanner[0] == '\\' ) { - content.append( cut_ptr, cut_length ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); s32 amount_to_skip = 1; if ( tokleft > 1 && scanner[1] == '\n' ) @@ -7601,25 +7481,26 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) else move_fwd(); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); continue; } // Consectuive spaces if ( tokleft > 1 && char_is_space( scanner[0] ) && char_is_space( scanner[ 1 ] ) ) { - content.append( cut_ptr, cut_length ); + strbuilder_append_c_str_len( & content, cut_ptr, cut_length ); do { move_fwd(); } while ( tokleft && char_is_space( scanner[0] ) ); - last_cut = sptr( scanner ) - sptr( raw_text.Ptr ); + last_cut = rcast( sptr, scanner ) - rcast( sptr, raw_text.Ptr ); // Preserve only 1 space of formattting - if ( content.back() != ' ' ) - content.append( ' ' ); + char* last = strbuilder_back(content); + if ( last == nullptr || * last != ' ' ) + strbuilder_append_char( & content, ' ' ); continue; } @@ -7627,9 +7508,8 @@ String strip_formatting( StrC raw_text, bool preserve_newlines = true ) move_fwd(); } - if ( last_cut < raw_text.Len ) - { - content.append( cut_ptr, raw_text.Len - last_cut ); + if ( last_cut < raw_text.Len ) { + strbuilder_append_c_str_len( & content, cut_ptr, raw_text.Len - last_cut ); } #undef cut_ptr @@ -7645,79 +7525,79 @@ Code parse_array_decl() { push_scope(); - if ( check( TokType::Operator ) && currtok.Text[0] == '[' && currtok.Text[1] == ']' ) + if ( check( Tok_Operator ) && currtok.Text.Ptr[0] == '[' && currtok.Text.Ptr[1] == ']' ) { - Code array_expr = untyped_str( currtok ); - eat( TokType::Operator ); + Code array_expr = untyped_str( txt(" ") ); + eat( Tok_Operator ); // [] - Context.pop(); + parser_pop(& _ctx->parser); return array_expr; } - if ( check( TokType::BraceSquare_Open ) ) + if ( check( Tok_BraceSquare_Open ) ) { - eat( TokType::BraceSquare_Open ); + eat( Tok_BraceSquare_Open ); // [ if ( left == 0 ) { - log_failure( "Error, unexpected end of array declaration ( '[]' scope started )\n%s", Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Error, unexpected end of array declaration ( '[]' scope started )\n%s", parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } - if ( currtok.Type == TokType::BraceSquare_Close ) + if ( currtok.Type == Tok_BraceSquare_Close ) { - log_failure( "Error, empty array expression in definition\n%s", Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Error, empty array expression in definition\n%s", parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } Token untyped_tok = currtok; - while ( left && currtok.Type != TokType::BraceSquare_Close ) + while ( left && currtok.Type != Tok_BraceSquare_Close ) { eat( currtok.Type ); } - untyped_tok.Length = ( (sptr)prevtok.Text + prevtok.Length ) - (sptr)untyped_tok.Text; + untyped_tok.Text.Len = ( (sptr)prevtok.Text.Ptr + prevtok.Text.Len ) - (sptr)untyped_tok.Text.Ptr; - Code array_expr = untyped_str( untyped_tok ); + Code array_expr = untyped_str( tok_to_str(untyped_tok) ); // [ if ( left == 0 ) { - log_failure( "Error, unexpected end of array declaration, expected ]\n%s", Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Error, unexpected end of array declaration, expected ]\n%s", parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } - if ( currtok.Type != TokType::BraceSquare_Close ) + if ( currtok.Type != Tok_BraceSquare_Close ) { - log_failure( "%s: Error, expected ] in array declaration, not %s\n%s", ETokType::to_str( currtok.Type ), Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "%s: Error, expected ] in array declaration, not %s\n%s", toktype_to_str( currtok.Type ), parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } - eat( TokType::BraceSquare_Close ); + eat( Tok_BraceSquare_Close ); // [ ] // Its a multi-dimensional array - if ( check( TokType::BraceSquare_Open )) + if ( check( Tok_BraceSquare_Open )) { Code adjacent_arr_expr = parse_array_decl(); // [ ][ ]... - array_expr->Next = adjacent_arr_expr.ast; + array_expr->Next = adjacent_arr_expr; } - Context.pop(); + parser_pop(& _ctx->parser); return array_expr; } - Context.pop(); - return { nullptr }; + parser_pop(& _ctx->parser); + return NullCode; } internal inline @@ -7729,131 +7609,129 @@ CodeAttributes parse_attributes() s32 len = 0; // There can be more than one attribute. If there is flatten them to a single string. - // TODO(Ed): Support keeping an linked list of attributes similar to parameters - while ( left && currtok.is_attribute() ) + // TODO(Ed): Support chaining attributes (Use parameter linkage pattern) + while ( left && tok_is_attribute(currtok) ) { - if ( check( TokType::Attribute_Open ) ) + if ( check( Tok_Attribute_Open ) ) { - eat( TokType::Attribute_Open ); + eat( Tok_Attribute_Open ); // [[ - while ( left && currtok.Type != TokType::Attribute_Close ) + while ( left && currtok.Type != Tok_Attribute_Close ) { eat( currtok.Type ); } // [[ - eat( TokType::Attribute_Close ); + eat( Tok_Attribute_Close ); // [[ ]] - len = ( ( sptr )prevtok.Text + prevtok.Length ) - ( sptr )start.Text; + len = ( ( sptr )prevtok.Text.Ptr + prevtok.Text.Len ) - ( sptr )start.Text.Ptr; } - else if ( check( TokType::Decl_GNU_Attribute ) ) + else if ( check( Tok_Decl_GNU_Attribute ) ) { - eat( TokType::Decl_GNU_Attribute ); - eat( TokType::Capture_Start ); - eat( TokType::Capture_Start ); + eat( Tok_Decl_GNU_Attribute ); + eat( Tok_Capture_Start ); + eat( Tok_Capture_Start ); // __attribute__(( - while ( left && currtok.Type != TokType::Capture_End ) + while ( left && currtok.Type != Tok_Capture_End ) { eat( currtok.Type ); } // __attribute__(( - eat( TokType::Capture_End ); - eat( TokType::Capture_End ); + eat( Tok_Capture_End ); + eat( Tok_Capture_End ); // __attribute__(( )) - len = ( ( sptr )prevtok.Text + prevtok.Length ) - ( sptr )start.Text; + len = ( ( sptr )prevtok.Text.Ptr + prevtok.Text.Len ) - ( sptr )start.Text.Ptr; } - else if ( check( TokType::Decl_MSVC_Attribute ) ) + else if ( check( Tok_Decl_MSVC_Attribute ) ) { - eat( TokType::Decl_MSVC_Attribute ); - eat( TokType::Capture_Start ); + eat( Tok_Decl_MSVC_Attribute ); + eat( Tok_Capture_Start ); // __declspec( - while ( left && currtok.Type != TokType::Capture_End ) + while ( left && currtok.Type != Tok_Capture_End ) { eat( currtok.Type ); } // __declspec( - eat( TokType::Capture_End ); + eat( Tok_Capture_End ); // __declspec( ) - len = ( ( sptr )prevtok.Text + prevtok.Length ) - ( sptr )start.Text; + len = ( ( sptr )prevtok.Text.Ptr + prevtok.Text.Len ) - ( sptr )start.Text.Ptr; } - else if ( currtok.is_attribute() ) + else if ( tok_is_attribute(currtok) ) { eat( currtok.Type ); // // If its a macro based attribute, this could be a functional macro such as Unreal's UE_DEPRECATED(...) - if ( check( TokType::Capture_Start)) + if ( check( Tok_Capture_Start)) { - eat( TokType::Capture_Start ); + eat( Tok_Capture_Start ); s32 level = 0; - while (left && currtok.Type != TokType::Capture_End && level == 0) + while (left && currtok.Type != Tok_Capture_End && level == 0) { - if (currtok.Type == TokType::Capture_Start) + if (currtok.Type == Tok_Capture_Start) ++ level; - if (currtok.Type == TokType::Capture_End) + if (currtok.Type == Tok_Capture_End) --level; eat(currtok.Type); } - eat(TokType::Capture_End); + eat(Tok_Capture_End); } - len = ( ( sptr )prevtok.Text + prevtok.Length ) - ( sptr )start.Text; + len = ( ( sptr )prevtok.Text.Ptr + prevtok.Text.Len ) - ( sptr )start.Text.Ptr; // ( ... ) } } if ( len > 0 ) { - StrC attribute_txt = { len, start.Text }; - Context.pop(); + Str attribute_txt = { start.Text.Ptr, len }; + parser_pop(& _ctx->parser); - String name_stripped = strip_formatting( attribute_txt, strip_formatting_dont_preserve_newlines ); + StrBuilder name_stripped = parser_strip_formatting( attribute_txt, parser_strip_formatting_dont_preserve_newlines ); Code result = make_code(); - result->Type = ECode::PlatformAttributes; - result->Name = get_cached_string( name_stripped ); + result->Type = CT_PlatformAttributes; + result->Name = cache_str( strbuilder_to_str(name_stripped) ); result->Content = result->Name; // result->Token = return ( CodeAttributes )result; } - Context.pop(); - return { nullptr }; + parser_pop(& _ctx->parser); + return NullCode; } internal -Code parse_class_struct( TokType which, bool inplace_def = false ) +Code parse_class_struct( TokType which, bool inplace_def ) { - if ( which != TokType::Decl_Class && which != TokType::Decl_Struct ) - { - log_failure( "Error, expected class or struct, not %s\n%s", ETokType::to_str( which ), Context.to_string() ); - return CodeInvalid; + if ( which != Tok_Decl_Class && which != Tok_Decl_Struct ) { + log_failure( "Error, expected class or struct, not %s\n%s", toktype_to_str( which ), parser_to_strbuilder(_ctx->parser) ); + return InvalidCode; } - Token name { nullptr, 0, TokType::Invalid }; + Token name = NullToken; - AccessSpec access = AccessSpec::Default; - CodeType parent = { nullptr }; + AccessSpec access = AccessSpec_Default; + CodeTypename parent = { nullptr }; CodeBody body = { nullptr }; CodeAttributes attributes = { nullptr }; - ModuleFlag mflags = ModuleFlag::None; + ModuleFlag mflags = ModuleFlag_None; - CodeClass result = CodeInvalid; + Code result = InvalidCode; - if ( check(TokType::Module_Export) ) - { - mflags = ModuleFlag::Export; - eat( TokType::Module_Export ); + if ( check(Tok_Module_Export) ) { + mflags = ModuleFlag_Export; + eat( Tok_Module_Export ); } // @@ -7863,261 +7741,275 @@ Code parse_class_struct( TokType which, bool inplace_def = false ) attributes = parse_attributes(); // - if ( check( TokType::Identifier ) ) - { - name = parse_identifier(); - Context.Scope->Name = name; + if ( check( Tok_Identifier ) ) { + name = parse_identifier(nullptr); + _ctx->parser.Scope->Name = name.Text; } // local_persist - char interface_arr_mem[ kilobytes(4) ] {0}; - Array interfaces = Array::init_reserve( Arena::init_from_memory(interface_arr_mem, kilobytes(4) ), 4 ); + char interface_arr_mem[ kilobytes(4) ] = {0}; + Array(CodeTypename) interfaces; { + Arena arena = arena_init_from_memory( interface_arr_mem, kilobytes(4) ); + interfaces = array_init_reserve(CodeTypename, arena_allocator_info(& arena), 4 ); + } // TODO(Ed) : Make an AST_DerivedType, we'll store any arbitary derived type into there as a linear linked list of them. - if ( check( TokType::Assign_Classifer ) ) + if ( check( Tok_Assign_Classifer ) ) { - eat( TokType::Assign_Classifer ); + eat( Tok_Assign_Classifer ); // : - if ( currtok.is_access_specifier() ) - { - access = currtok.to_access_specifier(); + if ( tok_is_access_specifier(currtok) ) { + access = tok_to_access_specifier(currtok); // : eat( currtok.Type ); } - Token parent_tok = parse_identifier(); - parent = def_type( parent_tok ); + Token parent_tok = parse_identifier(nullptr); + parent = def_type( tok_to_str(parent_tok) ); // : - while ( check(TokType::Comma) ) + while ( check(Tok_Comma) ) { - eat( TokType::Comma ); + eat( Tok_Comma ); // : , - if ( currtok.is_access_specifier() ) - { + if ( tok_is_access_specifier(currtok) ) { eat(currtok.Type); } - Token interface_tok = parse_identifier(); + Token interface_tok = parse_identifier(nullptr); - interfaces.append( def_type( interface_tok ) ); + array_append( interfaces, def_type( tok_to_str(interface_tok) ) ); // : , ... } } - if ( check( TokType::BraceCurly_Open ) ) - { + if ( check( Tok_BraceCurly_Open ) ) { body = parse_class_struct_body( which, name ); } // : , ... { } - CodeComment inline_cmt = NoCode; + CodeComment inline_cmt = NullCode; if ( ! inplace_def ) { Token stmt_end = currtok; - eat( TokType::Statement_End ); + eat( Tok_Statement_End ); // : , ... { }; - if ( currtok_noskip.Type == TokType::Comment && currtok_noskip.Line == stmt_end.Line ) + if ( currtok_noskip.Type == Tok_Comment && currtok_noskip.Line == stmt_end.Line ) inline_cmt = parse_comment(); - // : , ... { }; + // : , ... { }; } - if ( which == TokType::Decl_Class ) - result = def_class( name, body, parent, access, attributes, mflags ); + if ( which == Tok_Decl_Class ) + result = cast(Code, def_class( tok_to_str(name), def_assign( body, parent, access, attributes, interfaces, scast(s32, array_num(interfaces)), mflags ) )); else - result = def_struct( name, body, (CodeType)parent, access, attributes, mflags ); + result = cast(Code, def_struct( tok_to_str(name), def_assign( body, (CodeTypename)parent, access, attributes, interfaces, scast(s32, array_num(interfaces)), mflags ) )); if ( inline_cmt ) - result->InlineCmt = inline_cmt; + result->InlineCmt = cast(Code, inline_cmt); - interfaces.free(); + array_free(interfaces); return result; } internal neverinline CodeBody parse_class_struct_body( TokType which, Token name ) { - using namespace ECode; push_scope(); - eat( TokType::BraceCurly_Open ); + eat( Tok_BraceCurly_Open ); // { CodeBody result = (CodeBody) make_code(); - if ( which == TokType::Decl_Class ) - result->Type = Class_Body; + if ( which == Tok_Decl_Class ) + result->Type = CT_Class_Body; else - result->Type = Struct_Body; + result->Type = CT_Struct_Body; - while ( left && currtok_noskip.Type != TokType::BraceCurly_Close ) + while ( left && currtok_noskip.Type != Tok_BraceCurly_Close ) { - Code member = Code::Invalid; + Code member = Code_Invalid; CodeAttributes attributes = { nullptr }; CodeSpecifiers specifiers = { nullptr }; bool expects_function = false; - // Context.Scope->Start = currtok_noskip; + // _ctx->parser.Scope->Start = currtok_noskip; - if ( currtok_noskip.Type == TokType::Preprocess_Hash ) - eat( TokType::Preprocess_Hash ); + if ( currtok_noskip.Type == Tok_Preprocess_Hash ) + eat( Tok_Preprocess_Hash ); switch ( currtok_noskip.Type ) { - case TokType::Statement_End: - { + case Tok_Statement_End: { // TODO(Ed): Convert this to a general warning procedure - log_fmt("Dangling end statement found %S\n", currtok_noskip.to_string()); - eat( TokType::Statement_End ); + log_fmt("Dangling end statement found %SB\n", tok_to_strbuilder(currtok_noskip)); + eat( Tok_Statement_End ); continue; } - case TokType::NewLine: + case Tok_NewLine: { member = fmt_newline; - eat( TokType::NewLine ); - break; - - case TokType::Comment: - member = parse_comment(); - break; - - case TokType::Access_Public: + eat( Tok_NewLine ); + break; + } + case Tok_Comment: { + member = cast(Code, parse_comment()); + break; + } + case Tok_Access_Public: { member = access_public; - eat( TokType::Access_Public ); - eat( TokType::Assign_Classifer ); + eat( Tok_Access_Public ); + eat( Tok_Assign_Classifer ); // public: - break; - - case TokType::Access_Protected: + break; + } + case Tok_Access_Protected: { member = access_protected; - eat( TokType::Access_Protected ); - eat( TokType::Assign_Classifer ); + eat( Tok_Access_Protected ); + eat( Tok_Assign_Classifer ); // protected: - break; - - case TokType::Access_Private: + break; + } + case Tok_Access_Private: { member = access_private; - eat( TokType::Access_Private ); - eat( TokType::Assign_Classifer ); + eat( Tok_Access_Private ); + eat( Tok_Assign_Classifer ); // private: - break; - - case TokType::Decl_Class: - member = parse_complicated_definition( TokType::Decl_Class ); + break; + } + case Tok_Decl_Class: { + member = parse_complicated_definition( Tok_Decl_Class ); // class - break; - - case TokType::Decl_Enum: - member = parse_complicated_definition( TokType::Decl_Enum ); + break; + } + case Tok_Decl_Enum: { + member = parse_complicated_definition( Tok_Decl_Enum ); // enum - break; - - case TokType::Decl_Friend: - member = parse_friend(); + break; + } + case Tok_Decl_Friend: { + member = cast(Code, parser_parse_friend()); // friend - break; - - case TokType::Decl_Operator: - member = parse_operator_cast(); + break; + } + case Tok_Decl_Operator: { + member = cast(Code, parser_parse_operator_cast(NullCode)); // operator () - break; - - case TokType::Decl_Struct: - member = parse_complicated_definition( TokType::Decl_Struct ); + break; + } + case Tok_Decl_Struct: { + member = parse_complicated_definition( Tok_Decl_Struct ); // struct - break; - - case TokType::Decl_Template: - member = parse_template(); + break; + } + case Tok_Decl_Template: { + member = cast(Code, parser_parse_template()); // template< ... > - break; - - case TokType::Decl_Typedef: - member = parse_typedef(); + break; + } + case Tok_Decl_Typedef: { + member = cast(Code, parser_parse_typedef()); // typedef - break; - - case TokType::Decl_Union: - member = parse_complicated_definition( TokType::Decl_Union ); + break; + } + case Tok_Decl_Union: { + member = parse_complicated_definition( Tok_Decl_Union ); // union - break; - - case TokType::Decl_Using: - member = parse_using(); + break; + } + case Tok_Decl_Using: { + member = cast(Code, parser_parse_using()); // using - break; + break; + } + case Tok_Operator: + { + //if ( currtok.Text[0] != '~' ) + //{ + // log_failure( "Operator token found in global body but not destructor unary negation\n%s", to_strbuilder(_ctx->parser) ); + // return InvalidCode; + //} - case TokType::Operator: - if ( currtok.Text[0] != '~' ) - { - log_failure( "Operator token found in global body but not destructor unary negation\n%s", Context.to_string() ); - return CodeInvalid; - } - - member = parse_destructor(); + member = cast(Code, parser_parse_destructor(NullCode)); // ~() - break; - - case TokType::Preprocess_Define: - member = parse_define(); + break; + } + case Tok_Preprocess_Define: { + member = cast(Code, parser_parse_define()); // #define - break; - - case TokType::Preprocess_Include: - member = parse_include(); + break; + } + case Tok_Preprocess_Include: + { + member = cast(Code, parse_include()); // #include - break; + break; + } - case TokType::Preprocess_If: - case TokType::Preprocess_IfDef: - case TokType::Preprocess_IfNotDef: - case TokType::Preprocess_ElIf: - member = parse_preprocess_cond(); + case Tok_Preprocess_If: + case Tok_Preprocess_IfDef: + case Tok_Preprocess_IfNotDef: + case Tok_Preprocess_ElIf: + member = cast(Code, parse_preprocess_cond()); // # break; - case TokType::Preprocess_Else: - member = preprocess_else; - eat( TokType::Preprocess_Else ); + case Tok_Preprocess_Else: { + member = cast(Code, preprocess_else); + eat( Tok_Preprocess_Else ); // #else - break; - - case TokType::Preprocess_EndIf: - member = preprocess_endif; - eat( TokType::Preprocess_EndIf ); + break; + } + case Tok_Preprocess_EndIf: { + member = cast(Code, preprocess_endif); + eat( Tok_Preprocess_EndIf ); // #endif - break; + break; + } - case TokType::Preprocess_Macro: - member = parse_simple_preprocess( TokType::Preprocess_Macro ); - // - break; + case Tok_Preprocess_Macro_Stmt: { + member = cast(Code, parse_simple_preprocess( Tok_Preprocess_Macro_Stmt )); + break; + } + case Tok_Preprocess_Macro_Expr: { + log_failure("Unbounded macro expression residing in class/struct body\n%S", parser_to_strbuilder(_ctx->parser)); + return InvalidCode; + } - case TokType::Preprocess_Pragma: - member = parse_pragma(); + // case Tok_Preprocess_Macro: + // // + // macro_found = true; + // goto Preprocess_Macro_Bare_In_Body; + // break; + + case Tok_Preprocess_Pragma: { + member = cast(Code, parse_pragma()); // #pragma - break; + break; + } - case TokType::Preprocess_Unsupported: - member = parse_simple_preprocess( TokType::Preprocess_Unsupported ); + case Tok_Preprocess_Unsupported: { + member = cast(Code, parse_simple_preprocess( Tok_Preprocess_Unsupported )); // # - break; + break; + } - case TokType::StaticAssert: + case Tok_StaticAssert: { member = parse_static_assert(); // static_assert - break; + break; + } - case TokType::Attribute_Open: - case TokType::Decl_GNU_Attribute: - case TokType::Decl_MSVC_Attribute: - #define Entry( attribute, str ) case TokType::attribute: + case Tok_Attribute_Open: + case Tok_Decl_GNU_Attribute: + case Tok_Decl_MSVC_Attribute: + #define Entry( attribute, str ) case attribute: GEN_DEFINE_ATTRIBUTE_TOKENS #undef Entry { @@ -8125,53 +8017,34 @@ CodeBody parse_class_struct_body( TokType which, Token name ) // } //! Fallthrough intended - case TokType::Spec_Consteval: - case TokType::Spec_Constexpr: - case TokType::Spec_Constinit: - case TokType::Spec_Explicit: - case TokType::Spec_ForceInline: - case TokType::Spec_Inline: - case TokType::Spec_Mutable: - case TokType::Spec_NeverInline: - case TokType::Spec_Static: - case TokType::Spec_Volatile: - case TokType::Spec_Virtual: + GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_TOK_SPECIFIERS_CASES: { - SpecifierT specs_found[16] { ESpecifier::NumSpecifiers }; + Specifier specs_found[16] = { Spec_NumSpecifiers }; s32 NumSpecifiers = 0; - while ( left && currtok.is_specifier() ) + while ( left && tok_is_specifier(currtok) ) { - SpecifierT spec = ESpecifier::to_type( currtok ); + Specifier spec = str_to_specifier( tok_to_str(currtok) ); b32 ignore_spec = false; switch ( spec ) { - case ESpecifier::Constexpr: - case ESpecifier::Constinit: - case ESpecifier::Explicit: - case ESpecifier::Inline: - case ESpecifier::ForceInline: - case ESpecifier::Mutable: - case ESpecifier::NeverInline: - case ESpecifier::Static: - case ESpecifier::Volatile: - case ESpecifier::Virtual: + GEN_PARSER_CLASS_STRUCT_BODY_ALLOWED_MEMBER_SPECIFIERS_CASES: break; - case ESpecifier::Consteval: + case Spec_Consteval: expects_function = true; break; - case ESpecifier::Const : + case Spec_Const : ignore_spec = true; break; default: - log_failure( "Invalid specifier %s for variable\n%s", ESpecifier::to_str(spec), Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Invalid specifier %S for class/struct member\n%S", spec_to_str(spec), strbuilder_to_str( parser_to_strbuilder(_ctx->parser)) ); + parser_pop(& _ctx->parser); + return InvalidCode; } // Every specifier after would be considered part of the type type signature @@ -8189,17 +8062,18 @@ CodeBody parse_class_struct_body( TokType which, Token name ) } // - if ( currtok.is_attribute() ) + if ( tok_is_attribute(currtok) ) { // Unfortuantely Unreal has code where there is attirbutes before specifiers CodeAttributes more_attributes = parse_attributes(); if ( attributes ) { - String fused = String::make_reserve( GlobalAllocator, attributes->Content.length() + more_attributes->Content.length() ); - fused.append_fmt( "%S %S", attributes->Content, more_attributes->Content ); + StrBuilder fused = strbuilder_make_reserve( _ctx->Allocator_Temp, attributes->Content.Len + more_attributes->Content.Len ); + strbuilder_append_fmt( & fused, "%SB %SB", attributes->Content, more_attributes->Content ); - attributes->Name = get_cached_string(fused); + Str attrib_name = strbuilder_to_str(fused); + attributes->Name = cache_str( attrib_name ); attributes->Content = attributes->Name; // } @@ -8207,37 +8081,38 @@ CodeBody parse_class_struct_body( TokType which, Token name ) attributes = more_attributes; } - if ( currtok.Type == TokType::Operator && currtok.Text[0] == '~' ) + if ( currtok.Type == Tok_Operator && currtok.Text.Ptr[0] == '~' ) { - member = parse_destructor( specifiers ); + member = cast(Code, parser_parse_destructor( specifiers )); // ~() break; } - if ( currtok.Type == TokType::Decl_Operator ) + if ( currtok.Type == Tok_Decl_Operator ) { - member = parse_operator_cast( specifiers ); + member = cast(Code, parser_parse_operator_cast( specifiers )); // operator () break; } } //! Fallthrough intentional - case TokType::Identifier: - case TokType::Spec_Const: - case TokType::Type_Unsigned: - case TokType::Type_Signed: - case TokType::Type_Short: - case TokType::Type_Long: - case TokType::Type_bool: - case TokType::Type_char: - case TokType::Type_int: - case TokType::Type_double: + case Tok_Identifier: + case Tok_Preprocess_Macro_Typename: + case Tok_Spec_Const: + case Tok_Type_Unsigned: + case Tok_Type_Signed: + case Tok_Type_Short: + case Tok_Type_Long: + case Tok_Type_bool: + case Tok_Type_char: + case Tok_Type_int: + case Tok_Type_double: { - if ( nexttok.Type == TokType::Capture_Start && name.Length && currtok.Type == TokType::Identifier ) + if ( nexttok.Type == Tok_Capture_Start && name.Text.Len && currtok.Type == Tok_Identifier ) { - if ( str_compare( name.Text, currtok.Text, name.Length ) == 0 ) + if ( c_str_compare_len( name.Text.Ptr, currtok.Text.Ptr, name.Text.Len ) == 0 ) { - member = parse_constructor( specifiers ); + member = cast(Code, parser_parse_constructor( specifiers )); // () break; } @@ -8252,49 +8127,44 @@ CodeBody parse_class_struct_body( TokType which, Token name ) default: Token untyped_tok = currtok; - - while ( left && currtok.Type != TokType::BraceCurly_Close ) + while ( left && currtok.Type != Tok_BraceCurly_Close ) { - untyped_tok.Length = ( (sptr)currtok.Text + currtok.Length ) - (sptr)untyped_tok.Text; + untyped_tok.Text.Len = ( (sptr)currtok.Text.Ptr + currtok.Text.Len ) - (sptr)untyped_tok.Text.Ptr; eat( currtok.Type ); } - - member = untyped_str( untyped_tok ); + member = untyped_str( tok_to_str(untyped_tok) ); // Something unknown break; } - if ( member == Code::Invalid ) + if ( member == Code_Invalid ) { - log_failure( "Failed to parse member\n%s", Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Failed to parse member\n%s", parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } - - result.append( member ); + body_append(result, member ); } - eat( TokType::BraceCurly_Close ); + eat( Tok_BraceCurly_Close ); // { } - Context.pop(); + parser_pop(& _ctx->parser); return result; } internal CodeComment parse_comment() { - StackNode scope { nullptr, currtok_noskip, NullToken, txt( __func__ ) }; - Context.push( & scope ); + push_scope(); CodeComment result = (CodeComment) make_code(); - result->Type = ECode::Comment; - result->Content = get_cached_string( currtok_noskip ); - result->Name = result->Content; + result->Type = CT_Comment; + result->Content = cache_str( tok_to_str(currtok_noskip) ); // result->Token = currtok_noskip; - eat( TokType::Comment ); + eat( Tok_Comment ); - Context.pop(); + parser_pop(& _ctx->parser); return result; } @@ -8305,19 +8175,19 @@ Code parse_complicated_definition( TokType which ) bool is_inplace = false; - TokArray tokens = Context.Tokens; + TokArray tokens = _ctx->parser.Tokens; s32 idx = tokens.Idx; s32 level = 0; - for ( ; idx < tokens.Arr.num(); idx++ ) + for ( ; idx < array_num(tokens.Arr); idx++ ) { - if ( tokens[ idx ].Type == TokType::BraceCurly_Open ) + if ( tokens.Arr[ idx ].Type == Tok_BraceCurly_Open ) level++; - if ( tokens[ idx ].Type == TokType::BraceCurly_Close ) + if ( tokens.Arr[ idx ].Type == Tok_BraceCurly_Close ) level--; - if ( level == 0 && tokens[ idx ].Type == TokType::Statement_End ) + if ( level == 0 && tokens.Arr[ idx ].Type == Tok_Statement_End ) break; } @@ -8326,68 +8196,68 @@ Code parse_complicated_definition( TokType which ) // Its a forward declaration only Code result = parse_forward_or_definition( which, is_inplace ); // ; - Context.pop(); + parser_pop(& _ctx->parser); return result; } - Token tok = tokens[ idx - 1 ]; - if ( tok.is_specifier() && is_trailing( ESpecifier::to_type(tok)) ) + Token tok = tokens.Arr[ idx - 1 ]; + if ( tok_is_specifier(tok) && spec_is_trailing( str_to_specifier( tok_to_str(tok))) ) { // (...) ...; s32 spec_idx = idx - 1; - Token spec = tokens[spec_idx]; - while ( spec.is_specifier() && is_trailing( ESpecifier::to_type(spec)) ) + Token spec = tokens.Arr[spec_idx]; + while ( tok_is_specifier(spec) && spec_is_trailing( str_to_specifier( tok_to_str(spec))) ) { -- spec_idx; - spec = tokens[spec_idx]; + spec = tokens.Arr[spec_idx]; } - if ( tokens[spec_idx].Type == TokType::Capture_End ) + if ( tokens.Arr[spec_idx].Type == Tok_Capture_End ) { // Forward declaration with trailing specifiers for a procedure - tok = tokens[spec_idx]; + tok = tokens.Arr[spec_idx]; - Code result = parse_operator_function_or_variable( false, { nullptr }, { nullptr } ); + Code result = parse_operator_function_or_variable( false, NullCode, NullCode ); // , or Name> ... - Context.pop(); + parser_pop(& _ctx->parser); return result; } - log_failure( "Unsupported or bad member definition after %s declaration\n%s", to_str(which), Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Unsupported or bad member definition after %s declaration\n%s", toktype_to_str(which), parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } - if ( tok.Type == TokType::Identifier ) + if ( tok.Type == Tok_Identifier ) { - tok = tokens[ idx - 2 ]; - bool is_indirection = tok.Type == TokType::Ampersand || tok.Type == TokType::Star; + tok = tokens.Arr[ idx - 2 ]; + bool is_indirection = tok.Type == Tok_Ampersand || tok.Type == Tok_Star; bool ok_to_parse = false; - if ( tok.Type == TokType::BraceCurly_Close ) + if ( tok.Type == Tok_BraceCurly_Close ) { // Its an inplace definition // { ... } ; ok_to_parse = true; is_inplace = true; } - else if ( tok.Type == TokType::Identifier && tokens[ idx - 3 ].Type == which ) + else if ( tok.Type == Tok_Identifier && tokens.Arr[ idx - 3 ].Type == which ) { // Its a variable with type ID using namespace. // ; ok_to_parse = true; } - else if ( tok.Type == TokType::Assign_Classifer - && ( ( tokens[idx - 5].Type == which && tokens[idx - 4].Type == TokType::Decl_Class ) - || ( tokens[idx - 4].Type == which)) + else if ( tok.Type == Tok_Assign_Classifer + && ( ( tokens.Arr[idx - 5].Type == which && tokens.Arr[idx - 4].Type == Tok_Decl_Class ) + || ( tokens.Arr[idx - 4].Type == which)) ) { // Its a forward declaration of an enum // : ; // : ; ok_to_parse = true; - Code result = parse_enum(); - Context.pop(); + Code result = cast(Code, parser_parse_enum( ! parser_inplace_def)); + parser_pop(& _ctx->parser); return result; } else if ( is_indirection ) @@ -8399,143 +8269,95 @@ Code parse_complicated_definition( TokType which ) if ( ! ok_to_parse ) { - log_failure( "Unsupported or bad member definition after %s declaration\n%s", to_str(which), Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Unsupported or bad member definition after %s declaration\n%s", toktype_to_str(which), parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } - Code result = parse_operator_function_or_variable( false, { nullptr }, { nullptr } ); + Code result = parse_operator_function_or_variable( false, NullCode, NullCode ); // , or Name> ... - Context.pop(); + parser_pop(& _ctx->parser); return result; } - else if ( tok.Type >= TokType::Type_Unsigned && tok.Type <= TokType::Type_MS_W64 ) + else if ( tok.Type >= Tok_Type_Unsigned && tok.Type <= Tok_Type_MS_W64 ) { - tok = tokens[ idx - 2 ]; + tok = tokens.Arr[ idx - 2 ]; - if ( tok.Type != TokType::Assign_Classifer - || ( ( tokens[idx - 5].Type != which && tokens[idx - 4].Type != TokType::Decl_Class ) - && ( tokens[idx - 4].Type != which)) + if ( tok.Type != Tok_Assign_Classifer + || ( ( tokens.Arr[idx - 5].Type != which && tokens.Arr[idx - 4].Type != Tok_Decl_Class ) + && ( tokens.Arr[idx - 4].Type != which)) ) { - log_failure( "Unsupported or bad member definition after %s declaration\n%s", to_str(which), Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Unsupported or bad member definition after %s declaration\n%s", toktype_to_str(which), parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } // Its a forward declaration of an enum class // : ; // : ; - Code result = parse_enum(); - Context.pop(); + Code result = cast(Code, parser_parse_enum( ! parser_inplace_def)); + parser_pop(& _ctx->parser); return result; } - else if ( tok.Type == TokType::BraceCurly_Close ) + else if ( tok.Type == Tok_BraceCurly_Close ) { // Its a definition Code result = parse_forward_or_definition( which, is_inplace ); // { ... }; - Context.pop(); + parser_pop(& _ctx->parser); return result; } - else if ( tok.Type == TokType::BraceSquare_Close ) + else if ( tok.Type == Tok_BraceSquare_Close ) { // Its an array definition - Code result = parse_operator_function_or_variable( false, { nullptr }, { nullptr } ); + Code result = parse_operator_function_or_variable( false, NullCode, NullCode ); // [ ... ]; - Context.pop(); + parser_pop(& _ctx->parser); return result; } else { - log_failure( "Unsupported or bad member definition after %s declaration\n%S", to_str(which).Ptr, Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Unsupported or bad member definition after %s declaration\n%SB", toktype_to_str(which).Ptr, parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } } -internal inline -CodeDefine parse_define() -{ - push_scope(); - eat( TokType::Preprocess_Define ); - // #define - - CodeDefine - define = (CodeDefine) make_code(); - define->Type = ECode::Preprocess_Define; - - if ( ! check( TokType::Identifier ) ) - { - log_failure( "Error, expected identifier after #define\n%s", Context.to_string() ); - Context.pop(); - return CodeInvalid; - } - - Context.Scope->Name = currtok; - define->Name = get_cached_string( currtok ); - eat( TokType::Identifier ); - // #define - - if ( ! check( TokType::Preprocess_Content )) - { - log_failure( "Error, expected content after #define %s\n%s", define->Name, Context.to_string() ); - Context.pop(); - return CodeInvalid; - } - - if ( currtok.Length == 0 ) - { - define->Content = get_cached_string( currtok ); - eat( TokType::Preprocess_Content ); - // #define - - Context.pop(); - return define; - } - - define->Content = get_cached_string( strip_formatting( currtok, strip_formatting_dont_preserve_newlines ) ); - eat( TokType::Preprocess_Content ); - // #define - - Context.pop(); - return define; -} - internal inline Code parse_assignment_expression() { Code expr = { nullptr }; - eat( TokType::Operator ); + eat( Tok_Operator ); // = Token expr_tok = currtok; - if ( currtok.Type == TokType::Statement_End && currtok.Type != TokType::Comma ) + if ( currtok.Type == Tok_Statement_End && currtok.Type != Tok_Comma ) { - log_failure( "Expected expression after assignment operator\n%s", Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Expected expression after assignment operator\n%s", parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } s32 level = 0; - while ( left && currtok.Type != TokType::Statement_End && (currtok.Type != TokType::Comma || level > 0) ) + while ( left && currtok.Type != Tok_Statement_End && (currtok.Type != Tok_Comma || level > 0) ) { - if (currtok.Type == TokType::BraceCurly_Open ) + if (currtok.Type == Tok_BraceCurly_Open ) level++; - if (currtok.Type == TokType::BraceCurly_Close ) + if (currtok.Type == Tok_BraceCurly_Close ) level--; - if (currtok.Type == TokType::Capture_Start) + if (currtok.Type == Tok_Capture_Start) level++; - else if (currtok.Type == TokType::Capture_End) + else if (currtok.Type == Tok_Capture_End) level--; eat( currtok.Type ); } - expr_tok.Length = ( ( sptr )currtok.Text + currtok.Length ) - ( sptr )expr_tok.Text - 1; - expr = untyped_str( expr_tok ); + expr_tok.Text.Len = ( ( sptr )currtok.Text.Ptr + currtok.Text.Len ) - ( sptr )expr_tok.Text.Ptr - 1; + expr = untyped_str( tok_to_str(expr_tok) ); // = return expr; } @@ -8543,32 +8365,32 @@ Code parse_assignment_expression() internal inline Code parse_forward_or_definition( TokType which, bool is_inplace ) { - Code result = CodeInvalid; + Code result = InvalidCode; switch ( which ) { - case TokType::Decl_Class: - result = parse_class( is_inplace ); + case Tok_Decl_Class: + result = cast(Code, parser_parse_class( is_inplace )); return result; - case TokType::Decl_Enum: - result = parse_enum( is_inplace ); + case Tok_Decl_Enum: + result = cast(Code, parser_parse_enum( is_inplace )); return result; - case TokType::Decl_Struct: - result = parse_struct( is_inplace ); + case Tok_Decl_Struct: + result = cast(Code, parser_parse_struct( is_inplace )); return result; - case TokType::Decl_Union: - result = parse_union( is_inplace ); + case Tok_Decl_Union: + result = cast(Code, parser_parse_union( is_inplace )); return result; default: log_failure( "Error, wrong token type given to parse_complicated_definition " "(only supports class, enum, struct, union) \n%s" - , Context.to_string() ); + , parser_to_strbuilder(_ctx->parser) ); - return CodeInvalid; + return InvalidCode; } } @@ -8578,99 +8400,97 @@ CodeFn parse_function_after_name( ModuleFlag mflags , CodeAttributes attributes , CodeSpecifiers specifiers - , CodeType ret_type + , CodeTypename ret_type , Token name ) { push_scope(); - CodeParam params = parse_params(); + CodeParams params = parse_params(parser_use_parenthesis); // ( ) - // TODO(Ed), Review old comment : These have to be kept separate from the return type's specifiers. - while ( left && currtok.is_specifier() ) + // TODO(Ed), Review old comment : These have to be kept separate from the return type's specifiers. + while ( left && tok_is_specifier(currtok) ) { - if ( specifiers.ast == nullptr ) + if ( specifiers == nullptr ) { - specifiers = def_specifier( ESpecifier::to_type(currtok) ); + specifiers = def_specifier( str_to_specifier( tok_to_str(currtok)) ); eat( currtok.Type ); continue; } - specifiers.append( ESpecifier::to_type(currtok) ); + specifiers_append(specifiers, str_to_specifier( tok_to_str(currtok)) ); eat( currtok.Type ); } // ( ) - CodeBody body = NoCode; - CodeComment inline_cmt = NoCode; - if ( check( TokType::BraceCurly_Open ) ) + CodeBody body = NullCode; + CodeComment inline_cmt = NullCode; + if ( check( Tok_BraceCurly_Open ) ) { - body = parse_function_body(); - if ( body == Code::Invalid ) + body = cast(CodeBody, parse_function_body()); + if ( cast(Code, body) == Code_Invalid ) { - Context.pop(); - return CodeInvalid; + parser_pop(& _ctx->parser); + return InvalidCode; } // ( ) { } } - else if ( check(TokType::Operator) && currtok.Text[0] == '=' ) + else if ( check(Tok_Operator) && currtok.Text.Ptr[0] == '=' ) { - eat(TokType::Operator); - specifiers.append( ESpecifier::Pure ); + eat(Tok_Operator); + specifiers_append(specifiers, Spec_Pure ); - eat( TokType::Number); + eat( Tok_Number); Token stmt_end = currtok; - eat( TokType::Statement_End ); + eat( Tok_Statement_End ); // ( ) = 0; - if ( currtok_noskip.Type == TokType::Comment && currtok_noskip.Line == stmt_end.Line ) + if ( currtok_noskip.Type == Tok_Comment && currtok_noskip.Line == stmt_end.Line ) inline_cmt = parse_comment(); // ( ) ; } else { Token stmt_end = currtok; - eat( TokType::Statement_End ); + eat( Tok_Statement_End ); // ( ) ; - if ( currtok_noskip.Type == TokType::Comment && currtok_noskip.Line == stmt_end.Line ) + if ( currtok_noskip.Type == Tok_Comment && currtok_noskip.Line == stmt_end.Line ) inline_cmt = parse_comment(); // ( ) ; } - using namespace ECode; - - String - name_stripped = String::make( GlobalAllocator, name ); - name_stripped.strip_space(); + StrBuilder + name_stripped = strbuilder_make_str( _ctx->Allocator_Temp, tok_to_str(name) ); + strbuilder_strip_space(name_stripped); CodeFn result = (CodeFn) make_code(); - result->Name = get_cached_string( name_stripped ); + result->Name = cache_str( strbuilder_to_str(name_stripped) ); result->ModuleFlags = mflags; if ( body ) { switch ( body->Type ) { - case Function_Body: - case Untyped: + case CT_Function_Body: + case CT_Untyped: break; default: { - log_failure("Body must be either of Function_Body or Untyped type, %s\n%s", body.debug_str(), Context.to_string()); - Context.pop(); - return CodeInvalid; + log_failure("Body must be either of Function_Body or Untyped type, %s\n%s", code_debug_str(body), parser_to_strbuilder(_ctx->parser)); + parser_pop(& _ctx->parser); + return InvalidCode; } } - result->Type = Function; + result->Type = CT_Function; result->Body = body; } else { - result->Type = Function_Fwd; + result->Type = CT_Function_Fwd; } if ( attributes ) @@ -8687,218 +8507,233 @@ CodeFn parse_function_after_name( if ( inline_cmt ) result->InlineCmt = inline_cmt; - Context.pop(); + parser_pop(& _ctx->parser); return result; } internal Code parse_function_body() { - using namespace ECode; push_scope(); - eat( TokType::BraceCurly_Open ); + eat( Tok_BraceCurly_Open ); CodeBody result = (CodeBody) make_code(); - result->Type = Function_Body; + result->Type = CT_Function_Body; // TODO : Support actual parsing of function body Token start = currtok_noskip; s32 level = 0; - while ( left && ( currtok_noskip.Type != TokType::BraceCurly_Close || level > 0 ) ) + while ( left && ( currtok_noskip.Type != Tok_BraceCurly_Close || level > 0 ) ) { - if ( currtok_noskip.Type == TokType::BraceCurly_Open ) + if ( currtok_noskip.Type == Tok_BraceCurly_Open ) level++; - else if ( currtok_noskip.Type == TokType::BraceCurly_Close && level > 0 ) + else if ( currtok_noskip.Type == Tok_BraceCurly_Close && level > 0 ) level--; eat( currtok_noskip.Type ); } - Token previous = prevtok; + Token past = prevtok; - s32 len = ( (sptr)prevtok.Text + prevtok.Length ) - (sptr)start.Text; + s32 len = ( (sptr)prevtok.Text.Ptr + prevtok.Text.Len ) - (sptr)start.Text.Ptr; if ( len > 0 ) { - result.append( def_execution( { len, start.Text } ) ); + Str str = { start.Text.Ptr, len }; + body_append( result, cast(Code, def_execution( str )) ); } - eat( TokType::BraceCurly_Close ); + eat( Tok_BraceCurly_Close ); - Context.pop(); - return result; + parser_pop(& _ctx->parser); + return cast(Code, result); } internal neverinline -CodeBody parse_global_nspace( CodeT which ) +CodeBody parse_global_nspace( CodeType which ) { - using namespace ECode; - push_scope(); - if ( which != Namespace_Body && which != Global_Body && which != Export_Body && which != Extern_Linkage_Body ) - return CodeInvalid; + if ( which != CT_Namespace_Body && which != CT_Global_Body && which != CT_Export_Body && which != CT_Extern_Linkage_Body ) + return InvalidCode; - if ( which != Global_Body ) - eat( TokType::BraceCurly_Open ); + if ( which != CT_Global_Body ) + eat( Tok_BraceCurly_Open ); // { CodeBody result = (CodeBody) make_code(); result->Type = which; - while ( left && currtok_noskip.Type != TokType::BraceCurly_Close ) + while ( left && currtok_noskip.Type != Tok_BraceCurly_Close ) { - Code member = Code::Invalid; + Code member = Code_Invalid; CodeAttributes attributes = { nullptr }; CodeSpecifiers specifiers = { nullptr }; bool expects_function = false; - // Context.Scope->Start = currtok_noskip; + // _ctx->parser.Scope->Start = currtok_noskip; - if ( currtok_noskip.Type == TokType::Preprocess_Hash ) - eat( TokType::Preprocess_Hash ); + if ( currtok_noskip.Type == Tok_Preprocess_Hash ) + eat( Tok_Preprocess_Hash ); + + b32 macro_found = false; switch ( currtok_noskip.Type ) { - case TokType::Statement_End: + case Tok_Comma: + { + log_failure("Dangling comma found: %SB\nContext:\n%SB", tok_to_strbuilder(currtok), parser_to_strbuilder(_ctx->parser)); + parser_pop( & _ctx->parser); + return InvalidCode; + } + break; + case Tok_Statement_End: { // TODO(Ed): Convert this to a general warning procedure - log_fmt("Dangling end statement found %S\n", currtok_noskip.to_string()); - eat( TokType::Statement_End ); + log_fmt("Dangling end statement found %SB\n", tok_to_strbuilder(currtok_noskip)); + eat( Tok_Statement_End ); continue; } - case TokType::NewLine: + case Tok_NewLine: // Empty lines are auto skipped by Tokens.current() member = fmt_newline; - eat( TokType::NewLine ); + eat( Tok_NewLine ); break; - case TokType::Comment: - member = parse_comment(); + case Tok_Comment: + member = cast(Code, parse_comment()); break; - case TokType::Decl_Class: - member = parse_complicated_definition( TokType::Decl_Class ); + case Tok_Decl_Class: + member = parse_complicated_definition( Tok_Decl_Class ); // class break; - case TokType::Decl_Enum: - member = parse_complicated_definition( TokType::Decl_Enum ); + case Tok_Decl_Enum: + member = parse_complicated_definition( Tok_Decl_Enum ); // enum break; - case TokType::Decl_Extern_Linkage: - if ( which == Extern_Linkage_Body ) - log_failure( "Nested extern linkage\n%s", Context.to_string() ); + case Tok_Decl_Extern_Linkage: + if ( which == CT_Extern_Linkage_Body ) + log_failure( "Nested extern linkage\n%s", parser_to_strbuilder(_ctx->parser) ); - member = parse_extern_link(); + member = cast(Code, parser_parse_extern_link()); // extern "..." { ... } break; - case TokType::Decl_Namespace: - member = parse_namespace(); + case Tok_Decl_Namespace: + member = cast(Code, parser_parse_namespace()); // namespace { ... } break; - case TokType::Decl_Struct: - member = parse_complicated_definition( TokType::Decl_Struct ); + case Tok_Decl_Struct: + member = parse_complicated_definition( Tok_Decl_Struct ); // struct ... break; - case TokType::Decl_Template: - member = parse_template(); + case Tok_Decl_Template: + member = cast(Code, parser_parse_template()); // template<...> ... break; - case TokType::Decl_Typedef: - member = parse_typedef(); + case Tok_Decl_Typedef: + member = cast(Code, parser_parse_typedef()); // typedef ... break; - case TokType::Decl_Union: - member = parse_complicated_definition( TokType::Decl_Union ); + case Tok_Decl_Union: + member = parse_complicated_definition( Tok_Decl_Union ); // union ... break; - case TokType::Decl_Using: - member = parse_using(); + case Tok_Decl_Using: + member = cast(Code, parser_parse_using()); // using ... break; - case TokType::Preprocess_Define: - member = parse_define(); + case Tok_Preprocess_Define: + member = cast(Code, parser_parse_define()); // #define ... break; - case TokType::Preprocess_Include: - member = parse_include(); + case Tok_Preprocess_Include: + member = cast(Code, parse_include()); // #include ... break; - case TokType::Preprocess_If: - case TokType::Preprocess_IfDef: - case TokType::Preprocess_IfNotDef: - case TokType::Preprocess_ElIf: - member = parse_preprocess_cond(); + case Tok_Preprocess_If: + case Tok_Preprocess_IfDef: + case Tok_Preprocess_IfNotDef: + case Tok_Preprocess_ElIf: + member = cast(Code, parse_preprocess_cond()); // # ... break; - case TokType::Preprocess_Else: - member = preprocess_else; - eat( TokType::Preprocess_Else ); + case Tok_Preprocess_Else: + member = cast(Code, preprocess_else); + eat( Tok_Preprocess_Else ); // #else break; - case TokType::Preprocess_EndIf: - member = preprocess_endif; - eat( TokType::Preprocess_EndIf ); + case Tok_Preprocess_EndIf: + member = cast(Code, preprocess_endif); + eat( Tok_Preprocess_EndIf ); // #endif break; - case TokType::Preprocess_Macro: - member = parse_simple_preprocess( TokType::Preprocess_Macro ); - // - break; + case Tok_Preprocess_Macro_Stmt: { + member = cast(Code, parse_simple_preprocess( Tok_Preprocess_Macro_Stmt )); + break; + } + case Tok_Preprocess_Macro_Expr: { + log_failure("Unbounded macro expression residing in class/struct body\n%S", parser_to_strbuilder(_ctx->parser)); + return InvalidCode; + } - case TokType::Preprocess_Pragma: - member = parse_pragma(); + case Tok_Preprocess_Pragma: { + member = cast(Code, parse_pragma()); // #pragma ... + } break; - case TokType::Preprocess_Unsupported: - member = parse_simple_preprocess( TokType::Preprocess_Unsupported ); + case Tok_Preprocess_Unsupported: { + member = cast(Code, parse_simple_preprocess( Tok_Preprocess_Unsupported )); // # ... + } break; - case TokType::StaticAssert: - member = parse_static_assert(); + case Tok_StaticAssert: { + member = cast(Code, parse_static_assert()); // static_assert( , ... ); + } break; - case TokType::Module_Export: - if ( which == Export_Body ) - log_failure( "Nested export declaration\n%s", Context.to_string() ); + case Tok_Module_Export: { + if ( which == CT_Export_Body ) + log_failure( "Nested export declaration\n%s", parser_to_strbuilder(_ctx->parser) ); - member = parse_export_body(); + member = cast(Code, parser_parse_export_body()); // export { ... } + } break; - case TokType::Module_Import: - { - not_implemented( context ); + case Tok_Module_Import: { // import ... + log_failure( "gen::%s: This function is not implemented" ); + return InvalidCode; } //! Fallthrough intentional - case TokType::Attribute_Open: - case TokType::Decl_GNU_Attribute: - case TokType::Decl_MSVC_Attribute: - #define Entry( attribute, str ) case TokType::attribute: + case Tok_Attribute_Open: + case Tok_Decl_GNU_Attribute: + case Tok_Decl_MSVC_Attribute: + #define Entry( attribute, str ) case attribute: GEN_DEFINE_ATTRIBUTE_TOKENS #undef Entry { @@ -8906,55 +8741,36 @@ CodeBody parse_global_nspace( CodeT which ) // } //! Fallthrough intentional - case TokType::Spec_Consteval: - case TokType::Spec_Constexpr: - case TokType::Spec_Constinit: - case TokType::Spec_Extern: - case TokType::Spec_ForceInline: - case TokType::Spec_Global: - case TokType::Spec_Inline: - case TokType::Spec_Internal_Linkage: - case TokType::Spec_NeverInline: - case TokType::Spec_Static: + GEN_PARSER_CLASS_GLOBAL_NSPACE_ALLOWED_MEMBER_TOK_SPECIFIER_CASES: { - SpecifierT specs_found[16] { ESpecifier::NumSpecifiers }; + Specifier specs_found[16] = { Spec_NumSpecifiers }; s32 NumSpecifiers = 0; - while ( left && currtok.is_specifier() ) + while ( left && tok_is_specifier(currtok) ) { - SpecifierT spec = ESpecifier::to_type( currtok ); + Specifier spec = str_to_specifier( tok_to_str(currtok) ); bool ignore_spec = false; switch ( spec ) { - case ESpecifier::Constexpr: - case ESpecifier::Constinit: - case ESpecifier::ForceInline: - case ESpecifier::Global: - case ESpecifier::External_Linkage: - case ESpecifier::Internal_Linkage: - case ESpecifier::Inline: - case ESpecifier::Mutable: - case ESpecifier::NeverInline: - case ESpecifier::Static: - case ESpecifier::Volatile: + GEN_PARSER_CLASS_GLOBAL_NSPACE_ALLOWED_MEMBER_SPECIFIER_CASES: break; - case ESpecifier::Consteval: + case Spec_Consteval: expects_function = true; break; - case ESpecifier::Const: + case Spec_Const: ignore_spec = true; break; default: - StrC spec_str = ESpecifier::to_str(spec); + Str spec_str = spec_to_str(spec); - log_failure( "Invalid specifier %.*s for variable\n%s", spec_str.Len, spec_str, Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Invalid specifier %S for variable\n%S", spec_str, strbuilder_to_str( parser_to_strbuilder(_ctx->parser)) ); + parser_pop(& _ctx->parser); + return InvalidCode; } if (ignore_spec) @@ -8972,53 +8788,57 @@ CodeBody parse_global_nspace( CodeT which ) // } //! Fallthrough intentional - case TokType::Identifier: - case TokType::Spec_Const: - case TokType::Type_Long: - case TokType::Type_Short: - case TokType::Type_Signed: - case TokType::Type_Unsigned: - case TokType::Type_bool: - case TokType::Type_char: - case TokType::Type_double: - case TokType::Type_int: + case Tok_Identifier: + case Tok_Preprocess_Macro_Typename: + case Tok_Spec_Const: + case Tok_Type_Long: + case Tok_Type_Short: + case Tok_Type_Signed: + case Tok_Type_Unsigned: + case Tok_Type_bool: + case Tok_Type_char: + case Tok_Type_double: + case Tok_Type_int: { - Code constructor_destructor = parse_global_nspace_constructor_destructor( specifiers ); - // Possible constructor implemented at global file scope. - if ( constructor_destructor ) + // This s only in a scope so that Preprocess_Macro_Bare_In_Body works without microsoft extension warnings { - member = constructor_destructor; - break; - } - - bool found_operator_cast_outside_class_implmentation = false; - s32 idx = Context.Tokens.Idx; - - for ( ; idx < Context.Tokens.Arr.num(); idx++ ) - { - Token tok = Context.Tokens[ idx ]; - - if ( tok.Type == TokType::Identifier ) + Code constructor_destructor = parse_global_nspace_constructor_destructor( specifiers ); + // Possible constructor implemented at global file scope. + if ( constructor_destructor ) { - idx++; - tok = Context.Tokens[ idx ]; - if ( tok.Type == TokType::Access_StaticSymbol ) - continue; + member = constructor_destructor; + break; + } + + bool found_operator_cast_outside_class_implmentation = false; + s32 idx = _ctx->parser.Tokens.Idx; + + for ( ; idx < array_num(_ctx->parser.Tokens.Arr); idx++ ) + { + Token tok = _ctx->parser.Tokens.Arr[ idx ]; + + if ( tok.Type == Tok_Identifier ) + { + idx++; + tok = _ctx->parser.Tokens.Arr[ idx ]; + if ( tok.Type == Tok_Access_StaticSymbol ) + continue; + + break; + } + + if ( tok.Type == Tok_Decl_Operator ) + found_operator_cast_outside_class_implmentation = true; break; } - if ( tok.Type == TokType::Decl_Operator ) - found_operator_cast_outside_class_implmentation = true; - - break; - } - - if ( found_operator_cast_outside_class_implmentation ) - { - member = parse_operator_cast( specifiers ); - // ::operator () { ... } - break; + if ( found_operator_cast_outside_class_implmentation ) + { + member = cast(Code, parser_parse_operator_cast( specifiers )); + // ::operator () { ... } + break; + } } member = parse_operator_function_or_variable( expects_function, attributes, specifiers ); @@ -9026,22 +8846,22 @@ CodeBody parse_global_nspace( CodeT which ) } } - if ( member == Code::Invalid ) + if ( member == Code_Invalid ) { - log_failure( "Failed to parse member\n%s", Context.to_string() ); - Context.pop(); - return CodeInvalid; + log_failure( "Failed to parse member\nToken: %SB\nContext:\n%SB", tok_to_strbuilder(currtok_noskip), parser_to_strbuilder(_ctx->parser) ); + parser_pop(& _ctx->parser); + return InvalidCode; } // log_fmt("Global Body Member: %s", member->debug_str()); - result.append( member ); + body_append(result, member ); } - if ( which != Global_Body ) - eat( TokType::BraceCurly_Close ); + if ( which != CT_Global_Body ) + eat( Tok_BraceCurly_Close ); // { } - Context.pop(); + parser_pop(& _ctx->parser); return result; } @@ -9062,28 +8882,28 @@ Code parse_global_nspace_constructor_destructor( CodeSpecifiers specifiers ) TODO(Ed): We could fix this by attempting to parse a type, but we would have to have a way to have it soft fail and rollback. */ - TokArray tokens = Context.Tokens; + TokArray tokens = _ctx->parser.Tokens; s32 idx = tokens.Idx; - Token nav = tokens[ idx ]; - for ( ; idx < tokens.Arr.num(); idx++, nav = tokens[ idx ] ) + Token nav = tokens.Arr[ idx ]; + for ( ; idx < array_num(tokens.Arr); idx++, nav = tokens.Arr[ idx ] ) { - if ( nav.Text[0] == '<' ) + if ( nav.Text.Ptr[0] == '<' ) { // Skip templated expressions as they mey have expressions with the () operators s32 capture_level = 0; s32 template_level = 0; - for ( ; idx < tokens.Arr.num(); idx++, nav = tokens[idx] ) + for ( ; idx < array_num(tokens.Arr); idx++, nav = tokens.Arr[idx] ) { - if (nav.Text[ 0 ] == '<') + if (nav.Text.Ptr[ 0 ] == '<') ++ template_level; - if (nav.Text[ 0 ] == '>') + if (nav.Text.Ptr[ 0 ] == '>') -- template_level; - if (nav.Type == TokType::Operator && nav.Text[1] == '>') + if (nav.Type == Tok_Operator && nav.Text.Ptr[1] == '>') -- template_level; - if ( nav.Type == ETokType::Capture_Start) + if ( nav.Type == Tok_Capture_Start) { if (template_level != 0 ) ++ capture_level; @@ -9091,42 +8911,42 @@ Code parse_global_nspace_constructor_destructor( CodeSpecifiers specifiers ) break; } - if ( template_level != 0 && nav.Type == ETokType::Capture_End) + if ( template_level != 0 && nav.Type == Tok_Capture_End) -- capture_level; } } - if ( nav.Type == TokType::Capture_Start ) + if ( nav.Type == Tok_Capture_Start ) break; } -- idx; - Token tok_right = tokens[idx]; + Token tok_right = tokens.Arr[idx]; Token tok_left = NullToken; - if (tok_right.Type != TokType::Identifier) + if (tok_right.Type != Tok_Identifier) { // We're not dealing with a constructor if there is no identifier right before the opening of a parameter's scope. return result; } -- idx; - tok_left = tokens[idx]; + tok_left = tokens.Arr[idx]; // ... bool possible_destructor = false; - if ( tok_left.Type == TokType::Operator && tok_left.Text[0] == '~') + if ( tok_left.Type == Tok_Operator && tok_left.Text.Ptr[0] == '~') { possible_destructor = true; -- idx; - tok_left = tokens[idx]; + tok_left = tokens.Arr[idx]; } - if ( tok_left.Type != TokType::Access_StaticSymbol ) + if ( tok_left.Type != Tok_Access_StaticSymbol ) return result; -- idx; - tok_left = tokens[idx]; + tok_left = tokens.Arr[idx]; // ... :: // We search toward the left until we find the next valid identifier @@ -9134,39 +8954,39 @@ Code parse_global_nspace_constructor_destructor( CodeSpecifiers specifiers ) s32 template_level = 0; while ( idx != tokens.Idx ) { - if (tok_left.Text[ 0 ] == '<') + if (tok_left.Text.Ptr[ 0 ] == '<') ++ template_level; - if (tok_left.Text[ 0 ] == '>') + if (tok_left.Text.Ptr[ 0 ] == '>') -- template_level; - if (tok_left.Type == TokType::Operator && tok_left.Text[1] == '>') + if (tok_left.Type == Tok_Operator && tok_left.Text.Ptr[1] == '>') -- template_level; - if ( template_level != 0 && tok_left.Type == ETokType::Capture_Start) + if ( template_level != 0 && tok_left.Type == Tok_Capture_Start) ++ capture_level; - if ( template_level != 0 && tok_left.Type == ETokType::Capture_End) + if ( template_level != 0 && tok_left.Type == Tok_Capture_End) -- capture_level; - if ( capture_level == 0 && template_level == 0 && tok_left.Type == TokType::Identifier ) + if ( capture_level == 0 && template_level == 0 && tok_left.Type == Tok_Identifier ) break; -- idx; - tok_left = tokens[idx]; + tok_left = tokens.Arr[idx]; } - bool is_same = str_compare( tok_right.Text, tok_left.Text, tok_right.Length ) == 0; - if (tok_left.Type == TokType::Identifier && is_same) + bool is_same = c_str_compare_len( tok_right.Text.Ptr, tok_left.Text.Ptr, tok_right.Text.Len ) == 0; + if (tok_left.Type == Tok_Identifier && is_same) { // We have found the pattern we desired if (possible_destructor) { // :: ~ ( - result = parse_destructor( specifiers ); + result = cast(Code, parser_parse_destructor( specifiers )); } else { // :: ( - result = parse_constructor( specifiers ); + result = cast(Code, parser_parse_constructor( specifiers )); } } @@ -9175,77 +8995,78 @@ Code parse_global_nspace_constructor_destructor( CodeSpecifiers specifiers ) // TODO(Ed): I want to eventually change the identifier to its own AST type. // This would allow distinction of the qualifier for a symbol :: -// This would also allow +// This would also allow internal Token parse_identifier( bool* possible_member_function ) { push_scope(); Token name = currtok; - Context.Scope->Name = name; - eat( TokType::Identifier ); + _ctx->parser.Scope->Name = name.Text; + eat( Tok_Identifier ); // - parse_template_args( name ); + parse_template_args( & name ); //