From 527fb3b13203d3ba2126a3bdc07d93073eef5d63 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Mon, 16 Dec 2024 10:36:55 -0500 Subject: [PATCH] Progress on docs, adjustments --- base/Readme.md | 66 +++++++++++++++++++++++----- base/base.cpp | 1 - base/components/gen/especifier.hpp | 13 +++++- base/helpers/base_codegen.hpp | 13 +++++- gen_c_library/c_library.cpp | 1 - gen_segmented/Readme.md | 4 +- gen_segmented/segmented.cpp | 1 - gen_singleheader/singleheader.cpp | 1 - gen_unreal_engine/Readme.md | 17 +++++++ gen_unreal_engine/enums/ETokType.csv | 4 +- gen_unreal_engine/unreal.cpp | 1 - 11 files changed, 101 insertions(+), 21 deletions(-) diff --git a/base/Readme.md b/base/Readme.md index fb50225..52619b6 100644 --- a/base/Readme.md +++ b/base/Readme.md @@ -14,7 +14,7 @@ If using the library's provided build scripts: .\build.ps1 base ``` -Standard formats: +## Content Overview * **base**: Files are in granular pieces separated into four directories: * **dependencies**: Originally from the c-zpl library and modified thereafter. @@ -123,15 +123,15 @@ The vast majority of macros should be single-line subsitutions that either add: There are ***five*** header files which are automatically generated using [base_codegen.hpp](./helpers/base_codegen.hpp) by [base.cpp](./base.cpp). They are all located in [components/gen](./components/gen/). -* [`ecodetypes.hpp`](./components/gen/ecode.hpp): `CodeType` enum definition and related implementaiton. Generation is based off of [`ECodeType.csv](./enums/ECodeTypes.csv). -* [`especifier.hpp`](./components/gen/especifier.hpp): `Specifier` enum definition, etc. Generated using [`ESpecifier.csv`](./enums/ESpecifier.csv). -* [`eoperator.hpp`](./components/gen/eoperator.hpp): `Operator` enum definition, etc. Generated using [`EOperator.hpp`](./enums/EOperator.csv). -* [`etoktype.cpp`](./components/gen/etoktype.cpp): `TokType` enum defininition, etc. Used by the lexer and parser backend. Uses two csvs: - * [`ETokType.csv`](./enums/ETokType.csv): Provides the enum entries and their strinng ids. - * [`AttributeTokens.csv`](./enums/AttributeTokens.csv): Provides tokens entries that should be considered as attributes by the lexer and parser. Sspecfiically macro attributes such as those use for exporting symbols. -* [`ast_inlines.hpp`](./components/gen/ast_inlines.hpp): Member trivial `operator` definitions for C++ code types. Does not use a csv. +* [ecodetypes.hpp](./components/gen/ecode.hpp): `CodeType` enum definition and related implementaiton. Generation is based off of [ECodeType.csv](./enums/ECodeTypes.csv). +* [especifier.hpp](./components/gen/especifier.hpp): `Specifier` enum definition, etc. Generated using [ESpecifier.csv](./enums/ESpecifier.csv). +* [eoperator.hpp](./components/gen/eoperator.hpp): `Operator` enum definition, etc. Generated using [EOperator.hpp](./enums/EOperator.csv). +* [etoktype.cpp](./components/gen/etoktype.cpp): `TokType` enum defininition, etc. Used by the lexer and parser backend. Uses two csvs: + * [ETokType.csv](./enums/ETokType.csv): Provides the enum entries and their strinng ids. + * [AttributeTokens.csv](./enums/AttributeTokens.csv): Provides tokens entries that should be considered as attributes by the lexer and parser. Sspecfiically macro attributes such as those use for exporting symbols. +* [ast_inlines.hpp](./components/gen/ast_inlines.hpp): Member trivial `operator` definitions for C++ code types. Does not use a csv. -[`misc.hpp`](./helpers/misc.hpp): Has shared functions used by the library generation meta-programs throughout this codebase. +[misc.hpp](./helpers/misc.hpp): Has shared functions used by the library generation meta-programs throughout this codebase. If using the library's provided build scripts: @@ -160,9 +160,53 @@ Names or Content fields are interned strings and thus showed be cached using `ca `def_operator` is the most sophisticated upfront constructor as it has multiple permutations of definitions that could be created that are not trivial to determine if valid. -The parser is documented under [`docs/Parsing.md`](../docs/Parsing.md) and [`docs/Parser_Algo.md`](../docs/Parser_Algo.md). +The parser is documented under [`docs/Parsing.md`](../docs/Parsing.md) and [`docs/Parser_Algo.md`](../docs/Parser_Algo.md). Read that and the entire library if you want to extend it. + +### Attributes + +To add additional macro attributes, all that has to be done is modifying [`AttributeTokens.csv`](./enums/AttributeTokens.csv). + +### Specifiers + +To add additional macro specifiers, the following needs to be done: + +1. Adjust [especifier.hpp](./components/gen/especifier.hpp) +2. Adjust [etoktype.cpp](./components/gen/etoktype.cpp) +3. Adjust [parser_case_macros.cpp](./components/parser_case_macros.cpp) + +If the specifier is a new trailing specifier on function definitions: + +Head into [base_codegen.hpp](./helpers/base_codegen.hpp): `gen_especifier`. There will be an `is_trailing` function that needs to be adjusted with an additional case for the user's new trailing specifier. + +### Code Types + +These require the following to be handled to the equivalent extent as the other types: + +1. Adjust [ECodeTypes.csv](./enums/ECodeTypes.csv) with the new types +2. Define a new `AST_` and `Code`. See + * [ast.hpp](./components/ast.hpp): Initial forwards and user defined conversion for Code. + * [ast_types.hpp](./components/ast_types.hpp): Define the `AST_` struct. + * [code_types.hpp](./components/code_types.hpp): Defne the `CodeType` struct. If its needs an iterator see: `struct CodeBody` & `struct CodeParams`. +3. [ast_case_macros.cpp](./components/ast_case_macros.cpp): Review cases here if the new code type needs to be considered. +4. [ast.cpp](./components/ast.cpp): Need to review + * `code_debug_str` + * `code_is_equal` + * `code_to_strbuilder_ptr` + * `code_validate_body` +5. [code_serialization.cpp](./components/code_serialization.cpp): Define serialization here. +6. [inlines.hpp](./components/inlines.hpp): Any inline definitions for the `struct Code` are defined here. +7. [interface.cpp](./components/interface.hpp): Define the `Code` upfront and parsing interface. +8. [interface.upfront.cpp](./components/interface.upfront.cpp): Define the upfront constructor implementation. +9. [interface.parsing.cpp](./components/interface.parsing.cpp): Define the parsing interface implementation. +10. [lexer.cpp](./components/lexer.cpp): Adjust the lexer as needed. +11. [parser.cpp](./components/parser.cpp): Adjust the parser as needed. ## A note on compilation and runtime generation speed -The library is designed to be fast to compile and generate code at runtime as fast as resonable possible on a debug build. +The library is designed to be fast to compile and generate code at runtime as fast as possible on a debug build. Its recommended that your metaprogam be compiled using a single translation unit (unity build). + +## Whats with the expression / executions support #ifd and enums? + +The library is a *work in progress* and those are unfinished hypotheticals for adding the ability to manage or parse the AST of expresions or execution scope code. +They are entirely untested and not meant to be used yet, futher there is no parsing support or an upfront interface for what CodeTypes are defined so far. diff --git a/base/base.cpp b/base/base.cpp index 6e87efc..e62a7af 100644 --- a/base/base.cpp +++ b/base/base.cpp @@ -1,6 +1,5 @@ #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS #define GEN_ENFORCE_STRONG_CODE_TYPES -#define GEN_EXPOSE_BACKEND #define GEN_C_LIKE_CPP 1 #include "gen.cpp" #include "helpers/push_ignores.inline.hpp" diff --git a/base/components/gen/especifier.hpp b/base/components/gen/especifier.hpp index bfbcdd1..32e08b8 100644 --- a/base/components/gen/especifier.hpp +++ b/base/components/gen/especifier.hpp @@ -72,7 +72,18 @@ inline Str spec_to_str( Specifier type ) inline bool spec_is_trailing( Specifier specifier ) { - return specifier > Spec_Virtual; + switch (specifier) { + case Spec_Const: + case Spec_Final: + case Spec_NoExceptions: + case Spec_Override: + case Spec_Pure: + case Spec_Volatile: + return true; + + default: + return false; + } } inline Specifier str_to_specifier( Str str ) diff --git a/base/helpers/base_codegen.hpp b/base/helpers/base_codegen.hpp index 10d1765..a1d104c 100644 --- a/base/helpers/base_codegen.hpp +++ b/base/helpers/base_codegen.hpp @@ -214,7 +214,18 @@ CodeBody gen_especifier( char const* path, bool use_c_definition = false ) inline bool spec_is_trailing( Specifier specifier ) { - return specifier > Spec_Virtual; + switch (specifier) { + case Spec_Const: + case Spec_Final: + case Spec_NoExceptions: + case Spec_Override: + case Spec_Pure: + case Spec_Volatile: + return true; + + default: + return false; + } } ))); diff --git a/gen_c_library/c_library.cpp b/gen_c_library/c_library.cpp index 812a081..1e060c8 100644 --- a/gen_c_library/c_library.cpp +++ b/gen_c_library/c_library.cpp @@ -2,7 +2,6 @@ #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS #define GEN_ENFORCE_STRONG_CODE_TYPES -#define GEN_EXPOSE_BACKEND #include "gen.cpp" #include "helpers/push_ignores.inline.hpp" diff --git a/gen_segmented/Readme.md b/gen_segmented/Readme.md index e8a350b..bd62fe2 100644 --- a/gen_segmented/Readme.md +++ b/gen_segmented/Readme.md @@ -8,11 +8,13 @@ # Segemented Library Generation +Create a segemented library using `segemented.cpp` + The principal (user) files are `gen.hpp` and `gen.cpp`. They contain includes for its various components: `components/.` Dependencies are bundled into `gen.dep.`. They are included in `gen.` before component includes. -Just like the `gen.` they include their components: `dependencies/.` +Just like the `gen.` they include their components: `dependencies/.`. The auxillary content (builder & scanner) is given their own files. If using the library's provided build scripts: diff --git a/gen_segmented/segmented.cpp b/gen_segmented/segmented.cpp index a613767..0a361a9 100644 --- a/gen_segmented/segmented.cpp +++ b/gen_segmented/segmented.cpp @@ -2,7 +2,6 @@ #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS #define GEN_ENFORCE_STRONG_CODE_TYPES -#define GEN_EXPOSE_BACKEND #define GEN_C_LIKE_CPP 1 #include "gen.cpp" #include "helpers/push_ignores.inline.hpp" diff --git a/gen_singleheader/singleheader.cpp b/gen_singleheader/singleheader.cpp index f3b4327..462a35a 100644 --- a/gen_singleheader/singleheader.cpp +++ b/gen_singleheader/singleheader.cpp @@ -1,6 +1,5 @@ #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS #define GEN_ENFORCE_STRONG_CODE_TYPES -#define GEN_EXPOSE_BACKEND #include "gen.cpp" #include "helpers/push_ignores.inline.hpp" diff --git a/gen_unreal_engine/Readme.md b/gen_unreal_engine/Readme.md index 7be3fd6..342f4d0 100644 --- a/gen_unreal_engine/Readme.md +++ b/gen_unreal_engine/Readme.md @@ -15,3 +15,20 @@ If using the library's provided build scripts: ```ps1 .\build.ps1 unreal ``` + +## Notables + +For the most part this follows the same conventions as `gen_segmented`. + +This generator uses a separate enumeration definitions for the following: + +* [AttributeTokens.csv](./enums/AttributeTokens.csv) : Add your own _API attributes, etc here that are encountered within the Engine. +* [ESpecifier.csv](./enums/ESpecifier.csv) : Adds the `FORCEINLINE` & `FORCEINLINE_DEBUGGABLE` specfiers (additions are made as they are encountered) +* [ETokType.csv](./enums/ETokType.csv) : Same modifications as ESpecifier.csv. + +A separate [parser_case_macros.cpp](./components/parser_case_macros.cpp) is used to accomodate for the new forceinline specifiers. + +The `global` macro the library uses is redefined as an empty substiution. + + +The expected usage of this library is to put into into a third-party plugin module to then use either in editor modules or in shell script done in some stage of hot-reloading or building the Unreal Engine or Project. diff --git a/gen_unreal_engine/enums/ETokType.csv b/gen_unreal_engine/enums/ETokType.csv index cd8b917..6feec9d 100644 --- a/gen_unreal_engine/enums/ETokType.csv +++ b/gen_unreal_engine/enums/ETokType.csv @@ -13,8 +13,8 @@ BraceCurly_Open, "{" BraceCurly_Close, "}" BraceSquare_Open, "[" BraceSquare_Close, "]" -Capture_Start, "(" -Capture_End, ")" +Paren_Open, "(" +Paren_Close, ")" Comment, "__comment__" Comment_End, "__comment_end__" Comment_Start, "__comment_start__" diff --git a/gen_unreal_engine/unreal.cpp b/gen_unreal_engine/unreal.cpp index 839a7a9..56e494c 100644 --- a/gen_unreal_engine/unreal.cpp +++ b/gen_unreal_engine/unreal.cpp @@ -1,6 +1,5 @@ #define GEN_DEFINE_LIBRARY_CODE_CONSTANTS #define GEN_ENFORCE_STRONG_CODE_TYPES -#define GEN_EXPOSE_BACKEND #include "gen.cpp" #include "helpers/push_ignores.inline.hpp"