Updated readmes

This commit is contained in:
Edward R. Gonzalez 2023-07-15 23:38:53 -04:00
parent 805e69bb40
commit 79c3459f08
6 changed files with 108 additions and 31 deletions

110
Readme.md
View File

@ -39,6 +39,8 @@ The majority of the dependency's implementation was derived from the [c-zpl libr
When gencpp is in a stable state, I will make a C variant with the same feature set. When gencpp is in a stable state, I will make a C variant with the same feature set.
A single-header version will also be generated for both. A single-header version will also be generated for both.
A `natvis` and `natstepfilter` are provided in the scripts directory.
***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.*** ***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.***
## Usage ## Usage
@ -205,20 +207,45 @@ Data layout of AST struct:
```cpp ```cpp
union { union {
AST* ArrStatic[AST::ArrS_Cap]; struct
Array< AST* > ArrDyn; {
StringCached Content; AST* Attributes; // Class, Enum, Function, Struct, Typedef, Union, Using, Variable
SpecifierT ArrSpecs[AST::ArrSpecs_Cap]; AST* Specs; // Function, Operator, Type symbol, Variable
union {
AST* ParentType; // Class, Struct
AST* ReturnType; // Function, Operator
AST* UnderlyingType; // Enum, Typedef
AST* ValueType; // Parameter, Variable
};
AST* Params; // Function, Operator, Template
union {
AST* ArrExpr; // Type Symbol
AST* Body; // Class, Enum, Function, Namespace, Struct, Union
AST* Declaration; // Friend, Template
AST* Value; // Parameter, Variable
};
};
StringCached Content; // Attributes, Comment, Execution, Include
SpecifierT ArrSpecs[AST::ArrSpecs_Cap]; // Specifiers
};
union {
AST* Prev;
AST* Front; // Used by CodeBody
AST* Last; // Used by CodeParam
};
union {
AST* Next;
AST* Back; // Used by CodeBody
}; };
AST* Parent; AST* Parent;
StringCached Name; StringCached Name;
CodeT Type; CodeT Type;
OperatorT Op;
ModuleFlag ModuleFlags; ModuleFlag ModuleFlags;
AccessSpec ParentAccess; union {
u32 StaticIndex; OperatorT Op;
bool DynamicEntries; AccessSpec ParentAccess;
u8 _Align_Pad[3]; s32 NumEntries;
};
``` ```
*`CodeT` is a typedef for `ECode::Type` which has an underlying type of `u32`* *`CodeT` is a typedef for `ECode::Type` which has an underlying type of `u32`*
@ -226,31 +253,29 @@ u8 _Align_Pad[3];
*`StringCahced` is a typedef for `String const`, to denote it is an interned string* *`StringCahced` is a typedef for `String const`, to denote it is an interned string*
*`String` is the dynamically allocated string type for the library* *`String` is the dynamically allocated string type for the library*
AST widths are setup to be AST_POD_Size. AST widths are setup to be AST_POD_Size.
The width dictates how much the static array can hold before it must give way to using an allocated array: The width dictates how much the static array can hold before it must give way to using an allocated array:
```cpp ```cpp
constexpr static constexpr static
uw ArrS_Cap = uw ArrSpecs_Cap =
( AST_POD_Size (
- sizeof(AST*) // Parent AST_POD_Size
- sizeof(StringCached) // Name - sizeof(AST*) * 3
- sizeof(CodeT) // Type - sizeof(StringCached)
- sizeof(OperatorT) // Op - sizeof(CodeT)
- sizeof(ModuleFlag) // ModuleFlags - sizeof(ModuleFlag)
- sizeof(AccessSpec) // ParentAccess - sizeof(s32)
- sizeof(u32) // StaticIndex )
- sizeof(bool) // DynamicEntries / sizeof(SpecifierT) -1; // -1 for 4 extra bytes (Odd num of AST*)
- sizeof(u8) * 3 ) // _Align_Pad
/ sizeof(AST*);
``` ```
*Ex: If the AST_POD_Size is 256 the capacity of the static array is 27.* *Ex: If the AST_POD_Size is 128 the capacity of the static array is 20.*
Data Notes: Data Notes:
* The allocator definitions used are exposed to the user incase they want to dictate memory usage * The allocator definitions used are exposed to the user incase they want to dictate memory usage
* You'll find the memory handling in `init`, `gen_string_allocator`, `get_cached_string`, `make_code`, and `make_code_entries`. * You'll find the memory handling in `init`, `gen_string_allocator`, `get_cached_string`, `make_code`.
* ASTs are wrapped for the user in a Code struct which is a warpper for a AST* type. * ASTs are wrapped for the user in a Code struct which is a warpper for a AST* type.
* Both AST and Code have member symbols but their data layout is enforced to be POD types. * Both AST and Code have member symbols but their data layout is enforced to be POD types.
* This library treats memory failures as fatal. * This library treats memory failures as fatal.
@ -258,15 +283,50 @@ Data Notes:
* `StringArenas`, `StringCache`, `Allocator_StringArena`, and `Allocator_StringTable` are the associated containers or allocators. * `StringArenas`, `StringCache`, `Allocator_StringArena`, and `Allocator_StringTable` are the associated containers or allocators.
* Strings used for seralization and file buffers are not contained by those used for cached strings. * Strings used for seralization and file buffers are not contained by those used for cached strings.
* They are currently using `Memory::GlobalAllocator`, which are tracked array of arenas that grows as needed (adds buckets when one runs out). * They are currently using `Memory::GlobalAllocator`, which are tracked array of arenas that grows as needed (adds buckets when one runs out).
* Memory within the buckets is not resused, so its inherently wasteful (most likely will give non-cached strings their own tailored alloator later) * Memory within the buckets is not resused, so its inherently wasteful (most likely will give non-cached strings their own tailored allocator later)
* Linked lists used children nodes on bodies, and parameters.
* Its intended to generate the AST in one go and serialize after. The contructors and serializer are designed to be a "one pass, front to back" setup.
Two generic templated containers are used throughout the library: Two generic templated containers are used throughout the library:
* `template< class Type> struct Array` * `template< class Type> struct Array`
* `template< class Type> struct HashTable` * `template< class Type> struct HashTable`
Both Code and AST definitions have a `template< class Type> Code/AST cast()`. Its just an alternative way to explictly cast to each other.
Otherwise the library is free of any templates. Otherwise the library is free of any templates.
The following CodeTypes are used which the user may optionally use strong typeing with if they enable: `GEN_ENFORCE_STRONG_CODE_TYPES`
* CodeBody : Has support for `for-range` iterating across Code objects.
* CodeAttributes
* CodeComment
* CodeClass
* CodeEnum
* CodeExec
* CodeExtern
* CodeInclude
* CodeFriend
* CodeFn
* CodeModule
* CodeNamespace
* CodeOperator
* CodeOpCast
* CodeParam : Has suppor for `for-range` iterating across parameters.
* CodeSpecifier : Has support for `for-range` iterating across specifiers.
* CodeStruct
* CodeTemplate
* CodeType
* CodeTypedef
* CodeUnion
* CodeUsing
* CodeUsingNamespace
* CodeVar
Each Code boy has an assoicated "filtered AST" with the naming convention: `AST_<CodeName>`
Unrelated fields of the AST for that node type are omitted and only necesary padding members are defined otherwise.
Retreiving a raw version of the ast can be done using the `raw()` function defined in each AST.
## There are three sets of interfaces for Code AST generation the library provides ## There are three sets of interfaces for Code AST generation the library provides
* Upfront * Upfront

View File

@ -6,8 +6,9 @@ All the library code is contained in two files: `gen.hpp` and `gen.cpp`
Feature Macros: Feature Macros:
* `GENCPP_ROLL_OWN_DEPENDENCIES` : Optional override so that user may define the dependencies themselves. * `GEN_ROLL_OWN_DEPENDENCIES` : Optional override so that user may define the dependencies themselves.
* `GEN_DEFINE_LIBRARY_CORE_CONSTANTS` : Optional typename codes as they are non-standard to C/C++ and not necessary to library usage * `GEN_DEFINE_LIBRARY_CORE_CONSTANTS` : Optional typename codes as they are non-standard to C/C++ and not necessary to library usage
* `GEN_ENFORCE_STRONG_CODE_TYPES` : Enforces casts to filtered code types.
* `GEN_FEATURE_PARSING` : Defines the parse constructors * `GEN_FEATURE_PARSING` : Defines the parse constructors
* `GEN_FEATURE_EDITOR` : Defines the file editing features for changing definitions based on ASTs * `GEN_FEATURE_EDITOR` : Defines the file editing features for changing definitions based on ASTs
* `GEN_FEATURE_SCANNER` : Defines the file scanning features for generating ASTs * `GEN_FEATURE_SCANNER` : Defines the file scanning features for generating ASTs
@ -37,8 +38,10 @@ AST with.
`StringCache` : Hash table for cached strings. (`StringCached` typedef used to denote strings managed by it) `StringCache` : Hash table for cached strings. (`StringCached` typedef used to denote strings managed by it)
`AST` : The node data strucuture for the code. `Code` : Wrapper for `AST` with functionality for handling it appropriately.
`Code` : Wrapper for `AST` with functionality for handling it appropriately. `AST` : The node data strucuture for the code.
`Code Types` : Codes with typed ASTs. Body, Param, and Specifier have unique implementation, the rest use `Define_CodeType`
`AST Types` : Filtered AST definitions.
#### Gen Interface #### Gen Interface

View File

@ -2798,13 +2798,13 @@ namespace gen
uw ArrSpecs_Cap = uw ArrSpecs_Cap =
( (
AST_POD_Size AST_POD_Size
- sizeof(AST*) * 4 - sizeof(AST*) * 3
- sizeof(StringCached) - sizeof(StringCached)
- sizeof(CodeT) - sizeof(CodeT)
- sizeof(ModuleFlag) - sizeof(ModuleFlag)
- sizeof(u32) - sizeof(s32)
) )
/ sizeof(SpecifierT); / sizeof(SpecifierT) -1; // -1 for 4 extra bytes
union { union {
struct struct

12
scripts/Readme.md Normal file
View File

@ -0,0 +1,12 @@
# Scripts
Build and cleanup scripts for the test deirectory are found here along with `natvis` and `natstepfilter` files for debugging.
The build works as follows:
* Compile and run the meta-program, it will dump files to the `test/gen` directory.
* Format the files using clang-format
* Build a program that uses some the generated definitions. (Have not done yet)
The `test/gen` directory has the meson.build config for the meta-program
The `test` directory has the one for the depdendent-program.

View File

@ -253,6 +253,7 @@
<Type Name="gen::AST_Specifiers"> <Type Name="gen::AST_Specifiers">
<DisplayString>{Name} Type: {Type}</DisplayString> <DisplayString>{Name} Type: {Type}</DisplayString>
<Expand> <Expand>
<Item Name="Specs">Specs</Item>
<Item Name="NumEntries">NumEntries</Item> <Item Name="NumEntries">NumEntries</Item>
<Item Name="Parent">Parent</Item> <Item Name="Parent">Parent</Item>
<Item Name="Prev">Prev</Item> <Item Name="Prev">Prev</Item>
@ -541,6 +542,7 @@
<DisplayString Condition="ast == nullptr">Null</DisplayString> <DisplayString Condition="ast == nullptr">Null</DisplayString>
<DisplayString Condition="ast != nullptr">{ast->Name} {ast->Type}</DisplayString> <DisplayString Condition="ast != nullptr">{ast->Name} {ast->Type}</DisplayString>
<Expand> <Expand>
<Item Name="Specs">ast->Specs</Item>
<Item Name="Parent">ast->Parent</Item> <Item Name="Parent">ast->Parent</Item>
<Item Name="Prev">ast->Prev</Item> <Item Name="Prev">ast->Prev</Item>
<Item Name="Next">ast->Next</Item> <Item Name="Next">ast->Next</Item>