This commit is contained in:
Edward R. Gonzalez 2024-12-16 21:48:01 -05:00
parent 04ae75c698
commit 9c968967e2
28 changed files with 65 additions and 65 deletions

2
.gitignore vendored
View File

@ -27,7 +27,7 @@ release/**
**/Unreal/*.h
**/Unreal/*.cpp
! **/Unreal/validate.unreal.cpp
project/auxillary/vis_ast/dependencies/temp
project/auxiliary/vis_ast/dependencies/temp
test/gen/original
singleheader/gen/scratch.hpp
test/gen/scratch.cpp

View File

@ -24,16 +24,16 @@ If using the library's provided build scripts:
* `<push/pop>.<name>.inline.<hpp>`: macros that are meant to be injected at specific locations of the library file/s.
* `misc.hpp`: Misc functionality used by the library generation metaprograms.
* `undef.macros.h`: Undefines all macros from library.
* **auxillary**: Non-essential tooling:
* **auxiliary**: Non-essential tooling:
* `Builder`: Similar conceptually to Jai programming language's *builder*, just opens a file and prepares a string buffer to serialize code into (`builder_print`, `builder_print_fmt`). Then write & close the file when completed (`builder_write`).
* **`Scanner`**: Interface to load up `Code` from files two basic funcctions are currently provided.
* **`Scanner`**: Interface to load up `Code` from files two basic functions are currently provided.
* `scan_file`: Used mainly by the library format generators to directly scan files into untyped `Code` (raw string content, pre-formatted no AST parsed).
* `parse_file`: Used to read file and then parsed to populate a `CodeBody` AST.
* CSV parsing via one or two columns simplified.
* **gen_segemetned**: Dependencies go into gen.dep.{hpp/cpp} and components into gen.{hpp/cpp}
* **gen_singleheader**: Everything into a single file: gen.hpp
* **gen_unreal_engine**: Like gen_segemented but the library is modified slightly to compile as a thirdparty library within an Unreal Engine plugin or module.
* **gen_c_library**: The library is heavily modifed into C11 compliant code. A segemented and single-header set of variants are generated.
* **gen_unreal_engine**: Like gen_segmented but the library is modified slightly to compile as a thirdparty library within an Unreal Engine plugin or module.
* **gen_c_library**: The library is heavily modifed into C11 compliant code. A segmented and single-header set of variants are generated.
Code not making up the core library is located in `auxiliary/<auxiliary_name>.<hpp/cpp>`. These are optional extensions or tools for the library.
@ -108,7 +108,7 @@ Any large macros used implementing the gen interface or parser are going to be p
The vast majority of macros should be single-line subsitutions that either add:
* Improvements to searching
* Inteniality of keyword usage
* Intentionality of keyword usage
* A feature that only the preprocessor has (ex: function name reflection or stringifying)
* Compatibility of statements or expressions bewteen C & C++ that cannot be parsed by gencpp itself.
* Masking highly verbose syntax (the latter is getting phased out).
@ -123,7 +123,7 @@ The vast majority of macros should be single-line subsitutions that either add:
There are ***five*** header files which are automatically generated using [base_codegen.hpp](./helpers/base_codegen.hpp) by [base.cpp](./base.cpp). They are all located in [components/gen](./components/gen/).
* [ecodetypes.hpp](./components/gen/ecode.hpp): `CodeType` enum definition and related implementaiton. Generation is based off of [ECodeType.csv](./enums/ECodeTypes.csv).
* [ecodetypes.hpp](./components/gen/ecode.hpp): `CodeType` enum definition and related implementation. Generation is based off of [ECodeType.csv](./enums/ECodeTypes.csv).
* [especifier.hpp](./components/gen/especifier.hpp): `Specifier` enum definition, etc. Generated using [ESpecifier.csv](./enums/ESpecifier.csv).
* [eoperator.hpp](./components/gen/eoperator.hpp): `Operator` enum definition, etc. Generated using [EOperator.hpp](./enums/EOperator.csv).
* [etoktype.cpp](./components/gen/etoktype.cpp): `TokType` enum defininition, etc. Used by the lexer and parser backend. Uses two csvs:

View File

@ -420,7 +420,7 @@ struct AST
Code Next;
Code Back;
};
Token* Token; // Reference to starting token, only avaialble if it was derived from parsing.
Token* Token; // Reference to starting token, only available if it was derived from parsing.
Code Parent;
CodeType Type;
// CodeFlag CodeFlags;

View File

@ -402,7 +402,7 @@ bool null__check( char const* context, char const* code_id, Code code ) {
#define null_check( context, code ) null__check( #context, #code, cast(Code, code) )
/*
The implementaiton of the upfront constructors involves doing three things:
The implementation of the upfront constructors involves doing three things:
* Validate the arguments given to construct the intended type of AST is valid.
* Construct said AST type.
* Lock the AST (set to readonly) and return the valid object.

View File

@ -4629,7 +4629,7 @@ CodeTypename parser_parse_type( bool from_template, bool* typedef_is_function )
// <Attributes> <Specifiers> <class, enum, struct, union> <Name>
}
// Decltype draft implementaiton
// Decltype draft implementation
#if 0
else if ( currtok.Type == Tok_DeclType )
{

View File

@ -178,7 +178,7 @@ enum EMacroFlags : u16
// ^^^ This a kludge because unreal has a macro that behaves as both a 'statement' and an attribute (UE_DEPRECATED, PRAGMA_ENABLE_DEPRECATION_WARNINGS, etc)
// TODO(Ed): We can keep the MF_Allow_As_Attribute flag for macros, however, we need to add the ability of AST_Attributes to chain themselves.
// Its thats already a thing in the standard language anyway
// & it would allow UE_DEPRECATED, (UE_PROPERTY / UE_FUNCTION) to chain themselves as attributes of a resolved member function/varaible definition
// & it would allow UE_DEPRECATED, (UE_PROPERTY / UE_FUNCTION) to chain themselves as attributes of a resolved member function/variable definition
MF_Allow_As_Attribute = bit(3),
// When a macro is encountered after attributs and specifiers while parsing a function, or variable:

View File

@ -37,8 +37,8 @@ GEN_NS_BEGIN
#include "components/interface.parsing.cpp"
#include "components/interface.untyped.cpp"
#include "auxillary/builder.cpp"
#include "auxillary/scanner.cpp"
#include "auxiliary/builder.cpp"
#include "auxiliary/scanner.cpp"
GEN_NS_END

View File

@ -33,8 +33,8 @@ GEN_NS_BEGIN
#include "components/gen/ast_inlines.hpp"
#include "components/header_end.hpp"
#include "auxillary/builder.hpp"
#include "auxillary/scanner.hpp"
#include "auxiliary/builder.hpp"
#include "auxiliary/scanner.hpp"
GEN_NS_END

View File

@ -7,9 +7,9 @@
# include "gen.hpp"
# include "helpers/push_ignores.inline.hpp"
# include "helpers/helper.hpp"
# include "auxillary/builder.hpp"
# include "auxillary/builder.cpp"
# include "auxillary/scanner.hpp"
# include "auxiliary/builder.hpp"
# include "auxiliary/builder.cpp"
# include "auxiliary/scanner.hpp"
#include <stdlib.h>

View File

@ -15,7 +15,7 @@ In order to abstract away constant use of `AST*` its wrapped in a Code type whic
When its the [C generated variant of the library](../gen_c_library/)
```c
typedef AST* Code;
tyepdef AST_<name>* Code<name>;
typedef AST_<name>* Code<name>;
...
```

View File

@ -6,7 +6,7 @@
# AST Types Documentation
While the Readme for docs covers the data layout per AST, this will focus on the AST types avaialble, and their nuances.
While the Readme for docs covers the data layout per AST, this will focus on the AST types available, and their nuances.
## Body
@ -37,7 +37,7 @@ s32 NumEntries;
The `Front` member represents the start of the link list and `Back` the end.
NumEntries is the number of entries in the body.
Parent should have a compatible CodeType type for the type of defintion used.
Parent should have a compatible CodeType type for the type of definition used.
Serialization:
@ -95,7 +95,7 @@ Serialization:
<Content>
```
The parser will perserve comments found if residing with a body or in accepted inline-to-definition locations.
The parser will preserve comments found if residing with a body or in accepted inline-to-definition locations.
Otherwise they will be skipped by the TokArray::__eat and TokArray::current( skip foramtting enabled ) functions.
The upfront constructor: `def_comment` expects to recieve a comment without the `//` or `/* */` parts. It will add them during construction.
@ -289,7 +289,7 @@ Serialization:
## Execution
Just represents an execution body. Equivalent to an untyped body.
Will be obsolute when function body parsing is implemented.
Will be obsolete when function body parsing is implemented.
Fields:

View File

@ -4,7 +4,7 @@
<- [docs - General](Readme.md)
# Parser's Algorithim
# Parser's Algorithm
gencpp uses a hand-written recursive descent parser. Both the lexer and parser currently handle a full C/C++ file in a single pass.
@ -399,7 +399,7 @@ Below is an outline of the general alogirithim used for these internal procedure
## `parse_function_after_name`
This is needed as a function defintion is not easily resolvable early on, as such this function handles resolving a function
This is needed as a function definition is not easily resolvable early on, as such this function handles resolving a function
after its been made ceratin that the type of declaration or definition is indeed for a function signature.
By the point this function is called the following are known : export module flag, attributes, specifiers, return type, & name
@ -548,7 +548,7 @@ In the future statements and expressions will be parsed.
### Implementation Constraints
* Cannot definitively distinguish nested namespaces with identical names
* Cannot distinguish nested namespaces with identical names
* Return type detection requires parser enhancement
* Template parameter validation is syntax-based only
* Future enhancement: Implement type parsing with rollback capability
@ -595,7 +595,7 @@ Notes:
## `parse_operator_after_ret_type`
This is needed as a operator defintion is not easily resolvable early on, as such this function handles resolving a operator after its been made ceratin that the type of declaration or definition is indeed for a operator signature.
This is needed as a operator definition is not easily resolvable early on, as such this function handles resolving a operator after its been made ceratin that the type of declaration or definition is indeed for a operator signature.
By the point this function is called the following are known : export module flag, attributes, specifiers, and return type
@ -639,7 +639,7 @@ By the point this function is called the following are known : export module fla
## `parse_operator_function_or_variable`
When this function is called, attribute and specifiers may have been resolved, however what comes next can still be either an operator, function, or varaible.
When this function is called, attribute and specifiers may have been resolved, however what comes next can still be either an operator, function, or variable.
1. Initial Type Resolution
1. Push parsing scope
@ -792,7 +792,7 @@ This will get changed heavily once we have better support for typename expressio
## `parse_variable_after_name`
This is needed as a variable defintion is not easily resolvable early on, it takes a long evaluation period before its known that the declaration or definition is a variable. As such this function handles resolving a variable.
This is needed as a variable definition is not easily resolvable early on, it takes a long evaluation period before its known that the declaration or definition is a variable. As such this function handles resolving a variable.
By the point this function is called the following are known : export module flag, attributes, specifiers, value type, name
1. Initialization Processing
@ -1090,7 +1090,7 @@ Limitations:
This implementatin will be updated in the future to properly handle functional typename signatures.
### Current Algorithim
### Current Algorithm
Anything that is in the qualifier capture of the function typename is treated as an expression abstracted as an untyped string
@ -1119,7 +1119,7 @@ Anything that is in the qualifier capture of the function typename is treated as
8. Check for varaidic argument (param pack) token:
1. Consume varadic argument token
### WIP - Alternative Algorithim
### WIP - Alternative Algorithm
Currently wrapped up via macro: `GEN_USE_NEW_TYPENAME_PARSING`
Anything that is in the qualifier capture of the function typename is treated as an expression abstracted as an untyped string
@ -1152,7 +1152,7 @@ Anything that is in the qualifier capture of the function typename is treated as
7. Check for varaidic argument (param pack) token:
1. Consume varadic argument token
### **Later: Algorithim based on typename expressions**
### **Later: Algorithm based on typename expressions**
## `parse_typedef`

View File

@ -10,7 +10,7 @@ The library features a naive single-pass parser, tailored for only what the libr
This parser does not, and should not do the compiler's job. By only supporting this minimal set of features, the parser is kept (so far) around ~7000 loc. I hope to keep it under 10-15k loc worst case.
You can think of this parser as *frontend parser* vs a *semantic parser*. Its intuitively similar to WYSIWYG. What you ***precerive*** as the syntax from the user-side before the compiler gets a hold of it, is what you get.
You can think of this parser as *frontend parser* vs a *semantic parser*. Its intuitively similar to WYSIWYG. What you ***perceive*** as the syntax from the user-side before the compiler gets a hold of it, is what you get.
User exposed interface:
@ -63,7 +63,7 @@ The preprocessor lines are stored as members of their associated scope they are
Any preprocessor definition abuse that changes the syntax of the core language is unsupported and will fail to parse if not kept within an execution scope (function body, or expression assignment).
Exceptions:
* varaible definitions are allowed for a preprocessed macro `extern MACRO();`
* variable definitions are allowed for a preprocessed macro `extern MACRO();`
* function definitions are allowed for a preprocessed macro: `neverinline MACRO() { ... }`
* Disable with: `#define GEN_PARSER_DISABLE_MACRO_FUNCTION_SIGNATURES`
* typedefs allow for a preprocessed macro: `typedef MACRO();`

View File

@ -260,7 +260,7 @@ def_global_body( args( ht_entry, array_ht_entry, hashtable ));
def_global_body( 3, ht_entry, array_ht_entry, hashtable );
```
If a more incremental approach is desired for the body ASTs, `Code def_body( CodeT type )` can be used to create an empty body.
If a more incremental approach is desired for the body ASTs, `Code def_body( CodeT type )` can be used to create an empty bodyss
When the members have been populated use: `code_validate_body` to verify that the members are valid entires for that type.
### Parse construction
@ -432,7 +432,7 @@ and have the desired specifiers assigned to them beforehand.
## Code generation and modification
There are two provided auxillary interfaces:
There are two provided auxiliary interfaces:
* Builder
* Scanner
@ -444,7 +444,7 @@ There are two provided auxillary interfaces:
* The code is provided via print( code ) function will be serialized to its buffer.
* When all serialization is finished, use the write() command to write the buffer to the file.
### Scanner Auxillary Interface
### Scanner
* The purpose is to scan or parse files
* Some with two basic functions to convert a fil to code: `scan_file` and `parse_file`

View File

@ -32,7 +32,7 @@ All free from tag identifiers will be prefixed with `gen_` or `GEN_` as the name
* If they are parsed, its dude to requiring some changes to either naming, or adding additonal definitions (container generation, typedefs, etc).
2. All scanned or parsed code is refactored (identifiers substs) and/or formatted.
3. Singleheader generated.
4. Segemented headers and source generated.
4. Segmented headers and source generated.
## Templated container generation

View File

@ -995,7 +995,7 @@ R"(#define AST_ArrSpecs_Cap \
/*
This thing makes a:
#define code_<interface_name>( code, ... ) _Generic( (code), \
<slots> of defintions that look like: <typeof(code)>: code__<interface_name>, \
<slots> of definitions that look like: <typeof(code)>: code__<interface_name>, \
default: gen_generic_selection (Fail case) \
) GEN_RESOLVED_FUNCTION_CALL( code, ... ) \
*/
@ -1256,7 +1256,7 @@ R"(#define <interface_name>( code ) _Generic( (code), \
break;
}
CodeBody parsed_header_builder = parse_file( path_base "auxillary/builder.hpp" );
CodeBody parsed_header_builder = parse_file( path_base "auxiliary/builder.hpp" );
CodeBody header_builder = def_body(CT_Global_Body);
for ( Code entry = parsed_header_builder.begin(); entry != parsed_header_builder.end(); ++ entry ) switch( entry->Type )
{
@ -1559,7 +1559,7 @@ R"(#define <interface_name>( code ) _Generic( (code), \
}
#pragma endregion Resolve Components
// THERE SHOULD BE NO NEW GENERIC CONTAINER DEFINTIONS PAST THIS POINT (It will not have slots for the generic selection generated macros)
// THERE SHOULD BE NO NEW GENERIC CONTAINER DEFINITIONS PAST THIS POINT (It will not have slots for the generic selection generated macros)
CodeBody containers = def_body(CT_Global_Body);
{
containers.append( def_pragma(code(region Containers)));
@ -1611,7 +1611,7 @@ R"(#define <interface_name>( code ) _Generic( (code), \
Code rf_array_string_cached = refactor_and_format(array_string_cached);
Code rf_header_end = refactor_and_format(header_end);
Code rf_header_builder = refactor_and_format(header_builder);
Code rf_header_scanner = refactor_and_format( scan_file( path_base "auxillary/scanner.hpp" ));
Code rf_header_scanner = refactor_and_format( scan_file( path_base "auxiliary/scanner.hpp" ));
Code r_src_dep_start = refactor(src_dep_start);
Code r_src_debug = refactor(src_debug);
@ -1644,8 +1644,8 @@ R"(#define <interface_name>( code ) _Generic( (code), \
CodeBody etoktype = gen_etoktype( path_base "enums/ETokType.csv", path_base "enums/AttributeTokens.csv", helper_use_c_definition );
Code rf_etoktype = refactor_and_format(etoktype);
Code rf_src_builder = refactor_and_format( scan_file( path_base "auxillary/builder.cpp" ));
Code rf_src_scanner = refactor_and_format( scan_file( path_base "auxillary/scanner.cpp" ));
Code rf_src_builder = refactor_and_format( scan_file( path_base "auxiliary/builder.cpp" ));
Code rf_src_scanner = refactor_and_format( scan_file( path_base "auxiliary/scanner.cpp" ));
#pragma endregion Refactored / Formatted
#pragma region Singleheader

View File

@ -6,15 +6,15 @@
* [docs](../docs/Readme.md)
# Segemented Library Generation
# segmented Library Generation
Create a segemented library using `segemented.cpp`
Create a segmented library using `segmented.cpp`
The principal (user) files are `gen.hpp` and `gen.cpp`.
They contain includes for its various components: `components/<component_name>.<hpp/cpp>`
Dependencies are bundled into `gen.dep.<hpp/cpp>`. They are included in `gen.<hpp/cpp>` before component includes.
Just like the `gen.<hpp/cpp>` they include their components: `dependencies/<dependency_name>.<hpp/cpp>`. The auxillary content (builder & scanner) is given their own files.
Just like the `gen.<hpp/cpp>` they include their components: `dependencies/<dependency_name>.<hpp/cpp>`. The auxiliary content (builder & scanner) is given their own files.
If using the library's provided build scripts:

View File

@ -231,7 +231,7 @@ int gen_main()
// gen_builder.hpp
{
Code builder = scan_file( path_base "auxillary/builder.hpp" );
Code builder = scan_file( path_base "auxiliary/builder.hpp" );
Builder header = builder_open( "gen/gen.builder.hpp" );
builder_print_fmt( & header, generation_notice );
@ -245,7 +245,7 @@ int gen_main()
// gen_builder.cpp
{
Code builder = scan_file( path_base "auxillary/builder.cpp" );
Code builder = scan_file( path_base "auxiliary/builder.cpp" );
Builder src = builder_open( "gen/gen.builder.cpp" );
builder_print_fmt( & src, generation_notice );
@ -258,7 +258,7 @@ int gen_main()
// gen_scanner.hpp
{
Code scanner = scan_file( path_base "auxillary/scanner.hpp" );
Code scanner = scan_file( path_base "auxiliary/scanner.hpp" );
Builder header = builder_open( "gen/gen.scanner.hpp" );
builder_print_fmt( & header, generation_notice );
@ -272,7 +272,7 @@ int gen_main()
// gen_scanner.cpp
{
Code scanner = scan_file( path_base "auxillary/scanner.cpp" );
Code scanner = scan_file( path_base "auxiliary/scanner.cpp" );
Builder src = builder_open( "gen/gen.scanner.cpp" );
builder_print_fmt( & src, generation_notice );

View File

@ -159,10 +159,10 @@ int gen_main()
header.print( header_end );
if ( generate_builder ) {
header.print( scan_file( path_base "auxillary/builder.hpp" ) );
header.print( scan_file( path_base "auxiliary/builder.hpp" ) );
}
if ( generate_scanner ) {
header.print( scan_file( path_base "auxillary/scanner.hpp" ) );
header.print( scan_file( path_base "auxiliary/scanner.hpp" ) );
}
header.print(fmt_newline);
@ -246,11 +246,11 @@ int gen_main()
header.print_fmt( "\n#pragma endregion Interface\n");
if ( generate_builder ) {
header.print( scan_file( path_base "auxillary/builder.cpp" ) );
header.print( scan_file( path_base "auxiliary/builder.cpp" ) );
}
if ( generate_scanner ) {
header.print( scan_file( path_base "auxillary/scanner.cpp" ) );
header.print( scan_file( path_base "auxiliary/scanner.cpp" ) );
}
header.print( fmt_newline);

View File

@ -337,7 +337,7 @@ int gen_main()
// gen_builder.hpp
{
Code builder = scan_file( path_base "auxillary/builder.hpp" );
Code builder = scan_file( path_base "auxiliary/builder.hpp" );
Builder
header = Builder::open( "gen/gen.builder.hpp" );
@ -356,7 +356,7 @@ int gen_main()
// gen_builder.cpp
{
Code builder = scan_file( path_base "auxillary/builder.cpp" );
Code builder = scan_file( path_base "auxiliary/builder.cpp" );
Builder
src = Builder::open( "gen/gen.builder.cpp" );
@ -374,7 +374,7 @@ int gen_main()
// gen_scanner.hpp
{
Code scanner = scan_file( path_base "auxillary/scanner.hpp" );
Code scanner = scan_file( path_base "auxiliary/scanner.hpp" );
Builder
header = Builder::open( "gen/gen.scanner.hpp" );
@ -393,7 +393,7 @@ int gen_main()
// gen.scanner.cpp
{
Code scanner = scan_file( path_base "auxillary/scanner.cpp" );
Code scanner = scan_file( path_base "auxiliary/scanner.cpp" );
Builder
src = Builder::open( "gen/gen.scanner.cpp" );

View File

@ -22,7 +22,7 @@ args:
c_lib : Build c11 library (singleheader & segmented)
c_lib_static : Build static c11 library
c_lib_dyn : Buidl dyanmic c11
segemented
segmented
singleheader
unreal
clang

View File

@ -106,10 +106,10 @@ Copy-Item -Verbose -Path $path_base\gen.hpp -Destination $path_rel
Copy-Item -Verbose -Path $path_base\gen.cpp -Destination $path_release_content
Copy-Item -Verbose -Path $path_base\gen.dep.hpp -Destination $path_release_content
Copy-Item -Verbose -Path $path_base\gen.dep.cpp -Destination $path_release_content
Copy-Item -Verbose -Path $path_base\auxillary\builder.hpp -Destination $path_release_content\auxillary
Copy-Item -Verbose -Path $path_base\auxillary\builder.cpp -Destination $path_release_content\auxillary
Copy-Item -Verbose -Path $path_base\auxillary\scanner.hpp -Destination $path_release_content\auxillary
Copy-Item -Verbose -Path $path_base\auxillary\scanner.cpp -Destination $path_release_content\auxillary
Copy-Item -Verbose -Path $path_base\auxiliary\builder.hpp -Destination $path_release_content\auxiliary
Copy-Item -Verbose -Path $path_base\auxiliary\builder.cpp -Destination $path_release_content\auxiliary
Copy-Item -Verbose -Path $path_base\auxiliary\scanner.hpp -Destination $path_release_content\auxiliary
Copy-Item -Verbose -Path $path_base\auxiliary\scanner.cpp -Destination $path_release_content\auxiliary
New-Item -ItemType Directory -Force -Path "$path_release_content\components"
New-Item -ItemType Directory -Force -Path "$path_release_content\components\gen"

View File

@ -1,3 +1,3 @@
# Test
The implementaiton here has been gutted and will be rewritten...
The implementation here has been gutted and will be rewritten...