Added varadic parameter support (upfront and parsing)

This commit is contained in:
Edward R. Gonzalez 2023-07-24 11:20:13 -04:00
parent 1d3050f157
commit d4c2cdf30e
3 changed files with 118 additions and 48 deletions

View File

@ -18,7 +18,6 @@ These build up a code AST to then serialize with a file builder.
* [On multithreading](#on-multi-threading) * [On multithreading](#on-multi-threading)
* [Extending the library](#extending-the-library) * [Extending the library](#extending-the-library)
* [TODO](#todo) * [TODO](#todo)
* [Thoughts](#thoughts)
## Notes ## Notes
@ -26,21 +25,29 @@ The project has reached an *alpha* state, all the current functionality works fo
The project has no external dependencies beyond: The project has no external dependencies beyond:
* `errno.h` (gen.cpp) * `errno.h` (gen.dep.cpp)
* `stat.h` (gen.cpp) * `stat.h` (gen.dep.cpp)
* `stdarg.h` (gen.hpp) * `stdarg.h` (gen.dep.hpp)
* `stddef.h` (gen.hpp * `stddef.h` (gen.dep.hpp
* `stdio.h` (gen.cpp) * `stdio.h` (gen.dep.cpp)
* `copyfile.h` (Mac, gen.cpp) * `copyfile.h` (Mac, gen.dep.cpp)
* `types.h` (Linux, gen.cpp) * `types.h` (Linux, gen.dep.cpp)
* `unistd.h` (Linux/Mac, gen.cpp) * `unistd.h` (Linux/Mac, gen.dep.cpp)
* `intrin.h` (Windows, gen.hpp) * `intrin.h` (Windows, gen.dep.hpp)
* `io.h` (Windows with gcc, gen.cpp) * `io.h` (Windows with gcc, gen.dep.cpp)
* `windows.h` (Windows, gen.cpp) * `windows.h` (Windows, gen.dep.cpp)
Dependencies for the project are wrapped within `GENCPP_ROLL_OWN_DEPENDENCIES` (Defining it will disable them). Dependencies for the project are wrapped within `GENCPP_ROLL_OWN_DEPENDENCIES` (Defining it will disable them).
The majority of the dependency's implementation was derived from the [c-zpl library](https://github.com/zpl-c/zpl). The majority of the dependency's implementation was derived from the [c-zpl library](https://github.com/zpl-c/zpl).
This library was written a subset of C++ where the following are avoided:
* RAII (Constructors/Destructors), lifetimes are managed using named static or regular functions.
* Language provide dynamic dispatch, RTTI
* Object-Oriented Inheritance
Member-functions are used as an ergonomic choice, along with a conserative use of operator overloads.
A `natvis` and `natstepfilter` are provided in the scripts directory. A `natvis` and `natstepfilter` are provided in the scripts directory.
***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.*** ***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.***
@ -160,11 +167,12 @@ This method is setup where all the metaprogram's code are the within the same fi
### *WHAT IS NOT PROVIDED* ### *WHAT IS NOT PROVIDED*
* Lambdas
* RTTI
* Exceptions * Exceptions
* Execution statement validation : Execution expressions are defined using the untyped API. * Execution statement validation : Execution expressions are defined using the untyped API.
* Parsing support for module specifiers and attributes. (Its a todo) * Lambdas (This naturally means its unsupported)
* RAII : This needs support for constructors/destructor parsing
* I haven't gotten around to yet, only useful (to me) for third-party scanning
* Multiple Inheritance
Keywords kept from "Modern C++": Keywords kept from "Modern C++":
@ -191,7 +199,7 @@ This means that the typename entry for the parameter AST would be either:
* A fundamental type, function, or pointer type. * A fundamental type, function, or pointer type.
Anything beyond this usage is not supported by parse_template for arguments (at least not intentionally). Anything beyond this usage is not supported by parse_template for arguments (at least not intentionally).
Use at your own mental peril... Use at your own mental peril.
*Concepts and Constraints are not supported, its usage is non-trivial substitution.* *Concepts and Constraints are not supported, its usage is non-trivial substitution.*
@ -536,17 +544,20 @@ The following are provided predefined by the library as they are commonly used:
* `spec_constexpr` * `spec_constexpr`
* `spec_constinit` * `spec_constinit`
* `spec_extern_linkage` (extern) * `spec_extern_linkage` (extern)
* `spec_final`
* `spec_global` (global macro) * `spec_global` (global macro)
* `spec_inline` * `spec_inline`
* `spec_internal_linkage` (internal macro) * `spec_internal_linkage` (internal macro)
* `spec_local_persist` (local_persist macro) * `spec_local_persist` (local_persist macro)
* `spec_mutable` * `spec_mutable`
* `spec_override`
* `spec_ptr` * `spec_ptr`
* `spec_ref` * `spec_ref`
* `spec_register` * `spec_register`
* `spec_rvalue` * `spec_rvalue`
* `spec_static_member` (static) * `spec_static_member` (static)
* `spec_thread_local` * `spec_thread_local`
* `spec_virtual`
* `spec_volatile` * `spec_volatile`
* `spec_type_signed` * `spec_type_signed`
* `spec_type_unsigned` * `spec_type_unsigned`
@ -663,16 +674,6 @@ Currently unsupported. The following changes would have to be made:
This library is relatively very small, and can be extended without much hassle. This library is relatively very small, and can be extended without much hassle.
The untyped codes and builder/editor/scanner can be technically be used to circumvent
any sort of constrictions the library has with: modern c++, templates, macros, etc.
Typical use case is for getting define constants an old C/C++ library with the scanner:
Code parse_defines() can emit a custom code AST with Macro_Constant type.
Another would be getting preprocessor or template metaprogramming Codes from Unreal Engine definitions, etc.
The rules for constructing the AST are largely bound the syntax rules for what can be composed with whichever version of C++ your targeting.
The convention you'll see used throughout the API of the library is as follows: The convention you'll see used throughout the API of the library is as follows:
1. Check name or parameters to make sure they are valid for the construction requested 1. Check name or parameters to make sure they are valid for the construction requested
@ -684,9 +685,12 @@ Names or Content fields are interned strings and thus showed be cached using `ge
`def_operator` is the most sophisticated constructor as it has multiple permutations of definitions that could be created that are not trivial to determine if valid. `def_operator` is the most sophisticated constructor as it has multiple permutations of definitions that could be created that are not trivial to determine if valid.
If extendeding parsing capability
# TODO # TODO
* Implement a context stack for the parsing, allows for accurate scope validation for the AST types. * Implement a context stack for the parsing, allows for accurate scope validation for the AST types. (Better errors)
* Right now the parsing errors require a debugger in most cases.
* Make a more robust test suite. * Make a more robust test suite.
* Generate a single-header library * Generate a single-header library
* Componetize the library, make a metaprogram using gencpp to bootstrap itself. * Componetize the library, make a metaprogram using gencpp to bootstrap itself.

View File

@ -37,6 +37,7 @@ global AllocatorInfo Allocator_TypeTable = heap();
#pragma endregion StaticData #pragma endregion StaticData
#pragma region Constants #pragma region Constants
global CodeType t_empty;
global CodeType t_auto; global CodeType t_auto;
global CodeType t_void; global CodeType t_void;
global CodeType t_int; global CodeType t_int;
@ -66,6 +67,11 @@ global CodeType t_f32;
global CodeType t_f64; global CodeType t_f64;
#endif #endif
global CodeParam param_varadic;
global CodeAttributes attrib_api_export;
global CodeAttributes attrib_api_import;
global Code access_public; global Code access_public;
global Code access_protected; global Code access_protected;
global Code access_private; global Code access_private;
@ -80,17 +86,20 @@ global CodeSpecifiers spec_consteval;
global CodeSpecifiers spec_constexpr; global CodeSpecifiers spec_constexpr;
global CodeSpecifiers spec_constinit; global CodeSpecifiers spec_constinit;
global CodeSpecifiers spec_extern_linkage; global CodeSpecifiers spec_extern_linkage;
global CodeSpecifiers spec_final;
global CodeSpecifiers spec_global; global CodeSpecifiers spec_global;
global CodeSpecifiers spec_inline; global CodeSpecifiers spec_inline;
global CodeSpecifiers spec_internal_linkage; global CodeSpecifiers spec_internal_linkage;
global CodeSpecifiers spec_local_persist; global CodeSpecifiers spec_local_persist;
global CodeSpecifiers spec_mutable; global CodeSpecifiers spec_mutable;
global CodeSpecifiers spec_override;
global CodeSpecifiers spec_ptr; global CodeSpecifiers spec_ptr;
global CodeSpecifiers spec_ref; global CodeSpecifiers spec_ref;
global CodeSpecifiers spec_register; global CodeSpecifiers spec_register;
global CodeSpecifiers spec_rvalue; global CodeSpecifiers spec_rvalue;
global CodeSpecifiers spec_static_member; global CodeSpecifiers spec_static_member;
global CodeSpecifiers spec_thread_local; global CodeSpecifiers spec_thread_local;
global CodeSpecifiers spec_virtual;
global CodeSpecifiers spec_volatile; global CodeSpecifiers spec_volatile;
#pragma endregion Constants #pragma endregion Constants
@ -1278,6 +1287,23 @@ internal void define_constants()
#endif #endif
# undef def_constant_code_type # undef def_constant_code_type
t_empty = (CodeType) make_code();
t_empty->Type = ECode::Typename;
t_empty->Name = get_cached_string( txt_StrC("") );
t_empty.set_global();
param_varadic = (CodeType) make_code();
param_varadic->Type = ECode::Parameters;
param_varadic->Name = get_cached_string( txt_StrC("...") );
param_varadic->ValueType = t_empty;
param_varadic.set_global();
attrib_api_export = def_attributes( code(GEN_API_Export_Code));
attrib_api_export.set_global();
attrib_api_import = def_attributes( code(GEN_API_Import_Code));
attrib_api_import.set_global();
access_private = make_code(); access_private = make_code();
access_private->Type = ECode::Access_Private; access_private->Type = ECode::Access_Private;
access_private->Name = get_cached_string( txt_StrC("private:") ); access_private->Name = get_cached_string( txt_StrC("private:") );
@ -1327,17 +1353,20 @@ internal void define_constants()
def_constant_spec( constexpr, ESpecifier::Constexpr ); def_constant_spec( constexpr, ESpecifier::Constexpr );
def_constant_spec( constinit, ESpecifier::Constinit ); def_constant_spec( constinit, ESpecifier::Constinit );
def_constant_spec( extern_linkage, ESpecifier::External_Linkage ); def_constant_spec( extern_linkage, ESpecifier::External_Linkage );
def_constant_spec( final, ESpecifier::Final );
def_constant_spec( global, ESpecifier::Global ); def_constant_spec( global, ESpecifier::Global );
def_constant_spec( inline, ESpecifier::Inline ); def_constant_spec( inline, ESpecifier::Inline );
def_constant_spec( internal_linkage, ESpecifier::Internal_Linkage ); def_constant_spec( internal_linkage, ESpecifier::Internal_Linkage );
def_constant_spec( local_persist, ESpecifier::Local_Persist ); def_constant_spec( local_persist, ESpecifier::Local_Persist );
def_constant_spec( mutable, ESpecifier::Mutable ); def_constant_spec( mutable, ESpecifier::Mutable );
def_constant_spec( override, ESpecifier::Override );
def_constant_spec( ptr, ESpecifier::Ptr ); def_constant_spec( ptr, ESpecifier::Ptr );
def_constant_spec( ref, ESpecifier::Ref ); def_constant_spec( ref, ESpecifier::Ref );
def_constant_spec( register, ESpecifier::Register ); def_constant_spec( register, ESpecifier::Register );
def_constant_spec( rvalue, ESpecifier::RValue ); def_constant_spec( rvalue, ESpecifier::RValue );
def_constant_spec( static_member, ESpecifier::Static ); def_constant_spec( static_member, ESpecifier::Static );
def_constant_spec( thread_local, ESpecifier::Thread_Local ); def_constant_spec( thread_local, ESpecifier::Thread_Local );
def_constant_spec( virtual, ESpecifier::Virtual );
def_constant_spec( volatile, ESpecifier::Volatile) def_constant_spec( volatile, ESpecifier::Volatile)
spec_local_persist = def_specifiers( 1, ESpecifier::Local_Persist ); spec_local_persist = def_specifiers( 1, ESpecifier::Local_Persist );
@ -3475,6 +3504,7 @@ namespace Parser
Entry( Type_char, "char" ) \ Entry( Type_char, "char" ) \
Entry( Type_int, "int" ) \ Entry( Type_int, "int" ) \
Entry( Type_double, "double" ) \ Entry( Type_double, "double" ) \
Entry( Varadic_Argument, "..." ) \
Entry( Attributes_Start, "__attrib_start__" ) Entry( Attributes_Start, "__attrib_start__" )
enum class TokType : u32 enum class TokType : u32
@ -3706,6 +3736,22 @@ namespace Parser
if (left) if (left)
move_forward(); move_forward();
if ( current == '.' )
{
move_forward();
if( current == '.' )
{
token.Length = 3;
token.Type = TokType::Varadic_Argument;
move_forward();
}
else
{
log_failure( "gen::lex: invalid varadic argument, expected '...' got '..%c'", current );
}
}
goto FoundToken; goto FoundToken;
case '&' : case '&' :
@ -4145,25 +4191,25 @@ struct ParseContext
char const* Fn; char const* Fn;
}; };
internal Code parse_function_body ( Parser::TokArray& toks, char const* context ); internal Code parse_function_body( Parser::TokArray& toks, char const* context );
internal Code parse_global_nspace ( Parser::TokArray& toks, char const* context ); internal Code parse_global_nspace( Parser::TokArray& toks, char const* context );
internal CodeClass parse_class ( Parser::TokArray& toks, char const* context ); internal CodeClass parse_class ( Parser::TokArray& toks, char const* context );
internal CodeEnum parse_enum ( Parser::TokArray& toks, char const* context ); internal CodeEnum parse_enum ( Parser::TokArray& toks, char const* context );
internal CodeBody parse_export_body ( Parser::TokArray& toks, char const* context ); internal CodeBody parse_export_body ( Parser::TokArray& toks, char const* context );
internal CodeBody parse_extern_link_body ( Parser::TokArray& toks, char const* context ); internal CodeBody parse_extern_link_body( Parser::TokArray& toks, char const* context );
internal CodeExtern parse_exten_link ( Parser::TokArray& toks, char const* context ); internal CodeExtern parse_exten_link ( Parser::TokArray& toks, char const* context );
internal CodeFriend parse_friend ( Parser::TokArray& toks, char const* context ); internal CodeFriend parse_friend ( Parser::TokArray& toks, char const* context );
internal CodeFn parse_function ( Parser::TokArray& toks, char const* context ); internal CodeFn parse_function ( Parser::TokArray& toks, char const* context );
internal CodeNamespace parse_namespace ( Parser::TokArray& toks, char const* context ); internal CodeNamespace parse_namespace ( Parser::TokArray& toks, char const* context );
internal CodeOpCast parse_operator_cast ( Parser::TokArray& toks, char const* context ); internal CodeOpCast parse_operator_cast ( Parser::TokArray& toks, char const* context );
internal CodeStruct parse_struct ( Parser::TokArray& toks, char const* context ); internal CodeStruct parse_struct ( Parser::TokArray& toks, char const* context );
internal CodeVar parse_variable ( Parser::TokArray& toks, char const* context ); internal CodeVar parse_variable ( Parser::TokArray& toks, char const* context );
internal CodeTemplate parse_template ( Parser::TokArray& toks, char const* context ); internal CodeTemplate parse_template ( Parser::TokArray& toks, char const* context );
internal CodeType parse_type ( Parser::TokArray& toks, char const* context ); internal CodeType parse_type ( Parser::TokArray& toks, char const* context );
internal CodeTypedef parse_typedef ( Parser::TokArray& toks, char const* context ); internal CodeTypedef parse_typedef ( Parser::TokArray& toks, char const* context );
internal CodeUnion parse_union ( Parser::TokArray& toks, char const* context ); internal CodeUnion parse_union ( Parser::TokArray& toks, char const* context );
internal CodeUsing parse_using ( Parser::TokArray& toks, char const* context ); internal CodeUsing parse_using ( Parser::TokArray& toks, char const* context );
internal inline internal inline
Code parse_array_decl( Parser::TokArray& toks, char const* context ) Code parse_array_decl( Parser::TokArray& toks, char const* context )
@ -4339,6 +4385,13 @@ CodeParam parse_params( Parser::TokArray& toks, char const* context, bool use_te
CodeType type = { nullptr }; CodeType type = { nullptr };
Code value = { nullptr }; Code value = { nullptr };
if ( check( TokType::Varadic_Argument) )
{
eat( TokType::Varadic_Argument );
return param_varadic;
}
type = parse_type( toks, context ); type = parse_type( toks, context );
if ( type == Code::Invalid ) if ( type == Code::Invalid )
return CodeInvalid; return CodeInvalid;
@ -4396,6 +4449,13 @@ CodeParam parse_params( Parser::TokArray& toks, char const* context, bool use_te
Code type = { nullptr }; Code type = { nullptr };
Code value = { nullptr }; Code value = { nullptr };
if ( check( TokType::Varadic_Argument) )
{
eat( TokType::Varadic_Argument );
result.append( param_varadic );
continue;
}
type = parse_type( toks, context ); type = parse_type( toks, context );
if ( type == Code::Invalid ) if ( type == Code::Invalid )
return CodeInvalid; return CodeInvalid;

View File

@ -1942,6 +1942,7 @@ StrC token_fmt_impl( sw num, ... )
constexpr s32 LexAllocator_Size = GEN_LEX_ALLOCATOR_SIZE; constexpr s32 LexAllocator_Size = GEN_LEX_ALLOCATOR_SIZE;
constexpr s32 Builder_StrBufferReserve = GEN_BUILDER_STR_BUFFER_RESERVE; constexpr s32 Builder_StrBufferReserve = GEN_BUILDER_STR_BUFFER_RESERVE;
extern CodeType t_empty; // Used with varaidc parameters. (Exposing just in case its useful for another circumstance)
extern CodeType t_auto; extern CodeType t_auto;
extern CodeType t_void; extern CodeType t_void;
extern CodeType t_int; extern CodeType t_int;
@ -1951,8 +1952,10 @@ StrC token_fmt_impl( sw num, ... )
extern CodeType t_class; extern CodeType t_class;
extern CodeType t_typename; extern CodeType t_typename;
extern Code attrib_api_export; extern CodeParam param_varadic;
extern Code attrib_api_import;
extern CodeAttributes attrib_api_export;
extern CodeAttributes attrib_api_import;
extern Code access_public; extern Code access_public;
extern Code access_protected; extern Code access_protected;
@ -1968,17 +1971,20 @@ StrC token_fmt_impl( sw num, ... )
extern CodeSpecifiers spec_constexpr; extern CodeSpecifiers spec_constexpr;
extern CodeSpecifiers spec_constinit; extern CodeSpecifiers spec_constinit;
extern CodeSpecifiers spec_extern_linkage; extern CodeSpecifiers spec_extern_linkage;
extern CodeSpecifiers spec_final;
extern CodeSpecifiers spec_global; extern CodeSpecifiers spec_global;
extern CodeSpecifiers spec_inline; extern CodeSpecifiers spec_inline;
extern CodeSpecifiers spec_internal_linkage; extern CodeSpecifiers spec_internal_linkage;
extern CodeSpecifiers spec_local_persist; extern CodeSpecifiers spec_local_persist;
extern CodeSpecifiers spec_mutable; extern CodeSpecifiers spec_mutable;
extern CodeSpecifiers spec_override;
extern CodeSpecifiers spec_ptr; extern CodeSpecifiers spec_ptr;
extern CodeSpecifiers spec_ref; extern CodeSpecifiers spec_ref;
extern CodeSpecifiers spec_register; extern CodeSpecifiers spec_register;
extern CodeSpecifiers spec_rvalue; extern CodeSpecifiers spec_rvalue;
extern CodeSpecifiers spec_static_member; extern CodeSpecifiers spec_static_member;
extern CodeSpecifiers spec_thread_local; extern CodeSpecifiers spec_thread_local;
extern CodeSpecifiers spec_virtual;
extern CodeSpecifiers spec_volatile; extern CodeSpecifiers spec_volatile;
#pragma endregion Constants #pragma endregion Constants