Added varadic parameter support (upfront and parsing)

Readme.md

@@ -18,7 +18,6 @@ These build up a code AST to then serialize with a file builder.
 * [On multithreading](#on-multi-threading)
 * [Extending the library](#extending-the-library)
 * [TODO](#todo)
-* [Thoughts](#thoughts)
 
 ## Notes
 
@@ -26,21 +25,29 @@ The project has reached an *alpha* state, all the current functionality works fo
 
 The project has no external dependencies beyond:
 
-* `errno.h`    (gen.cpp)
+* `errno.h`    (gen.dep.cpp)
-* `stat.h`     (gen.cpp)
+* `stat.h`     (gen.dep.cpp)
-* `stdarg.h`   (gen.hpp)
+* `stdarg.h`   (gen.dep.hpp)
-* `stddef.h`   (gen.hpp
+* `stddef.h`   (gen.dep.hpp
-* `stdio.h`    (gen.cpp)
+* `stdio.h`    (gen.dep.cpp)
-* `copyfile.h` (Mac, gen.cpp)
+* `copyfile.h` (Mac, gen.dep.cpp)
-* `types.h`    (Linux, gen.cpp)
+* `types.h`    (Linux, gen.dep.cpp)
-* `unistd.h`   (Linux/Mac, gen.cpp)
+* `unistd.h`   (Linux/Mac, gen.dep.cpp)
-* `intrin.h`   (Windows, gen.hpp)
+* `intrin.h`   (Windows, gen.dep.hpp)
-* `io.h`       (Windows with gcc, gen.cpp)
+* `io.h`       (Windows with gcc, gen.dep.cpp)
-* `windows.h`  (Windows, gen.cpp)
+* `windows.h`  (Windows, gen.dep.cpp)
 
 Dependencies for the project are wrapped within `GENCPP_ROLL_OWN_DEPENDENCIES` (Defining it will disable them).  
 The majority of the dependency's implementation was derived from the [c-zpl library](https://github.com/zpl-c/zpl).
 
+This library was written a subset of C++ where the following are avoided:
+
+* RAII (Constructors/Destructors), lifetimes are managed using named static or regular functions.
+* Language provide dynamic dispatch, RTTI
+* Object-Oriented Inheritance
+
+Member-functions are used as an ergonomic choice, along with a conserative use of operator overloads.
+
 A `natvis` and `natstepfilter` are provided in the scripts directory.
 
 ***The editor and scanner have not been implemented yet. The scanner will come first, then the editor.***
@@ -160,11 +167,12 @@ This method is setup where all the metaprogram's code are the within the same fi
 
 ### *WHAT IS NOT PROVIDED*
 
-* Lambdas
-* RTTI
 * Exceptions
-* Execution statement validation              : Execution expressions are defined using the untyped API.
+* Execution statement validation : Execution expressions are defined using the untyped API.
-* Parsing support for module specifiers and attributes. (Its a todo)
+  * Lambdas (This naturally means its unsupported)
+* RAII : This needs support for constructors/destructor parsing
+  * I haven't gotten around to yet, only useful (to me) for third-party scanning
+* Multiple Inheritance
 
 Keywords kept from "Modern C++":
 
@@ -191,7 +199,7 @@ This means that the typename entry for the parameter AST would be either:
 * A fundamental type, function, or pointer type.
 
 Anything beyond this usage is not supported by parse_template for arguments (at least not intentionally).  
-Use at your own mental peril...
+Use at your own mental peril.
 
 *Concepts and Constraints are not supported, its usage is non-trivial substitution.*
 
@@ -536,17 +544,20 @@ The following are provided predefined by the library as they are commonly used:
 * `spec_constexpr`
 * `spec_constinit`
 * `spec_extern_linkage` (extern)
+* `spec_final`
 * `spec_global` (global macro)
 * `spec_inline`
 * `spec_internal_linkage` (internal macro)
 * `spec_local_persist` (local_persist macro)
 * `spec_mutable`
+* `spec_override`
 * `spec_ptr`
 * `spec_ref`
 * `spec_register`
 * `spec_rvalue`
 * `spec_static_member` (static)
 * `spec_thread_local`
+* `spec_virtual`
 * `spec_volatile`
 * `spec_type_signed`
 * `spec_type_unsigned`
@@ -663,16 +674,6 @@ Currently unsupported. The following changes would have to be made:
 
 This library is relatively very small, and can be extended without much hassle.
 
-The untyped codes and builder/editor/scanner can be technically be used to circumvent
-any sort of constrictions the library has with: modern c++, templates, macros, etc.
-
-Typical use case is for getting define constants an old C/C++ library with the scanner:  
-Code parse_defines() can emit a custom code AST with Macro_Constant type.
-
-Another would be getting preprocessor or template metaprogramming Codes from Unreal Engine definitions, etc.
-
-The rules for constructing the AST are largely bound the syntax rules for what can be composed with whichever version of C++ your targeting.
-
 The convention you'll see used throughout the API of the library is as follows:
 
 1. Check name or parameters to make sure they are valid for the construction requested
@@ -684,9 +685,12 @@ Names or Content fields are interned strings and thus showed be cached using `ge
 
 `def_operator` is the most sophisticated constructor as it has multiple permutations of definitions that could be created that are not trivial to determine if valid.
 
+If extendeding parsing capability
+
 # TODO
 
-* Implement a context stack for the parsing, allows for accurate scope validation for the AST types.
+* Implement a context stack for the parsing, allows for accurate scope validation for the AST types. (Better errors)
+  * Right now the parsing errors require a debugger in most cases.
 * Make a more robust test suite.
 * Generate a single-header library
   * Componetize the library, make a metaprogram using gencpp to bootstrap itself.

project/gen.cpp

@@ -37,6 +37,7 @@ global AllocatorInfo Allocator_TypeTable        = heap();
 #pragma endregion StaticData
 
 #pragma region Constants
+global CodeType t_empty;
 global CodeType t_auto;
 global CodeType t_void;
 global CodeType t_int;
@@ -66,6 +67,11 @@ global CodeType t_f32;
 global CodeType t_f64;
 #endif
 
+global CodeParam param_varadic;
+
+global CodeAttributes attrib_api_export;
+global CodeAttributes attrib_api_import;
+
 global Code access_public;
 global Code access_protected;
 global Code access_private;
@@ -80,17 +86,20 @@ global CodeSpecifiers spec_consteval;
 global CodeSpecifiers spec_constexpr;
 global CodeSpecifiers spec_constinit;
 global CodeSpecifiers spec_extern_linkage;
+global CodeSpecifiers spec_final;
 global CodeSpecifiers spec_global;
 global CodeSpecifiers spec_inline;
 global CodeSpecifiers spec_internal_linkage;
 global CodeSpecifiers spec_local_persist;
 global CodeSpecifiers spec_mutable;
+global CodeSpecifiers spec_override;
 global CodeSpecifiers spec_ptr;
 global CodeSpecifiers spec_ref;
 global CodeSpecifiers spec_register;
 global CodeSpecifiers spec_rvalue;
 global CodeSpecifiers spec_static_member;
 global CodeSpecifiers spec_thread_local;
+global CodeSpecifiers spec_virtual;
 global CodeSpecifiers spec_volatile;
 #pragma endregion Constants
 
@@ -1278,6 +1287,23 @@ internal void define_constants()
 #endif
 #	undef def_constant_code_type
 
+	t_empty = (CodeType) make_code();
+	t_empty->Type    = ECode::Typename;
+	t_empty->Name    = get_cached_string( txt_StrC("") );
+	t_empty.set_global();
+
+	param_varadic            = (CodeType) make_code();
+	param_varadic->Type      = ECode::Parameters;
+	param_varadic->Name      = get_cached_string( txt_StrC("...") );
+	param_varadic->ValueType = t_empty;
+	param_varadic.set_global();
+
+	attrib_api_export = def_attributes( code(GEN_API_Export_Code));
+	attrib_api_export.set_global();
+
+	attrib_api_import = def_attributes( code(GEN_API_Import_Code));
+	attrib_api_import.set_global();
+
 	access_private       = make_code();
 	access_private->Type = ECode::Access_Private;
 	access_private->Name = get_cached_string( txt_StrC("private:") );
@@ -1327,17 +1353,20 @@ internal void define_constants()
 	def_constant_spec( constexpr,        ESpecifier::Constexpr );
 	def_constant_spec( constinit,        ESpecifier::Constinit );
 	def_constant_spec( extern_linkage,   ESpecifier::External_Linkage );
+	def_constant_spec( final, 		     ESpecifier::Final );
 	def_constant_spec( global,           ESpecifier::Global );
 	def_constant_spec( inline,           ESpecifier::Inline );
 	def_constant_spec( internal_linkage, ESpecifier::Internal_Linkage );
 	def_constant_spec( local_persist,    ESpecifier::Local_Persist );
 	def_constant_spec( mutable,          ESpecifier::Mutable );
+	def_constant_spec( override,         ESpecifier::Override );
 	def_constant_spec( ptr,              ESpecifier::Ptr );
 	def_constant_spec( ref,              ESpecifier::Ref );
 	def_constant_spec( register,         ESpecifier::Register );
 	def_constant_spec( rvalue,           ESpecifier::RValue );
 	def_constant_spec( static_member,    ESpecifier::Static );
 	def_constant_spec( thread_local,     ESpecifier::Thread_Local );
+	def_constant_spec( virtual, 		 ESpecifier::Virtual );
 	def_constant_spec( volatile, 	     ESpecifier::Volatile)
 
 	spec_local_persist = def_specifiers( 1, ESpecifier::Local_Persist );
@@ -3475,6 +3504,7 @@ namespace Parser
 	Entry( Type_char, 			   "char" )             \
 	Entry( Type_int, 			   "int" )              \
 	Entry( Type_double, 		   "double" )           \
+	Entry( Varadic_Argument,       "..." )              \
 	Entry( Attributes_Start,       "__attrib_start__" )
 
 	enum class TokType : u32
@@ -3706,6 +3736,22 @@ namespace Parser
 
 					if (left)
 						move_forward();
+
+					if ( current == '.' )
+					{
+						move_forward();
+						if( current == '.' )
+						{
+							token.Length = 3;
+							token.Type = TokType::Varadic_Argument;
+							move_forward();
+						}
+						else
+						{
+							log_failure( "gen::lex: invalid varadic argument, expected '...' got '..%c'", current );
+						}
+					}
+
 					goto FoundToken;
 
 				case '&' :
@@ -4145,25 +4191,25 @@ struct ParseContext
 	char const*   Fn;
 };
 
-internal Code parse_function_body    ( Parser::TokArray& toks, char const* context );
+internal Code parse_function_body( Parser::TokArray& toks, char const* context );
-internal Code parse_global_nspace    ( Parser::TokArray& toks, char const* context );
+internal Code parse_global_nspace( Parser::TokArray& toks, char const* context );
 
-internal CodeClass     parse_class            ( Parser::TokArray& toks, char const* context );
+internal CodeClass     parse_class           ( Parser::TokArray& toks, char const* context );
-internal CodeEnum      parse_enum             ( Parser::TokArray& toks, char const* context );
+internal CodeEnum      parse_enum            ( Parser::TokArray& toks, char const* context );
-internal CodeBody      parse_export_body      ( Parser::TokArray& toks, char const* context );
+internal CodeBody      parse_export_body     ( Parser::TokArray& toks, char const* context );
-internal CodeBody      parse_extern_link_body ( Parser::TokArray& toks, char const* context );
+internal CodeBody      parse_extern_link_body( Parser::TokArray& toks, char const* context );
-internal CodeExtern    parse_exten_link       ( Parser::TokArray& toks, char const* context );
+internal CodeExtern    parse_exten_link      ( Parser::TokArray& toks, char const* context );
-internal CodeFriend    parse_friend           ( Parser::TokArray& toks, char const* context );
+internal CodeFriend    parse_friend          ( Parser::TokArray& toks, char const* context );
-internal CodeFn        parse_function         ( Parser::TokArray& toks, char const* context );
+internal CodeFn        parse_function        ( Parser::TokArray& toks, char const* context );
-internal CodeNamespace parse_namespace        ( Parser::TokArray& toks, char const* context );
+internal CodeNamespace parse_namespace       ( Parser::TokArray& toks, char const* context );
-internal CodeOpCast    parse_operator_cast    ( Parser::TokArray& toks, char const* context );
+internal CodeOpCast    parse_operator_cast   ( Parser::TokArray& toks, char const* context );
-internal CodeStruct    parse_struct           ( Parser::TokArray& toks, char const* context );
+internal CodeStruct    parse_struct          ( Parser::TokArray& toks, char const* context );
-internal CodeVar       parse_variable         ( Parser::TokArray& toks, char const* context );
+internal CodeVar       parse_variable        ( Parser::TokArray& toks, char const* context );
-internal CodeTemplate  parse_template         ( Parser::TokArray& toks, char const* context );
+internal CodeTemplate  parse_template        ( Parser::TokArray& toks, char const* context );
-internal CodeType      parse_type             ( Parser::TokArray& toks, char const* context );
+internal CodeType      parse_type            ( Parser::TokArray& toks, char const* context );
-internal CodeTypedef   parse_typedef          ( Parser::TokArray& toks, char const* context );
+internal CodeTypedef   parse_typedef         ( Parser::TokArray& toks, char const* context );
-internal CodeUnion     parse_union            ( Parser::TokArray& toks, char const* context );
+internal CodeUnion     parse_union           ( Parser::TokArray& toks, char const* context );
-internal CodeUsing     parse_using            ( Parser::TokArray& toks, char const* context );
+internal CodeUsing     parse_using           ( Parser::TokArray& toks, char const* context );
 
 internal inline
 Code parse_array_decl( Parser::TokArray& toks, char const* context )
@@ -4339,6 +4385,13 @@ CodeParam parse_params( Parser::TokArray& toks, char const* context, bool use_te
 	CodeType type  = { nullptr };
 	Code     value = { nullptr };
 
+	if ( check( TokType::Varadic_Argument) )
+	{
+		eat( TokType::Varadic_Argument );
+
+		return param_varadic;
+	}
+
 	type = parse_type( toks, context );
 	if ( type == Code::Invalid )
 		return CodeInvalid;
@@ -4396,6 +4449,13 @@ CodeParam parse_params( Parser::TokArray& toks, char const* context, bool use_te
 		Code type  = { nullptr };
 		Code value = { nullptr };
 
+		if ( check( TokType::Varadic_Argument) )
+		{
+			eat( TokType::Varadic_Argument );
+			result.append( param_varadic );
+			continue;
+		}
+
 		type = parse_type( toks, context );
 		if ( type == Code::Invalid )
 			return CodeInvalid;

project/gen.hpp

@@ -1942,6 +1942,7 @@ StrC token_fmt_impl( sw num, ... )
 	constexpr s32 LexAllocator_Size         = GEN_LEX_ALLOCATOR_SIZE;
 	constexpr s32 Builder_StrBufferReserve  = GEN_BUILDER_STR_BUFFER_RESERVE;
 
+	extern CodeType t_empty; // Used with varaidc parameters. (Exposing just in case its useful for another circumstance)
 	extern CodeType t_auto;
 	extern CodeType t_void;
 	extern CodeType t_int;
@@ -1951,8 +1952,10 @@ StrC token_fmt_impl( sw num, ... )
 	extern CodeType t_class;
 	extern CodeType t_typename;
 
-	extern Code attrib_api_export;
+	extern CodeParam param_varadic;
-	extern Code attrib_api_import;
+
+	extern CodeAttributes attrib_api_export;
+	extern CodeAttributes attrib_api_import;
 
 	extern Code access_public;
 	extern Code access_protected;
@@ -1968,17 +1971,20 @@ StrC token_fmt_impl( sw num, ... )
 	extern CodeSpecifiers spec_constexpr;
 	extern CodeSpecifiers spec_constinit;
 	extern CodeSpecifiers spec_extern_linkage;
+	extern CodeSpecifiers spec_final;
 	extern CodeSpecifiers spec_global;
 	extern CodeSpecifiers spec_inline;
 	extern CodeSpecifiers spec_internal_linkage;
 	extern CodeSpecifiers spec_local_persist;
 	extern CodeSpecifiers spec_mutable;
+	extern CodeSpecifiers spec_override;
 	extern CodeSpecifiers spec_ptr;
 	extern CodeSpecifiers spec_ref;
 	extern CodeSpecifiers spec_register;
 	extern CodeSpecifiers spec_rvalue;
 	extern CodeSpecifiers spec_static_member;
 	extern CodeSpecifiers spec_thread_local;
+	extern CodeSpecifiers spec_virtual;
 	extern CodeSpecifiers spec_volatile;
 #pragma endregion Constants