ed/gencpp
1
0
Fork 0
mirror of https://github.com/Ed94/gencpp.git synced 2024-11-10 02:54:53 -08:00
Code Issues Packages Projects Releases Wiki Activity

Progress on parser documentation

Browse Source
This commit is contained in:
Edward R. Gonzalez 2023-11-21 23:36:56 -05:00
parent f67f9547df
commit a667eb4afe
6 changed files with 273 additions and 41 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/AST_Types.md
View File

@ -75,6 +75,7 @@ Fields:
StringCached Content; StringCached Content;
Code Prev; Code Prev;
Code Next; Code Next;
parser::Token* Tok;
Code Parent; Code Parent;
StringCached Name; StringCached Name;
CodeT Type; CodeT Type;
@ -102,6 +103,7 @@ CodeType ParentType;
CodeBody Body; CodeBody Body;
CodeType Prev; // Used to store references to interfaces CodeType Prev; // Used to store references to interfaces
CodeType Next; // Used to store references to interfaces CodeType Next; // Used to store references to interfaces
parser::Token* Tok;
Code Parent; Code Parent;
StringCached Name; StringCached Name;
CodeT Type; CodeT Type;

137
docs/Parser_Algo.md
View File

@ -106,7 +106,7 @@ internal
} }
``` ```
Below is an outline of the general alogirithim used for these internal procedures. The intention is provide a basic briefing to aid the user in traversing the actual code definitions. These appear in the same order as they are in the `parser.cpp` file Below is an outline of the general alogirithim used for these internal procedures. The intention is to provide a basic briefing to aid the user in traversing the actual code definitions. These appear in the same order as they are in the `parser.cpp` file
## `parse_array_decl` ## `parse_array_decl`
@ -127,30 +127,155 @@ Below is an outline of the general alogirithim used for these internal procedure
## `parse_class_struct` ## `parse_class_struct`
1. Check for export module specifier
2. class or struct keyword
3. `parse_attributes`
4. If identifier : `parse_identifier`
5. Parse inherited parent or interfaces
6. If opening curly brace : `parse_class_struct_body`
7. If not an inplace definition
1. End statement
2. Check for inline comment
## `parse_class_struct_body` ## `parse_class_struct_body`
1. Opening curly brace
2. Parse the body (Possible options):
1. Newline : ast constant
2. Comment : `parse_comment`
3. Access_Public : ast constant
4. Access_Protected : ast constant
5. Access_Private : ast constant
6. Decl_Class : `parse_complicated_definition`
7. Decl_Enum : `parse_complicated_definition`
8. Decl_Friend : `parse_friend`
9. Decl_Operator : `parse_operator_cast`
10. Decl_Struct : `parse_complicated_definition`
11. Decl_Template : `parse_template`
12. Decl_Typedef : `parse_typedef`
13. Decl_Union : `parse_complicated_definition`
14. Decl_Using : `parse_using`
15. Operator == '~'
1. `parse_destructor`
16. Preprocess_Define : `parse_define`
17. Preprocess_Include : `parse_include`
18. Preprocess_Conditional (if, ifdef, ifndef, elif, else, endif) : `parse_preprocess_cond` or else/endif ast constant
19. Preprocess_Macro : `parse_simple_preprocess`
20. Preprocess_Pragma : `parse_pragma`
21. Preprocess_Unsupported : `parse_simple_preprocess`
22. StaticAssert : `parse_static_assert`
23. The following compound into a resolved definition or declaration:
1. Attributes (Standard, GNU, MSVC) : `parse_attributes`
2. Specifiers (consteval, constexpr, constinit, forceinline, inline, mutable, neverinline, static, volatile)
3. Possible Destructor : `parse_destructor`
4. Possible User defined operator cast : `parse_operator_cast`
5. Possible Constructor : `parse_constructor`
6. Something that has the following: (identifier, const, unsigned, signed, short, long, bool, char, int, double)
1. Possible Constructor `parse_constructor`
2. Possible Operator, Function, or varaible : `parse_operator_function_or_variable`
24. Something completely unknown (will just make untyped...) : `parse_untyped`
## `parse_comment` ## `parse_comment`
1. Just wrap the token into a cached string ( the lexer did the processing )
## `parse_compilcated_definition` ## `parse_compilcated_definition`
This is a helper function used by the following functions to help resolve a declaration or definition:
* `parse_class_struct_body`
* `parse_global_nspace`
* `parse_union`
A portion of the code in `parse_typedef` is very similar to this as both have to resolve a similar issue.
1. Look ahead to the termination token (End statement)
2. Check to see if it fits the pattern for a forward declare
3. If the previous token was an identifier ( `token[-1]` ):
1. Look back one more token : `[-2]`
2. If the token has a closing brace its an inplace definition
3. If the `token[-2]` is an identifier & `token[-3]` is the declaration type, its a variable using a namespaced type.
4. If the `token[-2]` is an indirection, then its a variable using a namespaced/forwarded type.
5. If any of the above is the case, `parse_operator_function_or_variable`
4. If the previous token was a closing curly brace, its a definition : `parse_forward_or_definition`
5. If the previous token was a closing square brace, its an array definition : `parse_operator_function_or_variable`
## `parse_define` ## `parse_define`
1. Define directive
2. Get identifier
3. Get Content
## `parse_forward_or_definition` ## `parse_forward_or_definition`
* Parse any of the following for either a forward declaration or definition:
1. Decl_Class : `parse_class`
2. Decl_Enum : `parse_enum`
3. Decl_Struct : `parse_struct`
4. Decl_Union : `parse_union`
## `parse_function_after_name` ## `parse_function_after_name`
This is needed as a function defintion is not easily resolvable early on, as such this function handles resolving a function
after its been made ceratin that the type of declaration or definition is indeed for a function signature.
By the point this function is called the following are known : export module flag, attributes, specifiers, return type, & name
1. `parse_parameters`
2. parse postfix specifiers (we do not check if the specifier here is correct or not to be here... yet)
3. If there is a body : `parse_body`
4. Otherwise :
1. Statment end
2. Check for inline comment
## `parse_function_body` ## `parse_function_body`
Currently there is no actual parsing of the function body. Any content with the braces is shoved into an execution AST node.
In the future statements and expressions will be parsed.
1. Open curly brace
2. Grab all tokens between the brace and the closing brace, shove them in a execution AST node.
3. Closing curly brace
## `parse_global_nspace` ## `parse_global_nspace`
1. Make sure the type provided to the helper function is a `Namespace_Body`, `Global_Body`, `Export_Body`, `Extern_Linkage_body`. 1. Make sure this is being called for a valid type (namespace, global body, export body, linkage body)
2. If its not a `Global_Body` eat the opening brace for the scope. 2. If its not a global body, consume the opening curly brace
3. ` 3. Parse the body (Possible options):
1. NewLine : ast constant
2. Comment : `parse_comment`
3. Decl_Cass : `parse_complicated_definition`
4. Decl_Enum : `parse_complicated_definition`
5. Decl_Extern_Linkage : `parse_extern_link`
6. Decl_Namespace : `parse_namespace`
7. Decl_Struct : `parse_complicated_definition`
8. Decl_Template : `parse_template`
9. Decl_Typedef : `parse_typedef`
10. Decl_Union : `parse_complicated_definition`
11. Decl_Using : `parse_using`
12. Preprocess_Define : `parse_define`
13. Preprocess_Include : `parse_include`
14. Preprocess_If, IfDef, IfNotDef, Elif : `parse_preprocess_cond`
15. Preprocess_Else : ast constant
16. Preprocess_Endif : ast constant
17. Preprocess_Macro : `parse_simple_preprocess`
18. Preprocess_Pragma : `parse_pragma`
19. Preprocess_Unsupported : `parse_simple_preprocess`
20. StaticAssert : `parse_static_assert`
21. Module_Export : `parse_export_body`
22. Module_Import : NOT_IMPLEMENTED
23. The following compound into a resolved definition or declaration:
1. Attributes ( Standard, GNU, MSVC, Macro ) : `parse_attributes`
2. Specifiers ( consteval, constexpr, constinit, extern, forceinline, global, inline, internal_linkage, neverinline, static )
3. Is either ( identifier, const specifier, long, short, signed, unsigned, bool, char, double, int)
1. If its an operator cast (definition outside class) : `parse_operator_cast`
2. Its an operator, function, or varaible : `parse_operator_function_or_varaible`
4. If its not a global body, consuem the closing curly brace
## `parse_identifier` ## `parse_identifier`
## `parse_include` ## `parse_include`
## `parse_operator_after_ret_type` ## `parse_operator_after_ret_type`
@ -224,7 +349,7 @@ Below is an outline of the general alogirithim used for these internal procedure
5. Decl_Struct 5. Decl_Struct
6. Decl_Union 6. Decl_Union
7. Preprocess_Define 7. Preprocess_Define
8. Preprocess_Conditional 8. Preprocess_Conditional (if, ifdef, ifndef, elif, else, endif)
9. Preprocess_Macro 9. Preprocess_Macro
10. Preprocess_Pragma 10. Preprocess_Pragma
11. Unsupported preprocess directive 11. Unsupported preprocess directive

10
project/components/ast.hpp
View File

@ -11,6 +11,7 @@ struct AST_Body;
struct AST_Attributes; struct AST_Attributes;
struct AST_Comment; struct AST_Comment;
struct AST_Constructor; struct AST_Constructor;
// struct AST_BaseClass;
struct AST_Class; struct AST_Class;
struct AST_Define; struct AST_Define;
struct AST_Destructor; struct AST_Destructor;
@ -76,6 +77,7 @@ struct Code;
struct CodeBody; struct CodeBody;
// These are to offer ease of use and optionally strong type safety for the AST. // These are to offer ease of use and optionally strong type safety for the AST.
struct CodeAttributes; struct CodeAttributes;
// struct CodeBaseClass;
struct CodeComment; struct CodeComment;
struct CodeClass; struct CodeClass;
struct CodeConstructor; struct CodeConstructor;
@ -203,12 +205,14 @@ struct Code
#ifdef GEN_ENFORCE_STRONG_CODE_TYPES #ifdef GEN_ENFORCE_STRONG_CODE_TYPES
# define operator explicit operator # define operator explicit operator
#endif #endif
operator CodeBody() const;
operator CodeAttributes() const; operator CodeAttributes() const;
// operator CodeBaseClass() const;
operator CodeComment() const; operator CodeComment() const;
operator CodeConstructor() const;
operator CodeDestructor() const;
operator CodeClass() const; operator CodeClass() const;
operator CodeConstructor() const;
operator CodeDefine() const; operator CodeDefine() const;
operator CodeDestructor() const;
operator CodeExec() const; operator CodeExec() const;
operator CodeEnum() const; operator CodeEnum() const;
operator CodeExtern() const; operator CodeExtern() const;
@ -230,7 +234,6 @@ struct Code
operator CodeUnion() const; operator CodeUnion() const;
operator CodeUsing() const; operator CodeUsing() const;
operator CodeVar() const; operator CodeVar() const;
operator CodeBody() const;
#undef operator #undef operator
}; };
@ -273,6 +276,7 @@ struct AST
operator Code(); operator Code();
operator CodeBody(); operator CodeBody();
operator CodeAttributes(); operator CodeAttributes();
// operator CodeBaseClass();
operator CodeComment(); operator CodeComment();
operator CodeConstructor(); operator CodeConstructor();
operator CodeDestructor(); operator CodeDestructor();

23
project/components/ast_types.hpp
View File

@ -15,7 +15,7 @@ struct AST_Body
char _PAD_[ sizeof(SpecifierT) * AST::ArrSpecs_Cap + sizeof(AST*) ]; char _PAD_[ sizeof(SpecifierT) * AST::ArrSpecs_Cap + sizeof(AST*) ];
Code Front; Code Front;
Code Back; Code Back;
parser::Token* Token; parser::Token* Tok;
Code Parent; Code Parent;
StringCached Name; StringCached Name;
CodeT Type; CodeT Type;
@ -32,7 +32,7 @@ struct AST_Attributes
}; };
Code Prev; Code Prev;
Code Next; Code Next;
parser::Token* Token; parser::Token* Tok;
Code Parent; Code Parent;
StringCached Name; StringCached Name;
CodeT Type; CodeT Type;
@ -40,6 +40,23 @@ struct AST_Attributes
}; };
static_assert( sizeof(AST_Attributes) == sizeof(AST), "ERROR: AST_Attributes is not the same size as AST"); static_assert( sizeof(AST_Attributes) == sizeof(AST), "ERROR: AST_Attributes is not the same size as AST");
#if 0
struct AST_BaseClass
{
union {
char _PAD_[ sizeof(SpecifierT) * AST::ArrSpecs_Cap + sizeof(AST*) ];
};
Code Prev;
Code Next;
parser::Token* Tok;
Code Parent;
StringCached Name;
CodeT Type;
char _PAD_UNUSED_[ sizeof(ModuleFlag) + sizeof(u32) ];
};
static_assert( sizeof(AST_BaseClass) == sizeof(AST), "ERROR: AST_BaseClass is not the same size as AST");
#endif
struct AST_Comment struct AST_Comment
{ {
union { union {
@ -48,7 +65,7 @@ struct AST_Comment
}; };
Code Prev; Code Prev;
Code Next; Code Next;
parser::Token* Token; parser::Token* Tok;
Code Parent; Code Parent;
StringCached Name; StringCached Name;
CodeT Type; CodeT Type;

1
project/components/code_types.hpp
View File

@ -240,6 +240,7 @@ struct CodeStruct
} }
Define_CodeType( Attributes ); Define_CodeType( Attributes );
// Define_CodeType( BaseClass );
Define_CodeType( Comment ); Define_CodeType( Comment );
struct CodeConstructor struct CodeConstructor

129
project/components/parser.cpp
View File

@ -668,45 +668,54 @@ Code parse_class_struct( TokType which, bool inplace_def = false )
mflags = ModuleFlag::Export; mflags = ModuleFlag::Export;
eat( TokType::Module_Export ); eat( TokType::Module_Export );
} }
// <ModuleFlags>
eat( which ); eat( which );
// <ModuleFlags> <class/struct>
attributes = parse_attributes(); attributes = parse_attributes();
// <ModuleFlags> <class/struct> <Attributes>
if ( check( TokType::Identifier ) ) if ( check( TokType::Identifier ) )
{ {
name = parse_identifier(); name = parse_identifier();
Context.Scope->Name = name; Context.Scope->Name = name;
} }
// <ModuleFlags> <class/struct> <Attributes> <Name>
local_persist local_persist
char interface_arr_mem[ kilobytes(4) ] {0}; char interface_arr_mem[ kilobytes(4) ] {0};
Array<CodeType> interfaces = Array<CodeType>::init_reserve( Arena::init_from_memory(interface_arr_mem, kilobytes(4) ), 4 ); Array<CodeType> interfaces = Array<CodeType>::init_reserve( Arena::init_from_memory(interface_arr_mem, kilobytes(4) ), 4 );
// TODO(Ed) : Make an AST_DerivedType, we'll store any arbitary derived type into there as a linear linked list of them.
if ( check( TokType::Assign_Classifer ) ) if ( check( TokType::Assign_Classifer ) )
{ {
eat( TokType::Assign_Classifer ); eat( TokType::Assign_Classifer );
// <ModuleFlags> <class/struct> <Attributes> <Name> :
if ( currtok.is_access_specifier() ) if ( currtok.is_access_specifier() )
{ {
access = currtok.to_access_specifier(); access = currtok.to_access_specifier();
// <ModuleFlags> <class/struct> <Attributes> <Name> : <Access Specifier>
} }
Token parent_tok = parse_identifier(); Token parent_tok = parse_identifier();
parent = def_type( parent_tok ); parent = def_type( parent_tok );
// <ModuleFlags> <class/struct> <Attributes> <Name> : <Access Specifier> <Parent/Interface Name>
while ( check(TokType::Comma) ) while ( check(TokType::Comma) )
{ {
eat(TokType::Access_Public); eat( TokType::Comma );
// <ModuleFlags> <class/struct> <Attributes> <Name> : <Access Specifier> <Name>,
if ( currtok.is_access_specifier() ) if ( currtok.is_access_specifier() )
{ {
eat(currtok.Type); eat(currtok.Type);
} }
Token interface_tok = parse_identifier(); Token interface_tok = parse_identifier();
interfaces.append( def_type( interface_tok ) ); interfaces.append( def_type( interface_tok ) );
// <ModuleFlags> <class/struct> <Attributes> <Name> : <Access Specifier> <Name>, ...
} }
} }
@ -714,15 +723,18 @@ Code parse_class_struct( TokType which, bool inplace_def = false )
{ {
body = parse_class_struct_body( which, name ); body = parse_class_struct_body( which, name );
} }
// <ModuleFlags> <class/struct> <Attributes> <Name> : <Access Specifier> <Name>, ... { <Body> }
CodeComment inline_cmt = NoCode; CodeComment inline_cmt = NoCode;
if ( ! inplace_def ) if ( ! inplace_def )
{ {
Token stmt_end = currtok; Token stmt_end = currtok;
eat( TokType::Statement_End ); eat( TokType::Statement_End );
// <ModuleFlags> <class/struct> <Attributes> <Name> : <Access Specifier> <Name>, ... { <Body> };
if ( currtok_noskip.Type == TokType::Comment && currtok_noskip.Line == stmt_end.Line ) if ( currtok_noskip.Type == TokType::Comment && currtok_noskip.Line == stmt_end.Line )
inline_cmt = parse_comment(); inline_cmt = parse_comment();
// <ModuleFlags> <class/struct> <Attributes> <Name> : <Access Specifier> <Name>, ... { <Body> }; <InlineCmt>
} }
if ( which == TokType::Decl_Class ) if ( which == TokType::Decl_Class )
@ -745,6 +757,7 @@ CodeBody parse_class_struct_body( TokType which, Token name )
push_scope(); push_scope();
eat( TokType::BraceCurly_Open ); eat( TokType::BraceCurly_Open );
// {
CodeBody CodeBody
result = (CodeBody) make_code(); result = (CodeBody) make_code();
@ -783,54 +796,66 @@ CodeBody parse_class_struct_body( TokType which, Token name )
member = access_public; member = access_public;
eat( TokType::Access_Public ); eat( TokType::Access_Public );
eat( TokType::Assign_Classifer ); eat( TokType::Assign_Classifer );
// public:
break; break;
case TokType::Access_Protected: case TokType::Access_Protected:
member = access_protected; member = access_protected;
eat( TokType::Access_Protected ); eat( TokType::Access_Protected );
eat( TokType::Assign_Classifer ); eat( TokType::Assign_Classifer );
// protected:
break; break;
case TokType::Access_Private: case TokType::Access_Private:
member = access_private; member = access_private;
eat( TokType::Access_Private ); eat( TokType::Access_Private );
eat( TokType::Assign_Classifer ); eat( TokType::Assign_Classifer );
// private:
break; break;
case TokType::Decl_Class: case TokType::Decl_Class:
member = parse_complicated_definition( TokType::Decl_Class ); member = parse_complicated_definition( TokType::Decl_Class );
// class
break; break;
case TokType::Decl_Enum: case TokType::Decl_Enum:
member = parse_complicated_definition( TokType::Decl_Enum ); member = parse_complicated_definition( TokType::Decl_Enum );
// enum
break; break;
case TokType::Decl_Friend: case TokType::Decl_Friend:
member = parse_friend(); member = parse_friend();
// friend
break; break;
case TokType::Decl_Operator: case TokType::Decl_Operator:
member = parse_operator_cast(); member = parse_operator_cast();
// operator <Type>()
break; break;
case TokType::Decl_Struct: case TokType::Decl_Struct:
member = parse_complicated_definition( TokType::Decl_Struct ); member = parse_complicated_definition( TokType::Decl_Struct );
// struct
break; break;
case TokType::Decl_Template: case TokType::Decl_Template:
member = parse_template(); member = parse_template();
// template< ... >
break; break;
case TokType::Decl_Typedef: case TokType::Decl_Typedef:
member = parse_typedef(); member = parse_typedef();
// typedef
break; break;
case TokType::Decl_Union: case TokType::Decl_Union:
member = parse_complicated_definition( TokType::Decl_Union ); member = parse_complicated_definition( TokType::Decl_Union );
// union
break; break;
case TokType::Decl_Using: case TokType::Decl_Using:
member = parse_using(); member = parse_using();
// using
break; break;
case TokType::Operator: case TokType::Operator:
@ -841,14 +866,17 @@ CodeBody parse_class_struct_body( TokType which, Token name )
} }
member = parse_destructor(); member = parse_destructor();
// ~<Name>()
break; break;
case TokType::Preprocess_Define: case TokType::Preprocess_Define:
member = parse_define(); member = parse_define();
// #define
break; break;
case TokType::Preprocess_Include: case TokType::Preprocess_Include:
member = parse_include(); member = parse_include();
// #include
break; break;
case TokType::Preprocess_If: case TokType::Preprocess_If:
@ -856,32 +884,39 @@ CodeBody parse_class_struct_body( TokType which, Token name )
case TokType::Preprocess_IfNotDef: case TokType::Preprocess_IfNotDef:
case TokType::Preprocess_ElIf: case TokType::Preprocess_ElIf:
member = parse_preprocess_cond(); member = parse_preprocess_cond();
break; // #<Condition>
case TokType::Preprocess_Macro:
member = parse_simple_preprocess( TokType::Preprocess_Macro );
break;
case TokType::Preprocess_Pragma:
member = parse_pragma();
break; break;
case TokType::Preprocess_Else: case TokType::Preprocess_Else:
member = preprocess_else; member = preprocess_else;
eat( TokType::Preprocess_Else ); eat( TokType::Preprocess_Else );
// #else
break; break;
case TokType::Preprocess_EndIf: case TokType::Preprocess_EndIf:
member = preprocess_endif; member = preprocess_endif;
eat( TokType::Preprocess_EndIf ); eat( TokType::Preprocess_EndIf );
// #endif
break;
case TokType::Preprocess_Macro:
member = parse_simple_preprocess( TokType::Preprocess_Macro );
// <Macro>
break;
case TokType::Preprocess_Pragma:
member = parse_pragma();
// #pragma
break; break;
case TokType::Preprocess_Unsupported: case TokType::Preprocess_Unsupported:
member = parse_simple_preprocess( TokType::Preprocess_Unsupported ); member = parse_simple_preprocess( TokType::Preprocess_Unsupported );
// #<UNKNOWN>
break; break;
case TokType::StaticAssert: case TokType::StaticAssert:
member = parse_static_assert(); member = parse_static_assert();
// static_assert
break; break;
case TokType::Attribute_Open: case TokType::Attribute_Open:
@ -892,6 +927,7 @@ CodeBody parse_class_struct_body( TokType which, Token name )
#undef Entry #undef Entry
{ {
attributes = parse_attributes(); attributes = parse_attributes();
// <Attributes>
} }
//! Fallthrough intended //! Fallthrough intended
case TokType::Spec_Consteval: case TokType::Spec_Consteval:
@ -942,16 +978,19 @@ CodeBody parse_class_struct_body( TokType which, Token name )
{ {
specifiers = def_specifiers( NumSpecifiers, specs_found ); specifiers = def_specifiers( NumSpecifiers, specs_found );
} }
// <Attributes> <Specifiers>
if ( currtok.Type == TokType::Operator && currtok.Text[0] == '~' ) if ( currtok.Type == TokType::Operator && currtok.Text[0] == '~' )
{ {
member = parse_destructor( specifiers ); member = parse_destructor( specifiers );
// <Attribute> <Specifiers> ~<Name>()
break; break;
} }
if ( currtok.Type == TokType::Decl_Operator ) if ( currtok.Type == TokType::Decl_Operator )
{ {
member = parse_operator_cast( specifiers ); member = parse_operator_cast( specifiers );
// <Attributes> <Specifiers> operator <Type>()
break; break;
} }
} }
@ -972,11 +1011,15 @@ CodeBody parse_class_struct_body( TokType which, Token name )
if ( str_compare( name.Text, currtok.Text, name.Length ) == 0 ) if ( str_compare( name.Text, currtok.Text, name.Length ) == 0 )
{ {
member = parse_constructor(); member = parse_constructor();
// <Attributes> <Specifiers> <Name>()
break; break;
} }
} }
member = parse_operator_function_or_variable( expects_function, attributes, specifiers ); member = parse_operator_function_or_variable( expects_function, attributes, specifiers );
// <Attributes> <Specifiers> operator <Op> ...
// or
// <Attributes> <Specifiers> <Name> ...
} }
break; break;
@ -990,6 +1033,7 @@ CodeBody parse_class_struct_body( TokType which, Token name )
} }
member = untyped_str( untyped_tok ); member = untyped_str( untyped_tok );
// Something unknown
break; break;
} }
@ -1004,6 +1048,7 @@ CodeBody parse_class_struct_body( TokType which, Token name )
} }
eat( TokType::BraceCurly_Close ); eat( TokType::BraceCurly_Close );
// { <Members> }
Context.pop(); Context.pop();
return result; return result;
} }
@ -1053,6 +1098,7 @@ Code parse_complicated_definition( TokType which )
{ {
// Its a forward declaration only // Its a forward declaration only
Code result = parse_forward_or_definition( which, is_inplace ); Code result = parse_forward_or_definition( which, is_inplace );
// <class, enum, struct, or union> <Name>;
Context.pop(); Context.pop();
return result; return result;
} }
@ -1074,9 +1120,9 @@ Code parse_complicated_definition( TokType which )
ok_to_parse = true; ok_to_parse = true;
is_inplace = true; is_inplace = true;
} }
else if ( tok.Type == TokType::Identifier && tokens[ idx - 3 ].Type == TokType::Decl_Struct ) else if ( tok.Type == TokType::Identifier && tokens[ idx - 3 ].Type == which )
{ {
// Its a variable with type ID using struct namespace. // Its a variable with type ID using <which> namespace.
// <which> <type_identifier> <identifier>; // <which> <type_identifier> <identifier>;
ok_to_parse = true; ok_to_parse = true;
} }
@ -1095,22 +1141,23 @@ Code parse_complicated_definition( TokType which )
} }
Code result = parse_operator_function_or_variable( false, { nullptr }, { nullptr } ); Code result = parse_operator_function_or_variable( false, { nullptr }, { nullptr } );
// <Attributes> <Specifiers> <ReturnType/ValueType> <operator <Op>, or Name> ...
Context.pop(); Context.pop();
return result; return result;
} }
else if ( tok.Type == TokType::BraceCurly_Close ) else if ( tok.Type == TokType::BraceCurly_Close )
{ {
// Its a definition // Its a definition
// <which> { ... };
Code result = parse_forward_or_definition( which, is_inplace ); Code result = parse_forward_or_definition( which, is_inplace );
// <which> { ... };
Context.pop(); Context.pop();
return result; return result;
} }
else if ( tok.Type == TokType::BraceSquare_Close) else if ( tok.Type == TokType::BraceSquare_Close)
{ {
// Its an array definition // Its an array definition
// <which> <type_identifier> <identifier> [ ... ];
Code result = parse_operator_function_or_variable( false, { nullptr }, { nullptr } ); Code result = parse_operator_function_or_variable( false, { nullptr }, { nullptr } );
// <which> <type_identifier> <identifier> [ ... ];
Context.pop(); Context.pop();
return result; return result;
} }
@ -1127,6 +1174,7 @@ CodeDefine parse_define()
{ {
push_scope(); push_scope();
eat( TokType::Preprocess_Define ); eat( TokType::Preprocess_Define );
// #define
CodeDefine CodeDefine
define = (CodeDefine) make_code(); define = (CodeDefine) make_code();
@ -1142,6 +1190,7 @@ CodeDefine parse_define()
Context.Scope->Name = currtok; Context.Scope->Name = currtok;
define->Name = get_cached_string( currtok ); define->Name = get_cached_string( currtok );
eat( TokType::Identifier ); eat( TokType::Identifier );
// #define <Name>
if ( ! check( TokType::Preprocess_Content )) if ( ! check( TokType::Preprocess_Content ))
{ {
@ -1154,6 +1203,7 @@ CodeDefine parse_define()
{ {
define->Content = get_cached_string( currtok ); define->Content = get_cached_string( currtok );
eat( TokType::Preprocess_Content ); eat( TokType::Preprocess_Content );
// #define <Name> <Content>
Context.pop(); Context.pop();
return define; return define;
@ -1161,6 +1211,7 @@ CodeDefine parse_define()
define->Content = get_cached_string( strip_formatting( currtok, strip_formatting_dont_preserve_newlines ) ); define->Content = get_cached_string( strip_formatting( currtok, strip_formatting_dont_preserve_newlines ) );
eat( TokType::Preprocess_Content ); eat( TokType::Preprocess_Content );
// #define <Name> <Content>
Context.pop(); Context.pop();
return define; return define;
@ -1212,8 +1263,9 @@ CodeFn parse_function_after_name(
{ {
push_scope(); push_scope();
CodeParam params = parse_params(); CodeParam params = parse_params();
// <Attributes> <Specifiers> <ReturnType> <Name> ( <Parameters> )
// These have to be kept separate from the return type's specifiers. // TODO(Ed), Review old comment : These have to be kept separate from the return type's specifiers.
while ( left && currtok.is_specifier() ) while ( left && currtok.is_specifier() )
{ {
if ( specifiers.ast == nullptr ) if ( specifiers.ast == nullptr )
@ -1226,6 +1278,7 @@ CodeFn parse_function_after_name(
specifiers.append( ESpecifier::to_type(currtok) ); specifiers.append( ESpecifier::to_type(currtok) );
eat( currtok.Type ); eat( currtok.Type );
} }
// <Attributes> <Specifiers> <ReturnType> <Name> ( <Paraemters> ) <Specifiers>
CodeBody body = NoCode; CodeBody body = NoCode;
CodeComment inline_cmt = NoCode; CodeComment inline_cmt = NoCode;
@ -1237,14 +1290,17 @@ CodeFn parse_function_after_name(
Context.pop(); Context.pop();
return CodeInvalid; return CodeInvalid;
} }
// <Attributes> <Specifiers> <ReturnType> <Name> ( <Paraemters> ) <Specifiers> { <Body> }
} }
else else
{ {
Token stmt_end = currtok; Token stmt_end = currtok;
eat( TokType::Statement_End ); eat( TokType::Statement_End );
// <Attributes> <Specifiers> <ReturnType> <Name> ( <Paraemters> ) <Specifiers>;
if ( currtok_noskip.Type == TokType::Comment && currtok_noskip.Line == stmt_end.Line ) if ( currtok_noskip.Type == TokType::Comment && currtok_noskip.Line == stmt_end.Line )
inline_cmt = parse_comment(); inline_cmt = parse_comment();
// <Attributes> <Specifiers> <ReturnType> <Name> ( <Paraemters> ) <Specifiers>; <InlineCmt>
} }
using namespace ECode; using namespace ECode;
@ -1349,6 +1405,7 @@ CodeBody parse_global_nspace( CodeT which )
if ( which != Global_Body ) if ( which != Global_Body )
eat( TokType::BraceCurly_Open ); eat( TokType::BraceCurly_Open );
// {
CodeBody CodeBody
result = (CodeBody) make_code(); result = (CodeBody) make_code();
@ -1381,10 +1438,12 @@ CodeBody parse_global_nspace( CodeT which )
case TokType::Decl_Class: case TokType::Decl_Class:
member = parse_complicated_definition( TokType::Decl_Class ); member = parse_complicated_definition( TokType::Decl_Class );
// class
break; break;
case TokType::Decl_Enum: case TokType::Decl_Enum:
member = parse_complicated_definition( TokType::Decl_Enum ); member = parse_complicated_definition( TokType::Decl_Enum );
// enum
break; break;
case TokType::Decl_Extern_Linkage: case TokType::Decl_Extern_Linkage:
@ -1392,38 +1451,47 @@ CodeBody parse_global_nspace( CodeT which )
log_failure( "Nested extern linkage\n%s", Context.to_string() ); log_failure( "Nested extern linkage\n%s", Context.to_string() );
member = parse_extern_link(); member = parse_extern_link();
// extern "..." { ... }
break; break;
case TokType::Decl_Namespace: case TokType::Decl_Namespace:
member = parse_namespace(); member = parse_namespace();
// namespace <Name> { ... }
break; break;
case TokType::Decl_Struct: case TokType::Decl_Struct:
member = parse_complicated_definition( TokType::Decl_Struct ); member = parse_complicated_definition( TokType::Decl_Struct );
// struct ...
break; break;
case TokType::Decl_Template: case TokType::Decl_Template:
member = parse_template(); member = parse_template();
// template<...> ...
break; break;
case TokType::Decl_Typedef: case TokType::Decl_Typedef:
member = parse_typedef(); member = parse_typedef();
// typedef ...
break; break;
case TokType::Decl_Union: case TokType::Decl_Union:
member = parse_complicated_definition( TokType::Decl_Union ); member = parse_complicated_definition( TokType::Decl_Union );
// union ...
break; break;
case TokType::Decl_Using: case TokType::Decl_Using:
member = parse_using(); member = parse_using();
// using ...
break; break;
case TokType::Preprocess_Define: case TokType::Preprocess_Define:
member = parse_define(); member = parse_define();
// #define ...
break; break;
case TokType::Preprocess_Include: case TokType::Preprocess_Include:
member = parse_include(); member = parse_include();
// #include ...
break; break;
case TokType::Preprocess_If: case TokType::Preprocess_If:
@ -1431,32 +1499,39 @@ CodeBody parse_global_nspace( CodeT which )
case TokType::Preprocess_IfNotDef: case TokType::Preprocess_IfNotDef:
case TokType::Preprocess_ElIf: case TokType::Preprocess_ElIf:
member = parse_preprocess_cond(); member = parse_preprocess_cond();
break; // #<Conditional> ...
case TokType::Preprocess_Macro:
member = parse_simple_preprocess( TokType::Preprocess_Macro );
break;
case TokType::Preprocess_Pragma:
member = parse_pragma();
break; break;
case TokType::Preprocess_Else: case TokType::Preprocess_Else:
member = preprocess_else; member = preprocess_else;
eat( TokType::Preprocess_Else ); eat( TokType::Preprocess_Else );
// #else
break; break;
case TokType::Preprocess_EndIf: case TokType::Preprocess_EndIf:
member = preprocess_endif; member = preprocess_endif;
eat( TokType::Preprocess_EndIf ); eat( TokType::Preprocess_EndIf );
// #endif
break;
case TokType::Preprocess_Macro:
member = parse_simple_preprocess( TokType::Preprocess_Macro );
// <Macro>
break;
case TokType::Preprocess_Pragma:
member = parse_pragma();
// #pragma ...
break; break;
case TokType::Preprocess_Unsupported: case TokType::Preprocess_Unsupported:
member = parse_simple_preprocess( TokType::Preprocess_Unsupported ); member = parse_simple_preprocess( TokType::Preprocess_Unsupported );
// #<UNSUPPORTED> ...
break; break;
case TokType::StaticAssert: case TokType::StaticAssert:
member = parse_static_assert(); member = parse_static_assert();
// static_assert( <Conditional Expression>, ... );
break; break;
case TokType::Module_Export: case TokType::Module_Export:
@ -1464,13 +1539,16 @@ CodeBody parse_global_nspace( CodeT which )
log_failure( "Nested export declaration\n%s", Context.to_string() ); log_failure( "Nested export declaration\n%s", Context.to_string() );
member = parse_export_body(); member = parse_export_body();
// export { ... }
break; break;
case TokType::Module_Import: case TokType::Module_Import:
{ {
not_implemented( context ); not_implemented( context );
// import ...
} }
//! Fallthrough intentional //! Fallthrough intentional
case TokType::Attribute_Open:
case TokType::Decl_GNU_Attribute: case TokType::Decl_GNU_Attribute:
case TokType::Decl_MSVC_Attribute: case TokType::Decl_MSVC_Attribute:
#define Entry( attribute, str ) case TokType::attribute: #define Entry( attribute, str ) case TokType::attribute:
@ -1478,6 +1556,7 @@ CodeBody parse_global_nspace( CodeT which )
#undef Entry #undef Entry
{ {
attributes = parse_attributes(); attributes = parse_attributes();
// <Attributes>
} }
//! Fallthrough intentional //! Fallthrough intentional
case TokType::Spec_Consteval: case TokType::Spec_Consteval:
@ -1542,6 +1621,7 @@ CodeBody parse_global_nspace( CodeT which )
{ {
specifiers = def_specifiers( NumSpecifiers, specs_found ); specifiers = def_specifiers( NumSpecifiers, specs_found );
} }
// <Attributes> <Specifiers>
} }
//! Fallthrough intentional //! Fallthrough intentional
case TokType::Identifier: case TokType::Identifier:
@ -1581,10 +1661,12 @@ CodeBody parse_global_nspace( CodeT which )
if ( found_operator_cast ) if ( found_operator_cast )
{ {
member = parse_operator_cast(); member = parse_operator_cast();
// <Attributes> <Specifiers> <Name>::operator <Type>() { ... }
break; break;
} }
member = parse_operator_function_or_variable( expects_function, attributes, specifiers ); member = parse_operator_function_or_variable( expects_function, attributes, specifiers );
// <Attributes> <Specifiers> ...
} }
} }
@ -1600,6 +1682,7 @@ CodeBody parse_global_nspace( CodeT which )
if ( which != Global_Body ) if ( which != Global_Body )
eat( TokType::BraceCurly_Close ); eat( TokType::BraceCurly_Close );
// { <Body> }
return result; return result;
} }