ed/gencpp
1
0
Fork 0
mirror of https://github.com/Ed94/gencpp.git synced 2024-11-13 20:24:52 -08:00
Code Issues Packages Projects Releases Wiki Activity
528ef72a51
gencpp/project/components/etoktype.cpp

164 lines
6.2 KiB
C++
Raw Normal View History

ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
namespace Parser
{
/*
This is a simple lexer that focuses on tokenizing only tokens relevant to the library.
It will not be capable of lexing C++ code with unsupported features.
For the sake of scanning files, it can scan preprocessor directives
WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union.. This language truly is a mess.
2023-07-31 21:42:08 -07:00
__Attributes_Start is only used to indicate the start of the user_defined attribute list.
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
*/
Reorganization of files, refactors, doc updates (WIP) Removing the gen. namespace from the files for components, dependencies, and file_processors. They are only necessary if the include directory is transparent, and in my case those are not. Made a docs directory. I'm offloading information from the main readme to there along with additional informationn I end up elaborating on down the line. Enum tables were moved to their own directory (project/enums). Library will not compile for now. Major refactor occuring with parsing related components.
2023-07-29 02:52:06 -07:00
#ifndef GEN_DEFINE_ATTRIBUTE_TOKENS
# define GEN_DEFINE_ATTRIBUTE_TOKENS \
Started to implement context stack for parser.
2023-07-28 18:44:31 -07:00
Entry( API_Export, "GEN_API_Export_Code" ) \
Entry( API_Import, "GEN_API_Import_Code" )
#endif
Naive preprocessor support initial implementation (compiles and runs, not heavily tested)
2023-07-29 22:21:04 -07:00
# define Define_TokType \
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
Entry( Invalid, "INVALID" ) \
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Entry( Access_Private, "private" ) \
Entry( Access_Protected, "protected" ) \
Entry( Access_Public, "public" ) \
Entry( Access_MemberSymbol, "." ) \
Entry( Access_StaticSymbol, "::") \
Entry( Ampersand, "&" ) \
Entry( Ampersand_DBL, "&&" ) \
Entry( Assign_Classifer, ":" ) \
Entry( Attribute_Open, "[[" ) \
Entry( Attribute_Close, "]]" ) \
Entry( BraceCurly_Open, "{" ) \
Entry( BraceCurly_Close, "}" ) \
Entry( BraceSquare_Open, "[" ) \
Entry( BraceSquare_Close, "]" ) \
Entry( Capture_Start, "(" ) \
Entry( Capture_End, ")" ) \
WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union.. This language truly is a mess.
2023-07-31 21:42:08 -07:00
Entry( Comment, "__comment__" ) \
Entry( Char, "__character__" ) \
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Entry( Comma, "," ) \
Entry( Decl_Class, "class" ) \
Naive preprocessor support initial implementation (compiles and runs, not heavily tested)
2023-07-29 22:21:04 -07:00
Entry( Decl_GNU_Attribute, "__attribute__" ) \
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Entry( Decl_MSVC_Attribute, "__declspec" ) \
Entry( Decl_Enum, "enum" ) \
Entry( Decl_Extern_Linkage, "extern" ) \
Entry( Decl_Friend, "friend" ) \
Entry( Decl_Module, "module" ) \
Entry( Decl_Namespace, "namespace" ) \
Entry( Decl_Operator, "operator" ) \
Entry( Decl_Struct, "struct" ) \
Entry( Decl_Template, "template" ) \
Entry( Decl_Typedef, "typedef" ) \
Entry( Decl_Using, "using" ) \
Entry( Decl_Union, "union" ) \
WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union.. This language truly is a mess.
2023-07-31 21:42:08 -07:00
Entry( Identifier, "__identifier__" ) \
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Entry( Module_Import, "import" ) \
Entry( Module_Export, "export" ) \
WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union.. This language truly is a mess.
2023-07-31 21:42:08 -07:00
Entry( Number, "__number__" ) \
Entry( Operator, "__operator__" ) \
Progress toward preprocessor parsing, lexing works, parsing does not.
2023-07-30 15:55:57 -07:00
Entry( Preprocess_Define, "define") \
Entry( Preprocess_If, "if") \
Entry( Preprocess_IfDef, "ifdef") \
Entry( Preprocess_IfNotDef, "ifndef") \
Entry( Preprocess_ElIf, "elif") \
Entry( Preprocess_Else, "else") \
Entry( Preprocess_EndIf, "endif") \
Naive preprocessor support initial implementation (compiles and runs, not heavily tested)
2023-07-29 22:21:04 -07:00
Entry( Preprocess_Include, "include" ) \
Progress toward preprocessor parsing, lexing works, parsing does not.
2023-07-30 15:55:57 -07:00
Entry( Preprocess_Pragma, "pragma") \
WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union.. This language truly is a mess.
2023-07-31 21:42:08 -07:00
Entry( Preprocess_Content, "__macro_content__") \
Entry( Preprocess_Macro, "__macro__") \
Entry( Preprocess_Unsupported, "__unsupported__" ) \
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Entry( Spec_Alignas, "alignas" ) \
Entry( Spec_Const, "const" ) \
Entry( Spec_Consteval, "consteval" ) \
Entry( Spec_Constexpr, "constexpr" ) \
Entry( Spec_Constinit, "constinit" ) \
Entry( Spec_Explicit, "explicit" ) \
Entry( Spec_Extern, "extern" ) \
Entry( Spec_Final, "final" ) \
Entry( Spec_Global, "global" ) \
Entry( Spec_Inline, "inline" ) \
Entry( Spec_Internal_Linkage, "internal" ) \
Entry( Spec_LocalPersist, "local_persist" ) \
Entry( Spec_Mutable, "mutable" ) \
Entry( Spec_Override, "override" ) \
Entry( Spec_Static, "static" ) \
Entry( Spec_ThreadLocal, "thread_local" ) \
Entry( Spec_Volatile, "volatile") \
Entry( Star, "*" ) \
Entry( Statement_End, ";" ) \
WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union.. This language truly is a mess.
2023-07-31 21:42:08 -07:00
Entry( StaticAssert, "static_assert" ) \
Entry( String, "__string__" ) \
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Entry( Type_Unsigned, "unsigned" ) \
Entry( Type_Signed, "signed" ) \
Entry( Type_Short, "short" ) \
Entry( Type_Long, "long" ) \
Entry( Type_char, "char" ) \
Entry( Type_int, "int" ) \
Entry( Type_double, "double" ) \
WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union.. This language truly is a mess.
2023-07-31 21:42:08 -07:00
Entry( Type_MS_int8, "__int8" ) \
Entry( Type_MS_int16, "__int16" ) \
Entry( Type_MS_int32, "__int32" ) \
Entry( Type_MS_int64, "__int64" ) \
Entry( Type_MS_W64, "_W64" ) \
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Entry( Varadic_Argument, "..." ) \
WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union.. This language truly is a mess.
2023-07-31 21:42:08 -07:00
Entry( __Attributes_Start, "__attrib_start__" )
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
namespace ETokType
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
{
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
enum Type : u32
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
{
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
# define Entry( Name_, Str_ ) Name_,
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Define_TokType
Reorganization of files, refactors, doc updates (WIP) Removing the gen. namespace from the files for components, dependencies, and file_processors. They are only necessary if the include directory is transparent, and in my case those are not. Made a docs directory. I'm offloading information from the main readme to there along with additional informationn I end up elaborating on down the line. Enum tables were moved to their own directory (project/enums). Library will not compile for now. Major refactor occuring with parsing related components.
2023-07-29 02:52:06 -07:00
GEN_DEFINE_ATTRIBUTE_TOKENS
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
# undef Entry
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
NumTokens,
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
};
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
internal inline
Type to_type( StrC str_tok )
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
{
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
local_persist
StrC lookup[(u32)NumTokens] =
{
# define Entry( Name_, Str_ ) { sizeof(Str_), Str_ },
Define_TokType
Reorganization of files, refactors, doc updates (WIP) Removing the gen. namespace from the files for components, dependencies, and file_processors. They are only necessary if the include directory is transparent, and in my case those are not. Made a docs directory. I'm offloading information from the main readme to there along with additional informationn I end up elaborating on down the line. Enum tables were moved to their own directory (project/enums). Library will not compile for now. Major refactor occuring with parsing related components.
2023-07-29 02:52:06 -07:00
GEN_DEFINE_ATTRIBUTE_TOKENS
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
# undef Entry
};
for ( u32 index = 0; index < (u32)NumTokens; index++ )
{
s32 lookup_len = lookup[index].Len - 1;
char const* lookup_str = lookup[index].Ptr;
if ( lookup_len != str_tok.Len )
continue;
if ( str_compare( str_tok.Ptr, lookup_str, lookup_len ) == 0 )
return scast(Type, index);
}
return Invalid;
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
}
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
internal inline
char const* to_str( Type type )
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
{
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
local_persist
char const* lookup[(u32)NumTokens] =
{
# define Entry( Name_, Str_ ) Str_,
Define_TokType
Reorganization of files, refactors, doc updates (WIP) Removing the gen. namespace from the files for components, dependencies, and file_processors. They are only necessary if the include directory is transparent, and in my case those are not. Made a docs directory. I'm offloading information from the main readme to there along with additional informationn I end up elaborating on down the line. Enum tables were moved to their own directory (project/enums). Library will not compile for now. Major refactor occuring with parsing related components.
2023-07-29 02:52:06 -07:00
GEN_DEFINE_ATTRIBUTE_TOKENS
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
# undef Entry
};
return lookup[(u32)type];
}
# undef Define_TokType
};
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
using TokType = ETokType::Type;
ECode, ESpecifier, and ETokType are now all generated. There is a redundant pattern for generating all three (as expected). I'll use it to define a general way of doing this sort of behavior.
2023-07-26 23:51:36 -07:00
Fixes from latest refactor of toktype enum.
2023-07-27 14:12:58 -07:00
} // Parser
Reference in New Issue Copy Permalink