gencpp/project/auxillary/scanner.hpp

160 lines
3.6 KiB
C++
Raw Permalink Normal View History

#ifdef GEN_INTELLISENSE_DIRECTIVES
# pragma once
# include "gen.hpp"
#endif
2023-08-21 18:40:23 -07:00
// This is a simple file reader that reads the entire file into memory.
// It has an extra option to skip the first few lines for undesired includes.
// This is done so that includes can be kept in dependency and component files so that intellisense works.
Code scan_file( char const* path )
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
{
FileInfo file;
FileError error = file_open_mode( & file, EFileMode_READ, path );
if ( error != EFileError_NONE )
{
GEN_FATAL( "scan_file: Could not open: %s", path );
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
}
sw fsize = file_size( & file );
if ( fsize <= 0 )
{
GEN_FATAL("scan_file: %s is empty", path );
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
}
String str = String::make_reserve( GlobalAllocator, fsize );
file_read( & file, str, fsize );
str.get_header().Length = fsize;
// Skip GEN_INTELLISENSE_DIRECTIVES preprocessor blocks
// Its designed so that the directive should be the first thing in the file.
// Anything that comes before it will also be omitted.
{
#define current (*scanner)
#define matched 0
#define move_fwd() do { ++ scanner; -- left; } while (0)
const StrC directive_start = txt( "ifdef" );
const StrC directive_end = txt( "endif" );
const StrC def_intellisense = txt("GEN_INTELLISENSE_DIRECTIVES" );
bool found_directive = false;
char const* scanner = str.Data;
s32 left = fsize;
while ( left )
{
// Processing directive.
if ( current == '#' )
{
move_fwd();
while ( left && char_is_space( current ) )
move_fwd();
if ( ! found_directive )
{
if ( left && str_compare( scanner, directive_start.Ptr, directive_start.Len ) == matched )
{
scanner += directive_start.Len;
left -= directive_start.Len;
while ( left && char_is_space( current ) )
move_fwd();
2023-08-21 19:48:05 -07:00
if ( left && str_compare( scanner, def_intellisense.Ptr, def_intellisense.Len ) == matched )
{
scanner += def_intellisense.Len;
left -= def_intellisense.Len;
found_directive = true;
}
}
// Skip to end of line
while ( left && current != '\r' && current != '\n' )
move_fwd();
move_fwd();
if ( left && current == '\n' )
move_fwd();
continue;
}
if ( left && str_compare( scanner, directive_end.Ptr, directive_end.Len ) == matched )
{
scanner += directive_end.Len;
left -= directive_end.Len;
// Skip to end of line
while ( left && current != '\r' && current != '\n' )
move_fwd();
move_fwd();
if ( left && current == '\n' )
move_fwd();
// sptr skip_size = fsize - left;
if ( (scanner + 2) >= ( str.Data + fsize ) )
{
mem_move( str, scanner, left );
str.get_header().Length = left;
break;
}
mem_move( str, scanner, left );
str.get_header().Length = left;
break;
}
}
move_fwd();
}
#undef move_fwd
#undef matched
#undef current
}
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
file_close( & file );
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
return untyped_str( str );
}
#if 0
struct CodeFile
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
{
using namespace Parser;
String FilePath;
TokArray Tokens;
Array<ParseFailure> ParseFailures;
Code CodeRoot;
};
namespace Parser
{
struct ParseFailure
{
String Reason;
Code Node;
};
}
CodeFile scan_file( char const* path )
{
using namespace Parser;
CodeFile
result = {};
result.FilePath = String::make( GlobalAllocator, path );
Code code = scan_file( path );
result.CodeRoot = code;
ParseContext context = parser_get_last_context();
result.Tokens = context.Tokens;
result.ParseFailures = context.Failures;
return result;
}
#endif