gencpp/project/auxillary/scanner.hpp

128 lines
2.3 KiB
C++
Raw Normal View History

2023-08-21 18:40:23 -07:00
#pragma once
#include "gen.hpp"
// This is a simple file reader that reads the entire file into memory.
// It has an extra option to skip the first few lines for undesired includes.
// This is done so that includes can be kept in dependency and component files so that intellisense works.
Code scan_file( char const* path, bool skip_header_includes = true )
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
{
FileInfo file;
FileError error = file_open_mode( & file, EFileMode_READ, path );
if ( error != EFileError_NONE )
{
GEN_FATAL( "scan_file: Could not open: %s", path );
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
}
sw fsize = file_size( & file );
if ( fsize <= 0 )
{
GEN_FATAL("scan_file: %s is empty", path );
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
}
String str = String::make_reserve( GlobalAllocator, fsize );
file_read( & file, str, fsize );
str.get_header().Length = fsize;
if ( skip_header_includes )
{
#define current (*scanner)
StrC toks[] {
txt( "pragma once" ),
txt( "include" )
};
char* scanner = str;
while ( current != '\r' && current != '\n' )
{
for ( StrC tok : toks )
{
if ( current == '#' )
{
++ scanner;
}
if ( strncmp( scanner, tok.Ptr, tok.Len ) == 0 )
{
scanner += tok.Len;
while ( current != '\r' && current != '\n' )
{
++ scanner;
}
// Skip the line
2023-08-21 18:40:23 -07:00
sptr skip_size = sptr( scanner - str.Data );
if ( current == '\r' )
{
skip_size += 2;
scanner += 2;
}
else
{
skip_size += 1;
scanner += 1;
}
sptr new_length = sptr( str.get_header().Length ) - skip_size;
// scanner -= skip_size;
mem_move( str, scanner, new_length );
str.get_header().Length = new_length;
scanner = str;
}
}
++ scanner;
}
#undef current
}
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
file_close( & file );
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
return untyped_str( str );
}
#if 0
Progress towards bootstrap/singleheader generation I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want. Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures. When it comes to the cpp files, I may leave those alone as the macros largely help ith readability. Replacing those macros is expensive and most likely not worth it. The macros under consideration with replacing using the library bootstrap are: * Define_Types * Define_Operators * Define_Specifiers * GEN_Define_Attribute_Tokens * Define_CodeType * Using_Code * Define_TokType * def_constant_spec ? * Helper Macros for def_**_body functions ? * AST unallowed types case macros? (The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...) The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing). This would allow the user to specify custom attribute macros as well with greater ease. I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell). The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out. (Most likely may want to for various purposes) The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components. This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 19:19:21 -07:00
struct Policy
{
// Nothing for now.
};
struct SymbolInfo
{
StringCached File;
char const* Marker;
Code Signature;
};
struct Scanner
{
struct RequestEntry
{
SymbolInfo Info;
};
struct Receipt
{
StringCached File;
Code Defintion;
bool Result;
};
AllocatorInfo MemAlloc;
static void set_allocator( AllocatorInfo allocator );
Array<FileInfo> Files;
String Buffer;
Array<RequestEntry> Requests;
void add_files( s32 num, char const** files );
void add( SymbolInfo signature, Policy policy );
bool process_requests( Array<Receipt> out_receipts );
};
#endif