Compare commits

...
34 Commits
Author SHA1 Message Date
ed ca6f2eb83c fix 2026-10-16 13:36:33 -04:00
ed b5b7b9ab4b Added new string template formatter
- Based on a template formatter I have in used/evolved in other projects.
- Will eventually replace the current token formatter used in untyped code.
- Has some x86 simd support.. (opt-in if avail).
2026-10-16 13:30:42 -04:00
ed 8ba02de6ed Parser: Started to support proper errors as values, error recovery, and verbose testing of upfront and parsing interface.
Also fix to c_library generation for Genric_ on code body.
2026-10-10 20:23:19 -04:00
ed ed755e34f1 adding some control flow macros (not using yet) 2026-10-06 08:29:51 -04:00
ed 050019c8c0 more tests and fixes 2026-10-06 07:23:25 -04:00
ed cc7fdac795 more tests and fixes 2026-10-05 00:25:48 -04:00
ed 4140e7e237 setting up some tests... 2026-10-04 18:10:34 -04:00
ed 1a60fd54fc progress 2026-10-04 13:15:04 -04:00
ed b754adcd96 WIP: Starting to improve failure/error handling (errors as values) 2026-10-03 20:33:50 -04:00
ed 61d53362e8 reviewing project state, doing some tests. 2026-10-03 14:46:07 -04:00
ed f044c6d5eb cross-platform path for refactor_exe in void refactor_file( char const* path, char const* refactor_script ) 2026-08-06 15:06:10 -04:00
ed a2b5163341 add .ai to ignore 2026-03-22 11:15:02 -04:00
ed 286552c3c3 Merge remote-tracking branch 'origin/main' into dev
# Conflicts:
#	base/components/code_types.hpp
2025-10-31 22:19:46 -04:00
ed 9ff5908e40 Update LICENSE 2025-10-31 22:18:21 -04:00
ed 822a3b07dc misc changes (technically, old was never pushed...) 2025-06-15 17:56:26 -04:00
ed 685bba36d5 .gitignore/ai 2025-04-23 19:52:27 -04:00
ed 2ad164dc39 CodeSpecifiers fixes 2025-04-02 21:19:15 -04:00
ed 346e8e3305 fixes for CodeSpecifiers cpp mappings 2025-04-02 21:12:31 -04:00
ed bfc754e66c progress 2025-03-19 13:01:51 -04:00
ed 84f4fc5ae9 lexer/parser revamp progress: got c11 ver compiling (synchup before continuation) 2025-03-19 12:09:07 -04:00
ed ad5cb6597b progress on parser and lexer revamp 2025-03-19 11:48:40 -04:00
ed 1c7dd4ab32 WIP Broken: Working towards major changes to lexer
Lexer tokens now tracked using TokenSlice. ParserContext as the id for current token during traversal.
Last progress on errors needs updates to interface.parsing.cpp
Not adding message support yet as that will need to revamped with the logging, right now just focused on getting both lexer on parser to use proper info data structures.
Thinking of separating ParseContext from the general lib Context, plan is to just have the parser context have its allocator and other references it needs.
So there will seem to be redundant parameter passing to some procedures for now.
The big endgoal of this other than the parser's compression is the ability to support mult-threading.
Immediate concern other than making sure everything necessary is only within ParseContext, etc is things related to logging, or otherwise that is not thread dependent.
Those can get guarded but I don't have full intuition on what will have that (most likely the library's provided allocator/s as well will need guards introduced).
I'll concern myself more with charting those out once things are at least lifted properly. Worst case a trivial situation can be achived by the user by just abusing multiple contextes/allocators/etc as we already have in place.
2025-03-18 03:26:14 -04:00
ed 1e7fdcec16 preparing to revamp lexer 2025-03-17 01:20:56 -04:00
ed 2ed36506b1 progress on modularizing parser paths 2025-03-17 01:09:46 -04:00
ed 790087aa3c progress on modularizing parser paths 2025-03-16 23:13:46 -04:00
ed 441a46daaa changing to just pass lib Context instead of ParseContext.
If I want to make it more granular I can later.
2025-02-19 15:48:56 -05:00
ed 26623075ad started to work on modularizing parser code paths 2025-02-19 12:10:11 -05:00
ed 7ea90ef349 intellisense still broken... 2025-02-19 11:22:00 -05:00
ed 6d531fdf97 Fixing intellisense 2025-02-19 11:08:24 -05:00
ed 3b81eea688 Doing some initial prep for parser overhaul 2025-02-19 10:50:55 -05:00
ed 844d431e1c Setup Context's logger and fallback. (Not yet used in implementation). 2025-02-19 10:50:36 -05:00
ed 727b54c341 Added CodeSpecifiers::index_of for cpp, fix token bug with () operator in parse_operator_after_ret_type 2025-02-15 21:22:33 -05:00
ed ef72d27f3e made optionals for c11 actually work... 2025-02-11 19:24:42 -05:00
ed 75b1d42cca changes based off of metadesk genc_c11 2025-02-11 18:59:09 -05:00
62 changed files with 7651 additions and 1632 deletions

No files matched your search

+17 -35
View File
@@ -1,44 +1,26 @@
BSD 3-Clause License
MIT License
Copyright (c) 2023, Edward R. Gonzalez
Copyright (c) 2025 Edward R. Gonzalez
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Source URL: https://github.com/Ed94/gencpp
Acknowledgements
* The dependencies for gencpp source are derived from the zpl library: https://github.com/zpl-c/zpl
Special thanks to:
* The Handmade Community.
* Casey Muratori, Ginger Bill (Bill Hall), Mr. 4th (Allen Webster), Ryan Fluery: Influnced conceptually how to handle staged metaprograming.
* Jonathan Blow: Jai's metaprogramming influenced the design of this library.
* My friends for putting up with discord spam on this library.
+1 -1
View File
@@ -3,7 +3,7 @@
# include "helpers/push_ignores.inline.hpp"
# include "components/header_start.hpp"
# include "components/types.hpp"
# include "components/gen/ecode.hpp"
# include "components/gen/ecodetypes.hpp"
# include "components/gen/eoperator.hpp"
# include "components/gen/especifier.hpp"
# include "components/ast.hpp"
+1 -1
View File
@@ -3,7 +3,7 @@
# include "helpers/push_ignores.inline.hpp"
# include "components/header_start.hpp"
# include "components/types.hpp"
# include "components/gen/ecode.hpp"
# include "components/gen/ecodetypes.hpp"
# include "components/gen/eoperator.hpp"
# include "components/gen/especifier.hpp"
# include "components/ast.hpp"
+4 -6
View File
@@ -1,9 +1,6 @@
#ifdef INTELLISENSE_DIRECTIVES
#pragma once
#include "types.hpp"
#include "gen/ecode.hpp"
#include "gen/eoperator.hpp"
#include "gen/especifier.hpp"
#include "parser_types.hpp"
#endif
/*
@@ -406,7 +403,8 @@ struct AST
Code PostNameMacro; // Only used with parameters for specifically UE_REQUIRES (Thanks Unreal)
};
};
StrCached Content; // Attributes, Comment, Execution, Include
StrCached Content; // Attributes, Comment, Execution, Include
TokenSlice ContentToks; // TODO(Ed): Use a token slice for content
struct {
Specifier ArrSpecs[AST_ArrSpecs_Cap]; // Specifiers
Code NextSpecs; // Specifiers; If ArrSpecs is full, then NextSpecs is used.
@@ -422,7 +420,7 @@ struct AST
Code Next;
Code Back;
};
Token* Token; // Reference to starting token, only available if it was derived from parsing.
Token* Token; // Reference to starting token, only available if it was derived from parsing. // TODO(Ed): Change this to a token slice.
Code Parent;
CodeType Type;
// CodeFlag CodeFlags;
+2 -1
View File
@@ -1077,7 +1077,8 @@ struct AST_Typedef
struct
{
CodeComment InlineCmt;
char _PAD_PROPERTIES_[ sizeof(AST*) * 2 ];
CodeAttributes Attributes;
char _PAD_PROPERTIES_[ sizeof(AST*) ];
Code UnderlyingType;
char _PAD_PROPERTIES_2_[ sizeof(AST*) * 3 ];
};
+10 -11
View File
@@ -38,13 +38,13 @@ void body_to_strbuilder_export( CodeBody body, StrBuilder* result )
GEN_ASSERT(result != nullptr);
strbuilder_append_fmt( result, "export\n{\n" );
Code curr = cast(Code, body);
Code curr = body->Front;
s32 left = body->NumEntries;
while ( left-- )
{
code_to_strbuilder_ref(curr, result);
// strbuilder_append_fmt( result, "%SB", code_to_strbuilder(curr) );
++curr;
curr = curr->Next;
}
strbuilder_append_fmt( result, "};\n" );
@@ -158,7 +158,7 @@ void class_to_strbuilder_def( CodeClass self, StrBuilder* result )
if ( self->ParentType )
{
Str access_level = access_spec_to_str( self->ParentAccess );
strbuilder_append_fmt( result, " : %S %SB", self->Name, access_level, typename_to_strbuilder(self->ParentType) );
strbuilder_append_fmt( result, " : %S %SB", access_level, typename_to_strbuilder(self->ParentType) );
CodeTypename interface = cast(CodeTypename, self->ParentType->Next);
if ( interface )
@@ -263,12 +263,12 @@ void destructor_to_strbuilder_def(CodeDestructor self, StrBuilder* result )
else if ( self->Specs )
{
if ( specifiers_has(self->Specs, Spec_Virtual ) )
strbuilder_append_fmt( result, "virtual ~%S()", self->Parent->Name );
strbuilder_append_fmt( result, "virtual ~%S()", self->Parent->Parent->Name );
else
strbuilder_append_fmt( result, "~%S()", self->Parent->Name );
strbuilder_append_fmt( result, "~%S()", self->Parent->Parent->Name );
}
else
strbuilder_append_fmt( result, "~%S()", self->Parent->Name );
strbuilder_append_fmt( result, "~%S()", self->Parent->Parent->Name );
strbuilder_append_fmt( result, "\n{\n%SB\n}\n", code_to_strbuilder(self->Body) );
}
@@ -629,9 +629,6 @@ void code_op_to_strbuilder_def(CodeOperator self, StrBuilder* result )
if ( self->Attributes )
strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) );
if ( self->Attributes )
strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) );
if ( self->Specs )
{
for ( Specifier* spec = begin_CodeSpecifiers(self->Specs); spec != end_CodeSpecifiers(self->Specs); spec = next_CodeSpecifiers(self->Specs, spec) )
@@ -644,8 +641,7 @@ void code_op_to_strbuilder_def(CodeOperator self, StrBuilder* result )
}
}
if ( self->Attributes || self->Specs )
{
if ( self->Attributes || self->Specs ) {
strbuilder_append_str( result, txt("\n") );
}
@@ -1021,6 +1017,9 @@ void typedef_to_strbuilder_ref(CodeTypedef self, StrBuilder* result )
if ( bitfield_is_set( u32, self->ModuleFlags, ModuleFlag_Export ))
strbuilder_append_str( result, txt("export ") );
if ( self->Attributes )
strbuilder_append_fmt( result, "%SB ", attributes_to_strbuilder(self->Attributes) );
strbuilder_append_str( result, txt("typedef "));
// Determines if the typedef is a function typename
+8 -6
View File
@@ -252,7 +252,8 @@ struct CodeSpecifiers
#if ! GEN_C_LIKE_CPP
Using_Code( CodeSpecifiers );
bool append( Specifier spec ) { return specifiers_append(* this, spec); }
s32 has( Specifier spec ) { return specifiers_has(* this, spec); }
bool has( Specifier spec ) { return specifiers_has(* this, spec); }
s32 index_of( Specifier spec ) { return specifiers_index_of(* this, spec); }
s32 remove( Specifier to_remove ) { return specifiers_remove(* this, to_remove); }
StrBuilder to_strbuilder() { return specifiers_to_strbuilder(* this ); }
void to_strbuilder( StrBuilder& result ) { return specifiers_to_strbuilder_ref(* this, & result); }
@@ -1073,11 +1074,12 @@ forceinline bool has_entries (CodeParams params ) {
forceinline StrBuilder to_strbuilder(CodeParams params ) { return params_to_strbuilder(params); }
forceinline void to_strbuilder(CodeParams params, StrBuilder& result ) { return params_to_strbuilder_ref(params, & result); }
forceinline bool append (CodeSpecifiers specifiers, Specifier spec) { return specifiers_append(specifiers, spec); }
forceinline s32 has (CodeSpecifiers specifiers, Specifier spec) { return specifiers_has(specifiers, spec); }
forceinline s32 remove (CodeSpecifiers specifiers, Specifier to_remove ) { return specifiers_remove(specifiers, to_remove); }
forceinline StrBuilder to_strbuilder(CodeSpecifiers specifiers) { return specifiers_to_strbuilder(specifiers); }
forceinline void to_strbuilder(CodeSpecifiers specifiers, StrBuilder& result) { return specifiers_to_strbuilder_ref(specifiers, & result); }
forceinline bool append (CodeSpecifiers specifiers, Specifier spec) { return specifiers_append(specifiers, spec); }
forceinline bool has (CodeSpecifiers specifiers, Specifier spec) { return specifiers_has(specifiers, spec); }
forceinline s32 index_of (CodeSpecifiers specifiers, Specifier spec) { return specifiers_index_of(specifiers, spec); }
forceinline s32 remove (CodeSpecifiers specifiers, Specifier to_remove ) { return specifiers_remove(specifiers, to_remove); }
forceinline StrBuilder to_strbuilder (CodeSpecifiers specifiers) { return specifiers_to_strbuilder(specifiers); }
forceinline void to_strbuilder (CodeSpecifiers specifiers, StrBuilder& result) { return specifiers_to_strbuilder_ref(specifiers, & result); }
forceinline void add_interface (CodeStruct self, CodeTypename interface) { return struct_add_interface(self, interface); }
forceinline StrBuilder to_strbuilder (CodeStruct self) { return struct_to_strbuilder(self); }
+2
View File
@@ -6,6 +6,8 @@
#pragma region Constants
// Predefined typename codes. Are set to readonly and are setup during gen::init()
GEN_API extern Token NullToken;
GEN_API extern Macro enum_underlying_macro;
GEN_API extern Code access_public;
+2 -2
View File
@@ -1,6 +1,6 @@
#ifdef INTELLISENSE_DIRECTIVES
#pragma once
#include "interface.hpp"
#include "constants.hpp"
#endif
#pragma region Serialization
@@ -38,7 +38,7 @@ void body_to_strbuilder_ref( CodeBody body, StrBuilder* result )
{
code_to_strbuilder_ref(curr, result);
// strbuilder_append_fmt( result, "%SB", code_to_strbuilder(curr) );
++curr;
curr = curr->Next;
}
}
+28 -7
View File
@@ -3,8 +3,8 @@
#include "code_serialization.cpp"
#endif
internal void parser_init();
internal void parser_deinit();
internal void parser_init(Context* ctx);
internal void parser_deinit(Context* ctx);
internal
void* fallback_allocator_proc( void* allocator_data, AllocType type, ssize size, ssize alignment, void* old_memory, ssize old_size, u64 flags )
@@ -71,6 +71,14 @@ void* fallback_allocator_proc( void* allocator_data, AllocType type, ssize size,
return nullptr;
}
internal
void fallback_logger(LogEntry entry)
{
GEN_ASSERT(entry.msg.Len > 0);
GEN_ASSERT(entry.msg.Ptr);
log_fmt("%S: %S", loglevel_to_str(entry.level), entry.msg);
}
internal
void define_constants()
{
@@ -283,6 +291,19 @@ void init(Context* ctx)
ctx->InitSize_Fallback_Allocator_Bucket_Size = megabytes(8);
}
if (ctx->InitSize_StrCacheTable == 0)
{
ctx->InitSize_StrCacheTable = kilobytes(8);
}
if (ctx->InitSize_MacrosTable == 0)
{
ctx->InitSize_MacrosTable = kilobytes(8);
}
if (ctx->Logger == nullptr) {
ctx->Logger = & fallback_logger;
}
// Override the current context (user has to put it back if unwanted).
_ctx = ctx;
@@ -298,7 +319,7 @@ void init(Context* ctx)
}
// Setup the code pool and code entries arena.
{
Pool code_pool = pool_init( ctx->Allocator_Pool, ctx->CodePool_NumBlocks, sizeof(AST) );
Pool code_pool = pool_init( ctx->Allocator_Pool, ctx->CodePool_NumBlocks, size_of(AST) );
if ( code_pool.PhysicalStart == nullptr )
GEN_FATAL( "gen::init: Failed to initialize the code pool" );
array_append( ctx->CodePools, code_pool );
@@ -311,18 +332,18 @@ void init(Context* ctx)
}
// Setup the hash tables
{
ctx->StrCache = hashtable_init(StrCached, ctx->Allocator_DyanmicContainers);
ctx->StrCache = hashtable_init_reserve(StrCached, ctx->Allocator_DyanmicContainers, ctx->InitSize_StrCacheTable);
if ( ctx->StrCache.Entries == nullptr )
GEN_FATAL( "gen::init: Failed to initialize the StringCache");
ctx->Macros = hashtable_init(Macro, ctx->Allocator_DyanmicContainers);
ctx->Macros = hashtable_init_reserve(Macro, ctx->Allocator_DyanmicContainers, ctx->InitSize_MacrosTable);
if (ctx->Macros.Hashes == nullptr || ctx->Macros.Entries == nullptr) {
GEN_FATAL( "gen::init: Failed to initialize the PreprocessMacros table" );
}
}
define_constants();
parser_init();
parser_init(ctx);
++ context_counter;
}
@@ -371,7 +392,7 @@ void deinit(Context* ctx)
while ( left--, left );
array_free( ctx->Fallback_AllocatorBuckets);
}
parser_deinit();
parser_deinit(ctx);
if (_ctx == ctx)
_ctx = nullptr;
+106 -65
View File
@@ -15,24 +15,13 @@
\▓▓▓▓▓▓ \▓▓▓▓▓▓▓\▓▓ \▓▓ \▓▓▓▓▓▓\▓▓ \▓▓ \▓▓▓▓ \▓▓▓▓▓▓▓\▓▓ \▓▓ \▓▓▓▓▓▓▓ \▓▓▓▓▓▓▓ \▓▓▓▓▓▓▓
*/
#if 0
enum LogLevel : u32
struct ParseInfo
{
Info,
Warning,
Panic,
ParseMessage* messages;
LexedInfo lexed;
Code result;
};
struct LogEntry
{
Str msg;
u32 line_num;
void* data;
};
typedef void LoggerCallback(LogEntry entry);
#endif
// Note(Ed): This is subject to heavily change
// with upcoming changes to the library's fallback (default) allocations strategy;
// and major changes to lexer/parser context usage.
@@ -64,9 +53,16 @@ struct Context
u32 InitSize_LexerTokens;
u32 SizePer_StringArena;
u32 InitSize_StrCacheTable;
u32 InitSize_MacrosTable;
// TODO(Ed): Symbol Table
// Keep track of all resolved symbols (naemspaced identifiers)
// Logging
LoggerProc* Logger;
// Parser
// Used by the lexer to persistently treat all these identifiers as preprocessor defines.
@@ -89,11 +85,9 @@ struct Context
StringTable StrCache;
// TODO(Ed): This needs to be just handled by a parser context
Array(Token) Lexer_Tokens;
// TODO(Ed): Active parse context vs a parse result need to be separated conceptually
ParseContext parser;
ParseInfo parse_info;
// TODO(Ed): Formatting - This will eventually be in a separate struct when in the process of serialization of the builder.
s32 temp_serialize_indent;
@@ -104,6 +98,37 @@ struct Context
// An implicit context interface will be provided instead as wrapper procedures as convience.
GEN_API extern Context* _ctx;
// TODO(Ed): Swap all usage of this with logger_fmt (then rename logger_fmt to log_fmt)
inline
ssize log_fmt(char const* fmt, ...)
{
ssize res;
va_list va;
va_start(va, fmt);
res = c_str_fmt_out_va(fmt, va);
va_end(va);
return res;
}
inline
void logger_fmt(Context* ctx, LogLevel level, char const* fmt, ...)
{
local_persist thread_local
PrintF_Buffer buf = struct_zero_init();
va_list va;
va_start(va, fmt);
ssize res = c_str_fmt_va(buf, GEN_PRINTF_MAXLEN, fmt, va) -1;
va_end(va);
StrBuilder msg = strbuilder_make_length(ctx->Allocator_Temp, buf, res);
LogEntry entry = { strbuilder_to_str(msg), level };
ctx->Logger(entry);
}
// Initialize the library. There first ctx initialized must exist for lifetime of other contextes that come after as its the one that
GEN_API void init(Context* ctx);
@@ -114,7 +139,7 @@ GEN_API void deinit(Context* ctx);
// Retrieves the active context (not usually needed, but here in case...)
GEN_API Context* get_context();
// Clears the allocations, but doesn't free the memoery, then calls init() again.
// Clears the allocations, but doesn't free the memory, then calls init() again.
// Ease of use.
GEN_API void reset(Context* ctx);
@@ -334,58 +359,73 @@ forceinline CodeBody def_union_body ( s32 num, Code* codes )
#pragma region Parsing
#if 0
struct StackNode
struct ParseStackNode
{
StackNode* Prev;
ParseStackNode* prev;
Token Start;
Token Name; // The name of the AST node (if parsed)
Str FailedProc; // The name of the procedure that failed
};
// Stack nodes are allocated the error's allocator
struct Error
{
StrBuilder message;
StackNode* context_stack;
TokenSlice tokens;
Token* start;
Str name; // The name of the AST node (if parsed)
Str proc_name; // The name of the procedure
Code code_rel; // Relevant AST node
// TODO(Ed): When an error occurs, the parse stack is not released and instead the scope is left dangling.
};
struct ParseInfo
{
Arena FileMem;
Arena TokMem;
Arena CodeMem;
FileContents FileContent;
Array<Token> Tokens;
Array<Error> Errors;
// Errors are allocated to a dedicated general arena.
struct ParseOpts {
AllocatorInfo backing_msgs;
AllocatorInfo backing_tokens;
AllocatorInfo backing_ast;
};
CodeBody parse_file( Str path );
#endif
ParseInfo wip_parse_str( LexedInfo lexed, ParseOpts* opts GEN_PARAM_DEFAULT );
GEN_API CodeClass parse_class ( Str class_def );
GEN_API CodeConstructor parse_constructor ( Str constructor_def );
GEN_API CodeDefine parse_define ( Str define_def );
GEN_API CodeDestructor parse_destructor ( Str destructor_def );
GEN_API CodeEnum parse_enum ( Str enum_def );
GEN_API CodeBody parse_export_body ( Str export_def );
GEN_API CodeExtern parse_extern_link ( Str exten_link_def );
GEN_API CodeFriend parse_friend ( Str friend_def );
GEN_API CodeFn parse_function ( Str fn_def );
GEN_API CodeBody parse_global_body ( Str body_def );
GEN_API CodeNS parse_namespace ( Str namespace_def );
GEN_API CodeOperator parse_operator ( Str operator_def );
GEN_API CodeOpCast parse_operator_cast( Str operator_def );
GEN_API CodeStruct parse_struct ( Str struct_def );
GEN_API CodeTemplate parse_template ( Str template_def );
GEN_API CodeTypename parse_type ( Str type_def );
GEN_API CodeTypedef parse_typedef ( Str typedef_def );
GEN_API CodeUnion parse_union ( Str union_def );
GEN_API CodeUsing parse_using ( Str using_def );
GEN_API CodeVar parse_variable ( Str var_def );
struct Opts_parse {
Context* ctx;
ParseInfo* info;
};
typedef Opts_parse Opts_parse_global_body;
GEN_API void parse_class_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_constructor_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_define_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_destructor_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_enum_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_export_body_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_extern_link_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_friend_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_function_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_global_body_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_namespace_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_operator_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_operator_cast_base(Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_struct_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_template_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_type_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_typedef_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_union_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_using_base (Context* ctx, Str def, ParseInfo* info);
GEN_API void parse_variable_base (Context* ctx, Str def, ParseInfo* info);
GEN_API CodeClass parse_class (Str class_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeConstructor parse_constructor (Str constructor_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeDefine parse_define (Str define_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeDestructor parse_destructor (Str destructor_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeEnum parse_enum (Str enum_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeBody parse_export_body (Str export_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeExtern parse_extern_link (Str exten_link_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeFriend parse_friend (Str friend_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeFn parse_function (Str fn_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeBody parse_global_body (Str body_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeNS parse_namespace (Str namespace_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeOperator parse_operator (Str operator_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeOpCast parse_operator_cast(Str operator_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeStruct parse_struct (Str struct_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeTemplate parse_template (Str template_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeTypename parse_type (Str type_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeTypedef parse_typedef (Str typedef_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeUnion parse_union (Str union_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeUsing parse_using (Str using_def, Opts_parse opts GEN_PARAM_DEFAULT);
GEN_API CodeVar parse_variable (Str var_def, Opts_parse opts GEN_PARAM_DEFAULT);
#pragma endregion Parsing
@@ -395,9 +435,10 @@ GEN_API ssize token_fmt_va( char* buf, usize buf_size, s32 num_tokens, va_list v
//! Do not use directly. Use the token_fmt macro instead.
Str token_fmt_impl( ssize, ... );
GEN_API Code untyped_str( Str content);
GEN_API Code untyped_str ( Str content);
GEN_API Code untyped_fmt ( char const* fmt, ... );
GEN_API Code untyped_token_fmt( s32 num_tokens, char const* fmt, ... );
GEN_API Code untyped_toks ( TokenSlice tokens );
#pragma endregion Untyped text
+385 -230
View File
@@ -1,6 +1,6 @@
#ifdef INTELLISENSE_DIRECTIVES
#pragma once
#include "gen/etoktype.cpp"
#include "gen/etoktype.hpp"
#include "interface.upfront.cpp"
#include "lexer.cpp"
#include "parser.cpp"
@@ -8,43 +8,91 @@
// Publically Exposed Interface
CodeClass parse_class( Str def )
ParseInfo wip_parse_str(LexedInfo lexed, ParseOpts* opts)
{
check_parse_args( def );
// TODO(Ed): Lift this.
Context* ctx = _ctx; ParseInfo info = struct_zero(ParseInfo);
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
if (lexed.tokens.num == 0 && lexed.tokens.ptr == nullptr) {
ctx->parser = struct_zero(ParseContext);
if (check_parse_args(lexed.text) == false) {
info.messages = ctx->parser.messages;
goto done;
}
lexed = lex(ctx, lexed.text);
}
info.lexed = lexed;
_ctx->parser.Tokens = toks;
push_scope();
CodeClass result = (CodeClass) parse_class_struct( Tok_Decl_Class, parser_not_inplace_def );
parser_pop(& _ctx->parser);
return result;
// TODO(Ed): ParseInfo should be set to the parser context.
ctx->parser = struct_zero(ParseContext);
ctx->parser.tokens = lexed.tokens; {
ctx->parser.lexed_messages = lexed.messages;
ParseStackNode scope = NullScope; parser_push(& ctx->parser, & scope);
CodeBody result = parse_global_nspace(ctx,CT_Global_Body); (void)result;
parser_pop(& ctx->parser);
}
done:
return info;
}
CodeConstructor parse_constructor( Str def )
internal b32 parse_base_begin(Context* ctx, Str def, ParseInfo* info, char const* function_name) {
if (ctx == nullptr || info == nullptr) GEN_FATAL("parse base: null context or info");
info[0] = struct_zero(ParseInfo);
info->result = Code_Invalid;
ctx->parser = struct_zero(ParseContext);
if (_check_parse_args(ctx, def, function_name) == false) {
info->messages = ctx->parser.messages;
return false;
}
info->lexed = lex(ctx, def);
ctx->parser.lexed_messages = info->lexed.messages;
if (info->lexed.tokens.num == 0) {
info->messages = ctx->parser.messages;
return false;
}
ctx->parser.tokens = info->lexed.tokens;
return true;
}
void parse_class_base(Context* ctx, Str def, ParseInfo* info) {
CodeClass result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_class") == false) return;
/*Parse*/ {
ParseStackNode scope = NullScope; parser_push(& ctx->parser, & scope);
result = (CodeClass) parse_class_struct( ctx, Tok_Decl_Class, parser_not_inplace_def );
parser_pop(& ctx->parser);
}
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeClass parse_class(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_class: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_class_base(ctx, def, info); return cast(CodeClass, info->result);
}
void parse_constructor_base(Context* ctx, Str def, ParseInfo* info)
{
check_parse_args( def );
CodeConstructor result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_constructor") == false) return;
/*Parse*/ {
ParseStackNode scope = NullScope; parser_push(& ctx->parser, & scope);
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
// TODO(Ed): Constructors can have prefix attributes
CodeSpecifiers specifiers = NullCode;
Specifier specs_found[ 16 ] = { Spec_NumSpecifiers };
s32 NumSpecifiers = 0;
while ( left && tok_is_specifier(currtok) )
{
Specifier spec = str_to_specifier( currtok.Text );
b32 ignore_spec = false;
switch ( spec )
// TODO(Ed): Constructors can have prefix attributes
CodeSpecifiers specifiers = NullCode;
Specifier specs_found[ 16 ] = { Spec_NumSpecifiers };
s32 NumSpecifiers = 0;
while ( left && tok_is_specifier(currtok) )
{
b32 ignore_spec = false;
Specifier spec = str_to_specifier( currtok.Text );
switch ( spec ) {
case Spec_Constexpr :
case Spec_Explicit:
case Spec_Inline :
@@ -57,261 +105,367 @@ CodeConstructor parse_constructor( Str def )
break;
default :
log_failure( "Invalid specifier %s for variable\n%S", spec_to_str( spec ), parser_to_strbuilder(_ctx->parser) );
parser_pop(& _ctx->parser);
return InvalidCode;
log_failure( "Invalid specifier %s for variable\n%S", spec_to_str( spec ), parser_to_strbuilder(& ctx->parser, ctx->Allocator_Temp) );
parser_pop(& ctx->parser);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
return;
}
// Every specifier after would be considered part of the type type signature
if (ignore_spec) break;
specs_found[ NumSpecifiers ] = spec;
++ NumSpecifiers;
eat( currtok.Type );
}
// Every specifier after would be considered part of the type type signature
if (ignore_spec)
break;
specs_found[ NumSpecifiers ] = spec;
NumSpecifiers++;
eat( currtok.Type );
if ( NumSpecifiers ) {
specifiers = def_specifiers_arr( NumSpecifiers, specs_found );
// <specifiers> ...
}
result = parser_parse_constructor(ctx, specifiers);
parser_pop(& ctx->parser);
}
if ( NumSpecifiers )
{
specifiers = def_specifiers_arr( NumSpecifiers, specs_found );
// <specifiers> ...
}
_ctx->parser.Tokens = toks;
CodeConstructor result = parser_parse_constructor( specifiers );
return result;
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeDefine parse_define( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
push_scope();
CodeDefine result = parser_parse_define();
parser_pop(& _ctx->parser);
return result;
CodeConstructor parse_constructor(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_constructor: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_constructor_base(ctx, def, info); return cast(CodeConstructor, info->result);
}
CodeDestructor parse_destructor( Str def )
{
check_parse_args( def );
void parse_define_base(Context* ctx, Str def, ParseInfo* info) {
CodeDefine result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_define") == false) return;
/*Parse*/ {
ParseStackNode scope = NullScope; parser_push(& ctx->parser, & scope);
result = parser_parse_define(ctx);
parser_pop(& ctx->parser);
}
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
CodeDefine parse_define(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_define: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_define_base(ctx, def, info); return cast(CodeDefine, info->result);
}
void parse_destructor_base(Context* ctx, Str def, ParseInfo* info) {
CodeDestructor result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_destructor") == false) return;
// TODO(Ed): Destructors can have prefix attributes
// TODO(Ed): Destructors can have virtual
_ctx->parser.Tokens = toks;
CodeDestructor result = parser_parse_destructor(NullCode);
return result;
result = parser_parse_destructor(ctx, NullCode);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeEnum parse_enum( Str def )
{
check_parse_args( def );
CodeDestructor parse_destructor(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_destructor: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_destructor_base(ctx, def, info); return cast(CodeDestructor, info->result);
}
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
{
parser_pop(& _ctx->parser);
return InvalidCode;
void parse_enum_base(Context* ctx, Str def, ParseInfo* info) {
CodeEnum result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_enum") == false) return;
result = parser_parse_enum(ctx, parser_not_inplace_def);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeEnum parse_enum(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_enum: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_enum_base(ctx, def, info); return cast(CodeEnum, info->result);
}
void parse_export_body_base(Context* ctx, Str def, ParseInfo* info) {
CodeBody result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_export_body") == false) return;
result = parser_parse_export_body(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeBody parse_export_body(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_export_body: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_export_body_base(ctx, def, info); return cast(CodeBody, info->result);
}
void parse_extern_link_base(Context* ctx, Str def, ParseInfo* info) {
CodeExtern result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_extern_link") == false) return;
result = parser_parse_extern_link(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeExtern parse_extern_link(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_extern_link: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_extern_link_base(ctx, def, info); return cast(CodeExtern, info->result);
}
void parse_friend_base(Context* ctx, Str def, ParseInfo* info) {
CodeFriend result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_friend") == false) return;
result = parser_parse_friend(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeFriend parse_friend(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_friend: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_friend_base(ctx, def, info); return cast(CodeFriend, info->result);
}
void parse_function_base(Context* ctx, Str def, ParseInfo* info) {
CodeFn result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_function") == false) return;
result = (CodeFn) parser_parse_function(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeFn parse_function(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts);
Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_function: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = & ctx->parse_info;
parse_function_base(ctx, def, info); return cast(CodeFn, info->result);
}
void parse_global_body_base( Context* ctx, Str def, ParseInfo* info )
{
LexedInfo lexed; CodeBody body;
if ( ctx == nullptr || info == nullptr ) GEN_FATAL( "parse_global_body_base: null context or info" );
info[0] = struct_zero( ParseInfo );
lexed = struct_zero( LexedInfo );
body = (CodeBody) make_code();
body->Type = CT_Global_Body;
ctx->parser = struct_zero( ParseContext );
if ( check_parse_args( def ) == false ) {
info->result = cast( Code, body );
info->messages = ctx->parser.messages;
return;
}
_ctx->parser.Tokens = toks;
return parser_parse_enum( parser_not_inplace_def);
lexed = lex( ctx, def );
info->lexed = lexed;
ctx->parser.lexed_messages = lexed.messages;
if ( lexed.tokens.num == 0 ) {
info->result = cast( Code, body );
info->messages = ctx->parser.messages;
return;
}
ctx->parser.tokens = lexed.tokens; {
ParseStackNode scope = NullScope;
parser_push( & ctx->parser, & scope );
body = parse_global_nspace( ctx, CT_Global_Body );
parser_pop( & ctx->parser );
}
info->result = cast( Code, body );
info->messages = ctx->parser.messages;
}
CodeBody parse_export_body( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_export_body();
CodeBody parse_global_body( Str def, Opts_parse opts ) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if ( ctx == nullptr ) ctx = _ctx; if ( ctx == nullptr ) GEN_FATAL( "parse_global_body: no context" );
ParseInfo* info = resolved.info; if ( info == nullptr ) info = & ctx->parse_info;
parse_global_body_base( ctx, def, info ); return cast( CodeBody, info->result );
}
CodeExtern parse_extern_link( Str def )
{
check_parse_args( def );
void parse_namespace_base(Context* ctx, Str def, ParseInfo* info) {
CodeNS result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_namespace") == false) return;
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_extern_link();
result = parser_parse_namespace(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeFriend parse_friend( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_friend();
CodeNS parse_namespace(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_namespace: no context");
ParseInfo* info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_namespace_base(ctx, def, info); return cast(CodeNS, info->result);
}
CodeFn parse_function( Str def )
{
check_parse_args( def );
void parse_operator_base(Context* ctx, Str def, ParseInfo* info) {
CodeOperator result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_operator") == false) return;
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return (CodeFn) parser_parse_function();
result = (CodeOperator) parser_parse_operator(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeBody parse_global_body( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
push_scope();
CodeBody result = parse_global_nspace( CT_Global_Body );
parser_pop(& _ctx->parser);
return result;
CodeOperator parse_operator(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_operator: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_operator_base(ctx, def, info); return cast(CodeOperator, info->result);
}
CodeNS parse_namespace( Str def )
{
check_parse_args( def );
void parse_operator_cast_base(Context* ctx, Str def, ParseInfo* info) {
CodeOpCast result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_operator_cast") == false) return;
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_namespace();
result = parser_parse_operator_cast(ctx, NullCode);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeOperator parse_operator( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return (CodeOperator) parser_parse_operator();
CodeOpCast parse_operator_cast(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_operator_cast: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_operator_cast_base(ctx, def, info); return cast(CodeOpCast, info->result);
}
CodeOpCast parse_operator_cast( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_operator_cast(NullCode);
void parse_struct_base(Context* ctx, Str def, ParseInfo* info) {
CodeStruct result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_struct") == false) return;
/*Parse*/ {
ParseStackNode scope = NullScope;
parser_push(& ctx->parser, & scope);
result = (CodeStruct) parse_class_struct( ctx, Tok_Decl_Struct, parser_not_inplace_def );
parser_pop(& ctx->parser);
}
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeStruct parse_struct( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
push_scope();
CodeStruct result = (CodeStruct) parse_class_struct( Tok_Decl_Struct, parser_not_inplace_def );
parser_pop(& _ctx->parser);
return result;
CodeStruct parse_struct(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_struct: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_struct_base(ctx, def, info); return cast(CodeStruct, info->result);
}
CodeTemplate parse_template( Str def )
{
check_parse_args( def );
void parse_template_base(Context* ctx, Str def, ParseInfo* info) {
CodeTemplate result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_template") == false) return;
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_template();
result = parser_parse_template(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeTypename parse_type( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_type( parser_not_from_template, nullptr);
CodeTemplate parse_template(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_template: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_template_base(ctx, def, info); return cast(CodeTemplate, info->result);
}
CodeTypedef parse_typedef( Str def )
{
check_parse_args( def );
void parse_type_base(Context* ctx, Str def, ParseInfo* info) {
CodeTypename result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_type") == false) return;
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_typedef();
result = parser_parse_type(ctx, parser_not_from_template, nullptr);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeUnion parse_union( Str def )
{
check_parse_args( def );
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_union( parser_not_inplace_def);
CodeTypename parse_type(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_type: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_type_base(ctx, def, info); return cast(CodeTypename, info->result);
}
CodeUsing parse_using( Str def )
{
check_parse_args( def );
void parse_typedef_base(Context* ctx, Str def, ParseInfo* info) {
CodeTypedef result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_typedef") == false) return;
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
_ctx->parser.Tokens = toks;
return parser_parse_using();
result = parser_parse_typedef(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeVar parse_variable( Str def )
{
check_parse_args( def );
CodeTypedef parse_typedef(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_typedef: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_typedef_base(ctx, def, info); return cast(CodeTypedef, info->result);
}
TokArray toks = lex( def );
if ( toks.Arr == nullptr )
return InvalidCode;
void parse_union_base(Context* ctx, Str def, ParseInfo* info) {
CodeUnion result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_union") == false) return;
_ctx->parser.Tokens = toks;
return parser_parse_variable();
result = parser_parse_union(ctx, parser_not_inplace_def);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeUnion parse_union(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_union: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_union_base(ctx, def, info); return cast(CodeUnion, info->result);
}
void parse_using_base(Context* ctx, Str def, ParseInfo* info) {
CodeUsing result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_using") == false) return;
result = parser_parse_using(ctx);
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeUsing parse_using(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_using: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_using_base(ctx, def, info); return cast(CodeUsing, info->result);
}
void parse_variable_base(Context* ctx, Str def, ParseInfo* info) {
CodeVar result = InvalidCode;
if (parse_base_begin(ctx, def, info, "parse_variable") == false) return;
if (parser_slice_is_formatting_only(& ctx->parser)) {
parser_record_failure(ctx, txt("parse_variable: only formatting tokens"));
}
else {
result = parser_parse_variable(ctx);
}
info->result = cast(Code, result);
info->messages = ctx->parser.messages;
}
CodeVar parse_variable(Str def, Opts_parse opts) {
Opts_parse resolved = get_optional(opts); Context* ctx = resolved.ctx; ParseInfo* info;
if (ctx == nullptr) ctx = _ctx; if (ctx == nullptr) GEN_FATAL("parse_variable: no context");
info = resolved.info; if (info == nullptr) info = &ctx->parse_info;
parse_variable_base(ctx, def, info); return cast(CodeVar, info->result);
}
// Undef helper macros
@@ -326,6 +480,7 @@ CodeVar parse_variable( Str def )
#undef left
#undef check
#undef push_scope
#undef NullScope
#undef def_assign
// Here for C Variant
+13
View File
@@ -176,3 +176,16 @@ Code untyped_token_fmt( s32 num_tokens, char const* fmt, ... )
return result;
}
Code untyped_toks( TokenSlice tokens )
{
if ( tokens.num == 0 ) {
log_failure( "untyped_toks: empty token slice" );
return InvalidCode;
}
Code
result = make_code();
result->Type = CT_Untyped;
result->ContentToks = tokens;
return result;
}
+60 -21
View File
@@ -473,8 +473,10 @@ CodeComment def_comment( Str content )
return (CodeComment) result;
}
CodeConstructor def_constructor( Opts_def_constructor p )
CodeConstructor def_constructor( Opts_def_constructor opt )
{
Opts_def_constructor p = get_optional(opt);
if ( p.params && p.params->Type != CT_Parameters ) {
log_failure("gen::def_constructor: params must be of Parameters type - %s", code_debug_str((Code)p.params));
GEN_DEBUG_TRAP();
@@ -510,8 +512,10 @@ CodeConstructor def_constructor( Opts_def_constructor p )
return result;
}
CodeClass def_class( Str name, Opts_def_struct p )
CodeClass def_class( Str name, Opts_def_struct opt )
{
Opts_def_struct p = get_optional(opt);
if ( ! name_check( def_class, name ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -561,8 +565,10 @@ CodeClass def_class( Str name, Opts_def_struct p )
return result;
}
CodeDefine def_define( Str name, MacroType type, Opts_def_define p )
CodeDefine def_define( Str name, MacroType type, Opts_def_define opt )
{
Opts_def_define p = get_optional(opt);
if ( ! name_check( def_define, name ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -585,8 +591,10 @@ CodeDefine def_define( Str name, MacroType type, Opts_def_define p )
return result;
}
CodeDestructor def_destructor( Opts_def_destructor p )
CodeDestructor def_destructor( Opts_def_destructor opt )
{
Opts_def_destructor p = get_optional(opt);
if ( p.specifiers && p.specifiers->Type != CT_Specifiers ) {
log_failure( "gen::def_destructor: specifiers was not a 'Specifiers' type: %s", code_debug_str(p.specifiers) );
GEN_DEBUG_TRAP();
@@ -619,8 +627,10 @@ CodeDestructor def_destructor( Opts_def_destructor p )
return result;
}
CodeEnum def_enum( Str name, Opts_def_enum p )
CodeEnum def_enum( Str name, Opts_def_enum opt )
{
Opts_def_enum p = get_optional(opt);
if ( ! name_check( def_enum, name ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -742,8 +752,10 @@ CodeFriend def_friend( Code declaration )
return result;
}
CodeFn def_function( Str name, Opts_def_function p )
CodeFn def_function( Str name, Opts_def_function opt )
{
Opts_def_function p = get_optional(opt);
if ( ! name_check( def_function, name )) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -802,8 +814,10 @@ CodeFn def_function( Str name, Opts_def_function p )
return result;
}
CodeInclude def_include( Str path, Opts_def_include p )
CodeInclude def_include( Str path, Opts_def_include opt )
{
Opts_def_include p = get_optional(opt);
if ( path.Len <= 0 || path.Ptr == nullptr ) {
log_failure( "gen::def_include: Invalid path provided - %d" );
GEN_DEBUG_TRAP();
@@ -821,8 +835,10 @@ CodeInclude def_include( Str path, Opts_def_include p )
return result;
}
CodeModule def_module( Str name, Opts_def_module p )
CodeModule def_module( Str name, Opts_def_module opt )
{
Opts_def_module p = get_optional(opt);
if ( ! name_check( def_module, name )) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -835,8 +851,10 @@ CodeModule def_module( Str name, Opts_def_module p )
return result;
}
CodeNS def_namespace( Str name, CodeBody body, Opts_def_namespace p )
CodeNS def_namespace( Str name, CodeBody body, Opts_def_namespace opt )
{
Opts_def_namespace p = get_optional(opt);
if ( ! name_check( def_namespace, name )) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -859,8 +877,10 @@ CodeNS def_namespace( Str name, CodeBody body, Opts_def_namespace p )
return result;
}
CodeOperator def_operator( Operator op, Str nspace, Opts_def_operator p )
CodeOperator def_operator( Operator op, Str nspace, Opts_def_operator opt )
{
Opts_def_operator p = get_optional(opt);
if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) {
log_failure( "gen::def_operator: PlatformAttributes was provided but its not of attributes type: %s", code_debug_str(p.attributes) );
GEN_DEBUG_TRAP();
@@ -926,8 +946,10 @@ CodeOperator def_operator( Operator op, Str nspace, Opts_def_operator p )
return result;
}
CodeOpCast def_operator_cast( CodeTypename type, Opts_def_operator_cast p )
CodeOpCast def_operator_cast( CodeTypename type, Opts_def_operator_cast opt )
{
Opts_def_operator_cast p = get_optional(opt);
if ( ! null_check( def_operator_cast, type )) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -959,8 +981,10 @@ CodeOpCast def_operator_cast( CodeTypename type, Opts_def_operator_cast p )
return result;
}
CodeParams def_param( CodeTypename type, Str name, Opts_def_param p )
CodeParams def_param( CodeTypename type, Str name, Opts_def_param opt )
{
Opts_def_param p = get_optional(opt);
if ( ! name_check( def_param, name ) || ! null_check( def_param, type ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -1034,8 +1058,10 @@ CodeSpecifiers def_specifier( Specifier spec )
return result;
}
CodeStruct def_struct( Str name, Opts_def_struct p )
CodeStruct def_struct( Str name, Opts_def_struct opt )
{
Opts_def_struct p = get_optional(opt);
if ( p.attributes && p.attributes->Type != CT_PlatformAttributes ) {
log_failure( "gen::def_struct: attributes was not a `PlatformAttributes` type - %s", code_debug_str(cast(Code, p.attributes)) );
GEN_DEBUG_TRAP();
@@ -1076,8 +1102,10 @@ CodeStruct def_struct( Str name, Opts_def_struct p )
return result;
}
CodeTemplate def_template( CodeParams params, Code declaration, Opts_def_template p )
CodeTemplate def_template( CodeParams params, Code declaration, Opts_def_template opt )
{
Opts_def_template p = get_optional(opt);
if ( ! null_check( def_template, declaration ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -1108,8 +1136,10 @@ CodeTemplate def_template( CodeParams params, Code declaration, Opts_def_templat
return result;
}
CodeTypename def_type( Str name, Opts_def_type p )
CodeTypename def_type( Str name, Opts_def_type opt )
{
Opts_def_type p = get_optional(opt);
if ( ! name_check( def_type, name )) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -1128,7 +1158,7 @@ CodeTypename def_type( Str name, Opts_def_type p )
return InvalidCode;
}
if ( p.array_expr && p.array_expr->Type != CT_Untyped ) {
log_failure( "gen::def_type: arrayexpr is not of untyped type - %s", code_debug_str((Code)p.array_expr) );
log_failure( "gen::def_type: arrayexpr is not of untyped type - %S", code_debug_str((Code)p.array_expr) );
GEN_DEBUG_TRAP();
return InvalidCode;
}
@@ -1143,8 +1173,10 @@ CodeTypename def_type( Str name, Opts_def_type p )
return result;
}
CodeTypedef def_typedef( Str name, Code type, Opts_def_typedef p )
CodeTypedef def_typedef( Str name, Code type, Opts_def_typedef opt )
{
Opts_def_typedef p = get_optional(opt);
if ( ! null_check( def_typedef, type ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -1186,6 +1218,7 @@ CodeTypedef def_typedef( Str name, Code type, Opts_def_typedef p )
result = (CodeTypedef) make_code();
result->Type = CT_Typedef;
result->ModuleFlags = p.mflags;
result->Attributes = p.attributes;
result->UnderlyingType = type;
if ( name.Len <= 0 )
@@ -1206,8 +1239,10 @@ CodeTypedef def_typedef( Str name, Code type, Opts_def_typedef p )
return result;
}
CodeUnion def_union( Str name, CodeBody body, Opts_def_union p )
CodeUnion def_union( Str name, CodeBody body, Opts_def_union opt )
{
Opts_def_union p = get_optional(opt);
if ( ! null_check( def_union, body ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
@@ -1233,9 +1268,11 @@ CodeUnion def_union( Str name, CodeBody body, Opts_def_union p )
return result;
}
CodeUsing def_using( Str name, CodeTypename type, Opts_def_using p )
CodeUsing def_using( Str name, CodeTypename type, Opts_def_using opt )
{
if ( ! name_check( def_using, name ) || null_check( def_using, type ) ) {
Opts_def_using p = get_optional(opt);
if ( ! name_check( def_using, name ) || ! null_check( def_using, type ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
}
@@ -1274,8 +1311,10 @@ CodeUsing def_using_namespace( Str name )
return result;
}
CodeVar def_variable( CodeTypename type, Str name, Opts_def_variable p )
CodeVar def_variable( CodeTypename type, Str name, Opts_def_variable opt )
{
Opts_def_variable p = get_optional(opt);
if ( ! name_check( def_variable, name ) || ! null_check( def_variable, type ) ) {
GEN_DEBUG_TRAP();
return InvalidCode;
+176 -146
View File
@@ -1,14 +1,12 @@
#ifdef INTELLISENSE_DIRECTIVES
#pragma once
#include "interface.upfront.cpp"
#include "gen/etoktype.cpp"
#include "gen/etoktype.hpp"
#endif
StrBuilder tok_to_strbuilder(Token tok)
{
StrBuilder result = strbuilder_make_reserve( _ctx->Allocator_Temp, kilobytes(4) );
StrBuilder tok_to_strbuilder(AllocatorInfo ainfo, Token tok) {
StrBuilder result = strbuilder_make_reserve( ainfo, kilobytes(4) );
Str type_str = toktype_to_str( tok.Type );
strbuilder_append_fmt( & result, "Line: %d Column: %d, Type: %.*s Content: %.*s"
, tok.Line, tok.Column
, type_str.Len, type_str.Ptr
@@ -17,59 +15,70 @@ StrBuilder tok_to_strbuilder(Token tok)
return result;
}
bool lex__eat( TokArray* self, TokType type );
bool lex__eat(Context* ctx, ParseContext* self, TokType type );
Token* lex_current(TokArray* self, bool skip_formatting )
{
if ( skip_formatting )
{
while ( self->Arr[self->Idx].Type == Tok_NewLine || self->Arr[self->Idx].Type == Tok_Comment )
self->Idx++;
}
return & self->Arr[self->Idx];
forceinline
b32 lex_index_in_range(ParseContext const* self, s32 idx) {
return self->tokens.ptr != nullptr
&& idx >= 0
&& idx < self->tokens.num;
}
Token* lex_peek(TokArray self, bool skip_formatting)
{
s32 idx = self.Idx;
if ( skip_formatting )
{
while ( self.Arr[idx].Type == Tok_NewLine )
idx++;
return & self.Arr[idx];
}
return & self.Arr[idx];
forceinline
Token* lex_token_at(ParseContext const* self, s32 idx) {
Token* result = & NullToken;
if (lex_index_in_range(self, idx)) result = & self->tokens.ptr[idx];
return result;
}
Token* lex_previous(TokArray self, bool skip_formatting)
{
s32 idx = self.Idx;
if ( skip_formatting )
{
while ( self.Arr[idx].Type == Tok_NewLine )
idx --;
return & self.Arr[idx];
Token* lex_current(ParseContext* self, bool skip_formatting) {
if (skip_formatting) while (lex_index_in_range(self, self->token_id)) {
TokType type = self->tokens.ptr[self->token_id].Type;
if (type != Tok_NewLine && type != Tok_Comment) break;
++ self->token_id;
}
return & self.Arr[idx - 1];
return lex_token_at(self, self->token_id);
}
Token* lex_next(TokArray self, bool skip_formatting)
{
s32 idx = self.Idx;
if ( skip_formatting )
{
while ( self.Arr[idx].Type == Tok_NewLine )
idx++;
return & self.Arr[idx + 1];
Token* lex_peek(ParseContext const* self, bool skip_formatting) {
s32 idx = self->token_id;
if (skip_formatting) {
while (lex_index_in_range(self, idx) && self->tokens.ptr[idx].Type == Tok_NewLine)
++ idx;
}
return & self.Arr[idx + 1];
return lex_token_at(self, idx);
}
enum
Token* lex_previous(ParseContext const* self, bool skip_formatting)
{
Token* result = & NullToken;
s32 idx = self->token_id;
if (skip_formatting)
{
if (self->tokens.num > 0 && idx >= self->tokens.num)
idx = self->tokens.num - 1;
while (lex_index_in_range(self, idx) && self->tokens.ptr[idx].Type == Tok_NewLine && idx > 0)
-- idx;
if (lex_index_in_range(self, idx) && self->tokens.ptr[idx].Type != Tok_NewLine)
result = &self->tokens.ptr[idx];
goto done;
}
if (self->tokens.num > 0 && idx >= self->tokens.num) result = lex_token_at(self, self->tokens.num - 1);
else result = lex_token_at(self, idx - 1);
done:
return result;
}
Token* lex_next(ParseContext const* self, bool skip_formatting) {
s32 idx = self->token_id;
if (skip_formatting) {
while (lex_index_in_range(self, idx) && self->tokens.ptr[idx].Type == Tok_NewLine)
-- idx;
}
return lex_token_at(self, idx + 1);
}
enum {
Lex_Continue,
Lex_ReturnNull,
};
@@ -90,19 +99,16 @@ void lexer_move_forward( LexContext* ctx )
#define move_forward() lexer_move_forward(ctx)
forceinline
void lexer_skip_whitespace( LexContext* ctx )
{
void lexer_skip_whitespace( LexContext* ctx ) {
while ( ctx->left && char_is_space( * ctx->scanner ) )
move_forward();
}
#define skip_whitespace() lexer_skip_whitespace(ctx)
forceinline
void lexer_end_line( LexContext* ctx )
{
void lexer_end_line( LexContext* ctx ) {
while ( ctx->left && (* ctx->scanner) == ' ' )
move_forward();
if ( ctx->left && (* ctx->scanner) == '\r' ) {
move_forward();
move_forward();
@@ -113,7 +119,9 @@ void lexer_end_line( LexContext* ctx )
#define end_line() lexer_end_line(ctx)
// TODO(Ed): We need to to attempt to recover from a lex failure?
s32 lex_preprocessor_define( LexContext* ctx )
internal void lexer_record_failure_fmt(Context* lib_ctx, LexContext* ctx, char const* fmt, ...);
s32 lex_preprocessor_define(Context* lib_ctx, LexContext* ctx)
{
Token name = { { ctx->scanner, 1 }, Tok_Identifier, ctx->line, ctx->column, TF_Preprocess };
move_forward();
@@ -137,7 +145,7 @@ s32 lex_preprocessor_define( LexContext* ctx )
);
// GEN_DEBUG_TRAP();
}
array_append( _ctx->Lexer_Tokens, name );
array_append(ctx->tokens, name);
if ( ctx->left && (* ctx->scanner) == '(' )
{
@@ -152,7 +160,7 @@ s32 lex_preprocessor_define( LexContext* ctx )
}
Token opening_paren = { { ctx->scanner, 1 }, Tok_Paren_Open, ctx->line, ctx->column, TF_Preprocess };
array_append( _ctx->Lexer_Tokens, opening_paren );
array_append(ctx->tokens, opening_paren);
move_forward();
Token last_parameter = {};
@@ -168,7 +176,7 @@ s32 lex_preprocessor_define( LexContext* ctx )
move_forward();
move_forward();
array_append(_ctx->Lexer_Tokens, parameter);
array_append(ctx->tokens, parameter);
skip_whitespace();
last_parameter = parameter;
@@ -178,7 +186,7 @@ s32 lex_preprocessor_define( LexContext* ctx )
}
if (* ctx->scanner != ')' )
{
log_failure("lex_preprocessor_define(%d, %d): Expected a ')' after '...' (varaidc macro param) %S\n"
lexer_record_failure_fmt(lib_ctx, ctx, "lex_preprocessor_define(%d, %d): Expected a ')' after '...' (varaidc macro param) %S\n"
, ctx->line
, ctx->column
, name.Text
@@ -202,12 +210,12 @@ s32 lex_preprocessor_define( LexContext* ctx )
move_forward();
parameter.Text.Len++;
}
array_append(_ctx->Lexer_Tokens, parameter);
array_append(ctx->tokens, parameter);
skip_whitespace();
last_parameter = parameter;
}
else {
log_failure("lex_preprocessor_define(%d, %d): Expected a '_' or alpha character for a parameter name for %S\n"
lexer_record_failure_fmt(lib_ctx, ctx, "lex_preprocessor_define(%d, %d): Expected a '_' or alpha character for a parameter name for %S\n"
, ctx->line
, ctx->column
, name.Text
@@ -220,7 +228,7 @@ s32 lex_preprocessor_define( LexContext* ctx )
// There should be a comma
if ( * ctx->scanner != ',' ) {
log_failure("lex_preprocessor_define(%d, %d): Expected a comma after parameter %S for %S\n"
lexer_record_failure_fmt(lib_ctx, ctx, "lex_preprocessor_define(%d, %d): Expected a comma after parameter %S for %S\n"
, ctx->line
, ctx->column
, last_parameter.Text
@@ -229,12 +237,12 @@ s32 lex_preprocessor_define( LexContext* ctx )
return Lex_ReturnNull;
}
Token comma = { { ctx->scanner, 1 }, Tok_Comma, ctx->line, ctx->column, TF_Preprocess };
array_append(_ctx->Lexer_Tokens, comma);
array_append(ctx->tokens, comma);
move_forward();
}
if ( * ctx->scanner != ')' ) {
log_failure("lex_preprocessor_define(%d, %d): Expected a ')' after last_parameter %S for %S (ran out of characters...)\n"
lexer_record_failure_fmt(lib_ctx, ctx, "lex_preprocessor_define(%d, %d): Expected a ')' after last_parameter %S for %S (ran out of characters...)\n"
, ctx->line
, ctx->column
, last_parameter.Text
@@ -243,7 +251,7 @@ s32 lex_preprocessor_define( LexContext* ctx )
return Lex_ReturnNull;
}
Token closing_paren = { { ctx->scanner, 1 }, Tok_Paren_Close, ctx->line, ctx->column, TF_Preprocess };
array_append(_ctx->Lexer_Tokens, closing_paren);
array_append(ctx->tokens, closing_paren);
move_forward();
}
else if ( registered_macro && macro_is_functional( * registered_macro) ) {
@@ -264,11 +272,11 @@ s32 lex_preprocessor_define( LexContext* ctx )
}
// TODO(Ed): We need to to attempt to recover from a lex failure?
s32 lex_preprocessor_directive( LexContext* ctx )
s32 lex_preprocessor_directive(Context* lib_ctx, LexContext* ctx)
{
char const* hash = ctx->scanner;
Token hash_tok = { { hash, 1 }, Tok_Preprocess_Hash, ctx->line, ctx->column, TF_Preprocess };
array_append( _ctx->Lexer_Tokens, hash_tok );
array_append(ctx->tokens, hash_tok);
move_forward();
skip_whitespace();
@@ -317,11 +325,11 @@ s32 lex_preprocessor_directive( LexContext* ctx )
}
else
{
log_failure( "gen::Parser::lex: Invalid escape sequence '\\%c' (%d, %d)"
lexer_record_failure_fmt(lib_ctx, ctx, "gen::Parser::lex: Invalid escape sequence '\\%c' (%d, %d)"
" in preprocessor directive (%d, %d)\n%.100s"
, (* ctx->scanner), ctx->line, ctx->column
, ctx->token.Line, ctx->token.Column, ctx->token.Text );
break;
return Lex_ReturnNull;
}
}
@@ -344,14 +352,14 @@ s32 lex_preprocessor_directive( LexContext* ctx )
ctx->token.Text.Len = ctx->token.Text.Len + ctx->token.Text.Ptr - hash;
ctx->token.Text.Ptr = hash;
array_append( _ctx->Lexer_Tokens, ctx->token );
array_append(ctx->tokens, ctx->token);
return Lex_Continue; // Skip found token, its all handled here.
}
if ( ctx->token.Type == Tok_Preprocess_Else || ctx->token.Type == Tok_Preprocess_EndIf )
{
ctx->token.Flags |= TF_Preprocess_Cond;
array_append( _ctx->Lexer_Tokens, ctx->token );
array_append(ctx->tokens, ctx->token);
end_line();
return Lex_Continue;
}
@@ -360,13 +368,13 @@ s32 lex_preprocessor_directive( LexContext* ctx )
ctx->token.Flags |= TF_Preprocess_Cond;
}
array_append( _ctx->Lexer_Tokens, ctx->token );
array_append(ctx->tokens, ctx->token);
skip_whitespace();
if ( ctx->token.Type == Tok_Preprocess_Define )
{
u32 result = lex_preprocessor_define(ctx); // handles: #define <name>( <params> ) - define's content handled later on within this scope.
u32 result = lex_preprocessor_define(lib_ctx, ctx); // handles: #define <name>( <params> ) - define's content handled later on within this scope.
if (result != Lex_Continue)
return Lex_ReturnNull;
}
@@ -379,9 +387,9 @@ s32 lex_preprocessor_directive( LexContext* ctx )
if ( (* ctx->scanner) != '"' && (* ctx->scanner) != '<' )
{
StrBuilder directive_str = strbuilder_fmt_buf( _ctx->Allocator_Temp, "%.*s", min( 80, ctx->left + preprocess_content.Text.Len ), ctx->token.Text.Ptr );
StrBuilder directive_str = strbuilder_fmt_buf( ctx->allocator_temp, "%.*s", min( 80, ctx->left + preprocess_content.Text.Len ), ctx->token.Text.Ptr );
log_failure( "gen::Parser::lex: Expected '\"' or '<' after #include, not '%c' (%d, %d)\n%s"
lexer_record_failure_fmt(lib_ctx, ctx, "gen::Parser::lex: Expected '\"' or '<' after #include, not '%c' (%d, %d)\n%s"
, (* ctx->scanner)
, preprocess_content.Line
, preprocess_content.Column
@@ -411,7 +419,7 @@ s32 lex_preprocessor_directive( LexContext* ctx )
move_forward();
}
array_append( _ctx->Lexer_Tokens, preprocess_content );
array_append(ctx->tokens, preprocess_content);
return Lex_Continue; // Skip found token, its all handled here.
}
@@ -446,16 +454,15 @@ s32 lex_preprocessor_directive( LexContext* ctx )
}
else
{
StrBuilder directive_str = strbuilder_make_length( _ctx->Allocator_Temp, ctx->token.Text.Ptr, ctx->token.Text.Len );
StrBuilder content_str = strbuilder_fmt_buf( _ctx->Allocator_Temp, "%.*s", min( 400, ctx->left + preprocess_content.Text.Len ), preprocess_content.Text.Ptr );
StrBuilder directive_str = strbuilder_make_length( ctx->allocator_temp, ctx->token.Text.Ptr, ctx->token.Text.Len );
StrBuilder content_str = strbuilder_fmt_buf( ctx->allocator_temp, "%.*s", min( 400, ctx->left + preprocess_content.Text.Len ), preprocess_content.Text.Ptr );
log_failure( "gen::Parser::lex: Invalid escape sequence '\\%c' (%d, %d)"
lexer_record_failure_fmt(lib_ctx, ctx, "gen::Parser::lex: Invalid escape sequence '\\%c' (%d, %d)"
" in preprocessor directive '%s' (%d, %d)\n%s"
, (* ctx->scanner), ctx->line, ctx->column
, directive_str, preprocess_content.Line, preprocess_content.Column
, content_str );
return Lex_ReturnNull;
break;
}
}
@@ -475,14 +482,14 @@ s32 lex_preprocessor_directive( LexContext* ctx )
preprocess_content.Text.Len++;
}
array_append( _ctx->Lexer_Tokens, preprocess_content );
array_append(ctx->tokens, preprocess_content);
return Lex_Continue; // Skip found token, its all handled here.
}
void lex_found_token( LexContext* ctx )
{
if ( ctx->token.Type != Tok_Invalid ) {
array_append( _ctx->Lexer_Tokens, ctx->token );
array_append(ctx->tokens, ctx->token);
return;
}
@@ -508,7 +515,7 @@ void lex_found_token( LexContext* ctx )
}
ctx->token.Type = type;
array_append( _ctx->Lexer_Tokens, ctx->token );
array_append(ctx->tokens, ctx->token);
return;
}
if ( ( type <= Tok_Star && type >= Tok_Spec_Alignas)
@@ -517,13 +524,13 @@ void lex_found_token( LexContext* ctx )
{
ctx->token.Type = type;
ctx->token.Flags |= TF_Specifier;
array_append( _ctx->Lexer_Tokens, ctx->token );
array_append(ctx->tokens, ctx->token);
return;
}
if ( type != Tok_Invalid )
{
ctx->token.Type = type;
array_append( _ctx->Lexer_Tokens, ctx->token );
array_append(ctx->tokens, ctx->token);
return;
}
@@ -534,11 +541,16 @@ void lex_found_token( LexContext* ctx )
ctx->token.Type = macrotype_to_toktype(macro->Type);
b32 is_functional = macro_is_functional(* macro);
resolved_to_macro = has_args ? is_functional : ! is_functional;
if ( ! resolved_to_macro && GEN_BUILD_DEBUG ) {
log_fmt("Info(%d, %d): %S identified as a macro but usage here does not resolve to one (interpreting as identifier)\n"
, ctx->token.Line
if ( ! resolved_to_macro && GEN_BUILD_DEBUG && ! bitfield_is_set(MacroFlags, macro->Flags, MF_Allow_As_Identifier) ) {
log_fmt("Info(%d, %d): %S identified as a macro but usage here does not resolve to one (functional=%d, has_args=%d, flags=%u, next='%.*s')\n"
, ctx->token.Line
, ctx->token.Column
, macro->Name
, is_functional
, has_args
, macro->Flags
, min(16, ctx->left)
, ctx->scanner
);
}
}
@@ -561,50 +573,66 @@ void lex_found_token( LexContext* ctx )
ctx->token.Type = Tok_Identifier;
}
array_append( _ctx->Lexer_Tokens, ctx->token );
array_append(ctx->tokens, ctx->token);
}
// TODO(Ed): We should dynamically allocate the lexer's array in Allocator_DyanmicContainers.
// TODO(Ed): We need to to attempt to recover from a lex failure?
internal
void lexer_record_failure(Context* lib_ctx, LexContext* ctx, Str text) {
LexerMessage* msg = rcast(LexerMessage*, alloc(lib_ctx->Allocator_DyanmicContainers, size_of(LexerMessage)));
if (msg == nullptr) GEN_FATAL("lexer_record_failure: allocation failed");
msg->next = ctx->messages;
msg->content = cache_str(text);
msg->level = LL_Error;
ctx->messages = msg;
}
internal
void lexer_record_failure_fmt(Context* lib_ctx, LexContext* ctx, char const* fmt, ...) {
va_list va;
char* buf;
Str text;
va_start(va, fmt);
buf = c_str_fmt_buf_va(fmt, va);
va_end(va);
text.Ptr = buf;
text.Len = c_str_len(buf);
lexer_record_failure(lib_ctx, ctx, text);
}
neverinline
// TokArray lex( Array<Token> tokens, Str content )
TokArray lex( Str content )
LexedInfo lex(Context* lib_ctx, Str content)
{
LexContext c; LexContext* ctx = & c;
c.content = content;
c.left = content.Len;
c.scanner = content.Ptr;
LexedInfo info = struct_zero(LexedInfo);
char const* word = c.scanner;
s32 word_length = 0;
LexContext c = struct_zero(LexContext); LexContext* ctx = & c;
c.allocator_temp = lib_ctx->Allocator_Temp;
c.content = content;
c.left = content.Len;
c.scanner = content.Ptr;
c.line = 1;
c.column = 1;
c.tokens = array_init_reserve(Token, lib_ctx->Allocator_DyanmicContainers, lib_ctx->InitSize_LexerTokens );
c.line = 1;
c.column = 1;
skip_whitespace();
if ( c.left <= 0 )
{
log_failure( "gen::lex: no tokens found (only whitespace provided)" );
TokArray null_array = {};
return null_array;
}
array_clear(_ctx->Lexer_Tokens);
// TODO(Ed): Re-implement to new constraints:
// 1. Ability to continue on error
// 2. Return a lexed info.
b32 preprocess_args = true;
skip_whitespace();
if ( c.left <= 0 ) {
lexer_record_failure(lib_ctx, ctx, txt("gen::lex: no tokens found (only whitespace provided)"));
info.messages = c.messages;
goto Lex_Done;
}
while (c.left )
{
#if 0
if (Tokens.num())
{
log_fmt("\nLastTok: %SB", Tokens.back().to_strbuilder());
}
#endif
{
Token thanks_c = { { c.scanner, 0 }, Tok_Invalid, c.line, c.column, TF_Null };
c.token = thanks_c;
}
c.token = struct_init(Token) { { c.scanner, 0 }, Tok_Invalid, c.line, c.column, TF_Null };
bool is_define = false;
@@ -623,7 +651,7 @@ TokArray lex( Str content )
c.token.Type = Tok_NewLine;
c.token.Text.Len++;
array_append( _ctx->Lexer_Tokens, c.token );
array_append(c.tokens, c.token);
continue;
}
}
@@ -638,7 +666,7 @@ TokArray lex( Str content )
{
case '#':
{
s32 result = lex_preprocessor_directive( ctx );
s32 result = lex_preprocessor_directive(lib_ctx, ctx);
switch ( result )
{
case Lex_Continue:
@@ -662,7 +690,7 @@ TokArray lex( Str content )
c.token.Text.Len++;
move_forward();
array_append( _ctx->Lexer_Tokens, c.token );
array_append(c.tokens, c.token);
}
}
continue;
@@ -670,8 +698,8 @@ TokArray lex( Str content )
case Lex_ReturnNull:
{
TokArray tok_array = {};
return tok_array;
info.messages = c.messages;
goto Lex_Done;
}
}
}
@@ -698,9 +726,11 @@ TokArray lex( Str content )
}
else
{
StrBuilder context_str = strbuilder_fmt_buf( _ctx->Allocator_Temp, "%s", c.scanner, min( 100, c.left ) );
StrBuilder context_str = strbuilder_fmt_buf( lib_ctx->Allocator_Temp, "%s", c.scanner, min( 100, c.left ) );
log_failure( "gen::lex: invalid varadic argument, expected '...' got '..%c' (%d, %d)\n%s", (* ctx->scanner), c.line, c.column, context_str );
lexer_record_failure_fmt(lib_ctx, ctx, "gen::lex: invalid varadic argument, expected '...' got '..%c' (%d, %d)\n%s", (* ctx->scanner), c.line, c.column, context_str );
info.messages = c.messages;
goto Lex_Done;
}
}
@@ -1119,7 +1149,7 @@ TokArray lex( Str content )
move_forward();
c.token.Text.Len++;
}
array_append( _ctx->Lexer_Tokens, c.token );
array_append(c.tokens, c.token);
continue;
}
else if ( (* ctx->scanner) == '*' )
@@ -1155,7 +1185,7 @@ TokArray lex( Str content )
move_forward();
c.token.Text.Len++;
}
array_append( _ctx->Lexer_Tokens, c.token );
array_append(c.tokens, c.token);
// end_line();
continue;
}
@@ -1243,30 +1273,27 @@ TokArray lex( Str content )
}
else
{
s32 start = max( 0, array_num(_ctx->Lexer_Tokens) - 100 );
s32 start = max( 0, array_num(c.tokens) - 100 );
log_fmt("\n%d\n", start);
for ( s32 idx = start; idx < array_num(_ctx->Lexer_Tokens); idx++ )
for ( s32 idx = start; idx < array_num(c.tokens); idx++ )
{
log_fmt( "Token %d Type: %s : %.*s\n"
, idx
, toktype_to_str( _ctx->Lexer_Tokens[ idx ].Type ).Ptr
, _ctx->Lexer_Tokens[ idx ].Text.Len, _ctx->Lexer_Tokens[ idx ].Text.Ptr
, toktype_to_str( c.tokens[ idx ].Type ).Ptr
, c.tokens[ idx ].Text.Len, c.tokens[ idx ].Text.Ptr
);
}
StrBuilder context_str = strbuilder_fmt_buf( _ctx->Allocator_Temp, "%.*s", min( 100, c.left ), c.scanner );
log_failure( "Failed to lex token '%c' (%d, %d)\n%s", (* ctx->scanner), c.line, c.column, context_str );
// Skip to next whitespace since we can't know if anything else is valid until then.
while ( c.left && ! char_is_space( (* ctx->scanner) ) ) {
move_forward();
}
StrBuilder context_str = strbuilder_fmt_buf( lib_ctx->Allocator_Temp, "%.*s", min( 100, c.left ), c.scanner );
lexer_record_failure_fmt(lib_ctx, ctx, "Failed to lex token '%c' (%d, %d)\n%s", (* ctx->scanner), c.line, c.column, context_str );
info.messages = c.messages;
goto Lex_Done;
}
FoundToken:
{
lex_found_token( ctx );
TokType last_type = array_back(_ctx->Lexer_Tokens)->Type;
TokType last_type = array_back(c.tokens)->Type;
if ( last_type == Tok_Preprocess_Macro_Stmt || last_type == Tok_Preprocess_Macro_Expr )
{
Token thanks_c = { { c.scanner, 0 }, Tok_Invalid, c.line, c.column, TF_Null };
@@ -1281,21 +1308,24 @@ TokArray lex( Str content )
c.token.Text.Len++;
move_forward();
array_append( _ctx->Lexer_Tokens, c.token );
array_append(c.tokens, c.token);
continue;
}
}
}
}
if ( array_num(_ctx->Lexer_Tokens) == 0 ) {
log_failure( "Failed to lex any tokens" );
TokArray tok_array = {};
return tok_array;
if ( array_num(c.tokens) == 0 ) {
lexer_record_failure(lib_ctx, ctx, txt("gen::lex: failed to lex any tokens"));
info.messages = c.messages;
goto Lex_Done;
}
TokArray result = { _ctx->Lexer_Tokens, 0 };
return result;
info.messages = c.messages;
info.text = content;
Lex_Done:
info.tokens = struct_init(TokenSlice) { pcast(Token*, c.tokens), scast(s32, array_num(c.tokens)) };
return info;
}
#undef move_forward
+1361 -779
View File
File diff suppressed because it is too large. Load diff
+73 -20
View File
@@ -1,7 +1,7 @@
#ifdef INTELLISENSE_DIRECTIVES
#pragma once
#include "types.hpp"
#include "gen/ecode.hpp"
#include "gen/ecodetypes.hpp"
#include "gen/eoperator.hpp"
#include "gen/especifier.hpp"
#include "gen/etoktype.hpp"
@@ -37,8 +37,6 @@ struct Token
u32 Flags;
};
constexpr Token NullToken { {}, Tok_Invalid, 0, 0, TF_Null };
forceinline
AccessSpec tok_to_access_specifier(Token tok) {
return scast(AccessSpec, tok.Type);
@@ -89,7 +87,28 @@ bool tok_is_end_definition(Token tok) {
return bitfield_is_set( u32, tok.Flags, TF_EndDefinition );
}
StrBuilder tok_to_strbuilder(Token tok);
StrBuilder tok_to_strbuilder(AllocatorInfo ainfo, Token tok);
struct TokenSlice
{
Token* ptr;
s32 num;
#if GEN_COMPILER_CPP
forceinline operator Token* () const { return ptr; }
forceinline Token& operator[]( ssize index ) const { return ptr[index]; }
#endif
};
forceinline
Str token_range_to_str(Token start, Token end)
{
Str result = {
start.Text.Ptr,
(scast(sptr, rcast(uptr, end.Text.Ptr)) + end.Text.Len) - scast(sptr, rcast(uptr, start.Text.Ptr))
};
return result;
}
struct TokArray
{
@@ -97,30 +116,54 @@ struct TokArray
s32 Idx;
};
typedef struct LexerMessage LexerMessage;
struct LexerMessage
{
LexerMessage* next;
Str content;
LogLevel level;
};
struct LexContext
{
AllocatorInfo allocator_temp;
LexerMessage* messages;
Str content;
s32 left;
char const* scanner;
s32 line;
s32 column;
// StringTable defines;
Token token;
Array(Token) tokens;
};
struct StackNode
struct LexedInfo
{
StackNode* Prev;
LexerMessage* messages;
Str text;
TokenSlice tokens;
};
Token* Start;
Str Name; // The name of the AST node (if parsed)
Str ProcName; // The name of the procedure
typedef struct ParseStackNode ParseStackNode;
typedef struct ParseMessage ParseMessage;
struct ParseMessage
{
ParseMessage* Next;
ParseStackNode* Scope;
Str Content;
LogLevel Level;
};
struct ParseContext
{
TokArray Tokens;
StackNode* Scope;
ParseMessage* messages;
ParseStackNode* scope;
// TokArray Tokens;
TokenSlice tokens;
s32 token_id;
LexerMessage* lexed_messages;
b32 token_mismatch;
};
enum MacroType : u16
@@ -168,26 +211,36 @@ Str macrotype_to_str( MacroType type )
enum EMacroFlags : u16
{
MF_Functional = bit(0), // Macro has parameters (args expected to be passed)
MF_Expects_Body = bit(1), // Expects to assign a braced scope to its body.
// Macro has parameters (args expected to be passed)
MF_Functional = bit(0),
// Expects to assign a braced scope to its body.
MF_Expects_Body = bit(1),
// lex__eat wil treat this macro as an identifier if the parser attempts to consume it as one.
// ^^^ This is a kludge because we don't support push/pop macro pragmas rn.
// This is a kludge because we don't support push/pop macro pragmas rn.
MF_Allow_As_Identifier = bit(2),
// When parsing identifiers, it will allow the consumption of the macro parameters (as its expected to be a part of constructing the identifier)
// Example of a decarator macro from stb_sprintf.h:
// STBSP__PUBLICDEC int STB_SPRINTF_DECORATE(sprintf)(char* buf, char const *fmt, ...) STBSP__ATTRIBUTE_FORMAT(2,3);
// ^^ STB_SPRINTF_DECORATE is decorating sprintf
MF_Identifier_Decorator = bit(3),
// lex__eat wil treat this macro as an attribute if the parser attempts to consume it as one.
// ^^^ This a kludge because unreal has a macro that behaves as both a 'statement' and an attribute (UE_DEPRECATED, PRAGMA_ENABLE_DEPRECATION_WARNINGS, etc)
// This a kludge because unreal has a macro that behaves as both a 'statement' and an attribute (UE_DEPRECATED, PRAGMA_ENABLE_DEPRECATION_WARNINGS, etc)
// TODO(Ed): We can keep the MF_Allow_As_Attribute flag for macros, however, we need to add the ability of AST_Attributes to chain themselves.
// Its thats already a thing in the standard language anyway
// & it would allow UE_DEPRECATED, (UE_PROPERTY / UE_FUNCTION) to chain themselves as attributes of a resolved member function/variable definition
MF_Allow_As_Attribute = bit(3),
MF_Allow_As_Attribute = bit(4),
// When a macro is encountered after attributes and specifiers while parsing a function, or variable:
// It will consume the macro and treat it as resolving the definition. (Yes this is for Unreal Engine)
// It will consume the macro and treat it as resolving the definition.
// (MUST BE OF MT_Statement TYPE)
MF_Allow_As_Definition = bit(4),
MF_Allow_As_Definition = bit(5),
MF_Allow_As_Specifier = bit(5), // Created for Unreal's PURE_VIRTUAL
// Created for Unreal's PURE_VIRTUAL
MF_Allow_As_Specifier = bit(6),
MF_Null = 0,
MF_UnderlyingType = GEN_U16_MAX,
+3 -1
View File
@@ -1,6 +1,6 @@
#ifdef INTELLISENSE_DIRECTIVES
#pragma once
#include "../gen.hpp"
#include "interface.hpp"
#endif
#pragma region StaticData
@@ -8,6 +8,8 @@ GEN_API global Context* _ctx;
GEN_API global u32 context_counter;
#pragma region Constants
GEN_API read_only global Token NullToken = { { 0, 0 }, Tok_Invalid, 0, 0, TF_Null };
GEN_API global Macro enum_underlying_macro;
GEN_API global Code Code_Global;
+47 -4
View File
@@ -1,6 +1,18 @@
#ifdef INTELLISENSE_DIRECTIVES
#pragma once
#include "header_start.hpp"
#include "dependencies/platform.hpp"
#include "dependencies/macros.hpp"
#include "dependencies/basic_types.hpp"
#include "dependencies/debug.hpp"
#include "dependencies/memory.hpp"
#include "dependencies/string_ops.hpp"
#include "dependencies/printing.hpp"
#include "dependencies/containers.hpp"
#include "dependencies/hashing.hpp"
#include "dependencies/strings.hpp"
#include "dependencies/filesystem.hpp"
#include "dependencies/timing.hpp"
#include "dependencies/parsing.hpp"
#endif
/*
@@ -19,14 +31,45 @@
*/
using LogFailType = ssize(*)(char const*, ...);
enum LogLevel //: u32
{
LL_Null,
LL_Note,
LL_Warning,
LL_Error,
LL_Fatal,
LL_UnderlyingType = GEN_U32_MAX,
};
typedef enum LogLevel LogLevel;
Str loglevel_to_str(LogLevel level)
{
local_persist
Str lookup[] = {
{ "Null", sizeof("Null") - 1 },
{ "Note", sizeof("Note") - 1 },
{ "Warning", sizeof("Info") - 1 },
{ "Error", sizeof("Error") - 1 },
{ "Fatal", sizeof("Fatal") - 1 },
};
return lookup[level];
}
typedef struct LogEntry LogEntry;
struct LogEntry
{
Str msg;
LogLevel level;
};
typedef void LoggerProc(LogEntry entry);
// By default this library will either crash or exit if an error is detected while generating codes.
// Even if set to not use GEN_FATAL, GEN_FATAL will still be used for memory failures as the library is unusable when they occur.
#ifdef GEN_DONT_USE_FATAL
#define log_failure log_fmt
#define log_failure( ... ) log_fmt( __VA_ARGS__ )
#else
#define log_failure GEN_FATAL
#define log_failure( ... ) GEN_FATAL( __VA_ARGS__ )
#endif
enum AccessSpec : u32
+38 -7
View File
@@ -17,6 +17,39 @@ template <class TType> struct RemovePtr<TType*> { typedef TType Type; };
template <class TType> using TRemovePtr = typename RemovePtr<TType>::Type;
#pragma region Slice
#if 0
#define Slice(Type) Slice<Type>
template<class Type> struct Slice;
template<class Type>
Type* slice_get(Slice<Type> self, ssize id) {
GEN_ASSERT(id > -1);
GEN_ASSERT(id < self.len);
return self.ptr[id];
}
template<class Type>
struct Slice
{
Type* ptr;
ssize len;
#if GEN_COMPILER_CPP
forceinline operator Token* () const { return ptr; }
forceinline Token& operator[]( ssize index ) const { return ptr[index]; }
forceinline Type* begin() { return ptr; }
forceinline Type* end() { return ptr + len; }
#endif
#if ! GEN_C_LIKE_CPP && GEN_COMPILER_CPP
forceinline Type& back() { return ptr[len - 1]; }
#endif
};
#endif
#pragma endregion Slice
#pragma region Array
#define Array(Type) Array<Type>
@@ -26,10 +59,8 @@ template <class TType> using TRemovePtr = typename RemovePtr<TType>::Type;
struct ArrayHeader;
#if GEN_COMPILER_CPP
template<class Type> struct Array;
# define get_array_underlying_type(array) typename TRemovePtr<typeof(array)>:: DataType
#endif
template<class Type> struct Array;
#define get_array_underlying_type(array) typename TRemovePtr<typeof(array)>:: DataType
usize array_grow_formula(ssize value);
@@ -59,12 +90,12 @@ struct ArrayHeader {
usize Num;
};
#if GEN_COMPILER_CPP
template<class Type>
struct Array
{
Type* Data;
#if ! GEN_C_LIKE_CPP
#pragma region Member Mapping
forceinline static Array init(AllocatorInfo allocator) { return array_init<Type>(allocator); }
forceinline static Array init_reserve(AllocatorInfo allocator, ssize capacity) { return array_init_reserve<Type>(allocator, capacity); }
@@ -88,6 +119,7 @@ struct Array
forceinline bool resize(usize num) { return array_resize<Type>(this, num); }
forceinline bool set_capacity(usize new_capacity) { return array_set_capacity<Type>(this, new_capacity); }
#pragma endregion Member Mapping
#endif
forceinline operator Type*() { return Data; }
forceinline operator Type const*() const { return Data; }
@@ -99,9 +131,8 @@ struct Array
using DataType = Type;
};
#endif
#if GEN_COMPILER_CPP && 0
#if 0
template<class Type> bool append(Array<Type>& array, Array<Type> other) { return append( & array, other ); }
template<class Type> bool append(Array<Type>& array, Type value) { return append( & array, value ); }
template<class Type> bool append(Array<Type>& array, Type* items, usize item_num) { return append( & array, items, item_num ); }
-3
View File
@@ -1,9 +1,6 @@
#ifdef INTELLISENSE_DIRECTIVES
# pragma once
# include "dependencies/platform.hpp"
# include "dependencies/macros.hpp"
# include "basic_types.hpp"
# include "macros.hpp"
#endif
#pragma region Debug
+1 -1
View File
@@ -187,7 +187,7 @@ struct FileContents
{
AllocatorInfo allocator;
void* data;
ssize size;
ssize size;
};
constexpr b32 file_zero_terminate = true;
+95 -12
View File
@@ -38,6 +38,16 @@
#ifndef local_persist
#define local_persist static // Local Persisting variables
#endif
#ifndef read_only
# if GEN_COMPILER_MSVC || ( defined(GEN_COMPILER_CLANG) && GEN_COMPILER_CLANG && GEN_SYSTEM_WINDOWS )
# pragma section(".rdata$", read)
# define read_only __declspec(allocate(".rdata$"))
# elif defined(GEN_COMPILER_CLANG) && GEN_COMPILER_CLANG
# define read_only __attribute__((section(".rodata")))
# else
# define read_only
# endif
#endif
#ifndef bit
#define bit( Value ) ( 1 << Value )
@@ -47,6 +57,11 @@
#define bitfield_is_set( Type, Field, Mask ) ( (scast(Type, Mask) & scast(Type, Field)) == scast(Type, Mask) )
#endif
#ifndef GEN_ASSERT_SLICE
#define GEN_ASSERT_SLICE( slice ) \
GEN_ASSERT( ((slice).Len >= 0) & (((slice).Len == 0) | ((slice).Ptr != nullptr)) )
#endif
// Mainly intended for forcing the base library to utilize only C-valid constructs or type coercion
#ifndef GEN_C_LIKE_CPP
#define GEN_C_LIKE_CPP 0
@@ -98,9 +113,9 @@
#define src_line_str stringize(__LINE__)
#ifndef do_once
#define do_once() \
local_persist int __do_once_counter_##src_line_str = 0; \
for(; __do_once_counter_##src_line_str != 1; __do_once_counter_##src_line_str = 1 ) \
#define do_once() \
local_persist int __do_once_counter_##src_line_str = 0; \
for(; __do_once_counter_##src_line_str != 1; __do_once_counter_##src_line_str = 1 ) \
#define do_once_defer( expression ) \
local_persist int __do_once_counter_##src_line_str = 0; \
@@ -198,21 +213,16 @@
#ifndef forceinline
# if GEN_COMPILER_MSVC
# define forceinline __forceinline
# define neverinline __declspec( noinline )
# elif GEN_COMPILER_GCC
# define forceinline inline __attribute__((__always_inline__))
# define neverinline __attribute__( ( __noinline__ ) )
# elif GEN_COMPILER_CLANG
# if __has_attribute(__always_inline__)
# define forceinline inline __attribute__((__always_inline__))
# define neverinline __attribute__( ( __noinline__ ) )
# else
# define forceinline
# define neverinline
# endif
# else
# define forceinline
# define neverinline
# endif
#endif
@@ -303,10 +313,28 @@
# define GEN_PARAM_DEFAULT
#endif
#if GEN_COMPILER_CPP
#define struct_init(type, value) {value}
#else
#define struct_init(type, value) {value}
#ifndef struct_init
# if GEN_COMPILER_CPP
# define struct_init(type)
# else
# define struct_init(type) (type)
# endif
#endif
#ifndef struct_zero
# if GEN_COMPILER_CPP
# define struct_zero(type) {}
# else
# define struct_zero(type) (type) {0}
# endif
#endif
#ifndef struct_zero_init
# if GEN_COMPILER_CPP
# define struct_zero_init() {}
# else
# define struct_zero_init() {0}
# endif
#endif
#if 0
@@ -319,4 +347,59 @@
# define GEN_OPITMIZE_MAPPINGS_END
#endif
#ifndef get_optional
# if GEN_COMPILER_C
# define get_optional(opt) opt ? *opt : (typeof(*opt)){0}
# else
# define get_optional(opt) opt
# endif
#endif
#ifndef each_iter
#define each_iter( type, iter, end ) \
for ( type iter = 0; iter < (end); ++ iter )
#endif
#ifndef index_iter
#define index_iter( type, iter, begin, op, end ) \
for ( type iter = (begin); iter op (end); ( (begin) < (end) ? ++ iter : -- iter ) )
#endif
#ifndef range_iter
#define range_iter( iter, op, range ) \
for ( typeof( (range).p0 ) iter = (range).p0; iter op (range).p1; ( (range).p0 < (range).p1 ? ++ iter : -- iter ) )
#endif
#ifndef defer
#define defer( expr ) \
for ( u32 once = 0; once != 1; ++ once, (expr) )
#endif
#ifndef defer_rewind
#define defer_rewind( cursor ) \
for ( struct { typeof( cursor ) sp; u32 once; } defer_state = { (cursor), 0 }; \
defer_state.once != 1; \
++ defer_state.once, (cursor) = defer_state.sp )
#endif
#ifndef defer_info
#define defer_info( type, expr, ... ) \
for ( type info = { __VA_ARGS__ }; info.once != 1; ++ info.once, (expr) )
#endif
#ifndef scope
#define scope( begin, end ) \
for ( u32 once = ( (begin), 0 ); once != 1; ++ once, (end) )
#endif
#ifndef scope_info
#define scope_info( type, begin, end ) \
for ( type info = (begin); info.once != 1; ++ info.once, (end) )
#endif
#ifndef do_while
#define do_while( cond ) \
for ( u32 once = 0; once != 1 || (cond); once = 1 )
#endif
#pragma endregion Macros
+2 -6
View File
@@ -134,12 +134,6 @@ GEN_API void* heap_allocator_proc( void* allocator_data, AllocType type, ssize s
//! The heap allocator backed by operating system's memory manager.
constexpr AllocatorInfo heap( void ) { AllocatorInfo allocator = { heap_allocator_proc, nullptr }; return allocator; }
//! Helper to allocate memory using heap allocator.
#define malloc( sz ) alloc( heap(), sz )
//! Helper to free memory allocated by heap allocator.
#define mfree( ptr ) free( heap(), ptr )
struct VirtualMemory
{
void* data;
@@ -185,6 +179,8 @@ void arena_check (Arena* arena);
void arena_free (Arena* arena);
ssize arena_size_remaining(Arena* arena, ssize alignment);
// TODO(Ed): Add arena_pos, arena_pop, and arena_pop_to
struct Arena
{
AllocatorInfo Backing;
+1 -1
View File
@@ -1,6 +1,6 @@
#ifdef INTELLISENSE_DIRECTIVES
# pragma once
# include "strbuilder_ops.cpp"
# include "string_ops.cpp"
#endif
#pragma region Printing
+1 -14
View File
@@ -1,6 +1,6 @@
#ifdef INTELLISENSE_DIRECTIVES
# pragma once
# include "strbuilder_ops.hpp"
# include "string_ops.hpp"
#endif
#pragma region Printing
@@ -26,17 +26,4 @@ GEN_API ssize c_str_fmt_file_va ( FileInfo* f, char const* fmt, va_list va );
constexpr
char const* Msg_Invalid_Value = "INVALID VALUE PROVIDED";
inline
ssize log_fmt(char const* fmt, ...)
{
ssize res;
va_list va;
va_start(va, fmt);
res = c_str_fmt_out_va(fmt, va);
va_end(va);
return res;
}
#pragma endregion Printing
+63
View File
@@ -0,0 +1,63 @@
#ifdef INTELLISENSE_DIRECTIVES
#pragma once
#include "debug.hpp"
#endif
#pragma region SIMD
// Lane names use bit widths: sixteen unsigned 8-bit lanes or four unsigned 32-bit lanes.
// Vector operations are available on Windows x64.
#if defined(GEN_SYSTEM_WINDOWS) && (defined(_M_X64) || defined(__x86_64__))
typedef __m128i u8x16;
typedef __m128i u32x4;
// Unaligned loads require 16 or 8 readable bytes respectively. load8 zeros lanes 8 through 15.
internal forceinline u8x16 u8x16_load (void const* address) { return _mm_loadu_si128(rcast(__m128i const*, address)); }
internal forceinline u8x16 u8x16_load8(void const* address) { return _mm_loadl_epi64(rcast(__m128i const*, address)); }
internal forceinline u8x16 u8x16_splat(u8 value) { return _mm_set1_epi8(scast(char, value)); }
internal forceinline u8x16 u8x16_equal(u8x16 lhs, u8x16 rhs) { return _mm_cmpeq_epi8(lhs, rhs); }
// Extract each lane's high bit. Bit 0 represents lane 0 at the lowest address.
internal forceinline u16 u8x16_mask(u8x16 value) { return scast(u16, _mm_movemask_epi8(value)); }
// Reads 16 bytes without requiring alignment.
internal forceinline u32x4 u32x4_load(void const* address) { return _mm_loadu_si128(rcast(__m128i const*, address)); }
internal forceinline u32x4 u32x4_splat(u32 value) { return _mm_set1_epi32(scast(s32, value)); }
internal forceinline u32x4 u32x4_equal(u32x4 lhs, u32x4 rhs) { return _mm_cmpeq_epi32(lhs, rhs); }
// Extract each lane's high bit into bits 0 through 3. Higher result bits are zero.
internal forceinline u8 u32x4_mask(u32x4 value) { return scast(u8, _mm_movemask_ps(_mm_castsi128_ps(value))); }
#endif
// Requires a nonzero mask. Returns the lowest set-bit index, from 0 through 31.
s32 u32_mask_first_set(u32 mask);
#if defined(GEN_SYSTEM_WINDOWS)
internal forceinline
s32 u32_mask_first_set(u32 mask) {
GEN_ASSERT(mask != 0);
unsigned long bit_index = 0;
_BitScanForward(& bit_index, scast(unsigned long, mask));
return scast(s32, bit_index);
}
#endif
#if ! defined(GEN_SYSTEM_WINDOWS)
internal forceinline
s32 u32_mask_first_set(u32 mask) {
GEN_ASSERT(mask != 0);
s32 bit = 0;
u32 bits = mask;
while ((bits & 1u) == 0u) {
bits >>= 1;
bit += 1;
}
return bit;
}
#endif
#pragma endregion SIMD
+150
View File
@@ -1,5 +1,6 @@
#ifdef INTELLISENSE_DIRECTIVES
# pragma once
# include "simd.hpp"
# include "hashing.cpp"
#endif
@@ -174,4 +175,153 @@ StrBuilder strbuilder_visualize_whitespace(StrBuilder const str)
return result;
}
#if defined(GEN_SYSTEM_WINDOWS) && (defined(_M_X64) || defined(__x86_64__))
internal
ssize strbuilder_find_template_byte(char const* text, ssize length, char needle)
{
ssize idx = 0;
for (; (length - idx) >= 16; )
{
char const* at = text + idx;
u8x16 chunk = u8x16_load(at);
u8x16 splat = u8x16_splat(scast(u8, needle));
u16 mask = u8x16_mask(u8x16_equal(chunk, splat));
if (mask != 0) {
s32 bit_index = u32_mask_first_set(mask);
return idx + scast(ssize, bit_index);
}
idx += 16;
}
for (; (length - idx) >= 8; )
{
char const* at = text + idx;
u8x16 chunk = u8x16_load8(at);
u8x16 splat = u8x16_splat(scast(u8, needle));
u16 mask = u8x16_mask(u8x16_equal(chunk, splat)) & 0xFF;
if (mask != 0) {
s32 bit_index = u32_mask_first_set(mask);
return idx + scast(ssize, bit_index);
}
idx += 8;
}
for (; idx < length; ++ idx) {
if (text[idx] == needle) return idx;
}
return length;
}
#else
internal
ssize strbuilder_find_template_byte(char const* text, ssize length, char needle) {
for (ssize idx = 0; idx < length; ++ idx)
{
if (text[idx] == needle)
return idx;
}
return length;
}
#endif
#if defined(GEN_SYSTEM_WINDOWS) && (defined(_M_X64) || defined(__x86_64__))
internal
StrTemplateSlot* strbuilder_find_template_slot(Slice_StrTemplateSlot slots, Str key)
{
u32 key_hash = crc32(key.Ptr, key.Len);
ssize idx = 0;
for (; (slots.Len - idx) >= 4; idx += 4)
{
u32 packed[4];
packed[0] = crc32(slots.Ptr[idx + 0].Key.Ptr, slots.Ptr[idx + 0].Key.Len);
packed[1] = crc32(slots.Ptr[idx + 1].Key.Ptr, slots.Ptr[idx + 1].Key.Len);
packed[2] = crc32(slots.Ptr[idx + 2].Key.Ptr, slots.Ptr[idx + 2].Key.Len);
packed[3] = crc32(slots.Ptr[idx + 3].Key.Ptr, slots.Ptr[idx + 3].Key.Len);
u32x4 packed_vec = u32x4_load(packed);
u32x4 want_vec = u32x4_splat(key_hash);
u8 mask = u32x4_mask(u32x4_equal(packed_vec, want_vec));
for (s32 lane = 0; lane < 4; ++ lane)
{
if ((mask & (1u << lane)) != 0) {
StrTemplateSlot* slot = & slots.Ptr[idx + lane];
if (str_are_equal(slot[0].Key, key)) return slot;
}
}
}
for (; idx < slots.Len; ++ idx)
{
StrTemplateSlot* slot = & slots.Ptr[idx];
u32 slot_hash = crc32(slot[0].Key.Ptr, slot[0].Key.Len);
if (slot_hash == key_hash) {
if (str_are_equal(slot[0].Key, key)) return slot;
}
}
return nullptr;
}
#else
internal
StrTemplateSlot* strbuilder_find_template_slot(Slice_StrTemplateSlot slots, Str key) {
u32 key_hash = crc32(key.Ptr, key.Len);
for (ssize idx = 0; idx < slots.Len; ++ idx)
{
StrTemplateSlot* slot = & slots.Ptr[idx];
u32 slot_hash = crc32(slot[0].Key.Ptr, slot[0].Key.Len);
if (slot_hash == key_hash && str_are_equal(slot[0].Key, key))
return slot;
}
return nullptr;
}
#endif
bool strbuilder_append_template(StrBuilder* output, Slice_StrTemplateSlot slots, Str format)
{
GEN_ASSERT(output != nullptr && output[0] != nullptr);
StrBuilderHeader* header = strbuilder_get_header(output[0]);
GEN_ASSERT((header[0].Length >= 0) & (header[0].Capacity >= header[0].Length));
GEN_ASSERT_SLICE(format);
GEN_ASSERT_SLICE(slots);
for (ssize slot_idx = 0; slot_idx < slots.Len; ++ slot_idx) {
GEN_ASSERT_SLICE(slots.Ptr[slot_idx].Key);
GEN_ASSERT_SLICE(slots.Ptr[slot_idx].Value);
}
bool result = false;
if (format.Len == 0) {
result = true;
goto finish;
}
for (ssize cursor = 0; cursor < format.Len;)
{
if (format.Ptr[cursor] == '<')
{
ssize key_at = cursor + 1;
ssize key_max = format.Len - key_at;
ssize key_len = strbuilder_find_template_byte(format.Ptr + key_at, key_max, '>');
if (key_len == key_max) goto finish;
Str key = { format.Ptr + key_at, key_len };
StrTemplateSlot* slot = strbuilder_find_template_slot(slots, key);
if (slot == nullptr) goto finish;
if (slot[0].Value.Len > 0) {
bool appended = strbuilder_append_c_str_len(output, slot[0].Value.Ptr, slot[0].Value.Len);
if (appended == false) goto finish;
}
cursor = key_at + key_len + 1;
continue;
}
ssize literal_max = format.Len - cursor;
ssize literal_len = strbuilder_find_template_byte(format.Ptr + cursor, literal_max, '<');
if (literal_len > 0) {
bool appended = strbuilder_append_c_str_len(output, format.Ptr + cursor, literal_len);
if (appended == false) goto finish;
}
cursor += literal_len;
}
result = true;
finish:
return result;
}
#pragma endregion StrBuilder
+36 -7
View File
@@ -113,6 +113,8 @@ Str to_str_from_c_str( char const* bad_str ) {
// I kept it for simplicty of porting but its not necessary to keep it that way.
#pragma region StrBuilder
struct StrBuilderHeader;
struct StrTemplateSlot;
struct Slice_StrTemplateSlot;
#if GEN_COMPILER_C
typedef char* StrBuilder;
@@ -122,12 +124,39 @@ struct StrBuilder;
forceinline usize strbuilder_grow_formula(usize value);
GEN_API StrBuilder strbuilder_make_reserve (AllocatorInfo allocator, ssize capacity);
GEN_API StrBuilder strbuilder_make_length (AllocatorInfo allocator, char const* str, ssize length);
GEN_API bool strbuilder_make_space_for (StrBuilder* str, char const* to_append, ssize add_len);
GEN_API bool strbuilder_append_c_str_len (StrBuilder* str, char const* c_str_to_append, ssize length);
GEN_API void strbuilder_trim (StrBuilder str, char const* cut_set);
GEN_API StrBuilder strbuilder_visualize_whitespace(StrBuilder const str);
GEN_API StrBuilder strbuilder_make_reserve (AllocatorInfo allocator, ssize capacity);
GEN_API StrBuilder strbuilder_make_length (AllocatorInfo allocator, char const* str, ssize length);
GEN_API bool strbuilder_make_space_for (StrBuilder* str, char const* to_append, ssize add_len);
GEN_API bool strbuilder_append_c_str_len(StrBuilder* str, char const* c_str_to_append, ssize length);
struct StrTemplateSlot { Str Key; Str Value; };
struct Slice_StrTemplateSlot { StrTemplateSlot* Ptr; ssize Len; };
// format and slot slices must not alias output. output may reallocate.
GEN_API bool strbuilder_append_template(StrBuilder* output, Slice_StrTemplateSlot slots, Str format);
#ifndef strtmpl_slot
# if GEN_COMPILER_C
# define strtmpl_slot(key, value) (GEN_NS StrTemplateSlot){ txt(key), (value) }
# else
# define strtmpl_slot(key, value) GEN_NS StrTemplateSlot{ txt(key), (value) }
# endif
#endif
#ifndef strtmpl_slots
# define strtmpl_slots( ... ) __VA_ARGS__
#endif
#ifndef strtmpl_append
#define strtmpl_append( output, result, entries, format ) do { \
GEN_NS StrTemplateSlot gencpp_tmpl_slot_array[] = { entries }; \
GEN_NS Slice_StrTemplateSlot gencpp_tmpl_slot_slice = { gencpp_tmpl_slot_array, count_of(gencpp_tmpl_slot_array) }; \
(result) = GEN_NS strbuilder_append_template((output), gencpp_tmpl_slot_slice, (format)); \
} while ( 0 )
#endif
GEN_API void strbuilder_trim (StrBuilder str, char const* cut_set);
GEN_API StrBuilder strbuilder_visualize_whitespace(StrBuilder const str);
StrBuilder strbuilder_make_c_str (AllocatorInfo allocator, char const* str);
StrBuilder strbuilder_make_str (AllocatorInfo allocator, Str str);
@@ -320,7 +349,7 @@ inline
StrBuilder strbuilder_fmt_buf(AllocatorInfo allocator, char const* fmt, ...)
{
local_persist thread_local
PrintF_Buffer buf = struct_init(PrintF_Buffer, {0});
PrintF_Buffer buf = struct_zero_init();
va_list va;
va_start(va, fmt);
+1
View File
@@ -13,6 +13,7 @@ GEN_NS_BEGIN
#include "dependencies/printing.hpp"
#include "dependencies/containers.hpp"
#include "dependencies/hashing.hpp"
#include "dependencies/simd.hpp"
#include "dependencies/strings.hpp"
#include "dependencies/filesystem.hpp"
#include "dependencies/timing.hpp"
-3
View File
@@ -11,9 +11,6 @@
#include "helpers/push_ignores.inline.hpp"
#include "components/header_start.hpp"
// Has container defines pushed
#include "gen.dep.hpp"
GEN_NS_BEGIN
#include "components/types.hpp"
+15 -3
View File
@@ -37,7 +37,11 @@ void clang_format_file( char const* path, char const* style_path )
strbuilder_append_str( & command, cf_verbose );
strbuilder_append_string( & command, style_arg );
strbuilder_append_string( & command, resolved_path );
system( command );
int command_result = system( command );
if ( command_result != 0 ) {
log_fmt( "clang-format failed with status %d\n", command_result );
process_exit( 1 );
}
}
// Will refactor a file with the given script at the provided path.
@@ -48,12 +52,20 @@ void refactor_file( char const* path, char const* refactor_script )
GEN_ASSERT_NOT_NULL(path);
GEN_ASSERT_NOT_NULL(refactor_script);
StrBuilder command = strbuilder_make_str(_ctx->Allocator_Temp, txt("refactor "));
Str refactor_exe = GEN_SYSTEM_WINDOWS ?
txt("\"..\\scripts\\helpers\\refactor.exe\" ")
: txt("../scripts/helpers/refactor ");
StrBuilder command = strbuilder_make_str(_ctx->Allocator_Temp, refactor_exe);
// strbuilder_append_str( & command, txt("-debug ") );
strbuilder_append_str( & command, txt("-num=1 ") );
strbuilder_append_fmt( & command, "-src=%s ", path );
strbuilder_append_fmt( & command,"-spec=%s ", refactor_script );
system(command);
int command_result = system( command );
if ( command_result != 0 ) {
log_fmt( "refactor failed with status %d\n", command_result );
process_exit( 1 );
}
log_fmt("\n");
}
+4
View File
@@ -141,6 +141,7 @@
#undef GEN_ASSERT
#undef GEN_ASSERT_MSG
#undef GEN_ASSERT_NOT_NULL
#undef GEN_ASSERT_SLICE
#undef GEN_PANIC
#undef GEN_FATAL
@@ -165,6 +166,9 @@
#undef get_hashtable_underlying_type
#undef txt
#undef strtmpl_slot
#undef strtmpl_slots
#undef strtmpl_append
#undef NOMINMAX
#undef VC_EXTRALEAN
+3 -3
View File
@@ -15,6 +15,9 @@ You can think of this parser as *frontend parser* vs a *semantic parser*. Its in
User exposed interface:
```cpp
CodeBody parse_global_body ( Str body_def, Opts_parse_global_body opts );
void parse_global_body_base( Context* ctx, Str body_def, ParseInfo* info );
CodeClass parse_class ( Str class_def );
CodeConstructor parse_constructor ( Str constructor_def );
CodeDefine parse_define ( Str define_def );
@@ -24,7 +27,6 @@ CodeBody parse_export_body ( Str export_def );
CodeExtern parse_extern_link ( Str exten_link_def );
CodeFriend parse_friend ( Str friend_def );
CodeFn parse_function ( Str fn_def );
CodeBody parse_global_body ( Str body_def );
CodeNS parse_namespace ( Str namespace_def );
CodeOperator parse_operator ( Str operator_def );
CodeOpCast parse_operator_cast( Str operator_def );
@@ -155,5 +157,3 @@ typedef HashTable(Macro) MacroTable;
* typedefs have attributes with the type (`parse_type`)
* Parsing attributes can be extended to support user defined macros by defining `GEN_DEFINE_ATTRIBUTE_TOKENS` (see `gen.hpp` for the formatting)
* This is useful for example: parsing Unreal `Module_API` macros.
**The lexer & parser do not gracefully attempt to continue when it comes across incorrect code, and doesn't properly track errors into a listing (yet).**
+184 -34
View File
@@ -57,6 +57,68 @@ Code refactor_and_format( Code code ) {
return code_refactor_and_format(code, scratch_file, path_refactor_script, path_format_style );
}
internal Code rewrite_parse_wrapper_signature(Code source, Str function_name)
{
Str scanned = source->Content;
Str needle = StrBuilder::fmt_buf(_ctx->Allocator_Temp, "%S(", function_name).to_str();
Str option_param = txt("Opts_parse opts");
ssize function_at = -1;
ssize option_at = -1;
ssize last_offset = scanned.Len - needle.Len;
for (ssize scan_offset = 0; scan_offset <= last_offset; ++ scan_offset) {
char const* scan_at = scanned.Ptr + scan_offset;
if (c_str_compare_len(scan_at, needle.Ptr, needle.Len) == 0) {
if (scan_offset > 0) {
ssize prev_at = scan_offset - 1;
if (scanned.Ptr[prev_at] == '_') continue;
}
function_at = scan_offset;
break;
}
}
if (function_at < 0) GEN_FATAL("c_library: parser wrapper signature missing");
for (ssize params_end = function_at + needle.Len;
params_end < scanned.Len && scanned.Ptr[params_end] != ')';
++ params_end
) {
ssize option_fit_end = params_end + option_param.Len;
if (option_fit_end > scanned.Len) break;
char const* params_at = scanned.Ptr + params_end;
if (c_str_compare_len(params_at, option_param.Ptr, option_param.Len) == 0) {
option_at = params_end;
break;
}
}
if (option_at < 0) GEN_FATAL("c_library: parser wrapper options parameter missing for %S", function_name);
Str new_name = StrBuilder::fmt_buf(_ctx->Allocator_Temp, "%S_opts", function_name).to_str();
Str pointer_param = txt("Opts_parse* opts");
ssize name_end = function_at + function_name.Len;
char const* before_opts_ptr = scanned.Ptr + name_end;
ssize before_opts_len = option_at - name_end;
Str before_name = { scanned.Ptr, function_at };
Str before_opts = { before_opts_ptr, before_opts_len };
ssize after_opts_at = option_at + option_param.Len;
char const* after_opts_ptr = scanned.Ptr + after_opts_at;
ssize after_opts_len = scanned.Len - after_opts_at;
Str after_opts = { after_opts_ptr, after_opts_len };
ssize opts_star_len = 1;
ssize rebuilt_cap = scanned.Len;
rebuilt_cap += new_name.Len;
rebuilt_cap += opts_star_len;
StrBuilder rebuilt = StrBuilder::make_reserve(_ctx->Allocator_Temp, rebuilt_cap);
strbuilder_append_str(& rebuilt, before_name);
strbuilder_append_str(& rebuilt, new_name);
strbuilder_append_str(& rebuilt, before_opts);
strbuilder_append_str(& rebuilt, pointer_param);
strbuilder_append_str(& rebuilt, after_opts);
return untyped_str(strbuilder_to_str(rebuilt));
}
constexpr bool helper_use_c_definition = true;
int gen_main()
@@ -79,6 +141,15 @@ int gen_main()
(Macro { txt("do_once_defer"), MT_Expression, MF_Functional }),
(Macro { txt("do_once_start"), MT_Statement, MF_Null }),
(Macro { txt("do_once_end"), MT_Statement, MF_Null }),
(Macro { txt("each_iter"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("index_iter"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("range_iter"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("defer"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("defer_rewind"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("defer_info"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("scope"), MT_Statement, MF_Functional | MF_Expects_Body | MF_Allow_As_Identifier }),
(Macro { txt("scope_info"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("do_while"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("labeled_scope_start"), MT_Statement, MF_Null }),
(Macro { txt("labeled_scope_end"), MT_Statement, MF_Null }),
(Macro { txt("compiler_decorated_func_name"), MT_Expression, MF_Null }),
@@ -124,12 +195,16 @@ int gen_main()
(Macro { txt("cast_to_str"), MT_Expression, MF_Functional }),
(Macro { txt("current"), MT_Expression, MF_Null }),
(Macro { txt("txt"), MT_Expression, MF_Functional }),
(Macro { txt("strtmpl_slot"), MT_Expression, MF_Functional }),
(Macro { txt("strtmpl_slots"), MT_Expression, MF_Functional }),
(Macro { txt("strtmpl_append"), MT_Statement, MF_Functional }),
(Macro { txt("GEN_FILE_OPEN_PROC"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("GEN_FILE_READ_AT_PROC"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("GEN_FILE_WRITE_AT_PROC"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("GEN_FILE_SEEK_PROC"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("GEN_FILE_CLOSE_PROC"), MT_Statement, MF_Functional | MF_Expects_Body }),
(Macro { txt("log_failure"), MT_Expression, MF_Null }),
(Macro { txt("log_failure"), MT_Statement, MF_Functional }),
(Macro { txt("NullScope"), MT_Expression, MF_Null }),
(Macro { txt("operator"), MT_Expression, MF_Null }),
(Macro { txt("InvalidCode"), MT_Expression, MF_Null }),
(Macro { txt("NullCode"), MT_Expression, MF_Null }),
@@ -137,8 +212,8 @@ int gen_main()
(Macro { txt("gen_main"), MT_Statement, MF_Null })
));
register_macros( args(
(Macro { txt("name"), MT_Expression, MF_Functional }),
(Macro { txt("code"), MT_Expression, MF_Functional }),
(Macro { txt("name"), MT_Expression, MF_Functional | MF_Allow_As_Identifier }),
(Macro { txt("code"), MT_Expression, MF_Functional | MF_Allow_As_Identifier }),
(Macro { txt("args"), MT_Expression, MF_Functional }),
(Macro { txt("code_str"), MT_Expression, MF_Functional }),
(Macro { txt("code_fmt"), MT_Expression, MF_Functional }),
@@ -196,6 +271,7 @@ int gen_main()
Code header_debug = scan_file( path_base "dependencies/debug.hpp" );
Code header_string_ops = scan_file( path_base "dependencies/string_ops.hpp" );
Code header_hashing = scan_file( path_base "dependencies/hashing.hpp" );
Code header_simd = scan_file( path_base "dependencies/simd.hpp" );
Code header_timing = scan_file( path_base "dependencies/timing.hpp" );
CodeBody parsed_header_memory = parse_file( path_base "dependencies/memory.hpp" );
@@ -494,10 +570,10 @@ do \
case CT_Union_Fwd:
{
Str type_str = codetype_to_keyword_str(entry->Type);
Str formated_tmpl = token_fmt_impl( 3,
"type", type_str
, "name", entry->Name,
stringize(
Str formated_tmpl = token_fmt_impl( 3
, "type", type_str
, "name", entry->Name
, stringize(
typedef <type> <name> <name>;
));
CodeTypedef tdef = parse_typedef(formated_tmpl);
@@ -597,10 +673,10 @@ do \
}
Str type_str = codetype_to_keyword_str(entry->Type);
Str formated_tmpl = token_fmt_impl( 3,
"type", type_str
, "name", entry->Name,
stringize(
Str formated_tmpl = token_fmt_impl( 3
, "type", type_str
, "name", entry->Name
, stringize(
typedef <type> <name> <name>;
));
CodeTypedef tdef = parse_typedef(formated_tmpl);
@@ -739,10 +815,10 @@ do \
{
// pop the array entry
array_token->NumEntries -= 1;
Code next = array_entry->Next;
Code prev = array_entry->Prev;
next->Prev = array_entry->Prev;
prev->Next = next;
Code next = array_entry->Next;
Code prev = array_entry->Prev;
next->Prev = array_entry->Prev;
prev->Next = next;
if ( array_token->Front == array_entry )
array_token->Front = next;
@@ -767,20 +843,7 @@ do \
break;
case CT_Variable:
{
CodeVar var = cast(CodeVar, entry);
if (var->Specs && var->Specs.has(Spec_Constexpr)) {
Code define_ver = untyped_str(token_fmt(
"name", var->Name
, "value", var->Value->Content
, "type", var->ValueType.to_strbuilder().to_str()
, "#define <name> (<type>) <value>\n"
));
parser_types.append(define_ver);
continue;
}
parser_types.append(entry);
}
break;
default:
@@ -965,6 +1028,18 @@ R"(#define AST_ArrSpecs_Cap \
txt("CodeUsing"),
txt("CodeVar"),
};
// append, entry, and validate_body. Code is the untyped handle. CodeBody is the list / body pod.
Str code_body_slots[] = {
txt("Code"),
txt("CodeBody"),
};
// has_entries reads NumEntries. These pods keep that word as an entry count.
Str code_has_entries_slots[] = {
txt("Code"),
txt("CodeBody"),
txt("CodeParams"),
txt("CodeDefineParams"),
};
CodeBody parsed_code_types = parse_file( path_base "components/code_types.hpp" );
CodeBody code_types = def_body(CT_Global_Body);
@@ -1037,9 +1112,18 @@ R"(#define <interface_name>( code ) _Generic( (code), \
, tmpl_def_start
));
// Append slots
for(Str type : code_typenames ) {
generic_selector.append_fmt("%S : %S,\\\n", type, fn->Name );
// Append slots. Restricted procedures omit static types that cannot call them.
if (actual_name.is_equal(txt("append")) || actual_name.is_equal(txt("entry")) || actual_name.is_equal(txt("validate_body"))) {
for (Str type : code_body_slots)
generic_selector.append_fmt("%S : %S,\\\n", type, fn->Name);
}
else if (actual_name.is_equal(txt("has_entries"))) {
for (Str type : code_has_entries_slots)
generic_selector.append_fmt("%S : %S,\\\n", type, fn->Name);
}
else {
for (Str type : code_typenames)
generic_selector.append_fmt("%S : %S,\\\n", type, fn->Name);
}
generic_selector.append(txt("default: gen_generic_selection_fail \\\n"));
@@ -1155,10 +1239,36 @@ R"(#define <interface_name>( code ) _Generic( (code), \
for ( CodeParams opt_param : fn->Params ) if (opt_param->ValueType->Name.starts_with(txt("Opts_")))
{
// Convert the definition to use a default struct: https://vxtwitter.com/vkrajacic/status/1749816169736073295
if (fn->Name.starts_with(txt("parse_")))
{
Str new_name = StrBuilder::fmt_buf (_ctx->Allocator_Temp, "%S_opts", fn->Name).to_str();
StrBuilder params_str = StrBuilder::make_reserve(_ctx->Allocator_Temp, 32);
for (CodeParams other_param : fn->Params) {
if (other_param == opt_param) break;
params_str.append_fmt("%S, ", other_param->Name);
}
opt_param->ValueType->Specs = def_specifier( Spec_Ptr );
Code fn_macro = untyped_str( token_fmt(
"def_name", fn->Name
, "def__name", new_name
, "params", params_str.to_str()
, "opts_type", opt_param->ValueType->Name
, "#define <def_name>( <params> ... ) <def__name>( <params> & (<opts_type>) { __VA_ARGS__ } )\n"
));
fn->Name = cache_str( new_name );
interface.append( fn );
interface.append( fn_macro );
handled = true;
break;
}
Str prefix = txt("def_");
Str actual_name = { fn->Name.Ptr + prefix.Len, fn->Name.Len - prefix.Len };
Str new_name = StrBuilder::fmt_buf(_ctx->Allocator_Temp, "def__%S", actual_name ).to_str();
opt_param->ValueType->Specs = def_specifier(Spec_Ptr);
// Resolve define's arguments
b32 has_args = fn->Params->NumEntries > 1;
StrBuilder params_str = StrBuilder::make_reserve(_ctx->Allocator_Temp, 32);
@@ -1172,10 +1282,10 @@ R"(#define <interface_name>( code ) _Generic( (code), \
}
char const* tmpl_fn_macro = nullptr;
if (params_str.length() > 0 ) {
tmpl_fn_macro= "#define <def_name>( <params> ... ) <def__name>( <params> (<opts_type>) { __VA_ARGS__ } )\n";
tmpl_fn_macro= "#define <def_name>( <params> ... ) <def__name>( <params> & (<opts_type>) { __VA_ARGS__ } )\n";
}
else {
tmpl_fn_macro= "#define <def_name>( ... ) <def__name>( (<opts_type>) { __VA_ARGS__ } )\n";
tmpl_fn_macro= "#define <def_name>( ... ) <def__name>( & (<opts_type>) { __VA_ARGS__ } )\n";
}
Code fn_macro = untyped_str(token_fmt(
"def_name", fn->Name
@@ -1337,7 +1447,17 @@ R"(#define <interface_name>( code ) _Generic( (code), \
);
// Append slots
for(Str type : code_typenames ) {
generic_selector.append_fmt("%S : %S,\\\n", type, fn->Name );
bool appended = false;
strtmpl_append(& generic_selector, appended, strtmpl_slots(
strtmpl_slot("type", type),
strtmpl_slot("fn", fn->Name)
),
code(<type> : <fn>,\\\n)
);
if (appended == false) {
GEN_FATAL("builder_print selector template failed");
return 1;
}
}
generic_selector.append(txt("default: gen_generic_selection_fail \\\n"));
generic_selector.append(txt("\t)\tGEN_RESOLVED_FUNCTION_CALL( builder, (Code)code )"));
@@ -1420,6 +1540,32 @@ R"(#define <interface_name>( code ) _Generic( (code), \
Code src_code_serialization = scan_file( path_base "components/code_serialization.cpp" );
Code src_parsing_interface = scan_file( path_base "components/interface.parsing.cpp" );
{
Str parser_names[] = {
txt("parse_class"),
txt("parse_constructor"),
txt("parse_define"),
txt("parse_destructor"),
txt("parse_enum"),
txt("parse_export_body"),
txt("parse_extern_link"),
txt("parse_friend"),
txt("parse_function"),
txt("parse_global_body"),
txt("parse_namespace"),
txt("parse_operator"),
txt("parse_operator_cast"),
txt("parse_struct"),
txt("parse_template"),
txt("parse_type"),
txt("parse_typedef"),
txt("parse_union"),
txt("parse_using"),
txt("parse_variable"),
};
for (s32 idx = 0; idx < (s32)(sizeof(parser_names) / sizeof(parser_names[0])); ++idx)
src_parsing_interface = rewrite_parse_wrapper_signature(src_parsing_interface, parser_names[idx]);
}
Code src_untyped = scan_file( path_base "components/interface.untyped.cpp" );
Code src_parser_case_macros = scan_file( path_base "components/parser_case_macros.cpp" );
@@ -1504,6 +1650,7 @@ R"(#define <interface_name>( code ) _Generic( (code), \
Str actual_name = { fn->Name.Ptr + prefix.Len, fn->Name.Len - prefix.Len };
Str new_name = StrBuilder::fmt_buf(_ctx->Allocator_Temp, "def__%S", actual_name ).to_str();
fn->Name = cache_str(new_name);
opt_param->ValueType->Specs = def_specifier(Spec_Ptr);
}
src_upfront.append(fn);
}
@@ -1658,6 +1805,7 @@ R"(#define <interface_name>( code ) _Generic( (code), \
Code r_header_string_ops = refactor(header_string_ops);
Code rf_containers = refactor_and_format(containers);
Code r_header_hashing = refactor(header_hashing);
Code r_header_simd = refactor(header_simd);
Code rf_header_strings = refactor_and_format(header_strings);
Code rf_header_filesystem = refactor_and_format(header_filesystem);
Code r_header_timing = refactor(header_timing);
@@ -1744,6 +1892,7 @@ R"(#define <interface_name>( code ) _Generic( (code), \
header.print( fmt_newline);
header.print( rf_containers);
header.print( r_header_hashing );
header.print( r_header_simd );
header.print( rf_header_strings);
header.print( rf_header_filesystem);
header.print( r_header_timing );
@@ -1893,6 +2042,7 @@ R"(#define <interface_name>( code ) _Generic( (code), \
header.print( fmt_newline);
header.print( rf_containers);
header.print( r_header_hashing );
header.print( r_header_simd );
header.print( rf_header_strings);
header.print( rf_header_filesystem);
header.print( r_header_timing );
+28 -3
View File
@@ -20,6 +20,7 @@
word global, gen_global
word internal, gen_internal
word local_persist, gen_local_persist
word read_only, gen_read_only
word bit, gen_bit
word bitfield_is_set, gen_bitfield_is_set
word cast, gen_cast
@@ -32,6 +33,15 @@ word stringize, gen_stringize
word do_once, gen_do_once
word do_once_start, gen_do_once_start
word do_once_end, gen_do_once_end
word each_iter, gen_each_iter
word index_iter, gen_index_iter
word range_iter, gen_range_iter
word defer, gen_defer
word defer_rewind, gen_defer_rewind
word defer_info, gen_defer_info
word scope, gen_scope
word scope_info, gen_scope_info
word do_while, gen_do_while
word labeled_scope_start, gen_labeled_scope_start
word labeled_scope_end, gen_labeled_scope_end
word compiler_decorated_func_name, gen_compiler_decorated_func_name
@@ -53,14 +63,17 @@ word enum_underlying, gen_enum_underlying
word nullptr, gen_nullptr
word struct_init, gen_struct_init
word hash, gen_hash
word txt, gen_txt
// Basic Types
word u8, gen_u8
word u8x16, gen_u8x16
word s8, gen_s8
word u16, gen_u16
word s16, gen_s16
word u32, gen_u32
word u32x4, gen_u32x4
word s32, gen_s32
word u64, gen_u64
word s64, gen_s64
@@ -80,6 +93,13 @@ word to_sptr, gen_to_sptr
word to_mem_ptr, gen_to_mem_ptr
word to_mem_ptr_const, gen_to_mem_ptr_const
// SIMD
namespace u8x16_, gen_u8x16_
namespace u32x4_, gen_u32x4_
word u32_mask_first_set, gen_u32_mask_first_set
// Debug
word assert_handler, gen_assert_handler
@@ -219,10 +239,13 @@ namespace str_, gen_str_
word cast_to_str, gen_cast_to_str
word StrBuilderHeader, gen_StrBuilderHeader
word StrBuilder, gen_StrBuilder
word StrBuilderHeader, gen_StrBuilderHeader
word StrBuilder, gen_StrBuilder
word StrTemplateSlot, gen_StrTemplateSlot
word Slice_StrTemplateSlot, gen_Slice_StrTemplateSlot
namespace strbuilder_, gen_strbuilder_
namespace strtmpl_, gen_strtmpl_
word StrCached, gen_StrCached
@@ -410,6 +433,8 @@ namespace var_, gen_var_
word _ctx, gen__ctx
word get_context, gen_get_context
word init, gen_init
word deinit, gen_deinit
word reset, gen_reset
@@ -532,7 +557,7 @@ namespace Lexer_, gen_Lexer_
word LexContext, gen_LexContext
word lex, gen_lex
word StackNode, gen_StackNode
word ParseStackNode, gen_ParseStackNode
word ParseContext, gen_ParseContext
// namespace parse_, gen_parse_
+2
View File
@@ -50,6 +50,7 @@ int gen_main()
Code printing = scan_file( path_base "dependencies/printing.hpp" );
Code containers = scan_file( path_base "dependencies/containers.hpp" );
Code hashing = scan_file( path_base "dependencies/hashing.hpp" );
Code simd = scan_file( path_base "dependencies/simd.hpp" );
Code strings = scan_file( path_base "dependencies/strings.hpp" );
Code filesystem = scan_file( path_base "dependencies/filesystem.hpp" );
Code timing = scan_file( path_base "dependencies/timing.hpp" );
@@ -70,6 +71,7 @@ int gen_main()
builder_print( header, printing );
builder_print( header, containers );
builder_print( header, hashing );
builder_print( header, simd );
builder_print( header, strings );
builder_print( header, filesystem );
builder_print( header, timing );
+2
View File
@@ -81,6 +81,7 @@ int gen_main()
Code printing = scan_file( path_base "dependencies/printing.hpp" );
Code containers = scan_file( path_base "dependencies/containers.hpp" );
Code hashing = scan_file( path_base "dependencies/hashing.hpp" );
Code simd = scan_file( path_base "dependencies/simd.hpp" );
Code strings = scan_file( path_base "dependencies/strings.hpp" );
Code filesystem = scan_file( path_base "dependencies/filesystem.hpp" );
Code timing = scan_file( path_base "dependencies/timing.hpp" );
@@ -97,6 +98,7 @@ int gen_main()
header.print( printing );
header.print( containers );
header.print( hashing );
header.print( simd );
header.print( strings );
header.print( filesystem );
header.print( timing );
+2
View File
@@ -144,6 +144,7 @@ int gen_main()
Code printing = scan_file( path_base "dependencies/printing.hpp" );
Code containers = scan_file( path_base "dependencies/containers.hpp" );
Code hashing = scan_file( path_base "dependencies/hashing.hpp" );
Code simd = scan_file( path_base "dependencies/simd.hpp" );
Code strings = scan_file( path_base "dependencies/strings.hpp" );
Code filesystem = scan_file( path_base "dependencies/filesystem.hpp" );
Code timing = scan_file( path_base "dependencies/timing.hpp" );
@@ -166,6 +167,7 @@ int gen_main()
header.print( printing );
header.print( containers );
header.print( hashing );
header.print( simd );
header.print( strings );
header.print( filesystem );
header.print( timing );
+5
View File
@@ -25,6 +25,11 @@ args:
segmented
singleheader
unreal
test
parser_bounds
lexer_failures
parse_body_messages
dogfood_parse
clang
msvc : By default this project builds with clang, specifying msvc will build with MSVC.
debug
+234 -133
View File
@@ -10,22 +10,76 @@ $vendor_toolchain = Join-Path $PSScriptRoot 'helpers/vendor_toolchain.ps1'
Import-Module $misc
$path_root = Get-ScriptRepoRoot
function stop-stage {
param([string]$stage, [int]$code_exit = 1)
Write-Host "FAILED: $stage" -ForegroundColor Red
exit $code_exit
}
function invoke-stageTool {
param(
[string]$stage,
[bool]$compiled,
[string]$executable,
[string]$path_work
)
if (-not $compiled) {
stop-stage "$stage compile/link"
}
if (-not (Test-Path -LiteralPath $executable)) {
stop-stage "$stage missing output $executable"
}
$code_exit = 1
push-location $path_work
try {
& $executable
$code_exit = $LASTEXITCODE
}
finally {
pop-location
}
if ($code_exit -ne 0) {
stop-stage "$stage run" $code_exit
}
Write-Host "PASSED: $stage"
}
if ($env:GENCPP_SOURCE_ROOT) {
$path_root = (Resolve-Path -LiteralPath $env:GENCPP_SOURCE_ROOT).Path
$probe = Join-Path $path_root "base\base.cpp"
if (-not (Test-Path -LiteralPath $probe)) {
stop-stage "GENCPP_SOURCE_ROOT is not a gencpp tree: $path_root"
}
}
else {
$path_root = Get-ScriptRepoRoot
}
Write-Host "SOURCE_ROOT: $path_root"
if ($null -eq $is_windows) {
$is_windows = [System.Environment]::OSVersion.Platform -eq 'Win32NT'
}
if ($null -eq $is_linux) {
$is_linux = [System.Environment]::OSVersion.Platform -eq 'Unix'
}
Push-Location $path_root
#region Arguments
$vendor = $null
$release = $null
[bool] $verbose = $false
[bool] $base = $false
[bool] $segmented = $false
[bool] $singleheader = $false
[bool] $c_lib = $false
[bool] $c_lib_static = $false
[bool] $c_lib_dyn = $false
[bool] $unreal = $false
[bool] $test = $false
$vendor = $null
$release = $null
[bool] $verbose = $false
[bool] $base = $false
[bool] $segmented = $false
[bool] $singleheader = $false
[bool] $c_lib = $false
[bool] $c_lib_static = $false
[bool] $c_lib_dyn = $false
[bool] $unreal = $false
[bool] $test = $false
[bool] $parser_bounds = $false
[bool] $lexer_failures = $false
[bool] $parse_body_messages = $false
[bool] $dogfood_parse = $false
[array] $vendors = @( "clang", "msvc" )
@@ -33,24 +87,28 @@ Push-Location $path_root
if ( $args ) { $args | ForEach-Object {
switch ($_){
{ $_ -in $vendors } { $vendor = $_; break }
"verbose" { $verbose = $true }
"release" { $release = $true }
"debug" { $release = $false }
"base" { $base = $true }
"segmented" { $segmented = $true }
"singleheader" { $singleheader = $true }
"c_lib" { $c_lib = $true }
"c_lib_static" { $c_lib_static = $true }
"c_lib_dyn" { $c_lib_dyn = $true }
"unreal" { $unreal = $true }
"test" { $test = $true }
{ $_ -in $vendors } { $vendor = $_; break }
"verbose" { $verbose = $true }
"release" { $release = $true }
"debug" { $release = $false }
"base" { $base = $true }
"segmented" { $segmented = $true }
"singleheader" { $singleheader = $true }
"c_lib" { $c_lib = $true }
"c_lib_static" { $c_lib_static = $true }
"c_lib_dyn" { $c_lib_dyn = $true }
"unreal" { $unreal = $true }
"test" { $test = $true }
"parser_bounds" { $parser_bounds = $true }
"lexer_failures" { $lexer_failures = $true }
"parse_body_messages" { $parse_body_messages = $true }
"dogfood_parse" { $dogfood_parse = $true }
}
}}
#endregion Arguments
#region Configuration
if ($IsWindows) {
if ($is_windows) {
# This library was really designed to only run on 64-bit systems.
# (Its a development tool after all)
& $devshell -arch amd64
@@ -64,6 +122,7 @@ if ( $vendor -eq $null ) {
if ( $release -eq $null ) {
write-host "No build type specified, assuming debug"
$release = $false
$debug = $true
}
elseif ( $release -eq $false ) {
$debug = $true
@@ -72,16 +131,20 @@ else {
$optimize = $true
}
$cannot_build = $base -eq $false
$cannot_build = $cannot_build -and $segmented -eq $false
$cannot_build = $cannot_build -and $singleheader -eq $false
$cannot_build = $cannot_build -and $c_lib -eq $false
$cannot_build = $cannot_build -and $c_lib_static -eq $false
$cannot_build = $cannot_build -and $c_lib_dyn -eq $false
$cannot_build = $cannot_build -and $unreal -eq $false
$cannot_build = $cannot_build -and $test -eq $false
$cannot_build = $base -eq $false
$cannot_build = $cannot_build -and $segmented -eq $false
$cannot_build = $cannot_build -and $singleheader -eq $false
$cannot_build = $cannot_build -and $c_lib -eq $false
$cannot_build = $cannot_build -and $c_lib_static -eq $false
$cannot_build = $cannot_build -and $c_lib_dyn -eq $false
$cannot_build = $cannot_build -and $unreal -eq $false
$cannot_build = $cannot_build -and $test -eq $false
$cannot_build = $cannot_build -and $parser_bounds -eq $false
$cannot_build = $cannot_build -and $lexer_failures -eq $false
$cannot_build = $cannot_build -and $parse_body_messages -eq $false
$cannot_build = $cannot_build -and $dogfood_parse -eq $false
if ( $cannot_build ) {
throw "No build target specified. One must be specified, this script will not assume one"
Stop-stage "No build target specified"
}
. $vendor_toolchain
@@ -125,18 +188,7 @@ if ( $base )
$executable = join-path $path_build "base.exe"
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
Push-Location $path_base
if ( Test-Path( $executable ) ) {
write-host "`nRunning base"
$time_taken = Measure-Command { & $executable
| ForEach-Object {
write-host `t $_ -ForegroundColor Green
}
}
write-host "`nbase completed in $($time_taken.TotalMilliseconds) ms"
}
Pop-Location
invoke-stageTool -stage "base" -compiled $result -executable $executable -path_work $path_base
}
if ( $segmented )
@@ -163,18 +215,7 @@ if ( $segmented )
$executable = join-path $path_build "segmented.exe"
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
Push-Location $path_segmented
if ( Test-Path( $executable ) ) {
write-host "`nRunning segmented"
$time_taken = Measure-Command { & $executable
| ForEach-Object {
write-host `t $_ -ForegroundColor Green
}
}
write-host "`nSegmented completed in $($time_taken.TotalMilliseconds) ms"
}
Pop-Location
invoke-stageTool -stage "segmented" -compiled $result -executable $executable -path_work $path_segmented
}
if ( $singleheader )
@@ -201,18 +242,7 @@ if ( $singleheader )
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
Push-Location $path_singleheader
if ( Test-Path( $executable ) ) {
write-host "`nRunning singleheader generator"
$time_taken = Measure-Command { & $executable
| ForEach-Object {
write-host `t $_ -ForegroundColor Green
}
}
write-host "`nSingleheader generator completed in $($time_taken.TotalMilliseconds) ms"
}
Pop-Location
invoke-stageTool -stage "singleheader" -compiled $result -executable $executable -path_work $path_singleheader
}
if ( $c_lib -or $c_lib_static -or $c_lib_dyn )
@@ -239,18 +269,7 @@ if ( $c_lib -or $c_lib_static -or $c_lib_dyn )
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
Push-Location $path_c_library
if ( Test-Path( $executable ) ) {
write-host "`nRunning c_library generator"
$time_taken = Measure-Command { & $executable
| ForEach-Object {
write-host `t $_ -ForegroundColor Green
}
}
write-host "`nc_library generator completed in $($time_taken.TotalMilliseconds) ms"
}
Pop-Location
invoke-stageTool -stage "c_lib" -compiled $result -executable $executable -path_work $path_c_library
}
if ( $c_lib_static )
@@ -267,6 +286,7 @@ if ( $c_lib_static )
$linker_args = @()
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $path_lib
if (-not $result) { Stop-stage "c_lib_static compile/link" }
}
if ( $c_lib_dyn )
@@ -285,6 +305,7 @@ if ( $c_lib_dyn )
$linker_args = @()
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $path_dll
if (-not $result) { Stop-stage "c_lib_dyn compile/link" }
}
if ( $unreal )
@@ -311,23 +332,27 @@ if ( $unreal )
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
Push-Location $path_unreal
if ( Test-Path( $executable ) ) {
write-host "`nRunning unreal variant generator"
$time_taken = Measure-Command { & $executable
| ForEach-Object {
write-host `t $_ -ForegroundColor Green
}
}
write-host "`n Unreal variant generator completed in $($time_taken.TotalMilliseconds) ms"
}
Pop-Location
invoke-stageTool -stage "unreal" -compiled $result -executable $executable -path_work $path_unreal
. $refactor_unreal
}
# C Library testing
if ($test) {
$consumer_headers = @(
(Join-Path $path_c_library 'gen\gen_singleheader.h'),
(Join-Path $path_singleheader 'gen\gen.hpp')
)
$header_hashes_before = @{}
foreach ($header in $consumer_headers) {
if (-not (Test-Path -LiteralPath $header -PathType Leaf)) {
Stop-stage "test missing generated header $header"
}
$header_hashes_before[$header] = (Get-FileHash -LiteralPath $header -Algorithm SHA256).Hash
Write-Host "HEADER_SHA256_GENERATED: $($header_hashes_before[$header]) $header"
}
}
if ( $test -and $true )
{
$path_test_c = join-path $path_test c_library
@@ -356,18 +381,7 @@ if ( $test -and $true )
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
Push-Location $path_test_c
if ( Test-Path( $executable ) ) {
write-host "`nRunning c_library test"
$time_taken = Measure-Command { & $executable
| ForEach-Object {
write-host `t $_ -ForegroundColor Green
}
}
write-host "`nc_library generator completed in $($time_taken.TotalMilliseconds) ms"
}
Pop-Location
invoke-stageTool -stage "test-c11" -compiled $result -executable $executable -path_work $path_test_c
}
if ( $test -and $false )
@@ -398,30 +412,18 @@ if ( $test -and $false )
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
Push-Location $path_test_c
if ( Test-Path( $executable ) ) {
write-host "`nRunning c_library test"
$time_taken = Measure-Command { & $executable
| ForEach-Object {
write-host `t $_ -ForegroundColor Green
}
}
write-host "`nc_library generator completed in $($time_taken.TotalMilliseconds) ms"
}
Pop-Location
}
if ($test -and $false)
if ($test)
{
$path_test_cpp = join-path $path_test cpp_library
$path_build = join-path $path_test_cpp build
$path_gen = join-path $path_test_cpp gen
if ( -not(Test-Path($path_build) )) {
New-Item -ItemType Directory -Path $path_build
new-item -ItemType Directory -Path $path_build
}
if ( -not(Test-Path($path_gen) )) {
New-Item -ItemType Directory -Path $path_gen
new-item -ItemType Directory -Path $path_gen
}
$path_singleheader_include = join-path $path_singleheader gen
@@ -431,25 +433,124 @@ if ($test -and $false)
$compiler_args = @()
$compiler_args += ( $flag_define + 'GEN_TIME' )
$compiler_args += $flag_all_cpp
$compiler_args += $flag_cpp17
$compiler_args += ($flag_define + "GEN_BUILD_DEBUG=1")
$linker_args = @(
$flag_link_win_subsystem_console
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
invoke-stageTool -stage "test-cpp" -compiled $result -executable $executable -path_work $path_test_cpp
}
if ($test) {
foreach ($header in $consumer_headers) {
$hash = (Get-FileHash -LiteralPath $header -Algorithm SHA256).Hash
write-host "HEADER_SHA256_CONSUMED: $hash $header"
if ($hash -ne $header_hashes_before[$header]) {
Stop-stage "test header changed after generation $header"
}
}
}
Push-Location $path_test_cpp
if ( Test-Path( $executable ) ) {
write-host "`nRunning cpp_library test"
$time_taken = Measure-Command { & $executable
| ForEach-Object {
write-host `t $_ -ForegroundColor Green
}
}
write-host "`nc_library generator completed in $($time_taken.TotalMilliseconds) ms"
}
Pop-Location
if ($parser_bounds)
{
$path_probe = join-path $path_test parser_bounds
$path_build = join-path $path_probe build
if ( -not(Test-Path($path_build) )) {
new-item -ItemType Directory -Path $path_build | Out-Null
}
$includes = @( $path_base )
$unit = join-path $path_probe "test.cpp"
$executable = join-path $path_build "parser_bounds.exe"
$compiler_args = @()
$compiler_args += ( $flag_define + 'GEN_TIME' )
$compiler_args += $flag_cpp17
$linker_args = @(
$flag_link_win_subsystem_console
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
invoke-stageTool -stage "parser_bounds" -compiled $result -executable $executable -path_work $path_probe
}
if ($lexer_failures)
{
$path_probe = join-path $path_test lexer_failures
$path_build = join-path $path_probe build
if ( -not(Test-Path($path_build) )) {
new-item -ItemType Directory -Path $path_build | Out-Null
}
$includes = @( $path_base )
$unit = join-path $path_probe "test.cpp"
$executable = join-path $path_build "lexer_failures.exe"
$compiler_args = @()
$compiler_args += ( $flag_define + 'GEN_TIME' )
$compiler_args += $flag_cpp17
$linker_args = @(
$flag_link_win_subsystem_console
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
invoke-stageTool -stage "lexer_failures" -compiled $result -executable $executable -path_work $path_probe
}
if ($parse_body_messages)
{
$path_probe = join-path $path_test parse_body_messages
$path_build = join-path $path_probe build
if ( -not(Test-Path($path_build) )) {
new-item -ItemType Directory -Path $path_build | Out-Null
}
$includes = @( $path_base )
$unit = join-path $path_probe "test.cpp"
$executable = join-path $path_build "parse_body_messages.exe"
$compiler_args = @()
$compiler_args += ( $flag_define + 'GEN_TIME' )
$compiler_args += $flag_cpp17
$linker_args = @(
$flag_link_win_subsystem_console
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
invoke-stageTool -stage "parse_body_messages" -compiled $result -executable $executable -path_work $path_probe
}
if ($dogfood_parse)
{
$path_probe = join-path $path_test dogfood_parse
$path_build = join-path $path_probe build
if ( -not(Test-Path($path_build) )) {
new-item -ItemType Directory -Path $path_build | Out-Null
}
$includes = @( $path_base )
$unit = join-path $path_probe "test.cpp"
$executable = join-path $path_build "dogfood_parse.exe"
$compiler_args = @()
$compiler_args += ( $flag_define + 'GEN_TIME' )
$compiler_args += $flag_cpp17
$linker_args = @(
$flag_link_win_subsystem_console
)
$result = build-simple $path_build $includes $compiler_args $linker_args $unit $executable
invoke-stageTool -stage "dogfood_parse" -compiled $result -executable $executable -path_work $path_probe
}
#endregion Building
Pop-Location # $path_root
pop-location # $path_root
write-host "PASSED: build.ci.ps1"
exit 0
+1 -1
View File
@@ -1,3 +1,3 @@
cls
$build = Join-Path $PSScriptRoot 'build.ci.ps1'
& $build @args
exit $LASTEXITCODE
+36 -20
View File
@@ -1,26 +1,30 @@
$misc = Join-Path $PSScriptRoot 'helpers/misc.psm1'
Import-Module $misc
$path_root = Get-ScriptRepoRoot
$path_base = Join-Path $path_root base
$path_base_build = Join-Path $path_base build
$path_c_library = Join-Path $path_root gen_c_library
$path_c_library_build = Join-Path $path_c_library build
$path_c_library_gen = Join-Path $path_c_library gen
$path_segmented = Join-Path $path_root gen_segmented
$path_segmented_build = Join-Path $path_segmented build
$path_segmented_gen = Join-Path $path_segmented gen
$path_singleheader = Join-Path $path_root gen_singleheader
$path_singleheader_build = Join-Path $path_singleheader build
$path_singleheader_gen = Join-Path $path_singleheader gen
$path_unreal = Join-Path $path_root gen_unreal_engine
$path_unreal_build = Join-Path $path_unreal build
$path_unreal_gen = Join-Path $path_unreal gen
$path_test = Join-Path $path_root test
$path_test_build = Join-Path $path_test build
$path_test_gen = Join-Path $path_test gen
$path_x64 = Join-Path $path_root x64
$path_release = Join-Path $path_root release
$path_root = Get-ScriptRepoRoot
$path_base = Join-Path $path_root base
$path_base_build = Join-Path $path_base build
$path_c_library = Join-Path $path_root gen_c_library
$path_c_library_build = Join-Path $path_c_library build
$path_c_library_gen = Join-Path $path_c_library gen
$path_segmented = Join-Path $path_root gen_segmented
$path_segmented_build = Join-Path $path_segmented build
$path_segmented_gen = Join-Path $path_segmented gen
$path_singleheader = Join-Path $path_root gen_singleheader
$path_singleheader_build = Join-Path $path_singleheader build
$path_singleheader_gen = Join-Path $path_singleheader gen
$path_unreal = Join-Path $path_root gen_unreal_engine
$path_unreal_build = Join-Path $path_unreal build
$path_unreal_gen = Join-Path $path_unreal gen
$path_test = Join-Path $path_root test
$path_test_build = Join-Path $path_test build
$path_test_gen = Join-Path $path_test gen
$path_parser_bounds_build = Join-Path $path_test "parser_bounds\build"
$path_lexer_failures_build = Join-Path $path_test "lexer_failures\build"
$path_parse_body_messages_build = Join-Path $path_test "parse_body_messages\build"
$path_dogfood_parse_build = Join-Path $path_test "dogfood_parse\build"
$path_x64 = Join-Path $path_root x64
$path_release = Join-Path $path_root release
if ( Test-Path $path_base_build) {
Remove-Item $path_base_build -Recurse -Verbose
@@ -58,6 +62,18 @@ if ( Test-Path $path_test_build ) {
if ( Test-Path $path_test_gen ) {
Remove-Item $path_test_gen -Recurse -Verbose
}
if ( Test-Path $path_parser_bounds_build ) {
Remove-Item $path_parser_bounds_build -Recurse -Verbose
}
if ( Test-Path $path_lexer_failures_build ) {
Remove-Item $path_lexer_failures_build -Recurse -Verbose
}
if ( Test-Path $path_parse_body_messages_build ) {
Remove-Item $path_parse_body_messages_build -Recurse -Verbose
}
if ( Test-Path $path_dogfood_parse_build ) {
Remove-Item $path_dogfood_parse_build -Recurse -Verbose
}
if ( Test-Path $path_x64) {
Remove-Item $path_x64 -Recurse -Verbose
}
+28 -6
View File
@@ -3,7 +3,7 @@
import-module $target_arch
if ($IsWindows) {
if ($is_windows) {
# This HandmadeHero implementation is only designed for 64-bit systems
& $devshell -arch amd64
}
@@ -27,6 +27,12 @@ function run-archiver
{
param( $archiver, $library, $lib_args )
if (-not (Get-Command $archiver -ErrorAction SilentlyContinue)) {
Write-Host "FAILED: archiver launch $archiver" -ForegroundColor Red
return $false
}
Write-Host ("ARCHIVER_ARGV: " + ($lib_args -join " "))
write-host "`Creating library $library"
if ( $verbose ) {
write-host "Lib manager config:"
@@ -60,9 +66,15 @@ function run-compiler
{
param( $compiler, $unit, $compiler_args )
if (-not (Get-Command $compiler -ErrorAction SilentlyContinue)) {
Write-Host "FAILED: compiler launch $compiler" -ForegroundColor Red
return $false
}
if ( $analysis ) {
$compiler_args += $flag_syntax_only
}
Write-Host ("COMPILER_ARGV: " + ($compiler_args -join " "))
write-host "`Compiling $unit"
if ( $verbose ) {
@@ -97,6 +109,12 @@ function run-linker
{
param( $linker, $binary, $linker_args )
if (-not (Get-Command $linker -ErrorAction SilentlyContinue)) {
Write-Host "FAILED: linker launch $linker" -ForegroundColor Red
return $false
}
Write-Host ("LINKER_ARGV: " + ($linker_args -join " "))
write-host "`Linking $binary"
if ( $verbose ) {
write-host "Linker config:"
@@ -131,7 +149,9 @@ if ( $vendor -match "clang" )
# https://clang.llvm.org/docs/ClangCommandLineReference.html
$flag_all_c = @('-x', 'c')
$flag_c11 = '-std=c11'
$flag_all_cpp = '-x c++'
$flag_cpp17 = '-std=c++17'
$flag_all_cpp = @('-x', 'c++')
$flag_charset_utf8 = '-utf-8'
$flag_compile = '-c'
$flag_color_diagnostics = '-fcolor-diagnostics'
$flag_no_color_diagnostics = '-fno-color-diagnostics'
@@ -146,12 +166,12 @@ if ( $vendor -match "clang" )
$flag_library = '-l'
$flag_library_path = '-L'
$flag_linker = '-Wl,'
if ( $IsWindows ) {
if ( $is_windows ) {
$flag_link_dll = '/DLL'
$flag_link_mapfile = '/MAP:'
$flag_link_optimize_references = '/OPT:REF'
}
if ( $IsLinux ) {
if ( $is_linux ) {
$flag_link_mapfile = '--Map='
$flag_link_optimize_references = '--gc-sections'
}
@@ -190,7 +210,7 @@ if ( $vendor -match "clang" )
)
# https://learn.microsoft.com/en-us/cpp/c-runtime-library/crt-library-features?view=msvc-170
if ( $IsWindows ) {
if ( $is_windows ) {
$libraries = @(
'Kernel32' # For Windows API
# 'msvcrt', # For the C Runtime (Dynamically Linked)
@@ -370,7 +390,9 @@ if ( $vendor -match "msvc" )
# https://learn.microsoft.com/en-us/cpp/build/reference/compiler-options-listed-by-category?view=msvc-170
$flag_all_c = '/TC'
$flag_c11 = '/std:c11'
$flag_cpp17 = '/std:c++17'
$flag_all_cpp = '/TP'
$flag_charset_utf8 = '/utf-8'
$flag_compile = '/c'
$flag_debug = '/Zi'
$flag_define = '/D'
@@ -404,7 +426,7 @@ if ( $vendor -match "msvc" )
$flag_optimize_intrinsics = '/Oi'
$flag_optimized_debug_forceinline = '/d2Obforceinline'
$flag_optimized_debug = '/Zo'
$flag_
# $flag_
# $flag_out_name = '/OUT:'
$flag_path_interm = '/Fo'
$flag_path_debug = '/Fd'
+267
View File
@@ -0,0 +1,267 @@
param(
[Parameter(Mandatory = $true)][Alias('MirrorRoot')][string]$path_mirror,
[ValidateSet(
"measure",
"positive",
"negative-compile",
"negative-link",
"negative-launch",
"negative-stale",
"negative-stale-header",
"negative-format",
"negative-refactor",
"negative-assert"
)]
[string]$mode = "positive",
[Alias('SourceRoot')][string]$path_source_root = "C:\projects\gencpp",
[Alias('ReuseMirror')][switch]$reuse_mirror
)
$ErrorActionPreference = "Stop"
$path_source = (Resolve-Path -LiteralPath $path_source_root).Path
$path_mirror = [System.IO.Path]::GetFullPath($path_mirror)
$path_verification_root = Join-Path $path_mirror 'test\verification'
if ($path_mirror -eq $path_source) { throw "MirrorRoot must not be the project checkout" }
$prefix_source = $path_source.TrimEnd('\') + '\'
$prefix_mirror = $path_mirror.TrimEnd('\') + '\'
if ($path_source.StartsWith($prefix_mirror, [System.StringComparison]::OrdinalIgnoreCase) -or
$path_mirror.StartsWith($prefix_source, [System.StringComparison]::OrdinalIgnoreCase)) {
throw "path_mirror and path_source_root must not overlap"
}
if ($reuse_mirror -and $mode -ne "positive") { throw "reuse_mirror is for the positive mode only" }
if ($reuse_mirror -and -not (Test-Path -LiteralPath (Join-Path $path_mirror 'base\base.cpp'))) {
throw "reuse_mirror needs an existing baseline mirror"
}
function Copy-Snapshot([string]$path_destination)
{
if (Test-Path -LiteralPath $path_destination) { Remove-Item -LiteralPath $path_destination -Recurse -Force }
New-Item -ItemType Directory -Path $path_destination | Out-Null
& robocopy $path_source $path_destination /E /NFL /NDL /NJH /NJS /XD .git
if ($LASTEXITCODE -ge 8) { throw "robocopy failed: $LASTEXITCODE" }
$global:LASTEXITCODE = 0
@(
"gen_segmented\gen",
"gen_segmented\build",
"gen_singleheader\gen",
"gen_singleheader\build",
"gen_c_library\gen",
"gen_c_library\build",
"gen_unreal_engine\gen",
"gen_unreal_engine\build",
"test\c_library\build",
"test\c_library\gen",
"test\cpp_library\build",
"test\cpp_library\gen",
"base\build"
) | ForEach-Object {
$path_item = Join-Path $path_destination $_
if (Test-Path -LiteralPath $path_item) { Remove-Item -LiteralPath $path_item -Recurse -Force }
}
}
function Copy-VerificationFixture([string]$name_fixture, [string]$path_destination) {
$path_fixture = Join-Path $path_verification_root $name_fixture
if (-not (Test-Path -LiteralPath $path_fixture -PathType Leaf)) {
throw "Missing verification fixture: $path_fixture"
}
Copy-Item -LiteralPath $path_fixture -Destination $path_destination -Force
}
function Invoke-Driver([string]$path_tree, [string[]]$list_targets) {
$path_driver = Join-Path $path_tree "scripts\build.ci.ps1"
$name_log = if ($reuse_mirror) { 'verify_driver.repeat.log' } else { 'verify_driver.log' }
$path_log = Join-Path $path_tree $name_log
$path_previous_root = $env:GENCPP_SOURCE_ROOT
$env:GENCPP_SOURCE_ROOT = $path_tree
$line_args = $list_targets -join ' '
try {
$text_output = & powershell.exe -NoProfile -Command "& '$path_driver' clang debug $line_args; exit `$LASTEXITCODE" *>&1
$code_exit = $LASTEXITCODE
}
finally {
$env:GENCPP_SOURCE_ROOT = $path_previous_root
}
$text_output | Tee-Object -FilePath $path_log | Out-Host
$script:path_last_driver_log = $path_log
$script:code_last_driver = $code_exit
return $code_exit
}
function Get-DriverLogText {
if (-not $script:path_last_driver_log) { return "" }
if (-not (Test-Path -LiteralPath $script:path_last_driver_log)) { return "" }
return Get-Content -LiteralPath $script:path_last_driver_log -Raw
}
function Assert-PositiveOutputs([string]$path_tree) {
$list_outputs = @(
'base\components\gen\ecodetypes.hpp',
'base\components\gen\eoperator.hpp',
'base\components\gen\especifier.hpp',
'base\components\gen\etoktype.hpp',
'base\components\gen\ast_inlines.hpp',
'gen_segmented\gen\gen.hpp',
'gen_segmented\gen\gen.cpp',
'gen_segmented\gen\gen.dep.hpp',
'gen_segmented\gen\gen.dep.cpp',
'gen_segmented\gen\gen.builder.hpp',
'gen_segmented\gen\gen.builder.cpp',
'gen_segmented\gen\gen.scanner.hpp',
'gen_segmented\gen\gen.scanner.cpp',
'gen_singleheader\gen\gen.hpp',
'gen_c_library\gen\gen.h',
'gen_c_library\gen\gen.c',
'gen_c_library\gen\gen.dep.h',
'gen_c_library\gen\gen.dep.c',
'gen_c_library\gen\gen_singleheader.h'
)
$list_hashes = foreach ($path_relative in $list_outputs) {
$path_output = Join-Path $path_tree $path_relative
if (-not (Test-Path -LiteralPath $path_output -PathType Leaf)) { throw "Missing output: $path_output" }
if ((Get-Item -LiteralPath $path_output).Length -eq 0) { throw "Empty output: $path_output" }
"{0} {1}" -f (Get-FileHash -LiteralPath $path_output -Algorithm SHA256).Hash, $path_relative
}
$text_log = (Get-DriverLogText).Replace('\\', '\')
foreach ($path_relative in @('gen_singleheader\gen\gen.hpp', 'gen_c_library\gen\gen_singleheader.h')) {
$path_header = Join-Path $path_tree $path_relative
if ($text_log -notmatch [regex]::Escape($path_header)) { throw "Compiler trace missing $path_header" }
}
$name_hash_file = if ($reuse_mirror) { 'verify_outputs.repeat.sha256' } else { 'verify_outputs.sha256' }
$list_hashes | Set-Content -LiteralPath (Join-Path $path_tree $name_hash_file)
Write-Host "PASSED: outputs and consumer header traces ($name_hash_file)"
}
$list_targets = @("base", "segmented", "singleheader", "c_lib", "test")
if (-not $reuse_mirror) { Copy-Snapshot $path_mirror }
if ($mode -eq "measure" -or $mode -eq "positive") {
$code_exit = Invoke-Driver $path_mirror $list_targets
if ($mode -eq "positive" -and $code_exit -ne 0) { exit $code_exit }
if ($mode -eq "positive") { Assert-PositiveOutputs $path_mirror }
exit $code_exit
}
if ($mode -eq "negative-compile") {
Copy-VerificationFixture 'failure_compile.cpp' (Join-Path $path_mirror 'base\base.cpp')
$code_exit = Invoke-Driver $path_mirror @("base")
if ($code_exit -eq 0) { throw "negative-compile returned 0" }
exit 0
}
if ($mode -eq "negative-link") {
Copy-VerificationFixture 'failure_link.cpp' (Join-Path $path_mirror 'base\base.cpp')
$code_exit = Invoke-Driver $path_mirror @("base")
$text_log = Get-DriverLogText
if ($code_exit -eq 0 -or $text_log -notmatch "Linking failed") {
throw "negative-link did not fail at the linker"
}
exit 0
}
if ($mode -eq "negative-launch") {
$path_toolchain = Join-Path $path_mirror "scripts\helpers\vendor_toolchain.ps1"
$text_toolchain = Get-Content -LiteralPath $path_toolchain -Raw
$text_patched = $text_toolchain.Replace(
"`$linker = 'lld-link'",
"`$linker = 'gencpp-missing-linker'"
)
if ($text_patched -eq $text_toolchain) { throw "negative-launch could not replace the linker path" }
Set-Content -LiteralPath $path_toolchain -Value $text_patched -NoNewline
$code_exit = Invoke-Driver $path_mirror @("base")
$text_log = Get-DriverLogText
if ($code_exit -eq 0 -or $text_log -notmatch "FAILED: linker launch gencpp-missing-linker") {
throw "negative-launch did not report a missing linker"
}
exit 0
}
if ($mode -eq "negative-stale") {
$path_stale_source = Join-Path $path_mirror "stale_marker.c"
$path_stale_executable = Join-Path $path_mirror "stale_marker.exe"
Copy-VerificationFixture 'stale_executable.c' $path_stale_source
& clang -o $path_stale_executable $path_stale_source
if ($LASTEXITCODE -ne 0) { throw "failed to compile stale marker" }
$path_base_build = Join-Path $path_mirror "base\build"
New-Item -ItemType Directory -Path $path_base_build -Force | Out-Null
Copy-Item -LiteralPath $path_stale_executable -Destination (Join-Path $path_base_build "base.exe") -Force
Copy-VerificationFixture 'failure_compile.cpp' (Join-Path $path_mirror 'base\base.cpp')
$code_exit = Invoke-Driver $path_mirror @("base")
if ($code_exit -eq 0) { throw "negative-stale returned 0" }
if ($code_exit -eq 73) { throw "negative-stale ran the stale executable" }
exit 0
}
if ($mode -eq "negative-stale-header") {
$code_exit = Invoke-Driver $path_mirror @("base", "segmented", "singleheader", "c_lib")
if ($code_exit -ne 0) { throw "negative-stale-header setup generation failed: $code_exit" }
Copy-Item -LiteralPath $script:path_last_driver_log -Destination (Join-Path $path_mirror 'verify_driver.setup.log')
$path_header = Join-Path $path_mirror 'gen_c_library\gen\gen_singleheader.h'
if (-not (Test-Path -LiteralPath $path_header)) { throw 'negative-stale-header missing setup header' }
Copy-VerificationFixture 'failure_compile.cpp' (Join-Path $path_mirror 'gen_c_library\c_library.cpp')
$code_exit = Invoke-Driver $path_mirror @('c_lib', 'test')
$text_log = Get-DriverLogText
if ($code_exit -eq 0 -or $text_log -notmatch 'FAILED: c_lib compile/link') {
throw 'negative-stale-header did not fail at c_lib'
}
if ($text_log -match 'Compiling .*test[.]c|PASSED: test-c11|PASSED: test-cpp') {
throw 'negative-stale-header consumed a prior header'
}
exit 0
}
if ($mode -eq "negative-format") {
$path_fake_tools = Join-Path $path_mirror "fake-tools"
New-Item -ItemType Directory -Path $path_fake_tools -Force | Out-Null
Copy-VerificationFixture 'failure_format.cmd' (Join-Path $path_fake_tools 'clang-format.cmd')
$path_saved = $env:PATH
$env:PATH = "$path_fake_tools;$env:PATH"
try {
$code_exit = Invoke-Driver $path_mirror @("base")
}
finally {
$env:PATH = $path_saved
}
$text_log = Get-DriverLogText
if ($code_exit -eq 0) { throw "negative-format returned 0" }
if ($text_log -notmatch "clang-format failed") { throw "negative-format log missing clang-format failed" }
exit 0
}
if ($mode -eq "negative-refactor") {
$path_fake_source = Join-Path $path_mirror 'failing_refactor.c'
$path_fake_executable = Join-Path $path_mirror 'scripts\helpers\refactor.exe'
Copy-VerificationFixture 'failure_refactor.c' $path_fake_source
& clang -o $path_fake_executable $path_fake_source
if ($LASTEXITCODE -ne 0) { throw 'negative-refactor failed to compile the stub' }
$code_exit = Invoke-Driver $path_mirror @('c_lib')
$text_log = Get-DriverLogText
if ($code_exit -eq 0 -or $text_log -notmatch 'refactor failed') {
throw 'negative-refactor did not propagate subprocess failure'
}
exit 0
}
if ($mode -eq "negative-assert") {
$path_header = Join-Path $path_mirror "gen_singleheader\gen\gen.hpp"
if (-not (Test-Path -LiteralPath $path_header)) {
$code_exit = Invoke-Driver $path_mirror $list_targets
if ($code_exit -ne 0) { throw "negative-assert setup generation failed: $code_exit" }
}
Copy-VerificationFixture 'failure_assert.cpp' (Join-Path $path_mirror 'test\cpp_library\test.cpp')
$code_exit = Invoke-Driver $path_mirror @("test")
$text_log = Get-DriverLogText
if ($code_exit -eq 0) { throw "negative-assert returned 0" }
if ($text_log -notmatch "FAIL forced assertion") { throw "negative-assert log missing forced assertion marker" }
exit 0
}
throw "unhandled mode $mode"
+38 -2
View File
@@ -1,3 +1,39 @@
# Test
# Baseline verification
The implementation here has been gutted and will be rewritten...
The implementation smoke is `verify_baseline.ps1`. It is not a parser corpus.
From the project root, in a fresh PowerShell process:
```powershell
.\scripts\verify_baseline.ps1 -Mode positive -MirrorRoot ..\gencpp-verify-positive
.\scripts\verify_baseline.ps1 -Mode positive -MirrorRoot ..\gencpp-verify-positive -ReuseMirror
```
The first command generates base component tables, segmented C++, single-header C++, and both C11 layouts.
It compiles and runs the C++17 and C11 single-header host smokes. The second command reuses the same mirror.
The verification wrapper copies them into the disposable mirror; it does not embed C or C++ source text.
## Parser bounds probe
`test/parser_bounds/test.cpp` is a GEN_TIME program. It includes `base/gen.cpp`. It is not part of `verify_baseline.ps1`.
From the project root:
```powershell
.\scripts\build.ps1 clang parser_bounds
```
The script builds and runs the probe. A pass prints `parser bounds passed` and `PASSED: parser_bounds`.
The probe covers helper positions, formatting skip at the slice end, empty and whitespace `parse_variable`, and two direct end-of-slice reads.
It does not prove in-slice syntax errors return.
## Lexer failure probe
```powershell
.\scripts\build.ps1 clang lexer_failures
```
A pass prints `lexer failures passed` and `PASSED: lexer_failures`.
The probe covers nine lexer failures, six successful sibling inputs, and a baseline variable after each failure.
It does not prove in-slice parser errors return.
+90
View File
@@ -0,0 +1,90 @@
gen_internal void expect_simd_lanes()
{
#if defined(GEN_SYSTEM_WINDOWS) && (defined(_M_X64) || defined(__x86_64__))
gen_u8 storage[32];
gen_mem_set(storage, 0x11, 32);
gen_u8 pattern[16] = {
0x00, 0x80, 0xFF, 0x01,
0x02, 0x03, 0x04, 0x05,
0x06, 0x07, 0x08, 0x09,
0x0A, 0x0B, 0x0C, 0x0D
};
gen_mem_copy(storage + 1, pattern, 16);
gen_u8x16 loaded = gen_u8x16_load(storage + 1);
gen_u8x16 again = gen_u8x16_load(storage + 1);
gen_u16 same = gen_u8x16_mask(gen_u8x16_equal(loaded, again));
gen_u16 zero_mask = gen_u8x16_mask(gen_u8x16_equal(loaded, gen_u8x16_splat(0x00)));
gen_u16 high_bit = gen_u8x16_mask(gen_u8x16_equal(loaded, gen_u8x16_splat(0x80)));
gen_u16 top = gen_u8x16_mask(gen_u8x16_equal(loaded, gen_u8x16_splat(0xFF)));
expect(same == 0xFFFF, gen_txt("simd u8 equal load"));
expect(zero_mask == 0x0001, gen_txt("simd byte 0"));
expect(high_bit == 0x0002, gen_txt("simd byte 0x80"));
expect(top == 0x0004, gen_txt("simd byte 0xFF"));
gen_u8 high_lane[16];
gen_mem_set(high_lane, 0x22, 16);
high_lane[15] = 0xFF;
gen_u8x16 high_loaded = gen_u8x16_load(high_lane);
gen_u16 lane15 = gen_u8x16_mask(gen_u8x16_equal(high_loaded, gen_u8x16_splat(0xFF)));
expect(lane15 == 0x8000, gen_txt("simd lane 15"));
expect(gen_u32_mask_first_set(lane15) == 15, gen_txt("simd first set lane 15"));
gen_u8 window[24];
gen_mem_set(window, 0xEE, 24);
window[1] = 0x00;
window[2] = 0x80;
window[3] = 0xFF;
window[4] = 0x11;
window[5] = 0x22;
window[6] = 0x33;
window[7] = 0x44;
window[8] = 0x55;
gen_u8x16 loaded8 = gen_u8x16_load8(window + 1);
gen_u16 upper = gen_u8x16_mask(gen_u8x16_equal(loaded8, gen_u8x16_splat(0x00)));
gen_u16 low_ff = gen_u8x16_mask(gen_u8x16_equal(loaded8, gen_u8x16_splat(0xFF)));
gen_u16 low_80 = gen_u8x16_mask(gen_u8x16_equal(loaded8, gen_u8x16_splat(0x80)));
gen_u16 byte7 = gen_u8x16_mask(gen_u8x16_equal(loaded8, gen_u8x16_splat(0x55)));
expect((upper & 0xFF00) == 0xFF00, gen_txt("simd load8 upper zero"));
expect((upper & 0x0001) == 0x0001, gen_txt("simd load8 low zero"));
expect(low_ff == 0x0004, gen_txt("simd load8 low 0xFF"));
expect(low_80 == 0x0002, gen_txt("simd load8 low 0x80"));
expect(byte7 == 0x0080, gen_txt("simd load8 byte 7"));
gen_u16 sentinel = gen_u8x16_mask(gen_u8x16_equal(loaded8, gen_u8x16_splat(0xEE)));
expect(sentinel == 0, gen_txt("simd load8 ignores tail"));
gen_u8x16 loaded16 = gen_u8x16_load(window + 1);
gen_u16 tail = gen_u8x16_mask(gen_u8x16_equal(loaded16, gen_u8x16_splat(0xEE)));
expect(tail == 0xFF00, gen_txt("simd load16 sees load8 tail"));
gen_u32 hashes[4] = { 0x00000000u, 0x7FFFFFFFu, 0x80000000u, 0xFFFFFFFFu };
gen_u32x4 hash_loaded = gen_u32x4_load(hashes);
gen_u8 lane0 = gen_u32x4_mask(gen_u32x4_equal(hash_loaded, gen_u32x4_splat(0x00000000u)));
gen_u8 lane1 = gen_u32x4_mask(gen_u32x4_equal(hash_loaded, gen_u32x4_splat(0x7FFFFFFFu)));
gen_u8 lane2 = gen_u32x4_mask(gen_u32x4_equal(hash_loaded, gen_u32x4_splat(0x80000000u)));
gen_u8 lane3 = gen_u32x4_mask(gen_u32x4_equal(hash_loaded, gen_u32x4_splat(0xFFFFFFFFu)));
expect(lane0 == 0x01, gen_txt("simd u32 lane 0"));
expect(lane1 == 0x02, gen_txt("simd u32 lane 1"));
expect(lane2 == 0x04, gen_txt("simd u32 bit31 lane"));
expect(lane3 == 0x08, gen_txt("simd u32 lane 3"));
expect((lane2 & 0xF0) == 0, gen_txt("simd u32 mask low bits"));
gen_u32 all_values[4] = { 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u };
gen_u32x4 all_loaded = gen_u32x4_load(all_values);
gen_u8 all_mask = gen_u32x4_mask(gen_u32x4_equal(all_loaded, gen_u32x4_splat(0x80000000u)));
expect(all_mask == 0x0F, gen_txt("simd u32 all bit31"));
gen_u8 none_mask = gen_u32x4_mask(gen_u32x4_equal(all_loaded, gen_u32x4_splat(0x7FFFFFFFu)));
expect(none_mask == 0x00, gen_txt("simd u32 none"));
gen_u8 hash_storage[20];
gen_mem_set (hash_storage, 0xA5, 20);
gen_mem_copy(hash_storage + 1, hashes, 16);
gen_u32x4 unaligned = gen_u32x4_load(hash_storage + 1);
gen_u8 unaligned_mask = gen_u32x4_mask(gen_u32x4_equal(unaligned, gen_u32x4_splat(0x80000000u)));
expect(unaligned_mask == 0x04, gen_txt("simd u32 unaligned bit31"));
expect(gen_u32_mask_first_set(0x1u) == 0, gen_txt("simd first set bit 0"));
expect(gen_u32_mask_first_set(0x14u) == 2, gen_txt("simd first set lowest"));
expect(gen_u32_mask_first_set(0x8000u) == 15, gen_txt("simd first set bit 15"));
expect(gen_u32_mask_first_set(0x80000000u) == 31, gen_txt("simd first set bit 31"));
expect(gen_u32_mask_first_set(0xFFFFFFFFu) == 0, gen_txt("simd first set all bits"));
#endif
}
+631
View File
@@ -0,0 +1,631 @@
gen_internal gen_Str bounded_format(char* storage, gen_ssize storage_len, char const* bytes, gen_ssize len) {
gen_mem_set(storage, '!', storage_len);
if (len > 0)
gen_mem_copy(storage, bytes, len);
storage[storage_len - 1] = '\0';
gen_Str result = { storage, len };
return result;
}
gen_internal void expect_template_expansion(gen_Str label, gen_Str format, gen_StrTemplateSlot* slots, gen_ssize slot_count, gen_Str expansion) {
gen_StrBuilder output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_append_str(& output, gen_txt("KEEP")), label);
gen_Slice_StrTemplateSlot slice = { 0, 0 };
if (slot_count != 0) {
slice.Ptr = slots;
slice.Len = slot_count;
}
bool appended = gen_strbuilder_append_template(& output, slice, format);
expect(appended, label);
gen_Str got = gen_strbuilder_to_str(output);
expect(got.Len == 4 + expansion.Len, label);
expect(gen_str_starts_with(got, gen_txt("KEEP")), label);
if (got.Len >= 4 + expansion.Len) {
gen_Str tail = { got.Ptr + 4, expansion.Len };
expect(gen_str_are_equal(tail, expansion), label);
}
gen_StrBuilderHeader* header = gen_strbuilder_get_header(output);
expect(header->Allocator.Proc == gen_heap_allocator_proc, label);
gen_strbuilder_free(& output);
}
gen_internal void expect_template_text(gen_Str label, gen_Str format, gen_StrTemplateSlot* slots, gen_ssize slot_count, gen_Str expansion) {
expect_template_expansion(label, format, slots, slot_count, expansion);
}
gen_internal void expect_literal_boundary(gen_ssize mark) {
char format_storage[64];
char format_bytes [40];
for (gen_ssize idx = 0; idx < mark; ++ idx) format_bytes[idx] = 'L';
gen_Str format = bounded_format(format_storage, 64, format_bytes, mark);
gen_Str expansion = { format_bytes, mark };
expect(format_storage[mark] == '!', gen_txt("literal boundary"));
expect_template_expansion(gen_txt("literal boundary"), format, 0, 0, expansion);
}
gen_internal void expect_open_boundary(gen_ssize mark) {
char format_storage [64];
char format_bytes [48];
char expansion_bytes[48];
gen_ssize format_len = 0;
for (gen_ssize idx = 0; idx < mark; ++ idx)
format_bytes[format_len ++] = 'L';
format_bytes[format_len ++] = '<';
format_bytes[format_len ++] = 'k';
format_bytes[format_len ++] = '>';
format_bytes[format_len ++] = 'Z';
gen_Str format = bounded_format(format_storage, 64, format_bytes, format_len);
gen_StrTemplateSlot slot = {0};
slot.Key = gen_txt("k");
slot.Value = gen_txt("V");
gen_ssize expansion_len = 0;
for (gen_ssize idx = 0; idx < mark; ++ idx)
expansion_bytes[expansion_len ++] = 'L';
expansion_bytes[expansion_len ++] = 'V';
expansion_bytes[expansion_len ++] = 'Z';
gen_Str expansion = { expansion_bytes, expansion_len };
expect(format.Ptr[mark] == '<', gen_txt("open boundary"));
expect(format_storage[format_len] == '!', gen_txt("open boundary"));
expect_template_expansion(gen_txt("open boundary"), format, & slot, 1, expansion);
}
gen_internal void expect_close_boundary(gen_ssize mark) {
char format_storage[64];
char format_bytes [48];
char key_storage [40];
gen_ssize key_len = mark - 1;
gen_mem_set(key_storage, '!', 40);
for (gen_ssize idx = 0; idx < key_len; ++ idx)
key_storage[idx] = 'k';
key_storage[39] = '\0';
gen_Str key = { key_storage, key_len };
gen_ssize format_len = 0;
format_bytes[format_len ++] = '<';
for (gen_ssize idx = 0; idx < key_len; ++ idx)
format_bytes[format_len ++] = 'k';
format_bytes[format_len ++] = '>';
format_bytes[format_len ++] = 'Z';
gen_Str format = bounded_format(format_storage, 64, format_bytes, format_len);
gen_StrTemplateSlot slot = {0};
slot.Key = key;
slot.Value = gen_txt("V");
expect(format.Ptr [mark] == '>', gen_txt("close boundary"));
expect(key.Ptr [key.Len] == '!', gen_txt("close boundary"));
expect(format_storage[format_len] == '!', gen_txt("close boundary"));
expect_template_expansion(gen_txt("close boundary"), format, & slot, 1, gen_txt("VZ"));
}
gen_internal void expect_template_shapes()
{
char literal_storage[8];
char literal_bytes [6] = { 'a', 'b', 'c', 'D', 'E', 'F' };
gen_Str literal = bounded_format(literal_storage, 8, literal_bytes, 3);
expect(literal_storage[3] == '!', gen_txt("literal sentinel"));
expect_template_expansion(gen_txt("literal bounded"), literal, 0, 0, gen_txt("abc"));
char empty_storage[4];
gen_Str empty_format = bounded_format(empty_storage, 4, empty_storage, 0);
expect(empty_storage[0] == '!', gen_txt("empty sentinel"));
expect_template_expansion(gen_txt("empty format"), empty_format, 0, 0, gen_txt(""));
gen_StrTemplateSlot name_slot = {0};
name_slot.Key = gen_txt("name");
name_slot.Value = gen_txt("world");
expect_template_text(gen_txt("one slot"), gen_txt("hello <name>!"), & name_slot, 1, gen_txt("hello world!"));
gen_StrTemplateSlot repeat_slot = {0};
repeat_slot.Key = gen_txt("w");
repeat_slot.Value = gen_txt("v");
expect_template_text(gen_txt("repeated slot"), gen_txt("<w>-<w>-<w>"), & repeat_slot, 1, gen_txt("v-v-v"));
gen_StrTemplateSlot adjacent[2] = {0};
adjacent[0].Key = gen_txt("a");
adjacent[0].Value = gen_txt("X");
adjacent[1].Key = gen_txt("b");
adjacent[1].Value = gen_txt("Y");
expect_template_text(gen_txt("adjacent slots"), gen_txt("<a><b><a>"), adjacent, 2, gen_txt("XYX"));
gen_StrTemplateSlot raw_slots[3] = {0};
raw_slots[0].Key = gen_txt("raw");
raw_slots[0].Value = gen_txt("a<b>c");
raw_slots[1].Key = gen_txt("angles");
raw_slots[1].Value = gen_txt("<>");
raw_slots[2].Key = gen_txt("b");
raw_slots[2].Value = gen_txt("SHOULD_NOT");
expect_template_text(gen_txt("value angles literal"), gen_txt("<raw><angles>"), raw_slots, 3, gen_txt("a<b>c<>"));
char empty_value_storage[2] = { '!', '!' };
gen_StrTemplateSlot empty_slot = {0};
empty_slot.Key = gen_txt("e");
empty_slot.Value = (gen_Str){ empty_value_storage, 0 };
expect_template_text(gen_txt("empty value"), gen_txt("A<e>B"), & empty_slot, 1, gen_txt("AB"));
gen_StrTemplateSlot empty_key = {0};
empty_key.Key = (gen_Str){ 0, 0 };
empty_key.Value = gen_txt("Z");
expect_template_text(gen_txt("empty key"), gen_txt("<>"), & empty_key, 1, gen_txt("Z"));
gen_StrTemplateSlot null_value = {0};
null_value.Key = gen_txt("e");
null_value.Value = (gen_Str){ 0, 0 };
expect(null_value.Value.Ptr == 0, gen_txt("null empty value pointer"));
expect(null_value.Value.Len == 0, gen_txt("null empty value length"));
expect_template_text(gen_txt("null empty value"), gen_txt("A<e>B"), & null_value, 1, gen_txt("AB"));
char nul_value_bytes[4] = { 'a', '\0', 'b', '!' };
char nul_expected [5] = { '[', 'a', '\0', 'b', ']' };
gen_StrTemplateSlot nul_slot = {0};
nul_slot.Key = gen_txt("n");
nul_slot.Value = (gen_Str){ nul_value_bytes, 3 };
gen_Str nul_expansion = { nul_expected, 5 };
expect_template_text(gen_txt("embedded nul"), gen_txt("[<n>]"), & nul_slot, 1, nul_expansion);
gen_StrTemplateSlot duffle[2] = {0};
duffle[0].Key = gen_txt("operands[0]");
duffle[0].Value = gen_txt("eax");
duffle[1].Key = gen_txt("mod_rm.bits");
duffle[1].Value = gen_txt("11");
expect_template_text(gen_txt("duffle keys"), gen_txt("mov <operands[0]>, <mod_rm.bits>;"), duffle, 2, gen_txt("mov eax, 11;"));
}
gen_internal void fill_lookup_slots(gen_StrTemplateSlot* slots, gen_ssize slot_count) {
gen_Str keys[9] = {
gen_txt("k0"), gen_txt("k1"), gen_txt("k2"), gen_txt("k3"), gen_txt("k4"),
gen_txt("k5"), gen_txt("k6"), gen_txt("k7"), gen_txt("k8")
};
gen_Str values[9] = {
gen_txt("V0"), gen_txt("V1"), gen_txt("V2"), gen_txt("V3"), gen_txt("V4"),
gen_txt("V5"), gen_txt("V6"), gen_txt("V7"), gen_txt("V8")
};
for (gen_ssize idx = 0; idx < slot_count; ++ idx) {
slots[idx].Key = keys[idx];
slots[idx].Value = values[idx];
}
}
gen_internal void expect_template_lookup()
{
gen_StrTemplateSlot slots[9] = {0};
expect_template_text(gen_txt("lookup 0"), gen_txt("none"), 0, 0, gen_txt("none"));
fill_lookup_slots(slots, 1);
expect_template_text(gen_txt("lookup 1"), gen_txt("<k0>"), slots, 1, gen_txt("V0"));
fill_lookup_slots(slots, 2);
expect_template_text(gen_txt("lookup 2"), gen_txt("<k1><k0>"), slots, 2, gen_txt("V1V0"));
fill_lookup_slots(slots, 3);
expect_template_text(gen_txt("lookup 3"), gen_txt("<k2>-<k0>-<k1>"), slots, 3, gen_txt("V2-V0-V1"));
fill_lookup_slots(slots, 4);
expect_template_text(gen_txt("lookup 4"), gen_txt("<k3><k0><k2><k1>"), slots, 4, gen_txt("V3V0V2V1"));
fill_lookup_slots(slots, 5);
expect_template_text(gen_txt("lookup 5 tail"), gen_txt("<k4><k0><k3>"), slots, 5, gen_txt("V4V0V3"));
fill_lookup_slots(slots, 8);
expect_template_text(gen_txt("lookup 8 lanes"), gen_txt("<k7><k4><k3><k0>"), slots, 8, gen_txt("V7V4V3V0"));
fill_lookup_slots(slots, 9);
expect_template_text(gen_txt("lookup 9 tail"), gen_txt("<k8><k5><k6><k4>"), slots, 9, gen_txt("V8V5V6V4"));
gen_StrTemplateSlot dup_small[3] = {0};
dup_small[0].Key = gen_txt("dup"); dup_small[0].Value = gen_txt("FIRST");
dup_small[1].Key = gen_txt("other"); dup_small[1].Value = gen_txt("X");
dup_small[2].Key = gen_txt("dup"); dup_small[2].Value = gen_txt("SECOND");
expect_template_text(gen_txt("duplicate first wins"), gen_txt("<dup><other>"), dup_small, 3, gen_txt("FIRSTX"));
gen_StrTemplateSlot dup_lanes[4] = {0};
dup_lanes[0].Key = gen_txt("same"); dup_lanes[0].Value = gen_txt("A");
dup_lanes[1].Key = gen_txt("same"); dup_lanes[1].Value = gen_txt("B");
dup_lanes[2].Key = gen_txt("other"); dup_lanes[2].Value = gen_txt("C");
dup_lanes[3].Key = gen_txt("same"); dup_lanes[3].Value = gen_txt("D");
expect_template_text(gen_txt("duplicate lane first wins"), gen_txt("<same><other>"), dup_lanes, 4, gen_txt("AC"));
gen_StrTemplateSlot dup_tail[5] = {0};
dup_tail[0].Key = gen_txt("dup"); dup_tail[0].Value = gen_txt("FIRST");
dup_tail[1].Key = gen_txt("k1"); dup_tail[1].Value = gen_txt("V1");
dup_tail[2].Key = gen_txt("k2"); dup_tail[2].Value = gen_txt("V2");
dup_tail[3].Key = gen_txt("k3"); dup_tail[3].Value = gen_txt("V3");
dup_tail[4].Key = gen_txt("dup"); dup_tail[4].Value = gen_txt("TAIL");
expect_template_text(gen_txt("duplicate tail first wins"), gen_txt("<dup>"), dup_tail, 5, gen_txt("FIRST"));
}
gen_internal void expect_template_boundaries() {
gen_ssize marks[12] = { 7, 8, 9, 15, 16, 17, 23, 24, 25, 31, 32, 33 };
for (gen_ssize idx = 0; idx < gen_count_of(marks); ++ idx) {
expect_literal_boundary(marks[idx]);
expect_open_boundary(marks[idx]);
expect_close_boundary(marks[idx]);
}
}
gen_internal void expect_template_growth(gen_ssize printf_maxlen)
{
gen_ssize huge_len = printf_maxlen + 64;
char* huge = gen_rcast(char*, gen_alloc(gen_heap(), huge_len + 1));
expect(huge != 0, gen_txt("growth alloc"));
if (huge == 0) return;
gen_mem_set(huge, 'Q', huge_len);
huge[huge_len] = '!';
gen_Str value = { huge, huge_len };
gen_StrTemplateSlot slot = {0};
slot.Key = gen_txt("blob");
slot.Value = value;
char format_storage[16];
gen_Str format = gen_txt("<blob>");
gen_StrBuilder output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_capacity(output) == 1, gen_txt("growth initial capacity"));
expect(gen_strbuilder_append_str(& output, gen_txt("KEEP")), gen_txt("growth prefix"));
gen_Slice_StrTemplateSlot slice = { & slot, 1 };
bool appended = gen_strbuilder_append_template(& output, slice, format);
expect(appended, gen_txt("growth append"));
gen_Str got = gen_strbuilder_to_str(output);
expect(got.Len == 4 + huge_len, gen_txt("growth length"));
expect(got.Len > printf_maxlen, gen_txt("growth beyond printf max"));
expect(gen_str_starts_with(got, gen_txt("KEEP")), gen_txt("growth keeps prefix"));
gen_ssize capacity = gen_strbuilder_capacity(output);
expect(capacity >= got.Len, gen_txt("growth capacity"));
if (got.Len == 4 + huge_len && capacity >= got.Len) {
gen_b32 payload_matches = true;
for (gen_ssize idx = 0; idx < got.Len - 4; ++ idx) {
if (got.Ptr[4 + idx] != 'Q') payload_matches = false;
}
expect(payload_matches, gen_txt("growth bytes"));
}
gen_StrBuilderHeader* header = gen_strbuilder_get_header(output);
expect(header->Allocator.Proc == gen_heap_allocator_proc, gen_txt("growth allocator"));
expect(huge[huge_len] == '!', gen_txt("growth sentinel"));
gen_strbuilder_free(& output);
gen_allocator_free(gen_heap(), huge);
}
gen_internal void expect_template_crc_collision() {
gen_Str plumless = gen_txt("plumless");
gen_Str buckeroo = gen_txt("buckeroo");
gen_u32 plum_hash = gen_crc32(plumless.Ptr, plumless.Len);
gen_u32 buck_hash = gen_crc32(buckeroo.Ptr, buckeroo.Len);
expect(plum_hash == buck_hash, gen_txt("crc collision equal"));
expect(plum_hash == 0x4ddb0c25, gen_txt("crc collision value"));
expect(gen_str_are_equal(plumless, buckeroo) == false, gen_txt("crc collision keys differ"));
gen_StrTemplateSlot scalar[2] = {0};
scalar[0].Key = plumless; scalar[0].Value = gen_txt("WRONG");
scalar[1].Key = buckeroo; scalar[1].Value = gen_txt("RIGHT");
expect_template_text(gen_txt("crc scalar 2"), gen_txt("<buckeroo>"), scalar, 2, gen_txt("RIGHT"));
gen_StrTemplateSlot lanes[4] = {0};
lanes[0].Key = plumless; lanes[0].Value = gen_txt("WRONG");
lanes[1].Key = gen_txt("k1"); lanes[1].Value = gen_txt("V1");
lanes[2].Key = buckeroo; lanes[2].Value = gen_txt("RIGHT");
lanes[3].Key = gen_txt("k3"); lanes[3].Value = gen_txt("V3");
expect_template_text(gen_txt("crc lane reject"), gen_txt("<buckeroo>"), lanes, 4, gen_txt("RIGHT"));
gen_StrTemplateSlot tail[5] = {0};
tail[0].Key = plumless; tail[0].Value = gen_txt("WRONG");
tail[1].Key = gen_txt("k1"); tail[1].Value = gen_txt("V1");
tail[2].Key = gen_txt("k2"); tail[2].Value = gen_txt("V2");
tail[3].Key = gen_txt("k3"); tail[3].Value = gen_txt("V3");
tail[4].Key = buckeroo; tail[4].Value = gen_txt("TAIL");
expect_template_text(gen_txt("crc tail after false hit"), gen_txt("<buckeroo>"), tail, 5, gen_txt("TAIL"));
}
gen_internal void* resize_fail_allocator_proc(void* allocator_data, gen_AllocType type, gen_ssize size, gen_ssize alignment
, void* old_memory, gen_ssize old_size
, gen_u64 flags
){
if (type == EAllocation_RESIZE) {
gen_s32* attempts = gen_rcast(gen_s32*, allocator_data);
attempts[0] = attempts[0] + 1;
return 0;
}
return gen_heap_allocator_proc(0, type, size, alignment, old_memory, old_size, flags);
}
gen_internal void expect_template_resize_fail(gen_ssize printf_maxlen)
{
gen_ssize huge_len = printf_maxlen + 64;
char* huge = gen_rcast(char*, gen_alloc(gen_heap(), huge_len + 1));
expect(huge != 0, gen_txt("resize fail alloc"));
if (huge == 0) return;
gen_mem_set(huge, 'L', huge_len);
huge[huge_len] = '!';
gen_Str huge_text = { huge, huge_len };
gen_s32 literal_attempts = 0;
gen_AllocatorInfo literal_alloc = {0};
literal_alloc.Proc = resize_fail_allocator_proc;
literal_alloc.Data = & literal_attempts;
gen_StrBuilder literal_output = gen_strbuilder_make_reserve(literal_alloc, 4);
expect(gen_strbuilder_append_str(& literal_output, gen_txt("KEEP")), gen_txt("resize fail literal prefix"));
gen_Slice_StrTemplateSlot literal_slots = { 0, 0 };
bool literal_appended = gen_strbuilder_append_template(& literal_output, literal_slots, huge_text);
expect(literal_appended == false, gen_txt("resize fail literal"));
gen_Str literal_got = gen_strbuilder_to_str(literal_output);
expect(literal_got.Len == 4, gen_txt("resize fail literal length"));
expect(gen_str_are_equal(literal_got, gen_txt("KEEP")), gen_txt("resize fail literal keep"));
expect(literal_attempts > 0, gen_txt("resize fail literal attempts"));
gen_strbuilder_free(& literal_output);
gen_s32 value_attempts = 0;
gen_AllocatorInfo value_alloc = {0};
value_alloc.Proc = resize_fail_allocator_proc;
value_alloc.Data = & value_attempts;
gen_StrTemplateSlot slot = {0};
slot.Key = gen_txt("blob");
slot.Value = huge_text;
char format_storage[16];
gen_Str format = gen_txt("<blob>");
gen_StrBuilder value_output = gen_strbuilder_make_reserve(value_alloc, 4);
expect(gen_strbuilder_append_str(& value_output, gen_txt("KEEP")), gen_txt("resize fail value prefix"));
gen_Slice_StrTemplateSlot value_slots = { & slot, 1 };
bool value_appended = gen_strbuilder_append_template(& value_output, value_slots, format);
expect(value_appended == false, gen_txt("resize fail value"));
gen_Str value_got = gen_strbuilder_to_str(value_output);
expect(value_got.Len == 4, gen_txt("resize fail value length"));
expect(gen_str_are_equal(value_got, gen_txt("KEEP")), gen_txt("resize fail value keep"));
expect(value_attempts > 0, gen_txt("resize fail value attempts"));
gen_strbuilder_free(& value_output);
gen_allocator_free(gen_heap(), huge);
}
gen_internal gen_Str counted_template_value (gen_s32* reads) { reads[0] = reads[0] + 1; return gen_txt("ONCE"); }
gen_internal gen_Str counted_template_format(gen_s32* reads) { reads[0] = reads[0] + 1; return gen_txt("<k>:<k>"); }
gen_internal gen_StrBuilder* counted_template_output(gen_StrBuilder* output, gen_s32* reads) { reads[0] = reads[0] + 1; return output; }
gen_internal bool* counted_template_result(bool* result, gen_s32* reads) { reads[0] = reads[0] + 1; return result; }
gen_internal void expect_template_slice()
{
gen_StrTemplateSlot slots[2] = {
gen_strtmpl_slot("a", gen_txt("X")),
gen_strtmpl_slot("b", gen_txt("Y"))
};
gen_Slice_StrTemplateSlot slice = { slots, 2 };
gen_StrBuilder output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_append_str(& output, gen_txt("KEEP")), gen_txt("slice prefix"));
bool appended = gen_strbuilder_append_template(& output, slice, gen_txt("<b><a><b>"));
expect(appended, gen_txt("slice append"));
expect(gen_str_are_equal(gen_strbuilder_to_str(output), gen_txt("KEEPYXY")), gen_txt("slice text"));
expect(gen_str_are_equal(slots[0].Key, gen_txt("a")), gen_txt("slice key"));
expect(gen_str_are_equal(slots[0].Value, gen_txt("X")), gen_txt("slice value"));
gen_strbuilder_free(& output);
slice.Ptr[0].Value = gen_txt("Z");
gen_StrBuilder mutated_output = gen_strbuilder_make_reserve(gen_heap(), 1);
bool mutated_appended = gen_strbuilder_append_template(& mutated_output, slice, gen_txt("<b><a><b>"));
expect(mutated_appended, gen_txt("slice mutate append"));
expect(gen_str_are_equal(gen_strbuilder_to_str(mutated_output), gen_txt("YZY")), gen_txt("slice mutate text"));
gen_strbuilder_free(& mutated_output);
gen_Slice_StrTemplateSlot empty_slice = { 0, 0 };
gen_StrBuilder empty_output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_append_str(& empty_output, gen_txt("KEEP")), gen_txt("zero slice prefix"));
bool empty_appended = gen_strbuilder_append_template(& empty_output, empty_slice, gen_txt("none"));
expect(empty_appended, gen_txt("zero slice append"));
expect(gen_str_are_equal(gen_strbuilder_to_str(empty_output), gen_txt("KEEPnone")), gen_txt("zero slice text"));
expect(empty_slice.Ptr == 0, gen_txt("zero slice pointer"));
expect(empty_slice.Len == 0, gen_txt("zero slice length"));
gen_strbuilder_free(& empty_output);
}
gen_internal void expect_template_wrapper() {
gen_StrBuilder output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_append_str(& output, gen_txt("KEEP")), gen_txt("wrapper prefix"));
bool appended = false;
gen_strtmpl_append(& output, appended, gen_strtmpl_slots(
gen_strtmpl_slot("name", gen_txt("world")),
gen_strtmpl_slot("tail", gen_txt("!"))
),
gen_txt(
"hello <name>\n"
"<name><tail>"
)
);
expect(appended, gen_txt("wrapper multiline result"));
gen_Str got = gen_strbuilder_to_str(output);
expect(gen_str_starts_with(got, gen_txt("KEEP")), gen_txt("wrapper prefix kept"));
expect(gen_str_are_equal (got, gen_txt("KEEPhello world\nworld!")), gen_txt("wrapper multiline text"));
gen_strbuilder_free(& output);
}
gen_internal void expect_template_wrapper_scope()
{
gen_StrBuilder then_output = gen_strbuilder_make_reserve(gen_heap(), 1);
gen_StrBuilder else_output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_append_str(& then_output, gen_txt("KEEP")), gen_txt("wrapper then prefix"));
expect(gen_strbuilder_append_str(& else_output, gen_txt("KEEP")), gen_txt("wrapper else prefix"));
bool then_result = false;
bool else_result = false;
gen_s32 selector = 1;
gen_s32 after_branch = 0;
if (selector == 1)
gen_strtmpl_append(& then_output, then_result, gen_strtmpl_slots(gen_strtmpl_slot("arm", gen_txt("THEN"))),
gen_txt("<arm>")
);
else
gen_strtmpl_append(& else_output, else_result, gen_strtmpl_slots(gen_strtmpl_slot("arm", gen_txt("ELSE"))),
gen_txt("<arm>")
);
after_branch = 1;
expect(after_branch == 1, gen_txt("wrapper then continues"));
expect(then_result, gen_txt("wrapper then assigned"));
expect(else_result == false, gen_txt("wrapper else skipped"));
expect(gen_str_are_equal(gen_strbuilder_to_str(then_output), gen_txt("KEEPTHEN")), gen_txt("wrapper then text"));
expect(gen_str_are_equal(gen_strbuilder_to_str(else_output), gen_txt("KEEP")), gen_txt("wrapper else untouched"));
gen_strbuilder_free(& then_output);
gen_strbuilder_free(& else_output);
then_output = gen_strbuilder_make_reserve(gen_heap(), 1);
else_output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_append_str(& then_output, gen_txt("KEEP")), gen_txt("wrapper skipped then prefix"));
expect(gen_strbuilder_append_str(& else_output, gen_txt("KEEP")), gen_txt("wrapper taken else prefix"));
then_result = false;
else_result = false;
selector = 0;
after_branch = 0;
if (selector == 1) gen_strtmpl_append(& then_output, then_result, gen_strtmpl_slots(gen_strtmpl_slot("arm", gen_txt("THEN"))), gen_txt("<arm>"));
else gen_strtmpl_append(& else_output, else_result, gen_strtmpl_slots(gen_strtmpl_slot("arm", gen_txt("ELSE"))), gen_txt("<arm>"));
after_branch = 1;
expect(after_branch == 1, gen_txt("wrapper else continues"));
expect(then_result == false, gen_txt("wrapper then skipped"));
expect(else_result, gen_txt("wrapper else assigned"));
expect(gen_str_are_equal(gen_strbuilder_to_str(then_output), gen_txt("KEEP")), gen_txt("wrapper then untouched"));
expect(gen_str_are_equal(gen_strbuilder_to_str(else_output), gen_txt("KEEPELSE")), gen_txt("wrapper else text"));
gen_strbuilder_free(& then_output);
gen_strbuilder_free(& else_output);
}
gen_internal void expect_template_wrapper_once() {
gen_s32 value_reads = 0;
gen_s32 format_reads = 0;
gen_s32 output_reads = 0;
gen_s32 result_reads = 0;
gen_StrBuilder output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_append_str(& output, gen_txt("KEEP")), gen_txt("wrapper once prefix"));
bool appended = false;
gen_strtmpl_append(
counted_template_output(& output, & output_reads),
counted_template_result(& appended, & result_reads)[0],
gen_strtmpl_slots(
gen_strtmpl_slot("k", counted_template_value(& value_reads)),
gen_strtmpl_slot("tail", gen_txt("Z"))
),
counted_template_format(& format_reads)
);
expect(appended, gen_txt("wrapper once result"));
expect(value_reads == 1, gen_txt("wrapper value once"));
expect(format_reads == 1, gen_txt("wrapper format once"));
expect(output_reads == 1, gen_txt("wrapper output once"));
expect(result_reads == 1, gen_txt("wrapper result once"));
expect(gen_str_are_equal(gen_strbuilder_to_str(output), gen_txt("KEEPONCE:ONCE")), gen_txt("wrapper once text"));
gen_strbuilder_free(& output);
}
gen_internal void expect_template_wrapper_resize_fail() {
gen_s32 attempts = 0;
gen_AllocatorInfo alloc = {0};
alloc.Proc = resize_fail_allocator_proc;
alloc.Data = & attempts;
gen_StrBuilder output = gen_strbuilder_make_reserve(alloc, 4);
expect(gen_strbuilder_capacity(output) == 4, gen_txt("wrapper resize capacity"));
expect(gen_strbuilder_append_str(& output, gen_txt("KEEP")), gen_txt("wrapper resize prefix"));
bool appended = true;
gen_strtmpl_append(& output, appended,
gen_strtmpl_slots(gen_strtmpl_slot("blob", gen_txt("X"))),
gen_txt("<blob>")
);
expect(appended == false, gen_txt("wrapper resize assigned false"));
expect(gen_str_are_equal(gen_strbuilder_to_str(output), gen_txt("KEEP")), gen_txt("wrapper resize keep"));
expect(attempts > 0, gen_txt("wrapper resize attempts"));
gen_strbuilder_free(& output);
}
gen_internal void expect_template_slot_literal()
{
gen_Str value = gen_txt("V");
gen_StrTemplateSlot slot = gen_strtmpl_slot("k", value);
expect(gen_str_are_equal(slot.Key, gen_txt("k")), gen_txt("slot literal key slice"));
expect(slot.Key.Ptr[slot.Key.Len] == '\0', gen_txt("slot literal key terminator"));
expect(slot.Value.Ptr == value.Ptr, gen_txt("slot literal value pointer"));
expect(slot.Value.Len == value.Len, gen_txt("slot literal value length"));
gen_Str assigned = gen_txt("W");
slot = gen_strtmpl_slot("key", assigned);
expect(gen_str_are_equal(slot.Key, gen_txt("key")), gen_txt("slot assign key slice"));
expect(slot.Value.Ptr == assigned.Ptr, gen_txt("slot assign value pointer"));
expect(slot.Value.Len == assigned.Len, gen_txt("slot assign value length"));
expect_template_text(gen_txt("slot assign expansion"), gen_txt("<key>"), & slot, 1, gen_txt("W"));
gen_StrTemplateSlot row[2] = {
gen_strtmpl_slot("a", gen_txt("X")),
gen_strtmpl_slot("b", gen_txt("Y"))
};
expect(gen_str_are_equal(row[0].Value, gen_txt("X")), gen_txt("slot array value slice"));
expect(gen_str_are_equal(row[1].Key, gen_txt("b")), gen_txt("slot array key slice"));
expect_template_text(gen_txt("slot array entry"), gen_txt("<b><a>"), row, 2, gen_txt("YX"));
gen_StrBuilder output = gen_strbuilder_make_reserve(gen_heap(), 1);
expect(gen_strbuilder_append_str(& output, gen_txt("KEEP")), gen_txt("slot wrapper prefix"));
bool appended = false;
gen_strtmpl_append(& output, appended, gen_strtmpl_slots(
gen_strtmpl_slot("name", gen_txt("world")),
gen_strtmpl_slot("tail", gen_txt("!"))
),
gen_txt("<name><tail>")
);
expect(appended, gen_txt("slot wrapper result"));
expect(gen_str_are_equal(gen_strbuilder_to_str(output), gen_txt("KEEPworld!")), gen_txt("slot wrapper text"));
gen_strbuilder_free(& output);
gen_StrTemplateSlot long_key = gen_strtmpl_slot("operands[0]", gen_txt("eax"));
expect(gen_str_are_equal(long_key.Key, gen_txt("operands[0]")), gen_txt("slot literal key slice"));
expect(long_key.Key.Ptr[long_key.Key.Len] == '\0', gen_txt("slot literal key excludes nul"));
expect(gen_str_are_equal(long_key.Value, gen_txt("eax")), gen_txt("slot literal value slice"));
gen_s32 value_reads = 0;
gen_StrTemplateSlot once = gen_strtmpl_slot("k", counted_template_value(& value_reads));
expect(value_reads == 1, gen_txt("slot value once"));
expect(gen_str_are_equal(once.Value, gen_txt("ONCE")), gen_txt("slot value once text"));
expect(once.Key.Len == 1, gen_txt("slot value once key"));
char nul_bytes[4] = { 'a', '\0', 'b', '!' };
gen_Str nul_value = { nul_bytes, 3 };
gen_StrTemplateSlot nul_slot = gen_strtmpl_slot("n", nul_value);
expect(nul_slot.Value.Ptr == nul_bytes, gen_txt("slot nul pointer"));
expect(nul_slot.Value.Len == 3, gen_txt("slot nul length"));
expect(nul_slot.Value.Ptr[1] == '\0', gen_txt("slot nul byte"));
char nul_expected[5] = { '[', 'a', '\0', 'b', ']' };
gen_Str nul_expansion = { nul_expected, 5 };
expect_template_text(gen_txt("slot nul expansion"), gen_txt("[<n>]"), & nul_slot, 1, nul_expansion);
gen_Str angle_value = gen_txt("a<b>c");
gen_StrTemplateSlot angle_slot = gen_strtmpl_slot("raw", angle_value);
expect(angle_slot.Value.Ptr == angle_value.Ptr, gen_txt("slot angle pointer"));
expect(angle_slot.Value.Len == angle_value.Len, gen_txt("slot angle length"));
expect_template_text(gen_txt("slot angle expansion"), gen_txt("<raw>"), & angle_slot, 1, gen_txt("a<b>c"));
gen_Str empty_value = { 0, 0 };
gen_StrTemplateSlot empty_slot = gen_strtmpl_slot("e", empty_value);
expect(empty_slot.Value.Ptr == 0, gen_txt("slot null value pointer"));
expect(empty_slot.Value.Len == 0, gen_txt("slot null value length"));
expect_template_text(gen_txt("slot null value expansion"), gen_txt("A<e>B"), & empty_slot, 1, gen_txt("AB"));
gen_Str percent_value = gen_txt("%d:%S");
gen_StrTemplateSlot percent_slot = gen_strtmpl_slot("fmt", percent_value);
expect(percent_slot.Value.Ptr == percent_value.Ptr, gen_txt("slot percent value pointer"));
expect(gen_str_are_equal(percent_slot.Value, gen_txt("%d:%S")), gen_txt("slot percent value text"));
expect_template_text(gen_txt("slot percent literal"), gen_txt("pre %d:%S <fmt>"), & percent_slot, 1, gen_txt("pre %d:%S %d:%S"));
}
gen_internal void expect_text_template(gen_ssize printf_maxlen) {
expect_template_shapes();
expect_template_lookup();
expect_template_crc_collision();
expect_template_boundaries();
expect_template_growth(printf_maxlen);
expect_template_resize_fail(printf_maxlen);
expect_template_slice();
expect_template_wrapper();
expect_template_wrapper_scope();
expect_template_wrapper_once();
expect_template_wrapper_resize_fail();
expect_template_slot_literal();
}
+1045 -22
View File
File diff suppressed because it is too large. Load diff
+91
View File
@@ -0,0 +1,91 @@
internal void expect_simd_lanes()
{
#if defined(GEN_SYSTEM_WINDOWS) && (defined(_M_X64) || defined(__x86_64__))
using namespace gen;
u8 storage[32];
mem_set(storage, 0x11, 32);
u8 pattern[16] = {
0x00, 0x80, 0xFF, 0x01,
0x02, 0x03, 0x04, 0x05,
0x06, 0x07, 0x08, 0x09,
0x0A, 0x0B, 0x0C, 0x0D
};
mem_copy(storage + 1, pattern, 16);
u8x16 loaded = u8x16_load(storage + 1);
u8x16 again = u8x16_load(storage + 1);
u16 same = u8x16_mask(u8x16_equal(loaded, again));
expect(same == 0xFFFF, txt("simd u8 equal load"));
u16 zero_mask = u8x16_mask(u8x16_equal(loaded, u8x16_splat(0x00)));
expect(zero_mask == 0x0001, txt("simd byte 0"));
u16 high_bit = u8x16_mask(u8x16_equal(loaded, u8x16_splat(0x80)));
expect(high_bit == 0x0002, txt("simd byte 0x80"));
u16 top = u8x16_mask(u8x16_equal(loaded, u8x16_splat(0xFF)));
expect(top == 0x0004, txt("simd byte 0xFF"));
u8 high_lane[16];
mem_set(high_lane, 0x22, 16);
high_lane[15] = 0xFF;
u8x16 high_loaded = u8x16_load(high_lane);
u16 lane15 = u8x16_mask(u8x16_equal(high_loaded, u8x16_splat(0xFF)));
expect(lane15 == 0x8000, txt("simd lane 15"));
expect(u32_mask_first_set(lane15) == 15, txt("simd first set lane 15"));
u8 window[24];
mem_set(window, 0xEE, 24);
window[1] = 0x00;
window[2] = 0x80;
window[3] = 0xFF;
window[4] = 0x11;
window[5] = 0x22;
window[6] = 0x33;
window[7] = 0x44;
window[8] = 0x55;
u8x16 loaded8 = u8x16_load8(window + 1);
u16 upper = u8x16_mask(u8x16_equal(loaded8, u8x16_splat(0x00)));
expect((upper & 0xFF00) == 0xFF00, txt("simd load8 upper zero"));
expect((upper & 0x0001) == 0x0001, txt("simd load8 low zero"));
u16 low_ff = u8x16_mask(u8x16_equal(loaded8, u8x16_splat(0xFF)));
expect(low_ff == 0x0004, txt("simd load8 low 0xFF"));
u16 low_80 = u8x16_mask(u8x16_equal(loaded8, u8x16_splat(0x80)));
expect(low_80 == 0x0002, txt("simd load8 low 0x80"));
u16 byte7 = u8x16_mask(u8x16_equal(loaded8, u8x16_splat(0x55)));
expect(byte7 == 0x0080, txt("simd load8 byte 7"));
u16 sentinel = u8x16_mask(u8x16_equal(loaded8, u8x16_splat(0xEE)));
expect(sentinel == 0, txt("simd load8 ignores tail"));
u8x16 loaded16 = u8x16_load(window + 1);
u16 tail = u8x16_mask(u8x16_equal(loaded16, u8x16_splat(0xEE)));
expect(tail == 0xFF00, txt("simd load16 sees load8 tail"));
u32 hashes[4] = { 0x00000000u, 0x7FFFFFFFu, 0x80000000u, 0xFFFFFFFFu };
u32x4 hash_loaded = u32x4_load(hashes);
u8 lane0 = u32x4_mask(u32x4_equal(hash_loaded, u32x4_splat(0x00000000u)));
u8 lane1 = u32x4_mask(u32x4_equal(hash_loaded, u32x4_splat(0x7FFFFFFFu)));
u8 lane2 = u32x4_mask(u32x4_equal(hash_loaded, u32x4_splat(0x80000000u)));
u8 lane3 = u32x4_mask(u32x4_equal(hash_loaded, u32x4_splat(0xFFFFFFFFu)));
expect(lane0 == 0x01, txt("simd u32 lane 0"));
expect(lane1 == 0x02, txt("simd u32 lane 1"));
expect(lane2 == 0x04, txt("simd u32 bit31 lane"));
expect(lane3 == 0x08, txt("simd u32 lane 3"));
expect((lane2 & 0xF0) == 0, txt("simd u32 mask low bits"));
u32 all_values[4] = { 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u };
u32x4 all_loaded = u32x4_load(all_values);
u8 all_mask = u32x4_mask(u32x4_equal(all_loaded, u32x4_splat(0x80000000u)));
expect(all_mask == 0x0F, txt("simd u32 all bit31"));
u8 none_mask = u32x4_mask(u32x4_equal(all_loaded, u32x4_splat(0x7FFFFFFFu)));
expect(none_mask == 0x00, txt("simd u32 none"));
u8 hash_storage[20];
mem_set(hash_storage, 0xA5, 20);
mem_copy(hash_storage + 1, hashes, 16);
u32x4 unaligned = u32x4_load(hash_storage + 1);
u8 unaligned_mask = u32x4_mask(u32x4_equal(unaligned, u32x4_splat(0x80000000u)));
expect(unaligned_mask == 0x04, txt("simd u32 unaligned bit31"));
expect(u32_mask_first_set(0x1u) == 0, txt("simd first set bit 0"));
expect(u32_mask_first_set(0x14u) == 2, txt("simd first set lowest"));
expect(u32_mask_first_set(0x8000u) == 15, txt("simd first set bit 15"));
expect(u32_mask_first_set(0x80000000u) == 31, txt("simd first set bit 31"));
expect(u32_mask_first_set(0xFFFFFFFFu) == 0, txt("simd first set all bits"));
#endif
}
+651
View File
@@ -0,0 +1,651 @@
internal gen::Str bounded_format(char* storage, gen::ssize storage_len, char const* bytes, gen::ssize len)
{
gen::mem_set(storage, '!', storage_len);
if (len > 0)
gen::mem_copy(storage, bytes, len);
storage[storage_len - 1] = '\0';
gen::Str result = { storage, len };
return result;
}
internal void expect_template_expansion(gen::Str label, gen::Str format, gen::StrTemplateSlot* slots, gen::ssize slot_count, gen::Str expansion)
{
using namespace gen;
StrBuilder output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_append_str(& output, txt("KEEP")), label);
Slice_StrTemplateSlot slice = { nullptr, 0 };
if (slot_count != 0) {
slice.Ptr = slots;
slice.Len = slot_count;
}
bool appended = strbuilder_append_template(& output, slice, format);
expect(appended, label);
Str got = strbuilder_to_str(output);
expect(got.Len == 4 + expansion.Len, label);
expect(str_starts_with(got, txt("KEEP")), label);
if (got.Len >= 4 + expansion.Len) {
Str tail = { got.Ptr + 4, expansion.Len };
expect(str_are_equal(tail, expansion), label);
}
StrBuilderHeader* header = strbuilder_get_header(output);
expect(header->Allocator.Proc == heap_allocator_proc, label);
strbuilder_free(& output);
}
internal void expect_template_text(gen::Str label, gen::Str format, gen::StrTemplateSlot* slots, gen::ssize slot_count, gen::Str expansion)
{
expect_template_expansion(label, format, slots, slot_count, expansion);
}
internal void expect_literal_boundary(gen::ssize mark) {
using namespace gen;
char format_storage[64];
char format_bytes[40];
for (ssize idx = 0; idx < mark; ++ idx)
format_bytes[idx] = 'L';
Str format = bounded_format(format_storage, 64, format_bytes, mark);
Str expansion = { format_bytes, mark };
expect(format_storage[mark] == '!', txt("literal boundary"));
expect_template_expansion(txt("literal boundary"), format, nullptr, 0, expansion);
}
internal void expect_open_boundary(gen::ssize mark)
{
using namespace gen;
char format_storage[64];
char format_bytes[48];
char expansion_bytes[48];
ssize format_len = 0;
for (ssize idx = 0; idx < mark; ++ idx)
format_bytes[format_len ++] = 'L';
format_bytes[format_len ++] = '<';
format_bytes[format_len ++] = 'k';
format_bytes[format_len ++] = '>';
format_bytes[format_len ++] = 'Z';
Str format = bounded_format(format_storage, 64, format_bytes, format_len);
StrTemplateSlot slot = {};
slot.Key = txt("k");
slot.Value = txt("V");
ssize expansion_len = 0;
for (ssize idx = 0; idx < mark; ++ idx)
expansion_bytes[expansion_len ++] = 'L';
expansion_bytes[expansion_len ++] = 'V';
expansion_bytes[expansion_len ++] = 'Z';
Str expansion = { expansion_bytes, expansion_len };
expect(format.Ptr[mark] == '<', txt("open boundary"));
expect(format_storage[format_len] == '!', txt("open boundary"));
expect_template_expansion(txt("open boundary"), format, & slot, 1, expansion);
}
internal void expect_close_boundary(gen::ssize mark) {
using namespace gen;
char format_storage[64];
char format_bytes[48];
char key_storage[40];
ssize key_len = mark - 1;
mem_set(key_storage, '!', 40);
for (ssize idx = 0; idx < key_len; ++ idx)
key_storage[idx] = 'k';
key_storage[39] = '\0';
Str key = { key_storage, key_len };
ssize format_len = 0;
format_bytes[format_len ++] = '<';
for (ssize idx = 0; idx < key_len; ++ idx)
format_bytes[format_len ++] = 'k';
format_bytes[format_len ++] = '>';
format_bytes[format_len ++] = 'Z';
Str format = bounded_format(format_storage, 64, format_bytes, format_len);
StrTemplateSlot slot = {};
slot.Key = key;
slot.Value = txt("V");
expect(format.Ptr[mark] == '>', txt("close boundary"));
expect(key.Ptr[key.Len] == '!', txt("close boundary"));
expect(format_storage[format_len] == '!', txt("close boundary"));
expect_template_expansion(txt("close boundary"), format, & slot, 1, txt("VZ"));
}
internal void expect_template_shapes()
{
using namespace gen;
char literal_storage[8];
char literal_bytes[6] = { 'a', 'b', 'c', 'D', 'E', 'F' };
Str literal = bounded_format(literal_storage, 8, literal_bytes, 3);
expect(literal_storage[3] == '!', txt("literal sentinel"));
expect_template_expansion(txt("literal bounded"), literal, nullptr, 0, txt("abc"));
char empty_storage[4];
Str empty_format = bounded_format(empty_storage, 4, empty_storage, 0);
expect(empty_storage[0] == '!', txt("empty sentinel"));
expect_template_expansion(txt("empty format"), empty_format, nullptr, 0, txt(""));
StrTemplateSlot name_slot = {};
name_slot.Key = txt("name");
name_slot.Value = txt("world");
expect_template_text(txt("one slot"), txt("hello <name>!"), & name_slot, 1, txt("hello world!"));
StrTemplateSlot repeat_slot = {};
repeat_slot.Key = txt("w");
repeat_slot.Value = txt("v");
expect_template_text(txt("repeated slot"), txt("<w>-<w>-<w>"), & repeat_slot, 1, txt("v-v-v"));
StrTemplateSlot adjacent[2] = {};
adjacent[0].Key = txt("a");
adjacent[0].Value = txt("X");
adjacent[1].Key = txt("b");
adjacent[1].Value = txt("Y");
expect_template_text(txt("adjacent slots"), txt("<a><b><a>"), adjacent, 2, txt("XYX"));
StrTemplateSlot raw_slots[3] = {};
raw_slots[0].Key = txt("raw");
raw_slots[0].Value = txt("a<b>c");
raw_slots[1].Key = txt("angles");
raw_slots[1].Value = txt("<>");
raw_slots[2].Key = txt("b");
raw_slots[2].Value = txt("SHOULD_NOT");
expect_template_text(txt("value angles literal"), txt("<raw><angles>"), raw_slots, 3, txt("a<b>c<>"));
char empty_value_storage[2] = { '!', '!' };
StrTemplateSlot empty_slot = {};
empty_slot.Key = txt("e");
empty_slot.Value = { empty_value_storage, 0 };
expect_template_text(txt("empty value"), txt("A<e>B"), & empty_slot, 1, txt("AB"));
StrTemplateSlot empty_key = {};
empty_key.Key = { nullptr, 0 };
empty_key.Value = txt("Z");
expect_template_text(txt("empty key"), txt("<>"), & empty_key, 1, txt("Z"));
StrTemplateSlot null_value = {};
null_value.Key = txt("e");
null_value.Value = { nullptr, 0 };
expect(null_value.Value.Ptr == nullptr, txt("null empty value pointer"));
expect(null_value.Value.Len == 0, txt("null empty value length"));
expect_template_text(txt("null empty value"), txt("A<e>B"), & null_value, 1, txt("AB"));
char nul_value_bytes[4] = { 'a', '\0', 'b', '!' };
char nul_expected[5] = { '[', 'a', '\0', 'b', ']' };
StrTemplateSlot nul_slot = {};
nul_slot.Key = txt("n");
nul_slot.Value = { nul_value_bytes, 3 };
gen::Str nul_expansion = { nul_expected, 5 };
expect_template_text(txt("embedded nul"), txt("[<n>]"), & nul_slot, 1, nul_expansion);
StrTemplateSlot duffle[2] = {};
duffle[0].Key = txt("operands[0]");
duffle[0].Value = txt("eax");
duffle[1].Key = txt("mod_rm.bits");
duffle[1].Value = txt("11");
expect_template_text(txt("duffle keys"), txt("mov <operands[0]>, <mod_rm.bits>;"), duffle, 2, txt("mov eax, 11;"));
}
internal void fill_lookup_slots(gen::StrTemplateSlot* slots, gen::ssize slot_count) {
gen::Str keys[9] = {
txt("k0"), txt("k1"), txt("k2"), txt("k3"), txt("k4"),
txt("k5"), txt("k6"), txt("k7"), txt("k8")
};
gen::Str values[9] = {
txt("V0"), txt("V1"), txt("V2"), txt("V3"), txt("V4"),
txt("V5"), txt("V6"), txt("V7"), txt("V8")
};
for (gen::ssize idx = 0; idx < slot_count; ++ idx) {
slots[idx].Key = keys[idx];
slots[idx].Value = values[idx];
}
}
internal void expect_template_lookup()
{
using namespace gen;
StrTemplateSlot slots[9] = {};
expect_template_text(txt("lookup 0"), txt("none"), nullptr, 0, txt("none"));
fill_lookup_slots(slots, 1);
expect_template_text(txt("lookup 1"), txt("<k0>"), slots, 1, txt("V0"));
fill_lookup_slots(slots, 2);
expect_template_text(txt("lookup 2"), txt("<k1><k0>"), slots, 2, txt("V1V0"));
fill_lookup_slots(slots, 3);
expect_template_text(txt("lookup 3"), txt("<k2>-<k0>-<k1>"), slots, 3, txt("V2-V0-V1"));
fill_lookup_slots(slots, 4);
expect_template_text(txt("lookup 4"), txt("<k3><k0><k2><k1>"), slots, 4, txt("V3V0V2V1"));
fill_lookup_slots(slots, 5);
expect_template_text(txt("lookup 5 tail"), txt("<k4><k0><k3>"), slots, 5, txt("V4V0V3"));
fill_lookup_slots(slots, 8);
expect_template_text(txt("lookup 8 lanes"), txt("<k7><k4><k3><k0>"), slots, 8, txt("V7V4V3V0"));
fill_lookup_slots(slots, 9);
expect_template_text(txt("lookup 9 tail"), txt("<k8><k5><k6><k4>"), slots, 9, txt("V8V5V6V4"));
StrTemplateSlot dup_small[3] = {};
dup_small[0].Key = txt("dup"); dup_small[0].Value = txt("FIRST");
dup_small[1].Key = txt("other"); dup_small[1].Value = txt("X");
dup_small[2].Key = txt("dup"); dup_small[2].Value = txt("SECOND");
expect_template_text(txt("duplicate first wins"), txt("<dup><other>"), dup_small, 3, txt("FIRSTX"));
StrTemplateSlot dup_lanes[4] = {};
dup_lanes[0].Key = txt("same"); dup_lanes[0].Value = txt("A");
dup_lanes[1].Key = txt("same"); dup_lanes[1].Value = txt("B");
dup_lanes[2].Key = txt("other"); dup_lanes[2].Value = txt("C");
dup_lanes[3].Key = txt("same"); dup_lanes[3].Value = txt("D");
expect_template_text(txt("duplicate lane first wins"), txt("<same><other>"), dup_lanes, 4, txt("AC"));
StrTemplateSlot dup_tail[5] = {};
dup_tail[0].Key = txt("dup"); dup_tail[0].Value = txt("FIRST");
dup_tail[1].Key = txt("k1"); dup_tail[1].Value = txt("V1");
dup_tail[2].Key = txt("k2"); dup_tail[2].Value = txt("V2");
dup_tail[3].Key = txt("k3"); dup_tail[3].Value = txt("V3");
dup_tail[4].Key = txt("dup"); dup_tail[4].Value = txt("TAIL");
expect_template_text(txt("duplicate tail first wins"), txt("<dup>"), dup_tail, 5, txt("FIRST"));
}
namespace gen {
internal void expect_template_boundaries() {
ssize marks[12] = { 7, 8, 9, 15, 16, 17, 23, 24, 25, 31, 32, 33 };
for (ssize idx = 0; idx < count_of(marks); ++ idx) {
expect_literal_boundary(marks[idx]);
expect_open_boundary(marks[idx]);
expect_close_boundary(marks[idx]);
}
}
}
internal void expect_template_growth(gen::ssize printf_maxlen)
{
using namespace gen;
ssize huge_len = printf_maxlen + 64;
char* huge = rcast(char*, alloc(heap(), huge_len + 1));
expect(huge != nullptr, txt("growth alloc"));
if (huge == nullptr) return;
mem_set(huge, 'Q', huge_len);
huge[huge_len] = '!';
Str value = { huge, huge_len };
StrTemplateSlot slot = {};
slot.Key = txt("blob");
slot.Value = value;
Str format = txt("<blob>");
StrBuilder output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_capacity(output) == 1, txt("growth initial capacity"));
expect(strbuilder_append_str(& output, txt("KEEP")), txt("growth prefix"));
Slice_StrTemplateSlot slice = { & slot, 1 };
bool appended = strbuilder_append_template(& output, slice, format);
expect(appended, txt("growth append"));
Str got = strbuilder_to_str(output);
expect(got.Len == 4 + huge_len, txt("growth length"));
expect(got.Len > printf_maxlen, txt("growth beyond printf max"));
expect(str_starts_with(got, txt("KEEP")), txt("growth keeps prefix"));
ssize capacity = strbuilder_capacity(output);
expect(capacity >= got.Len, txt("growth capacity"));
if (got.Len == 4 + huge_len && capacity >= got.Len) {
b32 payload_matches = true;
for (ssize idx = 0; idx < got.Len - 4; ++ idx) {
if (got.Ptr[4 + idx] != 'Q') payload_matches = false;
}
expect(payload_matches, txt("growth bytes"));
}
StrBuilderHeader* header = strbuilder_get_header(output);
expect(header->Allocator.Proc == heap_allocator_proc, txt("growth allocator"));
expect(huge[huge_len] == '!', txt("growth sentinel"));
strbuilder_free(& output);
allocator_free(heap(), huge);
}
internal void expect_template_crc_collision()
{
using namespace gen;
Str plumless = txt("plumless");
Str buckeroo = txt("buckeroo");
u32 plum_hash = crc32(plumless.Ptr, plumless.Len);
u32 buck_hash = crc32(buckeroo.Ptr, buckeroo.Len);
expect(plum_hash == buck_hash, txt("crc collision equal"));
expect(plum_hash == 0x4ddb0c25, txt("crc collision value"));
expect(str_are_equal(plumless, buckeroo) == false, txt("crc collision keys differ"));
StrTemplateSlot scalar[2] = {};
scalar[0].Key = plumless; scalar[0].Value = txt("WRONG");
scalar[1].Key = buckeroo; scalar[1].Value = txt("RIGHT");
expect_template_text(txt("crc scalar 2"), txt("<buckeroo>"), scalar, 2, txt("RIGHT"));
StrTemplateSlot lanes[4] = {};
lanes[0].Key = plumless; lanes[0].Value = txt("WRONG");
lanes[1].Key = txt("k1"); lanes[1].Value = txt("V1");
lanes[2].Key = buckeroo; lanes[2].Value = txt("RIGHT");
lanes[3].Key = txt("k3"); lanes[3].Value = txt("V3");
expect_template_text(txt("crc lane reject"), txt("<buckeroo>"), lanes, 4, txt("RIGHT"));
StrTemplateSlot tail[5] = {};
tail[0].Key = plumless; tail[0].Value = txt("WRONG");
tail[1].Key = txt("k1"); tail[1].Value = txt("V1");
tail[2].Key = txt("k2"); tail[2].Value = txt("V2");
tail[3].Key = txt("k3"); tail[3].Value = txt("V3");
tail[4].Key = buckeroo; tail[4].Value = txt("TAIL");
expect_template_text(txt("crc tail after false hit"), txt("<buckeroo>"), tail, 5, txt("TAIL"));
}
internal
void* resize_fail_allocator_proc(void* allocator_data, gen::AllocType type, gen::ssize size, gen::ssize alignment
, void* old_memory, gen::ssize old_size
, gen::u64 flags
){
if (type == gen::EAllocation_RESIZE) {
gen::s32* attempts = rcast(gen::s32*, allocator_data);
attempts[0] = attempts[0] + 1;
return nullptr;
}
return gen::heap_allocator_proc(nullptr, type, size, alignment, old_memory, old_size, flags);
}
internal void expect_template_resize_fail(gen::ssize printf_maxlen)
{
using namespace gen;
ssize huge_len = printf_maxlen + 64;
char* huge = rcast(char*, alloc(heap(), huge_len + 1));
expect(huge != nullptr, txt("resize fail alloc"));
if (huge == nullptr) return;
mem_set(huge, 'L', huge_len);
huge[huge_len] = '!';
Str huge_text = { huge, huge_len };
s32 literal_attempts = 0;
AllocatorInfo literal_alloc = {};
literal_alloc.Proc = resize_fail_allocator_proc;
literal_alloc.Data = & literal_attempts;
StrBuilder literal_output = strbuilder_make_reserve(literal_alloc, 4);
expect(strbuilder_append_str(& literal_output, txt("KEEP")), txt("resize fail literal prefix"));
Slice_StrTemplateSlot literal_slots = { nullptr, 0 };
bool literal_appended = strbuilder_append_template(& literal_output, literal_slots, huge_text);
expect(literal_appended == false, txt("resize fail literal"));
Str literal_got = strbuilder_to_str(literal_output);
expect(literal_got.Len == 4, txt("resize fail literal length"));
expect(str_are_equal(literal_got, txt("KEEP")), txt("resize fail literal keep"));
expect(literal_attempts > 0, txt("resize fail literal attempts"));
strbuilder_free(& literal_output);
s32 value_attempts = 0;
AllocatorInfo value_alloc = {};
value_alloc.Proc = resize_fail_allocator_proc;
value_alloc.Data = & value_attempts;
StrTemplateSlot slot = {};
slot.Key = txt("blob");
slot.Value = huge_text;
Str format = txt("<blob>");
StrBuilder value_output = strbuilder_make_reserve(value_alloc, 4);
expect(strbuilder_append_str(& value_output, txt("KEEP")), txt("resize fail value prefix"));
Slice_StrTemplateSlot value_slots = { & slot, 1 };
bool value_appended = strbuilder_append_template(& value_output, value_slots, format);
expect(value_appended == false, txt("resize fail value"));
Str value_got = strbuilder_to_str(value_output);
expect(value_got.Len == 4, txt("resize fail value length"));
expect(str_are_equal(value_got, txt("KEEP")), txt("resize fail value keep"));
expect(value_attempts > 0, txt("resize fail value attempts"));
strbuilder_free(& value_output);
allocator_free(heap(), huge);
}
internal gen::Str counted_template_value (gen::s32* reads) { reads[0] = reads[0] + 1; return txt("ONCE"); }
internal gen::Str counted_template_format(gen::s32* reads) { reads[0] = reads[0] + 1; return txt("<k>:<k>"); }
internal gen::StrBuilder* counted_template_output(gen::StrBuilder* output, gen::s32* reads) { reads[0] = reads[0] + 1; return output; }
internal bool* counted_template_result(bool* result, gen::s32* reads) { reads[0] = reads[0] + 1; return result; }
internal void expect_template_slice() {
using namespace gen;
StrTemplateSlot slots[2] = {
strtmpl_slot("a", txt("X")),
strtmpl_slot("b", txt("Y"))
};
Slice_StrTemplateSlot slice = { slots, 2 };
StrBuilder output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_append_str(& output, txt("KEEP")), txt("slice prefix"));
bool appended = strbuilder_append_template(& output, slice, txt("<b><a><b>"));
expect(appended, txt("slice append"));
expect(str_are_equal(strbuilder_to_str(output), txt("KEEPYXY")), txt("slice text"));
expect(str_are_equal(slots[0].Key, txt("a")), txt("slice key"));
expect(str_are_equal(slots[0].Value, txt("X")), txt("slice value"));
strbuilder_free(& output);
slice.Ptr[0].Value = txt("Z");
StrBuilder mutated_output = strbuilder_make_reserve(heap(), 1);
bool mutated_appended = strbuilder_append_template(& mutated_output, slice, txt("<b><a><b>"));
expect(mutated_appended, txt("slice mutate append"));
expect(str_are_equal(strbuilder_to_str(mutated_output), txt("YZY")), txt("slice mutate text"));
strbuilder_free(& mutated_output);
Slice_StrTemplateSlot empty_slice = { nullptr, 0 };
StrBuilder empty_output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_append_str(& empty_output, txt("KEEP")), txt("zero slice prefix"));
bool empty_appended = strbuilder_append_template(& empty_output, empty_slice, txt("none"));
expect(empty_appended, txt("zero slice append"));
expect(str_are_equal(strbuilder_to_str(empty_output), txt("KEEPnone")), txt("zero slice text"));
expect(empty_slice.Ptr == nullptr, txt("zero slice pointer"));
expect(empty_slice.Len == 0, txt("zero slice length"));
strbuilder_free(& empty_output);
}
internal void expect_template_wrapper() {
using namespace gen;
StrBuilder output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_append_str(& output, txt("KEEP")), txt("wrapper prefix"));
bool appended = false;
strtmpl_append(& output, appended,
strtmpl_slots(
strtmpl_slot("name", txt("world")),
strtmpl_slot("tail", txt("!"))
),
txt("hello <name>\n<name><tail>")
);
expect(appended, txt("wrapper multiline result"));
Str got = strbuilder_to_str(output);
expect(str_starts_with(got, txt("KEEP")), txt("wrapper prefix kept"));
expect(str_are_equal (got, txt("KEEPhello world\nworld!")), txt("wrapper multiline text"));
strbuilder_free(& output);
}
internal void expect_template_wrapper_scope()
{
using namespace gen;
StrBuilder then_output = strbuilder_make_reserve(heap(), 1);
StrBuilder else_output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_append_str(& then_output, txt("KEEP")), txt("wrapper then prefix"));
expect(strbuilder_append_str(& else_output, txt("KEEP")), txt("wrapper else prefix"));
bool then_result = false;
bool else_result = false;
s32 selector = 1;
s32 after_branch = 0;
if (selector == 1) strtmpl_append(& then_output, then_result, strtmpl_slots(strtmpl_slot("arm", txt("THEN"))), txt("<arm>"));
else strtmpl_append(& else_output, else_result, strtmpl_slots(strtmpl_slot("arm", txt("ELSE"))), txt("<arm>"));
after_branch = 1;
expect(after_branch == 1, txt("wrapper then continues"));
expect(then_result, txt("wrapper then assigned"));
expect(else_result == false, txt("wrapper else skipped"));
expect(str_are_equal(strbuilder_to_str(then_output), txt("KEEPTHEN")), txt("wrapper then text"));
expect(str_are_equal(strbuilder_to_str(else_output), txt("KEEP")), txt("wrapper else untouched"));
strbuilder_free(& then_output);
strbuilder_free(& else_output);
then_output = strbuilder_make_reserve(heap(), 1);
else_output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_append_str(& then_output, txt("KEEP")), txt("wrapper skipped then prefix"));
expect(strbuilder_append_str(& else_output, txt("KEEP")), txt("wrapper taken else prefix"));
then_result = false;
else_result = false;
selector = 0;
after_branch = 0;
if (selector == 1) strtmpl_append(& then_output, then_result, strtmpl_slots(strtmpl_slot("arm", txt("THEN"))), txt("<arm>"));
else strtmpl_append(& else_output, else_result, strtmpl_slots(strtmpl_slot("arm", txt("ELSE"))), txt("<arm>"));
after_branch = 1;
expect(after_branch == 1, txt("wrapper else continues"));
expect(then_result == false, txt("wrapper then skipped"));
expect(else_result, txt("wrapper else assigned"));
expect(str_are_equal(strbuilder_to_str(then_output), txt("KEEP")), txt("wrapper then untouched"));
expect(str_are_equal(strbuilder_to_str(else_output), txt("KEEPELSE")), txt("wrapper else text"));
strbuilder_free(& then_output);
strbuilder_free(& else_output);
}
internal void expect_template_wrapper_once() {
using namespace gen;
s32 value_reads = 0;
s32 format_reads = 0;
s32 output_reads = 0;
s32 result_reads = 0;
StrBuilder output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_append_str(& output, txt("KEEP")), txt("wrapper once prefix"));
bool appended = false;
strtmpl_append(
counted_template_output(& output, & output_reads),
counted_template_result(& appended, & result_reads)[0],
strtmpl_slots(
strtmpl_slot("k", counted_template_value(& value_reads)),
strtmpl_slot("tail", txt("Z"))
),
counted_template_format(& format_reads)
);
expect(appended, txt("wrapper once result"));
expect(value_reads == 1, txt("wrapper value once"));
expect(format_reads == 1, txt("wrapper format once"));
expect(output_reads == 1, txt("wrapper output once"));
expect(result_reads == 1, txt("wrapper result once"));
expect(str_are_equal(strbuilder_to_str(output), txt("KEEPONCE:ONCE")), txt("wrapper once text"));
strbuilder_free(& output);
}
internal void expect_template_wrapper_resize_fail() {
using namespace gen;
s32 attempts = 0;
AllocatorInfo alloc = {};
alloc.Proc = resize_fail_allocator_proc;
alloc.Data = & attempts;
StrBuilder output = strbuilder_make_reserve(alloc, 4);
expect(strbuilder_capacity(output) == 4, txt("wrapper resize capacity"));
expect(strbuilder_append_str(& output, txt("KEEP")), txt("wrapper resize prefix"));
bool appended = true;
strtmpl_append(
& output,
appended,
strtmpl_slots(strtmpl_slot("blob", txt("X"))),
txt("<blob>")
);
expect(appended == false, txt("wrapper resize assigned false"));
expect(str_are_equal(strbuilder_to_str(output), txt("KEEP")), txt("wrapper resize keep"));
expect(attempts > 0, txt("wrapper resize attempts"));
strbuilder_free(& output);
}
internal void expect_template_slot_literal()
{
using namespace gen;
Str value = txt("V");
StrTemplateSlot slot = strtmpl_slot("k", value);
expect(str_are_equal(slot.Key, txt("k")), txt("slot literal key text"));
expect(slot.Key.Ptr[slot.Key.Len] == '\0', txt("slot literal key terminator"));
expect(slot.Value.Ptr == value.Ptr, txt("slot literal value pointer"));
expect(slot.Value.Len == value.Len, txt("slot literal value length"));
Str assigned = txt("W");
slot = strtmpl_slot("key", assigned);
expect(str_are_equal(slot.Key, txt("key")), txt("slot assign key"));
expect(slot.Value.Ptr == assigned.Ptr, txt("slot assign value pointer"));
expect(slot.Value.Len == assigned.Len, txt("slot assign value length"));
expect_template_text(txt("slot assign expansion"), txt("<key>"), & slot, 1, txt("W"));
StrTemplateSlot row[2] = {
strtmpl_slot("a", txt("X")),
strtmpl_slot("b", txt("Y"))
};
expect(str_are_equal(row[0].Value, txt("X")), txt("slot array value"));
expect(str_are_equal(row[1].Key, txt("b")), txt("slot array key"));
expect_template_text(txt("slot array entry"), txt("<b><a>"), row, 2, txt("YX"));
StrBuilder output = strbuilder_make_reserve(heap(), 1);
expect(strbuilder_append_str(& output, txt("KEEP")), txt("slot wrapper prefix"));
bool appended = false;
strtmpl_append(
& output,
appended,
strtmpl_slots(
strtmpl_slot("name", txt("world")),
strtmpl_slot("tail", txt("!"))
),
txt("<name><tail>")
);
expect(appended, txt("slot wrapper result"));
expect(str_are_equal(strbuilder_to_str(output), txt("KEEPworld!")), txt("slot wrapper text"));
strbuilder_free(& output);
StrTemplateSlot long_key = strtmpl_slot("operands[0]", txt("eax"));
expect(str_are_equal(long_key.Key, txt("operands[0]")), txt("slot literal key equal"));
expect(long_key.Key.Ptr[long_key.Key.Len] == '\0', txt("slot literal key excludes nul"));
expect(str_are_equal(long_key.Value, txt("eax")), txt("slot literal value"));
s32 value_reads = 0;
StrTemplateSlot once = strtmpl_slot("k", counted_template_value(& value_reads));
expect(value_reads == 1, txt("slot value once"));
expect(str_are_equal(once.Value, txt("ONCE")), txt("slot value once text"));
expect(once.Key.Len == 1, txt("slot value once key"));
char nul_bytes[4] = { 'a', '\0', 'b', '!' };
Str nul_value = { nul_bytes, 3 };
StrTemplateSlot nul_slot = strtmpl_slot("n", nul_value);
expect(nul_slot.Value.Ptr == nul_bytes, txt("slot nul pointer"));
expect(nul_slot.Value.Len == 3, txt("slot nul length"));
expect(nul_slot.Value.Ptr[1] == '\0', txt("slot nul byte"));
char nul_expected[5] = { '[', 'a', '\0', 'b', ']' };
Str nul_expansion = { nul_expected, 5 };
expect_template_text(txt("slot nul expansion"), txt("[<n>]"), & nul_slot, 1, nul_expansion);
Str angle_value = txt("a<b>c");
StrTemplateSlot angle_slot = strtmpl_slot("raw", angle_value);
expect(angle_slot.Value.Ptr == angle_value.Ptr, txt("slot angle pointer"));
expect(angle_slot.Value.Len == angle_value.Len, txt("slot angle length"));
expect_template_text(txt("slot angle expansion"), txt("<raw>"), & angle_slot, 1, txt("a<b>c"));
Str empty_value = { nullptr, 0 };
StrTemplateSlot empty_slot = strtmpl_slot("e", empty_value);
expect(empty_slot.Value.Ptr == nullptr, txt("slot null value pointer"));
expect(empty_slot.Value.Len == 0, txt("slot null value length"));
expect_template_text(txt("slot null value expansion"), txt("A<e>B"), & empty_slot, 1, txt("AB"));
Str percent_value = txt("%d:%S");
StrTemplateSlot percent_slot = strtmpl_slot("fmt", percent_value);
expect(percent_slot.Value.Ptr == percent_value.Ptr, txt("slot percent value pointer"));
expect(str_are_equal(percent_slot.Value, txt("%d:%S")), txt("slot percent value text"));
expect_template_text(txt("slot percent literal"), txt("pre %d:%S <fmt>"), & percent_slot, 1, txt("pre %d:%S %d:%S"));
}
internal void expect_text_template(gen::ssize printf_maxlen) {
expect_template_shapes();
expect_template_lookup();
expect_template_crc_collision();
gen::expect_template_boundaries();
expect_template_growth(printf_maxlen);
expect_template_resize_fail(printf_maxlen);
expect_template_slice();
expect_template_wrapper();
expect_template_wrapper_scope();
expect_template_wrapper_once();
expect_template_wrapper_resize_fail();
expect_template_slot_literal();
}
+950 -15
View File
@@ -1,29 +1,964 @@
#define GEN_DONT_USE_FATAL
#define GEN_IMPLEMENTATION
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.hpp"
global int g_failures = 0;
internal void expect(int condition, gen::Str name) {
if (condition == false) { gen::log_fmt("FAIL %S\n", name); ++ g_failures; }
}
internal void expect_fragments(gen::Str text, gen::Str label) {
expect(text.Ptr != nullptr && text.Len > 0, label);
expect(gen::str_contains(text, txt("baseline_message")), label);
expect(gen::str_contains(text, txt("char")), label);
expect(gen::str_contains(text, txt("const")), label);
expect(gen::str_contains(text, txt("*")), label);
expect(gen::str_contains(text, txt("GENCPP_BASELINE")), label);
}
internal void expect_variable(gen::CodeVar var, gen::Str label) {
expect(var.ast != nullptr, label); if (var.ast == nullptr) return;
expect(var->Type == gen::CT_Variable, label);
expect(gen::str_contains(var->Name, txt("baseline_message")), label);
expect(var->ValueType.ast != nullptr, label);
expect(var->Value.ast != nullptr, label);
if (var->Value.ast != nullptr) {
expect(gen::str_contains(var->Value->Content, txt("GENCPP_BASELINE")), label);
}
}
internal void expect_parse_result(gen::Code result, gen::ParseInfo* info, gen::CodeType expected, gen::Str label) {
expect(result.ast != nullptr, label);
if (result.ast != nullptr) expect(result->Type == expected, label);
expect(info->result.ast == result.ast, label);
expect(info->lexed.tokens.num > 0, label);
expect(info->lexed.messages == nullptr && info->messages == nullptr, label);
}
#include "dependencies/strings.cpp"
#include "dependencies/simd.cpp"
int main()
{
using namespace gen;
Context ctx = {};
gen::init( & ctx);
Context ctx = {}; init(&ctx);
expect_text_template(GEN_PRINTF_MAXLEN);
expect_simd_lanes();
Builder src_hello = builder_open("gen/hello.cpp");
CodeVar hello_var = parse_variable( code(
constexpr char const* hello_gencpp_str = "HELLO GENCPP !";
));
src_hello.print(hello_var);
src_hello.write();
Opts_def_type type_opts = {};
type_opts.specifiers = def_specifiers(args(Spec_Const, Spec_Ptr));
CodeTypename type = def_type(name(char), type_opts);
Opts_def_variable opts = {}; opts.value = untyped_str(code("\"GENCPP_BASELINE\""));
CodeVar upfront = def_variable(type, name(baseline_message), opts);
expect_variable(upfront, txt("cpp upfront"));
expect_fragments(strbuilder_to_str(code_to_strbuilder((Code)upfront)), txt("cpp upfront serialized"));
CodeBody body = parse_file("gen/hello.cpp");
for ( Code code : body ) switch (code->Type) {
case CT_Variable:
CodeVar var = cast(CodeVar, code);
log_fmt("%S", var->Value);
break;
Opts_def_type type_options = {};
Code array_expr = untyped_str(txt("3"));
CodeSpecifiers type_specs = def_specifier(Spec_Const);
CodeAttributes type_attrs = def_attributes(txt("[[maybe_unused]]"));
type_options.type_tag = Tag_Struct;
type_options.array_expr = array_expr;
type_options.specifiers = type_specs;
type_options.attributes = type_attrs;
CodeTypename option_type = def_type(name(Payload), type_options);
expect(option_type.ast != nullptr, txt("cpp type options"));
if (option_type.ast != nullptr) {
expect(option_type->TypeTag == Tag_Struct, txt("cpp type tag"));
expect(option_type->ArrExpr.ast == array_expr.ast, txt("cpp type array expression"));
expect(option_type->Specs.ast == type_specs.ast, txt("cpp type specifiers"));
expect(option_type->Attributes.ast == type_attrs.ast, txt("cpp type attributes"));
Str type_text = strbuilder_to_str(code_to_strbuilder((Code)option_type));
expect(str_contains(type_text, txt("struct Payload")), txt("cpp type tag serialization"));
expect(str_contains(type_text, txt("const")), txt("cpp type specifier serialization"));
expect(str_contains(type_text, txt("[[maybe_unused]]")), txt("cpp type attribute serialization"));
}
gen::deinit(& ctx);
Opts_def_variable variable_options = {};
Code value = untyped_str(code("\"UPFRONT_OPTION\""));
CodeSpecifiers variable_specs = def_specifier(Spec_Static);
CodeAttributes variable_attrs = def_attributes(txt("[[maybe_unused]]"));
variable_options.value = value;
variable_options.specifiers = variable_specs;
variable_options.attributes = variable_attrs;
variable_options.mflags = ModuleFlag_Export;
CodeVar option_variable = def_variable(option_type, name(option_payload), variable_options);
expect(option_variable.ast != nullptr, txt("cpp variable options"));
if (option_variable.ast != nullptr) {
expect(option_variable->ValueType.ast == option_type.ast, txt("cpp variable type"));
expect(option_variable->Value.ast == value.ast, txt("cpp variable value"));
expect(option_variable->Specs.ast == variable_specs.ast, txt("cpp variable specifiers"));
expect(option_variable->Attributes.ast == variable_attrs.ast, txt("cpp variable attributes"));
expect(option_variable->ModuleFlags == ModuleFlag_Export, txt("cpp variable module flags"));
Str variable_text = strbuilder_to_str(code_to_strbuilder((Code)option_variable));
expect(str_contains(variable_text, txt("option_payload")), txt("cpp variable name serialization"));
expect(str_contains(variable_text, txt("UPFRONT_OPTION")), txt("cpp variable value serialization"));
expect(str_contains(variable_text, txt("static")), txt("cpp variable specifier serialization"));
expect(str_contains(variable_text, txt("[[maybe_unused]]")), txt("cpp variable attribute serialization"));
expect(str_contains(variable_text, txt("[ 3 ]")), txt("cpp variable array serialization"));
expect(str_contains(variable_text, txt("export")), txt("cpp variable module flag serialization"));
}
CodeVar class_member = def_variable(def_type(name(u32)), name(class_option_member));
CodeBody class_body = def_class_body(args(class_member));
CodeTypename class_parent = def_type(name(ClassOptionBase));
CodeTypename class_interfaces[] = { def_type(name(IClassOption)) };
Opts_def_struct class_options = {};
class_options.body = class_body;
class_options.parent = class_parent;
class_options.parent_access = AccessSpec_Public;
class_options.attributes = def_attributes(txt("[[maybe_unused]]"));
class_options.interfaces = class_interfaces;
class_options.num_interfaces = 1;
class_options.specifiers = def_specifier(Spec_Final);
class_options.mflags = ModuleFlag_Export;
CodeClass option_class = def_class(name(ClassOption), class_options);
expect(option_class.ast != nullptr, txt("cpp class options"));
if (option_class.ast != nullptr) {
expect(option_class->Body.ast == class_body.ast, txt("cpp class body"));
expect(option_class->ParentType.ast == class_parent.ast, txt("cpp class parent"));
expect(option_class->ParentAccess == AccessSpec_Public, txt("cpp class parent access"));
expect(option_class->Attributes.ast == class_options.attributes.ast, txt("cpp class attributes"));
expect(option_class->ParentType->Next.ast == ((Code)class_interfaces[0]).ast, txt("cpp class interface"));
expect(option_class->Specs.ast == class_options.specifiers.ast, txt("cpp class specifiers"));
expect(option_class->ModuleFlags == ModuleFlag_Export, txt("cpp class module flags"));
Str class_text = strbuilder_to_str(code_to_strbuilder((Code)option_class));
expect(str_contains(class_text, txt("ClassOption")), txt("cpp class name serialization"));
expect(str_contains(class_text, txt("ClassOptionBase")), txt("cpp class parent serialization"));
expect(str_contains(class_text, txt("IClassOption")), txt("cpp class interface serialization"));
expect(str_contains(class_text, txt("public")), txt("cpp class access serialization"));
expect(str_contains(class_text, txt("class_option_member")), txt("cpp class body serialization"));
expect(str_contains(class_text, txt("[[maybe_unused]]")), txt("cpp class attribute serialization"));
expect(str_contains(class_text, txt("final")), txt("cpp class final specifier serialization"));
expect(str_contains(class_text, txt("export")), txt("cpp class module flag serialization"));
}
CodeVar struct_member = def_variable(def_type(name(u32)), name(struct_option_member));
CodeBody struct_body = def_struct_body(args(struct_member));
CodeTypename struct_parent = def_type(name(StructOptionBase));
CodeTypename struct_interfaces[] = { def_type(name(IStructOption)) };
Opts_def_struct struct_options = {};
struct_options.body = struct_body;
struct_options.parent = struct_parent;
struct_options.parent_access = AccessSpec_Public;
struct_options.attributes = def_attributes(txt("[[maybe_unused]]"));
struct_options.interfaces = struct_interfaces;
struct_options.num_interfaces = 1;
struct_options.specifiers = def_specifier(Spec_Final);
struct_options.mflags = ModuleFlag_Export;
CodeStruct option_struct = def_struct(name(StructOption), struct_options);
expect(option_struct.ast != nullptr, txt("cpp struct options"));
if (option_struct.ast != nullptr) {
expect(option_struct->Body.ast == struct_body.ast, txt("cpp struct body"));
expect(option_struct->ParentType.ast == struct_parent.ast, txt("cpp struct parent"));
expect(option_struct->ParentAccess == AccessSpec_Public, txt("cpp struct parent access"));
expect(option_struct->Attributes.ast == struct_options.attributes.ast, txt("cpp struct attributes"));
expect(option_struct->ParentType->Next.ast == ((Code)struct_interfaces[0]).ast, txt("cpp struct interface"));
expect(option_struct->Specs.ast == struct_options.specifiers.ast, txt("cpp struct specifiers"));
expect(option_struct->ModuleFlags == ModuleFlag_Export, txt("cpp struct module flags"));
Str struct_text = strbuilder_to_str(code_to_strbuilder((Code)option_struct));
expect(str_contains(struct_text, txt("StructOption")), txt("cpp struct name serialization"));
expect(str_contains(struct_text, txt("StructOptionBase")), txt("cpp struct parent serialization"));
expect(str_contains(struct_text, txt("IStructOption")), txt("cpp struct interface serialization"));
expect(str_contains(struct_text, txt("public")), txt("cpp struct access serialization"));
expect(str_contains(struct_text, txt("struct_option_member")), txt("cpp struct body serialization"));
expect(str_contains(struct_text, txt("[[maybe_unused]]")), txt("cpp struct attribute serialization"));
expect(str_contains(struct_text, txt("final")), txt("cpp struct final specifier serialization"));
expect(str_contains(struct_text, txt("export")), txt("cpp struct module flag serialization"));
}
CodeBody enum_body = def_enum_body(args(untyped_str(txt("Option_None"))));
Opts_def_enum enum_options = {};
enum_options.body = enum_body;
enum_options.type = def_type(name(u32));
enum_options.specifier = EnumDecl_Class;
enum_options.attributes = def_attributes(txt("[[maybe_unused]]"));
enum_options.mflags = ModuleFlag_Export;
CodeEnum option_enum = def_enum(name(OptionEnum), enum_options);
expect(option_enum.ast != nullptr, txt("cpp enum options"));
if (option_enum.ast != nullptr) {
expect(option_enum->Body.ast == enum_body.ast, txt("cpp enum body"));
expect(option_enum->UnderlyingType.ast == enum_options.type.ast, txt("cpp enum underlying type"));
expect(option_enum->Type == CT_Enum_Class, txt("cpp enum specifier"));
expect(option_enum->Attributes.ast == enum_options.attributes.ast, txt("cpp enum attributes"));
expect(option_enum->ModuleFlags == ModuleFlag_Export, txt("cpp enum module flags"));
Str enum_text = strbuilder_to_str(code_to_strbuilder((Code)option_enum));
expect(str_contains(enum_text, txt("export")), txt("cpp enum module flag serialization"));
expect(str_contains(enum_text, txt("enum class")), txt("cpp enum specifier serialization"));
expect(str_contains(enum_text, txt("[[maybe_unused]]")), txt("cpp enum attribute serialization"));
expect(str_contains(enum_text, txt("u32")), txt("cpp enum type serialization"));
expect(str_contains(enum_text, txt("Option_None")), txt("cpp enum body serialization"));
}
Code enum_macro = untyped_str(txt("ENUM_BASE_TYPE"));
CodeEnum macro_enum = def_enum(name(MacroOptionEnum), {
.body = def_enum_body(args(untyped_str(txt("Macro_None")))),
.type_macro = enum_macro,
});
expect(macro_enum.ast != nullptr, txt("cpp enum type macro option"));
if (macro_enum.ast != nullptr) {
expect(macro_enum->UnderlyingTypeMacro.ast == enum_macro.ast, txt("cpp enum type macro"));
expect(str_contains(strbuilder_to_str(code_to_strbuilder((Code)macro_enum)), txt("ENUM_BASE_TYPE")), txt("cpp enum type macro serialization"));
}
CodeBody union_body = def_union_body(args(untyped_str(txt("u32 union_option_value;"))));
Opts_def_union union_options = {};
union_options.attributes = def_attributes(txt("[[maybe_unused]]"));
union_options.mflags = ModuleFlag_Export;
CodeUnion option_union = def_union(name(OptionUnion), union_body, union_options);
expect(option_union.ast != nullptr, txt("cpp union options"));
if (option_union.ast != nullptr) {
expect(option_union->Body.ast == union_body.ast, txt("cpp union body"));
expect(option_union->Attributes.ast == union_options.attributes.ast, txt("cpp union attributes"));
expect(option_union->ModuleFlags == ModuleFlag_Export, txt("cpp union module flags"));
Str union_text = strbuilder_to_str(code_to_strbuilder((Code)option_union));
expect(str_contains(union_text, txt("OptionUnion")), txt("cpp union name serialization"));
expect(str_contains(union_text, txt("union_option_value")), txt("cpp union body serialization"));
expect(str_contains(union_text, txt("[[maybe_unused]]")), txt("cpp union attribute serialization"));
expect(str_contains(union_text, txt("export")), txt("cpp union module flag serialization"));
}
CodeAttributes typedef_attrs = def_attributes(txt("[[maybe_unused]]"));
Code typedef_type = (Code)def_type(name(u32));
Opts_def_typedef typedef_options = {};
typedef_options.attributes = typedef_attrs;
typedef_options.mflags = ModuleFlag_Export;
CodeTypedef option_typedef = def_typedef(name(OptionTypedef), typedef_type, typedef_options);
expect(option_typedef.ast != nullptr, txt("cpp typedef options"));
if (option_typedef.ast != nullptr) {
expect(option_typedef->UnderlyingType.ast == typedef_type.ast, txt("cpp typedef underlying type"));
expect(option_typedef->Attributes.ast == typedef_attrs.ast, txt("cpp typedef attributes"));
expect(option_typedef->ModuleFlags == ModuleFlag_Export, txt("cpp typedef module flags"));
Str typedef_text = strbuilder_to_str(code_to_strbuilder((Code)option_typedef));
expect(str_contains(typedef_text, txt("OptionTypedef")), txt("cpp typedef name serialization"));
expect(str_contains(typedef_text, txt("u32")), txt("cpp typedef type serialization"));
expect(str_contains(typedef_text, txt("[[maybe_unused]]")), txt("cpp typedef attribute serialization"));
expect(str_contains(typedef_text, txt("export")), txt("cpp typedef module flag serialization"));
}
Opts_def_using using_options = {};
using_options.attributes = def_attributes(txt("[[maybe_unused]]"));
using_options.mflags = ModuleFlag_Export;
CodeTypename using_type = def_type(name(u32));
CodeUsing option_using = def_using(name(OptionUsing), using_type, using_options);
expect(option_using.ast != nullptr, txt("cpp using options"));
if (option_using.ast != nullptr) {
expect(option_using->UnderlyingType.ast == using_type.ast, txt("cpp using underlying type"));
expect(option_using->Attributes.ast == using_options.attributes.ast, txt("cpp using attributes"));
expect(option_using->ModuleFlags == ModuleFlag_Export, txt("cpp using module flags"));
Str using_text = strbuilder_to_str(code_to_strbuilder((Code)option_using));
expect(str_contains(using_text, txt("OptionUsing")), txt("cpp using name serialization"));
expect(str_contains(using_text, txt("[[maybe_unused]]")), txt("cpp using attribute serialization"));
expect(str_contains(using_text, txt("export")), txt("cpp using module flag serialization"));
}
CodeParams template_params = def_param(def_type(name(class)), name(T));
CodeTemplate option_template = def_template(template_params, (Code)option_struct, { .mflags = ModuleFlag_Export });
expect(option_template.ast != nullptr, txt("cpp template options"));
if (option_template.ast != nullptr) {
expect(option_template->Params.ast == template_params.ast, txt("cpp template parameters"));
expect(option_template->Declaration.ast == ((Code)option_struct).ast, txt("cpp template declaration"));
expect(option_template->ModuleFlags == ModuleFlag_Export, txt("cpp template module flags"));
Str template_text = strbuilder_to_str(code_to_strbuilder((Code)option_template));
expect(str_contains(template_text, txt("export")), txt("cpp template module flag serialization"));
expect(str_contains(template_text, txt("template")), txt("cpp template serialization"));
expect(str_contains(template_text, txt("StructOption")), txt("cpp template declaration serialization"));
}
Macro source_macro = { txt("OptionParamSource"), MT_Expression, MF_Functional };
register_macro(source_macro);
CodeDefine parsed_macro_source = parse_define(txt("#define OptionParamSource(param) param\n"));
expect(parsed_macro_source.ast != nullptr, txt("cpp define parameter source"));
Opts_def_define define_options = {};
define_options.params = parsed_macro_source->Params;
define_options.content = txt("param + 1");
define_options.flags = MF_Functional | MF_Allow_As_Identifier;
CodeDefine option_define = def_define(name(OptionMacro), MT_Expression, define_options);
expect(option_define.ast != nullptr, txt("cpp define options"));
if (option_define.ast != nullptr) {
expect(option_define->Params.ast == define_options.params.ast, txt("cpp define parameters"));
expect(str_contains(option_define->Body->Content, txt("param + 1")), txt("cpp define content"));
Macro* option_macro = lookup_macro(name(OptionMacro));
expect(option_macro != nullptr, txt("cpp define registration"));
if (option_macro != nullptr) expect(option_macro->Flags == define_options.flags, txt("cpp define flags"));
Str define_text = strbuilder_to_str(code_to_strbuilder((Code)option_define));
expect(str_contains(define_text, txt("OptionMacro")), txt("cpp define name serialization"));
expect(str_contains(define_text, txt("OptionMacro( param)")), txt("cpp define parameters serialization"));
expect(str_contains(define_text, txt("param + 1")), txt("cpp define content serialization"));
}
Opts_def_define no_register_options = {};
no_register_options.dont_register_to_preprocess_macros = true;
CodeDefine no_register_define = def_define(name(OptionMacroNoRegister), MT_Expression, no_register_options);
expect(no_register_define.ast != nullptr, txt("cpp define registration suppression"));
expect(lookup_macro(name(OptionMacroNoRegister)) == nullptr, txt("cpp define suppression flag"));
CodeInclude local_include = def_include(txt("option_local.h"));
expect(local_include.ast != nullptr, txt("cpp local include option"));
if (local_include.ast != nullptr) {
expect(str_contains(local_include->Content, txt("\"option_local.h\"")), txt("cpp quoted include"));
}
Opts_def_include foreign_include_options = {};
foreign_include_options.foreign = true;
CodeInclude foreign_include = def_include(txt("option_system.h"), foreign_include_options);
expect(foreign_include.ast != nullptr, txt("cpp foreign include option"));
if (foreign_include.ast != nullptr) {
expect(str_contains(foreign_include->Content, txt("<option_system.h>")), txt("cpp foreign include path"));
Str include_text = strbuilder_to_str(code_to_strbuilder((Code)foreign_include));
expect(str_contains(include_text, txt("#include <option_system.h>")), txt("cpp foreign include directive serialization"));
}
CodeModule option_module = def_module(name(OptionModule), { .mflags = ModuleFlag_Export });
expect(option_module.ast != nullptr, txt("cpp module options"));
if (option_module.ast != nullptr) {
expect(option_module->ModuleFlags == ModuleFlag_Export, txt("cpp module flags"));
expect(str_contains(strbuilder_to_str(code_to_strbuilder((Code)option_module)), txt("export")), txt("cpp module serialization"));
}
CodeVar namespace_member = def_variable(def_type(name(u32)), name(namespace_option_value));
CodeBody namespace_body = def_namespace_body(args(namespace_member));
CodeNS option_namespace = def_namespace(name(OptionNamespace), namespace_body, { .mflags = ModuleFlag_Export });
expect(option_namespace.ast != nullptr, txt("cpp namespace options"));
if (option_namespace.ast != nullptr) {
expect(option_namespace->Body.ast == namespace_body.ast, txt("cpp namespace body"));
expect(option_namespace->ModuleFlags == ModuleFlag_Export, txt("cpp namespace flags"));
Str namespace_text = strbuilder_to_str(code_to_strbuilder((Code)option_namespace));
expect(str_contains(namespace_text, txt("OptionNamespace")), txt("cpp namespace name serialization"));
expect(str_contains(namespace_text, txt("namespace_option_value")), txt("cpp namespace body serialization"));
expect(str_contains(namespace_text, txt("export")), txt("cpp namespace serialization"));
}
Code parameter_value = untyped_str(code(7));
Opts_def_param parameter_options = {};
parameter_options.value = parameter_value;
CodeParams option_parameter = def_param(def_type(name(u32)), name(option_parameter_value), parameter_options);
expect(option_parameter.ast != nullptr, txt("cpp parameter options"));
if (option_parameter.ast != nullptr) {
expect(option_parameter->Value.ast == parameter_value.ast, txt("cpp parameter value"));
Str parameter_text = strbuilder_to_str(code_to_strbuilder((Code)option_parameter));
expect(str_contains(parameter_text, txt("= 7")), txt("cpp parameter value serialization"));
}
CodeParams function_params = def_param(def_type(name(u32)), name(function_option_arg));
CodeTypename function_return = def_type(name(FunctionOptionReturn));
CodeBody function_body = def_function_body(args(untyped_str(txt("return function_option_arg;"))));
Opts_def_function function_options = {};
function_options.params = function_params;
function_options.ret_type = function_return;
function_options.body = function_body;
function_options.specs = def_specifier(Spec_Inline);
function_options.attrs = def_attributes(txt("[[maybe_unused]]"));
function_options.mflags = ModuleFlag_Export;
CodeFn option_function = def_function(name(OptionFunction), function_options);
expect(option_function.ast != nullptr, txt("cpp function options"));
if (option_function.ast != nullptr) {
expect(option_function->Params.ast == function_params.ast, txt("cpp function parameters"));
expect(option_function->ReturnType.ast == function_return.ast, txt("cpp function return type"));
expect(option_function->Body.ast == function_body.ast, txt("cpp function body"));
expect(option_function->Specs.ast == function_options.specs.ast, txt("cpp function specifiers"));
expect(option_function->Attributes.ast == function_options.attrs.ast, txt("cpp function attributes"));
expect(option_function->ModuleFlags == ModuleFlag_Export, txt("cpp function module flags"));
Str function_text = strbuilder_to_str(code_to_strbuilder((Code)option_function));
expect(str_contains(function_text, txt("OptionFunction")), txt("cpp function name serialization"));
expect(str_contains(function_text, txt("FunctionOptionReturn OptionFunction(")), txt("cpp function return type serialization"));
expect(str_contains(function_text, txt("function_option_arg")), txt("cpp function parameters serialization"));
expect(str_contains(function_text, txt("return function_option_arg")), txt("cpp function body serialization"));
expect(str_contains(function_text, txt("inline")), txt("cpp function specifier serialization"));
expect(str_contains(function_text, txt("[[maybe_unused]]")), txt("cpp function attribute serialization"));
expect(str_contains(function_text, txt("export")), txt("cpp function module flag serialization"));
}
CodeParams constructor_params = def_param(def_type(name(u32)), name(constructor_option_arg));
Code initializer_list = untyped_str(txt("option_member( 0 )"));
CodeBody constructor_body = def_function_body(args(untyped_str(txt("option_member = constructor_option_arg;"))));
Opts_def_constructor constructor_options = {};
constructor_options.params = constructor_params;
constructor_options.initializer_list = initializer_list;
constructor_options.body = constructor_body;
CodeConstructor option_constructor = def_constructor(constructor_options);
expect(option_constructor.ast != nullptr, txt("cpp constructor options"));
if (option_constructor.ast != nullptr) {
expect(option_constructor->Params.ast == constructor_params.ast, txt("cpp constructor parameters"));
expect(option_constructor->InitializerList.ast == initializer_list.ast, txt("cpp constructor initializer list"));
expect(option_constructor->Body.ast == ((Code)constructor_body).ast, txt("cpp constructor body"));
}
CodeBody constructor_class_body = def_class_body(args(option_constructor));
CodeClass constructor_class = def_class(name(ConstructorOptions), { .body = constructor_class_body });
expect(constructor_class.ast != nullptr, txt("cpp constructor serialization parent"));
if (constructor_class.ast != nullptr) {
Str constructor_text = strbuilder_to_str(code_to_strbuilder((Code)constructor_class));
expect(str_contains(constructor_text, txt("ConstructorOptions(")), txt("cpp constructor name serialization"));
expect(str_contains(constructor_text, txt("u32 constructor_option_arg")), txt("cpp constructor parameter serialization"));
expect(str_contains(constructor_text, txt("option_member( 0 )")), txt("cpp constructor initializer serialization"));
expect(str_contains(constructor_text, txt("constructor_option_arg")), txt("cpp constructor body serialization"));
}
Code destructor_body = untyped_str(txt("release_option_resource();"));
CodeSpecifiers destructor_specs = def_specifier(Spec_Virtual);
Opts_def_destructor destructor_options = {};
destructor_options.body = destructor_body;
destructor_options.specifiers = destructor_specs;
CodeDestructor option_destructor = def_destructor(destructor_options);
expect(option_destructor.ast != nullptr, txt("cpp destructor options"));
if (option_destructor.ast != nullptr) {
expect(option_destructor->Body.ast == destructor_body.ast, txt("cpp destructor body"));
expect(option_destructor->Specs.ast == destructor_specs.ast, txt("cpp destructor specifiers"));
CodeBody destructor_class_body = def_class_body(args(option_destructor));
CodeClass destructor_class = def_class(name(DestructorOptions), { .body = destructor_class_body });
expect(destructor_class.ast != nullptr, txt("cpp destructor serialization parent"));
if (destructor_class.ast != nullptr) {
Str destructor_text = strbuilder_to_str(code_to_strbuilder((Code)destructor_class));
expect(str_contains(destructor_text, txt("virtual ~DestructorOptions()")), txt("cpp destructor signature serialization"));
expect(str_contains(destructor_text, txt("release_option_resource();")), txt("cpp destructor body serialization"));
}
}
CodeTypename operator_type = def_type(name(OperatorOptionType));
CodeTypename operator_return = def_type(name(OperatorOptionReturn));
CodeParams operator_lhs = def_param(operator_type, name(operator_option_lhs));
CodeParams operator_rhs = def_param(operator_type, name(operator_option_rhs));
CodeParams operator_params = def_params(args(operator_lhs, operator_rhs));
CodeBody operator_body = def_function_body(args(untyped_str(txt("return operator_option_lhs;"))));
Opts_def_operator operator_options = {};
operator_options.params = operator_params;
operator_options.ret_type = operator_return;
operator_options.body = operator_body;
operator_options.specifiers = def_specifier(Spec_Inline);
operator_options.attributes = def_attributes(txt("[[maybe_unused]]"));
operator_options.mflags = ModuleFlag_Export;
CodeOperator option_operator = def_operator(Op_Add, txt(""), operator_options);
expect(option_operator.ast != nullptr, txt("cpp operator options"));
if (option_operator.ast != nullptr) {
expect(option_operator->Params.ast == operator_params.ast, txt("cpp operator parameters"));
expect(option_operator->ReturnType.ast == operator_return.ast, txt("cpp operator return type"));
expect(option_operator->Body.ast == operator_body.ast, txt("cpp operator body"));
expect(option_operator->Specs.ast == operator_options.specifiers.ast, txt("cpp operator specifiers"));
expect(option_operator->Attributes.ast == operator_options.attributes.ast, txt("cpp operator attributes"));
expect(option_operator->ModuleFlags == ModuleFlag_Export, txt("cpp operator module flags"));
Str operator_text = strbuilder_to_str(code_to_strbuilder((Code)option_operator));
expect(str_contains(operator_text, txt("OperatorOptionReturn")), txt("cpp operator return type serialization"));
expect(str_contains(operator_text, txt("operator +")), txt("cpp operator name serialization"));
expect(str_contains(operator_text, txt("operator_option_lhs")), txt("cpp operator parameters serialization"));
expect(str_contains(operator_text, txt("operator_option_rhs")), txt("cpp operator second parameter serialization"));
expect(str_contains(operator_text, txt("return operator_option_lhs")), txt("cpp operator body serialization"));
expect(str_contains(operator_text, txt("inline")), txt("cpp operator specifier serialization"));
expect(str_contains(operator_text, txt("[[maybe_unused]]")), txt("cpp operator attribute serialization"));
expect(str_contains(operator_text, txt("export")), txt("cpp operator module flag serialization"));
if (!str_starts_with(operator_text, txt("export [[maybe_unused]] inline\nOperatorOptionReturn"))) log_fmt("%S\n", operator_text);
expect(str_starts_with(operator_text, txt("export [[maybe_unused]] inline\nOperatorOptionReturn")), txt("cpp operator one-attribute prefix serialization"));
}
CodeTypename cast_type = def_type(name(s32));
CodeBody cast_body = def_function_body(args(untyped_str(txt("return 0;"))));
CodeSpecifiers cast_specs = def_specifier(Spec_Const);
Opts_def_operator_cast cast_options = {};
cast_options.body = cast_body;
cast_options.specs = cast_specs;
CodeOpCast option_cast = def_operator_cast(cast_type, cast_options);
expect(option_cast.ast != nullptr, txt("cpp operator cast options"));
if (option_cast.ast != nullptr) {
expect(option_cast->ValueType.ast == cast_type.ast, txt("cpp operator cast type"));
expect(option_cast->Body.ast == cast_body.ast, txt("cpp operator cast body"));
expect(option_cast->Specs.ast == cast_specs.ast, txt("cpp operator cast specifiers"));
Str cast_text = strbuilder_to_str(code_to_strbuilder((Code)option_cast));
expect(str_contains(cast_text, txt("operator s32")), txt("cpp operator cast serialization"));
expect(str_contains(cast_text, txt("const")), txt("cpp operator cast specifier serialization"));
expect(str_contains(cast_text, txt("return 0")), txt("cpp operator cast body serialization"));
}
Code untyped = code_str(char const* baseline_message = "GENCPP_BASELINE";);
Str untyped_text = strbuilder_to_str(code_to_strbuilder(untyped));
CodeVar parsed_untyped = parse_variable(untyped_text);
expect(untyped.ast != nullptr, txt("cpp untyped"));
expect_fragments(untyped_text, txt("cpp untyped content"));
expect_variable(parsed_untyped, txt("cpp untyped reparsed"));
CodeVar parsed = parse_variable(code(char const* baseline_message = "GENCPP_BASELINE"; ));
expect_variable(parsed, txt("cpp parsed"));
expect_fragments(strbuilder_to_str(code_to_strbuilder((Code)parsed)), txt("cpp parsed serialized"));
CodeClass parser_class = parse_class(txt("class ParseClass {};"));
CodeConstructor parser_constructor = parse_constructor(txt("ParseClass() {}"));
CodeDefine parser_define = parse_define(txt("#define PARSE_DEFINE 1"));
CodeDestructor parser_destructor = parse_destructor(txt("~ParseClass();"));
CodeEnum parser_enum = parse_enum(txt("enum ParseEnum { ParseValue };"));
CodeBody parser_export_body = parse_export_body(txt("{ int export_value; }"));
CodeExtern parser_extern = parse_extern_link(txt("extern \"C\" { int extern_value; }"));
CodeFriend parser_friend = parse_friend(txt("friend int parse_friend_function();"));
CodeFn parser_function = parse_function(txt("int parse_function();"));
CodeBody parser_global_body = parse_global_body(txt("struct ParseGlobal { int value; };"));
CodeNS parser_namespace = parse_namespace(txt("namespace ParseNamespace {}"));
CodeOperator parser_operator = parse_operator(txt("s32 operator +(s32 lhs, s32 rhs);"));
CodeOpCast parser_operator_cast = parse_operator_cast(txt("operator s32() const;"));
CodeStruct parser_struct = parse_struct(txt("struct ParseStruct {};"));
CodeTemplate parser_template = parse_template(txt("template<class T> struct ParseTemplate {};"));
CodeTypename parser_type = parse_type(txt("const s32*"));
CodeTypedef parser_typedef = parse_typedef(txt("typedef s32 ParseTypedef;"));
CodeUnion parser_union = parse_union(txt("union ParseUnion { s32 value; };"));
CodeUsing parser_using = parse_using(txt("using ParseUsing = s32;"));
CodeVar parser_variable = parse_variable(txt("s32 parse_variable;"));
expect(parser_class.ast != nullptr && parser_class->Type == CT_Class, txt("cpp parse class fixture"));
expect(parser_constructor.ast != nullptr && parser_constructor->Type == CT_Constructor, txt("cpp parse constructor fixture"));
expect(parser_define.ast != nullptr && parser_define->Type == CT_Preprocess_Define, txt("cpp parse define fixture"));
expect(parser_destructor.ast != nullptr && parser_destructor->Type == CT_Destructor_Fwd, txt("cpp parse destructor fixture"));
expect(parser_enum.ast != nullptr && parser_enum->Type == CT_Enum, txt("cpp parse enum fixture"));
expect(parser_export_body.ast != nullptr && parser_export_body->Type == CT_Export_Body, txt("cpp parse export body fixture"));
expect(parser_extern.ast != nullptr && parser_extern->Type == CT_Extern_Linkage, txt("cpp parse extern fixture"));
expect(parser_friend.ast != nullptr && parser_friend->Type == CT_Friend, txt("cpp parse friend fixture"));
expect(parser_function.ast != nullptr && parser_function->Type == CT_Function_Fwd, txt("cpp parse function fixture"));
expect(parser_global_body.ast != nullptr && parser_global_body->Type == CT_Global_Body, txt("cpp parse global body fixture"));
expect(parser_namespace.ast != nullptr && parser_namespace->Type == CT_Namespace, txt("cpp parse namespace fixture"));
expect(parser_operator.ast != nullptr && parser_operator->Type == CT_Operator_Fwd, txt("cpp parse operator fixture"));
expect(parser_operator_cast.ast != nullptr && parser_operator_cast->Type == CT_Operator_Cast_Fwd, txt("cpp parse operator cast fixture"));
expect(parser_struct.ast != nullptr && parser_struct->Type == CT_Struct, txt("cpp parse struct fixture"));
expect(parser_template.ast != nullptr && parser_template->Type == CT_Template, txt("cpp parse template fixture"));
expect(parser_type.ast != nullptr && parser_type->Type == CT_Typename, txt("cpp parse type fixture"));
expect(parser_typedef.ast != nullptr && parser_typedef->Type == CT_Typedef, txt("cpp parse typedef fixture"));
expect(parser_union.ast != nullptr && parser_union->Type == CT_Union, txt("cpp parse union fixture"));
expect(parser_using.ast != nullptr && parser_using->Type == CT_Using, txt("cpp parse using fixture"));
expect(parser_variable.ast != nullptr && parser_variable->Type == CT_Variable, txt("cpp parse variable fixture"));
CodeBody parsed_global_options = parse_global_body(txt("struct ParseGlobalOptions { s32 value; };"));
Code global_options_code = parsed_global_options;
expect(parsed_global_options.ast != nullptr, txt("cpp parse global body options"));
expect(ctx.parse_info.result.ast == global_options_code.ast, txt("cpp default parse info result"));
expect(ctx.parse_info.lexed.tokens.num > 0, txt("cpp default parse info tokens"));
expect(ctx.parse_info.lexed.messages == nullptr && ctx.parse_info.messages == nullptr, txt("cpp default parse info messages"));
ParseInfo caller_info = {};
Opts_parse parse_options = { & ctx, & caller_info };
CodeVar parsed_option_variable = parse_variable(txt("s32 parsed_option_variable;"), parse_options);
Code parsed_option_code = parsed_option_variable;
expect(parsed_option_variable.ast != nullptr && parsed_option_variable->Type == CT_Variable, txt("cpp explicit parse variable options"));
expect(str_contains(parsed_option_variable->Name, txt("parsed_option_variable")), txt("cpp explicit parse variable name"));
expect(caller_info.result.ast == parsed_option_code.ast, txt("cpp explicit parse info result"));
expect(caller_info.lexed.tokens.num > 0, txt("cpp explicit parse info tokens"));
ParseInfo lexer_info = {};
Opts_parse lexer_options = { & ctx, & lexer_info };
CodeVar parsed_lexer_prefix = parse_variable(txt("s32 parsed_lexer_prefix;@"), lexer_options);
Code parsed_lexer_code = parsed_lexer_prefix;
expect(parsed_lexer_prefix.ast != nullptr && parsed_lexer_prefix->Type == CT_Variable, txt("cpp parse lexer prefix result"));
expect(lexer_info.result.ast == parsed_lexer_code.ast, txt("cpp parse lexer prefix info result"));
expect(lexer_info.lexed.tokens.num > 0, txt("cpp parse lexer prefix retained tokens"));
expect(lexer_info.lexed.messages != nullptr, txt("cpp parse lexer prefix diagnostic"));
expect_parse_result(parse_class (txt("class ParseClass {};"), parse_options), & caller_info, CT_Class, txt("cpp class options matrix"));
expect_parse_result(parse_constructor (txt("ParseClass() {}"), parse_options), & caller_info, CT_Constructor, txt("cpp constructor options matrix"));
expect_parse_result(parse_define (txt("#define PARSE_DEFINE 1"), parse_options), & caller_info, CT_Preprocess_Define, txt("cpp define options matrix"));
expect_parse_result(parse_destructor (txt("~ParseClass();"), parse_options), & caller_info, CT_Destructor_Fwd, txt("cpp destructor options matrix"));
expect_parse_result(parse_enum (txt("enum ParseEnum { ParseValue };"), parse_options), & caller_info, CT_Enum, txt("cpp enum options matrix"));
expect_parse_result(parse_export_body (txt("{ int export_value; }"), parse_options), & caller_info, CT_Export_Body, txt("cpp export body options matrix"));
expect_parse_result(parse_extern_link (txt("extern \"C\" { int extern_value; }"), parse_options), & caller_info, CT_Extern_Linkage, txt("cpp extern options matrix"));
expect_parse_result(parse_friend (txt("friend int parse_friend_function();"), parse_options), & caller_info, CT_Friend, txt("cpp friend options matrix"));
expect_parse_result(parse_function (txt("int parse_function();"), parse_options), & caller_info, CT_Function_Fwd, txt("cpp function options matrix"));
expect_parse_result(parse_global_body (txt("struct ParseGlobal { int value; };"), parse_options), & caller_info, CT_Global_Body, txt("cpp global body options matrix"));
expect_parse_result(parse_namespace (txt("namespace ParseNamespace {}"), parse_options), & caller_info, CT_Namespace, txt("cpp namespace options matrix"));
expect_parse_result(parse_operator (txt("s32 operator +(s32 lhs, s32 rhs);"), parse_options), & caller_info, CT_Operator_Fwd, txt("cpp operator options matrix"));
expect_parse_result(parse_operator_cast(txt("operator s32() const;"), parse_options), & caller_info, CT_Operator_Cast_Fwd, txt("cpp operator cast options matrix"));
expect_parse_result(parse_struct (txt("struct ParseStruct {};"), parse_options), & caller_info, CT_Struct, txt("cpp struct options matrix"));
expect_parse_result(parse_template (txt("template<class T> struct ParseTemplate {};"), parse_options), & caller_info, CT_Template, txt("cpp template options matrix"));
expect_parse_result(parse_type (txt("const s32*"), parse_options), & caller_info, CT_Typename, txt("cpp type options matrix"));
expect_parse_result(parse_typedef (txt("typedef s32 ParseTypedef;"), parse_options), & caller_info, CT_Typedef, txt("cpp typedef options matrix"));
expect_parse_result(parse_union (txt("union ParseUnion { s32 value; };"), parse_options), & caller_info, CT_Union, txt("cpp union options matrix"));
expect_parse_result(parse_using (txt("using ParseUsing = s32;"), parse_options), & caller_info, CT_Using, txt("cpp using options matrix"));
expect_parse_result(parse_variable (txt("s32 parse_variable;"), parse_options), & caller_info, CT_Variable, txt("cpp variable options matrix"));
ParseInfo comma_info = {};
Opts_parse comma_options = { &ctx, &comma_info };
CodeBody parsed_comma_body = parse_global_body(txt("struct ParseComma { s32 value; };\n,"), comma_options);
Code parsed_comma_code = parsed_comma_body;
expect(parsed_comma_body.ast != nullptr && parsed_comma_body->Type == CT_Global_Body, txt("cpp comma parser result"));
expect(comma_info.result.ast == parsed_comma_code.ast, txt("cpp comma ParseInfo result"));
expect(comma_info.lexed.tokens.num > 0 && comma_info.lexed.messages == nullptr, txt("cpp comma retained tokens"));
expect(comma_info.messages != nullptr, txt("cpp comma parser diagnostic"));
if (comma_info.messages != nullptr)
expect(str_contains(comma_info.messages->Content, txt("Dangling comma")), txt("cpp comma diagnostic content"));
ParseInfo recovery_info = {};
Opts_parse recovery_options = { &ctx, &recovery_info };
CodeBody recovered_body = parse_global_body(txt("s32 before; s32 broken(s32 value; s32 after;"), recovery_options);
Str recovery_text = { nullptr, 0 };
if (recovered_body.ast != nullptr)
recovery_text = strbuilder_to_str(code_to_strbuilder((Code)recovered_body));
expect(recovered_body.ast != nullptr && recovered_body->Type == CT_Global_Body, txt("cpp file-scope recovery body"));
expect(recovery_info.result.ast == ((Code)recovered_body).ast, txt("cpp file-scope recovery ParseInfo result"));
expect(recovery_info.messages != nullptr, txt("cpp file-scope recovery diagnostic"));
if (recovery_info.messages != nullptr)
expect(str_contains(recovery_info.messages->Content, txt("Expected")), txt("cpp file-scope recovery mismatch text"));
if (recovery_info.messages != nullptr)
expect(recovery_info.messages->Next == nullptr, txt("cpp file-scope recovery single diagnostic"));
expect(str_contains(recovery_text, txt("before")), txt("cpp file-scope recovery keeps preceding member"));
expect(str_contains(recovery_text, txt("after")), txt("cpp file-scope recovery keeps following member"));
Code recovered_member = { nullptr };
if (recovered_body.ast != nullptr) recovered_member = recovered_body->Front;
Code recovered_before = recovered_member;
expect(recovered_before != nullptr && recovered_before->Type == CT_Variable, txt("cpp mismatch recovery keeps preceding member typed"));
if (recovered_before)
expect(str_are_equal(recovered_before->Name, txt("before")), txt("cpp mismatch recovery preceding member name"));
if (recovered_member) recovered_member = recovered_member->Next;
expect(recovered_member != nullptr && recovered_member->Type == CT_Untyped, txt("cpp file-scope recovery stores malformed member as untyped"));
if (recovered_member)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(recovered_member)), txt("s32 broken(s32 value;")), txt("cpp file-scope recovery retains exact mismatch span"));
Code recovered_after = recovered_member ? recovered_member->Next : Code{ nullptr };
expect(recovered_after != nullptr && recovered_after->Type == CT_Variable, txt("cpp file-scope recovery keeps following member typed"));
if (recovered_after)
expect(str_are_equal(recovered_after->Name, txt("after")), txt("cpp file-scope recovery following member name"));
ParseInfo invalid_member_info = {};
Opts_parse invalid_member_options = { &ctx, &invalid_member_info };
CodeBody invalid_member_body = parse_global_body(txt("s32 before; = { 1; 2 }; s32 after;"), invalid_member_options);
Str invalid_member_text = { nullptr, 0 };
if (invalid_member_body.ast != nullptr)
invalid_member_text = strbuilder_to_str(code_to_strbuilder((Code)invalid_member_body));
expect(invalid_member_body.ast != nullptr && invalid_member_body->Type == CT_Global_Body, txt("cpp invalid member recovery body"));
expect(invalid_member_info.result.ast == ((Code)invalid_member_body).ast, txt("cpp invalid member recovery ParseInfo result"));
expect(invalid_member_info.messages != nullptr, txt("cpp invalid member recovery diagnostic"));
if (invalid_member_info.messages != nullptr) {
expect(str_contains(invalid_member_info.messages->Content, txt("Failed to parse member")), txt("cpp invalid member recovery diagnostic text"));
expect(invalid_member_info.messages->Next == nullptr, txt("cpp invalid member recovery single diagnostic"));
}
expect(str_contains(invalid_member_text, txt("before")), txt("cpp invalid member recovery keeps preceding member"));
expect(str_contains(invalid_member_text, txt("after")), txt("cpp invalid member recovery keeps following member"));
Code invalid_member_node = { nullptr };
if (invalid_member_body.ast != nullptr) invalid_member_node = invalid_member_body->Front;
Code invalid_member_before = invalid_member_node;
expect(invalid_member_before != nullptr && invalid_member_before->Type == CT_Variable, txt("cpp invalid member keeps preceding member typed"));
if (invalid_member_before)
expect(str_are_equal(invalid_member_before->Name, txt("before")), txt("cpp invalid member preceding member name"));
if (invalid_member_node) invalid_member_node = invalid_member_node->Next;
expect(invalid_member_node != nullptr && invalid_member_node->Type == CT_Untyped, txt("cpp invalid member recovery stores member as untyped"));
if (invalid_member_node)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(invalid_member_node)), txt("= { 1; 2 };")), txt("cpp invalid member recovery retains exact source span"));
Code invalid_member_after = invalid_member_node ? invalid_member_node->Next : Code{ nullptr };
expect(invalid_member_after != nullptr && invalid_member_after->Type == CT_Variable, txt("cpp invalid member recovery keeps following member typed"));
if (invalid_member_after)
expect(str_are_equal(invalid_member_after->Name, txt("after")), txt("cpp invalid member recovery following member name"));
ParseInfo import_recovery_info = {};
Opts_parse import_recovery_options = { &ctx, &import_recovery_info };
CodeBody import_recovery_body = parse_global_body(txt("s32 before; import foo; s32 after;"), import_recovery_options);
Code import_recovery_node = { nullptr };
if (import_recovery_body.ast != nullptr) import_recovery_node = import_recovery_body->Front;
Code import_recovery_before = import_recovery_node;
expect(import_recovery_before != nullptr && import_recovery_before->Type == CT_Variable, txt("cpp import recovery keeps preceding member typed"));
if (import_recovery_before)
expect(str_are_equal(import_recovery_before->Name, txt("before")), txt("cpp import recovery preceding member name"));
if (import_recovery_node) import_recovery_node = import_recovery_node->Next;
expect(import_recovery_body.ast != nullptr && import_recovery_body->Type == CT_Global_Body, txt("cpp import recovery body"));
expect(import_recovery_info.messages != nullptr, txt("cpp import recovery diagnostic"));
if (import_recovery_info.messages != nullptr)
expect(import_recovery_info.messages->Next == nullptr, txt("cpp import recovery single diagnostic"));
expect(import_recovery_node != nullptr && import_recovery_node->Type == CT_Untyped, txt("cpp import recovery preserves member as untyped"));
if (import_recovery_node)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(import_recovery_node)), txt("import foo;")), txt("cpp import recovery exact source span"));
Code import_recovery_after = import_recovery_node ? import_recovery_node->Next : Code{ nullptr };
expect(import_recovery_after != nullptr && import_recovery_after->Type == CT_Variable, txt("cpp import recovery keeps following member typed"));
if (import_recovery_after)
expect(str_are_equal(import_recovery_after->Name, txt("after")), txt("cpp import recovery following member name"));
Macro unbounded_macro = { txt("UnboundedRecoveryMacro"), MT_Expression, MF_Null };
register_macro(unbounded_macro);
ParseInfo macro_recovery_info = {};
Opts_parse macro_recovery_options = { &ctx, &macro_recovery_info };
CodeBody macro_recovery_body = parse_global_body(txt("s32 before; UnboundedRecoveryMacro; s32 after;"), macro_recovery_options);
Code macro_recovery_node = { nullptr };
if (macro_recovery_body.ast != nullptr) macro_recovery_node = macro_recovery_body->Front;
Code macro_recovery_before = macro_recovery_node;
expect(macro_recovery_before != nullptr && macro_recovery_before->Type == CT_Variable, txt("cpp macro recovery keeps preceding member typed"));
if (macro_recovery_before)
expect(str_are_equal(macro_recovery_before->Name, txt("before")), txt("cpp macro recovery preceding member name"));
if (macro_recovery_node) macro_recovery_node = macro_recovery_node->Next;
expect(macro_recovery_body.ast != nullptr && macro_recovery_body->Type == CT_Global_Body, txt("cpp unbounded macro recovery body"));
expect(macro_recovery_info.messages != nullptr, txt("cpp unbounded macro recovery diagnostic"));
if (macro_recovery_info.messages != nullptr)
expect(macro_recovery_info.messages->Next == nullptr, txt("cpp unbounded macro recovery single diagnostic"));
expect(macro_recovery_node != nullptr && macro_recovery_node->Type == CT_Untyped, txt("cpp unbounded macro recovery preserves member as untyped"));
if (macro_recovery_node)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(macro_recovery_node)), txt("UnboundedRecoveryMacro;")), txt("cpp unbounded macro recovery exact source span"));
Code macro_recovery_after = macro_recovery_node ? macro_recovery_node->Next : Code{ nullptr };
expect(macro_recovery_after != nullptr && macro_recovery_after->Type == CT_Variable, txt("cpp unbounded macro recovery keeps following member typed"));
if (macro_recovery_after)
expect(str_are_equal(macro_recovery_after->Name, txt("after")), txt("cpp unbounded macro recovery following member name"));
ParseInfo eof_recovery_info = {};
Opts_parse eof_recovery_options = { &ctx, &eof_recovery_info };
CodeBody eof_recovery_body = parse_global_body(txt("s32 before; s32 broken(s32 value"), eof_recovery_options);
Code eof_recovery_node = { nullptr };
if (eof_recovery_body.ast != nullptr) eof_recovery_node = eof_recovery_body->Front;
Code eof_recovery_before = eof_recovery_node;
expect(eof_recovery_before != nullptr && eof_recovery_before->Type == CT_Variable, txt("cpp EOF recovery keeps preceding member typed"));
if (eof_recovery_before)
expect(str_are_equal(eof_recovery_before->Name, txt("before")), txt("cpp EOF recovery preceding member name"));
if (eof_recovery_node) eof_recovery_node = eof_recovery_node->Next;
expect(eof_recovery_body.ast != nullptr && eof_recovery_body->Type == CT_Global_Body, txt("cpp end-of-slice recovery body"));
expect(eof_recovery_info.messages != nullptr, txt("cpp end-of-slice recovery diagnostic"));
if (eof_recovery_info.messages != nullptr)
expect(eof_recovery_info.messages->Next == nullptr, txt("cpp end-of-slice recovery single diagnostic"));
expect(eof_recovery_node != nullptr && eof_recovery_node->Type == CT_Untyped, txt("cpp end-of-slice member is untyped"));
if (eof_recovery_node)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(eof_recovery_node)), txt("s32 broken(s32 value")), txt("cpp end-of-slice recovery exact source span"));
if (eof_recovery_node)
expect(eof_recovery_node->Next == nullptr, txt("cpp end-of-slice recovery has no following node"));
ParseInfo comma_recovery_info = {};
Opts_parse comma_recovery_options = { &ctx, &comma_recovery_info };
CodeBody comma_recovery_body = parse_global_body(txt("s32 before; , s32 after;"), comma_recovery_options);
Code comma_recovery_node = { nullptr };
if (comma_recovery_body.ast != nullptr) comma_recovery_node = comma_recovery_body->Front;
Code comma_recovery_before = comma_recovery_node;
expect(comma_recovery_before != nullptr && comma_recovery_before->Type == CT_Variable, txt("cpp comma recovery keeps preceding member typed"));
if (comma_recovery_before)
expect(str_are_equal(comma_recovery_before->Name, txt("before")), txt("cpp comma recovery preceding member name"));
if (comma_recovery_node) comma_recovery_node = comma_recovery_node->Next;
expect(comma_recovery_body.ast != nullptr && comma_recovery_body->Type == CT_Global_Body, txt("cpp comma recovery body"));
expect(comma_recovery_info.messages != nullptr, txt("cpp comma recovery diagnostic"));
if (comma_recovery_info.messages != nullptr) {
expect(str_contains(comma_recovery_info.messages->Content, txt("Dangling comma")), txt("cpp comma recovery diagnostic text"));
expect(comma_recovery_info.messages->Next == nullptr, txt("cpp comma recovery single diagnostic"));
}
expect(comma_recovery_node != nullptr && comma_recovery_node->Type == CT_Untyped, txt("cpp comma recovery preserves token as untyped"));
if (comma_recovery_node)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(comma_recovery_node)), txt(",")), txt("cpp comma recovery exact source span"));
Code comma_recovery_after = comma_recovery_node ? comma_recovery_node->Next : Code{ nullptr };
expect(comma_recovery_after != nullptr && comma_recovery_after->Type == CT_Variable, txt("cpp comma recovery keeps following member typed"));
if (comma_recovery_after)
expect(str_are_equal(comma_recovery_after->Name, txt("after")), txt("cpp comma recovery following member name"));
CodeBody recovery_reuse = parse_global_body(txt("s32 reused_after_recovery;"), recovery_options);
Str recovery_reuse_text = { nullptr, 0 };
if (recovery_reuse.ast != nullptr)
recovery_reuse_text = strbuilder_to_str(code_to_strbuilder((Code)recovery_reuse));
expect(recovery_reuse.ast != nullptr && recovery_reuse->Type == CT_Global_Body, txt("cpp parser context reuse after recovery"));
expect(recovery_info.messages == nullptr, txt("cpp parser context reuse clears prior diagnostics"));
expect(str_contains(recovery_reuse_text, txt("reused_after_recovery")), txt("cpp parser context reuse parses later input"));
ParseInfo nested_struct_info = {};
Opts_parse nested_struct_options = { &ctx, &nested_struct_info };
CodeStruct nested_struct = parse_struct(txt("struct NestedRecovery { s32 before; s32 broken(s32 value; s32 after; };"), nested_struct_options);
expect(nested_struct.ast != nullptr && nested_struct->Type == CT_Struct, txt("cpp nested struct recovery result"));
expect(nested_struct_info.result.ast == ((Code)nested_struct).ast, txt("cpp nested struct recovery ParseInfo result"));
expect(nested_struct_info.messages != nullptr, txt("cpp nested struct recovery diagnostic"));
if (nested_struct_info.messages != nullptr)
expect(str_contains(nested_struct_info.messages->Content, txt("Expected")), txt("cpp nested struct recovery diagnostic text"));
Code nested_struct_member = nested_struct.ast != nullptr && nested_struct->Body.ast != nullptr ? nested_struct->Body->Front : Code{ nullptr };
expect(nested_struct_member != nullptr && nested_struct_member->Type == CT_Variable, txt("cpp nested struct recovery keeps preceding member typed"));
if (nested_struct_member != nullptr)
expect(str_are_equal(nested_struct_member->Name, txt("before")), txt("cpp nested struct recovery preceding member name"));
if (nested_struct_member != nullptr) nested_struct_member = nested_struct_member->Next;
expect(nested_struct_member != nullptr && nested_struct_member->Type == CT_Untyped, txt("cpp nested struct recovery retains failed member"));
if (nested_struct_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(nested_struct_member)), txt("s32 broken(s32 value;")), txt("cpp nested struct recovery exact failed span"));
Code nested_struct_after = nested_struct_member != nullptr ? nested_struct_member->Next : Code{ nullptr };
expect(nested_struct_after != nullptr && nested_struct_after->Type == CT_Variable, txt("cpp nested struct recovery keeps following member typed"));
if (nested_struct_after != nullptr)
expect(str_are_equal(nested_struct_after->Name, txt("after")), txt("cpp nested struct recovery following member name"));
Str nested_struct_text = { nullptr, 0 };
if (nested_struct.ast != nullptr)
nested_struct_text = strbuilder_to_str(code_to_strbuilder((Code)nested_struct));
expect(str_contains(nested_struct_text, txt("s32 after")), txt("cpp nested struct recovery serializes following member"));
ParseInfo nested_namespace_info = {};
Opts_parse nested_namespace_options = { &ctx, &nested_namespace_info };
CodeNS nested_namespace = parse_namespace(txt("namespace NestedRecovery { s32 before; s32 broken(s32 value; s32 after; }"), nested_namespace_options);
expect(nested_namespace.ast != nullptr && nested_namespace->Type == CT_Namespace, txt("cpp nested namespace recovery result"));
expect(nested_namespace_info.result.ast == ((Code)nested_namespace).ast, txt("cpp nested namespace recovery ParseInfo result"));
expect(nested_namespace_info.messages != nullptr, txt("cpp nested namespace recovery diagnostic"));
if (nested_namespace_info.messages != nullptr)
expect(str_contains(nested_namespace_info.messages->Content, txt("Expected")), txt("cpp nested namespace recovery diagnostic text"));
Code nested_namespace_member = nested_namespace.ast != nullptr && nested_namespace->Body.ast != nullptr ? nested_namespace->Body->Front : Code{ nullptr };
expect(nested_namespace_member != nullptr && nested_namespace_member->Type == CT_Variable, txt("cpp nested namespace recovery keeps preceding member typed"));
if (nested_namespace_member != nullptr)
expect(str_are_equal(nested_namespace_member->Name, txt("before")), txt("cpp nested namespace recovery preceding member name"));
if (nested_namespace_member != nullptr) nested_namespace_member = nested_namespace_member->Next;
expect(nested_namespace_member != nullptr && nested_namespace_member->Type == CT_Untyped, txt("cpp nested namespace recovery retains failed member"));
if (nested_namespace_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(nested_namespace_member)), txt("s32 broken(s32 value;")), txt("cpp nested namespace recovery exact failed span"));
Code nested_namespace_after = nested_namespace_member != nullptr ? nested_namespace_member->Next : Code{ nullptr };
expect(nested_namespace_after != nullptr && nested_namespace_after->Type == CT_Variable, txt("cpp nested namespace recovery keeps following member typed"));
if (nested_namespace_after != nullptr)
expect(str_are_equal(nested_namespace_after->Name, txt("after")), txt("cpp nested namespace recovery following member name"));
Str nested_namespace_text = { nullptr, 0 };
if (nested_namespace.ast != nullptr)
nested_namespace_text = strbuilder_to_str(code_to_strbuilder((Code)nested_namespace));
expect(str_contains(nested_namespace_text, txt("s32 after")), txt("cpp nested namespace recovery serializes following member"));
ParseInfo nested_brace_info = {};
Opts_parse nested_brace_options = { &ctx, &nested_brace_info };
CodeStruct nested_brace_struct = parse_struct(txt("struct NestedBraceRecovery { s32 before; s32 broken(s32 { 1; s32 after; };"), nested_brace_options);
expect(nested_brace_struct.ast != nullptr && nested_brace_struct->Type == CT_Struct, txt("cpp nested brace recovery result"));
expect(nested_brace_info.result.ast == ((Code)nested_brace_struct).ast, txt("cpp nested brace recovery ParseInfo result"));
expect(nested_brace_info.messages != nullptr, txt("cpp nested brace recovery diagnostic"));
Code nested_brace_member = nested_brace_struct.ast != nullptr && nested_brace_struct->Body.ast != nullptr ? nested_brace_struct->Body->Front : Code{ nullptr };
expect(nested_brace_member != nullptr && nested_brace_member->Type == CT_Variable, txt("cpp nested brace recovery keeps preceding member typed"));
if (nested_brace_member != nullptr) nested_brace_member = nested_brace_member->Next;
expect(nested_brace_member != nullptr && nested_brace_member->Type == CT_Untyped, txt("cpp nested brace recovery retains failed member"));
if (nested_brace_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(nested_brace_member)), txt("s32 broken(s32 { 1;")), txt("cpp nested brace recovery exact failed span"));
Code nested_brace_after = nested_brace_member != nullptr ? nested_brace_member->Next : Code{ nullptr };
expect(nested_brace_after != nullptr && nested_brace_after->Type == CT_Variable, txt("cpp nested brace recovery keeps following member typed"));
if (nested_brace_after != nullptr)
expect(str_are_equal(nested_brace_after->Name, txt("after")), txt("cpp nested brace recovery following member name"));
Str nested_brace_text = { nullptr, 0 };
if (nested_brace_struct.ast != nullptr)
nested_brace_text = strbuilder_to_str(code_to_strbuilder((Code)nested_brace_struct));
expect(str_contains(nested_brace_text, txt("s32 after")), txt("cpp nested brace recovery serializes following member"));
ParseInfo nested_namespace_brace_info = {};
Opts_parse nested_namespace_brace_options = { &ctx, &nested_namespace_brace_info };
CodeNS nested_namespace_brace = parse_namespace(txt("namespace NestedBraceRecovery { s32 before; s32 broken(s32 { 1; s32 after; }"), nested_namespace_brace_options);
expect(nested_namespace_brace.ast != nullptr && nested_namespace_brace->Type == CT_Namespace, txt("cpp nested namespace brace recovery result"));
expect(nested_namespace_brace_info.result.ast == ((Code)nested_namespace_brace).ast, txt("cpp nested namespace brace recovery ParseInfo result"));
Code nested_namespace_brace_member = nested_namespace_brace.ast != nullptr && nested_namespace_brace->Body.ast != nullptr ? nested_namespace_brace->Body->Front : Code{ nullptr };
expect(nested_namespace_brace_member != nullptr && nested_namespace_brace_member->Type == CT_Variable, txt("cpp nested namespace brace recovery keeps preceding member typed"));
if (nested_namespace_brace_member != nullptr) nested_namespace_brace_member = nested_namespace_brace_member->Next;
expect(nested_namespace_brace_member != nullptr && nested_namespace_brace_member->Type == CT_Untyped, txt("cpp nested namespace brace recovery retains failed member"));
if (nested_namespace_brace_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(nested_namespace_brace_member)), txt("s32 broken(s32 { 1;")), txt("cpp nested namespace brace recovery exact failed span"));
Code nested_namespace_brace_after = nested_namespace_brace_member != nullptr ? nested_namespace_brace_member->Next : Code{ nullptr };
expect(nested_namespace_brace_after != nullptr && nested_namespace_brace_after->Type == CT_Variable, txt("cpp nested namespace brace recovery keeps following member typed"));
if (nested_namespace_brace_after != nullptr)
expect(str_are_equal(nested_namespace_brace_after->Name, txt("after")), txt("cpp nested namespace brace recovery following member name"));
Str nested_namespace_brace_text = { nullptr, 0 };
if (nested_namespace_brace.ast != nullptr)
nested_namespace_brace_text = strbuilder_to_str(code_to_strbuilder((Code)nested_namespace_brace));
expect(str_contains(nested_namespace_brace_text, txt("s32 after")), txt("cpp nested namespace brace recovery serializes following member"));
ParseInfo nested_brace_tail_info = {};
Opts_parse nested_brace_tail_options = { &ctx, &nested_brace_tail_info };
CodeStruct nested_brace_tail_struct = parse_struct(txt("struct NestedBraceTailRecovery { s32 before; s32 broken(s32 { 1; 2 } foo{ 3; s32 after; };"), nested_brace_tail_options);
expect(nested_brace_tail_struct.ast != nullptr && nested_brace_tail_struct->Type == CT_Struct, txt("cpp nested brace tail recovery result"));
expect(nested_brace_tail_info.result.ast == ((Code)nested_brace_tail_struct).ast, txt("cpp nested brace tail recovery ParseInfo result"));
expect(nested_brace_tail_info.messages != nullptr, txt("cpp nested brace tail recovery diagnostic"));
Code nested_brace_tail_member = nested_brace_tail_struct.ast != nullptr && nested_brace_tail_struct->Body.ast != nullptr ? nested_brace_tail_struct->Body->Front : Code{ nullptr };
if (nested_brace_tail_member != nullptr) nested_brace_tail_member = nested_brace_tail_member->Next;
expect(nested_brace_tail_member != nullptr && nested_brace_tail_member->Type == CT_Untyped, txt("cpp nested brace tail recovery retains failed member"));
if (nested_brace_tail_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(nested_brace_tail_member)), txt("s32 broken(s32 { 1; 2 } foo{ 3;")), txt("cpp nested brace tail recovery exact failed span"));
Code nested_brace_tail_after = nested_brace_tail_member != nullptr ? nested_brace_tail_member->Next : Code{ nullptr };
expect(nested_brace_tail_after != nullptr && nested_brace_tail_after->Type == CT_Variable, txt("cpp nested brace tail recovery keeps following member typed"));
if (nested_brace_tail_after != nullptr)
expect(str_are_equal(nested_brace_tail_after->Name, txt("after")), txt("cpp nested brace tail recovery following member name"));
ParseInfo nested_namespace_tail_info = {};
Opts_parse nested_namespace_tail_options = { &ctx, &nested_namespace_tail_info };
CodeNS nested_namespace_tail = parse_namespace(txt("namespace NestedBraceTailRecovery { s32 before; s32 broken(s32 { 1; 2 } foo{ 3; s32 after; }"), nested_namespace_tail_options);
expect(nested_namespace_tail.ast != nullptr && nested_namespace_tail->Type == CT_Namespace, txt("cpp nested namespace brace tail recovery result"));
expect(nested_namespace_tail_info.result.ast == ((Code)nested_namespace_tail).ast, txt("cpp nested namespace brace tail recovery ParseInfo result"));
Code nested_namespace_tail_member = nested_namespace_tail.ast != nullptr && nested_namespace_tail->Body.ast != nullptr ? nested_namespace_tail->Body->Front : Code{ nullptr };
if (nested_namespace_tail_member != nullptr) nested_namespace_tail_member = nested_namespace_tail_member->Next;
expect(nested_namespace_tail_member != nullptr && nested_namespace_tail_member->Type == CT_Untyped, txt("cpp nested namespace brace tail recovery retains failed member"));
if (nested_namespace_tail_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(nested_namespace_tail_member)), txt("s32 broken(s32 { 1; 2 } foo{ 3;")), txt("cpp nested namespace brace tail recovery exact failed span"));
Code nested_namespace_tail_after = nested_namespace_tail_member != nullptr ? nested_namespace_tail_member->Next : Code{ nullptr };
expect(nested_namespace_tail_after != nullptr && nested_namespace_tail_after->Type == CT_Variable, txt("cpp nested namespace brace tail recovery keeps following member typed"));
if (nested_namespace_tail_after != nullptr)
expect(str_are_equal(nested_namespace_tail_after->Name, txt("after")), txt("cpp nested namespace brace tail recovery following member name"));
ParseInfo nested_inner_close_info = {};
Opts_parse nested_inner_close_options = { &ctx, &nested_inner_close_info };
CodeStruct nested_inner_close_struct = parse_struct(txt("struct NestedInnerCloseRecovery { s32 before; s32 broken(s32 { 1; 2 } foo{ 3; s32 after; struct Inner { s32 x; }; s32 tail; };"), nested_inner_close_options);
expect(nested_inner_close_struct.ast != nullptr && nested_inner_close_struct->Type == CT_Struct, txt("cpp nested inner-close recovery result"));
Code nested_inner_close_member = nested_inner_close_struct.ast != nullptr && nested_inner_close_struct->Body.ast != nullptr ? nested_inner_close_struct->Body->Front : Code{ nullptr };
if (nested_inner_close_member != nullptr) nested_inner_close_member = nested_inner_close_member->Next;
expect(nested_inner_close_member != nullptr && nested_inner_close_member->Type == CT_Untyped, txt("cpp nested inner-close recovery retains failed member"));
if (nested_inner_close_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(nested_inner_close_member)), txt("s32 broken(s32 { 1; 2 } foo{ 3;")), txt("cpp nested inner-close recovery exact failed span"));
Code nested_inner_close_after = nested_inner_close_member != nullptr ? nested_inner_close_member->Next : Code{ nullptr };
expect(nested_inner_close_after != nullptr && nested_inner_close_after->Type == CT_Variable, txt("cpp nested inner-close recovery keeps following member typed"));
if (nested_inner_close_after != nullptr)
expect(str_are_equal(nested_inner_close_after->Name, txt("after")), txt("cpp nested inner-close recovery following member name"));
Code nested_inner_close_inner = nested_inner_close_after != nullptr ? nested_inner_close_after->Next : Code{ nullptr };
expect(nested_inner_close_inner != nullptr && nested_inner_close_inner->Type == CT_Struct, txt("cpp nested inner-close recovery parses nested struct"));
Code nested_inner_close_tail = nested_inner_close_inner != nullptr ? nested_inner_close_inner->Next : Code{ nullptr };
expect(nested_inner_close_tail != nullptr && nested_inner_close_tail->Type == CT_Variable, txt("cpp nested inner-close recovery parses trailing member"));
ParseInfo inner_before_sync_info = {};
Opts_parse inner_before_sync_options = { &ctx, &inner_before_sync_info };
CodeStruct inner_before_sync = parse_struct(txt("struct NestedInnerBeforeSync { s32 before; s32 broken(foo{ struct Inner { s32 x; }; s32 after; };"), inner_before_sync_options);
expect(inner_before_sync.ast != nullptr && inner_before_sync->Type == CT_Struct, txt("cpp inner-before-sync recovery result"));
Code inner_before_sync_member = inner_before_sync.ast != nullptr && inner_before_sync->Body.ast != nullptr ? inner_before_sync->Body->Front : Code{ nullptr };
if (inner_before_sync_member != nullptr) inner_before_sync_member = inner_before_sync_member->Next;
expect(inner_before_sync_member != nullptr && inner_before_sync_member->Type == CT_Untyped, txt("cpp inner-before-sync recovery retains failed member"));
if (inner_before_sync_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(inner_before_sync_member)), txt("s32 broken(foo{ struct Inner { s32 x; };")), txt("cpp inner-before-sync recovery exact failed span"));
Code inner_before_sync_after = inner_before_sync_member != nullptr ? inner_before_sync_member->Next : Code{ nullptr };
expect(inner_before_sync_after != nullptr && inner_before_sync_after->Type == CT_Variable, txt("cpp inner-before-sync recovery keeps following member typed"));
if (inner_before_sync_after != nullptr)
expect(str_are_equal(inner_before_sync_after->Name, txt("after")), txt("cpp inner-before-sync recovery following member name"));
ParseInfo global_namespace_recovery_info = {};
Opts_parse global_namespace_recovery_options = { &ctx, &global_namespace_recovery_info };
CodeBody global_namespace_recovery = parse_global_body(txt("namespace NestedScopeInGlobal { s32 before; s32 broken(s32 { 1; 2 } foo{ 3; s32 after; } struct Later { s32 value; };"), global_namespace_recovery_options);
expect(global_namespace_recovery.ast != nullptr && global_namespace_recovery->Type == CT_Global_Body, txt("cpp global namespace recovery body"));
expect(global_namespace_recovery_info.result.ast == ((Code)global_namespace_recovery).ast, txt("cpp global namespace recovery ParseInfo result"));
Code global_namespace_node = global_namespace_recovery.ast != nullptr ? global_namespace_recovery->Front : Code{ nullptr };
expect(global_namespace_node != nullptr && global_namespace_node->Type == CT_Namespace, txt("cpp global namespace recovery preserves namespace"));
Code global_namespace_member = global_namespace_node != nullptr && global_namespace_node->Body.ast != nullptr ? global_namespace_node->Body->Front : Code{ nullptr };
if (global_namespace_member != nullptr) global_namespace_member = global_namespace_member->Next;
expect(global_namespace_member != nullptr && global_namespace_member->Type == CT_Untyped, txt("cpp global namespace recovery retains failed member"));
if (global_namespace_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(global_namespace_member)), txt("s32 broken(s32 { 1; 2 } foo{ 3;")), txt("cpp global namespace recovery exact failed span"));
Code global_namespace_after = global_namespace_member != nullptr ? global_namespace_member->Next : Code{ nullptr };
expect(global_namespace_after != nullptr && global_namespace_after->Type == CT_Variable, txt("cpp global namespace recovery keeps following member typed"));
if (global_namespace_after != nullptr)
expect(str_are_equal(global_namespace_after->Name, txt("after")), txt("cpp global namespace recovery following member name"));
Code global_later_struct = global_namespace_node != nullptr ? global_namespace_node->Next : Code{ nullptr };
expect(global_later_struct != nullptr && global_later_struct->Type == CT_Struct, txt("cpp global namespace recovery preserves following global struct"));
if (global_later_struct != nullptr)
expect(str_are_equal(global_later_struct->Name, txt("Later")), txt("cpp global namespace recovery following global name"));
ParseInfo global_struct_recovery_info = {};
Opts_parse global_struct_recovery_options = { &ctx, &global_struct_recovery_info };
CodeBody global_struct_recovery = parse_global_body(txt("struct GlobalNestedRecovery { s32 before; s32 broken(s32 value; s32 after; }; struct LaterGlobal { s32 value; };"), global_struct_recovery_options);
expect(global_struct_recovery.ast != nullptr && global_struct_recovery->Type == CT_Global_Body, txt("cpp global struct recovery body"));
Code global_struct_node = global_struct_recovery.ast != nullptr ? global_struct_recovery->Front : Code{ nullptr };
Code global_struct_before = global_struct_node != nullptr && global_struct_node->Body.ast != nullptr ? global_struct_node->Body->Front : Code{ nullptr };
Code global_struct_member = global_struct_before != nullptr ? global_struct_before->Next : Code{ nullptr };
Code global_struct_after = global_struct_member != nullptr ? global_struct_member->Next : Code{ nullptr };
Code global_later_class = global_struct_node != nullptr ? global_struct_node->Next : Code{ nullptr };
expect(global_struct_node != nullptr && global_struct_node->Type == CT_Struct, txt("cpp global struct recovery preserves partial struct"));
expect(global_struct_member != nullptr && global_struct_member->Type == CT_Untyped, txt("cpp global struct recovery retains failed member"));
if (global_struct_member != nullptr)
expect(str_are_equal(strbuilder_to_str(code_to_strbuilder(global_struct_member)), txt("s32 broken(s32 value;")), txt("cpp global struct recovery exact failed span"));
expect(global_struct_after != nullptr && global_struct_after->Type == CT_Variable, txt("cpp global struct recovery keeps following member"));
if (global_struct_after != nullptr)
expect(str_are_equal(global_struct_after->Name, txt("after")), txt("cpp global struct recovery following member name"));
expect(global_later_class != nullptr && global_later_class->Type == CT_Struct, txt("cpp global struct recovery keeps following global struct"));
ParseInfo nested_owner_info = {};
Opts_parse nested_owner_options = { &ctx, &nested_owner_info };
CodeStruct nested_owner = parse_struct(txt("struct NestedOwner { s32 before; struct InnerRecovery { s32 inner_before; s32 broken(s32 value; s32 inner_after; }; s32 after; };"), nested_owner_options);
expect(nested_owner.ast != nullptr && nested_owner->Type == CT_Struct, txt("cpp nested owner recovery result"));
expect(nested_owner_info.result.ast == ((Code)nested_owner).ast, txt("cpp nested owner recovery ParseInfo result"));
expect(nested_owner_info.messages != nullptr, txt("cpp nested owner recovery diagnostic"));
Code nested_owner_before = nested_owner.ast != nullptr && nested_owner->Body.ast != nullptr ? nested_owner->Body->Front : Code{ nullptr };
Code nested_owner_inner = nested_owner_before != nullptr ? nested_owner_before->Next : Code{ nullptr };
Code nested_owner_after = nested_owner_inner != nullptr ? nested_owner_inner->Next : Code{ nullptr };
Code nested_owner_inner_before = nested_owner_inner != nullptr && nested_owner_inner->Body.ast != nullptr ? nested_owner_inner->Body->Front : Code{ nullptr };
Code nested_owner_inner_failed = nested_owner_inner_before != nullptr ? nested_owner_inner_before->Next : Code{ nullptr };
Code nested_owner_inner_after = nested_owner_inner_failed != nullptr ? nested_owner_inner_failed->Next : Code{ nullptr };
expect(nested_owner_before != nullptr && nested_owner_before->Type == CT_Variable, txt("cpp nested owner keeps preceding member typed"));
expect(nested_owner_inner != nullptr && nested_owner_inner->Type == CT_Struct, txt("cpp nested owner retains inner struct AST"));
expect(nested_owner_inner_before != nullptr && nested_owner_inner_before->Type == CT_Variable, txt("cpp nested inner recovery keeps preceding member typed"));
expect(nested_owner_inner_failed != nullptr && nested_owner_inner_failed->Type == CT_Untyped, txt("cpp nested inner recovery retains failed member"));
expect(nested_owner_inner_after != nullptr && nested_owner_inner_after->Type == CT_Variable, txt("cpp nested inner recovery keeps following member typed"));
expect(nested_owner_after != nullptr && nested_owner_after->Type == CT_Variable, txt("cpp nested owner keeps following member typed"));
#if ! GEN_BUILD_DEBUG
CodeVar wrong_array_value = def_variable(def_type(name(u32)), name(invalid_array_expr));
Opts_def_type wrong_array_options = {};
wrong_array_options.array_expr = (Code)wrong_array_value;
CodeTypename invalid_array_type = def_type(name(InvalidArrayOption), wrong_array_options);
expect(((Code)invalid_array_type).ast == Code_Invalid.ast, txt("cpp wrong type invalid code"));
CodeBody missing_union_body = {};
CodeUnion missing_union = def_union(name(MissingUnionBody), missing_union_body);
expect(((Code)missing_union).ast == Code_Invalid.ast, txt("cpp missing body invalid code"));
#endif
deinit(&ctx);
if (g_failures != 0) { log_fmt("cpp baseline smoke failed: %d\n", g_failures); return 1; }
log_fmt("cpp baseline smoke passed\n");
return 0;
}
+170
View File
@@ -0,0 +1,170 @@
#define GEN_TIME
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.cpp"
using namespace gen;
global int g_failures = 0;
internal void expect(int condition, Str label) {
if (condition == false) { log_fmt("FAIL %S\n", label); ++g_failures; }
}
internal Str body_text(CodeBody body) {
return strbuilder_to_str(code_to_strbuilder(cast(Code, body)));
}
internal b32 body_has_variable(CodeBody body, Str name) { for (Code member : body) {
if (member->Type == CT_Variable && member->Name.is_equal(name)) return true;
if ((member->Type == CT_Struct || member->Type == CT_Union) && member->Body.ast != nullptr) {
if (body_has_variable(cast(CodeBody, member->Body), name))
return true;
}
} return false; }
internal Str bad_at(Str good) {
StrBuilder built = strbuilder_make_reserve(_ctx->Allocator_Temp, good.Len + 2);
strbuilder_append_str(& built, good);
strbuilder_append_str(& built, txt("\n@"));
return strbuilder_to_str(built);
}
internal void expect_context_good(Context* ctx, Str label) {
Str def = txt("struct Context\n{\n\tContext* parent;\n};\n\nContext* make_context();\n");
CodeBody body = parse_global_body(def);
Code found = InvalidCode;
for (Code code : body) if (code->Type == CT_Struct && code->Name.is_equal(txt("Context")))
found = code;
expect(found.ast != nullptr, label); if (found.ast != nullptr) {
Code member = InvalidCode;
for (Code code : cast(CodeBody, found->Body)) {
if (code->Type == CT_Variable && code->Name.is_equal(txt("parent")))
member = code;
}
expect(member.ast != nullptr, label);
}
expect(ctx->parse_info.messages == nullptr, label);
expect(ctx->parse_info.lexed.messages == nullptr, label);
}
internal void expect_context_bad(Str label) {
Context ctx = {}; init(& ctx);
Str good = txt("struct Context\n{\n\tContext* parent;\n};\n\nContext* make_context();\n");
CodeBody body = parse_global_body(bad_at(good));
Str text = body_text(body);
expect(str_contains(text, txt("Context")), label);
expect(str_contains(text, txt("parent")), label);
expect(ctx.parse_info.lexed.messages != nullptr, label);
if (ctx.parse_info.lexed.messages != nullptr) expect(str_contains(ctx.parse_info.lexed.messages->content, txt("Failed to lex token")), label);
expect_context_good(&ctx, txt("context-reuse"));
deinit(& ctx);
}
internal Str vec_def() {
Str type = txt("Vec2f");
Str unit_type = txt("f32");
return token_fmt("type", type, "unit_type", unit_type, stringize(
struct <type>
{
union {
struct {
<unit_type> x;
<unit_type> y;
};
<unit_type> Basis[2];
};
};
));
}
internal void expect_vec_good(Context* ctx, Str label) {
Str def = vec_def();
expect(str_contains(def, txt("struct Vec2f")), txt("vec-def-struct"));
expect(str_contains(def, txt("union")), txt("vec-def-union"));
expect(str_contains(def, txt("f32 x;")), txt("vec-def-x"));
expect(str_contains(def, txt("f32 y;")), txt("vec-def-y"));
expect(str_contains(def, txt("f32 Basis[2]")), txt("vec-def-Basis"));
CodeBody body = parse_global_body(def);
expect(body_has_variable(body, txt("x")), txt("vec-ast-x"));
expect(body_has_variable(body, txt("y")), txt("vec-ast-y"));
Str text = body_text(body);
expect(str_contains(text, txt("Vec2f")), txt("vec-name"));
expect(str_contains(text, txt("x")), txt("vec-x"));
expect(str_contains(text, txt("y")), txt("vec-y"));
expect(ctx->parse_info.messages == nullptr, txt("vec-parser-messages"));
expect(ctx->parse_info.lexed.messages == nullptr, txt("vec-lexer-messages"));
}
internal void expect_vec_bad(Str label) {
Context ctx = {}; init(& ctx);
CodeBody body = parse_global_body(bad_at(vec_def()));
Str text = body_text(body);
expect(str_contains(text, txt("Vec2f")), label);
expect(ctx.parse_info.lexed.messages != nullptr, label);
if (ctx.parse_info.lexed.messages != nullptr) expect(str_contains(ctx.parse_info.lexed.messages->content, txt("Failed to lex token")), label);
expect_vec_good(& ctx, txt("vec-reuse"));
deinit(& ctx);
}
internal Str unreal_def() { return txt("UCLASS()\nclass UGasaFixture\n{\n\tUPROPERTY()\n\tint Health;\n};\n"); }
internal void expect_unreal_good(Context* ctx, Str label) {
CodeBody body = parse_global_body(unreal_def());
Code found = InvalidCode;
for (Code code : body) if (code->Type == CT_Class && code->Name.is_equal(txt("UGasaFixture")))
found = code;
expect(found.ast != nullptr, label); if (found.ast != nullptr) {
b32 saw_prop = false;
b32 saw_health = false;
for (Code member : cast(CodeBody, found->Body)) {
if ( member->Type == CT_Untyped && member->Name.starts_with(txt("UPROPERTY"))) saw_prop = true;
if (saw_prop && member->Type == CT_Variable && member->Name.is_equal(txt("Health"))) saw_health = true;
}
expect(saw_prop, label);
expect(saw_health, label);
}
expect(ctx->parse_info.messages == nullptr, label);
expect(ctx->parse_info.lexed.messages == nullptr, label);
}
internal void expect_unreal_bad(Str label) {
Context ctx = {}; init(& ctx);
register_macros(args(
(Macro { txt("UCLASS"), MT_Statement, MF_Functional }),
(Macro { txt("UPROPERTY"), MT_Statement, MF_Functional })
));
CodeBody body = parse_global_body(bad_at(unreal_def()));
Str text = body_text(body);
expect(str_contains(text, txt("UGasaFixture")), label);
expect(ctx.parse_info.lexed.messages != nullptr, label);
if (ctx.parse_info.lexed.messages != nullptr) expect(str_contains(ctx.parse_info.lexed.messages->content, txt("Failed to lex token")), label);
expect_unreal_good(&ctx, txt("unreal-reuse"));
deinit(& ctx);
}
int main()
{
Context ctx = {}; init(& ctx);
expect_context_good(&ctx, txt("context"));
deinit(& ctx);
expect_context_bad(txt("context-at"));
init(& ctx);
expect_vec_good(& ctx, txt("vec"));
deinit(& ctx);
expect_vec_bad(txt("vec-at"));
init(& ctx);
register_macros(args(
(Macro { txt("UCLASS"), MT_Statement, MF_Functional }),
(Macro { txt("UPROPERTY"), MT_Statement, MF_Functional })
));
expect_unreal_good(& ctx, txt("unreal"));
deinit(& ctx);
expect_unreal_bad(txt("unreal-at"));
if (g_failures != 0) { log_fmt("dogfood parse fixtures failed: %d\n", g_failures); return 1; }
log_fmt("dogfood parse fixtures passed\n");
return 0;
}
+82
View File
@@ -0,0 +1,82 @@
#define GEN_TIME
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.cpp"
using namespace gen;
global int g_failures = 0;
internal void expect(int condition, Str label) {
if (condition == false) { log_fmt("FAIL %S\n", label); ++g_failures; }
}
internal void expect_public(Str def, Str needle, Str label) {
Context ctx = {}; init(& ctx);
ParseInfo info = {}; Opts_parse options = { & ctx, & info };
Code parsed = parse_variable(def, options);
expect(info.result.ast == parsed.ast, label);
expect(info.lexed.messages != nullptr, label);
if (info.lexed.messages != nullptr) expect(str_contains(info.lexed.messages->content, needle), label);
if (info.lexed.tokens.num == 0) expect(parsed == Code_Invalid, label);
CodeVar again = parse_variable(txt("char const* baseline_message = \"GENCPP_BASELINE\";"), options); expect(again.ast != nullptr, label);
expect(info.lexed.messages == nullptr && info.messages == nullptr, label);
if (again.ast != nullptr) {
expect(again->Type == CT_Variable, label);
expect(str_contains(again->Name, txt("baseline_message")), label);
}
deinit(&ctx);
}
internal void expect_lex_fail(Str def, Str needle, Str label) {
Context ctx = {}; init(&ctx);
LexedInfo lexed = lex(&ctx, def);
expect(lexed.tokens.ptr != nullptr, label);
expect(lexed.messages != nullptr, label);
if (lexed.messages != nullptr) expect(str_contains(lexed.messages->content, needle), label);
deinit(&ctx);
}
internal void expect_lex_ok(Str def, Str label) {
Context ctx = {}; init(&ctx);
LexedInfo lexed = lex(&ctx, def);
expect(lexed.tokens.ptr != nullptr, label);
expect(lexed.tokens.num > 0, label);
expect(lexed.messages == nullptr, label);
deinit(&ctx);
}
int main()
{
expect_public(txt("#define FOO(... x)"), txt("Expected a ')' after '...'"), txt("L01-public"));
expect_public(txt("#define FOO(1)"), txt("Expected a '_' or alpha character"), txt("L02-public"));
expect_public(txt("#define FOO(a b)"), txt("Expected a comma after parameter"), txt("L03-public"));
expect_public(txt("#define FOO(a,"), txt("Expected a ')' after last_parameter"), txt("L04-public"));
expect_public(txt("#foo \\q"), txt("in preprocessor directive ("), txt("L05-public"));
expect_public(txt("#include foo"), txt("after #include"), txt("L06-public"));
expect_public(txt("#pragma \\q"), txt("in preprocessor directive '"), txt("L07-public"));
expect_public(txt("..x"), txt("invalid varadic argument"), txt("L08-public"));
expect_public(txt("@"), txt("Failed to lex token"), txt("L09-public"));
expect_lex_fail(txt("#define FOO(... x)"), txt("Expected a ')' after '...'"), txt("L01-lex"));
expect_lex_fail(txt("#define FOO(1)"), txt("Expected a '_' or alpha character"), txt("L02-lex"));
expect_lex_fail(txt("#define FOO(a b)"), txt("Expected a comma after parameter"), txt("L03-lex"));
expect_lex_fail(txt("#define FOO(a,"), txt("Expected a ')' after last_parameter"), txt("L04-lex"));
expect_lex_fail(txt("#foo \\q"), txt("in preprocessor directive ("), txt("L05-lex"));
expect_lex_fail(txt("#include foo"), txt("after #include"), txt("L06-lex"));
expect_lex_fail(txt("#pragma \\q"), txt("in preprocessor directive '"), txt("L07-lex"));
expect_lex_fail(txt("..x"), txt("invalid varadic argument"), txt("L08-lex"));
expect_lex_fail(txt("@"), txt("Failed to lex token"), txt("L09-lex"));
expect_lex_ok(txt("#define FOO(a, b) a"), txt("S01"));
expect_lex_ok(txt("#define FOO(...)"), txt("S02"));
expect_lex_ok(txt("#include \"a.h\""), txt("S03"));
expect_lex_ok(txt("#pragma once"), txt("S04"));
expect_lex_ok(txt("#foo bar"), txt("S05"));
expect_lex_ok(txt("..."), txt("S06"));
if (g_failures != 0) { log_fmt("lexer failures failed: %d\n", g_failures); return 1; }
log_fmt("lexer failures passed\n");
return 0;
}
+126
View File
@@ -0,0 +1,126 @@
#define GEN_TIME
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.cpp"
using namespace gen;
global int g_failures = 0;
internal void expect(int condition, Str label) {
if (condition == false) { log_fmt("FAIL %S\n", label); ++g_failures; }
}
internal Str body_text(CodeBody body) {
return strbuilder_to_str(code_to_strbuilder(cast(Code, body)));
}
internal void expect_good_base(Context* ctx, Str label) {
ParseInfo info = {};
Str def = txt("struct Context { int a; int b; };");
parse_global_body_base(ctx, def, &info);
CodeBody body = cast(CodeBody, info.result);
Str text = body_text(body);
expect(body.ast != nullptr, label);
expect(str_contains(text, txt("Context")), label);
expect(str_contains(text, txt("a")), label);
expect(str_contains(text, txt("b")), label);
expect(info.messages == nullptr, label);
expect(info.lexed.messages == nullptr, label);
expect(info.lexed.tokens.ptr != nullptr, label);
expect(info.lexed.tokens.num > 0, label);
}
internal void expect_at(Str label) {
Context ctx = {};
ParseInfo info = {};
Str def = txt("struct Context { int a; int b; };\n@");
init(&ctx);
parse_global_body_base(&ctx, def, &info);
CodeBody body = cast(CodeBody, info.result);
Str text = body_text(body);
expect(body.ast != nullptr, label);
expect(str_contains(text, txt("Context")), label);
expect(info.lexed.messages != nullptr, label);
if (info.lexed.messages != nullptr)
expect(str_contains(info.lexed.messages->content, txt("Failed to lex token")), label);
expect(info.lexed.tokens.ptr != nullptr, label);
expect(info.lexed.tokens.num > 0, label);
expect_good_base(&ctx, txt("at-reuse"));
deinit(&ctx);
}
internal void expect_comma(Str label) {
Context ctx = {};
ParseInfo info = {};
Str def = txt("struct Context { int a; int b; };\n,");
init(&ctx);
parse_global_body_base(&ctx, def, &info);
CodeBody body = cast(CodeBody, info.result);
Str text = body_text(body);
expect(str_contains(text, txt("Context")), label);
expect(info.messages != nullptr, label);
if (info.messages != nullptr)
expect(str_contains(info.messages->Content, txt("Dangling comma")), label);
expect_good_base(&ctx, txt("comma-reuse"));
deinit(&ctx);
}
internal void expect_space(Str label) {
Context ctx = {};
ParseInfo info = {};
init(&ctx);
parse_global_body_base(&ctx, txt(" \t\n"), &info);
CodeBody body = cast(CodeBody, info.result);
expect(body.ast != nullptr, label);
expect(body->Type == CT_Global_Body, label);
expect(info.lexed.tokens.num == 0, label);
expect(info.lexed.messages != nullptr, label);
if (info.lexed.messages != nullptr)
expect(str_contains(info.lexed.messages->content, txt("no tokens found")), label);
expect_good_base(&ctx, txt("space-reuse"));
deinit(&ctx);
}
internal void expect_empty(Str label) {
Context ctx = {};
ParseInfo info = {};
Str def = { nullptr, 0 };
init(&ctx);
parse_global_body_base(&ctx, def, &info);
CodeBody body = cast(CodeBody, info.result);
expect(body.ast != nullptr, label);
expect(info.messages != nullptr, label);
if (info.messages != nullptr)
expect(str_contains(info.messages->Content, txt("length must greater than 0")), label);
expect_good_base(&ctx, txt("empty-reuse"));
deinit(&ctx);
}
internal void expect_wrapper(Str label) {
Context ctx = {};
CodeBody body;
Str text;
init(&ctx);
body = parse_global_body(txt("struct Context { int a; int b; };"));
text = body_text(body);
expect(str_contains(text, txt("Context")), label);
expect(ctx.parse_info.messages == nullptr, label);
expect(str_contains(body_text(cast(CodeBody, ctx.parse_info.result)), txt("Context")), label);
deinit(&ctx);
}
int main()
{
Context ctx = {}; init(& ctx);
expect_good_base(&ctx, txt("good"));
deinit(& ctx);
expect_at(txt("at"));
expect_comma(txt("comma"));
expect_space(txt("space"));
expect_empty(txt("empty"));
expect_wrapper(txt("wrapper"));
if (g_failures != 0) { log_fmt("parse body messages failed: %d\n", g_failures); return 1; }
log_fmt("parse body messages passed\n");
return 0;
}
+249
View File
@@ -0,0 +1,249 @@
#define GEN_TIME
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.cpp"
using namespace gen;
global s32 g_failures = 0;
internal inline void expect(int condition, Str label) {
if (condition == false) { log_fmt("FAIL %S\n", label); ++ g_failures; }
}
internal inline Token canary_at(Str text) {
Token tok = NullToken;
tok.Type = Tok_Comma;
tok.Text = text;
tok.Line = 99;
tok.Column = 99;
return tok;
}
internal inline Token ident(Str text) {
Token tok = NullToken;
tok.Type = Tok_Identifier;
tok.Text = text;
tok.Line = 1;
tok.Column = 1;
return tok;
}
internal b32 canaries_intact(Token* storage, s32 logical) {
Token expected = canary_at(txt("CANARY"));
for (s32 id = 0; id < 3; ++id) {
Token got = storage[logical + id];
if (got.Type != expected.Type || got.Text.Ptr != expected.Text.Ptr || got.Line != 99)
return false;
}
return true;
}
internal inline b32 points_at_canary(Token* got, Token* storage, s32 logical) {
return got == & storage[logical] || got == & storage[logical + 1] || got == & storage[logical + 2];
}
internal void plant(Token* storage, s32 logical, Token* src, s32 src_num) {
for (s32 id = 0; id < src_num; ++id) storage[id] = src[id];
for (s32 id = 0; id < 3; ++id) storage[logical + id] = canary_at(txt("CANARY"));
}
internal void bind(ParseContext* parser, Token* storage, s32 logical, s32 token_id) {
parser->tokens.ptr = storage;
parser->tokens.num = logical;
parser->token_id = token_id;
parser->scope = nullptr;
parser->messages = nullptr;
}
internal void expect_helper(ParseContext* parser, Token* storage, s32 logical, Str label) {
s32 before = parser->token_id;
Token* cur = lex_current (parser, lex_dont_skip_formatting);
Token* peek = lex_peek (parser, lex_dont_skip_formatting);
Token* next = lex_next (parser, lex_dont_skip_formatting);
Token* prev = lex_previous(parser, lex_dont_skip_formatting);
expect(parser->token_id == before, label);
expect(canaries_intact(storage, logical), label);
expect(points_at_canary(cur, storage, logical) == false, label);
expect(points_at_canary(peek, storage, logical) == false, label);
expect(points_at_canary(next, storage, logical) == false, label);
expect(points_at_canary(prev, storage, logical) == false, label);
expect(cur != nullptr && peek != nullptr && next != nullptr && prev != nullptr, label);
}
internal void expect_row_empty(Str label) {
Token storage[3] = {}; plant(storage, 0, nullptr, 0);
ParseContext parser = {}; bind(& parser, storage, 0, 0);
expect_helper(& parser, storage, 0, label);
expect(lex_current (& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
expect(lex_peek (& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
expect(lex_next (& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
expect(lex_previous(& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
}
internal void expect_row_single(Str label) {
Token src [1] = { ident(txt("only")) };
Token storage[4] = {}; plant(storage, 1, src, 1);
ParseContext parser = {}; bind(& parser, storage, 1, 0);
expect_helper(& parser, storage, 1, label);
expect(lex_current (& parser, lex_dont_skip_formatting)->Text.Ptr == src[0].Text.Ptr, label);
expect(lex_peek (& parser, lex_dont_skip_formatting)->Text.Ptr == src[0].Text.Ptr, label);
expect(lex_next (& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
expect(lex_previous(& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
}
internal void expect_row_ends(Str label) {
Token src [2] = { ident(txt("first")), ident(txt("last")) };
Token storage[5] = {}; plant(storage, 2, src, 2);
ParseContext parser = {};
bind(& parser, storage, 2, 0);
expect_helper(& parser, storage, 2, label);
expect(lex_current (& parser, lex_dont_skip_formatting)->Text.Ptr == src[0].Text.Ptr, label);
expect(lex_next (& parser, lex_dont_skip_formatting)->Text.Ptr == src[1].Text.Ptr, label);
expect(lex_previous(& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
bind(& parser, storage, 2, 1);
expect_helper(& parser, storage, 2, label);
expect(lex_current (& parser, lex_dont_skip_formatting)->Text.Ptr == src[1].Text.Ptr, label);
expect(lex_next (& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
expect(lex_previous(& parser, lex_dont_skip_formatting)->Text.Ptr == src[0].Text.Ptr, label);
bind(& parser, storage, 2, 2);
expect_helper(& parser, storage, 2, label);
expect(lex_current (& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
expect(lex_peek (& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
expect(lex_next (& parser, lex_dont_skip_formatting)->Type == Tok_Invalid, label);
expect(lex_previous(& parser, lex_dont_skip_formatting)->Text.Ptr == src[1].Text.Ptr, label);
}
internal void expect_skip_end(Str label)
{
Token newlines[2] = { NullToken, NullToken };
newlines[0].Type = Tok_NewLine; newlines[0].Text = txt("\n");
newlines[1].Type = Tok_NewLine; newlines[1].Text = txt("\n");
Token storage[5] = {}; plant(storage, 2, newlines, 2);
ParseContext parser = {}; bind(& parser, storage, 2, 0);
Token* got = lex_current(& parser, lex_skip_formatting);
expect(got->Type == Tok_Invalid, label);
expect(parser.token_id == 2, label);
expect(canaries_intact(storage, 2), label);
expect(points_at_canary(got, storage, 2) == false, label);
bind(& parser, storage, 2, 0); got = lex_peek(& parser, lex_skip_formatting);
expect(parser.token_id == 0, label);
expect(got->Type == Tok_Invalid, label);
expect(canaries_intact(storage, 2), label);
Token comments[1] = { NullToken };
comments[0].Type = Tok_Comment; comments[0].Text = txt("// note");
plant(storage, 1, comments, 1); bind(& parser, storage, 1, 0);
got = lex_current(& parser, lex_skip_formatting);
expect(got->Type == Tok_Invalid, label);
expect(parser.token_id == 1, label);
expect(canaries_intact(storage, 1), label);
expect(points_at_canary(got, storage, 1) == false, label);
}
internal void expect_boundary(Str def, Str label, Str mark, b32 expect_lexer_message) {
Context ctx = {}; init(& ctx);
CodeVar parsed = parse_variable(def);
expect(cast(Code, parsed) == Code_Invalid, label);
ParseMessage* parse_message = _ctx->parse_info.messages;
LexerMessage* lexer_message = _ctx->parse_info.lexed.messages;
if (expect_lexer_message) {
expect(lexer_message != nullptr, label);
expect(parse_message == nullptr, label);
if (lexer_message != nullptr) expect(str_contains(lexer_message->content, mark), label);
}
else {
expect(parse_message != nullptr, label);
expect(lexer_message == nullptr, label);
}
if (expect_lexer_message == false && parse_message != nullptr) {
expect(parse_message->Scope == nullptr, label);
expect(parse_message->Content.Ptr != nullptr && parse_message->Content.Len > 0, label);
expect(str_contains(parse_message->Content, mark), label);
}
CodeVar again = parse_variable(txt("char const* baseline_message = \"GENCPP_BASELINE\";"));
expect(again.ast != nullptr, label);
if (again.ast != nullptr) {
expect(again->Type == CT_Variable, label);
expect(str_contains(again->Name, txt("baseline_message")), label);
}
deinit(& ctx);
}
internal void expect_eat_end(Str label) {
Context ctx = {}; init(&ctx);
Token src [1] = { ident(txt("x")) };
Token storage[4] = {};
plant(storage, 1, src, 1); bind(& ctx.parser, storage, 1, 1);
b32 ate = lex__eat(&ctx, & ctx.parser, Tok_Identifier);
expect(ate == false, label);
expect(ctx.parser.token_id <= ctx.parser.tokens.num, label);
expect(canaries_intact(storage, 1), label);
deinit(& ctx);
}
internal void expect_direct(Str label) {
Context ctx = {}; init(& ctx);
Token src [2] = { ident(txt("int")), ident(txt("name")) };
Token storage[5] = {}; plant(storage, 2, src, 2);
bind(& ctx.parser, storage, 2, 0);
Code got = parse_operator_function_or_variable(& ctx, false, NullCode, NullCode);
expect(canaries_intact(storage, 2), label);
(void)got;
Token op[2] = {};
op[0] = NullToken;
op[0].Type = Tok_Decl_Operator;
op[0].Text = txt("operator");
op[0].Line = 1;
op[1] = ident(operator_to_str(Op_Delete));
plant(storage, 2, op, 2);
ctx.parser = {}; bind(& ctx.parser, storage, 2, 0);
CodeOperator op_got = parse_operator_after_ret_type(& ctx, ModuleFlag_None, NullCode, NullCode, NullCode);
expect(cast(Code, op_got) == Code_Invalid, label);
expect(canaries_intact(storage, 2), label);
deinit(& ctx);
}
internal void expect_builder(Str label) {
Token storage[3] = {}; plant(storage, 0, nullptr, 0);
ParseContext parser = {}; bind(& parser, storage, 0, 0);
StrBuilder built = parser_to_strbuilder(& parser, _ctx->Allocator_Temp);
Str text = strbuilder_to_str(built);
expect(str_contains(text, txt("empty")), label);
expect(canaries_intact(storage, 0), label);
strbuilder_free(& built);
}
int main()
{
using namespace gen;
Context ctx = {}; init(& ctx);
expect_row_empty (txt("empty"));
expect_row_single(txt("single"));
expect_row_ends (txt("ends"));
expect_skip_end (txt("skip"));
expect_builder (txt("builder"));
deinit(& ctx);
Str null_def = { nullptr, 0 };
Str null_ptr = { nullptr, 4 };
expect_boundary(null_def, txt("null-len"), txt("gen::parse_variable: length must greater than 0"), false);
expect_boundary(null_ptr, txt("null-ptr"), txt("gen::parse_variable: def was null"), false);
expect_boundary(txt(""), txt("zero-len"), txt("gen::parse_variable: length must greater than 0"), false);
expect_boundary(txt(" \n\t"), txt("whitespace"), txt("no tokens found"), true);
expect_boundary(txt("// note\n"), txt("comment"), txt("only formatting tokens"), false);
expect_eat_end (txt("eat"));
expect_direct (txt("direct"));
if (g_failures != 0) {
log_fmt("parser bounds failed: %d\n", g_failures); return 1;
}
log_fmt("parser bounds passed\n");
return 0;
}
+10
View File
@@ -0,0 +1,10 @@
#define GEN_IMPLEMENTATION
#define GEN_DEFINE_LIBRARY_CODE_CONSTANTS
#define GEN_ENFORCE_STRONG_CODE_TYPES
#include "gen.hpp"
int main()
{
gen::log_fmt("FAIL forced assertion\n");
return 1;
}
+3
View File
@@ -0,0 +1,3 @@
int main( {
return 1;
}
+2
View File
@@ -0,0 +1,2 @@
@echo off
exit /b 1
+7
View File
@@ -0,0 +1,7 @@
extern "C" void baseline_missing_link_target(void);
int main()
{
baseline_missing_link_target();
return 0;
}
+4
View File
@@ -0,0 +1,4 @@
int main(void)
{
return 1;
}
+4
View File
@@ -0,0 +1,4 @@
int main(void)
{
return 73;
}