Progress towards bootstrap/singleheader generation

I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want.

Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures.

When it comes to the cpp files, I may leave those alone as the macros largely help ith readability.
Replacing those macros is expensive and most likely not worth it.

The macros under consideration with replacing using the library bootstrap are:

* Define_Types
* Define_Operators
* Define_Specifiers
* GEN_Define_Attribute_Tokens
* Define_CodeType
* Using_Code
* Define_TokType
* def_constant_spec ?
* Helper Macros for def_**_body functions ?
  * AST unallowed types case macros?
(The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...)

The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing).
This would allow the user to specify custom attribute macros as well with greater ease.

I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell).

The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out.
(Most likely may want to for various purposes)

The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components.
This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
This commit is contained in:
Edward R. Gonzalez 2023-07-24 22:19:21 -04:00
parent 387787b88d
commit 9c81504178
19 changed files with 450 additions and 106 deletions

3
.gitignore vendored
View File

@ -5,6 +5,9 @@ build/*
**/*.gen.*
gencpp.hpp
gencpp.cpp
# Build results
[Dd]ebug/
[Dd]ebugPublic/

9
.vscode/launch.json vendored
View File

@ -22,6 +22,15 @@
"args": [],
"cwd": "${workspaceFolder}/test/gen/",
"visualizerFile": "${workspaceFolder}/scripts/gencpp.natvis"
},
{
"type": "cppvsdbg",
"request": "launch",
"name": "Debug bootstrap vsdbg",
"program": "${workspaceFolder}/project/build/gencpp_bootstrap.exe",
"args": [],
"cwd": "${workspaceFolder}/project/",
"visualizerFile": "${workspaceFolder}/scripts/gencpp.natvis"
}
]
}

View File

@ -7,7 +7,7 @@ These build up a code AST to then serialize with a file builder.
General goal is to have a less than 15k sloc library that takes at most a couple of hours to learn and make use of.
*Why 15?* Assuming a seasoned coder of C++ can read and understand around 1000-2000 lines of code per hour, 15,000 could be understood in under 16-18 hours
*Why 15k ?* Assuming a seasoned coder of C++ can read and understand around 1000-2000 lines of code per hour, 15,000 could be understood in under 16-18 hours
and have confidence in modifying for their use case.
This code base attempts follow the [handmade philosophy](https://handmade.network/manifesto),
@ -162,7 +162,18 @@ struct ArrayHeader
## Building
An example of building is provided in the test directory.
An example of building is provided within project, singleheader, and test.
**Project**
`gen.bootstrap.cpp` generates a segmented version of the library following a more traditional cpp convention.
With the exception that: *The component hpp and cpp files are all included into their respective header and source*
**Singleheader**
`gen.singleheader.cpp` generated a single-header version of the library following the convention shown in popular libraries such as: gb, stb, and zpl.
**Test**
There are two meson build files the one within test is the program's build specification.
The other one in the gen directory within test is the metaprogram's build specification.

View File

@ -14,15 +14,10 @@ Both libraries use *pre-generated* (self-hosting I guess) version of the library
The default `gen.bootstrap.cpp` located in the project folder is meant to be produce a standard segmeneted library, where the components of the library
have relatively dedicated header and source files. With dependencies included at the top of the file and each header starting with a pragma once.
`gen.singleheader.cpp` in the single header folder with its own `meson.build` generates the library as a single header `gen.hpp`.
Following the same convention seen in the gb, stb, and zpl libraries.
Use those to get a general idea of how to make your own tailored version.
If the naming convention is undesired, the `gencpp.refactor` script can be used with the [refactor]()
## gen.hpp
Feature Macros:
* `GEN_DONT_USE_NAMESPACE` : By default, the library is wrapped in a `gen` namespace, this will disable that expose it to the global scope.
@ -36,70 +31,3 @@ Feature Macros:
`GEN_USE_RECURSIVE_AST_DUPLICATION` is available but its not well tested and should not need to be used.
If constructing ASTs properly. There should be no modification of ASTs, and thus this would never become an issue.
(I will probably remove down the line...)
Due to the design of `gen.hpp` to support being written alongside runtime intended code (in the same file), all the code is wrapped in a `GEN_TIME` `#ifdef` and then wrapped further in a `gen` namespace to avoid pollution of the global scope.
*Note: Its possible with the scanner feature to support parsing runtime files that use "generic" macros or identifiers with certain patterns.
This can be used to auto-queue generation of dependent definitions for the symbols used.*
### Organization
Dependencies : Mostly from the c-zpl library.
log_failure definition : based on whether to always use fatal on all errors
Major enum definitions and their associated functions used with the AST data
* `ECode` : Used to tag ASTs by their type
* `EOperator` : Used to tag operator overloads with their op type
* `ESpecifier` : Used with specifier ASTs for all specifiers the user may tag an associated
AST with.
* `AccessSpec` : Used with class and struct ASTs to denote the public, protected, or private fields.
* `EnumT` : Used with def_enum to determine if constructing a regular enum or an enum class.
* `ModuleFlag` : Used with any valid definition that can have export or import related keywords associated with it.
#### Data Structures
`StringCache` : Hash table for cached strings. (`StringCached` typedef used to denote strings managed by it)
`Code` : Wrapper for `AST` with functionality for handling it appropriately.
`AST` : The node data structure for the code.
`Code Types` : Codes with typed ASTs. Body, Param, and Specifier have unique implementation, the rest use `Define_CodeType`
`AST Types` : Filtered AST definitions.
#### Gen Interface
First set of forwards are either backend functions used for various aspects of AST generation or configurations allocators used for different containers.
Interface forwards defined in order of: Upfront, Parsing, Untyped.
From there forwards for the File handlers are defined: Builder, Editor, Scanner.
#### Macros
General helper macros are defined along with the optional DSL macros.
#### Constants
Constants including optional ones are defined.
#### Inlines
Inlined functions related to the AST datatype that required forwards for gen interface functions are defined.
## gen.cpp
* Dependencies
* Static data
* AST Body case macros are next.
* AST implementation
* Gen interface begins with its `init`, `deinit`, etc.. Until `make_code_entires`
* operator__validate defined, which will be used to verify if an operator definition is constructible.
* Allocator interface for data arrays, code pool, code entries, string arenas, and string table.
* Helper macros used throughout the constructor API
* Upfront constructors, following the same order as shown in the header.
* Parsing constructors, it defines its own lexer, and has many helper functions for parsing not exposed through the header.
* Untyped constructors
* Builder
* Editor
* Scanner

View File

@ -0,0 +1,17 @@
/*
gencpp: An attempt at "simple" staged metaprogramming for c/c++.
See Readme.md for more information from the project repository.
Public Address:
https://github.com/Ed94/gencpp
*/
#if ! defined(GEN_DONT_ENFORCE_GEN_TIME_GUARD) && ! defined(GEN_TIME)
# error Gen.hpp : GEN_TIME not defined
#endif
//! If its desired to roll your own dependencies, define GEN_ROLL_OWN_DEPENDENCIES before including this file.
// Dependencies are derived from the c-zpl library: https://github.com/zpl-c/zpl
#ifndef GEN_ROLL_OWN_DEPENDENCIES
# include "dependencies/gen.dep.hpp"
#endif

View File

@ -1,7 +1,37 @@
#pragma once
#include "gen.hpp"
namespace gen {
GEN_NS_BEGIN
Code scan_file( char const* path )
{
FileInfo file;
FileError error = file_open_mode( & file, EFileMode_READ, path );
if ( error != EFileError_NONE )
{
fatal( "scan_file: Could not open genc.macro.h: %s", path );
}
sw fsize = file_size( & file );
if ( fsize <= 0 )
{
fatal("scan_file: %s is empty", path );
}
String str = String::make_reserve( GlobalAllocator, fsize );
file_read( & file, str, fsize );
str.get_header().Length = fsize;
file_close( & file );
return untyped_str( str );
}
struct Policy
{
// Nothing for now.
};
struct SymbolInfo
{
@ -40,5 +70,4 @@ struct Scanner
bool process_requests( Array<Receipt> out_receipts );
};
// namespace gen
}
GEN_NS_END

View File

@ -2,13 +2,71 @@
#define GEN_ENFORCE_STRONG_CODE_TYPES
#define GEN_EXPOSE_BACKEND
#include "gen.cpp"
#include "filesystem/gen.scanner.hpp"
using namespace gen;
bool namespace_by_default = true;
constexpr StrC nspace_default = txt_StrC(R"(
#if defined(GEN_DONT_USE_NAMESPACE) && ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN
# define GEN_NS_END
#elif ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN namespace gen {
# define GEN_NS_END }
#endif
)");
constexpr StrC nspace_non_default = txt_StrC(R"(
#if ! defined(GEN_USE_NAMESPACE) && ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN
# define GEN_NS_END
#elif ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN namespace gen {
# define GEN_NS_END }
#endif
)");
int gen_main()
{
gen::init();
Code push_ignores = scan_file( "helpers/gen.push_ignores.inline.hpp" );
Code pop_ignores = scan_file( "helpers/gen.pop_ignores.inline.hpp" );
Builder gen_deps_header;
{
}
Builder gen_deps_impl;
{
}
Builder gen_header;
{
Code header_start = scan_file( "components/gen.header_start.hpp" );
Code nspace_macro = untyped_str( namespace_by_default ? nspace_default : nspace_non_default );
gen_header.open( "gencpp.hpp" );
gen_header.print_fmt("#pragma once\n\n");
gen_header.print( push_ignores );
gen_header.print( header_start );
gen_header.print( nspace_macro );
gen_header.print_fmt( "GEN_NS_BEGIN\n");
gen_header.print_fmt( "\nGEN_NS_END\n\n");
gen_header.print( pop_ignores );
gen_header.write();
}
Builder gen_impl;
{
}
gen::deinit();
return 0;
}

View File

@ -1,10 +1,11 @@
#include "helpers/gen.push_ignores.inline.hpp"
// ReSharper disable CppClangTidyClangDiagnosticSwitchEnum
#if ! defined(GEN_DONT_ENFORCE_GEN_TIME_GUARD) && ! defined(GEN_TIME)
#error Gen.hpp : GEN_TIME not defined
#endif
#include "helpers/gen.push_ignores.inline.hpp"
//! If its desired to roll your own dependencies, define GEN_ROLL_OWN_DEPENDENCIES before including this file.
//! Dependencies are derived from the c-zpl library: https://github.com/zpl-c/zpl

View File

@ -8,25 +8,8 @@
*/
#pragma once
#if ! defined(GEN_DONT_ENFORCE_GEN_TIME_GUARD) && ! defined(GEN_TIME)
#error Gen.hpp : GEN_TIME not defined
#endif
#include "helpers/gen.push_ignores.inline.hpp"
//! If its desired to roll your own dependencies, define GEN_ROLL_OWN_DEPENDENCIES before including this file.
// Dependencies are derived from the c-zpl library: https://github.com/zpl-c/zpl
#ifndef GEN_ROLL_OWN_DEPENDENCIES
# include "dependencies/gen.dep.hpp"
#endif
#if defined(GEN_DONT_USE_NAMESPACE) && ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN
# define GEN_NS_END
#elif ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN namespace gen {
# define GEN_NS_END }
#endif
#include "components/gen.header_start.hpp"
GEN_NS_BEGIN

163
scripts/.clang-format Normal file
View File

@ -0,0 +1,163 @@
# Format Style Options - Created with Clang Power Tools
---
AccessModifierOffset: -4
AlignAfterOpenBracket: BlockIndent
AlignArrayOfStructures: Right
AlignConsecutiveAssignments:
Enabled: true
AcrossEmptyLines: false
AcrossComments: true
AlignCompound: true
PadOperators: true
AlignConsecutiveBitFields: AcrossComments
AlignConsecutiveDeclarations: AcrossComments
AlignConsecutiveMacros: AcrossComments
AlignEscapedNewlines: Right
AlignOperands: DontAlign
AlignTrailingComments: true
AllowAllArgumentsOnNextLine: false
AllowAllConstructorInitializersOnNextLine: false
AllowAllParametersOfDeclarationOnNextLine: false
AllowShortBlocksOnASingleLine: Never
AllowShortCaseLabelsOnASingleLine: false
AllowShortLambdasOnASingleLine: None
AllowShortEnumsOnASingleLine: false
AllowShortFunctionsOnASingleLine: None
AllowShortIfStatementsOnASingleLine: Never
AllowShortLoopsOnASingleLine: false
AlwaysBreakAfterReturnType: None
AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: Yes
BinPackArguments: false
BinPackParameters: false
BitFieldColonSpacing: Both
BraceWrapping:
AfterCaseLabel: false
AfterClass: false
AfterControlStatement: false
AfterEnum: false
AfterFunction: false
AfterNamespace: false
AfterObjCDeclaration: false
AfterStruct: false
AfterUnion: false
AfterExternBlock: false
BeforeCatch: false
BeforeElse: false
IndentBraces: false
SplitEmptyFunction: false
SplitEmptyRecord: false
SplitEmptyNamespace: false
BeforeLambdaBody: false
BeforeWhile: false
# BreakAfterAttributes: Always
# BreakArrays: false
# BreakBeforeInlineASMColon: OnlyMultiline
BreakBeforeBinaryOperators: NonAssignment
BreakBeforeBraces: Allman
BreakBeforeInheritanceComma: true
BreakInheritanceList: BeforeComma
BreakBeforeConceptDeclarations: true
BreakBeforeTernaryOperators: true
BreakConstructorInitializers: BeforeComma
BreakStringLiterals: true
ColumnLimit: 180
CompactNamespaces: true
ConstructorInitializerAllOnOneLineOrOnePerLine: true
ConstructorInitializerIndentWidth : 4
ContinuationIndentWidth: 4
Cpp11BracedListStyle: false
DeriveLineEnding: true
ExperimentalAutoDetectBinPacking: false
FixNamespaceComments: true
IncludeBlocks: Preserve
IndentCaseBlocks: true
IndentCaseLabels: true
IndentExternBlock: AfterExternBlock
IndentGotoLabels: true
IndentPPDirectives: AfterHash
IndentRequires: true
IndentWidth: 4
IndentWrappedFunctionNames: false
# InsertNewlineAtEOF: true
InsertTrailingCommas: Wrapped
LambdaBodyIndentation: OuterScope
Language: Cpp
MaxEmptyLinesToKeep: 4
NamespaceIndentation: All
PointerAlignment: Left
QualifierAlignment: Leave
ReferenceAlignment: Left
ReflowComments: true
# RequiresExpressionIndentation: OuterScope
SeparateDefinitionBlocks: Always
ShortNamespaceLines: 40
SortIncludes: true
SortUsingDeclarations: true
SpaceAfterCStyleCast: false
SpaceAfterLogicalNot: true
SpaceAfterTemplateKeyword: false
SpaceAroundPointerQualifiers: Default
SpaceBeforeAssignmentOperators: true
SpaceBeforeCaseColon: true
SpaceBeforeCpp11BracedList: true
SpaceBeforeCtorInitializerColon: true
SpaceBeforeInheritanceColon: true
SpaceBeforeParens: ControlStatementsExceptControlMacros
SpaceBeforeRangeBasedForLoopColon: true
SpaceBeforeSquareBrackets: false
SpacesBeforeTrailingComments: 4
SpaceInEmptyBlock: true
SpaceInEmptyParentheses: false
SpacesInAngles: true
SpacesInCStyleCastParentheses: true
SpacesInConditionalStatement: true
SpacesInContainerLiterals: true
SpacesInLineCommentPrefix:
Minimum: 1
Maximum: 20
SpacesInParentheses: true
SpacesInSquareBrackets: true
Standard: c++17
TabWidth: 4
UseTab: ForIndentation
...

View File

@ -39,3 +39,27 @@ Push-Location $path_root
& ninja $args_ninja
Pop-Location
Push-location $path_project
# Run meta-program
$gencpp_bootstrap = Join-Path $path_project_build gencpp_bootstrap.exe
Write-Host `nRunning gencpp bootstrap...
& $gencpp_bootstrap
# Format generated files
Write-Host `nBeginning format...
$formatParams = @(
'-i' # In-place
'-style=file:../scripts/.clang-format'
'-verbose'
)
$include = @('gencpp.hpp', 'gencpp.cpp')
$exclude = $null
$targetFiles = @(Get-ChildItem -Recurse -Path $path_project -Include $include -Exclude $exclude | Select-Object -ExpandProperty FullName)
clang-format $formatParams $targetFiles
Write-Host "`nFormatting complete"
Pop-Location

View File

@ -28,13 +28,29 @@ if ( Test-Path $path_gen_build )
Remove-Item $path_gen_build -Recurse
}
[string[]] $include = '*.h', '*.hpp', '*.cpp'
[string[]] $include = 'gencpp.hpp', 'gencpp.cpp'
[string[]] $exclude =
$files = Get-ChildItem -Recurse -Path $path_project -Include $include -Exclude $exclude
if ( $files )
{
Remove-Item $files
}
$files = Get-ChildItem -Recurse -Path $path_singleheader -Include $include -Exclude $exclude
if ( $files )
{
Remove-Item $files
}
$include = '*.h', '*.hpp', '*.cpp'
$exclude =
$files = Get-ChildItem -Recurse -Path $path_gen -Include $include -Exclude $exclude
if ( $files )
{
Remove-Item $files
}

View File

@ -39,3 +39,27 @@ Push-Location $path_root
& ninja $args_ninja
Pop-Location
Push-location $path_singleheader
# Run meta-program
$gencpp_singleheader = Join-Path $path_singleheader_build gencpp_singleheader.exe
Write-Host `nRunning gencpp bootstrap...
& $gencpp_singleheader
# Format generated files
Write-Host `nBeginning format...
$formatParams = @(
'-i' # In-place
'-style=file:../scripts/.clang-format'
'-verbose'
)
$include = @('gencpp.hpp')
$exclude = $null
$targetFiles = @(Get-ChildItem -Recurse -Path $path_project -Include $include -Exclude $exclude | Select-Object -ExpandProperty FullName)
clang-format $formatParams $targetFiles
Write-Host "`nFormatting complete"
Pop-Location

6
singleheader/Readme.md Normal file
View File

@ -0,0 +1,6 @@
# Singleheader
`gen.singleheader.cpp` with its own `meson.build` generates the library as a single header `gen.hpp`.
Following the same convention seen in the gb, stb, and zpl libraries.
( Currently WIP )

View File

@ -0,0 +1,11 @@
/*
gencpp: An attempt at "simple" staged metaprogramming for c/c++.
See Readme.md for more information from the project repository.
Public Address:
https://github.com/Ed94/gencpp
*/
#if ! defined(GEN_DONT_ENFORCE_GEN_TIME_GUARD) && ! defined(GEN_TIME)
# error Gen.hpp : GEN_TIME not defined
#endif

View File

@ -2,10 +2,76 @@
#define GEN_ENFORCE_STRONG_CODE_TYPES
#define GEN_EXPOSE_BACKEND
#include "gen.cpp"
#include "filesystem/gen.scanner.hpp"
using namespace gen;
bool namespace_by_default = true;
constexpr StrC nspace_default = txt_StrC(R"(
#if defined(GEN_DONT_USE_NAMESPACE) && ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN
# define GEN_NS_END
#elif ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN namespace gen {
# define GEN_NS_END }
#endif
)");
constexpr StrC nspace_non_default = txt_StrC(R"(
#if ! defined(GEN_USE_NAMESPACE) && ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN
# define GEN_NS_END
#elif ! defined(GEN_NS_BEGIN)
# define GEN_NS_BEGIN namespace gen {
# define GEN_NS_END }
#endif
)");
constexpr StrC gen_implementation_guard = txt_StrC(R"(
#pragma region GENCPP IMPLEMENTATION GUARD
#if ! defined(GEN_IMPLEMENTATION)
# define GEN_IMPLEMENTATION
)");
constexpr StrC gen_implementation_end = txt_StrC(R"(
#endif
#pragma endregion GENCPP IMPLEMENTATION GUARD
)");
int gen_main()
{
gen::init();
Code push_ignores = scan_file( "../project/helpers/gen.push_ignores.inline.hpp" );
Code pop_ignores = scan_file( "../project/helpers/gen.pop_ignores.inline.hpp" );
Code header_start = scan_file( "components/gen.header_start.hpp" );
Code nspace_macro = untyped_str( namespace_by_default ? nspace_default : nspace_non_default );
Builder gen_header;
gen_header.open( "gencpp.hpp" );
gen_header.print( push_ignores );
// Headers
gen_header.print_fmt("#pragma once\n\n");
gen_header.print( header_start );
gen_header.print( nspace_macro );
gen_header.print_fmt( "GEN_NS_BEGIN\n");
gen_header.print_fmt( "\nGEN_NS_END\n");
// Implementation
gen_header.print_fmt( "%s\n\n", (char const*) gen_implementation_guard );
gen_header.print_fmt( "GEN_NS_BEGIN\n");
gen_header.print_fmt( "\nGEN_NS_END\n");
gen_header.print_fmt( "%s\n", (char const*) gen_implementation_end );
gen_header.print( pop_ignores );
gen_header.write();
gen::deinit();
return 0;
}
}

View File

@ -1,5 +0,0 @@
# Singleheader generator
This will require the scanner to be implemented before it can be done properly.

View File

@ -1,4 +1,4 @@
project( 'gen_singleheader', 'c', 'cpp', default_options : ['buildtype=debug'] )
project( 'gencpp_singleheader', 'c', 'cpp', default_options : ['buildtype=debug'] )
includes = include_directories(
[
@ -15,4 +15,4 @@ endif
add_project_arguments('-DGEN_TIME', language : ['c', 'cpp'])
executable( 'gen_singlehader', sources, include_directories : includes )
executable( 'gencpp_singleheader', sources, include_directories : includes )