Update readme, fix an issue with stb_image.refactor script (needed newline at eof), some script changes.

This commit is contained in:
Edward R. Gonzalez 2023-03-31 16:17:02 -04:00
parent 6e91555a20
commit 6bc4346652
8 changed files with 101 additions and 53 deletions

View File

@ -2,18 +2,18 @@
Refactor c/c++ files (and problably others) with ease. Refactor c/c++ files (and problably others) with ease.
## Parameters : ## Parameters
* `-num` : Used if more than one source file is provided (if used, number of destination files provided MUST MATCH). * `-num` : Used if more than one source file is provided (if used, number of destination files provided MUST MATCH).
* `-src` : Source file to refactor * `-src` : Source file to refactor
* `-dst` : Destination file after the refactor (omit to use the same as source) * `-dst` : Destination file after the refactor (omit to use the same as source)
* `-spec` : Specification containing rules to use for the refactor. * `-spec` : Specification containing rules to use for the refactor.
* `-debug` : Use only if on debug build and desire to attach to process. * `-debug` : Use if on debug build and desire to attach to process, or to get a more verbose log.
## Syntax : ## Syntax
* `not` Omit word or namespace. * `not` Omit word or namespace.
* `include` Preprocessor include <file> related identifiers. * `include` Preprocessor include `<file path>` related identifiers.
* `word` Fixed sized identifier. * `word` Fixed sized identifier.
* `namespace` Variable sized identifiers, mainly intended to redefine c-namespace of an identifier. * `namespace` Variable sized identifiers, mainly intended to redefine c-namespace of an identifier.
* `,` is used to delimit arguments (if doing a find and replace). * `,` is used to delimit arguments (if doing a find and replace).
@ -30,20 +30,60 @@ The main benefit for using this over alts is its problably more ergonomic and pe
There are other programs more robust for doing that sort of thing but I was not able to find something this simple. There are other programs more robust for doing that sort of thing but I was not able to find something this simple.
### Note ### Example script
See `scripts/template_reafactor.ps1` and the `test/*.refactor` related scripts in on intended usage.
This is not setup to really be ergonomically used directly from a virtual terminal, instead run from a script after globbing the files you want.
There is a desire also to get this setup as a single-header library and also alternative with a minimalist GUI for simple refactors.
### Notes
* Building for debug provides some nice output with context on a per-line basis. * Building for debug provides some nice output with context on a per-line basis.
* Release will only show errors for asserts (that will kill the refactor early). * Release will only show errors for asserts (that will kill the refactor early).
* If the refactor crashes, the files previously written to will retain their changes. * If the refactor crashes, the files previously written to will retain their changes.
Make sure to have the code backed up on a VCS or in some other way. * Make sure to have the code backed up on a VCS or in some other way.
* This was compiled using meson with ninja and clang on windows 11. The ZPL library used however should work fine on the other major os platforms and compiler venders.
* The scripts used for building and otherwise are in the scripts directory and are all in powershell (with exception to the meson.build). Techncially there should be a powershell package available on other platorms but worst case it should be pretty easy to port these scripts to w/e shell script you'd perfer. * The scripts used for building and otherwise are in the scripts directory and are all in powershell (with exception to the meson.build). Techncially there should be a powershell package available on other platorms but worst case it should be pretty easy to port these scripts to w/e shell script you'd perfer.
TODO: ## Building
The project has all build configuration in the `scripts` directory.
* `build.ci.ps1` is intended for a continuous intergration setup (GH-worfklow for now).
* `build.ps1` is just a wrap of build.ci that just calls cls.
* `clean.ps1` will clean the workspace of all generated files.
* `get_sources.ps1` is used to glob sources since meson devs refuse to add dynamic retrival of sources for a build.
The project uses [meson](https://github.com/mesonbuild/meson) as the build tool.
Compiler : clang
OS: Windows 11 (windows-latest for github actions)
There should theoretically not be anything stopping it from building on other plaforms.
The library's main dependency is [zpl](https://github.com/zpl-c) which seems to support all major platforms.
## Testing
If the `test` parameter is provided to the build scripts, the project and thirdparty code will be refactored based on the specificiation files `*.refactor` residing in `test`.
With the refactors applied a meson configuraiton is setup (`meson.build` in test) and run to build. So long as it compiles, the feature set of the current version should work fine.
* There is an extra file `stb_image` that is currently be parsed but its unused.
* Planned for use in the namespace addition todo.
## TODO:
* Possibly come up with a better name. * Possibly come up with a better name.
* Test to see how much needs to be ported for other platforms (if at all) * Test to see how much needs to be ported for other platforms (if at all)
* Provide binaries in the release page for github. (debug and release builds) * Provide binaries in the release page for github. (debug and release builds)
* Directive to ignore comments (with a way to specify the comment signature). Right now comments that meet the signature of words or namespaces are refactored. * Ability to run and not emit any changes to files unless all files sucesffully are refactored.
* Would fix issue where a refactor overwrites files but failed to complete
* Can have a heavy memory cost, so most likely do not want on by default.
* Directive to ignore comments (with a way to specify the comment signature). Right now comments that
meet the signature of words or namespaces are refactored.
* Make comments ignored by default, and just have ability to specify custom comments.
* Would need a directive to add refactors to comments.
* Directive to add cpp namespaces on specific lines of a file, or near specific signatures.
* This can honestly be done also with placing words on specific lines..
* Provide a GUI build. * Provide a GUI build.
* Provide as a single-header library. * Provide as a single-header library.

View File

@ -3,4 +3,4 @@ __VERSION 1
not comments not comments
namespace stbi_ namespace stbi_
not namepsace stbi__ not namepsace stbi__

View File

@ -30,11 +30,11 @@ The program execution is pretty much outlined quite clearly in `int main()`.
There are constraints for specific variables; There are constraints for specific variables;
* `Path_Size_Largest` : Longest path size is set to 1 KB of characters. * `Path_Size_Largest` : Longest path size is set to 1 KB of characters.
* `Token_Max_Length` : Set to 1 KB characters as well. * `Token_Max_Length` : Set to 128 KB.
* `Array_Reserve_Num` : Is set to 4 KB. * `Array_Reserve_Num` : Is set to 4 KB.
* Initial Global arena size : Set to 2 megabytes. * Initial Global arena size : Set to 2 megabytes.
The `Path_Size_Largest` and `Token_Max_Length` are compile-time constraints that the runtime will not have a fallback for, if 1 KB is not enough it will need to be changed for your use case. The `Path_Size_Largest` and `Token_Max_Length` are compile-time constraints that the runtime will not have a fallback for, if current size is not enough it will need to be changed for your use case.
`Array_Reserve_Num` is used to dictate the assumed amount of tokens will be held in total for any of spec's arrays holding ignores and refactor entries. If any of the array's exceed 4 KB they will grow triggering a resize which will bog down the speed of the refactor. Adjust if you think you can increase or lower for use case. `Array_Reserve_Num` is used to dictate the assumed amount of tokens will be held in total for any of spec's arrays holding ignores and refactor entries. If any of the array's exceed 4 KB they will grow triggering a resize which will bog down the speed of the refactor. Adjust if you think you can increase or lower for use case.

View File

@ -7,7 +7,7 @@
namespace Spec namespace Spec
{ {
ct uw Array_Reserve_Num = zpl_kilobytes(4); ct uw Array_Reserve_Num = zpl_kilobytes(4);
ct uw Token_Max_Length = zpl_kilobytes(1); ct uw Token_Max_Length = 128;
namespace StaticData namespace StaticData
{ {

View File

@ -162,6 +162,11 @@ void parse_options( int num, char** arguments )
} }
/*
Refactor will problably have the execution and arg parsing (main and opts)
moved to a separate file.
*/
zpl_arena Refactor_Buffer; zpl_arena Refactor_Buffer;
void refactor() void refactor()
@ -211,21 +216,32 @@ void refactor()
#define pos (IO::Current_Size - left) #define pos (IO::Current_Size - left)
#define move_forward( Amount_ ) \ struct Snapshot
if ( left - Amount_ <= 0 ) \ {
goto End_Search; \ char const* Src;
\ sw Left;
left -= Amount_; \ uw Col;
col += Amount_; \ uw Line;
src += Amount_ \ };
#define move_back( Amount_ ) \ #define move_forward( Amount_ ) \
left += Amount_; \ if ( left - Amount_ <= 0 ) \
col -= Amount_; \ goto End_Search; \
src -= Amount_ \ \
line += src[0] == '\n'; \
left -= Amount_; \
col = (col + Amount_) * src[0] != '\n'; \
src += Amount_ \
#define restore( Snapshot_ ) \
src = Snapshot_.Src; \
left = Snapshot_.Left; \
col = Snapshot_.Col; \
line = Snapshot_.Line \
do do
{ {
// Check for comments if ignoring.
if ( Spec::Ignore_Comments && src[0] == '/' && left - 2 > 0 ) if ( Spec::Ignore_Comments && src[0] == '/' && left - 2 > 0 )
{ {
if ( src[1] == '/' ) if ( src[1] == '/' )
@ -259,37 +275,33 @@ void refactor()
{ {
Spec::Entry* ignore = Spec::Ignore_Includes; Spec::Entry* ignore = Spec::Ignore_Includes;
sw ignores_left = zpl_array_count( Spec::Ignore_Includes); sw ignores_left = zpl_array_count( Spec::Ignore_Includes);
sw rewind = 0; Snapshot backup = { src, left, col, line };
if ( '#' != src[0] ) if ( '#' != src[0] )
break; break;
move_forward( 1 ); move_forward( 1 );
rewind++;
// Ignore whitespace // Ignore whitespace
while ( zpl_char_is_space( src[0] ) ) while ( zpl_char_is_space( src[0] ) )
{ {
move_forward( 1 ); move_forward( 1 );
rewind++;
} }
if ( zpl_strncmp( include_sig, src, sizeof(include_sig) - 1 ) != 0 ) if ( zpl_strncmp( include_sig, src, sizeof(include_sig) - 1 ) != 0 )
{ {
move_back( rewind ); restore( backup );
break; break;
} }
const u32 sig_size = sizeof(include_sig) - 1; const u32 sig_size = sizeof(include_sig) - 1;
move_forward( sig_size ); move_forward( sig_size );
rewind += sig_size;
// Ignore whitespace // Ignore whitespace
while ( zpl_char_is_space( src[0] ) || src[0] == '\"' || src[0] == '<' ) while ( zpl_char_is_space( src[0] ) || src[0] == '\"' || src[0] == '<' )
{ {
move_forward(1); move_forward(1);
rewind++;
} }
for ( ; ignores_left; ignores_left--, ignore++ ) for ( ; ignores_left; ignores_left--, ignore++ )
@ -318,7 +330,7 @@ void refactor()
} }
} }
move_back( rewind ); restore( backup );
} }
while (false); while (false);
@ -402,37 +414,33 @@ void refactor()
{ {
Spec::Entry* include = Spec::Includes; Spec::Entry* include = Spec::Includes;
sw includes_left = zpl_array_count ( Spec::Includes); sw includes_left = zpl_array_count ( Spec::Includes);
sw rewind = 0; Snapshot backup = { src, left, col, line };
if ( '#' != src[0] ) if ( '#' != src[0] )
break; break;
move_forward( 1 ); move_forward( 1 );
rewind++;
// Ignore whitespace // Ignore whitespace
while ( zpl_char_is_space( src[0] ) ) while ( zpl_char_is_space( src[0] ) )
{ {
move_forward( 1 ); move_forward( 1 );
rewind++;
} }
if ( zpl_strncmp( include_sig, src, sizeof(include_sig) - 1 ) != 0 ) if ( zpl_strncmp( include_sig, src, sizeof(include_sig) - 1 ) != 0 )
{ {
move_back( rewind ); restore( backup );
break; break;
} }
const u32 sig_size = sizeof(include_sig) - 1; const u32 sig_size = sizeof(include_sig) - 1;
move_forward( sig_size ); move_forward( sig_size );
rewind += sig_size;
// Ignore whitespace // Ignore whitespace
while ( zpl_char_is_space( src[0] ) || src[0] == '\"' || src[0] == '<' ) while ( zpl_char_is_space( src[0] ) || src[0] == '\"' || src[0] == '<' )
{ {
move_forward( 1 ); move_forward( 1 );
rewind++;
} }
for ( ; includes_left; includes_left--, include++ ) for ( ; includes_left; includes_left--, include++ )
@ -473,7 +481,7 @@ void refactor()
} }
} }
move_back( rewind ); restore( backup );
} }
while (false); while (false);

View File

@ -31,7 +31,7 @@ if ( -not( Test-Path $path_build ) )
# Start-Process meson $args_meson -NoNewWindow -Wait -WorkingDirectory $path_scripts # Start-Process meson $args_meson -NoNewWindow -Wait -WorkingDirectory $path_scripts
Push-Location $path_scripts Push-Location $path_scripts
& meson $args_meson Invoke-Expression "& meson $args_meson"
Pop-Location Pop-Location
} }
@ -44,7 +44,7 @@ if ( $type )
# Start-Process meson $args_meson -NoNewWindow -Wait -WorkingDirectory $path_scripts # Start-Process meson $args_meson -NoNewWindow -Wait -WorkingDirectory $path_scripts
Push-Location $path_scripts Push-Location $path_scripts
meson $args_meson Invoke-Expression "& meson $args_meson"
Pop-Location Pop-Location
} }

View File

@ -8,7 +8,6 @@ $path_project = Join-Path $path_root project
$path_test = Join-Path $path_root test $path_test = Join-Path $path_root test
$path_thirdparty = Join-Path $path_root thirdparty $path_thirdparty = Join-Path $path_root thirdparty
$file_spec = Join-Path $path_test zpl.refactor
$refactor = Join-Path $path_build refactor.exe $refactor = Join-Path $path_build refactor.exe
# Gather the files to be formatted. # Gather the files to be formatted.
@ -27,12 +26,7 @@ foreach ( $file in $targetFiles )
write-host "Beginning thirdpary refactor..." write-host "Beginning thirdpary refactor..."
$refactors = @(@()) $file_spec = Join-Path $path_test zpl.refactor
if ( $args.Contains( "debug" ) )
{
$refactorParams += "-debug"
}
$refactorParams = @( $refactorParams = @(
"-src=$(Join-Path $path_thirdparty "zpl.h")", "-src=$(Join-Path $path_thirdparty "zpl.h")",
@ -40,22 +34,28 @@ $refactorParams = @(
"-spec=$($file_spec)" "-spec=$($file_spec)"
) )
& $refactor $refactorParams
$refactors = @(@())
$file_spec = Join-Path $path_test "stb_image.refactor"
if ( $args.Contains( "debug" ) ) if ( $args.Contains( "debug" ) )
{ {
$refactorParams += "-debug" $refactorParams += "-debug"
} }
write-host "`zpl refactor: " $refactorParams
& $refactor $refactorParams
$file_spec = Join-Path $path_test "stb_image.refactor"
$refactorParams = @( $refactorParams = @(
"-src=$(Join-Path $path_thirdparty "stb_image.h")", "-src=$(Join-Path $path_thirdparty "stb_image.h")",
"-dst=$(Join-Path $path_test "stb_image.refactored.h")", "-dst=$(Join-Path $path_test "stb_image.refactored.h")",
"-spec=$($file_spec)" "-spec=$($file_spec)"
) )
if ( $args.Contains( "debug" ) )
{
$refactorParams += "-debug"
}
write-host "`n`nstb_image refactor: " $refactorParams
& $refactor $refactorParams & $refactor $refactorParams
Write-Host "`nRefactoring complete`n" Write-Host "`nRefactoring complete`n"

View File

@ -35,4 +35,4 @@ $refactorParams = @(
& $refactor $refactorParams & $refactor $refactorParams
Write-Host "`nRefactoring complete`n`n" Write-Host "`nRefactoring complete`n`n"