Update readme, fix an issue with stb_image.refactor script (needed newline at eof), some script changes.

Readme.md

@@ -2,18 +2,18 @@
 
 Refactor c/c++ files (and problably others) with ease.
 
-## Parameters : 
+## Parameters 
 
 * `-num` : Used if more than one source file is provided (if used, number of destination files provided MUST MATCH).
 * `-src` : Source file to refactor
 * `-dst` : Destination file after the refactor (omit to use the same as source)
 * `-spec` : Specification containing rules to use for the refactor.
-* `-debug` : Use only if on debug build and desire to attach to process.
+* `-debug` : Use if on debug build and desire to attach to process, or to get a more verbose log.
 
-## Syntax : 
+## Syntax
 
 * `not` Omit word or namespace.
-* `include` Preprocessor include <file> related identifiers.
+* `include` Preprocessor include `<file path>` related identifiers.
 * `word` Fixed sized identifier.
 * `namespace` Variable sized identifiers, mainly intended to redefine c-namespace of an identifier.
 * `,` is used to delimit arguments (if doing a find and replace).
@@ -30,20 +30,60 @@ The main benefit for using this over alts is its problably more ergonomic and pe
 
 There are other programs more robust for doing that sort of thing but I was not able to find something this simple.
 
-### Note
+### Example script
+
+See `scripts/template_reafactor.ps1` and the `test/*.refactor` related scripts in on intended usage.
+
+This is not setup to really be ergonomically used directly from a virtual terminal, instead run from a script after globbing the files you want.
+
+There is a desire also to get this setup as a single-header library and also alternative with a minimalist GUI for simple refactors.
+
+### Notes
 
 * Building for debug provides some nice output with context on a per-line basis.  
 * Release will only show errors for asserts (that will kill the refactor early).  
 * If the refactor crashes, the files previously written to will retain their changes.
-Make sure to have the code backed up on a VCS or in some other way.
+  * Make sure to have the code backed up on a VCS or in some other way.
-* This was compiled using meson with ninja and clang on windows 11. The ZPL library used however should work fine on the other major os platforms and compiler venders.
 * The scripts used for building and otherwise are in the scripts directory and are all in powershell (with exception to the meson.build). Techncially there should be a powershell package available on other platorms but worst case it should be pretty easy to port these scripts to w/e shell script you'd perfer.
 
-TODO:  
+## Building
+
+The project has all build configuration in the `scripts` directory.  
+
+* `build.ci.ps1` is intended for a continuous intergration setup (GH-worfklow for now).  
+* `build.ps1` is just a wrap of build.ci that just calls cls.
+* `clean.ps1` will clean the workspace of all generated files.
+* `get_sources.ps1` is used to glob sources since meson devs refuse to add dynamic retrival of sources for a build.
+
+The project uses [meson](https://github.com/mesonbuild/meson) as the build tool.  
+Compiler : clang  
+OS: Windows 11 (windows-latest for github actions)
+
+There should theoretically not be anything stopping it from building on other plaforms.
+The library's main dependency is [zpl](https://github.com/zpl-c) which seems to support all major platforms.
+
+## Testing
+
+If the `test` parameter is provided to the build scripts, the project and thirdparty code will be refactored based on the specificiation files `*.refactor` residing in `test`.
+
+With the refactors applied a meson configuraiton is setup (`meson.build` in test) and run to build. So long as it compiles, the feature set of the current version should work fine.
+
+* There is an extra file `stb_image` that is currently be parsed but its unused.
+  * Planned for use in the namespace addition todo.
+
+## TODO:  
 
 * Possibly come up with a better name.
 * Test to see how much needs to be ported for other platforms (if at all)
 * Provide binaries in the release page for github. (debug and release builds)
-* Directive to ignore comments (with a way to specify the comment signature). Right now comments that meet the signature of words or namespaces are refactored.
+* Ability to run and not emit any changes to files unless all files sucesffully are refactored.
+  * Would fix issue where a refactor overwrites files but failed to complete
+  * Can have a heavy memory cost, so most likely do not want on by default.
+* Directive to ignore comments (with a way to specify the comment signature). Right now comments that
+meet the signature of words or namespaces are refactored.
+* Make comments ignored by default, and just have ability to specify custom comments.
+  * Would need a directive to add refactors to comments.
+* Directive to add cpp namespaces on specific lines of a file, or near specific signatures.  
+  * This can honestly be done also with placing words on specific lines..  
 * Provide a GUI build.
 * Provide as a single-header library.

project/Readme.md

@@ -30,11 +30,11 @@ The program execution is pretty much outlined quite clearly in `int main()`.
 There are constraints for specific variables;
 
 * `Path_Size_Largest` : Longest path size is set to 1 KB of characters.
-* `Token_Max_Length` : Set to 1 KB characters as well.
+* `Token_Max_Length` : Set to 128 KB.
 * `Array_Reserve_Num` : Is set to 4 KB. 
 * Initial Global arena size : Set to 2 megabytes.
 
-The `Path_Size_Largest` and `Token_Max_Length` are compile-time constraints that the runtime will not have a fallback for, if 1 KB is not enough it will need to be changed for your use case.
+The `Path_Size_Largest` and `Token_Max_Length` are compile-time constraints that the runtime will not have a fallback for, if current size is not enough it will need to be changed for your use case.
 
 `Array_Reserve_Num` is used to dictate the assumed amount of tokens will be held in total for any of spec's arrays holding ignores and refactor entries. If any of the array's exceed 4 KB they will grow triggering a resize which will bog down the speed of the refactor. Adjust if you think you can increase or lower for use case.
 

project/Spec.cpp

@@ -7,7 +7,7 @@
 namespace Spec
 {
 	ct uw Array_Reserve_Num = zpl_kilobytes(4);
-	ct uw Token_Max_Length  = zpl_kilobytes(1); 
+	ct uw Token_Max_Length  = 128; 
 
 	namespace StaticData
 	{

project/refactor.cpp

@@ -162,6 +162,11 @@ void parse_options( int num, char** arguments )
 }
 
 
+/*
+	Refactor will problably have the execution and arg parsing (main and opts) 
+	moved to a separate file.
+*/
+
 zpl_arena Refactor_Buffer;
 
 void refactor()
@@ -211,21 +216,32 @@ void refactor()
 
 	#define pos (IO::Current_Size - left)
 
+	struct Snapshot
+	{
+		char const* Src;
+		sw          Left;
+		uw          Col;
+		uw          Line;
+	};
+
 	#define move_forward( Amount_ )               \
 		if ( left - Amount_ <= 0 )                \
 			goto End_Search;                      \
 		                                          \
+		line += src[0] == '\n';                   \
 		left -= Amount_;                          \
-		col  += Amount_;            \
+		col   = (col + Amount_) * src[0] != '\n'; \
 		src  += Amount_                           \
 
-	#define move_back( Amount_ )    \
+	#define restore( Snapshot_ ) \
-		left += Amount_;            \
+		src  = Snapshot_.Src;    \
-		col  -= Amount_;            \
+		left = Snapshot_.Left;   \
-		src  -= Amount_             \
+		col  = Snapshot_.Col;    \
+		line = Snapshot_.Line    \
 
 	do
 	{
+		// Check for comments if ignoring.
 		if ( Spec::Ignore_Comments && src[0] == '/' && left - 2 > 0 )
 		{
 			if ( src[1] == '/' )
@@ -259,37 +275,33 @@ void refactor()
 		{
 			Spec::Entry* ignore       = Spec::Ignore_Includes;
 			sw           ignores_left = zpl_array_count( Spec::Ignore_Includes);
-			sw           rewind       = 0;
+			Snapshot      backup      = { src, left, col, line };
 
 			if ( '#' != src[0] )
 				break;
 
 			move_forward( 1 );
-			rewind++;
 
 			// Ignore whitespace
 			while ( zpl_char_is_space( src[0] ) )
 			{
 				move_forward( 1 );
-				rewind++;
 			}
 
 			if ( zpl_strncmp( include_sig, src, sizeof(include_sig) - 1 ) != 0 )
 			{
-				move_back( rewind );
+				restore( backup );
 				break;
 			}
 
 			const u32 sig_size = sizeof(include_sig) - 1;
 
 			move_forward( sig_size );
-			rewind += sig_size;
 
 			// Ignore whitespace
 			while ( zpl_char_is_space( src[0] ) || src[0] == '\"' || src[0] == '<' )
 			{
 				move_forward(1);
-				rewind++;
 			}
 
 			for ( ; ignores_left; ignores_left--, ignore++ )
@@ -318,7 +330,7 @@ void refactor()
 				}
 			}
 
-			move_back( rewind );
+			restore( backup );
 		} 
 		while (false);
 
@@ -402,37 +414,33 @@ void refactor()
 		{
 			Spec::Entry* include       = Spec::Includes;
 			sw           includes_left = zpl_array_count ( Spec::Includes);
-			sw           rewind       = 0;
+			Snapshot     backup        = { src, left, col, line };
 
 			if ( '#' != src[0] )
 				break;
 
 			move_forward( 1 );
-			rewind++;
 
 			// Ignore whitespace
 			while ( zpl_char_is_space( src[0] ) )
 			{
 				move_forward( 1 );
-				rewind++;
 			}
 
 			if ( zpl_strncmp( include_sig, src, sizeof(include_sig) - 1 ) != 0 )
 			{
-				move_back( rewind );
+				restore( backup );
 				break;
 			}
 
 			const u32 sig_size = sizeof(include_sig) - 1;
 
 			move_forward( sig_size );
-			rewind += sig_size;
 
 			// Ignore whitespace
 			while ( zpl_char_is_space( src[0] ) || src[0] == '\"' || src[0] == '<' )
 			{
 				move_forward( 1 );
-				rewind++;
 			}
 
 			for ( ; includes_left; includes_left--, include++ )
@@ -473,7 +481,7 @@ void refactor()
 				}
 			}
 
-			move_back( rewind );
+			restore( backup );
 		} 
 		while (false);
 

scripts/build.ci.ps1

@@ -31,7 +31,7 @@ if ( -not( Test-Path $path_build ) )
 
 	# Start-Process meson $args_meson -NoNewWindow -Wait -WorkingDirectory $path_scripts
 	Push-Location $path_scripts
-	& meson $args_meson
+	Invoke-Expression "& meson $args_meson"
 	Pop-Location
 }
 
@@ -44,7 +44,7 @@ if ( $type )
 
 	# Start-Process meson $args_meson -NoNewWindow -Wait -WorkingDirectory $path_scripts
 	Push-Location $path_scripts
-	meson $args_meson
+	Invoke-Expression "& meson $args_meson"
 	Pop-Location
 }
 

scripts/refactor_and_format.ps1

@@ -8,7 +8,6 @@ $path_project    = Join-Path $path_root project
 $path_test       = Join-Path $path_root test
 $path_thirdparty = Join-Path $path_root thirdparty
 
-$file_spec = Join-Path $path_test zpl.refactor
 $refactor  = Join-Path $path_build refactor.exe
 
 # Gather the files to be formatted.
@@ -27,12 +26,7 @@ foreach ( $file in $targetFiles )
 
 write-host "Beginning thirdpary refactor..."
 
-$refactors = @(@())
+$file_spec = Join-Path $path_test zpl.refactor
-
-if ( $args.Contains( "debug" ) )
-{
-    $refactorParams += "-debug"
-}
 
 $refactorParams = @(
     "-src=$(Join-Path $path_thirdparty "zpl.h")",
@@ -40,22 +34,28 @@ $refactorParams = @(
     "-spec=$($file_spec)"
 )
 
-& $refactor $refactorParams
-
-$refactors = @(@())
-$file_spec = Join-Path $path_test "stb_image.refactor"
-
 if ( $args.Contains( "debug" ) )
 {
     $refactorParams += "-debug"
 }
 
+write-host "`zpl refactor: " $refactorParams
+& $refactor $refactorParams
+
+$file_spec = Join-Path $path_test "stb_image.refactor"
+
 $refactorParams = @(
     "-src=$(Join-Path $path_thirdparty "stb_image.h")",
     "-dst=$(Join-Path $path_test "stb_image.refactored.h")",
     "-spec=$($file_spec)"
 )
 
+if ( $args.Contains( "debug" ) )
+{
+    $refactorParams += "-debug"
+}
+
+write-host "`n`nstb_image refactor: " $refactorParams
 & $refactor $refactorParams
 
 Write-Host "`nRefactoring complete`n"