Commit Graph

33 Commits

Author SHA1 Message Date
ed ed3246c6b0 Fixes for typedef serialization of functions..
Also fix for HashTable<>::rehash_fast not having finished implemenation...

The typedef fix is a sort of hack (like how parsing the rest of the language feels like tbh...).
I might make a def_typedef_fn to make it clearer how to define function typedefs using the upfront interface.
2023-08-07 20:16:04 -04:00
ed c2f8c8aeb1 Added constructor and destructor supported (UNTESTED)
Just compiles and generates...

Also fixed a bug where parsing didn't have a token for virtual specifiers...
2023-08-07 03:10:45 -04:00
ed c2319b9651 Fixes for test.singleheader_ast.cpp, also added a bench for it.
On a Ryzen R9 5950 it takes 11 ms to generate AST and 21 ms to serialize to file.
2023-08-06 17:46:17 -04:00
ed a4f9596d3b Fixes to serialization, reduced Define_CodeType macro
Now the execution code is generated in bootstrap/singleheader gen.
2023-08-06 17:19:57 -04:00
ed 97750388ad No longer using components/temp/ast_inlines (switched to helper function to avoid macro usage)
Increased the arg count support of num_args to 100.
2023-08-06 14:58:43 -04:00
ed 00f6c45f15 Fixes for serializations found with last commit's test.
Should be fine to move on to next major feature....
2023-08-06 13:28:19 -04:00
ed 34f286d218 Library can now construct into AST and serialization itself (singleheader).
Still need to validate if they match.
2023-08-04 16:12:13 -04:00
ed d36c3fa847 Single header generates again, some more cleanup.
Looking into properly dealing with empty lines...

I want to preserve the text's empty lines in the AST for serialization purposes (perserve formatting for gapes between definitions).
Don't want to introduce the possibility of it breaking though, so will have to ignore empty_lines in a general way (if they are in a bad spot).
Attempted to cover that by having TokArray::current() auto-skip empty lines and eat as well if the type doesn't match.
2023-08-03 23:18:33 -04:00
ed 5d7dfaf666 Heavy refactor..
Isolating large macros to their own directory (components/temp).
- Plan is to remove them soon with proper generation.

Added additional component files, separating the old data_structures header for a set of ast headers.
Header_end also had its inlines extracted out.
Necessary to complete the macro isolation.

ZPL parser dependencies were removed from the core library along with builder, its now generated in bootstrap as pare of making a gen_builder set of files.

Singleheader will be changed in next few commits to reflect this as well (By making builder deps and components a conditional option).

Tests are most likely all broken for now.
2023-08-03 11:01:43 -04:00
ed a8a9b681f0 test completes singleheader ast construction and serailizes with corruption 2023-08-02 14:01:56 -04:00
ed b96b0821c1 Fixes towards parsing (getting to line 12575 now of the singleheader. 2023-08-02 12:39:35 -04:00
ed 4c8a0f0005 Iterations on serialization improvements. 2023-08-01 20:56:00 -04:00
ed 684569750d first serialization of singlehearder without asserts. (Still failing after around 4k lines. 2023-08-01 16:07:47 -04:00
ed 0f16d1131e Got past parsing, fixing serialization 2023-08-01 14:02:54 -04:00
ed 528ef72a51 More progress on parsing
Made it to line 2597 of self parsing its singleheader

Complex global or member defintions are now supported.
2023-08-01 05:17:24 -04:00
ed 21a8f3bb39 WIP: It can parse to around ~2k lines. Need to improve its ability to detect when a forward declare of a class/enum/struct/union..
This language truly is a mess.
2023-08-01 00:42:08 -04:00
ed 2b63fc27cd Progress toward preprocessor parsing, lexing works, parsing does not. 2023-07-30 18:55:57 -04:00
ed bfbfae466f Naive preprocessor support initial implementation (compiles and runs, not heavily tested) 2023-07-30 01:21:04 -04:00
ed 03df940085 Improved parser scope errors.
Added the caret for indicating where the error is.
2023-07-29 17:14:02 -04:00
ed f09bb99fdf Fixes for test generation (sanity, soa). 2023-07-29 13:15:53 -04:00
ed 108b16739f bootstrap and singleheader compile and generate. 2023-07-29 12:25:38 -04:00
ed 689646c393 Finished iniital refactor pass. Comples, but has runtime issues. 2023-07-29 06:32:16 -04:00
ed c5afede7b5 Reorganization of files, refactors, doc updates (WIP)
Removing the gen. namespace from the files for components, dependencies, and file_processors.
They are only necessary if the include directory is transparent, and in my case those are not.

Made a docs directory. I'm offloading information from the main readme to there along with additional informationn I end up elaborating on down the line.
Enum tables were moved to their own directory (project/enums).

Library will not compile for now. Major refactor occuring with parsing related components.
2023-07-29 05:52:06 -04:00
ed 31e1c38c18 Started to implement context stack for parser. 2023-07-28 21:44:31 -04:00
ed d977c82f37 Fixes from latest refactor of toktype enum. 2023-07-27 17:12:58 -04:00
ed b00c1ae522 ECode, ESpecifier, and ETokType are now all generated.
There is a redundant pattern for generating all three (as expected).

I'll use it to define a general way of doing this sort of behavior.
2023-07-27 02:51:36 -04:00
ed cf65638979 Started to generate the enums from csv (ECode, EOperator, ESpecifier).
- Changed the zpl csv parser to only accept hex values with 0x perfix. it was messing with the add term.
- Small changes to the clang format config.
2023-07-26 14:21:20 -04:00
ed a11117c97c Fixes to the parsing constructors 2023-07-25 19:23:21 -04:00
ed 62b0ed2112 Finished initial implmentation bootstrap generation and singleheader implementation. 2023-07-25 15:12:51 -04:00
ed 9c81504178 Progress towards bootstrap/singleheader generation
I will most likely need to refactor some of the components & dependencies files to get the desired gneration implementation the way I want.

Specficially I want to be able to eliminate macros I'm using for enums and common patterns for implmeentation of the data structures.

When it comes to the cpp files, I may leave those alone as the macros largely help ith readability.
Replacing those macros is expensive and most likely not worth it.

The macros under consideration with replacing using the library bootstrap are:

* Define_Types
* Define_Operators
* Define_Specifiers
* GEN_Define_Attribute_Tokens
* Define_CodeType
* Using_Code
* Define_TokType
* def_constant_spec ?
* Helper Macros for def_**_body functions ?
  * AST unallowed types case macros?
(The last three I'm unsure about as they work fine, and neeeding the debugger steps there is a rare scenario...)

The enums could be manually generated and have its fields derived from a CSV (which is what genc is currently doing).
This would allow the user to specify custom attribute macros as well with greater ease.

I may maually inline ProcessModuleFlags, as its not even necessary as any specific symbol with a module flag will only use the export value. Import is only used on modules themselves (from what I can tell).

The Parser::lex function could be offloaded to its own file in case the user wants to swap the entire thing out.
(Most likely may want to for various purposes)

The problem with extracting any definitions out of a component file currently is that will lead to splintering that componnet to multiple other components.
This is necessary as the proper scanner is not implemented yet (only a reduimentary scan_file proc is made so far).
2023-07-24 22:19:21 -04:00
ed 387787b88d Formatting fixes 2023-07-24 20:59:20 -04:00
ed 74ea502de5 "compentiazation" of gen.hpp and gen.cpp 2023-07-24 17:45:27 -04:00
ed fcca15b4b9 Making new files, preparing to offload code to them. 2023-07-24 14:30:57 -04:00