Renamed BAPFS -> RDP, RDP completed.

This commit is contained in:
ed committed 2022-07-14 17:12:25 -04:00
1 parent 0acd6ecbaf
commit d4ee6574b2
53 files changed
+1617 -140

No files matched your search

File renamed without changes.

Before

Width:  |  Height:  |  Size: 495 KiB

After

Width:  |  Height:  |  Size: 495 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 467 KiB

@@ -0,0 +1,35 @@
[remap]
importer="texture"
type="StreamTexture"
path="res://.import/RegM_Class_cover_small.png-9128ac026427f18f59811eadf663fe9b.stex"
metadata={
"vram_texture": false
}
[deps]
source_file="res://Assets/Branding/RegM_Class_cover_small.png"
dest_files=[ "res://.import/RegM_Class_cover_small.png-9128ac026427f18f59811eadf663fe9b.stex" ]
[params]
compress/mode=0
compress/lossy_quality=0.7
compress/hdr_mode=0
compress/bptc_ldr=0
compress/normal_map=0
flags/repeat=0
flags/filter=true
flags/mipmaps=false
flags/anisotropic=false
flags/srgb=2
process/fix_alpha_border=true
process/premult_alpha=false
process/HDR_as_SRGB=false
process/invert_color=false
process/normal_map_invert_y=false
stream=false
size_limit=0
detect_3d=true
svg/scale=1.0
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -724,7 +724,7 @@ class Parser:
return node
# > ClassDeclaration
# SuperClass
# ClassExtension
# : extends Identifier
# ;
func parse_ClassExtension():
@@ -1256,8 +1256,7 @@ onready var TextOut = GScene.get_node("TextOutput")
onready var FDialog = GScene.get_node("Letter_FDialog")
onready var FD_Btn = GScene.get_node("ParseLetterFile_Btn")
func tout(text):
TextOut.insert_text_at_cursor(text)
func parse_file(path):
var \
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -1,92 +1,92 @@
Following the first lecture of "Building a Parser from scratch"
By Dmitry Soshnikov.
Lecture 1:
Phases:
Data - Text Content
Processor - Tokenizer
Data - Tokens
Processor - Parser
Data - AST
Example of syntaxes :
S-Expression :
(class Point
(begin
(def constructor (self x y)
(begin
(set (prop self x) x)
(set (prop self y) y)
)
)
(def calc (self)
(+ (prop self x)
(prop self y)
)
)
)
)
(var p (new Point 10 20))
((prop p calc) p)
User Syntax :
class Point
{
def constructor( x, y )
{
this.x = x;
this.y = y;
}
def calc() {
return this.x + this.y;
}
}
let
p = new Point(10, 20);
p.calc();
Tokenizer - Lexial Analysis : Uses Regular Expressions (Optimal)
Parser - Syntactic Analysis : Uses Backus-Naur Form
Backus-Naur Example :
Program
: StatementList
;
StatementList
: BlockStatement
| IfStatement
| FunctionDeclaration
...
;
FunctionDeclaration
: def Identifier ( Arguments ) BlockStatement
;
Hand-written parsers :
Use recursive descent.
Automatically generated
All kinds of stuff...
Following the first lecture of "Building a Parser from scratch"
By Dmitry Soshnikov.
Lecture 1:
Phases:
Data - Text Content
Processor - Tokenizer
Data - Tokens
Processor - Parser
Data - AST
Example of syntaxes :
S-Expression :
(class Point
(begin
(def constructor (self x y)
(begin
(set (prop self x) x)
(set (prop self y) y)
)
)
(def calc (self)
(+ (prop self x)
(prop self y)
)
)
)
)
(var p (new Point 10 20))
((prop p calc) p)
User Syntax :
class Point
{
def constructor( x, y )
{
this.x = x;
this.y = y;
}
def calc() {
return this.x + this.y;
}
}
let
p = new Point(10, 20);
p.calc();
Tokenizer - Lexial Analysis : Uses Regular Expressions (Optimal)
Parser - Syntactic Analysis : Uses Backus-Naur Form
Backus-Naur Example :
Program
: StatementList
;
StatementList
: BlockStatement
| IfStatement
| FunctionDeclaration
...
;
FunctionDeclaration
: def Identifier ( Arguments ) BlockStatement
;
Hand-written parsers :
Use recursive descent.
Automatically generated
All kinds of stuff...
@@ -1,11 +1,13 @@
[gd_scene load_steps=3 format=2]
[gd_scene load_steps=4 format=2]
[ext_resource path="res://Assets/Styles/EditorTheme.tres" type="Theme" id=1]
[ext_resource path="res://Assets/Branding/RDP_Class_cover_small.png" type="Texture" id=2]
[ext_resource path="res://RDP/Scripts/RDP_Viewer.gd" type="Script" id=3]
[node name="Control" type="Control"]
[node name="RDP_Panel" type="Panel"]
anchor_right = 1.0
anchor_bottom = 1.0
script = ExtResource( 3 )
[node name="CourseBrand" type="TextureRect" parent="."]
anchor_right = 1.0
@@ -15,8 +17,18 @@ texture = ExtResource( 2 )
expand = true
stretch_mode = 6
[node name="TextOutput" type="TextEdit" parent="."]
[node name="Tokens_TOut" type="TextEdit" parent="."]
anchor_left = 0.25
anchor_right = 0.5
anchor_bottom = 1.0
grow_horizontal = 0
theme = ExtResource( 1 )
readonly = true
highlight_current_line = true
show_line_numbers = true
[node name="AST_TOut" type="TextEdit" parent="."]
anchor_left = 0.5
anchor_right = 1.0
anchor_bottom = 1.0
grow_horizontal = 0
@@ -27,7 +39,6 @@ show_line_numbers = true
minimap_draw = true
[node name="Letter_FDialog" type="FileDialog" parent="."]
visible = true
anchor_left = 0.35
anchor_top = 0.15
anchor_right = 0.45
@@ -48,3 +59,12 @@ margin_left = 2.0
margin_right = 0.071991
margin_bottom = 14.6
text = "Parse a letter file"
[node name="Back_Btn" type="Button" parent="."]
anchor_top = 0.95
anchor_right = 0.247
anchor_bottom = 0.992
margin_left = 2.0
margin_right = 0.071991
margin_bottom = 2.59998
text = "Course Directory"
+13
View File
@@ -0,0 +1,13 @@
# Building a parser from scratch
[Img](https://i.imgur.com/rEzWwGs.png)
## Lecutres
Contains gd scripts that show the resulting progress at the end of each of their corresponding lectures.
## Scripts
Contains a final set of scripts for the course that cleans up the implementation to a final state.
+230
View File
@@ -0,0 +1,230 @@
extends Object
class_name Lexer
const TokenType : Dictionary = \
{
# Comments
cmt_SL = "Comment Single Line",
cmt_ML = "Comment Multi-Line",
# Formatting
fmt_S = "Formatting String",
# Delimiters
delim_Comma = "Comma Delimiter",
delim_SMR = "Symbol Member Resolution",
# Statements
def_End = "Statement End",
def_BStart = "Block Start",
def_BEnd = "Block End",
def_Var = "Variable Declaration",
def_Class = "Class",
# Iteration
def_While = "While",
def_Do = "Do-While",
def_For = "For",
# Procedures
def_Proc = "Procedure Declaration",
def_Return = "Return",
# Conditional
def_If = "If Statement",
def_Else = "Else Statement",
# Expressions
expr_PStart = "Parenthesis Start",
expr_PEnd = "Parenthesis End",
expr_SBStart = "Bracket Start",
expr_SBEnd = "Bracket End",
expr_New = "New Expression",
expr_Super = "Super Expression",
expr_Extends = "Class Extension",
# Operators
# Logical
op_Relational = "Relational",
op_Equality = "Equality",
op_LAnd = "Logical And",
op_LOr = "Logical Or",
op_LNot = "Logical Not",
# Arithmetic
op_CAssign = "ComplexAssignment",
op_Assign = "Assignment",
op_Additive = "AdditiveOperator",
op_Multiplicative = "MultiplicativeOperator",
# Literals
literal_BTrue = "True",
literal_BFalse = "False",
literal_Number = "Number",
literal_String = "String",
literal_Null = "Null Value",
# Symbols
sym_This = "This Reference",
sym_Identifier = "User Identifier",
}
const Spec : Dictionary = \
{
# Comments
TokenType.cmt_SL : "^\\/\\/.*",
TokenType.cmt_ML : "^\\/\\*[\\s\\S]*?\\*\\/",
# Formatting
TokenType.fmt_S : "^\\s+",
# Delimiters
TokenType.delim_Comma : "^,",
TokenType.delim_SMR : "^\\.",
# Statements
TokenType.def_End : "^;",
TokenType.def_BStart : "^{",
TokenType.def_BEnd : "^}",
TokenType.def_Var : "^\\blet\\b",
TokenType.def_Class : "^\\bclass\\b",
# Iteration
TokenType.def_While : "^\\bwhile\\b",
TokenType.def_Do : "^\\bdo\\b",
TokenType.def_For : "^\\bfor\\b",
# Procedures
TokenType.def_Proc : "^\\bdef\\b",
TokenType.def_Return : "^\\breturn\\b",
# Conditional
TokenType.def_If : "^\\bif\\b",
TokenType.def_Else : "^\\belse\\b",
# Expressions
TokenType.expr_PStart : "^\\(",
TokenType.expr_PEnd : "^\\)",
TokenType.expr_SBStart : "^\\[",
TokenType.expr_SBEnd : "^\\]",
TokenType.expr_New : "^\\bnew\\b",
TokenType.expr_Super : "^\\bsuper\\b",
TokenType.expr_Extends : "^\\bextends\\b",
#Operators
# Logical
TokenType.op_Relational : "^[>\\<]=?",
TokenType.op_Equality : "^[=!]=",
TokenType.op_LAnd : "^&&",
TokenType.op_LOr : "^\\|\\|",
TokenType.op_LNot : "^!",
# Arithmetic
TokenType.op_CAssign : "^[*\\/\\+\\-]=",
TokenType.op_Assign : "^=",
TokenType.op_Additive : "^[+\\-]",
TokenType.op_Multiplicative : "^[*\\/]",
# Literals
TokenType.literal_BTrue : "^\\btrue\\b",
TokenType.literal_BFalse : "^\\bfalse\\b",
TokenType.literal_Number : "^\\d+",
TokenType.literal_String : "^\"[^\"]*\"",
TokenType.literal_Null : "^\\bnull\\b",
# Symbols
TokenType.sym_This : "^\\bthis\\b",
TokenType.sym_Identifier : "^\\w+"
}
class Token:
var Type : String
var Value : String
var SourceText : String
var Cursor : int
var SpecRegex : Dictionary
var Tokens : Array
var TokenIndex : int = 0
func compile_regex():
for type in TokenType.values() :
var \
regex = RegEx.new()
regex.compile( Spec[type] )
SpecRegex[type] = regex
# SpecRegex[type].compile( Spec[type] )
func init(programSrcText):
SourceText = programSrcText
Cursor = 0
TokenIndex = 0
if SpecRegex.size() == 0 :
compile_regex()
tokenize()
func next_Token():
var nextToken = null
if Tokens.size() > TokenIndex :
nextToken = Tokens[TokenIndex]
TokenIndex += 1
return nextToken
func reached_EndOfText():
return Cursor >= SourceText.length()
func tokenize():
Tokens.clear()
while reached_EndOfText() == false :
var srcLeft = SourceText.substr(Cursor)
var token = Token.new()
var error = true
for type in TokenType.values() :
var result = SpecRegex[type].search( srcLeft )
if result == null || result.get_start() != 0 :
continue
# Skip Comments
if type == TokenType.cmt_SL || type == TokenType.cmt_ML :
Cursor += result.get_string().length()
error = false
break
# Skip Whitespace
if type == TokenType.fmt_S :
var addVal = result.get_string().length()
Cursor += addVal
error = false
break
token.Type = type
token.Value = result.get_string()
Cursor += ( result.get_string().length() )
Tokens.append( token )
error = false
break;
if error :
var assertStrTmplt = "next_token: Source text not understood by tokenizer at Cursor pos: {value} -: {txt}"
var assertStr = assertStrTmplt.format({"value" : Cursor, "txt" : srcLeft})
assert(true != true, assertStr)
return
File diff suppressed because it is too large. Load diff
+56
View File
@@ -0,0 +1,56 @@
extends Panel
var Lexer = preload("Lexer.gd").new()
var Parser = preload("Parser.gd").new()
onready var Tokens_TOut = get_node("Tokens_TOut")
onready var AST_TOut = get_node("AST_TOut")
onready var FDialog = get_node("Letter_FDialog")
onready var FD_Btn = get_node("ParseLetterFile_Btn")
onready var Back_Btn = get_node("Back_Btn")
func tokens_out(text):
Tokens_TOut.insert_text_at_cursor(text)
func ast_out(text):
AST_TOut.insert_text_at_cursor(text)
func parse_file(path):
var \
file = File.new()
file.open(path, File.READ)
var programDescription = file.get_as_text()
file.close()
Lexer.init(programDescription)
for token in Lexer.Tokens :
var string = "[" + token.Type + "] " + token.Value + "\n"
tokens_out( string )
var ast = Parser.parse(Lexer)
var json = JSON.print(ast.to_Dictionary(), '\t')
ast_out(json + "\n")
ast_out("Finished Parsing!\n")
func fd_btn_pressed():
FDialog.popup()
func fdialog_FSelected(path):
Tokens_TOut.text = ""
AST_TOut.text = ""
parse_file(path)
func backBtn_pressed():
queue_free()
# Called when the node enters the scene tree for the first time.
func _ready():
FDialog.connect("file_selected", self, "fdialog_FSelected")
FD_Btn.connect("pressed", self, "fd_btn_pressed")
Back_Btn.connect("pressed", self, "backBtn_pressed")
+5
View File
@@ -0,0 +1,5 @@
[gd_scene format=2]
[node name="RegM_Panel" type="Panel"]
anchor_right = 1.0
anchor_bottom = 1.0
+49
View File
@@ -0,0 +1,49 @@
[gd_scene load_steps=5 format=2]
[ext_resource path="res://Scripts/CourseDirectory.gd" type="Script" id=1]
[ext_resource path="res://Assets/Branding/RDP_Class_cover_small.png" type="Texture" id=2]
[ext_resource path="res://Assets/Branding/RegM_Class_cover_small.png" type="Texture" id=3]
[ext_resource path="res://Assets/Styles/EditorTheme.tres" type="Theme" id=4]
[node name="Panel" type="Panel"]
anchor_right = 1.0
anchor_bottom = 1.0
script = ExtResource( 1 )
[node name="HBox" type="HBoxContainer" parent="."]
anchor_left = 0.15
anchor_top = 0.25
anchor_right = 0.85
anchor_bottom = 0.75
alignment = 1
[node name="RDP_Btn" type="Button" parent="HBox"]
margin_left = 84.0
margin_right = 339.0
margin_bottom = 150.0
rect_min_size = Vector2( 255, 150 )
size_flags_horizontal = 0
size_flags_vertical = 0
theme = ExtResource( 4 )
icon = ExtResource( 2 )
icon_align = 1
expand_icon = true
[node name="VSeparator" type="VSeparator" parent="HBox"]
margin_left = 343.0
margin_right = 373.0
margin_bottom = 300.0
theme = ExtResource( 4 )
custom_constants/separation = 30
[node name="RegM_Btn" type="Button" parent="HBox"]
margin_left = 377.0
margin_right = 632.0
margin_bottom = 150.0
rect_min_size = Vector2( 255, 150 )
size_flags_horizontal = 0
size_flags_vertical = 0
theme = ExtResource( 4 )
icon = ExtResource( 3 )
icon_align = 1
expand_icon = true
+21
View File
@@ -0,0 +1,21 @@
extends Panel
onready var RDP_Viewer = load("res://RDP/RDP_Viewer.tscn")
onready var RegM_Viewer = load("res://RegM/RegM_Viewer.tscn")
onready var RDP_Btn = get_node("HBox/RDP_Btn")
onready var RegM_Btn = get_node("HBox/RegM_Btn")
func rdp_pressed():
add_child( RDP_Viewer.instance() )
func regM_pressed():
add_child( RegM_Viewer.instance() )
# Called when the node enters the scene tree for the first time.
func _ready():
RDP_Btn.connect("pressed", self, "rdp_pressed")
RegM_Btn.connect("pressed", self, "regM_pressed")
File renamed without changes.
View File
File renamed without changes.

Before

Width:  |  Height:  |  Size: 3.2 KiB

After

Width:  |  Height:  |  Size: 3.2 KiB

File renamed without changes.
+11 -3
View File
@@ -8,17 +8,25 @@
config_version=4
_global_script_classes=[ {
"base": "Object",
"class": "Lexer",
"language": "GDScript",
"path": "res://RDP/Scripts/Lexer.gd"
} ]
_global_script_class_icons={
"Lexer": ""
}
[application]
config/name="Parser"
run/main_scene="res://Persistent.tscn"
boot_splash/image="res://Assets/Branding/RDP_Class_cover_small.png"
config/icon="res://Assets/Branding/RDP_Class_cover_small.png"
[autoload]
GScene="*res://Lectures/AST_Viewer.tscn"
GScript="*res://Lectures/Lecture.18.gd"
GScene="*res://Scenes/CourseDirectory.tscn"
[gui]
+1 -1
View File
@@ -30,7 +30,7 @@ if not exist Engine\gd\bin\godot.windows.opt.tools.64.exe (
)
timeout 10
start Engine\gd\bin\godot.windows.opt.tools.64.exe -e Editor/project.godot
start Engine\gd\bin\godot.windows.opt.tools.64.exe -e App/project.godot
timeout 30
taskkill /f /im godot.windows.opt.tools.64.exe
+1 -1
View File
@@ -4,4 +4,4 @@
A set of material done while following [Dmitry Soshnikov's](https://t.co/UL7NQyykPu) courses
Currently I'm in the process of completing
![Building a parser from scratch](https://i.imgur.com/rEzWwGs.png)
![Automata: RegExp Machines](https://i.imgur.com/NWwBjhN.png)
+1 -1
View File
@@ -12,4 +12,4 @@ cd Builds
mkdir %SUBFILENAME%
cd ..\Engine\gd\bin
godot.windows.opt.64.exe --export "Windows Desktop" "..\Builds\%SUBFILENAME%\LangStudies.exe" --path "..\..\..\Editor"
godot.windows.opt.64.exe --export "Windows Desktop" "..\Builds\%SUBFILENAME%\LangStudies.exe" --path "..\..\..\App"
+1 -1
View File
@@ -1,3 +1,3 @@
start Engine\gd\bin\godot.windows.opt.tools.64.exe -e Editor/project.godot
start Engine\gd\bin\godot.windows.opt.tools.64.exe -e App/project.godot
+1 -1
View File
@@ -1,3 +1,3 @@
start Engine\gd\bin\godot.windows.tools.64.exe --path Editor/
start Engine\gd\bin\godot.windows.tools.64.exe --path App/
+1 -1
View File
@@ -1,3 +1,3 @@
start Engine\gd\bin\godot.windows.opt.64.exe --path Editor/
start Engine\gd\bin\godot.windows.opt.64.exe --path App/
+1 -1
View File
@@ -1,3 +1,3 @@
start Engine\gd\bin\godot.windows.opt.tools.64.exe --path Editor/
start Engine\gd\bin\godot.windows.opt.tools.64.exe --path App/
-12
View File
@@ -1,12 +0,0 @@
Godot Engine v3.3.4.stable.official.faf3f883d - https://godotengine.org
OpenGL ES 3.0 Renderer: NVIDIA GeForce GTX 1080/PCIe/SSE2
OpenGL ES Batching: ON
**ERROR**: Condition "err" is true. Returned: err
At: modules/gdscript/gdscript.cpp:815:load_source_code() - Condition "err" is true. Returned: err
**ERROR**: Cannot load source code from file 'C:/Projects/Study/LangStudies/Editor/Lecture.3.gd'.
At: modules/gdscript/gdscript.cpp:2241:load() - Condition "err != OK" is true. Returned: RES()
**ERROR**: Failed loading resource: C:/Projects/Study/LangStudies/Editor/Lecture.3.gd. Make sure resources have been imported by opening the project in the editor at least once.
At: core/io/resource_loader.cpp:282:_load() - Condition "found" is true. Returned: RES()
**ERROR**: Can't load script: C:/Projects/Study/LangStudies/Editor/Lecture.3.gd
At: main/main.cpp:1658:start() - Condition "script_res.is_null()" is true. Returned: false
-7
View File
@@ -1,7 +0,0 @@
Godot Engine v3.3.4.stable.official.faf3f883d - https://godotengine.org
OpenGL ES 3.0 Renderer: NVIDIA GeForce GTX 1080/PCIe/SSE2
OpenGL ES Batching: ON
Project is missing: C:/Projects/Study/Parsing/Editor/project.godot
**ERROR**: Condition "default_certs != __null" is true.
At: modules/mbedtls/crypto_mbedtls.cpp:201:load_default_certificates() - Condition "default_certs != __null" is true.
-12
View File
@@ -1,12 +0,0 @@
Godot Engine v3.3.4.stable.official.faf3f883d - https://godotengine.org
OpenGL ES 3.0 Renderer: NVIDIA GeForce GTX 1080/PCIe/SSE2
OpenGL ES Batching: ON
**ERROR**: Condition "err" is true. Returned: err
At: modules/gdscript/gdscript.cpp:815:load_source_code() - Condition "err" is true. Returned: err
**ERROR**: Cannot load source code from file 'C:/Projects/Study/LangStudies/Editor/Lecture.3.gd'.
At: modules/gdscript/gdscript.cpp:2241:load() - Condition "err != OK" is true. Returned: RES()
**ERROR**: Failed loading resource: C:/Projects/Study/LangStudies/Editor/Lecture.3.gd. Make sure resources have been imported by opening the project in the editor at least once.
At: core/io/resource_loader.cpp:282:_load() - Condition "found" is true. Returned: RES()
**ERROR**: Can't load script: C:/Projects/Study/LangStudies/Editor/Lecture.3.gd
At: main/main.cpp:1658:start() - Condition "script_res.is_null()" is true. Returned: false