diff --git a/README.md b/README.md index 1adbf516..7d79280a 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ with any information you can gather, like dump files (along with the build you used), instructions to reproduce, test executables, and so on. You can download pre-built binaries for the debugger -[here](https://github.com/EpicGames/raddebugger/releases). +[here](https://github.com/EpicGamesExt/raddebugger/releases). The RAD Debugger project aims to simplify the debugger by simplifying and unifying the underlying debug info format. In that pursuit we've built the RAD @@ -90,13 +90,12 @@ You should see the following output: ``` [debug mode] [msvc compile] -[default mode, assuming `raddbg` build] metagen_main.c -searching C:\devel\raddebugger/src... 299 files found -parsing metadesk... 12 metadesk files parsed -gathering tables... 37 tables found +searching C:\devel\raddebugger/src... 309 files found +parsing metadesk... 15 metadesk files parsed +gathering tables... 96 tables found generating layer code... -raddbg.cpp +raddbg_main.c ``` If everything worked correctly, there will be a `build` folder in the root @@ -169,7 +168,8 @@ After setting up the codebase and building, the following directories will also exist: - `build`: All build artifacts. Not checked in to version control. -- `local`: Local files, used for local build configuration input files. +- `local`: Local files, used for local build configuration input files. Not + checked in to version control. ## Codebase Introduction @@ -213,55 +213,70 @@ A list of the layers in the codebase and their associated namespaces is below: processes. Runs in lockstep with attached processes. When it runs, attached processes are halted. When attached processes are running, it is halted. Driven by a debugger frontend on another thread. -- `dasm` (`DASM_`): An asynchronous disassembly decoder and cache. Users ask for - disassembly for a particular virtual address range in a process, and threads - implemented in this layer decode and cache the disassembly for that range. +- `dasm_cache` (`DASM_`): An asynchronous disassembly decoder and cache. Users + ask for disassembly for some data, with a particular architecture, and other + various parameters, and threads implemented in this layer decode and cache the + disassembly for that data with those parameters. - `dbgi` (`DI_`): An asynchronous debug info loader and cache. Loads debug info stored in the RDI format. Users ask for debug info for a particular path, and on separate threads, this layer loads the associated debug info file. If necessary, it will launch a separate conversion process to convert original debug info into the RDI format. -- `demon` (`DEMON_`): An abstraction layer for local-machine, low-level process +- `dbg_engine` (`D_`): Implements the core debugger system, without any + graphical components. This contains top-level logic for things like stepping, + launching, freezing threads, mid-run breakpoint addition, some caching layers, + and so on. +- `demon` (`DMN_`): An abstraction layer for local-machine, low-level process control. The abstraction is used to provide a common interface for process control on target platforms. Used to implement part of `ctrl`. -- `df/core` (`DF_`): The debugger's non-graphical frontend. Implements a - debugger "entity cache" (where "entities" include processes, threads, modules, - breakpoints, source files, targets, and so on). Implements a command loop - for driving process control, which is used to implement stepping commands and - user breakpoints. Implements extractors and caches for various entity-related - data, like full thread unwinds and local variable maps. Also implements core - building blocks for evaluation and evaluation visualization. -- `df/gfx` (`DF_`): The debugger's graphical frontend. Builds on top of - `df/core` to provide all graphical features, including windows, panels, all - of the various debugger interfaces, and evaluation visualization. -- `draw` (`D_`): Implements a high-level graphics drawing API for the debugger's - purposes, using the underlying `render` abstraction layer. Provides high-level - APIs for various draw commands, but takes care of batching them, and so on. -- `eval` (`EVAL_`): Implements a compiler for an expression language built for - evaluation of variables, registers, and so on from debugger-attached processes - and/or debug info. Broken into several phases mostly corresponding to - traditional compiler phases - lexer, parser, type-checker, IR generation, and - IR evaluation. -- `font_cache` (`F_`): Implements a cache of rasterized font data, both in CPU- - side data for text shaping, and in GPU texture atlases for rasterized glyphs. - All cache information is sourced from the `font_provider` abstraction layer. +- `draw` (`DR_`): Implements a high-level graphics drawing API for the + debugger's purposes, using the underlying `render` abstraction layer. Provides + high-level APIs for various draw commands, but takes care of batching them, + and so on. +- `eval` (`E_`): Implements a compiler for an expression language built for + evaluation of variables, registers, types, and more, from debugger-attached + processes, debug info, debugger state, and files. Broken into several phases + mostly corresponding to traditional compiler phases - lexer, parser, + type-checker, IR generation, and IR evaluation. +- `eval_visualization` (`EV_`): Implements the core non-graphical evaluation + visualization engine, which can be used to visualize evaluations (provided by + the `eval` layer) in a number of ways. Implements core data structures and + transforms for the `Watch` view. +- `file_stream` (`FS_`): Provides asynchronous file loading, storing the + artifacts inside of the cache implemented by the `hash_store` layer, and + hot-reloading the contents of files when they change. Allows callers to map + file paths to data hashes, which can then be used to obtain the file's data. +- `font_cache` (`FNT_`): Implements a cache of rasterized font data, both in + CPU-side data for text shaping, and in GPU texture atlases for rasterized + glyphs. All cache information is sourced from the `font_provider` abstraction + layer. - `font_provider` (`FP_`): An abstraction layer for various font file decoding and font rasterization backends. +- `fuzzy_search` (`FZY_`): Provides a fuzzy searching engine for doing + large, asynchronous fuzzy searches. Used by the debugger for implementing + things like the symbol lister or the `Procedures` view, which search across + all loaded debug info records, using fuzzy matching rules. - `geo_cache` (`GEO_`): Implements an asynchronously-filled cache for GPU geometry data, filled by data sourced in the `hash_store` layer's cache. Used - for asynchronously preparing data for memory visualization in the debugger. + for asynchronously preparing data for visualization. - `hash_store` (`HS_`): Implements a cache for general data blobs, keyed by a - 128-bit hash of the data. Used as a general data store by other layers. + 128-bit hash of the data. Also implements a 128-bit key cache on top, where + the keys refer to a unique identity, associated with a 128-bit hash, where the + hash may change across time. Used as a general data store by other layers. - `lib_raddbg_markup` (`RADDBG_`): Standalone library for marking up user - programs to work with various features in the `raddbg` debugger. Does not - depend on `base`, and can be independently relocated to other codebases. -- `lib_rdi_make` (`RDIM_`): Standalone library for constructing RDI debug info - data. Does not depend on `base`, and can be independently relocated - to other codebases. + programs to work with various features in the debugger. Does not depend on + `base`, and can be independently relocated to other codebases. - `lib_rdi_format` (`RDI_`): Standalone library which defines the core RDI types and helper functions for reading and writing the RDI debug info file format. Does not depend on `base`, and can be independently relocated to other codebases. +- `lib_rdi_make` (`RDIM_`): Standalone library for constructing RDI debug info + data. Does not depend on `base`, and can be independently relocated + to other codebases. +- `mdesk` (`MD_`): Code for parsing Metadesk files (stored as `.mdesk`), which + is the JSON-like (technically a JSON superset) text format used for the + debugger's user and project configuration files, view rules, and metacode, + which is parsed and used to generate code with the `metagen` layer. - `metagen` (`MG_`): A metaprogram which is used to generate primarily code and data tables. Consumes Metadesk files, stored with the extension `.mdesk`, and generates C code which is then included by hand-written C code. Currently, it @@ -279,6 +294,9 @@ A list of the layers in the codebase and their associated namespaces is below: - `msf` (`MSF_`): Code for parsing and/or writing the MSF file format. - `mule` (no namespace): Test executables for battle testing debugger functionality. +- `mutable_text` (`MTX_`): Implements an asynchronously-filled-and-mutated + cache for text buffers which are mutated across time. In the debugger, this is + used to implement the `Output` view. - `natvis` (no namespace): NatVis files for type visualization of the codebase's types in other debuggers. - `os/core` (`OS_`): An abstraction layer providing core, non-graphical @@ -287,20 +305,28 @@ A list of the layers in the codebase and their associated namespaces is below: - `os/gfx` (`OS_`): An abstraction layer, building on `os/core`, providing graphical operating system features under an abstract API, which is implemented per-target-operating-system. -- `os/socket` (`OS_`): An abstraction layer, building on `os/core`, providing - networking operating system features under an abstract API, which is - implemented per-target-operating-system. +- `path` (`PATH_`): Small helpers for manipulating file path strings. - `pdb` (`PDB_`): Code for parsing and/or writing the PDB file format. - `pe` (`PE_`): Code for parsing and/or writing the PE (Portable Executable) file format. -- `raddbg` (no namespace): The layer which ties everything together for the main - graphical debugger. Not much "meat", just drives `df`, implements command line - options, and so on. -- `rdi_from_pdb` (`P2R_`): Our implementation of PDB-to-RDI conversion. -- `rdi_from_dwarf` (`D2R_`): Our in-progress implementation of DWARF-to-RDI - conversion. +- `raddbg` (`RD_`): The layer which ties everything together for the main + graphical debugger. Implements the debugger's graphical frontend, all of the + debugger-specific UI, the debugger executable's command line interface, and + all of the built-in visualizers. +- `rdi_breakpad_from_pdb` (`P2B_`): Our implementation, using the codebase's RDI + technology, for extracting information from PDBs and generating Breakpad text + dumps. - `rdi_dump` (no namespace): A dumper utility program for dumping textualizations of RDI debug info files. +- `rdi_format` (no namespace): A layer which includes the `lib_rdi_format` layer + and bundles it with codebase-specific helpers, to easily include the library + in codebase programs, and have it be integrated with codebase constructs. +- `rdi_from_dwarf` (`D2R_`): Our in-progress implementation of DWARF-to-RDI + conversion. +- `rdi_from_pdb` (`P2R_`): Our implementation of PDB-to-RDI conversion. +- `rdi_make` (no namespace): A layer which includes the `lib_rdi_make` layer and + bundles it with codebase-specific helpers, to easily include the library in + codebase programs, and have it be integrated with codebase constructs. - `regs` (`REGS_`): Types, helper functions, and metadata for registers on supported architectures. Used in reading/writing registers in `demon`, or in looking up register metadata. @@ -309,19 +335,19 @@ A list of the layers in the codebase and their associated namespaces is below: level drawing API - this layer is strictly for minimally abstracting on an as-needed basis. Higher level drawing features are implemented in the `draw` layer. -- `scratch` (no namespace): Scratch space for small and transient test or sample - programs. +- `scratch` (no namespace): Scratch space for small and transient test programs. +- `task_system` (`TS_`): Implements a system for kicking off asynchronous tasks + and obtaining artifacts from those tasks once they've completed. - `texture_cache` (`TEX_`): Implements an asynchronously-filled cache for GPU texture data, filled by data sourced in the `hash_store` layer's cache. Used - for asynchronously preparing data for memory visualization in the debugger. -- `txti` (`TXTI_`): Machinery for asynchronously-loaded, asynchronously hot- - reloaded, asynchronously parsed, and asynchronously mutated source code files. - Used by the debugger to visualize source code files. Users ask for text lines, - tokens, and metadata, and it is prepared on background threads. -- `type_graph` (`TG_`): Code for analyzing and navigating type structures from - RDI debug info files, with the additional capability of constructing - synthetic types *not* found in debug info. Used in `eval` and for various - visualization features. + for asynchronously preparing data for visualization. +- `text_cache` (`TXT_`): Implements an asynchronously-filled cache for textual + analysis data (tokens, line ranges, and so on), filled by data sourced in the + `hash_store` layer's cache. Used for asynchronously preparing data for + visualization (like for the source code viewer). +- `third_party` (no namespace): External code from other projects, which some + layers in the codebase depend on. All external code is included and built + directly within the codebase. - `ui` (`UI_`): Machinery for building graphical user interfaces. Provides a core immediate mode hierarchical user interface data structure building API, and has helper layers for building some higher-level widgets. diff --git a/src/ico/ico.c b/src/ico/ico.c deleted file mode 100644 index 8ac6bce5..00000000 --- a/src/ico/ico.c +++ /dev/null @@ -1,2 +0,0 @@ -// Copyright (c) 2024 Epic Games Tools -// Licensed under the MIT license (https://opensource.org/license/mit/) diff --git a/src/ico/ico.h b/src/ico/ico.h deleted file mode 100644 index 8230898d..00000000 --- a/src/ico/ico.h +++ /dev/null @@ -1,43 +0,0 @@ -// Copyright (c) 2024 Epic Games Tools -// Licensed under the MIT license (https://opensource.org/license/mit/) - -#ifndef ICO_H -#define ICO_H - -//////////////////////////////// -//~ rjf: ICO File Format Types - -#pragma pack(push, 1) - -typedef struct ICO_Header ICO_Header; -struct ICO_Header -{ - U16 reserved_padding; // must be 0 - U16 image_type; // if 1 -> ICO, if 2 -> CUR - U16 num_images; -}; - -typedef struct ICO_Entry ICO_Entry; -struct ICO_Entry -{ - U8 image_width_px; - U8 image_height_px; - U8 num_colors; - U8 reserved_padding; // should be 0 - union - { - U16 ico_color_planes; // in ICO - U16 cur_hotspot_x_px; // in CUR - }; - union - { - U16 ico_bits_per_pixel; // in ICO - U16 cur_hotspot_y_px; // in CUR - }; - U32 image_data_size; - U32 image_data_off; -}; - -#pragma pack(pop) - -#endif // ICO_H diff --git a/src/raddbg/raddbg_core.c b/src/raddbg/raddbg_core.c index 4878785f..4b79b5c2 100644 --- a/src/raddbg/raddbg_core.c +++ b/src/raddbg/raddbg_core.c @@ -10609,6 +10609,35 @@ rd_init(CmdLine *cmdln) U8 *opl = ptr+data.size; // rjf: read header +#pragma pack(push, 1) + typedef struct ICO_Header ICO_Header; + struct ICO_Header + { + U16 reserved_padding; // must be 0 + U16 image_type; // if 1 -> ICO, if 2 -> CUR + U16 num_images; + }; + typedef struct ICO_Entry ICO_Entry; + struct ICO_Entry + { + U8 image_width_px; + U8 image_height_px; + U8 num_colors; + U8 reserved_padding; // should be 0 + union + { + U16 ico_color_planes; // in ICO + U16 cur_hotspot_x_px; // in CUR + }; + union + { + U16 ico_bits_per_pixel; // in ICO + U16 cur_hotspot_y_px; // in CUR + }; + U32 image_data_size; + U32 image_data_off; + }; +#pragma pack(pop) ICO_Header hdr = {0}; if(ptr+sizeof(hdr) < opl) { diff --git a/src/raddbg/raddbg_main.c b/src/raddbg/raddbg_main.c index 78aed49d..dc7f19e8 100644 --- a/src/raddbg/raddbg_main.c +++ b/src/raddbg/raddbg_main.c @@ -538,7 +538,6 @@ #include "base/base_inc.h" #include "os/os_inc.h" #include "task_system/task_system.h" -#include "ico/ico.h" #include "rdi_format/rdi_format_local.h" #include "rdi_make/rdi_make_local.h" #include "mdesk/mdesk.h" @@ -578,7 +577,6 @@ #include "base/base_inc.c" #include "os/os_inc.c" #include "task_system/task_system.c" -#include "ico/ico.c" #include "rdi_format/rdi_format_local.c" #include "rdi_make/rdi_make_local.c" #include "mdesk/mdesk.c"