mirror of
https://github.com/Ed94/raddebugger.git
synced 2026-06-12 23:31:38 -07:00
Ryan Fleury
78 lines
4.5 KiB
C
78 lines
4.5 KiB
C
// Copyright (c) 2024 Epic Games Tools
|
|
// Licensed under the MIT license (https://opensource.org/license/mit/)
|
|
|
|
#ifndef CTRL_INC_H
|
|
#define CTRL_INC_H
|
|
|
|
////////////////////////////////
|
|
//~ NOTE(rjf): Control Layer Overview (2023/8/29)
|
|
//
|
|
// This layer's purpose is to provide access to the asynchronously-running, low
|
|
// level parts of a debugger, running on the debugger client. This primarily
|
|
// consists of process control, using the Demon layer (the lower level
|
|
// abstraction layer for process control, across multiple OSes), but including
|
|
// higher-level concepts, like stepping, breakpoint resolution, conditional
|
|
// breakpoint evaluation, and so on. Right now, this just includes process
|
|
// control *local to the debugger client machine*. But in the future, this can
|
|
// also include communication to multiple target machines, all running their
|
|
// own process controller, using the Demon layer.
|
|
//
|
|
// This part of a debugger must run asynchronously to prevent blocking the UI -
|
|
// ideally our debugger is designed such that, if targets are running, the
|
|
// debugger frontend is still usable for a variety of purposes. So, in short,
|
|
// the asynchronously-running "control thread", implemented by this layer, is
|
|
// tasked with communicating with a separately executing "user thread". This
|
|
// communication happens in two directions - `user -> ctrl`, and the reverse,
|
|
// `ctrl -> user`.
|
|
//
|
|
// In the case of `user -> ctrl` communication, this is done with a ring buffer
|
|
// of "messages" (`CTRL_Msg`), pushed via `ctrl_u2c_push_msgs`. These messages
|
|
// include commands like: launching targets, attaching to targets, killing
|
|
// targets, detaching from targets, stepping/running, or single stepping.
|
|
//
|
|
// In the case of `ctrl -> user` communication, this is done with a ring buffer
|
|
// of "events" (`CTRL_Event`), popped via `ctrl_c2u_pop_events`. These events
|
|
// include information about what happened during the execution of targets -
|
|
// including: process/module/thread creation, process/module/thread deletion,
|
|
// debug strings, thread name events, memory allocation events, and stop events
|
|
// (where stops can be caused by: user breakpoints, traps set for stepping,
|
|
// exceptions, halts, or errors).
|
|
//
|
|
// The various stepping algorithms are implemented with two concepts: (a) the
|
|
// "trap net", and (b) "spoofs".
|
|
//
|
|
// A "trap net" is a term which refers to a set of addresses paired with a set
|
|
// of behavioral flags. Before targets run, trap instructions are written to
|
|
// these addresses. After targets stop, these addresses are reset to their
|
|
// original bytes. These trap instructions cause the debugger's targets to
|
|
// stop executing, and based on which behavioral flags are associated with
|
|
// the instruction causing the stop, the control thread may adjust parameters
|
|
// used for running, then continue execution, or it will not resume target
|
|
// execution, and will report stopped events. These behavioral flags can
|
|
// include: single-stepping the stopped thread to execute the instruction at
|
|
// the trap location, saving a stack pointer "check value" (where this check
|
|
// value is compared against when making decisions about whether to continue
|
|
// running or not), and so on. It's complicated to unpack why exactly these
|
|
// behaviors are useful, but the TL;DR of it is that they are used for a
|
|
// variety of stepping behaviors. For example, when doing a "step into" step,
|
|
// a `call` instruction can have a trap set at it, and will be marked with
|
|
// a "single-step-after" trap flag, as well as the "end stepping" trap flag,
|
|
// such that the step operation will complete after the `call` has executed.
|
|
//
|
|
// A "spoof" is a feature the control layer uses to detect when some thread
|
|
// returns from a particular sub-callstack. This is useful when implementing
|
|
// "step over" in functions that may be recursive. In short, unlike a trap,
|
|
// which writes a trap instruction (like `int3`) into an instruction stream,
|
|
// a spoof overwrites a *return address* on some thread's *stack*. This return
|
|
// address is not a valid address for executing code -- it is simply a value
|
|
// that the debugger can recognize, such that it is notified when the thread
|
|
// returns from some level in a callstack. When the thread exits some function,
|
|
// it will return to the "spoofed" address, and it will immediately hit an
|
|
// exception, because the spoofed address will not be a valid address for
|
|
// code execution. At that point, the debugger can move the thread back to
|
|
// the pre-spoof return address, and resume execution.
|
|
|
|
#include "ctrl_core.h"
|
|
|
|
#endif // CTRL_INC_H
|