manual_slop/docs/guide_hot_reload.md

Hot Reload (State-Preserving Module Reloading)

Top | Architecture | Tools & IPC | Simulations

---

Overview

Manual Slop's Hot Reload enables selective, state-preserving reloading of Python modules at runtime. Developers can iterate on UI logic and other registered modules without restarting the application or losing the current session state. A hot reload is triggered manually via a keyboard shortcut (Ctrl+Alt+R) or a GUI button.

This guide covers:

1. Architecture — Why state-preserving reload matters
2. Components — HotReloader, HotModule, registration protocol
3. Reload Lifecycle — Capture → Reload → Restore (or Rollback)
4. Error Handling — Visual error tint feedback
5. Registration — How a module opts into hot reload
6. Testing — The integration test pattern

Key constraint: Hot Reload is for stateless logic modules (renderers, formatters, pure functions) and delegation-pattern modules (functions that take app: App as a parameter and don't mutate app state directly). It is not safe for modules that hold long-lived mutable references to the App instance or to other singletons, because importlib.reload() creates a new module object but doesn't replace existing references held by callers.

The safe pattern is: the App class contains thin delegation wrappers that call module-level functions. The module-level functions can be reloaded freely. The wrappers never need to be reloaded because they only call through to the function references, which the Hot Reload mechanism can swap atomically.

---

Architecture

Hot Reload lives at the boundary between the GUI delegation pattern and the runtime module system. It enables the development workflow shown below.

┌──────────────────────────────────────────────┐
│  App class (src/gui_2.py)                    │
│  - Holds long-lived state (history, comms,  │
│    selected files, discussion entries)        │
│  - Defines thin delegation wrappers:         │
│      def _render_xxx(self): render_xxx(self)│
│  - Owns the reload() trigger                │
└──────────────────┬───────────────────────────┘
                   │ delegates to
                   ▼
┌──────────────────────────────────────────────┐
│  Module-level functions (e.g. render_xxx)   │
│  - Pure: def render_xxx(app: App): ...       │
│  - Stateless: no module-level mutable state  │
│  - Re-registerable on import                 │
└──────────────────┬───────────────────────────┘
                   │ managed by
                   ▼
┌──────────────────────────────────────────────┐
│  HotReloader (src/hot_reloader.py)           │
│  - HOT_MODULES dict: name -> HotModule       │
│  - capture_state / restore_state             │
│  - reload(module_name) -> bool               │
│  - Tracks last_error, is_error_state         │
└──────────────────────────────────────────────┘

Why the delegation pattern matters:

• The App's _render_xxx wrappers are bound methods that call the module-level function by name (render_xxx(self)).
• After importlib.reload(), the module-level render_xxx is replaced with a new function object.
• Subsequent calls to app._render_xxx() use Python's late binding: the wrapper's render_xxx reference is resolved at call time, picking up the new function.
• The App's state (history, comms, etc.) is untouched.

Without the delegation pattern, hot reload would be unsafe. If App._render_xxx were a method defined directly on the class, reloading the module wouldn't update the bound method on existing App instances.

---

Components

HotModule (Data Class)

Describes a module that's eligible for hot reload.

@dataclass
class HotModule:
    name: str                                          # Module identifier (e.g., "src.gui_2_render")
    file_path: str                                     # Source file path (for error display)
    state_keys: list[str] = field(default_factory=list) # App attributes to preserve
    delegation_targets: list[str] = field(default_factory=list)  # Function names that the App calls into this module

Field	Type	Purpose
name	str	Unique identifier. Typically matches the Python module name (e.g., src.gui_2).
file_path	str	Absolute path to the source file. Used in error messages.
state_keys	list[str]	Names of attributes on the App to capture before reload and restore on error.
delegation_targets	list[str]	Names of the module-level functions that the App calls. Currently informational; future hot-swap mechanism.

HotReloader (Class)

The reloader itself. Uses class-level state (not instance state) so it can be invoked without dependency injection.

class HotReloader:
    HOT_MODULES: dict[str, HotModule] = {}      # Registered modules
    last_error: str | None = None                # Last reload error (formatted traceback)
    is_error_state: bool = False                # True if last reload failed

Public Methods:

@classmethod
def register(cls, module: HotModule) -> None:
    """Register a module for hot reload. Raises ValueError if already registered."""

@classmethod
def capture_state(cls, app: Any, state_keys: list[str]) -> dict[str, Any]:
    """Deep-copy the specified App attributes. Returns a dict of {key: value}."""

@classmethod
def restore_state(cls, app: Any, state: dict[str, Any]) -> None:
    """Restore previously-captured state to the App."""

@classmethod
def reload(cls, module_name: str, app: Any) -> bool:
    """Reload a registered module. On error, restore state and return False. Returns True on success."""

@classmethod
def reload_all(cls, app: Any) -> bool:
    """Reload all registered modules. Returns True only if ALL succeed."""

Class-level state design: HOT_MODULES, last_error, and is_error_state are class attributes (not instance attributes). This is intentional — it means:

• The reloader has no constructor state.
• Any code can check HotReloader.is_error_state without instantiating.
• The reloader is process-global; it tracks the most recent reload operation.

This is a single-developer, single-instance tool, so global state is appropriate. A multi-tenant or multi-window tool would refactor this into instance state.

---

Reload Lifecycle

The reload() method follows a strict capture-reload-restore-or-rollback protocol.

Sequence

1. Lookup module in HOT_MODULES
   - If not registered: set last_error, is_error_state = True, return False

2. state = capture_state(app, hm.state_keys)
   - Deep copy of each listed App attribute
   - Attributes not present on App are skipped silently

3. try:
     importlib.reload(sys.modules[module_name])
     # OR importlib.import_module if not yet loaded
     last_error = None
     is_error_state = False
     return True
   except Exception:
     # 4. Rollback
     restore_state(app, state)
     last_error = traceback.format_exc()
     is_error_state = True
     return False

Step 2: capture_state

@classmethod
def capture_state(cls, app: Any, state_keys: list[str]) -> dict[str, Any]:
    return {key: copy.deepcopy(getattr(app, key, None)) for key in state_keys if hasattr(app, key)}

• Uses copy.deepcopy() to ensure the captured state is independent of the App's live state.
• getattr(app, key, None) returns None if the attribute is missing — this is the default fallback.
• hasattr(app, key) filters out the None defaults to avoid storing phantom keys.
• If state_keys is empty (the default for HotModule), the captured state is {} and rollback is a no-op.

Step 3: importlib.reload

import importlib
import sys
if module_name in sys.modules:
    old_module = sys.modules[module_name]
    importlib.reload(old_module)
else:
    importlib.import_module(module_name)

• If the module is already imported, importlib.reload re-executes the module's top-level code in the existing module's namespace.
• If not yet imported, it just imports normally.
• Reload is synchronous and blocking. For long reloads, the GUI may briefly stutter. This is acceptable for a developer tool.

Step 4: Rollback on Error

If the reload raises an exception:

• The captured state is restored via setattr(app, key, value) for each captured key.
• The App's state is exactly as it was before the reload attempt.
• The error traceback is captured in last_error for display in the GUI.

Note: Rollback restores the App's state attributes but does not restore the module to its pre-reload state. If the module's top-level code partially executed (e.g., defined some new functions but failed on a later import), the module is in a partially-reloaded state. This is a known limitation; the next reload attempt (after the developer fixes the source) will overwrite the partial state.

---

Error Handling

When a reload fails, the GUI provides visual error feedback:

1. The status bar shows a red tint for 3 seconds.
2. The "Hot Reload" button shows a "Failed" state until the next reload attempt.
3. The last_error is displayed in a popup or notification panel (configurable).

The error is captured via Python's standard traceback.format_exc() and stored in HotReloader.last_error. The GUI's _render_hot_reload_status reads this attribute on each frame and renders accordingly.

Failure modes:

Failure	Handling
Module not registered	last_error = "Module <name> not registered", is_error_state = True
ImportError on reload	last_error = traceback, state restored
AttributeError during top-level code	Same as ImportError
Module raises during side effects (e.g., registering a callback)	Same as ImportError

Recovery: The next reload attempt (after the developer fixes the source) starts fresh. is_error_state is set to False only on a successful reload.

---

Registration

To opt a module into hot reload, register it at module import time:

# At the top of src/gui_2.py (or in a separate registration file)
from src.hot_reloader import HotReloader, HotModule

HotReloader.register(HotModule(
    name="src.gui_2",
    file_path="C:/projects/manual_slop/src/gui_2.py",
    state_keys=[
        "ai_input",
        "project_system_prompt",
        "global_system_prompt",
        "discussion_history",
        "selected_files",
    ],
    delegation_targets=[
        "render_main_window",
        "render_context_panel",
        "render_ai_settings",
        "render_discussion_hub",
        "render_mma_dashboard",
    ],
))

When to register: At module import time, ideally near the top of the file. The registration is idempotent — calling register() twice with the same name raises ValueError to prevent accidental double-registration.

What to put in state_keys: Attributes that:

• Are mutated by user interaction (text inputs, selected files, etc.)
• Are expensive to reconstruct (e.g., discussion history)
• Should survive a reload even if the module's code is broken

For pure-renderer modules that don't touch app state, state_keys can be empty.

What to put in delegation_targets: Currently informational. The intent is that future versions of the reloader could atomically swap the function references held by the App, providing even faster recovery. For now, this list serves as documentation.

---

Triggering a Reload

Keyboard Shortcut

Ctrl+Alt+R (configurable via config.toml → [hot_reload].trigger_key).

The keyboard handler is in src/gui_2.py and calls HotReloader.reload_all(app).

GUI Button

The "Hot Reload" button in the debug panel (or the View menu, depending on theme) triggers the same call.

Programmatic

From any code with a reference to the App:

from src.hot_reloader import HotReloader
HotReloader.reload("src.gui_2", app)

Or to reload all registered modules:

HotReloader.reload_all(app)

---

What Can and Cannot Be Reloaded

Safe to Reload

• Pure renderer functions — def render_xxx(app: App): ... with no module-level mutable state
• Formatters — Markdown rendering, syntax highlighting, diff generation
• Pure utility functions — Path resolution, file classification, token estimation
• Theme constants — Color values, geometry settings (re-read on next render)
• Tool implementations — Read-only MCP tools

Unsafe to Reload

• App class itself — Has bound methods that hold references to the old class
• Stateful singletons — ai_client (module-level globals like _send_lock would be reset)
• MCP client — Holds state for active tool calls
• Hooks — Would disconnect the running GUI from the hook server

For these, a full application restart is required. The reloader does not enforce this — it allows any registered module to be reloaded. The developer is responsible for not registering unsafe modules.

---

Testing

Unit Tests

• tests/test_hot_reloader.py — HotReloader.register, capture_state, restore_state, reload, error paths
• tests/test_hot_reload_integration.py — End-to-end with a test App instance

Test Pattern

def test_hot_reload_preserves_state(tmp_path, monkeypatch):
    # Set up a test App
    app = MockApp()
    app.user_input = "Original text"
    app.selected_index = 5
    
    # Register a module that mutates a different attribute
    hm = HotModule(
        name="test_module",
        file_path=str(tmp_path / "test_module.py"),
        state_keys=["user_input", "selected_index"],
    )
    HotReloader.register(hm)
    
    # Write a module that mutates app.foo on import
    (tmp_path / "test_module.py").write_text("app.foo = 'new value'")
    
    # Reload
    success = HotReloader.reload("test_module", app)
    
    # Verify: app.user_input and app.selected_index are preserved
    assert app.user_input == "Original text"
    assert app.selected_index == 5
    assert app.foo == "new value"
    assert success

For the error path:

def test_hot_reload_rolls_back_on_error(tmp_path):
    app = MockApp()
    app.user_input = "Original"
    hm = HotModule(name="bad_module", state_keys=["user_input"])
    HotReloader.register(hm)
    
    # Module raises on import
    (tmp_path / "bad_module.py").write_text("raise RuntimeError('boom')")
    
    success = HotReloader.reload("bad_module", app)
    
    assert not success
    assert HotReloader.is_error_state
    assert "RuntimeError" in HotReloader.last_error
    assert app.user_input == "Original"  # Rolled back

Integration Tests

• tests/test_hot_reload_integration.py — Full GUI integration: register a renderer, mutate a file, trigger reload, verify the new renderer is called on the next frame.

---

Limitations

1. No Hot Function Swap: The delegation_targets field is currently informational. Function references held by the App are updated only via Python's late binding, which works for top-level function calls but not for callbacks registered elsewhere.

2. State Restoration Is Shallow: capture_state does copy.deepcopy() of the listed attributes, but if the App holds references to objects that themselves hold module-level state (e.g., a callback registered with ai_client.set_tool_preset), the reloader can't update those. The deep copy preserves the existing object reference, but the object's internal state may be inconsistent with the reloaded module.

3. Module-Level Singletons Are Reset: If a reloaded module had a singleton (e.g., _initialized = False) that was set during the first import, the reload re-executes the top-level code and may reset the singleton. Code that depends on the singleton's previous state will break.

4. Thread Safety: HotReloader.HOT_MODULES is a class-level dict without internal locking. Concurrent registration from multiple threads is unsafe. In practice, registration happens at import time (single-threaded) and reloads are user-initiated (single-threaded from the GUI's perspective).

5. No Dependency Tracking: Reloading src.gui_2 doesn't reload its imports. If src.gui_2 imports a helper from src.imgui_scopes and the helper has been edited, the helper's changes are not picked up until src.imgui_scopes is itself reloaded. reload_all() mitigates this but reloads every registered module, which is slower.

6. GUI Frame Dependency: The visual error feedback requires the GUI to redraw. If the reload happens during a frame (extremely rare in practice), the error display may be delayed by one frame.

---

Future Work

• Atomic Function Swap — Implement delegation_targets as a live list; the App's wrappers read from this list at call time. Would enable hot-swapping of individual functions without reloading the entire module.
• Dependency Graph — Track which modules depend on which, and reload in dependency order.
• Background Reload — Run importlib.reload on a background thread to avoid frame stutter for large modules.
• Incremental Hot Reload — Use AST parsing to reload only the changed function, leaving unchanged functions at their original bytecode.
• Reload Notifications — Push reload events to the comms log so other systems (MMA, RAG) can refresh their views.

See guide_architecture.md for the overall architectural pattern and guide_context_curation.md#context-snapshotting-per-take for the related HistoryManager undo/redo mechanism.