docs
This commit is contained in:
45
docs/Readme.md
Normal file
45
docs/Readme.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# Manual Slop
|
||||
|
||||
A GUI orchestrator for local LLM-driven coding sessions, built to prevent the AI from running wild and to provide total transparency into the context and execution state.
|
||||
|
||||
## Core Management Panels
|
||||
|
||||
### Projects Panel
|
||||
The heart of context management.
|
||||
- **Configuration:** You specify the Git Directory (for commit tracking) and a Main Context File (the markdown file containing your project's notes and schema).
|
||||
- **Word-Wrap Toggle:** Dynamically swaps text rendering in large read-only panels (Responses, Comms Log) between unwrapped (ideal for viewing precise code formatting) and wrapped (ideal for prose).
|
||||
- **Project Switching:** Switch between different <project>.toml profiles to instantly swap out your entire active file list, discussion history, and settings.
|
||||
|
||||
### Discussion History
|
||||
Manages your conversational branches, preventing context poisoning across different tasks.
|
||||
- **Discussions Sub-Menu:** Allows you to create separate timelines for different tasks (e.g., "Refactoring Auth" vs. "Adding API Endpoints").
|
||||
- **Git Commit Tracking:** Clicking "Update Commit" reads HEAD from your project's git directory and stamps the discussion.
|
||||
- **Entry Management:** Each turn has a Role (User, AI, System). You can toggle entries between **Read** and **Edit** modes, collapse them, or hit [+ Max] to open them in the Global Text Viewer.
|
||||
- **Auto-Add:** If toggled, anything sent from the "Message" panel and returned to the "Response" panel is automatically appended to the current discussion history.
|
||||
|
||||
### Files & Screenshots
|
||||
Controls what is explicitly fed into the context compiler.
|
||||
- **Base Dir:** Defines the root for path resolution and tool constraints.
|
||||
- **Paths:** Explicit files or wildcard globs (e.g., src/**/*.rs).
|
||||
- When generating a request, these files are summarized symbolically (summarize.py) to conserve tokens, unless the AI explicitly decides to read their full contents via its internal tools.
|
||||
|
||||
## Interaction Panels
|
||||
|
||||
### Provider
|
||||
Switch between API backends (Gemini, Anthropic) on the fly. Clicking "Fetch Models" queries the active provider for the latest model list.
|
||||
|
||||
### Message & Response
|
||||
- **Message:** Your input field.
|
||||
- **Gen + Send:** Compiles the markdown context and dispatches the background thread to the AI.
|
||||
- **MD Only:** Dry-runs the compiler so you can inspect the generated <project>_00N.md without triggering an API charge.
|
||||
- **Response:** The read-only output. Flashes green when a new response arrives.
|
||||
|
||||
### Global Text Viewer & Script Outputs
|
||||
- **Last Script Output:** Whenever the AI executes a background script, this window pops up, flashing blue. It contains both the executed script and the stdout/stderr.
|
||||
- **Text Viewer:** A large, resizable global popup invoked anytime you click a [+] or [+ Maximize] button in the UI. Used for deep-reading long logs, discussion entries, or script bodies.
|
||||
|
||||
## System Prompts
|
||||
Provides two text inputs for overriding default instructions:
|
||||
1. **Global:** Applied across every project you load.
|
||||
2. **Project:** Specific to the active workspace.
|
||||
These are concatenated onto the strict tool-usage guidelines the agent is initialized with.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
85
docs/guide_architecture.md
Normal file
85
docs/guide_architecture.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# Guide: Architecture
|
||||
|
||||
Overview of the package design, state management, and code-path layout.
|
||||
|
||||
---
|
||||
|
||||
The purpose of this software is to alleviate the pain points of using AI as a local co-pilot by encapsulating the workflow into a resilient, strictly controlled state machine. It manages context generation, API throttling, human-in-the-loop tool execution, and session-long logging.
|
||||
|
||||
There are two primary state boundaries used:
|
||||
|
||||
* The GUI State (Main Thread, Retained-Mode via Dear PyGui)
|
||||
* The AI State (Daemon Thread, stateless execution loop)
|
||||
|
||||
All synchronization between these boundaries is managed via lock-protected queues and events.
|
||||
|
||||
## Code Paths
|
||||
|
||||
### Lifetime & Application Boot
|
||||
|
||||
The application lifetime is localized within App.run in gui.py.
|
||||
|
||||
1. __init__ parses the global config.toml (which sets the active provider, theme, and project paths).
|
||||
2. It immediately hands off to project_manager.py to deserialize the active <project>.toml which hydrates the session's files, discussion histories, and prompts.
|
||||
3. Dear PyGui's dpg contexts are bootstrapped with docking_viewport=True, allowing individual GUI panels to exist as native OS windows.
|
||||
4. The main thread enters a blocking while dpg.is_dearpygui_running() render loop.
|
||||
5. On shutdown (clean exit), it performs a dual-flush: _flush_to_project() commits the UI state back to the <project>.toml, and _flush_to_config() commits the global state to config.toml. The viewport layout is automatically serialized to dpg_layout.ini.
|
||||
|
||||
### Context Shaping & Aggregation
|
||||
|
||||
Before making a call to an AI Provider, the current state of the workspace is resolved into a dense Markdown representation.
|
||||
This occurs inside ggregate.run.
|
||||
|
||||
If using the default workflow, ggregate.py hashes through the following process:
|
||||
1. **Glob Resolution:** Iterates through config["files"]["paths"] and unpacks any wildcards (e.g., src/**/*.rs) against the designated ase_dir.
|
||||
2. **Summarization Pass:** Instead of concatenating raw file bodies (which would quickly overwhelm the ~200k token limit over multiple rounds), the files are passed to summarize.py.
|
||||
3. **AST Parsing:** summarize.py runs a heuristic pass. For Python files, it uses the standard st module to read structural nodes (Classes, Methods, Imports, Constants). It outputs a compact Markdown table.
|
||||
4. **Markdown Generation:** The final <project>_00N.md string is constructed, comprising the truncated AST summaries, the user's current project system prompt, and the active discussion branch.
|
||||
5. The Markdown file is persisted to disk (./md_gen/ by default) for auditing.
|
||||
|
||||
### AI Communication & The Tool Loop
|
||||
|
||||
The communication model is unified under i_client.py, which normalizes the Gemini and Anthropic SDKs into a singular interface send(md_content, user_message, base_dir, file_items).
|
||||
|
||||
The loop is defined as follows:
|
||||
|
||||
1. **Prompt Injection:** The aggregated Markdown context and system prompt are injected. (Gemini injects this directly into system_instruction at chat instantiation to prevent history bloat; Anthropic chunks this into cache_control: ephemeral blocks).
|
||||
2. **Execution Loop:** A MAX_TOOL_ROUNDS (default 10) bounded loop begins.
|
||||
3. The AI provider is polled.
|
||||
4. If the provider's stop_reason is ool_use:
|
||||
1. The loop parses the requested tool (either a read-only MCP tool or the destructive PowerShell tool).
|
||||
2. If PowerShell, it dispatches a blocking event to the Main Thread (see *On Tool Execution & Concurrency*).
|
||||
3. Once the result is retrieved, the loop executes a **Dynamic Refresh** (_reread_file_items). Any files currently tracked by the project are pulled from the disk fresh.
|
||||
4. The tool result, appended with the fresh [FILES UPDATED] block, is sent back to the provider.
|
||||
5. Once the model outputs standard text, the loop terminates and yields the string back to the GUI callback.
|
||||
|
||||
### On Tool Execution & Concurrency
|
||||
|
||||
When the AI calls a safe MCP tool (like
|
||||
ead_file or search_files), the daemon thread immediately executes it via mcp_client.py and returns the result.
|
||||
|
||||
However, when the AI requests
|
||||
un_powershell, the operation halts:
|
||||
|
||||
1. The Daemon Thread instantiates a ConfirmDialog object containing the payload and calls .wait(). This blocks the thread on a hreading.Event().
|
||||
2. The ConfirmDialog instance is safely placed in a _pending_dialog_lock.
|
||||
3. The Main Thread, during its next frame cycle, pops the dialog from the lock and renders an OS-level modal window using dpg.window(modal=True).
|
||||
4. The user can inspect the script, modify it in the text box, or reject it entirely.
|
||||
5. Upon the user clicking "Approve & Run", the main thread triggers the hreading.Event, unblocking the Daemon Thread.
|
||||
6. The Daemon Thread passes the script to shell_runner.py, captures stdout, stderr, and exit_code, logs it to session_logger.py, and returns it to the LLM.
|
||||
|
||||
### On Context History Pruning (Anthropic)
|
||||
|
||||
Because the Anthropic API requires sending the entire conversation history on every request, long sessions will inevitably hit the invalid_request_error: prompt is too long.
|
||||
|
||||
To solve this, i_client.py implements an aggressive pruning algorithm:
|
||||
1. _strip_stale_file_refreshes: It recursively sweeps backward through the history dict and strips out large [FILES UPDATED] data blocks from old turns, preserving only the most recent snapshot.
|
||||
2. _trim_anthropic_history: If the estimated token count still exceeds _ANTHROPIC_MAX_PROMPT_TOKENS (~180,000), it slices off the oldest user/assistant message pairs from the beginning of the history array.
|
||||
3. The loop guarantees that at least the System prompt, Tool Definitions, and the final user prompt are preserved.
|
||||
|
||||
### Session Persistence
|
||||
|
||||
All I/O bound session data is recorded sequentially. session_logger.py hooks into the execution loops and records:
|
||||
- logs/comms_<ts>.log: A JSON-L structured timeline of every raw payload sent/received.
|
||||
- logs/toolcalls_<ts>.log: A sequential markdown record detailing every AI tool invocation and its exact stdout result.
|
||||
- scripts/generated/: Every .ps1 script approved and executed by the shell runner is physically written to disk for version control transparency.
|
||||
48
docs/guide_tools.md
Normal file
48
docs/guide_tools.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Guide: Tooling
|
||||
|
||||
Overview of the tool dispatch and execution model.
|
||||
|
||||
---
|
||||
|
||||
The agent is provided two classes of tools: Read-Only MCP Tools, and a Destructive Execution Loop.
|
||||
|
||||
## 1. Read-Only Context (MCP Tools)
|
||||
|
||||
Implemented in mcp_client.py. These tools allow the AI to selectively expand its knowledge of the codebase without requiring the user to dump entire 10,000-line files into the static context prefix.
|
||||
|
||||
### Security & Scope
|
||||
Every filesystem MCP tool passes its arguments through _resolve_and_check. This function ensures that the requested path falls under one of the allowed directories defined in the GUI's Base Dir configurations.
|
||||
If the AI attempts to read or search a path outside the project bounds, the tool safely catches the constraint violation and returns ACCESS DENIED.
|
||||
|
||||
### Supplied Tools:
|
||||
*
|
||||
ead_file(path): Returns the raw UTF-8 text of a file.
|
||||
* list_directory(path): Returns a formatted table of a directory's contents, showing file vs dir and byte sizes.
|
||||
* search_files(path, pattern): Executes an absolute glob search (e.g., **/*.py) to find specific files.
|
||||
* get_file_summary(path): Invokes the local summarize.py heuristic parser to get the AST structure of a file without reading the whole body.
|
||||
* web_search(query): Queries DuckDuckGo's raw HTML endpoint and returns the top 5 results (Titles, URLs, Snippets) using a native HTMLParser to avoid heavy dependencies.
|
||||
* etch_url(url): Downloads a target webpage and strips out all scripts, styling, and structural HTML, returning only the raw prose content (clamped to 40,000 characters).
|
||||
|
||||
## 2. Destructive Execution (
|
||||
un_powershell)
|
||||
|
||||
The core manipulation mechanism. This is a single, heavily guarded tool.
|
||||
|
||||
### Flow
|
||||
1. The AI generates a
|
||||
un_powershell payload containing a PowerShell script.
|
||||
2. The AI background thread calls confirm_and_run_callback (injected by gui.py).
|
||||
3. The background thread blocks completely, creating a modal popup on the main GUI thread.
|
||||
4. The user reads the script and chooses to Approve or Reject.
|
||||
5. If Approved, shell_runner.py executes the script using -NoProfile -NonInteractive -Command within the specified ase_dir.
|
||||
6. The combined stdout, stderr, and EXIT CODE are captured and returned to the AI in the tool result block.
|
||||
|
||||
### AI Guidelines
|
||||
The core system prompt explicitly guides the AI on how to use this tool safely:
|
||||
* Prefer targeted replacements (using PowerShell's .Replace()) over full rewrites where possible.
|
||||
* If a file is large and complex (requiring specific escape characters), do not attempt an inline python -c script. Instead, use a PowerShell here-string (@'...'@) to write a temporary python helper script to disk, execute the python script, and then delete it.
|
||||
|
||||
### Synthetic Context Refresh
|
||||
Immediately after **any** tool call turn finishes, i_client runs _reread_file_items. It fetches the latest disk state of all files in the current project context and appends them as a synthetic [FILES UPDATED] message to the tool result.
|
||||
This means if the AI writes to a file, it instantly "sees" the modification in its next turn without having to waste a cycle calling
|
||||
ead_file.
|
||||
Reference in New Issue
Block a user