chore(conductor): Mark track 'Implement context visualization and memory management improvements' as complete

This reverts commit 1581380a43.

.gitignore

@@ -6,3 +6,4 @@ md_gen
 scripts/generated
 logs
 dpg_layout.ini
+.env

GEMINI.md

@@ -0,0 +1,47 @@
+# Project Overview
+
+**Manual Slop** is a local GUI application designed as an experimental, "manual" AI coding assistant. It allows users to curate and send context (files, screenshots, and discussion history) to AI APIs (Gemini and Anthropic). The AI can then execute PowerShell scripts within the project directory to modify files, requiring explicit user confirmation before execution.
+
+**Main Technologies:**
+*   **Language:** Python 3.11+
+*   **Package Management:** `uv`
+*   **GUI Framework:** Dear PyGui (`dearpygui`), ImGui Bundle (`imgui-bundle`)
+*   **AI SDKs:** `google-genai` (Gemini), `anthropic`
+*   **Configuration:** TOML (`tomli-w`)
+
+**Architecture:**
+*   **`gui.py`:** The main entry point and Dear PyGui application logic. Handles all panels, layouts, user input, and confirmation dialogs.
+*   **`ai_client.py`:** A unified wrapper for both Gemini and Anthropic APIs. Manages sessions, tool/function-call loops, token estimation, and context history management.
+*   **`aggregate.py`:** Responsible for building the `file_items` context. It reads project configurations, collects files and screenshots, and builds the context into markdown format to send to the AI.
+*   **`mcp_client.py`:** Implements MCP-like tools (e.g., `read_file`, `list_directory`, `search_files`, `web_search`) as native functions that the AI can call. Enforces a strict allowlist for file access.
+*   **`shell_runner.py`:** A sandboxed subprocess wrapper that executes PowerShell scripts (`powershell -NoProfile -NonInteractive -Command`) provided by the AI.
+*   **`project_manager.py`:** Manages per-project TOML configurations (`manual_slop.toml`), serializes discussion entries, and integrates with git (e.g., fetching current commit).
+*   **`session_logger.py`:** Handles timestamped logging of communication history (JSON-L) and tool calls (saving generated `.ps1` files).
+
+# Building and Running
+
+*   **Setup:** The application uses `uv` for dependency management. Ensure `uv` is installed.
+*   **Credentials:** You must create a `credentials.toml` file in the root directory to store your API keys:
+    ```toml
+    [gemini]
+    api_key = "****"
+    [anthropic]
+    api_key = "****"
+    ```
+*   **Run the Application:**
+    ```powershell
+    uv run .\gui.py
+    ```
+
+# Development Conventions
+
+*   **Configuration Management:** The application uses two tiers of configuration:
+    *   `config.toml`: Global settings (UI theme, active provider, list of project paths).
+    *   `manual_slop.toml`: Per-project settings (files to track, discussion history, specific system prompts).
+*   **Tool Execution:** The AI acts primarily by generating PowerShell scripts. These scripts MUST be confirmed by the user via a GUI modal before execution. The AI also has access to read-only MCP-style file exploration tools and web search capabilities.
+*   **Context Refresh:** After every tool call that modifies the file system, the application automatically refreshes the file contents in the context using the files' `mtime` to optimize reads.
+*   **UI State Persistence:** Window layouts and docking arrangements are automatically saved to and loaded from `dpg_layout.ini`.
+*   **Code Style:**
+    *   Use type hints where appropriate.
+    *   Internal methods and variables are generally prefixed with an underscore (e.g., `_flush_to_project`, `_do_generate`).
+*   **Logging:** All API communications are logged to `logs/comms_<ts>.log`. All executed scripts are saved to `scripts/generated/`.

MainContext.md

@@ -12,16 +12,16 @@ Is a local GUI tool for manually curating and sending context to AI APIs. It agg
 - `uv` - package/env management
 
 **Files:**
-- `gui.py` - main GUI, `App` class, all panels, all callbacks, confirmation dialog, layout persistence, rich comms rendering
+- `gui.py` - main GUI, `App` class, all panels, all callbacks, confirmation dialog, layout persistence, rich comms rendering; `[+ Maximize]` buttons in `ConfirmDialog` and `win_script_output` now pass text directly as `user_data` / read from `self._last_script` / `self._last_output` instance vars instead of `dpg.get_value(tag)` — fixes glitch when word-wrap is ON or dialog is dismissed before viewer opens
-- `ai_client.py` - unified provider wrapper, model listing, session management, send, tool/function-call loop, comms log, provider error classification
+- `ai_client.py` - unified provider wrapper, model listing, session management, send, tool/function-call loop, comms log, provider error classification, token estimation, and aggressive history truncation
-- `aggregate.py` - reads config, collects files/screenshots/discussion, writes numbered `.md` files to `output_dir`
+- `aggregate.py` - reads config, collects files/screenshots/discussion, builds `file_items` with `mtime` for cache optimization, writes numbered `.md` files to `output_dir` using `build_markdown_from_items` to avoid double I/O; `run()` returns `(markdown_str, path, file_items)` tuple; `summary_only=False` by default (full file contents sent, not heuristic summaries)
 - `shell_runner.py` - subprocess wrapper that runs PowerShell scripts sandboxed to `base_dir`, returns stdout/stderr/exit code as a string
 - `session_logger.py` - opens timestamped log files at session start; writes comms entries as JSON-L and tool calls as markdown; saves each AI-generated script as a `.ps1` file
 - `project_manager.py` - per-project .toml load/save, entry serialisation (entry_to_str/str_to_entry with @timestamp support), default_project/default_discussion factories, migrate_from_legacy_config, flat_config for aggregate.run(), git helpers (get_git_commit, get_git_log)
 - `theme.py` - palette definitions, font loading, scale, load_from_config/save_to_config
 - `gemini.py` - legacy standalone Gemini wrapper (not used by the main GUI; superseded by `ai_client.py`)
 - `file_cache.py` - stub; Anthropic Files API path removed; kept so stale imports don't break
-- `mcp_client.py` - MCP-style read-only file tools (read_file, list_directory, search_files, get_file_summary); allowlist enforced against project file_items + base_dirs; dispatched by ai_client tool-use loop for both Anthropic and Gemini
+- `mcp_client.py` - MCP-style tools (read_file, list_directory, search_files, get_file_summary, web_search, fetch_url); allowlist enforced against project file_items + base_dirs for file tools; web tools are unrestricted; dispatched by ai_client tool-use loop for both Anthropic and Gemini
 - `summarize.py` - local heuristic summariser (no AI); .py via AST, .toml via regex, .md headings, generic preview; used by mcp_client.get_file_summary and aggregate.build_summary_section
 - `config.toml` - global-only settings: [ai] provider+model+system_prompt, [theme] palette+font+scale, [projects] paths array + active path
 - `manual_slop.toml` - per-project file: [project] name+git_dir+system_prompt+main_context, [output] namespace+output_dir, [files] base_dir+paths, [screenshots] base_dir+paths, [discussion] roles+active+[discussion.discussions.<name>] git_commit+last_updated+history
@@ -87,7 +87,7 @@ Is a local GUI tool for manually curating and sending context to AI APIs. It agg
 - All tool calls (script + result/rejection) are appended to `_tool_log` and displayed in the Tool Calls panel
 
 **Dynamic file context refresh (ai_client.py):**
-- After the last tool call in each round, all project files from `file_items` are re-read from disk via `_reread_file_items()`. The `file_items` variable is reassigned so subsequent rounds see fresh content.
+- After the last tool call in each round, project files from `file_items` are checked via `_reread_file_items()`. It uses `mtime` to only re-read modified files, returning only the `changed` files to build a minimal `[FILES UPDATED]` block.
 - For Anthropic: the refreshed file contents are injected as a `text` block appended to the `tool_results` user message, prefixed with `[FILES UPDATED]` and an instruction not to re-read them.
 - For Gemini: refreshed file contents are appended to the last function response's `output` string as a `[SYSTEM: FILES UPDATED]` block. On the next tool round, stale `[FILES UPDATED]` blocks are stripped from history and old tool outputs are truncated to `_history_trunc_limit` characters to control token growth.
 - `_build_file_context_text(file_items)` formats the refreshed files as markdown code blocks (same format as the original context)
@@ -141,10 +141,12 @@ Entry layout: index + timestamp + direction + kind + provider/model header row,
 - `log_tool_call(script, result, script_path)` writes the script to `scripts/generated/<ts>_<seq:04d>.ps1` and appends a markdown record to the toolcalls log without the script body (just the file path + result); uses a `threading.Lock` for the sequence counter
 - `close_session()` flushes and closes both file handles; called just before `dpg.destroy_context()`
 
-**Anthropic prompt caching:**
+**Anthropic prompt caching & history management:**
 - System prompt + context are combined into one string, chunked into <=120k char blocks, and sent as the `system=` parameter array. Only the LAST chunk gets `cache_control: ephemeral`, so the entire system prefix is cached as one unit.
 - Last tool in `_ANTHROPIC_TOOLS` (`run_powershell`) has `cache_control: ephemeral`; this means the tools prefix is cached together with the system prefix after the first request.
 - The user message is sent as a plain `[{"type": "text", "text": user_message}]` block with NO cache_control. The context lives in `system=`, not in the first user message.
+- `_add_history_cache_breakpoint` places `cache_control:ephemeral` on the last content block of the second-to-last user message, using the 4th cache breakpoint to cache the conversation history prefix.
+- `_trim_anthropic_history` uses token estimation (`_CHARS_PER_TOKEN = 3.5`) to keep the prompt under `_ANTHROPIC_MAX_PROMPT_TOKENS = 180_000`. It strips stale file refreshes from old turns, and drops oldest turn pairs if still over budget.
 - The tools list is built once per session via `_get_anthropic_tools()` and reused across all API calls within the tool loop, avoiding redundant Python-side reconstruction.
 - `_strip_cache_controls()` removes stale `cache_control` markers from all history entries before each API call, ensuring only the stable system/tools prefix consumes cache breakpoint slots.
 - Cache stats (creation tokens, read tokens) are surfaced in the comms log usage dict and displayed in the Comms History panel
@@ -180,13 +182,15 @@ Entry layout: index + timestamp + direction + kind + provider/model header row,
 **MCP file tools (mcp_client.py + ai_client.py):**
 - Four read-only tools exposed to the AI as native function/tool declarations: `read_file`, `list_directory`, `search_files`, `get_file_summary`
 - Access control: `mcp_client.configure(file_items, extra_base_dirs)` is called before each send; builds an allowlist of resolved absolute paths from the project's `file_items` plus the `base_dir`; any path that is not explicitly in the list or not under one of the allowed directories returns `ACCESS DENIED`
-- `mcp_client.dispatch(tool_name, tool_input)` is the single dispatch entry point used by both Anthropic and Gemini tool-use loops
+- `mcp_client.dispatch(tool_name, tool_input)` is the single dispatch entry point used by both Anthropic and Gemini tool-use loops; `TOOL_NAMES` set now includes all six tool names
 - Anthropic: MCP tools appear before `run_powershell` in the tools list (no `cache_control` on them; only `run_powershell` carries `cache_control: ephemeral`)
 - Gemini: MCP tools are included in the `FunctionDeclaration` list alongside `run_powershell`
 - `get_file_summary` uses `summarize.summarise_file()` — same heuristic used for the initial `<context>` block, so the AI gets the same compact structural view it already knows
 - `list_directory` sorts dirs before files; shows name, type, and size
 - `search_files` uses `Path.glob()` with the caller-supplied pattern (supports `**/*.py` style)
 - `read_file` returns raw UTF-8 text; errors (not found, access denied, decode error) are returned as error strings rather than exceptions, so the AI sees them as tool results
+- `web_search(query)` queries DuckDuckGo HTML endpoint and returns the top 5 results (title, URL, snippet) as a formatted string; uses a custom `_DDGParser` (HTMLParser subclass)
+- `fetch_url(url)` fetches a URL, strips HTML tags/scripts via `_TextExtractor` (HTMLParser subclass), collapses whitespace, and truncates to 40k chars to prevent context blowup; handles DuckDuckGo redirect links automatically
 - `summarize.py` heuristics: `.py` → AST imports + ALL_CAPS constants + classes+methods + top-level functions; `.toml` → table headers + top-level keys; `.md` → h1–h3 headings with indentation; all others → line count + first 8 lines preview
 - Comms log: MCP tool calls log `OUT/tool_call` with `{"name": ..., "args": {...}}` and `IN/tool_result` with `{"name": ..., "output": ...}`; rendered in the Comms History panel via `_render_payload_tool_call` (shows each arg key/value) and `_render_payload_tool_result` (shows output)
 
@@ -199,7 +203,9 @@ Entry layout: index + timestamp + direction + kind + provider/model header row,
 
 ### Gemini Context Management
 - Gemini uses explicit caching via `client.caches.create()` to store the `system_instruction` + tools as an immutable cached prefix with a 1-hour TTL. The cache is created once per chat session.
+- Proactively rebuilds cache at 90% of `_GEMINI_CACHE_TTL = 3600` to avoid stale-reference errors.
 - When context changes (detected via `md_content` hash), the old cache is deleted, a new cache is created, and chat history is migrated to a fresh chat session pointing at the new cache.
+- Trims history by dropping oldest pairs if input tokens exceed `_GEMINI_MAX_INPUT_TOKENS = 900_000`.
 - If cache creation fails (e.g., content is under the minimum token threshold — 1024 for Flash, 4096 for Pro), the system falls back to inline `system_instruction` in the chat config. Implicit caching may still provide cost savings in this case.
 - The `<context>` block lives inside `system_instruction`, NOT in user messages, preventing history bloat across turns.
 - On cleanup/exit, active caches are deleted via `ai_client.cleanup()` to prevent orphaned billing.
@@ -244,3 +250,34 @@ Documentation has been completely rewritten matching the strict, structural form
 - `docs/guide_architecture.md`: Details the Python implementation algorithms, queue management for UI rendering, the specific AST heuristics used for context aggregation, and the distinct algorithms for trimming Anthropic history vs Gemini state caching.
 - `docs/Readme.md`: The core interface manual.
 - `docs/guide_tools.md`: Security architecture for `_is_allowed` paths and definitions of the read-only vs destructive tool pipeline.
+
+
+
+
+## Updates (2026-02-22 — ai_client.py & aggregate.py)
+
+### mcp_client.py — Web Tools Added
+- `web_search(query)` and `fetch_url(url)` added as two new MCP tools alongside the existing four file tools.
+- `TOOL_NAMES` set updated to include all six tool names for dispatch routing.
+- `MCP_TOOL_SPECS` list extended with full JSON schema definitions for both web tools.
+- Both tools are declared in `_build_anthropic_tools()` and `_gemini_tool_declaration()` so they are available to both providers.
+- Web tools bypass the `_is_allowed` path check (no filesystem access); file tools retain the allowlist enforcement.
+
+### aggregate.py — run() double-I/O elimination
+- `run()` now calls `build_file_items()` once, then passes the result to `build_markdown_from_items()` instead of calling `build_files_section()` separately. This avoids reading every file twice per send.
+- `build_markdown_from_items()` accepts a `summary_only` flag (default `False`); when `False` it inlines full file content; when `True` it delegates to `summarize.build_summary_markdown()` for compact structural summaries.
+- `run()` returns a 3-tuple `(markdown_str, output_path, file_items)` — the `file_items` list is passed through to `gui.py` as `self.last_file_items` for dynamic context refresh after tool calls.
+
+
+## Updates (2026-02-22 — gui.py [+ Maximize] bug fix)
+
+### Problem
+Three `[+ Maximize]` buttons were reading their text content via `dpg.get_value(tag)` at click time:
+1. `ConfirmDialog.show()` — passed `f"{self._tag}_script"` as `user_data` and called `dpg.get_value(u)` in the lambda. If the dialog was dismissed before the viewer opened, the item no longer existed and the call would fail silently or crash.
+2. `win_script_output` Script `[+ Maximize]` — used `user_data="last_script_text"` and `dpg.get_value(u)`. When word-wrap is ON, `last_script_text` is hidden (`show=False`); in some DPG versions `dpg.get_value` on a hidden `input_text` returns `""`.
+3. `win_script_output` Output `[+ Maximize]` — same issue with `"last_script_output"`.
+
+### Fix
+- `ConfirmDialog.show()`: changed `user_data` to `self._script` (the actual text string captured at button-creation time) and the callback to `lambda s, a, u: _show_text_viewer("Confirm Script", u)`. The text is now baked in at dialog construction, not read from a potentially-deleted widget.
+- `App._append_tool_log()`: added `self._last_script = script` and `self._last_output = result` assignments so the latest values are always available as instance state.
+- `win_script_output` buttons: both `[+ Maximize]` buttons now use `lambda s, a, u: _show_text_viewer("...", self._last_script/output)` directly, bypassing DPG widget state entirely.

aggregate.py

@@ -98,24 +98,28 @@ def build_file_items(base_dir: Path, files: list[str]) -> list[dict]:
         entry    : str   (original config entry string)
         content  : str   (file text, or error string)
         error    : bool
+        mtime    : float (last modification time, for skip-if-unchanged optimization)
     """
     items = []
     for entry in files:
         paths = resolve_paths(base_dir, entry)
         if not paths:
-            items.append({"path": None, "entry": entry, "content": f"ERROR: no files matched: {entry}", "error": True})
+            items.append({"path": None, "entry": entry, "content": f"ERROR: no files matched: {entry}", "error": True, "mtime": 0.0})
             continue
         for path in paths:
             try:
                 content = path.read_text(encoding="utf-8")
+                mtime = path.stat().st_mtime
                 error = False
             except FileNotFoundError:
                 content = f"ERROR: file not found: {path}"
+                mtime = 0.0
                 error = True
             except Exception as e:
                 content = f"ERROR: {e}"
+                mtime = 0.0
                 error = True
-            items.append({"path": path, "entry": entry, "content": content, "error": error})
+            items.append({"path": path, "entry": entry, "content": content, "error": error, "mtime": mtime})
     return items
 
 def build_summary_section(base_dir: Path, files: list[str]) -> str:
@@ -126,6 +130,40 @@ def build_summary_section(base_dir: Path, files: list[str]) -> str:
     items = build_file_items(base_dir, files)
     return summarize.build_summary_markdown(items)
 
+def _build_files_section_from_items(file_items: list[dict]) -> str:
+    """Build the files markdown section from pre-read file items (avoids double I/O)."""
+    sections = []
+    for item in file_items:
+        path = item.get("path")
+        entry = item.get("entry", "unknown")
+        content = item.get("content", "")
+        if path is None:
+            sections.append(f"### `{entry}`\n\n```text\n{content}\n```")
+            continue
+        suffix = path.suffix.lstrip(".") if hasattr(path, "suffix") else "text"
+        lang = suffix if suffix else "text"
+        original = entry if "*" not in entry else str(path)
+        sections.append(f"### `{original}`\n\n```{lang}\n{content}\n```")
+    return "\n\n---\n\n".join(sections)
+
+
+def build_markdown_from_items(file_items: list[dict], screenshot_base_dir: Path, screenshots: list[str], history: list[str], summary_only: bool = False) -> str:
+    """Build markdown from pre-read file items instead of re-reading from disk."""
+    parts = []
+    # STATIC PREFIX: Files and Screenshots must go first to maximize Cache Hits
+    if file_items:
+        if summary_only:
+            parts.append("## Files (Summary)\n\n" + summarize.build_summary_markdown(file_items))
+        else:
+            parts.append("## Files\n\n" + _build_files_section_from_items(file_items))
+    if screenshots:
+        parts.append("## Screenshots\n\n" + build_screenshots_section(screenshot_base_dir, screenshots))
+    # DYNAMIC SUFFIX: History changes every turn, must go last
+    if history:
+        parts.append("## Discussion History\n\n" + build_discussion_section(history))
+    return "\n\n---\n\n".join(parts)
+
+
 def build_markdown(base_dir: Path, files: list[str], screenshot_base_dir: Path, screenshots: list[str], history: list[str], summary_only: bool = False) -> str:
     parts = []
     # STATIC PREFIX: Files and Screenshots must go first to maximize Cache Hits
@@ -141,7 +179,7 @@ def build_markdown(base_dir: Path, files: list[str], screenshot_base_dir: Path,
         parts.append("## Discussion History\n\n" + build_discussion_section(history))
     return "\n\n---\n\n".join(parts)
 
-def run(config: dict) -> tuple[str, Path]:
+def run(config: dict) -> tuple[str, Path, list[dict]]:
     namespace = config.get("project", {}).get("name")
     if not namespace:
         namespace = config.get("output", {}).get("namespace", "project")
@@ -155,11 +193,11 @@ def run(config: dict) -> tuple[str, Path]:
     output_dir.mkdir(parents=True, exist_ok=True)
     increment = find_next_increment(output_dir, namespace)
     output_file = output_dir / f"{namespace}_{increment:03d}.md"
-    # Provide full files to trigger Gemini's 32k cache threshold and give the AI immediate context
+    # Build file items once, then construct markdown from them (avoids double I/O)
-    markdown = build_markdown(base_dir, files, screenshot_base_dir, screenshots, history,
-                              summary_only=False)
-    output_file.write_text(markdown, encoding="utf-8")
     file_items = build_file_items(base_dir, files)
+    markdown = build_markdown_from_items(file_items, screenshot_base_dir, screenshots, history,
+                                         summary_only=False)
+    output_file.write_text(markdown, encoding="utf-8")
     return markdown, output_file, file_items
 
 def main():

ai_client.py

@@ -13,6 +13,7 @@ during chat creation to avoid massive history bloat.
 # ai_client.py
 import tomllib
 import json
+import time
 import datetime
 from pathlib import Path
 import file_cache
@@ -34,6 +35,12 @@ def set_model_params(temp: float, max_tok: int, trunc_limit: int = 8000):
 _gemini_client = None
 _gemini_chat = None
 _gemini_cache = None
+_gemini_cache_md_hash: int | None = None
+_gemini_cache_created_at: float | None = None
+
+# Gemini cache TTL in seconds. Caches are created with this TTL and
+# proactively rebuilt at 90% of this value to avoid stale-reference errors.
+_GEMINI_CACHE_TTL = 3600
 
 _anthropic_client = None
 _anthropic_history: list[dict] = []
@@ -216,6 +223,7 @@ def cleanup():
 
 def reset_session():
     global _gemini_client, _gemini_chat, _gemini_cache
+    global _gemini_cache_md_hash, _gemini_cache_created_at
     global _anthropic_client, _anthropic_history
     global _CACHED_ANTHROPIC_TOOLS
     if _gemini_client and _gemini_cache:
@@ -226,6 +234,8 @@ def reset_session():
     _gemini_client = None
     _gemini_chat = None
     _gemini_cache = None
+    _gemini_cache_md_hash = None
+    _gemini_cache_created_at = None
     _anthropic_client = None
     _anthropic_history = []
     _CACHED_ANTHROPIC_TOOLS = None
@@ -276,37 +286,53 @@ def _list_anthropic_models() -> list[str]:
 
 TOOL_NAME = "run_powershell"
 
+_agent_tools: dict = {}
+
+def set_agent_tools(tools: dict):
+    global _agent_tools, _CACHED_ANTHROPIC_TOOLS
+    _agent_tools = tools
+    _CACHED_ANTHROPIC_TOOLS = None
+
 def _build_anthropic_tools() -> list[dict]:
     """Build the full Anthropic tools list: run_powershell + MCP file tools."""
     mcp_tools = []
     for spec in mcp_client.MCP_TOOL_SPECS:
-        mcp_tools.append({
+        if _agent_tools.get(spec["name"], True):
-            "name":        spec["name"],
+            mcp_tools.append({
-            "description": spec["description"],
+                "name":        spec["name"],
-            "input_schema": spec["parameters"],
+                "description": spec["description"],
-        })
+                "input_schema": spec["parameters"],
-    powershell_tool = {
+            })
-        "name": TOOL_NAME,
+    
-        "description": (
+    tools_list = mcp_tools
-            "Run a PowerShell script within the project base_dir. "
+    if _agent_tools.get(TOOL_NAME, True):
-            "Use this to create, edit, rename, or delete files and directories. "
+        powershell_tool = {
-            "The working directory is set to base_dir automatically. "
+            "name": TOOL_NAME,
-            "Always prefer targeted edits over full rewrites where possible. "
+            "description": (
-            "stdout and stderr are returned to you as the result."
+                "Run a PowerShell script within the project base_dir. "
-        ),
+                "Use this to create, edit, rename, or delete files and directories. "
-        "input_schema": {
+                "The working directory is set to base_dir automatically. "
-            "type": "object",
+                "Always prefer targeted edits over full rewrites where possible. "
-            "properties": {
+                "stdout and stderr are returned to you as the result."
-                "script": {
+            ),
-                    "type": "string",
+            "input_schema": {
-                    "description": "The PowerShell script to execute."
+                "type": "object",
-                }
+                "properties": {
+                    "script": {
+                        "type": "string",
+                        "description": "The PowerShell script to execute."
+                    }
+                },
+                "required": ["script"]
             },
-            "required": ["script"]
+            "cache_control": {"type": "ephemeral"},
-        },
+        }
-        "cache_control": {"type": "ephemeral"},
+        tools_list.append(powershell_tool)
-    }
+    elif tools_list:
-    return mcp_tools + [powershell_tool]
+        # Anthropic requires the LAST tool to have cache_control for the prefix caching to work optimally on tools
+        tools_list[-1]["cache_control"] = {"type": "ephemeral"}
+        
+    return tools_list
 
 
 _ANTHROPIC_TOOLS = _build_anthropic_tools()
@@ -328,6 +354,8 @@ def _gemini_tool_declaration():
 
     # MCP file tools
     for spec in mcp_client.MCP_TOOL_SPECS:
+        if not _agent_tools.get(spec["name"], True):
+            continue
         props = {}
         for pname, pdef in spec["parameters"].get("properties", {}).items():
             props[pname] = types.Schema(
@@ -345,27 +373,28 @@ def _gemini_tool_declaration():
         ))
 
     # PowerShell tool
-    declarations.append(types.FunctionDeclaration(
+    if _agent_tools.get(TOOL_NAME, True):
-        name=TOOL_NAME,
+        declarations.append(types.FunctionDeclaration(
-        description=(
+            name=TOOL_NAME,
-            "Run a PowerShell script within the project base_dir. "
+            description=(
-            "Use this to create, edit, rename, or delete files and directories. "
+                "Run a PowerShell script within the project base_dir. "
-            "The working directory is set to base_dir automatically. "
+                "Use this to create, edit, rename, or delete files and directories. "
-            "stdout and stderr are returned to you as the result."
+                "The working directory is set to base_dir automatically. "
-        ),
+                "stdout and stderr are returned to you as the result."
-        parameters=types.Schema(
+            ),
-            type=types.Type.OBJECT,
+            parameters=types.Schema(
-            properties={
+                type=types.Type.OBJECT,
-                "script": types.Schema(
+                properties={
-                    type=types.Type.STRING,
+                    "script": types.Schema(
-                    description="The PowerShell script to execute."
+                        type=types.Type.STRING,
-                )
+                        description="The PowerShell script to execute."
-            },
+                    )
-            required=["script"]
+                },
-        ),
+                required=["script"]
-    ))
+            ),
+        ))
 
-    return types.Tool(function_declarations=declarations)
+    return types.Tool(function_declarations=declarations) if declarations else None
 
 
 def _run_script(script: str, base_dir: str) -> str:
@@ -383,12 +412,15 @@ def _run_script(script: str, base_dir: str) -> str:
 
 # ------------------------------------------------------------------ dynamic file context refresh
 
-def _reread_file_items(file_items: list[dict]) -> list[dict]:
+def _reread_file_items(file_items: list[dict]) -> tuple[list[dict], list[dict]]:
     """
-    Re-read every file in file_items from disk, returning a fresh list.
+    Re-read file_items from disk, but only files whose mtime has changed.
-    This is called after tool calls so the AI sees updated file contents.
+    Returns (all_items, changed_items) — all_items is the full refreshed list,
+    changed_items contains only the files that were actually modified since
+    the last read (used to build a minimal [FILES UPDATED] block).
     """
     refreshed = []
+    changed = []
     for item in file_items:
         path = item.get("path")
         if path is None:
@@ -397,11 +429,20 @@ def _reread_file_items(file_items: list[dict]) -> list[dict]:
         from pathlib import Path as _P
         p = _P(path) if not isinstance(path, _P) else path
         try:
+            current_mtime = p.stat().st_mtime
+            prev_mtime = item.get("mtime", 0.0)
+            if current_mtime == prev_mtime:
+                refreshed.append(item)  # unchanged — skip re-read
+                continue
             content = p.read_text(encoding="utf-8")
-            refreshed.append({**item, "content": content, "error": False})
+            new_item = {**item, "content": content, "error": False, "mtime": current_mtime}
+            refreshed.append(new_item)
+            changed.append(new_item)
         except Exception as e:
-            refreshed.append({**item, "content": f"ERROR re-reading {p}: {e}", "error": True})
+            err_item = {**item, "content": f"ERROR re-reading {p}: {e}", "error": True, "mtime": 0.0}
-    return refreshed
+            refreshed.append(err_item)
+            changed.append(err_item)
+    return refreshed, changed
 
 
 def _build_file_context_text(file_items: list[dict]) -> str:
@@ -453,26 +494,52 @@ def _ensure_gemini_client():
         _gemini_client = genai.Client(api_key=creds["gemini"]["api_key"])
 
 
+
+def _get_gemini_history_list(chat):
+    if not chat: return []
+    # google-genai SDK stores the mutable list in _history
+    if hasattr(chat, "_history"):
+        return chat._history
+    if hasattr(chat, "history"):
+        return chat.history
+    if hasattr(chat, "get_history"):
+        return chat.get_history()
+    return []
+
 def _send_gemini(md_content: str, user_message: str, base_dir: str, file_items: list[dict] | None = None) -> str:
-    global _gemini_chat, _gemini_cache
+    global _gemini_chat, _gemini_cache, _gemini_cache_md_hash, _gemini_cache_created_at
     from google.genai import types
     try:
         _ensure_gemini_client(); mcp_client.configure(file_items or [], [base_dir])
         sys_instr = f"{_get_combined_system_prompt()}\n\n<context>\n{md_content}\n</context>"
         tools_decl = [_gemini_tool_declaration()]
-        
+
         # DYNAMIC CONTEXT: Check if files/context changed mid-session
         current_md_hash = hash(md_content)
         old_history = None
-        if _gemini_chat and getattr(_gemini_chat, "_last_md_hash", None) != current_md_hash:
+        if _gemini_chat and _gemini_cache_md_hash != current_md_hash:
-            old_history = list(_gemini_chat.history) if _gemini_chat.history else []
+            old_history = list(_get_gemini_history_list(_gemini_chat)) if _get_gemini_history_list(_gemini_chat) else []
             if _gemini_cache:
                 try: _gemini_client.caches.delete(name=_gemini_cache.name)
                 except: pass
             _gemini_chat = None
             _gemini_cache = None
+            _gemini_cache_created_at = None
             _append_comms("OUT", "request", {"message": "[CONTEXT CHANGED] Rebuilding cache and chat session..."})
 
+        # CACHE TTL: Proactively rebuild before the cache expires server-side.
+        # If we don't, send_message() will reference a deleted cache and fail.
+        if _gemini_chat and _gemini_cache and _gemini_cache_created_at:
+            elapsed = time.time() - _gemini_cache_created_at
+            if elapsed > _GEMINI_CACHE_TTL * 0.9:
+                old_history = list(_get_gemini_history_list(_gemini_chat)) if _get_gemini_history_list(_gemini_chat) else []
+                try: _gemini_client.caches.delete(name=_gemini_cache.name)
+                except: pass
+                _gemini_chat = None
+                _gemini_cache = None
+                _gemini_cache_created_at = None
+                _append_comms("OUT", "request", {"message": f"[CACHE TTL] Rebuilding cache (expired after {int(elapsed)}s)..."})
+
         if not _gemini_chat:
             chat_config = types.GenerateContentConfig(
                 system_instruction=sys_instr,
@@ -488,9 +555,10 @@ def _send_gemini(md_content: str, user_message: str, base_dir: str, file_items:
                     config=types.CreateCachedContentConfig(
                         system_instruction=sys_instr,
                         tools=tools_decl,
-                        ttl="3600s",
+                        ttl=f"{_GEMINI_CACHE_TTL}s",
                     )
                 )
+                _gemini_cache_created_at = time.time()
                 chat_config = types.GenerateContentConfig(
                     cached_content=_gemini_cache.name,
                     temperature=_temperature,
@@ -499,35 +567,38 @@ def _send_gemini(md_content: str, user_message: str, base_dir: str, file_items:
                 )
                 _append_comms("OUT", "request", {"message": f"[CACHE CREATED] {_gemini_cache.name}"})
             except Exception as e:
-                _gemini_cache = None  # Ensure clean state on failure
+                _gemini_cache = None
-                
+                _gemini_cache_created_at = None
+                _append_comms("OUT", "request", {"message": f"[CACHE FAILED] {type(e).__name__}: {e} — falling back to inline system_instruction"})
+
             kwargs = {"model": _model, "config": chat_config}
             if old_history:
                 kwargs["history"] = old_history
-            
+
             _gemini_chat = _gemini_client.chats.create(**kwargs)
-            _gemini_chat._last_md_hash = current_md_hash
+            _gemini_cache_md_hash = current_md_hash
         
         _append_comms("OUT", "request", {"message": f"[ctx {len(md_content)} + msg {len(user_message)}]"})
         payload, all_text = user_message, []
-        
-        for r_idx in range(MAX_TOOL_ROUNDS + 2):
-            # Strip stale file refreshes and truncate old tool outputs in Gemini history
-            if _gemini_chat and _gemini_chat.history:
-                for msg in _gemini_chat.history:
-                    if msg.role == "user" and hasattr(msg, "parts"):
-                        for p in msg.parts:
-                            if hasattr(p, "function_response") and p.function_response and hasattr(p.function_response, "response"):
-                                r = p.function_response.response
-                                if isinstance(r, dict) and "output" in r:
-                                    val = r["output"]
-                                    if isinstance(val, str):
-                                        if "[SYSTEM: FILES UPDATED]" in val:
-                                            val = val.split("[SYSTEM: FILES UPDATED]")[0].strip()
-                                        if _history_trunc_limit > 0 and len(val) > _history_trunc_limit:
-                                            val = val[:_history_trunc_limit] + "\n\n... [TRUNCATED BY SYSTEM TO SAVE TOKENS.]"
-                                        r["output"] = val
 
+        # Strip stale file refreshes and truncate old tool outputs ONCE before
+        # entering the tool loop (not per-round — history entries don't change).
+        if _gemini_chat and _get_gemini_history_list(_gemini_chat):
+            for msg in _get_gemini_history_list(_gemini_chat):
+                if msg.role == "user" and hasattr(msg, "parts"):
+                    for p in msg.parts:
+                        if hasattr(p, "function_response") and p.function_response and hasattr(p.function_response, "response"):
+                            r = p.function_response.response
+                            if isinstance(r, dict) and "output" in r:
+                                val = r["output"]
+                                if isinstance(val, str):
+                                    if "[SYSTEM: FILES UPDATED]" in val:
+                                        val = val.split("[SYSTEM: FILES UPDATED]")[0].strip()
+                                    if _history_trunc_limit > 0 and len(val) > _history_trunc_limit:
+                                        val = val[:_history_trunc_limit] + "\n\n... [TRUNCATED BY SYSTEM TO SAVE TOKENS.]"
+                                    r["output"] = val
+
+        for r_idx in range(MAX_TOOL_ROUNDS + 2):
             resp = _gemini_chat.send_message(payload)
             txt = "\n".join(p.text for c in resp.candidates if getattr(c, "content", None) for p in c.content.parts if hasattr(p, "text") and p.text)
             if txt: all_text.append(txt)
@@ -543,23 +614,25 @@ def _send_gemini(md_content: str, user_message: str, base_dir: str, file_items:
             
             # Guard: if Gemini reports input tokens approaching the limit, drop oldest history pairs
             total_in = usage.get("input_tokens", 0)
-            if total_in > _GEMINI_MAX_INPUT_TOKENS and _gemini_chat and _gemini_chat.history:
+            if total_in > _GEMINI_MAX_INPUT_TOKENS and _gemini_chat and _get_gemini_history_list(_gemini_chat):
-                hist = _gemini_chat.history
+                hist = _get_gemini_history_list(_gemini_chat)
                 dropped = 0
                 # Drop oldest pairs (user+model) but keep at least the last 2 entries
                 while len(hist) > 4 and total_in > _GEMINI_MAX_INPUT_TOKENS * 0.7:
-                    # Rough estimate: each dropped message saves ~(chars/4) tokens
+                    # Drop in pairs (user + model) to maintain alternating roles required by Gemini
                     saved = 0
-                    for p in hist[0].parts:
+                    for _ in range(2):
-                        if hasattr(p, "text") and p.text:
+                        if not hist: break
-                            saved += len(p.text) // 4
+                        for p in hist[0].parts:
-                        elif hasattr(p, "function_response") and p.function_response:
+                            if hasattr(p, "text") and p.text:
-                            r = getattr(p.function_response, "response", {})
+                                saved += len(p.text) // 4
-                            if isinstance(r, dict):
+                            elif hasattr(p, "function_response") and p.function_response:
-                                saved += len(str(r.get("output", ""))) // 4
+                                r = getattr(p.function_response, "response", {})
-                    hist.pop(0)
+                                if isinstance(r, dict):
-                    total_in -= max(saved, 100)
+                                    saved += len(str(r.get("output", ""))) // 4
-                    dropped += 1
+                        hist.pop(0)
+                        dropped += 1
+                    total_in -= max(saved, 200)
                 if dropped > 0:
                     _append_comms("OUT", "request", {"message": f"[GEMINI HISTORY TRIMMED: dropped {dropped} old entries to stay within token budget]"})
 
@@ -579,12 +652,12 @@ def _send_gemini(md_content: str, user_message: str, base_dir: str, file_items:
 
                 if i == len(calls) - 1:
                     if file_items:
-                        file_items = _reread_file_items(file_items)
+                        file_items, changed = _reread_file_items(file_items)
-                        ctx = _build_file_context_text(file_items)
+                        ctx = _build_file_context_text(changed)
                         if ctx:
                             out += f"\n\n[SYSTEM: FILES UPDATED]\n\n{ctx}"
                     if r_idx == MAX_TOOL_ROUNDS: out += "\n\n[SYSTEM: MAX ROUNDS. PROVIDE FINAL ANSWER.]"
-                
+
                 f_resps.append(types.Part.from_function_response(name=name, response={"output": out}))
                 log.append({"tool_use_id": name, "content": out})
             
@@ -616,7 +689,15 @@ _FILE_REFRESH_MARKER = "[FILES UPDATED"
 
 
 def _estimate_message_tokens(msg: dict) -> int:
-    """Rough token estimate for a single Anthropic message dict."""
+    """
+    Rough token estimate for a single Anthropic message dict.
+    Caches the result on the dict as '_est_tokens' so repeated calls
+    (e.g., from _trim_anthropic_history) don't re-scan unchanged messages.
+    Call _invalidate_token_estimate() when a message's content is modified.
+    """
+    cached = msg.get("_est_tokens")
+    if cached is not None:
+        return cached
     total_chars = 0
     content = msg.get("content", "")
     if isinstance(content, str):
@@ -634,7 +715,14 @@ def _estimate_message_tokens(msg: dict) -> int:
                     total_chars += len(_json.dumps(inp, ensure_ascii=False))
             elif isinstance(block, str):
                 total_chars += len(block)
-    return max(1, int(total_chars / _CHARS_PER_TOKEN))
+    est = max(1, int(total_chars / _CHARS_PER_TOKEN))
+    msg["_est_tokens"] = est
+    return est
+
+
+def _invalidate_token_estimate(msg: dict):
+    """Remove the cached token estimate so the next call recalculates."""
+    msg.pop("_est_tokens", None)
 
 
 def _estimate_prompt_tokens(system_blocks: list[dict], history: list[dict]) -> int:
@@ -646,7 +734,7 @@ def _estimate_prompt_tokens(system_blocks: list[dict], history: list[dict]) -> i
         total += max(1, int(len(text) / _CHARS_PER_TOKEN))
     # Tool definitions (rough fixed estimate — they're ~2k tokens for our set)
     total += 2500
-    # History messages
+    # History messages (uses cached estimates for unchanged messages)
     for msg in history:
         total += _estimate_message_tokens(msg)
     return total
@@ -681,6 +769,7 @@ def _strip_stale_file_refreshes(history: list[dict]):
             cleaned.append(block)
         if len(cleaned) < len(content):
             msg["content"] = cleaned
+            _invalidate_token_estimate(msg)
 
 
 def _trim_anthropic_history(system_blocks: list[dict], history: list[dict]):
@@ -772,6 +861,28 @@ def _strip_cache_controls(history: list[dict]):
                 if isinstance(block, dict):
                     block.pop("cache_control", None)
 
+def _add_history_cache_breakpoint(history: list[dict]):
+    """
+    Place cache_control:ephemeral on the last content block of the
+    second-to-last user message. This uses one of the 4 allowed Anthropic
+    cache breakpoints to cache the conversation prefix so the full history
+    isn't reprocessed on every request.
+    """
+    user_indices = [i for i, m in enumerate(history) if m.get("role") == "user"]
+    if len(user_indices) < 2:
+        return  # Only one user message (the current turn) — nothing stable to cache
+    target_idx = user_indices[-2]
+    content = history[target_idx].get("content")
+    if isinstance(content, list) and content:
+        last_block = content[-1]
+        if isinstance(last_block, dict):
+            last_block["cache_control"] = {"type": "ephemeral"}
+    elif isinstance(content, str):
+        history[target_idx]["content"] = [
+            {"type": "text", "text": content, "cache_control": {"type": "ephemeral"}}
+        ]
+
+
 def _repair_anthropic_history(history: list[dict]):
     """
     If history ends with an assistant message that contains tool_use blocks
@@ -809,23 +920,36 @@ def _send_anthropic(md_content: str, user_message: str, base_dir: str, file_item
         _ensure_anthropic_client()
         mcp_client.configure(file_items or [], [base_dir])
 
-        system_text = _get_combined_system_prompt() + f"\n\n<context>\n{md_content}\n</context>"
+        # Split system into two cache breakpoints:
-        system_blocks = _build_chunked_context_blocks(system_text)
+        # 1. Stable system prompt (never changes — always a cache hit)
+        # 2. Dynamic file context (invalidated only when files change)
+        stable_prompt = _get_combined_system_prompt()
+        stable_blocks = [{"type": "text", "text": stable_prompt, "cache_control": {"type": "ephemeral"}}]
+        context_text = f"\n\n<context>\n{md_content}\n</context>"
+        context_blocks = _build_chunked_context_blocks(context_text)
+        system_blocks = stable_blocks + context_blocks
 
         user_content = [{"type": "text", "text": user_message}]
 
         # COMPRESS HISTORY: Truncate massive tool outputs from previous turns
         for msg in _anthropic_history:
             if msg.get("role") == "user" and isinstance(msg.get("content"), list):
+                modified = False
                 for block in msg["content"]:
                     if isinstance(block, dict) and block.get("type") == "tool_result":
                         t_content = block.get("content", "")
                         if _history_trunc_limit > 0 and isinstance(t_content, str) and len(t_content) > _history_trunc_limit:
                             block["content"] = t_content[:_history_trunc_limit] + "\n\n... [TRUNCATED BY SYSTEM TO SAVE TOKENS. Original output was too large.]"
+                            modified = True
+                if modified:
+                    _invalidate_token_estimate(msg)
 
         _strip_cache_controls(_anthropic_history)
         _repair_anthropic_history(_anthropic_history)
         _anthropic_history.append({"role": "user", "content": user_content})
+        # Use the 4th cache breakpoint to cache the conversation history prefix.
+        # This is placed on the second-to-last user message (the last stable one).
+        _add_history_cache_breakpoint(_anthropic_history)
 
         n_chunks = len(system_blocks)
         _append_comms("OUT", "request", {
@@ -850,13 +974,16 @@ def _send_anthropic(md_content: str, user_message: str, base_dir: str, file_item
                     ),
                 })
 
+            def _strip_private_keys(history):
+                return [{k: v for k, v in m.items() if not k.startswith("_")} for m in history]
+
             response = _anthropic_client.messages.create(
                 model=_model,
                 max_tokens=_max_tokens,
                 temperature=_temperature,
                 system=system_blocks,
                 tools=_get_anthropic_tools(),
-                messages=_anthropic_history,
+                messages=_strip_private_keys(_anthropic_history),
             )
 
             # Convert SDK content block objects to plain dicts before storing in history
@@ -939,10 +1066,10 @@ def _send_anthropic(md_content: str, user_message: str, base_dir: str, file_item
                         "content":     output,
                     })
 
-            # Refresh file context after tool calls and inject into tool result message
+            # Refresh file context after tool calls — only inject CHANGED files
             if file_items:
-                file_items = _reread_file_items(file_items)
+                file_items, changed = _reread_file_items(file_items)
-                refreshed_ctx = _build_file_context_text(file_items)
+                refreshed_ctx = _build_file_context_text(changed)
                 if refreshed_ctx:
                     tool_results.append({
                         "type": "text",

conductor/code_styleguides/python.md

@@ -0,0 +1,37 @@
+# Google Python Style Guide Summary
+
+This document summarizes key rules and best practices from the Google Python Style Guide.
+
+## 1. Python Language Rules
+- **Linting:** Run `pylint` on your code to catch bugs and style issues.
+- **Imports:** Use `import x` for packages/modules. Use `from x import y` only when `y` is a submodule.
+- **Exceptions:** Use built-in exception classes. Do not use bare `except:` clauses.
+- **Global State:** Avoid mutable global state. Module-level constants are okay and should be `ALL_CAPS_WITH_UNDERSCORES`.
+- **Comprehensions:** Use for simple cases. Avoid for complex logic where a full loop is more readable.
+- **Default Argument Values:** Do not use mutable objects (like `[]` or `{}`) as default values.
+- **True/False Evaluations:** Use implicit false (e.g., `if not my_list:`). Use `if foo is None:` to check for `None`.
+- **Type Annotations:** Strongly encouraged for all public APIs.
+
+## 2. Python Style Rules
+- **Line Length:** Maximum 80 characters.
+- **Indentation:** 4 spaces per indentation level. Never use tabs.
+- **Blank Lines:** Two blank lines between top-level definitions (classes, functions). One blank line between method definitions.
+- **Whitespace:** Avoid extraneous whitespace. Surround binary operators with single spaces.
+- **Docstrings:** Use `"""triple double quotes"""`. Every public module, function, class, and method must have a docstring.
+  - **Format:** Start with a one-line summary. Include `Args:`, `Returns:`, and `Raises:` sections.
+- **Strings:** Use f-strings for formatting. Be consistent with single (`'`) or double (`"`) quotes.
+- **`TODO` Comments:** Use `TODO(username): Fix this.` format.
+- **Imports Formatting:** Imports should be on separate lines and grouped: standard library, third-party, and your own application's imports.
+
+## 3. Naming
+- **General:** `snake_case` for modules, functions, methods, and variables.
+- **Classes:** `PascalCase`.
+- **Constants:** `ALL_CAPS_WITH_UNDERSCORES`.
+- **Internal Use:** Use a single leading underscore (`_internal_variable`) for internal module/class members.
+
+## 4. Main
+- All executable files should have a `main()` function that contains the main logic, called from a `if __name__ == '__main__':` block.
+
+**BE CONSISTENT.** When editing code, match the existing style.
+
+*Source: [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html)*

conductor/index.md

@@ -0,0 +1,14 @@
+# Project Context
+
+## Definition
+- [Product Definition](./product.md)
+- [Product Guidelines](./product-guidelines.md)
+- [Tech Stack](./tech-stack.md)
+
+## Workflow
+- [Workflow](./workflow.md)
+- [Code Style Guides](./code_styleguides/)
+
+## Management
+- [Tracks Registry](./tracks.md)
+- [Tracks Directory](./tracks/)

conductor/product-guidelines.md

@@ -0,0 +1,15 @@
+# Product Guidelines: Manual Slop
+
+## Documentation Style
+- **Strict & In-Depth:** Documentation must follow an old-school, highly detailed technical breakdown style (similar to VEFontCache-Odin). Focus on architectural design, state management, algorithmic details, and structural formats rather than just surface-level usage.
+
+## UX & UI Principles
+- **USA Graphics Company Values:** Embrace high information density and tactile interactions. 
+- **Arcade Aesthetics:** Utilize arcade game-style visual feedback for state updates (e.g., blinking notifications for tool execution and AI responses) to make the experience fun, visceral, and engaging.
+- **Explicit Control & Expert Focus:** The interface should not hold the user's hand. It must prioritize explicit manual confirmation for destructive actions while providing dense, unadulterated access to logs and context.
+- **Multi-Viewport Capabilities:** Leverage dockable, floatable panels to allow users to build custom workspaces suitable for multi-monitor setups.
+
+## Code Standards & Architecture
+- **Strict State Management:** There must be a rigorous separation between the Main GUI rendering thread and daemon execution threads. The UI should *never* hang during AI communication or script execution. Use lock-protected queues and events for synchronization.
+- **Comprehensive Logging:** Aggressively log all actions, API payloads, tool calls, and executed scripts. Maintain timestamped JSON-L and markdown logs to ensure total transparency and debuggability.
+- **Dependency Minimalism:** Limit external dependencies where possible. For instance, prefer standard library modules (like `urllib` and `html.parser` for web tools) over heavy third-party packages.

conductor/product.md

@@ -0,0 +1,15 @@
+# Product Guide: Manual Slop
+
+## Vision
+To serve as an expert-level utility for personal developer use on small projects, providing full, manual control over vendor API metrics, agent capabilities, and context memory usage. 
+
+## Primary Use Cases
+- **Full Control over Vendor APIs:** Exposing detailed API metrics and configuring deep agent capabilities directly within the GUI.
+- **Context & Memory Management:** Better visualization and management of token usage and context memory, allowing developers to optimize prompt limits manually.
+- **Manual "Vibe Coding" Assistant:** Serving as an auxiliary, multi-provider assistant that natively interacts with the codebase via sandboxed PowerShell scripts and MCP-like file tools, emphasizing manual developer oversight and explicit confirmation.
+
+## Key Features
+- **Multi-Provider Integration:** Supports both Gemini and Anthropic with seamless switching.
+- **Explicit Execution Control:** All AI-generated PowerShell scripts require explicit human confirmation via interactive UI dialogs before execution.
+- **Detailed History Management:** Rich discussion history with branching, timestamping, and specific git commit linkage per conversation.
+- **In-Depth Toolset Access:** MCP-like file exploration, URL fetching, search, and dynamic context aggregation embedded within a multi-viewport Dear PyGui/ImGui interface.

conductor/setup_state.json

@@ -0,0 +1 @@
+{"last_successful_step": "3.3_initial_track_generated"}

conductor/tech-stack.md

@@ -0,0 +1,16 @@
+# Technology Stack: Manual Slop
+
+## Core Language
+- **Python 3.11+**
+
+## GUI Frameworks
+- **Dear PyGui:** For immediate/retained mode GUI rendering and node mapping.
+- **ImGui Bundle (`imgui-bundle`):** To provide advanced multi-viewport and dockable panel capabilities on top of Dear ImGui.
+
+## AI Integration SDKs
+- **google-genai:** For Google Gemini API interaction and explicit context caching.
+- **anthropic:** For Anthropic Claude API interaction, supporting ephemeral prompt caching.
+
+## Configuration & Tooling
+- **tomli-w:** For writing TOML configuration files.
+- **uv:** An extremely fast Python package and project manager.

conductor/tracks.md

@@ -0,0 +1,8 @@
+# Project Tracks
+
+This file tracks all major tracks for the project. Each track has its own detailed plan in its respective folder.
+
+---
+
+- [x] **Track: Implement context visualization and memory management improvements**
+  *Link: [./tracks/context_management_20260223/](./tracks/context_management_20260223/)*

conductor/tracks/context_management_20260223/index.md

@@ -0,0 +1,5 @@
+# Track context_management_20260223 Context
+
+- [Specification](./spec.md)
+- [Implementation Plan](./plan.md)
+- [Metadata](./metadata.json)

conductor/tracks/context_management_20260223/metadata.json

@@ -0,0 +1,8 @@
+{
+  "track_id": "context_management_20260223",
+  "type": "feature",
+  "status": "new",
+  "created_at": "2026-02-23T10:00:00Z",
+  "updated_at": "2026-02-23T10:00:00Z",
+  "description": "Implement context visualization and memory management improvements"
+}

conductor/tracks/context_management_20260223/plan.md

@@ -0,0 +1,19 @@
+# Implementation Plan
+
+## Phase 1: Context Memory and Token Visualization [checkpoint: a88311b]
+- [x] Task: Implement token usage summary widget e34ff7e
+    - [ ] Sub-task: Write Tests
+    - [ ] Sub-task: Implement Feature
+- [x] Task: Expose history truncation controls in the Discussion panel 94fe904
+    - [ ] Sub-task: Write Tests
+    - [ ] Sub-task: Implement Feature
+- [x] Task: Conductor - User Manual Verification 'Phase 1: Context Memory and Token Visualization' (Protocol in workflow.md) a88311b
+
+## Phase 2: Agent Capability Configuration [checkpoint: 1ac6eb9]
+- [x] Task: Add UI toggles for available tools per-project 1677d25
+    - [x] Sub-task: Write Tests
+    - [x] Sub-task: Implement Feature
+- [x] Task: Wire tool toggles to AI provider tool declaration payload 92aa33c
+    - [ ] Sub-task: Write Tests
+    - [ ] Sub-task: Implement Feature
+- [x] Task: Conductor - User Manual Verification 'Phase 2: Agent Capability Configuration' (Protocol in workflow.md) 1ac6eb9

conductor/tracks/context_management_20260223/spec.md

@@ -0,0 +1,9 @@
+# Specification: Context Visualization and Memory Management
+
+## Overview
+This track implements UI improvements and structural changes to Manual Slop to provide explicit visualization of context memory usage and token consumption, fulfilling the "Expert systems level utility" and "Full control" product goals.
+
+## Core Objectives
+1. **Token Visualization:** Expose token usage metrics in real-time within the GUI (e.g., in a dedicated metrics panel or augmented Comms panel).
+2. **Context Memory Management:** Provide tools to manually flush, persist, or truncate history to manage token budgets per-discussion.
+3. **Agent Capability Toggles:** Expose explicit configuration options for agent capabilities (e.g., toggle MCP tools on/off) from the UI.

conductor/workflow.md

@@ -0,0 +1,333 @@
+# Project Workflow
+
+## Guiding Principles
+
+1. **The Plan is the Source of Truth:** All work must be tracked in `plan.md`
+2. **The Tech Stack is Deliberate:** Changes to the tech stack must be documented in `tech-stack.md` *before* implementation
+3. **Test-Driven Development:** Write unit tests before implementing functionality
+4. **High Code Coverage:** Aim for >80% code coverage for all modules
+5. **User Experience First:** Every decision should prioritize user experience
+6. **Non-Interactive & CI-Aware:** Prefer non-interactive commands. Use `CI=true` for watch-mode tools (tests, linters) to ensure single execution.
+
+## Task Workflow
+
+All tasks follow a strict lifecycle:
+
+### Standard Task Workflow
+
+1. **Select Task:** Choose the next available task from `plan.md` in sequential order
+
+2. **Mark In Progress:** Before beginning work, edit `plan.md` and change the task from `[ ]` to `[~]`
+
+3. **Write Failing Tests (Red Phase):**
+   - Create a new test file for the feature or bug fix.
+   - Write one or more unit tests that clearly define the expected behavior and acceptance criteria for the task.
+   - **CRITICAL:** Run the tests and confirm that they fail as expected. This is the "Red" phase of TDD. Do not proceed until you have failing tests.
+
+4. **Implement to Pass Tests (Green Phase):**
+   - Write the minimum amount of application code necessary to make the failing tests pass.
+   - Run the test suite again and confirm that all tests now pass. This is the "Green" phase.
+
+5. **Refactor (Optional but Recommended):**
+   - With the safety of passing tests, refactor the implementation code and the test code to improve clarity, remove duplication, and enhance performance without changing the external behavior.
+   - Rerun tests to ensure they still pass after refactoring.
+
+6. **Verify Coverage:** Run coverage reports using the project's chosen tools. For example, in a Python project, this might look like:
+   ```bash
+   pytest --cov=app --cov-report=html
+   ```
+   Target: >80% coverage for new code. The specific tools and commands will vary by language and framework.
+
+7. **Document Deviations:** If implementation differs from tech stack:
+   - **STOP** implementation
+   - Update `tech-stack.md` with new design
+   - Add dated note explaining the change
+   - Resume implementation
+
+8. **Commit Code Changes:**
+   - Stage all code changes related to the task.
+   - Propose a clear, concise commit message e.g, `feat(ui): Create basic HTML structure for calculator`.
+   - Perform the commit.
+
+9. **Attach Task Summary with Git Notes:**
+   - **Step 9.1: Get Commit Hash:** Obtain the hash of the *just-completed commit* (`git log -1 --format="%H"`).
+   - **Step 9.2: Draft Note Content:** Create a detailed summary for the completed task. This should include the task name, a summary of changes, a list of all created/modified files, and the core "why" for the change.
+   - **Step 9.3: Attach Note:** Use the `git notes` command to attach the summary to the commit.
+     ```bash
+     # The note content from the previous step is passed via the -m flag.
+     git notes add -m "<note content>" <commit_hash>
+     ```
+
+10. **Get and Record Task Commit SHA:**
+    - **Step 10.1: Update Plan:** Read `plan.md`, find the line for the completed task, update its status from `[~]` to `[x]`, and append the first 7 characters of the *just-completed commit's* commit hash.
+    - **Step 10.2: Write Plan:** Write the updated content back to `plan.md`.
+
+11. **Commit Plan Update:**
+    - **Action:** Stage the modified `plan.md` file.
+    - **Action:** Commit this change with a descriptive message (e.g., `conductor(plan): Mark task 'Create user model' as complete`).
+
+### Phase Completion Verification and Checkpointing Protocol
+
+**Trigger:** This protocol is executed immediately after a task is completed that also concludes a phase in `plan.md`.
+
+1.  **Announce Protocol Start:** Inform the user that the phase is complete and the verification and checkpointing protocol has begun.
+
+2.  **Ensure Test Coverage for Phase Changes:**
+    -   **Step 2.1: Determine Phase Scope:** To identify the files changed in this phase, you must first find the starting point. Read `plan.md` to find the Git commit SHA of the *previous* phase's checkpoint. If no previous checkpoint exists, the scope is all changes since the first commit.
+    -   **Step 2.2: List Changed Files:** Execute `git diff --name-only <previous_checkpoint_sha> HEAD` to get a precise list of all files modified during this phase.
+    -   **Step 2.3: Verify and Create Tests:** For each file in the list:
+        -   **CRITICAL:** First, check its extension. Exclude non-code files (e.g., `.json`, `.md`, `.yaml`).
+        -   For each remaining code file, verify a corresponding test file exists.
+        -   If a test file is missing, you **must** create one. Before writing the test, **first, analyze other test files in the repository to determine the correct naming convention and testing style.** The new tests **must** validate the functionality described in this phase's tasks (`plan.md`).
+
+3.  **Execute Automated Tests with Proactive Debugging:**
+    -   Before execution, you **must** announce the exact shell command you will use to run the tests.
+    -   **Example Announcement:** "I will now run the automated test suite to verify the phase. **Command:** `CI=true npm test`"
+    -   Execute the announced command.
+    -   If tests fail, you **must** inform the user and begin debugging. You may attempt to propose a fix a **maximum of two times**. If the tests still fail after your second proposed fix, you **must stop**, report the persistent failure, and ask the user for guidance.
+
+4.  **Propose a Detailed, Actionable Manual Verification Plan:**
+    -   **CRITICAL:** To generate the plan, first analyze `product.md`, `product-guidelines.md`, and `plan.md` to determine the user-facing goals of the completed phase.
+    -   You **must** generate a step-by-step plan that walks the user through the verification process, including any necessary commands and specific, expected outcomes.
+    -   The plan you present to the user **must** follow this format:
+
+        **For a Frontend Change:**
+        ```
+        The automated tests have passed. For manual verification, please follow these steps:
+
+        **Manual Verification Steps:**
+        1.  **Start the development server with the command:** `npm run dev`
+        2.  **Open your browser to:** `http://localhost:3000`
+        3.  **Confirm that you see:** The new user profile page, with the user's name and email displayed correctly.
+        ```
+
+        **For a Backend Change:**
+        ```
+        The automated tests have passed. For manual verification, please follow these steps:
+
+        **Manual Verification Steps:**
+        1.  **Ensure the server is running.**
+        2.  **Execute the following command in your terminal:** `curl -X POST http://localhost:8080/api/v1/users -d '{"name": "test"}'`
+        3.  **Confirm that you receive:** A JSON response with a status of `201 Created`.
+        ```
+
+5.  **Await Explicit User Feedback:**
+    -   After presenting the detailed plan, ask the user for confirmation: "**Does this meet your expectations? Please confirm with yes or provide feedback on what needs to be changed.**"
+    -   **PAUSE** and await the user's response. Do not proceed without an explicit yes or confirmation.
+
+6.  **Create Checkpoint Commit:**
+    -   Stage all changes. If no changes occurred in this step, proceed with an empty commit.
+    -   Perform the commit with a clear and concise message (e.g., `conductor(checkpoint): Checkpoint end of Phase X`).
+
+7.  **Attach Auditable Verification Report using Git Notes:**
+    -   **Step 7.1: Draft Note Content:** Create a detailed verification report including the automated test command, the manual verification steps, and the user's confirmation.
+    -   **Step 7.2: Attach Note:** Use the `git notes` command and the full commit hash from the previous step to attach the full report to the checkpoint commit.
+
+8.  **Get and Record Phase Checkpoint SHA:**
+    -   **Step 8.1: Get Commit Hash:** Obtain the hash of the *just-created checkpoint commit* (`git log -1 --format="%H"`).
+    -   **Step 8.2: Update Plan:** Read `plan.md`, find the heading for the completed phase, and append the first 7 characters of the commit hash in the format `[checkpoint: <sha>]`.
+    -   **Step 8.3: Write Plan:** Write the updated content back to `plan.md`.
+
+9. **Commit Plan Update:**
+    - **Action:** Stage the modified `plan.md` file.
+    - **Action:** Commit this change with a descriptive message following the format `conductor(plan): Mark phase '<PHASE NAME>' as complete`.
+
+10.  **Announce Completion:** Inform the user that the phase is complete and the checkpoint has been created, with the detailed verification report attached as a git note.
+
+### Quality Gates
+
+Before marking any task complete, verify:
+
+- [ ] All tests pass
+- [ ] Code coverage meets requirements (>80%)
+- [ ] Code follows project's code style guidelines (as defined in `code_styleguides/`)
+- [ ] All public functions/methods are documented (e.g., docstrings, JSDoc, GoDoc)
+- [ ] Type safety is enforced (e.g., type hints, TypeScript types, Go types)
+- [ ] No linting or static analysis errors (using the project's configured tools)
+- [ ] Works correctly on mobile (if applicable)
+- [ ] Documentation updated if needed
+- [ ] No security vulnerabilities introduced
+
+## Development Commands
+
+**AI AGENT INSTRUCTION: This section should be adapted to the project's specific language, framework, and build tools.**
+
+### Setup
+```bash
+# Example: Commands to set up the development environment (e.g., install dependencies, configure database)
+# e.g., for a Node.js project: npm install
+# e.g., for a Go project: go mod tidy
+```
+
+### Daily Development
+```bash
+# Example: Commands for common daily tasks (e.g., start dev server, run tests, lint, format)
+# e.g., for a Node.js project: npm run dev, npm test, npm run lint
+# e.g., for a Go project: go run main.go, go test ./..., go fmt ./...
+```
+
+### Before Committing
+```bash
+# Example: Commands to run all pre-commit checks (e.g., format, lint, type check, run tests)
+# e.g., for a Node.js project: npm run check
+# e.g., for a Go project: make check (if a Makefile exists)
+```
+
+## Testing Requirements
+
+### Unit Testing
+- Every module must have corresponding tests.
+- Use appropriate test setup/teardown mechanisms (e.g., fixtures, beforeEach/afterEach).
+- Mock external dependencies.
+- Test both success and failure cases.
+
+### Integration Testing
+- Test complete user flows
+- Verify database transactions
+- Test authentication and authorization
+- Check form submissions
+
+### Mobile Testing
+- Test on actual iPhone when possible
+- Use Safari developer tools
+- Test touch interactions
+- Verify responsive layouts
+- Check performance on 3G/4G
+
+## Code Review Process
+
+### Self-Review Checklist
+Before requesting review:
+
+1. **Functionality**
+   - Feature works as specified
+   - Edge cases handled
+   - Error messages are user-friendly
+
+2. **Code Quality**
+   - Follows style guide
+   - DRY principle applied
+   - Clear variable/function names
+   - Appropriate comments
+
+3. **Testing**
+   - Unit tests comprehensive
+   - Integration tests pass
+   - Coverage adequate (>80%)
+
+4. **Security**
+   - No hardcoded secrets
+   - Input validation present
+   - SQL injection prevented
+   - XSS protection in place
+
+5. **Performance**
+   - Database queries optimized
+   - Images optimized
+   - Caching implemented where needed
+
+6. **Mobile Experience**
+   - Touch targets adequate (44x44px)
+   - Text readable without zooming
+   - Performance acceptable on mobile
+   - Interactions feel native
+
+## Commit Guidelines
+
+### Message Format
+```
+<type>(<scope>): <description>
+
+[optional body]
+
+[optional footer]
+```
+
+### Types
+- `feat`: New feature
+- `fix`: Bug fix
+- `docs`: Documentation only
+- `style`: Formatting, missing semicolons, etc.
+- `refactor`: Code change that neither fixes a bug nor adds a feature
+- `test`: Adding missing tests
+- `chore`: Maintenance tasks
+
+### Examples
+```bash
+git commit -m "feat(auth): Add remember me functionality"
+git commit -m "fix(posts): Correct excerpt generation for short posts"
+git commit -m "test(comments): Add tests for emoji reaction limits"
+git commit -m "style(mobile): Improve button touch targets"
+```
+
+## Definition of Done
+
+A task is complete when:
+
+1. All code implemented to specification
+2. Unit tests written and passing
+3. Code coverage meets project requirements
+4. Documentation complete (if applicable)
+5. Code passes all configured linting and static analysis checks
+6. Works beautifully on mobile (if applicable)
+7. Implementation notes added to `plan.md`
+8. Changes committed with proper message
+9. Git note with task summary attached to the commit
+
+## Emergency Procedures
+
+### Critical Bug in Production
+1. Create hotfix branch from main
+2. Write failing test for bug
+3. Implement minimal fix
+4. Test thoroughly including mobile
+5. Deploy immediately
+6. Document in plan.md
+
+### Data Loss
+1. Stop all write operations
+2. Restore from latest backup
+3. Verify data integrity
+4. Document incident
+5. Update backup procedures
+
+### Security Breach
+1. Rotate all secrets immediately
+2. Review access logs
+3. Patch vulnerability
+4. Notify affected users (if any)
+5. Document and update security procedures
+
+## Deployment Workflow
+
+### Pre-Deployment Checklist
+- [ ] All tests passing
+- [ ] Coverage >80%
+- [ ] No linting errors
+- [ ] Mobile testing complete
+- [ ] Environment variables configured
+- [ ] Database migrations ready
+- [ ] Backup created
+
+### Deployment Steps
+1. Merge feature branch to main
+2. Tag release with version
+3. Push to deployment service
+4. Run database migrations
+5. Verify deployment
+6. Test critical paths
+7. Monitor for errors
+
+### Post-Deployment
+1. Monitor analytics
+2. Check error logs
+3. Gather user feedback
+4. Plan next iteration
+
+## Continuous Improvement
+
+- Review workflow weekly
+- Update based on pain points
+- Document lessons learned
+- Optimize for user happiness
+- Keep things simple and maintainable

config.toml

@@ -1,6 +1,6 @@
 [ai]
-provider = "anthropic"
+provider = "gemini"
-model = "claude-sonnet-4-6"
+model = "gemini-3.1-pro-preview"
 temperature = 0.6000000238418579
 max_tokens = 12000
 history_trunc_limit = 8000
@@ -10,11 +10,11 @@ system_prompt = "DO NOT EVER make a shell script unless told to. DO NOT EVER mak
 palette = "10x Dark"
 font_path = "C:/Users/Ed/AppData/Local/uv/cache/archive-v0/WSthkYsQ82b_ywV6DkiaJ/pygame_gui/data/FiraCode-Regular.ttf"
 font_size = 18.0
-scale = 1.1
+scale = 1.0
 
 [projects]
 paths = [
     "manual_slop.toml",
     "C:/projects/forth/bootslop/bootslop.toml",
 ]
-active = "C:/projects/forth/bootslop/bootslop.toml"
+active = "manual_slop.toml"

docs/Readme.md

@@ -29,7 +29,7 @@ Controls what is explicitly fed into the context compiler.
 
 - **Base Dir:** Defines the root for path resolution and tool constraints.
 - **Paths:** Explicit files or wildcard globs (e.g., src/**/*.rs). 
-- When generating a request, these files are summarized symbolically (summarize.py) to conserve tokens, unless the AI explicitly decides to read their full contents via its internal tools.
+- When generating a request, full file contents are inlined into the context by default (`summary_only=False`). The AI can also call `get_file_summary` via its MCP tools to get a compact structural view of any file on demand.
 
 ## Interaction Panels
 
@@ -46,8 +46,9 @@ Switch between API backends (Gemini, Anthropic) on the fly. Clicking "Fetch Mode
 
 ### Global Text Viewer & Script Outputs
 
-- **Last Script Output:** Whenever the AI executes a background script, this window pops up, flashing blue. It contains both the executed script and the stdout/stderr.
+- **Last Script Output:** Whenever the AI executes a background script, this window pops up, flashing blue. It contains both the executed script and the stdout/stderr. The `[+ Maximize]` buttons read directly from stored instance variables (`_last_script`, `_last_output`) rather than DPG widget tags, so they work correctly regardless of word-wrap state.
 - **Text Viewer:** A large, resizable global popup invoked anytime you click a [+] or [+ Maximize] button in the UI. Used for deep-reading long logs, discussion entries, or script bodies.
+- **Confirm Dialog:** The `[+ Maximize]` button in the script approval modal passes the script text directly as `user_data` at button-creation time, so it remains safe to click even after the dialog has been dismissed.
 
 ## System Prompts
 

docs/guide_architecture.md

@@ -1,4 +1,4 @@
-# Guide: Architecture
+# Guide: Architecture
 
 Overview of the package design, state management, and code-path layout.
 
@@ -33,10 +33,9 @@ This occurs inside aggregate.run.
 If using the default workflow, aggregate.py hashes through the following process:
 
 1. **Glob Resolution:** Iterates through config["files"]["paths"] and unpacks any wildcards (e.g., src/**/*.rs) against the designated base_dir.
-2. **Summarization Pass:** Instead of concatenating raw file bodies (which would quickly overwhelm the ~200k token limit over multiple rounds), the files are passed to summarize.py.
+2. **File Item Build:** `build_file_items()` reads each resolved file once, storing path, content, and `mtime`. This list is returned alongside the markdown so `ai_client.py` can use it for dynamic context refresh after tool calls without re-reading from disk.
-3. **AST Parsing:** summarize.py runs a heuristic pass. For Python files, it uses the standard ast module to read structural nodes (Classes, Methods, Imports, Constants). It outputs a compact Markdown table.
+3. **Markdown Generation:** `build_markdown_from_items()` assembles the final `<project>_00N.md` string. By default (`summary_only=False`) it inlines full file contents. If `summary_only=True`, it delegates to `summarize.build_summary_markdown()` which uses AST-based heuristics to produce compact structural summaries instead.
-4. **Markdown Generation:** The final <project>_00N.md string is constructed, comprising the truncated AST summaries, the user's current project system prompt, and the active discussion branch. 
+4. The Markdown file is persisted to disk (`./md_gen/` by default) for auditing. `run()` returns a 3-tuple `(markdown_str, output_path, file_items)`.
-5. The Markdown file is persisted to disk (./md_gen/ by default) for auditing.
 
 ### AI Communication & The Tool Loop
 
@@ -85,3 +84,4 @@ All I/O bound session data is recorded sequentially. session_logger.py hooks int
 - logs/comms_<ts>.log: A JSON-L structured timeline of every raw payload sent/received.
 - logs/toolcalls_<ts>.log: A sequential markdown record detailing every AI tool invocation and its exact stdout result.
 - scripts/generated/: Every .ps1 script approved and executed by the shell runner is physically written to disk for version control transparency.
+

docs/guide_tools.md

@@ -12,17 +12,22 @@ Implemented in mcp_client.py. These tools allow the AI to selectively expand its
 
 ### Security & Scope
 
-Every filesystem MCP tool passes its arguments through _resolve_and_check. This function ensures that the requested path falls under one of the allowed directories defined in the GUI's Base Dir configurations. 
+Every **filesystem** MCP tool passes its arguments through `_resolve_and_check`. This function ensures that the requested path falls under one of the allowed directories defined in the GUI's Base Dir configurations.
 If the AI attempts to read or search a path outside the project bounds, the tool safely catches the constraint violation and returns ACCESS DENIED.
 
+The two **web tools** (`web_search`, `fetch_url`) bypass this check entirely — they have no filesystem access and are unrestricted.
+
 ### Supplied Tools:
 
-* read_file(path): Returns the raw UTF-8 text of a file.
+**Filesystem tools** (access-controlled via `_resolve_and_check`):
-* list_directory(path): Returns a formatted table of a directory's contents, showing file vs dir and byte sizes.
+* `read_file(path)`: Returns the raw UTF-8 text of a file.
-* search_files(path, pattern): Executes an absolute glob search (e.g., **/*.py) to find specific files.
+* `list_directory(path)`: Returns a formatted table of a directory's contents, showing file vs dir and byte sizes.
-* get_file_summary(path): Invokes the local summarize.py heuristic parser to get the AST structure of a file without reading the whole body.
+* `search_files(path, pattern)`: Executes a glob search (e.g., `**/*.py`) within an allowed directory.
-* web_search(query): Queries DuckDuckGo's raw HTML endpoint and returns the top 5 results (Titles, URLs, Snippets) using a native HTMLParser to avoid heavy dependencies.
+* `get_file_summary(path)`: Invokes the local `summarize.py` heuristic parser to get the AST structure of a file without reading the whole body.
-* fetch_url(url): Downloads a target webpage and strips out all scripts, styling, and structural HTML, returning only the raw prose content (clamped to 40,000 characters).
+
+**Web tools** (unrestricted — no filesystem access):
+* `web_search(query)`: Queries DuckDuckGo's raw HTML endpoint and returns the top 5 results (title, URL, snippet) using a native `_DDGParser` (HTMLParser subclass) to avoid heavy dependencies.
+* `fetch_url(url)`: Downloads a target webpage and strips out all scripts, styling, and structural HTML via `_TextExtractor`, returning only the raw prose content (clamped to 40,000 characters). Automatically resolves DuckDuckGo redirect links.
 
 ## 2. Destructive Execution (run_powershell)
 

gui.py

@@ -1,4 +1,4 @@
-# gui.py
+# gui.py
 """
 Note(Gemini):
 The main DearPyGui interface orchestrator.
@@ -47,6 +47,30 @@ def hide_tk_root() -> Tk:
     root.wm_attributes("-topmost", True)
     return root
 
+def get_total_token_usage() -> dict:
+    """Returns aggregated token usage across the entire session from comms log."""
+    usage = {
+        "input_tokens": 0,
+        "output_tokens": 0,
+        "cache_read_input_tokens": 0,
+        "cache_creation_input_tokens": 0
+    }
+    for entry in ai_client.get_comms_log():
+        if entry.get("kind") == "response" and "usage" in entry.get("payload", {}):
+            u = entry["payload"]["usage"]
+            for k in usage.keys():
+                usage[k] += u.get(k, 0) or 0
+    return usage
+
+def truncate_entries(entries: list[dict], max_pairs: int) -> list[dict]:
+    """Truncates history to the last N pairs of User/AI messages."""
+    if max_pairs <= 0:
+        return []
+    target_count = max_pairs * 2
+    if len(entries) <= target_count:
+        return entries
+    return entries[-target_count:]
+
 
 # ------------------------------------------------------------------ comms rendering helpers
 
@@ -301,9 +325,9 @@ class ConfirmDialog:
                 with dpg.group(horizontal=True):
                     dpg.add_text("Script:")
                     dpg.add_button(
-                        label="[+ Maximize]", 
+                        label="[+ Maximize]",
-                        user_data=f"{self._tag}_script",
+                        user_data=self._script,
-                        callback=lambda s, a, u: _show_text_viewer("Confirm Script", dpg.get_value(u))
+                        callback=lambda s, a, u: _show_text_viewer("Confirm Script", u)
                     )
                 dpg.add_input_text(
                     tag=f"{self._tag}_script",
@@ -432,6 +456,8 @@ class App:
         self._pending_dialog_lock = threading.Lock()
 
         self._tool_log: list[tuple[str, str]] = []
+        self._last_script: str = ""
+        self._last_output: str = ""
 
         # Comms log entries queued from background thread for main-thread rendering
         self._pending_comms: list[dict] = []
@@ -563,6 +589,13 @@ class App:
             dpg.set_value("auto_add_history", proj.get("discussion", {}).get("auto_add", False))
         if dpg.does_item_exist("project_word_wrap"):
             dpg.set_value("project_word_wrap", proj.get("project", {}).get("word_wrap", True))
+        
+        agent_tools = proj.get("agent", {}).get("tools", {})
+        for t_name in ["run_powershell", "read_file", "list_directory", "search_files", "get_file_summary", "web_search", "fetch_url"]:
+            tag = f"tool_toggle_{t_name}"
+            if dpg.does_item_exist(tag):
+                dpg.set_value(tag, agent_tools.get(t_name, True))
+
         self.cb_word_wrap_toggled(app_data=proj.get("project", {}).get("word_wrap", True))
 
     def _save_active_project(self):
@@ -711,6 +744,15 @@ class App:
         for entry in entries:
             self._comms_entry_count += 1
             self._append_comms_entry(entry, self._comms_entry_count)
+        if entries:
+            self._update_token_usage()
+
+    def _update_token_usage(self):
+        if not dpg.does_item_exist("ai_token_usage"):
+            return
+        usage = get_total_token_usage()
+        total = usage["input_tokens"] + usage["output_tokens"]
+        dpg.set_value("ai_token_usage", f"Tokens: {total} (In: {usage['input_tokens']} Out: {usage['output_tokens']})")
 
     def _append_comms_entry(self, entry: dict, idx: int):
         if not dpg.does_item_exist("comms_scroll"):
@@ -748,6 +790,8 @@ class App:
         return output
 
     def _append_tool_log(self, script: str, result: str):
+        self._last_script = script
+        self._last_output = result
         self._tool_log.append((script, result))
         self._rebuild_tool_log()
         
@@ -830,6 +874,13 @@ class App:
         if dpg.does_item_exist("project_word_wrap"):
             proj["project"]["word_wrap"] = dpg.get_value("project_word_wrap")
 
+        # Agent tools
+        proj.setdefault("agent", {}).setdefault("tools", {})
+        for t_name in ["run_powershell", "read_file", "list_directory", "search_files", "get_file_summary", "web_search", "fetch_url"]:
+            tag = f"tool_toggle_{t_name}"
+            if dpg.does_item_exist(tag):
+                proj["agent"]["tools"][t_name] = dpg.get_value(tag)
+
         # Discussion
         self._flush_disc_entries_to_project()
         disc_sec = proj.setdefault("discussion", {})
@@ -1152,6 +1203,7 @@ class App:
         if global_sp: combined_sp.append(global_sp.strip())
         if project_sp: combined_sp.append(project_sp.strip())
         ai_client.set_custom_system_prompt("\n\n".join(combined_sp))
+        ai_client.set_agent_tools(self.project.get("agent", {}).get("tools", {}))
         temp = dpg.get_value("ai_temperature") if dpg.does_item_exist("ai_temperature") else 0.0
         max_tok = dpg.get_value("ai_max_tokens") if dpg.does_item_exist("ai_max_tokens") else 8192
         trunc = dpg.get_value("ai_history_trunc") if dpg.does_item_exist("ai_history_trunc") else 8000
@@ -1213,6 +1265,7 @@ class App:
         with self._pending_comms_lock:
             self._pending_comms.clear()
         self._comms_entry_count = 0
+        self._update_token_usage()
         if dpg.does_item_exist("comms_scroll"):
             dpg.delete_item("comms_scroll", children_only=True)
 
@@ -1315,6 +1368,12 @@ class App:
         self.disc_entries.clear()
         self._rebuild_disc_list()
 
+    def cb_disc_truncate(self):
+        pairs = dpg.get_value("disc_truncate_pairs") if dpg.does_item_exist("disc_truncate_pairs") else 2
+        self.disc_entries = truncate_entries(self.disc_entries, pairs)
+        self._rebuild_disc_list()
+        self._update_status(f"history truncated to {pairs} pairs")
+
     def cb_disc_collapse_all(self):
         for i, entry in enumerate(self.disc_entries):
             tag = f"disc_content_{i}"
@@ -1655,6 +1714,15 @@ class App:
                 default_value=self.project.get("project", {}).get("word_wrap", True),
                 callback=self.cb_word_wrap_toggled
             )
+            dpg.add_separator()
+            dpg.add_text("Agent Capabilities")
+            agent_tools = self.project.get("agent", {}).get("tools", {})
+            for t_name in ["run_powershell", "read_file", "list_directory", "search_files", "get_file_summary", "web_search", "fetch_url"]:
+                dpg.add_checkbox(
+                    tag=f"tool_toggle_{t_name}",
+                    label=f"Enable {t_name}",
+                    default_value=agent_tools.get(t_name, True)
+                )
 
         # ---- Files panel ----
         with dpg.window(
@@ -1732,6 +1800,9 @@ class App:
                 dpg.add_button(label="+ Entry", callback=self.cb_disc_append_entry)
                 dpg.add_button(label="-All", callback=self.cb_disc_collapse_all)
                 dpg.add_button(label="+All", callback=self.cb_disc_expand_all)
+                dpg.add_text("Keep Pairs:", color=(160, 160, 160))
+                dpg.add_input_int(tag="disc_truncate_pairs", default_value=2, width=120, min_value=1)
+                dpg.add_button(label="Truncate", callback=self.cb_disc_truncate)
                 dpg.add_button(label="Clear All", callback=self.cb_disc_clear)
                 dpg.add_button(label="Save", callback=self.cb_disc_save)
             dpg.add_checkbox(
@@ -1860,6 +1931,8 @@ class App:
             with dpg.group(horizontal=True):
                 dpg.add_text("Status: idle", tag="ai_status", color=(200, 220, 160))
                 dpg.add_spacer(width=16)
+                dpg.add_text("Tokens: 0 (In: 0 Out: 0)", tag="ai_token_usage", color=(180, 255, 180))
+                dpg.add_spacer(width=16)
                 dpg.add_button(label="Clear", callback=self.cb_clear_comms)
             dpg.add_separator()
             with dpg.group(horizontal=True):
@@ -1917,8 +1990,7 @@ class App:
                 dpg.add_text("Script:")
                 dpg.add_button(
                     label="[+ Maximize]",
-                    user_data="last_script_text",
+                    callback=lambda s, a, u: _show_text_viewer("Last Script", self._last_script),
-                    callback=lambda s, a, u: _show_text_viewer("Last Script", dpg.get_value(u))
                 )
             dpg.add_input_text(
                 tag="last_script_text",
@@ -1934,8 +2006,7 @@ class App:
                 dpg.add_text("Output:")
                 dpg.add_button(
                     label="[+ Maximize]",
-                    user_data="last_script_output",
+                    callback=lambda s, a, u: _show_text_viewer("Last Output", self._last_output),
-                    callback=lambda s, a, u: _show_text_viewer("Last Output", dpg.get_value(u))
                 )
             dpg.add_input_text(
                 tag="last_script_output",
@@ -2119,4 +2190,8 @@ def main():
 
 
 if __name__ == "__main__":
-    main()
+    main()
+
+
+
+

project_manager.py

@@ -100,6 +100,17 @@ def default_project(name: str = "unnamed") -> dict:
         "output":      {"output_dir": "./md_gen"},
         "files":       {"base_dir": ".", "paths": []},
         "screenshots": {"base_dir": ".", "paths": []},
+        "agent":       {
+            "tools": {
+                "run_powershell": True,
+                "read_file": True,
+                "list_directory": True,
+                "search_files": True,
+                "get_file_summary": True,
+                "web_search": True,
+                "fetch_url": True
+            }
+        },
         "discussion":  {
             "roles":       ["User", "AI", "Vendor API", "System"],
             "active":      "main",

pyproject.toml

@@ -10,3 +10,8 @@ dependencies = [
     "anthropic",
     "tomli-w"
 ]
+
+[dependency-groups]
+dev = [
+    "pytest>=9.0.2",
+]

setup_gemini.ps1

@@ -0,0 +1 @@
+Get-Content .env | ForEach-Object { $name, $value = $_.Split('=', 2); [Environment]::SetEnvironmentVariable($name, $value, "Process") }

tests/test_agent_capabilities.py

@@ -0,0 +1,17 @@
+import pytest
+
+def test_agent_capabilities_config():
+    # A dummy test to fulfill the Red Phase for Agent Capability Configuration.
+    # The new function in gui.py should be get_active_tools() or we check the project dict.
+    from project_manager import default_project
+    
+    proj = default_project("test_proj")
+    
+    # We expect 'agent' config to exist in a default project and list tools
+    assert "agent" in proj
+    assert "tools" in proj["agent"]
+    
+    # By default, all tools should probably be True or defined
+    tools = proj["agent"]["tools"]
+    assert "run_powershell" in tools
+    assert tools["run_powershell"] is True

tests/test_agent_tools_wiring.py

@@ -0,0 +1,23 @@
+import pytest
+from ai_client import set_agent_tools, _build_anthropic_tools
+
+def test_agent_tools_wiring():
+    # Only enable read_file and run_powershell
+    agent_tools = {
+        "run_powershell": True,
+        "read_file": True,
+        "list_directory": False,
+        "search_files": False,
+        "get_file_summary": False,
+        "web_search": False,
+        "fetch_url": False
+    }
+    set_agent_tools(agent_tools)
+    
+    anth_tools = _build_anthropic_tools()
+    tool_names = [t["name"] for t in anth_tools]
+    
+    assert "read_file" in tool_names
+    assert "run_powershell" in tool_names
+    assert "list_directory" not in tool_names
+    assert "web_search" not in tool_names

tests/test_history_truncation.py

@@ -0,0 +1,22 @@
+import pytest
+
+def test_history_truncation():
+    # A dummy test to fulfill the Red Phase for the history truncation controls.
+    # The new function in gui.py should be cb_disc_truncate_history or a related utility.
+    from project_manager import str_to_entry, entry_to_str
+    
+    entries = [
+        {"role": "User", "content": "1", "collapsed": False, "ts": "10:00:00"},
+        {"role": "AI", "content": "2", "collapsed": False, "ts": "10:01:00"},
+        {"role": "User", "content": "3", "collapsed": False, "ts": "10:02:00"},
+        {"role": "AI", "content": "4", "collapsed": False, "ts": "10:03:00"}
+    ]
+    
+    # We expect a new function truncate_entries(entries, max_pairs) to exist
+    from gui import truncate_entries
+    
+    truncated = truncate_entries(entries, max_pairs=1)
+    # Keeping the last pair (user + ai)
+    assert len(truncated) == 2
+    assert truncated[0]["content"] == "3"
+    assert truncated[1]["content"] == "4"

tests/test_token_usage.py

@@ -0,0 +1,35 @@
+import pytest
+
+def test_token_usage_aggregation():
+    # A dummy test to fulfill the Red Phase for the new token usage widget.
+    # We will implement a function in gui.py or ai_client.py to aggregate tokens.
+    from ai_client import _comms_log, clear_comms_log, _append_comms
+    
+    clear_comms_log()
+    
+    _append_comms("IN", "response", {
+        "usage": {
+            "input_tokens": 100,
+            "output_tokens": 50,
+            "cache_read_input_tokens": 10,
+            "cache_creation_input_tokens": 5
+        }
+    })
+    
+    _append_comms("IN", "response", {
+        "usage": {
+            "input_tokens": 200,
+            "output_tokens": 100,
+            "cache_read_input_tokens": 20,
+            "cache_creation_input_tokens": 0
+        }
+    })
+    
+    # We expect a new function get_total_token_usage() to exist
+    from gui import get_total_token_usage
+    
+    totals = get_total_token_usage()
+    assert totals["input_tokens"] == 300
+    assert totals["output_tokens"] == 150
+    assert totals["cache_read_input_tokens"] == 30
+    assert totals["cache_creation_input_tokens"] == 5