gemini "fixes"

MainContext.md

@@ -87,9 +87,9 @@ Is a local GUI tool for manually curating and sending context to AI APIs. It agg
 - All tool calls (script + result/rejection) are appended to `_tool_log` and displayed in the Tool Calls panel
 
 **Dynamic file context refresh (ai_client.py):**
-- After every tool call round, all project files from `file_items` are re-read from disk via `_reread_file_items()`
+- After the last tool call in each round, all project files from `file_items` are re-read from disk via `_reread_file_items()`. The `file_items` variable is reassigned so subsequent rounds see fresh content.
-- For Anthropic: the refreshed file contents are injected as a `text` block appended to the `tool_results` user message, prefixed with `[FILES UPDATED]` and an instruction not to re-read them
+- For Anthropic: the refreshed file contents are injected as a `text` block appended to the `tool_results` user message, prefixed with `[FILES UPDATED]` and an instruction not to re-read them.
-- For Gemini: files are re-read (updating the `file_items` list in place) but cannot be injected into tool results due to Gemini's structured function response format
+- For Gemini: refreshed file contents are appended to the last function response's `output` string as a `[SYSTEM: FILES UPDATED]` block. On the next tool round, stale `[FILES UPDATED]` blocks are stripped from history and old tool outputs are truncated to `_history_trunc_limit` characters to control token growth.
 - `_build_file_context_text(file_items)` formats the refreshed files as markdown code blocks (same format as the original context)
 - The `tool_result_send` comms log entry filters out the injected text block (only logs actual `tool_result` entries) to keep the comms panel clean
 - `file_items` flows from `aggregate.build_file_items()` → `gui.py` `self.last_file_items` → `ai_client.send(file_items=...)` → `_send_anthropic(file_items=...)` / `_send_gemini(file_items=...)`
@@ -142,9 +142,11 @@ Entry layout: index + timestamp + direction + kind + provider/model header row,
 - `close_session()` flushes and closes both file handles; called just before `dpg.destroy_context()`
 
 **Anthropic prompt caching:**
-- System prompt sent as an array with `cache_control: ephemeral` on the text block
+- System prompt + context are combined into one string, chunked into <=120k char blocks, and sent as the `system=` parameter array. Only the LAST chunk gets `cache_control: ephemeral`, so the entire system prefix is cached as one unit.
-- Last tool in `_ANTHROPIC_TOOLS` has `cache_control: ephemeral`; system + tools prefix is cached together after the first request
+- Last tool in `_ANTHROPIC_TOOLS` (`run_powershell`) has `cache_control: ephemeral`; this means the tools prefix is cached together with the system prefix after the first request.
-- First user message content[0] is the `<context>` block with `cache_control: ephemeral`; content[1] is the user question without cache control
+- The user message is sent as a plain `[{"type": "text", "text": user_message}]` block with NO cache_control. The context lives in `system=`, not in the first user message.
+- The tools list is built once per session via `_get_anthropic_tools()` and reused across all API calls within the tool loop, avoiding redundant Python-side reconstruction.
+- `_strip_cache_controls()` removes stale `cache_control` markers from all history entries before each API call, ensuring only the stable system/tools prefix consumes cache breakpoint slots.
 - Cache stats (creation tokens, read tokens) are surfaced in the comms log usage dict and displayed in the Comms History panel
 
 **Data flow:**
@@ -190,15 +192,17 @@ Entry layout: index + timestamp + direction + kind + provider/model header row,
 
 **Known extension points:**
 - Add more providers by adding a section to `credentials.toml`, a `_list_*` and `_send_*` function in `ai_client.py`, and the provider name to the `PROVIDERS` list in `gui.py`
-- System prompt support could be added as a field in the project `.toml` and passed in `ai_client.send()`
 - Discussion history excerpts could be individually toggleable for inclusion in the generated md
 - `MAX_TOOL_ROUNDS` in `ai_client.py` caps agentic loops at 10 rounds; adjustable
 - `COMMS_CLAMP_CHARS` in `gui.py` controls the character threshold for clamping heavy payload fields in the Comms History panel
 - Additional project metadata (description, tags, created date) could be added to `[project]` in the per-project toml
 
 ### Gemini Context Management
-- Investigating ways to prevent context duplication in _gemini_chat history, as currently <context>{md_content}</context> is prepended to the user message on every single request, causing history bloat.
+- Gemini uses explicit caching via `client.caches.create()` to store the `system_instruction` + tools as an immutable cached prefix with a 1-hour TTL. The cache is created once per chat session.
-- Discussing explicit Gemini Context Caching API (client.caches.create()) to store read-only file context and avoid re-reading files across sessions.
+- When context changes (detected via `md_content` hash), the old cache is deleted, a new cache is created, and chat history is migrated to a fresh chat session pointing at the new cache.
+- If cache creation fails (e.g., content is under the minimum token threshold — 1024 for Flash, 4096 for Pro), the system falls back to inline `system_instruction` in the chat config. Implicit caching may still provide cost savings in this case.
+- The `<context>` block lives inside `system_instruction`, NOT in user messages, preventing history bloat across turns.
+- On cleanup/exit, active caches are deleted via `ai_client.cleanup()` to prevent orphaned billing.
 
 ### Latest Changes
 - Removed `Config` panel from the GUI to streamline per-project configuration.

aggregate.py

@@ -126,9 +126,8 @@ def build_summary_section(base_dir: Path, files: list[str]) -> str:
     items = build_file_items(base_dir, files)
     return summarize.build_summary_markdown(items)
 
-def build_markdown(base_dir: Path, files: list[str], screenshot_base_dir: Path, screenshots: list[str], history: list[str], summary_only: bool = False) -> str:
+def build_static_markdown(base_dir: Path, files: list[str], screenshot_base_dir: Path, screenshots: list[str], summary_only: bool = False) -> str:
     parts = []
-    # STATIC PREFIX: Files and Screenshots must go first to maximize Cache Hits
     if files:
         if summary_only:
             parts.append("## Files (Summary)\n\n" + build_summary_section(base_dir, files))
@@ -136,12 +135,12 @@ def build_markdown(base_dir: Path, files: list[str], screenshot_base_dir: Path,
             parts.append("## Files\n\n" + build_files_section(base_dir, files))
     if screenshots:
         parts.append("## Screenshots\n\n" + build_screenshots_section(screenshot_base_dir, screenshots))
-    # DYNAMIC SUFFIX: History changes every turn, must go last
+    return "\n\n---\n\n".join(parts) if parts else ""
-    if history:
-        parts.append("## Discussion History\n\n" + build_discussion_section(history))
-    return "\n\n---\n\n".join(parts)
 
-def run(config: dict) -> tuple[str, Path]:
+def build_dynamic_markdown(history: list[str]) -> str:
+    return "## Discussion History\n\n" + build_discussion_section(history) if history else ""
+
+def run(config: dict) -> tuple[str, str, Path, list[dict]]:
     namespace = config.get("project", {}).get("name")
     if not namespace:
         namespace = config.get("output", {}).get("namespace", "project")
@@ -155,21 +154,22 @@ def run(config: dict) -> tuple[str, Path]:
     output_dir.mkdir(parents=True, exist_ok=True)
     increment = find_next_increment(output_dir, namespace)
     output_file = output_dir / f"{namespace}_{increment:03d}.md"
-    # Provide full files to trigger Gemini's 32k cache threshold and give the AI immediate context
+    
-    markdown = build_markdown(base_dir, files, screenshot_base_dir, screenshots, history,
+    static_md = build_static_markdown(base_dir, files, screenshot_base_dir, screenshots, summary_only=False)
-                              summary_only=False)
+    dynamic_md = build_dynamic_markdown(history)
+    
+    markdown = f"{static_md}\n\n---\n\n{dynamic_md}" if static_md and dynamic_md else static_md or dynamic_md
     output_file.write_text(markdown, encoding="utf-8")
+    
     file_items = build_file_items(base_dir, files)
-    return markdown, output_file, file_items
+    return static_md, dynamic_md, output_file, file_items
 
 def main():
     with open("config.toml", "rb") as f:
         import tomllib
         config = tomllib.load(f)
-    markdown, output_file, _ = run(config)
+    static_md, dynamic_md, output_file, _ = run(config)
     print(f"Written: {output_file}")
 
 if __name__ == "__main__":
     main()
-
-

ai_client.py

@@ -217,6 +217,7 @@ def cleanup():
 def reset_session():
     global _gemini_client, _gemini_chat, _gemini_cache
     global _anthropic_client, _anthropic_history
+    global _CACHED_ANTHROPIC_TOOLS
     if _gemini_client and _gemini_cache:
         try:
             _gemini_client.caches.delete(name=_gemini_cache.name)
@@ -227,6 +228,7 @@ def reset_session():
     _gemini_cache = None
     _anthropic_client = None
     _anthropic_history = []
+    _CACHED_ANTHROPIC_TOOLS = None
     file_cache.reset_client()
 
 
@@ -309,6 +311,15 @@ def _build_anthropic_tools() -> list[dict]:
 
 _ANTHROPIC_TOOLS = _build_anthropic_tools()
 
+_CACHED_ANTHROPIC_TOOLS = None
+
+def _get_anthropic_tools() -> list[dict]:
+    """Return the Anthropic tools list, rebuilding only once per session."""
+    global _CACHED_ANTHROPIC_TOOLS
+    if _CACHED_ANTHROPIC_TOOLS is None:
+        _CACHED_ANTHROPIC_TOOLS = _build_anthropic_tools()
+    return _CACHED_ANTHROPIC_TOOLS
+
 
 def _gemini_tool_declaration():
     from google.genai import types
@@ -442,96 +453,67 @@ def _ensure_gemini_client():
         _gemini_client = genai.Client(api_key=creds["gemini"]["api_key"])
 
 
-def _send_gemini(md_content: str, user_message: str, base_dir: str, file_items: list[dict] | None = None) -> str:
+def _send_gemini(static_md: str, dynamic_md: str, user_message: str, base_dir: str, file_items: list[dict] | None = None) -> str:
-    global _gemini_chat
+    global _gemini_chat, _gemini_cache
     from google.genai import types
     try:
         _ensure_gemini_client(); mcp_client.configure(file_items or [], [base_dir])
-        sys_instr = f"{_get_combined_system_prompt()}\n\n<context>\n{md_content}\n</context>"
+        sys_instr = f"{_get_combined_system_prompt()}\n\n<context>\n{static_md}\n</context>"
         tools_decl = [_gemini_tool_declaration()]
         
-        global _gemini_cache, _gemini_chat
+        current_md_hash = hash(static_md)
-        
-        # DYNAMIC CONTEXT: Check if files/context changed mid-session
-        current_md_hash = hash(md_content)
         old_history = None
         if _gemini_chat and getattr(_gemini_chat, "_last_md_hash", None) != current_md_hash:
             old_history = list(_gemini_chat.history) if _gemini_chat.history else []
             if _gemini_cache:
                 try: _gemini_client.caches.delete(name=_gemini_cache.name)
                 except: pass
-            _gemini_chat = None
+            _gemini_chat, _gemini_cache = None, None
-            _gemini_cache = None
+            _append_comms("OUT", "request", {"message": "[STATIC CONTEXT CHANGED] Rebuilding cache and chat session..."})
-            _append_comms("OUT", "request", {"message": "[CONTEXT CHANGED] Rebuilding cache and chat session..."})
 
         if not _gemini_chat:
             chat_config = types.GenerateContentConfig(
-                system_instruction=sys_instr,
+                system_instruction=sys_instr, tools=tools_decl, temperature=_temperature, max_output_tokens=_max_tokens,
-                tools=tools_decl,
-                temperature=_temperature,
-                max_output_tokens=_max_tokens,
                 safety_settings=[types.SafetySetting(category="HARM_CATEGORY_DANGEROUS_CONTENT", threshold="BLOCK_ONLY_HIGH")]
             )
             try:
-                # Gemini requires 1024 (Flash) or 4096 (Pro) tokens to cache.
+                _gemini_cache = _gemini_client.caches.create(model=_model, config=types.CreateCachedContentConfig(system_instruction=sys_instr, tools=tools_decl, ttl="3600s"))
-                _gemini_cache = _gemini_client.caches.create(
-                    model=_model,
-                    config=types.CreateCachedContentConfig(
-                        system_instruction=sys_instr,
-                        tools=tools_decl,
-                        ttl="3600s",
-                    )
-                )
                 chat_config = types.GenerateContentConfig(
-                    cached_content=_gemini_cache.name,
+                    cached_content=_gemini_cache.name, temperature=_temperature, max_output_tokens=_max_tokens,
-                    temperature=_temperature,
-                    max_output_tokens=_max_tokens,
                     safety_settings=[types.SafetySetting(category="HARM_CATEGORY_DANGEROUS_CONTENT", threshold="BLOCK_ONLY_HIGH")]
                 )
                 _append_comms("OUT", "request", {"message": f"[CACHE CREATED] {_gemini_cache.name}"})
-            except Exception as e:
+            except Exception: _gemini_cache = None
-                # Fallback if under token limit or API error
-                pass
                 
             kwargs = {"model": _model, "config": chat_config}
-            if old_history:
+            if old_history: kwargs["history"] = old_history
-                kwargs["history"] = old_history
-            
             _gemini_chat = _gemini_client.chats.create(**kwargs)
             _gemini_chat._last_md_hash = current_md_hash
 
-        # COMPRESS HISTORY: Truncate massive tool outputs from previous turns to stop token leaks
+        import re
-        if _gemini_chat and getattr(_gemini_chat, "history", None):
-            for msg in _gemini_chat.history:
-                if msg.role == "user" and hasattr(msg, "parts"):
-                    for p in msg.parts:
-                        if hasattr(p, "function_response") and p.function_response and hasattr(p.function_response, "response"):
-                            r = p.function_response.response
-                            if isinstance(r, dict) and "output" in r:
-                                val = r["output"]
-                                if isinstance(val, str):
-                                    if "[SYSTEM: FILES UPDATED]" in val:
-                                        val = val.split("[SYSTEM: FILES UPDATED]")[0].strip()
-                                    if _history_trunc_limit > 0 and len(val) > _history_trunc_limit:
-                                        val = val[:_history_trunc_limit] + "\n\n... [TRUNCATED BY SYSTEM TO SAVE TOKENS. Original output was too large.]"
-                                    r["output"] = val
-
-        _append_comms("OUT", "request", {"message": f"[ctx {len(md_content)} + msg {len(user_message)}]"})
-        payload, all_text = user_message, []
-        
-        for r_idx in range(MAX_TOOL_ROUNDS + 2):
-            # Strip stale file refreshes from Gemini history
         if _gemini_chat and _gemini_chat.history:
             for msg in _gemini_chat.history:
                 if msg.role == "user" and hasattr(msg, "parts"):
                     for p in msg.parts:
+                        if hasattr(p, "text") and p.text and "<discussion>" in p.text:
+                            p.text = re.sub(r"<discussion>.*?</discussion>\n\n", "", p.text, flags=re.DOTALL)
                         if hasattr(p, "function_response") and p.function_response and hasattr(p.function_response, "response"):
                             r = p.function_response.response
-                                if isinstance(r, dict) and "output" in r:
+                            r_dict = r if isinstance(r, dict) else getattr(r, "__dict__", {})
-                                    val = r["output"]
+                            val = r_dict.get("output") if isinstance(r_dict, dict) else getattr(r, "output", None)
-                                    if isinstance(val, str) and "[SYSTEM: FILES UPDATED]" in val:
+                            if isinstance(val, str):
-                                        r["output"] = val.split("[SYSTEM: FILES UPDATED]")[0].strip()
+                                if "[SYSTEM: FILES UPDATED]" in val: val = val.split("[SYSTEM: FILES UPDATED]")[0].strip()
+                                if _history_trunc_limit > 0 and len(val) > _history_trunc_limit:
+                                    val = val[:_history_trunc_limit] + "\n\n... [TRUNCATED BY SYSTEM TO SAVE TOKENS.]"
+                                if isinstance(r, dict): r["output"] = val
+                                else: setattr(r, "output", val)
 
+        full_user_msg = f"<discussion>\n{dynamic_md}\n</discussion>\n\n{user_message}" if dynamic_md else user_message
+        _append_comms("OUT", "request", {"message": f"[ctx {len(static_md)} static + {len(dynamic_md)} dynamic + msg {len(user_message)}]"})
+        
+        payload, all_text = full_user_msg, []
+        
+        for r_idx in range(MAX_TOOL_ROUNDS + 2):
             resp = _gemini_chat.send_message(payload)
             txt = "\n".join(p.text for c in resp.candidates if getattr(c, "content", None) for p in c.content.parts if hasattr(p, "text") and p.text)
             if txt: all_text.append(txt)
@@ -539,11 +521,29 @@ def _send_gemini(md_content: str, user_message: str, base_dir: str, file_items:
             calls = [p.function_call for c in resp.candidates if getattr(c, "content", None) for p in c.content.parts if hasattr(p, "function_call") and p.function_call]
             usage = {"input_tokens": getattr(resp.usage_metadata, "prompt_token_count", 0), "output_tokens": getattr(resp.usage_metadata, "candidates_token_count", 0)}
             cached_tokens = getattr(resp.usage_metadata, "cached_content_token_count", None)
-            if cached_tokens:
+            if cached_tokens: usage["cache_read_input_tokens"] = cached_tokens
-                usage["cache_read_input_tokens"] = cached_tokens
             reason = resp.candidates[0].finish_reason.name if resp.candidates and hasattr(resp.candidates[0], "finish_reason") else "STOP"
             
             _append_comms("IN", "response", {"round": r_idx, "stop_reason": reason, "text": txt, "tool_calls": [{"name": c.name, "args": dict(c.args)} for c in calls], "usage": usage})
+            
+            total_in = usage.get("input_tokens", 0)
+            if total_in > _GEMINI_MAX_INPUT_TOKENS and _gemini_chat and _gemini_chat.history:
+                hist = list(_gemini_chat.history)
+                dropped = 0
+                while len(hist) > 4 and total_in > _GEMINI_MAX_INPUT_TOKENS * 0.7:
+                    saved = sum(len(p.text)//4 for p in hist[0].parts if hasattr(p, "text") and p.text)
+                    for p in hist[0].parts:
+                        if hasattr(p, "function_response") and p.function_response:
+                            r = getattr(p.function_response, "response", {})
+                            val = r.get("output", "") if isinstance(r, dict) else getattr(r, "output", "")
+                            saved += len(str(val)) // 4
+                    hist.pop(0)
+                    total_in -= max(saved, 100)
+                    dropped += 1
+                if dropped > 0:
+                    _gemini_chat.history = hist
+                    _append_comms("OUT", "request", {"message": f"[GEMINI HISTORY TRIMMED: dropped {dropped} old entries to stay within token budget]"})
+
             if not calls or r_idx > MAX_TOOL_ROUNDS: break
 
             f_resps, log = [], []
@@ -560,7 +560,8 @@ def _send_gemini(md_content: str, user_message: str, base_dir: str, file_items:
 
                 if i == len(calls) - 1:
                     if file_items:
-                        ctx = _build_file_context_text(_reread_file_items(file_items))
+                        file_items = _reread_file_items(file_items)
+                        ctx = _build_file_context_text(file_items)
                         if ctx: out += f"\n\n[SYSTEM: FILES UPDATED]\n\n{ctx}"
                     if r_idx == MAX_TOOL_ROUNDS: out += "\n\n[SYSTEM: MAX ROUNDS. PROVIDE FINAL ANSWER.]"
                 
@@ -586,6 +587,10 @@ _CHARS_PER_TOKEN = 3.5
 # Anthropic's limit is 200k. We leave headroom for the response + tool schemas.
 _ANTHROPIC_MAX_PROMPT_TOKENS = 180_000
 
+# Gemini models have a 1M context window but we cap well below to leave headroom.
+# If the model reports input tokens exceeding this, we trim old history.
+_GEMINI_MAX_INPUT_TOKENS = 900_000
+
 # Marker prefix used to identify stale file-refresh injections in history
 _FILE_REFRESH_MARKER = "[FILES UPDATED"
 
@@ -628,78 +633,41 @@ def _estimate_prompt_tokens(system_blocks: list[dict], history: list[dict]) -> i
 
 
 def _strip_stale_file_refreshes(history: list[dict]):
-    """
-    Remove [FILES UPDATED ...] text blocks from all history turns EXCEPT
-    the very last user message. These are stale snapshots from previous
-    tool rounds that bloat the context without providing value.
-    """
     if len(history) < 2:
         return
-    # Find the index of the last user message — we keep its file refresh intact
+    last_user_idx = next((i for i in range(len(history)-1, -1, -1) if history[i].get("role") == "user"), -1)
-    last_user_idx = -1
-    for i in range(len(history) - 1, -1, -1):
-        if history[i].get("role") == "user":
-            last_user_idx = i
-            break
     for i, msg in enumerate(history):
         if msg.get("role") != "user" or i == last_user_idx:
             continue
         content = msg.get("content")
         if not isinstance(content, list):
             continue
-        cleaned = []
+        cleaned = [b for b in content if not (isinstance(b, dict) and b.get("type") == "text" and b.get("text", "").startswith(_FILE_REFRESH_MARKER))]
-        for block in content:
-            if isinstance(block, dict) and block.get("type") == "text":
-                text = block.get("text", "")
-                if text.startswith(_FILE_REFRESH_MARKER):
-                    continue  # drop this stale file refresh block
-            cleaned.append(block)
         if len(cleaned) < len(content):
             msg["content"] = cleaned
 
 
-def _trim_anthropic_history(system_blocks: list[dict], history: list[dict]):
+def _trim_anthropic_history(system_blocks: list[dict], history: list[dict]) -> int:
-    """
-    Trim the Anthropic history to fit within the token budget.
-    Strategy:
-      1. Strip stale file-refresh injections from old turns.
-      2. If still over budget, drop oldest turn pairs (user + assistant).
-    Returns the number of messages dropped.
-    """
-    # Phase 1: strip stale file refreshes
     _strip_stale_file_refreshes(history)
-
     est = _estimate_prompt_tokens(system_blocks, history)
     if est <= _ANTHROPIC_MAX_PROMPT_TOKENS:
         return 0
-
-    # Phase 2: drop oldest turn pairs until within budget
     dropped = 0
     while len(history) > 3 and est > _ANTHROPIC_MAX_PROMPT_TOKENS:
-        # Protect history[0] (original user prompt). Drop from history[1] (assistant) and history[2] (user)
         if history[1].get("role") == "assistant" and len(history) > 2 and history[2].get("role") == "user":
-            removed_asst = history.pop(1)
+            est -= _estimate_message_tokens(history.pop(1))
-            removed_user = history.pop(1)
+            est -= _estimate_message_tokens(history.pop(1))
             dropped += 2
-            est -= _estimate_message_tokens(removed_asst)
-            est -= _estimate_message_tokens(removed_user)
-            # Also drop dangling tool_results if the next message is an assistant and the removed user was just tool results
             while len(history) > 2 and history[1].get("role") == "assistant" and history[2].get("role") == "user":
-                content = history[2].get("content", [])
+                c = history[2].get("content", [])
-                if isinstance(content, list) and content and isinstance(content[0], dict) and content[0].get("type") == "tool_result":
+                if isinstance(c, list) and c and isinstance(c[0], dict) and c[0].get("type") == "tool_result":
-                    r_a = history.pop(1)
+                    est -= _estimate_message_tokens(history.pop(1))
-                    r_u = history.pop(1)
+                    est -= _estimate_message_tokens(history.pop(1))
                     dropped += 2
-                    est -= _estimate_message_tokens(r_a)
+                else: break
-                    est -= _estimate_message_tokens(r_u)
         else:
-                    break
+            est -= _estimate_message_tokens(history.pop(1))
-        else:
-            # Edge case fallback: drop index 1 (protecting index 0)
-            removed = history.pop(1)
             dropped += 1
-            est -= _estimate_message_tokens(removed)
-
     return dropped
 
 
@@ -779,17 +747,19 @@ def _repair_anthropic_history(history: list[dict]):
     })
 
 
-def _send_anthropic(md_content: str, user_message: str, base_dir: str, file_items: list[dict] | None = None) -> str:
+def _send_anthropic(static_md: str, dynamic_md: str, user_message: str, base_dir: str, file_items: list[dict] | None = None) -> str:
     try:
         _ensure_anthropic_client()
         mcp_client.configure(file_items or [], [base_dir])
 
-        system_text = _get_combined_system_prompt() + f"\n\n<context>\n{md_content}\n</context>"
+        system_text = _get_combined_system_prompt() + f"\n\n<context>\n{static_md}\n</context>"
         system_blocks = _build_chunked_context_blocks(system_text)
         
+        if dynamic_md:
+            system_blocks.append({"type": "text", "text": f"<discussion>\n{dynamic_md}\n</discussion>"})
+
         user_content = [{"type": "text", "text": user_message}]
 
-        # COMPRESS HISTORY: Truncate massive tool outputs from previous turns
         for msg in _anthropic_history:
             if msg.get("role") == "user" and isinstance(msg.get("content"), list):
                 for block in msg["content"]:
@@ -800,181 +770,96 @@ def _send_anthropic(md_content: str, user_message: str, base_dir: str, file_item
 
         _strip_cache_controls(_anthropic_history)
         _repair_anthropic_history(_anthropic_history)
+        
+        user_content[-1]["cache_control"] = {"type": "ephemeral"}
         _anthropic_history.append({"role": "user", "content": user_content})
 
         n_chunks = len(system_blocks)
         _append_comms("OUT", "request", {
-            "message": (
+            "message": (f"[system {n_chunks} chunk(s), {len(static_md)} static + {len(dynamic_md)} dynamic chars context] "
-                f"[system {n_chunks} chunk(s), {len(md_content)} chars context] "
+                        f"{user_message[:200]}{'...' if len(user_message) > 200 else ''}"),
-                f"{user_message[:200]}{'...' if len(user_message) > 200 else ''}"
-            ),
         })
 
         all_text_parts = []
 
-        # We allow MAX_TOOL_ROUNDS, plus 1 final loop to get the text synthesis
         for round_idx in range(MAX_TOOL_ROUNDS + 2):
-            # Trim history to fit within token budget before each API call
             dropped = _trim_anthropic_history(system_blocks, _anthropic_history)
             if dropped > 0:
                 est_tokens = _estimate_prompt_tokens(system_blocks, _anthropic_history)
-                _append_comms("OUT", "request", {
+                _append_comms("OUT", "request", {"message": f"[HISTORY TRIMMED: dropped {dropped} old messages to fit token budget. Estimated {est_tokens} tokens remaining.]"})
-                    "message": (
-                        f"[HISTORY TRIMMED: dropped {dropped} old messages to fit token budget. "
-                        f"Estimated {est_tokens} tokens remaining. {len(_anthropic_history)} messages in history.]"
-                    ),
-                })
 
             response = _anthropic_client.messages.create(
-                model=_model,
+                model=_model, max_tokens=_max_tokens, temperature=_temperature,
-                max_tokens=_max_tokens,
+                system=system_blocks, tools=_get_anthropic_tools(), messages=_anthropic_history,
-                temperature=_temperature,
-                system=system_blocks,
-                tools=_build_anthropic_tools(),
-                messages=_anthropic_history,
             )
 
-            # Convert SDK content block objects to plain dicts before storing in history
             serialised_content = [_content_block_to_dict(b) for b in response.content]
-
+            _anthropic_history.append({"role": "assistant", "content": serialised_content})
-            _anthropic_history.append({
-                "role": "assistant",
-                "content": serialised_content,
-            })
 
             text_blocks = [b.text for b in response.content if hasattr(b, "text") and b.text]
-            if text_blocks:
+            if text_blocks: all_text_parts.append("\n".join(text_blocks))
-                all_text_parts.append("\n".join(text_blocks))
 
-            tool_use_blocks = [
+            tool_use_blocks = [{"id": b.id, "name": b.name, "input": b.input} for b in response.content if getattr(b, "type", None) == "tool_use"]
-                {"id": b.id, "name": b.name, "input": b.input}
-                for b in response.content
-                if getattr(b, "type", None) == "tool_use"
-            ]
 
-            usage_dict: dict = {}
+            usage_dict = {}
             if response.usage:
-                usage_dict["input_tokens"]  = response.usage.input_tokens
+                usage_dict.update({"input_tokens": response.usage.input_tokens, "output_tokens": response.usage.output_tokens})
-                usage_dict["output_tokens"] = response.usage.output_tokens
+                if getattr(response.usage, "cache_creation_input_tokens", None) is not None:
-                cache_creation = getattr(response.usage, "cache_creation_input_tokens", None)
+                    usage_dict["cache_creation_input_tokens"] = response.usage.cache_creation_input_tokens
-                cache_read     = getattr(response.usage, "cache_read_input_tokens",     None)
+                if getattr(response.usage, "cache_read_input_tokens", None) is not None:
-                if cache_creation is not None:
+                    usage_dict["cache_read_input_tokens"] = response.usage.cache_read_input_tokens
-                    usage_dict["cache_creation_input_tokens"] = cache_creation
-                if cache_read is not None:
-                    usage_dict["cache_read_input_tokens"] = cache_read
 
-            _append_comms("IN", "response", {
+            _append_comms("IN", "response", {"round": round_idx, "stop_reason": response.stop_reason, "text": "\n".join(text_blocks), "tool_calls": tool_use_blocks, "usage": usage_dict})
-                "round":       round_idx,
-                "stop_reason": response.stop_reason,
-                "text":        "\n".join(text_blocks),
-                "tool_calls":  tool_use_blocks,
-                "usage":       usage_dict,
-            })
 
-            if response.stop_reason != "tool_use" or not tool_use_blocks:
+            if response.stop_reason != "tool_use" or not tool_use_blocks: break
-                break
+            if round_idx > MAX_TOOL_ROUNDS: break
-
-            if round_idx > MAX_TOOL_ROUNDS:
-                # The model ignored the MAX ROUNDS warning and kept calling tools.
-                # Force abort to prevent infinite loop.
-                break
 
             tool_results = []
             for block in response.content:
-                if getattr(block, "type", None) != "tool_use":
+                if getattr(block, "type", None) != "tool_use": continue
-                    continue
+                b_name, b_id, b_input = getattr(block, "name", None), getattr(block, "id", ""), getattr(block, "input", {})
-                b_name = getattr(block, "name", None)
-                b_id   = getattr(block, "id",   "")
-                b_input = getattr(block, "input", {})
                 if b_name in mcp_client.TOOL_NAMES:
                     _append_comms("OUT", "tool_call", {"name": b_name, "id": b_id, "args": b_input})
-                    output = mcp_client.dispatch(b_name, b_input)
+                    out = mcp_client.dispatch(b_name, b_input)
-                    _append_comms("IN", "tool_result", {"name": b_name, "id": b_id, "output": output})
-                    tool_results.append({
-                        "type":        "tool_result",
-                        "tool_use_id": b_id,
-                        "content":     output,
-                    })
                 elif b_name == TOOL_NAME:
-                    script = b_input.get("script", "")
+                    scr = b_input.get("script", "")
-                    _append_comms("OUT", "tool_call", {
+                    _append_comms("OUT", "tool_call", {"name": TOOL_NAME, "id": b_id, "script": scr})
-                        "name":   TOOL_NAME,
+                    out = _run_script(scr, base_dir)
-                        "id":     b_id,
+                else: out = f"ERROR: unknown tool '{b_name}'"
-                        "script": script,
+                
-                    })
+                _append_comms("IN", "tool_result", {"name": b_name, "id": b_id, "output": out})
-                    output = _run_script(script, base_dir)
+                tool_results.append({"type": "tool_result", "tool_use_id": b_id, "content": out})
-                    _append_comms("IN", "tool_result", {
-                        "name":   TOOL_NAME,
-                        "id":     b_id,
-                        "output": output,
-                    })
-                    tool_results.append({
-                        "type":        "tool_result",
-                        "tool_use_id": b_id,
-                        "content":     output,
-                    })
 
-            # Refresh file context after tool calls and inject into tool result message
             if file_items:
                 file_items = _reread_file_items(file_items)
                 refreshed_ctx = _build_file_context_text(file_items)
                 if refreshed_ctx:
-                    tool_results.append({
+                    tool_results.append({"type": "text", "text": f"[{_FILE_REFRESH_MARKER} — current contents below. Do NOT re-read these files with PowerShell.]\n\n{refreshed_ctx}"})
-                        "type": "text",
-                        "text": (
-                            "[FILES UPDATED — current contents below. "
-                            "Do NOT re-read these files with PowerShell.]\n\n"
-                            + refreshed_ctx
-                        ),
-                    })
 
             if round_idx == MAX_TOOL_ROUNDS:
-                tool_results.append({
+                tool_results.append({"type": "text", "text": "SYSTEM WARNING: MAX TOOL ROUNDS REACHED. YOU MUST PROVIDE YOUR FINAL ANSWER NOW WITHOUT CALLING ANY MORE TOOLS."})
-                    "type": "text",
-                    "text": "SYSTEM WARNING: MAX TOOL ROUNDS REACHED. YOU MUST PROVIDE YOUR FINAL ANSWER NOW WITHOUT CALLING ANY MORE TOOLS."
-                })
 
-            _anthropic_history.append({
+            _anthropic_history.append({"role": "user", "content": tool_results})
-                "role":    "user",
+            _append_comms("OUT", "tool_result_send", {"results": [{"tool_use_id": r["tool_use_id"], "content": r["content"]} for r in tool_results if r.get("type") == "tool_result"]})
-                "content": tool_results,
-            })
-
-            _append_comms("OUT", "tool_result_send", {
-                "results": [
-                    {"tool_use_id": r["tool_use_id"], "content": r["content"]}
-                    for r in tool_results if r.get("type") == "tool_result"
-                ],
-            })
 
         final_text = "\n\n".join(all_text_parts)
         return final_text if final_text.strip() else "(No text returned by the model)"
-
+    except ProviderError: raise
-    except ProviderError:
+    except Exception as exc: raise _classify_anthropic_error(exc) from exc
-        raise
-    except Exception as exc:
-        raise _classify_anthropic_error(exc) from exc
 
 
 # ------------------------------------------------------------------ unified send
 
 def send(
-    md_content: str,
+    static_md: str,
+    dynamic_md: str,
     user_message: str,
     base_dir: str = ".",
     file_items: list[dict] | None = None,
 ) -> str:
-    """
+    """Send a message to the active provider."""
-    Send a message to the active provider.
-
-    md_content  : aggregated markdown string from aggregate.run()
-    user_message: the user question / instruction
-    base_dir    : project base directory (for PowerShell tool calls)
-    file_items  : list of file dicts from aggregate.build_file_items() for
-                  dynamic context refresh after tool calls
-    """
     if _provider == "gemini":
-        return _send_gemini(md_content, user_message, base_dir, file_items)
+        return _send_gemini(static_md, dynamic_md, user_message, base_dir, file_items)
     elif _provider == "anthropic":
-        return _send_anthropic(md_content, user_message, base_dir, file_items)
+        return _send_anthropic(static_md, dynamic_md, user_message, base_dir, file_items)
     raise ValueError(f"unknown provider: {_provider}")
-

config.toml

@@ -1,6 +1,6 @@
 [ai]
-provider = "gemini"
+provider = "anthropic"
-model = "gemini-3.1-pro-preview"
+model = "claude-sonnet-4-6"
 temperature = 0.6000000238418579
 max_tokens = 12000
 history_trunc_limit = 8000
@@ -17,4 +17,4 @@ paths = [
     "manual_slop.toml",
     "C:/projects/forth/bootslop/bootslop.toml",
 ]
-active = "manual_slop.toml"
+active = "C:/projects/forth/bootslop/bootslop.toml"

docs/Readme.md

@@ -8,6 +8,8 @@ A GUI orchestrator for local LLM-driven coding sessions, built to prevent the AI
 
 The heart of context management. 
 
+> **Note:** The Config panel has been removed. Output directory and auto-add history settings are now integrated into the Projects and Discussion History panels respectively.
+
 - **Configuration:** You specify the Git Directory (for commit tracking) and a Main Context File (the markdown file containing your project's notes and schema).
 - **Word-Wrap Toggle:** Dynamically swaps text rendering in large read-only panels (Responses, Comms Log) between unwrapped (ideal for viewing precise code formatting) and wrapped (ideal for prose).
 - **Project Switching:** Switch between different <project>.toml profiles to instantly swap out your entire active file list, discussion history, and settings.

docs/guide_architecture.md

@@ -44,14 +44,15 @@ The communication model is unified under ai_client.py, which normalizes the Gemi
 
 The loop is defined as follows:
 
-1. **Prompt Injection:** The aggregated Markdown context and system prompt are injected. (Gemini injects this directly into system_instruction at chat instantiation to prevent history bloat; Anthropic chunks this into cache_control: ephemeral blocks).
+1. **Prompt Injection:** The aggregated Markdown context and system prompt are injected. For Gemini, the system_instruction and tools are stored in an explicit cache via `client.caches.create()` with a 1-hour TTL; if cache creation fails (under minimum token threshold), it falls back to inline system_instruction. When context changes mid-session, the old cache is deleted and a new one is created. For Anthropic, the system prompt + context are sent as `system=` blocks with `cache_control: ephemeral` on the last chunk, and tools carry `cache_control: ephemeral` on the last tool definition.
-2. **Execution Loop:** A MAX_TOOL_ROUNDS (default 10) bounded loop begins.
+2. **Execution Loop:** A MAX_TOOL_ROUNDS (default 10) bounded loop begins. The tools list for Anthropic is built once per session and reused.
 3. The AI provider is polled.
-4. If the provider's stop_reason is 	ool_use:
+4. If the provider's stop_reason is tool_use:
    1. The loop parses the requested tool (either a read-only MCP tool or the destructive PowerShell tool).
    2. If PowerShell, it dispatches a blocking event to the Main Thread (see *On Tool Execution & Concurrency*).
-   3. Once the result is retrieved, the loop executes a **Dynamic Refresh** (_reread_file_items). Any files currently tracked by the project are pulled from the disk fresh.
+   3. Once the last tool result in the batch is retrieved, the loop executes a **Dynamic Refresh** (`_reread_file_items`). Any files currently tracked by the project are pulled from disk fresh. The `file_items` variable is reassigned so subsequent tool rounds see the updated content.
-   4. The tool result, appended with the fresh [FILES UPDATED] block, is sent back to the provider.
+   4. For Anthropic: the refreshed file contents are appended as a text block to the tool_results user message. For Gemini: the refreshed contents are appended to the last function response's output string. In both cases, the block is prefixed with `[FILES UPDATED]` / `[SYSTEM: FILES UPDATED]`.
+   5. On subsequent rounds, stale file-refresh blocks from previous turns are stripped from history to prevent token accumulation. For Gemini, old tool outputs exceeding `_history_trunc_limit` characters are also truncated.
 5. Once the model outputs standard text, the loop terminates and yields the string back to the GUI callback.
 
 ### On Tool Execution & Concurrency

docs/guide_tools.md

@@ -46,5 +46,8 @@ The core system prompt explicitly guides the AI on how to use this tool safely:
 
 ### Synthetic Context Refresh
 
-Immediately after **any** tool call turn finishes, ai_client runs _reread_file_items. It fetches the latest disk state of all files in the current project context and appends them as a synthetic [FILES UPDATED] message to the tool result. 
+After the **last** tool call in each round finishes (when multiple tools are called in a single round, the refresh happens once after all of them), ai_client runs `_reread_file_items`. It fetches the latest disk state of all files in the current project context. The `file_items` variable is reassigned so subsequent tool rounds within the same request use the fresh content.
-This means if the AI writes to a file, it instantly "sees" the modification in its next turn without having to waste a cycle calling read_file.
+
+For Anthropic, the refreshed contents are injected as a text block in the `tool_results` user message. For Gemini, they are appended to the last function response's output string. In both cases, the block is prefixed with `[FILES UPDATED]` / `[SYSTEM: FILES UPDATED]`.
+
+On the next tool round, stale file-refresh blocks from previous rounds are stripped from history to prevent token accumulation. This means if the AI writes to a file, it instantly "sees" the modification in its next turn without having to waste a cycle calling `read_file`, and the cost of carrying the full file snapshot is limited to one round.

gui.py

@@ -121,10 +121,19 @@ def _add_kv_row(parent: str, key: str, val, val_color=None):
 
 
 def _render_usage(parent: str, usage: dict):
-    """Render Anthropic usage dict as a compact token table."""
+    """Render Anthropic usage dict as a compact token table, with true totals."""
     if not usage:
         return
     dpg.add_text("usage:", color=_SUBHDR_COLOR, parent=parent)
+    
+    cache_read = usage.get("cache_read_input_tokens", 0)
+    cache_create = usage.get("cache_creation_input_tokens", 0)
+    raw_input = usage.get("input_tokens", 0)
+    total_in = cache_read + cache_create + raw_input
+    
+    if total_in > raw_input:
+        _add_kv_row(parent, "  total_input_tokens", total_in, _NUM_COLOR)
+        
     order = [
         "input_tokens",
         "cache_read_input_tokens",
@@ -855,7 +864,7 @@ class App:
         }
         theme.save_to_config(self.config)
 
-    def _do_generate(self) -> tuple[str, Path, list]:
+    def _do_generate(self) -> tuple[str, str, Path, list]:
         self._flush_to_project()
         self._save_active_project()
         self._flush_to_config()
@@ -1110,8 +1119,9 @@ class App:
 
     def cb_md_only(self):
         try:
-            md, path, _file_items = self._do_generate()
+            s_md, d_md, path, _file_items = self._do_generate()
-            self.last_md = md
+            self.last_static_md = s_md
+            self.last_dynamic_md = d_md
             self.last_md_path = path
             self._update_status(f"md written: {path.name}")
         except Exception as e:
@@ -1134,8 +1144,9 @@ class App:
         if self.send_thread and self.send_thread.is_alive():
             return
         try:
-            md, path, file_items = self._do_generate()
+            s_md, d_md, path, file_items = self._do_generate()
-            self.last_md = md
+            self.last_static_md = s_md
+            self.last_dynamic_md = d_md
             self.last_md_path = path
             self.last_file_items = file_items
         except Exception as e:
@@ -1152,6 +1163,7 @@ class App:
         if global_sp: combined_sp.append(global_sp.strip())
         if project_sp: combined_sp.append(project_sp.strip())
         ai_client.set_custom_system_prompt("\n\n".join(combined_sp))
+        
         temp = dpg.get_value("ai_temperature") if dpg.does_item_exist("ai_temperature") else 0.0
         max_tok = dpg.get_value("ai_max_tokens") if dpg.does_item_exist("ai_max_tokens") else 8192
         trunc = dpg.get_value("ai_history_trunc") if dpg.does_item_exist("ai_history_trunc") else 8000
@@ -1162,7 +1174,7 @@ class App:
             if auto_add:
                 self._queue_history_add("User", user_msg)
             try:
-                response = ai_client.send(self.last_md, user_msg, base_dir, self.last_file_items)
+                response = ai_client.send(getattr(self, "last_static_md", ""), getattr(self, "last_dynamic_md", ""), user_msg, base_dir, self.last_file_items)
                 self._update_response(response)
                 self._update_status("done")
                 self._trigger_blink = True

manual_slop.toml

@@ -1,7 +1,7 @@
 [project]
 name = "manual_slop"
 git_dir = "C:/projects/manual_slop"
-system_prompt = "Make sure to update MainContext.md every time.\nMake destructive modifications to the project, ITS OK, I HAVE GIT HISTORY TO MANAGE THE PROJECTS."
+system_prompt = "Make sure to update MainContext.md every time.\nMake destructive modifications to the project, ITS OK, I HAVE GIT HISTORY TO MANAGE THE PROJECTS.\nAvoid reading manual_slop.toml its expensive as it has the history of multiple dicussions.\n"
 main_context = "C:/projects/manual_slop/MainContext.md"
 word_wrap = true
 
@@ -147,7 +147,7 @@ history = [
 
 [discussion.discussions."docs writeup"]
 git_commit = "bf2d09f3fd817d64fbf6b4aa667e2b635b6fbc0e"
-last_updated = "2026-02-22T10:34:24"
+last_updated = "2026-02-22T11:08:58"
 history = [
     "@2026-02-22T08:56:39\nUser:\nLets write extensive documentation in the same style that I used for my VEFontCache-Oodin project.\nI added it's directories to your context.",
     "@2026-02-22T08:56:58\nAI:\n(No text returned)",

project_manager.py

@@ -154,4 +154,3 @@ def flat_config(proj: dict, disc_name: str | None = None) -> dict:
             "history": disc_data.get("history", []),
         },
     }
-

session_logger.py

@@ -133,5 +133,3 @@ def log_tool_call(script: str, result: str, script_path: str | None):
         pass
 
     return str(ps1_path) if ps1_path else None
-
-