Private
Public Access
docs: scrub gemini_cli references from 12 docs (provider count 8->7)
Cleaned: docs/guide_ai_client.md, docs/guide_architecture.md, docs/guide_models.md, docs/guide_simulations.md, docs/guide_context_aggregation.md, docs/guide_tools.md, docs/Readme.md, conductor/tech-stack.md, conductor/product.md, conductor/product-guidelines.md, conductor/workflow.md, conductor/code_styleguides/error_handling.md. Provider list citations updated to 7 (gemini, anthropic, deepseek, minimax, qwen, grok, llama). guide_meta_boundary.md intentionally retained (its gemini_cli references are the meta-tooling GEMINI_CLI_HOOK_CONTEXT env var, NOT the provider; per spec GAP-A12).
This commit is contained in:
+182
-182
@@ -6,14 +6,14 @@
|
||||
|
||||
## Overview
|
||||
|
||||
`src/ai_client.py` (~166KB) is the **unified LLM client** for 8 providers. It abstracts the differences between providers (Gemini, Anthropic, DeepSeek, MiniMax, Gemini CLI, Qwen, Grok, Llama) behind a single `send()` function.
|
||||
`src/ai_client.py` (~166KB) is the **unified LLM client** for 7 providers. It abstracts the differences between providers (Gemini, Anthropic, DeepSeek, MiniMax, Qwen, Grok, Llama) behind a single `send()` function.
|
||||
|
||||
The module is a **stateful singleton** — all provider state is held in module-level globals. There is no class wrapping; the module itself is the abstraction layer.
|
||||
|
||||
The 8 providers split into 3 API shapes:
|
||||
The 7 providers split into 3 API shapes:
|
||||
- **Native SDK**: Gemini (google-genai), Anthropic (anthropic), Qwen (DashScope)
|
||||
- **OpenAI-compatible**: MiniMax, Grok, Llama (Ollama/OpenRouter/custom), DeepSeek
|
||||
- **Subprocess**: Gemini CLI
|
||||
- **Subprocess**:
|
||||
|
||||
The OpenAI-compatible vendors all call the shared helper in `src/openai_compatible.py` (added 2026-06-06 by the `qwen_llama_grok_integration_20260606` track; see "Shared OpenAI-Compatible Helper" section below). The MiniMax provider's `_send_minimax` was refactored to use this helper (Phase 4 of the same track, 231 → 75 lines, 68% reduction).
|
||||
|
||||
@@ -21,7 +21,7 @@ The OpenAI-compatible vendors all call the shared helper in `src/openai_compatib
|
||||
|
||||
## Module-Level Imports
|
||||
|
||||
> **Important:** The provider SDKs are **NOT** imported at module level. `import google.genai`, `import anthropic`, `import openai`, `import dashscope`, and `import fastapi` are heavy (~430-955ms each on cold load) and are now obtained via `src.module_loader._require_warmed("google.genai")` and similar calls, after the `WarmupManager` has loaded them in the background. The module-level globals you see in the State section (`_gemini_client`, `_anthropic_client`, etc.) are typed as `Optional` because they're populated by `_require_warmed()` on first use, not at import time. (Updated 2026-07-02: there are 8 providers, not 5 — the original "5 SDKs" count predated the qwen/grok/llama additions.)
|
||||
> **Important:** The provider SDKs are **NOT** imported at module level. `import google.genai`, `import anthropic`, `import openai`, `import dashscope`, and `import fastapi` are heavy (~430-955ms each on cold load) and are now obtained via `src.module_loader._require_warmed("google.genai")` and similar calls, after the `WarmupManager` has loaded them in the background. The module-level globals you see in the State section (`_gemini_client`, `_anthropic_client`, etc.) are typed as `Optional` because they're populated by `_require_warmed()` on first use, not at import time. (Updated 2026-07-02: there are 7 providers, not 5 — the original "5 SDKs" count predated the qwen/grok/llama additions.)
|
||||
|
||||
This change was part of the 2026-06-06 `startup_speedup_20260606` track. Before: `import src.ai_client` took ~1800ms. After: ~161ms. The remaining cost is the bare module skeleton.
|
||||
|
||||
@@ -29,19 +29,19 @@ This change was part of the 2026-06-06 `startup_speedup_20260606` track. Before:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ ai_client.send(md_content, user_message, ...) │
|
||||
│ │
|
||||
│ 1. _send_lock.acquire() — serialize all calls │
|
||||
│ 2. Read _provider / _model │
|
||||
│ ai_client.send(md_content, user_message, ...) │
|
||||
│ │
|
||||
│ 1. _send_lock.acquire() — serialize all calls │
|
||||
│ 2. Read _provider / _model │
|
||||
│ 3. Route to provider-specific _send_<provider>() │
|
||||
│ 4. Return str response │
|
||||
│ 4. Return str response │
|
||||
└─────────────────┬───────────────────────────────┘
|
||||
│ dispatches based on _provider
|
||||
▼
|
||||
┌────────┬─────────┬────────┬──────────┐
|
||||
▼ ▼ ▼ ▼ ▼
|
||||
_gemini _anthropic _deepseek _minimax _gemini_cli
|
||||
(subprocess)
|
||||
│ dispatches based on _provider
|
||||
▼
|
||||
┌────────┬─────────┬────────┬──────────┐
|
||||
▼ ▼ ▼ ▼ ▼
|
||||
_gemini _anthropic _deepseek _minimax _gemini_cli
|
||||
(subprocess)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -94,18 +94,18 @@ _gemini_cli_adapter: Optional[GeminiCliAdapter] = None
|
||||
|
||||
```python
|
||||
def send(
|
||||
md_content: str,
|
||||
user_message: str,
|
||||
base_dir: str = ".",
|
||||
file_items: list[dict] | None = None,
|
||||
discussion_history: str = "",
|
||||
stream: bool = False,
|
||||
pre_tool_callback: Optional[Callable] = None,
|
||||
qa_callback: Optional[Callable] = None,
|
||||
enable_tools: bool = True,
|
||||
stream_callback: Optional[Callable] = None,
|
||||
patch_callback: Optional[Callable] = None,
|
||||
rag_engine: Optional[Any] = None,
|
||||
md_content: str,
|
||||
user_message: str,
|
||||
base_dir: str = ".",
|
||||
file_items: list[dict] | None = None,
|
||||
discussion_history: str = "",
|
||||
stream: bool = False,
|
||||
pre_tool_callback: Optional[Callable] = None,
|
||||
qa_callback: Optional[Callable] = None,
|
||||
enable_tools: bool = True,
|
||||
stream_callback: Optional[Callable] = None,
|
||||
patch_callback: Optional[Callable] = None,
|
||||
rag_engine: Optional[Any] = None,
|
||||
) -> Result[str]:
|
||||
```
|
||||
|
||||
@@ -147,7 +147,7 @@ ai_client.set_model_params(temp=0.7, max_tok=4096, top_p=0.9, trunc_limit=4000)
|
||||
### Session Management
|
||||
|
||||
```python
|
||||
ai_client.reset_session() # Clears all provider state, history, cache
|
||||
ai_client.reset_session() # Clears all provider state, history, cache
|
||||
```
|
||||
|
||||
### Event Hooks
|
||||
@@ -171,10 +171,10 @@ ai_client.events.on("my_event", my_handler)
|
||||
### Comms Log
|
||||
|
||||
```python
|
||||
ai_client._append_comms(direction, kind, payload) # Add entry
|
||||
ai_client.get_comms_log() # Read all
|
||||
ai_client.clear_comms_log() # Clear
|
||||
ai_client.get_token_stats(md_content) # Estimate token usage
|
||||
ai_client._append_comms(direction, kind, payload) # Add entry
|
||||
ai_client.get_comms_log() # Read all
|
||||
ai_client.clear_comms_log() # Clear
|
||||
ai_client.get_token_stats(md_content) # Estimate token usage
|
||||
```
|
||||
|
||||
### Provider Error Taxonomy — Legacy (Pre-Refactor)
|
||||
@@ -187,12 +187,12 @@ ai_client.get_token_stats(md_content) # Estimate token usage
|
||||
|
||||
```python
|
||||
class ProviderError(Exception):
|
||||
kind: str # "quota" | "rate_limit" | "auth" | "balance" | "network" | "unknown"
|
||||
provider: str
|
||||
original: Exception
|
||||
kind: str # "quota" | "rate_limit" | "auth" | "balance" | "network" | "unknown"
|
||||
provider: str
|
||||
original: Exception
|
||||
|
||||
def ui_message(self) -> str:
|
||||
"""Returns a user-friendly error message."""
|
||||
def ui_message(self) -> str:
|
||||
"""Returns a user-friendly error message."""
|
||||
```
|
||||
|
||||
`ProviderError` was raised by provider-specific `_send_*` functions on failure.
|
||||
@@ -210,30 +210,30 @@ All providers follow the same high-level pattern in `_send_*`:
|
||||
|
||||
```python
|
||||
def _send_<provider>(md_content, user_message, ...):
|
||||
for round in range(MAX_TOOL_ROUNDS + 2): # up to 10 rounds
|
||||
response = provider_api_call(md_content, user_message, history, tools)
|
||||
comms_log(direction="IN", kind="response", payload=response)
|
||||
for round in range(MAX_TOOL_ROUNDS + 2): # up to 10 rounds
|
||||
response = provider_api_call(md_content, user_message, history, tools)
|
||||
comms_log(direction="IN", kind="response", payload=response)
|
||||
|
||||
if not has_function_calls(response):
|
||||
return extract_text(response)
|
||||
if not has_function_calls(response):
|
||||
return extract_text(response)
|
||||
|
||||
for call in response.function_calls:
|
||||
if pre_tool_callback and pre_tool_callback(...) is rejected:
|
||||
return rejection_message
|
||||
tool_result = dispatch(call.name, call.args, base_dir)
|
||||
append_tool_result_to_history(call, tool_result)
|
||||
for call in response.function_calls:
|
||||
if pre_tool_callback and pre_tool_callback(...) is rejected:
|
||||
return rejection_message
|
||||
tool_result = dispatch(call.name, call.args, base_dir)
|
||||
append_tool_result_to_history(call, tool_result)
|
||||
|
||||
# Context refresh: re-read all tracked files (mtime check)
|
||||
_reread_file_items(file_items)
|
||||
# Context refresh: re-read all tracked files (mtime check)
|
||||
_reread_file_items(file_items)
|
||||
|
||||
# Truncate tool outputs at _history_trunc_limit
|
||||
truncate_tool_outputs(history)
|
||||
# Truncate tool outputs at _history_trunc_limit
|
||||
truncate_tool_outputs(history)
|
||||
|
||||
# Cumulative byte check
|
||||
if cumulative_tool_bytes > 500_000:
|
||||
inject_warning()
|
||||
# Cumulative byte check
|
||||
if cumulative_tool_bytes > 500_000:
|
||||
inject_warning()
|
||||
|
||||
return final_response
|
||||
return final_response
|
||||
```
|
||||
|
||||
The constants:
|
||||
@@ -273,7 +273,7 @@ The constants:
|
||||
- **History trimming**: similar to Anthropic (drop turn pairs at threshold)
|
||||
- **History repair**: `_repair_minimax_history`
|
||||
|
||||
### Gemini CLI
|
||||
###
|
||||
|
||||
- **Subprocess adapter**: `GeminiCliAdapter` in `src/gemini_cli_adapter.py`
|
||||
- **Persistent session**: CLI maintains its own session ID
|
||||
@@ -288,9 +288,9 @@ The constants:
|
||||
|
||||
```python
|
||||
if total_in > _GEMINI_MAX_INPUT_TOKENS * 0.4:
|
||||
while len(hist) > 4 and total_in > _GEMINI_MAX_INPUT_TOKENS * 0.3:
|
||||
hist.pop(0) # Assistant
|
||||
hist.pop(0) # User
|
||||
while len(hist) > 4 and total_in > _GEMINI_MAX_INPUT_TOKENS * 0.3:
|
||||
hist.pop(0) # Assistant
|
||||
hist.pop(0) # User
|
||||
```
|
||||
|
||||
### Anthropic (180K limit)
|
||||
@@ -314,8 +314,8 @@ No built-in trimming (relies on the caller to keep history short).
|
||||
### Gemini Server-Side Cache
|
||||
|
||||
```python
|
||||
_gemini_cache_md_hash: Optional[str] = None # Hash of cached content
|
||||
_gemini_cache_created_at: Optional[float] = None # Monotonic time
|
||||
_gemini_cache_md_hash: Optional[str] = None # Hash of cached content
|
||||
_gemini_cache_created_at: Optional[float] = None # Monotonic time
|
||||
```
|
||||
|
||||
The cache decision is a 3-way branch on each `_send_gemini` call:
|
||||
@@ -344,8 +344,8 @@ After the last tool call in each round, `_reread_file_items(file_items)` checks
|
||||
2. If unchanged: pass through as-is
|
||||
3. If changed: re-read content, store `old_content` for diffing, update `mtime`
|
||||
4. Changed files are diffed via `_build_file_diff_text`:
|
||||
- Files ≤ 200 lines: emit full content
|
||||
- Files > 200 lines with `old_content`: emit `difflib.unified_diff`
|
||||
- Files ≤ 200 lines: emit full content
|
||||
- Files > 200 lines with `old_content`: emit `difflib.unified_diff`
|
||||
5. Diff is appended to the last tool's output as `[SYSTEM: FILES UPDATED]\n\n{diff}`
|
||||
6. Stale `[FILES UPDATED]` blocks are stripped from older history turns by `_strip_stale_file_refreshes`
|
||||
|
||||
@@ -359,19 +359,19 @@ For Tier 4: when an error occurs, `qa_callback` may be invoked to get a Tier 4 A
|
||||
|
||||
```python
|
||||
def run_tier4_analysis(stderr: str) -> str:
|
||||
"""Stateless Tier 4 QA analysis of an error message."""
|
||||
# Uses a dedicated system prompt for error triage
|
||||
# Returns analysis text (root cause, suggested fix)
|
||||
# Does NOT modify any code — analysis only
|
||||
"""Stateless Tier 4 QA analysis of an error message."""
|
||||
# Uses a dedicated system prompt for error triage
|
||||
# Returns analysis text (root cause, suggested fix)
|
||||
# Does NOT modify any code — analysis only
|
||||
```
|
||||
|
||||
For Tier 4 patch generation:
|
||||
|
||||
```python
|
||||
def run_tier4_patch_generation(error: str, file_context: str) -> str:
|
||||
"""Generate a unified diff patch from an error and file context."""
|
||||
# Returns the patch as a string
|
||||
# The caller (typically the patch modal) presents it for human review
|
||||
"""Generate a unified diff patch from an error and file context."""
|
||||
# Returns the patch as a string
|
||||
# The caller (typically the patch modal) presents it for human review
|
||||
```
|
||||
|
||||
---
|
||||
@@ -416,10 +416,10 @@ def run_tier4_patch_generation(error: str, file_context: str) -> str:
|
||||
|
||||
```python
|
||||
def test_set_provider():
|
||||
from src import ai_client
|
||||
ai_client.set_provider("anthropic", "claude-3-5-sonnet-latest")
|
||||
assert ai_client.get_provider() == "anthropic"
|
||||
ai_client.reset_session() # Cleanup
|
||||
from src import ai_client
|
||||
ai_client.set_provider("anthropic", "claude-3-5-sonnet-latest")
|
||||
assert ai_client.get_provider() == "anthropic"
|
||||
ai_client.reset_session() # Cleanup
|
||||
```
|
||||
|
||||
### Mocked Tests
|
||||
@@ -428,12 +428,12 @@ def test_set_provider():
|
||||
from unittest.mock import patch
|
||||
|
||||
def test_send_routes_to_provider(monkeypatch):
|
||||
with patch.object(ai_client, "_send_anthropic", return_value="mocked") as m:
|
||||
ai_client.set_provider("anthropic", "claude-3-5-sonnet-latest")
|
||||
result = ai_client.send("system", "user")
|
||||
assert result == "mocked"
|
||||
m.assert_called_once()
|
||||
ai_client.reset_session()
|
||||
with patch.object(ai_client, "_send_anthropic", return_value="mocked") as m:
|
||||
ai_client.set_provider("anthropic", "claude-3-5-sonnet-latest")
|
||||
result = ai_client.send("system", "user")
|
||||
assert result == "mocked"
|
||||
m.assert_called_once()
|
||||
ai_client.reset_session()
|
||||
```
|
||||
|
||||
### Integration (real API)
|
||||
@@ -451,7 +451,7 @@ canonical reference is
|
||||
### Result-Based Returns
|
||||
|
||||
All `_send_<vendor>_result()` functions (8 vendors: Gemini, Anthropic,
|
||||
DeepSeek, MiniMax, Gemini CLI, Qwen, Llama, Grok — plus the
|
||||
DeepSeek, MiniMax, Qwen, Llama, Grok — plus the
|
||||
`_send_llama_native` Ollama adapter) return `Result[str]` with `errors: list[ErrorInfo]`. SDK
|
||||
exceptions are caught at the boundary (`src/openai_compatible.py`,
|
||||
`src/qwen_adapter.py`) and converted to `ErrorInfo` dataclasses. The
|
||||
@@ -469,10 +469,10 @@ meaning — do not overload `UNKNOWN` when a new failure mode surfaces
|
||||
### Public API
|
||||
|
||||
- **`ai_client.send(...)`** — the public API. Returns
|
||||
`Result[str]` (with `errors: list[ErrorInfo]` as a side-channel field).
|
||||
Accepts 13+ parameters including 8 callbacks.
|
||||
Internally calls `_send_<vendor>()` for the active provider (the
|
||||
vendor functions return `Result[str]` directly).
|
||||
`Result[str]` (with `errors: list[ErrorInfo]` as a side-channel field).
|
||||
Accepts 13+ parameters including 8 callbacks.
|
||||
Internally calls `_send_<vendor>()` for the active provider (the
|
||||
vendor functions return `Result[str]` directly).
|
||||
|
||||
### Example
|
||||
|
||||
@@ -482,9 +482,9 @@ from src.result_types import ErrorKind
|
||||
|
||||
r = ai_client.send("system prompt", "user message")
|
||||
if not r.ok:
|
||||
for err in r.errors:
|
||||
log.error(err.ui_message())
|
||||
# err.kind is one of ErrorKind.*; err.source is "ai_client.<vendor>"
|
||||
for err in r.errors:
|
||||
log.error(err.ui_message())
|
||||
# err.kind is one of ErrorKind.*; err.source is "ai_client.<vendor>"
|
||||
# use r.data regardless (it's the zero-initialized "" on failure)
|
||||
print(r.data)
|
||||
```
|
||||
@@ -492,10 +492,10 @@ print(r.data)
|
||||
### Migration Notes for Existing Callers
|
||||
|
||||
- All production call sites and tests now use `send()`. The
|
||||
legacy `send()` function was removed in the
|
||||
`public_api_migration_and_ui_polish_20260615` track.
|
||||
legacy `send()` function was removed in the
|
||||
`public_api_migration_and_ui_polish_20260615` track.
|
||||
- Tests that mock `ai_client._send_<vendor>` should use the
|
||||
`Result(data=...)` return value pattern.
|
||||
`Result(data=...)` return value pattern.
|
||||
|
||||
### See Also (in-doc)
|
||||
|
||||
@@ -532,34 +532,34 @@ Added 2026-06-06 by the `qwen_llama_grok_integration_20260606` track. Operates o
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class NormalizedResponse:
|
||||
text: str
|
||||
tool_calls: list[dict[str, Any]]
|
||||
usage_input_tokens: int
|
||||
usage_output_tokens: int
|
||||
usage_cache_read_tokens: int
|
||||
usage_cache_creation_tokens: int
|
||||
raw_response: Any
|
||||
text: str
|
||||
tool_calls: list[dict[str, Any]]
|
||||
usage_input_tokens: int
|
||||
usage_output_tokens: int
|
||||
usage_cache_read_tokens: int
|
||||
usage_cache_creation_tokens: int
|
||||
raw_response: Any
|
||||
|
||||
@dataclass
|
||||
class OpenAICompatibleRequest:
|
||||
messages: list[dict[str, Any]]
|
||||
model: str
|
||||
temperature: float = 0.0
|
||||
top_p: float = 1.0
|
||||
max_tokens: int = 8192
|
||||
tools: Optional[list[dict[str, Any]]] = None
|
||||
tool_choice: str = "auto"
|
||||
stream: bool = False
|
||||
stream_callback: Optional[Callable[[str], None]] = None
|
||||
messages: list[dict[str, Any]]
|
||||
model: str
|
||||
temperature: float = 0.0
|
||||
top_p: float = 1.0
|
||||
max_tokens: int = 8192
|
||||
tools: Optional[list[dict[str, Any]]] = None
|
||||
tool_choice: str = "auto"
|
||||
stream: bool = False
|
||||
stream_callback: Optional[Callable[[str], None]] = None
|
||||
```
|
||||
|
||||
### The Function
|
||||
|
||||
```python
|
||||
def send_openai_compatible(
|
||||
client: Any, # openai.OpenAI client with vendor-specific base_url + auth
|
||||
request: OpenAICompatibleRequest,
|
||||
*, capabilities: "VendorCapabilities", # from src/ai_client.py #region: Vendor Capabilities
|
||||
client: Any, # openai.OpenAI client with vendor-specific base_url + auth
|
||||
request: OpenAICompatibleRequest,
|
||||
*, capabilities: "VendorCapabilities", # from src/ai_client.py #region: Vendor Capabilities
|
||||
) -> NormalizedResponse:
|
||||
```
|
||||
|
||||
@@ -577,16 +577,16 @@ The function:
|
||||
```python
|
||||
# _send_grok, _send_llama (single-shot placeholders), _send_minimax (with restored tool loop)
|
||||
def _send_grok(md_content, user_message, base_dir, file_items=None, discussion_history="", stream=False, ...):
|
||||
client = _ensure_grok_client() # openai.OpenAI(api_key=..., base_url="https://api.x.ai/v1")
|
||||
with _grok_history_lock:
|
||||
# ... build messages, append user, system + context ...
|
||||
request = OpenAICompatibleRequest(
|
||||
messages=messages, model=_model, stream=stream,
|
||||
stream_callback=stream_callback,
|
||||
)
|
||||
caps = get_capabilities("grok", _model)
|
||||
response = send_openai_compatible(client, request, capabilities=caps)
|
||||
# ... append to history, return response.text ...
|
||||
client = _ensure_grok_client() # openai.OpenAI(api_key=..., base_url="https://api.x.ai/v1")
|
||||
with _grok_history_lock:
|
||||
# ... build messages, append user, system + context ...
|
||||
request = OpenAICompatibleRequest(
|
||||
messages=messages, model=_model, stream=stream,
|
||||
stream_callback=stream_callback,
|
||||
)
|
||||
caps = get_capabilities("grok", _model)
|
||||
response = send_openai_compatible(client, request, capabilities=caps)
|
||||
# ... append to history, return response.text ...
|
||||
```
|
||||
|
||||
### Qwen Adapter (`src/qwen_adapter.py`)
|
||||
@@ -610,28 +610,28 @@ Added 2026-06-11 by the `qwen_llama_grok_followup_20260611` track. Wraps `send_o
|
||||
|
||||
```python
|
||||
def run_with_tool_loop(
|
||||
client: Any,
|
||||
request: OpenAICompatibleRequest | Callable[[int], OpenAICompatibleRequest],
|
||||
*,
|
||||
capabilities: "VendorCapabilities",
|
||||
pre_tool_callback: Optional[Callable] = None,
|
||||
qa_callback: Optional[Callable] = None,
|
||||
stream_callback: Optional[Callable[[str], None]] = None,
|
||||
patch_callback: Optional[Callable] = None,
|
||||
base_dir: str,
|
||||
vendor_name: str,
|
||||
history_lock: Optional[threading.Lock] = None,
|
||||
history: Optional[list] = None,
|
||||
trim_func: Optional[Callable] = None,
|
||||
send_func: Optional[Callable[[int], "NormalizedResponse"]] = None,
|
||||
on_pre_dispatch: Optional[Callable] = None,
|
||||
client: Any,
|
||||
request: OpenAICompatibleRequest | Callable[[int], OpenAICompatibleRequest],
|
||||
*,
|
||||
capabilities: "VendorCapabilities",
|
||||
pre_tool_callback: Optional[Callable] = None,
|
||||
qa_callback: Optional[Callable] = None,
|
||||
stream_callback: Optional[Callable[[str], None]] = None,
|
||||
patch_callback: Optional[Callable] = None,
|
||||
base_dir: str,
|
||||
vendor_name: str,
|
||||
history_lock: Optional[threading.Lock] = None,
|
||||
history: Optional[list] = None,
|
||||
trim_func: Optional[Callable] = None,
|
||||
send_func: Optional[Callable[[int], "NormalizedResponse"]] = None,
|
||||
on_pre_dispatch: Optional[Callable] = None,
|
||||
) -> str:
|
||||
```
|
||||
|
||||
**Two extensions** were added beyond the original signature:
|
||||
|
||||
1. `request` accepts a `Callable[[int], OpenAICompatibleRequest]` (per-round history rebuild). Use this when the vendor mutates history between rounds (e.g., MiniMax's per-round append).
|
||||
2. `send_func + on_pre_dispatch` allows vendored call paths (e.g., Gemini CLI's `GeminiCliAdapter`) to share the loop + dispatch without going through `send_openai_compatible`.
|
||||
2. `send_func + on_pre_dispatch` allows vendored call paths (e.g., 's `GeminiCliAdapter`) to share the loop + dispatch without going through `send_openai_compatible`.
|
||||
|
||||
**Vendors applied** (as of 2026-06-11):
|
||||
- `_send_minimax` (was inline, now uses helper)
|
||||
@@ -657,7 +657,7 @@ Added 2026-06-11. When `_llama_base_url` is `localhost` / `127.0.0.1` (Ollama de
|
||||
The dispatcher check is in `_send_llama` at the function head:
|
||||
```python
|
||||
if "localhost" in _llama_base_url or "127.0.0.1" in _llama_base_url:
|
||||
return _send_llama_native(...)
|
||||
return _send_llama_native(...)
|
||||
```
|
||||
|
||||
For OpenRouter, custom URLs, and other cloud Llama endpoints, the existing OpenAI-compat path is unchanged.
|
||||
@@ -714,11 +714,11 @@ The test in `tests/test_aggregate_caching.py` ensures the first N characters of
|
||||
|
||||
```python
|
||||
def test_aggregate_stable_to_volatile_ordering():
|
||||
ctrl = mock_app_controller()
|
||||
turn1 = aggregate.build_initial_context(ctrl, user_message="first")
|
||||
turn2 = aggregate.build_initial_context(ctrl, user_message="second")
|
||||
N = aggregate.stable_prefix_length(ctrl)
|
||||
assert turn1[:N] == turn2[:N], f"Stable prefix mismatch: {turn1[:N]!r} != {turn2[:N]!r}"
|
||||
ctrl = mock_app_controller()
|
||||
turn1 = aggregate.build_initial_context(ctrl, user_message="first")
|
||||
turn2 = aggregate.build_initial_context(ctrl, user_message="second")
|
||||
N = aggregate.stable_prefix_length(ctrl)
|
||||
assert turn1[:N] == turn2[:N], f"Stable prefix mismatch: {turn1[:N]!r} != {turn2[:N]!r}"
|
||||
```
|
||||
|
||||
**The test is the contract.** If a new layer is added in the wrong position, the test fails; the agent must move the layer to the stable position or update the test with written justification.
|
||||
@@ -729,17 +729,17 @@ def test_aggregate_stable_to_volatile_ordering():
|
||||
|
||||
```python
|
||||
def _send_anthropic(messages, *, cache_prefix_chars=None):
|
||||
if cache_prefix_chars is not None:
|
||||
content_blocks = cache_prefix_blocks(messages, cache_prefix_chars)
|
||||
else:
|
||||
content_blocks = messages
|
||||
if cache_prefix_chars is not None:
|
||||
content_blocks = cache_prefix_blocks(messages, cache_prefix_chars)
|
||||
else:
|
||||
content_blocks = messages
|
||||
|
||||
response = anthropic_client.messages.create(
|
||||
model=model,
|
||||
max_tokens=8192,
|
||||
messages=[{"role": "user", "content": content_blocks}],
|
||||
)
|
||||
return _result_with_usage(response.content, response.usage, messages)
|
||||
response = anthropic_client.messages.create(
|
||||
model=model,
|
||||
max_tokens=8192,
|
||||
messages=[{"role": "user", "content": content_blocks}],
|
||||
)
|
||||
return _result_with_usage(response.content, response.usage, messages)
|
||||
```
|
||||
|
||||
**The `cache_prefix_blocks` helper** splits the message at the given char offsets and marks each prefix with `cache_control: {"type": "ephemeral"}`. Max 3 prefix blocks (provider limit is 4 breakpoints per request).
|
||||
@@ -750,17 +750,17 @@ def _send_anthropic(messages, *, cache_prefix_chars=None):
|
||||
|
||||
```python
|
||||
def _send_gemini(messages, *, cache_ttl_seconds=3600):
|
||||
if cache_ttl_seconds > 0:
|
||||
cached_content = genai_client.caches.create(
|
||||
model=model, contents=stable_prefix_messages, ttl=f"{cache_ttl_seconds}s",
|
||||
)
|
||||
response = genai_client.models.generate_content(
|
||||
model=model, contents=volatile_messages,
|
||||
config=genai.types.GenerateContentConfig(cached_content=cached_content.name),
|
||||
)
|
||||
else:
|
||||
response = genai_client.models.generate_content(model=model, contents=messages)
|
||||
return _result_with_usage(response.text, response.usage_metadata, messages)
|
||||
if cache_ttl_seconds > 0:
|
||||
cached_content = genai_client.caches.create(
|
||||
model=model, contents=stable_prefix_messages, ttl=f"{cache_ttl_seconds}s",
|
||||
)
|
||||
response = genai_client.models.generate_content(
|
||||
model=model, contents=volatile_messages,
|
||||
config=genai.types.GenerateContentConfig(cached_content=cached_content.name),
|
||||
)
|
||||
else:
|
||||
response = genai_client.models.generate_content(model=model, contents=messages)
|
||||
return _result_with_usage(response.text, response.usage_metadata, messages)
|
||||
```
|
||||
|
||||
**The default TTL is 1 hour**; configurable per-discussion via the GUI.
|
||||
@@ -783,21 +783,21 @@ No application-side control; the provider handles caching. The GUI just shows "C
|
||||
```python
|
||||
@dataclass
|
||||
class DiscussionCacheState:
|
||||
discussion_id: str
|
||||
provider: str
|
||||
cached_at: datetime
|
||||
expires_at: Optional[datetime] # None for OpenAI implicit
|
||||
hit_count: int = 0
|
||||
tokens_cached: int = 0
|
||||
last_invalidated_at: Optional[datetime] = None
|
||||
caching_enabled: bool = True
|
||||
discussion_id: str
|
||||
provider: str
|
||||
cached_at: datetime
|
||||
expires_at: Optional[datetime] # None for OpenAI implicit
|
||||
hit_count: int = 0
|
||||
tokens_cached: int = 0
|
||||
last_invalidated_at: Optional[datetime] = None
|
||||
caching_enabled: bool = True
|
||||
```
|
||||
|
||||
**The Hook API additions:**
|
||||
|
||||
```
|
||||
GET /api/cache # list all discussion cache states
|
||||
GET /api/cache/<discussion_id> # get one
|
||||
GET /api/cache # list all discussion cache states
|
||||
GET /api/cache/<discussion_id> # get one
|
||||
POST /api/cache/<discussion_id>/invalidate
|
||||
POST /api/cache/<discussion_id>/disable
|
||||
POST /api/cache/<discussion_id>/enable
|
||||
@@ -809,15 +809,15 @@ POST /api/cache/<discussion_id>/enable
|
||||
|
||||
```python
|
||||
def _send_claude_code(message, model, *, allowed_tools=None, max_turns=1):
|
||||
options = ClaudeAgentOptions(
|
||||
model=None if not model or model == "default" else model,
|
||||
max_turns=max_turns,
|
||||
tools=list(allowed_tools) if allowed_tools else [],
|
||||
allowed_tools=list(allowed_tools) if allowed_tools else [],
|
||||
cwd=os.getcwd(),
|
||||
)
|
||||
# ... claude_agent_sdk.query(prompt=message, options=options)
|
||||
return _result_with_usage(text, usage, message)
|
||||
options = ClaudeAgentOptions(
|
||||
model=None if not model or model == "default" else model,
|
||||
max_turns=max_turns,
|
||||
tools=list(allowed_tools) if allowed_tools else [],
|
||||
allowed_tools=list(allowed_tools) if allowed_tools else [],
|
||||
cwd=os.getcwd(),
|
||||
)
|
||||
# ... claude_agent_sdk.query(prompt=message, options=options)
|
||||
return _result_with_usage(text, usage, message)
|
||||
```
|
||||
|
||||
### The cross-references
|
||||
|
||||
Reference in New Issue
Block a user