|
|
|
@@ -0,0 +1,229 @@
|
|
|
|
|
# Track Specification: Vendor Refresh + xAI Rename
|
|
|
|
|
|
|
|
|
|
## Overview
|
|
|
|
|
|
|
|
|
|
Two intertwined changes to the AI client vendor layer:
|
|
|
|
|
|
|
|
|
|
1. **Rename `grok` → `xai`** across all 5 layers (public identifiers, internal function names, dispatch branches, cost_tracker regex, tests + docs). xAI is the vendor (the company, api.x.ai); Grok is a model brand sold by xAI. The current code names the vendor after the model brand — a misnomer. Hard cutover on the credentials key (no backward-compatible fallback; clear error message).
|
|
|
|
|
|
|
|
|
|
2. **Full model registry sync** for all 7 models.dev-mapped vendors against the canonical model lists at `https://models.dev/labs/<lab>`. Drop stale entries, add missing ones, update pricing/context/capabilities. `gemini_cli` stays as-is (not a models.dev vendor; it delegates to gemini models).
|
|
|
|
|
|
|
|
|
|
Canonical source: `https://models.dev/` (open-source database of AI model specs, pricing, features). Lab pages list canonical models by author. The sync uses the lab page as the source of truth for the model ID, context window, reasoning/tool/structured flags, and pricing.
|
|
|
|
|
|
|
|
|
|
## Current State Audit (as of commit `9a308c36`)
|
|
|
|
|
|
|
|
|
|
### The `grok` vendor (lines referenced from `src/ai_client.py`)
|
|
|
|
|
|
|
|
|
|
**Layer 1 — Public identifiers:**
|
|
|
|
|
- `PROVIDERS` list (`ai_client.py:62`): includes `"grok"` between `"qwen"` and `"llama"`
|
|
|
|
|
- `VendorCapabilities` registry (`ai_client.py:267-270`): 4 entries with `vendor='grok'` — wildcard + `grok-2` + `grok-2-vision` + `grok-beta`. All stale (xAI's current models are grok-4.3, grok-build-0.1, grok-4.20 variants).
|
|
|
|
|
- `provider_state.py:68`: `"_PROVIDER_HISTORIES"` dict has `"grok": ProviderHistory()`
|
|
|
|
|
- `ai_client.py:2635`: `creds.get("grok", {}).get("api_key")` — credentials.toml key is `[grok]`
|
|
|
|
|
|
|
|
|
|
**Layer 2 — Internal function names:**
|
|
|
|
|
- `_grok_client` global (`ai_client.py:152`), reset at `:612`
|
|
|
|
|
- `_ensure_grok_client()` (`ai_client.py:2630-2639`) — builds `openai.OpenAI(api_key=..., base_url="https://api.x.ai/v1")`
|
|
|
|
|
- `_send_grok()` (`ai_client.py:2641-2724`) — the dispatch function; docstring references "Grok history using _grok_history_lock" (stale: the lock moved to provider_state)
|
|
|
|
|
- `_build_grok_request()` (nested in `_send_grok`, `ai_client.py:2700`)
|
|
|
|
|
- `_list_grok_models()` (`ai_client.py:2726-2727`) — returns `list_models_for_vendor("grok")`
|
|
|
|
|
|
|
|
|
|
**Layer 3 — Dispatch branches:**
|
|
|
|
|
- `list_models()` (`ai_client.py:634`): `elif provider == "grok": return _list_grok_models()`
|
|
|
|
|
- `send()` (`ai_client.py:3424-3426`): `elif p == "grok": res = _send_grok(...)`
|
|
|
|
|
- Import list (`ai_client.py:3363` and `:996`): `_send_grok` in `__all__`-style import blocks
|
|
|
|
|
|
|
|
|
|
**Layer 4 — Cost tracker regex (`src/cost_tracker.py:53-55`):**
|
|
|
|
|
- `(r"grok-2", {"input_per_mtok": 2.00, "output_per_mtok": 10.00})`
|
|
|
|
|
- `(r"grok-2-vision", {"input_per_mtok": 2.00, "output_per_mtok": 10.00})`
|
|
|
|
|
- `(r"grok-beta", {"input_per_mtok": 5.00, "output_per_mtok": 15.00})`
|
|
|
|
|
- All three match stale model IDs. The cost tracker uses model-name regex (not vendor name), so after the rename the patterns should match the new xAI model IDs (grok-4.3, grok-4.20-*, grok-build-0.1).
|
|
|
|
|
|
|
|
|
|
**Layer 5 — Tests + docs:**
|
|
|
|
|
- `tests/test_grok_provider.py` — provider dispatch test; references `_send_grok`, `set_provider("grok", "grok-2")`, `get_capabilities("grok", "grok-2-vision")`
|
|
|
|
|
- `tests/test_vendor_capabilities.py:88` — `get_capabilities('grok', 'grok-2-vision')`
|
|
|
|
|
- `tests/test_providers_source_of_truth.py:12` — `EXPECTED_PROVIDERS` list includes `"grok"`
|
|
|
|
|
- `tests/test_provider_curation.py:6` — same
|
|
|
|
|
- `tests/test_provider_state.py:20,31,32,86` — `EXPECTED_PROVIDERS` tuple includes `"grok"`; `get_history("grok")` calls
|
|
|
|
|
- `tests/test_provider_state_migration.py:16` — same
|
|
|
|
|
- `docs/guide_ai_client.md` — multiple references to `_send_grok`, the 8-provider list, the grok vendor
|
|
|
|
|
- `docs/guide_models.md:59` — `PROVIDERS` constant description lists `"grok"`
|
|
|
|
|
- `docs/Readme.md:34,39` — provider list mentions `grok`
|
|
|
|
|
|
|
|
|
|
### The 8 vendors' current registry state (vs. models.dev canonical)
|
|
|
|
|
|
|
|
|
|
| Our vendor | models.dev lab | Our model count | models.dev model count | Gap |
|
|
|
|
|
|---|---|---|---|---|
|
|
|
|
|
| `gemini` | google | 5 (wildcard + 4) | 24 | Missing 20; has stale pricing on some |
|
|
|
|
|
| `anthropic` | anthropic | 12 (wildcard + 11) | 22 | Missing 10; some stale pricing |
|
|
|
|
|
| `gemini_cli` | (none) | 0 (delegates to gemini) | — | N/A — untouched |
|
|
|
|
|
| `deepseek` | deepseek | 4 (wildcard + 3) | 5 | Missing `deepseek-chat`, `deepseek-v4-flash`, `deepseek-v4-pro`; has stale `deepseek-v3` |
|
|
|
|
|
| `minimax` | minimax | 5 (wildcard + 4) | 7 | Missing `MiniMax-M3`, `MiniMax-M2.7-highspeed`, `MiniMax-M2.5-highspeed` |
|
|
|
|
|
| `qwen` | alibaba | 8 (wildcard + 7) | 32 | Missing 24; many stale (qwen-turbo/plus/max/long are old names) |
|
|
|
|
|
| `grok` → `xai` | xai | 4 (wildcard + 3, all stale) | 4 | All 3 named entries stale; new set is grok-4.3, grok-build-0.1, grok-4.20-non-reasoning, grok-4.20-reasoning |
|
|
|
|
|
| `llama` | meta | 9 (wildcard + 8) | 3 | Has 8 stale Llama 3.x entries; models.dev only lists 3 (Llama 4 Maverick, Llama 4 Scout, Llama-3.3-70B) |
|
|
|
|
|
|
|
|
|
|
### Already Implemented (DO NOT re-implement)
|
|
|
|
|
|
|
|
|
|
- `VendorCapabilities` dataclass with 22 fields (`ai_client.py:228-253`) — frozen dataclass, the registry shape is correct
|
|
|
|
|
- `register()` / `get_capabilities()` / `list_models_for_vendor()` (`ai_client.py:255-260`) — registry mechanics work
|
|
|
|
|
- Wildcard `*` fallback in `get_capabilities()` — already handles unknown model IDs per-vendor
|
|
|
|
|
- `provider_state.py` `ProviderHistory` + `_PROVIDER_HISTORIES` dict — the per-provider history isolation pattern is correct; only the `"grok"` key needs renaming to `"xai"`
|
|
|
|
|
- `_send_grok`'s OpenAI-compatible dispatch via `run_with_tool_loop` — the dispatch logic is correct; only the name and the `vendor_name="grok"` kwarg change
|
|
|
|
|
- `_ensure_grok_client`'s `base_url="https://api.x.ai/v1"` — already correct (xAI's API endpoint); only the function name and the creds key change
|
|
|
|
|
|
|
|
|
|
### Gaps to Fill (This Track's Scope)
|
|
|
|
|
|
|
|
|
|
**Gap 1 (Rename):** The `grok` vendor string is a misnomer (xAI is the vendor; Grok is a model brand). It must become `xai` in all 5 layers. The `grok` string survives ONLY as a model name (e.g., `grok-4.3`), which is correct.
|
|
|
|
|
|
|
|
|
|
**Gap 2 (Stale xAI models):** The 3 named xAI entries (`grok-2`, `grok-2-vision`, `grok-beta`) are obsolete. models.dev lists 4 current xAI models: `grok-4.3`, `grok-build-0.1`, `grok-4.20-0309-non-reasoning`, `grok-4.20-0309-reasoning`.
|
|
|
|
|
|
|
|
|
|
**Gap 3 (Stale/missing models across other 6 vendors):** The registry is out of sync with models.dev for gemini, anthropic, deepseek, minimax, qwen, and llama. Each has stale entries (old model IDs no longer on models.dev) and missing entries (new model IDs on models.dev not in our registry). Pricing and context windows also need updating to current values.
|
|
|
|
|
|
|
|
|
|
**Gap 4 (Cost tracker regex stale):** `cost_tracker.py:53-55` matches `grok-2`, `grok-2-vision`, `grok-beta` — all stale. After the model sync, the regex should match the new xAI model IDs.
|
|
|
|
|
|
|
|
|
|
## Goals
|
|
|
|
|
|
|
|
|
|
1. The `grok` vendor is renamed to `xai` everywhere. No `grok` vendor string survives in `src/`, `tests/`, or `docs/` (except as a model name within xAI's registry entries).
|
|
|
|
|
2. The `VendorCapabilities` registry is fully synced with models.dev for all 7 mapped vendors. Every canonical model on the lab page has a registry entry; no stale entries remain.
|
|
|
|
|
3. The cost tracker regex matches current model IDs.
|
|
|
|
|
4. All tests pass (renamed + updated). The 6+ test files that reference `grok` are renamed/updated to reference `xai`.
|
|
|
|
|
5. Docs reflect the rename and the updated vendor list.
|
|
|
|
|
|
|
|
|
|
## Functional Requirements
|
|
|
|
|
|
|
|
|
|
### FR1: Rename `grok` → `xai` (all 5 layers)
|
|
|
|
|
|
|
|
|
|
**FR1.1 — Public identifiers:**
|
|
|
|
|
- `PROVIDERS` list (`ai_client.py:62`): `"grok"` → `"xai"` (position unchanged, between `"qwen"` and `"llama"`)
|
|
|
|
|
- `VendorCapabilities` registry: all `vendor='grok'` → `vendor='xai'`
|
|
|
|
|
- `provider_state.py:68`: `"grok": ProviderHistory()` → `"xai": ProviderHistory()`
|
|
|
|
|
- `ai_client.py:2635` creds read: `creds.get("grok", {})` → `creds.get("xai", {})`. Error message becomes: `"xAI API key not found in credentials.toml — the [grok] section was renamed to [xai] in the vendor refresh track. Update your credentials.toml."`
|
|
|
|
|
|
|
|
|
|
**FR1.2 — Internal function names:**
|
|
|
|
|
- `_grok_client` → `_xai_client` (global at `:152`, reset at `:612`)
|
|
|
|
|
- `_ensure_grok_client` → `_ensure_xai_client`
|
|
|
|
|
- `_send_grok` → `_send_xai`
|
|
|
|
|
- `_build_grok_request` → `_build_xai_request`
|
|
|
|
|
- `_list_grok_models` → `_list_xai_models`
|
|
|
|
|
- `vendor_name="grok"` kwarg in `run_with_tool_loop` call → `vendor_name="xai"`
|
|
|
|
|
- `source="ai_client.grok"` in `_classify_openai_compatible_error` → `source="ai_client.xai"`
|
|
|
|
|
- Docstring references in `_send_xai` updated ("Grok history" → "xAI history"; remove stale `_grok_history_lock` reference)
|
|
|
|
|
|
|
|
|
|
**FR1.3 — Dispatch branches:**
|
|
|
|
|
- `list_models()` (`:634`): `elif provider == "grok": return _list_grok_models()` → `elif provider == "xai": return _list_xai_models()`
|
|
|
|
|
- `send()` (`:3424-3426`): `elif p == "grok": res = _send_grok(...)` → `elif p == "xai": res = _send_xai(...)`
|
|
|
|
|
- Import blocks (`:996`, `:3363`): `_send_grok` → `_send_xai`, `_list_grok_models` → `_list_xai_models`
|
|
|
|
|
|
|
|
|
|
**FR1.4 — Cost tracker regex (`src/cost_tracker.py:53-55`):**
|
|
|
|
|
- Remove the 3 stale patterns (`grok-2`, `grok-2-vision`, `grok-beta`)
|
|
|
|
|
- Add patterns matching the new xAI model IDs with current pricing from models.dev (see FR2.7 for the per-model pricing)
|
|
|
|
|
|
|
|
|
|
**FR1.5 — Tests:**
|
|
|
|
|
- `tests/test_grok_provider.py` → renamed to `tests/test_xai_provider.py`; all `grok` references → `xai`; model IDs updated to current xAI models
|
|
|
|
|
- `tests/test_vendor_capabilities.py:88`: `get_capabilities('grok', 'grok-2-vision')` → `get_capabilities('xai', '<current-vision-capable-xai-model>')` or drop the vision-specific assertion if no current xAI model has vision (per models.dev, none of the 4 current xAI models list vision — the assertion should be updated to test a current capability like `web_search=True`)
|
|
|
|
|
- `tests/test_providers_source_of_truth.py:12`: `EXPECTED_PROVIDERS` — `"grok"` → `"xai"`
|
|
|
|
|
- `tests/test_provider_curation.py:6`: same
|
|
|
|
|
- `tests/test_provider_state.py:20,31,32,86`: `EXPECTED_PROVIDERS` tuple — `"grok"` → `"xai"`; `get_history("grok")` → `get_history("xai")`
|
|
|
|
|
- `tests/test_provider_state_migration.py:16`: same
|
|
|
|
|
|
|
|
|
|
**FR1.6 — Docs:**
|
|
|
|
|
- `docs/guide_ai_client.md`: all `_send_grok` → `_send_xai`, "grok" vendor references → "xai", the 8-provider list updated
|
|
|
|
|
- `docs/guide_models.md:59`: `PROVIDERS` description — `"grok"` → `"xai"`
|
|
|
|
|
- `docs/Readme.md:34,39`: provider list mentions — `"grok"` → `"xai"`
|
|
|
|
|
|
|
|
|
|
### FR2: Full model registry sync (7 vendors)
|
|
|
|
|
|
|
|
|
|
Sync rule: for each models.dev-mapped vendor, the registry contains (a) one wildcard `*` entry as fallback and (b) one entry per canonical model on the lab page. Pricing: use models.dev's listed price when non-zero; when models.dev shows `$0.00 / $0.00` (meaning "no canonical lab-direct price; served by third-party providers"), preserve our existing cost data if we have it for that model family, otherwise set `cost_input_per_mtok=0.0, cost_output_per_mtok=0.0` with a `notes=` explaining the pricing is provider-dependent.
|
|
|
|
|
|
|
|
|
|
**FR2.1 — gemini (google lab, 24 models):**
|
|
|
|
|
Replace the 5 current entries with a wildcard + 24 entries. Notable additions: `gemini-3.5-flash`, `gemini-3.1-flash-lite`, `gemini-3.1-pro-preview-customtools`, `gemini-3-flash-preview`, `gemini-3-pro-preview`, `gemini-flash-latest`, `gemini-flash-lite-latest`, `gemini-2.5-pro`, `gemini-2.5-flash-tts`, `gemini-2.5-pro-tts`, `gemini-2.0-flash`, `gemini-2.0-flash-lite`, the Gemma 4 variants, the Nano Banana image models, `gemini-embedding-001`. Drop nothing that's still on models.dev (our current 4 named entries — gemini-3.1-pro-preview, gemini-3-flash-preview, gemini-2.5-flash, gemini-2.5-flash-lite — are all still listed, so they stay with updated pricing). Update pricing from models.dev: e.g., `gemini-2.5-flash` is `$0.21/$1.80` (we have `$0.15/$0.60` — stale); `gemini-2.5-flash-lite` is `$0.07/$0.28` (we have `$0.075/$0.30` — close); `gemini-3.1-pro-preview` is `$2.00/$12.00` (we have `$3.50/$10.50` — stale); `gemini-3-flash-preview` is `$0.50/$3.00` (we have `$0.15/$0.60` — stale).
|
|
|
|
|
|
|
|
|
|
**FR2.2 — anthropic (anthropic lab, 22 models):**
|
|
|
|
|
Replace the 12 current entries with a wildcard + 22 entries. Notable additions: `claude-sonnet-5`, `claude-fable-5`, `claude-opus-4-8`, `claude-opus-4-7`, `claude-sonnet-4-6`, `claude-opus-4-6`, `claude-opus-4-5` (latest alias), `claude-haiku-4-5` (latest alias), `claude-sonnet-4-5` (latest alias), `claude-opus-4-1` (latest alias), `claude-opus-4` (latest alias), `claude-sonnet-4` (latest alias), `claude-3-7-sonnet-20250219`, `claude-3-5-haiku-20241022`, `claude-3-5-sonnet-20241022`, `claude-3-haiku-20240307`. Our current entries with dated IDs (`claude-sonnet-4-5-20250929`, `claude-sonnet-4-20250514`, `claude-opus-4-1-20250805`, `claude-opus-4-20250514`, `claude-opus-4-5-20251101`, `claude-haiku-4-5-20251001`) stay (they're on models.dev). Our non-dated aliases (`claude-sonnet-4-6`, `claude-opus-4-6`, `claude-opus-4-7`, `claude-opus-4-8`, `claude-fable-5`) stay. Update pricing: e.g., `claude-opus-4-6` is `$5.00/$25.00` on models.dev (we have `$15.00/$75.00` — stale, that was opus-4-1 pricing); `claude-haiku-3-5` is `$0.80/$4.00` (we don't have it — add). Context windows: the newer Claude 5/4.6/4.7/4.8 models have 1,000,000 context (we have 200,000 for all — stale for the new ones).
|
|
|
|
|
|
|
|
|
|
**FR2.3 — deepseek (deepseek lab, 5 models):**
|
|
|
|
|
Replace the 4 current entries with a wildcard + 5 entries. Add `deepseek-v4-flash` (1M context, reasoning), `deepseek-v4-pro` (1M context, reasoning), `deepseek-chat` (1M context, no reasoning — this is the V3.2 chat model). Update `deepseek-reasoner` to 1M context (we have 32768 — stale) and pricing `$0.14/$0.28` (we have `$0.55/$2.19` — stale, that was old reasoner pricing). `deepseek-r1` stays at 128K context. Drop `deepseek-v3` (no longer on models.dev as a standalone — it's been superseded by `deepseek-chat`).
|
|
|
|
|
|
|
|
|
|
**FR2.4 — minimax (minimax lab, 7 models):**
|
|
|
|
|
Replace the 5 current entries with a wildcard + 7 entries. Add `MiniMax-M3` (512K context), `MiniMax-M2.7-highspeed` (204K context, `$0.33/$1.32`), `MiniMax-M2.5-highspeed` (204K context, `$0.19/$1.24`). Update pricing: `MiniMax-M2.7` is `$0.18/$0.72` (we have `$0.20/$0.20` — stale); `MiniMax-M2.5` is `$0.00/$0.00` on models.dev (provider-dependent — keep our `$0.20/$0.20` or set to 0 with note); `MiniMax-M2.1` is `$0.27/$1.10` (we have `$0.20/$0.20` — stale); `MiniMax-M2` is `$0.20/$1.00` (we have `$0.20/$0.20` — output stale). Update context windows: M2.7/M2.5/M2.1 are 204,800 (we have 131,072 — stale); M3 is 512,000; M2 is 196,608.
|
|
|
|
|
|
|
|
|
|
**FR2.5 — qwen (alibaba lab, 32 models):**
|
|
|
|
|
Replace the 8 current entries with a wildcard + 32 entries. This is the largest delta. Add all 32 Qwen models from models.dev. Our current entries (`qwen-turbo`, `qwen-plus`, `qwen-max`, `qwen-long`, `qwen-vl-plus`, `qwen-vl-max`, `qwen-audio`) — most are still on models.dev under those names or close variants. `qwen-long` is NOT on the current models.dev list (it's been superseded) — drop it. Update pricing and context per models.dev. Notable additions: `qwen3.7-plus`, `qwen3.7-max`, `qwen3.6-flash`, `qwen3.6-27b`, `qwen3.6-plus`, the Qwen3.5 family, `qwen3-max`, `qwen3-vl-plus`, `qwen3-coder-flash`, `qwen3-coder-plus`, `qwen-flash`, `qwen-turbo`, `qwen3-235b-a22b`, `qwen3-32b`, the Qwen3-Coder instruct models, `qwen-omni-turbo`, `qwq-plus`, `qwen-max`, `qwen2.5-vl-72b-instruct`.
|
|
|
|
|
|
|
|
|
|
**FR2.6 — llama (meta lab, 3 models):**
|
|
|
|
|
Replace the 9 current entries with a wildcard + 3 entries. models.dev only lists 3 Meta models: `llama-4-maverick-17b-instruct` (1M context, `$0.12/$0.60`), `llama-4-scout-17b-instruct` (3.5M context, `$0.17/$0.66`), `llama-3.3-70b-instruct` (128K context, `$0.00/$0.00` — open weight, provider-dependent). Drop all 8 of our stale Llama 3.x entries (`llama-3.1-8b-instant`, `llama-3.1-70b-versatile`, `llama-3.1-405b-reasoning`, `llama-3.2-1b-preview`, `llama-3.2-3b-preview`, `llama-3.2-11b-vision-preview`, `llama-3.2-90b-vision-preview`, `llama-3.3-70b-specdec`) — none are on models.dev's current Meta lab page. Note: our `llama` vendor dispatches to Ollama/OpenRouter/custom URLs, so the dropped models may still be *callable* via the wildcard fallback; they just won't have explicit registry entries.
|
|
|
|
|
|
|
|
|
|
**FR2.7 — xai (xai lab, 4 models):**
|
|
|
|
|
Replace the 4 stale `grok` entries with a wildcard + 4 entries (all with `vendor='xai'`):
|
|
|
|
|
- `xai` wildcard `*`: context_window=1000000, cost from grok-4.3, web_search=True, x_search=True, reasoning=True, tool_calling=True, structured_output=True
|
|
|
|
|
- `xai/grok-4.3`: 1,000,000 context, reasoning=True, tool_calling=True, structured_output=True, web_search=True, x_search=True. models.dev price `$0.00/$0.00` (provider-dependent). Per the FR2 preamble sync rule: set `cost_input_per_mtok=0.0, cost_output_per_mtok=0.0` with `notes='pricing provider-dependent; see https://docs.x.ai/docs/models'`. Do NOT invent pricing.
|
|
|
|
|
- `xai/grok-build-0.1`: 256,000 context, reasoning=True, tool_calling=True, structured_output=True. Coding model.
|
|
|
|
|
- `xai/grok-4.20-0309-non-reasoning`: 1,000,000 context, reasoning=False, tool_calling=True, structured_output=True. models.dev price `$1.25/$2.50`.
|
|
|
|
|
- `xai/grok-4.20-0309-reasoning`: 1,000,000 context, reasoning=True, tool_calling=True, structured_output=True. models.dev price `$1.25/$2.50`.
|
|
|
|
|
|
|
|
|
|
**FR2.8 — Cost tracker (`src/cost_tracker.py`):**
|
|
|
|
|
Update the xAI regex patterns to match the new model IDs. Remove `grok-2`, `grok-2-vision`, `grok-beta`. Add patterns for `grok-4.3`, `grok-build-0.1`, `grok-4.20-0309-non-reasoning`, `grok-4.20-0309-reasoning` with the pricing from FR2.7. The regex matches on model name (not vendor name), so patterns like `r"grok-4\.20"` would catch both reasoning variants if they share pricing.
|
|
|
|
|
|
|
|
|
|
### FR3: `gemini_cli` untouched
|
|
|
|
|
|
|
|
|
|
`gemini_cli` is not on models.dev. It delegates to gemini models via the CLI subprocess. Its registry entry (if any) and dispatch stay as-is. No changes.
|
|
|
|
|
|
|
|
|
|
## Non-Functional Requirements
|
|
|
|
|
|
|
|
|
|
- **NFR1 (Type safety):** The `VendorCapabilities` dataclass is already `@dataclass(frozen=True)`. The rename and sync must not introduce `dict[str, Any]`, `Any`, or `Optional[T]` in the touched code. Registry entries are constructed via `register(VendorCapabilities(...))` calls with keyword args — keep that pattern.
|
|
|
|
|
- **NFR2 (No behavior change beyond vendor data):** The `_send_xai` dispatch logic is identical to `_send_grok` — only names change. The `run_with_tool_loop` call signature is unchanged. The `base_url="https://api.x.ai/v1"` is already correct. No SDK swap, no endpoint change.
|
|
|
|
|
- **NFR3 (Tests pass):** All renamed/updated tests pass. The `live_gui` tests that exercise provider switching (if any reference grok) must be updated. Run the batch, not just isolation.
|
|
|
|
|
- **NFR4 (Atomic per-task commits):** Each plan task is one commit. The rename is mechanical and can be one commit; the registry sync is per-vendor (one commit per vendor) for safe rollback.
|
|
|
|
|
- **NFR5 (Docs in sync):** Per the Documentation Refresh Protocol (workflow.md), the docs touching the vendor list are updated in the same track.
|
|
|
|
|
- **NFR6 (Scope discipline):** No new `src/<thing>.py` files. All changes go in existing files (`ai_client.py`, `cost_tracker.py`, `provider_state.py`). No new vendors added (scope is B, not C).
|
|
|
|
|
|
|
|
|
|
## Architecture Reference
|
|
|
|
|
|
|
|
|
|
- **`docs/guide_ai_client.md`** — the AI client reference; §"State" lists the 8 providers and the `_send_<provider>` functions; §"Vendor Capabilities" documents the registry. Both sections need updating.
|
|
|
|
|
- **`docs/guide_models.md`** — §"PROVIDERS" documents the constant. Needs the `grok` → `xai` update.
|
|
|
|
|
- **`conductor/code_styleguides/data_oriented_design.md` §8.5** — the Python Type Promotion Mandate. The registry entries are typed `VendorCapabilities` (frozen dataclass) — already compliant. The sync must not regress to `dict[str, Any]`.
|
|
|
|
|
- **`conductor/code_styleguides/error_handling.md`** — the credentials-not-found error is a `ValueError` raised at the boundary (`_ensure_xai_client`). This is the existing pattern; keep it. The error message is updated to guide the user to rename their `[grok]` section.
|
|
|
|
|
- **`src/provider_state.py`** — the `ProviderHistory` pattern. The `"grok"` key in `_PROVIDER_HISTORIES` is renamed to `"xai"`. The `EXPECTED_PROVIDERS` tuple in tests is the source-of-truth assertion for this dict's keys.
|
|
|
|
|
|
|
|
|
|
## Risks
|
|
|
|
|
|
|
|
|
|
**R1 (User credentials breakage):** Hard cutover means any user with `[grok]` in credentials.toml gets an error on first use after upgrade. **Mitigation:** the error message explicitly names the fix ("rename [grok] to [xai]"). This is the user's chosen behavior (hard cutover, option B). Likelihood: certain for users with xAI configured. Impact: 1-line config edit to resolve. The user accepted this.
|
|
|
|
|
|
|
|
|
|
**R2 (Test breakage cascade):** 6+ test files reference `grok`. The rename must touch all of them atomically (in one commit) or tests fail mid-rename. **Mitigation:** the plan does the full rename in one phase, one commit, all layers at once. Likelihood: low if the plan is followed. Impact: test suite red until the commit lands.
|
|
|
|
|
|
|
|
|
|
**R3 (Registry sync data quality):** models.dev shows `$0.00 / $0.00` for many models (meaning "no canonical lab-direct price; provider-dependent"). Our registry needs *some* number for the cost tracking feature. **Mitigation:** the sync rule (FR2 preamble) preserves our existing cost data for provider-dependent models, or sets 0.0 with a `notes=` field. This is documented; the implementer follows the rule. Likelihood: medium (judgment calls on which models keep which cost). Impact: cost tracking shows 0 for some models — acceptable (the feature degrades gracefully; the user can override).
|
|
|
|
|
|
|
|
|
|
**R4 (Llama model drop is user-visible):** Dropping 8 Llama 3.x registry entries means the GUI's model picker won't list them (though they're still callable via the wildcard fallback if the user types the model ID). **Mitigation:** this is the correct behavior per the full-sync decision (option A). The dropped models are stale; models.dev doesn't list them. The user accepted full sync. Likelihood: certain. Impact: user must type old model IDs manually if they want them; acceptable.
|
|
|
|
|
|
|
|
|
|
**R5 (Scope creep into `_send_xai` logic):** The rename touches `_send_xai`'s body. A worker might be tempted to "improve" the dispatch logic while there. **Mitigation:** the plan explicitly scopes the change to renaming only; the dispatch logic is correct and untouched. The SSDL/docstring is updated but the code path is identical.
|
|
|
|
|
|
|
|
|
|
## Verification
|
|
|
|
|
|
|
|
|
|
**V1 (Rename completeness):** After Phase 1, run `grep -r "grok" src/ tests/ docs/` (case-sensitive, whole-word on `src/`). The only matches in `src/` should be the `grok-4.3` / `grok-build-0.1` / `grok-4.20-*` model ID strings inside `VendorCapabilities(vendor='xai', model='grok-...', ...)` calls and the cost_tracker regex. No `grok` vendor string, no `_send_grok`, no `_grok_client`, no `vendor='grok'`.
|
|
|
|
|
|
|
|
|
|
**V2 (Registry sync correctness):** For each of the 7 vendors, `list_models_for_vendor("<vendor>")` returns exactly the set of model IDs on the models.dev lab page (plus the wildcard is registered but not listed). A test asserts this per vendor.
|
|
|
|
|
|
|
|
|
|
**V3 (Test suite green):** The full test suite passes in batch (not isolation). Specifically: `test_xai_provider.py` (renamed), `test_vendor_capabilities.py`, `test_providers_source_of_truth.py`, `test_provider_curation.py`, `test_provider_state.py`, `test_provider_state_migration.py`, `test_cost_tracker.py` (if it exists).
|
|
|
|
|
|
|
|
|
|
**V4 (Cost tracker patterns match new models):** `cost_tracker.estimate_cost("grok-4.3", ...)` returns the synced price; `cost_tracker.estimate_cost("grok-2", ...)` returns 0 (pattern removed).
|
|
|
|
|
|
|
|
|
|
**V5 (Docs grep clean):** `grep -r "grok" docs/guide_ai_client.md docs/guide_models.md docs/Readme.md` returns only model-name references (e.g., "grok-4.3"), no vendor-name references.
|
|
|
|
|
|
|
|
|
|
## Out of Scope
|
|
|
|
|
|
|
|
|
|
- **Adding new vendors** (OpenAI, Mistral, Cohere, Moonshot, Zhipu, Xiaomi, etc.) — this is scope option C, rejected. Each new vendor needs a `_send_<vendor>` function, credentials handling, tests. Future track.
|
|
|
|
|
- **`gemini_cli` changes** — not on models.dev; delegates to gemini models; untouched.
|
|
|
|
|
- **`_send_xai` dispatch logic changes** — the OpenAI-compatible dispatch is correct; only names change.
|
|
|
|
|
- **Credentials.toml auto-migration** — hard cutover (user's choice); no fallback, no auto-migrate.
|
|
|
|
|
- **GUI changes** — the GUI reads `PROVIDERS` dynamically; renaming the list entry propagates automatically. No `gui_2.py` changes expected (confirmed: no `grok` references in `gui_2.py` per the audit).
|
|
|
|
|
- **`app_controller.py` changes** — confirmed: no `grok` references in `app_controller.py` per the audit.
|
|
|
|
|
- **Pricing for provider-dependent models** — models.dev shows `$0.00` for models with no canonical lab-direct price. Resolving the actual per-provider pricing for every third-party provider of every model is out of scope. The sync rule (preserve existing or set 0.0 with note) handles this.
|
|
|
|
|
|
|
|
|
|
## See Also
|
|
|
|
|
|
|
|
|
|
- `docs/guide_ai_client.md` — the AI client reference (to be updated)
|
|
|
|
|
- `docs/guide_models.md` — the models/data-structures reference (to be updated)
|
|
|
|
|
- `https://models.dev/labs` — the canonical lab list (22 labs)
|
|
|
|
|
- `https://models.dev/labs/<lab>` — per-lab canonical model pages (the sync source)
|
|
|
|
|
- `conductor/code_styleguides/data_oriented_design.md` §8.5 — the type promotion mandate (registry entries must stay typed)
|
|
|
|
|
- `conductor/code_styleguides/error_handling.md` — the boundary-error pattern (credentials-not-found ValueError)
|
ed