# Track: Qwen, Llama & Grok Follow-Up (Post-Phase 5)

**Status:** Active (initializing)
**Initialized:** 2026-06-11
**Owner:** Tier 2 Tech Lead
**Priority:** High (architectural consolidation + UX payoff; user is rightly concerned that the parent track shipped with gaps)

---

## Why This Track Exists

The parent track `qwen_llama_grok_integration_20260606` (status: 50/79 tasks done, Phase 6 in progress) shipped 5 phases cleanly but **left meaningful gaps** that the Tier 2 Tech Lead did not surface until the Phase 5 checkpoint. This track captures the deferred work, ordered by impact.

**The Tier 2's failure mode** (called out by the user 2026-06-11): "you never even told me until now and then you just say 'oh yeah we're done btw, fuck you' thats what it feels like." Rightly called. This track exists to fix that.

---

## Goals (Priority Order)

| Priority | Goal | Rationale |
|---|---|---|
| **A (architectural)** | Lift the tool-call loop into a shared `run_with_tool_loop()` helper. Apply to all 4 new vendors + the 4 existing vendors. | Today only `_send_minimax` has a working tool loop. Qwen/Grok/Llama are single-shot (regression). Anthropic/Gemini/Gemini-cli/DeepSeek already have inline tool loops (4-way duplication). Lifting gives one place to fix bugs + add new behavior. |
| **A (architectural)** | Move `PROVIDERS` out of `src/models.py`. | `src/models.py` is for MMA data models (Tickets, Tracks, FileItem). The vendor list is an AI client concern. The audit script `audit_no_models_config_io.py` enforces config I/O rules; PROVIDERS has no analogous enforcement. Move to `src/ai_client.py` (or new `src/ai_client_providers.py`); add an audit script that enforces the move. |
| **A (UX payoff)** | Apply the remaining 8 of 9 UX adaptations from parent track spec §6: tools toggle (tool_calling), cache panel (caching), stream progress (streaming), fetch models (model_discovery), token budget max (context_window), cost panel × 3. | The pattern is established (adaptation 1 shipped in parent Phase 5); the helper `_get_active_capabilities()` is in place; the remaining 8 are mechanical applications. |
| **B (local-first)** | Promote local models from "one of 3 backends" to first-class. | Add `local_backend: bool` capability field (separate from `cost_tracking`). Native Ollama (`/api/chat`) as the default for Llama (not the OpenAI-compatible fallback). Add Meta Llama API as a 4th backend. Add a "Local Model" UI badge. |
| **B (matrix expansion)** | Land the v2 matrix fields: `local`, `reasoning`, `structured_output`, `code_execution`, `web_search`, `x_search`, `file_search`, `mcp_support`, `audio`, `video`, `grounding`, `computer_use`. | These are the 12 fields documented in parent spec §3.1.1 after the Grok consultation. None wired today. Each addition is registry + UI adaptation. |
| **C (provider coverage)** | Migrate Anthropic / Gemini / DeepSeek onto the capability matrix. | Anthropic has prompt caching, extended thinking, Computer Use (high-value UX). Gemini has Grounding with Google Search, native video. DeepSeek has reasoning models. None of these capabilities are exposed in the GUI today. |
| **C (codepath consolidation)** | Reduce `src/ai_client.py` line count (currently 2784). | The 8 vendors' inline patterns have grown. Lifting history management, reasoning content extraction, error classification per HTTP code into shared helpers would cut ~30-40% of the file. |

### Non-Goals (this track)

- **Not** changing the matrix schema beyond the 7 v1 + 12 v2 = 19 fields (no further fields in this track)
- **Not** changing the shared `send_openai_compatible` helper (it works; the tool loop is separate)
- **Not** changing the `vendor_capabilities.py` lookup pattern (it works; registry is the source of truth)
- **Not** adding new vendors (the parent track added Qwen/Grok/Llama; this track only consolidates what's there)
- **Not** cleaning up the existing sprawl (the 3 stray `src/` files `vendor_capabilities.py`, `openai_compatible.py`, `qwen_adapter.py` — see Deferred Work below)
- **Not** refactoring `src/ai_client.py` to a smaller line count (it's 2784 lines and the user said large files are fine)
- **Not** lifting history management into a `VendorHistory` class (out of scope; the existing per-vendor pattern works)
- **Not** lifting reasoning content extraction into a shared helper (out of scope; the per-vendor extraction is short)
- **Not** lifting error classification into a per-HTTP-code helper (out of scope; the per-vendor classifiers are short)

### Deferred Work (separate tracks; out of scope for this one)

The user explicitly stated (2026-06-11): "I know I have to setup audit tracks and refactor tracks down the line to prune and cleanup the codebase but I also know thats not feasible while just trying to get you todo the right thing for this new way of handling vendors or models."

Three follow-up tracks are documented as DEFERRED (not in scope for this track):

1. **`namespace_cleanup_20260611`** — Audit the codebase for file sprawl. Specifically:
   - Move `src/vendor_capabilities.py` content into `src/ai_client.py` (the file is in scope to MODIFY for the v2 fields in this track, but moving it as a whole is the cleanup track's job)
   - Move `src/openai_compatible.py` content into `src/ai_client.py`
   - Move `src/qwen_adapter.py` content into `src/ai_client.py`
   - Audit OTHER modules for similar sprawl: `src/imgui_scopes.py`, `src/markdown_helper.py`, `src/markdown_table.py`, `src/io_pool.py`, `src/external_editor.py`, `src/performance_monitor.py`, `src/session_logger.py`, etc. Some may legitimately be sub-systems that should be namespace-isolated; others may be helpers that should fold into a parent.

2. **`ai_client_codepath_consolidation_20260611`** — Reduce `src/ai_client.py` line count from 2784 by:
   - Lifting history management into a `VendorHistory` class (each vendor has its own lock + history list; the per-vendor boilerplate is ~30 lines × 8 vendors = 240 lines of duplication)
   - Lifting reasoning content extraction into a shared helper
   - Lifting error classification into a per-HTTP-code helper
   - Lifting the per-vendor client init into a uniform pattern
   - The line count reduction is estimated at 30-40% (~1000 lines saved)
   - **Note:** the user explicitly said large files are FINE, so this codepath consolidation is about REDUCING DUPLICATION, not about reducing file size. The file can stay large; we just want less repetition.

3. **`mcp_architecture_refactor_20260606`** (already specced) — Splits `src/mcp_client.py` (2,205 lines) into 6 sub-MCPs (`mcp_file_io.py`, `mcp_python.py`, `mcp_c.py`, `mcp_cpp.py`, `mcp_web.py`, `mcp_analysis.py`). This is the OPPOSITE direction of the user's preference (the user wants things in one file, not split). **Note:** this track is already specced in the parent tracks.md; whether to actually execute it (vs. abort it) is a separate decision. The user may want to abort this track.

### Naming Convention Reference (HARD RULE, per `AGENTS.md`)

New `src/<thing>.py` files may only be created on the user's explicit request. If you find yourself about to create one, **ASK FIRST** — don't just create it. Defaults:
- Helpers and sub-systems go in the parent module
- E.g., AI-client-specific code goes in `src/ai_client.py`; MCP-client code goes in `src/mcp_client.py`
- Even if the parent file is already 3K+ lines, the helper still goes there
- The only new files this project ever creates (per typical track) are: `scripts/audit_*.py`, `tests/test_*.py`, and `docs/*.md`

See `AGENTS.md` "File Size and Naming Convention" for the full rule. This rule was added 2026-06-11 after the user called out the LLM training data bias against large files.

---

## Architecture

### A.1 Tool Loop Lift

**Naming convention (HARD RULE, per `AGENTS.md`):** `run_with_tool_loop` lives IN `src/ai_client.py`, not in a new `src/tool_loop.py`. New `src/<thing>.py` files may only be created on the user's explicit request. The only new files in this track are: `scripts/audit_*.py`, `tests/test_*.py`, and `docs/*.md`. See `AGENTS.md` "File Size and Naming Convention" for the full rule.

Today:
```python
# in _send_minimax (only):
for _round in range(MAX_TOOL_ROUNDS + 2):
    request = OpenAICompatibleRequest(...)
    response = send_openai_compatible(client, request, capabilities=caps)
    if not response.tool_calls: return response.text
    results = asyncio.run(_execute_tool_calls_concurrently(response.tool_calls, ...))
    # ... append results to history ...

# in _send_qwen, _send_grok, _send_llama: no loop (single-shot, regression)
# in _send_anthropic, _send_gemini, _send_gemini_cli, _send_deepseek: inline loop (4-way duplication)
```

After (all in `src/ai_client.py`):
```python
# added near _execute_tool_calls_concurrently at src/ai_client.py:754
def run_with_tool_loop(
    client, request, capabilities, *,
    pre_tool_callback, qa_callback, patch_callback,
    base_dir, vendor_name, history_lock, history, trim_func,
) -> str:
    """Wraps send_openai_compatible with a tool-call loop. Works for any
    OpenAI-compatible vendor; vendor-specific logic (history mgmt,
    trim, message format) is injected via parameters."""
    ...

# in each _send_<vendor>:
response = run_with_tool_loop(
    client=_ensure_<vendor>_client(),
    request=OpenAICompatibleRequest(...),
    capabilities=get_capabilities(vendor, _model),
    pre_tool_callback=..., qa_callback=..., patch_callback=...,
    base_dir=base_dir, vendor_name="<vendor>",
    history_lock=_<vendor>_history_lock,
    history=_<vendor>_history,
    trim_func=_<vendor>_trim_history,
)
```

The helper takes history management as injected parameters (each vendor has its own lock and history list). The tool dispatch (`_execute_tool_calls_concurrently`) takes a `vendor_name` string.

**Audit enforcement:** the new `scripts/audit_no_inline_tool_loops.py` fails if any `_send_<vendor>()` has an inline `for _round_idx in range(MAX_TOOL_ROUNDS` pattern.

### A.2 PROVIDERS Move

Today:
```python
# src/models.py:79
PROVIDERS: List[str] = ["gemini", "anthropic", "gemini_cli", "deepseek", "minimax", "qwen", "grok", "llama"]
```

After:
```python
# src/ai_client.py (new location) or src/ai_client_providers.py (new file)
PROVIDERS: List[str] = ["gemini", "anthropic", "gemini_cli", "deepseek", "minimax", "qwen", "grok", "llama"]

# src/models.py: import from src.ai_client or keep as re-export shim for backward compat
```

The audit script: add `scripts/audit_providers_source_of_truth.py` that verifies PROVIDERS is not declared in `src/models.py`. Fails the build if regressed.

### A.3 UX Adaptations 2-9

Same pattern as the shipped adaptation 1 (Screenshot button iff vision). For each render site:
```python
caps = app._get_active_capabilities()
imgui.begin_disabled(not caps.<field>)
... UI ...
imgui.end_disabled()
if not caps.<field>:
    imgui.same_line()
    imgui.text_disabled("(reason)")
```

### B.1 Local-First Architecture

**Per user feedback (2026-06-11):** "I want to put more emphasis and supporting local models and separating local model vending vis online/cloud vendors of models." Local models must be first-class, not "one of 3 backends."

- Add `local: bool` to `VendorCapabilities` (default False)
- Set True for Llama (when base_url is localhost/127.0.0.1)
- **Native Ollama adapter (in `src/ai_client.py`, NOT a new file):** `ollama_chat()` function lives alongside the existing `_send_llama`. The Ollama backend routes to native `/api/chat` (with `think`, `images` array) instead of OpenAI-compatible `/v1/chat/completions`. Native is the DEFAULT for localhost.
- **Meta Llama API as 4th backend (in `src/ai_client.py`):** `meta_llama_chat()` function. **Prerequisite:** verify the URL `https://llama.developer.meta.com/docs/overview` is reachable; it returned 400 in the parent's session. If unreachable on track start, DEFER the Meta backend to a separate follow-up; the native Ollama + 3 existing backends still ship.
- **GUI: "Local Model" badge** in the AI Settings panel when `caps.local` is True
- **Cost panel: 4th state "Local (no cost)"** distinct from "Free (local)" and "—" (replaces adaption 8's "Free (local)" wording per the v2 matrix; the original parent Phase 5 wording was "Free (local)" which was OK but the follow-up's v2 matrix adds an explicit `local` field that lets the UI be cleaner)

**Naming convention (HARD RULE):** `ollama_chat()` and `meta_llama_chat()` live in `src/ai_client.py` (NOT new `src/llama_ollama_native.py` and `src/llama_meta_api.py`). Per `AGENTS.md` "File Size and Naming Convention" — new top-level `src/<thing>.py` files require explicit user request.

### B.2 Matrix Expansion (v2)

Add to `VendorCapabilities` (the 12 v2 fields):
- `local: bool` (B.1)
- `reasoning: bool` (xAI `reasoning_effort`, Anthropic extended thinking, Ollama `think`)
- `structured_output: bool` (response_format / format)
- `code_execution: bool` (xAI code_interpreter, Anthropic Computer Use, Gemini Code Execution)
- `web_search: bool` (xAI web_search, Gemini Grounding)
- `x_search: bool` (xAI X/Twitter search, xAI-specific)
- `file_search: bool` (xAI file_search, Anthropic PDF, Gemini file API)
- `mcp_support: bool` (xAI mcp_calls, Anthropic MCP)
- `audio: bool` (Qwen-Audio, Gemini audio)
- `video: bool` (Gemini video)
- `grounding: bool` (Gemini Grounding with Google Search)
- `computer_use: bool` (Anthropic Computer Use)

Each new field is a registry update + a UI adaptation. The matrix schema grows; the GUI filters based on the matrix.

**UI adaptations for v2 fields** (one per field, in `src/gui_2.py`):
- `reasoning` → "Reasoning" toggle (controls `reasoning_effort` for xAI, etc.)
- `structured_output` → "JSON output" toggle
- `code_execution` → "Code execution" panel (when True)
- `web_search`, `x_search` → Search tool UI
- `file_search` → File search panel
- `mcp_support` → MCP integration toggle
- `audio` → Audio attachment button (replaces the absent-but-deferred audio_input)
- `video` → Video attachment button
- `grounding` → "Grounding" toggle
- `computer_use` → "Computer Use" toggle

Most of these UI adaptations are small (5-10 line additions per field). They can ship in a batch commit per field, or one big commit at the end of Phase 4.

### C.1 Anthropic / Gemini / DeepSeek Migration

Per the deferred follow-up track `anthropic_gemini_deepseek_capability_matrix_20260606` (parent spec §13.1.A). The capability matrix entries for these vendors can be populated:
- `anthropic/*` with `caching: True` (prompt caching), `extended_thinking: True`, `pdf: True`, `computer_use: True`
- `gemini/*` with `caching: True` (explicit cache), `grounding: True`, `video: True`, `audio: True`
- `deepseek/*` with `reasoning: True` (R1), `low_cost: True`

The implementations (`_send_anthropic`, `_send_gemini`, `_send_deepseek`) keep their unique per-vendor code paths. The matrix entries are the source of truth for the UI.

---

## Phase Plan (5 phases, 4 weeks of work)

### Phase 1: Tool Loop Lift (1-2 weeks)
- T1.1: Write red tests for `run_with_tool_loop` (5 tests covering: no tool calls returns immediately, tool calls dispatch, max rounds limit, history appending, error in tool call doesn't crash)
- T1.2: Implement `run_with_tool_loop` in `src/ai_client.py` (NOT a new file; per the naming convention HARD RULE)
- T1.3: Apply to `_send_minimax` (replace inline loop)
- T1.4: Apply to `_send_qwen`, `_send_grok`, `_send_llama` (add the missing loop)
- T1.5: Apply to `_send_anthropic`, `_send_gemini`, `_send_gemini_cli`, `_send_deepseek` (consolidate)
- T1.6: Verify all 8 vendors' existing tests still pass
- T1.7: Audit script `scripts/audit_no_inline_tool_loops.py` to enforce the pattern

### Phase 2: PROVIDERS Move (1 week)
- T2.1: Move `PROVIDERS` to `src/ai_client.py` (or new `src/ai_client_providers.py`)
- T2.2: Update all 5 import sites (gui_2.py, app_controller.py, etc.) to point to new location
- T2.3: Add `scripts/audit_providers_source_of_truth.py` to enforce the move
- T2.4: Verify all 38+ tests pass

### Phase 3: UX Adaptations 2-9 (1-2 weeks)
- T3.1: Apply adaptation 2 (tools toggle iff tool_calling)
- T3.2: Apply adaptation 3 (cache panel iff caching)
- T3.3: Apply adaptation 4 (stream progress iff streaming)
- T3.4: Apply adaptation 5 (fetch models iff model_discovery)
- T3.5: Apply adaptation 6 (token budget max = context_window)
- T3.6: Apply adaptation 7 (cost panel: estimate)
- T3.7: Apply adaptation 8 (cost panel: "Free (local)" for localhost)
- T3.8: Apply adaptation 9 (cost panel: "—" for other cost_tracking=false)
- T3.9: Verify live_gui tests pass

### Phase 4: Local-First + Matrix Expansion (1-2 weeks)
- T4.1: Add `local: bool` to VendorCapabilities; update registry for Llama
- T4.2: Native Ollama adapter (in `src/ai_client.py` as `ollama_chat` + `_send_llama_native`); replace OpenAI-compatible for Ollama backend
- T4.3: Meta Llama API adapter (in `src/ai_client.py` as `meta_llama_chat`); add as 4th Llama backend (DEFER if URL still 400)
- T4.4: GUI: "Local Model" badge
- T4.5: Add v2 fields (local, reasoning, structured_output, code_execution, web_search, x_search, file_search, mcp_support, audio, video, grounding, computer_use)
- T4.6: Update all vendor registry entries with the new fields
- T4.7: Add UI adaptations for the new fields (e.g., "Reasoning" toggle, "Code execution" panel)

### Phase 5: Anthropic / Gemini / DeepSeek Migration (1-2 weeks)
- T5.1: Populate Anthropic matrix entries (caching, extended_thinking, pdf, computer_use)
- T5.2: Populate Gemini matrix entries (caching, grounding, video, audio)
- T5.3: Populate DeepSeek matrix entries (reasoning, low_cost)
- T5.4: UI adaptations for the new capabilities
- T5.5: Docs + archive

---

## Testing Strategy

- All new helpers (`run_with_tool_loop`) get TDD: Red tests first, then implementation
- All UX adaptations get a test that verifies the render function reads the capability
- All audit scripts get a self-test (the script can detect its own absence)
- Live_gui tests run in batch (per the docs_sync lessons: bisect in batch, not isolation)

---

## Risks

- **Tool loop lift risk:** Anthropic and Gemini have unique tool-use formats (Anthropic uses `tool_use` blocks; Gemini uses `functionCall`). Lifting requires careful preservation. Mitigation: keep the per-vendor `tool_format_converter` injection as a parameter.
- **PROVIDERS move risk:** 5 import sites to update; some might use `from src.models import PROVIDERS` and break. Mitigation: search-and-replace audit, run full test suite after.
- **UX adaptation risk:** Same as parent Phase 5 — touching 260KB of GUI code is high risk. Mitigation: ship 1-2 per commit, run live_gui batch after each.

---

## Open Questions

1. **Meta Llama API spec verification:** The 400 error on `https://llama.developer.meta.com/docs/overview` last session. Re-verify on Phase 4 start. If still 400, **defer the Meta backend** to a separate follow-up; the native Ollama + 3 existing backends still ship.
2. **Local model as separate UI mode?** Should the GUI have a "Local / Cloud / All" filter on the provider dropdown, or just show the local badge per-vendor? Default: per-vendor badge (Phase 4 minimum). The filter is a future-track enhancement.
3. **PROVIDERS location:** **RESOLVED (2026-06-11):** `src/ai_client.py` (NOT a new `src/ai_client_providers.py`). The PROVIDERS list is small (8 entries); creating a new file for a single constant is over-engineering. The vendor list is logically part of the AI client.

---

## See Also

- Parent track: `conductor/tracks/qwen_llama_grok_integration_20260606/`
- Parent spec: `conductor/tracks/qwen_llama_grok_integration_20260606/spec.md`
- Parent Phase 5 report: `docs/reports/qwen_llama_grok_integration_20260610.md` (TBD)
- `docs/guide_ai_client.md` — the doc that needs updating in Phase 6 of the parent track

---

## Status

- T0: Spec drafted (this file)
- T1: Phase 1 (tool loop lift) ready to start
