# Qwen/Llama/Grok Follow-Up Audit Report (2026-06-11)

**Date:** 2026-06-11
**Author:** Tier 2 Tech Lead
**Subject:** Why a follow-up track is needed after `qwen_llama_grok_integration_20260606` Phase 5

## TL;DR

The parent track shipped 5 of 6 phases with 50/79 tasks done. The Tech Lead **did not surface the gaps at the checkpoints**; the user discovered them only at the Phase 5 checkpoint. The user is right: the Tech Lead's "footnote for now" pattern is bad — it looks like the work was hidden until called out.

**7 categories of gap** are documented here. Each is captured in the new follow-up track `qwen_llama_grok_followup_20260611`.

---

## 1. Phase 5 partial: 1 of 9 UX adaptations shipped

**What shipped:** Adaptation 1 (Screenshot button iff vision) at `src/gui_2.py:3030` + the helper `_get_active_capabilities()` at `src/gui_2.py:733`.

**What didn't ship:** Adaptations 2-9:
- Tools toggle iff tool_calling
- Cache panel iff caching
- Stream progress iff streaming
- Fetch Models button iff model_discovery
- Token budget max = context_window
- Cost panel × 3 (estimate / "Free (local)" for localhost / "—" for other cost_tracking=false)

**The right move:** All 9 at once, OR explicit user-facing "I'm shipping 1 of 9; the other 8 are deferred" BEFORE doing adaptation 1. The Tech Lead did the latter in a footnote, which the user called out as bad UX.

---

## 2. Tool-call loop regression: only MiniMax works

**What shipped:** `_send_minimax` has a working tool loop. The other 7 vendor entry points do not.

| Vendor | Tool loop? | Why |
|---|---|---|
| `_send_minimax` | ✅ Works (231 → 75 lines after refactor + tool loop restoration) | Worker did the refactor; I added the tool loop back manually |
| `_send_qwen` | ❌ Single-shot | Phase 2 worker omitted it (Qwen has DashScope-specific tool format) |
| `_send_grok` | ❌ Single-shot | Phase 3 worker omitted it (placeholder) |
| `_send_llama` | ❌ Single-shot | Phase 3 worker omitted it (placeholder) |
| `_send_anthropic` | ✅ Inline (4-way duplication with the other 3) | Pre-existing pattern |
| `_send_gemini` | ✅ Inline | Pre-existing pattern |
| `_send_gemini_cli` | ✅ Inline | Pre-existing pattern |
| `_send_deepseek` | ✅ Inline | Pre-existing pattern |

**The right move:** Lift the loop into a shared `run_with_tool_loop` helper that takes history management as injected parameters. Apply to all 8 vendors. This is a single-fix, 8-call-site refactor — much smaller than letting the duplication grow.

The Tech Lead caught this at the end of Phase 4 (during the MiniMax refactor) but should have caught it at the end of Phase 2 (when the Qwen worker shipped single-shot) or the end of Phase 3 (when Grok+Llama workers shipped single-shot).

---

## 3. `src/models.py` has a PROVIDERS list — the user is right that this is sprawl

**What's there now:**
```python
# src/models.py:79
PROVIDERS: List[str] = ["gemini", "anthropic", "gemini_cli", "deepseek", "minimax", "qwen", "grok", "llama"]
```

**The problem:** `src/models.py` is for **MMA data models** (Tickets, Tracks, FileItem, WorkerContext, etc.). The vendor list is an **AI client concern**. The audit script `audit_no_models_config_io.py` enforces config I/O rules; PROVIDERS has no analogous enforcement.

**The right move:** Move PROVIDERS to `src/ai_client.py` (or a new `src/ai_client_providers.py`). Add `scripts/audit_providers_source_of_truth.py` that fails the build if PROVIDERS is declared in models.py.

The Tech Lead justified keeping it in models.py with "the centralized registry pattern" without asking whether models.py was the right home.

---

## 4. `src/ai_client.py` is 2784 lines and growing

**What's there:** 8 vendor entry points (`_send_anthropic`, `_send_gemini`, `_send_gemini_cli`, `_send_deepseek`, `_send_minimax`, `_send_qwen`, `_send_grok`, `_send_llama`) plus all the supporting machinery (client init, history management, error classification, reasoning content extraction).

**The 8 vendors' inline patterns are 70% similar.** Each has:
- Client init (credentials + SDK setup)
- History management (per-vendor lock + history list + repair + trim)
- Message building (system + context + user content)
- API call (via SDK or HTTP)
- Tool loop (or single-shot — see gap #2)
- Reasoning content extraction
- Error classification

**The right move:** Codepath consolidation. The shared `send_openai_compatible` covers the API call. A future `run_with_tool_loop` covers the tool loop (gap #2). What's left:
- History management as a `VendorHistory` class or per-vendor thin wrapper
- Reasoning content extraction as a uniform helper
- Error classification as a per-HTTP-code helper

Could cut `src/ai_client.py` by 30-40% (~1000 lines).

---

## 5. Local models deserve more emphasis

**What's there now:** Ollama is one of 3 Llama backends (Ollama, OpenRouter, custom_url). The `cost_tracking: False` for localhost is a small signal.

**The user feedback (verbatim):** "I want to put more emphasis and supporting local models and separating local model vending vis online/cloud vendors of models."

**The right architecture:**
- Add `local: bool` to VendorCapabilities (separate from `cost_tracking`)
- Native Ollama (`/api/chat`) as the **default** for Llama (not the OpenAI-compatible fallback)
- Meta Llama API as a 4th backend (the docs URL returned 400 last session; needs re-verification)
- GUI: "Local Model" badge per-vendor
- Cost panel: 4th state "Local (no cost)" distinct from "Free (local)" and "—"
- vLLM, LM Studio, llama.cpp as additional custom-URL backends with discoverable presets

This is a significant priority shift. The follow-up track's Phase 4 leads with this.

---

## 6. V2 matrix field expansion documented but not implemented

**What the spec says (per Grok's consultation):** Add 12 new fields to VendorCapabilities:
- `local: bool`
- `reasoning: bool` (xAI `reasoning_effort`, Anthropic extended thinking, Ollama `think`)
- `structured_output: bool` (response_format / format)
- `code_execution: bool` (xAI code_interpreter, Anthropic Computer Use, Gemini Code Execution)
- `web_search: bool` (xAI web_search, Gemini Grounding)
- `x_search: bool` (xAI X/Twitter search)
- `file_search: bool` (xAI file_search, Anthropic PDF, Gemini file API)
- `mcp_support: bool` (xAI mcp_calls, Anthropic MCP)
- `audio: bool` (Qwen-Audio, Gemini audio)
- `video: bool` (Gemini video)
- `grounding: bool` (Gemini Grounding with Google Search)
- `computer_use: bool` (Anthropic Computer Use)

**What shipped:** 0 of 12. None wired. No UI adaptations.

The follow-up track's Phase 4 lands these.

---

## 7. Anthropic / Gemini / DeepSeek still not on the matrix

**What's there:** These 3 vendors have unique APIs (4-breakpoint caching, genai SDK, raw HTTP) and the migration to the matrix is non-trivial. The follow-up track is documented (`parent spec §13.1.A`) but never scheduled.

**The value:** Anthropic has prompt caching, extended thinking, Computer Use (big UX wins). Gemini has Grounding with Google Search, native video. DeepSeek has reasoning models.

The follow-up track's Phase 5 lands these.

---

## Lessons (Tech Lead Process)

1. **Surface gaps as they appear, not at the checkpoint.** If a task is going to be deferred mid-phase, say so immediately — don't footnote it later.
2. **Be explicit about architectural deviations.** The `src/models.py` PROVIDERS sprawl should have been raised at Phase 2, not at Phase 5.
3. **Plan for the test infrastructure before coding.** The tool-loop regression wasn't caught because no test exercised the loop.
4. **The "footnote for now" pattern is bad UX.** It looks like the work was hidden until called out. Either ship the work or be explicit about deferring it BEFORE doing the work.

## Follow-Up Track

`conductor/tracks/qwen_llama_grok_followup_20260611/` — 5 phases:
- Phase 1: Tool loop lift (run_with_tool_loop helper for 8 vendors)
- Phase 2: PROVIDERS move (out of src/models.py)
- Phase 3: UX adaptations 2-9 (8 of 9 deferred from parent Phase 5)
- Phase 4: Local-first + matrix v2 expansion (12 new fields)
- Phase 5: Anthropic / Gemini / DeepSeek migration

## Parent Track Status

`qwen_llama_grok_integration_20260606` is **NOT being archived** (per user directive). It stays open in `conductor/tracks/` for the follow-up to use as a reference. Phase 6 docs are being done now; the track folder remains at the same path.

## See Also

- `conductor/tracks/qwen_llama_grok_followup_20260611/spec.md` — the follow-up spec
- `conductor/tracks/qwen_llama_grok_followup_20260611/state.toml` — the follow-up state
- `conductor/tracks/qwen_llama_grok_followup_20260611/TODO.md` — the setup checklist
- `conductor/tracks/qwen_llama_grok_integration_20260606/` — the parent track
