ed/manual_slop
Private
Public Access
0
0
Fork 0
Code Activity

docs(workflow,agents): remove 'large files are bad' propaganda; add naming rule

Browse Source 
The user called out the LLM training data bias: 'small files are
good, large files are bad.' This is wrong for production codebases.
Unreal has 15K+ line files; OS kernels, game engines, compilers all
routinely have 10K+ line files. File size is a non-issue. Cognitive
load is managed via naming, regions, and navigation tools (the
manual-slop MCP) — NOT via file splitting.

Updates:

1. AGENTS.md (master agent guidance):
   - Added 'File Size and Naming Convention' section
   - Added the hard rule: 'New namespaced src/<thing>.py files may
     only be created on the user's explicit request. If you find
     yourself about to create one, ASK FIRST.'
   - Defaults: helpers and sub-systems go in the parent module

2. conductor/workflow.md (Guiding Principles):
   - Removed 'Do NOT perform large file writes directamente' from
     principle 7 (it was a delegating rule, but 'large file writes'
     carried the propaganda)
   - Added principle 8: 'File Naming Convention (HARD RULE)' that
     references AGENTS.md
   - Re-phrased principle 9 (Research-First) to clarify it's about
     navigation efficiency, not file size

3. conductor/code_styleguides/python.md:
   - Removed the 'extremely large files that violate the Anti-OOP
     rule by necessity' framing
   - Added the new rule about new src/<thing>.py files

4. .opencode/agents/tier3-worker.md and .opencode/agents/tier4-qa.md:
   - Re-phrased 'Do NOT read full large files' to 'Use skeleton
     tools to navigate any file regardless of size. File size is
     not a concern; the right tools are.'
   - Added the new rule about not creating new src/<thing>.py
     files unless user explicitly requests it

5. conductor/tracks/qwen_llama_grok_followup_20260611/plan.md:
   - Updated the 'Naming Convention' section to reference the new
     'user explicit request' rule

This is docs-only. No code changes. The rule is now codified:
agents must ASK FIRST before creating new top-level src/ files.
This commit is contained in:
ed
2026-06-11 10:07:07 -04:00
parent 4e4a56fd08
commit 51edbdef20
6 changed files with 151 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conductor/code_styleguides/python.md
+5 -1
View File
@@ -198,7 +198,11 @@ To minimize token usage and enhance visual scanning for human reviewers, heavily
## 14. Logical Region Blocks
For extremely large files that violate the "Anti-OOP" rule by necessity (e.g., `App` class holding global UI state), use `#region: Section Name` and `#endregion: Section Name` tags (or `# --- Section Name ---` for visual grouping) to strictly organize methods and state properties. This establishes a predictable structure that MCP tools and agents can leverage for contextual masking.
For files where many related methods/properties live in a single class (e.g., the `App` class in `src/gui_2.py` holding global UI state; the `src/ai_client.py` module holding 8 vendor entry points and supporting machinery), use `#region: Section Name` and `#endregion: Section Name` tags (or `# --- Section Name ---` for visual grouping) to strictly organize methods and state properties. This establishes a predictable structure that MCP tools and agents can leverage for contextual masking.
**Removed anti-pattern (2026-06-11):** the prior version of this section said "extremely large files that violate the Anti-OOP rule by necessity." That framing was wrong. Files are not "large" in any absolute sense; production codebases (Unreal, OS kernels, game engines) routinely have 10K+ line files. The "Anti-OOP" rule is about data-vs-behavior separation, not file size. The `App` class in `src/gui_2.py` is not "violating" anything by being large; it's the natural shape of a class that owns the GUI orchestration. The `#region` convention is for navigability, not as a workaround for "files that got too big."
**Hard rule on new `src/<thing>.py` files (added 2026-06-11):** New namespaced `src/<thing>.py` files may only be created on the user's explicit request. If you find yourself about to create one, ASK FIRST — don't just create it. Rationale: the user is the only one who can authorize a new top-level namespace. Defaults: helpers and sub-systems go in the parent module. E.g., AI-client-specific helpers go in `src/ai_client.py`; app-controller helpers go in `src/app_controller.py`; MCP-client helpers go in `src/mcp_client.py`. Even if the parent file is already 3K+ lines, the helper still goes there. If a new top-level `src/<thing>.py` is genuinely warranted (e.g., a truly new system that doesn't fit any existing parent), propose it in the next checkpoint or status note and wait for the user's explicit "yes, create it." See `AGENTS.md` "File Size and Naming Convention" for the full rule.
## 15. Modular Controller Pattern
conductor/tracks/qwen_llama_grok_followup_20260611/plan.md
+118 -35
View File
@@ -12,25 +12,116 @@
---
## Naming Convention (HARD RULE)
**The user's directive (2026-06-11):** `src/ai_client.py` is the AI vendor/API system layer. All new AI-client-related code lives IN `src/ai_client.py`, not in new `src/` files. The only new files in this track are:
- `scripts/audit_*.py` (audit scripts, namespace-isolated by directory)
- `tests/test_*.py` (test files, namespace-isolated by directory)
- `docs/*.md` (documentation)
**New `src/<thing>.py` files may only be created on the user's explicit request.** If you find yourself about to create one, **ASK FIRST** — don't just create it. See `AGENTS.md` "File Size and Naming Convention" for the full rule.
**No new `src/*.py` files** in this follow-up. Everything else goes into `src/ai_client.py`:
- `run_with_tool_loop` function (lives alongside the existing `_execute_tool_calls_concurrently` at `src/ai_client.py:754`)
- `ollama_chat` function (lives alongside the existing `_send_llama` at `src/ai_client.py:~2400`)
- `meta_llama_chat` function (lives alongside `ollama_chat`)
The agent (LLM) may push back on this rule if there's a strong reason (e.g., a helper needs to be imported by another module). For AI-client internals, there is no such reason — `ai_client.py` is already the single source of truth for the system.
**Audit of existing sprawl (separate track):** the parent track created 3 stray `src/` files that should arguably be in `src/ai_client.py`:
- `src/vendor_capabilities.py` (Phase 1)
- `src/openai_compatible.py` (Phase 1)
- `src/qwen_adapter.py` (Phase 2)
The follow-up track does NOT clean these up. A separate track `namespace_cleanup_20260611` is proposed (deferred; not in scope) to audit and consolidate ALL such sprawl in the codebase.
---
## File Structure (delta from parent track)
**No new `src/*.py` files.** All implementation code goes into `src/ai_client.py`.
| File | Action | Responsibility |
|---|---|---|
| `src/tool_loop.py` | Create | `run_with_tool_loop(client, request, capabilities, *, pre_tool_callback, qa_callback, patch_callback, base_dir, vendor_name, history_lock, history, trim_func=None) -> str` |
| `src/llama_ollama_native.py` | Create | Native Ollama adapter (`/api/chat`); replaces OpenAI-compatible for Ollama backend |
| `src/llama_meta_api.py` | Create | Meta Llama API adapter; new 4th backend |
| `src/ai_client.py` | Modify | (1) Add `run_with_tool_loop` function; (2) add `ollama_chat` and `meta_llama_chat` functions; (3) add v2 matrix fields handling; (4) add `_send_llama_native` and route Ollama to it; (5) move PROVIDERS from `src/models.py`; (6) apply `run_with_tool_loop` to all 8 vendors |
| `src/vendor_capabilities.py` | Modify | Add 12 v2 fields; populate v2 entries for all vendors (this file is in scope to modify but NOT in scope to delete; cleanup is the `namespace_cleanup_20260611` track's job) |
| `src/models.py` | Modify | Remove `PROVIDERS` declaration (moves to `src/ai_client.py`); keep re-export shim for backward compat |
| `src/gui_2.py` | Modify | Apply UX adaptations 2-9; add "Local Model" badge |
| `scripts/audit_no_inline_tool_loops.py` | Create | Fails if any `_send_<vendor>()` has an inline tool loop |
| `scripts/audit_providers_source_of_truth.py` | Create | Fails if `PROVIDERS` is declared in `src/models.py` |
| `tests/test_tool_loop.py` | Create | 5+ tests for `run_with_tool_loop` |
| `tests/test_llama_ollama_native.py` | Create | 3+ tests for native Ollama adapter |
| `tests/test_llama_meta_api.py` | Create | 2+ tests for Meta Llama API adapter |
| `src/ai_client.py` | Modify | (1) Use `run_with_tool_loop` in all 8 vendors; (2) move PROVIDERS; (3) add native Ollama + Meta Llama; (4) add v2 matrix fields |
| `src/vendor_capabilities.py` | Modify | Add 12 v2 fields; populate v2 entries for all vendors |
| `src/models.py` | Modify | Remove `PROVIDERS` (moves to `src/ai_client.py` or `src/ai_client_providers.py`); keep re-export shim for backward compat |
| `src/gui_2.py` | Modify | Apply UX adaptations 2-9; add "Local Model" badge |
| `tests/test_ai_client_tool_loop.py` | Create | 5+ tests for `run_with_tool_loop` (imported from `src.ai_client`) |
| `tests/test_ai_client_llama_ollama_native.py` | Create | 3+ tests for native Ollama adapter (imported from `src.ai_client`) |
| `docs/guide_ai_client.md` | Modify (Phase 5) | Document `run_with_tool_loop`; v2 fields; local-first |
| `docs/guide_models.md` | Modify (Phase 5) | Document new PROVIDERS location; v2 fields |
**Why not namespace via sub-directory?** Python doesn't support sub-directories for projects well. The user explicitly said the project can't lean on sub-directory organization. The convention is: large files are OK; agents should use the manual-slop MCP (`get_file_slice`, `get_file_summary`, `py_get_skeleton`, `py_get_code_outline`) to navigate efficiently.
---
## Functions to Add to `src/ai_client.py`
### `run_with_tool_loop` (Phase 1)
Add near `_execute_tool_calls_concurrently` at `src/ai_client.py:754`. Signature:
```python
def run_with_tool_loop(
client: Any,
request: OpenAICompatibleRequest,
*,
capabilities: VendorCapabilities,
pre_tool_callback: Optional[Callable] = None,
qa_callback: Optional[Callable] = None,
stream_callback: Optional[Callable[[str], None]] = None,
patch_callback: Optional[Callable] = None,
base_dir: str,
vendor_name: str,
history_lock: Optional[threading.Lock] = None,
history: Optional[list] = None,
trim_func: Optional[Callable] = None,
) -> str:
# ... (see Task 1.2 for body) ...
```
### `ollama_chat` (Phase 4)
Add near `_send_llama` at `src/ai_client.py:~2400`. Signature:
```python
def ollama_chat(
model: str,
messages: list[dict[str, Any]],
*,
think: str = "low",
images: Optional[list[str]] = None,
tools: Optional[list[dict[str, Any]]] = None,
base_url: str = "http://localhost:11434",
timeout: int = 120,
) -> dict[str, Any]:
payload: dict[str, Any] = {"model": model, "messages": messages, "stream": False}
if think:
payload["think"] = think
if images:
payload["images"] = images
if tools:
payload["tools"] = tools
resp = requests.post(f"{base_url}/api/chat", json=payload, timeout=timeout)
return resp.json()
```
### `meta_llama_chat` (Phase 4; deferred to a follow-up)
Add near `ollama_chat`. **Prerequisite:** verify the Meta Llama API docs URL (https://llama.developer.meta.com/docs/overview) is reachable — it returned 400 in the parent's session. If unreachable, defer the Meta backend to a separate follow-up track; in this track only the native Ollama adapter ships.
### `_send_llama_native` (Phase 4)
Add in `src/ai_client.py` near `_send_llama`. Routes Ollama backend to native adapter:
```python
def _send_llama_native(md_content, user_message, base_dir, file_items=None, discussion_history="", stream=False, pre_tool_callback=None, qa_callback=None, stream_callback=None, patch_callback=None) -> str:
base_url = _llama_base_url.replace("/v1", "")
# ... (uses ollama_chat; see Task 4.3 for body) ...
```
Then modify `_send_llama` to route localhost/127.0.0.1 to `_send_llama_native`:
---
# Phase 1: Tool Loop Lift
@@ -41,7 +132,7 @@
## Task 1.1: Write red tests for `run_with_tool_loop`
**Files:** Create `tests/test_tool_loop.py`
**Files:** Create `tests/test_ai_client_tool_loop.py`
- [ ] **Step 1: Create the file with 5 tests**
@@ -49,7 +140,7 @@
from unittest.mock import MagicMock, patch
import pytest
from src.openai_compatible import NormalizedResponse, OpenAICompatibleRequest
from src.tool_loop import run_with_tool_loop
from src.ai_client import run_with_tool_loop
from src.vendor_capabilities import VendorCapabilities
@pytest.fixture
@@ -147,39 +238,29 @@ def test_run_with_tool_loop_does_not_crash_on_tool_error(caps: VendorCapabilitie
- [ ] **Step 2: Run, confirm 5 tests fail with ModuleNotFoundError**
```
cd C:\projects\manual_slop && uv run pytest tests/test_tool_loop.py -v
cd C:\projects\manual_slop && uv run pytest tests/test_ai_client_tool_loop.py -v
```
Expected: 5 failed, all with `ModuleNotFoundError: No module named 'src.tool_loop'`.
Expected: 5 failed, all with `AttributeError: module 'src.ai_client' has no attribute 'run_with_tool_loop'`.
- [ ] **Step 3: Commit (red)**
```bash
git add tests/test_tool_loop.py
git commit -m "test(tool_loop): add red tests for run_with_tool_loop shared helper"
git add tests/test_ai_client_tool_loop.py
git commit -m "test(ai_client): add red tests for run_with_tool_loop shared helper"
```
---
## Task 1.2: Implement `src/tool_loop.py`
## Task 1.2: Implement `run_with_tool_loop` in `src/ai_client.py`
**Files:** Create `src/tool_loop.py`
**Files:** Modify `src/ai_client.py` (add function near `_execute_tool_calls_concurrently` at line 754)
- [ ] **Step 1: Create the file**
- [ ] **Step 1: Add the function**
Find a good location — right after `_execute_tool_calls_concurrently` at `src/ai_client.py:801` (or wherever the function ends). Insert:
```python
from __future__ import annotations
import asyncio
import threading
from typing import Any, Callable, Optional
from src.openai_compatible import OpenAICompatibleRequest, send_openai_compatible
from src.vendor_capabilities import VendorCapabilities
from src.ai_client import (
MAX_TOOL_ROUNDS,
_execute_tool_calls_concurrently,
)
def run_with_tool_loop(
client: Any,
request: OpenAICompatibleRequest,
@@ -234,10 +315,12 @@ def run_with_tool_loop(
return response_text
```
Use `manual-slop_py_update_definition` to add this function to `src/ai_client.py`. The new function will appear right after the existing `_execute_tool_calls_concurrently`.
- [ ] **Step 2: Run the 5 Red tests; confirm they now pass**
```
cd C:\projects\manual_slop && uv run pytest tests/test_tool_loop.py -v
cd C:\projects\manual_slop && uv run pytest tests/test_ai_client_tool_loop.py -v
```
Expected: 5 passed.
@@ -245,8 +328,8 @@ Expected: 5 passed.
- [ ] **Step 3: Commit (green)**
```bash
git add src/tool_loop.py
git commit -m "feat(tool_loop): add run_with_tool_loop shared helper for all 8 vendors"
git add src/ai_client.py
git commit -m "feat(ai_client): add run_with_tool_loop shared helper for all 8 vendors"
```
---
@@ -1244,7 +1327,7 @@ git commit -m "feat(capability_matrix): populate v2 fields for all 22 existing e
- [ ] **Step 1: Run regression batch (38+ tests + the 3 new ollama tests)**
```
cd C:\projects\manual_slop && uv run pytest tests/test_tool_loop.py tests/test_llama_ollama_native.py tests/test_qwen_provider.py tests/test_grok_provider.py tests/test_llama_provider.py tests/test_minimax_provider.py tests/test_vendor_capabilities.py tests/test_openai_compatible.py tests/test_cost_tracker.py -q
cd C:\projects\manual_slop && uv run pytest tests/test_ai_client_tool_loop.py tests/test_ai_client_llama_ollama_native.py tests/test_qwen_provider.py tests/test_grok_provider.py tests/test_llama_provider.py tests/test_minimax_provider.py tests/test_vendor_capabilities.py tests/test_openai_compatible.py tests/test_cost_tracker.py -q
```
Expected: 41+ passed (3 new + 38 existing).
conductor/workflow.md
+2 -1
View File
@@ -40,7 +40,8 @@ with open('file.py', 'w', encoding='utf-8', newline='') as f:
4. **High Code Coverage:** Aim for >80% code coverage for all modules
5. **User Experience First:** Every decision should prioritize user experience
6. **Non-Interactive & CI-Aware:** Prefer non-interactive commands. Use `CI=true` for watch-mode tools (tests, linters) to ensure single execution.
7. **MMA Tiered Delegation is Mandatory:** The Conductor acts as a Tier 1/2 Orchestrator. You MUST delegate all non-trivial coding to Tier 3 Workers and all error analysis to Tier 4 QA Agents. Do NOT perform large file writes directly.
7. **MMA Tiered Delegation is Mandatory:** The Conductor acts as a Tier 1/2 Orchestrator. You MUST delegate all non-trivial coding to Tier 3 Workers and all error analysis to Tier 4 QA Agents. Do NOT write non-trivial code directly.
8. **File Naming Convention (HARD RULE, added 2026-06-11):** New `src/<thing>.py` files may only be created on the user's explicit request. Helpers and sub-systems go in the parent module. E.g., AI-client-specific code goes in `src/ai_client.py`; MCP-client code goes in `src/mcp_client.py`. If you find yourself about to create a new `src/<thing>.py` file, ASK FIRST. See `AGENTS.md` "File Size and Naming Convention" for the full rule.
8. **Mandatory Research-First Protocol:** Before reading the full content of any file over 50 lines, you MUST use `get_file_summary`, `py_get_skeleton`, `py_get_code_outline`, or `py_get_docstring` to map the architecture and identify specific target ranges. Use `get_git_diff` to understand recent changes. Use `py_find_usages` to locate where symbols are used.
9. **Architecture Documentation Fallback:** When uncertain about threading, event flow, data structures, or module interactions, consult the deep-dive docs in `docs/` (last refreshed: 2026-06-02 via the comprehensive documentation refresh track, **8 new guides added**):
- **[docs/guide_architecture.md](../docs/guide_architecture.md):** Thread domains, cross-thread patterns, AI client multi-provider (Gemini, Anthropic, DeepSeek, Gemini CLI, MiniMax), HITL Execution Clutch.