diff --git a/conductor/code_styleguides/python.md b/conductor/code_styleguides/python.md index 34d04674..b1821814 100644 --- a/conductor/code_styleguides/python.md +++ b/conductor/code_styleguides/python.md @@ -137,16 +137,16 @@ When refactoring a class to functions: | Class | File:Line | State held | |---|---|---| -| `App` | `src/gui_2.py:307` | GUI state (show_windows, active_discussion, disc_entries), delegation proxies | -| `AppController` | `src/app_controller.py:795` | 11 locks, all subsystem managers, presets/personas/RAG state | +| `App` | `src/gui_2.py:314` | GUI state (show_windows, active_discussion, disc_entries), delegation proxies | +| `AppController` | `src/app_controller.py:801` | 11 locks, all subsystem managers, presets/personas/RAG state | | `ConductorEngine` | `src/multi_agent_conductor.py:112` | TrackDAG, ExecutionEngine, WorkerPool, tier_usage | | `WorkerPool` | `src/multi_agent_conductor.py:52` | active workers dict, semaphore, lock | -| `RAGEngine` | `src/rag_engine.py:123` | embedding provider, chroma client/collection | +| `RAGEngine` | `src/rag_engine.py:125` | embedding provider, chroma client/collection | | `BaseEmbeddingProvider` + subclasses (`LocalEmbeddingProvider`, `GeminiEmbeddingProvider`) | `src/rag_engine.py:74,78,87` | loaded model state | | `EventEmitter` | `src/events.py:40` | listeners dict | | `AsyncEventQueue` | `src/events.py:77` | asyncio.Queue | | `HistoryManager` | `src/history.py:71` | undo/redo stack (100-snapshot capacity) | -| `HookServer` + `HookServerInstance` + `HookHandler` + `WebSocketServer` | `src/api_hooks.py:856,130,155,908` | HTTP server thread, port binding, event queue | +| `HookServer` + `HookServerInstance` + `HookHandler` + `WebSocketServer` | `src/api_hooks.py:941,171,208,993` | HTTP server thread, port binding, event queue | | `HotReloader` + `HotModule` | `src/hot_reloader.py:21,15` | HOT_MODULES registry, last_error, is_error_state | **NOT exempt** (these are dataclasses / data carriers / context managers, not stateful subsystems): diff --git a/docs/Readme.md b/docs/Readme.md index 631d2d05..d7e009e2 100644 --- a/docs/Readme.md +++ b/docs/Readme.md @@ -35,8 +35,8 @@ This documentation suite provides comprehensive technical reference for the Manu | [API Hooks](guide_api_hooks.md) | `src/api_hooks.py` + `src/api_hook_client.py` reference: HookServer on `127.0.0.1:8999`, ApiHookClient Python wrapper, 8+ endpoints (`/status`, `/api/gui`, `/api/ask`, `/api/gui/mma_status`, `/api/performance`, `/api/comms`, `/api/diagnostics`), Remote Confirmation Protocol via `/api/ask` (synchronous blocking HITL), `custom_callback` action for invoking any registered App method | | [MCP Client](guide_mcp_client.md) | `src/mcp_client.py` reference: 45 native tools (File I/O, Python AST, C/C++ AST, Analysis, Network, Runtime, Beads), 3-layer security model (Allowlist Construction, Path Validation, Resolution Gate), `dispatch()`/`async_dispatch()` entry points, ExternalMCPManager for external MCP servers (Stdio + SSE), JSON-RPC 2.0 engine, public API, configuration | | [App Controller](guide_app_controller.md) | `src/app_controller.py` reference: headless orchestrator owning AppState and all subsystem managers (PresetManager, PersonaManager, ContextPresetManager, ToolPresetManager, ToolBiasEngine, RAGEngine, HistoryManager, WorkspaceManager, HookServer, HotReloader, PathManager), `_predefined_callbacks` and `_gettable_fields` registries for Hook API, SyncEventQueue bridge, preset/persona/context coordination, headless mode | -| [MMA Engine](guide_multi_agent_conductor.md) | `src/multi_agent_conductor.py` + `src/dag_engine.py` reference: TrackDAG with cycle detection (iterative DFS) and topological sort (Kahn's variant), ExecutionEngine with Auto-Queue / Step Mode state machine, MultiAgentConductor with WorkerPool (configurable concurrency, default 4), mma_exec.py sub-agent invocation for Token Firewall, parse_plan_md utility, Beads mode delegation | -| [Data Models](guide_models.md) | `src/models.py` reference: centralized data model registry using pydantic + dataclasses, model categories (Core, AI, Preset, Persona, Context, MMA, UI State, Logging, Hook, Workspace, RAG), `AGENT_TOOL_NAMES` canonical 45-tool list, `PROVIDERS` constant, `parse_plan_md` utility, validation patterns, SDM tags, serialization strategies (TOML, JSON-L) | +| [MMA Engine](guide_multi_agent_conductor.md) | `src/multi_agent_conductor.py` + `src/dag_engine.py` reference: TrackDAG with cycle detection (iterative DFS) and topological sort (Kahn's variant), ExecutionEngine with Auto-Queue / Step Mode state machine, MultiAgentConductor with WorkerPool (configurable concurrency, default 4), the WorkerPool's internal `run_worker_lifecycle` subprocess template (NOT the meta-tooling `mma_exec.py` — that's deprecated; see `guide_meta_boundary.md`), parse_plan_md utility (now in `src/mma.py`), Beads mode delegation | +| [Data Models](guide_models.md) | `src/models.py` is now a ~1.5KB legacy re-export shim (`Metadata = TrackMetadata` alias + `PROVIDERS` lazy `__getattr__`). Data models moved to per-system files per `module_taxonomy_refactor_20260627`: `src/mma.py` (TrackMetadata, Ticket, Track, WorkerContext), `src/project_files.py` (FileItem), `src/type_aliases.py` (typed boundary + per-aggregate dataclasses: Metadata, CommsLogEntry, HistoryMessage, ToolDefinition, SessionInsights, DiscussionSettings, CustomSlice, MMAUsageStats, ProviderPayload, UIPanelConfig, PathInfo, FileItemsDiff, JsonPrimitive/JsonValue), `src/mcp_tool_specs.py` (typed ToolSpec registry, 45 tools), `src/result_types.py` (Result[T], ErrorInfo, ErrorKind). `VendorCapabilities` lives in `src/ai_client.py` `#region: Vendor Capabilities`. `PROVIDERS` constant in `src/ai_client.py` (8 providers: gemini, anthropic, gemini_cli, deepseek, minimax, qwen, grok, llama). | | [Discussions](guide_discussions.md) | The Discussion system: 23-operation matrix A1-A7 (per-entry) + B1-B11 (discussion-level) + C1-C5 (undo/redo), Take naming convention (`_take_`), branching at any entry (`project_manager.branch_discussion`), promotion to top-level (`project_manager.promote_take`), user-managed role list (`app.disc_roles`), per-role filter linked to MMA persona focus, `_disc_entries_lock` thread-safety contract, Hook API session endpoints | | [State Lifecycle](guide_state_lifecycle.md) | Undo/redo via `HistoryManager` + `UISnapshot` (13 captured fields, 100-snapshot capacity, debounced change detection at render frame), reset flow (`_handle_reset_session` — clears 30+ fields, replaces project, preserves `active_project_path` per the 2026-06-08 regression fix), `App.__getattr__`/`__setattr__` state delegation to Controller, 8-thread io_pool with 11 lock-protected regions (per `IO_POOL_MAX_WORKERS = 8` in `src/io_pool.py:20`; bumped 4→8 in 4a338486 on 2026-06-06), hot-reload integration | | [Context Aggregation](guide_context_aggregation.md) | The `aggregate.py` (518-line) pipeline: 3 aggregation strategies (`auto`/`summarize`/`full`), 7 per-file view modes (`full`/`summary`/`skeleton`/`outline`/`masked`/`custom`/`none`), full `FileItem` schema (9 fields + `__post_init__` normalizer), `ContextPreset` schema and `ContextPresetManager`, Tier 3 worker variant (`build_tier3_context` with FuzzyAnchor re-resolution and focus-file handling), `force_full`/`auto_aggregate` short-circuits, output file numbering, cache strategy (static prefix + dynamic history) | diff --git a/docs/guide_agent_memory_dimensions.md b/docs/guide_agent_memory_dimensions.md index 9a6b926a..9505a058 100644 --- a/docs/guide_agent_memory_dimensions.md +++ b/docs/guide_agent_memory_dimensions.md @@ -77,7 +77,7 @@ A `ContextPreset` is a named, persisted set of `FileItem`s. Both persist in the **The shape rule.** Curation is per-file, per-discussion, structural. Edited at the Structural File Editor. Persisted in TOML. The file's `FileItem` is the single source of truth for "how do I render this file in the AI's context." -**See:** `docs/guide_context_curation.md`; `src/models.py:510-559` (FileItem schema); `src/context_presets.py` (ContextPresetManager). +**See:** `docs/guide_context_curation.md`; `src/project_files.py` (FileItem dataclass, moved out of `src/models.py` per `module_taxonomy_refactor_20260627`); `src/context_presets.py` (ContextPresetManager). --- diff --git a/docs/guide_ai_client.md b/docs/guide_ai_client.md index 0559de92..a4b56350 100644 --- a/docs/guide_ai_client.md +++ b/docs/guide_ai_client.md @@ -21,7 +21,7 @@ The OpenAI-compatible vendors all call the shared helper in `src/openai_compatib ## Module-Level Imports -> **Important:** The 5 provider SDKs are **NOT** imported at module level. `import google.genai`, `import anthropic`, `import openai`, and `import fastapi` are heavy (~430-955ms each on cold load) and are now obtained via `src.module_loader._require_warmed("google.genai")` and similar calls, after the `WarmupManager` has loaded them in the background. The module-level globals you see in the State section (`_gemini_client`, `_anthropic_client`, etc.) are typed as `Optional` because they're populated by `_require_warmed()` on first use, not at import time. +> **Important:** The provider SDKs are **NOT** imported at module level. `import google.genai`, `import anthropic`, `import openai`, `import dashscope`, and `import fastapi` are heavy (~430-955ms each on cold load) and are now obtained via `src.module_loader._require_warmed("google.genai")` and similar calls, after the `WarmupManager` has loaded them in the background. The module-level globals you see in the State section (`_gemini_client`, `_anthropic_client`, etc.) are typed as `Optional` because they're populated by `_require_warmed()` on first use, not at import time. (Updated 2026-07-02: there are 8 providers, not 5 — the original "5 SDKs" count predated the qwen/grok/llama additions.) This change was part of the 2026-06-06 `startup_speedup_20260606` track. Before: `import src.ai_client` took ~1800ms. After: ~161ms. The remaining cost is the bare module skeleton. @@ -52,7 +52,7 @@ All state is module-level globals. The most important: | Variable | Type | Purpose | |---|---|---| -| `_provider: str` | `"gemini" \| "anthropic" \| "deepseek" \| "minimax" \| "gemini_cli"` | Active provider | +| `_provider: str` | `"gemini" \| "anthropic" \| "gemini_cli" \| "deepseek" \| "minimax" \| "qwen" \| "grok" \| "llama"` | Active provider (8 total) | | `_model: str` | `str` | Active model name | | `_temperature: float` | `0.0` | Sampling temperature | | `_top_p: float` | `1.0` | Nucleus sampling | @@ -129,10 +129,13 @@ Returns the model's response as a `Result[str, ErrorInfo]` (the data-oriented er ```python from src import ai_client ai_client.set_provider("gemini", "gemini-3-flash-preview") -ai_client.set_provider("anthropic", "claude-3-5-sonnet-latest") -ai_client.set_provider("deepseek", "deepseek-chat") -ai_client.set_provider("minimax", "grok-2-latest") -ai_client.set_provider("gemini_cli", "gemini-2.0-flash") +ai_client.set_provider("anthropic", "claude-sonnet-4-5-20250929") +ai_client.set_provider("deepseek", "deepseek-v3") +ai_client.set_provider("minimax", "MiniMax-M2") +ai_client.set_provider("gemini_cli", "gemini-2.5-flash") +ai_client.set_provider("qwen", "qwen-plus") +ai_client.set_provider("grok", "grok-2") +ai_client.set_provider("llama", "llama-3.1-8b-instant") ``` ### Parameter Setters @@ -686,9 +689,9 @@ Added 2026-06-11. The `VendorCapabilities` dataclass in `src/vendor_capabilities ### PROVIDERS Location (Phase 2) -The `PROVIDERS` list moved from `src/models.py` to `src/ai_client.py:56` per the AGENTS.md HARD RULE (no new `src/.py` files). A PEP 562 `__getattr__` re-export in `src/models.py:261` maintains backward compatibility (lazy import; breaks the circular dependency where `src/ai_client.py` imports `ToolPreset` from `src/models.py`). +The `PROVIDERS` list moved from `src/models.py` to `src/ai_client.py:62` per the AGENTS.md HARD RULE (no new `src/.py` files; system code lives in the system module). A PEP 562 `__getattr__` re-export in `src/models.py:31` maintains backward compatibility (lazy import; breaks the circular dependency where `src/ai_client.py` imports `ToolPreset` from `src/tool_presets.py` and `src/models.py` is now a thin shim). -Audit: `scripts/audit_providers_source_of_truth.py` fails if `PROVIDERS` is declared in `src/models.py`. +Audit: `scripts/audit_providers_source_of_truth.py` fails if `PROVIDERS` is declared as a literal in `src/models.py`. ### Tests diff --git a/docs/guide_context_aggregation.md b/docs/guide_context_aggregation.md index 65057ee8..ef6c321a 100644 --- a/docs/guide_context_aggregation.md +++ b/docs/guide_context_aggregation.md @@ -87,7 +87,7 @@ The `auto` strategy is the *only* one that respects `config.project.summary_only ## The FileItem Schema (Full) -`src/models.py:510-559 FileItem` is the **per-file curation memory** that nagent_review identified as Manual Slop's strongest dimension. The dataclass has 9 mutable fields + a `__post_init__` normalizer: +`src/project_files.py:FileItem` is the **per-file curation memory** that nagent_review identified as Manual Slop's strongest dimension (moved out of `src/models.py` per `module_taxonomy_refactor_20260627`). The dataclass has 9 mutable fields + a `__post_init__` normalizer: ```python @dataclass @@ -139,7 +139,7 @@ Multiple slices in a file are joined with `\n\n`. ## The ContextPreset Schema -`src/models.py:909-937 ContextPreset` is a *named, persisted set* of `FileItem`s — a reusable "context composition": +`src/context_presets.py:ContextPreset` is a *named, persisted set* of `FileItem`s — a reusable "context composition" (moved out of `src/models.py`; see [guide_models.md](guide_models.md) for the current per-system file map): ```python @dataclass @@ -381,10 +381,10 @@ For very large codebases (1000+ files), the bottleneck is the tree-sitter parsin ## Cross-References - **The pipeline source:** `src/aggregate.py` (518 lines) -- **FileItem schema:** `src/models.py:510-559 FileItem` -- **ContextPreset schema:** `src/models.py:909-937 ContextPreset` +- **FileItem schema:** `src/project_files.py:FileItem` (moved out of `src/models.py`) +- **ContextPreset schema:** `src/context_presets.py:ContextPreset` (moved out of `src/models.py`) - **ContextPresetManager:** `src/context_presets.py` (30 lines) -- **AI client consumption:** `src/ai_client.py:_send_` × 5, see `guide_ai_client.md` +- **AI client consumption:** `src/ai_client.py:_send_` × 8 (gemini, anthropic, gemini_cli, deepseek, minimax, qwen, grok, llama), see `guide_ai_client.md` - **Tier 3 worker consumption:** `src/multi_agent_conductor.py:run_worker_lifecycle`, see `guide_multi_agent_conductor.md` - **Per-file curation features:** `guide_context_curation.md` (Fuzzy Anchors, AST Inspector, Granular AST Control) - **Cache strategy:** `guide_architecture.md §"Cache Hit Strategy"`, `guide_ai_client.md §"Caching"` diff --git a/docs/guide_discussions.md b/docs/guide_discussions.md index 7ca8a10d..38627d18 100644 --- a/docs/guide_discussions.md +++ b/docs/guide_discussions.md @@ -28,7 +28,7 @@ This is a *deliberate* design choice. Manual Slop treats the discussion as user- ### The Entry Dict -The smallest unit of a discussion is the **entry**, a `dict[str, Any]` with this shape (`src/models.py:parse_history_entries` builds it; `src/gui_2.py:render_discussion_entry` reads it): +The smallest unit of a discussion is the **entry**, a `dict[str, Any]` with this shape (`parse_history_entries` builds it — moved out of `src/models.py` to `src/mma.py` per `module_taxonomy_refactor_20260627`; `src/gui_2.py:render_discussion_entry` reads it): | Field | Type | Source | Purpose | |---|---|---|---| @@ -343,7 +343,7 @@ The POST endpoint allows external automation to *replace* the entire discussion. ## Cross-References -- **Discussion data model:** `src/models.py:196 parse_history_entries`, `src/models.py:909 ContextPreset`, `src/models.py:510 FileItem` +- **Discussion data model:** `parse_history_entries` (moved out of `src/models.py` to `src/mma.py`), `ContextPreset` (now in `src/context_presets.py`), `FileItem` (now in `src/project_files.py`) - **Discussion persistence:** `src/project_manager.py:429 branch_discussion`, `src/project_manager.py:447 promote_take`, `src/project_manager.py:396 calculate_track_progress` - **Discussion switching/management:** `src/app_controller.py:3199 _switch_discussion`, `src/app_controller.py:3225 _flush_disc_entries_to_project`, `src/app_controller.py:3286 _handle_reset_session`, `src/app_controller.py:3357 _handle_compress_discussion`, `src/app_controller.py:3503 _branch_discussion`, `src/app_controller.py:3521 _rename_discussion`, `src/app_controller.py:3537 _delete_discussion` - **GUI render functions:** `src/gui_2.py:175 truncate_entries`, `src/gui_2.py:735 _take_snapshot`, `src/gui_2.py:754 _apply_snapshot`, `src/gui_2.py:3770 render_discussion_entry`, `src/gui_2.py:4227 render_discussion_entries`, `src/gui_2.py:4239 render_discussion_entry_controls`, `src/gui_2.py:4317 render_discussion_roles`, `src/gui_2.py:4330 render_discussion_selector` diff --git a/docs/guide_gui_2.md b/docs/guide_gui_2.md index f8146816..72c7a7af 100644 --- a/docs/guide_gui_2.md +++ b/docs/guide_gui_2.md @@ -100,7 +100,7 @@ The App holds dozens of state attributes. The most important: | `self._tool_log` | `deque` | Tool call log | | `show_command_palette` | `bool` | Command palette visibility | -### Render Entry Point: `_gui_func` (line 754) +### Render Entry Point: `_gui_func` (line 1062) The main render loop, called by imgui-bundle's Hello ImGui runner every frame: @@ -130,7 +130,7 @@ def _gui_func(self) -> None: render_error_tint(self) ``` -### `render_main_interface` (line 1259) +### `render_main_interface` (line 1898) The "main content" renderer. Iterates over the major panels and calls the right render function for each: diff --git a/docs/guide_mma.md b/docs/guide_mma.md index 48fe542b..9a982cd1 100644 --- a/docs/guide_mma.md +++ b/docs/guide_mma.md @@ -17,7 +17,9 @@ Tier 4: QA — stateless error analysis, no fixes --- -## Data Structures (`models.py`) +## Data Structures (`src/mma.py`) + +> **Note (updated 2026-07-02):** the MMA data structures (`Ticket`, `Track`, `WorkerContext`, `TrackMetadata`, `TicketStatus`, `TicketPriority`, `TrackCheckpoint`) moved out of `src/models.py` into `src/mma.py` per `module_taxonomy_refactor_20260627`. The old heading "models.py" is retained as a section anchor for readers following older references; the live location is `src/mma.py`. ### Ticket diff --git a/docs/guide_multi_agent_conductor.md b/docs/guide_multi_agent_conductor.md index dbdb0fd4..826d04a1 100644 --- a/docs/guide_multi_agent_conductor.md +++ b/docs/guide_multi_agent_conductor.md @@ -72,7 +72,7 @@ class TrackDAG: self.ticket_map = {t.id: t for t in tickets} # dict[str, Ticket] (O(1) lookup) ``` -The `Ticket` dataclass itself is defined in `src/models.py` (see [guide_models.md](guide_models.md)). It carries the dependencies and status directly — there is no separate `TicketNode` wrapper, no `edges` dict, no `reverse_edges` dict. Adjacency lists are computed on demand inside `cascade_blocks()` and `topological_sort()`. +The `Ticket` dataclass itself is defined in `src/mma.py` (moved out of `src/models.py` per `module_taxonomy_refactor_20260627`; see [guide_models.md](guide_models.md)). It carries the dependencies and status directly — there is no separate `TicketNode` wrapper, no `edges` dict, no `reverse_edges` dict. Adjacency lists are computed on demand inside `cascade_blocks()` and `topological_sort()`. ### Public Methods (actual signatures from `src/dag_engine.py:41-163`) @@ -192,7 +192,7 @@ class ExecutionEngine: --- -## The `ConductorEngine` (in `src/multi_agent_conductor.py:116+`) +## The `ConductorEngine` (in `src/multi_agent_conductor.py:112+`) The actual class is named `ConductorEngine`, not `MultiAgentConductor`. It owns the DAG, the engine, and a `WorkerPool`; pushes state to the GUI; and runs the main async dispatch loop. @@ -263,7 +263,7 @@ async def _push_state(self, status: str = "running", active_tier: str = None) -> --- -## The `WorkerPool` (in `src/multi_agent_conductor.py:50-114`) +## The `WorkerPool` (in `src/multi_agent_conductor.py:52-110`) A `dict[str, Thread]` + `threading.Lock` + `threading.Semaphore`, NOT a `ThreadPoolExecutor` wrapper. diff --git a/docs/guide_personas.md b/docs/guide_personas.md index 195d7cd4..3c985981 100644 --- a/docs/guide_personas.md +++ b/docs/guide_personas.md @@ -23,7 +23,7 @@ This guide covers: ## Data Model -### `Persona` (`src/models.py:704`) +### `Persona` (defined in `src/personas.py`; moved out of `src/models.py` per `module_taxonomy_refactor_20260627`) ```python class Persona: @@ -138,8 +138,7 @@ aggregation_strategy = "summarize" `PersonaManager` is the CRUD interface. ```python -from src.personas import PersonaManager -from src.models import Persona +from src.personas import PersonaManager, Persona manager = PersonaManager(project_root=Path("/path/to/project")) all_personas = manager.load_all() # Merged global + project diff --git a/docs/guide_rag.md b/docs/guide_rag.md index b07329b7..fb937e86 100644 --- a/docs/guide_rag.md +++ b/docs/guide_rag.md @@ -69,7 +69,7 @@ class RAGEngine: ... ``` -**Construction**: Takes a `RAGConfig` (from `src/models.py`) and a `base_dir`. The config specifies the embedding provider type, the vector store path, the chunk size, and the chunk overlap. +**Construction**: Takes a `RAGConfig` (defined in `src/mcp_client.py` — moved out of `src/models.py` per `module_taxonomy_refactor_20260627`) and a `base_dir`. The config specifies the embedding provider type, the vector store path, the chunk size, and the chunk overlap. **Internal state**: - `embedding_provider: BaseEmbeddingProvider` — set by `_init_embedding_provider` @@ -322,7 +322,7 @@ chunk_size = 1000 chunk_overlap = 200 ``` -### `RAGConfig` + `VectorStoreConfig` Schema (`src/models.py`) +### `RAGConfig` + `VectorStoreConfig` Schema (defined in `src/mcp_client.py`; moved out of `src/models.py`) ```python @dataclass diff --git a/docs/guide_workspace_profiles.md b/docs/guide_workspace_profiles.md index 74a53335..c4b2622b 100644 --- a/docs/guide_workspace_profiles.md +++ b/docs/guide_workspace_profiles.md @@ -64,7 +64,7 @@ Workspace Profiles live at the boundary between the ImGui render loop and the pe ## Data Model -### `WorkspaceProfile` (`src/models.py`) +### `WorkspaceProfile` (defined in `src/workspace_manager.py`; moved out of `src/models.py` per `module_taxonomy_refactor_20260627`) A snapshot of window state at a moment in time.