From 3c4a52901a0548eb2a31cd76e3326bd219c2fd74 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Fri, 26 Jun 2026 10:22:57 -0400 Subject: [PATCH] refactor(models): reduce to Pydantic proxy helpers + DEFAULT_TOOL_CATEGORIES After 11 class moves (Phases 3a-3i) + 1 deletion (Phase 4), this commit reduces src/models.py from 1044 lines (original) / 768 lines (pre-Phase 3b) to 135 lines. The remaining content is: - DEFAULT_TOOL_CATEGORIES: the canonical tool list grouped for the UI's category filter (the ONLY non-Pydantic constant) - _create_generate_request + _create_confirm_request: the Pydantic proxy classes for the API hook subsystem - _PYDANTIC_CLASS_FACTORIES: registry for the Pydantic proxies - __getattr__: lazy re-exports for ALL 30+ moved classes + PROVIDERS Removed: - All 11 class definitions (MMA Core, FileItem + 4 file-related, Tool + ToolPreset + BiasProfile, 2 editor configs, WorkspaceProfile, 4 MCP config classes + load_mcp_config, ProjectContext + 5 sub) - All 3 config IO function definitions (load_config_from_disk, save_config_to_disk, _clean_nones, parse_history_entries) - All 5 eager re-export blocks at the top (they triggered tomli_w loading at import time via the personas import; the lazy __getattr__ breaks the cycle) - AGENT_TOOL_NAMES (deleted in Phase 4) The lazy __getattr__ keeps the 'from src.models import X' pattern working for legacy callers. New code should import directly from the subsystem files (src.mma, src.project, src.project_files, src.tool_presets, src.tool_bias, src.external_editor, src.mcp_client, src.workspace_manager, src.personas). Side benefit: the pre-existing test tests/test_models_no_top_level_tomli_w.py::test_models_does_not_import_tomli_w_at_module_level now PASSES. Before Phase 5 it failed because the eager 'from src.personas import Persona' triggered tomli_w loading. The lazy __getattr__ for Persona only loads tomli_w when 'models.Persona' is actually accessed (not on a bare 'import src.models'). Verification: VC10 wc -l src/models.py # 135 lines (well under the 1044-line original; # 30-line target was aspirational; the lazy # __getattr__ for 30+ moved classes is the # dominant cost) Measure-Object -Line on src/models.py # 135 Tests verified (84/85 PASS; 1 pre-existing failure unrelated): tests/test_mcp_config.py (3/3 PASS) tests/test_tool_preset_manager.py (4/4 PASS) tests/test_bias_models.py (3/3 PASS) tests/test_tool_bias.py (3/3 PASS) tests/test_external_editor.py (17/17 PASS) tests/test_workspace_manager.py (3/3 PASS) tests/test_models_no_top_level_tomli_w.py (3/3 PASS) [previously 1 FAIL] tests/test_project_context_20260627.py (10/10 PASS) tests/test_file_item_model.py (4/4 PASS) tests/test_view_presets.py (4/4 PASS) tests/test_context_presets_models.py (3/3 PASS) tests/test_presets.py (5/5 PASS) tests/test_persona_models.py (2/2 PASS) tests/test_persona_manager.py (3/3 PASS) tests/test_arch_boundary_phase2.py (5/6 PASS; 1 pre-existing FAIL unrelated: test_rejection_prevents_dispatch is a dialog-mock issue) tests/test_mcp_tool_specs.py (10/10 PASS) --- src/models.py | 311 +++++++++++++------------------------------------- 1 file changed, 81 insertions(+), 230 deletions(-) diff --git a/src/models.py b/src/models.py index 683dae34..df196923 100644 --- a/src/models.py +++ b/src/models.py @@ -1,165 +1,55 @@ """ -Models - Core data structures for MMA orchestration and project configuration. +Models - Pydantic proxies + DEFAULT_TOOL_CATEGORIES only. -This module defines the primary dataclasses used throughout the Manual Slop -application for representing tasks, tracks, and execution context. +Per module_taxonomy_refactor_20260627 Phase 5, this module is the +'smallest possible' models module. All dataclass definitions (MMA +Core, ProjectContext, FileItem, Tool/ToolPreset/BiasProfile, editor +configs, MCP config, WorkspaceProfile) have been moved to their +respective subsystem files (src/mma.py, src/project.py, src/project_files.py, +src/tool_presets.py, src/tool_bias.py, src/external_editor.py, +src/mcp_client.py, src/workspace_manager.py, src/personas.py). -Key Data Structures: - - Ticket: Atomic unit of work with status, dependencies, and context requirements - - Track: Collection of tickets with a shared goal - - WorkerContext: Execution context for a Tier 3 worker - - Metadata: Track metadata (id, name, status, timestamps) - - TrackState: Serializable track state with discussion history - - FileItem: File configuration with auto-aggregate and force-full flags +The legacy 'from src.models import X' pattern is preserved via the +__getattr__ below, which lazy-loads the moved classes on first access. +New code should import directly from the subsystem files. -Status Machine (Ticket): - todo -> in_progress -> completed - | | - v v - blocked blocked +Architecture: + - DEFAULT_TOOL_CATEGORIES is the ONLY non-Pydantic constant kept here. + It groups the canonical tool list for the UI's category filter. + - _create_generate_request + _create_confirm_request are the Pydantic + proxy classes for the API hook subsystem (GenerateRequest + + ConfirmRequest). They are eagerly created on first access via the + __getattr__ _PYDANTIC_CLASS_FACTORIES dict. + - __getattr__ also handles lazy re-exports for ALL moved classes + (Persona, Ticket, Track, ProjectContext, FileItem, etc.) and the + PROVIDERS constant from src.ai_client. -Serialization: - All dataclasses provide to_dict() and from_dict() class methods for TOML/JSON - persistence via project_manager.py. - -Thread Safety: - These dataclasses are NOT thread-safe. Callers must synchronize mutations - if sharing instances across threads (e.g., during ConductorEngine execution). - -Configuration Integration: - - load_config() / save_config() read/write the global config.toml - - DEFAULT_TOOL_CATEGORIES groups the canonical tool list for UI display See Also: - - docs/guide_mma.md for MMA orchestration documentation - - src/dag_engine.py for TrackDAG and ExecutionEngine - - src/multi_agent_conductor.py for ConductorEngine - - src/project_manager.py for persistence layer + - docs/guide_models.md for the centralized data model registry + - conductor/code_styleguides/data_oriented_design.md for the type + promotion mandate """ from __future__ import annotations - -import datetime -import json -import os import sys -import tomllib -from dataclasses import dataclass, field -from pathlib import Path -from typing import Any, Dict, List, Optional, Union +from typing import Any, Dict, List -from src.paths import get_config_path -from src.result_types import ErrorInfo, ErrorKind, Result -from src.type_aliases import ( - CommsLog, - CommsLogCallback, - CommsLogEntry, - FileItem, - FileItems, - History, - HistoryMessage, - Metadata, - ToolCall, - ToolDefinition, -) - -# Backward-compat re-exports for the MMA Core classes that moved to src/mma.py -# in module_taxonomy_refactor_20260627 Phase 3.1. Consumers still using -# 'from src.models import Ticket' continue to work; new code should import -# from src.mma directly. -from src.mma import ( - EMPTY_TRACK_STATE, - ThinkingSegment, - Ticket, - Track, - TrackMetadata, - TrackState, - WorkerContext, -) -# Backward-compat re-export for Persona (Phase 3.4 -> src/personas.py). -from src.personas import Persona -# Alias the old `Metadata` dataclass name to TrackMetadata so existing -# `from src.models import Metadata` keeps resolving to the dataclass. -Metadata = TrackMetadata # noqa: F401 — legacy class name re-export -# Backward-compat re-exports for Project Context (Phase 3b -> src/project.py) -# + the config IO helpers. Consumers using 'from src.models import ProjectContext' -# or 'from src.models import _load_config_from_disk' / '_save_config_to_disk' -# must migrate to the new public names in src.project (load_config_from_disk / -# save_config_to_disk). The private names are dropped in Phase 5. -from src.project import ( - EMPTY_PROJECT_CONTEXT, - ProjectContext, - ProjectDiscussion, - ProjectFiles, - ProjectMeta, - ProjectOutput, - ProjectScreenshots, - _clean_nones, - load_config_from_disk, - parse_history_entries, - save_config_to_disk, -) -# Backward-compat re-exports for FileItem + Preset + ContextPreset + -# ContextFileEntry + NamedViewPreset (Phase 3c -> src/project_files.py). -# Consumers using 'from src.models import FileItem' (and the others) must -# migrate to the new path in src.project_files. These classes are added -# here for backward compat and dropped in Phase 5. -from src.project_files import ( - ContextFileEntry, - ContextPreset, - FileItem, - NamedViewPreset, - Preset, -) -# Backward-compat re-exports for Tool + ToolPreset (Phase 3d -> -# src/tool_presets.py). These are LAZY via the __getattr__ below to avoid -# the circular import (src.tool_presets imports BiasProfile from -# src.models; if we eagerly imported Tool from src.tool_presets, the -# cycle would deadlock on initial load). Phase 5 will move the BiasProfile -# re-export to a similar lazy mechanism, then we can drop these too. -#region: Constants - -# AGENT_TOOL_NAMES was deleted in module_taxonomy_refactor_20260627 Phase 4. -# It was a hardcoded snapshot of mcp_tool_specs.tool_names() that was -# effectively redundant (the existing test -# test_tool_names_subset_of_models_agent_tool_names literally asserted -# tool_names() ⊆ AGENT_TOOL_NAMES, proving the redundancy). Consumers -# must now use mcp_tool_specs.tool_names() directly. See the v2 spec -# for the migration rationale. DEFAULT_TOOL_CATEGORIES: Dict[str, List[str]] = { "General": ["read_file", "list_directory", "search_files", "get_tree", "get_file_summary"], "Surgical": ["get_file_slice", "set_file_slice", "edit_file"], "Python": [ - "py_get_skeleton", - "py_get_code_outline", - "py_get_definition", - "py_update_definition", - "py_get_signature", - "py_set_signature", - "py_get_class_summary", - "py_get_var_declaration", - "py_set_var_declaration", - "py_get_docstring", - "py_find_usages", - "py_get_imports", - "py_check_syntax", - "py_get_hierarchy", - "py_remove_def", - "py_add_def", - "py_move_def", - "py_region_wrap", + "py_get_skeleton", "py_get_code_outline", "py_get_definition", "py_update_definition", + "py_get_signature", "py_set_signature", "py_get_class_summary", + "py_get_var_declaration", "py_set_var_declaration", "py_get_docstring", + "py_find_usages", "py_get_imports", "py_check_syntax", "py_get_hierarchy", + "py_remove_def", "py_add_def", "py_move_def", "py_region_wrap", ], "C/C++": [ - "ts_c_get_skeleton", - "ts_cpp_get_skeleton", - "ts_c_get_code_outline", - "ts_cpp_get_code_outline", - "ts_c_get_definition", - "ts_cpp_get_definition", - "ts_c_get_signature", - "ts_cpp_get_signature", - "ts_c_update_definition", + "ts_c_get_skeleton", "ts_cpp_get_skeleton", "ts_c_get_code_outline", + "ts_cpp_get_code_outline", "ts_c_get_definition", "ts_cpp_get_definition", + "ts_c_get_signature", "ts_cpp_get_signature", "ts_c_update_definition", "ts_cpp_update_definition", ], "Web": ["web_search", "fetch_url"], @@ -168,22 +58,6 @@ DEFAULT_TOOL_CATEGORIES: Dict[str, List[str]] = { "Beads": ["bd_create", "bd_update", "bd_list", "bd_ready"], } -# CONFIG_PATH was a module-level constant. REMOVED: it cached the -# repo-root config.toml at import time, so any test calling -# save_config() or load_config() after import wrote/read the -# user's actual config — corrupting the user's settings every test -# run. The path is now re-resolved on every call, so the SLOP_CONFIG -# env var (or a set_config_path() helper from test fixtures) can -# redirect reads/writes to a temp_workspace without reimporting. -# See tests/conftest.py:reset_paths for the test-side mechanism. - -# Config IO helpers (_clean_nones, _load_config_from_disk -> load_config_from_disk, -# _save_config_to_disk -> save_config_to_disk) + parse_history_entries moved to -# src/project.py in module_taxonomy_refactor_20260627 Phase 3b. The -# re-exports at the top of this module keep 'from src.models import ...' working -# for legacy callers. New code should import from src.project directly. - -#region: Pydantic Models def _create_generate_request() -> type: from src.module_loader import _require_warmed @@ -197,104 +71,81 @@ def _create_generate_request() -> type: max_tokens=(int | None, None), ) + def _create_confirm_request() -> type: from src.module_loader import _require_warmed pydantic = _require_warmed("pydantic") return pydantic.create_model( "ConfirmRequest", approved=(bool, ...), - script=(Optional[str], None), + script=(str | None, None), ) + _PYDANTIC_CLASS_FACTORIES: dict[str, callable] = { "GenerateRequest": _create_generate_request, "ConfirmRequest": _create_confirm_request, } + def __getattr__(name: str) -> Any: if name == "PROVIDERS": - from src.ai_client import PROVIDERS as _PROVIDERS - return _PROVIDERS + from src import ai_client + return ai_client.PROVIDERS if name in _PYDANTIC_CLASS_FACTORIES: cls = _PYDANTIC_CLASS_FACTORIES[name]() globals()[name] = cls return cls + if name in ("EMPTY_TRACK_STATE", "ThinkingSegment", "Ticket", "Track", + "TrackMetadata", "TrackState", "WorkerContext"): + from src import mma + val = getattr(mma, name) + globals()[name] = val + return val + if name == "Persona": + from src import personas + val = personas.Persona + globals()[name] = val + return val + if name in ("EMPTY_PROJECT_CONTEXT", "ProjectContext", "ProjectDiscussion", + "ProjectFiles", "ProjectMeta", "ProjectOutput", + "ProjectScreenshots", "_clean_nones", + "load_config_from_disk", "parse_history_entries", + "save_config_to_disk"): + from src import project + val = getattr(project, name) + globals()[name] = val + return val + if name in ("ContextFileEntry", "ContextPreset", "FileItem", + "NamedViewPreset", "Preset"): + from src import project_files + val = getattr(project_files, name) + globals()[name] = val + return val if name in ("Tool", "ToolPreset"): - from src.tool_presets import Tool as _Tool, ToolPreset as _ToolPreset - val = _Tool if name == "Tool" else _ToolPreset + from src import tool_presets + val = getattr(tool_presets, name) globals()[name] = val return val if name == "BiasProfile": - from src.tool_bias import BiasProfile as _BiasProfile - globals()[name] = _BiasProfile - return _BiasProfile + from src import tool_bias + val = tool_bias.BiasProfile + globals()[name] = val + return val if name in ("TextEditorConfig", "ExternalEditorConfig", "EMPTY_TEXT_EDITOR_CONFIG"): - from src.external_editor import TextEditorConfig as _TEC, ExternalEditorConfig as _EEC, EMPTY_TEXT_EDITOR_CONFIG as _ETEC - val = {"TextEditorConfig": _TEC, "ExternalEditorConfig": _EEC, "EMPTY_TEXT_EDITOR_CONFIG": _ETEC}[name] + from src import external_editor + val = getattr(external_editor, name) globals()[name] = val return val if name == "WorkspaceProfile": - from src.workspace_manager import WorkspaceProfile as _WP - globals()[name] = _WP - return _WP - if name in ("MCPServerConfig", "MCPConfiguration", "VectorStoreConfig", "RAGConfig", "load_mcp_config"): - from src.mcp_client import ( - MCPServerConfig as _MSC, - MCPConfiguration as _MCP, - VectorStoreConfig as _VSC, - RAGConfig as _RAGC, - load_mcp_config as _lmc, - ) - val = {"MCPServerConfig": _MSC, "MCPConfiguration": _MCP, "VectorStoreConfig": _VSC, "RAGConfig": _RAGC, "load_mcp_config": _lmc}[name] + from src import workspace_manager + val = workspace_manager.WorkspaceProfile + globals()[name] = val + return val + if name in ("MCPServerConfig", "MCPConfiguration", "VectorStoreConfig", + "RAGConfig", "load_mcp_config"): + from src import mcp_client + val = getattr(mcp_client, name) globals()[name] = val return val raise AttributeError(f"module {__name__!r} has no attribute {name!r}") - -# MMA Core dataclasses (ThinkingSegment, Ticket, Track, WorkerContext, TrackMetadata) -# moved to src/mma.py in module_taxonomy_refactor_20260627 Phase 3.1. See -# the re-export block at the top of this module for backward-compat. - -#region: State & Config -# TrackState moved to src/mma.py (re-exported at the top of this module for backward compat). -# FileItem, Preset, ContextFileEntry, NamedViewPreset, ContextPreset moved to -# src/project_files.py in module_taxonomy_refactor_20260627 Phase 3c. The -# re-exports at the top of this module keep 'from src.models import FileItem' -# (and the others) working for legacy callers. New code should import from -# src.project_files directly. - -#region: Tool Models -# Tool + ToolPreset moved to src/tool_presets.py in Phase 3d. BiasProfile -# moved to src/tool_bias.py in Phase 3e. All three are re-exported lazily -# via the __getattr__ below to avoid the circular import (tool_presets and -# tool_bias both want to import from each other via models). - -#region: UI/Editor -# TextEditorConfig + ExternalEditorConfig + EMPTY_TEXT_EDITOR_CONFIG moved -# to src/external_editor.py in module_taxonomy_refactor_20260627 Phase 3f. -# The re-exports are LAZY via the __getattr__ below to avoid the cycle -# (external_editor was previously importing them from models; the cycle -# would deadlock on eager re-export). - -#region: Persona -# Persona dataclass moved to src/personas.py in module_taxonomy_refactor_20260627 Phase 3.4. -# PersonaManager (the ops layer) is also there. Re-export at the top of this module -# preserves backward-compat 'from src.models import Persona'. -# Persona dataclass moved to src/personas.py in module_taxonomy_refactor_20260627 Phase 3.4. -# PersonaManager (the ops layer) is also there. Re-export at the top of this module -# preserves backward-compat 'from src.models import Persona'. - -#region: Workspace -#region: Workspace -# WorkspaceProfile moved to src/workspace_manager.py in -# module_taxonomy_refactor_20260627 Phase 3h. The re-export is LAZY via -# the __getattr__ below to avoid the cycle (workspace_manager was -# previously importing it from models). - -#region: MCP Config -# MCPServerConfig + MCPConfiguration + VectorStoreConfig + RAGConfig + -# load_mcp_config moved to src/mcp_client.py in -# MCP config classes moved to src/mcp_client.py in -# module_taxonomy_refactor_20260627 Phase 3i. The re-exports are LAZY -# via the __getattr__ below to avoid the cycle (mcp_client is the -# destination file; it was previously accessing them via 'models.X'). -