ed/manual_slop
Private
Public Access
0
0
Fork 0
Code Activity
Files
0ffeccc7d301cc42975a37b9719aba2e3d22ea55
manual_slop/conductor/tech-stack.md
T

15 KiB
Raw Blame History

Technology Stack: Manual Slop

Core Language

  • Python 3.11+

GUI Frameworks

  • Dear PyGui: For immediate/retained mode GUI rendering and node mapping.
  • ImGui Bundle (imgui-bundle): To provide advanced multi-viewport and dockable panel capabilities on top of Dear ImGui. Includes imgui-node-editor for complex graph-based visualizations, imgui_markdown for rich text rendering, and ImGuiColorTextEdit for syntax-highlighted code blocks.

Web & Service Frameworks

  • FastAPI: High-performance REST API framework for providing the headless backend service.
  • websockets: Lightweight asynchronous WebSocket server for real-time event streaming and remote orchestration.
  • Uvicorn: ASGI server for serving the FastAPI application.

AI Integration SDKs

  • google-genai: For Google Gemini API interaction and explicit context caching.
  • anthropic: For Anthropic Claude API interaction, supporting ephemeral prompt caching.
  • DeepSeek (Dedicated SDK): Integrated for high-performance codegen and reasoning (Phase 2).
  • Gemini CLI: Integrated as a headless backend provider, utilizing a custom subprocess adapter and bridge script for tool execution control. Achieves full functional parity with direct SDK usage, including real-time token counting and detailed subprocess observability.
  • MiniMax: Alternative provider support, integrated via the OpenAI-compatible SDK.
  • Gemini 3.1 Pro Preview: Tier 1 Orchestrator model for complex reasoning.
  • Gemini 3-Flash Preview: High-reasoning low-latency model for Tier 2 Tech Lead.
  • Gemini 2.5 Flash Lite: Ultra-low-cost, high-speed model for Tier 3 Workers and Tier 4 QA.
  • DeepSeek-V3: Tier 3 Worker model optimized for code implementation.
  • DeepSeek-R1: Specialized reasoning model for complex logical chains and "thinking" traces.
  • Gemini Embedding 001: Default embedding model for RAG vector store.

Configuration & Tooling

  • src/paths.py: Centralized module for path resolution. Supports project-specific conductor directory overrides via project TOML ([conductor].dir), enabling isolated track management per project. If not specified, conductor paths default to ./conductor relative to each project's TOML file. All paths are resolved to absolute objects. Provides Path Resolution Metadata, exposing the source of each resolved path (default, environment variable, or configuration file) for high-fidelity GUI display. Supports Runtime Re-Resolution via reset_resolved(), allowing path changes to be applied immediately without an application restart. Path configuration (logs, scripts) can also be configured via config.toml or environment variables, eliminating hardcoded filesystem dependencies.

  • src/presets.py: Implements PresetManager for high-performance CRUD operations on system prompt presets stored in TOML format (presets.toml, project_presets.toml). Supports dynamic path resolution, scope-based inheritance, and foundational base prompt customization.

  • src/personas.py: Implements PersonaManager for high-performance CRUD operations on unified agent personas stored in TOML format (personas.toml, project_personas.toml). Handles consolidation of model settings, prompts, and tool biases.

  • src/context_presets.py: Implements ContextPresetManager for managing complex context compositions (files, view modes, screenshots) within the project configuration. Supports validation of file existence and relative path mapping.

  • src/tool_bias.py: Implements the ToolBiasEngine for semantic tool description nudging and dynamic tooling strategy generation.

  • src/tool_presets.py: Extends ToolPresetManager to handle nested Tool models, weights, and global BiasProfile persistence within tool_presets.toml.

  • src/mcp_client.py: Implements the native tool registry (45 tools) and the ExternalMCPManager for orchestrating third-party Model Context Protocol servers. Provides dynamic tool discovery and validation. See docs/guide_mcp_client.md for the complete 3-layer security model (Allowlist → Validate → Resolve) and tool inventory.

    • StdioMCPServer: Manages local MCP servers via asynchronous subprocess pipes (stdin/stdout/stderr).
    • RemoteMCPServer (SSE): Provides a foundation for remote MCP integration via Server-Sent Events.
    • JSON-RPC 2.0 Engine: Handles asynchronous message routing, request/response matching, and error handling for all external MCP communication.
    • AST-Based C/C++ Tools: Provides ts_c_get_skeleton, ts_cpp_get_skeleton, ts_c_get_code_outline, and ts_cpp_get_code_outline for structural analysis of C/C++ codebases using tree-sitter.
    • AST-Based Python Tools (15): py_get_skeleton, py_get_code_outline, py_get_definition, py_update_definition, py_get_signature, py_set_signature, py_get_class_summary, py_get_var_declaration, py_set_var_declaration, py_get_hierarchy, py_get_docstring, py_get_imports, py_find_usages, py_check_syntax, plus structural mutators py_remove_def, py_add_def, py_move_def, py_region_wrap.
    • Network Tools: web_search (DuckDuckGo HTML scrape), fetch_url (HTML → text).
    • Beads Tools (4): bd_list, bd_create, bd_update, bd_ready — interface to the Beads/Dolt backend.

  • src/api_hooks.py + src/api_hook_client.py: Implements the Hook API and Python client wrapper for external automation. See docs/guide_api_hooks.md.

    • HookServer: FastAPI/Uvicorn server on 127.0.0.1:8999, started by AppController when --enable-test-hooks is set. Exposes 8+ REST endpoints (/status, /api/gui, /api/ask, /api/mma_status, /api/performance, /api/comms, /api/diagnostics).
    • ApiHookClient: Python client with retry logic, health-check polling, and timeout configuration. Used by all live_gui tests, the WorkerPool, and external scripts.
    • /api/ask Protocol: Non-blocking, ID-based challenge/response for synchronous HITL approvals from external contexts.
    • _predefined_callbacks and _gettable_fields: AppController-owned registries that the Hook API consumes to expose any App method as a custom_callback action.

  • src/rag_engine.py: Core RAG implementation managing the vector store lifecycle, chunking strategies (character-based and AST-aware), and multi-provider search. Integrates with ChromaDB for local persistence and provides a bridge for external MCP retrieval tools.

  • src/beads_client.py: Python client for interacting with the Beads / Dolt backend. Handles repository initialization, bead creation, status updates, and graph queries.

  • bd / dolt: External CLI dependencies for versioned, graph-based issue tracking.

  • src/history.py: Implements the core HistoryManager and UISnapshot logic for the non-provider undo/redo system. Manages state stacks with a fixed capacity and provides jumping capabilities.

  • src/workspace_manager.py: Implements the WorkspaceManager and WorkspaceProfile data models for saving, loading, and merging ImGui docking layouts and window states across global and project-specific configurations.

  • src/paths.py: Centralized module for path resolution.

  • tree-sitter / AST Parsing: For deterministic AST parsing and automated generation of curated "Skeleton Views" and "Targeted Views" (extracting specific functions and their dependencies). Supports Python, C, and C++. Features an integrated AST cache with mtime-based invalidation to minimize re-parsing overhead. Supplemented by SummaryCache which provides persistent, hash-based (SHA256) caching with LRU eviction for AI-generated file summaries.

  • pydantic / dataclasses: For defining strict state schemas (Tracks, Tickets) used in linear orchestration.

  • tomli-w: For writing TOML configuration files.

  • tomllib: For native TOML parsing (Python 3.11+).

  • LogRegistry & LogPruner: Custom components for session metadata persistence and automated filesystem cleanup within the logs/sessions/ taxonomy.

  • psutil: For system and process monitoring (CPU/Memory telemetry).

  • uv: An extremely fast Python package and project manager.

  • pywin32: For custom OS window frame manipulation on Windows (e.g., minimizing, maximizing, closing, and dragging the borderless ImGui window).

  • pytest: For unit and integration testing, leveraging custom fixtures for live GUI verification.

  • Taxonomy & Artifacts: Enforces a clean root by organizing core implementation into a src/ directory, and redirecting session logs and artifacts to configurable directories (defaulting to logs/sessions/ and scripts/generated/). Temporary test data and test logs are siloed in tests/artifacts/ and tests/logs/.

  • ApiHookClient: A dedicated IPC client for automated GUI interaction and state inspection. Now supports simulating complex interactions including drag and right_click operations. Full API surface: click(), set_value(), select_tab(), select_list_item(), push_event(), get_value(), get_status(), reset_session(), wait_for_event(). See docs/guide_api_hooks.md for the complete method reference and /api/ask protocol.

  • mma-exec / mma.ps1: Python-based execution engine and PowerShell wrapper for managing the 4-Tier MMA hierarchy and automated documentation mapping.

  • dag_engine.py: A native Python utility implementing TrackDAG and ExecutionEngine for dependency resolution, cycle detection, transitive blocking propagation, and programmable task execution loops. Optimized using Kahn's Algorithm and iterative DFS to eliminate recursion overhead and provide O(V+E) performance.

  • multi_agent_conductor.py: Orchestrates the concurrent execution of implementation tracks using a non-blocking ConductorEngine and a thread-safe WorkerPool. Employs configurable concurrency limits and thread-local context isolation to manage multi-agent state.

  • Thread-Local Context Isolation: Utilizes threading.local() for managing per-thread AI client context (e.g., source tier tagging), ensuring thread safety during concurrent multi-agent execution.

  • Asynchronous Tool Execution Engine: Refactored MCP tool dispatch and AI client loops to use asyncio.gather and asyncio.to_thread, enabling parallel execution of independent tool calls within a single AI turn to reduce latency.

Architectural Patterns

  • Centralized Registry Management: Consolidation of critical application constants (e.g., PROVIDERS, AGENT_TOOL_NAMES) into src/models.py as a single source of truth, eliminating redundant list definitions across the UI and Controller.

  • Event-Driven Metrics: Uses a custom EventEmitter to decouple API lifecycle events from UI rendering, improving performance and responsiveness.

  • Synchronous Event Queue: Employs a SyncEventQueue based on queue.Queue to manage communication between the UI and backend agents, maintaining responsiveness through a threaded execution model.

  • Synchronous IPC Approval Flow: A specialized bridge mechanism that allows headless AI providers (like Gemini CLI) to synchronously request and receive human approval for tool calls and manual ticket transitions (Step Mode) via the GUI's REST API hooks.

  • High-Fidelity Selectable Labels: Implements a pattern for making read-only UI text selectable by wrapping imgui.input_text with imgui.InputTextFlags_.read_only. Includes a specialized _render_selectable_label helper that resets frame backgrounds, borders, and padding to mimic standard labels while enabling OS-level clipboard support (Ctrl+C).

  • Hybrid Markdown Rendering: Employs a custom MarkdownRenderer that orchestrates imgui_markdown for standard text and headers while intercepting code blocks to render them via cached ImGuiColorTextEdit instances. This ensures high-performance rich text rendering with robust syntax highlighting and stateful text selection.

  • Hybrid Shader Pipeline: Utilizes an optimized ImDrawList-based batching technique to simulate UI effects (shadows, soft borders) instead of complex GPU post-processing, eliminating the overhead of heavy GPU-resident shaders.

  • Interface-Driven Development (IDD): Enforces a "Stub-and-Resolve" pattern where cross-module dependencies are resolved by generating signatures/contracts before implementation.

  • UI Concern Isolation: Enforces a strict separation between the GUI layer (Tkinter/ImGui) and the business logic (AppController). All platform-native UI actions, such as file and directory selection, are handled exclusively within the GUI layer.

  • UI Delegation Pattern: Employs module-level rendering functions decoupled from the App class. This enables selective hot-reloading of UI logic by swapping function references at runtime while maintaining a stable state object (app: App).

  • Manual Hot-Reload Pipeline: Implements a HotReloader utility that manages module invalidation and state preservation, triggered by keyboard shortcuts (Ctrl+Alt+R) or GUI controls.

  • src/command_palette.py + src/commands.py: Implements the keyboard-driven Command Palette (Ctrl+Shift+P). See docs/guide_command_palette.md and docs/guide_gui_2.md.

    • CommandRegistry: Decorator-based command registration (@registry.register). 32+ built-in commands including _toggle_command_palette, _open_command_palette, theme switching, view presets, persona application.
    • fuzzy_match: Subsequence matching with score (consecutive bonus, start-of-word bonus, length penalty).
    • render_palette_modal: Centered popup with input field, keyboard navigation (Up/Down/Enter/Esc), and live result filtering.
    • defensive try/except wrapping: All action callbacks wrapped to prevent GUI crashes from buggy commands.

Per-Source-File Deep Dives

For the largest source files, consult the dedicated guides in docs/:

ter/ImGui) and the business logic (AppController). All platform-native UI actions, such as file and directory selection, are handled exclusively within the GUI layer.

View Git Blame Copy Permalink