ed/manual_slop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refinement of upcoming tracks

Browse Source
This commit is contained in:
Edward R. Gonzalez
2026-03-06 15:41:33 -05:00
parent 3ce6a2ec8a
commit fca40fd8da
24 changed files with 2388 additions and 391 deletions
Download Patch File Download Diff File Expand all files Collapse all files

280
conductor/tracks/conductor_path_configurable_20260306/plan.md
View File

@@ -1,63 +1,237 @@
# Implementation Plan: Conductor Path Configuration (conductor_path_configurable_20260306)
# Implementation Plan: Conductor Path Configuration (conductor_path_configurable_20260306)
## Phase 1: Centralized Path Resolver
- [ ] Task: Create path resolver module
- WHERE: src/orchestrator_pm.py or new module (src/paths.py)
- WHAT: Single function to get all configurable paths
- HOW: Check env vars, then config, then defaults
- SAFETY: Immutable after first call
> **Reference:** [Spec](./spec.md) | [Architecture Guide](../../../docs/guide_architecture.md)
>
> **CRITICAL:** This is Phase 0 infrastructure. Complete this before other Phase 3 tracks.
## Phase 2: Define Config Schema
- [ ] Task: Define path configuration schema
- WHERE: src/paths.py
- WHAT: Define default paths and env var names
- HOW: Constants for each path
- SAFETY: None
## Phase 1: Create Centralized Path Module
Focus: Establish the single source of truth for all paths
## Phase 3: Update Orchestrator
- [ ] Task: Update orchestrator_pm.py
- WHERE: src/orchestrator_pm.py
- WHAT: Use centralized resolver
- HOW: Import and use get_path('conductor')
- [ ] Task: Update project_manager.py
- WHERE: src/project_manager.py
- WHAT: Use centralized paths
- HOW: Import from paths module
- [ ] Task 1.1: Initialize MMA Environment
- Run `activate_skill mma-orchestrator` before starting
- [ ] Task 1.2: Create src/paths.py module
- WHERE: `src/paths.py` (new file)
- WHAT: Centralized path resolution module
- HOW:
```python
from pathlib import Path
import os
import tomllib
from typing import Optional
_CONFIG_PATH: Path = Path(os.environ.get("SLOP_CONFIG", "config.toml"))
_RESOLVED: dict[str, Path] = {}
def _resolve_path(env_var: str, config_key: str, default: str) -> Path:
if env_var in os.environ:
return Path(os.environ[env_var])
try:
with open(_CONFIG_PATH, "rb") as f:
cfg = tomllib.load(f)
if "paths" in cfg and config_key in cfg["paths"]:
return Path(cfg["paths"][config_key])
except FileNotFoundError:
pass
return Path(default)
def get_conductor_dir() -> Path:
if "conductor_dir" not in _RESOLVED:
_RESOLVED["conductor_dir"] = _resolve_path("SLOP_CONDUCTOR_DIR", "conductor_dir", "conductor")
return _RESOLVED["conductor_dir"]
def get_logs_dir() -> Path:
if "logs_dir" not in _RESOLVED:
_RESOLVED["logs_dir"] = _resolve_path("SLOP_LOGS_DIR", "logs_dir", "logs/sessions")
return _RESOLVED["logs_dir"]
def get_scripts_dir() -> Path:
if "scripts_dir" not in _RESOLVED:
_RESOLVED["scripts_dir"] = _resolve_path("SLOP_SCRIPTS_DIR", "scripts_dir", "scripts/generated")
return _RESOLVED["scripts_dir"]
def get_config_path() -> Path:
return _CONFIG_PATH
def get_tracks_dir() -> Path:
return get_conductor_dir() / "tracks"
def get_track_state_dir(track_id: str) -> Path:
return get_tracks_dir() / track_id
def get_archive_dir() -> Path:
return get_conductor_dir() / "archive"
def reset_resolved() -> None:
"""For testing only - clear cached resolutions."""
_RESOLVED.clear()
```
- CODE STYLE: 1-space indentation
- SAFETY: Lazy resolution prevents import-order issues
- [ ] Task 1.3: Write unit tests for paths module
- WHERE: `tests/test_paths.py` (new file)
- WHAT: Test path resolution logic
- HOW: Test defaults, env vars, config overrides, precedence
- PATTERN: Mock `os.environ`, create temp config files
## Phase 2: Update Core Modules
Focus: Migrate orchestrator and project_manager to use paths module
- [ ] Task 2.1: Update orchestrator_pm.py
- WHERE: `src/orchestrator_pm.py` line 10
- WHAT: Replace `CONDUCTOR_PATH` with path function
- HOW:
```python
# OLD: CONDUCTOR_PATH: Path = Path("conductor")
# NEW:
from src import paths
# Then use paths.get_conductor_dir() where needed
```
- SAFETY: Check all usages of `CONDUCTOR_PATH` in the file
- [ ] Task 2.2: Update project_manager.py
- WHERE: `src/project_manager.py` lines 240, 252, 297
- WHAT: Replace hardcoded "conductor" with path functions
- HOW:
```python
from src import paths
# save_track_state: track_dir = paths.get_track_state_dir(track_id)
# load_track_state: state_file = paths.get_track_state_dir(track_id) / "state.toml"
# get_all_tracks: tracks_dir = paths.get_tracks_dir()
```
- SAFETY: Maintain `base_dir` parameter for backward compatibility if needed
## Phase 3: Update Session Logger
Focus: Migrate session_logger.py to use paths module
- [ ] Task 3.1: Update session_logger.py paths
- WHERE: `src/session_logger.py` lines 26-27
- WHAT: Replace module-level constants with lazy resolution
- HOW:
```python
# OLD:
# _LOG_DIR: Path = Path("./logs/sessions")
# _SCRIPTS_DIR: Path = Path("./scripts/generated")
# NEW:
from src import paths
# In functions, use paths.get_logs_dir() and paths.get_scripts_dir()
```
- SAFETY: Module-level initialization may need to become function-level
- [ ] Task 3.2: Handle open_session() path resolution
- WHERE: `src/session_logger.py` open_session function
- WHAT: Resolve paths at call time, not import time
- HOW: Call `paths.get_logs_dir()` inside `open_session()`
- SAFETY: Verify no import-time side effects
## Phase 4: Update App Controller
- [ ] Task: Update app_controller.py
- WHERE: src/app_controller.py
- WHAT: Use paths for logs, conductor
- HOW: Import from paths module
Focus: Migrate all scattered path references in app_controller.py
- [ ] Task 4.1: Update app_controller.py log paths
- WHERE: `src/app_controller.py` lines 643, 674, 1241
- WHAT: Replace hardcoded "logs/sessions" with path functions
- HOW:
```python
from src import paths
# cb_load_prior_log: initialdir=str(paths.get_logs_dir())
# cb_prune_logs: LogRegistry(paths.get_logs_dir() / "log_registry.toml")
# _render_log_management: Path(paths.get_logs_dir())
```
- SAFETY: Convert Path to str where file dialog expects string
- [ ] Task 4.2: Update app_controller.py conductor paths
- WHERE: `src/app_controller.py` lines 1907, 1937
- WHAT: Replace hardcoded "conductor" with path functions
- HOW:
```python
from src import paths
# _render_projects_panel: base = paths.get_conductor_dir()
# _render_projects_panel: track_dir = paths.get_tracks_dir() / track_id
```
- SAFETY: None
## Phase 5: Update GUI
- [ ] Task: Update gui_2.py
- WHERE: src/gui_2.py
- WHAT: Use centralized paths
- HOW: Import from paths module
Focus: Migrate gui_2.py path references
## Phase 6: Update Other Modules
- [ ] Task: Update aggregate.py
- WHERE: src/aggregate.py
- WHAT: Use config path from resolver
- HOW: Import from paths module
- [ ] Task: Update session_logger.py
- WHERE: src/session_logger.py
- WHAT: Use scripts_dir from resolver
- HOW: Import from paths module
- [ ] Task: Update other files with hardcoded paths
- [ ] Task 5.1: Update gui_2.py log path
- WHERE: `src/gui_2.py` line 776
- WHAT: Replace hardcoded LogRegistry path
- HOW:
```python
from src import paths
# LogRegistry(paths.get_logs_dir() / "log_registry.toml")
```
- SAFETY: None
## Phase 7: Config & Env Vars
- [ ] Task: Add paths to config.toml
- WHERE: config.toml
- WHAT: Add path configuration section
- HOW: toml section [paths]
- [ ] Task: Document environment variables
- WHERE: docs/ or README
- WHAT: Document all path env vars
## Phase 6: Configuration Support
Focus: Add paths section to config.toml
## Phase 8: Verification
- [ ] Task: Test with custom paths
- [ ] Task: Test default behavior
- [ ] Task: Run test suite
- [ ] Task: Conductor - Phase Verification
- [ ] Task 6.1: Add [paths] section to config.toml
- WHERE: `config.toml`
- WHAT: Add documented paths section
- HOW:
```toml
# Path Configuration (optional - defaults shown)
# Override with environment variables: SLOP_CONDUCTOR_DIR, SLOP_LOGS_DIR, SLOP_SCRIPTS_DIR
[paths]
# conductor_dir = "conductor"
# logs_dir = "logs/sessions"
# scripts_dir = "scripts/generated"
```
- SAFETY: Comments-only defaults don't change behavior
- [ ] Task 6.2: Document environment variables
- WHERE: `conductor/tech-stack.md` or `README.md`
- WHAT: Add documentation for path environment variables
- HOW: Table of env vars with descriptions
## Phase 7: Verification
Focus: Ensure all changes work correctly
- [ ] Task 7.1: Run full test suite
- COMMAND: `uv run pytest tests/ -v --timeout=60`
- EXPECTED: All existing tests pass
- BATCHING: Run in batches of 4 files max
- [ ] Task 7.2: Test with custom paths
- HOW: Set env vars, verify app uses custom paths
- VERIFY: Check logs go to custom dir, tracks load from custom dir
- [ ] Task 7.3: Test default behavior unchanged
- HOW: Run without env vars, verify defaults work
- VERIFY: All paths resolve to original defaults
- [ ] Task 7.4: Conductor - Phase Verification
- Run: `uv run pytest tests/test_paths.py -v`
- Manual: Start app, verify no path-related errors
## Implementation Notes
### Import Order Considerations
- `paths.py` must not import from other src modules
- Other modules can safely import from `paths.py`
- `session_logger.py` needs careful handling due to module-level state
### Backward Compatibility
- All existing `base_dir` parameters continue to work
- Default behavior is unchanged when no config/env overrides
- Existing `SLOP_CONFIG` env var pattern is preserved
### Files Modified
- `src/paths.py` (new)
- `src/orchestrator_pm.py`
- `src/project_manager.py`
- `src/session_logger.py`
- `src/app_controller.py`
- `src/gui_2.py`
- `config.toml`
- `tests/test_paths.py` (new)
### Code Style Checklist
- [ ] 1-space indentation throughout
- [ ] CRLF line endings on Windows
- [ ] No comments unless documenting public API
- [ ] Type hints on all public functions
- [ ] Follow existing module patterns

199
conductor/tracks/conductor_path_configurable_20260306/spec.md
View File

@@ -1,36 +1,185 @@
# Track Specification: Conductor Path Configuration (conductor_path_configurable_20260306)
# Track Specification: Conductor Path Configuration (conductor_path_configurable_20260306)
## Overview
Eliminate all hardcoded paths in the application. Make directory paths configurable via config.toml or environment variables, allowing the running app to use different directories from development setup.
Eliminate all hardcoded paths in the application. Make directory paths configurable via `config.toml` or environment variables, allowing the running app to use different directories from development setup. This is **Phase 0 - Critical Infrastructure** that must be completed before other Phase 3 tracks.
## Current State Audit
### Already Implemented
- CONDUCTOR_PATH in orchestrator_pm.py is hardcoded to Path(\"conductor\")
- project_manager.py uses Path(base_dir) / \"conductor\" / \"tracks\"
- app_controller.py hardcodes \"conductor\" and \"conductor/tracks\"
- logs/sessions hardcoded in app_controller.py, gui_2.py
- config.toml hardcoded in aggregate.py (but models.py has SLOP_CONFIG env var)
- scripts/generated hardcoded in session_logger.py
### Gaps to Fill
- No config-based path overrides
- No environment variable support for most paths
### Already Implemented (DO NOT re-implement)
## Goals
- Make all directory paths configurable
- Support config.toml and environment variables
- Backward compatible (defaults remain)
- Centralized path resolver
#### Environment Variable Pattern (models.py)
- **`CONFIG_PATH`**: `Path(os.environ.get("SLOP_CONFIG", "config.toml"))` - This pattern exists and should be replicated.
#### Hardcoded Path Inventory
| File | Line | Current Hardcode | Variable/Usage |
|------|------|------------------|----------------|
| `src/orchestrator_pm.py` | 10 | `"conductor"` | `CONDUCTOR_PATH: Path = Path("conductor")` |
| `src/project_manager.py` | 240 | `"conductor"` | `track_dir = Path(base_dir) / "conductor" / "tracks" / track_id` |
| `src/project_manager.py` | 252 | `"conductor"` | `state_file = Path(base_dir) / "conductor" / "tracks" / track_id / "state.toml"` |
| `src/project_manager.py` | 297 | `"conductor"` | `tracks_dir = Path(base_dir) / "conductor" / "tracks"` |
| `src/app_controller.py` | 1907 | `"conductor"` | `base = Path("conductor")` |
| `src/app_controller.py` | 1937 | `"conductor"` | `track_dir = Path("conductor/tracks") / track_id` |
| `src/app_controller.py` | 643 | `"logs/sessions"` | `initialdir="logs/sessions"` |
| `src/app_controller.py` | 674 | `"logs/sessions"` | `LogRegistry("logs/sessions/log_registry.toml")` |
| `src/app_controller.py` | 1241 | `"logs/sessions"` | `log_dir = Path("logs/sessions")` |
| `src/gui_2.py` | 776 | `"logs/sessions"` | `LogRegistry("logs/sessions/log_registry.toml")` |
| `src/session_logger.py` | 26 | `"./logs/sessions"` | `_LOG_DIR: Path = Path("./logs/sessions")` |
| `src/session_logger.py` | 27 | `"./scripts/generated"` | `_SCRIPTS_DIR: Path = Path("./scripts/generated")` |
#### Notes on Existing Implementation
- `session_logger.py` has module-level path constants that are set once at import time
- `project_manager.py` uses `base_dir` parameter but hardcodes `"conductor"` as subdirectory
- `app_controller.py` has multiple scattered hardcodes that need consolidation
### Gaps to Fill (This Track's Scope)
- No centralized path configuration module
- No `config.toml` section for paths
- No environment variable support for logs, scripts, or conductor directories
- Duplicate path definitions across files
## Architectural Constraints
### Single Source of Truth
- All paths MUST be resolved through a single `paths.py` module
- No file should hardcode directory strings directly
### Initialization Order
- Path resolution MUST happen before any module imports that use paths
- `session_logger.py` imports at module level - may need lazy initialization
### Backward Compatibility
- Default paths MUST remain unchanged (relative to project root)
- Existing `SLOP_CONFIG` env var pattern MUST be preserved
- All existing function signatures that take `base_dir` MUST continue to work
### Thread Safety
- Path resolution is read-only after initialization
- No locks needed once paths are resolved
## Architecture Reference
### Key Integration Points
| File | Lines | Purpose |
|------|-------|---------|
| `src/models.py` | 27-28 | `CONFIG_PATH` pattern to replicate |
| `src/orchestrator_pm.py` | 10 | `CONDUCTOR_PATH` to replace |
| `src/project_manager.py` | 238-297 | Track state paths to update |
| `src/session_logger.py` | 26-27 | `_LOG_DIR`, `_SCRIPTS_DIR` to update |
| `src/app_controller.py` | 643, 674, 1241, 1907, 1937 | Multiple hardcodes to fix |
| `src/gui_2.py` | 776 | LogRegistry path to update |
| `config.toml` | N/A | Add `[paths]` section |
### Proposed Module Structure
```python
# src/paths.py (NEW FILE)
from pathlib import Path
import os
from typing import Optional
import tomllib
_CONFIG_PATH: Path = Path(os.environ.get("SLOP_CONFIG", "config.toml"))
_RESOLVED: dict[str, Path] = {}
def _resolve_path(env_var: str, config_key: str, default: Path) -> Path:
"""Resolve path from env var, config, or default."""
if env_var in os.environ:
return Path(os.environ[env_var])
if _CONFIG_PATH.exists():
with open(_CONFIG_PATH, "rb") as f:
cfg = tomllib.load(f)
if "paths" in cfg and config_key in cfg["paths"]:
return Path(cfg["paths"][config_key])
return default
def get_conductor_dir() -> Path: ...
def get_logs_dir() -> Path: ...
def get_scripts_dir() -> Path: ...
def get_config_path() -> Path: ...
```
## Functional Requirements
- Config file support in config.toml for all paths
- Environment variable support for all paths
- Runtime path resolution
- All modules use centralized path resolver
### FR1: Config File Support
Add `[paths]` section to `config.toml`:
```toml
[paths]
conductor_dir = "conductor"
logs_dir = "logs/sessions"
scripts_dir = "scripts/generated"
# config_file uses SLOP_CONFIG env var (existing)
```
### FR2: Environment Variable Support
| Env Var | Config Key | Default |
|---------|------------|---------|
| `SLOP_CONDUCTOR_DIR` | `conductor_dir` | `"conductor"` |
| `SLOP_LOGS_DIR` | `logs_dir` | `"logs/sessions"` |
| `SLOP_SCRIPTS_DIR` | `scripts_dir` | `"scripts/generated"` |
| `SLOP_CONFIG` | (existing) | `"config.toml"` |
### FR3: Centralized Path Module
Create `src/paths.py` with:
- `get_conductor_dir() -> Path`
- `get_logs_dir() -> Path`
- `get_scripts_dir() -> Path`
- `get_config_path() -> Path`
- `get_tracks_dir() -> Path` (conductor/tracks)
- `get_track_state_dir(track_id: str) -> Path`
### FR4: Module Migration
Update all modules to import from `paths.py`:
- `orchestrator_pm.py`: Use `paths.get_conductor_dir()`
- `project_manager.py`: Use `paths.get_tracks_dir()`
- `session_logger.py`: Use `paths.get_logs_dir()`, `paths.get_scripts_dir()`
- `app_controller.py`: Use all path functions
- `gui_2.py`: Use `paths.get_logs_dir()`
## Non-Functional Requirements
| Requirement | Constraint |
|-------------|------------|
| Import Order | `paths.py` must be importable without side effects |
| Lazy Resolution | Path resolution on first access, not at import |
| No Breaking Changes | All existing code continues to work |
| Default Behavior | Unchanged when no config/env overrides |
## Testing Requirements
### Unit Tests
- Test each path function returns expected default
- Test env var override for each path
- Test config.toml override for each path
- Test env var takes precedence over config
### Integration Tests
- Verify app starts with custom paths
- Verify tracks load from custom conductor dir
- Verify logs write to custom logs dir
### Test Isolation
- Reset `_RESOLVED` dict between tests
- Mock `os.environ` for env var tests
- Use temp config files for config tests
## Out of Scope
- Runtime path changes (paths are resolved once at startup)
- Path validation (directory existence checks)
- Relative path resolution (always resolve to absolute)
## Acceptance Criteria
- [ ] config.toml has options for: conductor_dir, logs_dir, config_file, scripts_dir
- [ ] Environment variables work: CONDUCTOR_DIR, LOGS_DIR, SLOP_CONFIG, SCRIPTS_DIR
- [ ] `src/paths.py` module created with all path functions
- [ ] `config.toml` has `[paths]` section
- [ ] All 4 environment variables work
- [ ] Default paths remain unchanged
- [ ] All modules use resolved paths
- [ ] Backward compatible
- [ ] `orchestrator_pm.py` uses `paths.get_conductor_dir()`
- [ ] `project_manager.py` uses path functions
- [ ] `session_logger.py` uses path functions
- [ ] `app_controller.py` uses path functions
- [ ] `gui_2.py` uses path functions
- [ ] All existing tests pass
- [ ] New path resolution tests pass
- [ ] 1-space indentation maintained