manual_slop/conductor/archive/logging_refactor_20260226/spec.md

Specification: Logging Reorganization and Automated Pruning

Overview

Currently, gui_2.py and the test suites generate a large number of log files in a flat logs/ directory. These logs accumulate quickly, especially during incremental development and testing. This track aims to organize logs into session-based sub-directories and implement a heuristic-based pruning system to keep the log directory clean while preserving valuable sessions.

Functional Requirements

1. Session-Based Organization:
   • Logs must be stored in sub-directories within logs/.
   • Sub-directory naming convention: YYYYMMDD_HHMMSS[_Label] (e.g., 20260226_143005_feature_x).
   • The "Label" should be included if a project or track is active at session start.
2. Central Registry:
   • A logs/log_registry.toml file will track session metadata, including:
     • Session ID / Path
     • Start Time
     • Whitelist Status (Manual/Auto)
     • Metrics (message count, errors detected, total size).
3. Automated Pruning Heuristic:
   • Pruning triggers on application startup (gui_2.py).
   • Target: Logs older than 24 hours.
   • Exemption: Whitelisted logs are never auto-pruned.
   • Insignificance Criteria: Non-whitelisted logs under a specific size threshold (heuristic: ~2 KB) or with zero significant interactions will be purged.
4. Whitelisting System:
   • Auto-Whitelisting: Sessions are marked as "rich" if they meet any of these:
     • Complexity: > 10 messages/interactions.
     • Diagnostics: Contains ERROR, WARNING, EXCEPTION.
     • Major Events: User created a new project or initialized a track.
   • Manual Whitelisting: The user can "star" a session via the GUI (persisted in the registry).

Non-Functional Requirements

• Performance: Pruning and registry updates must be asynchronous or extremely fast to avoid delaying app startup.
• Safety: Ensure the pruning logic is conservative to prevent accidental data loss of important debug information.

Acceptance Criteria

• New logs are created in session-specific folders.
• The log_registry.toml correctly identifies and tracks sessions.
• On startup, non-whitelisted logs older than 1 day are successfully pruned.
• Whitelisted logs (due to complexity or errors) remain untouched.
• (Bonus) The GUI displays a basic list of sessions with their "starred" status.

Out of Scope

• Migrating the entire backlog of existing flat logs (focus is on new sessions).
• Implementing a full-blown log viewer (basic metadata view only).