From 9a66b7697ec8c5cb45a8929e98d2e43496fc1336 Mon Sep 17 00:00:00 2001 From: Ed_ Date: Thu, 26 Feb 2026 08:46:25 -0500 Subject: [PATCH] chore(conductor): Add new track 'Review logging used throughout the project' --- conductor/tracks.md | 5 +++ .../tracks/logging_refactor_20260226/index.md | 5 +++ .../logging_refactor_20260226/metadata.json | 8 ++++ .../tracks/logging_refactor_20260226/plan.md | 39 +++++++++++++++++ .../tracks/logging_refactor_20260226/spec.md | 42 +++++++++++++++++++ 5 files changed, 99 insertions(+) create mode 100644 conductor/tracks/logging_refactor_20260226/index.md create mode 100644 conductor/tracks/logging_refactor_20260226/metadata.json create mode 100644 conductor/tracks/logging_refactor_20260226/plan.md create mode 100644 conductor/tracks/logging_refactor_20260226/spec.md diff --git a/conductor/tracks.md b/conductor/tracks.md index d68de94..3eb8863 100644 --- a/conductor/tracks.md +++ b/conductor/tracks.md @@ -40,3 +40,8 @@ This file tracks all major tracks for the project. Each track has its own detail --- +- [ ] **Track: Review logging used throughout the project. THe log directory has several categories of logs and they are getting quite large in number. We need sub-directoreis and we need a way to prune logs that aren't valuable to keep.** +*Link: [./tracks/logging_refactor_20260226/](./tracks/logging_refactor_20260226/)* + +--- + diff --git a/conductor/tracks/logging_refactor_20260226/index.md b/conductor/tracks/logging_refactor_20260226/index.md new file mode 100644 index 0000000..f0c688c --- /dev/null +++ b/conductor/tracks/logging_refactor_20260226/index.md @@ -0,0 +1,5 @@ +# Track logging_refactor_20260226 Context + +- [Specification](./spec.md) +- [Implementation Plan](./plan.md) +- [Metadata](./metadata.json) diff --git a/conductor/tracks/logging_refactor_20260226/metadata.json b/conductor/tracks/logging_refactor_20260226/metadata.json new file mode 100644 index 0000000..b402658 --- /dev/null +++ b/conductor/tracks/logging_refactor_20260226/metadata.json @@ -0,0 +1,8 @@ +{ + "track_id": "logging_refactor_20260226", + "type": "chore", + "status": "new", + "created_at": "2026-02-26T08:45:00Z", + "updated_at": "2026-02-26T08:45:00Z", + "description": "Review logging used throughout the project. The log directory has several categories of logs and they are getting quite large in number. We need sub-directories and we need a way to prune logs that aren't valuable to keep." +} diff --git a/conductor/tracks/logging_refactor_20260226/plan.md b/conductor/tracks/logging_refactor_20260226/plan.md new file mode 100644 index 0000000..9d5ab59 --- /dev/null +++ b/conductor/tracks/logging_refactor_20260226/plan.md @@ -0,0 +1,39 @@ +# Implementation Plan: Logging Reorganization and Automated Pruning + +## Phase 1: Session Organization & Registry Foundation +- [ ] Task: Initialize MMA Environment (Protocol: `activate_skill mma-orchestrator`) +- [ ] Task: Implement `LogRegistry` to manage `log_registry.toml` + - [ ] Define TOML schema for session metadata. + - [ ] Create methods to register sessions and update whitelist status. +- [ ] Task: Implement Session-Based Directory Creation + - [ ] Create utility to generate Session IDs: `YYYYMMDD_HHMMSS[_Label]`. + - [ ] Update logging initialization to create and use session sub-directories. +- [ ] Task: Conductor - User Manual Verification 'Phase 1: Foundation' (Protocol in workflow.md) + +## Phase 2: Pruning Logic & Heuristics +- [ ] Task: Implement `LogPruner` Core Logic + - [ ] Implement time-based filtering (older than 24h). + - [ ] Implement size-based heuristic for "insignificance" (~2 KB). +- [ ] Task: Implement Auto-Whitelisting Heuristics + - [ ] Implement content scanning for `ERROR`, `WARNING`, `EXCEPTION`. + - [ ] Implement complexity detection (message count > 10). +- [ ] Task: Integrate Pruning into App Startup + - [ ] Hook the pruner into `gui_2.py` startup sequence. + - [ ] Ensure pruning runs asynchronously to prevent startup lag. +- [ ] Task: Conductor - User Manual Verification 'Phase 2: Pruning' (Protocol in workflow.md) + +## Phase 3: GUI Integration & Manual Control +- [ ] Task: Add "Log Management" UI Panel + - [ ] Display a list of recent sessions from the registry. + - [ ] Add "Star/Unstar" toggle for manual whitelisting. +- [ ] Task: Display Session Metrics in UI + - [ ] Show size, message count, and status (Whitelisted/Pending Prune). +- [ ] Task: Conductor - User Manual Verification 'Phase 3: GUI' (Protocol in workflow.md) + +## Phase 4: Final Verification & Cleanup +- [ ] Task: Comprehensive Integration Testing + - [ ] Verify that empty old logs are deleted. + - [ ] Verify that complex/error-filled old logs are preserved. +- [ ] Task: Final Refactoring and Documentation + - [ ] Ensure all new classes and methods follow project style. +- [ ] Task: Conductor - User Manual Verification 'Phase 4: Final' (Protocol in workflow.md) diff --git a/conductor/tracks/logging_refactor_20260226/spec.md b/conductor/tracks/logging_refactor_20260226/spec.md new file mode 100644 index 0000000..f046332 --- /dev/null +++ b/conductor/tracks/logging_refactor_20260226/spec.md @@ -0,0 +1,42 @@ +# Specification: Logging Reorganization and Automated Pruning + +## Overview +Currently, `gui_2.py` and the test suites generate a large number of log files in a flat `logs/` directory. These logs accumulate quickly, especially during incremental development and testing. This track aims to organize logs into session-based sub-directories and implement a heuristic-based pruning system to keep the log directory clean while preserving valuable sessions. + +## Functional Requirements +1. **Session-Based Organization:** + - Logs must be stored in sub-directories within `logs/`. + - Sub-directory naming convention: `YYYYMMDD_HHMMSS[_Label]` (e.g., `20260226_143005_feature_x`). + - The "Label" should be included if a project or track is active at session start. +2. **Central Registry:** + - A `logs/log_registry.toml` file will track session metadata, including: + - Session ID / Path + - Start Time + - Whitelist Status (Manual/Auto) + - Metrics (message count, errors detected, total size). +3. **Automated Pruning Heuristic:** + - Pruning triggers on application startup (`gui_2.py`). + - **Target:** Logs older than 24 hours. + - **Exemption:** Whitelisted logs are never auto-pruned. + - **Insignificance Criteria:** Non-whitelisted logs under a specific size threshold (heuristic: ~2 KB) or with zero significant interactions will be purged. +4. **Whitelisting System:** + - **Auto-Whitelisting:** Sessions are marked as "rich" if they meet any of these: + - Complexity: > 10 messages/interactions. + - Diagnostics: Contains `ERROR`, `WARNING`, `EXCEPTION`. + - Major Events: User created a new project or initialized a track. + - **Manual Whitelisting:** The user can "star" a session via the GUI (persisted in the registry). + +## Non-Functional Requirements +- **Performance:** Pruning and registry updates must be asynchronous or extremely fast to avoid delaying app startup. +- **Safety:** Ensure the pruning logic is conservative to prevent accidental data loss of important debug information. + +## Acceptance Criteria +- [ ] New logs are created in session-specific folders. +- [ ] The `log_registry.toml` correctly identifies and tracks sessions. +- [ ] On startup, non-whitelisted logs older than 1 day are successfully pruned. +- [ ] Whitelisted logs (due to complexity or errors) remain untouched. +- [ ] (Bonus) The GUI displays a basic list of sessions with their "starred" status. + +## Out of Scope +- Migrating the entire backlog of existing flat logs (focus is on new sessions). +- Implementing a full-blown log viewer (basic metadata view only).