new track

conductor/tracks.md

@@ -84,6 +84,9 @@ This file tracks all major tracks for the project. Each track has its own detail
 19. [ ] **Track: Manual UX Validation & Review**
     *Link: [./tracks/manual_ux_validation_20260302/](./tracks/manual_ux_validation_20260302/)*
 
+13. [~] **Track: Enhanced Context Control & Cache Awareness**
+    *Link: [./tracks/enhanced_context_control_20260307/](./tracks/enhanced_context_control_20260307/)*
+
 ---
 
 ## Completed / Archived

conductor/tracks/enhanced_context_control_20260307/metadata.json

@@ -0,0 +1,9 @@
+{
+  "id": "enhanced_context_control_20260307",
+  "name": "Enhanced Context Control & Cache Awareness",
+  "status": "planned",
+  "created_at": "2026-03-07T00:00:00Z",
+  "updated_at": "2026-03-07T00:00:00Z",
+  "type": "feature",
+  "priority": "high"
+}

conductor/tracks/enhanced_context_control_20260307/plan.md

@@ -0,0 +1,35 @@
+# Implementation Plan: Enhanced Context Control & Cache Awareness (enhanced_context_control_20260307)
+
+> **Reference:** [Spec](./spec.md) | [Architecture Guide](../../../docs/guide_architecture.md)
+
+## Phase 1: Data Model & Project Configuration
+Focus: Update the underlying structures to support per-file flags.
+
+- [ ] Task 1.1: Update `FileItem` dataclass/model to include `auto_aggregate` and `force_full` flags.
+- [ ] Task 1.2: Modify `project_manager.py` to parse and serialize these new flags in `manual_slop.toml` or `project.toml`. Ensure backward compatibility with existing simple file lists.
+
+## Phase 2: Context Builder Updates
+Focus: Make the context aggregation logic respect the new flags.
+
+- [ ] Task 2.1: Update `aggregate.py` to filter out files where `auto_aggregate` is False.
+- [ ] Task 2.2: Modify skeleton generation logic in `aggregate.py` to send full content when `force_full` is True.
+- [ ] Task 2.3: Add support for manual 'Context' role injections to bypass `auto_aggregate` restrictions for specific turns.
+
+## Phase 3: Gemini Cache Tracking
+Focus: Track and expose API cache state.
+
+- [ ] Task 3.1: Modify `ai_client.py`'s Gemini cache logic to record which file paths/hashes are currently in the active server-side cache.
+- [ ] Task 3.2: Create an event payload (e.g., via `comms_log_callback` or a dedicated queue event) to push the active cache state to the GUI.
+
+## Phase 4: UI Refactoring
+Focus: Update the Files & Media panel and event handlers.
+
+- [ ] Task 4.1: Refactor the Files & Media panel in `gui_2.py` from a list to an ImGui table displaying the files, their checkboxes (Auto-Aggregate, Force Full), and the Cached indicator dot.
+- [ ] Task 4.2: Implement handlers in `_process_pending_gui_tasks` to receive cache state updates and render the indicator dots.
+- [ ] Task 4.3: Wire the table checkboxes to update the in-memory models and trigger project saves (`_save_active_project()`).
+
+## Phase 5: Testing & Verification
+Focus: Ensure stability and adherence to the architecture.
+
+- [ ] Task 5.1: Write unit tests verifying configuration parsing, flag-respecting context aggregation, and cache tracking.
+- [ ] Task 5.2: Perform a manual UI walkthrough to confirm the table displays correctly, checkboxes toggle state, and cache indicators light up when the Gemini API responds.

conductor/tracks/enhanced_context_control_20260307/spec.md

@@ -0,0 +1,42 @@
+# Track Specification: Enhanced Context Control & Cache Awareness (enhanced_context_control_20260307)
+
+## Overview
+Give developers granular control over how files are included in the AI context and provide visibility into the active Gemini cache state. This involves moving away from a simple list of files to a structured format with per-file flags (`auto_aggregate`, `force_full`), revamping the UI to display this state, and updating the context builders and API clients to respect and expose these details.
+
+## Core Requirements
+
+### 1. `project.toml` Schema Update
+- Migrate the `tracked_files` list to a more structured format (or preserve list for compatibility but support dictionaries/objects per file).
+- Support per-file flags:
+  - `auto_aggregate` (bool, default true): Whether to automatically include this file in context aggregation.
+  - `force_full` (bool, default false): Whether to send the full file content, overriding skeleton extraction.
+
+### 2. Files & Media Panel Refactoring
+- Replace the existing simple list/checkboxes in the GUI (`src/gui_2.py`) with a structured table.
+- Columns should include: File Name, Auto-Aggregate (checkbox), Force Full (checkbox), and a 'Cached' indicator (e.g., a green dot).
+- The GUI must reflect real-time updates from the background threads using the established event queue (`_process_pending_gui_tasks`).
+
+### 3. 'Context' Role for Manual Injections
+- Implement a 'Context' role that allows manual file injections into discussions.
+- Context amnesia needs to respect these manual inclusions or properly categorize them.
+
+### 4. `aggregate.py` Updates
+- `build_file_items()` and tier-specific context builders must respect the `auto_aggregate` and `force_full` flags.
+- If `auto_aggregate` is false, the file is omitted unless manually injected.
+- If `force_full` is true, bypass skeleton extraction (like `ASTParser.get_skeleton()`) and include the full file content.
+
+### 5. `ai_client.py` Cache Tracking
+- Add state tracking for the active Gemini cache (e.g., tracking which file hashes/paths are currently embedded in the `CachedContent`).
+- Expose this state back to the UI (via `AsyncEventQueue` and `mma_state_update` or a dedicated `"refresh_api_metrics"` action) so the GUI can render the 'Cached' indicator dots.
+- Ensure thread safety (`_send_lock` and appropriate variable locks) when updating and reading cache state.
+
+## Architectural Constraints
+- Follow the 1-space indentation rule for Python.
+- Obey the decoupling of GUI (main thread) and asyncio background workers. All UI state mutations must occur via `_process_pending_gui_tasks`.
+- No new third-party dependencies unless strictly necessary.
+
+## Key Integration Points
+- `src/project_manager.py`: TOML serialization/deserialization for tracked files.
+- `src/gui_2.py`: The "Files & Media" panel and `_process_pending_gui_tasks`.
+- `src/aggregate.py`: Context building logic.
+- `src/ai_client.py`: Gemini API cache tracking.