manual_slop/conductor/tracks/rag_support_20260308/plan.md

Implementation Plan: RAG Support

Phase 1: Foundation & Vector Store Integration

• Task: Define the RAG architecture and configuration schema.
  • Update src/models.py to include RAGConfig and VectorStoreConfig.
  • Implement configuration loading/saving in AppController.
• Task: Integrate a local vector store.
  • Add chromadb or qdrant-client to requirements.txt.
  • Create src/rag_engine.py to manage the vector database lifecycle (init, add, search, delete).
• Task: Implement embedding providers.
  • Implement Gemini embedding wrapper in src/rag_engine.py.
  • Implement local embedding wrapper (e.g., using sentence-transformers) in src/rag_engine.py.
• Task: Write unit tests for vector store operations and embedding generation.
• Task: Conductor - User Manual Verification 'Phase 1: Foundation & Vector Store' (Protocol in workflow.md)

Phase 2: Indexing & Retrieval Logic

• Task: Implement the indexing pipeline.
  • Implement file chunking strategies (e.g., character-based, AST-aware) in src/rag_engine.py.
  • Create a background indexing task in AppController.
  • Implement auto-indexing logic triggered by Context Hub changes.
• Task: Implement the retrieval pipeline.
  • Implement similarity search with configurable top-k and threshold.
  • Implement "Native Retrieval" logic for Gemini (leveraging ai_client.py).
• Task: Update ai_client.py to support RAG.
  • Add a retrieve_context() step to the send() loop.
  • Format and inject retrieved fragments into the model's system prompt or context block.
• Task: Write integration tests for the indexing and retrieval flow.
• Task: Conductor - User Manual Verification 'Phase 2: Indexing & Retrieval Logic' (Protocol in workflow.md)

Phase 3: GUI Integration & Visualization

• Task: Implement the RAG Settings panel in src/gui_2.py.
  • Add UI controls for choosing the RAG source, embedding model, and retrieval parameters.
  • Add a "Rebuild Index" button and status progress bar.
• Task: Implement retrieval visualization in the Discussion history.
  • Display "Retrieved Context" blocks with expandable summaries.
  • Add "Source" buttons to each block that open the file at the specific chunk's location.
• Task: Implement auto-start/indexing status indicators in the GUI.
• Task: Write visual regression tests or simulation scripts to verify the RAG UI components.
• Task: Conductor - User Manual Verification 'Phase 3: GUI Integration & Visualization' (Protocol in workflow.md)

Phase 4: Refinement & Advanced RAG

• Task: Implement support for external RAG APIs/MCP servers.
  • Create a bridge in src/rag_engine.py to call external RAG tools via the MCP interface.
• Task: Optimize indexing performance for large projects (e.g., incremental updates, parallel chunking).
• Task: Perform a final end-to-end verification with a large codebase.
• Task: Conductor - User Manual Verification 'Phase 4: Refinement & Advanced RAG' (Protocol in workflow.md)