diff --git a/conductor/tracks.md b/conductor/tracks.md index 64a0ff6..efb8f60 100644 --- a/conductor/tracks.md +++ b/conductor/tracks.md @@ -66,6 +66,14 @@ This file tracks all major tracks for the project. Each track has its own detail --- +### Model Providers + +12. [ ] **Track: OpenAI Provider Integration** + *Link: [./tracks/openai_integration_20260308/](./tracks/openai_integration_20260308/)* + *Goal: Add support for OpenAI as a first-class model provider (GPT-4o, GPT-4o-mini, o1, o3-mini). Achieve functional parity with Gemini/Anthropic, including Vision, Structured Output, and response streaming.* + +--- + ## Phase 3: Future Horizons *Initialized: 2026-03-06* *Updated: 2026-03-08* diff --git a/conductor/tracks/openai_integration_20260308/index.md b/conductor/tracks/openai_integration_20260308/index.md new file mode 100644 index 0000000..6aa4126 --- /dev/null +++ b/conductor/tracks/openai_integration_20260308/index.md @@ -0,0 +1,5 @@ +# Track openai_integration_20260308 Context + +- [Specification](./spec.md) +- [Implementation Plan](./plan.md) +- [Metadata](./metadata.json) diff --git a/conductor/tracks/openai_integration_20260308/metadata.json b/conductor/tracks/openai_integration_20260308/metadata.json new file mode 100644 index 0000000..7425290 --- /dev/null +++ b/conductor/tracks/openai_integration_20260308/metadata.json @@ -0,0 +1,8 @@ +{ + "track_id": "openai_integration_20260308", + "type": "feature", + "status": "new", + "created_at": "2026-03-08T13:47:00Z", + "updated_at": "2026-03-08T13:47:00Z", + "description": "Add support for openai vendor (GPT/codex)." +} diff --git a/conductor/tracks/openai_integration_20260308/plan.md b/conductor/tracks/openai_integration_20260308/plan.md new file mode 100644 index 0000000..e734911 --- /dev/null +++ b/conductor/tracks/openai_integration_20260308/plan.md @@ -0,0 +1,44 @@ +# Implementation Plan: OpenAI Provider Integration + +## Phase 1: Core Client Implementation +- [ ] Task: Define OpenAI state and initialize client in `src/ai_client.py`. + - [ ] Import `openai` (ensure it's added to `requirements.txt` if missing). + - [ ] Add `_openai_client` and `_openai_history` module-level variables. + - [ ] Implement `_classify_openai_error(exc)` for structured error mapping. +- [ ] Task: Implement `_send_openai` tool-call loop. + - [ ] Implement function/tool definition conversion to OpenAI format. + - [ ] Implement the core Chat Completions API call with streaming support. + - [ ] Implement tool result packaging and the recursion loop (up to `MAX_TOOL_ROUNDS`). + - [ ] Integrate `_reread_file_items` for context refresh after tool rounds. +- [ ] Task: Implement Multi-modal (Vision) support. + - [ ] Add logic to `_send_openai` to process `screenshot` inputs by encoding them as base64 data URIs. +- [ ] Task: Implement model discovery. + - [ ] Implement `_list_openai_models()` using the `client.models.list()` API. +- [ ] Task: Update `ai_client.py` utility functions. + - [ ] Update `send()` dispatcher to route to `_send_openai`. + - [ ] Update `reset_session()` to clear `_openai_history`. +- [ ] Task: Conductor - User Manual Verification 'Phase 1: Core Client Implementation' (Protocol in workflow.md) + +## Phase 2: Configuration & Credentials +- [ ] Task: Update credential loading in `src/ai_client.py`. + - [ ] Update `_load_credentials()` to include `openai` in the error message and loading logic. +- [ ] Task: Add cost tracking for OpenAI models in `src/cost_tracker.py`. + - [ ] Add regex patterns and rates for `gpt-4o`, `gpt-4o-mini`, `o1`, and `o3-mini` to `MODEL_PRICING`. +- [ ] Task: Verify that `credentials.toml` works with the new provider section. +- [ ] Task: Conductor - User Manual Verification 'Phase 2: Configuration & Credentials' (Protocol in workflow.md) + +## Phase 3: GUI & Controller Integration +- [ ] Task: Register `openai` as a provider. + - [ ] Add `openai` to `PROVIDERS` in `src/gui_2.py`. + - [ ] Add `openai` to `PROVIDERS` in `src/app_controller.py`. +- [ ] Task: Ensure model fetching works in the GUI. + - [ ] Verify that clicking "Fetch Models" in the AI Settings panel correctly populates OpenAI models when the provider is selected. +- [ ] Task: Conductor - User Manual Verification 'Phase 3: GUI & Controller Integration' (Protocol in workflow.md) + +## Phase 4: MMA Integration & Final Verification +- [ ] Task: Verify MMA compatibility. + - [ ] Test a simple multi-agent workflow where a Tier 3 worker is configured to use `gpt-4o-mini`. + - [ ] Verify that tool calls, history, and context injection work as expected within the tiered architecture. +- [ ] Task: Run full regression suite. + - [ ] Ensure adding OpenAI hasn't introduced side effects for Gemini or Anthropic providers. +- [ ] Task: Conductor - User Manual Verification 'Phase 4: MMA Integration & Final Verification' (Protocol in workflow.md) diff --git a/conductor/tracks/openai_integration_20260308/spec.md b/conductor/tracks/openai_integration_20260308/spec.md new file mode 100644 index 0000000..c5ead4f --- /dev/null +++ b/conductor/tracks/openai_integration_20260308/spec.md @@ -0,0 +1,59 @@ +# Specification: OpenAI Provider Integration + +## Overview +This track introduces support for OpenAI as a first-class model provider. It involves implementing a dedicated client in `src/ai_client.py`, updating configuration models, enhancing the GUI for provider selection, and integrating OpenAI into the tiered MMA architecture. + +## Functional Requirements + +### 1. Core AI Client (`src/ai_client.py`) +- **OpenAI Integration:** + - Implement `_send_openai()` to handle communication with OpenAI's Chat Completions API. + - Implement a tool-call loop similar to `_send_gemini` and `_send_anthropic`. + - Support for function calling (using `tools` and `tool_choice`). + - Support for multi-modal input (Vision) by encoding screenshots as base64 data URIs in the message payload. + - Implement response streaming support compatible with the existing GUI mechanism. +- **State Management:** + - Define module-level `_openai_client` and `_openai_history`. + - Ensure `ai_client.reset_session()` clears OpenAI-specific history. +- **Error Handling:** + - Implement `_classify_openai_error()` to map OpenAI API exceptions to `ProviderError` types (`quota`, `rate_limit`, `auth`, `balance`, `network`). +- **Model Discovery:** + - Implement `_list_openai_models()` to fetch available models from the API. + +### 2. Configuration & Authentication +- **Credentials:** + - Update `_load_credentials()` to support an `[openai]` section in `credentials.toml`. + - Update the `FileNotFoundError` message in `_load_credentials()` to include the OpenAI example. +- **Cost Tracking:** + - Update `src/cost_tracker.py` with pricing for `gpt-4o`, `gpt-4o-mini`, `o1`, and `o3-mini`. + +### 3. GUI & Controller Integration +- **Provider Lists:** + - Add `openai` to the `PROVIDERS` list in `src/gui_2.py` and `src/app_controller.py`. +- **Model Fetching:** + - Ensure `AppController._fetch_models()` correctly dispatches to `ai_client.list_models("openai")`. +- **Settings UI:** + - The AI Settings panel should automatically handle the new provider and its models once added to the lists. + +### 4. MMA Orchestration +- **Tier Support:** + - Verify that agents in all tiers (1-4) can be configured to use OpenAI models via the `mma_tier_usage` dict in `AppController`. + - Ensure `run_worker_lifecycle` in `src/multi_agent_conductor.py` correctly passes OpenAI model names to `ai_client.send()`. + +## Non-Functional Requirements +- **Consistency:** Follow the established pattern of module-level singleton state in `ai_client.py`. +- **Latency:** Ensure the tool-call loop overhead is minimal. +- **Security:** Rigorously protect the OpenAI API key; ensure it is never logged or exposed in the GUI. + +## Acceptance Criteria +- [ ] OpenAI can be selected as a provider in the AI Settings panel. +- [ ] Models like `gpt-4o` and `gpt-4o-mini` are listed and selectable. +- [ ] Agents can successfully use tools (e.g., `read_file`, `run_powershell`) using OpenAI models. +- [ ] Screenshots are correctly processed and described by vision-capable OpenAI models. +- [ ] Response streaming is functional in the Discussion panel. +- [ ] Estimated costs for OpenAI calls are displayed in the MMA Dashboard. + +## Out of Scope +- Support for the legacy `gpt-3.5-turbo-instruct` or other non-chat models. +- Batch API support. +- Fine-tuning management.