docs: Add AI Server IPC design spec

2026-05-13 08:33:09 -04:00
parent 93c5320fa0
commit 0c79e76bad
1 changed files with 110 additions and 0 deletions
@@ -0,0 +1,110 @@
+# AI Server IPC Design
+
+## Overview
+
+Decouple heavy AI SDK imports (google.genai, anthropic) from the GUI process via a subprocess command queue. GUI starts instantly (~0.5s) while AI server loads in background (~1.2s one-time cost).
+
+## Architecture
+
+```
+GUI Process                    AI Server Process
+-----------+                 +------------------+
+| Command   |----pipe/json--->| AI processing    |
+| Queue     |                 | (google.genai,  |
+-----------+                 |  anthropic)     |
+| Response  |<---pipe/json-----|------------------+
+| Queue     |                 |                 |
+-----------+                 +------------------+
+```
+
+## Command Queue (Input to AI Server)
+
+Format: JSON lines on stdin
+```json
+{"id": "uuid", "method": "send", "params": {...}}
+{"id": "uuid", "method": "list_models", "params": {}}
+```
+
+## Response Queue (Output from AI Server)
+
+Format: JSON lines on stdout
+```json
+{"id": "uuid", "result": {...}}
+{"id": "uuid", "error": "message"}
+```
+
+## Commands
+
+| Method | Params | Description |
+|--------|--------|-------------|
+| `send` | `{history, model, provider, tools}` | Send AI request |
+| `list_models` | `{provider}` | List available models |
+| `cleanup` | `{}` | Cleanup sessions |
+| `reset_session` | `{}` | Reset conversation history |
+| `set_provider` | `{provider, model}` | Switch provider |
+| `set_credentials` | `{creds}` | Set API credentials |
+
+## AI Server Lifecycle
+
+1. **Spawn**: `subprocess.Popen(["python", "-m", "src.ai_server"])`
+2. **Startup**: Load google.genai, anthropic SDKs (~1.2s)
+3. **Ready**: Send `{"type": "ready"}` to GUI
+4. **Process**: Read commands, write responses
+5. **Shutdown**: On GUI exit or disconnect
+
+## GUI Response
+
+- **Immediate return** from command queue operations
+- **Background thread reads** response queue
+- **No polling** - blocking read with timeout
+- **Lock-free** queue operations
+
+## Status Indicator
+
+GUI tracks AI server state:
+- `init` - Server starting
+- `ready` - Server loaded, accepting requests
+- `busy` - Processing request
+- `error` - Server error state
+
+Panels that need AI show "Initializing..." tint when `status != ready`.
+
+## Implementation
+
+### Files
+
+- `src/ai_server.py` - Subprocess AI server (new)
+- `src/ai_client_proxy.py` - Queue client for GUI (new)
+- Modify `src/ai_client.py` - Route via proxy when AI server enabled
+
+### ai_server.py
+
+```
+- stdin reader loop
+- Command dispatcher
+- Provider wrappers (google.genai, anthropic)
+- stdout writer
+```
+
+### ai_client_proxy.py
+
+```
+- Command queue (subprocess.stdin)
+- Response queue (subprocess.stdout reader thread)
+- Request/response matching by ID
+- Timeout handling
+```
+
+## Error Handling
+
+- **Server crash**: GUI detects via broken pipe, auto-restart server
+- **Timeout**: Requests timeout after 60s, return error
+- **Queue full**: Backpressure, return busy status
+
+## Startup Sequence
+
+1. GUI starts, shows immediate (~0.5s)
+2. Spawn ai_server subprocess
+3. Server loads SDKs (~1.2s)
+4. Server sends `{"type": "ready"}`
+5. GUI enables AI panels