docs: Add AI Server IPC design spec
This commit is contained in:
@@ -0,0 +1,110 @@
|
|||||||
|
# AI Server IPC Design
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Decouple heavy AI SDK imports (google.genai, anthropic) from the GUI process via a subprocess command queue. GUI starts instantly (~0.5s) while AI server loads in background (~1.2s one-time cost).
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
GUI Process AI Server Process
|
||||||
|
+-----------+ +------------------+
|
||||||
|
| Command |----pipe/json--->| AI processing |
|
||||||
|
| Queue | | (google.genai, |
|
||||||
|
+-----------+ | anthropic) |
|
||||||
|
| Response |<---pipe/json-----|------------------+
|
||||||
|
| Queue | | |
|
||||||
|
+-----------+ +------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
## Command Queue (Input to AI Server)
|
||||||
|
|
||||||
|
Format: JSON lines on stdin
|
||||||
|
```json
|
||||||
|
{"id": "uuid", "method": "send", "params": {...}}
|
||||||
|
{"id": "uuid", "method": "list_models", "params": {}}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Response Queue (Output from AI Server)
|
||||||
|
|
||||||
|
Format: JSON lines on stdout
|
||||||
|
```json
|
||||||
|
{"id": "uuid", "result": {...}}
|
||||||
|
{"id": "uuid", "error": "message"}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
| Method | Params | Description |
|
||||||
|
|--------|--------|-------------|
|
||||||
|
| `send` | `{history, model, provider, tools}` | Send AI request |
|
||||||
|
| `list_models` | `{provider}` | List available models |
|
||||||
|
| `cleanup` | `{}` | Cleanup sessions |
|
||||||
|
| `reset_session` | `{}` | Reset conversation history |
|
||||||
|
| `set_provider` | `{provider, model}` | Switch provider |
|
||||||
|
| `set_credentials` | `{creds}` | Set API credentials |
|
||||||
|
|
||||||
|
## AI Server Lifecycle
|
||||||
|
|
||||||
|
1. **Spawn**: `subprocess.Popen(["python", "-m", "src.ai_server"])`
|
||||||
|
2. **Startup**: Load google.genai, anthropic SDKs (~1.2s)
|
||||||
|
3. **Ready**: Send `{"type": "ready"}` to GUI
|
||||||
|
4. **Process**: Read commands, write responses
|
||||||
|
5. **Shutdown**: On GUI exit or disconnect
|
||||||
|
|
||||||
|
## GUI Response
|
||||||
|
|
||||||
|
- **Immediate return** from command queue operations
|
||||||
|
- **Background thread reads** response queue
|
||||||
|
- **No polling** - blocking read with timeout
|
||||||
|
- **Lock-free** queue operations
|
||||||
|
|
||||||
|
## Status Indicator
|
||||||
|
|
||||||
|
GUI tracks AI server state:
|
||||||
|
- `init` - Server starting
|
||||||
|
- `ready` - Server loaded, accepting requests
|
||||||
|
- `busy` - Processing request
|
||||||
|
- `error` - Server error state
|
||||||
|
|
||||||
|
Panels that need AI show "Initializing..." tint when `status != ready`.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
### Files
|
||||||
|
|
||||||
|
- `src/ai_server.py` - Subprocess AI server (new)
|
||||||
|
- `src/ai_client_proxy.py` - Queue client for GUI (new)
|
||||||
|
- Modify `src/ai_client.py` - Route via proxy when AI server enabled
|
||||||
|
|
||||||
|
### ai_server.py
|
||||||
|
|
||||||
|
```
|
||||||
|
- stdin reader loop
|
||||||
|
- Command dispatcher
|
||||||
|
- Provider wrappers (google.genai, anthropic)
|
||||||
|
- stdout writer
|
||||||
|
```
|
||||||
|
|
||||||
|
### ai_client_proxy.py
|
||||||
|
|
||||||
|
```
|
||||||
|
- Command queue (subprocess.stdin)
|
||||||
|
- Response queue (subprocess.stdout reader thread)
|
||||||
|
- Request/response matching by ID
|
||||||
|
- Timeout handling
|
||||||
|
```
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
- **Server crash**: GUI detects via broken pipe, auto-restart server
|
||||||
|
- **Timeout**: Requests timeout after 60s, return error
|
||||||
|
- **Queue full**: Backpressure, return busy status
|
||||||
|
|
||||||
|
## Startup Sequence
|
||||||
|
|
||||||
|
1. GUI starts, shows immediate (~0.5s)
|
||||||
|
2. Spawn ai_server subprocess
|
||||||
|
3. Server loads SDKs (~1.2s)
|
||||||
|
4. Server sends `{"type": "ready"}`
|
||||||
|
5. GUI enables AI panels
|
||||||
Reference in New Issue
Block a user