AI Server IPC Design

Overview

Decouple heavy AI SDK imports (google.genai, anthropic) from the GUI process via a subprocess command queue. GUI starts instantly (~0.5s) while AI server loads in background (~1.2s one-time cost).

Architecture

GUI Process                    AI Server Process
+-----------+                 +------------------+
| Command   |----pipe/json--->| AI processing    |
| Queue     |                 | (google.genai,  |
+-----------+                 |  anthropic)     |
| Response  |<---pipe/json-----|------------------+
| Queue     |                 |                 |
+-----------+                 +------------------+

Command Queue (Input to AI Server)

Format: JSON lines on stdin

{"id": "uuid", "method": "send", "params": {...}}
{"id": "uuid", "method": "list_models", "params": {}}

Response Queue (Output from AI Server)

Format: JSON lines on stdout

{"id": "uuid", "result": {...}}
{"id": "uuid", "error": "message"}

Commands

Method	Params	Description
`send`	`{history, model, provider, tools}`	Send AI request
`list_models`	`{provider}`	List available models
`cleanup`	`{}`	Cleanup sessions
`reset_session`	`{}`	Reset conversation history
`set_provider`	`{provider, model}`	Switch provider
`set_credentials`	`{creds}`	Set API credentials

AI Server Lifecycle

Spawn: subprocess.Popen(["python", "-m", "src.ai_server"])
Startup: Load google.genai, anthropic SDKs (~1.2s)
Ready: Send {"type": "ready"} to GUI
Process: Read commands, write responses
Shutdown: On GUI exit or disconnect

GUI Response

Immediate return from command queue operations
Background thread reads response queue
No polling - blocking read with timeout
Lock-free queue operations

Status Indicator

GUI tracks AI server state:

init - Server starting
ready - Server loaded, accepting requests
busy - Processing request
error - Server error state

Panels that need AI show "Initializing..." tint when status != ready.

Implementation

Files

src/ai_server.py - Subprocess AI server (new)
src/ai_client_proxy.py - Queue client for GUI (new)
Modify src/ai_client.py - Route via proxy when AI server enabled

ai_server.py

- stdin reader loop
- Command dispatcher
- Provider wrappers (google.genai, anthropic)
- stdout writer

ai_client_proxy.py

- Command queue (subprocess.stdin)
- Response queue (subprocess.stdout reader thread)
- Request/response matching by ID
- Timeout handling

Error Handling

Server crash: GUI detects via broken pipe, auto-restart server
Timeout: Requests timeout after 60s, return error
Queue full: Backpressure, return busy status

Startup Sequence

GUI starts, shows immediate (~0.5s)
Spawn ai_server subprocess
Server loads SDKs (~1.2s)
Server sends {"type": "ready"}
GUI enables AI panels

3.0 KiB Raw Blame History