Skip to main content

​
Overview

Autopilot is a native Rust + WGPU desktop application that provides a unified interface for AI coding agents. Codex is wired today; other adapters are present in the repo but not enabled by default.

​
Architecture Layers

UI Layer (WGPUI / Rust)
  - Autopilot UI components and chat panes
  - Immediate‑mode render tree with retained state
  - Hotbar + pane HUD

Desktop Host (winit + wgpu)
  - Window + GPU swapchain
  - Input routing (keyboard/mouse)
  - Render loop

App Core (Rust)
  - Session lifecycle and event routing
  - Codex message send/receive
  - Tool call formatting + UI state

Agent/Protocol Layer (Rust)
  - Codex app‑server client
  - ACP/event parsing
  - Unified event normalization

​
Core Event Flows

​
Unified Event Flow

  1. User sends a prompt in the UI.
  2. Desktop app forwards the message to Codex via the app‑server client.
  3. ACP/app events stream back from the agent.
  4. App core maps events into formatted UI state.
  5. WGPUI renders updated conversation items.

​
UI Event Flow

  1. Session starts and the app initializes UI state.
  2. Incoming events update the in‑memory view model.
  3. WGPUI re-renders the updated components immediately.

​
Key Components

​
App Core (Rust)

  • autopilot_app: session lifecycle and event routing
  • codex-client: app‑server client and ACP event parsing
  • autopilot_ui: WGPUI view state + components

​
Desktop Host (Rust)

  • apps/autopilot-desktop: winit + wgpu host, render loop, input routing

​
Session ID Management

Autopilot uses a two-phase session ID flow:
  1. Temporary session ID from connect_unified_agent
  2. Actual ACP session ID emitted via SessionStarted
All subsequent messages use the actual ACP session ID.

​
Implementation Status

Completed:
  • Native WGPUI desktop host and pane system
  • Codex app‑server integration and event stream handling
  • Tool call cards and markdown rendering
In progress:
  • Additional pane types and tooling UI
  • Improved event formatting and message history
Planned:
  • Agent selection UI
  • Conversation persistence and search

​
File Map (High Level)

  • apps/autopilot-desktop/src/ — desktop host (winit + wgpu)
  • crates/autopilot_app/ — app core + event routing
  • crates/autopilot_ui/ — WGPUI views + components
  • crates/codex-client/ — Codex app‑server client + event parsing