CLI, Terminal & Web/Desktop Interfaces

Omnigent's selling line is that a session follows you — start it in a terminal, keep watching it in the browser, glance at it on your phone. The interface layer makes that real with one architectural choice: nothing lives in the client. State lives on a server; every client — the CLI REPL, an attach watcher, the React app, the iOS WebView — is a thin replica that subscribes to one server-sent event stream and renders it. There is no "primary" UI. They are peers on the same feed.

1. One Entry Point, Many Harnesses

The console script is dead simple — two aliases, one callable (pyproject.toml:251-252):

omnigent = "omnigent.cli:main"
omni     = "omnigent.cli:main"

omnigent/__main__.py re-exports the same main for python -m omnigent. main() (omnigent/cli.py:1240) does light argv massaging before handing off to a Click group:

• Bare omnigent on a TTY rewrites to omnigent run; piped/CI with no TTY falls back to --help so it never hangs on stdin (omnigent/cli.py:1276-1277).
• A leading flag (omnigent --harness claude …) is rewritten to run --harness claude … (omnigent/cli.py:1284-1285).
• A bare agent path (omnigent myagent.yaml) becomes run myagent.yaml (omnigent/cli.py:1290-1291).
• A server URL as the first token is rejected with a hint to use --server (omnigent/cli.py:1293-1299); the removed top-level ad-hoc chat shape is rejected too (omnigent/cli.py:1301-1308).
• Always-on diagnostics are armed before Click dispatches so a crash is captured even without --log.

The Click group is declared at omnigent/cli.py:1160-1170; the full subcommand set is enumerated in _CLICK_SUBCOMMANDS (omnigent/cli.py:1177-1210).

1.1 The Command Surface

The meta-harness identity is visible right in the command list: a dozen sibling commands, each a thin launcher over a different upstream coding agent (Claude Code, Codex, Cursor, Goose, …), all routed through the same server-backed dispatch.

For Swisscheese: this is the shape you want — one run verb that takes --harness {claude,codex,…} and otherwise behaves identically. The supervisor types one command; the underlying agent is a flag, not a fork in the codebase.

1.2 run — the dispatch flow

run (omnigent/cli.py:6380) resolves config defaults (CLI > project config > global config), then derives smart first-run defaults: a bare run with no agent and no --harness picks a harness from whatever creds exist (Claude → polly, else Codex, else Pi) without persisting it, so adding Claude later silently promotes a Codex-only user (omnigent/cli.py:6443-6453). If nothing is configured it drops into the setup wizard.

Two facts about run define the cross-device experience:

1. It auto-opens the browser. An interactive run sets auto_open_conversation so the user discovers the web UI the moment the server is up (omnigent/cli.py:6455-6463). Headless -p one-shots stay quiet.
2. It is server-backed by default. The Click path always drives the "Omnigent server + REPL" topology: a local server is spawned, and the REPL connects to it as an HTTP client (omnigent/cli.py:6402-6406, _dispatch_run at omnigent/cli.py:5889-5915). --server <url> points the REPL at a remote server instead; with a local agent it spawns a local runner that tunnels to the remote server "so terminals/MCPs run on your laptop" (omnigent/cli.py:6351-6357).

So even a single-user omnigent run is already a two-party system — REPL client talking to a server — which is exactly why a browser can join the same session with zero extra machinery.

1.3 attach — the pure watcher

attach (omnigent/cli.py:6266) is the conceptual heart of the "follows-you" promise. It is a thin client that joins a live session and streams its I/O — it never spawns a server, runner, or harness, and errors loudly when there's nothing live (omnigent/cli.py:6272-6299). Its own docstring frames it as the CLI equivalent of the web UI's co-drive (omnigent/cli.py:6304-6306): it dispatches turns to the runner the host already bound, just like a second browser tab would.

run starts; attach watches/co-drives; resume reopens a stored one. Three verbs, three intents.

2. Auth & Config State

omnigent login <url> (omnigent/cli.py:12040) probes the server's auth mode from GET /v1/me and runs the matching flow without a flag (omnigent/cli.py:12085-12132):

• accounts → username/password POST to /auth/login.
• OIDC → opens the browser, polls a CLI ticket endpoint (/auth/cli-login → /auth/cli-poll), stores the JWT (omnigent/cli.py:12136-12159).
• header → proxy injects identity; nothing to do.
• Databricks-fronted → databricks auth login --host <ws>, store a pointer record (no bearer — Databricks tokens expire hourly).

Tokens land in ~/.omnigent/auth_tokens.json, keyed by normalized server URL, written 0o600 (omnigent/cli_auth.py:56-81). Two record shapes coexist: session JWTs (store_token, omnigent/cli_auth.py:84-106) and tokenless Databricks pointers (store_databricks_auth, omnigent/cli_auth.py:109-142) that mint fresh workspace bearers on demand. A successful login also records the server as the user-level default (omnigent/cli.py:12062-12067), so a later bare omnigent targets it.

Two diagnostic layers sit underneath every invocation:

• Always-on CLI log — omnigent/cli_diagnostics.py captures exceptions and lifecycle to ~/.omnigent/logs/cli-*.log even without --log. INFO level logs no prompt/message/tool content, and a redaction filter strips Authorization headers, bearer tokens, *_TOKEN / *_API_KEY / *SECRET* env vars, sk-*, dapi* from the formatted output (omnigent/cli_diagnostics.py:10-21).
• --debug-events — opt-in "SSE-to-UI debug pipeline": a Ctrl+E event-tape overlay, JSONL event log under ~/.omnigent/debug/, and pipeline-stage counters in the toolbar (omnigent/cli.py:6359-6368).

3. The Live-Sync Backbone: One SSE Stream

The whole system rests on one endpoint:

GET /v1/sessions/{session_id}/stream   →  text/event-stream

It's in the OpenAPI spec as "Stream Session" returning text/event-stream, and it's the single mechanism every client uses to follow a session in real time. The CLI REPL subscribes via self._client.sessions.stream(session_id) and pushes each event through _on_event (omnigent/repl/_repl.py:1900-1941); it auto-reconnects with backoff and treats peer-closed/read-timeout drops as ordinary background noise (omnigent/repl/_repl.py:1947-1961). All of omnigent/chat.py's client work goes over httpx (omnigent/chat.py:27).

The stream multiplexes two event families (omnigent/server/schemas.py:2013-2044):

• session.* — lifecycle: status, input-consumed, interrupted, created, presence.
• response.* — pass-through Responses-API events from the executor, multiplexed unchanged: text/reasoning deltas, tool calls, tool results.

A two-channel model decides durability (omnigent/server/schemas.py:2027-2037): transient events (text deltas, turn lifecycle, heartbeats, approval_required) are fire-and-forget on SSE and never persisted; persistent events (assistant messages, tool calls, tool results, compaction summaries) are persist-then-publish. A late-joining client therefore gets the durable history via a REST snapshot and the live tail via SSE — which is precisely how the browser hydrates (§5).

Streaming tools: yes. Tool calls and their results stream as first-class response.* events on the same SSE feed, so every client renders a tool invocation as it happens, not after it completes.

For Swisscheese: a single append-only event stream per session, with a transient/persistent split, is the cleanest substrate for a writer→reviewer→fixer pipeline. Reviewers subscribe read-only; nothing about the supervisor's view diverges from what the agent emitted.

4. Live Terminals: Mirrored over WebSocket

Session events ride SSE; live terminals ride a separate raw WebSocket — a deliberate split, because a PTY is a bidirectional byte pipe, not an event log.

A terminal registry tracks live tmux sessions keyed by (conversation_id, terminal_name, session_key) so one conversation can own several terminals and the same name can run parallel sessions (omnigent/terminals/registry.py:103-352). launch() registers a TerminalInstance and allocates a per-instance lock to serialize tmux ops (registry.py:159-277); get() and list_for_conversation() are synchronous map reads (registry.py:307-352).

The mirror itself is bridge_tmux_pty_to_websocket() (omnigent/terminals/ws_bridge.py:455-662). Its wire protocol (omnigent/terminals/ws_bridge.py:17-26):

• Server → client: every PTY read becomes a binary WS frame.
• Client → server: binary frames are raw input bytes written to the PTY (dropped when read-only); text frames are JSON control messages, currently only {"type":"resize","cols":N,"rows":M} applied via ioctl(TIOCSWINSZ).

Crucially, multiple clients can attach to the same tmux socket simultaneously — each gets its own task pair but reads the same underlying PTY, so a CLI pane and a browser tab see identical output in real time. Write access is gated: only the owner attaches read-write; others get read-only (tmux attach -r). The attach URL is built by terminal_attach_url() — wss when the base is https, else ws — targeting …/v1/sessions/{id}/resources/terminals/{tid}/attach (omnigent/native_terminal.py:27-46). (omnigent/native_terminal.py also binds a session's runner affinity so a native-CLI session's terminal stays with whichever runner owns the conversation.)

omnigent/_terminal_picker_theme.py is a 4-line shared-color module (PICKER_ACCENT = "#F43BA6", PICKER_MUTED) used by the prompt-toolkit/Rich terminal selectors.

5. The Web / Desktop App (ap-web/)

A React 18 + Vite + TypeScript SPA, Tailwind v4 + shadcn/ui. Entry is ap-web/src/main.tsx (loaded from ap-web/index.html:11), which mounts the React root and a TanStack QueryClient. The Vite build emits straight into the backend's static dir — outDir: …/omnigent/server/static/web-ui (ap-web/vite.config.ts:164) — so the FastAPI server serves its own UI. There is no separate deploy.

It ships in three skins around the same bundle:

• Browser — the SPA itself.
• Desktop — an Electron wrapper, a thin native shell whose main is ap-web/electron/src/main.js (ap-web/electron/package.json:7).
• iOS — a Swift WKWebView app under ap-web/ios/Omnigent/ (OmnigentApp.swift, OmnigentWebView.swift, plus native notification + connect/settings views). It's a native chrome over the same web UI.

This is the "phone" leg of the promise: the same session, rendered by the same React code, inside a native shell.

5.1 Transport — REST + SSE (not WebSocket for sync)

• REST: a hand-written fetch wrapper, not an openapi-generated client — ap-web/src/lib/sessionsApi.ts, which explicitly "mirrors omnigent/server/routes/sessions.py" (ap-web/src/lib/sessionsApi.ts:1-3). All calls go through authenticatedFetch (ap-web/src/lib/sessionsApi.ts:16). Endpoints: POST /v1/sessions (create, :412), GET /v1/sessions/{id} (snapshot, :695), …/items (paginated history), POST …/events (send a message/event, :867), PATCH … (model/effort), POST …/fork (:500), POST …/switch-agent.
• Live sync = SSE, the very same /v1/sessions/{id}/stream the CLI uses. openSessionStream() opens it with Accept: text/event-stream and hands the raw Response body to a parser (ap-web/src/lib/sessionsApi.ts:901-911). parseSseStream() in ap-web/src/lib/sse.ts parses event: <TYPE>\ndata: <JSON>\n\n frames and distinguishes a clean [DONE] close from a transport drop the caller reconnects on (ap-web/src/lib/sse.ts:125-149). The TS file is a hand-port of the Python sdks/python-client/omnigent_client/_sse.py and decodes the full response.* / session.* vocabulary, including session.presence.

So the browser and the terminal converge on the identical event stream — the cross-device guarantee falls out of that, not from any bespoke sync protocol.

5.2 State & rendering a live multi-agent session

State is a Zustand store, useChatStore (ap-web/src/store/chatStore.ts:681): conversationId, status, model/harness/skills/todos, a flat blocks[] array reduced from the SSE stream, optimistic pendingUserMessages, an abortController owning the live /stream fetch, and viewers[] for presence. TanStack Query handles snapshot/history pagination; the documented hydration order is open the live stream first, then fetch the snapshot, then dedupe (ap-web/src/lib/sessionsApi.ts:690) — late-join without gaps.

Rendering:

• Timeline — the flat blocks[] (built by a BlockStream reducer over SSE events) is grouped into bubbles by response id and rendered as a chat conversation.
• Live terminals — xterm.js (@xterm/xterm v6) wrapped by a pure-JS TerminalSession that owns the WebSocket and xterm lifecycle outside React (ap-web/src/components/blocks/TerminalSession.ts:1-15). Its header comment mirrors the Python wire protocol verbatim: server → binary PTY bytes → term.write; client → binary keystrokes + JSON resize. TerminalView builds the attach URL via resolveWebSocketUrl (wss: on https, else ws: — ap-web/src/lib/host.ts:143-147) and appends ?read_only=true for read-only attach (ap-web/src/components/blocks/TerminalView.tsx:417-424). This is the browser end of §4's multi-watcher PTY mirror.
• Agent graph — @xyflow/react is a dependency and the ai-elements/{node,edge}.tsx primitives import its Handle/Position, but it isn't wired into the live chat page; the multi-agent view is the timeline + a subagents panel, not a flow graph.

5.3 Cross-device sync & collaboration

Collaboration is presence-based, Google-Docs-style, and it is a side effect of holding the stream open. The server's presence registry tracks which users have a live SSE stream open anywhere in a session tree — root or any sub-agent conversation — so two users on different sub-agents of the same session still see each other (omnigent/server/presence.py:1-43). Multiple tabs from one user dedupe into a single viewer whose idle flag is the AND of its tabs. Every broadcast carries the full viewer list (never deltas), so missed/reordered events self-heal on the next reconnect snapshot (omnigent/server/presence.py:34-43). A _LEAVE_GRACE_S window absorbs the Databricks ingress' ~5-minute stream cap and page refreshes so avatars don't flicker (omnigent/server/presence.py:24-29).

On the UI side, session.presence SSE events replace viewers[] wholesale, and PresenceAvatars renders the circles — up to a few avatars plus a "+N" chip, idle viewers dimmed (ap-web/src/components/PresenceAvatars.tsx:15-67). The stream URL "is the entire presence uplink": an idle flip mid-view arrives as a reconnect carrying the new value (ap-web/src/lib/sessionsApi.ts:892-895).

What collaboration is: anyone with read access watches live; the session owner co-drives by sending events; fork and switch-agent are available. What it isn't: no shared cursors, no collaborative text editing, no "follow-me" camera. Sharing is "open the same live session," gated by a permission level (read / edit / manage / owner).

For Swisscheese: presence-by-stream-subscription is a low-cost way to show a human "who else is watching this agent." A reviewer opening the session is automatically a visible co-viewer — no separate presence service to run.

6. The Picture

flowchart LR
    subgraph clients[Clients — thin replicas]
      cli["CLI REPL<br/>omnigent run / attach<br/>(omnigent/repl/_repl.py)"]
      browser["Browser SPA<br/>React + Vite<br/>(ap-web/)"]
      desktop["Desktop<br/>Electron shell"]
      phone["Phone<br/>iOS WKWebView"]
    end

    subgraph server[Omnigent Server — FastAPI]
      rest["REST<br/>/v1/sessions/*<br/>(omnigent/server/routes/sessions.py)"]
      sse["SSE live-tail<br/>GET /v1/sessions/{id}/stream<br/>text/event-stream"]
      presence["Presence registry<br/>(omnigent/server/presence.py)"]
      store[("Session store<br/>persistent items")]
    end

    subgraph live[Live session]
      runner["Runner + harness<br/>(claude / codex / …)"]
      term["tmux PTY<br/>(omnigent/terminals/registry.py)"]
    end

    cli & browser & desktop & phone -->|"create / send / snapshot (REST)"| rest
    cli & browser & desktop & phone -->|"subscribe (SSE) → register as viewer"| sse
    cli & browser & desktop & phone -->|"WS binary PTY mirror<br/>…/terminals/{id}/attach"| term

    sse <--> presence
    rest --> store
    sse --> store
    rest --> runner
    runner -->|"response.* / session.* events"| sse
    runner --- term
    term -->|"binary PTY frames (omnigent/terminals/ws_bridge.py)"| browser

Every arrow into the server is the same for every client. The terminal is one box mirrored to N watchers; the session is one event stream fanned out to N subscribers; presence is just "who currently holds a stream open." That uniformity — not a sync engine — is what lets a session follow you across the terminal, the browser, and the phone.

7. Design Observations

Good ideas

• One stream, all clients. The CLI REPL and the React app consume the byte-identical SSE feed (/v1/sessions/{id}/stream); the TS parser is a deliberate hand-port of the Python one. Cross-device parity is structural, not bolted on.
• Transient/persistent split. Late joiners hydrate from a REST snapshot (persistent) and the live SSE tail (transient) with a documented "stream-first, then snapshot, then dedupe" order — no gaps, no double-render.
• Right transport per data shape. Event log → SSE (one-way, reconnectable, cheap). PTY → raw WebSocket (bidirectional bytes). The app doesn't force one pipe to do both.
• Presence as a free side effect. Holding the stream open is the presence signal. No separate presence service; full-list broadcasts self-heal.
• Meta-harness as a flag. A dozen coding agents are sibling launchers over one run dispatcher — the wrapper, not the workflow, changes per agent.
• Privacy-aware diagnostics. Always-on logging with content suppression at INFO and an aggressive secret-redaction filter.

Pitfalls / sharp edges

• No openapi-generated web client. ap-web/src/lib/sessionsApi.ts and ap-web/src/lib/sse.ts are hand-mirrors of the Python routes/SSE parser with no parity test — a server-side event-shape change can silently starve the reducer (the file's own comment flags this).
• SSE caps force reconnect churn. The Databricks ingress' ~5-minute stream cap means every viewer transparently reconnects on that cadence; the presence grace window exists specifically to mask the resulting flicker. Correct, but a lot of machinery to paper over an infra limit.
• Multi-writer PTY isn't serialized. Two read-write attachers type into the same tmux pane with interleaved keystrokes; the only guard is owner-vs-read-only at the permission layer.
• @xyflow/react dead weight. Shipped and imported by primitives but not used in the live session view — bundle cost without payoff today.