Skip to main content
Desktop capture currently supports macOS and Windows.

1. Backend Setup

Install

Create a Capture Session

Your backend creates a session and generates a short-lived token for the desktop client:

2. Client Setup

Install

Start Capture

The desktop client uses the token to stream screen and audio:

3. Backend Starts AI

When capture begins, your backend receives a webhook and starts AI processing:

4. What You Get

Your backend receives AI-ready events in real-time:
Build with these:
  • Screen-aware AI agents
  • Live meeting copilots
  • In-call assistance
  • Semantic search and replay

Architecture

Diagram showing the architecture of the system
  1. Backend creates a CaptureSession and mints a short-lived token
  2. Desktop client uses the token to stream screen + audio (never sees API key)
  3. VideoDB creates RTStreams (one per channel) when capture starts
  4. Backend receives webhook, starts transcript and indexing on RTStreams
  5. AI events flow back via WebSocket (real-time) or can be polled

Two Runtimes

Rule of thumb: Webhooks for correctness (durable, at-least-once). WebSocket for live UI (best-effort).

5. Example Applications

Claude Pair Programmer

AI coding assistant with screen and audio context

Bloom

Local-first screen recorder with AI indexing

Focusd

AI-powered productivity tracking

Call.md

Real-time meeting intelligence

6. Core Concepts

CaptureSession (cap-xxx)

The lifecycle container for one capture run. Created by backend, activated by desktop client. States: created → starting → active → stopping → stopped → exported

RTStream (rts-xxx)

A real-time media stream, one per captured channel. This is where you run AI:

Channel

A recordable source on the desktop:

Multi-Screen Capture

When multiple monitors are connected, each appears as a separate display:N channel. Use cap.displays on the backend to inspect available video channels:
Python
cap.displays returns a list of video channel objects. Each object includes an is_primary field that indicates which display was set as the primary video channel when capture started (via primary_video_channel_id). To capture multiple screens, pass all desired display channels to the desktop client:
Python
Each display produces its own RTStream on the backend. The primary display is used for the default muxed export video; non-primary displays are available as raw channel assets or can be exported separately (see Storage & Search).

Explore More

View All Examples on GitHub

Complete source code with quickstart guides, example apps, and implementation patterns

Real-time Context

Events you receive from capture

Storage & Search

Optional persistence and semantic search