Real-time Context

Capture clients emit typed local events in real-time. Backend RTStream payload fields can vary by pipeline; use webhooks for durable delivery and WebSockets for live UI.

Desktop capture currently supports macOS and Windows.

Quick Example

import videodb

conn = videodb.connect()
ws = conn.connect_websocket()
await ws.connect()

# Inspect backend events; payload fields are pipeline-specific
async for ev in ws.receive():
    print(ev)

import { connect } from 'videodb';

const conn = connect();
const ws = await conn.connectWebsocket();
await ws.connect();

// Inspect backend events; payload fields are pipeline-specific
for await (const ev of ws.receive()) {
    console.log(ev);
}

Event Types

The following backend payloads are illustrative observations, not a fixed Capture SDK schema. Inspect and validate the payloads your integration receives.

Transcript Events

Real-time speech-to-text from audio channels:

{
  "channel": "transcript",
  "rtstream_id": "rts-xxx",
  "rtstream_name": "mic:default",
  "data": {
    "text": "Let's schedule the meeting for Thursday",
    "is_final": true,
    "start": 1710000001234,
    "end": 1710000002345
  }
}

Field	Description
`text`	Transcribed speech
`is_final`	`true` for final, `false` for interim
`start/end`	Timestamps (ms)

Visual Index Events

Scene descriptions from screen capture:

{
  "channel": "visual_index",
  "rtstream_id": "rts-xxx",
  "rtstream_name": "display:1",
  "data": {
    "text": "User is viewing a Slack conversation with 3 unread messages",
    "start": 1710000012340,
    "end": 1710000018900
  }
}

Audio Index Events

Semantic understanding of audio:

{
  "channel": "audio_index",
  "rtstream_id": "rts-xxx",
  "rtstream_name": "mic:default",
  "data": {
    "text": "Discussion about scheduling a team meeting",
    "start": 1710000021500,
    "end": 1710000029200
  }
}

Alert Events

Custom detection rules firing:

{
  "channel": "alert",
  "rtstream_id": "rts-xxx",
  "data": {
    "label": "sensitive_content",
    "triggered": true,
    "confidence": 0.92,
    "start": 1710000045100,
    "end": 1710000047800
  }
}

RTStream Helper Channels

The helper APIs below are available for compatibility. For descriptor-backed V2 understanding and indexing, see Understand & Index RTStreams.

Channel	Source	Content
`capture_session`	Session lifecycle	Status changes
`transcript`	`start_transcript()`	Speech-to-text
`scene_index`	`index_visuals()`	Visual analysis
`audio_index`	`index_audio()`	Audio analysis
`alert`	`create_alert()`	Alert notifications

Connecting to RTStream Helper Events

This example uses RTStream helpers.

conn = videodb.connect()
ws = conn.connect_websocket()
await ws.connect()

# Pass ws.connection_id when starting AI operations
rtstream.start_transcript(ws_connection_id=ws.connection_id)
rtstream.index_visuals(prompt="...", ws_connection_id=ws.connection_id)
rtstream.index_audio(prompt="...", ws_connection_id=ws.connection_id)

const conn = connect();
const ws = await conn.connectWebsocket();
await ws.connect();

// Pass the WebSocket connection ID when starting AI operations
await rtstream.startTranscript(ws.connectionId);
await rtstream.indexVisuals({ prompt: "...", socketId: ws.connectionId });
await rtstream.indexAudio({ prompt: "...", socketId: ws.connectionId });

Webhooks

Durable delivery for session lifecycle events. Backend webhook fields and lifecycle payloads are illustrative server behavior, not a fixed Capture SDK schema.

Illustrative Webhook Envelope

{
  "version": "2",
  "event": "capture_session.active",
  "timestamp": "2026-01-20T12:34:56Z",
  "capture_session_id": "cap-xxx",
  "end_user_id": "user_abc",
  "status": "active",
  "data": {}
}

Illustrative Session Lifecycle Events

Event	Status	Key Data
`capture_session.created`	`created`	—
`capture_session.starting`	`starting`	—
`capture_session.active`	`active`	`rtstreams[]`
`capture_session.stopping`	`stopping`	—
`capture_session.stopped`	`stopped`	—
`capture_session.exported`	`exported`	`exported_video_id`
`capture_session.failed`	`failed`	`error` object

Illustrative `capture_session.active` Payload

This is where you start AI pipelines:

{
  "event": "capture_session.active",
  "capture_session_id": "cap-xxx",
  "status": "active",
  "data": {
    "rtstreams": [
      { "rtstream_id": "rts-1", "name": "mic:default", "media_types": ["audio"] },
      { "rtstream_id": "rts-2", "name": "system_audio:default", "media_types": ["audio"] },
      { "rtstream_id": "rts-3", "name": "display:1", "media_types": ["video"] }
    ]
  }
}

Resolving Capture RTStreams from category-style webhook values is not yet a supported released-SDK contract, so this runnable lookup example is deferred.

Delivery Semantics

Method	Guarantee	Handle
WebSocket	Best-effort	Reconnect on disconnect
Webhook	At-least-once	Deduplicate by event ID

Webhooks may deliver duplicates. Respond 2xx quickly, process asynchronously, implement idempotency.

Start Here

Core Concepts

Ingest

Understand

Act

Automate

MCP Server

Open Source Frameworks

Quick Example

Event Types

Transcript Events

Visual Index Events

Audio Index Events

Alert Events

RTStream Helper Channels

Connecting to RTStream Helper Events

Webhooks

Illustrative Webhook Envelope

Illustrative Session Lifecycle Events

Illustrative `capture_session.active` Payload

Delivery Semantics

Next Steps

Capture Overview

Storage & Search

​Quick Example

​Event Types

​Transcript Events

​Visual Index Events

​Audio Index Events

​Alert Events

​RTStream Helper Channels

​Connecting to RTStream Helper Events

​Webhooks

​Illustrative Webhook Envelope

​Illustrative Session Lifecycle Events

​Illustrative capture_session.active Payload

​Delivery Semantics

​Next Steps

Capture Overview

Storage & Search

Quick Example

Event Types

Transcript Events

Visual Index Events

Audio Index Events

Alert Events

RTStream Helper Channels

Connecting to RTStream Helper Events

Webhooks

Illustrative Webhook Envelope

Illustrative Session Lifecycle Events

Illustrative `capture_session.active` Payload

Delivery Semantics

Next Steps