> ## Documentation Index
> Fetch the complete documentation index at: https://docs.videodb.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Real-time Context

> Client events and backend RTStream event patterns for capture

Capture clients emit typed local events in real-time. Backend RTStream payload fields can vary by pipeline; use webhooks for durable delivery and WebSockets for live UI.

<Note>
  Desktop capture currently supports **macOS** and **Windows**.
</Note>

## Quick Example

<CodeGroup>
  ```python Python theme={null}
  import videodb

  conn = videodb.connect()
  ws = conn.connect_websocket()
  await ws.connect()

  # Inspect backend events; payload fields are pipeline-specific
  async for ev in ws.receive():
      print(ev)
  ```

  ```javascript Node.js theme={null}
  import { connect } from 'videodb';

  const conn = connect();
  const ws = await conn.connectWebsocket();
  await ws.connect();

  // Inspect backend events; payload fields are pipeline-specific
  for await (const ev of ws.receive()) {
      console.log(ev);
  }
  ```
</CodeGroup>

***

## Event Types

The following backend payloads are illustrative observations, not a fixed Capture SDK schema. Inspect and validate the payloads your integration receives.

### Transcript Events

Real-time speech-to-text from audio channels:

```json theme={null}
{
  "channel": "transcript",
  "rtstream_id": "rts-xxx",
  "rtstream_name": "mic:default",
  "data": {
    "text": "Let's schedule the meeting for Thursday",
    "is_final": true,
    "start": 1710000001234,
    "end": 1710000002345
  }
}
```

| Field | Description |
| :- | :- |
| `text` | Transcribed speech |
| `is_final` | `true` for final, `false` for interim |
| `start/end` | Timestamps (ms) |

### Visual Index Events

Scene descriptions from screen capture:

```json theme={null}
{
  "channel": "visual_index",
  "rtstream_id": "rts-xxx",
  "rtstream_name": "display:1",
  "data": {
    "text": "User is viewing a Slack conversation with 3 unread messages",
    "start": 1710000012340,
    "end": 1710000018900
  }
}
```

### Audio Index Events

Semantic understanding of audio:

```json theme={null}
{
  "channel": "audio_index",
  "rtstream_id": "rts-xxx",
  "rtstream_name": "mic:default",
  "data": {
    "text": "Discussion about scheduling a team meeting",
    "start": 1710000021500,
    "end": 1710000029200
  }
}
```

### Alert Events

Custom detection rules firing:

```json theme={null}
{
  "channel": "alert",
  "rtstream_id": "rts-xxx",
  "data": {
    "label": "sensitive_content",
    "triggered": true,
    "confidence": 0.92,
    "start": 1710000045100,
    "end": 1710000047800
  }
}
```

***

## RTStream Helper Channels

The helper APIs below are available for compatibility. For descriptor-backed V2 understanding and indexing, see [Understand & Index RTStreams](/pages/ingest/live-streams/rtstream-understand-index).

| Channel | Source | Content |
| :- | :- | :- |
| `capture_session` | Session lifecycle | Status changes |
| `transcript` | `start_transcript()` | Speech-to-text |
| `scene_index` | `index_visuals()` | Visual analysis |
| `audio_index` | `index_audio()` | Audio analysis |
| `alert` | `create_alert()` | Alert notifications |

### Connecting to RTStream Helper Events

This example uses RTStream helpers.

<CodeGroup>
  ```python Python theme={null}
  conn = videodb.connect()
  ws = conn.connect_websocket()
  await ws.connect()

  # Pass ws.connection_id when starting AI operations
  rtstream.start_transcript(ws_connection_id=ws.connection_id)
  rtstream.index_visuals(prompt="...", ws_connection_id=ws.connection_id)
  rtstream.index_audio(prompt="...", ws_connection_id=ws.connection_id)
  ```

  ```javascript Node.js theme={null}
  const conn = connect();
  const ws = await conn.connectWebsocket();
  await ws.connect();

  // Pass the WebSocket connection ID when starting AI operations
  await rtstream.startTranscript(ws.connectionId);
  await rtstream.indexVisuals({ prompt: "...", socketId: ws.connectionId });
  await rtstream.indexAudio({ prompt: "...", socketId: ws.connectionId });
  ```
</CodeGroup>

***

## Webhooks

Durable delivery for session lifecycle events.

Backend webhook fields and lifecycle payloads are illustrative server behavior, not a fixed Capture SDK schema.

### Illustrative Webhook Envelope

```json theme={null}
{
  "version": "2",
  "event": "capture_session.active",
  "timestamp": "2026-01-20T12:34:56Z",
  "capture_session_id": "cap-xxx",
  "end_user_id": "user_abc",
  "status": "active",
  "data": {}
}
```

### Illustrative Session Lifecycle Events

| Event | Status | Key Data |
| :- | :- | :- |
| `capture_session.created` | `created` | — |
| `capture_session.starting` | `starting` | — |
| `capture_session.active` | `active` | `rtstreams[]` |
| `capture_session.stopping` | `stopping` | — |
| `capture_session.stopped` | `stopped` | — |
| `capture_session.exported` | `exported` | `exported_video_id` |
| `capture_session.failed` | `failed` | `error` object |

### Illustrative `capture_session.active` Payload

This is where you start AI pipelines:

```json theme={null}
{
  "event": "capture_session.active",
  "capture_session_id": "cap-xxx",
  "status": "active",
  "data": {
    "rtstreams": [
      { "rtstream_id": "rts-1", "name": "mic:default", "media_types": ["audio"] },
      { "rtstream_id": "rts-2", "name": "system_audio:default", "media_types": ["audio"] },
      { "rtstream_id": "rts-3", "name": "display:1", "media_types": ["video"] }
    ]
  }
}
```

<Note>
  Resolving Capture RTStreams from category-style webhook values is not yet a supported released-SDK contract, so this runnable lookup example is deferred.
</Note>

***

## Delivery Semantics

| Method | Guarantee | Handle |
| :- | :- | :- |
| WebSocket | Best-effort | Reconnect on disconnect |
| Webhook | At-least-once | Deduplicate by event ID |

<Warning>
  Webhooks may deliver duplicates. Respond 2xx quickly, process asynchronously, implement idempotency.
</Warning>

***

## Next Steps

<CardGroup cols={2}>
  <Card icon="camera" title="Capture Overview" href="/pages/ingest/capture-sdks/overview">
    Architecture and quickstart
  </Card>

  <Card icon="search" title="Storage & Search" href="/pages/ingest/capture-sdks/storage-and-search">
    Export and persistence patterns
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.