Real-time APIs

Quick Example

The preferred RTStream interface follows the same Understand → Index → Retrieve flow as videos, but runs continuously on new stream windows.

import videodb

conn = videodb.connect()
coll = conn.get_collection()
rtstream = coll.get_rtstream("rts-xxx")

# 1. Continuously understand the stream with a VLM
understanding = rtstream.understand(
    segmentation={"type": "time", "window": "5s"},
    analyzers=[
        {
            "type": "vlm",
            "name": "scene",
            "sampling": {"frame_count": 2},
            "config": {
                "prompt": "Describe activity and detect unusual behavior"
            },
        }
    ],
    store=True,
)

# 2. Continuously index the VLM output
index = rtstream.index(
    name="activity_monitor",
    source=understanding.outputs["scene"],
    use_for=["semantic"],
)

# 3. Create a reusable event and attach an alert to the index
event_id = conn.create_event(
    event_prompt="Detect when someone enters restricted area",
    label="intrusion_detected",
)

alert_id = index.create_alert(
    event_id=event_id,
    callback_url="https://your-backend.com/webhooks/alerts",
)

Initial support for the new interface is VLM-only with time-based segmentation. See Understand & Index RTStreams for the complete interface.

Visual Indexing Compatibility

The index_visuals() helper remains available as an explicit compatibility API. It creates a continuous VLM scene index directly from stream frames.

scene_index = rtstream.index_visuals(
    prompt="Describe the scene and highlight congestion",
    batch_config={"type": "time", "value": 5, "frame_count": 2},
    name="traffic_monitor",
    ws_connection_id=ws.connection_id  # optional, for real-time events
)

const sceneIndex = await rtstream.indexVisuals({
    prompt: "Describe the scene and highlight congestion",
    batchConfig: { type: "time", value: 5, frameCount: 2 },
    name: "traffic_monitor",
    socketId: ws.connectionId  // optional
});

`batch_config` options

Field	Type	Description
`type`	str	Only `"time"` is supported
`value`	int	Window size in seconds
`frame_count`	int	Frames to extract per window

Examples:

# Every 5 seconds, extract 2 frames
{"type": "time", "value": 5, "frame_count": 2}

# Every 10 seconds, extract 5 frames
{"type": "time", "value": 10, "frame_count": 5}

Managing scene indexes

# List all indexes
indexes = rtstream.list_scene_indexes()

# Get specific index
scene_index = rtstream.get_scene_index(index_id)

# Poll scenes
scenes = scene_index.get_scenes(
    start=0, end=None, page=1, page_size=100
)

// List all indexes
const indexes = await rtstream.listSceneIndexes();

// Get specific index
const sceneIndex = await rtstream.getSceneIndex(indexId);

// Poll scenes
const scenes = await sceneIndex.getScenes();

Audio Indexing Compatibility

The index_audio() helper remains available for compatibility:

audio_index = rtstream.index_audio(
    prompt="Identify key speakers and main topics",
    batch_config={"type": "word", "value": 50}
)

const audioIndex = await rtstream.indexAudio({
    prompt: "Identify key speakers and main topics",
    batchConfig: { type: "word", value: 50 }
});

Audio batch_config

Type	Value	Description
`"word"`	count	Segment every N words
`"sentence"`	count	Segment every N sentences
`"time"`	seconds	Segment every N seconds

Transcription

Real-time speech-to-text:

# Start transcription
rtstream.start_transcript(ws_connection_id=ws.connection_id)

# Stop transcription
rtstream.stop_transcript()

# Poll transcripts
transcript = rtstream.get_transcript(
    start=0, page=1, page_size=100
)

// Start transcription
await rtstream.startTranscript(ws.connectionId);

// Stop transcription
await rtstream.stopTranscript("graceful");

// Poll transcripts
const transcript = await rtstream.getTranscript({
    start: 0, page: 1, pageSize: 100
});

Search

Query indexed content with natural language:

results = rtstream.search(
    query="white car moving fast",
    score_threshold=0.5
)

for shot in results.shots:
    print(f"Match at {shot.start}: {shot.text}")
    shot.play()  # Opens in browser

const results = await rtstream.search({
    query: "white car moving fast",
    scoreThreshold: 0.5
});

for (const shot of results.shots) {
    console.log(`Match at ${shot.start}: ${shot.text}`);
    await shot.play();
}

Search Results

Each shot contains:

Attribute	Description
`start`	Start timestamp
`end`	End timestamp
`text`	Content description
`search_score`	Relevance score (0-1)
`stream_url`	Playback URL

Stream Generation

Use generate_stream() to create a playable stream for a time range. You can optionally pass player_config to attach metadata to a shareable player URL.

# Generate stream with player metadata
player_url = rtstream.generate_stream(
    start=1711000000,
    end=1711003600,
    player_config={
        "title": "Live Feed Recording",
        "description": "Camera 1 - Front entrance",
        "slug": "cam1-front"
    }
)

# generate_stream() returns the player_url
# Both URLs are also stored on the instance
print(rtstream.player_url)   # Shareable player URL (same as return value)
print(rtstream.stream_url)   # Raw HLS stream URL (.m3u8)

// Generate stream with player metadata
const playerUrl = await rtstream.generateStream(
    1711000000,
    1711003600,
    {
        title: "Live Feed Recording",
        description: "Camera 1 - Front entrance",
        slug: "cam1-front"
    }
);

// generateStream() returns the playerUrl
// Both URLs are also stored on the instance
console.log(rtstream.playerUrl);   // Shareable player URL (same as return value)
console.log(rtstream.streamUrl);   // Raw HLS stream URL (.m3u8)

player_url vs stream_url

URL	Format	Use case
`player_url`	`https://console.videodb.io/player?v=...`	Shareable link with built-in player UI
`stream_url`	`https://stream.videodb.io/.../.m3u8`	Raw HLS stream for custom players (HLS.js, Video.js)

generate_stream() returns player_url and stores both URLs on the instance. Use player_url for sharing and embedding, stream_url for custom player integrations.

player_config

Key	Type	Description
`title`	str	Title displayed in the player
`description`	str	Description for the player page
`slug`	str	URL-friendly slug for the player link

player_config is optional. When provided, the player URL includes the metadata for sharing.

Embed Code

After generating a stream, you can produce embeddable HTML:

# Generate embed code (requires generate_stream() first)
rtstream.generate_stream(start=start_ts, end=end_ts)
embed_html = rtstream.get_embed_code()

// Generate embed code (requires generateStream() first)
await rtstream.generateStream(startTs, endTs);
const embedHtml = rtstream.getEmbedCode();

RTStream does not support auto_generate because generate_stream() requires explicit start and end parameters.

Events and Alerts

Events are reusable detection rules. Alerts wire events to indexes for notifications.

Create Event

event_id = conn.create_event(
    event_prompt="Detect pedestrians crossing the zebra",
    label="pedestrian_detected"
)

const eventId = await conn.createEvent(
    "Detect pedestrians crossing the zebra",
    "pedestrian_detected"
);

Create Alert

alert_id = scene_index.create_alert(
    event_id=event_id,
    callback_url="https://your-backend.com/webhooks/alerts",
    ws_connection_id=ws.connection_id  # optional
)

const alertId = await sceneIndex.createAlert(
    eventId,
    "https://your-backend.com/webhooks/alerts",
    ws.connectionId  // optional
);

Alert Delivery

Method	Latency	Use Case
Webhook	Under 1s	Server-to-server POST
WebSocket	Real-time	Frontend dashboards

Manage Alerts

# List alerts
alerts = scene_index.list_alerts()

# Enable/disable
scene_index.enable_alert(alert_id)
scene_index.disable_alert(alert_id)

// List alerts
const alerts = await sceneIndex.listAlerts();

// Enable/disable
await sceneIndex.enableAlert(alertId);
await sceneIndex.disableAlert(alertId);

WebSocket Events

Receive real-time events by passing ws_connection_id:

ws = conn.connect_websocket()
await ws.connect()

# Pass ws_connection_id to methods
rtstream.start_transcript(ws_connection_id=ws.connection_id)

# WebSocket messages are arbitrary decoded payloads; inspect before accessing fields.
async for ev in ws.receive():
    print(ev)

const ws = await conn.connectWebsocket();
await ws.connect();

// Pass the WebSocket connection ID to methods
await rtstream.startTranscript(ws.connectionId);

// WebSocket messages are arbitrary decoded payloads; inspect before accessing fields.
for await (const ev of ws.receive()) {
    console.log(ev);
}

Event Channels

The current V2 alert channel is produced by the V2 index in the quick example:

Channel	Source	Content
`alert`	`index.create_alert()`	Alert notifications

The following channel names are available through RTStream helper APIs:

Channel	Source	Content
`transcript`	`start_transcript()`	Real-time speech-to-text
`scene_index`	`index_visuals()`	Visual analysis
`audio_index`	`index_audio()`	Audio analysis

Start Here

Core Concepts

Ingest

Understand

Act

Automate

MCP Server

Open Source Frameworks

Quick Example

Visual Indexing Compatibility

`batch_config` options

Managing scene indexes

Audio Indexing Compatibility

Audio batch_config

Transcription

Search

Search Results

Stream Generation

player_url vs stream_url

player_config

Embed Code

Events and Alerts

Create Event

Create Alert

Alert Delivery

Manage Alerts

WebSocket Events

Event Channels

Next Steps

Stream Lifecycle

RTSP Ingest

​Quick Example

​Visual Indexing Compatibility

​batch_config options

​Managing scene indexes

​Audio Indexing Compatibility

​Audio batch_config

​Transcription

​Search

​Search Results

​Stream Generation

​player_url vs stream_url

​player_config

​Embed Code

​Events and Alerts

​Create Event

​Create Alert

​Alert Delivery

​Manage Alerts

​WebSocket Events

​Event Channels

​Next Steps

Stream Lifecycle

RTSP Ingest

Quick Example

Visual Indexing Compatibility

`batch_config` options

Managing scene indexes

Audio Indexing Compatibility

Audio batch_config

Transcription

Search

Search Results

Stream Generation

player_url vs stream_url

player_config

Embed Code

Events and Alerts

Create Event

Create Alert

Alert Delivery

Manage Alerts

WebSocket Events

Event Channels

Next Steps