Quickstart

Video was built for people to watch. VideoDB makes it understandable and actionable for people and AI agents. With one backend, your application can turn files, live streams, and cameras into structured, timestamped context - then retrieve exact moments and act on playable evidence. Every VideoDB workflow follows the same loop:

See

Upload a video, connect a live stream, or capture a desktop.

Understand

Create reusable artifacts from what was said and what was shown, then index the context you want to retrieve.

Act

Retrieve matching moments, play the evidence, or trigger an alert.

You will complete this loop with one recorded video, apply it continuously to a live stream, and see how the same foundation extends to desktop capture.

Set up VideoDB

Get an API key

Go to the VideoDB Console.
Copy your API key.
Add it to your environment:

export VIDEO_DB_API_KEY="your-api-key"

Install the SDK

pip install videodb

npm install videodb

Run the Node.js examples as ES modules by using an .mjs file or setting "type": "module" in package.json.

See, understand, and search a video

Start with one recorded video and complete the loop. By the end, a natural-language request will return timestamped matches and one stream containing only the relevant moments. Choose Python or Node.js, then run that language’s blocks in order in the same script or interactive session.

See: upload a video

Connect to your default collection, then upload a public video URL.

import videodb

conn = videodb.connect()
coll = conn.get_collection()

video = coll.upload(
    url="https://www.youtube.com/watch?v=LPZh9BOjkQs"
)

print(video.id, video.name, video.length)

import { connect } from "videodb";

const conn = connect();
const coll = await conn.getCollection();

const video = await coll.uploadURL({
    url: "https://www.youtube.com/watch?v=LPZh9BOjkQs"
});

console.log(video.id, video.name, video.length);

The media is now available to your application. VideoDB can also ingest local files, object-storage URLs, and other publicly accessible video URLs. Generate a playable stream for the complete uploaded video. Later, you will compile a second stream containing only matching evidence.

stream_url = video.generate_stream()
print(stream_url)

const streamUrl = await video.generateStream();
console.log(streamUrl);

Understand: create visual and spoken artifacts

Next, create reusable, timestamped artifacts from what was said and what was shown. One understanding run can create both: spoken_words produces a timed transcript, while the vlm prompt controls which visual details become structured context. Understanding and indexing run as background jobs; the wait calls below continue when each result is ready.

understanding = video.understand(
    analyzers=[
        {"type": "spoken_words", "name": "transcript"},
        {
            "type": "vlm",
            "name": "scene",
            "config": {
                "prompt": "Describe the visual content."
            },
        },
    ]
)
understanding.wait_until_complete()

const understanding = await video.understand([
    { type: "spoken_words", name: "transcript" },
    {
        type: "vlm",
        name: "scene",
        config: {
            prompt: "Describe the visual content."
        }
    }
]);
await understanding.waitUntilComplete();

Retrieve the completed analyzer handles:

transcript_analyzer = understanding.get_analyzer("transcript")
scene_analyzer = understanding.get_analyzer("scene")

const transcriptAnalyzer = await understanding.getAnalyzer("transcript");
const sceneAnalyzer = await understanding.getAnalyzer("scene");

Each artifact contains timestamped scenes. Inspect the first result from each analyzer:

for analyzer in [transcript_analyzer, scene_analyzer]:
    output = analyzer.get_output()
    first_scene = output["scenes"][0]

    print(f"{analyzer.name}: {len(output['scenes'])} scenes")
    print(first_scene["start"], first_scene["end"])
    print(first_scene["data"]["text"])
    print()

for (const analyzer of [transcriptAnalyzer, sceneAnalyzer]) {
    const output = await analyzer.getOutput();
    const firstScene = output.scenes[0];

    console.log(`${analyzer.name}: ${output.scenes.length} scenes`);
    console.log(firstScene.start, firstScene.end);
    console.log(firstScene.data.text);
    console.log();
}

Understand: index the artifacts

Artifacts record what VideoDB found. Index them once to make that context retrievable without rerunning the analysis. Create the transcript index:

transcript_index = video.index(
    name="quickstart_transcript",
    source=transcript_analyzer,
)
transcript_index.wait_until_complete()

print(transcript_index.index_id, transcript_index.status)

const transcriptIndex = await video.index(transcriptAnalyzer, {
    name: "quickstart_transcript"
});
await transcriptIndex.waitUntilComplete();

console.log(transcriptIndex.indexId, transcriptIndex.status);

Create the visual scene index:

scene_index = video.index(
    name="quickstart_scene",
    source=scene_analyzer,
)
scene_index.wait_until_complete()

print(scene_index.index_id, scene_index.status)

const sceneIndex = await video.index(sceneAnalyzer, {
    name: "quickstart_scene"
});
await sceneIndex.waitUntilComplete();

console.log(sceneIndex.indexId, sceneIndex.status);

VideoDB derives the index configuration from each artifact. You can choose explicit fields later when an application needs a specific schema for filtering or aggregation.

Act: find and play matching moments

Now ask for a moment - not a summary. Search both indexes by their returned IDs so the request uses the artifacts created in this run. Each returned Shot includes timestamps and the matched index text in its metadata.

results = video.semantic_search(
    query="an explanation supported by diagrams and on-screen text",
    index_ids=[transcript_index.index_id, scene_index.index_id],
    top_k=5,
)

for shot in results.get_shots():
    print(
        f"{shot.start:.1f}s–{shot.end:.1f}s: "
        f"{shot.metadata['embedded_text']}"
    )

const results = await video.semanticSearch(
    "an explanation supported by diagrams and on-screen text",
    {
        indexIds: [transcriptIndex.indexId, sceneIndex.indexId],
        topK: 5
    }
);

for (const shot of results.getShots()) {
    console.log(
        `${shot.start.toFixed(1)}s–${shot.end.toFixed(1)}s: ` +
        shot.metadata.embeddedText
    );
}

Compile the matching moments into one playable HLS stream:

evidence_url = results.compile()
print(evidence_url)

const evidenceUrl = await results.compile();
console.log(evidenceUrl);

You now have timestamped results and a stream that plays only the matching evidence - ready to show to a user or pass to another part of your application.

See, understand, and act on a live stream

Files finish processing; streams keep producing new windows. For live media, understanding and indexing run continuously while the RTStream is active. Copy these blocks in order into one file. The first block opens main() in each language, the middle blocks continue it, and the final block closes and runs it.

See: connect the stream

Connect the sample stream in the default collection and retain it for the continuous workflow.

import asyncio
import videodb

RTSP_URL = "rtsp://samples.rts.videodb.io:8554/intruder"


async def main():
    conn = videodb.connect()
    coll = conn.get_collection()

    rtstream = coll.connect_rtstream(
        url=RTSP_URL,
        name="Quickstart Live Stream",
        store=True,
    )

import { connect } from "videodb";

const RTSP_URL = "rtsp://samples.rts.videodb.io:8554/intruder";

async function main() {
    const conn = connect();
    const coll = await conn.getCollection();

    const rtstream = await coll.connectRTStream(
        RTSP_URL,
        "Quickstart Live Stream",
        ["video"],
        30,
        true
    );

Understand: analyze each window

Use one continuous VLM output named scene for each five-second window.

    understanding = rtstream.understand(
        segmentation={"type": "time", "window": "5s"},
        analyzers=[
            {
                "type": "vlm",
                "name": "scene",
                "sampling": {"frame_count": 2},
                "config": {
                    "prompt": "Describe the people, movement, and activity in this stream window."
                },
            }
        ],
        store=True,
    )

    const understanding = await rtstream.understand({
        segmentation: { type: "time", window: "5s" },
        analyzers: [
            {
                type: "vlm",
                name: "scene",
                sampling: { frame_count: 2 },
                config: {
                    prompt: "Describe the people, movement, and activity in this stream window."
                }
            }
        ],
        store: true
    });

Understand: index the scene output

Create a semantic index that receives each new scene record.

    index = rtstream.index(
        name="quickstart-live-scenes",
        source=understanding.outputs["scene"],
        use_for=["semantic"],
    )

    const index = await rtstream.index({
        name: "quickstart-live-scenes",
        source: {
            understanding_id: understanding.id,
            output: "scene"
        },
        useFor: ["semantic"]
    });

The Node.js index source identifies the continuous understanding run and its scene output explicitly.

Act: connect for live alerts

Open a WebSocket before attaching the alert so matches can reach this client.

    websocket = conn.connect_websocket(coll.id)
    await websocket.connect()

    const websocket = await conn.connectWebsocket(coll.id);
    await websocket.connect();

Act: define the event

Give the condition a reusable prompt and label.

    event_id = conn.create_event(
        event_prompt="Detect when a person appears in the monitored area.",
        label="person_appears",
    )

    const eventId = await conn.createEvent(
        "Detect when a person appears in the monitored area.",
        "person_appears"
    );

Act: create and receive the alert

Attach the event to the index, then print the first alert and its playable evidence URL.

    alert_id = index.create_alert(
        event_id=event_id,
        callback_url="https://example.com",
        ws_connection_id=websocket.connection_id,
    )

    async for message in websocket.receive():
        if message.get("channel") == "alert":
            alert = message["data"]
            print(alert["label"], alert["confidence"])
            print(alert["explanation"])
            print(alert["stream_url"])
            break

    const alertId = await index.createAlert(
        eventId,
        "https://example.com",
        websocket.connectionId
    );

    for await (const message of websocket.receive()) {
        if (message.channel === "alert") {
            const alert = message.data;
            console.log(alert.label, alert.confidence);
            console.log(alert.explanation);
            console.log(alert.stream_url);
            break;
        }
    }

The alert includes the event label, confidence, explanation, timestamps, and a playable stream URL for the triggering window.

Stop the live resources

Disable delivery, stop the continuous jobs and stream, then close the WebSocket.

    index.disable_alert(alert_id)
    index.stop()
    understanding.stop()
    rtstream.stop()
    await websocket.close()


if __name__ == "__main__":
    asyncio.run(main())

    await index.disableAlert(alertId);
    await index.stop();
    await understanding.stop();
    await rtstream.stop();
    await websocket.close();
}

main();

RTStream understanding is continuous, so the first indexed records and alerts can take several time windows to appear. callback_url remains required when WebSocket delivery is enabled. This public sample uses https://example.com as a placeholder; replace it with an endpoint you control before using your own stream. Production applications should also run the same cleanup during shutdown or interruption.

Extend the loop to a live desktop

Desktop capture turns a screen, microphone, or system-audio source into an RTStream - with explicit permission from the person using the device. Your backend keeps the API key; the desktop client receives only a short-lived token. Desktop capture currently supports macOS and Windows. Install the desktop capture runtime:

pip install "videodb[capture]"

npm install videodb

See: create an authorized capture

Create a CaptureSession on your backend and mint a token that expires after ten minutes. Deliver only the session ID and token to the desktop client through your authenticated application channel.

import videodb

conn = videodb.connect()
coll = conn.get_collection()

capture_session = coll.create_capture_session(
    end_user_id="quickstart-user",
    metadata={"app": "desktop-quickstart"},
)
client_token = conn.generate_client_token(expires_in=600)

# Pass this mapping to the desktop process through your authenticated launcher.
desktop_environment = {
    "VIDEODB_CAPTURE_SESSION_ID": capture_session.id,
    "VIDEODB_CAPTURE_TOKEN": client_token,
}

import { connect } from "videodb";

const conn = connect();
const coll = await conn.getCollection();

const captureSession = await coll.createCaptureSession({
    endUserId: "quickstart-user",
    metadata: { app: "desktop-quickstart" }
});
const clientToken = await conn.generateClientToken(600);

// Pass this mapping to the desktop process through your authenticated launcher.
const desktopEnvironment = {
    VIDEODB_CAPTURE_SESSION_ID: captureSession.id,
    VIDEODB_CAPTURE_TOKEN: clientToken
};

Run the next block in a separate desktop process with that environment. This keeps the short-lived token out of logs, URLs, and source code. Request screen permission, select the default display, and begin streaming it.

import asyncio
import os
from videodb.capture import CaptureClient


async def capture(session_id, client_token):
    client = CaptureClient(client_token=client_token)

    await client.request_permission("screen_capture")
    channels = await client.list_channels()
    display = channels.displays.default
    display.store = True
    display.is_primary = True

    await client.start_session(
        capture_session_id=session_id,
        channels=[display],
    )

    await asyncio.to_thread(input, "Capture started. Press Enter to stop.")
    await client.stop_session()
    await client.shutdown()


session_id = os.environ["VIDEODB_CAPTURE_SESSION_ID"]
client_token = os.environ["VIDEODB_CAPTURE_TOKEN"]
asyncio.run(capture(session_id, client_token))

import { CaptureClient } from "videodb/capture";

async function capture(sessionId, clientToken) {
    const client = new CaptureClient({ sessionToken: clientToken });

    await client.requestPermission("screen-capture");
    const channels = await client.listChannels();
    const display = channels.displays.default;

    await client.startSession({
        sessionId,
        channels: [{
            channelId: display.id,
            type: "video",
            store: true,
            isPrimary: true
        }]
    });

    console.log("Capture started. Press Enter to stop.");
    await new Promise((resolve) => process.stdin.once("data", resolve));
    await client.stopSession();
    await client.shutdown();
}

const sessionId = process.env.VIDEODB_CAPTURE_SESSION_ID;
const clientToken = process.env.VIDEODB_CAPTURE_TOKEN;
await capture(sessionId, clientToken);

Leave this desktop process running after it prints Capture started. Return to the original backend session and continue with Understand and Act; press Enter in the desktop process only after you finish searching the live capture.

Understand: index the live screen

While the desktop process remains active, fetch the session again on your backend. The captured display is now an RTStream, so it uses the same continuous understanding pipeline as any other live source.

capture_session = coll.get_capture_session(capture_session.id)
screen = capture_session.get_rtstream("screen")[0]

screen_understanding = screen.understand(
    segmentation={"type": "time", "window": "5s"},
    analyzers=[
        {
            "type": "vlm",
            "name": "scene",
            "sampling": {"frame_count": 2},
            "config": {"prompt": "Describe the visible desktop activity."},
        }
    ],
    store=True,
)

screen_index = screen.index(
    name="quickstart-desktop-scenes",
    source=screen_understanding.outputs["scene"],
    use_for=["semantic"],
)

const activeCapture = await coll.getCaptureSession(captureSession.id);
await activeCapture.refresh();
const screen = activeCapture.getRTStream("screen")[0];

const screenUnderstanding = await screen.understand({
    segmentation: { type: "time", window: "5s" },
    analyzers: [
        {
            type: "vlm",
            name: "scene",
            sampling: { frame_count: 2 },
            config: { prompt: "Describe the visible desktop activity." }
        }
    ],
    store: true
});

const screenIndex = await screen.index({
    name: "quickstart-desktop-scenes",
    source: {
        understanding_id: screenUnderstanding.id,
        output: "scene"
    },
    useFor: ["semantic"]
});

Act: find a captured moment

After a few windows have been indexed, search the live desktop and generate a playable stream for each matching moment.

matches = screen.search(
    query="source code open in an editor",
    index_id=screen_index.id,
    result_threshold=5,
)

for shot in matches.get_shots():
    print(shot.text)
    print(shot.generate_stream())

const matches = await screen.search({
    query: "source code open in an editor",
    indexId: screenIndex.id,
    resultThreshold: 5
});

for (const shot of matches.getShots()) {
    console.log(shot.text);
    console.log(await shot.generateStream());
}

Stop live desktop processing

Stop the continuous index before its understanding run. Then return to the separate desktop process and press Enter so it stops capture and shuts down its local client.

screen_index.stop()
screen_understanding.stop()

await screenIndex.stop();
await screenUnderstanding.stop();

The same live index can also drive events and alerts. When capture stops, stored channels can be exported as regular VideoDB media for playback and longer-term search.

Continue building

You have now completed the loop with recorded and live media, and seen how a desktop becomes another live source: see the media, turn it into timestamped understanding, and act on evidence.

Understanding artifacts

Configure analyzers, prompts, schemas, and multimodal inputs.

Search and retrieval

Choose semantic search, structured query, aggregation, Search, or Ask.

RTStream understanding

Build continuous understanding, retrieval, and alert workflows.

Start Here

Core Concepts

Ingest

Understand

Act

Automate

Open Source Frameworks

Set up VideoDB

Get an API key

Install the SDK

See, understand, and search a video

See: upload a video

Understand: create visual and spoken artifacts

Understand: index the artifacts

Act: find and play matching moments

See, understand, and act on a live stream

See: connect the stream

Understand: analyze each window

Understand: index the scene output

Act: connect for live alerts

Act: define the event

Act: create and receive the alert

Stop the live resources

Extend the loop to a live desktop

See: create an authorized capture

Understand: index the live screen

Act: find a captured moment

Stop live desktop processing

Continue building

Understanding artifacts

Search and retrieval

RTStream understanding

​Set up VideoDB

​Get an API key

​Install the SDK

​See, understand, and search a video

​See: upload a video

​Understand: create visual and spoken artifacts

​Understand: index the artifacts

​Act: find and play matching moments

​See, understand, and act on a live stream

​See: connect the stream

​Understand: analyze each window

​Understand: index the scene output

​Act: connect for live alerts

​Act: define the event

​Act: create and receive the alert

​Stop the live resources

​Extend the loop to a live desktop

​See: create an authorized capture

​Understand: index the live screen

​Act: find a captured moment

​Stop live desktop processing

​Continue building

Understanding artifacts

Search and retrieval

RTStream understanding

Set up VideoDB

Get an API key

Install the SDK

See, understand, and search a video

See: upload a video

Understand: create visual and spoken artifacts

Understand: index the artifacts

Act: find and play matching moments

See, understand, and act on a live stream

See: connect the stream

Understand: analyze each window

Understand: index the scene output

Act: connect for live alerts

Act: define the event

Act: create and receive the alert

Stop the live resources

Extend the loop to a live desktop

See: create an authorized capture

Understand: index the live screen

Act: find a captured moment

Stop live desktop processing

Continue building