Search and Retrieval

VideoDB gives you two ways to retrieve indexed video:

Use direct retrieval methods when you know how the data should be searched, filtered, or grouped.
Use intelligent search when you want VideoDB to interpret a natural-language request and choose the retrieval strategy.

Try the search notebook

Explore a runnable guide to searching and retrieving indexed video.

Direct retrieval methods

Method	Use it when you need	Returns
`semantic_search()`	Moments similar in meaning to a text query	`SearchResult` with timestamped `Shot` objects
`query()`	Exact filters, conditions, or sorting on one index	`SearchResult` with timestamped `Shot` objects
`aggregate()`	Counts, groups, facets, or metrics on one index	Aggregate rows as a `list` or `dict`

Let VideoDB choose the retrieval strategy

You do not always need to select a direct method yourself. Intelligent search interprets the request and uses the relevant retrieval capabilities internally.

search() plans retrieval for a natural-language request.
search(mode="deepsearch") performs deeper, multi-step retrieval and supports follow-up sessions.
ask() retrieves evidence and synthesizes a grounded answer.

One request, multiple video signals

Consider this request:

Find moments where someone discusses Nike while holding a phone outside a store.

A useful answer may depend on several indexed signals:

Signal	What it contributes
Transcript	Whether someone discusses Nike
Object detections	Whether a phone is present
Brand detections	Whether Nike appears in the scene
Scene understanding	Whether the person is outside a store
Temporal data	The exact start and end of the matching moment

With intelligent search, the application sends the goal rather than manually coordinating each retrieval step:

response = collection.search(
    query="Find moments where someone discusses Nike while holding a phone outside a store",
    top_k=10,
    return_fields=["scene", "transcript", "objects", "brands"],
)

for shot in response:
    print(shot.video_id, shot.start, shot.end)
    print(shot.generate_stream())

The result keeps the timestamped evidence and can hydrate the relevant index rows through return_fields.

Intelligent Search

Learn when to use Search, DeepSearch, and Ask.

Index selection at a glance

With direct retrieval, you choose the indexes. With intelligent search, VideoDB selects them based on your request and the available index schemas.

Method	Accepted selectors	Selection rule
`semantic_search()`	`index_names` and/or `index_ids`	Select zero, one, or many indexes. If neither is provided, VideoDB searches all semantic indexes in scope.
`query()`	`index_name` or `index_id`	Select exactly one index using either `index_name` or `index_id`.
`aggregate()`	`index_name` or `index_id`	Select exactly one index using either `index_name` or `index_id`.
`search()` (`default` and `deepsearch` modes)	None	Index selectors are not accepted. VideoDB chooses the indexes and retrieval strategy.
`ask()` (`default` and `deepsearch` modes)	None	Index selectors are not accepted. VideoDB chooses the indexes used as answer context.

Plural selectors (index_names, index_ids) support multi-index retrieval. Singular selectors (index_name, index_id) identify one index.

How index and field targeting works

An index name searches all applicable fields inside that index. Add a field path when you want to target one specific field.

# Search all semantic fields inside the scene index
results = collection.semantic_search(
    query="a busy street at night",
    index_names=["scene"],
)

# Search only outputs.location inside the scene index
results = collection.semantic_search(
    query="a busy street at night",
    index_names=["scene.outputs.location"],
)

Filter fields follow the same targeting pattern. Suppose both the transcript and ocr indexes contain a text field. An unqualified field checks text in every index in scope where that field is present:

response = collection.search(
    query="Find moments about emergency exits",
    filter=[
        {"field": "text", "op": "contains", "value": "EXIT"},
    ],
)

Qualify the field when the condition should use only text detected on screen by the ocr index:

response = collection.search(
    query="Find moments about emergency exits",
    filter=[
        {"field": "ocr.text", "op": "contains", "value": "EXIT"},
    ],
)

Form	Meaning
`index_name`	Search all applicable fields inside that index.
`field`	Apply the condition to every index in scope that contains the field.
`index_name.field`	Target one field inside one specific index.

For semantic refs, an exact index-name match wins first. Otherwise, VideoDB treats the first segment as the index name and the remainder as its field path. For query() and aggregate(), the singular index_name or index_id already limits the operation to one index.

Direct retrieval in detail

Use the direct methods when your application should control the retrieval operation and index selection.

Find similar moments with `semantic_search()`

Use semantic search when you know the meaning you want to match and which semantic indexes should be searched.

results = video.semantic_search(
    query="a customer reacting positively to a product",
    index_names=["scene"],
    top_k=10,
    score_threshold=0.7,
)

for shot in results:
    print(shot.start, shot.end, shot.search_score)

You can select indexes by name or ID. If you omit both selectors, VideoDB searches all semantic indexes in the video or collection scope.

results = collection.semantic_search(
    query="person presenting at a podium",
    index_ids=["idx_scene_123"],
    top_k=20,
)

Add a filter when semantic similarity should also satisfy an explicit condition:

results = collection.semantic_search(
    query="customer discussing a product launch",
    index_names=["scene"],
    filter=[
        {"field": "brand_names", "op": "contains", "value": "Nike"},
    ],
    top_k=20,
)

Filters must use fields supported by the indexes available to the search. Inspect an index’s field_schema before relying on a field or operator in production.

Apply exact conditions with `query()`

Use query when you know the index and the exact conditions to apply. Query does not interpret natural language or calculate semantic similarity.

results = collection.query(
    index_name="objects",
    filter=[
        {"field": "label", "op": "contains", "value": "cell phone"},
    ],
    limit=100,
    sort=[("max_score", "desc")],
)

Select exactly one index using index_name or index_id.

results = video.query(
    index_id="idx_objects_123",
    filter=[{"field": "label", "op": "==", "value": "person"}],
    limit=50,
)

Count and group with `aggregate()`

Use aggregation when you need analytics rather than matching moments. group_by is required and must reference an aggregatable field.

response = collection.aggregate(
    index_name="brands",
    group_by="brand_names",
    metric="count",
    limit=100,
)

for row in response["results"]:
    print(row)

Filters can narrow the rows included in the aggregation:

response = collection.aggregate(
    index_name="objects",
    filter=[{"field": "max_score", "op": ">=", "value": 0.8}],
    group_by="label",
    metric="count",
    limit=100,
)

counts = response["results"]

Unlike semantic search and query, aggregation returns an envelope rather than Shot objects. Read the aggregate rows from response["results"] and optional migration warnings with response.get("warnings", []).

Which direct method should I use?

If you know…	Use
The meaning you want to match	`semantic_search()`
The exact condition to apply	`query()`
What you want to count or group	`aggregate()`
Only the goal, not the retrieval strategy	Intelligent Search

Video and collection scope

The same methods are available at both scopes:

video.semantic_search(...)
collection.semantic_search(...)

video.query(...)
collection.query(...)

video.aggregate(...)
collection.aggregate(...)

Use a video method for one video. Use a collection method to search every indexed video in a collection.

Next steps

Intelligent Search

Let VideoDB plan retrieval or synthesize an answer.

Results and Evidence

Work with shots, timestamps, returned fields, and streams.

Collection Search

Apply retrieval methods across a video library.

Start Here

Core Concepts

Ingest

Understand

Act

Automate

Open Source Frameworks

Try the search notebook

Direct retrieval methods

Let VideoDB choose the retrieval strategy

One request, multiple video signals

Intelligent Search

Index selection at a glance

How index and field targeting works

Direct retrieval in detail

Find similar moments with `semantic_search()`

Apply exact conditions with `query()`

Count and group with `aggregate()`

Which direct method should I use?

Video and collection scope

Next steps

Intelligent Search

Results and Evidence

Collection Search

Try the search notebook

​Direct retrieval methods

​Let VideoDB choose the retrieval strategy

​One request, multiple video signals

Intelligent Search

​Index selection at a glance

​How index and field targeting works

​Direct retrieval in detail

​Find similar moments with semantic_search()

​Apply exact conditions with query()

​Count and group with aggregate()

​Which direct method should I use?

​Video and collection scope

​Next steps

Intelligent Search

Results and Evidence

Collection Search

Direct retrieval methods

Let VideoDB choose the retrieval strategy

One request, multiple video signals

Index selection at a glance

How index and field targeting works

Direct retrieval in detail

Find similar moments with `semantic_search()`

Apply exact conditions with `query()`

Count and group with `aggregate()`

Which direct method should I use?

Video and collection scope

Next steps