> ## Documentation Index
> Fetch the complete documentation index at: https://docs.videodb.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Index Fields
> Choose which artifact fields become semantic text, filters, aggregates, and sort keys.
`fields` controls how an artifact becomes an index.
An artifact may contain many fields. VideoDB stores the additional artifact data with the indexed records, but only selected fields are optimized for retrieval capabilities:
* direct semantic search
* exact/text matching
* structured filters
* aggregation
* sorting
Use `return_fields` at search time to choose which stored fields should come back in results.
***
## Example artifact
A VLM scene artifact may contain scene records like this:
```json theme={null}
{
"scene_id": "scene-000001",
"start": 12.4,
"end": 18.9,
"data": {
"scene_description": "A person walks through a retail aisle while holding a phone.",
"activity": "walking through store",
"setting": "retail aisle",
"frames": [{"timestamp": 14.2, "asset": {"type": "s3_object", "key": "..."}}]
}
}
```
A `fields` config tells VideoDB how to index values that exist on that artifact:
```python theme={null}
fields={
"semantic": ["scene_description", "activity", "setting"],
"filter": ["activity", "setting"],
"aggregate": ["activity", "setting"],
}
```
The `frames` field is still stored with the record. Request it later with `return_fields` when you need it in search results. Object labels and brand names usually live on separate `objects` and `brands` artifacts.
***
## Dotted paths for nested data
Field names may be **dotted paths** that reach inside nested objects, so you do not need to flatten your data. When a path crosses a list, values are collected from **every element**.
An object-detection scene looks like this:
```json theme={null}
{
"scene_id": "scene-0.000-8.634",
"start": 0.0,
"end": 8.634,
"data": {
"frames": [
{
"frame_id": "frame-scene-0.000-8.634-7.000",
"timestamp": 7.0,
"detections": [
{
"label": "person",
"score": 0.5856,
"box": {
"box": [0.0002, 0.2332, 0.6967, 0.9947],
"unit": "normalized"
}
},
{
"label": "dog",
"score": 0.5041,
"box": {
"box": [0.0018, 0.2346, 0.6908, 0.7378],
"unit": "normalized"
}
}
]
}
]
},
"metadata": {}
}
```
Index the nested values directly:
```python theme={null}
video.index(
name="objects",
source=objects,
use_for=["query", "aggregate"],
fields={
"filter": ["frames.detections.label", "frames.detections.score"],
"aggregate": ["frames.detections.label"],
"sort": ["frames.detections.score"],
},
)
```
`frames.detections.label` becomes the list of every detection label in the scene (`["person", "dog"]`); `frames.detections.score` becomes the list of every score. Bounding boxes remain nested under `frames[].detections[].box`, and the raw `frames` structure stays stored-only; request the objects index data with `return_fields` when you need timestamps or boxes.
Dotted paths work everywhere a field name does: `fields` groups, `query()` filters, `aggregate(group_by=...)`, and targeted semantic refs (`index_names=["scene.outputs.location"]`).
***
## Field groups
| Group | Values | Used for |
| ----------- | ----------------------------- | -------------------------------------------------------------------------------------------- |
| `semantic` | list of text-like field names | direct semantic/vector search with `semantic_search()`; also usable by high-level `search()` |
| `filter` | list of field names | structured `query()` filters |
| `aggregate` | list of field names | `aggregate()` count/group/facet operations |
| `sort` | list of field names | result ordering |
Do not add `clip` to `fields`. Clips are generated automatically from `video_id`, `start`, and `end` for moment results. Use `include_clip=False` at retrieval time if you do not need playable clips.
***
## Supported field value types
| Type | Example | Common use |
| -------------- | ----------------------------------- | ---------------------------------------------------------------------------- |
| `string` | `"Nike"` | filter, aggregate |
| `text` | `"A person walks through a store."` | semantic/text retrieval |
| `number` | `0.94` | filter, sort, aggregate |
| `boolean` | `true` | filter, aggregate |
| `string_array` | `["person", "phone"]` | contains filters, facets (an array of objects is reported as `string_array`) |
| `number_array` | `[0.91, 0.88]` | advanced filters, returned fields |
| `object` | `{...}` | stored record detail, returned fields; index inner values with dotted paths |
Nested values you filter or aggregate on should be declared with their dotted path. Declaring them projects the values into the queryable store at ingest time, so retrieval stays fast.
***
## Default fields
`fields` is optional per group. Any group you omit is **derived from your data**: well-known names get product defaults (`text`/`scene_description` → semantic, `language`/`brand_names` → filter + aggregate, `frames.detections.label` → filter + aggregate, `frames.detections.score` → filter + sort), and everything else is classified by shape — prose becomes semantic, scalars and scalar lists become filter + aggregate (numbers also sort), and nested structures stay stored-only.
Groups you declare are used verbatim, and an explicit empty group (`"filter": []`) opts out. See [Create an Index](/pages/understand/indexing-pipelines/create-an-index#default-fields-derived-from-your-data) for the full derivation tables.
```python theme={null}
# Declare only what you want to control; the rest is derived:
video.index(
name="scene",
source=analyzer,
use_for=["semantic", "query"],
fields={"semantic": ["outputs.description"]}, # filter/aggregate still derived
)
```
***
## User-provided temporal records
If you index your own timestamped records, choose fields the same way:
```python theme={null}
fields={
"semantic": ["summary"],
"filter": ["scene_type"],
"aggregate": ["scene_type"],
}
```
***
## Return fields at retrieval time
Indexing does not decide the result payload. Search does.
Use `return_fields` to choose which stored fields to include in each result:
```python theme={null}
results = video.semantic_search(
query="happy reunion outside a shop",
index_names=["scene"],
return_fields=["scene_description", "activity", "frames"],
)
```
For multi-index search, namespace fields by index name:
```python theme={null}
results = collection.search(
query="person talking about Nike while holding a phone",
return_fields={
"scene": ["scene_description", "frames"],
"transcript": ["text"],
"objects": ["object_labels", "frames"],
"brands": ["brand_names"],
},
)
```
For debugging or inspection, request all stored fields:
```python theme={null}
results = video.search(
query="product demo moments",
return_fields="all",
)
```
Returned fields can make responses larger. Keep them narrow for broad searches and request detailed fields only when you need to display or inspect them.
***
## Filter syntax
Query filters use fields that were indexed in `fields["filter"]`, including dotted paths.
| Filter type | Example |
| ------------------- | ---------------------------------------------------- |
| equality | `{"activity": "walking through store"}` |
| contains | `{"frames.detections.label": {"contains": "phone"}}` |
| numeric range | `{"frames.detections.score": {">=": 0.9}}` |
| boolean | `{"is_song": True}` |
| multiple conditions | `{"activity": "walking", "setting": "retail aisle"}` |
How filters behave on **list-valued fields** (anything a dotted path collected across a list, or a plain array field):
* equality means element membership: `{"frames.detections.label": "phone"}` matches a scene where **any** detection is a phone
* numeric comparators match **any element**: `{"frames.detections.score": {">=": 0.9}}` matches if any detection scores ≥ 0.9
* `contains` is a per-element substring match: `"car"` matches `["car", "truck"]` but not `["cartoon"]`
***
## Aggregation syntax
Aggregation uses fields that were indexed in `fields["aggregate"]`.
```python theme={null}
collection.aggregate(
index_name="brands",
group_by="brand_names",
)
# Dotted paths group by nested values, with one group per detection label
collection.aggregate(
index_name="objects",
group_by="frames.detections.label",
metric="count",
)
```
List-valued fields explode to one group per element (each detection label counts separately). Dict-valued fields (`{"person": 3, "phone": 1}`) explode to one group per key, and numeric metrics (`sum`/`avg`/`min`/`max`) apply to the dict's values.
Common metrics:
| Metric | Meaning |
| ------------ | ----------------------------- |
| `count` | count matching scenes/records |
| `sum(field)` | sum numeric field values |
| `avg(field)` | average numeric field values |
| `min(field)` | minimum value |
| `max(field)` | maximum value |
***
## Best practices
1. Put natural-language descriptions in `fields["semantic"]`.
2. Put exact labels, enums, numbers, and booleans in `fields["filter"]`.
3. Put fields you want to count or facet in `fields["aggregate"]`.
4. Put fields you want to order by in `fields["sort"]`.
5. Keep indexing focused on retrieval capabilities; request display/debug details with `return_fields` during search.
6. Reach into nested data with dotted paths (`frames.detections.label`) instead of reshaping your artifacts. Declare the paths in `fields` so the values are projected for fast retrieval.
***
## Next steps
Create indexes from artifacts.
Inspect fields, schemas, and records after indexing.