> ## Documentation Index
> Fetch the complete documentation index at: https://docs.videodb.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Accuracy Tips
> Better search results come from better indexing and smarter queries. This guide covers techniques to improve precision and recall.
## Quick Wins
1. **Use specific prompts** during indexing
2. **Choose the right retrieval mode** (semantic search vs structured query)
3. **Tune thresholds** based on your use case
4. **Combine multiple indexes** for layered search
***
## Understanding Precision and Recall
| Metric | Definition | Goal |
| :- | :- | :- |
| **Precision** | % of returned results that are relevant | Fewer false positives |
| **Recall** | % of relevant content that was returned | Fewer missed results |
**The trade-off:** Higher precision often means lower recall, and vice versa. Tune based on your priority.
***
## Indexing for Accuracy
### Specific Prompts Beat Generic Ones
```python Python theme={null}
# Pass one of these VLM analyzer configurations to video.understand(...).
# Too generic - low precision
generic_scene = {"type": "vlm", "name": "scene", "config": {"prompt": "Describe the scene"}}
# Better - focused on what matters
vehicle_scene = {
"type": "vlm",
"name": "scene",
"config": {"prompt": "Identify all vehicles with their color, make, and model"},
}
# Best - structured for your retrieval needs
structured_scene = {
"type": "vlm",
"name": "scene",
"config": {"prompt": """Describe this scene with:
- People: count, actions, clothing colors
- Vehicles: type, color, direction of travel
- Environment: indoor/outdoor, time of day
"""},
}
```
```javascript Node.js theme={null}
// Pass one of these VLM analyzer configurations to video.understand(...).
const genericScene = { type: "vlm", name: "scene", config: { prompt: "Describe the scene" } };
const vehicleScene = {
type: "vlm",
name: "scene",
config: { prompt: "Identify all vehicles with their color, make, and model" }
};
const structuredScene = {
type: "vlm",
name: "scene",
config: { prompt: `Describe this scene with:
- People: count, actions, clothing colors
- Vehicles: type, color, direction of travel
- Environment: indoor/outdoor, time of day` }
};
```
### Match Extraction to Content Type
| Content | Extraction | Reasoning |
| :- | :- | :- |
| Static shots | 1 frame/scene | Single frame captures all info |
| Action/motion | 3-5 frames/scene | Need temporal context |
| Quick cuts | Shot-based | Respect natural boundaries |
| Continuous | Time-based, short intervals | Capture changes |
```python Python theme={null}
# For interviews (mostly static)
interview_analyzer = {
"type": "vlm",
"name": "interview",
"sampling": {"frame_count": 1},
"config": {"prompt": "Describe the speaker and topic"},
}
interview_segmentation = {"type": "time", "seconds": 30}
# For action content (need motion)
action_analyzer = {
"type": "vlm",
"name": "action",
"sampling": {"frame_count": 4},
"config": {"prompt": "Describe the activity and movements"},
}
action_segmentation = {"type": "time", "seconds": 5}
```
```javascript Node.js theme={null}
// Pass each analyzer and segmentation pair to video.understand(...).
const interviewAnalyzer = {
type: "vlm",
name: "interview",
sampling: { frame_count: 1 },
config: { prompt: "Describe the speaker and topic" }
};
const interviewSegmentation = { type: "time", seconds: 30 };
const actionAnalyzer = {
type: "vlm",
name: "action",
sampling: { frame_count: 4 },
config: { prompt: "Describe the activity and movements" }
};
const actionSegmentation = { type: "time", seconds: 5 };
```
***
## Query Strategies
### Semantic Search vs Structured Query
| Query Type | Use Semantic Search | Use Structured Query |
| :- | :- | :- |
| Questions | ✓ "How does the engine work?" | |
| Concepts | ✓ "explains machine learning" | |
| Exact fields | | ✓ `event_type == "goal"` |
| Technical categories | | ✓ `camera_view == "road_ahead"` |
| Numeric metadata | | ✓ `score >= 0.8` |
```python Python theme={null}
# Semantic for conceptual queries
results = video.semantic_search(
query="explains the benefits",
index_names=["scene"],
top_k=10,
)
# Structured query for exact indexed fields
results = video.query(
index_name="events",
filter=[{"field": "event_type", "op": "==", "value": "quarterly_earnings"}],
limit=10,
)
```
```javascript Node.js theme={null}
// Semantic for conceptual queries
const semanticResults = await video.semanticSearch(
"explains the benefits",
{ indexNames: ["scene"], topK: 10 }
);
// Structured query for exact indexed fields
const exactResults = await video.query({
indexName: "events",
filter: [{ field: "event_type", op: "==", value: "quarterly_earnings" }],
limit: 10
});
```
### Threshold Tuning
| Parameter | Higher Value | Lower Value |
| :- | :- | :- |
| `score_threshold` | ↑ Precision, ↓ Recall | ↓ Precision, ↑ Recall |
| `top_k` | Allows more returned moments | Limits the returned moments |
```python Python theme={null}
# High precision (strict)
results = video.semantic_search(
query="CEO announcement",
index_names=["scene"],
score_threshold=0.5,
top_k=3,
)
# High recall (inclusive)
results = video.semantic_search(
query="CEO announcement",
index_names=["scene"],
score_threshold=0.1,
top_k=20,
)
```
```javascript Node.js theme={null}
// High precision (strict)
const highPrecisionResults = await video.semanticSearch(
"CEO announcement",
{ indexNames: ["scene"], topK: 3, scoreThreshold: 0.5 }
);
// High recall (inclusive)
const highRecallResults = await video.semanticSearch(
"CEO announcement",
{ indexNames: ["scene"], topK: 20, scoreThreshold: 0.1 }
);
```
***
## Evaluating Search Quality
### Set Up Ground Truth
Create test queries with known correct answers:
```python theme={null}
test_cases = [
{
"query": "six",
"expected_timestamps": [(4.0, 5.0), (14.0, 15.0), (24.0, 25.0)]
},
{
"query": "introduction",
"expected_timestamps": [(0.0, 30.0)]
}
]
```
### Measure Precision and Recall
```python theme={null}
def evaluate_search(results, expected):
"""Calculate precision and recall"""
returned = set((s.start, s.end) for s in results.get_shots())
expected_set = set(expected)
true_positives = len(returned & expected_set)
false_positives = len(returned - expected_set)
false_negatives = len(expected_set - returned)
precision = true_positives / (true_positives + false_positives) if returned else 0
recall = true_positives / (true_positives + false_negatives) if expected_set else 0
return {"precision": precision, "recall": recall}
# Evaluate
results = video.semantic_search(query="six", index_names=["scene"])
metrics = evaluate_search(results, [(4.0, 5.0), (14.0, 15.0), (24.0, 25.0)])
print(f"Precision: {metrics['precision']:.2f}, Recall: {metrics['recall']:.2f}")
```
***
## Advanced Techniques
### Multi-Index Search
Layer indexes for precise filtering:
```python theme={null}
# Search vehicles index
vehicle_results = video.semantic_search(
query="red car",
index_ids=[vehicle_index.index_id],
)
# Search motion index
motion_results = video.semantic_search(
query="speeding",
index_ids=[motion_index.index_id],
)
# Intersect exact timestamped shots for high precision.
motion_timestamps = {(shot.start, shot.end) for shot in motion_results.get_shots()}
final_shots = [
shot for shot in vehicle_results.get_shots()
if (shot.start, shot.end) in motion_timestamps
]
```
### Metadata Filtering
Pre-filter before semantic search:
```python theme={null}
# Narrow search space with metadata
results = coll.semantic_search(
query="product demo",
filter=[{"field": "category", "op": "==", "value": "marketing"}],
index_names=["scene"],
)
```
### Post-Processing with LLMs
For complex queries, use an LLM to refine results:
```python theme={null}
# Get broad results first
results = video.semantic_search(
query="numbers",
index_names=["scene"],
score_threshold=0.1 # Low threshold for high recall
)
# Refine with LLM
refined = llm.filter(
results,
criteria="Keep only results showing numbers greater than 10"
)
```
***
## Common Pitfalls
| Problem | Cause | Fix |
| :- | :- | :- |
| Missing relevant results | Threshold too high | Lower `score_threshold` |
| Too many irrelevant results | Threshold too low | Raise `score_threshold` |
| Semantic search misses an exact word or phrase | Exact terms need full-text matching | Use an [FTS index](/examples-and-tutorials/video-rag/keyword-search) |
| Poor visual search results | Generic prompt | Use specific, structured prompts |
| Inconsistent results | Wrong extraction config | Match extraction to content type |
***
## Iterative Improvement
1. **Start broad** - Low thresholds, high recall
2. **Evaluate** - Check precision on sample queries
3. **Refine** - Adjust thresholds, improve prompts
4. **Test** - Validate against ground truth
5. **Repeat** - Iterate until satisfied
***
## Next Steps
Optimize for speed and efficiency
Better extraction strategies
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.