> ## Documentation Index > Fetch the complete documentation index at: https://docs.videodb.io/llms.txt > Use this file to discover all available pages before exploring further. # Core Concepts Overview > VideoDB is the perception, memory, and action layer for AI agents operating on video and audio. Every workflow follows the same loop, whether you're processing files, live streams, or desktop capture. ## The Platform Loop ``` See (Ingest) → Process → Understand (Indexes) → Remember → Retrieve (Search) → Act ``` ```python theme={null} import videodb conn = videodb.connect() coll = conn.get_collection() # SEE: Ingest from any source video = coll.upload(url="https://example.com/video.mp4") # UNDERSTAND: Create a reusable scene artifact and index it understanding = video.understand( analyzers=[ { "type": "vlm", "name": "scene", "config": {"prompt": "Extract key moments"}, } ] ) understanding.wait_until_complete() scene = understanding.get_analyzer("scene") scene_index = video.index(name="key_moments", source=scene) scene_index.wait_until_complete() # RETRIEVE: Search with natural language results = video.semantic_search( query="important announcement", index_ids=[scene_index.index_id], top_k=5, ) # ACT: Generate outputs, trigger actions for shot in results.shots: print(f"{shot.start}s - {shot.end}s") shot.play() # Playable evidence ``` *** ## See (Ingest) Get video and audio from anywhere into VideoDB. | Source | Method | | :-------------- | :------------------------------------------------------- | | File URL | `coll.upload(url="https://...")` | | Local file | `coll.upload(file_path="./video.mp4")` | | RTSP stream | `coll.connect_rtstream(url="rtsp://...", name="Camera")` | | Desktop capture | Capture SDK (screen, mic, camera) | ```python theme={null} # File-based video = coll.upload(url="https://example.com/meeting.mp4") # Live stream rtstream = coll.connect_rtstream( name="Security Camera", url="rtsp://user:pass@host:554/stream" ) ``` *** ## Process Understanding configuration converts raw media into reusable, processable artifacts before indexing. * **Scene segmentation** - Time-based or shot-based * **Frame sampling** - Control which frames to analyze * **Audio chunking** - Word, sentence, or time-based segments ```python theme={null} # Time-based finite-video analysis: one frame every 10 seconds understanding = video.understand( segmentation={"type": "time", "seconds": 10}, analyzers=[ { "type": "vlm", "name": "scene", "sampling": {"frame_count": 1}, "config": {"prompt": "Describe the scene."}, } ], ) understanding.wait_until_complete() # RTStreams continuously produce VLM output descriptors. live_understanding = rtstream.understand( segmentation={"type": "time", "window": "5s"}, analyzers=[ { "type": "vlm", "name": "scene", "sampling": {"frame_count": 2}, "config": {"prompt": "Describe this stream window."}, } ], store=True, ) live_scene_index = rtstream.index( name="live_scene", source=live_understanding.outputs["scene"], use_for=["semantic"], ) ``` This is where cost control happens - sampling policies trade compute for recall. *** ## Understand (Indexes) Indexes are programmable interpretation layers. You define what to extract with prompts. * **Prompt-driven** - Natural language instructions * **Model-orchestrated** - LLMs and VLMs do the work * **Additive** - Multiple indexes on same media * **Multimodal** - Visual and spoken ```python theme={null} # Create visual and spoken artifacts in one finite understanding run. understanding = video.understand( analyzers=[ { "type": "vlm", "name": "visual", "config": {"prompt": "Identify key moments and describe activities"}, }, {"type": "spoken_words", "name": "transcript"}, ] ) understanding.wait_until_complete() visual = understanding.get_analyzer("visual") transcript = understanding.get_analyzer("transcript") visual_index = video.index(name="visual", source=visual) transcript_index = video.index(name="transcript", source=transcript) visual_index.wait_until_complete() transcript_index.wait_until_complete() ``` *** ## Remember Indexes are stored as episodic memory. This is automatic by default. **What gets stored:** * Transcripts and embeddings * Scene descriptions and tags * Structured metadata * Retrieval structures For a live stream, set `store=True` when you plan to index its continuous VLM output: ```python theme={null} live_understanding = rtstream.understand( segmentation={"type": "time", "window": "5s"}, analyzers=[ { "type": "vlm", "name": "scene", "sampling": {"frame_count": 1}, "config": {"prompt": "Describe this stream window."}, } ], store=True, ) live_scene_index = rtstream.index( name="live_scene", source=live_understanding.outputs["scene"], use_for=["semantic"], ) ``` *** ## Retrieve (Search) Search across indexed content with natural language. Results include playable evidence. ```python theme={null} # Single video results = video.semantic_search( query="product demo", index_ids=[visual_index.index_id], top_k=10, ) # Single stream results = rtstream.search("intrusion", index_id=live_scene_index.id) # Collection-wide results = coll.search("quarterly results", top_k=10) ``` **Results include:** * **Timestamps** - Exact start/end times * **Text** - What was detected * **Score** - Relevance ranking * **Stream URL** - Playable link ```python theme={null} for shot in results.shots: print(f"{shot.start}s - {shot.end}s") shot.play() # Verify the result ``` *** ## Act Go from understanding to automation and outputs. ### Event Detection Continuously understand and index the live VLM output before adding an alert: ```python theme={null} live_understanding = rtstream.understand( segmentation={"type": "time", "window": "5s"}, analyzers=[ { "type": "vlm", "name": "scene", "sampling": {"frame_count": 2}, "config": {"prompt": "Describe activity in this stream window."}, } ], store=True, ) live_scene_index = rtstream.index( name="live_scene", source=live_understanding.outputs["scene"], use_for=["semantic"], ) event_id = conn.create_event( event_prompt="Detect intruder", label="security_alert" ) live_scene_index.create_alert( event_id=event_id, callback_url="https://your-backend.com/alerts" ) ``` ### Programmable Editing Compose outputs using the 4-layer editor architecture: ```python theme={null} from videodb.editor import Timeline, Track, Clip, VideoAsset video_asset = VideoAsset(id=video.id, start=10) clip = Clip(asset=video_asset, duration=20) track = Track() track.add_clip(0, clip) timeline = Timeline(conn) timeline.add_track(track) output = timeline.generate_stream() ``` *** ## Architecture Patterns The loop applies to different use cases: | Use Case | See | Understand | Act | | :--------------- | :----------------- | :------------------------ | :---------------- | | Video Search | Upload files | Index with domain prompts | Search + retrieve | | Monitoring | Connect RTSP | Real-time indexing | Alerts + webhooks | | Desktop Agent | Capture SDK | Index screen/mic | Context for LLM | | Media Automation | Upload + transcode | Index for editing | Timeline + export | *** ## Next Steps Collections, Videos, RTStreams, and other core objects Turn media into searchable knowledge How search returns playable evidence Real-time detection and automation