TwelveLabs Integration

Human Monitoring is Expensive, Exhausting, and Doesn’t Scale

From baby monitors to warehouse cameras or endless CCTV feeds, many companies still depend on human eyes to monitor live video—a costly, tedious, and highly error-prone approach. Fatigue inevitably sets in, accuracy declines, and scaling up means hiring more personnel rather than enhancing systems. But in an AI-driven era, monitoring shouldn’t be manual, miss critical details, or become a bottleneck. Imagine receiving instant alerts when packages are stolen, notifying parents the moment a baby attempts to climb out of a crib, proactively highlighting safety risks before they escalate, or doctors instantly knowing when ICU patients need immediate attention. This is precisely the responsive monitoring VideoDB delivers through its real-time infrastructure. VideoDB now provide first party integration with TwelveLabs’s advanced Pegasus 1.2 AI model for precise frame understanding.

Live video to Instant Action

Real-time video analysis isn’t merely a nice-to-have—it’s a fundamental shift in capability.

Safety and Security: Transforming reactive measures into proactive alerts, potentially saving lives during emergencies like flash floods or security breaches.
Enterprise Productivity: Converting passive meeting archives into interactive, searchable knowledge repositories, vastly enhancing collaboration.
Content Platforms: Automatically tagging, chaptering, and moderating content in unprecedented detail, elevating user experiences.

The opportunities are immense, yet technical barriers have historically prevented many development teams from unlocking this potential.

Reality of Building Video AI

If you’ve ever attempted to build a robust video understanding system, you know the pain firsthand. You’re often stuck playing the role of a systems integrator, wrestling with:

API Spaghetti: Managing credentials, juggling rate limits, and navigating diverse SDKs across multiple video processing, AI modeling, and storage services.
Scaling Nightmares: Each component scales independently, causing bottlenecks and inefficiencies—particularly with resource-intensive GPU workloads.
Latency Issues: The delays involved in interactions between storage, AI models, and your applications undermine genuine real-time capabilities.

These challenges can quickly stall innovation, turning promising ideas into lengthy, frustrating engineering endeavors.

Simplifying Video AI Infrastructure with VideoDB

VideoDB is a purpose-built infrastructure for AI driven video management. It provides a unified, AI-native infrastructure handling the complete lifecycle of video content—from ingestion and indexing to alert management—all via a streamlined, developer-friendly API. Leveraging VideoDB, you can effortlessly ingest multiple real-time video streams, manage customizable indexes uniquely tailored for specific analyses, and simultaneously analyze diverse aspects of a single video stream. Additionally, VideoDB’s targeted alerting system triggers automated webhooks, ensuring rapid responses to critical events. And now, we’ve equipped that powerful infrastructure with an even more powerful intelligence.

Introducing the TwelveLabs Integration: Your Frame Understanding, Supercharged

We’re excited to announce our native integration with TwelveLabs—going far beyond a traditional partnership. We’ve seamlessly embedded TwelveLabs’ advanced AI, powered by the exceptional Pegasus 1.2 model, directly within VideoDB.

VideoDB and TwelveLabs integration architecture showing seamless AI embedding

What does that mean for you?

No additional accounts.
No extra API keys.
Zero integration headaches.

Now, accessing TwelveLabs’ sophisticated video understanding models, like Pegasus, is as easy as adding a single parameter to your indexing call. All the AI power you need is fully integrated into your VideoDB environment—effortlessly blending world-class intelligence with unmatched simplicity.

Real-World Use Cases: From Theory to Action

With VideoDB + TwelveLabs, sophisticated real-time video monitoring is just minutes away:

Real-World Use Cases in Action

🌊 Flash Flood Detection

Click image to open interactive notebook →

Imagine a camera monitoring a dry riverbed in a flood-prone area. With TwelveLabs integrated into VideoDB, you can continuously detect critical events—like rapidly rising floodwaters—in real-time. The moment Pegasus recognizes the signs of a flash flood, VideoDB immediately triggers life-saving alerts. This isn’t just video analysis; it’s proactive, intelligent disaster prevention.

👶🏻 Baby Crib Monitoring

Click image to open interactive notebook →

After a long day, parents deserve restful, worry-free sleep. With VideoDB’s real-time monitoring powered by TwelveLabs’ Pegasus, you’ll instantly know if your baby tries climbing out of the crib or needs immediate attention. Sleep easy, knowing VideoDB and TwelveLabs have you covered—every second of the night.

How It Works: One Line of Code to Unlock a New World

Ready to see how simple this is? To index a video stream with TwelveLabs’ powerful Pegasus model, you just specify it as the model_name.

# Your existing stream object
# flood_stream = coll.connect_rtstream(...)

# Index visuals using TwelveLabs' Pegasus model
flood_scene_index = flood_stream.index_visuals(
    batch_config={
        "type": "time",
        "value": 10,
        "frame_count": 6,
    },
    prompt="Monitor the dry riverbed and surrounding area. If moving water is detected across the land, identify it as a flash flood and describe the scene.",
    # This is the magic line!
    model_name="twelvelabs-pegasus-1.2",
    name="Flash_Flood_Detection_Index"
)

print("Scene Index ID:", flood_scene_index.id)

// Your existing stream object
const floodStream = await coll.connectRTStream(
    "rtsp://your-stream-url",
    "Flood Detection Stream"
);

// Index visuals using TwelveLabs' Pegasus model
const floodSceneIndex = await floodStream.indexVisuals({
    batchConfig: {
        type: "time",
        value: 10,
        frameCount: 6,
    },
    prompt: "Monitor the dry riverbed and surrounding area. If moving water is detected across the land, identify it as a flash flood and describe the scene.",
    // This is the magic line!
    modelName: "twelvelabs-pegasus-1.2",
    name: "Flash_Flood_Detection_Index"
});

console.log("Scene Index ID:", floodSceneIndex.id);

This integration lets you seamlessly test out TwelveLabs models right inside VideoDB’s indexing pipeline—giving you maximum flexibility, control, and ease. Now, you can instantly build real-time visual understanding apps without friction. We can’t wait to see what you create!

Try It Now: Interactive Notebooks

Ready to build your own real-time video intelligence system? Open these interactive notebooks directly in Google Colab and start experimenting with TwelveLabs’ Pegasus model:

Flash Flood Detection Notebook

Build a real-time flood detection system that monitors dry riverbeds and sends instant alerts

Baby Crib Monitoring Notebook

Create an AI-powered baby monitor that detects escape attempts and keeps parents informed

Overview

Agentic Workflows

Video Search and Understanding

Live Intelligence

Content Factory

Programmatic Editing

Safety & Compliance

Live video to Instant Action

Reality of Building Video AI

Simplifying Video AI Infrastructure with VideoDB

Introducing the TwelveLabs Integration: Your Frame Understanding, Supercharged

Real-World Use Cases: From Theory to Action

Real-World Use Cases in Action

🌊 Flash Flood Detection

👶🏻 Baby Crib Monitoring

How It Works: One Line of Code to Unlock a New World

Try It Now: Interactive Notebooks

Flash Flood Detection Notebook

Baby Crib Monitoring Notebook

​Live video to Instant Action

​Reality of Building Video AI

​Simplifying Video AI Infrastructure with VideoDB

​Introducing the TwelveLabs Integration: Your Frame Understanding, Supercharged

​Real-World Use Cases: From Theory to Action

​Real-World Use Cases in Action

​🌊 Flash Flood Detection

​👶🏻 Baby Crib Monitoring

​How It Works: One Line of Code to Unlock a New World

​Try It Now: Interactive Notebooks

Flash Flood Detection Notebook

Baby Crib Monitoring Notebook

Live video to Instant Action

Reality of Building Video AI

Simplifying Video AI Infrastructure with VideoDB

Introducing the TwelveLabs Integration: Your Frame Understanding, Supercharged

Real-World Use Cases: From Theory to Action

Real-World Use Cases in Action

🌊 Flash Flood Detection

👶🏻 Baby Crib Monitoring

How It Works: One Line of Code to Unlock a New World

Try It Now: Interactive Notebooks