AI-Powered Content Moderation

The Idea

Content moderation can be complex, often requiring multiple tools, manual timestamp extraction, and intricate integration work. Setting up these pipelines involves managing credentials, parsing responses, and stitching everything together. VideoDB simplifies this into a “Prompt-and-Filter” workflow using structured video understanding. The analyzer emits a structured moderation_label and reason for each timed segment, which the index can query and aggregate. The innovation is simple: instead of generic video descriptions, we give the AI a strict moderation role with a small structured schema. This turns unstructured video into timed data that can be filtered and counted. Want stricter moderation? Update the prompt. Need different criteria? Change a few lines. It’s content moderation reimagined for the prompt engineering era.

Setup

Install Dependencies

pip install videodb

npm install videodb

Connect to VideoDB

Get your API key from VideoDB Console. Free for first 50 uploads, no credit card required.

from videodb import connect

conn = connect(api_key="YOUR_API_KEY")
coll = conn.get_collection()

import { connect } from 'videodb';

const conn = await connect({ apiKey: process.env.VIDEO_DB_API_KEY });
const coll = await conn.getCollection();

Implementation

Step 1: Upload Video

We’ll use a Breaking Bad clip with mixed content to test the moderation workflow.

# Upload video from YouTube
video = coll.upload(url='https://www.youtube.com/watch?v=Xa7UaHgOGfM')
print(f"Uploaded Video ID: {video.id}")

# Preview the video
video.play()

// Upload video from YouTube
const video = await coll.uploadURL({ url: 'https://www.youtube.com/watch?v=Xa7UaHgOGfM' });
console.log(`Uploaded Video ID: ${video.id}`);

// Get the player URL
const playerUrl = video.playerUrl;
console.log(playerUrl);

Step 2: Understand Content with a Moderator Prompt

This is the core innovation. We give the AI a strict role as a Content Moderator with a schema containing moderation_label and reason. The prompt instructs the AI to analyze visual content for specific inappropriate elements and record the decision in those fields. This structured labeling transforms unstructured video into searchable, filterable data.

moderation_prompt = """
You are a Content Moderator. Analyze the visual content for inappropriate elements:
1. Violence (fighting, hitting, shooting)
2. Weapons (guns, knives)
3. Blood or gore
4. Drug use
5. Sexual content

Set moderation_label to exactly CONTENT_UNSAFE if any of these are present.
Otherwise, set moderation_label to exactly CONTENT_SAFE.
Set reason to a brief explanation of the decision.
"""
moderation_understanding = video.understand(
    segmentation={"type": "time", "seconds": 5},
    analyzers=[{
        "type": "vlm",
        "name": "moderation",
        "sampling": {"strategy": "uniform", "frame_count": 3},
        "config": {"prompt": moderation_prompt, "schema": {
            "moderation_label": "string", "reason": "string"}},
    }],
)
moderation_understanding.wait_until_complete()
moderation_analyzer = moderation_understanding.get_analyzer("moderation")
moderation_index = video.index(
    name="content_moderation",
    source=moderation_analyzer,
    use_for=["query", "aggregate"],
    fields={"filter": ["moderation_label"], "aggregate": ["moderation_label"]},
)
moderation_index.wait_until_complete()

const moderationPrompt = `
You are a Content Moderator. Analyze visual content for violence, weapons,
blood or gore, drug use, and sexual content.
Set moderation_label to exactly CONTENT_UNSAFE if any are present; otherwise
set it to exactly CONTENT_SAFE. Set reason to a brief explanation.
`;

const moderationUnderstanding = await video.understand(
  [{
    type: "vlm",
    name: "moderation",
    sampling: { strategy: "uniform", frame_count: 3 },
    config: {
      prompt: moderationPrompt,
      schema: { moderation_label: "string", reason: "string" },
    },
  }],
  { segmentation: { type: "time", seconds: 5 } },
);
await moderationUnderstanding.waitUntilComplete();
const moderationAnalyzer = await moderationUnderstanding.getAnalyzer("moderation");
const moderationIndex = await video.index(moderationAnalyzer, {
  name: "content_moderation",
  useFor: ["query", "aggregate"],
  fields: { filter: ["moderation_label"], aggregate: ["moderation_label"] },
});
await moderationIndex.waitUntilComplete();

Why this works: The structured moderation_label field supports exact filtering, while reason preserves the explanation for each timed decision.

Step 3: Review Moderation Output (Optional)

Want to see what the AI detected? Inspect the analyzer output to understand each timed moderation decision.

moderation_output = moderation_analyzer.get_output()
for scene in moderation_output["scenes"][:5]:
    data = scene.get("data", {})
    print(scene["start"], scene["end"], data.get("moderation_label"), data.get("reason"))

const moderationOutput = await moderationAnalyzer.getOutput();
for (const scene of moderationOutput.scenes.slice(0, 5)) {
  console.log(scene.start, scene.end, scene.data?.moderation_label, scene.data?.reason);
}

Sample output:

0 5.005 CONTENT_SAFE Title cards with a smoky background.
005 10.01 CONTENT_SAFE Two men in indoor settings with no inappropriate elements.
02 25.025 CONTENT_UNSAFE Implied physical confrontation and aggressive interaction.

Step 4: Filter for Safe Content

Now the magic happens. Aggregate the structured labels for a moderation summary, then query the timed records that are safe to include.

moderation_counts = video.aggregate(
    index_name=moderation_index.name,
    group_by="moderation_label",
    metric="count",
    limit=10,
)
safe_results = video.query(
    index_name=moderation_index.name,
    filter=[{"field": "moderation_label", "op": "==", "value": "CONTENT_SAFE"}],
    limit=100,
    sort=[("start", "asc")],
)
safe_shots = safe_results.get_shots()
print(f"Found {len(safe_shots)} safe segments")

const moderationCounts = await video.aggregate({
  indexName: moderationIndex.name,
  groupBy: "moderation_label",
  metric: "count",
  limit: 10,
});
const safeResults = await video.query({
  indexName: moderationIndex.name,
  filter: [{ field: "moderation_label", op: "==", value: "CONTENT_SAFE" }],
  limit: 100,
  sort: [["start", "asc"]],
});
console.log(`Found ${safeResults.getShots().length} safe segments`);

Step 5: Play the Clean Version

Compile the filtered results into a stream for playback.

clean_stream_url = safe_results.compile()
print("Stream URL:", clean_stream_url)
safe_results.play()

const cleanStreamUrl = await safeResults.compile();
console.log("Stream URL:", cleanStreamUrl);
await safeResults.play();

Here’s the result - a clean version with all inappropriate content removed:

What You Get

No external APIs or credentials required
Full control over moderation criteria through prompts
Instant filtering without video re-encoding
Granular 5-second scene analysis
Real-time playback of cleaned content
Customizable: change prompt to adjust moderation standards instantly

Perfect For

Educational platforms serving minor audiences
Family-friendly streaming services
Corporate training content libraries
Social media platforms with content policies
Broadcasting companies creating TV-safe edits
User-generated content platforms with safety requirements

The Result

What used to require multiple integrations, manual timestamp extraction, and complex video editing pipelines now works with prompt engineering. Change the moderation criteria by updating the prompt and running the moderation understanding again. Pure simplicity powered by VideoDB’s structured video understanding.

Explore Full Notebook

Open the complete implementation in Google Colab with detailed explanations and working code.

Profanity Detection

Detect and censor curse words with audio overlays

Keyword Search

Find and extract specific content from your videos

Overview

Agentic Workflows

Video Search and Understanding

Live Intelligence

Content Factory

Programmatic Editing

Safety & Compliance

The Idea

Setup

Install Dependencies

Connect to VideoDB

Implementation

Step 1: Upload Video

Step 2: Understand Content with a Moderator Prompt

Step 3: Review Moderation Output (Optional)

Step 4: Filter for Safe Content

Step 5: Play the Clean Version

What You Get

Perfect For

The Result

Explore Full Notebook

Profanity Detection

Keyword Search

​The Idea

​Setup

​Install Dependencies

​Connect to VideoDB

​Implementation

​Step 1: Upload Video

​Step 2: Understand Content with a Moderator Prompt

​Step 3: Review Moderation Output (Optional)

​Step 4: Filter for Safe Content

​Step 5: Play the Clean Version

​What You Get

​Perfect For

​The Result

Explore Full Notebook

​Related Tutorials

Profanity Detection

Keyword Search

The Idea

Setup

Install Dependencies

Connect to VideoDB

Implementation

Step 1: Upload Video

Step 2: Understand Content with a Moderator Prompt

Step 3: Review Moderation Output (Optional)

Step 4: Filter for Safe Content

Step 5: Play the Clean Version

What You Get

Perfect For

The Result

Related Tutorials