VideoDB Documentation

Build with VideoDB

AWS Rekognition and VideoDB - Intelligent Video Clips

⁠

Overview

👨‍🍳 Imagine effortlessly identifying individuals in videos and generating clips of them from episodes of a TV series or other videos. This is now achievable without manual effort, thanks to cutting-edge technology provided by AWS Rekognition API and VideoDB

⁠

Key components of this technique include:

AWS Rekognition API: The

FaceSearch⁠

endpoint scans the video and identifies the presence of individuals.

VideoDB: Designed specifically for videos, it stores videos, allowing extraction of clips and selective timeline streams effortlessly.

Most spicy thing about our dish 🌶️ ?

We have fully automated this process — it's accurate and fast. We gather timestamps of individuals’ appearances, extract video clips, and merge them seamlessly, without touching any video editor or manual effort.🚀

Setup

Install required packages:

boto3:

S3⁠

and

AWS rekognition api⁠

⁠

pytube: Download YouTube Videos.

videodb : VideoDB python SDK

!pip install boto3 pytube requests videodb

Helper functions

To work with AWS Rekognition API, we’ll need to download the video and image of persons we want to extract from that video.

download_video_yt : downloads a YouTube video at highest resolution, saving it with a specified filename.

download_file : downloads any file from a URL using wget, saving it with a specified name.

Downloading media

Let’s download a YouTube video and extract clips featuring a specific individual to create a new video focused on them. Here’s a

15-minute video⁠

from HBO's Silicon Valley show. Let’s download the video and image of Erlich Bachman GilFoyle using publicly available data.

⁠

⁠

Configuration

VideoDB Configuration

To connect to VideoDB, simply create a Connection object.

AWS Configuration

Setup your AWS secrets: aws_secret_key_id, aws_secret_access_key and aws_reigon

user_id - This is the AWS user ID associated with the face we are indexing. You can assign a random UUID or select an ID of your choice for this purpose.

collection_id - This refers to the AWS collection in which we will store the indexed faces.

bucket_name - This is the name of the S3 bucket where the video will be stored.

Rekognition API Workflow

Create a collection for faces to store

Create a new user with given user_id

Index Face from image and store index in collection_id

Associate Index face with newly created user

Upload a video to S3 Bucket and Start face search using

StartFaceSearch⁠

⁠

Extracting Shots from Video

Preparing Timestamps

When the Rekognition API detects indexed faces from a collection, it provides timestamps. Our goal is to merge timestamps that are part of the same scene. While the

AWS Segment API⁠

is one way to do this, we'll use a simpler approach. If the interval between two timestamps is smaller than a specified threshold, they will be grouped as a single shot. Additionally, we will apply a padding on both the right and left sides of each shot for better context.

You can experiment with both variables threshold and padding to get a better result.

Now that we have the timestamps of our favourite artist’s appearance in a video, We can move on to create clip using VideoDB Please follow next steps in this blog 👉

⁠

Instant Clips of Your Favorite Characters⁠

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.