videodb
VideoDB Documentation
videodb
VideoDB Documentation
Build with VideoDB

icon picker
AWS Rekognition and VideoDB - Intelligent Video Clips

Overview

👨‍🍳 Imagine effortlessly identifying individuals in videos and generating clips of them from episodes of a TV series or other videos. This is now achievable without manual effort, thanks to cutting-edge technology provided by AWS Rekognition API and VideoDB
image.png

Key components of this technique include:
AWS Rekognition API: The endpoint scans the video and identifies the presence of individuals.
VideoDB: Designed specifically for videos, it stores videos, allowing extraction of clips and selective timeline streams effortlessly.

light
Most spicy thing about our dish 🌶️ ?
We have fully automated this process — it's accurate and fast. We gather timestamps of individuals’ appearances, extract video clips, and merge them seamlessly, without touching any video editor or manual effort.🚀

Setup

Install required packages:
boto3:
and
pytube: Download YouTube Videos.
videodb : VideoDB python SDK
!pip install boto3 pytube requests videodb

Helper functions

To work with AWS Rekognition API, we’ll need to download the video and image of persons we want to extract from that video.
download_video_yt : downloads a YouTube video at highest resolution, saving it with a specified filename.
download_file : downloads any file from a URL using wget, saving it with a specified name.

Downloading media

Let’s download a YouTube video and extract clips featuring a specific individual to create a new video focused on them. Here’s a from HBO's Silicon Valley show. Let’s download the video and image of Erlich Bachman GilFoyle using publicly available data.
Screenshot 2023-12-29 at 1.18.55 PM.jpg

Configuration

VideoDB Configuration

To connect to VideoDB, simply create a Connection object.

AWS Configuration

Setup your AWS secrets: aws_secret_key_id, aws_secret_access_key and aws_reigon
user_id - This is the AWS user ID associated with the face we are indexing. You can assign a random UUID or select an ID of your choice for this purpose.
collection_id - This refers to the AWS collection in which we will store the indexed faces.
bucket_name - This is the name of the S3 bucket where the video will be stored.


Rekognition API Workflow

Create a collection for faces to store
Create a new user with given user_id
Index Face from image and store index in collection_id
Associate Index face with newly created user
Upload a video to S3 Bucket and Start face search using

Extracting Shots from Video

Preparing Timestamps

When the Rekognition API detects indexed faces from a collection, it provides timestamps. Our goal is to merge timestamps that are part of the same scene. While the is one way to do this, we'll use a simpler approach. If the interval between two timestamps is smaller than a specified threshold, they will be grouped as a single shot. Additionally, we will apply a padding on both the right and left sides of each shot for better context.
You can experiment with both variables threshold and padding to get a better result.

Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.