VideoDB Documentation
Examples and Tutorials
Eleven Labs x VideoDB: Adding AI Generated voiceovers to silent footage
Case: Automatically Creating a David Attenborough-style Voiceover for Silent Footage of the Underwater World
Overview
Voiceovers are the secret sauce that turns silent footage into captivating stories. They add depth, emotion, and excitement, elevating the viewing experience to new heights. With VideoDB's cutting-edge technology, creating dynamic voiceovers for your videos is easier than ever before.
Creating AI content and merging outputs from different tools can be tough and time-consuming. Doing it manually is even harder! What if you could do it all- in just a few lines of code?
VideoDB's cool tech helps you generate intelligent AI content and combine outputs from various tools quickly and automatically. No more long and tedious processes. Just simple, efficient, and awesome results!
Let’s try adding an automatically generated voiceover to in the style and voice of Sir David Attenborough. We shall do this using the powerful tech provided by Open AI, elevenlabs and VideoDB
Setup
📦 Installing packages
%pip install openai
%pip install videodb
🔑 API Keys
Before proceeding, ensure access to , , and API key. If not, sign up for API access on the respective platforms.
import os
os.environ["OPENAI_API_KEY"] = ""
os.environ["ELEVEN_LABS_API_KEY"] = ""
os.environ["VIDEO_DB_API_KEY"] = ""
🎙️ ElevenLab's Voice ID
You will also need ElevenLab's VoiceID of a Voice that you want to use.
For this demo, we will be using ElevenLabs has a large variety of voices to choose from (browse them
). Once finalized, copy the Voice ID from ElevenLabs and link it here.voiceover_artist_id = "VOICEOVER_ARTIST_ID"
Implementation
🌐 Step 1: Connect to VideoDB
Connect to VideoDB using your API key to establish a session for uploading and manipulating video files.
from videodb import connect
# Connect to VideoDB using your API key
conn = connect()
🎥 Step 2: Upload Video
# Upload a video by URL (replace the url with your video)
video = conn.upload(url='https://youtu.be/RcRjY5kzia8')
🔍 Step 3: Analyze Scenes and Generate Scene Descriptions
Start by analyzing the scenes within your Video using VideoDB's scene indexing capabilities. This will provide context for generating the script prompt.
video.index_scenes()
Let's view the description of first scene from the video
scenes = video.get_scenes()
print(f"{scenes[0]['start']} - {scenes[0]['end']}")
print(scenes[0]["response"])
Output:
0 - 9.033333333333333
The image presents a close-up, textured pattern reminiscent of organic forms. Dominated by a cool color palette of blue and turquoise hues, it gives the impression of looking at a magnified cluster of cells, crystalline structures, or perhaps a zoomed-in portion of aquatic flora. The shapes appear irregular yet symmetrical, like petals or leaves clustered tightly together. Light and shadow play across the surfaces, creating a dynamic interplay that suggests depth and complexity. The overall effect is one of natural beauty, with a soothing, almost hypnotic visual rhythm. The image is abstract enough to allow for multiple interpretations, depending on the viewer's perspective.
🎤 Step 4: Generate Voiceover Script with LLM
Combine scene descriptions with the script prompt, instructing LLM to create a voiceover script in David Attenborough's style.
This script prompt can be refined and tweaked to generate the most suitable output. Check out to explore more use cases.
import openai
client = openai.OpenAI()
script_prompt = "Here's the data from a scene index for a video about the underwater world. Study this and then generate a synced script based on the description below. Make sure the script is in the language, voice and style of Sir David Attenborough \n \n"
full_prompt = script_prompt
# run this prompt for each scene
for scene in scenes:
full_prompt += f"- {scene}\n"
openai_res = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "system", "content": full_prompt}],
)
voiceover_script = openai_res.choices[0].message.content
# If you have ElevenLab's paid plan remove the :2500 limit on
# voiceover script.
voiceover_script = voiceover_script[:2500]
🎤 Step 5: Generate Voiceover Audio with ElevenLabs
Utilize the generated script to synthesize AI-generated voiceover narration in David Attenborough's voice using ElevenLabs API.
import requests
# Call ElevenLabs API to generate voiceover
url = f"https://api.elevenlabs.io/v1/text-to-speech/{voiceover_artist_id}"
headers = {
"xi-api-key": os.environ.get("ELEVEN_LABS_API_KEY"),
"Content-Type": "application/json"
}
payload = {
"model_id": "eleven_monolingual_v1",
"text": voiceover_script,
"voice_settings": {
"stability": 0.5,
"similarity_boost": 0.5
}
}
elevenlabs_res = requests.request("POST", url, json=payload, headers=headers)
# Save the audio file
audio_file = "audio.mp3"
CHUNK_SIZE = 1024
with open(audio_file, 'wb') as f:
for chunk in elevenlabs_res.iter_content(chunk_size=CHUNK_SIZE):
if chunk:
f.write(chunk)
🎬 Step 6: Add Voiceover to Video with VideoDB
In order to use the voiceover generated above, let's upload the audio file (voiceover) to VideoDB first
audio = conn.upload(file_path=audio_file)
from videodb.timeline import Timeline
from videodb.asset import VideoAsset, AudioAsset
# Create a timeline object
timeline = Timeline(conn)
# Add the video asset to the timeline for playback
video_asset = VideoAsset(asset_id=video.id)
timeline.add_inline(asset=video_asset)
# Add the audio asset to the timeline for playback
audio_asset = AudioAsset(asset_id=audio.id)
timeline.add_overlay(start=0, asset=audio_asset)
🪄 Step 7: Review and Share
Preview the video with the integrated voiceover to ensure it functions correctly.
Once satisfied, generate a stream of the video and share the link for others to view and enjoy this wholesome creation!
from videodb import play_stream
stream_url = timeline.generate_stream()
play_stream(stream_url)
Output:
🎉 Conclusion
By leveraging advanced AI technologies, you can enhance the storytelling and immersive experience of your video content. Experiment with different prompts and scene analysis techniques to further improve the quality and accuracy of the voiceovers. Enjoy creating captivating narratives with AI-powered voiceovers using VideoDB!
For more such explorations, refer to the and join the VideoDB community on or for support and collaboration 🙌🏼
We're excited to see your creations, so we welcome you to share your creations via , or
Want to print your doc?
This is not the way.
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.