VideoDB Documentation
Director - Video Agent Framework
Agent Creation Playbook
This playbook will guide you through the process of creating your own agents within the Director framework. You'll learn:
🏛️ Understanding the Architecture
Before diving into agent creation, let's understand how Director works as a system. Director follows a modular architecture that enables seamless interaction between users and AI-powered video processing capabilities.
System Overview
Director's architecture is designed around three core principles:
Looking at the system architecture diagram below, you can see how these principles come together:
Understanding the Director Framework
Director consists of several key components working together:
The Reasoning Engine in Detail
The Reasoning Engine is the orchestrator of all agent activities. As shown in the architecture diagram, it:
Bringing It All Together
The architecture enables powerful workflows like:
This architectural foundation is what makes Director so powerful. When you create a new agent, it becomes part of this ecosystem, leveraging all these capabilities to perform its tasks efficiently.
With the fundamentals covered, let’s start building! 🚀
✍️ Planning Phase
The success of an agent heavily depends on thorough planning and requirements gathering. Before writing any code:
MUST-HAVE (v1)
- Core functionality requirements
- Essential error handling
- Basic user feedback
SHOULD-HAVE (v2)
- Enhanced features
- Performance optimizations
- Additional provider support
NICE-TO-HAVE (v3)
- Advanced customization
- Extra integration points
- Optional enhancements
TECHNICAL_LIMITS = {
"max_input_length": "Clear limits",
"rate_limits": "API constraints",
"storage_requirements": "Resource needs",
"performance_expectations": "Response times"
}
Investing time in this planning phase:
Remember: It's easier to adjust plans than refactor code. Take time to ask questions and challenge assumptions before implementation begins.
☑️ Pre-Development Checklist
1. Purpose
2. Background Check
3. Agent architecture
The I/O Contract defines how your agent interacts with the system, including the expected inputs, outputs, and how they are structured. It ensures consistency and clarity in communication between the agent, the system, and the end user. This contract is critical for integrating your agent with the infrastructure and enabling seamless interaction with other components.
Input Contract
The input contract specifies the parameters your agent expects to receive. These parameters can be simple (e.g., a string or number) or complex (e.g., a nested JSON object). The input contract is defined in two parts:
Simple Input Example
For agents with straightforward inputs, you can define the parameters directly in the run method. Here’s an example for a Slack agent:
Complex Input Example
For agents requiring structured inputs (e.g., video generation), you define a JSON schema. This schema is used to validate the input and provide clear documentation for API consumers. Here’s an example for a video generation agent:
In this example:
Output Contract
The output contract defines how your agent communicates results, errors, and progress updates. This includes:
AgentResponse
The AgentResponse object is used to return the result of the agent's execution. It includes:
Example:
Progress Updates
Use the output_message object to provide real-time updates during the agent's execution. For example:
Frontend Content Handling
Agents can return different types of content (e.g., text, video, images) using the output_message.content list. Each content type is represented by a specific model (e.g., TextContent, VideoContent). Example:
Plan and fix the steps to go from input → output. Keep the following factors in mind:
Look for composition opportunities with existing agents or tools
4. Session Management
class NewAgent(BaseAgent):
def __init__(self, session: Session, **kwargs):
self.agent_name = "unique_name"
self.description = "Clear, specific description"
self.parameters = self.get_parameters()
super().__init__(session=session, **kwargs)
Note: Parameter handling self.parameters changes based on the complexity of the parameters required for the agent:
#in case of simple docstring
self.parameters = self.get_parameters()
#in case of dictionary of complex parameters:
self.parameters = AGENT_PARAMETERS
Key Points of User Communication:
Think of this like a progress bar or status updates you see when installing software. It tells users what's happening behind the scenes.
This handles the actual content (videos, images, text) and its current state. Think of this like when you upload a file to Google Drive - you see both the file and its upload status.
This shows up in the chat as:
When the agent finishes, it returns three things:
5. Error Handling
Clear error responses help the reasoning engine interpret failures, guide users effectively, and determine the next appropriate actions in the workflow.
try:
# Main logic
self.output_message.actions.append("Current action...")
self.output_message.push_update()
except Exception as e:
logger.exception(f"Error in {self.agent_name}")
content.status = MsgStatus.error
content.status_message = "User-friendly error message"
self.output_message.publish()
return AgentResponse(status=AgentStatus.ERROR, message=str(e))
Want to print your doc?
This is not the way.
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.