Skip to main content

Record & Replay on GitHub

Complete source code, installation guide, and troubleshooting

​
What Is It?

Record & Replay turns a workflow you perform on your desktop into instructions an AI agent can reuse. It supports macOS and Windows. You demonstrate the task once while the app records the desktop interactions. It then combines the action log with optional visual context from VideoDB and compiles the result into a structured, human-readable agent skill. The result is not a fixed macro. The generated skill describes the task, starting state, inputs, actions, and checks an agent needs to repeat it. Values such as search terms, dates, choices, and file paths can become inputs instead of being copied as fixed values.
The Idea: Show the workflow once, let Record & Replay turn the demonstration into a reusable skill, then give that skill to your agent for future runs.

​
Why Use Record & Replay?

​
Demonstrate the Real Workflow

Perform the task in the app you already use. Record & Replay captures desktop accessibility events and can include screen video as visual reference.

​
How It Works

1

Start a Recording

Ask your agent to start recording a named workflow. A short lead-in can give you time to switch from the agent to the app where the task will happen.
2

Demonstrate the Task

Complete the workflow yourself. Native accessibility hooks capture the actions and interface metadata. When screen capture is available, VideoDB adds visual context.
3

Compile the Skill

The compiler combines the event log and available visual context, identifies reusable inputs, and generates structured and human-readable skill files.
4

Use It with an Agent

By default, the generated SKILL.md is installed in ~/.codex/skills/. You can point the installer at another global skills directory for agents that consume the same skill format.
Human-in-the-loop recording: You perform the demonstration yourself. Record & Replay creates the skill, but it does not include a standalone replay engine—the agent that consumes the skill performs the workflow later.

​
What the Skill Contains

A readable sequence of actions that explains how to complete the demonstrated workflow.
Values such as search queries, dates, dropdown choices, and file paths can be represented as inputs rather than copied as fixed values.
The generated skill includes checks and guidance that help the consuming agent confirm whether the workflow succeeded.
Each structured skill records a preferred execution strategy, such as browser, desktop app, terminal, file system, or a hybrid workflow.

​
Getting Started

Prerequisites
1

Clone and Install

2

Add Your API Key

Create a .env file in the project root:
3

Configure Your MCP Client

Add the server to your MCP client configuration:
4

Restart the Client

Restart your MCP client so the Record & Replay tools and skill resources become available.

​
Record Your First Workflow

Recording is guided by your agent:
  1. Ask the agent to record a named workflow.
  2. Switch to the target app and perform the task.
  3. Return to the MCP client and ask the agent to stop recording.
  4. Ask it to compile the recording into a skill.
  5. Confirm the global SKILL.md path reported by the agent.
Use a short lead-in before the workflow begins and trim the end of the recording if you need time to switch back to your MCP client. This keeps navigation to and from the client out of the generated instructions.

​
Platform Notes

Screen Recording, Microphone, Accessibility, and Input Monitoring permissions may be required. Run the permission helper before recording:
Record & Replay uses Windows UI Automation. No additional setup is required beyond the standard installation.

Complete Setup Guide on GitHub

Tool reference, skill output details, platform setup, and troubleshooting

Pair Programmer

Give coding agents real-time screen and audio context

OpenClaw Monitoring

Record, inspect, and search autonomous agent activity

Capture SDK Overview

Learn how VideoDB desktop capture works