walid@portfolio:~/lab/openmontage-setup$
cd../lab
01ideaSep 2026

The agent that makes the video

Setting up an open-source video production pipeline your coding agent directs end to end

You describe a video in one sentence and the agent runs research, script, scene plan, assets, edit and render — pausing for your approval at three gates and telling you the cost before it spends anything. There is a genuine zero-key path: free narration, free archival footage, free composition. One correction to the guide this came from: the project is AGPL-3.0, not MIT, which matters a great deal if you plan to build a business on top of it.

VideoClaude CodeRemotionFFmpegOpen sourcePipelinesOpenMontage ↗Remotion ↗Piper TTS ↗FFmpeg ↗Pexels API ↗Archive.org ↗
i
Read this first

Claude Code is the orchestrator — there is no autonomous “press go” mode. The agent pauses for your approval at the script, scene plan and asset stages, and tells you costs before spending. Nothing is uploaded or published anywhere: “publish” means a local export folder. The guide this is built from was verified by cloning the repo at commit 08e2151 (5 Sep 2026), running the setup, and reading the agent guide and pipeline files.

Section 1

What it actually is

An open-source video production system where your AI coding assistant is the director. You describe a video; the agent reads a pipeline manifest (YAML) and a set of stage “director skills” (Markdown), then calls the project’s Python tools to research the subject, write the script, plan the scenes, generate or fetch visuals, narrate, add music and captions, and render through Remotion or HyperFrames plus FFmpeg.

Eleven pipelines existed at the verified commit — animated explainer, documentary montage, cinematic, talking head, clip factory and more — and the current repository lists twelve. Every creative decision comes back to you for approval.

!
Correction — the licence is AGPL-3.0, not MIT

The guide states MIT twice. It is not: the repository ships the GNU Affero General Public License v3, both at the commit the guide was verified against and on the current default branch, and the project’s own README carries an AGPLv3 badge. The difference is not cosmetic — AGPL’s network clause means that if you run a modified version as a service for other people, you owe them your source. Fine for personal use; read it properly before building a product on it.

The shape of a run

One sentence in, one file out

The three gates are the safety story. Nothing paid happens, and no gated stage is marked complete, without your approval.
Section 1b

What works today, and what is still optional

Working now (verified)Optional / experimental
Zero-key path: Piper TTS narration, free archival and stock footage, Remotion + HyperFrames composition, FFmpeg, auto captions.Paid image, video, voice and music providers — fal, Kling, ElevenLabs, OpenAI, Google, Runway, HeyGen and others.
Instruction files for Claude Code, Cursor, Copilot, Codex and Windsurf.GPU-local video generation via a separate install target; requires an NVIDIA card.
Budget controls, approval gates and a local export bundle.A live storyboard board, reference-video analysis, and local LLMs via Ollama or LM Studio listed as coming soon.
Section 2

What you need

ItemWhyRequired
Python 3.10+Runs the tools and pipelines — a virtualenv is created for you.Yes
FFmpegEncoding, subtitles, audio mixing.Yes
Node.js 18+Remotion and HyperFrames rendering. The first render downloads a headless Chromium.Yes
Claude CodeThe agent that runs everything. Uses your existing plan — no separate model key needed. Cursor, Copilot, Codex and Windsurf also work.Yes
GitTo clone the repository.Yes
Free stock keysPEXELS_API_KEY, PIXABAY_API_KEY, UNSPLASH_ACCESS_KEY — more free footage and images for real-footage videos. The developer keys are free.Optional, free
Paid provider keysFAL_KEY, KLING_API_KEY, ELEVENLABS_API_KEY, OPENAI_API_KEY, GOOGLE_API_KEY, SUNO_API_KEY, RUNWAY_API_KEY, HEYGEN_API_KEY — AI-generated imagery and video, premium voices, custom music.Optional, costs money
NVIDIA GPUFree local video generation (Wan, Hunyuan, LTX and similar).Optional
Section 3

Install it

The setup target creates the virtualenv, installs the Python requirements, installs the Remotion composer via npm, installs free Piper TTS, warms the HyperFrames cache, and copies the example environment file.

terminal3 lines
git clone https://github.com/calesthio/OpenMontage.git
cd OpenMontage
make setup

No make? The manual equivalent

On Windows PowerShell the README gives the equivalent. If npm install fails with ERR_INVALID_ARG_TYPE, use npx --yes npm install instead.

macOS / Linux4 lines
python3 -m venv .venv && source .venv/bin/activate && \
python -m pip install -r requirements.txt && \
cd remotion-composer && npm install && cd .. && \
python -m pip install piper-tts && cp .env.example .env
Section 4

Connecting it to the agent

Nothing to install into Claude Code — the repository already contains its configuration. Open the folder and start the agent.

File or folderWhat it does
CLAUDE.mdAuto-read on start. Its whole job is to say: read AGENT_GUIDE.md before doing anything.
AGENT_GUIDE.md + PROJECT_CONTEXT.mdThe operating contract — pipelines, approval gates, costs, and where files go.
.claude/skills/ (50+)Skills loaded automatically: create-video, remotion, ffmpeg, text-to-speech, elevenlabs, heygen, video-edit, music, visual-style and more.
.claude/commands/Slash commands — a live storyboard board, ink-art, animated-drawing.
skills/pipelines/ + pipeline_defs/The stage-director instructions and the YAML manifests the agent follows.
tools/The Python tools the agent calls: image, video, audio, subtitles, compose.

To see the real capability envelope at any time — what your particular keys unlock — ask the agent “what can you make with my current setup?”. The onboarding skill answers that directly rather than guessing.

Section 5

Your first video, stage by stage

The fastest path is the zero-key animated explainer.

StageWhat happensYour part
IdeaYou type one sentence.Type it
ScriptWeb research → proposal → script with voice direction.Approve or revise — gated
StoryboardScene plan: shots, timing, on-screen text, visual style.Approve or revise — gated
VisualsFree archival and stock footage, Remotion scenes, or AI images and video if you added keys.Review the filmstrip — gated
VoicePiper TTS, free and offline, or a cloud voice if configured.Approve
EditEdit decisions, music plan from a free royalty-free library, captions.Automatic in most pipelines
RenderRemotion or HyperFrames → FFmpeg → a self-review pass using ffprobe, frames and audio.Automatic, then you watch it
Finishedprojects/<project-name>/renders/final.mp4Done

The zero-key first run

Straight from the project README — works with no API keys at all. Prefer real footage? Say so: “documentary montage, use real footage only, no narration.” The agent then pulls from Archive.org, NASA, Wikimedia and any free stock keys you added.

type-this.txt1 lines
Make a 45-second animated explainer about why the sky is blue
Section 6

The copy-paste prompt

The long-form version, with the budget and no-publish rules stated explicitly.

video-prompt.txt22 lines
Use OpenMontage to create a complete short-form video about:
[VIDEO IDEA]

Goal: a polished 30-60 second vertical (9:16) video.

Follow AGENT_GUIDE.md: pick the right pipeline, initialize the project
workspace, then run research → proposal → script → scene plan. Stop at
each approval gate and wait for me.

After I approve the script and scene plan, list the visual assets needed
and which tools you will use (free first: archival/stock footage, Remotion
scenes, Piper narration).

Create narration with the configured TTS. Add word-level captions.
Assemble, edit, and render the final video.

BEFORE using any paid API or provider, tell me exactly what will cost
money and how much, and ask for my approval. Stay under the budget cap
in config.yaml.

Do not publish or upload anything. Save the finished video locally and
tell me the exact path (projects/<name>/renders/final.mp4).
Section 7

Prompts worth trying

Nothing here auto-publishes; every one of them ends at a local file.

PromptWhat it uses
“Create a 30-second faceless YouTube Short about the craziest new AI technology.”Animated explainer pipeline: web research, script, Remotion scenes, Piper narration, captions. Add an image or video key for AI visuals. Say 9:16 for vertical output.
“Create a 60-second documentary-style video explaining how AI agents work.”Animated explainer with narration, or documentary montage if you say “use real footage only” — free archival and stock footage, music, optional narration.
“Turn this script into a polished vertical Reel with narration, visuals, captions, and fast pacing.” + paste the scriptSkips research entirely: your script → scene plan → assets → compose. Talking head or hybrid pipeline if you also supply your own footage.
Section 8

Where the video ends up

Project names are kebab-case from the title, e.g. why-the-sky-is-blue. The projects folder is git-ignored. A demo target renders zero-key demos, and the optional storyboard board shows the same folder live in your browser.

projects/<project-name>/10 lines
projects/<project-name>/
  artifacts/              research_brief, script, scene_plan… (JSON)
  assets/images/          generated images (PNG)
  assets/video/           generated or fetched clips (MP4)
  assets/audio/           narration segments + final mix (MP3/WAV)
  assets/music/           background track (MP3)
  assets/subtitles.srt
  renders/final.mp4       ← the finished video
  checkpoint_<stage>.json resumable state
  history/                keeps old runs
Section 9

What it costs

Provider prices come from the project README and change — check your own provider. The figures below were re-checked against the README rather than copied.

LayerCostNotes
The agent itselfYour Claude plan or API usageIt does the research, writing and orchestration. This is the part that is not free.
Zero-key path$0 in provider feesPiper TTS, Archive.org / NASA / Wikimedia footage, Remotion, HyperFrames, FFmpeg, captions.
Free stock keys$0Pexels, Pixabay and Unsplash developer keys are free to obtain.
AI image / video providersPay per generationThe README estimates roughly $0.15–$1.50 per video with a provider configured, and about $1–$3 for a full setup.
Premium voice / musicPay per useElevenLabs, OpenAI TTS, Google TTS, Suno — all optional.
Local GPU generation$0 in fees, your hardwareSeparate install target; an NVIDIA GPU is required.
!
The money guardrails are real, and they are configurable

config.yaml sets a $10 total cap, asks for approval on any single action over $0.50, and requires approval before any new paid tool. The default mode only warns on overrun — change it to a hard cap if you want the run to actually stop. Keys live in .env, which is git-ignored. All of that is genuinely built in, but the system stops being free the moment you add generation APIs.

Section 10

Troubleshooting

ProblemFix
“Python 3.10+ is required”Install Python 3.10 or newer (or uv) and re-run the setup — the Makefile picks it up.
npm install fails with ERR_INVALID_ARG_TYPERun npx --yes npm install inside remotion-composer, then continue.
First render fails downloading ChromiumRemotion fetches a headless Chromium on the first render. Allow the network access and retry — this is the one that bites in locked-down sandboxes.
The agent starts building without askingTell it to read AGENT_GUIDE.md first. The gates are binding per the manifest, and the checkpoint system will not mark a gated stage complete without your approval.
“[skip] piper-tts install failed” / no narrationRe-run the piper-tts install inside the virtualenv, or add a cloud TTS key instead.
Start here

Quick-start checklist

✓Clone the repository and change into it.✓Run the setup target — Python 3.10+, FFmpeg, Node 18+.✓Open the folder with your coding agent. Nothing extra to install.✓Configure keys: none required for the free path; optional ones go in .env.✓Pick a video idea.✓Generate the script and storyboard, and approve both gates.✓Create the assets — review the filmstrip, and approve paid steps only if you want them.✓Let it assemble and compose.✓Render. The self-review pass runs automatically.✓Collect the file at projects/<name>/renders/final.mp4.
i
Provenance

Verified against the project at commit 08e2151 (5 Sep 2026): cloned, set up, and read against the README, agent guide, config and pipeline definitions. The demo render was the only step blocked, and only by a sandbox without network access. Everything above was re-checked against the current repository before publishing — the cost figures, budget thresholds and zero-key tool list all hold; the licence did not.

next →
Nine agents, one voice
← all experiments