Remotion voiceover: AI text to speech and word-timed captions for Remotion videos

remotion-voiceover is an open-source Node.js command-line tool that turns a narration script into an AI voiceover for Remotion and HyperFrames videos. One command writes one WAV per line, a frame-timed voiceover.json, word-level @remotion/captions JSON, SRT and VTT subtitles, and a HyperFrames audio_request.json, with voice cloning and several voices per video. Code on GitHub (MIT).

Remotion voiceover demo: a script rendered to a male and a female cloned AI voice with word-by-word TikTok-style captions in a Remotion video

Watch: Remotion video with a male and a female cloned voice

remotion-voiceover-captions-demo.mp4 (rendered by the example Remotion project, 11.78 s = 353 frames at 30 fps)

captions.json (@remotion/captions) · voiceover.json · SRT · VTT

Transcript
hook, Ben (cloned male voice, frames 0–171): Your Remotion video finally has a real voice, and every caption is timed to the word.
cta, Ava (cloned female voice, frames 189–338): Write the script once, run one command, and render with perfectly synced voiceover.

Quick start

export MIRAGEA_API_KEY=...        # https://miragea.ai/workspace/api
npx github:RTY2345RR/remotion-voiceover generate voiceover.md --hyperframes
npx remotion render src/index.ts Voiceover out/video.mp4

Using an AI agent? Install https://miragea.ai/skills/miragea-audio/install.md

  1. Set the voice-engine key: Create a key at https://miragea.ai/workspace/api and export MIRAGEA_API_KEY or put it in .env.
  2. Write voiceover.md: Scenes as # headings, one spoken line per line, voices in front matter (voice id or clip URL to clone).
  3. Generate: Run npx github:RTY2345RR/remotion-voiceover generate voiceover.md to write public/voiceover/.
  4. Render: calculateMetadata reads voiceover.json; render with npx remotion render.

How it works

How remotion-voiceover works: script.md, plan and cache, Miragea TTS or voice cloning, measure WAV, lay out frames, write voiceover.json, captions.json, SRT, VTT, audio_request.json

Remotion TTS features

Remotion voiceover comparison

remotion-voiceovertemplate-tiktokremotion-superpowersHyperFrames media-use
Generates voiceoverYesNo (captions only)Yes (ElevenLabs)Yes (HeyGen or Kokoro)
Voice cloningYesNoNot documentedPaid HeyGen plan
Frame timings JSONYesNoNot documentedNo
@remotion/captions JSONYesYes (Whisper.cpp)Yes (Whisper)No
LicenceMITRemotion licenceMITApache-2.0

Other projects: facts from their public READMEs, November 2026. "Not documented" means we could not find it.

FAQ

How do I add a voiceover in Remotion?

Generate the audio into public/, then play it with an Audio element inside a Sequence. remotion-voiceover generates the audio and writes each line's start frame in voiceover.json.

Does Remotion have text to speech built in?

No. Remotion renders video from React and audio comes from files. remotion-voiceover is the Remotion TTS step that makes those files plus timings and captions.

How do I add captions or subtitles in Remotion?

Use @remotion/captions: pass the Caption[] JSON this tool writes to createTikTokStyleCaptions(). SRT and VTT are written too.

Is there a Remotion ElevenLabs alternative?

Yes. remotion-voiceover uses the Miragea voice engine for text to speech and voice cloning and writes the files a Remotion project needs.

Can I clone my own voice for Remotion videos?

Yes. Put a public WAV/MP3 clip URL and its exact words under voices: in the front matter and use [voice: name].

Can one video have several voices?

Yes. Define several voices and switch with [voice: ...] on a scene heading or a line.

Does it work with HyperFrames?

Yes. Add --hyperframes to write audio_request.json with audio paths and word timings.

What happens when I edit one line?

Only that line is voiced again; the rest are reused and timings are recomputed.

Related open-source voice tools

Voices by the Miragea voice engine (https://miragea.ai/v1). Demo reference voices: CMU ARCTIC (bdl, slt), © Carnegie Mellon University.