transcript-video

Generate synced transcript videos from audio files. Word-by-word text synchronized to audio with semantic phrase splitting, concept highlighting, and audio-reactive instrument panel visualizations.

https://github.com/ejfox/transcript-video

What it does

Give it an audio file. Get back a 1080p video with:

• Synced transcript — words appear timed to speech, split on natural phrase boundaries
• Concept tagging — highlight key terms in color (everything else is monochrome)
• Audio visualizations — spectrogram, waveforms, oscilloscopes at multiple time scales, Lissajous phase scope, EKG history strips, waterfall display
• QR codes — scannable provenance metadata per phrase (speaker, timestamp, context)
• Instrument panel layout — labeled modules with metadata (timebase, gain, frequency range, live dBFS readout)

Designed for video essay production — turn podcast clips, interviews, and lectures into footage you can drop into your NLE.

Install

npm install -g transcript-video

# Or clone and link
git clone https://github.com/ejfox/transcript-video.git
cd transcript-video
npm install
npx playwright install chromium
npm link

Requirements:

• Node.js 18+
• ffmpeg
• OPENAI_API_KEY env var (for Whisper word-level timestamps, ~$0.006/min)

Usage

# Basic — transcribe + render
transcript-video interview.mp3

# With a speaker label
transcript-video podcast.mp3 --speaker "ERIK DAVIS — HARVARD CSWR"

# First 5 minutes only
transcript-video lecture.mp3 --duration 300

# 4K output
transcript-video clip.mp3 --width 3840 --height 2160

# Preview in browser (no render)
transcript-video clip.mp3 --preview

# Use pre-existing word timestamps (skip API call)
transcript-video clip.mp3 --words clip-words.json

# Custom concept tags
transcript-video clip.mp3 --tags my-tags.json

Concept tags

Create a JSON file with word clusters to highlight. Only tagged words get color — everything else stays monochrome.

[
  {
    "label": "consciousness",
    "color": "#6eedf7",
    "words": ["consciousness", "sentient", "aware", "mind", "qualia"]
  },
  {
    "label": "technical",
    "color": "#ffaa00",
    "words": ["model", "training", "alignment", "neural", "parameter"]
  }
]

Pass it with --tags my-tags.json. Word matching is case-insensitive and supports partial stems (e.g., "meditat" matches "meditation", "meditative", "meditating").

How it works

1. Transcription — Sends audio to OpenAI Whisper API with timestamp_granularities[]=word to get per-word timing
2. Phrase splitting — Groups words into natural phrases based on pauses, punctuation, and clause boundaries (not mechanical N-word chunks)
3. Frame rendering — Playwright opens the template in headless Chrome, steps through time frame-by-frame, injects pre-decoded PCM audio data into the visualization engine, and screenshots each frame
4. Encoding — ffmpeg assembles frames + original audio into h264/aac MP4

Render speed is ~20-25fps (a 10-minute clip takes ~12 minutes to render).

Options

Flag	Default	Description
--words <file>	auto	Pre-computed word timestamps JSON
--tags <file>	built-in	Concept tag definitions
--speaker "LABEL"	filename	Speaker label on screen
--duration <sec>	full	Render only first N seconds
--width <px>	1920	Output width
--height <px>	1080	Output height
--fps <num>	30	Frames per second
--output <file>	<input>-transcript.mp4	Output path
--preview	false	Open in browser instead of rendering

Design

• Monochrome — all visualizations are white-on-black. Color is reserved exclusively for concept-tagged words
• Instrument panel — each visualization module is a labeled panel with metadata annotations (timebase, gain, frequency range)
• Fonts — Archivo Black (display), Monaspace Xenon (prose), Monaspace Krypton (labels) via CDN
• 16:9 locked — always renders at correct aspect ratio for YouTube/FCP

License

MIT