lidge-jun/video

Remotion Video Skill

JSON-driven timeline rendering with Remotion. Create, render, preview, and review videos. Triggers: "video", "Remotion", "animation", "mp4", "render video", "slides to video". Covers: JSON-driven timeline, theme system, resolution presets, TTS narration, render pipeline, FFmpeg processing, final polish.

---

Quick Reference

| Task | Command | | -------------- | ------------------------------------------------------------------------------------ | | Render | node scripts/pipeline.mjs --timeline <path> [--preset Landscape-1080p] | | Render+TTS | node scripts/pipeline.mjs --timeline timeline.draft.json (auto-detects narration) | | Skip TTS | node scripts/pipeline.mjs --timeline timeline.draft.json --skip-tts | | TTS only | node scripts/tts.mjs --batch timeline.draft.json [--provider supertone] | | TTS single | node scripts/tts.mjs --text "Hello" --output /tmp/tts-out.m4a [--provider gemini] | | TTS voices | node scripts/tts.mjs --list-voices [--provider supertone] | | Async | node scripts/pipeline.mjs --timeline <path> --async | | Status | node scripts/pipeline.mjs --status /tmp/remotion-render/render-result.json | | Preview | cd remotion-project && pnpm exec remotion studio | | Validate | node scripts/validate-artifact.mjs /tmp/remotion-render/TimelineVideo.mp4 --preset Landscape-1080p |

---

Pipeline Usage

Output goes to /tmp/remotion-render by default. Keep render output outside skills_ref/video/ — that directory is source code, not workspace.

# Sync (default) — blocks until render complete
node skills_ref/video/scripts/pipeline.mjs \
  --timeline timeline.json \
  --output /tmp/remotion-render

# With preset override (timeline.meta.preset is the source of truth)
node skills_ref/video/scripts/pipeline.mjs \
  --timeline timeline.json \
  --preset Portrait-1080p

# Async — returns immediately, renders in background
node skills_ref/video/scripts/pipeline.mjs \
  --timeline timeline.json \
  --async

# Check async status
node skills_ref/video/scripts/pipeline.mjs \
  --status /tmp/remotion-render/render-result.json

timeline.meta.preset is the source of truth for resolution. CLI --preset overrides it with a warning.

---

Timeline Authoring

Refer to the project's timeline schema TypeScript interface for the full type definition.

Minimal Example

{
  "meta": {
    "title": "My Video",
    "preset": "Landscape-1080p",
    "fps": 30,
    "totalDurationSec": 15
  },
  "elements": [
    {
      "type": "title",
      "startSec": 0,
      "durationSec": 5,
      "props": { "title": "Hello World", "subtitle": "A demo" },
      "transition": { "type": "fade" }
    },
    {
      "type": "content",
      "startSec": 5,
      "durationSec": 5,
      "props": { "header": "Key Points", "bulletPoints": ["Fast", "Safe", "Beautiful"] },
      "transition": { "type": "slide", "direction": "from-right" }
    },
    {
      "type": "code",
      "startSec": 10,
      "durationSec": 5,
      "props": { "code": "const x = 42;", "language": "typescript", "title": "Example" },
      "transition": { "type": "fade" }
    }
  ],
  "audio": []
}

Theme System (meta.theme)

Each video should define a unique aesthetic. Uses @remotion/google-fonts for cross-platform rendering. Default: Chakra Petch (display) + Outfit (body) + JetBrains Mono (code), dark blue + cyan accent.

{
  "meta": {
    "theme": {
      "aesthetic": "brutalist tech",
      "font": { "display": "Chakra Petch", "body": "Outfit" },
      "color": { "accent": "#FF6B35", "bg": "#0A0A0A" },
      "gradient": { "hero": "radial-gradient(circle at 20% 30%, rgba(255,107,53,0.2) 0%, transparent 60%)" }
    }
  }
}

Content Design Rules

Do:

• Use concise headers without emoji
• Write bullet points as short phrases (max 8 words), 3-4 per slide
• Vary slide types: title → content → code → content
• Mix transitions: fade/slide/wipe (avoid repeating same type 3+ times)
• Show real code on code slides
• Pick a theme aesthetic and commit

Avoid:

• Emoji in slide titles or headers
• More than 5 bullets on a single slide
• Generic headers like "Key Features", "Getting Started", "Summary"
• Inter, Roboto, Arial as fonts in meta.theme (because they signal generic/AI-generated content)

Content Density — Fill the Frame

The canvas is large (1920×1080 or 1080×1920). Sparse content creates dead space. Add more content blocks — not bigger fonts or more padding.

• Content slides: Use content + bulletPoints together; 4-5 bullets portrait, 3-4 landscape
• Code slides: Minimum 6 lines portrait, 4 landscape; add comments to fill space
• Title slides: Always include a subtitle
• General: Each slide should use ≥70% of canvas area; prefer fewer dense slides over many sparse ones

Visual Quality

Detailed anti-slop rules, per-slide-type layout guidelines, motion principles, and typography specs → reference/visual-quality.md

Shorts (Portrait-1080p)

• Max 8-10 elements for 60s
• First slide = hook (max 5 words) + descriptive subtitle
• Last slide = CTA or memorable closing + tagline
• 5-6 bullets per content slide, code 6-10 lines
• Use content + bulletPoints together on every content slide

---

TTS Integration

Three providers available. Default: Gemini.

| Provider | ID | Default Voice | Strengths | Env Key | | ---------------- | ------------ | ------------- | -------------------------------------- | ------------------- | | Gemini | gemini | Kore | 30 voices, tone via prompt | GEMINI_API_KEY | | Supertone | supertone | Andrew | 6 emotion styles, Korean-best | SUPERTONE_API_KEY | | Supertonic | supertonic | M1 | 0.22s gen, free, offline | none |

Workflow: Draft → TTS → Final → Render

1. Write timeline.draft.json with narration + optional voiceControl per element
2. Pipeline auto-detects narration → generates per-cut audio
3. Produces timeline.final.json with audio[] entries + corrected durationSec
4. Final timeline rendered with synced audio

Draft Timeline with Narration

{
  "elements": [
    {
      "id": "intro",
      "type": "title",
      "durationSec": 11,
      "narration": "Welcome to the analysis report.",
      "voiceControl": { "tonePrompt": "Calm, professional news anchor tone" },
      "props": { "title": "Tech Trends", "subtitle": "2026 Edition" }
    }
  ]
}

Duration Estimation

• Korean: ~6.5 chars/sec → Math.ceil(narration.length / 6.5) + 0.5
• Final durationSec auto-corrected by ffprobe measurement

Full TTS details (auth paths, priority chain, voiceControl fields, caching, audio path contract) → reference/tts-integration.md

---

Component Library

Slides

| Component | Props | Use For | | ----------------- | ------------------------------------------------------- | -------------------- | | TitleSlide | title, subtitle, animation? | Opening/closing | | ContentSlide | header, content, bulletPoints, animation? | Body content | | CodeSlide | code, language, title, animation? | Code demos | | DiagramSlide | src, title, caption, fit, animation? | Images/diagrams | | StatSlide | title, stats[] (value/suffix/label/trend/decimals) | KPI / count-up | | QuoteSlide | quote, author?, source? | Quotes | | ComparisonSlide | title, left{label,items,accent}, right{…} | Side-by-side compare | | VideoSlide | src, title?, startFrom?, playbackRate?, loop? | Inline video | | GifSlide | src, title?, fit? | Animated GIF | | LottieSlide | src, title? | Lottie animation | | ChartSlide | chartType, title, data{labels,datasets} | Bar/pie/line chart | | Caption | text, position, designTheme? | Timed subtitles |

Features

• Surface Card: Glassmorphism wrapper on content slides. Customize via meta.theme.card
• Animation: Optional animation: { enter: "scale-in", exit: "fade-out" } per element. Enter: scale-in, fade-in, slide-up, none. Exit: scale-out, slide-down, fade-out, none
• Transitions: fade, slide, wipe, flip, clock-wipe. Optional timing: "spring". Slide/wipe accept direction
• Korean fonts: NotoSansKR loaded as body primary. Stack: NotoSansKR, Outfit, sans-serif
• Charts: bar (staggered grow), pie (sweep), line (draw-on). Pure SVG, no external library
• Audio: fadeInSec/fadeOutSec, loop, trimStartSec

Full component details → reference/components.md

---

Resolution Presets

| Preset | Width | Height | Aspect | Use For | | ----------------- | ----- | ------ | ------ | ------------------- | | Landscape-720p | 1280 | 720 | 16:9 | Draft/preview | | Landscape-1080p | 1920 | 1080 | 16:9 | Standard delivery | | Portrait-1080p | 1080 | 1920 | 9:16 | TikTok/Reels/Shorts | | Square-1080p | 1080 | 1080 | 1:1 | Instagram/LinkedIn |

Default: Landscape-1080p. Agent picks based on user keywords (reels/shorts → Portrait).

---

FFmpeg Patterns

FFmpeg handles deterministic cuts, batch processing, and preprocessing outside of Remotion.

Extract segment by timestamp

ffmpeg -i raw.mp4 -ss 00:12:30 -to 00:15:45 -c copy segment_01.mp4

Batch cut from edit decision list

#!/bin/bash
# cuts.txt format: start,end,label
while IFS=, read -r start end label; do
  ffmpeg -i raw.mp4 -ss "$start" -to "$end" -c copy "segments/${label}.mp4"
done < cuts.txt

Concatenate segments

for f in segments/*.mp4; do echo "file '$f'"; done > concat.txt
ffmpeg -f concat -safe 0 -i concat.txt -c copy assembled.mp4

Create proxy for faster editing

ffmpeg -i raw.mp4 -vf "scale=960:-2" -c:v libx264 -preset ultrafast -crf 28 proxy.mp4

Extract audio for transcription

ffmpeg -i raw.mp4 -vn -acodec pcm_s16le -ar 16000 audio.wav

Normalize audio levels

ffmpeg -i segment.mp4 -af loudnorm=I=-16:TP=-1.5:LRA=11 -c:v copy normalized.mp4

Scene detection

# Detect scene changes (threshold 0.3 = moderate sensitivity)
ffmpeg -i input.mp4 -vf "select='gt(scene,0.3)',showinfo" -vsync vfr -f null - 2>&1 | grep showinfo

Silence detection (auto-cut dead air)

ffmpeg -i input.mp4 -af silencedetect=noise=-30dB:d=2 -f null - 2>&1 | grep silence

Social media reframing

# 16:9 → 9:16 (center crop for TikTok/Reels)
ffmpeg -i input.mp4 -vf "crop=ih*9/16:ih,scale=1080:1920" vertical.mp4

# 16:9 → 1:1 (center crop for Instagram)
ffmpeg -i input.mp4 -vf "crop=ih:ih,scale=1080:1080" square.mp4

---

ElevenLabs Voice (Supplementary)

For professional voiceover outside the Remotion TTS pipeline, ElevenLabs is available via direct API:

import os, requests

resp = requests.post(
    f"https://api.elevenlabs.io/v1/text-to-speech/<voice_id>",
    headers={
        "xi-api-key": os.environ["ELEVENLABS_API_KEY"],
        "Content-Type": "application/json"
    },
    json={
        "text": "Your narration text here",
        "model_id": "eleven_turbo_v2_5",
        "voice_settings": {"stability": 0.5, "similarity_boost": 0.75}
    }
)
with open("voiceover.mp3", "wb") as f:
    f.write(resp.content)

Use when: high-fidelity English narration, voice cloning, or emotion control beyond what Gemini/Supertone provide. Requires ELEVENLABS_API_KEY.

---

Final Polish (Descript / CapCut)

For tasks that code-driven rendering handles poorly — use a traditional editor as the last mile:

• Pacing: adjust cuts that feel too fast or slow
• Captions: auto-generated, then manually cleaned
• Color grading: basic correction and mood
• Final audio mix: balance voice, music, and SFX levels
• Export: platform-specific formats and quality settings

AI clears repetitive work. Final creative taste lives in this layer.

---

Render Validation — 3-Tier Gate

All 3 gates pass for a render to be considered successful.

| Gate | Checks | Role | | :----------- | :-------------------------------------- | :-------------- | | 1. Policy | No forbidden engine in logs | Supplementary | | 2. Execution | remotion render exit 0 | Primary | | 3. Artifact | ffprobe duration/codec/resolution valid | Final truth |

node scripts/validate-artifact.mjs /tmp/remotion-render/TimelineVideo.mp4 --preset Landscape-1080p

---

Dependencies

• Runtime: Node.js 20+, pnpm, ffmpeg, ffprobe
• Packages: Installed at ~/.remotion/node_modules/. Run setup-remotion.sh once
• Chromium: Auto-installed by remotion browser ensure (via ensure-remotion.mjs)
• TTS env: GEMINI_API_KEY for Gemini; SUPERTONE_API_KEY for Supertone; none for Supertonic
• Bootstrap: node scripts/ensure-remotion.mjs before first render
• Vertex AI auth: gcloud auth application-default login or set GOOGLE_APPLICATION_CREDENTIALS

---

Project Structure

skills_ref/video/
├── SKILL.md
├── reference/                    ← visual-quality.md, tts-integration.md, components.md
├── scripts/
│   ├── pipeline.mjs              ← CLI entrypoint (sync/async/TTS)
│   ├── tts.mjs                   ← TTS orchestrator (multi-provider)
│   ├── tts-providers/            ← gemini.mjs, supertone.mjs, supertonic.mjs
│   ├── ensure-remotion.mjs       ← runtime bootstrap
│   ├── validate-artifact.mjs     ← Gate 3: ffprobe validation
│   └── presets.mjs               ← resolution presets (ESM)
└── remotion-project/
    ├── public/example-timeline.json
    └── src/
        ├── components/           ← 11 slide components + barrel
        └── timeline/             ← JSON→React engine