remotion-render (Remotion)

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 17.0 / 30，结论 视情况而定。

下面是评测原文，英文。

Verdict: Situational (17.0 / 30). Source: remotion-dev/skills, skills/remotion-render, pinned at commit 30d4553. Install: npx skills@latest add remotion-dev/skills --skill remotion-render. Licence: MIT (Remotion skills repo).

What it is

The remotion-render skill (remotion-dev/skills): short guidance (about 30 lines plus a transparent-videos note) for rendering Remotion compositions with npx remotion render and still, and for transparent video: --image-format=png with an alpha pixel format, ProRes 4444 (.mov) for editors, VP9 yuva420p (.webm) for browsers.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

A Remotion project with composition id 'Promo'. Tasks (shell file only, nothing is run): a transparent video for Premiere/Final Cut, a transparent video for a web page, and a grab script with three frames in one render call plus a still. Prompts do not state the flags. The one-liner states the correct flags.

1. Transparent video for an editor — This is a Remotion project (composition id 'Promo'). Write render.sh with the single command that exports Promo as a transparent video I can import into Premiere or Final Cut. Do not run anything.
2. Transparent video for the browser — This is a Remotion project (composition id 'Promo'). Write render.sh with the single command that exports Promo as a transparent video I can play on a web page. Do not run anything.
3. Several frames and a still — This is a Remotion project (composition id 'Promo'). Write grab.sh with two commands: one that saves frames 0, 30 and 90 of Promo as PNG files into out/frames in a single render call, and one that saves frame 60 as out/hero.png. Do not run anything.

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "Remotion CLI: npx remotion render <comp> <out> renders a video, npx remotion still one frame, render --frames=0,30,90 --image-format=png several frames in one call; transparent video needs --image-format=png plus an alpha pixel format: --codec=prores --pixel-format=yuva444p10le --prores-profile=4444 (.mov) for editors, --codec=vp9 --pixel-format=yuva420p (.webm) for browsers."

Mean score: 4.48 without vs 4.85 with the skill (delta +0.37). Tokens per run: 32k vs 62k (1.96x); time 21s vs 29s. One-liner group: 5.0 (the skill's gain over the one-liner: -0.15; one-liner's own gain over no skill: +0.52); one-liner tokens 32k per run, so the skill costs 1.95x the one-liner.

What happened

• Control 4.48, skill 4.85, one-liner 5.0 (n=6 each). Codex already knows the right codec and pixel format for both transparent targets (ProRes 4444 with yuva444p10le, VP9 with yuva420p) and the multi-frame and still commands (5.0 in all 6 frames-task runs of every arm).
• The only gap is the PNG image-format flag, which transparent output needs: control left it out in 3 of 4 transparent runs. The skill added it in 3 of 4 (one ProRes run still omitted it), and the one-liner in 4 of 4.
• Tokens: skill 62k, one-liner and control 32k per run (about 1.9x control). The skill file was opened in only 1 of 6 treatment runs, so the extra cost comes mostly from the skill being installed (its description and instructions added to the context), for a one-flag gain.
• Treatment runs copy skill files into the project (5 new files per run).

Dimension scores

Limits of this test

• Nothing was rendered: the checks parse the flags in the shell file. The commands were not run against a real Remotion project (offline; no Chromium or FFmpeg).
• Three small tasks with a single correct command each, so the score is close to a ceiling for all arms; n=2 per cell.
• The PNG-format check is the only one that separates the arms.
• n = 2 per group per task: differences under about 1 point are noise.
• Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

A small, correct note that fixes one commonly forgotten flag, but Codex already writes nearly right commands without help and the skill roughly doubles the tokens. Worth installing only if you render transparent video often and want the flag handled every time.