remotion-best-practices (remotion-dev/skills)

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 22.9 / 30，结论 视情况而定。

下面是评测原文，英文。

Verdict: Situational (22.9 / 30). Source: remotion-dev/skills, skills/remotion-best-practices, pinned at commit 30d4553. Install: npx skills@latest add remotion-dev/skills --skill remotion-best-practices. Licence: Remotion License (company use above 3 people needs a paid license; evaluation only).

What it is

The remotion-best-practices skill, the router of Remotion's official skill set (remotion-dev/skills, version 4.0.539, Remotion License). SKILL.md (about 600 words) sends the agent to sub-skills: create a video, markup best practices (drive animation with useCurrentFrame() and interpolate(), take fps from useVideoConfig(), no CSS transitions or Tailwind animation classes), maps, captions, studio, render and more. About 12 sub-skills of Markdown, two small local .mjs scripts. It also tells the agent to open the Remotion Studio preview and not to render unless asked.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

Environment: the project template (Remotion project with composition id Main) is the same for all groups. The sandbox has no network and no Remotion tooling, so none of the groups could start Studio or render; they wrote source only. We then rendered every project on a separate offline machine with Remotion 4.0.539. The skill's 'open the preview first' step could not happen.

1. 15 s product teaser — In this Remotion project (React components in src/, composition id 'Main', rendered by npx remotion render src/index.ts Main) build a 15-second, 1920x1080, 30 fps product teaser for Tidepool, a monthly seaweed-snack subscription. Required on-screen text: the…
2. 20 s animated bar-chart explainer — In this Remotion project (React components in src/, composition id 'Main', rendered by npx remotion render src/index.ts Main) build a 20-second, 1920x1080, 30 fps animated explainer: a bar chart of monthly revenue (Aug $42k, Sep $58k, Oct $71k). Title 'Reven…
3. 30 s vertical captioned short — In this Remotion project (React components in src/, composition id 'Main', rendered by npx remotion render src/index.ts Main) build a 30-second vertical 1080x1920, 30 fps social short with burned-in captions. Captions, in order, each on screen for about 5 se…

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "Drive every animation with useCurrentFrame() and interpolate()/spring() (never CSS transition/animation), take fps from useVideoConfig() and convert seconds to frames with it, and structure the video as <Sequence>/<Series> scenes inside the composition."

Mean score: 4.91 without vs 4.91 with the skill (delta -0.01). Tokens per run: 159k vs 217k (1.36x); time 161s vs 177s. One-liner group: 4.89 (the skill's gain over the one-liner: +0.02; one-liner's own gain over no skill: -0.03); one-liner tokens 156k per run, so the skill costs 1.39x the one-liner.

What happened

• Video quality was the same. All 18 projects rendered at the right size, length and frame rate, and OCR found every required text (name, tagline, $14 / box, closing line, chart title, captions) in each of them. Mean score 4.91 without the skill, 4.89 with only a one-sentence version of its rules, 4.91 with the skill. The only points lost, in all groups, are the bar-chart explainer holding still for long stretches (15-23% of frames with motion) and two one-liner teasers with 48-52%.
• We viewed contact sheets of the teaser from all three groups: all look like finished product ads (cream and dark-green scenes, a pouch illustration, a price end card) and we could not rank them.
• The skill's main concrete rule was followed with or without it. Reading the source: in the 6 no-skill projects none calls useVideoConfig() (they hard-code 30 fps), in all 6 skill projects and all 6 one-liner projects it is used, and no project in any group uses CSS transitions or keyframes. So the one sentence carried the same rule, and the agents already animate with useCurrentFrame() and interpolate() unprompted.
• The skill is read every time (6/6 treatment runs) and costs more: 217k tokens per run against 159k without it (1.36x) and 156k with the one-liner (1.39x), 14 commands against 9. This is the smallest overhead of the three video skills we tested.
• Part of it could not run. The skill's first instruction is to start Remotion Studio and open a preview before building; with no network and no tooling that did not happen, so the result is the skill's text only.

Dimension scores

Limits of this test

• No Studio preview, no render and no Remotion CLI in the sandbox, so the skill's workflow was only partly exercised.
• Three code-drawn tasks (teaser, bar chart, vertical captioned short); video footage, audio, captions from speech, maps, Lottie, 3D and the SaaS rendering features were not tested.
• Renders are on Linux with a software GL and the check for motion is a crude frame-difference threshold; the chart task loses points in every group.
• The useVideoConfig counts come from reading the source and are not part of the 0-30 score.
• Remotion License: free for individuals and companies of up to three people, paid above that; we only evaluated and are not redistributing.
• n = 2 per group per task: differences under about 1 point are noise.
• Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 13 CRITICAL / 16 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

For one-off code-drawn videos the skill made no measurable difference: the agent already writes sound Remotion code, and one sentence of the skill's key rule reproduced its small structural effect at about 0.7x the cost. It is cheap enough to leave installed if you build Remotion projects often (the sub-skills cover maps, captions and rendering that we did not test), but there is no evidence here that it improves a simple video.