video (smixs/visual-skills)

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 20.8 / 30，结论 视情况而定。

下面是评测原文，英文。

Verdict: Situational (20.8 / 30). Source: smixs/visual-skills, video, pinned at commit 92be33a. Install: npx skills@latest add smixs/visual-skills --skill video. Licence: CC-BY-4.0.

What it is

The video skill from smixs/visual-skills (CC-BY-4.0, attribution required): it makes the agent act as director, screenwriter and editor when writing prompts for AI video generators (Seedance, Kling, Veo and similar). SKILL.md is thin by design (about 1,400 words) and orders the agent to read dramaturgy.md, universal-rules.md and one model-specific file (Seedance, Kling or Veo) before writing. Markdown only: 12 reference files, no scripts. It writes prompts; it does not render video.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

This skill produces prompts for external generators (Seedance, Kling), so we scored the prompts. No video file exists for this skill: generating one needs a Seedance/Kling/Veo account and key, which we did not use.

1. Prompts for a 15 s teaser — Write the generation prompts to make a 15-second, 16:9 product teaser for Tidepool, a monthly seaweed-snack subscription ($14 per box), with an AI video generator (Seedance). If one generation cannot cover 15 seconds, split it into clips and say how long each…
2. Prompt for an 8 s cinematic shot — Write the generation prompt for an 8-second cinematic 16:9 shot of a small paper boat on a rain-streaked window sill at dusk, for an AI video generator (Kling). One continuous shot, no cuts. Work autonomously; no questions. Save the result to prompts.md (copy-…
3. Prompts for a 30 s vertical short in clips — Write the generation prompts for a 30-second vertical 9:16 social short for Tidepool (seaweed snacks, $14 per box) for an AI video generator whose clips are at most 10 seconds long. Captions to appear (added later): Meet Tidepool. / Seaweed snacks, crisp and s…

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score. The checks are keyword and structure checks on prompts.md (for example 'names a camera move', 'mentions sound', 'puts a duration on every clip', 'says no cuts'). They cannot judge whether a prompt would produce a better film.

Results

The one-liner given to the third group: "Write each clip prompt like a director: lead with the subject and its action, give one concrete camera move and lens, lighting and the final image, keep the same named character/product anchor in every clip, and put the duration in seconds on every clip."

Mean score: 4.77 without vs 4.3 with the skill (delta -0.47). Tokens per run: 62k vs 172k (2.76x); time 76s vs 126s. One-liner group: 4.69 (the skill's gain over the one-liner: -0.39; one-liner's own gain over no skill: -0.08); one-liner tokens 51k per run, so the skill costs 3.35x the one-liner.

What happened

• By the keyword checks the skill came out slightly behind: mean 4.30 with the skill, against 4.77 with no skill and 4.69 with the one-line version. The gap comes mostly from the 8-second paper-boat prompt (4.11 and 3.67 against 4.56 in all four other runs), then from the teaser (4.25 against 4.75: it missed timecoded shots and the price-as-overlay note in places) and one clip plan that did not name camera moves.
• Part of that gap is a difference in taste, not an error. In the boat task both skill runs chose a locked-off camera so that one small action (a drop landing on the bow, two drops merging) carries the shot; the checker looks for words such as 'push-in' or 'dolly' and marked that as a missing camera move. Neither skill run mentioned sound, which the checker also wanted.
• The outputs do look different. Skill runs lay out a three-clip plan with per-clip beats, name the target model and its settings (duration, resolution, camera_fixed), keep one character/product anchor across clips and say what is added later in editing. No-skill runs on the teaser also gave full timecoded prompts, with audio and voice-over lines, and were not worse by any check; the one-liner runs were shorter (about 900 words against 1,300-1,500).
• The skill is read every time (6/6) and the cost is high: 172k tokens per run against 62k without the skill (2.8x) and 51k with the one-liner (3.4x), 14 commands against 3, and about 1.7x the time. The agent reads about six reference files before writing a single prompt.
• The one-line version was as good as no skill by these checks (4.69 against 4.77, within noise) at 0.8x the tokens.

Dimension scores

Limits of this test

• No video was generated, so we cannot say whether the skill's prompts produce better films; the scores measure prompt features only.
• The keyword checks penalise a legitimate style choice (locked camera) and ignore prompt quality; treat the 0.47-point gap as weak evidence.
• Three prompt tasks (15 s 16:9 teaser, 8 s cinematic shot, 30 s vertical split into clips); character-consistency across generations, image-to-video and editing modes were not tested.
• CC-BY-4.0: reuse requires crediting Serge Shima.
• n = 2 per group per task: differences under about 1 point are noise.
• Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

A careful, directorial prompt-writing guide that makes the agent plan clips, anchors and settings, but our keyword checks could not show an advantage, and it costs about 2.8x the tokens of simply asking. Worth trying if you will actually feed the prompts to Seedance, Kling or Veo and compare films; we could not do that part.