hyperframes (heygen-com/hyperframes)

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 19.9 / 30，结论 视情况而定。

下面是评测原文，英文。

Verdict: Situational (19.9 / 30). Source: heygen-com/hyperframes, skills/hyperframes, pinned at commit af0d68d. Install: npx skills@latest add heygen-com/hyperframes --skill hyperframes. Licence: Apache-2.0.

What it is

The hyperframes skill from heygen-com/hyperframes (Apache-2.0): a 1,100-word SKILL.md plus ~25 reference files and a few helper scripts, for making videos as HTML that is rendered to MP4 by seeking a paused GSAP timeline frame by frame. It sets a 12-rule render contract (every timed element is class="clip" with data-start/data-duration, one paused timeline registered on window.__timelines, nothing non-deterministic), an editable structure (one scene per sub-composition) and a work loop built around the npx hyperframes CLI (check, snapshot, preview, render).

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

Environment: the project template (index.html with the GSAP script tag, composition metadata) is the same for all groups. The sandbox has no network and no HyperFrames CLI, so none of the groups could run check, snapshot or render; they wrote source only. We then rendered every project on a separate offline machine with the pinned HyperFrames 0.8.226 (draft quality). This tests the skill's text, not its CLI loop.

1. 15 s product teaser — In this HyperFrames project (index.html root composition, GSAP timeline, rendered by npx hyperframes render) build a 15-second, 1920x1080, 30 fps product teaser for Tidepool, a monthly seaweed-snack subscription. Required on-screen text: the name 'Tidepool',…
2. 20 s animated bar-chart explainer — In this HyperFrames project (index.html root composition, GSAP timeline, rendered by npx hyperframes render) build a 20-second, 1920x1080, 30 fps animated explainer: a bar chart of monthly revenue (Aug $42k, Sep $58k, Oct $71k). Title 'Revenue, last 3 months…
3. 30 s vertical captioned short — In this HyperFrames project (index.html root composition, GSAP timeline, rendered by npx hyperframes render) build a 30-second vertical 1080x1920, 30 fps social short with burned-in captions. Captions, in order, each on screen for about 5 seconds starting at…

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "Build it as a HyperFrames HTML composition: a root with data-composition-id/data-width/data-height/data-duration, every timed element given class="clip" plus data-start and data-duration in seconds, one paused GSAP timeline registered as window.__timelines["<composition-id>"], and nothing non-deterministic (no Date.now(), unseeded Math.random(), timers, network or repeat:-1)."

Mean score: 4.9 without vs 4.89 with the skill (delta -0.01). Tokens per run: 128k vs 249k (1.95x); time 154s vs 208s. One-liner group: 4.91 (the skill's gain over the one-liner: -0.01; one-liner's own gain over no skill: +0.0); one-liner tokens 150k per run, so the skill costs 1.67x the one-liner.

What happened

• Video quality was the same. All 18 projects rendered to the right size, length and frame rate on the first try, and the required text (name, tagline, $14 / box, closing line, captions) was found on screen by OCR in every run that should have had it. Mean score 4.90 without the skill, 4.91 with only the one-line version of its rules, 4.89 with the skill. The only points lost, in all three groups, come from the bar-chart explainer holding still for long stretches (frames with motion 15-23%).
• The one-line version did as well as the skill. Giving the control agent a single sentence of the contract (clip class, data-start/data-duration, one paused timeline, no non-deterministic code) produced the same scores and the same look: we viewed contact sheets of the teaser, chart and vertical short from all three groups and could not rank them by quality.
• The skill is read every time (6/6 treatment runs) but changes how the project is built rather than how it looks. All 6 skill runs split the film into sub-compositions (compositions/*.html, one per scene) as its 'keep it editable' section asks; 0 of 12 runs without the skill did. Marking timed elements as clips varied by group, not by skill: the no-skill runs used 0-6 per project, the skill runs 1-9, and the one-liner runs 3-25. That is the part of the skill that matters if a person will open the project in HyperFrames Studio afterwards, and it is not scored here.
• It costs a lot. 249k tokens per run against 128k without the skill (1.94x) and 150k with the one-liner (1.66x), 15 commands per run against 8, and about 35% more time. The agent reads SKILL.md and several reference files first, and in autonomous mode it still writes SPEC.md, REVIEW.md and a README that nobody asked for (4.7 files per run against 1).
• Part of the skill could not run. SKILL.md's loop is check, snapshot, review, then render; with no CLI available the agents fell back to hand-written REVIEW.md files. A setup where the CLI works could show a different result, and might catch errors the first-pass projects here did not have.

Dimension scores

Limits of this test

• The CLI loop (check, snapshot, preview, render) was not available in the sandbox, so this scores the skill's guidance only; the skill's own premise is that the CLI closes the loop.
• Three tasks: a product teaser, a three-bar chart and a vertical captioned short, all drawn in code. Footage, audio, maps, registry blocks and the HeyGen-based media features were not tested.
• Draft-quality renders on a Linux machine; the check for motion is a crude frame-difference threshold and the same chart task loses points in every group.
• The structural counts (sub-compositions, clip elements) come from reading the source files and are not part of the 0-30 score.
• Licence is Apache-2.0, but the CLI the skill relies on is a separate npm package we did not audit or run on the test computer; it sends usage telemetry unless disabled.
• n = 2 per group per task: differences under about 1 point are noise.
• Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 27 CRITICAL / 26 HIGH / 88 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

For one-off code-drawn videos in this sandbox, the skill did not make the videos any better and cost about double the tokens; one sentence of its contract did the same job. Its value is in the project structure (scene per sub-composition, marked clips) for people who will keep editing in HyperFrames Studio, and in the CLI loop we could not exercise. Install it if you work in HyperFrames regularly; skip it for quick throwaway renders.