hyperframes-core (HeyGen HyperFrames)

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 25.8 / 30，结论 视情况而定。

下面是评测原文，英文。

Verdict: Situational (25.8 / 30). Source: heygen-com/hyperframes, skills/hyperframes-core, pinned at commit 207c8a4. Install: npx skills@latest add heygen-com/hyperframes --skill hyperframes-core. Licence: Apache-2.0 (repo LICENSE).

What it is

The hyperframes-core skill from heygen-com/hyperframes, tested at commit 207c8a4 (the skill was removed from the repo HEAD later, so this describes a retired version). It documents the HyperFrames composition contract: root div with data-composition-id/width/height/duration, timed clips with data-start/data-duration and class=clip, one paused GSAP timeline registered at window.__timelines[root id], render-time determinism rules, and sub-compositions in separate files.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

HyperFrames is described in the prompt as 'an open-source framework that renders an HTML page to MP4 by seeking a GSAP timeline frame by frame', with no contract details. Tasks: author a 12-second three-scene explainer, fix a broken composition (jitter, wrong timing), and build a reusable lower-third as a sub-composition. The one-liner states the full composition contract in one paragraph.

1. Three-scene explainer — Create a 12-second 1920x1080 video as a HyperFrames (an open-source framework that renders an HTML page to MP4 by seeking a GSAP timeline frame by frame; GSAP comes from a CDN script tag). Three scenes of 4 seconds each: a title card ('Meet Orbit'), a bullet l…
2. Fix a broken composition — index.html in this folder is a HyperFrames (an open-source framework that renders an HTML page to MP4 by seeking a GSAP timeline frame by frame; GSAP comes from a CDN script tag) composition that renders wrong (jittery, wrong scene timing, scene 2 invisible, d…
3. Reusable lower-third as a sub-composition — Build a HyperFrames (an open-source framework that renders an HTML page to MP4 by seeking a GSAP timeline frame by frame; GSAP comes from a CDN script tag) video of 8 seconds, 1920x1080. A reusable lower-third (name and job title that slide in from the left an…

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "HyperFrames composition contract: a standalone root <div data-composition-id data-width data-height data-duration> in <body>; timed elements carry data-start and data-duration (class=clip); build ONE gsap.timeline({paused:true}) and register it at window.__timelines['<root id>']; render-time determinism: no Math.random/Date.now/timers/rAF, no repeat:-1, never tween display/visibility/autoAlpha on a clip; sub-compositions live in separate files with data-composition-src, content wrapped in <template>, host id = inner id = timeline key."

Mean score: 4.09 without vs 4.94 with the skill (delta +0.85). Tokens per run: 239k vs 114k (0.47x); time 107s vs 79s. One-liner group: 4.62 (the skill's gain over the one-liner: +0.32; one-liner's own gain over no skill: +0.53); one-liner tokens 160k per run, so the skill costs 0.71x the one-liner.

What happened

• Control 4.09, skill 4.94, one-liner 4.62 (n=6 each). The strongest gap is the explainer: control wrote visibility/display tweens or autoAlpha on clips in 2/2 runs and did not register the timeline at window.__timelines in 2/2; both skill runs and both one-liner runs were clean (5.0).
• Fix task: the skill scored 4.82 against 3.95 for control and 3.86 for the one-liner. Control left the root without a duration and hard-coded px size in 2/2, and the root's duration did not cover the last clip; the one-liner runs hard-coded px size and one did not register the timeline.
• Sub-composition task: all three arms scored 5.0, so a bare model already handles data-composition-src and the template wrapper when asked for a reusable component.
• Tokens: skill 114k, one-liner 160k, control 239k per run. Without the contract, control spends many more steps (13 commands per run against 9) trying to work out the format; the skill is cheaper than no guidance at all.
• One check, 'every timed element has a duration', fails for 3 runs including skill runs; it counts only two clips in the fix task's file and is the same rule for all arms.

Dimension scores

Limits of this test

• Checks are static: HTML structure, GSAP calls and timeline registration were parsed, but no frames were rendered (the hyperframes CLI, Chrome and FFmpeg were not run).
• The skill was tested at the commit 207c8a4 because HEAD no longer contains it; the install command in the review may not work against the current repository.
• n=2 per cell; control's token count is inflated by a few very long runs (up to 440k).
• Prompts describe HyperFrames in one sentence; an agent with web access could look up the docs, but network was off.
• n = 2 per group per task: differences under about 1 point are noise.
• Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 0 HIGH / 21 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

Where the format is unknown, the skill roughly halves the chance of contract mistakes and saves tokens. A paragraph stating the contract recovers most of the benefit. Because the skill is no longer in the vendor's repo, treat the result as a record of that earlier version.