review-animations (emilkowalski/skills)
review-animations: makes a pull-request motion review find the planted problems and give exact fixes (control 2.9, skill 4.4, one-liner 4.9) at 1.2x the tokens; a one-sentence prompt of the checklist does better
24.9 / 30
Lift over no skill: +1.52
Recommended: the total 24.9/30 is at least 22, output quality 4.4 is at least 3.5, and the lift +1.52 is at least +1.0.
SOTA
PassTested with Codex.
Covers Codex, Claude Code
Codex CLI 0.166 (SOTA tier): 18 graded runs. A pass here also covers Claude Code under our tier rule; it says nothing about the second tier.
Scores
Each dimension is 0 to 5 and the six add up to the total out of 30. 5 is better on every dimension. Cost 5 means cheap and fast. Install 5 means easy.
- Trigger reliability
- 5.0
- Lift over no skill
- 3.5
- Output quality
- 4.4
- Token and time cost
- 4.0
- Install and portability
- 4.0
- Safety
- 4.0
Security review
SafeStatic scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Manual review: Safe to run in a no-network sandbox (no findings). A clean scan does not prove a skill safe.
Install
npx skills@latest add emilkowalski/skills --skill review-animations
Pinned commit: a81dbca07f2217b7103e510c0f13f8d5b017afc9
Tested: 2026-11-02
What we tested
- Review a motion diff — Review the animation changes in change.diff before I merge. Do not change any files.
- Review just the toast rules — Review only the .toast animation rules in change.diff. Do not change any files.
- Merge verdict — Should I merge the animation changes in change.diff as they are? Answer first, then why. Do not change any files.
Pros
- Lifts the mean score from 2.86 to 4.38 over no skill (+1.52).
- The skill was picked up without being named: Skill file read in 6/6 treatment runs.
Cons
- A one-sentence prompt scored 4.93, against 4.38 with the skill (-0.55).
- n=2 per task per arm; differences under about 1 point are noise.
Review
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below). A change.diff that adds animations with 7 planted problems. Tasks: review the whole diff before merge, review only the toast rules, and answer 'should I merge as is?' (answer first, then why). Prompts do not name any rule. The one-liner is the checklist in one sentence and asks for exact replacements.
Three-arm comparison
| Arm | Mean score (0-5) | Tokens per run |
|---|---|---|
| Without the skill | 2.86 | 53k |
| One-sentence prompt | 4.93 | 48k |
| With the skill | 4.38 | 63k |
Measured with Codex CLI over 18 runs, scored by rule-based checks with no LLM judge. Total 24.9 / 30, verdict Recommended.
Verdict: Recommended (24.9 / 30). Source: emilkowalski/skills, skills/review-animations, pinned at commit a81dbca. Install: npx skills@latest add emilkowalski/skills --skill review-animations. Licence: MIT (repo LICENSE).
What it is
The review-animations skill (emilkowalski/skills): a review checklist for animation changes in a diff: default to flagging ease-in/ease-in-out UI curves, transition: all, scale(0) starts, layout-property animation, animation on high-frequency actions, no-op stagger, missing reduced-motion, durations over 300 ms, with exact replacement values.
How we tested
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).
A change.diff that adds animations with 7 planted problems. Tasks: review the whole diff before merge, review only the toast rules, and answer 'should I merge as is?' (answer first, then why). Prompts do not name any rule. The one-liner is the checklist in one sentence and asks for exact replacements.
- Review a motion diff — Review the animation changes in change.diff before I merge. Do not change any files.
- Review just the toast rules — Review only the .toast animation rules in change.diff. Do not change any files.
- Merge verdict — Should I merge the animation changes in change.diff as they are? Answer first, then why. Do not change any files.
Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.
Results
The one-liner given to the third group: "Review the motion code against a high craft bar and default to flagging: ease-in or ease-in-out UI curves, transition: all, scale(0) starts, layout-property animation, animation on frequent actions, no-op stagger, missing reduced-motion, durations over 300 ms; give the exact replacement values for each finding."
| Task | Control score | One-liner | With skill | Control tokens | One-liner tokens | With-skill tokens | Skill read |
|---|---|---|---|---|---|---|---|
| t1-review-diff | 1.53 | 4.79 | 3.67 | 56k | 56k | 77k | 2/2 |
| t2-toast-only | 2.22 | 5.0 | 5.0 | 47k | 39k | 55k | 2/2 |
| t3-verdict | 4.82 | 5.0 | 4.46 | 55k | 48k | 55k | 2/2 |
Mean score: 2.86 without vs 4.38 with the skill (delta +1.52). Tokens per run: 53k vs 63k (1.18x); time 30s vs 32s. One-liner group: 4.93 (the skill's gain over the one-liner: -0.55; one-liner's own gain over no skill: +2.08); one-liner tokens 48k per run, so the skill costs 1.30x the one-liner.
What happened
- Control 2.86, skill 4.38, one-liner 4.93 (n=6 each). Without guidance the whole-diff review found 3 of 7 planted problems in one run and 1 of 7 in the other and gave no replacement values in 2 of 2 runs.
- Toast-only review: control missed
transition: alland gave no exact fix in both runs (2.2); skill and one-liner scored 5.0 in all four runs. - Whole-diff review: the skill found 5-6 of 7 problems but was inconsistent (4.18 and 3.16); it missed the high-frequency command palette in one run and the no-op stagger in the other. The one-liner scored 5.0 and 4.59.
- Verdict task: control already gives a well-backed answer (5.0 and 4.64), skill 4.64 and 4.29, one-liner 5.0 in both.
- Tokens: skill 63k, one-liner 48k, control 53k per run (1.19x control, 1.3x one-liner). The skill was read in 6/6 treatment runs.
Dimension scores
| Dimension | Score | Why |
|---|---|---|
| 1_output_quality | 4.38 | Mean score 4.38/5 with the skill (programmatic checks, see per-task table). |
| 2_lift_vs_no_skill | 3.52 | +1.52 points vs 2.86 without the skill (n=2 per task; under ~1 point is noise). |
| 3_trigger | 5.0 | Skill file read in 6/6 treatment runs. |
| 4_instructions_portability | 4 | SKILL.md checklist only. |
| 5_efficiency | 4 | 1.18x tokens (63k vs 53k per run) and 1.06x time (32s vs 30s). |
| 6_safety_hygiene | 4 | Prose only; see security.md. |
Limits of this test
- Recall checks on the review text: items found among the planted ones and presence of concrete values; the quality of explanations was not judged.
- The one-liner is a close paraphrase of the skill's own checklist.
- One fixture diff; n=2 per cell.
- n = 2 per group per task: differences under about 1 point are noise.
- Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.
Security
See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.
Bottom line
The skill clearly improves a motion review over no guidance, mainly by getting concrete replacements and catching the less obvious items. The same checklist as a single sentence did better and cheaper in this test, so the benefit is over no guidance rather than over a good prompt.
Transcript
Run log for review-animations: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run. Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened. t1-review-diff control-1: score 1.94, tokens 48194, secs 27.2, checks 1/3, opened nothing t1-review-diff control-2: score 1.12, tokens 64124, secs 37.9, checks 1/3, opened nothing t1-review-diff oneliner-1: score 4.59, tokens 65072, secs 52.4, checks 3/3, opened nothing t1-review-diff oneliner-2: score 5.0, tokens 48423, secs 41.8, checks 3/3, opened nothing t1-review-diff treatment-1: score 3.16, tokens 77459, secs 41.1, checks 2/3, opened review-animations t1-review-diff treatment-2: score 4.18, tokens 77187, secs 40.0, checks 3/3, opened review-animations t2-toast-only control-1: score 2.22, tokens 47474, secs 21.1, checks 0/2, opened nothing t2-toast-only control-2: score 2.22, tokens 47748, secs 25.5, checks 0/2, opened nothing t2-toast-only oneliner-1: score 5.0, tokens 31576, secs 16.0, checks 2/2, opened nothing
18 runs, 3 artifacts
Sample output
SOTA · With the skill: final message (t1-review-diff, run 1, Codex)
| Before | After | Why | | --- | --- | --- | | Toast starts at `scale(0)` ([diff:5](change.diff:5)) | Start at `scale(0.95)` with opacity `0`. | Growing from nothing creates an exaggerated entrance and distorts the toast’s content. | | Toast uses `all 600ms ease-in-out` ([diff:5](change.diff:5)) | Limit transitions to `transform, opacity`; shorten below 300ms and use `cubic-bezier(0.23, 1, 0.32, 1)`. | The entrance starts slowly and takes too long. `all` also allows future layout changes to animate accidentally. | | Sidebar animates `width 400ms` ([diff:7](change.diff:7)) | Remove the width transition; apply the collapsed width immediately. | Animating from 240px to 56px repeatedly recalculates layout and can reflow content throughout an already lengthy transition. | | Command items enter with a 500ms, 24px slide ([diff:10](change.diff:10)) | Remove the item animation and its delay rule. | Assuming `.cmdk-item` represents command-palette items, decorative entrances delay visual availability in a frequently used keyboard workflow. Adding stagger would worsen this. | | Movement has no reduced-motion override ([diff:5–12](change.diff:5)) | Under `prefers-reduced-motion: reduce`, remove scaling and translation; retain a short opacity transition where useful. | The supplied styles impose the same movement on users who request reduced motion. | | Hover transition is ungated ([diff:9](change.diff:9)) | Gate it with `@media (hover: hover) and (pointer: fine)`; prefer an immediate background change. | Touch can produce lingering hover states. Removing this frequent color animation also avoids repeated paint work. | **Feel-breaking regressions:** The toast’s zero-scale entrance and slow timing block approval. The command-item animation should be removed if these are command-palette results. **Performance:** The sidebar width transition adds avoidable layout work. Keep the toast’s transitions explicitly scoped. **Accessibility:** Reduced-motion handling and hover capability gating are missing. **Block.** Static review only; the workspace contains no runnable interface to verify motion visually. Locations above reference the supplied diff. No files changed.