Agent Skills

All skills

writing-skills (obra/superpowers)

writing-skills: gets the description rule right ('Use when...' 4.9 vs 4.4 without the skill), but a one-sentence rule gets 4.7 for a third of the tokens (2.5x)

SituationalSafe

17.6 / 30

Lift over no skill: +0.45

Situational: the total 17.6/30 is under 22 and the lift +0.45 is under +1.0. Recommended needs a total of 22/30, output quality of 3.5, and a lift of +1.0.

SOTA

Pass

Tested with Codex.

Covers Codex, Claude Code

Codex CLI 0.166 (SOTA tier): 18 graded runs. A pass here also covers Claude Code under our tier rule; it says nothing about the second tier.

Scores

Each dimension is 0 to 5 and the six add up to the total out of 30. 5 is better on every dimension. Cost 5 means cheap and fast. Install 5 means easy.

Trigger reliability
3.3
Lift over no skill
2.5
Output quality
4.9
Token and time cost
1.0
Install and portability
3.0
Safety
3.0

Security review

Safe

Static scan: 0 CRITICAL / 1 HIGH / 1 MEDIUM. Manual review: Safe to run in a no-network sandbox (scanner hits explained). A clean scan does not prove a skill safe.

Install

npx skills@latest add obra/superpowers --skill writing-skills

Source

Pinned commit: 2ce8aa15d05bd57213e6f746f953aafb7ded7a8b

Tested: 2026-11-02

What we tested

  • Write a commit-message skill — Write a skill (a SKILL.md) at skills/commit-writer/SKILL.md that makes our coding agents write commit messages following these rules: the subject is imperative, under 50 characters, with no trailing period; then a blank line and a body wrapped at 72 characters…
  • Tighten an AGENTS.md — AGENTS.md has grown to the point that nobody reads it. Rewrite it in place so it is as short as it can be without losing anything an agent actually needs.
  • Write a bug-triage skill — Write a skill (a SKILL.md) at skills/bug-triage/SKILL.md that has agents classify incoming bug reports as S1 (data loss or outage), S2 (a major feature is broken but a workaround exists) or S3 (minor or cosmetic), and ask for the missing reproduction steps fir…

Pros

  • High absolute quality with the skill: 4.88 out of 5.

Cons

  • Uses 2.56x the tokens of no skill.
  • n=2 per task per arm; differences under about 1 point are noise.

Review

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below). Three tasks: write a commit-message skill from four rules, tighten a bloated AGENTS.md without losing the essentials, and write a bug-triage skill (S1/S2/S3 plus 'ask for missing repro steps first'). Control gets only the task. The one-liner adds a sentence stating the description rule, short SKILL.md and one concrete example.

Three-arm comparison

ArmMean score (0-5)Tokens per run
Without the skill4.4364k
One-sentence prompt4.7166k
With the skill4.88163k

Measured with Codex CLI over 18 runs, scored by rule-based checks with no LLM judge. Total 17.6 / 30, verdict Situational.

Verdict: Situational (17.6 / 30). Source: obra/superpowers, skills/writing-skills, pinned at commit 2ce8aa1. Install: npx skills@latest add obra/superpowers --skill writing-skills. Licence: MIT (repo LICENSE).

What it is

The writing-skills skill (obra/superpowers): a guide for authoring agent skills, built around test-driven writing (baseline the failure, write the minimal skill, close loopholes). It sets rules for the frontmatter description (triggering conditions only, starting 'Use when', never summarising the workflow), file structure, token efficiency and cross-referencing. Ships a render-graphs.js helper and long reference files.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

Three tasks: write a commit-message skill from four rules, tighten a bloated AGENTS.md without losing the essentials, and write a bug-triage skill (S1/S2/S3 plus 'ask for missing repro steps first'). Control gets only the task. The one-liner adds a sentence stating the description rule, short SKILL.md and one concrete example.

  1. Write a commit-message skill — Write a skill (a SKILL.md) at skills/commit-writer/SKILL.md that makes our coding agents write commit messages following these rules: the subject is imperative, under 50 characters, with no trailing period; then a blank line and a body wrapped at 72 characters…
  2. Tighten an AGENTS.md — AGENTS.md has grown to the point that nobody reads it. Rewrite it in place so it is as short as it can be without losing anything an agent actually needs.
  3. Write a bug-triage skill — Write a skill (a SKILL.md) at skills/bug-triage/SKILL.md that has agents classify incoming bug reports as S1 (data loss or outage), S2 (a major feature is broken but a workaround exists) or S3 (minor or cosmetic), and ask for the missing reproduction steps fir…

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "A skill's description says only when to use it (triggering conditions, starting 'Use when', under about 400 characters, never summarising the workflow); keep SKILL.md short and imperative, move bulky reference out, and include one concrete example instead of narrative."

TaskControl scoreOne-linerWith skillControl tokensOne-liner tokensWith-skill tokensSkill read
t1-commit-skill4.465.05.075k65k249k2/2
t2-trim-agents4.644.144.6459k77k76k0/2
t3-triage-skill4.185.05.055k55k161k2/2

Mean score: 4.43 without vs 4.88 with the skill (delta +0.45). Tokens per run: 64k vs 163k (2.56x); time 34s vs 52s. One-liner group: 4.71 (the skill's gain over the one-liner: +0.17; one-liner's own gain over no skill: +0.29); one-liner tokens 66k per run, so the skill costs 2.45x the one-liner.

What happened

  • Skill 4.88, one-liner 4.71, control 4.43 (n=6 each). The gain is the description: control's descriptions summarise the workflow ('Triage incoming bug reports by requesting missing reproduction steps and classifying severity as S1, S2, or S3'), which is the anti-pattern the skill warns about, and both control bug-triage skills scored 5/7 on the quality check ('when to use it' wording and a concrete example). The skill and one-liner groups wrote 'Use when ...' descriptions and scored 7/7 in all 4 of their triage and commit-message runs.
  • Cost: 163k tokens per run for the skill against 64k control and 66k one-liner (2.5x). On the commit-skill task both treatment runs spent 243-255k tokens (and up to 229k on triage) reading the long reference files; on the AGENTS.md trimming task, which has nothing to do with skills, it used 68-85k.
  • On the commit-message skill, control omitted the 'explain why, wrapped at 72' rule in both runs (3/4 rules) while both skill and one-liner runs kept all four.
  • Tightening AGENTS.md (task 2) gives the same 4.64 across groups, with one one-liner run at 3.64 (it dropped the 'migrations are append-only' rule and kept the stale docs pointer): the skill did not make the trimmed file better here. A check that counts package.json commands restated in AGENTS.md fails in all 18 runs, so it is a defective check and was ignored.
  • Its helper (render-graphs.js, runs dot through execFileSync) was not run.

Dimension scores

DimensionScoreWhy
1_output_quality4.88

Mean score 4.88/5 with the skill (programmatic checks, see per-task table).

2_lift_vs_no_skill2.45

+0.45 points vs 4.43 without the skill (n=2 per task; under ~1 point is noise).

3_trigger3.3Skill file read in 4/6 treatment runs.
4_instructions_portability3

Long SKILL.md plus several large reference files and a Node/graphviz helper; heavy for its purpose.

5_efficiency12.56x tokens (163k vs 64k per run) and 1.52x time (52s vs 34s).
6_safety_hygiene3

Scanner clean-ish; helper invokes dot; mentions ~/.claude/skills paths in examples (see security.md).

Limits of this test

  • Rule-based checks on the generated SKILL.md files (frontmatter, description wording, length) and recall of the rules; no test of whether the written skills actually change an agent's behaviour, which is what the skill's own method demands.
  • One check (package.json commands in AGENTS.md) is defective and fails everywhere.
  • n=2 per cell.
  • n = 2 per group per task: differences under about 1 point are noise.
  • Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 1 HIGH / 1 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

It does teach the one rule that matters most for triggering (descriptions that say when to use, not what the skill does) and produces tidier skill files, but at 2.5x tokens, and a single sentence gets almost all of it. Read the guide; do not load it for every small skill edit.

Transcript

Run log for writing-skills: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run.
Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened.

t1-commit-skill control-1: score 4.46, tokens 75571, secs 36.1, checks 1/2, opened nothing
t1-commit-skill control-2: score 4.46, tokens 75427, secs 37.8, checks 1/2, opened nothing
t1-commit-skill oneliner-1: score 5.0, tokens 75487, secs 32.0, checks 2/2, opened nothing
t1-commit-skill oneliner-2: score 5.0, tokens 55779, secs 28.0, checks 2/2, opened nothing
t1-commit-skill treatment-1: score 5.0, tokens 255351, secs 72.7, checks 2/2, opened writing-skills
t1-commit-skill treatment-2: score 5.0, tokens 243348, secs 61.0, checks 2/2, opened writing-skills
t2-trim-agents control-1: score 4.64, tokens 50635, secs 30.0, checks 6/7, opened nothing
t2-trim-agents control-2: score 4.64, tokens 68044, secs 37.7, checks 6/7, opened nothing
t2-trim-agents oneliner-1: score 3.64, tokens 68983, secs 35.6, checks 4/7, opened nothing

18 runs, 3 artifacts

Sample output

SOTA · With the skill: skills/commit-writer/SKILL.md (t1-commit-skill, run 1, Codex)

--- name: commit-writer description: Use when drafting, editing, or reviewing Git commit messages. --- # Commit Writer Write commit messages that explain the reason for the change using this structure: 1. An imperative subject (for example, "Reject expired sessions") of fewer than 50 characters, including spaces. Do not add a trailing period. 2. A blank line. 3. A body explaining why the change is needed: the problem, motivation, or intended benefit. Do not merely list code changes. Wrap lines at 72 characters maximum, breaking at word boundaries. 4. A blank line followed by the actual ticket ID on the final line, such as `PROJ-123`. Nothing follows the ticket reference. Use the diff and task context to establish the motivation and ticket ID. If either is unavailable, ask for the missing information; do not invent it or substitute the example ticket ID. Never mention AI tools anywhere in the commit message, including generation credits, attribution trailers, or copied development notes. Before returning the message, check the imperative wording, subject length (49 characters maximum), blank lines, body line lengths, rationale, final ticket reference, and absence of AI tool mentions. Example: ```text Reject expired sessions Expired sessions let users retain access to private pages after their authorization ends. Enforce the session lifetime to protect that data. PROJ-123 ```

See Pro