writing-skills (obra/superpowers)

Name: writing-skills (obra/superpowers) review
Item: writing-skills (obra/superpowers)
Rating: 17.6
Author: Agent Skills

writing-skills：对照组 4.43、一句话提示 4.71、装 skill 4.88（0–5 分，Codex 实测，n=2/任务/组）。结论：视情况而定，相对不装 skill +0.45。

看情况安全

17.6 / 30

相对不装 skill 的提升：+0.45

视情况而定：总分 17.6/30 低于 22，并且提升 +0.45 低于 +1.0。推荐要求总分 22/30、产出质量 3.5、提升 +1.0。

前沿

通过

使用 Codex 测试。

覆盖 Codex, Claude Code

Codex CLI 0.166（SOTA 档）：18 次评分运行。按我们的分档规则，这档通过同时覆盖 Claude Code；它不代表第二档的结果。

分数

每一维 0 到 5 分，六维相加是满分 30 的总分。每一维都是 5 更好。成本 5 表示又便宜又快。安装 5 表示容易。

触发可靠性: 3.3
相对不装的提升: 2.5
产出质量: 4.9
令牌与时间成本: 1.0
安装与可移植性: 3.0
安全: 3.0

安全审查

安全

静态扫描：0 CRITICAL / 1 HIGH / 1 MEDIUM。人工审查：可在断网沙箱中运行。扫描没有命中不等于绝对安全。

安装

npx skills@latest add obra/superpowers --skill writing-skills

来源

固定提交: 2ce8aa15d05bd57213e6f746f953aafb7ded7a8b

测试日期: 2026-11-02

我们测了什么

Write a commit-message skill — Write a skill (a SKILL.md) at skills/commit-writer/SKILL.md that makes our coding agents write commit messages following these rules: the subject is imperative, under 50 characters, with no trailing period; then a blank line and a body wrapped at 72 characters…
Tighten an AGENTS.md — AGENTS.md has grown to the point that nobody reads it. Rewrite it in place so it is as short as it can be without losing anything an agent actually needs.
Write a bug-triage skill — Write a skill (a SKILL.md) at skills/bug-triage/SKILL.md that has agents classify incoming bug reports as S1 (data loss or outage), S2 (a major feature is broken but a workaround exists) or S3 (minor or cosmetic), and ask for the missing reproduction steps fir…

优点

装 skill 后绝对质量高：4.88 / 5。

缺点

token 消耗是不装 skill 时的 2.56 倍。
每个任务每组 n=2，差距在 1 分以内属于噪声。

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

组别	平均分（0–5）	每次 token
不装 skill	4.43	64k
一句话提示	4.71	66k
装了 skill	4.88	163k

Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 17.6 / 30，结论 视情况而定。

下面是评测原文，英文。

Verdict: Situational (17.6 / 30). Source: obra/superpowers, skills/writing-skills, pinned at commit 2ce8aa1. Install: npx skills@latest add obra/superpowers --skill writing-skills. Licence: MIT (repo LICENSE).

What it is

The writing-skills skill (obra/superpowers): a guide for authoring agent skills, built around test-driven writing (baseline the failure, write the minimal skill, close loopholes). It sets rules for the frontmatter description (triggering conditions only, starting 'Use when', never summarising the workflow), file structure, token efficiency and cross-referencing. Ships a render-graphs.js helper and long reference files.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

Three tasks: write a commit-message skill from four rules, tighten a bloated AGENTS.md without losing the essentials, and write a bug-triage skill (S1/S2/S3 plus 'ask for missing repro steps first'). Control gets only the task. The one-liner adds a sentence stating the description rule, short SKILL.md and one concrete example.

Write a commit-message skill — Write a skill (a SKILL.md) at skills/commit-writer/SKILL.md that makes our coding agents write commit messages following these rules: the subject is imperative, under 50 characters, with no trailing period; then a blank line and a body wrapped at 72 characters…
Tighten an AGENTS.md — AGENTS.md has grown to the point that nobody reads it. Rewrite it in place so it is as short as it can be without losing anything an agent actually needs.
Write a bug-triage skill — Write a skill (a SKILL.md) at skills/bug-triage/SKILL.md that has agents classify incoming bug reports as S1 (data loss or outage), S2 (a major feature is broken but a workaround exists) or S3 (minor or cosmetic), and ask for the missing reproduction steps fir…

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "A skill's description says only when to use it (triggering conditions, starting 'Use when', under about 400 characters, never summarising the workflow); keep SKILL.md short and imperative, move bulky reference out, and include one concrete example instead of narrative."

Task	Control score	One-liner	With skill	Control tokens	One-liner tokens	With-skill tokens	Skill read
t1-commit-skill	4.46	5.0	5.0	75k	65k	249k	2/2
t2-trim-agents	4.64	4.14	4.64	59k	77k	76k	0/2
t3-triage-skill	4.18	5.0	5.0	55k	55k	161k	2/2

Mean score: 4.43 without vs 4.88 with the skill (delta +0.45). Tokens per run: 64k vs 163k (2.56x); time 34s vs 52s. One-liner group: 4.71 (the skill's gain over the one-liner: +0.17; one-liner's own gain over no skill: +0.29); one-liner tokens 66k per run, so the skill costs 2.45x the one-liner.

What happened

Skill 4.88, one-liner 4.71, control 4.43 (n=6 each). The gain is the description: control's descriptions summarise the workflow ('Triage incoming bug reports by requesting missing reproduction steps and classifying severity as S1, S2, or S3'), which is the anti-pattern the skill warns about, and both control bug-triage skills scored 5/7 on the quality check ('when to use it' wording and a concrete example). The skill and one-liner groups wrote 'Use when ...' descriptions and scored 7/7 in all 4 of their triage and commit-message runs.
Cost: 163k tokens per run for the skill against 64k control and 66k one-liner (2.5x). On the commit-skill task both treatment runs spent 243-255k tokens (and up to 229k on triage) reading the long reference files; on the AGENTS.md trimming task, which has nothing to do with skills, it used 68-85k.
On the commit-message skill, control omitted the 'explain why, wrapped at 72' rule in both runs (3/4 rules) while both skill and one-liner runs kept all four.
Tightening AGENTS.md (task 2) gives the same 4.64 across groups, with one one-liner run at 3.64 (it dropped the 'migrations are append-only' rule and kept the stale docs pointer): the skill did not make the trimmed file better here. A check that counts package.json commands restated in AGENTS.md fails in all 18 runs, so it is a defective check and was ignored.
Its helper (render-graphs.js, runs dot through execFileSync) was not run.

Dimension scores

Dimension	Score	Why
1_output_quality	4.88	Mean score 4.88/5 with the skill (programmatic checks, see per-task table).
2_lift_vs_no_skill	2.45	+0.45 points vs 4.43 without the skill (n=2 per task; under ~1 point is noise).
3_trigger	3.3	Skill file read in 4/6 treatment runs.
4_instructions_portability	3	Long SKILL.md plus several large reference files and a Node/graphviz helper; heavy for its purpose.
5_efficiency	1	2.56x tokens (163k vs 64k per run) and 1.52x time (52s vs 34s).
6_safety_hygiene	3	Scanner clean-ish; helper invokes `dot`; mentions ~/.claude/skills paths in examples (see security.md).

Limits of this test

Rule-based checks on the generated SKILL.md files (frontmatter, description wording, length) and recall of the rules; no test of whether the written skills actually change an agent's behaviour, which is what the skill's own method demands.
One check (package.json commands in AGENTS.md) is defective and fails everywhere.
n=2 per cell.
n = 2 per group per task: differences under about 1 point are noise.
Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 1 HIGH / 1 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

It does teach the one rule that matters most for triggering (descriptions that say when to use, not what the skill does) and produces tidier skill files, but at 2.5x tokens, and a single sentence gets almost all of it. Read the guide; do not load it for every small skill edit.

记录

Run log for writing-skills: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run.
Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened.

t1-commit-skill control-1: score 4.46, tokens 75571, secs 36.1, checks 1/2, opened nothing
t1-commit-skill control-2: score 4.46, tokens 75427, secs 37.8, checks 1/2, opened nothing
t1-commit-skill oneliner-1: score 5.0, tokens 75487, secs 32.0, checks 2/2, opened nothing
t1-commit-skill oneliner-2: score 5.0, tokens 55779, secs 28.0, checks 2/2, opened nothing
t1-commit-skill treatment-1: score 5.0, tokens 255351, secs 72.7, checks 2/2, opened writing-skills
t1-commit-skill treatment-2: score 5.0, tokens 243348, secs 61.0, checks 2/2, opened writing-skills
t2-trim-agents control-1: score 4.64, tokens 50635, secs 30.0, checks 6/7, opened nothing
t2-trim-agents control-2: score 4.64, tokens 68044, secs 37.7, checks 6/7, opened nothing
t2-trim-agents oneliner-1: score 3.64, tokens 68983, secs 35.6, checks 4/7, opened nothing

18 次运行、3 个产物

样例产物

前沿 · 装了 skill：skills/commit-writer/SKILL.md（t1-commit-skill，第 1 次，Codex）

--- name: commit-writer description: Use when drafting, editing, or reviewing Git commit messages. --- # Commit Writer Write commit messages that explain the reason for the change using this structure: 1. An imperative subject (for example, "Reject expired sessions") of fewer than 50 characters, including spaces. Do not add a trailing period. 2. A blank line. 3. A body explaining why the change is needed: the problem, motivation, or intended benefit. Do not merely list code changes. Wrap lines at 72 characters maximum, breaking at word boundaries. 4. A blank line followed by the actual ticket ID on the final line, such as `PROJ-123`. Nothing follows the ticket reference. Use the diff and task context to establish the motivation and ticket ID. If either is unavailable, ask for the missing information; do not invent it or substitute the example ticket ID. Never mention AI tools anywhere in the commit message, including generation credits, attribution trailers, or copied development notes. Before returning the message, check the imperative wording, subject length (49 characters maximum), blank lines, body line lengths, rationale, final ticket reference, and absence of AI tool mentions. Example: ```text Reject expired sessions Expired sessions let users retain access to private pages after their authorization ends. Enforce the session lifetime to protect that data. PROJ-123 ```

查看 Pro