writing-for-agents (mattpocock/skills)
writing-for-agents：对照组 4.17、一句话提示 4.63、装 skill 4.52（0–5 分，Codex 实测，n=2/任务/组）。结论：视情况而定，相对不装 skill +0.35。
24.9 / 30
相对不装 skill 的提升：+0.35
视情况而定：提升 +0.35 低于 +1.0。推荐要求总分 22/30、产出质量 3.5、提升 +1.0。
前沿
通过使用 Codex 测试。
覆盖 Codex, Claude Code
Codex CLI 0.166（SOTA 档）：18 次评分运行。按我们的分档规则，这档通过同时覆盖 Claude Code；它不代表第二档的结果。
分数
每一维 0 到 5 分，六维相加是满分 30 的总分。每一维都是 5 更好。成本 5 表示又便宜又快。安装 5 表示容易。
- 触发可靠性
- 5.0
- 相对不装的提升
- 2.4
- 产出质量
- 4.5
- 令牌与时间成本
- 4.0
- 安装与可移植性
- 4.0
- 安全
- 5.0
安全审查
安全静态扫描：0 CRITICAL / 0 HIGH / 0 MEDIUM。人工审查：可在断网沙箱中运行。扫描没有命中不等于绝对安全。
安装
npx skills@latest add mattpocock/skills --skill writing-for-agents
固定提交: 4ae2317ddb22b046e26455e4f9c9b3aaaf62212d
测试日期: 2026-11-02
我们测了什么
- Write a commit-message skill — Write a skill (a SKILL.md) at skills/commit-writer/SKILL.md that makes our coding agents write commit messages following these rules: the subject is imperative, under 50 characters, with no trailing period; then a blank line and a body wrapped at 72 characters…
- Tighten an AGENTS.md — AGENTS.md has grown to the point that nobody reads it. Rewrite it in place so it is as short as it can be without losing anything an agent actually needs.
- Write a bug-triage skill — Write a skill (a SKILL.md) at skills/bug-triage/SKILL.md that has agents classify incoming bug reports as S1 (data loss or outage), S2 (a major feature is broken but a workaround exists) or S3 (minor or cosmetic), and ask for the missing reproduction steps fir…
优点
- 不点名也会被触发：Skill file read in 6/6 treatment runs.
- 装 skill 后绝对质量高：4.52 / 5。
缺点
- 一句话提示得 4.63，装 skill 得 4.52（差 -0.11）。
- 每个任务每组 n=2，差距在 1 分以内属于噪声。
评测
在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。
三组对比
| 组别 | 平均分（0–5） | 每次 token |
|---|---|---|
| 不装 skill | 4.17 | 70k |
| 一句话提示 | 4.63 | 76k |
| 装了 skill | 4.52 | 88k |
Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 24.9 / 30，结论 视情况而定。
下面是评测原文，英文。
Verdict: Situational (24.9 / 30). Source: mattpocock/skills, skills/productivity/writing-for-agents, pinned at commit 4ae2317. Install: npx skills@latest add mattpocock/skills --skill writing-for-agents. Licence: MIT (repo LICENSE).
What it is
The writing-for-agents skill (mattpocock/skills): guidance for writing instructions that agents read: put the trigger first in a description with one trigger per distinct branch, keep each meaning in a single source of truth, delete no-op instructions the model already follows and lines that restate the environment (scripts, config), end each step on a checkable completion criterion, and keep files short.
How we tested
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).
Same three tasks as for writing-skills: write a commit-message skill from four rules, tighten a bloated AGENTS.md without losing essentials (5 real rules buried among filler, duplicates and a stale pointer), and write a bug-triage skill. Control gets only the task. The one-liner gives the skill's main rules in one sentence. The generated SKILL.md files are scored with the same 7-point checklist as writing-skills, which expects a 'Use when ...' style description; this skill's own advice is 'trigger first', not that literal wording.
- Write a commit-message skill — Write a skill (a SKILL.md) at skills/commit-writer/SKILL.md that makes our coding agents write commit messages following these rules: the subject is imperative, under 50 characters, with no trailing period; then a blank line and a body wrapped at 72 characters…
- Tighten an AGENTS.md — AGENTS.md has grown to the point that nobody reads it. Rewrite it in place so it is as short as it can be without losing anything an agent actually needs.
- Write a bug-triage skill — Write a skill (a SKILL.md) at skills/bug-triage/SKILL.md that has agents classify incoming bug reports as S1 (data loss or outage), S2 (a major feature is broken but a workaround exists) or S3 (minor or cosmetic), and ask for the missing reproduction steps fir…
Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.
Results
The one-liner given to the third group: "Write for agents: put the trigger first in the description with one trigger per distinct branch, keep each meaning in a single source of truth, delete no-op instructions the model already follows and lines that merely restate the environment (scripts, config), end each step on a checkable completion criterion, and keep the file short."
| Task | Control score | One-liner | With skill | Control tokens | One-liner tokens | With-skill tokens | Skill read |
|---|---|---|---|---|---|---|---|
| t1-commit-skill | 4.06 | 4.59 | 4.38 | 65k | 55k | 84k | 2/2 |
| t2-trim-agents | 4.29 | 4.71 | 5.0 | 67k | 95k | 96k | 2/2 |
| t3-triage-skill | 4.18 | 4.59 | 4.18 | 75k | 75k | 84k | 2/2 |
Mean score: 4.17 without vs 4.52 with the skill (delta +0.35). Tokens per run: 70k vs 88k (1.27x); time 39s vs 37s. One-liner group: 4.63 (the skill's gain over the one-liner: -0.11; one-liner's own gain over no skill: +0.46); one-liner tokens 76k per run, so the skill costs 1.17x the one-liner.
What happened
- Means: skill 4.52, one-liner 4.63, control 4.17 (n=6 each). The clean win is the AGENTS.md task: skill 5.00 in both runs (all 5 essential rules kept, filler removed, stale pointer fixed, duplicates collapsed) against 4.29 control and 4.71 one-liner. Both one-liner runs deleted the 'migrations are append-only' rule while trimming; the skill runs kept it.
- On the two skill-writing tasks the skill does no better than control (4.18-4.59 vs 4.06-4.18): in all 4 treatment runs on those tasks the description was a workflow summary or did not say when to use the skill ('Triage incoming bug reports by requesting missing reproduction steps and classifying severity...', identical to control), so the 7-point checklist gave 5/7 or 6/7 just like control. The one-liner runs wrote trigger-style descriptions ('When an incoming bug report lacks reproduction steps, request them first...').
- Control omitted the 'body wrapped at 72, explain why' rule in both commit-skill runs (3/4 rules); skill and one-liner kept all four.
- Cost: 88k tokens per run for the skill vs 70k control and 76k one-liner (1.26x, 1.16x). Treatment runs copy 4 files per run into the project.
- A check that counts package.json scripts restated in AGENTS.md fails for the 2 control runs only here (3 matches), consistent with the skill's 'do not restate the environment' rule.
Dimension scores
| Dimension | Score | Why |
|---|---|---|
| 1_output_quality | 4.52 | Mean score 4.52/5 with the skill (programmatic checks, see per-task table). |
| 2_lift_vs_no_skill | 2.35 | +0.35 points vs 4.17 without the skill (n=2 per task; under ~1 point is noise). |
| 3_trigger | 5.0 | Skill file read in 6/6 treatment runs. |
| 4_instructions_portability | 4 | One short SKILL.md; no scripts. |
| 5_efficiency | 4 | 1.27x tokens (88k vs 70k per run) and 0.94x time (37s vs 39s). |
| 6_safety_hygiene | 5 | Prose only; no findings. |
Limits of this test
- Rule-based checks, partly borrowed from the writing-skills test: the 7-point skill checklist rewards 'Use when' wording that this skill does not require, which understates it on the two skill-writing tasks.
- Nobody tested whether the written skills or the trimmed AGENTS.md actually change an agent's behaviour.
- n=2 per cell.
- n = 2 per group per task: differences under about 1 point are noise.
- Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.
Security
See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.
Bottom line
Worth it for compacting an instruction file (it kept every real rule that a one-sentence prompt dropped), otherwise no better than the base model for writing new skills at 1.26x tokens.
记录
Run log for writing-for-agents: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run. Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened. t1-commit-skill control-1: score 4.06, tokens 55944, secs 45.4, checks 1/2, opened nothing t1-commit-skill control-2: score 4.06, tokens 75771, secs 39.9, checks 1/2, opened nothing t1-commit-skill oneliner-1: score 4.59, tokens 55800, secs 30.0, checks 2/2, opened nothing t1-commit-skill oneliner-2: score 4.59, tokens 55891, secs 31.8, checks 2/2, opened nothing t1-commit-skill treatment-1: score 4.18, tokens 84379, secs 38.2, checks 1/2, opened writing-for-agents t1-commit-skill treatment-2: score 4.59, tokens 84910, secs 43.3, checks 2/2, opened writing-for-agents t2-trim-agents control-1: score 3.93, tokens 67718, secs 37.8, checks 5/7, opened nothing t2-trim-agents control-2: score 4.64, tokens 68132, secs 36.7, checks 6/7, opened nothing t2-trim-agents oneliner-1: score 4.71, tokens 86525, secs 36.9, checks 6/7, opened nothing
18 次运行、3 个产物
样例产物
前沿 · 装了 skill：skills/commit-writer/SKILL.md（t1-commit-skill，第 1 次，Codex）
--- name: commit-writer description: Write or revise commit messages using the team's subject, rationale, and ticket-reference conventions. --- # Commit writer Base the message on the changes and their purpose in the available task context. Apply every rule below before returning the message: - Write an imperative subject (for example, "Fix" rather than "Fixed") under 50 characters, including spaces, with no trailing period. - Follow the subject with a blank line and a required body. Explain why the change is needed and its intended benefit, rather than listing what changed. Hard-wrap body lines at 72 characters or fewer. - End the message with the relevant ticket ID, such as `PROJ-123`, on its own line, separated from the body by a blank line. Use the actual ID from the task or branch context; if it is missing or ambiguous, ask for it rather than inventing one. - Keep the message focused on the change's rationale. Never mention AI tools, including in attribution or co-author trailers. Return the commit message as plain text when asked for a message.