setup-pre-commit (mattpocock/skills)

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 29.0 / 30，结论 推荐。

下面是评测原文，英文。

Verdict: Recommended (29.0 / 30). Source: mattpocock/skills, skills/misc/setup-pre-commit, pinned at commit 4ae2317. Install: npx skills@latest add mattpocock/skills --skill setup-pre-commit. Licence: MIT (repo LICENSE).

What it is

The setup-pre-commit skill (mattpocock/skills): sets up Husky v9 pre-commit hooks with lint-staged and Prettier: detect the package manager from the lockfile, write .husky/pre-commit (no shebang) that runs lint-staged then the repo's typecheck and test scripts through that package manager (skipping and reporting scripts that do not exist), add a .lintstagedrc with prettier --ignore-unknown --write, add prepare: husky, and create .prettierrc only when none exists.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

Three small repos, all with the same prompt 'Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing)': an npm repo with typecheck and test scripts, a pnpm repo (pnpm-lock.yaml) with no typecheck script, and an npm repo that already has a .prettierrc. One-liner = the skill's recipe in one sentence.

1. npm project with typecheck and test scripts — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).
2. pnpm project without a typecheck script — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).
3. Existing Prettier config must survive — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "Set up Husky v9 with lint-staged: detect the package manager from the lockfile, write .husky/pre-commit (no shebang) running npx lint-staged then the repo's typecheck and test scripts via that package manager (omit and tell me about any script that does not exist), a .lintstagedrc with prettier --ignore-unknown --write, a prepare: husky script, and create .prettierrc only if none exists."

Mean score: 1.41 without vs 5.0 with the skill (delta +3.59). Tokens per run: 82k vs 82k (0.99x); time 58s vs 43s. One-liner group: 5.0 (the skill's gain over the one-liner: +0.0; one-liner's own gain over no skill: +3.59); one-liner tokens 75k per run, so the skill costs 1.09x the one-liner.

What happened

• Control 1.41, one-liner 5.00, skill 5.00 (n=6 each). Without a tool named, control picked its own design: in the npm and pnpm repos (4 runs) it wrote a native git hook folder (.githooks/pre-commit plus a README and a prepare script) with no Husky, no lint-staged and no Prettier, and in the pnpm repo the hook runs npm test (checked in run 1) although the repo uses pnpm. Only the existing-.prettierrc repo got Husky (2 runs).
• Skill and one-liner runs are all correct on the details: pnpm repo used pnpm exec lint-staged and pnpm run test, left out the missing typecheck and said so, the existing .prettierrc was not rewritten, prepare: husky and the three devDependencies were added. The skill matters through the opinion (Husky + lint-staged + Prettier), not through hard engineering.
• No cost: 82k tokens per run for the skill, 82k control, 75k one-liner. The skill was read in 6/6 treatment runs without being named.
• The agent could not install or run anything offline, so none of the generated hooks was executed; both groups said hooks stay inactive until npm install.

Dimension scores

Limits of this test

• Rule-based checks on the files written; the hooks were never executed (no network, Husky not installed).
• Control's native-hook solution is a valid design; the checks measure whether the requested Husky/lint-staged recipe was produced, which control had no way to know.
• One check (existing .prettierrc kept) initially failed for every run because unchanged files are not copied out; it was changed to pass when the file is untouched and all 18 runs were re-scored under the same rule.
• n = 2 per group per task: differences under about 1 point are noise.
• Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

If you want the Husky + lint-staged + Prettier setup, the skill gets it right on all three repo shapes (pnpm detection, missing typecheck, existing Prettier config) for no extra tokens; a one-sentence recipe does the same. Install it to avoid re-explaining the recipe, expect no quality gain beyond that.