setup-pre-commit (mattpocock/skills)
setup-pre-commit: for a vague 'add commit hooks' request plain Codex builds a different (native git hook) setup, even calling npm in a pnpm repo (seen in one run); the skill and a one-sentence prompt both give the Husky + lint-staged setup (5.0 vs 1.4)
29.0 / 30
Lift over no skill: +3.59
Recommended: the total 29.0/30 is at least 22, output quality 5.0 is at least 3.5, and the lift +3.59 is at least +1.0.
SOTA
PassTested with Codex.
Covers Codex, Claude Code
Codex CLI 0.166 (SOTA tier): 18 graded runs. A pass here also covers Claude Code under our tier rule; it says nothing about the second tier.
Scores
Each dimension is 0 to 5 and the six add up to the total out of 30. 5 is better on every dimension. Cost 5 means cheap and fast. Install 5 means easy.
- Trigger reliability
- 5.0
- Lift over no skill
- 5.0
- Output quality
- 5.0
- Token and time cost
- 5.0
- Install and portability
- 4.0
- Safety
- 5.0
Security review
SafeStatic scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Manual review: Safe to run in a no-network sandbox (no findings). A clean scan does not prove a skill safe.
Install
npx skills@latest add mattpocock/skills --skill setup-pre-commit
Pinned commit: 4ae2317ddb22b046e26455e4f9c9b3aaaf62212d
Tested: 2026-11-02
What we tested
- npm project with typecheck and test scripts — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).
- pnpm project without a typecheck script — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).
- Existing Prettier config must survive — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).
Pros
- Lifts the mean score from 1.41 to 5.0 over no skill (+3.59).
- The skill was picked up without being named: Skill file read in 6/6 treatment runs.
- No token overhead: 0.99x the tokens of no skill.
- High absolute quality with the skill: 5.0 out of 5.
Cons
- A one-sentence prompt scored 5.0, against 5.0 with the skill (+0.00).
- Test caveat: One check was fixed after the first scoring (an untouched file must pass), and every run was re-scored.
- n=2 per task per arm; differences under about 1 point are noise.
Review
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below). Three small repos, all with the same prompt 'Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing)': an npm repo with typecheck and test scripts, a pnpm repo (pnpm-lock.yaml) with no typecheck script, and an npm repo that already has a .prettierrc. One-liner = the skill's recipe in one sentence.
Three-arm comparison
| Arm | Mean score (0-5) | Tokens per run |
|---|---|---|
| Without the skill | 1.41 | 82k |
| One-sentence prompt | 5.0 | 75k |
| With the skill | 5.0 | 82k |
Measured with Codex CLI over 18 runs, scored by rule-based checks with no LLM judge. Total 29.0 / 30, verdict Recommended.
Disclosure: One check was fixed after the first scoring (an untouched file must pass), and every run was re-scored.
Verdict: Recommended (29.0 / 30). Source: mattpocock/skills, skills/misc/setup-pre-commit, pinned at commit 4ae2317. Install: npx skills@latest add mattpocock/skills --skill setup-pre-commit. Licence: MIT (repo LICENSE).
What it is
The setup-pre-commit skill (mattpocock/skills): sets up Husky v9 pre-commit hooks with lint-staged and Prettier: detect the package manager from the lockfile, write .husky/pre-commit (no shebang) that runs lint-staged then the repo's typecheck and test scripts through that package manager (skipping and reporting scripts that do not exist), add a .lintstagedrc with prettier --ignore-unknown --write, add prepare: husky, and create .prettierrc only when none exists.
How we tested
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).
Three small repos, all with the same prompt 'Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing)': an npm repo with typecheck and test scripts, a pnpm repo (pnpm-lock.yaml) with no typecheck script, and an npm repo that already has a .prettierrc. One-liner = the skill's recipe in one sentence.
- npm project with typecheck and test scripts — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).
- pnpm project without a typecheck script — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).
- Existing Prettier config must survive — Add commit hooks. Set up pre-commit hooks for this repo (no network is available, so write the files and package.json changes yourself instead of installing).
Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.
Results
The one-liner given to the third group: "Set up Husky v9 with lint-staged: detect the package manager from the lockfile, write .husky/pre-commit (no shebang) running npx lint-staged then the repo's typecheck and test scripts via that package manager (omit and tell me about any script that does not exist), a .lintstagedrc with prettier --ignore-unknown --write, a prepare: husky script, and create .prettierrc only if none exists."
| Task | Control score | One-liner | With skill | Control tokens | One-liner tokens | With-skill tokens | Skill read |
|---|---|---|---|---|---|---|---|
| t1-npm | 0.0 | 5.0 | 5.0 | 76k | 75k | 78k | 2/2 |
| t2-pnpm | 0.0 | 5.0 | 5.0 | 103k | 66k | 87k | 2/2 |
| t3-existing-prettier | 4.23 | 5.0 | 5.0 | 66k | 83k | 78k | 2/2 |
Mean score: 1.41 without vs 5.0 with the skill (delta +3.59). Tokens per run: 82k vs 82k (0.99x); time 58s vs 43s. One-liner group: 5.0 (the skill's gain over the one-liner: +0.0; one-liner's own gain over no skill: +3.59); one-liner tokens 75k per run, so the skill costs 1.09x the one-liner.
What happened
- Control 1.41, one-liner 5.00, skill 5.00 (n=6 each). Without a tool named, control picked its own design: in the npm and pnpm repos (4 runs) it wrote a native git hook folder (.githooks/pre-commit plus a README and a prepare script) with no Husky, no lint-staged and no Prettier, and in the pnpm repo the hook runs
npm test(checked in run 1) although the repo uses pnpm. Only the existing-.prettierrc repo got Husky (2 runs). - Skill and one-liner runs are all correct on the details: pnpm repo used
pnpm exec lint-stagedandpnpm run test, left out the missing typecheck and said so, the existing .prettierrc was not rewritten,prepare: huskyand the three devDependencies were added. The skill matters through the opinion (Husky + lint-staged + Prettier), not through hard engineering. - No cost: 82k tokens per run for the skill, 82k control, 75k one-liner. The skill was read in 6/6 treatment runs without being named.
- The agent could not install or run anything offline, so none of the generated hooks was executed; both groups said hooks stay inactive until
npm install.
Dimension scores
| Dimension | Score | Why |
|---|---|---|
| 1_output_quality | 5.0 | Mean score 5.00/5 with the skill (programmatic checks, see per-task table). |
| 2_lift_vs_no_skill | 5 | +3.59 points vs 1.41 without the skill (n=2 per task; under ~1 point is noise). |
| 3_trigger | 5.0 | Skill file read in 6/6 treatment runs. |
| 4_instructions_portability | 4 | One short SKILL.md, no scripts. |
| 5_efficiency | 5 | 0.99x tokens (82k vs 82k per run) and 0.73x time (43s vs 58s). |
| 6_safety_hygiene | 5 | Prose only; it tells the agent to edit package.json and add hooks, which run on every commit. |
Limits of this test
- Rule-based checks on the files written; the hooks were never executed (no network, Husky not installed).
- Control's native-hook solution is a valid design; the checks measure whether the requested Husky/lint-staged recipe was produced, which control had no way to know.
- One check (existing .prettierrc kept) initially failed for every run because unchanged files are not copied out; it was changed to pass when the file is untouched and all 18 runs were re-scored under the same rule.
- n = 2 per group per task: differences under about 1 point are noise.
- Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.
Security
See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.
Bottom line
If you want the Husky + lint-staged + Prettier setup, the skill gets it right on all three repo shapes (pnpm detection, missing typecheck, existing Prettier config) for no extra tokens; a one-sentence recipe does the same. Install it to avoid re-explaining the recipe, expect no quality gain beyond that.
Transcript
Run log for setup-pre-commit: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run. Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened. t1-npm control-1: score 0.0, tokens 84825, secs 49.0, checks 0/7, opened nothing t1-npm control-2: score 0.0, tokens 67687, secs 62.1, checks 0/7, opened nothing t1-npm oneliner-1: score 5.0, tokens 66754, secs 38.4, checks 7/7, opened nothing t1-npm oneliner-2: score 5.0, tokens 83997, secs 38.6, checks 7/7, opened nothing t1-npm treatment-1: score 5.0, tokens 69607, secs 37.3, checks 7/7, opened setup-pre-commit t1-npm treatment-2: score 5.0, tokens 87911, secs 42.1, checks 7/7, opened setup-pre-commit t2-pnpm control-1: score 0.0, tokens 102203, secs 77.9, checks 0/8, opened nothing t2-pnpm control-2: score 0.0, tokens 104460, secs 67.1, checks 0/8, opened nothing t2-pnpm oneliner-1: score 5.0, tokens 66597, secs 34.4, checks 8/8, opened nothing
18 runs, 3 artifacts
Sample output
SOTA · With the skill: package.json (t1-npm, run 1, Codex)
{ "name": "app", "version": "1.0.0", "private": true, "scripts": { "prepare": "husky", "test": "vitest run", "typecheck": "tsc --noEmit" }, "devDependencies": { "husky": "^9.1.7", "lint-staged": "^15.5.2", "prettier": "^3.5.3", "typescript": "^5.4.0" } }