brainstorming (obra/superpowers)
brainstorming：对照组 2.33、一句话提示 4.67、装 skill 5.0（0–5 分，Codex 实测，n=2/任务/组）。结论：推荐，相对不装 skill +2.67。
27.7 / 30
相对不装 skill 的提升：+2.67
推荐：总分 27.7/30 不低于 22，产出质量 5.0 不低于 3.5，提升 +2.67 不低于 +1.0。
前沿
通过使用 Codex 测试。
覆盖 Codex, Claude Code
Codex CLI 0.166（SOTA 档）：18 次评分运行。按我们的分档规则，这档通过同时覆盖 Claude Code；它不代表第二档的结果。
分数
每一维 0 到 5 分，六维相加是满分 30 的总分。每一维都是 5 更好。成本 5 表示又便宜又快。安装 5 表示容易。
- 触发可靠性
- 5.0
- 相对不装的提升
- 4.7
- 产出质量
- 5.0
- 令牌与时间成本
- 5.0
- 安装与可移植性
- 4.0
- 安全
- 4.0
安全审查
注意静态扫描：18 CRITICAL / 19 HIGH / 11 MEDIUM。人工审查：可运行但有需要注意的地方，见 Pro 区的安全报告。扫描没有命中不等于绝对安全。
安装
npx skills@latest add obra/superpowers --skill brainstorming
固定提交: 2ce8aa15d05bd57213e6f746f953aafb7ded7a8b
测试日期: 2026-11-02
我们测了什么
- Feature request — I want to add a dark mode to my web app.
- Non-software creative work — I'm giving a 10-minute talk at a local meetup about my first year as a solo founder. Help me put it together.
- Build request — Let's build a small CLI tool that tracks my expenses.
优点
- 平均分从不装时的 2.33 提到 5.0（+2.67）。
- 不点名也会被触发：Skill file read in 6/6 treatment runs.
- 不增加 token 开销：约为不装 skill 时的 0.82 倍。
- 比把它的规则压成一句话的做法高 +0.33。
- 装 skill 后绝对质量高：5.0 / 5。
缺点
- 每个任务每组 n=2，差距在 1 分以内属于噪声。
评测
在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。
三组对比
| 组别 | 平均分（0–5） | 每次 token |
|---|---|---|
| 不装 skill | 2.33 | 51k |
| 一句话提示 | 4.67 | 45k |
| 装了 skill | 5.0 | 42k |
Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 27.7 / 30，结论 推荐。
下面是评测原文，英文。
Verdict: Recommended (27.7 / 30). Source: obra/superpowers, skills/brainstorming, pinned at commit 2ce8aa1. Install: npx skills@latest add obra/superpowers --skill brainstorming. Licence: MIT (repo LICENSE).
What it is
The brainstorming skill from obra/superpowers: 'you MUST use this before any creative work', in software or out of it. About 2,400 words of Markdown: read the context, ask the human what they actually want and why (one question at a time), propose 2-3 approaches with trade-offs, get a design approved before anything is built, plus an optional visual-companion server (a small local web server for showing mock-ups) and a reviewer prompt. Also a few helper scripts for that server.
How we tested
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).
The prompts are one-line, vague requests with an empty folder ('add a dark mode to my web app', 'help me put together a talk', 'let's build a small CLI that tracks my expenses'). The check is the process: did the agent stop and ask 1-3 questions in a short reply and write no code?
- Feature request — I want to add a dark mode to my web app.
- Non-software creative work — I'm giving a 10-minute talk at a local meetup about my first year as a solo founder. Help me put it together.
- Build request — Let's build a small CLI tool that tracks my expenses.
Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.
Results
The one-liner given to the third group: "Before building anything, read the existing context, ask one clarifying question about what they actually want and why, then lay out 2-3 approaches with trade-offs and a recommendation, and get approval of a short design before implementing."
| Task | Control score | One-liner | With skill | Control tokens | One-liner tokens | With-skill tokens | Skill read |
|---|---|---|---|---|---|---|---|
| t1-dark-mode | 3.0 | 4.0 | 5.0 | 64k | 52k | 51k | 2/2 |
| t2-talk | 3.0 | 5.0 | 5.0 | 29k | 37k | 32k | 2/2 |
| t3-cli | 1.0 | 5.0 | 5.0 | 57k | 44k | 40k | 2/2 |
Mean score: 2.33 without vs 5.0 with the skill (delta +2.67). Tokens per run: 51k vs 42k (0.82x); time 43s vs 21s. One-liner group: 4.67 (the skill's gain over the one-liner: +0.33; one-liner's own gain over no skill: +2.33); one-liner tokens 45k per run, so the skill costs 0.93x the one-liner.
What happened
- This is the clearest effect so far. Without the skill the agent asked no clarifying question in 5 of 6 runs; on the CLI request both control runs simply built the tool (4 files each, with tests) without a single question. With the skill every one of 6 runs asked 1-3 questions in a short reply and wrote nothing (mean 5.0 vs 2.33 out of 5).
- A one-sentence version of the skill's rule ('read context, ask one question about what they want and why, lay out 2-3 approaches, get approval before implementing') did almost as well: 4.67. In 5 of 6 runs it asked questions; once (dark-mode task) it went straight to a plan with no question.
- The skill was read in 6/6 runs and cut cost instead of raising it: 42k tokens per run against 51k without (0.82x), because the control builds things and the skill stops to ask. Against the one-liner it costs 0.93x.
- On the CLI request one skill reply also surfaced an 'opt out of the questions' line, quoting the skill's own rule to the user. That is the kind of behaviour you want to know about: the agent stops even when you are in a hurry.
- The skill's heavier parts (visual companion server, design-doc review loop, reviewer prompt) were never reached: the conversation ends at the first question, as designed.
Dimension scores
| Dimension | Score | Why |
|---|---|---|
| 1_output_quality | 5.0 | Mean score 5.00/5 with the skill (programmatic checks, see per-task table). |
| 2_lift_vs_no_skill | 4.67 | +2.67 points vs 2.33 without the skill (n=2 per task; under ~1 point is noise). |
| 3_trigger | 5.0 | Skill file read in 6/6 treatment runs. |
| 4_instructions_portability | 4 | Written for Claude Code-style skills; the visual companion needs node and a browser; the rest is plain Markdown. |
| 5_efficiency | 5 | 0.82x tokens (42k vs 51k per run) and 0.49x time (21s vs 43s). |
| 6_safety_hygiene | 4 | Markdown plus a local visual-companion server (node, binds 127.0.0.1, only started with user consent); its page loads a version-tagged logo image from the author's domain (a telemetry beacon) unless SUPERPOWERS_DISABLE_TELEMETRY=1. We set the variable and never started the server. |
Limits of this test
- We only test the first turn. We did not answer the questions and follow the skill through design approval and implementation, so we cannot say whether the full workflow produces better software.
- The behaviour is a trade-off, not an obvious win: if you want the agent to just build, a skill that insists on asking first costs you a round trip.
- Three vague prompts; a detailed specification may or may not trigger the same stop.
- n = 2 per group per task: differences under about 1 point are noise.
- Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.
Security
See security.md. Scan: 18 CRITICAL / 19 HIGH / 11 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.
Bottom line
For vague 'build me X' requests the skill reliably changes behaviour from building immediately to asking one good question, and it does so cheaply. But the same behaviour is reproduced by one sentence in your prompt, so the skill's value is that you do not have to remember to ask. Recommended if you want your agent to clarify before building; skip it if you prefer it to act first.
记录
Run log for brainstorming: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run. Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened. t1-dark-mode control-1: score 3.0, tokens 63726, secs 40.4, checks 2/3, opened nothing t1-dark-mode control-2: score 3.0, tokens 65953, secs 33.4, checks 2/3, opened nothing t1-dark-mode oneliner-1: score 5.0, tokens 44490, secs 20.0, checks 3/3, opened nothing t1-dark-mode oneliner-2: score 3.0, tokens 59849, secs 30.6, checks 2/3, opened nothing t1-dark-mode treatment-1: score 5.0, tokens 51573, secs 23.4, checks 3/3, opened brainstorming t1-dark-mode treatment-2: score 5.0, tokens 51475, secs 16.2, checks 3/3, opened brainstorming t2-talk control-1: score 4.0, tokens 29770, secs 32.1, checks 2/3, opened nothing t2-talk control-2: score 2.0, tokens 29728, secs 33.5, checks 1/3, opened nothing t2-talk oneliner-1: score 5.0, tokens 44545, secs 20.1, checks 3/3, opened nothing
18 次运行、3 个产物
样例产物
前沿 · 装了 skill：final message（t1-dark-mode，第 1 次，Codex）
I don’t see any app files in this workspace yet. Where is your web app’s source code? Once it’s available, I can inspect its styling. If you’d prefer I just dive in, say so; I’d default to a Light / Dark / System setting that remembers the user’s choice.