pptx (anthropics/skills)
pptx：对照组 4.99、一句话提示 4.86、装 skill 5.0（0–5 分，Codex 实测，n=2/任务/组）。结论：视情况而定，相对不装 skill +0.01。
24.0 / 30
相对不装 skill 的提升：+0.01
视情况而定：提升 +0.01 低于 +1.0。推荐要求总分 22/30、产出质量 3.5、提升 +1.0。
前沿
通过使用 Codex 测试。
覆盖 Codex, Claude Code
Codex CLI 0.166（SOTA 档）：17 次评分运行。按我们的分档规则，这档通过同时覆盖 Claude Code；它不代表第二档的结果。
分数
每一维 0 到 5 分，六维相加是满分 30 的总分。每一维都是 5 更好。成本 5 表示又便宜又快。安装 5 表示容易。
- 触发可靠性
- 5.0
- 相对不装的提升
- 2.0
- 产出质量
- 5.0
- 令牌与时间成本
- 3.0
- 安装与可移植性
- 4.0
- 安全
- 5.0
安全审查
安全静态扫描：0 CRITICAL / 1 HIGH / 6 MEDIUM。人工审查：可在断网沙箱中运行。扫描没有命中不等于绝对安全。
安装
npx skills@latest add anthropics/skills --skill pptx
固定提交: ca45fc496a3436251f191354091d557da31c15b5
测试日期: 2026-11-02
我们测了什么
- Create a pitch deck — Create pitch.pptx: a 5-slide 16:9 deck for the launch of a fictional product called Falcon: a title slide, a Problem slide, a Solution slide, a Traction slide with a bar chart of quarterly revenue (Q1 12, Q2 18, Q3 27, Q4 35, in $k), and a closing Ask slide. E…
- Insert and retitle slides — In team.pptx change the title of slide 2 to 'Revenue outlook' and insert a new slide right after it titled 'Risks' with three bullet points (supply delays, hiring, currency). Keep every other slide and its speaker notes unchanged. Save the result as team.pptx…
- Extract outline and table — From team.pptx write outline.txt (one slide title per line, in order) and table.csv (the table on the results slide, header row first). Do not modify team.pptx.
优点
- 不点名也会被触发：Skill file read in 6/6 treatment runs.
- 装 skill 后绝对质量高：5.0 / 5。
缺点
- 相对不装 skill 没有可测量的提升（+0.01，在噪声范围内）。
- 对照组已经 4.99 分：触到天花板，这个测试看不出提升。
- 测试说明：计划的 18 次里只完成 17 次：t3 的一句话提示组 n=1。测试机上 LibreOffice 反复崩溃，批次因此中止。
- 每个任务每组 n=2，差距在 1 分以内属于噪声。
评测
在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 17 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。
三组对比
| 组别 | 平均分（0–5） | 每次 token |
|---|---|---|
| 不装 skill | 4.99 | 114k |
| 一句话提示 | 4.86 | 224k |
| 装了 skill | 5.0 | 162k |
Codex CLI 实测，17 次运行，规则脚本打分，没有 LLM 评委。总分 24.0 / 30，结论 视情况而定。
测试披露： 计划的 18 次里只完成 17 次：t3 的一句话提示组 n=1。测试机上 LibreOffice 反复崩溃，批次因此中止。
下面是评测原文，英文。
Verdict: Situational (24.0 / 30). Source: anthropics/skills, skills/pptx, pinned at commit ca45fc4. Install: npx skills@latest add anthropics/skills --skill pptx. Licence: Proprietary (LICENSE.txt in skill dir): evaluated read-only, not redistributed.
What it is
The pptx skill from anthropics/skills: create decks with pptxgenjs (with a list of file-corrupting gotchas), edit decks by unzipping and editing slide XML with helper scripts (add_slide.py, clean.py, thumbnail.py), read content with markitdown, and validate with scripts/office/validate.py and LibreOffice. About 3,100 words in SKILL.md plus Python scripts and OOXML schemas; proprietary licence, evaluated read-only.
How we tested
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).
Three tasks: create a 5-slide 16:9 pitch deck (speaker notes on every slide, a native editable chart, real bullets, everything on the slide, opens in LibreOffice); in team.pptx retitle slide 2 and insert a 'Risks' slide with three bullets after it, keeping the other slides, the table and the speaker notes; extract the slide titles and the table to text files without modifying the deck. Both arms had the same offline copies of pptxgenjs and python-pptx. Checks use python-pptx plus a LibreOffice conversion done offline on our side. Run with 17 of 18 planned runs: the batch was stopped early (see limits), so the one-sentence group has one run fewer on the extraction task.
- Create a pitch deck — Create pitch.pptx: a 5-slide 16:9 deck for the launch of a fictional product called Falcon: a title slide, a Problem slide, a Solution slide, a Traction slide with a bar chart of quarterly revenue (Q1 12, Q2 18, Q3 27, Q4 35, in $k), and a closing Ask slide. E…
- Insert and retitle slides — In team.pptx change the title of slide 2 to 'Revenue outlook' and insert a new slide right after it titled 'Risks' with three bullet points (supply delays, hiring, currency). Keep every other slide and its speaker notes unchanged. Save the result as team.pptx…
- Extract outline and table — From team.pptx write outline.txt (one slide title per line, in order) and table.csv (the table on the results slide, header row first). Do not modify team.pptx.
Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.
Results
The one-liner given to the third group: "Build the deck as real PowerPoint objects: set the 16:9 layout first, use a native editable chart (not a picture), real bullets (never typed '•' characters), speaker notes through the notes API, keep every shape inside the slide, and confirm the file opens by converting it to PDF; when editing, change only what was asked, keep the other slides and their notes as they are, and leave no working files."
| Task | Control score | One-liner | With skill | Control tokens | One-liner tokens | With-skill tokens | Skill read |
|---|---|---|---|---|---|---|---|
| t1-new | 4.96 | 5.0 | 5.0 | 147k | 342k | 185k | 2/2 |
| t2-edit | 5.0 | 4.65 | 5.0 | 114k | 175k | 199k | 2/2 |
| t3-extract | 5.0 | 5.0 | 5.0 | 79k | 80k | 99k | 2/2 |
Mean score: 4.99 without vs 5.0 with the skill (delta +0.01). Tokens per run: 114k vs 162k (1.42x); time 67s vs 70s. One-liner group: 4.86 (the skill's gain over the one-liner: +0.14; one-liner's own gain over no skill: -0.12); one-liner tokens 224k per run, so the skill costs 0.72x the one-liner.
What happened
- No difference in score: 4.99 control, 4.86 one-sentence, 5.00 skill. Control already creates correct decks (5 slides, notes on every slide, a native chart, no typed bullets, shapes inside the slide, opens in LibreOffice) and does the edit with python-pptx without losing notes or the table.
- The skill cost more: 162k tokens per run against 114k (1.4x), and the one-sentence version 224k (2.0x), mostly on the create task (185k for the skill and 325-361k for the one-sentence version, against 118-176k for control).
- One failure, in the one-sentence group: its edited team.pptx opens in python-pptx but LibreOffice cannot convert it (reproduced twice on our side); the agent itself reported that LibreOffice crashed. The skill group's six decks all converted, and all six skill runs ran its validate.py.
- The skill was read in 6/6 runs, and in all 6 the agent used its markitdown, validate.py and LibreOffice wrapper (soffice.py). Control never called LibreOffice; 4 of 5 one-sentence runs called soffice directly.
Dimension scores
| Dimension | Score | Why |
|---|---|---|
| 1_output_quality | 5.0 | Mean score 5.00/5 with the skill (programmatic checks, see per-task table). |
| 2_lift_vs_no_skill | 2.01 | +0.01 points vs 4.99 without the skill (n=2 per task; under ~1 point is noise). |
| 3_trigger | 5.0 | Skill file read in 6/6 treatment runs. |
| 4_instructions_portability | 4 | Assumes pptxgenjs, python-pptx, markitdown and LibreOffice are installed; Linux-sandbox oriented. |
| 5_efficiency | 3 | 1.42x tokens (162k vs 114k per run) and 1.04x time (70s vs 67s). |
| 6_safety_hygiene | 5 | Scripts read: soffice.py (LibreOffice launcher; compiles a small LD_PRELOAD shim only on Linux when sockets are blocked), thumbnail.py, add_slide.py, clean.py and validators; no network, no credential access. |
Limits of this test
- The batch was stopped at 17 of 18 runs: LibreOffice crashed repeatedly on the test computer while agents (mostly in the skill and one-sentence groups) converted decks to PDF, and the user asked us to stop. All results come from runs that finished; one one-sentence extraction run is missing.
- Rule checks (structure, notes, chart, bounds, opens); visual quality of the slides was not judged, and the design guidance in the skill (colour palettes, layout ideas) was not exercised.
- Small decks; 5 slides and 3 slides. Template-driven editing, combine/split and comments were not tested.
- Conversions in the checks were run offline on our side; the agents' own LibreOffice use on the test computer is what caused the crashes.
- n = 2 per group per task: differences under about 1 point are noise.
- Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.
Security
See security.md. Scan: 0 CRITICAL / 1 HIGH / 6 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.
Bottom line
On small decks plain Codex with python-pptx already produced correct, valid files, so the skill's gotchas and validator added about 40% tokens without raising the score. Its validator and conversion steps are useful for larger or template-based decks, which this test did not cover. Be aware that the skill's own workflow calls LibreOffice, which can be unstable on some desktops.
记录
Run log for pptx: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run. Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened. t1-new control-1: score 4.92, tokens 117727, secs 118.4, checks 1/2, opened nothing t1-new control-2: score 5.0, tokens 176319, secs 129.0, checks 2/2, opened nothing t1-new oneliner-1: score 5.0, tokens 324776, secs 128.1, checks 2/2, opened nothing t1-new oneliner-2: score 5.0, tokens 360676, secs 134.3, checks 2/2, opened nothing t1-new treatment-1: score 5.0, tokens 183150, secs 123.4, checks 2/2, opened pptx t1-new treatment-2: score 5.0, tokens 187890, secs 110.7, checks 2/2, opened pptx t2-edit control-1: score 5.0, tokens 115865, secs 43.4, checks 2/2, opened nothing t2-edit control-2: score 5.0, tokens 113982, secs 42.2, checks 2/2, opened nothing t2-edit oneliner-1: score 4.31, tokens 147330, secs 67.2, checks 1/2, opened nothing
17 次运行、3 个产物
样例产物
前沿 · 装了 skill：create_pitch.js（t1-new，第 1 次，Codex）
const pptxgen=require('pptxgenjs'); const p=new pptxgen();p.layout='LAYOUT_WIDE';p.title='Falcon launch';p.author='Falcon';p.theme={headFontFace:'Arial',bodyFontFace:'Arial',lang:'en-US'}; const dark='192620',green='C2F36B',muted='56665C'; function t(s,text,x,y,w,h,size=20,color=dark,bold=false){s.addText(text,{x,y,w,h,fontSize:size,fontFace:'Arial',color,bold,margin:0});} function base(n,d=false){const s=p.addSlide();s.background={color:d?dark:'FFFFFF'};t(s,'FALCON / LAUNCH',.65,.35,6,.25,10,d?green:muted,true);t(s,String(n).padStart(2,'0'),12,6.9,.6,.25,10,d?'FFFFFF':muted);return s;} function list(s,items,x,y,w,h,color=dark,size=22){s.addText(items.map((text,i)=>({text,options:{bullet:{indent:size},hanging:4,breakLine:i<items.length-1}})),{x,y,w,h,fontFace:'Arial',fontSize:size,color,margin:0,paraSpaceAfter:23,valign:'top'});} function box(s,x,y,w,h,color){s.addShape(p.ShapeType.roundRect,{x,y,w,h,fill:{color},line:{color,transparency:100},radius:.15});} let s=base(1,true);t(s,'Falcon',.75,1.45,8,1.1,66,'FFFFFF',true);t(s,'Turn team plans into\nforward motion.',.75,2.85,8,1.5,34,'FFFFFF');t(s,'One workspace for priorities, owners, and progress.',.75,4.85,8,.7,20,'B9C8BE');s.addShape(p.ShapeType.chevron,{x:10,y:1.8,w:1.2,h:3.3,fill:{color:green},line:{color:green}});s.addShape(p.ShapeType.chevron,{x:11.4,y:2.5,w:.6,h:1.9,fill:{color:'71964A'},line:{color:'71964A'}});t(s,'Fictional product • Illustrative launch pitch',.75,6.5,8,.3,11,'B9C8BE');s.addNotes('Introduce Falcon as a fictional work-management product for growing teams. Its promise is to turn plans into coordinated action through shared priorities, owners, and progress. All business details are illustrative; quarterly revenue was supplied for this pitch.'); s=base(2);t(s,'Problem',.65,1,10,.65,40,dark,true);t(s,'Work moves faster than the tools tracking it.',.65,1.85,12,.6,25,muted);list(s,['Priorities scatter across chats, docs, and spreadsheets.','Unclear ownership delays decisions and handoffs.','Manual updates hide risks until deadlines slip.'],.8,3,7.1,3);box(s,9,2.85,3.55,3.15,'F0F5F1');t(s,'The cost',9.35,3.2,2.8,.4,20,muted,true);t(s,'More chasing.\nLess shipping.',9.35,3.95,2.8,1.3,30,dark,true);s.addNotes('Growing teams often keep priorities in multiple places, leaving owners and dependencies unclear. Explain how this creates time spent chasing updates. These are product assumptions for a fictional launch, not research findings.'); s=base(3);t(s,'Solution',.65,1,10,.65,40,dark,true);t(s,'One shared view. A clear next step.',.65,1.85,12,.6,25,muted);[['01','Align priorities',['Capture goals in one place.','Link tasks to outcomes.']],['02','Assign owners',['Give every task an owner.','Make deadlines visible.']],['03','Advance work',['Track progress together.','Flag blockers early.']]].forEach((a,i)=>{let x=.65+i*4.1;box(s,x,2.9,3.8,3.3,'F0F5F1');t(s,a[0],x+.3,3.2,1,.5,28,'638831',true);t(s,a[1],x+.3,4,3.2,.6,23,dark,true);list(s,a[2],x+.3,4.9,3.1,1.1,dark,16);});s.addNotes('Walk through the proposed workflow: align on priorities, assign owners, and advance work with visible progress and blockers. These are conceptual features of the fictional Falcon product. Emphasize the va [... excerpt: first 3200 of 5381 characters]