xlsx (anthropics/skills)

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、装 skill 两组，共 12 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

Codex CLI 实测，12 次运行，规则脚本打分，没有 LLM 评委。总分 20.3 / 30，结论 视情况而定。

下面是评测原文，英文。

Verdict: Situational (20.3 / 30). Source: anthropics/skills, skills/xlsx, pinned at commit ca45fc4. Install: npx skills@latest add anthropics/skills --skill xlsx. Licence: Proprietary (LICENSE.txt in skill dir): evaluated read-only, not redistributed.

What it is

The xlsx skill from Anthropic's skills repo: use openpyxl/pandas to build and edit Excel files, keep formulas live instead of pasting numbers, follow financial-model colour conventions, and run scripts/recalc.py (LibreOffice headless) to calculate formulas and scan for errors such as #REF!. SKILL.md says the tools are preinstalled.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

Environment: both groups got the same offline toolkit (python openpyxl/lxml/defusedxml, node docx, a python shim for python3), because the skill assumes it. LibreOffice is on the Mac but cannot create a profile inside the Codex sandbox.

1. Create a budget workbook — Create budget.xlsx with a sheet named 'Budget': header row Category, Q1, Q2, Q3, Q4, Total (bold, Arial font); at least five expense categories with sample numbers; the Total column and a Total row at the bottom must be live Excel formulas, not typed values.
2. Add a growth column — In sales.xlsx add a 'Growth %' column at the end of the Sales sheet (column F) showing each region's Q2 change relative to Q1 as a live formula. Keep the existing formulas and header formatting. Edit the file in place.
3. Summary sheet with SUMIF — In orders.xlsx add a 'Summary' sheet with the total Amount per region (North, South, East, West) as live formulas that reference the Orders sheet. Do not change the Orders sheet. Edit the file in place.

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

Mean score: 4.8 without vs 4.55 with the skill (delta -0.25). Tokens per run: 83k vs 120k (1.44x); time 44s vs 63s.

What happened

• Outcome was about the same: control 4.8 vs treatment 4.55 out of 5 (n=2, within noise). Both groups built correct live formulas (row and column totals, growth column, SUMIF per region) with correct recalculated values when we recalculated them ourselves.
• The skill's headline feature did not run. In 5 of the 6 treatment runs recalc.py was tried: the plain python call crashed with TypeError: ... 'ignore_cleanup_errors' (that argument needs Python 3.10+; the python3 the sandbox PATH resolves to is older), and after switching to Homebrew Python it stopped with "LibreOffice did not create a usable profile; formulas were NOT recalculated" inside the sandbox.
• The cost of finding that out: about 1.4x the tokens per run, and more commands (10-18 vs 4-6).
• A side effect of the workaround attempts: one treatment run left a .xlsx-tmp/ folder with a copy of the workbook in the project.
• The growth-column task lost the same point in all four runs: the file had no stored values for the new formulas (a viewer that does not recalculate would show blanks), because nobody could run a recalculation. In the SUMIF task three of four runs ended with stored values (the control runs wrote them into the XML themselves); one treatment run left none.
• In one treatment run the budget total row had no formulas (7 of 8 checks).

Dimension scores

Limits of this test

• Only three small workbook tasks; formatting conventions (blue inputs, black formulas), number formats and multi-sheet financial models were not scored.
• The failures are environment-driven (old system Python, LibreOffice in a sandbox). On a machine where recalc.py runs, the result could differ.
• Proprietary licence: behaviour described only.
• n = 2 per group per task: differences under about 1 point are noise.
• Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

In a Codex-style sandbox on a Mac the skill's recalculation script does not run, so you pay about 1.4x the tokens and get no better workbooks than without it. Its advice (live formulas, recalc, error scan) is sound, but the tooling needs Python 3.10+ and an unsandboxed LibreOffice.