pdf (Anthropic)

评测

在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。

三组对比

Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 20.6 / 30，结论 视情况而定。

下面是评测原文，英文。

Verdict: Situational (20.3 / 30). Source: anthropics/skills, skills/pdf, pinned at commit ca45fc4. Install: npx skills@latest add anthropics/skills --skill pdf. Licence: proprietary (use is governed by your agreement with Anthropic; this is not open source).

What it is

A guide for PDF work: read and extract text and tables, merge, split, rotate, watermark, create new PDFs, fill forms, encrypt, extract images, OCR scanned pages. SKILL.md covers the basics with Python snippets (pypdf, pdfplumber, reportlab) and command-line tools (pdftotext, qpdf). Two more files (forms.md, reference.md) and eight helper scripts handle forms and bounding boxes.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning), a fresh temporary project per run, skill installed or absent, two runs per group, network off. Inputs were PDFs we generated, so we could check the results with code:

1. Table extraction. Pull a 12-row table that spans two pages (header repeated on page two) into sales.csv and report the total revenue.
2. Merge, rotate, split. Combine two PDFs, rotate only the last page 90 degrees, save page one as cover.pdf, leave the originals alone.
3. Fill a form. Set three fields and tick a checkbox in a fillable PDF, keep fields editable.

The prompts never mentioned the skill.

What happened

It triggers reliably. The agent read SKILL.md in all six treatment runs, unprompted.

It made no difference to the outcome. Task 1 (2 of 2 checks) and task 2 (6 of 6) passed in all four runs of each task, with and without the skill. Task 3 failed once in each group, so both groups passed 5 of 6 runs overall.

Both failures were the same, and both agents said they had succeeded. The checkbox was saved as the text value Yes while its visual state stayed /Off. A PDF viewer shows it unticked, and the final messages in those two runs said the box was ticked. One failure was a control run, the other a treatment run. The treatment agent had read FORMS.md, which tells it to inspect field types and use each checkbox's checked_value, but it never got that far (next paragraph), so the skill did not prevent the error.

The scripts did not run as documented. FORMS.md says to run python scripts/check_fillable_fields .... macOS has no python, only python3, and the stock environment has no pypdf. The two treatment runs then diverged. One searched the disk, found pypdf inside the user's uv cache, ran the skill's own scripts through PYTHONPATH, and filled the form correctly. The other tried pip install (it failed offline), then wrote a Swift/PDFKit program and got the checkbox wrong. The control runs improvised in the same ways (Swift in one, pypdf in the other). So the skill's tooling only helped in the one run that managed to borrow a library from outside the project directory.

It doubles the cost. Treatment runs averaged 228,000 tokens against 116,000 without (1.98x) and took 72 seconds against 46. The agent reads the skill, then forms.md, then tries the scripts first. On the form task the treatment run that succeeded needed 35 commands and 434,000 tokens; the control runs needed 16 commands and about 185,000.

Portability notes

• SKILL.md refers to FORMS.md and REFERENCE.md, but the files are forms.md and reference.md. This worked in our runs because macOS file systems ignore case; on Linux those links would break.
• It assumes python, and several Python libraries, are installed. A clean machine usually has neither.
• The guide mentions pip install pytesseract pdf2image for OCR, which needs network access.

Safety review

We ran the scanner and then read SKILL.md, forms.md, reference.md and all eight scripts. The scripts import only standard-library modules plus pypdf, pdfplumber, pdf2image and PIL; they read an input file and write local files. No subprocess calls, sockets, environment reads, eval or downloads. No instructions to hide anything or to touch credentials. The finding worth flagging is the licence: it is proprietary, so the site should not label it open source. See security.md.

Scores

Who should use it

Modern agents already merge, rotate, split and extract tables from clean PDFs without help, as our control group showed, so for these jobs the skill mostly adds tokens. It might earn its keep on harder cases we did not test: scanned documents and OCR, password-protected files, forms without fillable fields (the bounding-box workflow), or an environment where the helper libraries are installed so the scripts work. If you install it, make sure python and pypdf exist first, or the guidance will not run as written.

Limits of this test

Two runs per group, three tasks, one model at low reasoning effort, offline sandbox with no Python PDF libraries in the stock environment. Runs shared the system temp folder, and one treatment run reused a virtual environment another run had created there, so isolation between runs was not perfect. A fully equipped machine could change the result in the skill's favour. The Codex tier counts for Codex and Claude Code under the site's rule.

Update (2 Nov 2026): one-sentence group added

Third group: no skill, with one sentence ("Use proper PDF tooling: extract tables by page with their header rows intact, merge/rotate/split pages without losing content, fill real form fields by name and keep the original file unchanged, then verify the output by reopening it."). 6 runs, rule-based checks, run on a Windows machine with a vendored pypdf/pdfplumber set available offline.

All six one-sentence runs passed every check, including the checkbox form fill that failed once in the control and once in the skill group. The skill cost 3.3x the one-sentence tokens. Caveat: the one-sentence runs ran on a different machine (Windows) from the first two groups (macOS) and had the PDF libraries available, which the Mac runs did not, so this comparison is confounded by environment. It does suggest the skill's 2x token overhead buys nothing on these three tasks.