seo-audit (coreyhaines31/marketingskills)
seo-audit：对照组 4.95、一句话提示 4.83、装 skill 4.84（0–5 分，Codex 实测，n=2/任务/组）。结论：视情况而定，相对不装 skill -0.11。
24.7 / 30
相对不装 skill 的提升：-0.11
视情况而定：提升 -0.11 低于 +1.0。推荐要求总分 22/30、产出质量 3.5、提升 +1.0。
前沿
通过使用 Codex 测试。
覆盖 Codex, Claude Code
Codex CLI 0.166（SOTA 档）：18 次评分运行。按我们的分档规则，这档通过同时覆盖 Claude Code；它不代表第二档的结果。
分数
每一维 0 到 5 分，六维相加是满分 30 的总分。每一维都是 5 更好。成本 5 表示又便宜又快。安装 5 表示容易。
- 触发可靠性
- 5.0
- 相对不装的提升
- 1.9
- 产出质量
- 4.8
- 令牌与时间成本
- 4.0
- 安装与可移植性
- 4.0
- 安全
- 5.0
安全审查
安全静态扫描：0 CRITICAL / 2 HIGH / 0 MEDIUM。人工审查：可在断网沙箱中运行。扫描没有命中不等于绝对安全。
安装
npx skills@latest add coreyhaines31/marketingskills --skill seo-audit
固定提交: 88c79f6528a9347420e57ceb1fff9a60de651768
测试日期: 2026-11-02
我们测了什么
- Audit a small static site — Audit the SEO of the static site in this folder (index.html, robots.txt, sitemap.xml; domain https://tidepool.example). Write the findings to audit.md, ordered by severity. Don't change the site files.
- Fix the on-page issues — Fix the on-page SEO problems in index.html and robots.txt (domain https://tidepool.example). Edit the files in place; keep the content otherwise the same.
- Top 5 for a non-technical owner — I'm not technical. Look at the site in this folder and tell me the 5 most important SEO fixes, ranked by impact, in plain language. Do not change any files.
优点
- 不点名也会被触发：Skill file read in 6/6 treatment runs.
- 装 skill 后绝对质量高：4.84 / 5。
缺点
- 相对不装 skill 没有可测量的提升（-0.11，在噪声范围内）。
- 一句话提示得 4.83，装 skill 得 4.84（差 +0.01）。
- 对照组已经 4.95 分：触到天花板，这个测试看不出提升。
- 每个任务每组 n=2，差距在 1 分以内属于噪声。
评测
在 Codex CLI（SOTA 档）上跑 3 个任务、对照、一句话提示、装 skill 三组，共 18 次运行，由规则脚本打分（没有 LLM 评委），断网沙箱、每次运行独立临时目录。具体做法见下方英文评测原文。
三组对比
| 组别 | 平均分（0–5） | 每次 token |
|---|---|---|
| 不装 skill | 4.95 | 88k |
| 一句话提示 | 4.83 | 96k |
| 装了 skill | 4.84 | 108k |
Codex CLI 实测，18 次运行，规则脚本打分，没有 LLM 评委。总分 24.7 / 30，结论 视情况而定。
下面是评测原文，英文。
Verdict: Situational (24.7 / 30). Source: coreyhaines31/marketingskills, skills/seo-audit, pinned at commit 88c79f6. Install: npx skills@latest add coreyhaines31/marketingskills --skill seo-audit. Licence: MIT (repo LICENSE).
What it is
The seo-audit skill (v2.2.2) from coreyhaines31/marketingskills. An audit framework for 'my SEO is bad', 'why am I not ranking', technical SEO and meta-tag reviews: check for a product-marketing context file, audit order (crawlability and indexation, technical foundations, on-page, content quality, authority), severity-ranked output, with seven reference files (title tags, international SEO, local SEO, rankings push, AI-writing detection and more). It also says fetched pages are untrusted data. Markdown only.
How we tested
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).
The test site is a small static page set (index.html, robots.txt, sitemap.xml for the fictional domain tidepool.example) with 12 planted issues for the audit task and 9 on-page issues for the fix task, including a Disallow: / robots file and a noindex meta tag. No network, so nothing was fetched.
- Audit a small static site — Audit the SEO of the static site in this folder (index.html, robots.txt, sitemap.xml; domain https://tidepool.example). Write the findings to audit.md, ordered by severity. Don't change the site files.
- Fix the on-page issues — Fix the on-page SEO problems in index.html and robots.txt (domain https://tidepool.example). Edit the files in place; keep the content otherwise the same.
- Top 5 for a non-technical owner — I'm not technical. Look at the site in this folder and tell me the 5 most important SEO fixes, ranked by impact, in plain language. Do not change any files.
Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.
Results
The one-liner given to the third group: "Audit crawlability and indexation (robots.txt, sitemap, canonicals), on-page elements (title, meta description, headings, image alt text, internal links) and structured data, and report findings by severity with a concrete fix for each."
| Task | Control score | One-liner | With skill | Control tokens | One-liner tokens | With-skill tokens | Skill read |
|---|---|---|---|---|---|---|---|
| t1-audit | 4.86 | 4.86 | 4.72 | 105k | 128k | 125k | 2/2 |
| t2-fix | 5.0 | 4.63 | 4.81 | 94k | 94k | 120k | 2/2 |
| t3-priorities | 5.0 | 5.0 | 5.0 | 64k | 63k | 78k | 2/2 |
Mean score: 4.95 without vs 4.84 with the skill (delta -0.11). Tokens per run: 88k vs 108k (1.23x); time 60s vs 62s. One-liner group: 4.83 (the skill's gain over the one-liner: +0.01; one-liner's own gain over no skill: -0.12); one-liner tokens 96k per run, so the skill costs 1.13x the one-liner.
What happened
- No quality gain. Mean score 4.95 without the skill, 4.83 with only a one-sentence version of its audit order, 4.84 with the skill. All three groups reached the planted-issue threshold in every run and the same top finding (robots.txt blocking everything plus the noindex tag) came first.
- In the audit task, planted-issue recall was 11 of 12 in 5 of 6 runs in every group; the missed item varied (missing image alt text, missing viewport tag, thin content). Both skill runs missed 'thin content', the one issue the control got in a run.
- In the fix task, both control runs fixed 9/9 on-page issues; the one-liner runs fixed 8/9 (missed the viewport tag) and the skill runs 8/9 once ('click here' link text left) and 9/9 once. Differences of one issue are noise at n=2.
- The skill was read in 6/6 treatment runs. It cost 108k tokens per run against 88k without it (1.23x) and 96k with the one-liner (1.13x), the smallest overhead of the skills in this round.
- For the non-technical 'top 5 fixes' task all groups scored 5/5 and all put the robots/noindex problem first.
Dimension scores
| Dimension | Score | Why |
|---|---|---|
| 1_output_quality | 4.84 | Mean score 4.84/5 with the skill (programmatic checks, see per-task table). |
| 2_lift_vs_no_skill | 1.89 | -0.11 points vs 4.95 without the skill (n=2 per task; under ~1 point is noise). |
| 3_trigger | 5.0 | Skill file read in 6/6 treatment runs. |
| 4_instructions_portability | 4 | Plain Markdown; optional product-marketing context file and sibling skills (programmatic-seo, schema, ai-seo) referenced but not required. |
| 5_efficiency | 4 | 1.23x tokens (108k vs 88k per run) and 1.03x time (62s vs 60s). |
| 6_safety_hygiene | 5 | Markdown only (scanner HIGH hits were a regex matching 'coast'); includes an explicit rule to treat fetched pages as untrusted data. |
Limits of this test
- A tiny static site with obvious planted problems: a competent agent finds them by reading three files. The skill's strengths (traffic-drop diagnosis, striking-distance keywords, local SEO, international/hreflang, crawl data) need real analytics or a live site and were not tested.
- No network: the skill's page-speed and live-crawl guidance could not run.
- Scoring counts planted issues found by regex; it does not judge prioritisation quality or how well a fix is explained.
- n = 2 per group per task: differences under about 1 point are noise.
- Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.
Security
See security.md. Scan: 0 CRITICAL / 2 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.
Bottom line
On a small site the skill does not beat plain Codex: both find the major problems and fix the on-page issues, and the skill costs about 1.2x the tokens. The skill is worth keeping for the references it carries (title-tag, local and international SEO) on messier real sites, but this test gives no evidence it makes a basic audit better.
记录
Run log for seo-audit: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run. Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened. t1-audit control-1: score 4.72, tokens 96625, secs 90.7, checks 3/3, opened nothing t1-audit control-2: score 5.0, tokens 114606, secs 114.4, checks 3/3, opened nothing t1-audit oneliner-1: score 4.72, tokens 119742, secs 95.1, checks 3/3, opened nothing t1-audit oneliner-2: score 5.0, tokens 137601, secs 93.1, checks 3/3, opened nothing t1-audit treatment-1: score 4.72, tokens 139627, secs 111.9, checks 3/3, opened seo-audit t1-audit treatment-2: score 4.72, tokens 110467, secs 100.8, checks 3/3, opened seo-audit t2-fix control-1: score 5.0, tokens 94756, secs 37.3, checks 2/2, opened nothing t2-fix control-2: score 5.0, tokens 94428, secs 36.3, checks 2/2, opened nothing t2-fix oneliner-1: score 4.63, tokens 109171, secs 60.9, checks 2/2, opened nothing
18 次运行、3 个产物
样例产物
前沿 · 装了 skill：audit.md（t1-audit，第 1 次，Codex）
# Tidepool SEO audit Date: 2026-11-02. Intended origin: `https://tidepool.example`. ## Scope and summary Reviewed the complete local `index.html`, `robots.txt`, and `sitemap.xml`, plus README and the folder inventory. No site files were changed. Findings below are ordered by severity; line numbers refer to the original files. The site is not ready for organic search: all crawling is blocked, and the homepage separately requests exclusion from the index. Fix both together before metadata improvements. Next, align HTTPS URLs, repair the sitemap, and verify the missing resources and destinations. The homepage also needs clearer product information and mobile configuration. This is a source audit, not a live crawl. HTTP status codes, redirects, response headers, rendered appearance, search rankings, traffic, and Core Web Vitals were not measured. Missing local files do not prove production 404 responses. The supplied `.example` origin is treated as the audit's intended URL convention, not evidence of a publicly deployed site. Assumed search intent: monthly seaweed snack subscriptions in the United States, based on the supplied copy; keyword demand was not researched. ## Findings, ordered by severity ### 1. Critical — robots.txt blocks the entire site - **Evidence:** `robots.txt:1–2` contains `User-agent: *` and `Disallow: /`. - **Impact:** Compliant crawlers cannot fetch the homepage, its linked pages, or resources. A robots block does not reliably remove a URL from search: externally discovered URLs can still appear without crawled content. - **Recommendation:** For this public marketing site, remove the blanket disallow or replace it with an empty `Disallow:` directive. Coordinate this with finding 2. Verify crawl access after deployment. - **Basis:** [Google robots.txt guidance](https://developers.google.com/search/docs/crawling-indexing/robots/intro). ### 2. Critical — homepage explicitly requests no indexing - **Evidence:** `index.html:6`: `<meta name="robots" content="noindex, follow">`. - **Impact:** Once crawlers can retrieve this page, `noindex` makes it ineligible for indexing. `follow` does not cancel `noindex`. Currently the robots block prevents crawlers from reading this directive, so fixing only one blocker is insufficient. - **Recommendation:** Remove `noindex` from the public homepage; removing the entire tag is sufficient for normal default indexing behavior. After deployment, also check for an `X-Robots-Tag: noindex` response header and use Search Console URL Inspection. - **Basis:** [Google's noindex documentation](https://developers.google.com/search/docs/crawling-indexing/block-indexing). ### 3. High — inconsistent HTTPS and canonical signals - **Evidence:** Both sitemap locations use `http://tidepool.example` (`sitemap.xml:3–4`); the blog link also uses HTTP (`index.html:20`). The HTML head (`index.html:3–8`) contains no canonical link, while the supplied intended origin is HTTPS. - **Impact:** The sitemap and internal link promote a different protocol from the intended URL. If both protocols serve content, this can create duplicate URL discovery and ambiguous consolidation. Missing canonical markup alone is not [... excerpt: first 3200 of 12260 characters]