SEO GitHub Action: Fail the Build on Broken Links

The expensive SEO mistakes are rarely clever. A template change drops the title tag from every product page. A redirect rule gets a second hop. Someone ships the staging noindex to production. None of these are hard to find with a crawler; the problem is that the crawl happens after the release, when the damage is already in the index. crawlcove-action moves that crawl to the pull request, where a regression is a red check instead of a lost week of traffic. This is how to set it up, and how to choose what should fail.

The three-line version

The action wraps the Crawl Cove CLI. The smallest useful workflow step is:

- uses: CrawlCove/crawlcove-action@v1
  with:
    url: https://preview-123.example.com/

That crawls up to 100 same-origin pages from the URL, fails the check on the first broken internal link, missing title, noindex page or two-hop redirect chain, and posts one comment on the pull request that it updates on every run rather than duplicating. The comment looks like this:

CrawlCove SEO crawl: failed Crawled 42 pages from https://preview-123.example.com/. The gated checks total 3, threshold 1.

Check	Count	Fails the build
Broken links	2	yes
Missing titles	1	yes
Noindex pages	0	yes
Redirect chains (2+ hops)	0	yes

The broken links and missing titles expand below the table, as the page that links and the page that fails, so the fix is one click away rather than a search.

The comment needs permissions: pull-requests: write on the job. Everything else works with the default token.

Getting a URL to crawl

The action is only as useful as the URL you give it, and there are three common shapes.

A hosted preview (Vercel, Netlify, Cloudflare Pages and similar). The host's GitHub integration sends a deployment_status event when a preview is up, and the event carries the URL. No secrets, no waiting loop:

name: SEO crawl (preview)

on:
  deployment_status:

permissions:
  contents: read
  pull-requests: write

jobs:
  crawl:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: CrawlCove/crawlcove-action@v1
        with:
          url: ${{ github.event.deployment_status.environment_url }}
          max-pages: 200
          fail-on: broken-links,missing-titles,redirect-chains

Note the fail-on list: noindex is left out, because preview deployments are usually noindex on purpose. More on that below.

A site built inside the job. For a Next.js site, or anything that can build and serve on localhost, there is no external host to wait for. Build, serve, wait for the port, crawl:

on:
  pull_request:

jobs:
  crawl:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run build
      - name: Serve the build
        run: |
          (npx next start -p 3000 > /dev/null 2>&1 &)
          for i in $(seq 1 30); do curl -fs http://127.0.0.1:3000/ > /dev/null && break; sleep 1; done
      - uses: CrawlCove/crawlcove-action@v1
        with:
          url: http://127.0.0.1:3000/
          max-pages: 300
          ignore-robots: 'true'

ignore-robots is set because a local build's robots.txt usually blocks everything, and a crawler that respects it would fetch nothing. Which brings up the third shape.

A staging site on a schedule. A WordPress staging install does not have pull requests, but it has a URL and a cron schedule. The repo's wordpress-staging.yml example runs the same crawl on a timer with comment off, so a nightly regression shows up as a failed workflow run rather than a PR comment.

Choosing what fails

Four checks are gated by default, and the right list depends on what the URL is.

• broken-links counts every link to a 4xx, 5xx or unreachable page, plus each such page itself. This is the one to keep on everywhere. A broken internal link is a regression on any environment.
• missing-titles is a page with no <title> at all. Also keep on. It is almost always a template bug, and template bugs multiply.
• noindex is a page carrying a robots noindex meta tag. On production this is the check that catches the shipped staging tag. On a preview it is noise, because the preview is meant to be noindex. Set fail-on per environment rather than switching the check off globally.
• redirect-chains counts URLs that went through two or more redirects. A single redirect is normal and is not flagged. Chains cost crawl budget and leak link equity, and they tend to appear exactly when a URL structure changes, which is exactly when a PR is open.

threshold sets how many findings across the selected checks it takes to fail, and defaults to 1. none for fail-on reports without ever failing, which is a reasonable first week while you see what the crawl finds on your site.

Reading the result from later steps

Every count is an output, so a later step can act on it without parsing the comment:

- uses: CrawlCove/crawlcove-action@v1
  id: seo
  with:
    url: ${{ github.event.deployment_status.environment_url }}
- run: echo "Broken links: ${{ steps.seo.outputs.broken-links }}"
- uses: actions/upload-artifact@v4
  with:
    name: seo-crawl
    path: ${{ steps.seo.outputs.report-path }}

report-path points at the full JSON report, one object per page, with field names that follow the Crawl Cove export spec. Anything that reads a Crawl Cove export reads this file.

exit-code is worth knowing too: 0 is clean, 1 is the gated checks reaching the threshold, and 2 means the crawl could not run, most often because robots.txt disallowed the start URL. The action deliberately reports 2 as a failure with a "could not run" summary rather than as a pass, because a crawl that fetched nothing has proved nothing.

What a CI crawl is not

A check on a pull request catches regressions on the pages the preview happens to have. It does not tell you which of the findings across a whole site matter most, it keeps no history between runs, and it knows nothing about how the pages perform in search. That is the audit, and it is a different job. When the check goes red and you want the whole picture, open the same site in the Crawl Cove desktop crawler: every page, every finding ranked by impact, history over time and Search Console data alongside. The action is the tripwire; the app is the investigation.

Related

• Run an SEO audit from the terminal covers the CLI the action runs, including cron crawls and the robots.txt rules.
• Redirect chains and loops explains why the two-hop rule is the right cut-off.
• Site migration SEO checklist is the manual version of what this action automates on the day a URL structure changes.