Commit 37a633f44c6 for woocommerce
commit 37a633f44c636c0b7ad49920837b8873f310e568
Author: Vlad Olaru <vlad.olaru@automattic.com>
Date: Mon Sep 7 14:46:14 2026 +0300
[CI] Add external link checks for docs Markdown (#68421)
* ci: add pinned lychee installer and docs link-check policies
WooCommerce has no automated check for dead links in docs/**/*.md.
PR #68407 proposed markdown-link-check, which follows redirects with no
destination validation, so a docs-only pull request could point the CI
runner at any host it chose.
Add lychee 0.24.2 as the checker, installed from the release tarball
with a hardcoded SHA-256, plus two policies: pr.toml requests only ten
exact approved hosts (matched by lychee against the parsed, normalized
URL), rejects loopback, private, and link-local addresses before the
allowlist is consulted, and follows no redirects; scheduled.toml checks
every host and follows redirects for use against reviewed trunk content.
A fixture of adversarial URLs and a `lychee --dump` diff script prove
the pull-request policy offline, so reviewers can see exactly which
hosts a docs change can reach without any network access.
Refs #43281
* ci: add weekly external link crawl for docs
Link rot happens over time in documents nobody touches, so a check that
only looks at changed files on pull requests never finds it. Issue
#43281 was opened because a user found such a dead link in shipped docs.
Add a weekly workflow that crawls every external link in docs/**/*.md on
trunk with the scheduled lychee policy. It runs only against reviewed
content, holds no token, and reports the way other nightly checks do: a
failed run with the lychee report in the job summary and a Slack notice
to the daily-checks channel.
A first local run over all 284 docs found 25 broken links in 14 files.
Refs #43281
* ci: check approved external links in changed docs
Pull requests that touch docs/**/*.md get no feedback about links they
introduce until the weekly crawl runs after merge.
Add a step to the Validate markdown job that runs lychee with the
pull-request policy on added or modified docs files. Only ten exact
approved hosts are requested, redirects are not followed, and only 400,
404, 410, and transport failures fail the step, so the gate stays
deterministic and other hosts are left to the weekly crawl. Changed
filenames travel as JSON through an environment variable and reach
lychee via `jq --raw-output0 | xargs -0`, never through a shell string.
Pin the job to ubuntu-latest for every author: it now requests
contributor-authored URLs and must never run inside the WooCommerce
runner group. The job-selection filter also matches the link-check
files, so policy changes run the offline policy test even when no
document changed.
Refs #43281
* ci: approve raw.githubusercontent.com in the docs link policy
lychee rewrites approved github.com links of the form
/<owner>/<repo>/blob/<ref>/<file>.md#<fragment> to
raw.githubusercontent.com at check time (its "fetch raw GitHub Markdown
files" quirk). The rewrite runs after the allowlist filter and is not a
redirect, so neither the include regex nor max_redirects = 0 stopped
the runner from contacting a host the policy did not list.
Approve raw.githubusercontent.com explicitly, since it is GitHub-operated
like the hosts already listed, and record why in the policy file and the
fixture so reviewers know `--dump` cannot show this rewrite.
Refs #43281
* ci: cap the weekly docs link crawl at 10 minutes
A full crawl of all docs takes about a minute, and the other scheduled
checks on hosted runners cap their jobs at five or ten minutes. Match
them so a wedged run cannot hold a runner for longer than its peers.
Refs #43281
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index c51f68aa2d2..d83ae35d266 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -92,6 +92,10 @@ jobs:
needs-markdown-validation:
- '*.md'
- '!(.github/**)/**/*.md'
+ - '.github/workflows/ci.yml'
+ - '.github/workflows/docs-link-check/**'
+ - '.github/workflows/scripts/docs-link-check-policy-test.sh'
+ - '.github/workflows/scripts/install-lychee.sh'
needs-syncpack-validation:
- '(.syncpackrc)|(pnpm-lock.yaml)|(pnpm-workspace.yaml)'
@@ -747,7 +751,9 @@ jobs:
name: 'Validate markdown'
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.base.ref == 'trunk' && needs.identify-jobs-to-run.outputs.needs-markdown-validation == 'true' }}
needs: [ 'identify-jobs-to-run' ]
- runs-on: ${{ github.event.pull_request.user.login == 'woocommercebot' && fromJSON('{"group":"WooCommerce Release Checks"}') || 'ubuntu-latest' }}
+ # Always GitHub-hosted: this job requests contributor-authored documentation links
+ # (see the docs link check below) and must never run inside the WooCommerce runner group.
+ runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
@@ -766,6 +772,21 @@ jobs:
needs-docs-build-validation:
- 'docs/**/*.md'
+ # JSON output keeps unusual filenames as data; it is consumed through `env`, never
+ # interpolated into a script.
+ - uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
+ id: docs-link-changes
+ with:
+ list-files: json
+ filters: |
+ needs-docs-link-check:
+ - added|modified: 'docs/**/*.md'
+ needs-docs-link-policy-check:
+ - '.github/workflows/ci.yml'
+ - '.github/workflows/docs-link-check/**'
+ - '.github/workflows/scripts/docs-link-check-policy-test.sh'
+ - '.github/workflows/scripts/install-lychee.sh'
+
- name: 'Setup - node'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
@@ -779,6 +800,29 @@ jobs:
shell: bash
run: pnpm dlx markdownlint-cli -- ${{ steps.target-changes.outputs.needs-markdown-linting_files }}
+ - name: 'Setup - lychee'
+ if: ${{ !cancelled() && ( steps.docs-link-changes.outputs.needs-docs-link-check == 'true' || steps.docs-link-changes.outputs.needs-docs-link-policy-check == 'true' ) }}
+ shell: bash
+ run: bash .github/workflows/scripts/install-lychee.sh
+
+ - name: 'Validate - docs link-check policy'
+ if: ${{ !cancelled() && steps.docs-link-changes.outputs.needs-docs-link-policy-check == 'true' }}
+ shell: bash
+ run: bash .github/workflows/scripts/docs-link-check-policy-test.sh
+
+ # Requests only approved hosts (docs-link-check/pr.toml) and follows no redirects.
+ # Links to other hosts are skipped here and covered by the weekly crawl in
+ # docs-link-check.yml.
+ - name: 'Validate - approved external doc links'
+ if: ${{ !cancelled() && steps.docs-link-changes.outputs.needs-docs-link-check == 'true' }}
+ timeout-minutes: 2
+ shell: bash
+ env:
+ DOCS_LINK_FILES_JSON: ${{ steps.docs-link-changes.outputs.needs-docs-link-check_files }}
+ run: |
+ jq --raw-output0 '.[]' <<< "$DOCS_LINK_FILES_JSON" | xargs -0 --no-run-if-empty \
+ lychee --config .github/workflows/docs-link-check/pr.toml --root-dir "$PWD" --
+
- name: 'Validate - lint docs-files'
if: ${{ always() && steps.target-changes.outputs.needs-docs-build-validation == 'true' }}
shell: bash
diff --git a/.github/workflows/docs-link-check.yml b/.github/workflows/docs-link-check.yml
new file mode 100644
index 00000000000..84af7b333b1
--- /dev/null
+++ b/.github/workflows/docs-link-check.yml
@@ -0,0 +1,73 @@
+name: 'Docs external link check'
+
+# Weekly crawl of every external link in docs/**/*.md on trunk. It runs only against
+# reviewed content, so no pull request can choose what this job requests. Broken links
+# fail the run, land in the job summary, and are announced on Slack.
+#
+# The pull-request gate for changed docs lives in ci.yml (`Validate markdown`) and is
+# restricted to approved hosts; see .github/workflows/docs-link-check/pr.toml.
+
+on:
+ schedule:
+ - cron: '0 6 * * 1' # Mondays at 06:00 UTC.
+ workflow_dispatch:
+
+permissions: {}
+
+concurrency:
+ group: 'docs-link-check'
+ cancel-in-progress: false
+
+jobs:
+ check:
+ name: 'Check external links in docs'
+ if: ${{ github.repository == 'woocommerce/woocommerce' }}
+ runs-on: ubuntu-latest
+ timeout-minutes: 10
+ permissions:
+ contents: read
+ steps:
+ - name: 'Checkout'
+ uses: 'actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0' # v7.0.0
+ with:
+ ref: trunk
+ persist-credentials: false
+ sparse-checkout: |
+ docs
+ .github/workflows
+ tools/monorepo-utils
+
+ - name: 'Install lychee'
+ run: bash .github/workflows/scripts/install-lychee.sh
+
+ - name: 'Check links'
+ run: |
+ lychee --config .github/workflows/docs-link-check/scheduled.toml \
+ --root-dir "$PWD" \
+ --format markdown --output "$RUNNER_TEMP/docs-link-report.md" \
+ 'docs/**/*.md'
+
+ - name: 'Publish report'
+ if: ${{ always() }}
+ run: |
+ if [[ -f "$RUNNER_TEMP/docs-link-report.md" ]]; then
+ cat "$RUNNER_TEMP/docs-link-report.md" >> "$GITHUB_STEP_SUMMARY"
+ fi
+
+ - name: 'Setup Node'
+ if: ${{ failure() }}
+ uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
+ with:
+ node-version-file: '.nvmrc'
+
+ - name: 'Setup PNPM'
+ if: ${{ failure() }}
+ uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
+
+ - name: 'Notify Slack on failure'
+ if: ${{ failure() }}
+ env:
+ SLACK_TOKEN: ${{ secrets.E2E_SLACK_TOKEN }}
+ SLACK_CHANNELS: ${{ secrets.DAILY_CHECKS_CHANNEL }}
+ run: |
+ pnpm utils slack ":link: Docs external link check found broken links or failed.\n<$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID|View the report>"
diff --git a/.github/workflows/docs-link-check/policy-expected.txt b/.github/workflows/docs-link-check/policy-expected.txt
new file mode 100644
index 00000000000..45f5de6e21f
--- /dev/null
+++ b/.github/workflows/docs-link-check/policy-expected.txt
@@ -0,0 +1,8 @@
+http://wordpress.org/plugins/woocommerce/
+https://developer.woocommerce.com/docs/
+https://github.com/Woo/Repo#frag
+https://github.com/auto/link
+https://github.com/woocommerce/woocommerce
+https://github.com/woocommerce/woocommerce/blob/trunk/README.md#readme
+https://github.com/x?y=1
+https://raw.githubusercontent.com/woocommerce/woocommerce/trunk/README.md
diff --git a/.github/workflows/docs-link-check/policy-fixture.md b/.github/workflows/docs-link-check/policy-fixture.md
new file mode 100644
index 00000000000..30e31a60371
--- /dev/null
+++ b/.github/workflows/docs-link-check/policy-fixture.md
@@ -0,0 +1,52 @@
+# Docs link-check policy fixture
+
+This file is input for `.github/workflows/scripts/docs-link-check-policy-test.sh`. It lists URLs that the pull-request link check must request and URLs it must never request. The expected result is `policy-expected.txt`.
+
+## Approved hosts (must be requested)
+
+- [exact host](https://github.com/woocommerce/woocommerce)
+- [plain http](http://wordpress.org/plugins/woocommerce/)
+- [uppercase host is normalized](https://GITHUB.COM/Woo/Repo#frag)
+- [approved docs host](https://developer.woocommerce.com/docs/)
+- [query string kept](https://github.com/x?y=1)
+- autolink: <https://github.com/auto/link>
+- [raw file host](https://raw.githubusercontent.com/woocommerce/woocommerce/trunk/README.md)
+- [blob link with a fragment](https://github.com/woocommerce/woocommerce/blob/trunk/README.md#readme) is requested from raw.githubusercontent.com at check time (see `pr.toml`); `--dump` shows the original URL
+
+## Lookalikes and out-of-policy hosts (must not be requested)
+
+- [user info before an approved host](https://github.com@evil.example/x)
+- [credentials](https://user:pw@github.com/x)
+- [non-default port](https://github.com:8443/x)
+- [suffix lookalike](https://github.com.evil.example/x)
+- [approved host in the path](https://evil.example/github.com/)
+- [www subdomain is not approved](https://www.woocommerce.com/)
+- [other subdomain is not approved](https://foo.wordpress.org/)
+- [trailing dot host](https://github.com./x)
+- [homoglyph host](https://gıthub.com/x)
+- [npm is not approved](https://www.npmjs.com/package/x)
+- [example domain](https://example.com/)
+
+## Private, loopback, and link-local (must not be requested)
+
+- [loopback](http://127.0.0.1/)
+- [localhost with port](http://localhost:8080/)
+- [link-local metadata](http://169.254.169.254/latest/meta-data/)
+- [private range](http://10.0.0.1/)
+- [ipv6 loopback](http://[::1]/)
+- [hex loopback](http://0x7f000001/)
+- [decimal loopback](http://2130706433/)
+- [placeholder .local host](http://store.local/)
+- [placeholder .test host](http://local.wordpress.test/)
+
+## Other schemes and non-network links (must not be requested)
+
+- [mail](mailto:docs@example.com)
+- [relative](../README.md)
+- [root-relative](/docs/apis/rest-api/)
+- [fragment](#approved-hosts-must-be-requested)
+- [ftp](ftp://github.com/x)
+
+```md
+[inside a code fence](https://github.com/should/not/appear)
+```
diff --git a/.github/workflows/docs-link-check/pr.toml b/.github/workflows/docs-link-check/pr.toml
new file mode 100644
index 00000000000..2c45f37a752
--- /dev/null
+++ b/.github/workflows/docs-link-check/pr.toml
@@ -0,0 +1,36 @@
+# lychee policy for the pull-request docs link check (`Validate markdown` job in ci.yml).
+#
+# Only URLs whose exact host is listed in `include` are requested. lychee matches the
+# regex against the parsed, normalized URL (lowercased punycode host, default port
+# elided, user info kept), so user info, non-default ports, subdomains, and lookalike
+# hosts never match. Loopback, private, and link-local addresses are rejected before
+# `include` is consulted and cannot be re-included.
+#
+# Adding a host is a policy change: it lets a docs-only pull request make the CI runner
+# contact that host. Keep the list to hosts operated by WooCommerce, WordPress.org, or GitHub.
+#
+# raw.githubusercontent.com is approved because lychee rewrites approved github.com links
+# of the form `/<owner>/<repo>/blob/<ref>/<file>.md#<fragment>` to that host after the
+# filter has run (lychee-lib `quirks`, "fetch raw GitHub Markdown files"). The rewrite is
+# not a redirect, so `max_redirects = 0` does not stop it, and `--dump` prints the original
+# URL, so the offline policy test cannot observe it either. Both hosts are GitHub-operated.
+#
+# `.github/workflows/scripts/docs-link-check-policy-test.sh` proves this file offline.
+
+no_progress = true
+scheme = ["http", "https"]
+exclude_all_private = true
+include = [
+ '^https?://(github\.com|developer\.woocommerce\.com|woocommerce\.com|developer\.wordpress\.org|woocommerce\.github\.io|wordpress\.org|qit\.woo\.com|user-images\.githubusercontent\.com|raw\.githubusercontent\.com|codex\.wordpress\.org|make\.wordpress\.org)/',
+]
+
+# Redirects are not followed at pull-request time, so an approved host can never lead the
+# runner elsewhere. A 3xx counts as alive here; the scheduled crawl follows redirects.
+max_redirects = 0
+
+# Only 400, 404, and 410 (plus transport failures) fail the check. Auth challenges, bot
+# blocks, rate limits, and server errors do not prove a link is dead.
+accept = "200..=399,401,403,405..=409,411..=599"
+timeout = 10
+max_retries = 2
+max_concurrency = 4
diff --git a/.github/workflows/docs-link-check/scheduled.toml b/.github/workflows/docs-link-check/scheduled.toml
new file mode 100644
index 00000000000..59fefdf9e47
--- /dev/null
+++ b/.github/workflows/docs-link-check/scheduled.toml
@@ -0,0 +1,18 @@
+# lychee policy for the scheduled docs external link crawl (docs-link-check.yml).
+#
+# This runs against already-reviewed content on trunk, so it checks every external host
+# and follows redirects. Results are a report, not a merge gate.
+
+no_progress = true
+scheme = ["http", "https"]
+exclude_all_private = true
+exclude_path = ["docs/_docu-tools"]
+
+# Placeholder hosts used in examples.
+exclude = ['^https?://([^/]+\.local|local\.wordpress\.test)/']
+
+# 403 and 429 are bot blocks and rate limits, not dead links.
+accept = "200..=206,403,429"
+timeout = 15
+max_retries = 2
+max_concurrency = 8
diff --git a/.github/workflows/scripts/docs-link-check-policy-test.sh b/.github/workflows/scripts/docs-link-check-policy-test.sh
new file mode 100755
index 00000000000..cd09f88252e
--- /dev/null
+++ b/.github/workflows/scripts/docs-link-check-policy-test.sh
@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+# Proves the pull-request docs link-check policy without network access.
+# `lychee --dump` applies the same URL filters as a real run and prints only
+# the URLs it would request, so diffing it against the expected list shows
+# exactly which hosts a docs change can make the CI runner contact.
+#
+# Usage: bash .github/workflows/scripts/docs-link-check-policy-test.sh
+# Set LYCHEE to a binary path when lychee is not on PATH.
+
+set -euo pipefail
+
+policy_dir="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )/../docs-link-check" && pwd )"
+lychee="${LYCHEE:-lychee}"
+
+diff -u "${policy_dir}/policy-expected.txt" <(
+ "${lychee}" --config "${policy_dir}/pr.toml" --dump --root-dir "${policy_dir}" "${policy_dir}/policy-fixture.md" | LC_ALL=C sort
+)
+
+printf 'PASS: docs link-check policy\n'
diff --git a/.github/workflows/scripts/install-lychee.sh b/.github/workflows/scripts/install-lychee.sh
new file mode 100755
index 00000000000..e08644d2018
--- /dev/null
+++ b/.github/workflows/scripts/install-lychee.sh
@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+
+# Installs a checksum-pinned lychee release for GitHub-hosted x86_64 Linux runners
+# and adds it to the job PATH. Bump both values together when upgrading.
+
+set -euo pipefail
+
+LYCHEE_VERSION='0.24.2'
+LYCHEE_SHA256='1f4e0ef7f6554a6ed33dd7ac144fb2e1bbed98598e7af973042fc5cd43951c9a'
+
+target='x86_64-unknown-linux-gnu'
+archive="lychee-${target}.tar.gz"
+download_dir="$( mktemp -d )"
+
+curl --silent --show-error --fail --location --retry 3 \
+ --output "${download_dir}/${archive}" \
+ "https://github.com/lycheeverse/lychee/releases/download/lychee-v${LYCHEE_VERSION}/${archive}"
+printf '%s %s\n' "${LYCHEE_SHA256}" "${download_dir}/${archive}" | sha256sum --check --strict --quiet -
+
+tar -xzf "${download_dir}/${archive}" -C "${download_dir}" "lychee-${target}/lychee"
+install -D "${download_dir}/lychee-${target}/lychee" "${RUNNER_TEMP}/lychee/bin/lychee"
+rm -rf "${download_dir}"
+
+printf '%s\n' "${RUNNER_TEMP}/lychee/bin" >> "${GITHUB_PATH}"
+"${RUNNER_TEMP}/lychee/bin/lychee" --version