Commit 913ec89d241 for woocommerce
commit 913ec89d241ac139f988014a6c2a4b9011c9c5e9
Author: Jamel Noel Reid <MrJnrman@users.noreply.github.com>
Date: Mon Nov 9 00:56:57 2026 -0500
Skip CI test jobs that already passed locally via gh local-ci receipts (#69176)
* [DEVPROD-1223] Add the local CI receipt consumer and .github/local-ci.json
* [DEVPROD-1223] Address pre-push review findings on the receipt consumer
* [DEVPROD-1223] Address a8c-review and Codex findings on the receipt consumer
* [DEVPROD-1223] Trust allowlisted authors and write permission, not only author_association
* [DEVPROD-1223] Move who-may-publish to the tool's allowlist; CI checks permission or membership
* [DEVPROD-1223] Remove spot checks; CI relies on write-access trust
* [DEVPROD-1223] Fix a stale comment
* [DEVPROD-1223] Ship the receipt consumer enabled
* [DEVPROD-1223] Document the consumer and the live-test runbook
* [DEVPROD-1223] Test the receipt consumer live instead of with unit tests
* [DEVPROD-1223] Point the file comment at the live runbook
* [DEVPROD-1223] Shorten the file comment
* [DEVPROD-1223] Trim comments and descriptions
* [DEVPROD-1223] Require confirmed write permission; drop the association fallback
* [DEVPROD-1223] Give each jest job four workers
* test change
* Add a Docker image for gh local-ci run --docker
The stock node image has no pnpm, and every job gets a fresh container,
so pnpm has to be in the image rather than installed by a setup step.
The Dockerfile pins Node 24.15.0 by digest to match .nvmrc and installs
pnpm 10.33.0 to match packageManager. The image is built locally; the
command is at the top of the Dockerfile.
* Have gh local-ci build the docker image from the Dockerfile
docker.build replaces the image tag that had to be built by hand. The tool
builds from .github/local-ci/Dockerfile as committed and tags the image by
that directory's git tree, so it rebuilds only when the Dockerfile changes.
* Revert the test change pushed by mistake
This reverts commit 96a8af2313da8ce582ab3e054db34e78d45155b9.
* [DEVPROD-1223] Skip CI test jobs that already passed locally via gh local-ci receipts
Adds the local CI receipt consumer, .github/local-ci.json and the
Dockerfile for gh local-ci run --docker. Carried over from
feature/DEVPROD-1223-local-ci-consumer onto current trunk without its
merge commits and test changes.
* [DEVPROD-1223] Let gh local-ci size jest's workers
A fixed --maxWorkers=4 oversubscribed the fast cores on an M5 Max, which has 6 of them beside 12 slower ones. With {workers} the tool sizes the count for each machine.
* [DEVPROD-1223] Sample receipts for parity and compare CI's result with the receipt's
A rule's sampleRate is the share of jobs with a passing receipt that CI
runs anyway. The draw is deterministic per head commit and job name. For
a sampled job, and for one whose newest receipt failed, a step after the
tests compares CI's result with the receipt's and writes an annotation
and a job summary section. It never changes the job's result.
The JavaScript unit rule samples 10%. sampleRate and {workers} need
gh local-ci v0.4.0 or later.
* [DEVPROD-1223] Let gh local-ci warm, skip prepare and check Node
plan.all lets setup warm the worktree, plan.fields.path lets a run skip a
prepare that would change nothing, and toolchain warns when the local Node
is not the one .nvmrc gives CI.
* [DEVPROD-1223] Check the Node pnpm runs jobs on, not the one on PATH
pnpm-workspace.yaml sets useNodeVersion, so every job runs on pnpm's Node whatever node is on PATH. node -v warned about a version the jobs never use.
---------
Co-authored-by: Ajitem Sahasrabuddhe <ajitem.s@outlook.com>
diff --git a/.github/actions/local-ci-receipt/AGENTS.md b/.github/actions/local-ci-receipt/AGENTS.md
new file mode 100644
index 00000000000..e8e001c150e
--- /dev/null
+++ b/.github/actions/local-ci-receipt/AGENTS.md
@@ -0,0 +1,158 @@
+# Live test runbook
+
+Run this for any change to `receipt.js`, `action.yml`, the `ci.yml` receipt
+step, its `if:` guards or the comparison step, or `.github/local-ci.json`. It is the action's only
+test: it proves the consumer end to end in real CI without touching trunk.
+Budget: ~30 minutes, mostly waiting. Everything created here is deleted at
+the end.
+
+## Preconditions
+
+- The change is committed on a feature branch (call it `FEATURE`).
+- `gh` is signed in as someone whose login is in `allowlist.json` of the
+ tool repository, and `gh local-ci auth` has been run once on this machine.
+- `gh local-ci version` works (the installed build must be current: in the
+ tool clone, `go build -o gh-local-ci . && gh extension install .`).
+- The repository variable `LOCAL_CI_RECEIPTS_DISABLED` is `0` (check with
+ `gh variable list`). If it is `1`, stop and ask; someone turned it off.
+
+## 1. Scratch base branch (stands in for trunk)
+
+```sh
+git checkout -b test/local-ci-base FEATURE
+```
+
+Edit `.github/local-ci.json`: set `"baseBranch": "test/local-ci-base"` and
+make sure `"enabled": true`. Commit (`--no-verify` is fine) and push:
+
+```sh
+git commit -am "Scratch base for the live receipt test (not for merge)" --no-verify
+git push -u origin test/local-ci-base
+```
+
+Why: the consumer reads the config from the PR's base branch and refuses a PR
+that does not target `baseBranch`. The scratch base carries the change under
+test *and* a config that names itself.
+
+## 2. Child branch with receipts
+
+```sh
+git checkout -b test/local-ci-child
+printf '\n// live receipt test marker\n' >> packages/js/number/src/index.ts
+git commit -am "Touch @woocommerce/number for the live receipt test" --no-verify
+gh local-ci plan # expect: merge base = the scratch base tip; 4 JavaScript unit jobs
+gh local-ci run --push # runs them, publishes 4 receipts, pushes the branch
+```
+
+`run` must end with `All 4 job(s) passed` and `pushed test/local-ci-child`.
+If it stops at authorisation or the allowlist, fix that before going on.
+
+## 3. Draft PR against the scratch base
+
+```sh
+gh pr create --draft --base test/local-ci-base --head test/local-ci-child \
+ --title "Scratch: live test of the local CI receipt consumer (do not merge)" \
+ --body "Throwaway. Base branch carries the consumer under test with enabled: true; head has receipts. Both branches will be deleted."
+```
+
+## 4. Positive path
+
+Wait for the `ci` workflow run on the head SHA (`gh run list --workflow ci.yml
+--branch test/local-ci-child`). For each `JavaScript - … [unit]` job:
+
+- conclusion `success`, duration about 10–15 s;
+- steps `Install Monorepo` and `Run tests (unit)` are `skipped`;
+- the job log contains `substituted=true reason=receipt … by <login> (has
+ <permission> permission)`.
+
+Read a job's log with
+`gh api repos/woocommerce/woocommerce/actions/jobs/<job id>/logs | grep substituted=`.
+Every other job of the run (e2e, performance) runs in full; that is
+expected.
+
+If any JavaScript job ran in full, the `reason=` line says why. Fix, commit
+on `FEATURE`, cherry-pick onto **both** scratch branches (the head config must
+stay byte-identical to the base config), run `gh local-ci run --push` on the
+child again (new head → new receipts), and re-check.
+
+## 5. Kill switch
+
+Set the repository variable to `1`, then trigger a **new** run (a re-run
+keeps the old variable value):
+
+```sh
+gh variable set LOCAL_CI_RECEIPTS_DISABLED --body 1
+git commit --allow-empty -m "Trigger a run with the kill switch set" --no-verify
+git push
+```
+
+Expected on every JavaScript job: `substituted=false reason=kill switch
+LOCAL_CI_RECEIPTS_DISABLED=1`, install and tests run. Then set it back:
+
+```sh
+gh variable set LOCAL_CI_RECEIPTS_DISABLED --body 0
+```
+
+## 6. Config change
+
+Edit the `$comment` in `.github/local-ci.json` on the child (any one-word
+change), commit, push. Expected on every JavaScript job:
+`substituted=false reason=.github/local-ci.json differs between the pull
+request and the base branch; config changes always get full CI`.
+
+## 7. Parity
+
+On both scratch branches, set `"sampleRate": 100` on the JavaScript unit
+rule (the head config must stay byte-identical to the base), commit and
+push the base, then run `gh local-ci run --push` on the child.
+
+- **Agree.** Every JavaScript unit job runs in full and logs
+ `substituted=false reason=parity sample: bucket N < sampleRate 100; …`.
+ The step `Compare with the local CI receipt` adds a
+ `local-ci-parity result=agree` notice and a job summary section.
+- **Local pass, CI fail.** On the child, add a throwaway test to
+ `packages/js/number` that expects `process.platform` to be `'darwin'`.
+ Run `gh local-ci run --push` natively on macOS (not `--docker`). Expected
+ on `@woocommerce/number`: an error annotation
+ `local-ci-parity result=mismatch local=success ci=failure`, the job red,
+ and the summary linking the receipt.
+- **Failed receipt.** Change the expectation to `'linux'` and run
+ `gh local-ci run`; the receipt fails. Push with `git push --no-verify`.
+ Expected: a warning annotation
+ `local-ci-parity result=mismatch local=failure ci=success kind=failed-receipt`.
+- **Inconclusive.** Re-run the job and cancel the run while `Run tests`
+ is going. Expected: `local-ci-parity result=inconclusive ci=cancelled`.
+- **Determinism.** Re-run one sampled job. Its bucket is unchanged and
+ matches the one-liner in README.md.
+- **Rate 0.** Set `"sampleRate": 0` on both branches and run again. Jobs
+ are substituted as in step 4, and the comparison step is skipped.
+
+Remove the throwaway test before cleaning up.
+
+## 8. Record and clean up
+
+Add a row per run to the evidence document in the tool repository
+(`docs/evidence-<date>.md`): run URL, what was set, the decision line, the
+per-job durations. Then:
+
+```sh
+gh pr close <number> --delete-branch --comment "Live test complete; results recorded."
+git push origin --delete test/local-ci-base
+git checkout FEATURE && git branch -D test/local-ci-child test/local-ci-base
+git ls-remote origin 'refs/local-ci/*' # must print nothing
+```
+
+Confirm `LOCAL_CI_RECEIPTS_DISABLED` is `0`.
+
+## Reading the result
+
+- A job that ran in full with `reason=no receipt named …` means the tool
+ published under a different name or config hash than CI expected: compare
+ the receipt's name and `external_id` on the commit
+ (`gh api repos/woocommerce/woocommerce/commits/<sha>/check-runs?app_id=4830646`)
+ with what `receipt.js` computes.
+- `author … has unknown repository permission; write permission is
+ required` means the permission lookup failed; check the token's
+ permissions on the job.
+- `lookup failed, running normally: …` is an API or parse error; the message
+ says which.
diff --git a/.github/actions/local-ci-receipt/README.md b/.github/actions/local-ci-receipt/README.md
new file mode 100644
index 00000000000..293906a0586
--- /dev/null
+++ b/.github/actions/local-ci-receipt/README.md
@@ -0,0 +1,90 @@
+# local-ci-receipt
+
+Decides, at the start of each `project-test-jobs` matrix job on a pull
+request, whether a receipt published by [`gh local-ci`](https://github.com/woocommerce/gh-local-ci)
+lets the job skip `Install Monorepo` and `Run tests`. A receipt is a check
+run created on the head commit by the Woo Local CI Checks GitHub App.
+
+The job is skipped only when the newest receipt of its name comes from that
+App, passed, was made under the config trunk has, and was published by the
+PR author, whose write permission on the repository is confirmed through
+the API. Anything else runs the job as always and logs the reason. Receipts apply to `pull_request` events only; merge queue
+runs and pushes to trunk run everything.
+
+- `action.yml` — composite action; `receipt.js` — the decision (`decide()`)
+ and the API reads (`lookup()`), and the comparison (`parityReport()`,
+ `report()`).
+- Contract: `.github/local-ci.json` (read from the base branch). Its
+ `sampleRate` and `{workers}` need gh local-ci v0.4.0 or later.
+- Kill switch: repository variable `LOCAL_CI_RECEIPTS_DISABLED=1`, effective
+ on the next *new* run (re-runs keep old variables).
+
+## Parity
+
+A rule's `sampleRate` in `.github/local-ci.json` is the percentage of jobs
+with a valid, passing receipt that run in CI anyway. The draw is
+deterministic: a job is sampled when
+`sha256(head sha + job name)`, read as a 32-bit big-endian number, mod 100
+is below the rate. A re-run samples the same jobs, and each push draws
+again. To recompute a draw by hand:
+
+```sh
+node -e 'const [s, j] = process.argv.slice(1); console.log(require("crypto").createHash("sha256").update(s + j).digest().readUInt32BE(0) % 100)' <full head sha> '<job name>'
+```
+
+A job whose newest receipt failed also runs in CI, as it always did. For
+both, the step `Compare with the local CI receipt` runs after the tests and
+compares CI's result with the receipt's. It writes a job summary section
+and one annotation, which the run page lists with every other job's:
+
+- `result=agree`: a notice.
+- `result=mismatch local=success ci=failure`: an error. The receipt would
+ have let a failing job through.
+- `result=mismatch local=failure ci=success`: a warning.
+- `result=inconclusive`: a notice. CI's tests were cancelled or never ran.
+
+Each annotation starts with
+`local-ci-parity result=… local=… ci=… base-moved=yes|no kind=sample|failed-receipt`,
+so the results can be searched. The comparison never changes the job's
+result: CI's own test run already decides it.
+
+### When CI and a receipt disagree
+
+1. Check `base-moved` first. CI tests the merge commit; the receipt tested
+ the head against an older base. A change on trunk can explain the
+ difference.
+2. Compare the receipt's log (the check run's output) with the CI log.
+3. Run the job again locally with `gh local-ci run --fresh`, then with
+ `--docker`. A failure only outside Docker points at the local toolchain.
+4. Open an issue on [gh-local-ci](https://github.com/woocommerce/gh-local-ci)
+ with the run URL.
+
+If mismatches repeat, set `LOCAL_CI_RECEIPTS_DISABLED=1` until the cause is
+known.
+
+## Testing
+
+This action is tested live, against real GitHub, not with unit tests: the
+only test that sees what GitHub actually sends is a real pull request, which
+is how the `author_association` problem was found. The **live test is
+required** for any change to `receipt.js`, `action.yml`, the `ci.yml` step,
+its two `if:` guards or the comparison step, or `.github/local-ci.json`. `pr-check-local-actions.yml`
+still validates `.github/local-ci.json` on every change to it.
+
+The live test uses a scratch branch as a stand-in for trunk, so nothing
+touches trunk. It takes about 30 minutes, mostly waiting for CI, and needs
+someone whose login is in the tool's `allowlist.json` with `gh local-ci auth`
+done. The full runbook, written so an agent can carry it out, is in
+`AGENTS.md`; the result of the first run is recorded in the tool repository
+(`docs/evidence-2026-09-28.md`).
+
+What it proves, in one PR:
+
+| Check | Expected in the job log |
+|---|---|
+| Positive path | `substituted=true reason=receipt … by <login> (has <permission> permission)`; `Install Monorepo` and `Run tests (unit)` skipped; job ≈ 10 s |
+| Kill switch (new run with the variable set) | `substituted=false reason=kill switch LOCAL_CI_RECEIPTS_DISABLED=1` |
+| Config changed in the PR | `substituted=false reason=.github/local-ci.json differs …` |
+| Receipts absent | `substituted=false reason=no receipt named …` |
+| Parity sample (`sampleRate: 100`) | `substituted=false reason=parity sample: bucket … < sampleRate 100 …`; tests run; a `local-ci-parity result=agree` notice |
+| Local pass, CI fail | a `local-ci-parity result=mismatch local=success ci=failure` error; job red |
diff --git a/.github/actions/local-ci-receipt/action.yml b/.github/actions/local-ci-receipt/action.yml
new file mode 100644
index 00000000000..d56ccbb721c
--- /dev/null
+++ b/.github/actions/local-ci-receipt/action.yml
@@ -0,0 +1,63 @@
+name: 'Look for a local CI receipt'
+description: 'Decides whether a `gh local-ci` receipt on the pull request head lets this job skip its install and test steps. Anything short of a verified, passing receipt yields `substituted=false` with a reason. With `ci-outcome`, it instead compares CI''s result with the receipt''s. See README.md.'
+
+inputs:
+ job-name:
+ description: 'The matrix job name. The receipt is the check run named `receiptPrefix + job-name`.'
+ required: true
+ github-token:
+ description: 'Needs `contents: read` and `checks: read`; the default `github.token` has both.'
+ required: true
+ disabled:
+ description: 'Pass the `LOCAL_CI_RECEIPTS_DISABLED` repository variable; `1` disables substitution.'
+ required: false
+ default: ''
+ ci-outcome:
+ description: 'Set after the tests to compare: the `Run tests` step outcome. Empty runs the lookup.'
+ required: false
+ default: ''
+ parity:
+ description: 'With `ci-outcome`: the lookup step''s `parity` output.'
+ required: false
+ default: ''
+
+outputs:
+ substituted:
+ description: '`true` when the install and test steps may be skipped.'
+ value: ${{ steps.lookup.outputs.substituted }}
+ reason:
+ description: 'Why.'
+ value: ${{ steps.lookup.outputs.reason }}
+ parity:
+ description: 'Set when CI runs the job and its result can be compared with the receipt''s (a sampled job, or a failed receipt); pass it to the comparison step.'
+ value: ${{ steps.lookup.outputs.parity }}
+
+runs:
+ using: 'composite'
+ steps:
+ - name: Look up the receipt
+ id: lookup
+ if: ${{ inputs.ci-outcome == '' }}
+ uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
+ env:
+ JOB_NAME: ${{ inputs.job-name }}
+ DISABLED: ${{ inputs.disabled }}
+ SCRIPT: ${{ github.action_path }}/receipt.js
+ with:
+ github-token: ${{ inputs.github-token }}
+ retries: 2
+ script: |
+ await require( process.env.SCRIPT ).lookup( { github, context, core } );
+
+ - name: Compare with the receipt
+ if: ${{ inputs.ci-outcome != '' }}
+ uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
+ env:
+ JOB_NAME: ${{ inputs.job-name }}
+ PARITY: ${{ inputs.parity }}
+ CI_OUTCOME: ${{ inputs.ci-outcome }}
+ SCRIPT: ${{ github.action_path }}/receipt.js
+ with:
+ github-token: ${{ inputs.github-token }}
+ script: |
+ await require( process.env.SCRIPT ).report( { core } );
diff --git a/.github/actions/local-ci-receipt/receipt.js b/.github/actions/local-ci-receipt/receipt.js
new file mode 100644
index 00000000000..2d8d7459bbc
--- /dev/null
+++ b/.github/actions/local-ci-receipt/receipt.js
@@ -0,0 +1,447 @@
+/**
+ * Decides whether a `gh local-ci` receipt (a check run on the head commit)
+ * lets this matrix job skip its install and test steps. Anything short of
+ * a verified, passing receipt yields `substituted=false` with a reason.
+ * A sampled job runs anyway, and so does one whose receipt failed; for
+ * both, `report()` compares CI's result with the receipt's after the tests.
+ * `decide()` and `parityReport()` are pure; `lookup()` and `report()` do
+ * the I/O. See README.md.
+ */
+
+const crypto = require( 'crypto' );
+
+const CONFIG_PATH = '.github/local-ci.json';
+const WRITE_PERMISSIONS = [ 'admin', 'maintain', 'write' ];
+const OFF_VALUES = [ '', '0', 'false', 'off', 'no' ];
+// Only job types whose remaining steps ci.yml guards.
+const SUBSTITUTABLE_TYPES = [ 'unit' ];
+const RECEIPT_VERSION = 'v1';
+// Local results worth comparing with CI's; `cancelled` is not a result.
+const COMPARABLE_CONCLUSIONS = [ 'success', 'failure', 'timed_out' ];
+const API_TIMEOUT_MS = 20000;
+
+/**
+ * Parses `v1;author=…;base=…;config=…`; null for any other version.
+ *
+ * @param {string} externalId
+ * @return {Object|null}
+ */
+function parseExternalId( externalId ) {
+ const parts = String( externalId || '' ).split( ';' );
+ if ( parts[ 0 ] !== RECEIPT_VERSION ) {
+ return null;
+ }
+ const meta = {};
+ for ( const kv of parts.slice( 1 ) ) {
+ const i = kv.indexOf( '=' );
+ if ( i > 0 ) {
+ meta[ kv.slice( 0, i ) ] = kv.slice( i + 1 );
+ }
+ }
+ return meta;
+}
+
+/**
+ * Anything but an explicit "off" value disables, so a typo fails safe.
+ *
+ * @param {*} value
+ * @return {boolean}
+ */
+function killSwitchSet( value ) {
+ return ! OFF_VALUES.includes( String( value === undefined || value === null ? '' : value ).trim().toLowerCase() );
+}
+
+/**
+ * Deterministic sample bucket for a (commit, job) pair: 0..99. The same
+ * pair always lands in the same bucket, so a re-run samples the same jobs
+ * and anyone can recompute the draw.
+ *
+ * @param {string} headSha
+ * @param {string} jobName
+ * @return {number}
+ */
+function sampleBucket( headSha, jobName ) {
+ const digest = crypto.createHash( 'sha256' ).update( headSha + jobName ).digest();
+ return digest.readUInt32BE( 0 ) % 100;
+}
+
+/**
+ * @param {string} jobName
+ * @return {string} The `[type]` suffix, or "".
+ */
+function jobType( jobName ) {
+ const m = /\[([^\]]+)\]\s*(\(optional\))?\s*$/.exec( jobName );
+ return m ? m[ 1 ] : '';
+}
+
+/**
+ * The rule that claims this job, matched the way gh local-ci matches it.
+ *
+ * @param {Object} config
+ * @param {string} jobName
+ * @return {Object|null}
+ */
+function ruleFor( config, jobName ) {
+ const type = jobType( jobName );
+ for ( const rule of config.eligible || [] ) {
+ if ( rule.type !== type ) {
+ continue;
+ }
+ if ( rule.namePrefix && ! jobName.startsWith( rule.namePrefix ) ) {
+ continue;
+ }
+ if ( ( rule.exclude || [] ).some( ( x ) => jobName.includes( x ) ) ) {
+ continue;
+ }
+ return rule;
+ }
+ return null;
+}
+
+/**
+ * Decides whether the job may be substituted. Pure.
+ *
+ * @param {Object} input
+ * @param {string} input.disabled Kill-switch variable value.
+ * @param {string} input.eventName
+ * @param {Object} input.pullRequest pull_request payload.
+ * @param {string|null} input.authorPermission Repository permission, or null.
+ * @param {string} input.jobName
+ * @param {Object|null} input.baseConfig `{ raw, parsed, sha256 }`, or null.
+ * @param {string|null} input.headConfigRaw
+ * @param {Array} input.checkRuns
+ * @return {{substituted: boolean, reason: string, parity?: Object}} `parity`
+ * is set when CI runs the job and its result can be compared with
+ * the receipt's.
+ */
+function decide( input ) {
+ const no = ( reason ) => ( { substituted: false, reason } );
+
+ if ( killSwitchSet( input.disabled ) ) {
+ return no( `kill switch LOCAL_CI_RECEIPTS_DISABLED=${ input.disabled }` );
+ }
+ if ( input.eventName !== 'pull_request' || ! input.pullRequest ) {
+ return no( `event is ${ input.eventName }, receipts apply to pull_request only` );
+ }
+ const pr = input.pullRequest;
+
+ if ( ! input.baseConfig || ! input.baseConfig.parsed ) {
+ return no( `no ${ CONFIG_PATH } on the base branch` );
+ }
+ const config = input.baseConfig.parsed;
+ if ( config.enabled !== true ) {
+ return no( 'local CI receipts are not enabled on the base branch' );
+ }
+ // Only the configured base branch's config is trusted.
+ const baseRef = pr.base && pr.base.ref;
+ if ( baseRef !== config.baseBranch ) {
+ return no( `pull request targets ${ baseRef }, receipts apply to ${ config.baseBranch } only` );
+ }
+ if ( input.headConfigRaw !== input.baseConfig.raw ) {
+ return no( `${ CONFIG_PATH } differs between the pull request and the base branch; config changes always get full CI` );
+ }
+
+ const login = String( ( pr.user && pr.user.login ) || '' );
+ // Write permission is what publishing a receipt requires; nothing weaker
+ // (author_association is viewer-dependent) is accepted in its place.
+ if ( ! WRITE_PERMISSIONS.includes( String( input.authorPermission || '' ).toLowerCase() ) ) {
+ return no( `author ${ login } has ${ input.authorPermission || 'unknown' } repository permission; write permission is required` );
+ }
+ const trust = `has ${ input.authorPermission } permission`;
+
+ const name = String( config.receiptPrefix || '' ) + input.jobName;
+ const appId = Number( config.app && config.app.appId );
+ const candidates = ( input.checkRuns || [] ).filter(
+ ( r ) => r && r.name === name && r.app && Number( r.app.id ) === appId
+ );
+ if ( candidates.length === 0 ) {
+ return no( `no receipt named "${ name }" from app ${ appId }` );
+ }
+ // Newest wins: a later failure supersedes an earlier success.
+ const receipt = candidates.reduce( ( a, b ) => ( Number( b.id ) > Number( a.id ) ? b : a ) );
+ const receiptRef = receipt.html_url || receipt.id;
+
+ const meta = parseExternalId( receipt.external_id );
+ if ( ! meta ) {
+ return no( `receipt ${ receipt.id } is not a ${ RECEIPT_VERSION } receipt` );
+ }
+ if ( String( meta.author || '' ).toLowerCase() !== login.toLowerCase() ) {
+ return no( `receipt was published by ${ meta.author || 'nobody' }, pull request is by ${ login }` );
+ }
+ if ( meta.config !== input.baseConfig.sha256 ) {
+ return no( 'receipt was made under a different local-ci.json than the base branch has' );
+ }
+
+ const rule = ruleFor( config, input.jobName );
+ if ( ! rule ) {
+ return no( 'no eligibility rule for this job in the base config' );
+ }
+ if ( ! SUBSTITUTABLE_TYPES.includes( jobType( input.jobName ) ) ) {
+ return no( `ci.yml does not guard the steps a ${ jobType( input.jobName ) } job needs; only ${ SUBSTITUTABLE_TYPES.join( '/' ) } jobs may be substituted` );
+ }
+ // The conclusion is checked after the receipt is known to be valid, so a
+ // failed one is still worth comparing: CI runs that job anyway.
+ const parity = ( kind, extra = {} ) => ( {
+ kind,
+ receipt: { id: receipt.id, url: receipt.html_url || '', conclusion: receipt.conclusion },
+ author: meta.author,
+ mode: meta.mode || '',
+ tool: meta.tool || '',
+ base: meta.base || '',
+ ciBase: ( pr.base && pr.base.sha ) || '',
+ ...extra,
+ } );
+ if ( receipt.conclusion !== 'success' ) {
+ const result = no( `newest receipt ${ receiptRef } concluded ${ receipt.conclusion }` );
+ if ( COMPARABLE_CONCLUSIONS.includes( receipt.conclusion ) ) {
+ result.parity = parity( 'failed-receipt' );
+ }
+ return result;
+ }
+
+ const bucket = sampleBucket( String( ( pr.head && pr.head.sha ) || '' ), input.jobName );
+ // A rate that is not a number fails closed: sample everything rather
+ // than nothing.
+ const rate = rule.sampleRate === undefined ? 0 : Number( rule.sampleRate );
+ if ( ! Number.isFinite( rate ) || bucket < rate ) {
+ return {
+ substituted: false,
+ reason: `parity sample: bucket ${ bucket } < sampleRate ${ rule.sampleRate }; running in CI to compare with receipt ${ receiptRef }`,
+ parity: parity( 'sample', { bucket, rate: rule.sampleRate } ),
+ };
+ }
+
+ let reason = `receipt ${ receiptRef } by ${ meta.author } (${ trust })`;
+ if ( meta.base && pr.base && meta.base !== pr.base.sha ) {
+ reason += ` (base moved since: tested against ${ meta.base.slice( 0, 11 ) }, base is now ${ String( pr.base.sha ).slice( 0, 11 ) })`;
+ }
+ return { substituted: true, reason };
+}
+
+/**
+ * @param {string} sha
+ * @return {string} The 11-character short form, or "unknown".
+ */
+function short( sha ) {
+ return sha ? String( sha ).slice( 0, 11 ) : 'unknown';
+}
+
+/**
+ * Compares CI's result with the receipt's. Pure.
+ *
+ * @param {Object} input
+ * @param {Object} input.parity `decide()`'s `parity`.
+ * @param {string} input.ciOutcome The `Run tests` step's outcome.
+ * @param {string} input.jobName
+ * @return {{result: string, level: string, marker: string, message: string, markdown: string}}
+ */
+function parityReport( { parity, ciOutcome, jobName } ) {
+ const local = parity.receipt.conclusion;
+ const ci = ciOutcome || 'unknown';
+ const baseMoved = Boolean( parity.base && parity.ciBase && parity.base !== parity.ciBase );
+ const localPassed = local === 'success';
+
+ let result;
+ let level;
+ let message;
+ if ( ci !== 'success' && ci !== 'failure' ) {
+ result = 'inconclusive';
+ level = 'notice';
+ message = `CI's tests did not finish (${ ci }), so there is nothing to compare with the receipt.`;
+ } else if ( localPassed === ( ci === 'success' ) ) {
+ result = 'agree';
+ level = 'notice';
+ message = `CI ${ localPassed ? 'passed' : 'failed' }, as the receipt did.`;
+ } else {
+ result = 'mismatch';
+ // A local pass that CI fails is the case that would let a bad change
+ // through on a receipt; the other way round costs only a re-run.
+ level = localPassed ? 'error' : 'warning';
+ message = `the receipt ${ localPassed ? 'passed' : 'failed' } and CI ${ ci === 'success' ? 'passed' : 'failed' }.`;
+ if ( baseMoved ) {
+ message += ' The base moved since the receipt, so the difference may come from the base branch.';
+ }
+ }
+
+ const marker = `local-ci-parity result=${ result } local=${ local } ci=${ ci } base-moved=${ baseMoved ? 'yes' : 'no' } kind=${ parity.kind }`;
+ const receiptLink = parity.receipt.url ? `[receipt ${ parity.receipt.id }](${ parity.receipt.url })` : `receipt ${ parity.receipt.id }`;
+ const why =
+ parity.kind === 'sample'
+ ? `sampled (bucket ${ parity.bucket } < sampleRate ${ parity.rate })`
+ : 'the newest receipt did not pass';
+ const base = baseMoved
+ ? `the receipt tested against ${ short( parity.base ) }; CI merged onto ${ short( parity.ciBase ) }`
+ : `the same base, ${ short( parity.ciBase ) }`;
+ const markdown = [
+ `### Local CI parity: ${ result }`,
+ '',
+ `${ jobName }: ${ message }`,
+ '',
+ `- **Receipt:** ${ receiptLink }, ${ local }, by ${ parity.author } (mode ${ parity.mode || 'unknown' }, tool ${ parity.tool || 'unknown' })`,
+ `- **CI:** ${ ci }`,
+ `- **Why CI ran it:** ${ why }`,
+ `- **Base:** ${ base }`,
+ '',
+ result === 'mismatch' ? 'See "When CI and a receipt disagree" in `.github/actions/local-ci-receipt/README.md`.' : '',
+ '',
+ `\`${ marker }\``,
+ '',
+ ].join( '\n' );
+
+ return { result, level, marker, message: `${ marker }: ${ message } ${ parity.receipt.url || parity.receipt.id }`, markdown };
+}
+
+/**
+ * Writes the comparison as an annotation and a job summary section. Never
+ * throws and never fails the job: CI's own result is already the job's.
+ *
+ * @param {Object} args
+ * @param {Object} args.core
+ * @return {Promise<Object|null>}
+ */
+async function report( { core } ) {
+ try {
+ const out = parityReport( {
+ parity: JSON.parse( process.env.PARITY ),
+ ciOutcome: process.env.CI_OUTCOME,
+ jobName: process.env.JOB_NAME,
+ } );
+ core[ out.level ]( out.message, { title: `Local CI parity: ${ out.result }` } );
+ await core.summary.addRaw( out.markdown ).write();
+ return out;
+ } catch ( e ) {
+ core.warning( `local-ci: parity report failed: ${ e && e.message ? e.message : e }` );
+ return null;
+ }
+}
+
+/**
+ * Rejects after API_TIMEOUT_MS so a slow API cannot stall the job.
+ *
+ * @param {Promise} promise
+ * @param {string} what
+ * @return {Promise}
+ */
+function within( promise, what ) {
+ let timer;
+ const timeout = new Promise( ( _, reject ) => {
+ timer = setTimeout( () => reject( new Error( `${ what } took longer than ${ API_TIMEOUT_MS }ms` ) ), API_TIMEOUT_MS );
+ } );
+ return Promise.race( [ promise, timeout ] ).finally( () => clearTimeout( timer ) );
+}
+
+/**
+ * The config file's raw bytes at a ref, or null when absent.
+ *
+ * @param {Object} github
+ * @param {Object} repo `{ owner, repo }`
+ * @param {string} ref
+ * @return {Promise<string|null>}
+ */
+async function rawFile( github, repo, ref ) {
+ try {
+ const res = await within(
+ github.rest.repos.getContent( { ...repo, path: CONFIG_PATH, ref, mediaType: { format: 'raw' } } ),
+ `reading ${ CONFIG_PATH } at ${ ref }`
+ );
+ return typeof res.data === 'string' ? res.data : null;
+ } catch ( e ) {
+ if ( e && e.status === 404 ) {
+ return null;
+ }
+ throw e;
+ }
+}
+
+/**
+ * The author's repository permission, or null when unavailable.
+ *
+ * @param {Object} github
+ * @param {Object} repo `{ owner, repo }`
+ * @param {string} username
+ * @return {Promise<string|null>}
+ */
+async function authorPermission( github, repo, username ) {
+ if ( ! username ) {
+ return null;
+ }
+ try {
+ const res = await within(
+ github.rest.repos.getCollaboratorPermissionLevel( { ...repo, username } ),
+ `reading ${ username }'s permission`
+ );
+ return ( res.data && res.data.permission ) || null;
+ } catch ( e ) {
+ return null;
+ }
+}
+
+/**
+ * Gathers the inputs, decides, writes the outputs. Never throws.
+ *
+ * @param {Object} args
+ * @param {Object} args.github
+ * @param {Object} args.context
+ * @param {Object} args.core
+ */
+async function lookup( { github, context, core } ) {
+ let result;
+ try {
+ const pr = context.payload.pull_request;
+ const input = {
+ disabled: process.env.DISABLED,
+ eventName: context.eventName,
+ pullRequest: pr,
+ authorPermission: null,
+ jobName: process.env.JOB_NAME,
+ baseConfig: null,
+ headConfigRaw: null,
+ checkRuns: [],
+ };
+ // No API calls when the switch is set, off a PR, or not enabled.
+ const wanted = ! killSwitchSet( input.disabled ) && input.eventName === 'pull_request' && pr;
+ if ( wanted ) {
+ const repo = { owner: context.repo.owner, repo: context.repo.repo };
+ const baseRaw = await rawFile( github, repo, pr.base.sha );
+ if ( baseRaw !== null ) {
+ input.baseConfig = {
+ raw: baseRaw,
+ parsed: JSON.parse( baseRaw ),
+ sha256: crypto.createHash( 'sha256' ).update( baseRaw ).digest( 'hex' ),
+ };
+ }
+ }
+ if ( wanted && input.baseConfig && input.baseConfig.parsed.enabled === true ) {
+ const repo = { owner: context.repo.owner, repo: context.repo.repo };
+ input.headConfigRaw = await rawFile( github, repo, pr.head.sha );
+ input.authorPermission = await authorPermission( github, repo, pr.user && pr.user.login );
+ core.info( `author permission lookup: ${ input.authorPermission || 'unavailable to this token' }` );
+ // `all`: every attempt, so the newest is chosen here.
+ const listed = await within(
+ github.rest.checks.listForRef( {
+ ...repo,
+ ref: pr.head.sha,
+ check_name: String( input.baseConfig.parsed.receiptPrefix || '' ) + input.jobName,
+ app_id: Number( input.baseConfig.parsed.app && input.baseConfig.parsed.app.appId ),
+ filter: 'all',
+ per_page: 100,
+ } ),
+ 'listing check runs'
+ );
+ input.checkRuns = ( listed.data && listed.data.check_runs ) || [];
+ }
+ result = decide( input );
+ } catch ( e ) {
+ result = { substituted: false, reason: `lookup failed, running normally: ${ e && e.message ? e.message : e }` };
+ }
+ if ( /lookup failed/.test( result.reason ) ) {
+ core.warning( `local-ci: ${ result.reason }` );
+ }
+ core.info( `substituted=${ result.substituted } reason=${ result.reason }` );
+ core.setOutput( 'substituted', String( result.substituted ) );
+ core.setOutput( 'reason', result.reason );
+ core.setOutput( 'parity', result.parity ? JSON.stringify( result.parity ) : '' );
+ return result;
+}
+
+module.exports = { decide, lookup, parityReport, report, parseExternalId, sampleBucket, ruleFor, jobType, killSwitchSet, CONFIG_PATH, SUBSTITUTABLE_TYPES };
diff --git a/.github/local-ci.json b/.github/local-ci.json
new file mode 100644
index 00000000000..b645a8eb994
--- /dev/null
+++ b/.github/local-ci.json
@@ -0,0 +1,60 @@
+{
+ "$comment": "Contract between `gh local-ci` (publisher) and the receipt lookup action in ci.yml (consumer). CI reads this file from trunk; a PR that changes it gets a full CI run. Who may publish is the allowlist compiled into gh local-ci, not this file.",
+ "version": 1,
+ "host": "github.com",
+ "repository": "woocommerce/woocommerce",
+ "app": { "clientId": "Iv23lix7ORqAqQLsB13K", "appId": 4830646 },
+ "baseBranch": "trunk",
+ "receiptPrefix": "local-ci/v1: ",
+
+ "enabled": true,
+
+ "plan": {
+ "$comment": "The same planner CI runs (ci.yml → pnpm utils ci-jobs) with the same inputs: a diff from the merge base (CI uses gh pr diff) and the pull_request event filter. GITHUB_ACTIONS=true makes it write its matrix to GITHUB_OUTPUT. fields.path is projectPath, the project's directory from the repository root, so gh local-ci can skip a rule's prepare when only files inside the tested projects changed.",
+ "command": [ "sh", "-c", "GITHUB_ACTIONS=true GITHUB_OUTPUT={out} pnpm utils ci-jobs --base-ref {mergeBase} --event pull_request" ],
+ "output": "github-output",
+ "fields": { "name": "name", "project": "projectName", "command": "command", "type": "testType", "path": "projectPath" },
+ "all": {
+ "$comment": "The same planner with no diff, for gh local-ci setup to warm every project a rule could test. Without --base-ref the planner treats every project as changed.",
+ "command": [ "sh", "-c", "GITHUB_ACTIONS=true GITHUB_OUTPUT={out} pnpm utils ci-jobs --event pull_request" ]
+ }
+ },
+
+ "eligible": [
+ {
+ "type": "unit",
+ "namePrefix": "JavaScript",
+ "sampleRate": 10,
+ "prepare": {
+ "$comment": "Mirrors setup-woocommerce-monorepo with install='<project>...' and build-type=dependencies. Re-validate and update mirrors.sha256 when that action changes.",
+ "env": { "CI": "true", "GITHUB_ACTIONS": "true", "BROWSERSLIST_IGNORE_OLD_DATA": "true" },
+ "clean": [ "plugins/woocommerce/assets/client/blocks" ],
+ "steps": [
+ [ "pnpm", "install", "{projects:--filter=%s...}", "--frozen-lockfile" ],
+ [ "pnpm", "--if-present", "--workspace-concurrency=Infinity", "--stream", "{projects:--filter=%s^...}", "/^build:project:.*$/" ]
+ ],
+ "mirrors": {
+ "action": ".github/actions/setup-woocommerce-monorepo/action.yml",
+ "sha256": "9aa17d624c6b2f4e5666763e9354b424c5c988f1b6e506d26ca8df40ad9c0fa0"
+ }
+ },
+ "$comment": "Every test:js is jest, which defaults to cores-1 workers per invocation. gh local-ci sizes {workers} for the machine it runs on, from a budget the jobs share; on a Mac with more than one core tier, from the fastest tier only. A fixed 4 per job oversubscribed an M5 Max's 6 fast cores: the seven jest jobs of one diff took 83 s, against 65 to 73 s sized by the tool.",
+ "run": [ "pnpm", "--filter={project}", "{command}", "--maxWorkers={workers}" ]
+ }
+ ],
+
+ "docker": {
+ "$comment": "For run --docker. gh local-ci builds the image from .github/local-ci/Dockerfile as committed, and rebuilds it only when that directory changes. The pnpm store lives on the per-clone HOME volume, so no mounts are needed.",
+ "build": { "context": ".github/local-ci" }
+ },
+
+ "lockfile": "pnpm-lock.yaml",
+
+ "toolchain": [
+ {
+ "$comment": "CI's Node: setup-woocommerce-monorepo runs setup-node with node-version-file .nvmrc. Locally every job runs through pnpm, which uses useNodeVersion from pnpm-workspace.yaml whatever node is on PATH, so the check asks pnpm, not PATH. pnpm is not checked: action-setup reads it from packageManager in package.json, and expectFrom takes a file that holds only the version. Nor is PHP: the JavaScript rule's build-type dependencies skips setup-php.",
+ "run": "pnpm exec node -v",
+ "expectFrom": ".nvmrc"
+ }
+ ]
+}
diff --git a/.github/local-ci/Dockerfile b/.github/local-ci/Dockerfile
new file mode 100644
index 00000000000..db9f4149135
--- /dev/null
+++ b/.github/local-ci/Dockerfile
@@ -0,0 +1,16 @@
+# The image `gh local-ci run --docker` cuts its containers from. The tool
+# builds it from this file as committed, with this directory as the context,
+# and rebuilds it only when something here changes. Keep the Node and pnpm
+# versions in step with .nvmrc and the packageManager field in package.json.
+#
+# Every job gets a fresh container and only named volumes survive between
+# them, so pnpm must be in the image itself: a setup step that installs it
+# would be gone by the time the jobs start.
+#
+# The full Debian image, not -slim, because node-gyp needs python3, make and g++.
+FROM node:24.15.0-bookworm@sha256:f22d6a1f082c02f292e86929b5b0442ac2e5eaf438a5dea9b1566601c3e05940
+
+# The standalone pnpm, as pnpm/action-setup installs it in CI, rather than
+# corepack, whose cache would sit under the build-time HOME and not the
+# /home/local-ci volume the tool mounts at run time.
+RUN npm install --global pnpm@10.33.0 && npm cache clean --force
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 80b88a3e306..0595d6782b7 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -325,9 +325,25 @@ jobs:
with:
ref: ${{ inputs.refName != '' && inputs.refName || '' }}
+ # Skip install and tests when a `gh local-ci` receipt vouches for this
+ # job on this commit. Off switch: repository variable
+ # LOCAL_CI_RECEIPTS_DISABLED=1. Rollback: revert this step, the two
+ # guards below and the comparison step after the tests. See
+ # .github/actions/local-ci-receipt/README.md.
+ - uses: './.github/actions/local-ci-receipt'
+ name: 'Look for a local CI receipt'
+ id: 'local-ci-receipt'
+ if: ${{ github.event_name == 'pull_request' }}
+ continue-on-error: true
+ with:
+ job-name: ${{ matrix.name }}
+ github-token: ${{ github.token }}
+ disabled: ${{ vars.LOCAL_CI_RECEIPTS_DISABLED }}
+
- uses: './.github/actions/setup-woocommerce-monorepo'
name: 'Install Monorepo'
id: 'install-monorepo'
+ if: ${{ steps.local-ci-receipt.outputs.substituted != 'true' }}
with:
install: '${{ matrix.projectName }}...'
build: ${{ ( matrix.usesSharedPluginBuild == true && inputs.artifactName == '' && 'false' ) || matrix.projectName }}
@@ -448,6 +464,8 @@ jobs:
path: '${{ runner.temp }}/last-run-info'
- name: 'Run tests (${{ matrix.testType }})'
+ id: 'run-tests'
+ if: ${{ steps.local-ci-receipt.outputs.substituted != 'true' }}
timeout-minutes: ${{ ( github.event_name == 'pull_request' && ( ( matrix.testType == 'e2e' && ( github.event.repository.private && 30 || 20 ) ) || ( matrix.testType == 'unit:php' && 25 ) ) ) || 360 }}
env:
E2E_ENV_KEY: ${{ secrets.E2E_ENV_KEY }}
@@ -495,6 +513,18 @@ jobs:
# Finally, run the tests
pnpm --filter="${{ matrix.projectName }}" ${{ matrix.command }} $lastRunFlag
+ # When the lookup ran the job anyway with a receipt in hand (a parity
+ # sample, or a failed receipt), report whether CI agreed with it.
+ - uses: './.github/actions/local-ci-receipt'
+ name: 'Compare with the local CI receipt'
+ if: ${{ always() && steps.local-ci-receipt.outputs.parity != '' }}
+ continue-on-error: true
+ with:
+ job-name: ${{ matrix.name }}
+ github-token: ${{ github.token }}
+ parity: ${{ steps.local-ci-receipt.outputs.parity }}
+ ci-outcome: ${{ steps.run-tests.outcome }}
+
- name: 'Upload Playwright last run info'
# always upload the last run info, even if the test run passed
if: ${{ always() && matrix.report.resultsPath != '' }}
diff --git a/.github/workflows/pr-check-local-actions.yml b/.github/workflows/pr-check-local-actions.yml
index 173fd6b7e94..7f346751436 100644
--- a/.github/workflows/pr-check-local-actions.yml
+++ b/.github/workflows/pr-check-local-actions.yml
@@ -5,6 +5,7 @@ on:
paths:
- '.github/actions/**'
- '.github/*.json'
+ - '.github/workflows/ci.yml'
- '.github/workflows/pr-check-local-actions.yml'
permissions:
@@ -31,3 +32,42 @@ jobs:
# case-sensitive pattern passes on macOS and Windows when it should not.
- name: 'Run the checks'
run: node .github/actions/assign-reviewers/test.js
+
+ local-ci-receipt:
+ name: 'Validate local-ci.json'
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - name: Checkout
+ uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+ with:
+ sparse-checkout: .github
+ persist-credentials: false
+
+ - name: 'Setup Node'
+ uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
+ with:
+ node-version-file: '.nvmrc'
+
+ # The consumer and the tool both read this file; a syntax error or a
+ # missing field would silently turn substitution off everywhere.
+ - name: 'Validate .github/local-ci.json'
+ run: |
+ node -e '
+ const c = JSON.parse( require( "fs" ).readFileSync( ".github/local-ci.json", "utf8" ) );
+ for ( const k of [ "version", "repository", "app", "baseBranch", "receiptPrefix", "plan", "eligible" ] ) {
+ if ( ! ( k in c ) ) throw new Error( "local-ci.json is missing " + k );
+ }
+ if ( c.repository !== process.env.GITHUB_REPOSITORY ) throw new Error( "local-ci.json repository is " + c.repository + ", not " + process.env.GITHUB_REPOSITORY );
+ if ( typeof c.app.appId !== "number" ) throw new Error( "app.appId must be a number" );
+ if ( typeof c.enabled !== "boolean" ) throw new Error( "enabled must be a boolean" );
+ const crypto = require( "crypto" );
+ for ( const r of c.eligible ) {
+ if ( r.sampleRate !== undefined && ! ( Number.isInteger( r.sampleRate ) && r.sampleRate >= 0 && r.sampleRate <= 100 ) ) throw new Error( r.type + ": sampleRate must be an integer 0..100" );
+ const m = r.prepare && r.prepare.mirrors;
+ if ( ! m ) continue;
+ const actual = crypto.createHash( "sha256" ).update( require( "fs" ).readFileSync( m.action ) ).digest( "hex" );
+ if ( actual !== m.sha256 ) throw new Error( r.type + ": " + m.action + " changed (sha256 " + actual.slice( 0, 12 ) + "…); re-validate its prepare steps and update mirrors.sha256" );
+ }
+ console.log( "local-ci.json ok: enabled=" + c.enabled + ", rules=" + c.eligible.map( ( r ) => r.type ).join( "," ) );
+ '