Commit 3359d40b43a for woocommerce
commit 3359d40b43a5ee142cec9d54a5e143cb812add22
Author: Vlad Olaru <vlad.olaru@automattic.com>
Date: Fri Aug 21 18:05:59 2026 +0300
[tests] [e2e] Make the Blocks e2e seed re-runnable and fail on a broken step (#67707)
* fix(e2e): make the Blocks attribute fixture re-runnable
attributes.sh creates the Color and Size product attributes unguarded, so
it exits non-zero the second time it runs:
Error: Slug "color" is already in use. Change it, please.
Error: Slug "size" is already in use. Change it, please.
That is not an edge case. index.sh opens with `wp site empty`, which
deletes posts, comments and terms — but product attributes are rows in
woocommerce_attribute_taxonomies, not terms, so they survive it. Every
re-seed of an existing environment therefore hits this.
Today the failure is only noise in the build log, because index.sh does
not `set -e`. It also blocks turning that on, which is the actual cost:
the seed cannot fail loudly while one of its steps fails routinely.
Create only the attributes that are missing, following the guard #65861
introduced for the customer fixture.
Verified both directions: with the attributes already present the script
skips them and exits 0, and with them deleted it recreates both.
* fix(e2e): fail the Blocks seed loudly when a step breaks
index.sh runs six provisioning steps and, without `set -e`, ignored the
exit code of all but one of them. A failure in `wp site empty`,
attributes.sh, products.sh or the parallel fan-out left the script
exiting 0 with a half-seeded site.
That matters more than a normal script because of what happens next:
test-env-setup.sh snapshots the database once index.sh returns, and that
snapshot is restored before every test in the suite. A silent failure
here does not fail one spec — it hands the whole Blocks suite a subtly
wrong environment, which surfaces later as unrelated flakiness.
`set -euo pipefail` covers every step, which also makes the explicit
`|| exit 1` on the sorting fixture redundant. This was not previously
possible because attributes.sh failed on every re-seed; the preceding
commit fixes that.
Verified: the seed passes twice in a row against the same environment,
and passes against one with the attributes removed. With a failure
injected into attributes.sh, index.sh now exits non-zero and stops
before importing any products, where it previously carried on.
* fix(e2e): enable strict mode in every Blocks seed child script
`set -euo pipefail` in index.sh does not reach the scripts it launches:
shell options are not inherited across a `bash child.sh` invocation, and
none of the children set their own. index.sh therefore only catches a
child that *exits* non-zero, and a bash script exits with the status of
its last command.
So a child could fail in the middle and still report success.
attributes.sh is the clearest case, introduced earlier in this branch: if
creating the Color attribute failed and creating Size then succeeded, the
script exited 0 and the seed carried on to import products whose
variations reference the missing pa_color. Verified with a stubbed `wp`
on PATH — exit 0 before this commit, exit 1 after.
The same held for every script under parallel/, where `xargs` does report
a failing child (status 123), but only if that child exits non-zero in
the first place.
Enabling strict mode in each child closes the gap, and leaves the seed's
guarantee matching what index.sh already claims. languages.sh also picks
up the shebang and trailing newline it was missing.
Verified after the change: the seed passes twice in a row and against an
environment with the attributes removed; a mid-script failure injected
into parallel/payment.sh now aborts the seed at the fan-out with 123
instead of being swallowed; and the Blocks specs still pass (24).
* fix(e2e): make the Blocks shipping and tax fixtures re-runnable
`index.sh` empties the site with `wp site empty` before seeding, which
removes posts and terms but nothing WooCommerce keeps in its own tables.
`shipping.sh` and `tax.sh` create their rows unconditionally, so every
re-seed of an existing environment appended a fresh set: zone 0 grew from
two shipping methods to four, and the tax table from three rates to six,
with both scripts exiting 0.
Because nothing failed, the damage surfaced far from the seed. With four
methods on zone 0, `cart-checkout-block-shipping.block_theme.spec.ts`
tripped Playwright strict mode — `getByRole( 'radio', { name: 'Flat rate
shipping $' } )` resolved to two elements — and the "toggle is not shown
when shipping methods are disabled" test in
`checkout-block.shopper.block_theme.spec.ts` disabled the first and last
of what it assumes are exactly two toggles, leaving two enabled.
Clear both fixtures before recreating them. Going through
`WC_Shipping_Zone::delete_shipping_method()` and `WC_Tax::_delete_tax_rate()`
rather than deleting rows directly also drops the per-instance
`woocommerce_<method>_<id>_settings` options and the tax rate locations,
and invalidates the shipping and taxes caches.
Refs #67512
* fix(e2e): fail the Blocks env setup loudly when a post-seed step breaks
`test-env-setup.sh` guarded only its call to `index.sh`, with `|| exit 1`.
The three steps that follow it — clearing `woocommerce_coming_soon`,
writing the site editor welcome guide preferences, and generating the test
translations — ran unguarded.
That is the same hole this branch just closed one level down, and it has
the same consequence: the Playwright global setup snapshots the database
after this script returns, and every test restores that snapshot, so a
silently skipped preference write is restored before every test in the
suite rather than failing one spec.
Turn on `set -euo pipefail` for the whole script. The `|| exit 1` on the
`index.sh` call becomes redundant and is dropped.
Refs #67512
* docs(e2e): credit the right script with the Blocks database snapshot
The comment at the top of `index.sh` said `test-env-setup.sh` snapshots the
database once the seed returns. It does not: it only removes the existing
snapshot so a new one gets built. The export lives in the Playwright global
setup, `tests/e2e/fixtures/blocks-setup.ts`, and the per-test restore in
`tests/e2e/utils/blocks/test.ts`.
The consequence the comment describes is real — a bad seed is captured and
replayed before every test — so point at the code that actually does it,
otherwise the next reader looks for the export in the wrong file.
Refs #67512
* fix(e2e): fail the Blocks seed when a product gallery image is missing
The sample data imports its images from a remote host: sample_products.xml
carries `wp:attachment_url` entries pointing at
woocommercecore.mystagingwebsite.com. If one of those fetches fails while
`wp import` still exits 0, the attachment lookups that follow resolve to an
empty string rather than erroring, and the gallery write happily stores
`,,` and exits 0 too.
`set -u` does not catch this: the variables are set, they are just empty.
The other lookups in this script are self-guarding by accident, because
`wp post meta update "" key value` fails outright and `set -e` stops the
seed. The image case is different because the empties are interpolated into
a string argument instead of being used as the target ID, so the command
succeeds with garbage.
Check the three IDs before writing. One check covers both gallery writes,
since the beanie write reuses the same three variables.
Refs #67512
* fix(e2e): name the Blocks seed step that failed in the parallel fan-out
The ten parallel scripts run through `xargs -P10 -n1 bash`, and xargs
reports any child exit between 1 and 125 as its own exit 123. So the seed
died with 123 and no indication of which step failed or what it really
exited with, while ten scripts wrote interleaved output to the same log.
Wrap each invocation so a failing step prints its own path and exit code
before the seed stops. xargs still collapses its own exit to 123, which
cannot be changed from this side, but the log now identifies the step.
Refs #67512
diff --git a/plugins/woocommerce/changelog/blocks-e2e-seed-idempotency b/plugins/woocommerce/changelog/blocks-e2e-seed-idempotency
new file mode 100644
index 00000000000..e304b1aaee0
--- /dev/null
+++ b/plugins/woocommerce/changelog/blocks-e2e-seed-idempotency
@@ -0,0 +1,4 @@
+Significance: patch
+Type: dev
+
+Make the Blocks e2e seed re-runnable and fail loudly on a broken step.
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/attributes.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/attributes.sh
index 9068060e8f2..62f41f55529 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/attributes.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/attributes.sh
@@ -1,11 +1,24 @@
#!/usr/bin/env bash
-wp wc product_attribute create \
- --name=Color \
- --slug=pa_color \
- --user=1
-
-wp wc product_attribute create \
- --name=Size \
- --slug=pa_size \
- --user=1
+set -euo pipefail
+
+# `wp site empty` does not remove product attributes: they live in the
+# woocommerce_attribute_taxonomies table rather than being terms, so they
+# survive it. Creating them again on a re-seed fails with
+# `Slug "color" is already in use`, which is why this script only creates the
+# attributes that are missing.
+existing_slugs="$(wp wc product_attribute list --field=slug --user=1)"
+
+if ! printf '%s\n' "$existing_slugs" | grep -qx 'pa_color'; then
+ wp wc product_attribute create \
+ --name=Color \
+ --slug=pa_color \
+ --user=1
+fi
+
+if ! printf '%s\n' "$existing_slugs" | grep -qx 'pa_size'; then
+ wp wc product_attribute create \
+ --name=Size \
+ --slug=pa_size \
+ --user=1
+fi
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/index.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/index.sh
index fbe00fb26da..3cd184c1779 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/index.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/index.sh
@@ -1,11 +1,12 @@
#!/usr/bin/env bash
-# This script deliberately does not `set -e`. Some steps below are not
-# idempotent: attributes.sh exits non-zero once the attributes it creates
-# already exist, which is every re-seed of an existing environment, since
-# `wp site empty` leaves attribute taxonomies in place. Under `set -e` the
-# seed would die there, before any product is imported. Guard steps
-# individually until those steps are made re-runnable.
+# The Playwright global setup (tests/e2e/fixtures/blocks-setup.ts) exports the
+# database once this seed has run, and every test restores that snapshot before
+# it starts. So a step that fails silently here does not fail one spec, it hands
+# the whole suite a subtly wrong environment. Every step must therefore be
+# re-runnable, so that a re-seed of an existing environment is not mistaken for
+# a real failure.
+set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -19,12 +20,16 @@ bash "$script_dir/attributes.sh"
bash "$script_dir/products.sh"
# Run all scripts in parallel at maximum 10 at a time.
-find "$script_dir"/parallel/*.sh -maxdepth 1 -type f | xargs -P10 -n1 bash
+#
+# `xargs` reports any child failure as its own exit 123, so it names neither the
+# script that failed nor the code it failed with — and with ten of them writing
+# to the same log at once, the output alone does not identify it either. Wrap
+# each step so a failure announces itself before the seed dies.
+find "$script_dir"/parallel/*.sh -maxdepth 1 -type f | xargs -P10 -n1 bash -c \
+ 'bash "$0" || { status=$?; echo "Seed step failed: $0 (exit $status)" >&2; exit "$status"; }'
# Add deterministic ratings and sales data for product collection sorting.
-# Guarded because test-env-setup.sh snapshots the database once this returns, so
-# a half-seeded site would be restored before every test in the suite.
-bash "$script_dir/product-collection-sort-data.sh" || exit 1
+bash "$script_dir/product-collection-sort-data.sh"
# Run rewrite script last to ensure all posts are created before running it.
bash "$script_dir/rewrite.sh"
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/blog-name.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/blog-name.sh
index 890321149bf..2aa9f7203b9 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/blog-name.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/blog-name.sh
@@ -1,3 +1,5 @@
#!/usr/bin/env bash
+set -euo pipefail
+
wp option update blogname 'WooCommerce Blocks E2E Test Suite'
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/coupon.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/coupon.sh
index 2f46d6c02bf..7fe1786caf2 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/coupon.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/coupon.sh
@@ -1,3 +1,5 @@
#!/usr/bin/env bash
+set -euo pipefail
+
wp wc shop_coupon create --code=TESTCOUPON --amount=10 --discount_type=fixed_cart --user=1
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/customer.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/customer.sh
index 58bbd6623a1..229e0c474d1 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/customer.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/customer.sh
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
+set -euo pipefail
+
if ! wp user get customer --field=ID --path=/var/www/html >/dev/null 2>&1; then
wp user create customer customer@woocommerceblockse2etestsuite.com \
--user_pass=password \
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/languages.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/languages.sh
index eb733bee8ed..869cf0c4313 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/languages.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/languages.sh
@@ -1 +1,5 @@
-wp language core install nl_NL
\ No newline at end of file
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+wp language core install nl_NL
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/pages.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/pages.sh
index 8262ac9b9a8..6a2a91655fe 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/pages.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/pages.sh
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
+set -euo pipefail
+
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && cd ../../pages && pwd)"
post_id=$(wp post create \
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/payment.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/payment.sh
index ac1e1adbbc0..45b89e44512 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/payment.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/payment.sh
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
+set -euo pipefail
+
wp option set --format=json woocommerce_cod_settings '{
"enabled":"yes",
"title":"Cash on delivery",
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/posts.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/posts.sh
index 020dfebb296..fa44f2803d2 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/posts.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/posts.sh
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
+set -euo pipefail
+
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && cd ../../posts/ && pwd)"
function create_post {
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/reviews.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/reviews.sh
index 2d978310899..ada7a59dcb2 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/reviews.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/reviews.sh
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
+set -euo pipefail
+
# Add reviews to Hoodie. Order matters: specs treat the last created review as most recent.
# Include three distinct ratings so latest / highest / lowest are different reviews.
hoodie_post_id=$(wp post list --post_type=product --field=ID --name="Hoodie" --format=ids)
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/shipping.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/shipping.sh
index 7b9c250b491..5eddc57681a 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/shipping.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/shipping.sh
@@ -1,5 +1,21 @@
#!/usr/bin/env bash
+set -euo pipefail
+
+# `wp site empty` does not remove shipping zone methods: they are rows in the
+# woocommerce_shipping_zone_methods table rather than posts or terms, so they
+# survive it. Creating them again on a re-seed leaves zone 0 with four methods
+# instead of two, at exit 0, and the specs that rely on the seeded shape then
+# fail somewhere far from the seed. Clear the zone first so this is re-runnable.
+wp eval "$(cat <<'PHP'
+$zone = WC_Shipping_Zones::get_zone( 0 );
+
+foreach ( $zone->get_shipping_methods() as $method ) {
+ $zone->delete_shipping_method( $method->instance_id );
+}
+PHP
+)"
+
wp wc shipping_zone_method create 0 \
--order=1 \
--enabled=true \
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/tax.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/tax.sh
index d588f45313b..97a6dd13024 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/tax.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/parallel/tax.sh
@@ -1,7 +1,25 @@
#!/usr/bin/env bash
+set -euo pipefail
+
wp option set woocommerce_calc_taxes yes
+# `wp site empty` does not remove tax rates either: they are rows in the
+# woocommerce_tax_rates table, so a re-seed turns three rates into six at
+# exit 0. The duplicates do not change any total, since WooCommerce still
+# applies one rate per priority per class, but they leave the fixture lying
+# about its own shape. Clear them first so this is re-runnable.
+wp eval "$(cat <<'PHP'
+global $wpdb;
+
+$rate_ids = $wpdb->get_col( "SELECT tax_rate_id FROM {$wpdb->prefix}woocommerce_tax_rates" );
+
+foreach ( $rate_ids as $rate_id ) {
+ WC_Tax::_delete_tax_rate( $rate_id );
+}
+PHP
+)"
+
wp wc tax create \
--user=1 \
--rate=20 \
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/products.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/products.sh
index 164e2d4dc61..5d84d65ef53 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/products.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/products.sh
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
+set -euo pipefail
+
###################################################################################################
# Import sample products and regenerate product lookup tables
###################################################################################################
@@ -25,6 +27,19 @@ hoodie_product_id=$(wp post list --post_type=product --field=ID --name="Hoodie"
image1=$(wp post list --post_type=attachment --field=ID --name="hoodie-with-logo-2.jpg" --format=ids)
image2=$(wp post list --post_type=attachment --field=ID --name="hoodie-green-1.jpg" --format=ids)
image3=$(wp post list --post_type=attachment --field=ID --name="hoodie-2.jpg" --format=ids)
+
+# The sample data pulls its images from a remote host, so the import can lose an
+# attachment while still exiting 0. A missing attachment resolves to an empty
+# string here, not an error, and `set -u` does not catch it because the variable
+# is set, just empty. Without this check the next line writes a `,,` gallery and
+# exits 0, and the gallery specs fail with no trace back to the import.
+for image_id in "$image1" "$image2" "$image3"; do
+ [ -n "$image_id" ] || {
+ echo "Missing gallery attachment; the sample-data image import did not complete." >&2
+ exit 1
+ }
+done
+
wp post meta update $hoodie_product_id _product_image_gallery "$image1,$image2,$image3"
# Create a tag, so we can add tests for tag-related blocks and templates.
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/rewrite.sh b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/rewrite.sh
index 84de3bcfaf5..f1d134dc5ec 100644
--- a/plugins/woocommerce/tests/e2e/bin/blocks/scripts/rewrite.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/scripts/rewrite.sh
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
+set -euo pipefail
+
# Currently, the rewrite rules don't work properly in the test environment: https://github.com/WordPress/gutenberg/issues/28201
chmod -c ugo+w /var/www/html
wp rewrite structure /%postname%/ --hard
diff --git a/plugins/woocommerce/tests/e2e/bin/blocks/test-env-setup.sh b/plugins/woocommerce/tests/e2e/bin/blocks/test-env-setup.sh
index 2382159fe94..c7630ba0ad1 100755
--- a/plugins/woocommerce/tests/e2e/bin/blocks/test-env-setup.sh
+++ b/plugins/woocommerce/tests/e2e/bin/blocks/test-env-setup.sh
@@ -1,4 +1,10 @@
#!/usr/bin/env bash
+
+# The steps after the seed are as easy to lose as the steps inside it: a
+# silently skipped preference write or translation build is restored before
+# every test along with everything else. Fail on the first broken one.
+set -euo pipefail
+
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Command prefix for running wp-cli against the single-container E2E environment
@@ -8,7 +14,7 @@ wp_cli="wp-env --config .wp-env.e2e.json run cli"
# Remove the database snapshot if it exists.
$wp_cli -- rm -f blocks_e2e.sql
# Run the main script in the container for better performance.
-$wp_cli -- bash wp-content/plugins/woocommerce/blocks-bin/playwright/scripts/index.sh || exit 1
+$wp_cli -- bash wp-content/plugins/woocommerce/blocks-bin/playwright/scripts/index.sh
# Disable the LYS Coming Soon banner.
$wp_cli -- wp option update woocommerce_coming_soon 'no'
# Dismiss the site editor welcome guide for the admin user so it does not