Commit 0719af8194f for woocommerce
commit 0719af8194f7e758abe802db9e5984c90447a29c
Author: malinajirka <malinajirka@gmail.com>
Date: Thu Jul 2 08:58:25 2026 +0200
Generate POS catalog feed in chunks (#65860)
* Detect and recover stuck POS catalog feed generation jobs
When a large catalog feed is generated, the whole walk runs inside a single
Action Scheduler action. If the server kills that process (request/PHP timeout
or out of memory) before it finishes, the catch block never runs, so the status
option stays pinned at `in_progress` forever. Nothing reconciles it:
validate_status() only had a staleness check for the `scheduled` state, and
force_regeneration() refused to touch an `in_progress` job. The catalog could
then never be regenerated.
Add a heartbeat (`updated_at`) that is refreshed when a job is scheduled, when
it transitions to `in_progress`, and after every processed batch. validate_status()
now treats an `in_progress` job whose heartbeat is older than a filterable timeout
(woocommerce_product_feed_in_progress_timeout, default 5 minutes) as invalid, so
both status polling and force_regeneration() automatically reschedule a fresh job.
A genuinely running job keeps its heartbeat fresh and is left untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Extend per-batch PHP time limit during POS catalog feed generation
ProductWalker already supported resetting PHP's execution time limit per batch
via add_time_limit(), but AsyncGenerator never called it, so generation relied
on whatever max_execution_time the host applied to the Action Scheduler request
(often as low as 30s). Reset the time limit after each batch via the new
woocommerce_product_feed_batch_time_limit filter (default 5 minutes) so a low
max_execution_time does not abort a large catalog mid-walk.
This is a mitigation, not a guarantee: it does not extend Action Scheduler's
300s failure period or a hard host request timeout. The heartbeat-based recovery
remains the safety net when a job is killed regardless of the cause.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Raise memory and execution time limits during POS catalog feed generation
The previous change reset PHP's execution time limit after each batch, but the
walker only does so once a batch completes, so the initial product query and the
first batch still ran under whatever (often low) limit the Action Scheduler
request started with. Raise the time limit up front as well, and raise the memory
limit before the walk begins, since large catalogs are memory heavy and out of
memory is a likely cause of the process being killed.
These raises are scoped to the generating process and never lower an existing
higher limit. They cannot override a hard host/server request timeout or Action
Scheduler's failure period, which remain the ceilings that only chunked
processing can remove.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Generate POS catalog feed in chunks across multiple actions
Previously the entire catalog feed was built inside a single Action Scheduler
action. On large catalogs that one run exceeded the host request timeout / out
of memory or Action Scheduler's 300s failure period, killing the job.
Process the feed in bounded chunks instead: each action walks up to
woocommerce_product_feed_chunk_size products (default 2500), persists its
progress, and schedules + dispatches the next chunk, so no single run is long
enough to be killed. The feed file is written across processes via new
FeedInterface lifecycle methods (begin/resume/flush/finalize) into the shared
upload directory, with the cursor (file name, page, entries written) tracked in
the status option. ProductWalker gains walk_chunk() for bounded, resumable walks.
New filters: woocommerce_product_feed_chunk_size (products per chunk) and
woocommerce_product_feed_batch_size (products per DB query). Internal cursor
fields are stripped from the REST response.
The heartbeat-based recovery from the previous change still applies: if a chunk
is killed, the stale heartbeat causes generation to restart cleanly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Simplify POS catalog feed lifecycle and adapt chunk size on stuck
Collapse the feed storage API: fold begin/can_resume/resume into start()
and finalize() into end(), merge ProductWalker::walk_chunk() back into
walk(), and drop the now-dead temp-dir path. Also fix entries_written
being double-counted across chunks (it is already cumulative).
Make the chunk size adaptive instead of fixed: generation now starts at a
large size so most catalogs finish in a single action (no inter-action
latency), and only steps the size down a ladder (100000 -> 2500 -> 1000)
when a run is detected stuck. The effective size is persisted per feed so
the next request from the app does not repeat the attempt that failed.
Stop resuming stuck jobs: a stale in-progress job is now discarded and
restarted fresh, which avoids appending to a partial file left by a
killed run. Feed-generation logs now include the chunk size.
* Drop resume-after-kill fallback from POS catalog feed generation
On a stuck job, generation already restarts from scratch (the partial is
discarded and a fresh feed begins). The defensive machinery to resume a
killed run mid-file was therefore only stitching together successful
chunks. Remove it: a continuation whose partial file has vanished now
fails (and recovers on the next poll) instead of silently falling back.
Also trim comments to business rationale only and simplify the feed
lifecycle (no reopen-after-flush in end(), single is_file guard on
resume).
* Consolidate and correct POS catalog feed changelog entries
Merge the two POS catalog changelog entries into one and drop the stale
claim that a stalled run resumes from the last completed chunk; it now
restarts from scratch.
* Add changefile(s) from automation for the following project(s): woocommerce
* Fix PHPStan errors in POS catalog feed walker and storage
Drop the dead null-coalesce fallback on the walker's return value (the
do-while always assigns $progress on its first iteration) and have
open_handle() return the validated resource so the feed's initial
fwrite() operates on a definite resource rather than resource|false|null.
* Regenerate POS catalog feed on failed status instead of stalling
A failed generation job (e.g. a continuation whose partial feed file
vanished) was a terminal state: validate_status() reported it as valid,
so a status poll returned `failed` unchanged and only an explicit
force_regeneration() could restart it. That pushed an implementation
detail onto the client.
Treat `failed` as invalid in validate_status() so an ordinary status
poll discards the partial feed and starts a fresh generation, the same
way stuck and expired jobs are already handled. The now-unreachable
`failed` case in force_regeneration() is removed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Strip internal updated_at heartbeat from feed status API response
The generate_feed endpoint already removes internal fields (action_id,
path, file_name, page, entries_written), but the new updated_at
heartbeat leaked through as a raw Unix timestamp. Add it to the unset
list and extend ApiControllerTest to assert every internal field is
omitted from the response.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Consolidate POS catalog chunked-generation changelog into one entry
There were two changelog files for the same feature: a real entry and a
Comment-only duplicate. Keep a single entry and fold the fuller wording
(chunk-size reduction, raised execution/memory limits) into it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix JsonFileFeed alignment warning and dead file_handle type
Align the $handle assignment to satisfy phpcs, and drop `false` from the
$file_handle property type: open_handle() now throws on a failed fopen()
rather than storing false, so the type is only ever resource|null.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add changefile(s) from automation for the following project(s): woocommerce
* Harden POS catalog feed deletion against path traversal
Feed identifiers and paths round-trip through the persisted status option
and FeedInterface::delete() is callable by extensions, so a corrupted or
hostile value must never escape the feed directory.
- JsonFileFeed: validate that a feed identifier is a plain `.json` file
name before concatenating it into a path. start() throws on a bad
resume identifier; delete() ignores one rather than removing a file
elsewhere.
- AsyncGenerator: discard_feed() now prefers the feed-owned file_name
(validated by delete()) and only falls back to a raw status path after
confirming it resolves inside the feed upload directory. The completed
case of force_regeneration() routes through discard_feed() too.
Addresses CodeRabbit review on PR #65860.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Simplify discard_feed() by deleting legacy feeds via their basename
* Surface failed POS catalog generation instead of silently retrying
* Widen POS catalog stuck timeout beyond the batch budget to avoid false recovery
* Stub create_feed() in discard_feed tests so deletion actually runs
The two discard_feed() tests mocked POSIntegration but never stubbed
create_feed(), which returns a FeedInterface. PHPUnit therefore auto-
returned a test double whose delete() is a silent no-op, so discard_feed()
never deleted anything:
- test_discard_feed_deletes_legacy_path_inside_feed_dir failed because the
real partial feed it created was never removed (assertFalse(file_exists)
saw the file still there).
- test_discard_feed_does_not_delete_path_outside_feed_dir passed only
trivially, since the no-op delete() can't remove the sentinel regardless,
so it never actually exercised the feed-directory confinement.
Stub create_feed() with a real JsonFileFeed (as the other tests already do)
so delete() runs for real: the inside-dir case now deletes the file and the
outside-dir case is confined to the feed directory and leaves the sentinel
untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Step POS catalog chunk size down on a forced rebuild of a stuck job too
force_regeneration() discarded and deleted a stuck in-progress job's status
before delegating to get_status(), so get_status() never saw the in-progress
state and never reduced the chunk size. A forced rebuild of a job that was
killed for being too large therefore restarted at the same size and was prone
to dying the same way, while an ordinary force=false poll would have stepped it
down.
Extract the "reduce when the invalid status was a stuck in-progress job" rule
into reduce_chunk_size_if_stuck() and call it from both get_status() and
force_regeneration(), so a stuck job adapts its chunk size the same way however
it is recovered. A genuine failure (state = failed) stays excluded — it is a
real error, not a size symptom. Extend the force_regeneration stalled-job test
to assert the step-down.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Document the 11.0.0 addition of FeedInterface::start() parameters
start() shipped in 10.5.0; the $resume_identifier and $entries_written
parameters were added in 11.0.0 for chunked generation. Add the matching
@since line so the docblock records when the signature changed, consistent
with flush() and delete() in the same interface.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Keep the "chunk" concept out of ProductWalker
A chunk is the AsyncGenerator's unit of work (one Action Scheduler action);
the walker just processes a bounded number of batches, and its signature
(`$start_page`, `$max_batches`) is already generic. Rename the local
`$batches_in_chunk` counter to `$batches_processed` and drop the chunk
references from the walk() docblock and loop comment so the walker no longer
leaks a caller-level concept.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Keep chunked POS feed generation backward compatible (#66112)
* Restore FeedInterface BC via a separate ResumableFeedInterface
The chunked POS catalog generation changed the released FeedInterface in
three backward-incompatible ways (start() signature, plus required
flush() and delete()), which fatals external implementers such as the
WooCommerce Stripe Gateway -- the same class of break that reverted
10.9.0.
Revert FeedInterface to its released shape and move the chunked lifecycle
to a new additive ResumableFeedInterface (open/flush/delete/
get_entry_count) that JsonFileFeed also implements. Restore the
lifecycle-owning ProductWalker::walk(): int and add the bounded
walk_batches(): WalkerProgress alongside it. AsyncGenerator and the POS
feed factory work against the resumable contract.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Update plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGenerator.php
Co-authored-by: Jaclyn Chen <watertranquil@gmail.com>
* Update plugins/woocommerce/src/Internal/ProductFeed/Storage/JsonFileFeed.php
Co-authored-by: Jaclyn Chen <watertranquil@gmail.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Jaclyn Chen <watertranquil@gmail.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: woocommercebot <woocommercebot@users.noreply.github.com>
Co-authored-by: Jaclyn Chen <watertranquil@gmail.com>
diff --git a/plugins/woocommerce/changelog/65860-feature-pos-catalog-chunked-generation b/plugins/woocommerce/changelog/65860-feature-pos-catalog-chunked-generation
new file mode 100644
index 00000000000..e7303aeda88
--- /dev/null
+++ b/plugins/woocommerce/changelog/65860-feature-pos-catalog-chunked-generation
@@ -0,0 +1,4 @@
+Significance: patch
+Type: fix
+
+Generate the POS catalog feed in chunks across several Action Scheduler actions so large catalogs no longer exceed the host request timeout or Action Scheduler's failure period mid-generation, and detect and restart stuck generation jobs that would otherwise stay pinned in progress forever.
\ No newline at end of file
diff --git a/plugins/woocommerce/src/Internal/ProductFeed/Feed/ProductWalker.php b/plugins/woocommerce/src/Internal/ProductFeed/Feed/ProductWalker.php
index 9d2351e8bfd..83f72675ee9 100644
--- a/plugins/woocommerce/src/Internal/ProductFeed/Feed/ProductWalker.php
+++ b/plugins/woocommerce/src/Internal/ProductFeed/Feed/ProductWalker.php
@@ -195,32 +195,70 @@ class ProductWalker {
}
/**
- * Walks through all products.
+ * Walks through every remaining product in one go, managing the feed lifecycle.
+ *
+ * This is the simple, single-process entry point: it starts the feed, walks every batch, ends the
+ * feed and returns the number of products processed. Callers that need to write a feed across
+ * several processes should own the feed lifecycle themselves and use {@see walk_batches()} instead.
*
* @since 10.5.0
*
- * @param callable $callback The callback to call after each batch of products is processed.
- * @return int The total number of products processed.
+ * @param callable|null $callback The callback to call after each batch of products is processed.
+ * @return int The number of products processed.
*/
public function walk( ?callable $callback = null ): int {
- $progress = null;
-
- // Instruct the feed to start.
$this->feed->start();
+ $progress = $this->walk_batches( $callback );
+ $this->feed->end();
+
+ return $progress->processed_items;
+ }
+
+ /**
+ * Walks through products, optionally limited to a bounded number of batches.
+ *
+ * The walker does not own the feed lifecycle: the caller is responsible for starting, flushing
+ * and ending the feed. This lets a feed be written across several processes by resuming the same
+ * feed between calls.
+ *
+ * Called with the defaults it walks every remaining page in one go; pass `$start_page` and
+ * `$max_batches` to process a bounded slice and resume later from `processed_batches`.
+ *
+ * @since 11.0.0
+ *
+ * @param callable|null $callback The callback to call after each batch of products is processed.
+ * @param int $start_page The 1-based page (batch) to start at.
+ * @param int $max_batches The maximum number of batches to process in this call.
+ * @return WalkerProgress Items/batches processed here, plus the overall total_count and
+ * total_batch_count so the caller knows whether the feed is complete.
+ */
+ public function walk_batches( ?callable $callback = null, int $start_page = 1, int $max_batches = PHP_INT_MAX ): WalkerProgress {
+ if ( $start_page < 1 ) {
+ $start_page = 1;
+ }
+ if ( $max_batches < 1 ) {
+ $max_batches = 1;
+ }
+
+ $progress = null;
+ $page = $start_page;
// Check how much memory is available at first.
$initial_available_memory = $this->memory_manager->get_available_memory();
+ $batches_processed = 0;
do {
- $result = $this->iterate( $this->query_args, $progress ? $progress->processed_batches + 1 : 1, $this->per_page );
+ $result = $this->iterate( $this->query_args, $page, $this->per_page );
$iterated = count( $result->products );
- // Only done when the progress is not set. Will be modified otherwise.
+ // Only build the progress object once; the total/total batch count is stable across pages.
if ( is_null( $progress ) ) {
$progress = WalkerProgress::from_wc_get_products_result( $result );
}
$progress->processed_items += $iterated;
++$progress->processed_batches;
+ ++$batches_processed;
+ ++$page;
if ( is_callable( $callback ) && $iterated > 0 ) {
$callback( $progress );
@@ -239,14 +277,15 @@ class ProductWalker {
// If `wc_get_products()` returns less than the batch size, it was the last page.
$iterated === $this->per_page
- // For the cases where the above is true, make sure that we do not exceed the total number of pages.
- && $progress->processed_batches < $progress->total_batch_count
- );
+ // Stop once this call has processed its requested number of batches.
+ && $batches_processed < $max_batches
- // Instruct the feed to end.
- $this->feed->end();
+ // For the cases where the above are true, make sure that we do not exceed the total number of pages.
+ && ( $progress->total_batch_count <= 0 || ( $page - 1 ) < $progress->total_batch_count )
+ );
- return $progress->processed_items;
+ // The do-while body always executes at least once and assigns $progress on the first iteration.
+ return $progress;
}
/**
diff --git a/plugins/woocommerce/src/Internal/ProductFeed/Feed/ResumableFeedInterface.php b/plugins/woocommerce/src/Internal/ProductFeed/Feed/ResumableFeedInterface.php
new file mode 100644
index 00000000000..9d4d16bca8e
--- /dev/null
+++ b/plugins/woocommerce/src/Internal/ProductFeed/Feed/ResumableFeedInterface.php
@@ -0,0 +1,71 @@
+<?php
+/**
+ * Resumable Feed Interface.
+ *
+ * @package Automattic\WooCommerce\Internal\ProductFeed
+ */
+
+declare(strict_types=1);
+
+namespace Automattic\WooCommerce\Internal\ProductFeed\Feed;
+
+/**
+ * Contract for feeds that can be written across multiple processes (chunked generation).
+ *
+ * This is deliberately kept separate from {@see FeedInterface}: a feed backend opts into chunked,
+ * resumable generation by implementing this interface *in addition to* FeedInterface. Existing
+ * FeedInterface implementations (including those provided by third-party integrations) remain
+ * valid and are unaffected, so adding this contract is a backwards-compatible change.
+ *
+ * @since 11.0.0
+ */
+interface ResumableFeedInterface extends FeedInterface {
+ /**
+ * Start a feed fresh, or resume one that a previous chunk began.
+ *
+ * A feed may be written across separate processes (one Action Scheduler action per chunk), so it
+ * lives in a stable, shared location identified by the returned value. Pass that identifier back on
+ * a later chunk to keep appending to the same feed; pass nothing to begin a new one.
+ *
+ * @since 11.0.0
+ *
+ * @param string|null $resume_identifier Identifier of an existing feed to resume, or null to start fresh.
+ * @param int $entries_written The number of entries already written by previous chunks, so
+ * separators are added correctly when resuming.
+ * @return string The identifier of the feed that was started, to be passed back by later chunks.
+ */
+ public function open( ?string $resume_identifier = null, int $entries_written = 0 ): string;
+
+ /**
+ * Persist the current chunk and release the file handle without finalizing the feed.
+ *
+ * Called at the end of a chunk that is not the last one, so a later chunk can resume.
+ *
+ * @since 11.0.0
+ *
+ * @return void
+ */
+ public function flush(): void;
+
+ /**
+ * Delete a feed (e.g. a partial feed left by an abandoned chunked generation).
+ *
+ * @since 11.0.0
+ *
+ * @param string $identifier The identifier returned by {@see open()}.
+ * @return void
+ */
+ public function delete( string $identifier ): void;
+
+ /**
+ * Get the number of entries that have been written to the feed.
+ *
+ * This reflects the rows actually written, which may be fewer than the number of products
+ * iterated, because the validator can silently drop entries before they are added.
+ *
+ * @since 11.0.0
+ *
+ * @return int Number of entries written to the feed.
+ */
+ public function get_entry_count(): int;
+}
diff --git a/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/ApiController.php b/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/ApiController.php
index fca3ab01edb..71074f03b21 100644
--- a/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/ApiController.php
+++ b/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/ApiController.php
@@ -114,13 +114,15 @@ class ApiController {
$response['completed_at'] = wc_rest_prepare_date_response( $response['completed_at'] );
}
- // Remove sensitive data from the response.
- if ( isset( $response['action_id'] ) ) {
- unset( $response['action_id'] );
- }
- if ( isset( $response['path'] ) ) {
- unset( $response['path'] );
- }
+ // Remove sensitive and internal data from the response.
+ unset(
+ $response['action_id'],
+ $response['path'],
+ $response['file_name'],
+ $response['page'],
+ $response['entries_written'],
+ $response['updated_at']
+ );
} catch ( \Exception $e ) {
wc_get_logger()->error(
'Feed generation failed',
diff --git a/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGenerator.php b/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGenerator.php
index 29ea697e1d0..52f2d1acfbf 100644
--- a/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGenerator.php
+++ b/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGenerator.php
@@ -12,6 +12,7 @@ namespace Automattic\WooCommerce\Internal\ProductFeed\Integrations\POSCatalog;
use ActionScheduler_AsyncRequest_QueueRunner;
use ActionScheduler_Store;
use Automattic\WooCommerce\Internal\ProductFeed\Feed\ProductWalker;
+use Automattic\WooCommerce\Internal\ProductFeed\Feed\ResumableFeedInterface;
use Automattic\WooCommerce\Internal\ProductFeed\Feed\WalkerProgress;
if ( ! defined( 'ABSPATH' ) ) {
@@ -54,6 +55,24 @@ class AsyncGenerator {
const STATE_COMPLETED = 'completed';
const STATE_FAILED = 'failed';
+ /**
+ * The number of products fetched per database batch.
+ *
+ * @var int
+ */
+ const BATCH_SIZE = 100;
+
+ /**
+ * The chunk sizes (products processed per action) tried in descending order.
+ *
+ * Generation starts at the first (largest) size so most catalogs finish in a single action. When a
+ * run gets stuck — likely killed because the size was too large for the host — the size steps down
+ * one rung and is persisted, so future runs do not repeat the attempt that failed.
+ *
+ * @var int[]
+ */
+ const CHUNK_SIZE_STEPS = array( 100000, 2500, 1000 );
+
/**
* Integration instance.
*
@@ -93,18 +112,31 @@ class AsyncGenerator {
* @return array The feed generation status.
*/
public function get_status( ?array $args = null ): array {
- // Determine the option key based on the integration ID and arguments.
$option_key = $this->get_option_key( $args );
$status = get_option( $option_key );
- // For existing jobs, make sure that everything in the status makes sense.
- if ( is_array( $status ) && ! $this->validate_status( $status ) ) {
- $status = false;
- }
-
- // If the status is an array, it means that there is nothing to schedule in this method.
if ( is_array( $status ) ) {
- return $status;
+ if ( $this->validate_status( $status ) ) {
+ return $status;
+ }
+
+ // Surface a failed generation to the client once, then clear it so the next poll starts a
+ // fresh run. The POS clients are built to read `failed`, stop, and let their own scheduling
+ // drive the next attempt, so the server must report the failure rather than silently retry
+ // it. Clearing the status (rather than leaving it sticky) matters because those clients poll
+ // again with force=false, which would otherwise keep re-reading the same failure forever.
+ if ( self::STATE_FAILED === ( $status['state'] ?? '' ) ) {
+ $this->discard_feed( $status );
+ delete_option( $option_key );
+ return $status;
+ }
+
+ // A stuck in-progress job most likely died because its chunk was too large for this host;
+ // step the size down so the restart is more likely to fit.
+ $this->reduce_chunk_size_if_stuck( $status, $option_key );
+
+ // Whatever made the status invalid (stuck, expired, …), discard the partial feed and start fresh.
+ $this->discard_feed( $status );
}
// Clear all previous actions to avoid race conditions.
@@ -121,28 +153,38 @@ class AsyncGenerator {
'args' => $args ?? array(),
);
- update_option(
- $option_key,
- $status
- );
+ update_option( $option_key, $status );
+
+ $this->schedule_generation_action( $option_key );
- // Start an immediate async action to generate the feed.
+ return $status;
+ }
+
+ /**
+ * Schedules (and immediately dispatches) an async action to process a feed generation chunk.
+ *
+ * @param string $option_key The option key for the feed generation status.
+ * @return void
+ */
+ private function schedule_generation_action( string $option_key ): void {
+ // Deliberately not enqueued as "unique": Action Scheduler's uniqueness check matches on hook +
+ // group only (not args) and treats a running action as a blocker, so a unique enqueue of the next
+ // chunk would be rejected while the current chunk's action is still running. Per-job de-duplication
+ // is handled by as_unschedule_all_actions() in get_status() instead.
as_enqueue_async_action(
self::FEED_GENERATION_ACTION,
array( $option_key ),
'woo-product-feed',
- true,
+ false,
1
);
- // Manually force an async request to be dispatched to process the action immediately.
+ // Force an async request so the action runs immediately.
if ( class_exists( ActionScheduler_AsyncRequest_QueueRunner::class ) && class_exists( ActionScheduler_Store::class ) ) {
$store = ActionScheduler_Store::instance();
$async_request = new ActionScheduler_AsyncRequest_QueueRunner( $store );
$async_request->dispatch();
}
-
- return $status;
}
/**
@@ -156,92 +198,91 @@ class AsyncGenerator {
public function feed_generation_action( string $option_key ) {
$status = get_option( $option_key );
- if ( ! is_array( $status ) || ! isset( $status['state'] ) || self::STATE_SCHEDULED !== $status['state'] ) {
+ // Only a scheduled (first chunk) or in-progress (continuation) job should be processed here.
+ if ( ! is_array( $status ) || ! in_array( $status['state'] ?? '', array( self::STATE_SCHEDULED, self::STATE_IN_PROGRESS ), true ) ) {
wc_get_logger()->error( 'Invalid feed generation status', array( 'status' => $status ) );
return;
}
+ $is_first_chunk = self::STATE_SCHEDULED === $status['state'];
+
+ // A continuation must know which feed file it is appending to. If it doesn't, the status is
+ // corrupt; bail and let the heartbeat-based recovery restart generation from scratch.
+ if ( ! $is_first_chunk && empty( $status['file_name'] ) ) {
+ wc_get_logger()->error( 'Invalid feed generation continuation status', array( 'status' => $status ) );
+ return;
+ }
+
$status['state'] = self::STATE_IN_PROGRESS;
$status['updated_at'] = time();
update_option( $option_key, $status );
+ $feed = null;
try {
- // Large catalogs are memory heavy, so give the process as much headroom as the
- // host allows before the heavy lifting begins. This only raises the limit for the
- // current process and never lowers an already higher limit.
- wp_raise_memory_limit( 'admin' );
-
- /**
- * Filters the per-batch PHP execution time limit (in seconds) for product feed generation.
- *
- * The execution time limit is set to this value up front and reset to it after each processed
- * batch, so that a low `max_execution_time` does not abort generation part-way through a large
- * catalog. Return 0 to leave the time limit untouched.
- *
- * This only affects PHP's own execution timeout. It does not extend Action Scheduler's
- * failure period (`action_scheduler_failure_period`, 300 seconds by default) nor any hard
- * server/host request timeout, so it is a mitigation rather than a guarantee for very large
- * catalogs.
- *
- * @param int $batch_time_limit The per-batch time limit in seconds.
- *
- * @since 11.0.0
- */
- $batch_time_limit = (int) apply_filters( 'woocommerce_product_feed_batch_time_limit', 5 * MINUTE_IN_SECONDS );
-
- // Raise the time limit up front too: the walker only resets it after each batch, so the
- // initial product query and the first batch would otherwise run under whatever (possibly
- // very low) limit the Action Scheduler request started with.
- if ( $batch_time_limit > 0 ) {
- wc_set_time_limit( $batch_time_limit );
+ $this->raise_resource_limits();
+
+ $feed = $this->integration->create_feed();
+
+ if ( $is_first_chunk ) {
+ $status['file_name'] = $feed->open();
+ $status['page'] = 1;
+ $status['processed'] = 0;
+ $status['entries_written'] = 0;
+ update_option( $option_key, $status );
+ } else {
+ $feed->open( (string) $status['file_name'], (int) ( $status['entries_written'] ?? 0 ) );
}
- $feed = $this->integration->create_feed();
$walker = ProductWalker::from_integration( $this->integration, $feed );
- $walker->add_time_limit( $batch_time_limit );
-
- // Add dynamic args to the mapper.
- $args = $status['args'] ?? array();
- if (
- isset( $args['_product_fields'] )
- && is_string( $args['_product_fields'] ) &&
- ! empty( $args['_product_fields'] )
- ) {
- $this->integration->get_product_mapper()->set_fields( $args['_product_fields'] );
- }
- if (
- isset( $args['_variation_fields'] )
- && is_string( $args['_variation_fields'] ) &&
- ! empty( $args['_variation_fields'] )
- ) {
- $this->integration->get_product_mapper()->set_variation_fields( $args['_variation_fields'] );
- }
-
- $walker->walk(
- function ( WalkerProgress $progress ) use ( &$status, $option_key ) {
- $status = $this->update_feed_progress( $status, $progress );
+ $walker->set_batch_size( $this->get_batch_size() );
+ $walker->add_time_limit( $this->get_batch_time_limit() );
+
+ $this->apply_mapper_args( $status['args'] ?? array() );
+
+ $start_page = max( 1, (int) ( $status['page'] ?? 1 ) );
+ $base_processed = (int) ( $status['processed'] ?? 0 );
+ $progress = $walker->walk_batches(
+ function ( WalkerProgress $progress ) use ( &$status, $option_key, $base_processed ) {
+ // Refresh progress and the heartbeat after every batch, so polling sees smooth progress
+ // within a chunk rather than a single jump at the chunk boundary.
+ $status = $this->update_progress( $status, $base_processed + $progress->processed_items, $progress->total_count );
update_option( $option_key, $status );
- }
+ },
+ $start_page,
+ $this->get_chunk_batch_count( $option_key )
);
- // Store the final details.
- $status['state'] = self::STATE_COMPLETED;
- $status['url'] = $feed->get_file_url();
- $status['path'] = $feed->get_file_path();
- $status['completed_at'] = time();
- update_option( $option_key, $status );
-
- // Schedule another action to delete the file after the expiry time.
- as_schedule_single_action(
- time() + self::FEED_EXPIRY,
- self::FEED_DELETION_ACTION,
- array(
- $option_key,
- $feed->get_file_path(),
- ),
- 'woo-product-feed',
- false
- );
+ // The feed's entry count is already cumulative across chunks (open() seeds it with the running
+ // total when resuming), so store it as-is rather than adding to the previous total.
+ $status = $this->update_progress( $status, $base_processed + $progress->processed_items, $progress->total_count );
+ $status['entries_written'] = $feed->get_entry_count();
+ $status['page'] = $start_page + $progress->processed_batches;
+
+ $is_complete = $progress->total_batch_count <= 0 || (int) $status['page'] > $progress->total_batch_count;
+
+ if ( $is_complete ) {
+ $feed->end();
+
+ $status['state'] = self::STATE_COMPLETED;
+ $status['progress'] = 100;
+ $status['url'] = $feed->get_file_url();
+ $status['path'] = $feed->get_file_path();
+ $status['completed_at'] = time();
+ update_option( $option_key, $status );
+
+ // Schedule deletion of the file after the expiry time.
+ as_schedule_single_action(
+ time() + self::FEED_EXPIRY,
+ self::FEED_DELETION_ACTION,
+ array( $option_key, $feed->get_file_path() ),
+ 'woo-product-feed',
+ false
+ );
+ } else {
+ $feed->flush();
+ update_option( $option_key, $status );
+ $this->schedule_generation_action( $option_key );
+ }
} catch ( \Throwable $e ) {
wc_get_logger()->error(
'Feed generation failed',
@@ -251,6 +292,11 @@ class AsyncGenerator {
)
);
+ // Release the file handle, if any, so it is not left dangling.
+ if ( $feed instanceof ResumableFeedInterface ) {
+ $feed->flush();
+ }
+
$status['state'] = self::STATE_FAILED;
$status['error'] = $e->getMessage();
$status['failed_at'] = time();
@@ -258,6 +304,223 @@ class AsyncGenerator {
}
}
+ /**
+ * Raises the memory and execution time limits for the current process before heavy work begins.
+ *
+ * These only affect the current process and never lower an already higher limit. They cannot
+ * override a hard host/server request timeout or Action Scheduler's failure period.
+ *
+ * @return void
+ */
+ private function raise_resource_limits(): void {
+ wp_raise_memory_limit( 'admin' );
+
+ // Raise the time limit up front: the walker only resets it after each batch, so the initial
+ // product query and the first batch would otherwise run under the request's (possibly low) limit.
+ $batch_time_limit = $this->get_batch_time_limit();
+ if ( $batch_time_limit > 0 ) {
+ wc_set_time_limit( $batch_time_limit );
+ }
+ }
+
+ /**
+ * Returns the per-batch PHP execution time limit (in seconds) for feed generation.
+ *
+ * @return int The per-batch time limit in seconds.
+ */
+ private function get_batch_time_limit(): int {
+ /**
+ * Filters the per-batch PHP execution time limit (in seconds) for product feed generation.
+ *
+ * The limit is set up front and reset after each processed batch, so a low `max_execution_time`
+ * does not abort generation part-way through a chunk. Return 0 to leave the time limit untouched.
+ * This only affects PHP's own execution timeout, not Action Scheduler's failure period nor any
+ * hard server/host request timeout.
+ *
+ * @param int $batch_time_limit The per-batch time limit in seconds.
+ *
+ * @since 11.0.0
+ */
+ return (int) apply_filters( 'woocommerce_product_feed_batch_time_limit', 5 * MINUTE_IN_SECONDS );
+ }
+
+ /**
+ * Returns the number of batches to process per chunk, derived from the effective chunk size.
+ *
+ * @param string $option_key The option key for the feed generation status.
+ * @return int The number of batches per chunk (at least 1).
+ */
+ private function get_chunk_batch_count( string $option_key ): int {
+ /**
+ * Filters the number of products processed per chunk during feed generation.
+ *
+ * Each chunk runs in its own Action Scheduler action and then schedules the next, keeping every
+ * run short enough to finish within Action Scheduler's failure period and the host's request
+ * timeout. Defaults to the effective chunk size, which starts large and shrinks if a run gets stuck.
+ *
+ * @param int $chunk_size The number of products to process per chunk.
+ *
+ * @since 11.0.0
+ */
+ $chunk_size = (int) apply_filters( 'woocommerce_product_feed_chunk_size', $this->get_effective_chunk_size( $option_key ) );
+ if ( $chunk_size < 1 ) {
+ $chunk_size = self::CHUNK_SIZE_STEPS[0];
+ }
+
+ return (int) max( 1, (int) ceil( $chunk_size / $this->get_batch_size() ) );
+ }
+
+ /**
+ * Returns the option key under which the effective chunk size is persisted.
+ *
+ * Stored separately from the status so a shrunk chunk size survives the status being cleared when a
+ * job completes, expires, or restarts, and carries over to the next request from the app.
+ *
+ * @param string $option_key The option key for the feed generation status.
+ * @return string The option key for the effective chunk size.
+ */
+ private function get_chunk_size_option_key( string $option_key ): string {
+ return $option_key . '_chunk_size';
+ }
+
+ /**
+ * Returns the effective chunk size (products per action) currently in use for a feed.
+ *
+ * @param string $option_key The option key for the feed generation status.
+ * @return int The effective chunk size, defaulting to the largest configured step.
+ */
+ private function get_effective_chunk_size( string $option_key ): int {
+ $chunk_size = (int) get_option( $this->get_chunk_size_option_key( $option_key ), self::CHUNK_SIZE_STEPS[0] );
+
+ return $chunk_size > 0 ? $chunk_size : self::CHUNK_SIZE_STEPS[0];
+ }
+
+ /**
+ * Steps the chunk size down when an invalidated status was a stuck in-progress job.
+ *
+ * A stuck job was most likely killed because its chunk was too large for the host, so a smaller chunk
+ * makes the restart more likely to fit. A genuine failure (state = failed) is a real error rather than
+ * a size symptom and is intentionally excluded. Both recovery paths — an ordinary poll
+ * ({@see get_status()}) and an explicit rebuild ({@see force_regeneration()}) — call this, so a stuck
+ * job adapts the same way however it is recovered.
+ *
+ * @param array $status The invalidated status being discarded.
+ * @param string $option_key The option key for the feed generation status.
+ * @return void
+ */
+ private function reduce_chunk_size_if_stuck( array $status, string $option_key ): void {
+ if ( self::STATE_IN_PROGRESS === ( $status['state'] ?? '' ) ) {
+ $this->reduce_chunk_size( $option_key );
+ }
+ }
+
+ /**
+ * Steps the effective chunk size down to the next-smaller configured size and persists it.
+ *
+ * Called when a job gets stuck. Once at the smallest configured size it stays there.
+ *
+ * @param string $option_key The option key for the feed generation status.
+ * @return int The new effective chunk size.
+ */
+ private function reduce_chunk_size( string $option_key ): int {
+ $current = $this->get_effective_chunk_size( $option_key );
+
+ // CHUNK_SIZE_STEPS is descending, so the first step smaller than the current size is the next rung down.
+ $next = $current;
+ foreach ( self::CHUNK_SIZE_STEPS as $step ) {
+ if ( $step < $current ) {
+ $next = $step;
+ break;
+ }
+ }
+
+ update_option( $this->get_chunk_size_option_key( $option_key ), $next );
+
+ wc_get_logger()->warning(
+ 'Product feed generation got stuck; reducing the chunk size for future runs.',
+ array(
+ 'option_key' => $option_key,
+ 'previous_chunk_size' => $current,
+ 'chunk_size' => $next,
+ )
+ );
+
+ return $next;
+ }
+
+ /**
+ * Returns the number of products fetched per database batch.
+ *
+ * @return int The batch size (at least 1).
+ */
+ private function get_batch_size(): int {
+ /**
+ * Filters the number of products fetched per database query during feed generation.
+ *
+ * This is the granularity within a chunk; see `woocommerce_product_feed_chunk_size` for how many
+ * products each Action Scheduler action processes.
+ *
+ * @param int $batch_size The number of products per database batch.
+ *
+ * @since 11.0.0
+ */
+ $batch_size = (int) apply_filters( 'woocommerce_product_feed_batch_size', self::BATCH_SIZE );
+
+ return (int) max( 1, $batch_size );
+ }
+
+ /**
+ * Updates the cumulative progress fields on the status and refreshes the heartbeat.
+ *
+ * @param array $status The current feed generation status.
+ * @param int $processed The cumulative number of products processed so far.
+ * @param int $total The total number of products to process.
+ * @return array The updated status.
+ */
+ private function update_progress( array $status, int $processed, int $total ): array {
+ $status['processed'] = $processed;
+ $status['total'] = $total;
+ $status['progress'] = $total > 0 ? round( ( $processed / $total ) * 100, 2 ) : 0;
+ $status['updated_at'] = time();
+ return $status;
+ }
+
+ /**
+ * Applies the dynamic field arguments to the product mapper.
+ *
+ * @param array $args The feed generation arguments.
+ * @return void
+ */
+ private function apply_mapper_args( array $args ): void {
+ if ( isset( $args['_product_fields'] ) && is_string( $args['_product_fields'] ) && '' !== $args['_product_fields'] ) {
+ $this->integration->get_product_mapper()->set_fields( $args['_product_fields'] );
+ }
+ if ( isset( $args['_variation_fields'] ) && is_string( $args['_variation_fields'] ) && '' !== $args['_variation_fields'] ) {
+ $this->integration->get_product_mapper()->set_variation_fields( $args['_variation_fields'] );
+ }
+ }
+
+ /**
+ * Deletes the feed file referenced by a status, if any.
+ *
+ * Completed feeds expose a full path; in-progress chunked feeds only track a file name.
+ *
+ * @param array $status The feed generation status.
+ * @return void
+ */
+ private function discard_feed( array $status ): void {
+ // A completed feed exposes a full path; an in-progress chunked feed only tracks a file name.
+ // Reduce either to a plain identifier and let ResumableFeedInterface::delete() validate it and confine
+ // the deletion to the feed directory, so a tampered path read back from the option can never escape it.
+ $identifier = ! empty( $status['file_name'] )
+ ? (string) $status['file_name']
+ : ( ! empty( $status['path'] ) ? wp_basename( (string) $status['path'] ) : '' );
+
+ if ( '' !== $identifier ) {
+ $this->integration->create_feed()->delete( $identifier );
+ }
+ }
+
/**
* Forces a regeneration of the feed.
*
@@ -271,33 +534,35 @@ class AsyncGenerator {
$option_key = $this->get_option_key( $args );
$status = get_option( $option_key );
- // If there is no option, there is nothing to force. If the option is invalid, we can restart.
+ // An invalid status (stale, expired, or a stalled in-progress job) always regenerates from
+ // scratch: discard any partial feed and clear the option so the restart starts clean.
if ( ! is_array( $status ) || ! $this->validate_status( $status ) ) {
+ if ( is_array( $status ) ) {
+ // A stuck in-progress job adapts its chunk size on a forced rebuild too, the same way an
+ // ordinary poll does, so the rebuild does not re-die at the size that just got it killed.
+ $this->reduce_chunk_size_if_stuck( $status, $option_key );
+ $this->discard_feed( $status );
+ delete_option( $option_key );
+ }
return $this->get_status( $args );
}
switch ( $status['state'] ?? '' ) {
case self::STATE_SCHEDULED:
- // If generation is scheduled, we can just let it be and return the current status.
- // It should start shortly.
+ // Generation is already scheduled and should start shortly; leave it be.
return $status;
case self::STATE_IN_PROGRESS:
- // A genuinely running job (its heartbeat is still fresh, otherwise validate_status()
- // above would have restarted it) cannot be interrupted mid-flight.
+ // A genuinely running job (fresh heartbeat) cannot be interrupted mid-flight.
throw new \Exception( 'Feed generation is already in progress and cannot be stopped.' );
case self::STATE_COMPLETED:
- // Delete the existing file, clear the option and let generation start again.
- wp_delete_file( (string) $status['path'] );
- delete_option( $option_key );
- return $this->get_status( $args );
-
- case self::STATE_FAILED:
- // Clear the failed status and restart generation.
+ $this->discard_feed( $status );
delete_option( $option_key );
return $this->get_status( $args );
+ // A failed job is invalid (see validate_status()), so it never reaches this switch; it is
+ // discarded and regenerated by the early return above.
default:
throw new \Exception( 'Unknown feed generation state.' );
}
@@ -341,23 +606,6 @@ class AsyncGenerator {
);
}
- /**
- * Updates the feed progress while the feed is being generated.
- *
- * @param array $status The last previously known status.
- * @param WalkerProgress $progress The progress of the walker.
- * @return array Updated status of the feed generation.
- */
- private function update_feed_progress( array $status, WalkerProgress $progress ): array {
- $status['progress'] = $progress->total_count > 0
- ? round( ( $progress->processed_items / $progress->total_count ) * 100, 2 )
- : 0;
- $status['processed'] = $progress->processed_items;
- $status['total'] = $progress->total_count;
- $status['updated_at'] = time();
- return $status;
- }
-
/**
* Validates the status of the feed generation.
*
@@ -368,14 +616,15 @@ class AsyncGenerator {
* @return bool True if the status is valid, false otherwise.
*/
private function validate_status( array $status ): bool {
- /**
- * For completed jobs, make sure the file still exists. Regenerate otherwise.
- *
- * The file should typically get deleted at the same time as the status is cleared.
- * However, something else could cause the file to disappear in the meantime (ex. manual delete).
- *
- * Also, if the cleanup job failed, the feed might appear as complete, but be expired.
- */
+ // A failed job is never served as-is. get_status() surfaces the failure to the client once and
+ // then clears it, so the client can react and its next poll starts a fresh run; force_regeneration()
+ // likewise treats it as invalid and regenerates. Either way it must not validate.
+ if ( self::STATE_FAILED === $status['state'] ) {
+ return false;
+ }
+
+ // For completed jobs, the file must still exist and not be expired (e.g. manually deleted, or a
+ // cleanup job that failed to clear an expired feed).
if ( self::STATE_COMPLETED === $status['state'] ) {
if ( ! file_exists( $status['path'] ) ) {
return false;
@@ -391,12 +640,8 @@ class AsyncGenerator {
}
/**
- * If the job has been scheduled more than 10 minutes ago but has not
- * transitioned to IN_PROGRESS yet, ActionScheduler is typically stuck.
- */
-
- /**
- * Allows the timeout for a feed to remain in `scheduled` state to be changed.
+ * Allows the timeout for a feed to remain in `scheduled` state to be changed. Past this point
+ * Action Scheduler is typically stuck and the job is regenerated.
*
* @param int $stuck_time The stuck time in seconds.
* @return int The stuck time in seconds.
@@ -413,33 +658,36 @@ class AsyncGenerator {
return false;
}
- /**
- * If the job is in progress but has not updated its heartbeat within the timeout, the
- * process was most likely killed (server/host timeout or out of memory) before it could
- * mark itself as failed. Without this check, such a job would stay `in_progress` forever
- * and no new feed could ever be generated.
- *
- * The heartbeat (`updated_at`) is refreshed when the job starts and after every processed
- * batch, so an active job keeps it fresh while a killed one does not.
- */
+ // An in-progress job that has not refreshed its heartbeat (`updated_at`, set on start and after
+ // every batch) within the timeout was most likely killed (host timeout or out of memory) before
+ // it could mark itself failed. Treat it as stuck so a new feed can be generated.
if ( self::STATE_IN_PROGRESS === $status['state'] ) {
$last_activity = $status['updated_at'] ?? $status['scheduled_at'] ?? 0;
/**
- * Allows the timeout for a feed to remain in `in_progress` state without a heartbeat
- * update to be changed. Past this point the job is treated as stuck and regenerated.
+ * Allows the heartbeat timeout for an `in_progress` feed to be changed. Past this point the
+ * job is treated as stuck and regenerated.
+ *
+ * The default is kept comfortably larger than the per-batch time budget on purpose. The
+ * heartbeat only refreshes between batches, so the longest gap a healthy job can produce is
+ * roughly one batch (`woocommerce_product_feed_batch_time_limit`). A timeout at or near that
+ * budget would let a single slow-but-valid batch look stuck, and recovery would then discard
+ * the partial the live process is still writing. Deriving it as a multiple (with a floor)
+ * keeps that margin even when the batch budget is raised via its own filter.
*
* @param int $stuck_time The stuck time in seconds.
* @return int The stuck time in seconds.
* @since 11.0.0
*/
- $in_progress_timeout = apply_filters( 'woocommerce_product_feed_in_progress_timeout', 5 * MINUTE_IN_SECONDS );
+ $in_progress_timeout = apply_filters(
+ 'woocommerce_product_feed_in_progress_timeout',
+ max( 15 * MINUTE_IN_SECONDS, 3 * $this->get_batch_time_limit() )
+ );
if ( time() - $last_activity > $in_progress_timeout ) {
return false;
}
}
- // All good.
return true;
}
}
diff --git a/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/POSIntegration.php b/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/POSIntegration.php
index 18f9ec9677c..79cb6b02567 100644
--- a/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/POSIntegration.php
+++ b/plugins/woocommerce/src/Internal/ProductFeed/Integrations/POSCatalog/POSIntegration.php
@@ -12,6 +12,7 @@ namespace Automattic\WooCommerce\Internal\ProductFeed\Integrations\POSCatalog;
use Automattic\WooCommerce\Container;
use Automattic\WooCommerce\Internal\ProductFeed\Feed\FeedInterface;
use Automattic\WooCommerce\Internal\ProductFeed\Feed\FeedValidatorInterface;
+use Automattic\WooCommerce\Internal\ProductFeed\Feed\ResumableFeedInterface;
use Automattic\WooCommerce\Internal\ProductFeed\Integrations\IntegrationInterface;
use Automattic\WooCommerce\Internal\ProductFeed\Storage\JsonFileFeed;
@@ -102,8 +103,15 @@ class POSIntegration implements IntegrationInterface {
/**
* {@inheritdoc}
+ *
+ * POS catalog feeds are generated in chunks across multiple processes, so this returns a feed that
+ * supports the resumable lifecycle. Narrowing the return type (a covariant override of the base
+ * {@see FeedInterface} contract) guarantees that at the language level for callers such as
+ * {@see AsyncGenerator}, instead of relying on a PHPDoc hint or a runtime check.
+ *
+ * @return ResumableFeedInterface
*/
- public function create_feed(): FeedInterface {
+ public function create_feed(): ResumableFeedInterface {
return new JsonFileFeed( 'pos-catalog-feed' );
}
diff --git a/plugins/woocommerce/src/Internal/ProductFeed/Storage/JsonFileFeed.php b/plugins/woocommerce/src/Internal/ProductFeed/Storage/JsonFileFeed.php
index 5580df1a75b..65c7f7fd4fa 100644
--- a/plugins/woocommerce/src/Internal/ProductFeed/Storage/JsonFileFeed.php
+++ b/plugins/woocommerce/src/Internal/ProductFeed/Storage/JsonFileFeed.php
@@ -11,6 +11,7 @@ namespace Automattic\WooCommerce\Internal\ProductFeed\Storage;
use Automattic\WooCommerce\Internal\Utilities\FilesystemUtil;
use Automattic\WooCommerce\Internal\ProductFeed\Feed\FeedInterface;
+use Automattic\WooCommerce\Internal\ProductFeed\Feed\ResumableFeedInterface;
use Exception;
// This file works directly with local files. That's fine.
@@ -23,7 +24,7 @@ use Exception;
*
* @since 10.5.0
*/
-class JsonFileFeed implements FeedInterface {
+class JsonFileFeed implements ResumableFeedInterface {
public const UPLOAD_DIR = 'product-feeds';
/**
@@ -57,7 +58,9 @@ class JsonFileFeed implements FeedInterface {
/**
* The file handle.
*
- * @var resource|false|null
+ * Only ever a resource or null: open_handle() throws instead of storing a failed fopen().
+ *
+ * @var resource|null
*/
private $file_handle = null;
@@ -75,13 +78,6 @@ class JsonFileFeed implements FeedInterface {
*/
private $file_url = null;
- /**
- * Indicates if the feed file is in a temp directory.
- *
- * @var bool
- */
- private $is_temp_filepath = false;
-
/**
* Cached upload directory details (path and URL), resolved once per feed instance.
*
@@ -99,59 +95,80 @@ class JsonFileFeed implements FeedInterface {
}
/**
- * Start the feed.
+ * {@inheritDoc}
+ *
+ * Simple one-shot entry point for non-resumable generation. This is a thin adapter over the
+ * resumable {@see open()}: it starts a fresh feed and discards the returned identifier. It exists
+ * to honor the base {@see FeedInterface} contract; chunked callers use {@see open()} directly.
*
* @return void
- * @throws Exception If the feed directory cannot be created.
+ * @throws Exception If the feed directory or file cannot be created/opened.
*/
public function start(): void {
- $this->entry_count = 0;
+ $this->open();
+ }
+
+ /**
+ * {@inheritDoc}
+ *
+ * A feed can be written across separate processes (and possibly servers), so it is created
+ * directly in the shared upload directory rather than a per-request temp directory.
+ *
+ * @param string|null $resume_identifier Identifier of an existing feed to resume, or null to start fresh.
+ * @param int $entries_written The number of entries already written by previous chunks.
+ * @return string The identifier of the feed that was started.
+ * @throws Exception If the feed directory or file cannot be created/opened, or a resumed feed is missing.
+ */
+ public function open( ?string $resume_identifier = null, int $entries_written = 0 ): string {
+ $upload_dir = $this->get_upload_dir();
+
$this->file_completed = false;
$this->file_url = null;
- /**
- * Allows the current time to be overridden before a feed is stored.
- *
- * @param int $time The current time.
- * @param FeedInterface $feed The feed instance.
- * @return int The current time.
- * @since 10.5.0
- */
- $current_time = apply_filters( 'woocommerce_product_feed_time', time(), $this );
- $hash_data = $this->base_name . gmdate( 'r', $current_time );
- $this->file_name = sprintf(
- '%s-%s-%s.json',
- $this->base_name,
- gmdate( 'Y-m-d', $current_time ),
- wp_hash( $hash_data )
- );
+ if ( null !== $resume_identifier ) {
+ if ( ! $this->is_valid_feed_identifier( $resume_identifier ) ) {
+ throw new Exception(
+ esc_html(
+ sprintf(
+ /* translators: %s: feed identifier */
+ __( 'Invalid feed file identifier: %s', 'woocommerce' ),
+ $resume_identifier
+ )
+ )
+ );
+ }
- // Start by trying to use a temp directory to generate the feed.
- $this->file_path = get_temp_dir() . DIRECTORY_SEPARATOR . $this->file_name;
- $this->file_handle = fopen( $this->file_path, 'w' );
- if ( false === $this->file_handle ) {
- // Fall back to immediately using the upload directory for generation.
- $upload_dir = $this->get_upload_dir();
- $this->file_path = $upload_dir['path'] . $this->file_name;
- $this->file_handle = fopen( $this->file_path, 'w' );
- } else {
- $this->is_temp_filepath = true;
- }
+ $this->file_name = $resume_identifier;
+ $this->file_path = $upload_dir['path'] . $resume_identifier;
- if ( false === $this->file_handle ) {
- throw new Exception(
- esc_html(
- sprintf(
- /* translators: %s: directory path */
- __( 'Unable to open feed file for writing: %s', 'woocommerce' ),
- $this->file_path
+ // The partial must still be there to append to. If it has vanished (e.g. cleaned up by the
+ // host), fail rather than write a corrupt feed; the caller restarts generation from scratch.
+ if ( ! is_file( $this->file_path ) ) {
+ throw new Exception(
+ esc_html(
+ sprintf(
+ /* translators: %s: file path */
+ __( 'Cannot resume feed; file does not exist: %s', 'woocommerce' ),
+ $this->file_path
+ )
)
- )
- );
+ );
+ }
+
+ // Seed the entry count so add_entry()'s separator accounts for entries already written.
+ $this->entry_count = $entries_written;
+ $this->open_handle( $this->file_path, 'a' );
+
+ return $this->file_name;
}
- // Open the array.
- fwrite( $this->file_handle, '[' );
+ $this->entry_count = 0;
+ $this->file_name = $this->generate_file_name();
+ $this->file_path = $upload_dir['path'] . $this->file_name;
+ $handle = $this->open_handle( $this->file_path, 'w' );
+ fwrite( $handle, '[' );
+
+ return $this->file_name;
}
/**
@@ -179,32 +196,130 @@ class JsonFileFeed implements FeedInterface {
}
/**
- * End the feed.
- *
- * @return void
+ * {@inheritDoc}
*/
public function end(): void {
if ( ! is_resource( $this->file_handle ) ) {
return;
}
- // Close the array and the file.
fwrite( $this->file_handle, ']' );
fclose( $this->file_handle );
-
- // Indicate that we have a complete file.
+ $this->file_handle = null;
$this->file_completed = true;
}
/**
- * Get the number of entries that have been added to the feed.
+ * {@inheritDoc}
+ */
+ public function flush(): void {
+ if ( is_resource( $this->file_handle ) ) {
+ fclose( $this->file_handle );
+ $this->file_handle = null;
+ }
+ }
+
+ /**
+ * {@inheritDoc}
+ *
+ * @param string $identifier The identifier returned by open().
+ * @return void
+ */
+ public function delete( string $identifier ): void {
+ // Never turn an identifier that is actually a path into a delete outside the feed directory.
+ if ( ! $this->is_valid_feed_identifier( $identifier ) ) {
+ return;
+ }
+
+ $path = $this->feed_file_path( $identifier );
+ if ( is_file( $path ) ) {
+ wp_delete_file( $path );
+ }
+ }
+
+ /**
+ * Checks that a feed identifier is a plain feed file name, not a path.
*
- * This reflects the rows actually written to the feed, which may be fewer
- * than the number of products iterated by `ProductWalker` because the
- * validator can silently drop entries before they reach `add_entry()`.
+ * Identifiers round-trip through the persisted status option and are accepted by the public
+ * {@see delete()}, so a corrupted or hostile value (e.g. containing `../`) must never be
+ * concatenated into a path that escapes the feed directory.
+ *
+ * @param string $identifier The feed file identifier to check.
+ * @return bool True if the identifier is a safe, plain `.json` file name.
+ */
+ private function is_valid_feed_identifier( string $identifier ): bool {
+ return '' !== $identifier
+ && wp_basename( $identifier ) === $identifier
+ && 'json' === strtolower( (string) pathinfo( $identifier, PATHINFO_EXTENSION ) );
+ }
+
+ /**
+ * Resolves a feed file's path from its identifier without creating the upload directory.
+ *
+ * Unlike {@see get_upload_dir()} (used when writing), this must not create the directory as a side effect.
+ * Callers must validate the identifier with {@see is_valid_feed_identifier()} first.
+ *
+ * @param string $identifier The feed file name.
+ * @return string The absolute path to the feed file.
+ */
+ private function feed_file_path( string $identifier ): string {
+ $upload_dir = wp_upload_dir( null, false );
+ return $upload_dir['basedir'] . DIRECTORY_SEPARATOR . self::UPLOAD_DIR . DIRECTORY_SEPARATOR . $identifier;
+ }
+
+ /**
+ * Opens the feed file handle, throwing if it cannot be opened.
+ *
+ * @param string $path The file path to open.
+ * @param string $mode The fopen() mode.
+ * @return resource The opened file handle.
+ * @throws Exception If the file cannot be opened.
+ */
+ private function open_handle( string $path, string $mode ) {
+ $handle = fopen( $path, $mode );
+ if ( false === $handle ) {
+ throw new Exception(
+ esc_html(
+ sprintf(
+ /* translators: %s: file path */
+ __( 'Unable to open feed file: %s', 'woocommerce' ),
+ $path
+ )
+ )
+ );
+ }
+
+ $this->file_handle = $handle;
+ return $handle;
+ }
+
+ /**
+ * Generate the feed file name based on the base name and the current time.
*
- * @since 10.9.0
- * @return int Number of entries added to the feed.
+ * @return string The feed file name.
+ */
+ private function generate_file_name(): string {
+ /**
+ * Allows the current time to be overridden before a feed is stored.
+ *
+ * @param int $time The current time.
+ * @param FeedInterface $feed The feed instance.
+ * @return int The current time.
+ * @since 10.5.0
+ */
+ $current_time = apply_filters( 'woocommerce_product_feed_time', time(), $this );
+ $hash_data = $this->base_name . gmdate( 'r', $current_time );
+
+ return sprintf(
+ '%s-%s-%s.json',
+ $this->base_name,
+ gmdate( 'Y-m-d', $current_time ),
+ wp_hash( $hash_data )
+ );
+ }
+
+ /**
+ * {@inheritDoc}
*/
public function get_entry_count(): int {
return $this->entry_count;
@@ -224,41 +339,15 @@ class JsonFileFeed implements FeedInterface {
/**
* {@inheritDoc}
*
- * @throws Exception If the feed file cannot be moved to the upload directory.
+ * @throws Exception If the upload directory cannot be created.
*/
public function get_file_url(): ?string {
if ( ! $this->file_completed ) {
return null;
}
- $upload_dir = $this->get_upload_dir();
-
- // Move the file to the upload directory if it is in temp.
- if ( $this->is_temp_filepath ) {
- $tmp_path = $this->file_path;
- $this->file_path = $upload_dir['path'] . $this->file_name;
- // phpcs:ignore WordPress.PHP.NoSilencedErrors.Discouraged
- if ( ! @copy( $tmp_path, $this->file_path ) ) {
- $error = error_get_last();
- $error_message = is_array( $error ) ? $error['message'] : 'Unknown error';
- throw new Exception(
- esc_html(
- sprintf(
- /* translators: %1$s: file path, %2$s: error message */
- __( 'Unable to move feed file %1$s to upload directory: %2$s', 'woocommerce' ),
- $this->file_path,
- $error_message
- )
- )
- );
- }
-
- unlink( $tmp_path );
-
- $this->is_temp_filepath = false;
- }
-
- // Generate the URL.
+ // Resolve the upload directory (also refreshes its .htaccess for file access) and build the URL.
+ $upload_dir = $this->get_upload_dir();
$this->file_url = $upload_dir['url'] . $this->file_name;
return $this->file_url;
diff --git a/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Feed/ProductWalkerTest.php b/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Feed/ProductWalkerTest.php
index e41a235daff..ac19e34a07c 100644
--- a/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Feed/ProductWalkerTest.php
+++ b/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Feed/ProductWalkerTest.php
@@ -206,7 +206,7 @@ class ProductWalkerTest extends \WC_Unit_Test_Case {
}
);
- // Make sure that the field is initiated, added to, and ended.
+ // walk() owns the feed lifecycle: it starts the feed once, adds entries, and ends it once.
$mock_feed->expects( $this->once() )->method( 'start' );
$mock_feed->expects( $this->once() )->method( 'end' );
$mock_feed->expects( $this->exactly( $number_of_products - $validation_compensation ) )
@@ -273,4 +273,72 @@ class ProductWalkerTest extends \WC_Unit_Test_Case {
$walker->set_batch_size( $batch_size );
$walker->walk( $walker_callback );
}
+
+ /**
+ * Test that a bounded walk processes a limited number of batches, resumes from the right page,
+ * stops at the last page, and never manages the feed lifecycle (start/end).
+ */
+ public function test_walk_processes_bounded_pages_and_resumes() {
+ $batch_size = 2;
+ $total = 6;
+ $total_pages = 3;
+
+ // Build the pages of products the loader will return, keyed by page number.
+ $pages = array();
+ for ( $page = 1; $page <= $total_pages; $page++ ) {
+ $products = array();
+ for ( $i = 0; $i < $batch_size; $i++ ) {
+ $products[] = WC_Helper_Product::create_simple_product();
+ }
+ $pages[ $page ] = (object) array(
+ 'products' => $products,
+ 'total' => $total,
+ 'max_num_pages' => $total_pages,
+ );
+ }
+
+ $mock_loader = $this->createMock( ProductLoader::class );
+ $this->test_container->replace( ProductLoader::class, $mock_loader );
+ $mock_loader->method( 'get_products' )->willReturnCallback(
+ function ( $args ) use ( $pages ) {
+ return $pages[ $args['page'] ];
+ }
+ );
+
+ $mock_memory_manager = $this->createMock( MemoryManager::class );
+ $this->test_container->replace( MemoryManager::class, $mock_memory_manager );
+ $mock_memory_manager->method( 'get_available_memory' )->willReturn( 90 );
+
+ // The chunked walk must NOT own the feed lifecycle.
+ $mock_feed = $this->createMock( FeedInterface::class );
+ $mock_feed->expects( $this->never() )->method( 'start' );
+ $mock_feed->expects( $this->never() )->method( 'end' );
+
+ $mock_mapper = $this->createMock( ProductMapperInterface::class );
+ $mock_mapper->method( 'map_product' )->willReturnCallback(
+ fn( WC_Product $product ) => array( 'id' => $product->get_id() )
+ );
+ $mock_validator = $this->createMock( FeedValidatorInterface::class );
+ $mock_validator->method( 'validate_entry' )->willReturn( array() );
+
+ $mock_integration = $this->createMock( IntegrationInterface::class );
+ $mock_integration->method( 'get_product_mapper' )->willReturn( $mock_mapper );
+ $mock_integration->method( 'get_feed_validator' )->willReturn( $mock_validator );
+ $mock_integration->method( 'get_product_feed_query_args' )->willReturn( array() );
+
+ $walker = ProductWalker::from_integration( $mock_integration, $mock_feed );
+ $walker->set_batch_size( $batch_size );
+
+ // First chunk: pages 1-2 (allotment of 2 batches reached).
+ $progress = $walker->walk_batches( null, 1, 2 );
+ $this->assertSame( $total, $progress->total_count );
+ $this->assertSame( $total_pages, $progress->total_batch_count );
+ $this->assertSame( 2, $progress->processed_batches );
+ $this->assertSame( 4, $progress->processed_items );
+
+ // Final chunk: resumes at page 3 and stops at the last page even though 2 batches were allowed.
+ $progress = $walker->walk_batches( null, 3, 2 );
+ $this->assertSame( 1, $progress->processed_batches );
+ $this->assertSame( 2, $progress->processed_items );
+ }
}
diff --git a/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Integrations/POSCatalog/ApiControllerTest.php b/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Integrations/POSCatalog/ApiControllerTest.php
index 48e2ab1cc30..44696a6f707 100644
--- a/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Integrations/POSCatalog/ApiControllerTest.php
+++ b/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Integrations/POSCatalog/ApiControllerTest.php
@@ -90,9 +90,13 @@ class ApiControllerTest extends \WC_Unit_Test_Case {
->with( $fields ? array( '_product_fields' => $fields ) : array() )
->willReturn(
array(
- 'action_id' => 6789,
- 'path' => '/tmp/random_path.json',
- 'url' => 'https://example.com/feed.json',
+ 'action_id' => 6789,
+ 'path' => '/tmp/random_path.json',
+ 'file_name' => 'pos-catalog-feed.json',
+ 'page' => 3,
+ 'entries_written' => 250,
+ 'updated_at' => time(),
+ 'url' => 'https://example.com/feed.json',
)
);
@@ -100,8 +104,12 @@ class ApiControllerTest extends \WC_Unit_Test_Case {
$response_data = $response->get_data();
$this->assertEquals( 200, $response->get_status() );
- $this->assertArrayNotHasKey( 'action_id', $response_data );
- $this->assertArrayNotHasKey( 'path', $response_data );
+
+ // Sensitive and internal-only fields must never be exposed to the client.
+ foreach ( array( 'action_id', 'path', 'file_name', 'page', 'entries_written', 'updated_at' ) as $internal_key ) {
+ $this->assertArrayNotHasKey( $internal_key, $response_data );
+ }
+
$this->assertArrayHasKey( 'url', $response_data );
$this->assertEquals( 'https://example.com/feed.json', $response_data['url'] );
}
diff --git a/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGeneratorTest.php b/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGeneratorTest.php
index b6d6fc75c9a..3b9f1fed419 100644
--- a/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGeneratorTest.php
+++ b/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Integrations/POSCatalog/AsyncGeneratorTest.php
@@ -4,9 +4,11 @@ declare( strict_types = 1 );
namespace Automattic\WooCommerce\Tests\Internal\ProductFeed\Integrations\POSCatalog;
use PHPUnit\Framework\MockObject\MockObject;
+use Automattic\WooCommerce\Internal\ProductFeed\Feed\FeedValidatorInterface;
use Automattic\WooCommerce\Internal\ProductFeed\Integrations\POSCatalog\AsyncGenerator;
use Automattic\WooCommerce\Internal\ProductFeed\Integrations\POSCatalog\POSIntegration;
use Automattic\WooCommerce\Internal\ProductFeed\Integrations\POSCatalog\ProductMapper;
+use Automattic\WooCommerce\Internal\ProductFeed\Storage\JsonFileFeed;
use ReflectionClass;
use WC_Helper_Product;
@@ -50,7 +52,10 @@ class AsyncGeneratorTest extends \WC_Unit_Test_Case {
$this->mock_integration = $this->createMock( POSIntegration::class );
$this->test_container->replace( POSIntegration::class, $this->mock_integration );
- $this->sut = $this->test_container->get( AsyncGenerator::class );
+ // Build a fresh generator per test bound to this test's mock. Resolving it from the container
+ // returns a cached singleton bound to the first test's mock, which would ignore later mocks.
+ $this->sut = new AsyncGenerator();
+ $this->sut->init( $this->mock_integration );
}
/**
@@ -60,6 +65,9 @@ class AsyncGeneratorTest extends \WC_Unit_Test_Case {
parent::tearDown();
delete_option( self::OPTION_KEY );
+ delete_option( self::OPTION_KEY . '_chunk_size' );
+ remove_all_filters( 'woocommerce_product_feed_chunk_size' );
+ remove_all_filters( 'woocommerce_product_feed_batch_size' );
// Always clear the timeout filter here so a failed assertion in a test that registers it cannot
// leak it into later tests in the same process (tearDown runs even when a test fails).
remove_all_filters( 'woocommerce_product_feed_in_progress_timeout' );
@@ -100,6 +108,11 @@ class AsyncGeneratorTest extends \WC_Unit_Test_Case {
->method( 'get_product_mapper' )
->willReturn( $mock_mapper );
+ // The integration produces a real (resumable) feed so generation runs end to end.
+ $this->mock_integration->method( 'create_feed' )->willReturnCallback(
+ fn() => new JsonFileFeed( 'pos-catalog-feed-test' )
+ );
+
// Trigger the action.
$this->sut->feed_generation_action( self::OPTION_KEY );
@@ -114,7 +127,8 @@ class AsyncGeneratorTest extends \WC_Unit_Test_Case {
public function test_validate_status_returns_false_for_expired_feed() {
$status = array(
'state' => AsyncGenerator::STATE_COMPLETED,
- 'path' => __FILE__, // We just need a path that exists.
+ // __FILE__ is just a path that exists.
+ 'path' => __FILE__,
'completed_at' => time() - AsyncGenerator::FEED_EXPIRY - 1,
);
@@ -130,7 +144,8 @@ class AsyncGeneratorTest extends \WC_Unit_Test_Case {
public function test_validate_status_returns_true_for_non_expired_feed() {
$status = array(
'state' => AsyncGenerator::STATE_COMPLETED,
- 'path' => __FILE__, // We just need a path that exists.
+ // __FILE__ is just a path that exists.
+ 'path' => __FILE__,
'completed_at' => time() + AsyncGenerator::FEED_EXPIRY,
);
@@ -140,6 +155,69 @@ class AsyncGeneratorTest extends \WC_Unit_Test_Case {
$this->assertTrue( $method->invoke( $this->sut, $status ) );
}
+ /**
+ * Test that validate_status treats a failed job as invalid, so a status poll surfaces the failure
+ * and clears it rather than serving a terminal failure as a valid status.
+ */
+ public function test_validate_status_returns_false_for_failed_feed() {
+ $status = array(
+ 'state' => AsyncGenerator::STATE_FAILED,
+ 'error' => 'Something went wrong',
+ 'failed_at' => time(),
+ );
+
+ $method = ( new ReflectionClass( $this->sut ) )->getMethod( 'validate_status' );
+ $method->setAccessible( true );
+
+ $this->assertFalse( $method->invoke( $this->sut, $status ) );
+ }
+
+ /**
+ * @testdox Should surface a failed status to the client once, discard its partial feed, and clear it so the next poll starts fresh.
+ */
+ public function test_get_status_surfaces_failed_then_clears_so_next_poll_restarts() {
+ $this->mock_integration->method( 'create_feed' )->willReturnCallback(
+ fn() => new JsonFileFeed( 'pos-catalog-feed-test' )
+ );
+
+ // A real partial feed file the failed status points at, so we can prove it is discarded.
+ $partial = new JsonFileFeed( 'pos-catalog-feed-test' );
+ $identifier = $partial->open();
+ $partial->flush();
+ $partial_path = wp_upload_dir()['basedir'] . '/' . JsonFileFeed::UPLOAD_DIR . '/' . $identifier;
+ $this->assertTrue( file_exists( $partial_path ) );
+
+ $key_method = ( new ReflectionClass( $this->sut ) )->getMethod( 'get_option_key' );
+ $key_method->setAccessible( true );
+ $option_key = $key_method->invoke( $this->sut, array() );
+
+ update_option(
+ $option_key,
+ array(
+ 'state' => AsyncGenerator::STATE_FAILED,
+ 'error' => 'Something went wrong',
+ 'failed_at' => time(),
+ 'file_name' => $identifier,
+ )
+ );
+
+ // The first poll surfaces the failure (with its error) to the client...
+ $status = $this->sut->get_status( array() );
+ $this->assertSame( AsyncGenerator::STATE_FAILED, $status['state'] );
+ $this->assertSame( 'Something went wrong', $status['error'] );
+
+ // ...and clears the stored status and the partial feed file.
+ $this->assertFalse( get_option( $option_key ), 'A failed poll should clear the stored status.' );
+ $this->assertFalse( file_exists( $partial_path ), 'A failed poll should discard the partial feed file.' );
+
+ // The next poll then starts a fresh generation.
+ $status = $this->sut->get_status( array() );
+ $this->assertSame( AsyncGenerator::STATE_SCHEDULED, $status['state'] );
+
+ as_unschedule_all_actions( AsyncGenerator::FEED_GENERATION_ACTION, array(), 'woo-product-feed' );
+ delete_option( $option_key );
+ }
+
/**
* Test that validate_status treats an in-progress job with a stale heartbeat as invalid.
*
@@ -204,10 +282,429 @@ class AsyncGeneratorTest extends \WC_Unit_Test_Case {
$method = ( new ReflectionClass( $this->sut ) )->getMethod( 'validate_status' );
$method->setAccessible( true );
- // With a 10 second timeout, a 30 second old heartbeat is considered stale. The filter is cleaned
- // up in tearDown() so it cannot leak into later tests if the assertion fails.
- add_filter( 'woocommerce_product_feed_in_progress_timeout', fn() => 10 );
+ // With a 10 second timeout, a 30 second old heartbeat is considered stale.
+ $callback = fn() => 10;
+ add_filter( 'woocommerce_product_feed_in_progress_timeout', $callback );
+ $this->assertFalse( $method->invoke( $this->sut, $status ) );
+ remove_filter( 'woocommerce_product_feed_in_progress_timeout', $callback );
+ }
+
+ /**
+ * @testdox Should keep an in-progress job valid when its heartbeat is older than one batch budget but within the stuck timeout, so a slow batch is not mistaken for stuck.
+ */
+ public function test_validate_status_keeps_slow_but_valid_in_progress_feed_within_stuck_timeout() {
+ // A heartbeat older than the 5-minute per-batch budget (so it would trip a timeout set at that
+ // budget) but well within the derived stuck timeout, mirroring one slow-but-valid batch.
+ $status = array(
+ 'state' => AsyncGenerator::STATE_IN_PROGRESS,
+ 'scheduled_at' => time() - 10 * MINUTE_IN_SECONDS,
+ 'updated_at' => time() - 6 * MINUTE_IN_SECONDS,
+ );
+
+ $method = ( new ReflectionClass( $this->sut ) )->getMethod( 'validate_status' );
+ $method->setAccessible( true );
+
+ $this->assertTrue(
+ $method->invoke( $this->sut, $status ),
+ 'A heartbeat within the stuck timeout (but older than one batch budget) must not be treated as stuck.'
+ );
+ }
+
+ /**
+ * @testdox Should scale the stuck timeout with the batch time limit so raising the batch budget keeps the safety margin.
+ */
+ public function test_validate_status_stuck_timeout_scales_with_batch_time_limit() {
+ $status = array(
+ 'state' => AsyncGenerator::STATE_IN_PROGRESS,
+ 'scheduled_at' => time() - 40 * MINUTE_IN_SECONDS,
+ 'updated_at' => time() - 30 * MINUTE_IN_SECONDS,
+ );
+
+ $method = ( new ReflectionClass( $this->sut ) )->getMethod( 'validate_status' );
+ $method->setAccessible( true );
+
+ // With the default batch budget the 30-minute-old heartbeat is past the derived stuck timeout...
$this->assertFalse( $method->invoke( $this->sut, $status ) );
+
+ // ...but raising the per-batch budget raises the derived stuck timeout (3x) with it, so the same
+ // heartbeat is no longer considered stuck.
+ $callback = fn() => 20 * MINUTE_IN_SECONDS;
+ add_filter( 'woocommerce_product_feed_batch_time_limit', $callback );
+ $this->assertTrue( $method->invoke( $this->sut, $status ) );
+ remove_filter( 'woocommerce_product_feed_batch_time_limit', $callback );
+ }
+
+ /**
+ * Deletes every existing product so chunk-count assertions are deterministic regardless of any
+ * products left in the (persistent) test database by other runs.
+ */
+ private function delete_all_products(): void {
+ $ids = get_posts(
+ array(
+ 'post_type' => array( 'product', 'product_variation' ),
+ 'post_status' => 'any',
+ 'numberposts' => -1,
+ 'fields' => 'ids',
+ )
+ );
+ foreach ( $ids as $id ) {
+ wp_delete_post( (int) $id, true );
+ }
+
+ // The object cache is not rolled back between tests, so a prior test's cached product-query
+ // counts can leak in. Flush so the walker sees the real product count.
+ wp_cache_flush();
+ }
+
+ /**
+ * Configures the mock integration with a real feed and lightweight mapper/validator so the
+ * chunked generation path can be exercised end to end.
+ */
+ private function setup_real_feed_integration(): void {
+ $this->delete_all_products();
+
+ $this->mock_integration->method( 'get_product_feed_query_args' )->willReturn( array() );
+ $this->mock_integration->method( 'create_feed' )->willReturnCallback(
+ fn() => new JsonFileFeed( 'pos-catalog-feed-test' )
+ );
+
+ $mapper = $this->createMock( ProductMapper::class );
+ $mapper->method( 'map_product' )->willReturnCallback(
+ fn( $product ) => array( 'id' => $product->get_id() )
+ );
+ $this->mock_integration->method( 'get_product_mapper' )->willReturn( $mapper );
+
+ // FeedValidator is final and cannot be mocked, so use a permissive anonymous validator.
+ $validator = new class() implements FeedValidatorInterface {
+ /**
+ * Accept every entry.
+ *
+ * @param array $row The entry to validate.
+ * @param \WC_Product $product The related product.
+ * @return string[] Validation issues.
+ */
+ public function validate_entry( array $row, \WC_Product $product ): array {
+ // Avoid parameter not used PHPCS errors.
+ unset( $row, $product );
+ return array();
+ }
+ };
+ $this->mock_integration->method( 'get_feed_validator' )->willReturn( $validator );
+ }
+
+ /**
+ * Test that progress is reported between chunks: after the first (non-final) chunk the status
+ * reflects the real total and the products processed so far, rather than the initial -1 total.
+ */
+ public function test_feed_generation_reports_progress_between_chunks() {
+ // One product per database batch, two products per chunk.
+ add_filter( 'woocommerce_product_feed_batch_size', fn() => 1 );
+ add_filter( 'woocommerce_product_feed_chunk_size', fn() => 2 );
+
+ $this->setup_real_feed_integration();
+
+ // Five products across three chunks (2 + 2 + 1).
+ for ( $i = 0; $i < 5; $i++ ) {
+ WC_Helper_Product::create_simple_product();
+ }
+
+ update_option( self::OPTION_KEY, array( 'state' => AsyncGenerator::STATE_SCHEDULED ) );
+
+ // Process only the first chunk.
+ $this->sut->feed_generation_action( self::OPTION_KEY );
+ $status = get_option( self::OPTION_KEY );
+
+ $this->assertSame( AsyncGenerator::STATE_IN_PROGRESS, $status['state'] );
+ $this->assertSame( 5, $status['total'] );
+ $this->assertSame( 2, $status['processed'] );
+ $this->assertEqualsWithDelta( 40.0, $status['progress'], 0.001 );
+
+ // Clean up the partial feed file.
+ $partial_path = wp_upload_dir()['basedir'] . '/' . JsonFileFeed::UPLOAD_DIR . '/' . $status['file_name'];
+ if ( file_exists( $partial_path ) ) {
+ wp_delete_file( $partial_path );
+ }
+ }
+
+ /**
+ * Test that a feed is generated across multiple chunks, resuming each time, and produces a
+ * single valid JSON file once complete.
+ */
+ public function test_feed_generation_completes_across_multiple_chunks() {
+ // Force the smallest possible chunks: one product per database batch, one batch per chunk.
+ add_filter( 'woocommerce_product_feed_batch_size', fn() => 1 );
+ add_filter( 'woocommerce_product_feed_chunk_size', fn() => 1 );
+
+ // Use a real feed so the chunked file lifecycle is exercised, with lightweight mapper/validator.
+ $this->setup_real_feed_integration();
+
+ // Three products means three chunks.
+ WC_Helper_Product::create_simple_product();
+ WC_Helper_Product::create_simple_product();
+ WC_Helper_Product::create_simple_product();
+
+ update_option( self::OPTION_KEY, array( 'state' => AsyncGenerator::STATE_SCHEDULED ) );
+
+ // Drive the chunks manually (Action Scheduler does this in production via the scheduled action).
+ $iterations = 0;
+ do {
+ $this->sut->feed_generation_action( self::OPTION_KEY );
+ $status = get_option( self::OPTION_KEY );
+ ++$iterations;
+ } while ( AsyncGenerator::STATE_IN_PROGRESS === $status['state'] && $iterations < 10 );
+
+ $this->assertSame( AsyncGenerator::STATE_COMPLETED, $status['state'] );
+ $this->assertSame( 3, $iterations );
+ $this->assertSame( 3, $status['processed'] );
+
+ // The resulting file must be a single valid JSON array with one entry per product.
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_get_contents_file_get_contents
+ $contents = file_get_contents( $status['path'] );
+ $decoded = json_decode( (string) $contents, true );
+ $this->assertIsArray( $decoded );
+ $this->assertCount( 3, $decoded );
+
+ wp_delete_file( $status['path'] );
+ }
+
+ /**
+ * Test that entries_written accumulates correctly across chunks rather than being double-counted.
+ *
+ * Each resumed chunk seeds the feed's entry count with the running total, so the feed already
+ * reports the cumulative count; the status must store that count as-is, not add it on top of the
+ * previous total (which would grow the value quadratically).
+ */
+ public function test_feed_generation_tracks_cumulative_entries_written_across_chunks() {
+ // One product per database batch, one batch per chunk: three products means three chunks.
+ add_filter( 'woocommerce_product_feed_batch_size', fn() => 1 );
+ add_filter( 'woocommerce_product_feed_chunk_size', fn() => 1 );
+
+ $this->setup_real_feed_integration();
+
+ WC_Helper_Product::create_simple_product();
+ WC_Helper_Product::create_simple_product();
+ WC_Helper_Product::create_simple_product();
+
+ update_option( self::OPTION_KEY, array( 'state' => AsyncGenerator::STATE_SCHEDULED ) );
+
+ $iterations = 0;
+ do {
+ $this->sut->feed_generation_action( self::OPTION_KEY );
+ $status = get_option( self::OPTION_KEY );
+ ++$iterations;
+ } while ( AsyncGenerator::STATE_IN_PROGRESS === $status['state'] && $iterations < 10 );
+
+ $this->assertSame( AsyncGenerator::STATE_COMPLETED, $status['state'] );
+ // Three products, one entry each: the cumulative count is 3, not 1 + 2 + 4 = 7.
+ $this->assertSame( 3, $status['entries_written'] );
+
+ wp_delete_file( $status['path'] );
+ }
+
+ /**
+ * Test that the next chunk is scheduled even when another action with the same hook and group is
+ * already pending or running. Action Scheduler's uniqueness check matches on hook + group only, so
+ * a "unique" enqueue would be blocked here and generation would stall after the first chunk.
+ */
+ public function test_feed_generation_schedules_next_chunk_despite_existing_action() {
+ add_filter( 'woocommerce_product_feed_batch_size', fn() => 1 );
+ add_filter( 'woocommerce_product_feed_chunk_size', fn() => 1 );
+
+ $this->setup_real_feed_integration();
+
+ // Three products means the first chunk is not the last, so a continuation must be scheduled.
+ WC_Helper_Product::create_simple_product();
+ WC_Helper_Product::create_simple_product();
+ WC_Helper_Product::create_simple_product();
+
+ // A pending action for a different feed that shares the same hook and group.
+ as_enqueue_async_action( AsyncGenerator::FEED_GENERATION_ACTION, array( 'other-feed' ), 'woo-product-feed' );
+
+ update_option( self::OPTION_KEY, array( 'state' => AsyncGenerator::STATE_SCHEDULED ) );
+
+ $this->sut->feed_generation_action( self::OPTION_KEY );
+
+ $status = get_option( self::OPTION_KEY );
+ $this->assertSame( AsyncGenerator::STATE_IN_PROGRESS, $status['state'] );
+ $this->assertTrue(
+ as_has_scheduled_action( AsyncGenerator::FEED_GENERATION_ACTION, array( self::OPTION_KEY ), 'woo-product-feed' ),
+ 'A follow-up chunk action should be scheduled for the in-progress feed.'
+ );
+
+ // Clean up scheduled actions and the partial feed file.
+ as_unschedule_all_actions( AsyncGenerator::FEED_GENERATION_ACTION, array(), 'woo-product-feed' );
+ $partial_path = wp_upload_dir()['basedir'] . '/' . JsonFileFeed::UPLOAD_DIR . '/' . $status['file_name'];
+ if ( file_exists( $partial_path ) ) {
+ wp_delete_file( $partial_path );
+ }
+ }
+
+ /**
+ * Test that polling a stalled in-progress job restarts it fresh and steps the chunk size down,
+ * so the next attempt (and future requests) use a smaller, more reliable size.
+ */
+ public function test_get_status_reduces_chunk_size_and_restarts_when_stuck() {
+ $this->mock_integration->method( 'create_feed' )->willReturnCallback(
+ fn() => new JsonFileFeed( 'pos-catalog-feed-test' )
+ );
+
+ // A real partial feed left behind by the stuck (first, single-pass) attempt.
+ $partial = new JsonFileFeed( 'pos-catalog-feed-test' );
+ $identifier = $partial->open();
+ $partial->flush();
+ $partial_path = wp_upload_dir()['basedir'] . '/' . JsonFileFeed::UPLOAD_DIR . '/' . $identifier;
+ $this->assertTrue( file_exists( $partial_path ) );
+
+ $method = ( new ReflectionClass( $this->sut ) )->getMethod( 'get_option_key' );
+ $method->setAccessible( true );
+ $option_key = $method->invoke( $this->sut, array() );
+
+ // A stuck in-progress job: stale heartbeat.
+ update_option(
+ $option_key,
+ array(
+ 'state' => AsyncGenerator::STATE_IN_PROGRESS,
+ 'updated_at' => time() - HOUR_IN_SECONDS,
+ 'file_name' => $identifier,
+ 'page' => 3,
+ 'processed' => 50000,
+ 'total' => 120000,
+ )
+ );
+
+ $result = $this->sut->get_status( array() );
+
+ // Restarted from scratch (counters reset, partial discarded), not resumed.
+ $this->assertSame( AsyncGenerator::STATE_SCHEDULED, $result['state'] );
+ $this->assertSame( 0, $result['processed'] );
+ $this->assertArrayNotHasKey( 'file_name', $result );
+ $this->assertFalse( file_exists( $partial_path ) );
+
+ // Chunk size stepped down one rung (100000 -> 2500) and persisted for future runs.
+ $this->assertSame( 2500, (int) get_option( $option_key . '_chunk_size' ) );
+
+ as_unschedule_all_actions( AsyncGenerator::FEED_GENERATION_ACTION, array(), 'woo-product-feed' );
+ delete_option( $option_key );
+ delete_option( $option_key . '_chunk_size' );
+ }
+
+ /**
+ * Test that the chunk size steps down through the configured ladder and stops at the smallest.
+ */
+ public function test_chunk_size_steps_down_through_ladder() {
+ $method = ( new ReflectionClass( $this->sut ) )->getMethod( 'get_option_key' );
+ $method->setAccessible( true );
+ $option_key = $method->invoke( $this->sut, array() );
+
+ $reduce = ( new ReflectionClass( $this->sut ) )->getMethod( 'reduce_chunk_size' );
+ $reduce->setAccessible( true );
+
+ $this->assertSame( 2500, $reduce->invoke( $this->sut, $option_key ) );
+ $this->assertSame( 1000, $reduce->invoke( $this->sut, $option_key ) );
+ // Already at the smallest configured size: it stays there.
+ $this->assertSame( 1000, $reduce->invoke( $this->sut, $option_key ) );
+
+ delete_option( $option_key . '_chunk_size' );
+ }
+
+ /**
+ * Test that force_regeneration on a stalled in-progress job starts fresh and discards the
+ * partial feed, rather than resuming it (which is what an ordinary status poll would do), and
+ * that it steps the chunk size down just like a force=false poll does.
+ */
+ public function test_force_regeneration_starts_fresh_for_stalled_job() {
+ $this->mock_integration->method( 'create_feed' )->willReturnCallback(
+ fn() => new JsonFileFeed( 'pos-catalog-feed-test' )
+ );
+
+ // A real partial feed file that force should discard.
+ $partial = new JsonFileFeed( 'pos-catalog-feed-test' );
+ $identifier = $partial->open();
+ $partial->flush();
+ $partial_path = wp_upload_dir()['basedir'] . '/' . JsonFileFeed::UPLOAD_DIR . '/' . $identifier;
+ $this->assertTrue( file_exists( $partial_path ) );
+
+ $key_method = ( new ReflectionClass( $this->sut ) )->getMethod( 'get_option_key' );
+ $key_method->setAccessible( true );
+ $option_key = $key_method->invoke( $this->sut, array() );
+
+ update_option(
+ $option_key,
+ array(
+ 'state' => AsyncGenerator::STATE_IN_PROGRESS,
+ 'updated_at' => time() - HOUR_IN_SECONDS,
+ 'file_name' => $identifier,
+ 'page' => 3,
+ 'processed' => 2500,
+ 'total' => 12000,
+ 'entries_written' => 2500,
+ )
+ );
+
+ $result = $this->sut->force_regeneration( array() );
+
+ // Fresh start: scheduled, counters reset.
+ $this->assertSame( AsyncGenerator::STATE_SCHEDULED, $result['state'] );
+ $this->assertSame( 0, $result['processed'] );
+ $this->assertArrayNotHasKey( 'file_name', $result );
+ // The partial feed file was discarded.
+ $this->assertFalse( file_exists( $partial_path ) );
+ // The stuck job's chunk size stepped down one rung (100000 -> 2500) and persisted for future runs.
+ $this->assertSame( 2500, (int) get_option( $option_key . '_chunk_size' ) );
+
+ as_unschedule_all_actions( AsyncGenerator::FEED_GENERATION_ACTION, array(), 'woo-product-feed' );
+ delete_option( $option_key );
+ delete_option( $option_key . '_chunk_size' );
+ }
+
+ /**
+ * Test that a continuation whose partial feed file has vanished fails rather than appending to a
+ * non-existent file, that the next status poll surfaces the failure to the client and clears it,
+ * and that the poll after that starts a fresh generation.
+ */
+ public function test_feed_generation_fails_when_partial_file_missing() {
+ add_filter( 'woocommerce_product_feed_batch_size', fn() => 1 );
+ add_filter( 'woocommerce_product_feed_chunk_size', fn() => 1 );
+
+ $this->setup_real_feed_integration();
+
+ WC_Helper_Product::create_simple_product();
+
+ // Use the real (derived) option key so the action and the follow-up status poll act on the same job.
+ $key_method = ( new ReflectionClass( $this->sut ) )->getMethod( 'get_option_key' );
+ $key_method->setAccessible( true );
+ $option_key = $key_method->invoke( $this->sut, array() );
+
+ // A continuation pointing at a partial file that does not exist.
+ update_option(
+ $option_key,
+ array(
+ 'state' => AsyncGenerator::STATE_IN_PROGRESS,
+ 'file_name' => 'pos-catalog-feed-missing.json',
+ 'page' => 5,
+ 'processed' => 999,
+ 'entries_written' => 999,
+ 'total' => 999,
+ 'updated_at' => time(),
+ )
+ );
+
+ $this->sut->feed_generation_action( $option_key );
+ $this->assertSame( AsyncGenerator::STATE_FAILED, get_option( $option_key )['state'] );
+
+ // The first poll surfaces the failure to the client and clears the stored status.
+ $status = $this->sut->get_status( array() );
+ $this->assertSame( AsyncGenerator::STATE_FAILED, $status['state'] );
+ $this->assertFalse( get_option( $option_key ) );
+
+ // The next poll then starts a fresh generation.
+ $status = $this->sut->get_status( array() );
+ $this->assertSame( AsyncGenerator::STATE_SCHEDULED, $status['state'] );
+ $this->assertSame( 0, $status['processed'] );
+ $this->assertArrayNotHasKey( 'file_name', $status );
+
+ as_unschedule_all_actions( AsyncGenerator::FEED_GENERATION_ACTION, array(), 'woo-product-feed' );
+ delete_option( $option_key );
}
/**
@@ -223,10 +720,61 @@ class AsyncGeneratorTest extends \WC_Unit_Test_Case {
$mock_mapper->method( 'map_product' )->willReturn( array() );
$this->mock_integration->method( 'get_product_mapper' )->willReturn( $mock_mapper );
+ // The integration produces a real (resumable) feed so generation runs end to end.
+ $this->mock_integration->method( 'create_feed' )->willReturnCallback(
+ fn() => new JsonFileFeed( 'pos-catalog-feed-test' )
+ );
+
$this->sut->feed_generation_action( self::OPTION_KEY );
$updated_status = get_option( self::OPTION_KEY );
$this->assertSame( AsyncGenerator::STATE_COMPLETED, $updated_status['state'] );
$this->assertArrayHasKey( 'updated_at', $updated_status );
}
+
+ /**
+ * Test that discarding a feed never deletes a path that lies outside the feed directory, even when
+ * the persisted status was corrupted or tampered with to point elsewhere.
+ */
+ public function test_discard_feed_does_not_delete_path_outside_feed_dir() {
+ $this->mock_integration->method( 'create_feed' )->willReturnCallback(
+ fn() => new JsonFileFeed( 'pos-catalog-feed-test' )
+ );
+
+ // A sentinel file outside the feed directory that a tampered status path points at.
+ $outside = wp_upload_dir()['basedir'] . '/not-a-feed.json';
+ // phpcs:ignore WordPress.WP.AlternativeFunctions
+ file_put_contents( $outside, 'keep' );
+
+ $method = ( new ReflectionClass( $this->sut ) )->getMethod( 'discard_feed' );
+ $method->setAccessible( true );
+ $method->invoke( $this->sut, array( 'path' => $outside ) );
+
+ $this->assertTrue( file_exists( $outside ), 'A path outside the feed directory must not be deleted.' );
+ wp_delete_file( $outside );
+ }
+
+ /**
+ * Test that discarding a legacy feed (path only, no file_name) deletes the file when it is inside
+ * the feed directory.
+ */
+ public function test_discard_feed_deletes_legacy_path_inside_feed_dir() {
+ $this->mock_integration->method( 'create_feed' )->willReturnCallback(
+ fn() => new JsonFileFeed( 'pos-catalog-feed-test' )
+ );
+
+ $partial = new JsonFileFeed( 'pos-catalog-feed-test' );
+ $identifier = $partial->open();
+ $partial->flush();
+
+ $path = wp_upload_dir()['basedir'] . '/' . JsonFileFeed::UPLOAD_DIR . '/' . $identifier;
+ $this->assertTrue( file_exists( $path ) );
+
+ $method = ( new ReflectionClass( $this->sut ) )->getMethod( 'discard_feed' );
+ $method->setAccessible( true );
+ // Legacy status: only a path, no file_name.
+ $method->invoke( $this->sut, array( 'path' => $path ) );
+
+ $this->assertFalse( file_exists( $path ) );
+ }
}
diff --git a/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Storage/JsonFileFeedTest.php b/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Storage/JsonFileFeedTest.php
index aceadc708fb..08292c3280c 100644
--- a/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Storage/JsonFileFeedTest.php
+++ b/plugins/woocommerce/tests/php/src/Internal/ProductFeed/Storage/JsonFileFeedTest.php
@@ -41,19 +41,16 @@ class JsonFileFeedTest extends \WC_Unit_Test_Case {
$feed->start();
$feed->end();
- // The file should be in `/tmp` at first.
+ // The file is written directly into the shared uploads directory.
$path = $feed->get_file_path();
- $this->assertStringStartsWith( get_temp_dir(), $path );
+ $this->assertStringContainsString( 'uploads/product-feeds', $path );
$this->assertStringContainsString( gmdate( 'Y-m-d', $current_time ), $path );
$this->assertStringContainsString( wp_hash( 'test-feed' . gmdate( 'r', $current_time ) ), $path );
$this->assertTrue( file_exists( $path ) );
$this->assertEquals( '[]', file_get_contents( $path ) );
- // Once a URL is retrieved, the file will be moved to the uploads dir.
- $url = $feed->get_file_url();
- $path2 = $feed->get_file_path();
+ $url = $feed->get_file_url();
$this->assertNotNull( $url );
- $this->assertStringContainsString( 'uploads/product-feeds', $path2 );
$this->assertStringEndsWith( '.json', (string) $url );
$this->assertStringContainsString( '/product-feeds/', (string) $url );
}
@@ -86,6 +83,128 @@ class JsonFileFeedTest extends \WC_Unit_Test_Case {
);
}
+ /**
+ * Test that a feed written across chunks (start fresh, flush, resume, end) produces a single
+ * valid JSON array.
+ */
+ public function test_chunked_feed_produces_valid_json_across_chunks() {
+ $chunk_one = array(
+ array( 'name' => 'First' ),
+ array( 'name' => 'Second' ),
+ );
+ $chunk_two = array(
+ array( 'name' => 'Third' ),
+ );
+
+ // First chunk: start the feed and write some entries, then flush (do not end).
+ $feed = new JsonFileFeed( 'test-feed' );
+ $identifier = $feed->open();
+ foreach ( $chunk_one as $entry ) {
+ $feed->add_entry( $entry );
+ }
+ $entries_written = $feed->get_entry_count();
+ $feed->flush();
+
+ // Second (final) chunk: a fresh instance resumes the same feed and ends it, mirroring how a
+ // subsequent Action Scheduler action would run in its own process.
+ $feed_two = new JsonFileFeed( 'test-feed' );
+ $feed_two->open( $identifier, $entries_written );
+ foreach ( $chunk_two as $entry ) {
+ $feed_two->add_entry( $entry );
+ }
+ $feed_two->end();
+
+ $path = $feed_two->get_file_path();
+ $this->assertNotNull( $path );
+ $this->assertSame(
+ wp_json_encode( array_merge( $chunk_one, $chunk_two ) ),
+ file_get_contents( $path )
+ );
+ }
+
+ /**
+ * Test that a chunked feed whose first chunk wrote no entries still produces a valid JSON array.
+ */
+ public function test_chunked_feed_handles_empty_first_chunk() {
+ $feed = new JsonFileFeed( 'test-feed' );
+ $identifier = $feed->open();
+ $feed->flush();
+
+ $feed_two = new JsonFileFeed( 'test-feed' );
+ $feed_two->open( $identifier, 0 );
+ $feed_two->add_entry( array( 'name' => 'Only' ) );
+ $feed_two->end();
+
+ $this->assertSame(
+ wp_json_encode( array( array( 'name' => 'Only' ) ) ),
+ file_get_contents( $feed_two->get_file_path() )
+ );
+ }
+
+ /**
+ * Test that open() resumes an existing partial feed by reusing its identifier, and throws when the
+ * partial file is missing rather than appending to a non-existent file.
+ */
+ public function test_open_resumes_existing_feed_and_throws_when_missing() {
+ // Resuming a just-created feed reuses the same identifier.
+ $feed = new JsonFileFeed( 'test-feed' );
+ $identifier = $feed->open();
+ $feed->flush();
+
+ $feed_two = new JsonFileFeed( 'test-feed' );
+ $this->assertSame( $identifier, $feed_two->open( $identifier ) );
+ $feed_two->flush();
+
+ // Resuming a feed that no longer exists throws.
+ $this->expectException( \Exception::class );
+ ( new JsonFileFeed( 'test-feed' ) )->open( 'does-not-exist.json' );
+ }
+
+ /**
+ * Test that delete removes a partial feed file.
+ */
+ public function test_delete_removes_partial_feed_file() {
+ $feed = new JsonFileFeed( 'test-feed' );
+ $identifier = $feed->open();
+ $feed->flush();
+
+ $path = wp_upload_dir()['basedir'] . '/' . JsonFileFeed::UPLOAD_DIR . '/' . $identifier;
+ $this->assertTrue( file_exists( $path ) );
+
+ $feed->delete( $identifier );
+ $this->assertFalse( file_exists( $path ) );
+ }
+
+ /**
+ * Test that resuming with an identifier that is actually a path (traversal attempt) is rejected
+ * rather than opening a file outside the feed directory.
+ */
+ public function test_open_rejects_resume_identifier_with_path() {
+ $this->expectException( \Exception::class );
+ ( new JsonFileFeed( 'test-feed' ) )->open( '../escape.json' );
+ }
+
+ /**
+ * Test that delete() ignores an identifier containing a path, so it cannot remove a file outside
+ * the feed directory, while a plain identifier still deletes normally.
+ */
+ public function test_delete_ignores_identifier_with_path() {
+ $feed = new JsonFileFeed( 'test-feed' );
+ $identifier = $feed->open();
+ $feed->flush();
+
+ $path = wp_upload_dir()['basedir'] . '/' . JsonFileFeed::UPLOAD_DIR . '/' . $identifier;
+ $this->assertTrue( file_exists( $path ) );
+
+ // An identifier that resolves back to the same file via a parent path must be rejected.
+ $feed->delete( '../' . JsonFileFeed::UPLOAD_DIR . '/' . $identifier );
+ $this->assertTrue( file_exists( $path ), 'A traversal identifier must not delete the file.' );
+
+ // The plain identifier still deletes.
+ $feed->delete( $identifier );
+ $this->assertFalse( file_exists( $path ) );
+ }
+
/**
* Test that get_entry_count reflects the number of rows written to the feed.
*/