Commit 344f0b6913b for woocommerce
commit 344f0b6913b3fbf41ea89c150fdde0edea99c2ab
Author: Abdalsalaam Halawa <abdalsalaamnafez@gmail.com>
Date: Mon Nov 9 09:50:36 2026 +0200
[Order Fulfillments] Add CSV header parsing for the fulfillments importer (#69356)
* Add CSV header parsing for the fulfillments importer
FulfillmentsCsvImporter reads the header row of an uploaded CSV: delimiter normalization, BOM stripping, header alias detection behind the woocommerce_fulfillments_csv_importer_column_aliases filter, a sample row and the data row count with the 5,000-row cap.
* Reject an enclosure fgetcsv() cannot take
A multi-character or non-string enclosure option now falls back to the double quote instead of reaching fgetcsv(), which throws on PHP 8.
* Share one CSV reader for fulfillments prepare and import
Blank rows, including empty fields and whitespace, are skipped for the header, the row total, and read_data_rows(). Byte offsets still move past them, and error row numbers stay the physical record numbers. CR-only files are rejected, non-UTF-8 text is converted from Windows-1252, and a capped total is marked so it is not shown as an exact count.
* Stop the fulfillments CSV count at the first extra row
Count data rows before the cap check, then stop once one more real row shows the file is larger than 5,001. Detect UTF-8 without mbstring, convert with iconv when that is the available converter, and return an error instead of raw bytes when conversion is impossible.
Co-authored-by: Abdalsalaam Halawa <abdalsalaamnafez@gmail.com>
* Decode fulfillments CSV cells one at a time
Each cell is checked for well-formed UTF-8 and only a cell that fails
the check is read as Windows-1252. A file that mixes both encodings now
decodes correctly, and the row reader no longer scans the whole file on
every call: a chunked run of 100 calls over a 5,000-row file drops from
about 5 s to under 0.1 s.
Drop the stream wrapper exception around is_readable(), so php://,
data:// and http:// paths are refused like any unreadable path; the test
wrapper answers url_stat() instead. Drop the shape check on records the
class builds itself.
* Reject UTF-16 fulfillments CSV files
A UTF-16 BOM, or a NUL byte in the first 8 KB, returns utf16_encoding
with a message asking the merchant to re-save the file as a UTF-8 CSV.
Excel's "Unicode Text" export produces these files; before this they
parsed into headers with a NUL after every character that nothing could
map, with no error.
* Test that prepare and a chunked read agree on blank rows
Drive read_data_rows() to EOF at chunk sizes 1 to 8 over a file with a
leading blank line, a ",,," row, blanks mid-file and a trailing double
newline, and assert the row count equals parse_headers()['total'] and
the row numbers match a single read. Also pin a blank record landing on
a chunk boundary.
* Trim cells and pad short rows to the header width
A data row with fewer fields than the header came back with a shorter
cells array, so a consumer indexing it by header position hit an
undefined key and, under strict_types, a TypeError at the first typed
parameter. The sample row was short the same way, so the mapping screen
had no preview for those columns. Rows and the sample are now padded
with empty strings up to count( headers ). A longer row keeps its extra
cells.
Every cell is trimmed on the shared read path, so "1042, 1Z999, ups"
yields the same values as "1042,1Z999,ups" on both the prepare and the
run side, and a tracking number or carrier slug never carries a stray
space into a URL or a lookup. Header cells are trimmed too, so the
headers array matches what the alias matcher already compared against.
diff --git a/docs/features/fulfillments/hooks.md b/docs/features/fulfillments/hooks.md
index c95c97773b7..50cfc0d1a33 100644
--- a/docs/features/fulfillments/hooks.md
+++ b/docs/features/fulfillments/hooks.md
@@ -5,7 +5,7 @@ sidebar_label: Hooks
# Order fulfillments hooks
-WooCommerce exposes lifecycle hooks, status filters, provider filters, notification actions, and email template hooks for order fulfillments. These hooks let extensions validate fulfillment data, synchronize external systems, customize tracking behavior, and adjust merchant or customer-facing output.
+WooCommerce exposes lifecycle hooks, status filters, provider filters, notification actions, email template hooks, and a CSV importer filter for order fulfillments. These hooks let extensions validate fulfillment data, synchronize external systems, customize tracking behavior, adjust merchant or customer-facing output, and accept header names from warehouse exports.
## Data lifecycle filters
@@ -149,3 +149,22 @@ add_filter( 'woocommerce_privacy_remove_fulfillment_personal_data_meta', functio
return $meta_keys;
}, 10, 2 );
```
+
+## CSV importer filters
+
+The fulfillments CSV importer recognizes five canonical columns. This filter lets a store accept the header names used by its warehouse or 3PL exports when the importer auto-detects the column mapping.
+
+Aliases are normalized before matching (lowercase, with runs of non-alphanumeric characters replaced by `_`), so `PO Number` and `po_number` are the same alias. Keys that are not one of the five canonical columns are ignored, non-string aliases are dropped, and if nothing usable is left the default aliases are used.
+
+| Hook | Purpose |
+| --- | --- |
+| `woocommerce_fulfillments_csv_importer_column_aliases` | Change the accepted header names for `order_number`, `tracking_number`, `shipment_provider`, `tracking_url`, and `items`. Receives the alias map and returns the modified map. |
+
+```php
+add_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', function( $aliases ) {
+ $aliases['order_number'][] = 'PO Number';
+ $aliases['shipment_provider'][] = 'Ship Via';
+
+ return $aliases;
+} );
+```
diff --git a/plugins/woocommerce/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvImporter.php b/plugins/woocommerce/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvImporter.php
new file mode 100644
index 00000000000..b7a24f572f4
--- /dev/null
+++ b/plugins/woocommerce/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvImporter.php
@@ -0,0 +1,910 @@
+<?php
+/**
+ * Fulfillments CSV importer service.
+ *
+ * @package Automattic\WooCommerce\Admin\Features\Fulfillments\Importer
+ */
+
+declare( strict_types=1 );
+
+namespace Automattic\WooCommerce\Admin\Features\Fulfillments\Importer;
+
+defined( 'ABSPATH' ) || exit;
+
+/**
+ * Reads a fulfillments CSV for the import wizard and for the chunked import run.
+ *
+ * The first non-blank record is the header. Later reads return only real data rows.
+ * Blank records still advance the file position and keep their physical record numbers
+ * so error messages can point at the row in the file.
+ *
+ * Cells are decoded one at a time: valid UTF-8 is kept and anything else is read as
+ * Windows-1252, so a file that mixes both still comes out right. UTF-16 and CR-only
+ * files are rejected with a message asking the merchant to re-save the file. Every cell
+ * is trimmed, and a data row shorter than the header is padded with empty strings, so a
+ * consumer can index a row by header position without a guard.
+ *
+ * @internal
+ * @since 11.4.0
+ */
+class FulfillmentsCsvImporter {
+
+ /**
+ * Canonical column keys recognized by the importer.
+ */
+ public const COL_ORDER_NUMBER = 'order_number';
+ public const COL_TRACKING_NUMBER = 'tracking_number';
+ public const COL_PROVIDER = 'shipment_provider';
+ public const COL_TRACKING_URL = 'tracking_url';
+ public const COL_ITEMS = 'items';
+
+ /**
+ * Every canonical column key, in the order the wizard lists them.
+ *
+ * @return array<int, string>
+ */
+ public static function canonical_columns(): array {
+ return array(
+ self::COL_ORDER_NUMBER,
+ self::COL_TRACKING_NUMBER,
+ self::COL_PROVIDER,
+ self::COL_TRACKING_URL,
+ self::COL_ITEMS,
+ );
+ }
+
+ /**
+ * Absolute path to the CSV file.
+ *
+ * @var string
+ */
+ private string $file;
+
+ /**
+ * Options.
+ *
+ * @var array{notify_customer:bool,delimiter:string,enclosure:string,update_existing:bool}
+ */
+ private array $options;
+
+ /**
+ * Set when a cell that is not UTF-8 could not be converted from Windows-1252.
+ *
+ * Only happens when neither mbstring nor iconv is loaded. mbstring maps every
+ * Windows-1252 byte, so with it loaded the flag never trips.
+ *
+ * @var bool
+ */
+ private bool $conversion_failed = false;
+
+ /**
+ * Maximum number of data rows accepted per file.
+ *
+ * The cap bounds the time the prepare request spends counting rows and the
+ * size of the session transient. Larger files must be split and imported in parts.
+ */
+ public const MAX_IMPORT_ROWS = 5000;
+
+ /**
+ * Spellings accepted for a tab delimiter, since multipart and JSON clients
+ * cannot always send a raw tab character.
+ */
+ private const TAB_SPELLINGS = array( '\t', '\\\\t', 'tab' );
+
+ /**
+ * Bytes read at a time while checking the file for UTF-16 and CR-only line endings.
+ */
+ private const INSPECT_CHUNK_BYTES = 8192;
+
+ /**
+ * Constructor.
+ *
+ * @since 11.4.0
+ *
+ * @param string $file Absolute path to the CSV file.
+ * @param array<string, mixed> $options Importer options. Header parsing and row reading use delimiter and
+ * enclosure; the other two are kept for the import run.
+ * - notify_customer (bool): Whether to fire customer notifications. Default false.
+ * - delimiter (string): Single-character CSV delimiter. Default ','.
+ * - enclosure (string): Single-byte CSV enclosure. A line break, a non-ASCII byte,
+ * the delimiter, or anything that is not one byte falls back to '"'.
+ * - update_existing (bool): Update fulfillment when one with the same tracking number
+ * already exists on the order. Default true.
+ */
+ public function __construct( string $file, array $options = array() ) {
+ $delimiter = self::normalize_delimiter( $options['delimiter'] ?? ',' );
+ $this->file = $file;
+ $this->options = array(
+ 'notify_customer' => ! empty( $options['notify_customer'] ),
+ 'delimiter' => $delimiter,
+ 'enclosure' => self::normalize_enclosure( $options['enclosure'] ?? '"', $delimiter ),
+ 'update_existing' => array_key_exists( 'update_existing', $options ) ? (bool) $options['update_existing'] : true,
+ );
+ }
+
+ /**
+ * Normalize a delimiter input to the single ASCII byte fgetcsv() needs.
+ *
+ * The spellings "\t", "\\t" and "tab" map to a tab character. Longer strings are
+ * clamped to their first byte. Empty, non-string, non-ASCII, line-break and
+ * double-quote inputs fall back to a comma.
+ *
+ * @since 11.4.0
+ *
+ * @param mixed $delimiter Raw delimiter input.
+ * @return string Delimiter string (defaults to ',').
+ */
+ public static function normalize_delimiter( $delimiter ): string {
+ if ( ! is_string( $delimiter ) || '' === $delimiter ) {
+ return ',';
+ }
+ if ( self::is_tab_spelling( $delimiter ) ) {
+ return "\t";
+ }
+ // substr() slices by byte; the first byte of a multibyte character is a
+ // malformed fragment that would make fgetcsv() silently mis-parse the file,
+ // so anything outside the ASCII range falls back to the default.
+ $first = substr( $delimiter, 0, 1 );
+ if ( ord( $first ) >= 0x80 || in_array( $first, array( "\r", "\n", '"' ), true ) ) {
+ return ',';
+ }
+ return $first;
+ }
+
+ /**
+ * Normalize an enclosure to the single byte fgetcsv() can use.
+ *
+ * Empty, non-string, and multi-byte values fall back to a double quote. So do
+ * line breaks, bytes outside ASCII, and the delimiter itself: fgetcsv() would
+ * accept them and then split quoted fields in the wrong places.
+ *
+ * @since 11.4.0
+ *
+ * @param mixed $enclosure Raw enclosure input.
+ * @param string $delimiter Delimiter already normalized to one byte.
+ * @return string Enclosure character.
+ */
+ public static function normalize_enclosure( $enclosure, string $delimiter ): string {
+ if ( ! is_string( $enclosure ) || 1 !== strlen( $enclosure ) ) {
+ return '"';
+ }
+ if ( $enclosure === $delimiter || ord( $enclosure ) >= 0x80 || "\r" === $enclosure || "\n" === $enclosure ) {
+ return '"';
+ }
+ return $enclosure;
+ }
+
+ /**
+ * Whether a delimiter input is one of the accepted spellings of a tab character.
+ *
+ * @since 11.4.0
+ *
+ * @param string $delimiter Raw delimiter input.
+ * @return bool
+ */
+ public static function is_tab_spelling( string $delimiter ): bool {
+ return in_array( strtolower( $delimiter ), self::TAB_SPELLINGS, true );
+ }
+
+ /**
+ * Parse the CSV header row and return metadata sufficient to drive the column-mapping UI.
+ *
+ * Streams the file, counting non-blank data rows and keeping the first as a sample.
+ * `sample` has at least as many entries as `headers`: a short row is padded with empty
+ * strings and a longer row keeps its extra cells. Header and data cells are trimmed.
+ * Blank records before the header and between data rows are skipped. Does not fail
+ * when required canonical columns cannot be auto-detected; the caller can present
+ * the mapping UI so the user resolves it manually.
+ *
+ * `total` is the exact number of non-blank data rows, except when the file has more
+ * than the cap. Counting stops once `MAX_IMPORT_ROWS + 1` data rows have been seen
+ * and one further data row proves the file is larger. `total` is then
+ * `MAX_IMPORT_ROWS + 1` and `total_capped` is true. That value is a sentinel, not
+ * an exact count. A file with exactly 5,001 data rows, including one that ends
+ * after blank records, has the same `total` and `total_capped` false. The unread
+ * tail of a larger file is not scanned. Callers reject either over-cap case with
+ * `total > MAX_IMPORT_ROWS`, and must not display `total` when `total_capped` is true.
+ *
+ * @since 11.4.0
+ *
+ * @param string $delimiter Delimiter override; falls back to the constructor delimiter when empty.
+ * @return array{
+ * headers?: array<int, string>,
+ * sample?: array<int, string>,
+ * total?: int,
+ * total_capped?: bool,
+ * detected_mapping?: array<int, string>,
+ * delimiter?: string,
+ * error?: array{code:string, message:string}
+ * }
+ */
+ public function parse_headers( string $delimiter = '' ): array {
+ $opened = $this->open_csv( $delimiter );
+ if ( isset( $opened['error'] ) ) {
+ return $opened;
+ }
+
+ $handle = $opened['handle'];
+ try {
+ $this->strip_bom( $handle );
+ $summary = $this->read_header_summary( $handle, $opened['delimiter'] );
+ if ( isset( $summary['error'] ) ) {
+ return $summary;
+ }
+ if ( $this->conversion_failed ) {
+ return $this->unsupported_encoding_error();
+ }
+
+ $header_map = $this->build_header_map( $summary['headers'] );
+ $detected_mapping = array();
+ foreach ( $header_map as $canonical => $col_index ) {
+ $detected_mapping[ (int) $col_index ] = (string) $canonical;
+ }
+
+ return array(
+ 'headers' => $summary['headers'],
+ 'sample' => $summary['sample'],
+ 'total' => $summary['total'],
+ 'total_capped' => $summary['total_capped'],
+ 'detected_mapping' => $detected_mapping,
+ 'delimiter' => $opened['delimiter'],
+ );
+ } finally {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- Closing the handle opened above.
+ fclose( $handle );
+ }
+ }
+
+ /**
+ * Read a window of real data rows.
+ *
+ * This is the reader the chunked import run should use. The first non-blank record
+ * is the header and is not returned in `rows`. `$offset` and `$limit` count only
+ * real data rows. Blank records, including ",,," and whitespace-only rows, are not
+ * returned and do not count, but `byte_offset` still moves past them to the next
+ * data row (or to the end of the file). Each returned row's `row_number` is its
+ * 1-based physical record number, counting the header and blank records. Each row's
+ * `cells` has at least `count( headers )` entries: a short row is padded with empty
+ * strings so `$cells[ $index ]` is always a string for a mapped column, and a longer
+ * row keeps its extra cells. Every cell is trimmed.
+ *
+ * Pass a previous result's `byte_offset` and `next_record_number` to resume without
+ * reading the skipped rows again. `byte_offset` 0 always starts at the beginning.
+ * A non-zero `byte_offset` must be a position this method returned. `offset` still
+ * skips that many further data rows after the seek.
+ *
+ * @since 11.4.0
+ *
+ * @param int $offset Real data rows to skip after the header or the seek point.
+ * @param int $limit Maximum number of real data rows to return.
+ * @param array<string, mixed> $options {
+ * Optional read state.
+ *
+ * @type int $byte_offset File position from a previous result. 0 starts from the beginning.
+ * @type int $record_number Physical record number to assign at `$byte_offset`. Required when resuming;
+ * pass the previous `next_record_number`. Ignored when `$byte_offset` is 0.
+ * @type string $delimiter Delimiter override. Empty uses the constructor delimiter.
+ * }
+ * @return array{
+ * headers?: array<int, string>,
+ * header_row_number?: int,
+ * rows?: array<int, array{row_number:int, cells:array<int, string>}>,
+ * byte_offset?: int,
+ * next_record_number?: int,
+ * consumed?: int,
+ * skipped?: int,
+ * eof?: bool,
+ * delimiter?: string,
+ * error?: array{code:string, message:string}
+ * }
+ */
+ public function read_data_rows( int $offset, int $limit, array $options = array() ): array {
+ $offset = max( 0, $offset );
+ $limit = max( 0, $limit );
+ $byte_offset = isset( $options['byte_offset'] ) ? max( 0, (int) $options['byte_offset'] ) : 0;
+ $delimiter = isset( $options['delimiter'] ) && is_string( $options['delimiter'] ) ? $options['delimiter'] : '';
+ $opened = $this->open_csv( $delimiter );
+ if ( isset( $opened['error'] ) ) {
+ return $opened;
+ }
+
+ $handle = $opened['handle'];
+ try {
+ // Byte offsets are positions after a leading BOM has been removed. Strip
+ // it on every open, including a resume, before seeking to one of them.
+ $this->strip_bom( $handle );
+ $result = $this->read_data_window( $handle, $opened['delimiter'], $offset, $limit, $byte_offset, $options );
+ if ( isset( $result['error'] ) ) {
+ return $result;
+ }
+ if ( $this->conversion_failed ) {
+ return $this->unsupported_encoding_error();
+ }
+ return $result;
+ } finally {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- Closing the handle opened above.
+ fclose( $handle );
+ }
+ }
+
+ /**
+ * Read one window of data rows from a handle already past its BOM.
+ *
+ * @param resource $handle Open handle.
+ * @param string $delimiter Delimiter passed to fgetcsv().
+ * @param int $offset Real data rows to skip.
+ * @param int $limit Maximum real data rows to return.
+ * @param int $byte_offset Resume position, or 0 to start after the header.
+ * @param array<string, mixed> $options Resume options. Uses record_number when $byte_offset is set.
+ * @return array<string, mixed>
+ */
+ private function read_data_window( $handle, string $delimiter, int $offset, int $limit, int $byte_offset, array $options ): array {
+ $this->conversion_failed = false;
+ $records = $this->read_records( $handle, $delimiter, 1 );
+ $header = $this->next_data_record( $records );
+ if ( null === $header ) {
+ return $this->error( 'empty_csv', __( 'CSV file is empty.', 'woocommerce' ) );
+ }
+
+ $last_row_number = (int) $header['row_number'];
+ $width = count( $header['cells'] );
+
+ if ( $byte_offset > 0 ) {
+ // Drop the header generator before seeking so it cannot read again from the new position.
+ unset( $records );
+ if ( 0 !== fseek( $handle, $byte_offset ) ) {
+ return $this->error( 'byte_offset_invalid', __( 'Could not resume reading the CSV file.', 'woocommerce' ) );
+ }
+ $record_number = isset( $options['record_number'] ) ? max( 1, (int) $options['record_number'] ) : 1;
+ $records = $this->read_records( $handle, $delimiter, $record_number );
+ $last_row_number = $record_number - 1;
+ }
+
+ $skipped = 0;
+ $eof = false;
+ $position = 0;
+ $next_record_number = $last_row_number + 1;
+ while ( $skipped < $offset ) {
+ $record = $this->next_data_record( $records, $last_row_number );
+ if ( null === $record ) {
+ $eof = true;
+ break;
+ }
+ ++$skipped;
+ }
+
+ $rows = array();
+ $consumed = 0;
+ if ( ! $eof ) {
+ while ( $consumed < $limit ) {
+ $record = $this->next_data_record( $records, $last_row_number );
+ if ( null === $record ) {
+ $eof = true;
+ break;
+ }
+ $rows[] = array(
+ 'row_number' => (int) $record['row_number'],
+ 'cells' => $this->pad_row( $record['cells'], $width ),
+ );
+ ++$consumed;
+ }
+ }
+
+ if ( ! $eof ) {
+ $upcoming = $this->next_data_record( $records, $last_row_number );
+ if ( null === $upcoming ) {
+ $eof = true;
+ } else {
+ $position = (int) $upcoming['start'];
+ $next_record_number = (int) $upcoming['row_number'];
+ }
+ }
+
+ if ( $eof ) {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_ftell -- Position of the handle opened above.
+ $end_position = ftell( $handle );
+ $position = false === $end_position ? 0 : (int) $end_position;
+ $next_record_number = $last_row_number + 1;
+ }
+
+ return array(
+ 'headers' => $header['cells'],
+ 'header_row_number' => (int) $header['row_number'],
+ 'rows' => $rows,
+ 'byte_offset' => $position,
+ 'next_record_number' => $next_record_number,
+ 'consumed' => $consumed,
+ 'skipped' => $skipped,
+ 'eof' => $eof,
+ 'delimiter' => $delimiter,
+ );
+ }
+
+ /**
+ * Build an error result.
+ *
+ * @param string $code Machine-readable error code.
+ * @param string $message Message safe to show to the merchant.
+ * @return array{error: array{code:string, message:string}}
+ */
+ private function error( string $code, string $message ): array {
+ return array(
+ 'error' => array(
+ 'code' => $code,
+ 'message' => $message,
+ ),
+ );
+ }
+
+ /**
+ * Open the CSV and reject a UTF-16 or CR-only file.
+ *
+ * The handle is left at byte 0. The caller strips a BOM.
+ *
+ * @param string $delimiter_override Delimiter override. Empty uses the constructor delimiter.
+ * @return array{handle: resource, delimiter: string}|array{error: array{code:string, message:string}}
+ */
+ private function open_csv( string $delimiter_override ): array {
+ $delimiter = '' === $delimiter_override ? $this->options['delimiter'] : self::normalize_delimiter( $delimiter_override );
+
+ if ( ! is_readable( $this->file ) ) {
+ return $this->error( 'file_not_readable', __( 'File is not readable.', 'woocommerce' ) );
+ }
+
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fopen -- Streaming a staged local CSV; WP_Filesystem has no line reader.
+ $handle = fopen( $this->file, 'rb' );
+ if ( false === $handle ) {
+ return $this->error( 'file_open_failed', __( 'Could not open file.', 'woocommerce' ) );
+ }
+
+ if ( $this->is_utf16( $handle ) ) {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- Closing the handle opened above.
+ fclose( $handle );
+ // Excel's "Unicode Text" export. fgetcsv() would return cells with a NUL after every character.
+ return $this->error(
+ 'utf16_encoding',
+ __( 'This file is saved as UTF-16 and cannot be read. Re-save it as a UTF-8 CSV and try again.', 'woocommerce' )
+ );
+ }
+
+ if ( $this->is_cr_only( $handle ) ) {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- Closing the handle opened above.
+ fclose( $handle );
+ // fgetcsv() only splits on LF. auto_detect_line_endings is deprecated, so a CR-only file would be one record.
+ return $this->error(
+ 'cr_line_endings',
+ __( 'This file uses old Mac line endings and cannot be read. Re-save it as CSV and try again.', 'woocommerce' )
+ );
+ }
+
+ rewind( $handle );
+ return array(
+ 'handle' => $handle,
+ 'delimiter' => $delimiter,
+ );
+ }
+
+ /**
+ * Whether the file is UTF-16, judged from its first bytes.
+ *
+ * A UTF-16 BOM, or a NUL byte anywhere in the first chunk, is enough: a UTF-8
+ * CSV never contains NUL, while UTF-16 text has one after every ASCII character.
+ *
+ * @param resource $handle Open file handle.
+ * @return bool
+ */
+ private function is_utf16( $handle ): bool {
+ rewind( $handle );
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fread -- Reading the handle opened by the caller.
+ $chunk = fread( $handle, self::INSPECT_CHUNK_BYTES );
+ if ( ! is_string( $chunk ) || '' === $chunk ) {
+ return false;
+ }
+ $bom = substr( $chunk, 0, 2 );
+ return "\xFF\xFE" === $bom || "\xFE\xFF" === $bom || false !== strpos( $chunk, "\0" );
+ }
+
+ /**
+ * Whether the file uses CR line endings and no LF.
+ *
+ * Stops at the first LF, so a normal CSV is not read to the end.
+ *
+ * @param resource $handle Open file handle.
+ * @return bool
+ */
+ private function is_cr_only( $handle ): bool {
+ $saw_cr = false;
+ rewind( $handle );
+ while ( ! feof( $handle ) ) {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fread -- Reading the handle opened by the caller.
+ $chunk = fread( $handle, self::INSPECT_CHUNK_BYTES );
+ if ( ! is_string( $chunk ) || '' === $chunk ) {
+ break;
+ }
+ if ( false !== strpos( $chunk, "\n" ) ) {
+ return false;
+ }
+ if ( false !== strpos( $chunk, "\r" ) ) {
+ $saw_cr = true;
+ }
+ }
+ return $saw_cr;
+ }
+
+ /**
+ * Read the header and count data rows, stopping once the cap is decided.
+ *
+ * The count is incremented before the cap check. After MAX_IMPORT_ROWS + 1
+ * data rows, one further data row sets total_capped and the read stops.
+ * Exactly that many data rows, or that many followed only by blanks, leaves
+ * total_capped false.
+ *
+ * @param resource $handle Open handle positioned after a BOM strip.
+ * @param string $delimiter Delimiter passed to fgetcsv().
+ * @return array{error: array{code:string, message:string}}|array{headers: array<int, string>, sample: array<int, string>, total: int, total_capped: bool}
+ */
+ private function read_header_summary( $handle, string $delimiter ): array {
+ $this->conversion_failed = false;
+ $records = $this->read_records( $handle, $delimiter, 1 );
+ $header = $this->next_data_record( $records );
+ if ( null === $header ) {
+ return $this->error( 'empty_csv', __( 'CSV file is empty.', 'woocommerce' ) );
+ }
+
+ $sample = array();
+ $total = 0;
+ $total_capped = false;
+ while ( true ) {
+ $record = $this->next_data_record( $records );
+ if ( null === $record ) {
+ break;
+ }
+ ++$total;
+ if ( 1 === $total ) {
+ $sample = $this->pad_row( $record['cells'], count( $header['cells'] ) );
+ }
+ if ( $total > self::MAX_IMPORT_ROWS ) {
+ $extra = $this->next_data_record( $records );
+ $total_capped = null !== $extra;
+ break;
+ }
+ }
+
+ return array(
+ 'headers' => $header['cells'],
+ 'sample' => $sample,
+ 'total' => $total,
+ 'total_capped' => $total_capped,
+ );
+ }
+
+ /**
+ * Error for text that is not UTF-8 and cannot be converted.
+ *
+ * @return array{error: array{code:string, message:string}}
+ */
+ private function unsupported_encoding_error(): array {
+ return $this->error(
+ 'unsupported_encoding',
+ __( 'This file is not valid UTF-8 and could not be converted. Save it as a UTF-8 CSV and try again.', 'woocommerce' )
+ );
+ }
+
+ /**
+ * Yield each physical CSV record, with blank rows flagged and cells decoded.
+ *
+ * @param resource $handle Open file handle positioned at the first record to read.
+ * @param string $delimiter Delimiter passed to fgetcsv().
+ * @param int $next_record_number Physical record number to assign to the first record read.
+ * @return \Generator<int, array{row_number:int, cells:array<int, string>, blank:bool, start:int}>
+ */
+ private function read_records( $handle, string $delimiter, int $next_record_number ): \Generator {
+ while ( true ) {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_ftell -- Position of the handle opened by the caller.
+ $start = ftell( $handle );
+ if ( false === $start ) {
+ break;
+ }
+
+ $record = fgetcsv( $handle, 0, $delimiter, $this->options['enclosure'], '' );
+ if ( ! is_array( $record ) ) {
+ break;
+ }
+
+ $cells = $this->normalize_cells( $record );
+ yield array(
+ 'row_number' => $next_record_number,
+ 'cells' => $cells,
+ 'blank' => $this->is_blank_row( $cells ),
+ 'start' => (int) $start,
+ );
+ ++$next_record_number;
+ }
+ }
+
+ /**
+ * Return the next non-blank record, moving the reader past any blanks in between.
+ *
+ * @param \Generator $records Record generator from read_records().
+ * @param int|null $last_row_number Set to the physical number of the last record pulled, including blanks.
+ * @return array{row_number:int, cells:array<int, string>, blank:bool, start:int}|null
+ */
+ private function next_data_record( \Generator $records, &$last_row_number = null ): ?array {
+ // Advance with next() rather than foreach. A second foreach rewinds, and rewind()
+ // either repeats the first record or throws once the generator has moved on.
+ while ( $records->valid() ) {
+ $record = $records->current();
+ $records->next();
+ $last_row_number = $record['row_number'];
+ if ( ! $record['blank'] ) {
+ return $record;
+ }
+ }
+ return null;
+ }
+
+ /**
+ * Turn one fgetcsv() record into trimmed UTF-8 strings.
+ *
+ * Leading and trailing ASCII whitespace is removed from every cell, so "1042, 1Z999, ups"
+ * yields the same values as "1042,1Z999,ups" and a tracking number or carrier slug
+ * never carries a stray space into a URL or a lookup.
+ *
+ * Each cell is judged on its own: valid UTF-8 is kept, anything else is read as
+ * Windows-1252. A Windows-1252 cell whose high bytes happen to form valid UTF-8,
+ * such as a literal "é", is kept as UTF-8; that is rarer than a file that mixes
+ * both encodings.
+ *
+ * @param array<int, string|null> $record Raw record.
+ * @return array<int, string>
+ */
+ private function normalize_cells( array $record ): array {
+ $cells = array();
+ foreach ( $record as $value ) {
+ $cell = is_scalar( $value ) ? trim( (string) $value ) : '';
+ if ( '' !== $cell && ! self::is_utf8( $cell ) ) {
+ $converted = $this->convert_windows_1252( $cell );
+ if ( null === $converted ) {
+ $this->conversion_failed = true;
+ $cell = '';
+ } else {
+ $cell = $converted;
+ }
+ }
+ $cells[] = $cell;
+ }
+ return $cells;
+ }
+
+ /**
+ * Pad a data row with empty strings up to the header width.
+ *
+ * A line with fewer fields than the header comes back from fgetcsv() as a shorter
+ * record, which would leave a mapped index undefined. Extra cells on a longer row are kept.
+ *
+ * @param array<int, string> $cells Decoded row.
+ * @param int $width Number of header cells.
+ * @return array<int, string>
+ */
+ private function pad_row( array $cells, int $width ): array {
+ return count( $cells ) < $width ? array_pad( $cells, $width, '' ) : $cells;
+ }
+
+ /**
+ * Whether a string is well-formed UTF-8.
+ *
+ * Both checks reject overlong sequences, surrogate halves and stray high bytes.
+ *
+ * @param string $value Bytes to check.
+ * @return bool
+ */
+ private static function is_utf8( string $value ): bool {
+ if ( function_exists( 'mb_check_encoding' ) ) {
+ return mb_check_encoding( $value, 'UTF-8' );
+ }
+ return 1 === preg_match( '//u', $value );
+ }
+
+ /**
+ * Convert one Windows-1252 cell to UTF-8.
+ *
+ * ASCII bytes are unchanged. mbstring is used when it is available, otherwise
+ * iconv. Null means neither converter could produce UTF-8.
+ *
+ * @param string $value Cell text in Windows-1252.
+ * @return string|null
+ */
+ private function convert_windows_1252( string $value ): ?string {
+ if ( '' === $value ) {
+ return '';
+ }
+ if ( function_exists( 'mb_convert_encoding' ) ) {
+ $converted = mb_convert_encoding( $value, 'UTF-8', 'Windows-1252' );
+ if ( is_string( $converted ) ) {
+ return $converted;
+ }
+ }
+ if ( function_exists( 'iconv' ) ) {
+ $converted = iconv( 'Windows-1252', 'UTF-8', $value );
+ if ( is_string( $converted ) ) {
+ return $converted;
+ }
+ }
+ return null;
+ }
+
+ /**
+ * Skip a leading UTF-8 BOM, rewinding if not present.
+ *
+ * @param resource $handle Open file handle positioned at byte 0.
+ */
+ private function strip_bom( $handle ): void {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fread -- Reading 3 bytes from an already-open handle.
+ $bom = fread( $handle, 3 );
+ if ( "\xEF\xBB\xBF" !== $bom ) {
+ rewind( $handle );
+ }
+ }
+
+ /**
+ * Invert a column-index-keyed mapping into a canonical-keyed header map.
+ *
+ * @since 11.4.0
+ *
+ * @param array<int, string> $mapping CSV column index => canonical column key. Unmapped slots may be "".
+ * @return array<string, int> Canonical column key => CSV column index. First wins on duplicates.
+ */
+ public static function mapping_to_header_map( array $mapping ): array {
+ $header_map = array();
+ foreach ( $mapping as $col_index => $canonical ) {
+ $canonical = is_string( $canonical ) ? trim( $canonical ) : '';
+ if ( '' === $canonical ) {
+ continue;
+ }
+ $index = (int) $col_index;
+ if ( ! isset( $header_map[ $canonical ] ) ) {
+ $header_map[ $canonical ] = $index;
+ }
+ }
+ return $header_map;
+ }
+
+ /**
+ * Map normalized header values to column indexes.
+ *
+ * @param array<int, string|null> $header Raw header row.
+ * @return array<string, int> Map of canonical key => column index.
+ */
+ private function build_header_map( array $header ): array {
+ $aliases = $this->get_column_aliases();
+ $map = array();
+
+ foreach ( $header as $index => $name ) {
+ $normalized = $this->normalize_header( (string) $name );
+ if ( '' === $normalized ) {
+ continue;
+ }
+ foreach ( $aliases as $canonical => $alias_list ) {
+ if ( in_array( $normalized, $alias_list, true ) && ! isset( $map[ $canonical ] ) ) {
+ $map[ $canonical ] = $index;
+ break;
+ }
+ }
+ }
+
+ return $map;
+ }
+
+ /**
+ * Determine which required columns are missing.
+ *
+ * @since 11.4.0
+ *
+ * @param array<string, int> $header_map Header map.
+ * @return array<int, string> Human-friendly names of missing columns.
+ */
+ public static function find_missing_required_columns( array $header_map ): array {
+ // Column keys are the literal header values merchants must put in their CSV; do not translate.
+ $required = array(
+ self::COL_ORDER_NUMBER => self::COL_ORDER_NUMBER,
+ self::COL_TRACKING_NUMBER => self::COL_TRACKING_NUMBER,
+ self::COL_PROVIDER => self::COL_PROVIDER,
+ );
+ $missing = array();
+ foreach ( $required as $key => $label ) {
+ if ( ! isset( $header_map[ $key ] ) ) {
+ $missing[] = $label;
+ }
+ }
+ return $missing;
+ }
+
+ /**
+ * Normalize a header cell to a lowercase snake-case string for fuzzy matching.
+ *
+ * @param string $name Raw header.
+ * @return string
+ */
+ private function normalize_header( string $name ): string {
+ $name = strtolower( trim( $name ) );
+ $name = preg_replace( '/[^a-z0-9]+/', '_', $name );
+ return trim( (string) $name, '_' );
+ }
+
+ /**
+ * Aliases recognized for each canonical column.
+ *
+ * @return array<string, array<int, string>>
+ */
+ private function get_column_aliases(): array {
+ $defaults = array(
+ self::COL_ORDER_NUMBER => array( 'order_number', 'order', 'order_id', 'order_no', 'order_num' ),
+ self::COL_TRACKING_NUMBER => array( 'tracking_number', 'tracking', 'tracking_no', 'tracking_num' ),
+ self::COL_PROVIDER => array( 'shipment_provider', 'provider', 'carrier', 'shipping_provider', 'shipping_carrier' ),
+ self::COL_TRACKING_URL => array( 'tracking_url', 'url' ),
+ self::COL_ITEMS => array( 'items', 'line_items' ),
+ );
+
+ /**
+ * Filter the header aliases recognized by the fulfillments CSV importer.
+ *
+ * Lets stores accept additional header names from third-party WMS/3PL exports.
+ * Keys are canonical column identifiers; values are arrays of accepted aliases.
+ * Aliases are normalized the same way as header cells (lowercase, non-alphanumeric
+ * runs collapsed to "_"), so "Order No." and "order_no" are the same alias. Only the
+ * importer's own canonical keys are honored: the wizard works from that fixed set,
+ * so a new key here would be detected and then rejected.
+ *
+ * @since 11.4.0
+ *
+ * @param array<string, array<int, string>> $aliases Default alias map.
+ */
+ $aliases = apply_filters( 'woocommerce_fulfillments_csv_importer_column_aliases', $defaults );
+
+ // Any callback can return anything; drop malformed entries instead of
+ // letting a non-array alias list fatal in build_header_map().
+ if ( ! is_array( $aliases ) ) {
+ return $defaults;
+ }
+
+ $canonical_columns = self::canonical_columns();
+ $sanitized = array();
+ foreach ( $aliases as $canonical => $alias_list ) {
+ if ( ! is_string( $canonical ) || ! in_array( $canonical, $canonical_columns, true ) || ! is_array( $alias_list ) ) {
+ continue;
+ }
+ $clean = array();
+ foreach ( $alias_list as $alias ) {
+ $normalized = is_string( $alias ) ? $this->normalize_header( $alias ) : '';
+ if ( '' !== $normalized ) {
+ $clean[] = $normalized;
+ }
+ }
+ if ( array() !== $clean ) {
+ $sanitized[ $canonical ] = $clean;
+ }
+ }
+
+ // A filter that leaves nothing usable would make every header undetectable.
+ return array() === $sanitized ? $defaults : $sanitized;
+ }
+
+ /**
+ * Whether every cell in a decoded row is empty.
+ *
+ * Cells arrive trimmed from normalize_cells(), so a blank line, a row of empty
+ * fields such as ",,,", and a whitespace-only row are all blank. A cell holding
+ * "0" is not.
+ *
+ * @param array<int, string> $row Decoded row.
+ * @return bool
+ */
+ private function is_blank_row( array $row ): bool {
+ foreach ( $row as $cell ) {
+ if ( '' !== $cell ) {
+ return false;
+ }
+ }
+ return true;
+ }
+}
diff --git a/plugins/woocommerce/tests/php/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvImporterTest.php b/plugins/woocommerce/tests/php/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvImporterTest.php
new file mode 100644
index 00000000000..9ff0c0906d7
--- /dev/null
+++ b/plugins/woocommerce/tests/php/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvImporterTest.php
@@ -0,0 +1,949 @@
+<?php declare( strict_types=1 );
+
+namespace Automattic\WooCommerce\Tests\Admin\Features\Fulfillments\Importer;
+
+require_once __DIR__ . '/FulfillmentsCsvReadTracker.php';
+
+use Automattic\WooCommerce\Admin\Features\Fulfillments\FulfillmentsController;
+use Automattic\WooCommerce\Admin\Features\Fulfillments\Importer\FulfillmentsCsvImporter;
+
+/**
+ * Tests for the Fulfillments CSV importer service.
+ */
+class FulfillmentsCsvImporterTest extends \WC_Unit_Test_Case {
+
+ /**
+ * Original value of the fulfillments feature flag.
+ *
+ * @var mixed
+ */
+ private static $original_fulfillments_flag;
+
+ /**
+ * Paths to temporary CSV files created by tests; cleaned up in tearDown.
+ *
+ * @var array<int, string>
+ */
+ private array $temp_files = array();
+
+ /**
+ * Bootstrap the fulfillments feature for the test run.
+ */
+ public static function setUpBeforeClass(): void {
+ parent::setUpBeforeClass();
+ if ( ! in_array( 'wcimp', stream_get_wrappers(), true ) ) {
+ stream_wrapper_register( 'wcimp', FulfillmentsCsvReadTracker::class );
+ }
+ self::$original_fulfillments_flag = get_option( 'woocommerce_feature_fulfillments_enabled' );
+ update_option( 'woocommerce_feature_fulfillments_enabled', 'yes' );
+ $controller = wc_get_container()->get( FulfillmentsController::class );
+ $controller->register();
+ $controller->initialize_fulfillments();
+ }
+
+ /**
+ * Restore the original feature flag value.
+ */
+ public static function tearDownAfterClass(): void {
+ if ( false === self::$original_fulfillments_flag ) {
+ delete_option( 'woocommerce_feature_fulfillments_enabled' );
+ } else {
+ update_option( 'woocommerce_feature_fulfillments_enabled', self::$original_fulfillments_flag );
+ }
+ parent::tearDownAfterClass();
+ }
+
+ /**
+ * Clean up any temp files between tests.
+ */
+ public function tearDown(): void {
+ foreach ( $this->temp_files as $path ) {
+ if ( file_exists( $path ) ) {
+ wp_delete_file( $path );
+ }
+ }
+ $this->temp_files = array();
+ parent::tearDown();
+ }
+
+ /**
+ * Write the given CSV content to a temp file and return its path.
+ *
+ * @param string $content CSV content.
+ * @return string
+ */
+ private function make_csv( string $content ): string {
+ $path = wp_tempnam( 'wc-fulfillments-import-' );
+ file_put_contents( $path, $content ); // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_file_put_contents -- Test fixture write.
+ $this->temp_files[] = $path;
+ return $path;
+ }
+
+ /**
+ * @testdox Header aliases (e.g. "Tracking" / "Carrier") are normalized to the canonical columns.
+ */
+ public function test_header_aliases_are_accepted(): void {
+ $csv = "Order,Tracking,Carrier\n1,ALIAS-1,ups\n";
+ $file = $this->make_csv( $csv );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame(
+ array(
+ 0 => FulfillmentsCsvImporter::COL_ORDER_NUMBER,
+ 1 => FulfillmentsCsvImporter::COL_TRACKING_NUMBER,
+ 2 => FulfillmentsCsvImporter::COL_PROVIDER,
+ ),
+ $parsed['detected_mapping']
+ );
+ }
+
+ /**
+ * @testdox Canonical keys the importer does not know are dropped from the column-aliases filter.
+ */
+ public function test_alias_filter_ignores_unknown_canonical_keys(): void {
+ $filter = function ( $aliases ) {
+ $aliases['warehouse_bay'] = array( 'bay' );
+ return $aliases;
+ };
+ add_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', $filter );
+
+ try {
+ $csv = "order_number,tracking_number,shipment_provider,bay\n1,UNKNOWN-KEY,ups,A1\n";
+ $sut = new FulfillmentsCsvImporter( $this->make_csv( $csv ) );
+ $parsed = $sut->parse_headers();
+ } finally {
+ remove_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', $filter );
+ }
+
+ // An unknown key would be auto-detected here even though the importer can never import it.
+ $this->assertNotContains( 'warehouse_bay', $parsed['detected_mapping'] );
+ }
+
+ /**
+ * @testdox Malformed values from the column-aliases filter are dropped instead of fataling the import.
+ */
+ public function test_malformed_alias_filter_output_is_tolerated(): void {
+ $filter = function ( $aliases ) {
+ // A misbehaving callback: non-array alias list, non-string entries, junk key.
+ $aliases[ FulfillmentsCsvImporter::COL_TRACKING_URL ] = 'not-an-array';
+ $aliases[ FulfillmentsCsvImporter::COL_ITEMS ] = array( 42, null, 'items' );
+ $aliases[0] = array( 'zero' );
+ return $aliases;
+ };
+ add_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', $filter );
+
+ try {
+ $csv = "order_number,tracking_number,shipment_provider,items\n1,ALIAS-BAD,ups,\n";
+ $parsed = ( new FulfillmentsCsvImporter( $this->make_csv( $csv ) ) )->parse_headers();
+ } finally {
+ remove_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', $filter );
+ }
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame(
+ array(
+ 0 => FulfillmentsCsvImporter::COL_ORDER_NUMBER,
+ 1 => FulfillmentsCsvImporter::COL_TRACKING_NUMBER,
+ 2 => FulfillmentsCsvImporter::COL_PROVIDER,
+ 3 => FulfillmentsCsvImporter::COL_ITEMS,
+ ),
+ $parsed['detected_mapping'],
+ 'Valid alias entries must keep working when the filter returns junk for others'
+ );
+ }
+
+ /**
+ * @testdox parse_headers returns headers, sample row, total and auto-detected mapping.
+ */
+ public function test_parse_headers_returns_metadata_and_mapping(): void {
+ $csv = "Order ID,Tracking,Carrier,URL,Items\n"
+ . "12345,1Z999AA,UPS,https://example.com,SKU-A:1\n"
+ . "67890,1Z000AA,UPS,https://example.com,SKU-B:1\n";
+ $file = $this->make_csv( $csv );
+
+ $sut = new FulfillmentsCsvImporter( $file );
+ $parsed = $sut->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( array( 'Order ID', 'Tracking', 'Carrier', 'URL', 'Items' ), $parsed['headers'] );
+ $this->assertSame( array( '12345', '1Z999AA', 'UPS', 'https://example.com', 'SKU-A:1' ), $parsed['sample'] );
+ $this->assertSame( 2, $parsed['total'] );
+ $this->assertFalse( $parsed['total_capped'], 'A file under the cap has an exact total' );
+ $this->assertSame( ',', $parsed['delimiter'] );
+
+ $mapping = $parsed['detected_mapping'];
+ $this->assertSame( FulfillmentsCsvImporter::COL_ORDER_NUMBER, $mapping[0] );
+ $this->assertSame( FulfillmentsCsvImporter::COL_TRACKING_NUMBER, $mapping[1] );
+ $this->assertSame( FulfillmentsCsvImporter::COL_PROVIDER, $mapping[2] );
+ $this->assertSame( FulfillmentsCsvImporter::COL_TRACKING_URL, $mapping[3] );
+ $this->assertSame( FulfillmentsCsvImporter::COL_ITEMS, $mapping[4] );
+ }
+
+ /**
+ * @testdox parse_headers honors an explicit non-comma delimiter.
+ */
+ public function test_parse_headers_honors_explicit_delimiter(): void {
+ $csv = "order_number;tracking_number;shipment_provider\n1;TRACK-1;ups\n";
+ $file = $this->make_csv( $csv );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file, array( 'delimiter' => ';' ) ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( ';', $parsed['delimiter'] );
+ $this->assertSame( array( 'order_number', 'tracking_number', 'shipment_provider' ), $parsed['headers'] );
+ }
+
+ /**
+ * @testdox An empty delimiter option falls back to comma.
+ */
+ public function test_empty_delimiter_falls_back_to_comma(): void {
+ $csv = "order_number,tracking_number,shipment_provider\n1,TRACK-1,ups\n";
+ $file = $this->make_csv( $csv );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file, array( 'delimiter' => '' ) ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( ',', $parsed['delimiter'] );
+ }
+
+ /**
+ * @testdox A multi-character delimiter is clamped to its first byte so fgetcsv() does not throw.
+ */
+ public function test_multi_character_delimiter_is_clamped_to_single_byte(): void {
+ $csv = "order_number;tracking_number;shipment_provider\n1;TRACK-1;ups\n";
+ $file = $this->make_csv( $csv );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file, array( 'delimiter' => ';;' ) ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( ';', $parsed['delimiter'] );
+ }
+
+ /**
+ * @testdox An enclosure fgetcsv() cannot take falls back to the double quote instead of throwing.
+ *
+ * @testWith ["ab"]
+ * [""]
+ * [["\""]]
+ *
+ * @param mixed $enclosure Enclosure option.
+ */
+ public function test_invalid_enclosure_falls_back_to_double_quote( $enclosure ): void {
+ $csv = "\"order_number\",\"tracking_number\",\"shipment_provider\"\n\"1\",\"TRACK, 1\",\"ups\"\n";
+ $file = $this->make_csv( $csv );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file, array( 'enclosure' => $enclosure ) ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed, 'An unusable enclosure must not fail the parse' );
+ $this->assertSame( array( 'order_number', 'tracking_number', 'shipment_provider' ), $parsed['headers'], 'Headers must be read with the default enclosure' );
+ $this->assertSame( array( '1', 'TRACK, 1', 'ups' ), $parsed['sample'], 'The enclosed comma must stay inside its field' );
+ }
+
+ /**
+ * @testdox A multibyte delimiter falls back to comma instead of a truncated byte fragment.
+ */
+ public function test_multibyte_delimiter_falls_back_to_comma(): void {
+ // Em dash and left double quotation mark, written as UTF-8 byte sequences.
+ $this->assertSame( ',', FulfillmentsCsvImporter::normalize_delimiter( "\xE2\x80\x94" ) );
+ $this->assertSame( ',', FulfillmentsCsvImporter::normalize_delimiter( "\xE2\x80\x9C" ) );
+ $this->assertSame( "\t", FulfillmentsCsvImporter::normalize_delimiter( "\t" ), 'ASCII control delimiters like tab must be preserved' );
+ $this->assertSame( '|', FulfillmentsCsvImporter::normalize_delimiter( '|' ) );
+ }
+
+ /**
+ * @testdox The spelled-out tab forms map to a tab character.
+ *
+ * @testWith ["\\t"]
+ * ["\\\\t"]
+ * ["tab"]
+ * ["TAB"]
+ *
+ * @param string $spelling Delimiter input.
+ */
+ public function test_tab_spellings_map_to_tab( string $spelling ): void {
+ $this->assertSame( "\t", FulfillmentsCsvImporter::normalize_delimiter( $spelling ) );
+ $this->assertTrue( FulfillmentsCsvImporter::is_tab_spelling( $spelling ) );
+ }
+
+ /**
+ * @testdox Line breaks and the enclosure character fall back to comma.
+ *
+ * @testWith ["\n"]
+ * ["\r"]
+ * ["\""]
+ *
+ * @param string $delimiter Delimiter input.
+ */
+ public function test_unusable_delimiters_fall_back_to_comma( string $delimiter ): void {
+ $this->assertSame( ',', FulfillmentsCsvImporter::normalize_delimiter( $delimiter ) );
+ $this->assertFalse( FulfillmentsCsvImporter::is_tab_spelling( $delimiter ) );
+ }
+
+ /**
+ * @testdox parse_headers reads a tab-separated file when the delimiter is spelled out.
+ */
+ public function test_parse_headers_honors_spelled_out_tab_delimiter(): void {
+ $file = $this->make_csv( "order_number\ttracking_number\tshipment_provider\n1\tTAB-1\tups\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers( 'tab' );
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( "\t", $parsed['delimiter'] );
+ $this->assertSame( array( 'order_number', 'tracking_number', 'shipment_provider' ), $parsed['headers'] );
+ $this->assertSame( array( '1', 'TAB-1', 'ups' ), $parsed['sample'] );
+ }
+
+ /**
+ * @testdox parse_headers strips a UTF-8 BOM from the first header cell.
+ */
+ public function test_parse_headers_strips_bom(): void {
+ $file = $this->make_csv( "\xEF\xBB\xBForder_number,tracking_number,shipment_provider\n1,BOM-1,ups\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( 'order_number', $parsed['headers'][0] );
+ $this->assertSame( FulfillmentsCsvImporter::COL_ORDER_NUMBER, $parsed['detected_mapping'][0] );
+ }
+
+ /**
+ * @testdox parse_headers reports zero rows and no sample for a header-only file.
+ */
+ public function test_parse_headers_reports_zero_rows_for_header_only_file(): void {
+ $file = $this->make_csv( "order_number,tracking_number,shipment_provider\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( 0, $parsed['total'] );
+ $this->assertSame( array(), $parsed['sample'] );
+ $this->assertSame( array( 'order_number', 'tracking_number', 'shipment_provider' ), $parsed['headers'] );
+ }
+
+ /**
+ * @testdox parse_headers skips blank lines before the header.
+ */
+ public function test_parse_headers_skips_leading_blank_lines(): void {
+ $file = $this->make_csv( "\n\norder_number,tracking_number,shipment_provider\n1,BLANK-1,ups\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( array( 'order_number', 'tracking_number', 'shipment_provider' ), $parsed['headers'] );
+ $this->assertSame( 1, $parsed['total'] );
+ $this->assertSame( array( '1', 'BLANK-1', 'ups' ), $parsed['sample'] );
+ }
+
+ /**
+ * @testdox parse_headers reports an empty CSV when the file holds only blank lines.
+ */
+ public function test_parse_headers_reports_empty_csv_for_blank_lines_only(): void {
+ $file = $this->make_csv( "\n\n,,\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+
+ $this->assertArrayHasKey( 'error', $parsed );
+ $this->assertSame( 'empty_csv', $parsed['error']['code'] );
+ }
+
+ /**
+ * @testdox parse_headers does not count blank data lines towards the total.
+ */
+ public function test_parse_headers_ignores_blank_data_lines(): void {
+ $file = $this->make_csv( "order_number,tracking_number,shipment_provider\n\n1,ROW-1,ups\n,,\n \n2,ROW-2,ups\n\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( 2, $parsed['total'] );
+ $this->assertSame( array( '1', 'ROW-1', 'ups' ), $parsed['sample'] );
+ }
+
+ /**
+ * @testdox A file larger than the cap stops at 5,001 rows and does not read the rest.
+ */
+ public function test_parse_headers_stops_before_the_rest_of_an_over_cap_file(): void {
+ $prefix = "order_number,tracking_number,shipment_provider\n";
+ for ( $i = 1; $i <= FulfillmentsCsvImporter::MAX_IMPORT_ROWS + 2; $i++ ) {
+ $prefix .= "1,OVER-{$i},ups\n";
+ }
+ $tail = str_repeat( "9,TAIL,ups\n", 200000 );
+ $path = $this->make_csv( $prefix . $tail );
+
+ FulfillmentsCsvReadTracker::$max_position = 0;
+ $parsed = ( new FulfillmentsCsvImporter( 'wcimp://' . $path ) )->parse_headers();
+
+ $this->assertSame( FulfillmentsCsvImporter::MAX_IMPORT_ROWS + 1, $parsed['total'] );
+ $this->assertTrue( $parsed['total_capped'], 'MAX+2 data rows must set the sentinel and stop' );
+ $this->assertLessThan( filesize( $path ), FulfillmentsCsvReadTracker::$max_position, 'The tail past the cap must not be read' );
+ $this->assertLessThan( strlen( $prefix ) + 65536, FulfillmentsCsvReadTracker::$max_position, 'Reading must stop within one read-ahead of the proving row' );
+ }
+
+ /**
+ * @testdox A file with exactly one row past the cap still reports that exact total.
+ */
+ public function test_parse_headers_exact_total_one_row_past_the_cap_is_not_a_sentinel(): void {
+ $csv = "order_number,tracking_number,shipment_provider\n";
+ for ( $i = 1; $i <= FulfillmentsCsvImporter::MAX_IMPORT_ROWS + 1; $i++ ) {
+ $csv .= "1,EXACT-{$i},ups\n";
+ }
+ $csv .= "\n,,,\n";
+
+ $parsed = ( new FulfillmentsCsvImporter( $this->make_csv( $csv ) ) )->parse_headers();
+
+ $this->assertSame( FulfillmentsCsvImporter::MAX_IMPORT_ROWS + 1, $parsed['total'] );
+ $this->assertFalse( $parsed['total_capped'], 'Exactly 5,001 data rows is a real total, not the over-cap sentinel' );
+ }
+
+ /**
+ * @testdox Blank rows do not count toward the import cap.
+ */
+ public function test_parse_headers_does_not_count_blank_rows_toward_the_cap(): void {
+ $csv = "order_number,tracking_number,shipment_provider\n";
+ for ( $i = 1; $i <= FulfillmentsCsvImporter::MAX_IMPORT_ROWS; $i++ ) {
+ $csv .= "1,CAP-{$i},ups\n,,,\n";
+ }
+
+ $parsed = ( new FulfillmentsCsvImporter( $this->make_csv( $csv ) ) )->parse_headers();
+
+ $this->assertSame( FulfillmentsCsvImporter::MAX_IMPORT_ROWS, $parsed['total'] );
+ $this->assertFalse( $parsed['total_capped'], 'Blank rows between data rows must not push the file over the cap' );
+ }
+
+ /**
+ * @testdox The first of two headers mapping to the same canonical column wins.
+ */
+ public function test_parse_headers_first_duplicate_header_wins(): void {
+ $file = $this->make_csv( "tracking_number,tracking,order_number,shipment_provider\nT-1,T-2,1,ups\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+
+ $this->assertSame( FulfillmentsCsvImporter::COL_TRACKING_NUMBER, $parsed['detected_mapping'][0] );
+ $this->assertArrayNotHasKey( 1, $parsed['detected_mapping'] );
+ $this->assertSame( FulfillmentsCsvImporter::COL_ORDER_NUMBER, $parsed['detected_mapping'][2] );
+ }
+
+ /**
+ * @testdox parse_headers handles CRLF line endings without leaking the carriage return into cells.
+ */
+ public function test_parse_headers_handles_crlf_line_endings(): void {
+ $file = $this->make_csv( "order_number,tracking_number,shipment_provider\r\n1,CRLF-1,ups\r\n2,CRLF-2,ups\r\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( array( 'order_number', 'tracking_number', 'shipment_provider' ), $parsed['headers'] );
+ $this->assertSame( array( '1', 'CRLF-1', 'ups' ), $parsed['sample'] );
+ $this->assertSame( 2, $parsed['total'] );
+ }
+
+ /**
+ * @testdox An alias added through the filter is matched regardless of its casing and punctuation.
+ */
+ public function test_alias_filter_entries_are_normalized(): void {
+ $filter = function ( $aliases ) {
+ $aliases[ FulfillmentsCsvImporter::COL_ORDER_NUMBER ][] = 'PO Number';
+ return $aliases;
+ };
+ add_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', $filter );
+
+ try {
+ $file = $this->make_csv( "po_number,tracking_number,shipment_provider\n1,PO-1,ups\n" );
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+ } finally {
+ remove_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', $filter );
+ }
+
+ $this->assertSame( FulfillmentsCsvImporter::COL_ORDER_NUMBER, $parsed['detected_mapping'][0] );
+ }
+
+ /**
+ * @testdox The default aliases are used when the filter leaves nothing usable.
+ */
+ public function test_alias_filter_without_usable_entries_falls_back_to_defaults(): void {
+ $filter = function () {
+ return array( FulfillmentsCsvImporter::COL_ORDER_NUMBER => array( 42, '', ' ' ) );
+ };
+ add_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', $filter );
+
+ try {
+ $file = $this->make_csv( "order_number,tracking_number,shipment_provider\n1,DEF-1,ups\n" );
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+ } finally {
+ remove_filter( 'woocommerce_fulfillments_csv_importer_column_aliases', $filter );
+ }
+
+ $this->assertSame(
+ array(
+ 0 => FulfillmentsCsvImporter::COL_ORDER_NUMBER,
+ 1 => FulfillmentsCsvImporter::COL_TRACKING_NUMBER,
+ 2 => FulfillmentsCsvImporter::COL_PROVIDER,
+ ),
+ $parsed['detected_mapping']
+ );
+ }
+
+ /**
+ * @testdox parse_headers reports an error when the file is empty.
+ */
+ public function test_parse_headers_reports_empty_csv(): void {
+ $file = $this->make_csv( '' );
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers( ',' );
+
+ $this->assertArrayHasKey( 'error', $parsed );
+ $this->assertSame( 'empty_csv', $parsed['error']['code'] );
+ }
+
+ /**
+ * @testdox parse_headers reports an error when the file is missing.
+ */
+ public function test_parse_headers_reports_missing_file(): void {
+ $parsed = ( new FulfillmentsCsvImporter( '/nonexistent/path/missing.csv' ) )->parse_headers( ',' );
+
+ $this->assertArrayHasKey( 'error', $parsed );
+ $this->assertSame( 'file_not_readable', $parsed['error']['code'] );
+ }
+
+ /**
+ * @testdox Rows whose cells all trim to empty are blank, including ",,," and whitespace.
+ */
+ public function test_blank_rows_include_empty_fields_and_whitespace(): void {
+ $file = $this->make_csv( ",,,\n \norder_number,tracking_number,shipment_provider\n , , \n1,ROW-1,ups\n\t\t\t\n0,0,ups\n,,,\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+ $read = ( new FulfillmentsCsvImporter( $file ) )->read_data_rows( 0, 10 );
+
+ $this->assertArrayNotHasKey( 'error', $parsed );
+ $this->assertSame( array( 'order_number', 'tracking_number', 'shipment_provider' ), $parsed['headers'] );
+ $this->assertSame( 2, $parsed['total'], 'Only the two non-blank data rows count' );
+ $this->assertSame( array( '1', 'ROW-1', 'ups' ), $parsed['sample'] );
+ $this->assertSame( $parsed['total'], $read['consumed'] );
+ $this->assertSame(
+ array(
+ array(
+ 'row_number' => 5,
+ 'cells' => array( '1', 'ROW-1', 'ups' ),
+ ),
+ array(
+ 'row_number' => 7,
+ 'cells' => array( '0', '0', 'ups' ),
+ ),
+ ),
+ $read['rows'],
+ 'Physical record numbers keep the leading blanks, the header, and the blank data rows'
+ );
+ $this->assertSame( 3, $read['header_row_number'] );
+ $this->assertTrue( $read['eof'] );
+ }
+
+ /**
+ * @testdox Offset fast-forward and a resumed byte offset both count only real data rows.
+ */
+ public function test_read_data_rows_skips_and_resumes_on_real_data_rows(): void {
+ $file = $this->make_csv( "\norder_number,tracking_number,shipment_provider\n\n1,ONE,ups\n,,,\n , , \n2,TWO,ups\n\n3,THREE,ups\n" );
+ $sut = new FulfillmentsCsvImporter( $file );
+
+ $first = $sut->read_data_rows( 0, 1 );
+ $this->assertSame( 4, $first['rows'][0]['row_number'], 'The first data row keeps its physical record number' );
+ $this->assertSame( array( '1', 'ONE', 'ups' ), $first['rows'][0]['cells'] );
+ $this->assertSame( 1, $first['consumed'] );
+ $this->assertSame( 0, $first['skipped'] );
+ $this->assertFalse( $first['eof'] );
+ $this->assertSame( 7, $first['next_record_number'], 'The saved record number moves past the blank rows after the first data row' );
+
+ $resumed = $sut->read_data_rows(
+ 0,
+ 1,
+ array(
+ 'byte_offset' => $first['byte_offset'],
+ 'record_number' => $first['next_record_number'],
+ )
+ );
+ $this->assertSame( array( '2', 'TWO', 'ups' ), $resumed['rows'][0]['cells'] );
+ $this->assertSame( 7, $resumed['rows'][0]['row_number'] );
+
+ $skipped = $sut->read_data_rows( 1, 1 );
+ $this->assertSame( $resumed['rows'], $skipped['rows'], 'Skipping one data row lands on the same record as the byte offset' );
+ $this->assertSame( 1, $skipped['skipped'] );
+ $this->assertSame( 1, $skipped['consumed'] );
+ $this->assertFalse( $skipped['eof'] );
+
+ $rest = $sut->read_data_rows(
+ 0,
+ 10,
+ array(
+ 'byte_offset' => $resumed['byte_offset'],
+ 'record_number' => $resumed['next_record_number'],
+ )
+ );
+ $this->assertSame( array( '3', 'THREE', 'ups' ), $rest['rows'][0]['cells'] );
+ $this->assertTrue( $rest['eof'], 'Trailing blank rows are not another data row' );
+ $this->assertSame( 1, $rest['consumed'] );
+ }
+
+ /**
+ * @testdox A CR-only file is rejected and the merchant is told to re-save it as CSV.
+ */
+ public function test_cr_only_line_endings_are_rejected(): void {
+ $file = $this->make_csv( "order_number,tracking_number,shipment_provider\r1,CR-1,ups\r2,CR-2,ups\r" );
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+ $read = ( new FulfillmentsCsvImporter( $file ) )->read_data_rows( 0, 10 );
+
+ $this->assertSame( 'cr_line_endings', $parsed['error']['code'] );
+ $this->assertStringContainsString( 'Re-save', $parsed['error']['message'] );
+ $this->assertStringContainsString( 'CSV', $parsed['error']['message'] );
+ $this->assertSame( 'cr_line_endings', $read['error']['code'] );
+ }
+
+ /**
+ * @testdox Windows-1252 cells are converted at the shared read boundary, and valid UTF-8 cells are left unchanged.
+ *
+ * Each cell is checked for well-formed UTF-8 on its own. Conversion uses
+ * mb_convert_encoding when that exists, otherwise iconv. This environment
+ * has mbstring, so the iconv-only branch and the neither-available branch
+ * are not executed. When neither converter exists, an invalid cell returns
+ * unsupported_encoding instead of the raw bytes.
+ */
+ public function test_character_encoding_is_normalized_for_prepare_and_row_reads(): void {
+ $utf8_file = $this->make_csv( "caf\xC3\xA9,tracking_number,shipment_provider\n1,Spediteur M\xC3\xBCller,ups\n" );
+ $utf8 = ( new FulfillmentsCsvImporter( $utf8_file ) )->parse_headers();
+ $this->assertSame( 'café', $utf8['headers'][0], 'Valid UTF-8 headers must not be reinterpreted as Windows-1252' );
+ $this->assertSame( 'Spediteur Müller', $utf8['sample'][1] );
+
+ $euro = "\xE2\x82\xAC";
+ $padding = str_repeat( 'a', 9000 );
+ $long = ( new FulfillmentsCsvImporter( $this->make_csv( "order_number,tracking_number,shipment_provider\n1,{$padding}{$euro}{$padding},ups\n" ) ) )->parse_headers();
+ $this->assertSame( $padding . $euro . $padding, $long['sample'][1], 'A multibyte character past a read boundary must stay valid UTF-8' );
+
+ $legacy_file = $this->make_csv( "order_number,tracking_number,shipment_provider\n1,\xC3\xA9,Spediteur M\xFCller\n" );
+ $legacy = ( new FulfillmentsCsvImporter( $legacy_file ) )->parse_headers();
+ $can_convert = function_exists( 'mb_convert_encoding' ) || function_exists( 'iconv' );
+ if ( ! $can_convert ) {
+ $this->assertSame( 'unsupported_encoding', $legacy['error']['code'], 'Invalid text must not be returned raw when it cannot be converted' );
+ return;
+ }
+
+ $legacy_rows = ( new FulfillmentsCsvImporter( $legacy_file ) )->read_data_rows( 0, 1 );
+ $this->assertSame( 'é', $legacy['sample'][1], 'A valid UTF-8 cell is kept even when another cell in the row is Windows-1252' );
+ $this->assertSame( 'Spediteur Müller', $legacy['sample'][2], 'A Windows-1252 cell is converted on its own' );
+ $this->assertSame( $legacy['sample'], $legacy_rows['rows'][0]['cells'], 'Prepare and the row reader must decode the same text' );
+ }
+
+ /**
+ * @testdox A resumed read keeps BOM-stripped offsets and decodes each cell on its own.
+ */
+ public function test_read_data_rows_resume_keeps_bom_offset_and_encoding(): void {
+ if ( ! function_exists( 'mb_convert_encoding' ) && ! function_exists( 'iconv' ) ) {
+ $this->markTestSkipped( 'Windows-1252 conversion is unavailable in this environment.' );
+ }
+
+ $file = $this->make_csv( "\xEF\xBB\xBForder_number,tracking_number,shipment_provider\n1,ONE,M\xFCller\n2,\xC3\xA9,TWO\n" );
+ $sut = new FulfillmentsCsvImporter( $file );
+
+ $first = $sut->read_data_rows( 0, 1 );
+ $this->assertSame( 'order_number', $first['headers'][0], 'The BOM must not remain on the header' );
+ $this->assertSame( 'Müller', $first['rows'][0]['cells'][2] );
+
+ $second = $sut->read_data_rows(
+ 0,
+ 1,
+ array(
+ 'byte_offset' => $first['byte_offset'],
+ 'record_number' => $first['next_record_number'],
+ )
+ );
+ $this->assertSame( 'order_number', $second['headers'][0], 'Resuming must strip the BOM before seeking' );
+ $this->assertSame( 'é', $second['rows'][0]['cells'][1], 'A valid UTF-8 cell in a later row is kept as it is' );
+ $this->assertSame( 'TWO', $second['rows'][0]['cells'][2] );
+ $this->assertSame( $first['next_record_number'], $second['rows'][0]['row_number'] );
+ }
+
+ /**
+ * @testdox An enclosure fgetcsv() would misread falls back to the double quote.
+ */
+ public function test_unusable_enclosure_values_fall_back_to_double_quote(): void {
+ $csv = "order_number,tracking_number,shipment_provider\n1042,\"1Z999,AA\",\"Acme, Inc\"\n";
+ foreach ( array( "\n", "\r", ',', "\x80" ) as $enclosure ) {
+ $parsed = ( new FulfillmentsCsvImporter( $this->make_csv( $csv ), array( 'enclosure' => $enclosure ) ) )->parse_headers();
+ $this->assertSame( array( '1042', '1Z999,AA', 'Acme, Inc' ), $parsed['sample'], 'A line break, the delimiter, or a non-ASCII enclosure must not split quoted fields' );
+ }
+
+ $this->assertSame( '"', FulfillmentsCsvImporter::normalize_enclosure( ';', ';' ) );
+ $semicolon = "\"order_number\";\"tracking_number\";\"shipment_provider\"\n\"1\";\"TRACK; 1\";\"ups\"\n";
+ $parsed = ( new FulfillmentsCsvImporter(
+ $this->make_csv( $semicolon ),
+ array(
+ 'delimiter' => ';',
+ 'enclosure' => ';',
+ )
+ ) )->parse_headers();
+ $this->assertSame( ';', $parsed['delimiter'] );
+ $this->assertSame( array( '1', 'TRACK; 1', 'ups' ), $parsed['sample'] );
+ }
+
+ /**
+ * @testdox A one-byte enclosure other than the delimiter is kept.
+ */
+ public function test_single_quote_enclosure_is_preserved(): void {
+ $csv = "'order_number','tracking_number','shipment_provider'\n'1','TRACK, 1','ups'\n";
+ $parsed = ( new FulfillmentsCsvImporter( $this->make_csv( $csv ), array( 'enclosure' => "'" ) ) )->parse_headers();
+
+ $this->assertSame( array( 'order_number', 'tracking_number', 'shipment_provider' ), $parsed['headers'] );
+ $this->assertSame( array( '1', 'TRACK, 1', 'ups' ), $parsed['sample'] );
+ }
+
+ /**
+ * @testdox mapping_to_header_map keeps the first column for a repeated canonical key.
+ */
+ public function test_mapping_to_header_map_keeps_the_first_duplicate_canonical_key(): void {
+ $map = FulfillmentsCsvImporter::mapping_to_header_map(
+ array(
+ 0 => 'order_number',
+ 1 => 'order_number',
+ 2 => '',
+ 3 => ' tracking_number ',
+ '4' => 'shipment_provider',
+ 5 => 12,
+ )
+ );
+
+ $this->assertSame(
+ array(
+ 'order_number' => 0,
+ 'tracking_number' => 3,
+ 'shipment_provider' => 4,
+ ),
+ $map,
+ 'Empty and non-string targets are ignored, and the first duplicate canonical key wins'
+ );
+ }
+
+ /**
+ * @testdox find_missing_required_columns reports order number, tracking number, and shipment provider.
+ */
+ public function test_find_missing_required_columns_reports_the_required_set(): void {
+ $this->assertSame(
+ array(),
+ FulfillmentsCsvImporter::find_missing_required_columns(
+ array(
+ 'order_number' => 0,
+ 'tracking_number' => 1,
+ 'shipment_provider' => 2,
+ 'tracking_url' => 3,
+ 'items' => 4,
+ )
+ )
+ );
+ $this->assertSame(
+ array( 'order_number', 'shipment_provider' ),
+ FulfillmentsCsvImporter::find_missing_required_columns(
+ array(
+ 'tracking_number' => 1,
+ 'tracking_url' => 3,
+ 'items' => 4,
+ )
+ ),
+ 'Optional columns do not satisfy a missing required column'
+ );
+ }
+
+ /**
+ * @testdox The prepare total and a chunked read agree on every blank-line shape and every chunk size.
+ */
+ public function test_prepare_total_matches_a_chunked_read_across_blank_line_shapes(): void {
+ $csv = "\n";
+ $csv .= ",,,\n";
+ $csv .= "order_number,tracking_number,shipment_provider\n";
+ $csv .= "1,ONE,ups\n";
+ $csv .= "\n";
+ $csv .= "2,TWO,ups\n";
+ $csv .= ",,,\n";
+ $csv .= "3,THREE,ups\n";
+ $csv .= "4,FOUR,ups\n";
+ $csv .= " , , \n";
+ $csv .= "\n";
+ $csv .= "5,FIVE,ups\n";
+ $csv .= "6,SIX,ups\n";
+ $csv .= "\n\n";
+ $file = $this->make_csv( $csv );
+ $sut = new FulfillmentsCsvImporter( $file );
+
+ $parsed = $sut->parse_headers();
+ $this->assertSame( 6, $parsed['total'], 'Six real data rows, blank records excluded' );
+
+ $whole = $sut->read_data_rows( 0, 100 );
+ $this->assertSame( array( 4, 6, 8, 9, 12, 13 ), array_column( $whole['rows'], 'row_number' ), 'Row numbers are the physical record numbers in the file' );
+ $this->assertSame( 3, $whole['header_row_number'] );
+
+ for ( $chunk = 1; $chunk <= 8; $chunk++ ) {
+ $rows = array();
+ $options = array();
+ $calls = 0;
+ do {
+ $result = $sut->read_data_rows( 0, $chunk, $options );
+ $this->assertArrayNotHasKey( 'error', $result, "Chunk size {$chunk} must read cleanly" );
+ $rows = array_merge( $rows, $result['rows'] );
+ $options = array(
+ 'byte_offset' => $result['byte_offset'],
+ 'record_number' => $result['next_record_number'],
+ );
+ ++$calls;
+ } while ( ! $result['eof'] && $calls < 50 );
+
+ $this->assertSame( $parsed['total'], count( $rows ), "Chunk size {$chunk} must return exactly the prepare total" );
+ $this->assertSame( $whole['rows'], $rows, "Chunk size {$chunk} must return the same rows and row numbers as one read" );
+ }
+ }
+
+ /**
+ * @testdox A blank record on the chunk boundary is neither returned nor counted on resume.
+ */
+ public function test_blank_record_on_a_chunk_boundary_is_skipped_once(): void {
+ $file = $this->make_csv( "order_number,tracking_number,shipment_provider\n1,ONE,ups\n2,TWO,ups\n,,,\n\n3,THREE,ups\n" );
+ $sut = new FulfillmentsCsvImporter( $file );
+
+ $first = $sut->read_data_rows( 0, 2 );
+ $this->assertSame( array( 2, 3 ), array_column( $first['rows'], 'row_number' ) );
+ $this->assertFalse( $first['eof'], 'A real row remains after the blank records' );
+ $this->assertSame( 6, $first['next_record_number'], 'The resume point is the next real row, past both blank records' );
+
+ $second = $sut->read_data_rows(
+ 0,
+ 2,
+ array(
+ 'byte_offset' => $first['byte_offset'],
+ 'record_number' => $first['next_record_number'],
+ )
+ );
+ $this->assertSame(
+ array(
+ array(
+ 'row_number' => 6,
+ 'cells' => array( '3', 'THREE', 'ups' ),
+ ),
+ ),
+ $second['rows']
+ );
+ $this->assertSame( 1, $second['consumed'] );
+ $this->assertTrue( $second['eof'] );
+ }
+
+ /**
+ * @testdox Cells are decoded one at a time, so UTF-8 and Windows-1252 text in the same file both come out right.
+ */
+ public function test_mixed_encoding_file_decodes_each_cell_on_its_own(): void {
+ if ( ! function_exists( 'mb_convert_encoding' ) && ! function_exists( 'iconv' ) ) {
+ $this->markTestSkipped( 'Windows-1252 conversion is unavailable in this environment.' );
+ }
+
+ $file = $this->make_csv( "order_number,tracking_number,shipment_provider\n101,1Z1,Spediteur M\xC3\xBCller\n102,1Z2,Caf\xE9\n" );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+ $this->assertSame( 'Spediteur Müller', $parsed['sample'][2], 'A valid UTF-8 cell must not be re-decoded because another row is Windows-1252' );
+
+ $rows = ( new FulfillmentsCsvImporter( $file ) )->read_data_rows( 0, 2 );
+ $this->assertSame( 'Spediteur Müller', $rows['rows'][0]['cells'][2] );
+ $this->assertSame( 'Café', $rows['rows'][1]['cells'][2], 'A Windows-1252 cell is converted on its own' );
+ }
+
+ /**
+ * @testdox A UTF-16 file is rejected with or without a BOM and the merchant is told to re-save it as UTF-8.
+ */
+ public function test_utf16_files_are_rejected(): void {
+ $text = "order_number,tracking_number,shipment_provider\n101,1Z1,ups\n";
+ $utf16 = '';
+ foreach ( str_split( $text ) as $char ) {
+ $utf16 .= $char . "\0";
+ }
+
+ foreach ( array( "\xFF\xFE" . $utf16, $utf16, "\xFE\xFF" . strrev( $utf16 ) ) as $content ) {
+ $file = $this->make_csv( $content );
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+ $read = ( new FulfillmentsCsvImporter( $file ) )->read_data_rows( 0, 10 );
+
+ $this->assertSame( 'utf16_encoding', $parsed['error']['code'] ?? null, 'UTF-16 must be rejected instead of parsed into unmappable headers' );
+ $this->assertStringContainsString( 'UTF-8', $parsed['error']['message'] );
+ $this->assertSame( 'utf16_encoding', $read['error']['code'] ?? null, 'The row reader must reject the same file' );
+ }
+ }
+
+ /**
+ * @testdox Stream paths without a readable file are rejected like any other unreadable path.
+ */
+ public function test_unreadable_stream_paths_are_rejected(): void {
+ foreach ( array( 'php://memory', 'data://text/plain,order_number%0A1%0A' ) as $path ) {
+ $parsed = ( new FulfillmentsCsvImporter( $path ) )->parse_headers();
+ $this->assertSame( 'file_not_readable', $parsed['error']['code'] ?? null, "{$path} must not bypass the readability check" );
+ }
+ }
+
+ /**
+ * @testdox A data row shorter than the header is padded with empty strings in the sample and in read rows.
+ */
+ public function test_short_rows_are_padded_to_the_header_width(): void {
+ $file = $this->make_csv(
+ "order_number,tracking_number,shipment_provider,tracking_url\n"
+ . "1042,1Z999\n"
+ . "1043,1Z998,ups,https://example.com/t,extra\n"
+ . "1044\n"
+ );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+ $this->assertSame( array( '1042', '1Z999', '', '' ), $parsed['sample'], 'The sample must have one entry per header cell' );
+ $this->assertSame( 3, $parsed['total'] );
+
+ $read = ( new FulfillmentsCsvImporter( $file ) )->read_data_rows( 0, 10 );
+ $this->assertCount( 3, $read['rows'] );
+ $this->assertSame( array( '1042', '1Z999', '', '' ), $read['rows'][0]['cells'], 'A two-cell row must be padded to four cells' );
+ $this->assertSame( array( '1043', '1Z998', 'ups', 'https://example.com/t', 'extra' ), $read['rows'][1]['cells'], 'A longer row keeps its extra cell' );
+ $this->assertSame( array( '1044', '', '', '' ), $read['rows'][2]['cells'], 'A one-cell row must be padded to four cells' );
+
+ $header_map = FulfillmentsCsvImporter::mapping_to_header_map( $parsed['detected_mapping'] );
+ foreach ( $read['rows'] as $row ) {
+ foreach ( $header_map as $canonical => $index ) {
+ $this->assertIsString( $row['cells'][ $index ], "Row {$row['row_number']} must have a string at the {$canonical} index" );
+ }
+ }
+
+ $first = ( new FulfillmentsCsvImporter( $file ) )->read_data_rows( 0, 1 );
+ $resumed = ( new FulfillmentsCsvImporter( $file ) )->read_data_rows(
+ 0,
+ 10,
+ array(
+ 'byte_offset' => $first['byte_offset'],
+ 'record_number' => $first['next_record_number'],
+ )
+ );
+ $this->assertSame( array( '1044', '', '', '' ), $resumed['rows'][1]['cells'], 'Padding must apply on the resume path too' );
+ }
+
+ /**
+ * @testdox Header and data cells are trimmed on both the prepare and the row-read path.
+ */
+ public function test_cells_are_trimmed_in_headers_sample_and_rows(): void {
+ $file = $this->make_csv(
+ " order_number , Tracking ,Carrier\n"
+ . "1042, 1Z999 , ups \n"
+ . "\t1043 ,\" 1Z998 \",\"usps\"\n"
+ );
+
+ $parsed = ( new FulfillmentsCsvImporter( $file ) )->parse_headers();
+ $this->assertSame( array( 'order_number', 'Tracking', 'Carrier' ), $parsed['headers'], 'Header cells must be trimmed' );
+ $this->assertSame( array( '1042', '1Z999', 'ups' ), $parsed['sample'], 'Sample cells must be trimmed' );
+ $this->assertSame(
+ array(
+ 0 => 'order_number',
+ 1 => 'tracking_number',
+ 2 => 'shipment_provider',
+ ),
+ $parsed['detected_mapping']
+ );
+
+ $read = ( new FulfillmentsCsvImporter( $file ) )->read_data_rows( 0, 10 );
+ $this->assertSame( array( '1042', '1Z999', 'ups' ), $read['rows'][0]['cells'] );
+ $this->assertSame( array( '1043', '1Z998', 'usps' ), $read['rows'][1]['cells'], 'Whitespace inside a quoted cell and a leading tab are trimmed too' );
+ $this->assertSame( $parsed['headers'], $read['headers'], 'Both paths must return the same trimmed headers' );
+ }
+}
diff --git a/plugins/woocommerce/tests/php/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvReadTracker.php b/plugins/woocommerce/tests/php/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvReadTracker.php
new file mode 100644
index 00000000000..28a81ede7ad
--- /dev/null
+++ b/plugins/woocommerce/tests/php/src/Admin/Features/Fulfillments/Importer/FulfillmentsCsvReadTracker.php
@@ -0,0 +1,172 @@
+<?php
+/**
+ * Records how far a capped CSV parse reads.
+ *
+ * @package Automattic\WooCommerce\Tests\Admin\Features\Fulfillments\Importer
+ */
+
+declare( strict_types=1 );
+
+namespace Automattic\WooCommerce\Tests\Admin\Features\Fulfillments\Importer;
+
+/**
+ * Stream wrapper that counts the furthest byte read from a real file.
+ */
+class FulfillmentsCsvReadTracker {
+
+ /**
+ * Highest file position read or seeked during the parse.
+ *
+ * @var int
+ */
+ public static $max_position = 0;
+
+ /**
+ * Stream context PHP assigns to the wrapper instance.
+ *
+ * @var resource|null
+ */
+ public $context;
+
+ /**
+ * Underlying file handle.
+ *
+ * @var resource|null
+ */
+ public $handle;
+
+ /**
+ * Strip the wcimp:// prefix from a stream URL.
+ *
+ * @param string $path Stream URL.
+ * @return string Real file path.
+ */
+ private static function real_path( string $path ): string {
+ $prefix = 'wcimp://';
+ return 0 === strpos( $path, $prefix ) ? substr( $path, strlen( $prefix ) ) : $path;
+ }
+
+ /**
+ * Stat the real file so is_readable() and file_exists() work on the stream URL.
+ *
+ * @param string $path Stream URL.
+ * @param int $flags Stat flags.
+ * @return array<int|string, int>|false
+ */
+ public function url_stat( $path, $flags ) {
+ $file = self::real_path( (string) $path );
+ if ( ! file_exists( $file ) ) {
+ return false;
+ }
+ $stat = ( $flags & STREAM_URL_STAT_LINK ) ? lstat( $file ) : stat( $file );
+ return is_array( $stat ) ? $stat : false;
+ }
+
+ /**
+ * Open the real file named after the wcimp:// prefix.
+ *
+ * @param string $path Stream URL.
+ * @param string $mode Open mode.
+ * @param int $options Stream options.
+ * @param string $opened_path Resolved path. Assigned by PHP when requested.
+ * @return bool
+ */
+ public function stream_open( $path, $mode, $options, &$opened_path ) {
+ unset( $mode, $options, $opened_path );
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fopen -- The wrapper has to open the fixture the parser streams.
+ $handle = fopen( self::real_path( (string) $path ), 'rb' );
+ if ( false === $handle ) {
+ return false;
+ }
+ $this->handle = $handle;
+ return true;
+ }
+
+ /**
+ * Read and record the position.
+ *
+ * @param int $count Maximum bytes.
+ * @return string
+ */
+ public function stream_read( $count ) {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fread -- Reading the fixture handle opened above.
+ $data = fread( $this->handle, (int) $count );
+ $this->note_position();
+ return is_string( $data ) ? $data : '';
+ }
+
+ /**
+ * Seek and record the position.
+ *
+ * @param int $offset Offset.
+ * @param int $whence Whence.
+ * @return bool
+ */
+ public function stream_seek( $offset, $whence ) {
+ $ok = 0 === fseek( $this->handle, (int) $offset, (int) $whence );
+ $this->note_position();
+ return $ok;
+ }
+
+ /**
+ * Current position.
+ *
+ * @return int
+ */
+ public function stream_tell() {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_ftell -- Position of the fixture handle.
+ $position = ftell( $this->handle );
+ return false === $position ? 0 : (int) $position;
+ }
+
+ /**
+ * Whether the underlying file is at the end.
+ *
+ * @return bool
+ */
+ public function stream_eof() {
+ return feof( $this->handle );
+ }
+
+ /**
+ * Stat the underlying file.
+ *
+ * @return array<int|string, int>|false
+ */
+ public function stream_stat() {
+ $stat = fstat( $this->handle );
+ return is_array( $stat ) ? $stat : false;
+ }
+
+ /**
+ * Close the underlying file.
+ */
+ public function stream_close() {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- Closing the fixture handle opened above.
+ fclose( $this->handle );
+ }
+
+ /**
+ * Accept stream option changes. fgetcsv() does not depend on them.
+ *
+ * @param int $option Option.
+ * @param int $arg1 First argument.
+ * @param int $arg2 Second argument.
+ * @return bool
+ */
+ public function stream_set_option( $option, $arg1, $arg2 ) {
+ unset( $option, $arg1, $arg2 );
+ return false;
+ }
+
+ /**
+ * Remember the furthest position reached.
+ */
+ private function note_position() {
+ // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_ftell -- Position of the fixture handle.
+ $position = ftell( $this->handle );
+ if ( false !== $position && (int) $position > self::$max_position ) {
+ self::$max_position = (int) $position;
+ }
+ }
+}