Commit 00f0b9689e for aom
commit 00f0b9689e09863e54cff53405e93634aa2b1cec
Author: James Zern <jzern@google.com>
Date: Wed Nov 19 12:19:24 2025 -0800
v256_intrinsics_c.h: fix AVX2/C mismatch w/gcc+ubsan
This also clears the `-Wmaybe-uninitialized` warnings with gcc and
`-fsanitize=undefined`.
This is a backport of the libavm change:
7f95277d98 Fix AVX/C mismatch bug for v256_wideshuffle_8
Fix AVX/C mismatch bug for v256_wideshuffle_8
The bug was not in SIMD. In the reference C implementation array
indexing had a ternary operator as a clever shortcut.
This operation confused GCC compiler which produced different results
compared to SIMD that was correct, clang also matches SIMD correctly.
Fix explicitly defines the index as int outside the array.
This caused the reported bug in https://gitlab.com/AOMediaCodec/avm/-/issues/95.
Bug: 42302489
Change-Id: I0c3e09275db4d004f3669bd739cfbeb956a3dbdb
diff --git a/aom_dsp/simd/v256_intrinsics_c.h b/aom_dsp/simd/v256_intrinsics_c.h
index 20d2709aba..fb096f84d5 100644
--- a/aom_dsp/simd/v256_intrinsics_c.h
+++ b/aom_dsp/simd/v256_intrinsics_c.h
@@ -693,11 +693,13 @@ SIMD_INLINE c_v256 c_v256_shuffle_8(c_v256 a, c_v256 pattern) {
SIMD_INLINE c_v256 c_v256_wideshuffle_8(c_v256 a, c_v256 b, c_v256 pattern) {
c_v256 t;
int c;
- for (c = 0; c < 32; c++)
- t.u8[c] = (pattern.u8[c] < 32
- ? b.u8
- : a.u8)[CONFIG_BIG_ENDIAN ? 31 - (pattern.u8[c] & 31)
- : pattern.u8[c] & 31];
+ for (c = 0; c < 32; c++) {
+ const bool from_b_flag = pattern.u8[c] < 32;
+ const uint8_t *const src = from_b_flag ? b.u8 : a.u8;
+ const int idx =
+ CONFIG_BIG_ENDIAN ? 31 - (pattern.u8[c] & 31) : (pattern.u8[c] & 31);
+ t.u8[c] = src[idx];
+ }
return t;
}