Commit 3eccd1e6 for tesseract
commit 3eccd1e605cef7f49f9961e3abf19605dd997a08
Author: Stefan Weil <sw@weilnetz.de>
Date: Sat Nov 14 12:10:29 2026 +0100
lstm: use AVX2 for tanh, the logistic function and the conversion to int8_t
A profile of the recognition with tessdata_fast/eng on x86_64 shows
that the table lookups of Tanh and Logistic, which were done value by
value with branches, take 17 % of the time, and the conversion of the
results to int8_t in NetworkIO::WriteTimeStepPart 10 %. That is more
than the integer matrix product (12 %).
Add the function pointers TanhVector, LogisticVector and
FloatToInt8Vector for whole vectors. The generic implementations use
the scalar functions. The new implementations for AVX2 with FAST_FLOAT
handle 8 values per pass without branches. They load the table entries
one by one, which is much faster than the gather instruction on CPUs
with the microcode mitigation for gather data sampling. They are used
if the integer matrix for AVX2 is selected.
The new functions do the same calculations in the same order, so the
results are exactly the same as before. Huge values are limited before
they are scaled, so no floating point exception is raised for them.
Recognition of 36 pages with tessdata_fast/eng is 1.35 times faster on
an Intel Core i3-1115G4 (g++ 14: 23.0 s -> 17.0 s), recognition of
12 pages with tessdata_best/eng 1.13 times (18.6 s -> 16.4 s) and
training 1.05 times (16.0 s -> 15.3 s for 1000 lines). The text results
and the results of the training are identical.
Add a unit test which compares the results with those of the scalar
functions, and add the results for x86_64 to benchmark/lstm/README.md.
Assisted-by: Claude Fable 5.2 (Anthropic)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
diff --git a/CMakeLists.txt b/CMakeLists.txt
index bdd19c7d..7a25f639 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -660,9 +660,10 @@ if(HAVE_AVX)
PROPERTIES COMPILE_FLAGS ${AVX_COMPILE_FLAGS})
endif(HAVE_AVX)
if(HAVE_AVX2)
- list(APPEND arch_files_opt src/arch/intsimdmatrixavx2.cpp
- src/arch/dotproductavx.cpp)
- set_source_files_properties(src/arch/intsimdmatrixavx2.cpp
+ list(APPEND arch_files_opt src/arch/functionsavx2.cpp
+ src/arch/intsimdmatrixavx2.cpp src/arch/dotproductavx.cpp)
+ set_source_files_properties(src/arch/functionsavx2.cpp
+ src/arch/intsimdmatrixavx2.cpp
PROPERTIES COMPILE_FLAGS ${AVX2_COMPILE_FLAGS})
endif(HAVE_AVX2)
if(HAVE_AVX512F)
@@ -764,6 +765,7 @@ if(ENABLE_PRECOMPILED_HEADERS)
src/arch/dotproductfma.cpp
src/arch/dotproductsse.cpp
src/arch/dotproductneon.cpp
+ src/arch/functionsavx2.cpp
src/arch/intsimdmatrixavx2.cpp
src/arch/intsimdmatrixsse.cpp
src/arch/intsimdmatrixneon.cpp
diff --git a/Makefile.am b/Makefile.am
index 5b429b05..04c4f238 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -170,7 +170,10 @@ endif
if HAVE_AVX2
libtesseract_avx2_la_CXXFLAGS = -mavx2
libtesseract_avx2_la_CXXFLAGS += -I$(top_srcdir)/src/ccutil
-libtesseract_avx2_la_SOURCES = src/arch/intsimdmatrixavx2.cpp
+libtesseract_avx2_la_CXXFLAGS += -I$(top_srcdir)/src/arch
+libtesseract_avx2_la_CXXFLAGS += -I$(top_srcdir)/src/lstm
+libtesseract_avx2_la_SOURCES = src/arch/functionsavx2.cpp
+libtesseract_avx2_la_SOURCES += src/arch/intsimdmatrixavx2.cpp
libtesseract_la_LIBADD += libtesseract_avx2.la
noinst_LTLIBRARIES += libtesseract_avx2.la
endif
diff --git a/benchmark/lstm/README.md b/benchmark/lstm/README.md
index f881cf5f..1eb29651 100644
--- a/benchmark/lstm/README.md
+++ b/benchmark/lstm/README.md
@@ -10,13 +10,17 @@ measured with them.
- `sample-summary.py` summarizes a profile which was made with the macOS tool
`sample`.
- `neon-kernel-variants.cpp` is a microbenchmark for the float matrix products
- (variants of `RowDotProductsNEON` and `CrossDotProductsNEON`).
+ on ARM64 (variants of `RowDotProductsNEON` and `CrossDotProductsNEON`).
- `neon-int-kernel-variants.cpp` is a microbenchmark for the `int8_t` matrix
product which is used for the recognition with integer (fast) models
(variants of `PartialMatrixDotVector8`).
-The two microbenchmarks are standalone programs for ARM64. How to build
-them is described at the beginning of each file.
+- `x86-kernel-variants.cpp` is the same microbenchmark for the float matrix
+ products on x86_64 (variants of `RowDotProductsFMA`, `CrossDotProductsFMA`,
+ `RowDotProductsAVX512F` and `CrossDotProductsAVX512F`).
+
+The microbenchmarks are standalone programs for ARM64 or x86_64. How to
+build them is described at the beginning of each file.
## Timing the training
@@ -167,3 +171,118 @@ and after the new NEON functions for the integer matrix product:
- Recognition with the integer models is about 1.27 times faster on M1 and
X925 and 1.55 times faster on A725. The text results are identical.
- The float models do not use the integer matrix product.
+
+## Results for x86_64
+
+Measured in November 2026 on an Intel Core i3-1115G4 (Tiger Lake, AVX512),
+Debian 13, g++ 14, FAST_FLOAT, a Release build with CMake without OpenMP.
+"Before" is the commit "arch: speed up the integer matrix product with
+NEON". "Products" adds the functions for several dot products at once for
+FMA and AVX512F, "functions" also adds the functions for tanh, the logistic
+function and the conversion to `int8_t` for whole vectors with AVX2
+(`src/arch/functionsavx2.cpp`).
+
+### Training time on x86_64 (1000 training lines)
+
+| Settings | Before | Products | Functions |
+| ------------------------------ | -----: | -------: | --------: |
+| Default (AVX512F on this CPU) | 24.3 s | 16.0 s | 15.3 s |
+| `DOTPRODUCT=fma` | 24.3 s | 15.7 s | 15.0 s |
+| `DOTPRODUCT=avx2` | 28.1 s | 27.2 s | |
+
+- The training is 1.6 times faster.
+- A CPU with AVX2 and FMA, but without AVX512F, now uses the functions for
+ FMA by default. Before it used `DotProductAVX` (the time for
+ `DOTPRODUCT=avx2`), so the gain is larger there.
+- `DOTPRODUCT=avx2` still calculates single dot products with the same
+ results as before.
+- The functions for whole vectors give exactly the same results as the
+ scalar functions, so they do not change the results of the training.
+
+### Recognition time on x86_64
+
+Time for the 36 pages which are described above (`tessdata_fast`) and for
+the first 12 of them (`tessdata_best`):
+
+| Models | Before | Products | Functions |
+| -------------------------- | -----: | -------: | --------: |
+| `tessdata_fast` (36 pages) | 23.0 s | 23.0 s | 17.0 s |
+| `tessdata_best` (12 pages) | 22.7 s | 18.6 s | 16.4 s |
+
+- Recognition is 1.35 times faster with the integer models and 1.4 times
+ faster with the float models. The text results are identical.
+
+### Float matrix products (`x86-kernel-variants.cpp`)
+
+Time for the largest layer (Lfx192), single dot products compared with the
+implemented variants (g++ 14):
+
+| Product | FMA single | FMA new | AVX512F single | AVX512F new |
+| --------------------------------------- | ---------: | ------: | -------------: | ----------: |
+| Forward, 4 gates, per timestep (4x2) | 12.1 us | 9.1 us | 17.6 us | 9.5 us |
+| Backward, 4 gates, per timestep (4x2) | 12.6 us | 9.0 us | 18.2 us | 9.7 us |
+| Gradients, 1 gate, 300 timesteps | 1.24 ms | 0.37 ms | 1.17 ms | 0.35 ms |
+
+- The gradients use blocks of 4 x 2 dot products for FMA (16 registers;
+ 4 x 4 is 1.3 times slower) and of 4 x 4 for AVX512F (32 registers).
+- The smaller layers gain more for the products with a vector (2 to 3
+ times).
+- The rest of a row (less than 8 or 16 elements) is loaded with a mask
+ instead of a scalar loop.
+- Registers with 512 bits are not faster than those with 256 bits on this
+ CPU.
+- The four sums of a pass are reduced together (`Sum4`). A profile showed
+ that single calls of `_mm512_reduce_add_ps` took 12 % of the training
+ time.
+
+### Tanh, logistic function and conversion to `int8_t`
+
+A profile of the recognition with `tessdata_fast` (see below) showed that
+the scalar table lookups for tanh and the logistic function took 17 % of
+the time and the conversion of the results to `int8_t` 10 %, more than the
+integer matrix product (12 %).
+
+Time for tanh of 192 random values:
+
+| Variant | Time |
+| ---------------------------------------------------- | -----: |
+| Scalar (`Tanh`, with branches) | 707 ns |
+| Scalar without branches | 434 ns |
+| AVX2 with the gather instruction | 442 ns |
+| AVX2, table entries loaded one by one (implemented) | 158 ns |
+
+- The gather instruction is slow on this CPU because of the microcode
+ mitigation for "gather data sampling".
+- All variants give the same results.
+
+### Integer matrix product
+
+A variant with the dot product instruction of AVX512VNNI (`VPDPBUSD`, 256
+bits) was only 1.1 times faster than the code for AVX2 for the integer
+matrix product on this CPU, which is not measurable for the whole
+recognition. Therefore it was not added.
+
+### Profiling on Linux without perf
+
+The CPU profiler of gperftools works with an unchanged binary (build with
+`-g` to get source lines):
+
+ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libprofiler.so.0 \
+ CPUPROFILE=/tmp/tess.prof CPUPROFILE_FREQUENCY=1000 \
+ build/bin/tesseract pages.txt out --tessdata-dir ../tessdata_fast -l eng
+ google-pprof --text build/bin/tesseract /tmp/tess.prof | head -30
+ google-pprof --lines --text build/bin/tesseract /tmp/tess.prof | head -40
+
+Profile of the recognition with `tessdata_fast` (12 pages, share of the
+samples with everything a function calls):
+
+| Function | Products | Functions |
+| ------------------------------------------ | -------: | --------: |
+| `LSTM::Forward` | 37 % | 22 % |
+| Tanh and logistic function | 17 % | 10 % |
+| Integer `matrixDotVector` | 12 % | 14 % |
+| Conversion to `int8_t` | 10 % | 2.4 % |
+| `Convolve::Forward` | 12 % | 17 % |
+| Page layout analysis (`SegmentPage`) | 21 % | 28 % |
+
+The shares in the last column refer to a total which is 1.35 times smaller.
diff --git a/src/arch/dotproduct.h b/src/arch/dotproduct.h
index 1c1c5c52..5de07e9f 100644
--- a/src/arch/dotproduct.h
+++ b/src/arch/dotproduct.h
@@ -17,6 +17,7 @@
#ifndef TESSERACT_ARCH_DOTPRODUCT_H_
#define TESSERACT_ARCH_DOTPRODUCT_H_
+#include <cstdint> // for int8_t
#include "tesstypes.h"
namespace tesseract {
@@ -61,6 +62,13 @@ void RowDotProductsAVX512F(const float *w, int w_stride, int num_rows,
const float *u, int n, float *v);
void CrossDotProductsAVX512F(const float *u, int num_u, const float *v,
int num_v, int n, float *out, int out_stride);
+
+// Non-linearity functions and conversion to int8_t for n values with AVX2
+// (src/arch/functionsavx2.cpp), see TanhVector, LogisticVector and
+// FloatToInt8Vector in simddetect.h.
+void TanhVectorAVX2(int n, const float *in, float *out);
+void LogisticVectorAVX2(int n, const float *in, float *out);
+void FloatToInt8VectorAVX2(int n, const float *in, int8_t *out);
#endif
} // namespace tesseract.
diff --git a/src/arch/functionsavx2.cpp b/src/arch/functionsavx2.cpp
new file mode 100644
index 00000000..db0689d3
--- /dev/null
+++ b/src/arch/functionsavx2.cpp
@@ -0,0 +1,140 @@
+///////////////////////////////////////////////////////////////////////
+// File: functionsavx2.cpp
+// Description: Non-linearity functions and conversion of floats to
+// int8_t for whole vectors with AVX2.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+// http://www.apache.org/licenses/LICENSE-2.0
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+///////////////////////////////////////////////////////////////////////
+
+#if !defined(__AVX2__)
+# if defined(__i686__) || defined(__x86_64__)
+# error Implementation only for AVX2 capable architectures
+# endif
+#else
+
+# include <immintrin.h>
+# include <cstdint>
+# include "dotproduct.h"
+# include "functions.h" // for Tanh, Logistic, TanhTable, LogisticTable
+
+namespace tesseract {
+
+# if defined(FAST_FLOAT)
+
+// The functions give exactly the same results as the scalar functions Tanh,
+// Logistic and IntCastRounded, because they do the same calculations in the
+// same order. This file must not be built with FMA, which would change that.
+
+// Returns the interpolated values of the table (TanhTable or LogisticTable)
+// for the 8 values of a, which must not be negative, like the scalar
+// functions do for x >= 0.
+static inline __m256 Lookup(const float *table, __m256 a) {
+ // Values which are too large for the table (the result is 1 for them) are
+ // limited first, so the indices are always valid and nothing overflows,
+ // also for huge values and for NaN (_mm256_min_ps returns the second value
+ // then).
+ const __m256 limit = _mm256_set1_ps((kTableSize - 1) / kScaleFactor);
+ __m256 scaled =
+ _mm256_mul_ps(_mm256_min_ps(a, limit), _mm256_set1_ps(kScaleFactor));
+ __m256i index = _mm256_min_epi32(_mm256_cvttps_epi32(scaled),
+ _mm256_set1_epi32(kTableSize - 2));
+ // Each load gets table[i] and table[i + 1]. This is faster than the gather
+ // instructions (much faster on CPUs with a microcode mitigation for them).
+ alignas(32) int32_t i[8];
+ _mm256_store_si256(reinterpret_cast<__m256i *>(i), index);
+ auto pair = [table](int32_t k) {
+ return _mm_castsi128_ps(
+ _mm_loadl_epi64(reinterpret_cast<const __m128i *>(table + k)));
+ };
+ __m128 p01 = _mm_unpacklo_ps(pair(i[0]), pair(i[1]));
+ __m128 p23 = _mm_unpacklo_ps(pair(i[2]), pair(i[3]));
+ __m128 p45 = _mm_unpacklo_ps(pair(i[4]), pair(i[5]));
+ __m128 p67 = _mm_unpacklo_ps(pair(i[6]), pair(i[7]));
+ __m256 lo = _mm256_set_m128(p45, p01);
+ __m256 hi = _mm256_set_m128(p67, p23);
+ __m256 t0 = _mm256_shuffle_ps(lo, hi, 0x44); // table[i]
+ __m256 t1 = _mm256_shuffle_ps(lo, hi, 0xee); // table[i + 1]
+ // Linear interpolation.
+ __m256 fraction = _mm256_sub_ps(scaled, _mm256_cvtepi32_ps(index));
+ __m256 result =
+ _mm256_add_ps(t0, _mm256_mul_ps(_mm256_sub_ps(t1, t0), fraction));
+ // The result is 1 for values which are too large for the table.
+ __m256 too_large = _mm256_cmp_ps(a, limit, _CMP_GE_OQ);
+ return _mm256_blendv_ps(result, _mm256_set1_ps(1.0f), too_large);
+}
+
+// Computes out[i] = Tanh(in[i]) for n values. in and out can be the same.
+void TanhVectorAVX2(int n, const float *in, float *out) {
+ const __m256 abs_mask = _mm256_castsi256_ps(_mm256_set1_epi32(0x7fffffff));
+ const __m256 zero = _mm256_setzero_ps();
+ int i = 0;
+ for (; i + 8 <= n; i += 8) {
+ __m256 x = _mm256_loadu_ps(in + i);
+ __m256 result = Lookup(TanhTable, _mm256_and_ps(x, abs_mask));
+ // Tanh(x) = -Tanh(-x) for x < 0.
+ __m256 negative = _mm256_cmp_ps(x, zero, _CMP_LT_OQ);
+ result = _mm256_blendv_ps(result, _mm256_sub_ps(zero, result), negative);
+ _mm256_storeu_ps(out + i, result);
+ }
+ for (; i < n; i++) {
+ out[i] = Tanh(in[i]);
+ }
+}
+
+// Computes out[i] = Logistic(in[i]) for n values. in and out can be the same.
+void LogisticVectorAVX2(int n, const float *in, float *out) {
+ const __m256 abs_mask = _mm256_castsi256_ps(_mm256_set1_epi32(0x7fffffff));
+ const __m256 one = _mm256_set1_ps(1.0f);
+ int i = 0;
+ for (; i + 8 <= n; i += 8) {
+ __m256 x = _mm256_loadu_ps(in + i);
+ __m256 result = Lookup(LogisticTable, _mm256_and_ps(x, abs_mask));
+ // Logistic(x) = 1 - Logistic(-x) for x < 0.
+ __m256 negative = _mm256_cmp_ps(x, _mm256_setzero_ps(), _CMP_LT_OQ);
+ result = _mm256_blendv_ps(result, _mm256_sub_ps(one, result), negative);
+ _mm256_storeu_ps(out + i, result);
+ }
+ for (; i < n; i++) {
+ out[i] = Logistic(in[i]);
+ }
+}
+
+// Computes out[i] = IntCastRounded(ClipToRange(in[i] * INT8_MAX, -INT8_MAX,
+// INT8_MAX)) for n values.
+void FloatToInt8VectorAVX2(int n, const float *in, int8_t *out) {
+ const __m256 scale = _mm256_set1_ps(INT8_MAX);
+ const __m256 lower = _mm256_set1_ps(-INT8_MAX);
+ const __m256 sign_mask = _mm256_castsi256_ps(_mm256_set1_epi32(INT32_MIN));
+ const __m256 half = _mm256_set1_ps(0.5f);
+ int i = 0;
+ for (; i + 8 <= n; i += 8) {
+ __m256 x = _mm256_mul_ps(_mm256_loadu_ps(in + i), scale);
+ // NaN gives INT8_MAX.
+ x = _mm256_max_ps(_mm256_min_ps(x, scale), lower);
+ // Round half away from zero: add 0.5 with the sign of x and truncate.
+ x = _mm256_add_ps(x, _mm256_or_ps(half, _mm256_and_ps(x, sign_mask)));
+ __m256i value = _mm256_cvttps_epi32(x);
+ __m128i value16 = _mm_packs_epi32(_mm256_castsi256_si128(value),
+ _mm256_extracti128_si256(value, 1));
+ _mm_storel_epi64(reinterpret_cast<__m128i *>(out + i),
+ _mm_packs_epi16(value16, value16));
+ }
+ for (; i < n; i++) {
+ out[i] = IntCastRounded(
+ ClipToRange<float>(in[i] * INT8_MAX, -INT8_MAX, INT8_MAX));
+ }
+}
+
+# endif // FAST_FLOAT
+
+} // namespace tesseract.
+
+#endif
diff --git a/src/arch/simddetect.cpp b/src/arch/simddetect.cpp
index 5b1f6b4e..ad94010e 100644
--- a/src/arch/simddetect.cpp
+++ b/src/arch/simddetect.cpp
@@ -21,6 +21,7 @@
#include <cstdlib> // for getenv
#include <numeric> // for std::inner_product
#include "dotproduct.h"
+#include "functions.h" // for Tanh, Logistic
#include "intsimdmatrix.h" // for IntSimdMatrix
#include "params.h" // for STRING_VAR
#include "simddetect.h"
@@ -104,6 +105,13 @@ DotProductFunction DotProduct;
RowDotProductsFunction RowDotProducts;
CrossDotProductsFunction CrossDotProducts;
+// The functions for whole vectors give the same results with all
+// implementations, so the fastest one for the CPU is used unless the generic
+// dot product is selected.
+VectorFunction TanhVector;
+VectorFunction LogisticVector;
+FloatToInt8Function FloatToInt8Vector;
+
static STRING_VAR(dotproduct, "auto", "Function used for calculation of dot product");
const SIMDDetect &SIMDDetect::GetDetector() {
@@ -203,7 +211,40 @@ static void CrossDotProductsGeneric(const TFloat *u, int num_u, const TFloat *v,
}
}
+// Computes out[i] = Tanh(in[i]) for n values.
+static void TanhVectorGeneric(int n, const TFloat *in, TFloat *out) {
+ for (int i = 0; i < n; ++i) {
+ out[i] = Tanh(in[i]);
+ }
+}
+
+// Computes out[i] = Logistic(in[i]) for n values.
+static void LogisticVectorGeneric(int n, const TFloat *in, TFloat *out) {
+ for (int i = 0; i < n; ++i) {
+ out[i] = Logistic(in[i]);
+ }
+}
+
+// Converts n floats in the range [-1, 1] to int8_t.
+static void FloatToInt8VectorGeneric(int n, const TFloat *in, int8_t *out) {
+ for (int i = 0; i < n; ++i) {
+ out[i] = IntCastRounded(
+ ClipToRange<TFloat>(in[i] * INT8_MAX, -INT8_MAX, INT8_MAX));
+ }
+}
+
static void SetDotProduct(DotProductFunction f, const IntSimdMatrix *m = nullptr) {
+ TanhVector = TanhVectorGeneric;
+ LogisticVector = LogisticVectorGeneric;
+ FloatToInt8Vector = FloatToInt8VectorGeneric;
+#if defined(FAST_FLOAT) && defined(HAVE_AVX2)
+ // The integer matrix for AVX2 is only selected for a CPU with AVX2.
+ if (m == &IntSimdMatrix::intSimdMatrixAVX2) {
+ TanhVector = TanhVectorAVX2;
+ LogisticVector = LogisticVectorAVX2;
+ FloatToInt8Vector = FloatToInt8VectorAVX2;
+ }
+#endif
RowDotProductsFunction rows = RowDotProductsGeneric;
CrossDotProductsFunction cross = CrossDotProductsGeneric;
#if defined(FAST_FLOAT) && defined(__ARM_ARCH_ISA_A64)
diff --git a/src/arch/simddetect.h b/src/arch/simddetect.h
index 1ed3cdcc..9a032cc4 100644
--- a/src/arch/simddetect.h
+++ b/src/arch/simddetect.h
@@ -44,6 +44,21 @@ using CrossDotProductsFunction = void (*)(const TFloat *u, int num_u,
TFloat *out, int out_stride);
extern CrossDotProductsFunction CrossDotProducts;
+// Function pointers for the non-linearity functions for whole vectors:
+// out[i] = Tanh(in[i]) or out[i] = Logistic(in[i]) (see functions.h) for n
+// values. in and out can be the same. All implementations give exactly the
+// same results.
+using VectorFunction = void (*)(int n, const TFloat *in, TFloat *out);
+extern TESS_API VectorFunction TanhVector;
+extern TESS_API VectorFunction LogisticVector;
+
+// Function pointer for the conversion of n floats in the range [-1, 1] to
+// int8_t: out[i] = IntCastRounded(ClipToRange(in[i] * INT8_MAX, -INT8_MAX,
+// INT8_MAX)). All implementations give the same results (unless a value
+// is NaN, which gives an undefined result).
+using FloatToInt8Function = void (*)(int n, const TFloat *in, int8_t *out);
+extern TESS_API FloatToInt8Function FloatToInt8Vector;
+
// Architecture detector. Add code here to detect any other architectures for
// SIMD-based faster dot product functions. Intended to be a single static
// object, but it does no real harm to have more than one.
diff --git a/src/lstm/functions.h b/src/lstm/functions.h
index f3cd05de..eeca10fd 100644
--- a/src/lstm/functions.h
+++ b/src/lstm/functions.h
@@ -19,6 +19,7 @@
#define TESSERACT_LSTM_FUNCTIONS_H_
#include "helpers.h"
+#include "simddetect.h" // for TanhVector, LogisticVector
#include "tesstypes.h"
// Setting this to 1 or more causes massive dumps of debug data: weights,
@@ -171,6 +172,20 @@ inline void FuncInplace(int n, TFloat *inout) {
inout[i] = f(inout[i]);
}
}
+// The functions with table lookups use the fastest implementation for whole
+// vectors (see simddetect.h), which gives the same results.
+template <>
+inline void FuncInplace<FFunc>(int n, TFloat *inout) {
+ LogisticVector(n, inout, inout);
+}
+template <>
+inline void FuncInplace<GFunc>(int n, TFloat *inout) {
+ TanhVector(n, inout, inout);
+}
+template <>
+inline void FuncInplace<HFunc>(int n, TFloat *inout) {
+ TanhVector(n, inout, inout);
+}
// Applies Func to u and multiplies the result by v component-wise,
// putting the product in out, all of size n.
template <class Func>
@@ -180,6 +195,20 @@ inline void FuncMultiply(const TFloat *u, const TFloat *v, int n, TFloat *out) {
out[i] = f(u[i]) * v[i];
}
}
+template <>
+inline void FuncMultiply<HFunc>(const TFloat *u, const TFloat *v, int n, TFloat *out) {
+ if (out == v) {
+ // The factors would be overwritten by the function values.
+ for (int i = 0; i < n; ++i) {
+ out[i] = Tanh(u[i]) * v[i];
+ }
+ return;
+ }
+ TanhVector(n, u, out);
+ for (int i = 0; i < n; ++i) {
+ out[i] *= v[i];
+ }
+}
// Applies the Softmax function in-place to inout, of size n.
template <typename T>
inline void SoftmaxInPlace(int n, T *inout) {
diff --git a/src/lstm/networkio.cpp b/src/lstm/networkio.cpp
index e72bb731..602865e5 100644
--- a/src/lstm/networkio.cpp
+++ b/src/lstm/networkio.cpp
@@ -665,12 +665,9 @@ void NetworkIO::WriteTimeStep(int t, const TFloat *input) {
void NetworkIO::WriteTimeStepPart(int t, int offset, int num_features, const TFloat *input) {
ASSERT_HOST(offset + num_features <= NumFeatures());
if (int_mode_) {
- int8_t *line = i_[t] + offset;
- for (int i = 0; i < num_features; ++i) {
- // Clip before the conversion to int: the output of a linear layer of an
- // untrusted model is not bounded to [-1, 1].
- line[i] = IntCastRounded(ClipToRange<TFloat>(input[i] * INT8_MAX, -INT8_MAX, INT8_MAX));
- }
+ // The values are clipped before the conversion to int: the output of a
+ // linear layer of an untrusted model is not bounded to [-1, 1].
+ FloatToInt8Vector(num_features, input, i_[t] + offset);
} else {
float *line = f_[t] + offset;
for (int i = 0; i < num_features; ++i) {
diff --git a/unittest/functions_test.cc b/unittest/functions_test.cc
index 2a4db168..65ce19de 100644
--- a/unittest/functions_test.cc
+++ b/unittest/functions_test.cc
@@ -15,8 +15,11 @@
#include "functions.h"
+#include "helpers.h" // for TRand
+
#include <cfenv>
#include <cmath>
+#include <vector>
namespace tesseract {
@@ -61,4 +64,65 @@ TEST(FunctionsTest, InterpolatesInRange) {
}
}
+// The functions for whole vectors (TanhVector, LogisticVector and
+// FloatToInt8Vector, which use SIMD code if the CPU supports it) must give
+// exactly the same results as the scalar functions for all lengths, also for
+// huge values and at the limits of the table.
+TEST(FunctionsTest, VectorFunctionsMatchScalar) {
+ std::vector<TFloat> values = {0.0f, -0.0f, 1e-30f, -1e-30f, 0.5f / 256,
+ 1.0f, -1.0f, 15.99f, -15.99f, 4095.0f / 256,
+ -4095.0f / 256, 16.0f, -16.0f, 1e30f,
+ -1e30f, 3e38f, -3e38f};
+ // Values around the entries of the table and the limits of int8_t.
+ for (int i = -4100; i <= 4100; i += 7) {
+ values.push_back(i / 256.0f);
+ values.push_back(std::nextafter(i / 256.0f, 100.0f));
+ values.push_back(std::nextafter(i / 256.0f, -100.0f));
+ }
+ for (int i = -260; i <= 260; ++i) {
+ values.push_back(i / 254.0f);
+ values.push_back(std::nextafter(i / 254.0f, 100.0f));
+ values.push_back(std::nextafter(i / 254.0f, -100.0f));
+ }
+ TRand random;
+ random.set_seed(1);
+ for (int i = 0; i < 4000; ++i) {
+ values.push_back(static_cast<TFloat>(random.SignedRand(20.0)));
+ }
+ std::feclearexcept(FE_ALL_EXCEPT);
+ const size_t kLengths[] = {values.size(), 1, 2, 7, 8, 9, 15, 16, 17, 31, 48};
+ for (size_t n : kLengths) {
+ // The results are written to the middle of a larger vector to detect
+ // writes outside of the n values.
+ const TFloat kGuard = 12345;
+ std::vector<TFloat> tanh_result(n + 2, kGuard);
+ std::vector<TFloat> logistic_result(n + 2, kGuard);
+ std::vector<int8_t> int_result(n + 2, 99);
+ TanhVector(n, values.data(), tanh_result.data() + 1);
+ LogisticVector(n, values.data(), logistic_result.data() + 1);
+ FloatToInt8Vector(n, values.data(), int_result.data() + 1);
+ for (size_t i = 0; i < n; ++i) {
+ TFloat x = values[i];
+ EXPECT_EQ(Tanh(x), tanh_result[i + 1]) << "x=" << x << ", n=" << n;
+ EXPECT_EQ(Logistic(x), logistic_result[i + 1]) << "x=" << x << ", n=" << n;
+ int expected =
+ IntCastRounded(ClipToRange<TFloat>(x * INT8_MAX, -INT8_MAX, INT8_MAX));
+ EXPECT_EQ(expected, int_result[i + 1]) << "x=" << x << ", n=" << n;
+ }
+ EXPECT_EQ(kGuard, tanh_result[0]);
+ EXPECT_EQ(kGuard, tanh_result[n + 1]);
+ EXPECT_EQ(kGuard, logistic_result[0]);
+ EXPECT_EQ(kGuard, logistic_result[n + 1]);
+ EXPECT_EQ(99, int_result[0]);
+ EXPECT_EQ(99, int_result[n + 1]);
+ // The functions can also be used in place.
+ std::vector<TFloat> inout(values.begin(), values.begin() + n);
+ TanhVector(n, inout.data(), inout.data());
+ for (size_t i = 0; i < n; ++i) {
+ EXPECT_EQ(tanh_result[i + 1], inout[i]) << "in place, i=" << i;
+ }
+ }
+ EXPECT_EQ(0, std::fetestexcept(FE_INVALID));
+}
+
} // namespace tesseract