Commit 6e5ad3d4 for tesseract
commit 6e5ad3d47e9a694ea6250205747ee30b6804b350
Author: Stefan Weil <sw@weilnetz.de>
Date: Fri Nov 6 20:14:45 2026 +0100
wordrec: add a per-page blob classification budget to bound classifier timeouts
The per-pass seam search budget (chop_max_seam_evals_pass) bounds the
seam search, but it does not bound the character classifier: a single
word can legitimately contain many blobs, and the cost of classifying
them grows with the number of words on the page. On inputs where
pageseg produces a large number of blob-like pieces with no inter-word
gaps (e.g. repeated vertical stripes) the chopper classifies tens of
thousands of blobs. OSS-Fuzz issue 581720522 (fuzzer-api-512x256,
Timeout) is such an input: ~75 words of up to 132 blobs each,
recursively split into 743 leaf recognitions, so the chopper
classifies 30129 blobs (179494 calls into IntegerMatcher::Match, 13.2M
calls into UpdateTablesForFeature), which exceeds the bot's 60 s
timeout.
Add a budget of blob classifications per page and language
(chop_max_classifications_pass). Tesseract::ResetPassBudgets() sets it
at the start of the page for the primary language and for all
sub-languages, together with the seam evaluation budget, and
Wordrec::call_matcher() counts it down. The new function is called
from recog_all_words(), ApplyBoxes() and recog_training_segmented(),
so the ambigs training path gets fresh budgets for each page, too.
The budget is enforced between words, not for single blobs: a word
which was started is always finished with real classifications. When
the budget of a language is used up, that language does not get any
further word (classify_word_and_language(), and the legacy fallback
of the combined mode in classify_word_pass1()). Other languages which
still have budget are tried as usual. If no language is left, a word
without a result gets a fake result as for a timeout, and a word with
a result from an earlier pass keeps it, so a budget which runs out in
the second pass does not destroy results of the first pass. The
diacritics reassignment and the fuzzy space search stop when no
language can classify any more. A warning is printed once per page
and language when the budget is used up.
The default is effectively unlimited (INT32_MAX), because a fixed
number cannot tell a degenerate image from a normal page: the fuzzer
input needs about three classifications per blob, which is the same
order as normal text. Synthetic 300 dpi pages of 460 to 940 words
needed 4000 to 8300 classifications with --oem 0, so a budget of 5000
would truncate the text of normal pages. The fuzzer therefore sets
the budget of 5000 itself (unittest/fuzzers/fuzzer-api.cpp).
Each language gets its own budget, so the worst case for a
multilingual page is the number of languages times the budget; a
value changed with SetVariable() after Init() applies to the primary
language only. The parameter is read at the start of the page, so
changing it between SetImage() and Recognize() takes effect. Negative
values are clamped to 0, which disables the recognition by the legacy
engine. The counter is atomic because PrerecAllWordsPar() classifies
blobs from several threads.
The help text of the parameter is short because it is shown by
`tesseract --print-parameters`; the full description is in
doc/tesseract.1.asc. A unit test (ClassificationBudgetTest) covers
the default, a budget of 0, a small budget, and a second language
taking over.
Assisted-by: OpenCode / qwen3.8-27b-thinking (Alibaba Cloud)
Assisted-by: Claude Fable 5.1 (Anthropic)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
diff --git a/doc/tesseract.1.asc b/doc/tesseract.1.asc
index dd35bba5..066ac078 100644
--- a/doc/tesseract.1.asc
+++ b/doc/tesseract.1.asc
@@ -704,6 +704,18 @@ model such as those from https://github.com/tesseract-ocr/tessdata).
when forming word hypotheses. Disabling may speed up recognition at the
cost of accuracy on fragmented characters.
+*chop_max_classifications_pass* (int, default: 2147483647) [Legacy]::
+ Maximum number of blob classifications which the legacy engine may use for
+ one page (all recognition passes). When this budget is used up, the legacy
+ engine finishes the current word, but does not recognize any further word of
+ the page: such words get no text in the first pass and keep their result in
+ later passes. This bounds the run time for degenerate images with a huge
+ number of blobs. A value which is too small truncates the text of normal
+ pages (a page of text needs several thousand classifications), so the
+ default is effectively unlimited. A value of 0 disables the recognition by
+ the legacy engine. Each language of a multi-language run (e.g. `-l eng+deu`)
+ has its own budget. The value is read at the start of each page.
+
DEBUG PARAMETERS
~~~~~~~~~~~~~~~~
diff --git a/src/ccmain/applybox.cpp b/src/ccmain/applybox.cpp
index 8c591840..c05a3d61 100644
--- a/src/ccmain/applybox.cpp
+++ b/src/ccmain/applybox.cpp
@@ -115,14 +115,10 @@ PAGE_RES *Tesseract::ApplyBoxes(const char *filename, bool find_segmentation,
return nullptr; // Can't do it.
}
- // Start a fresh per-pass seam-evaluation budget for the resegmentation
+ // Start fresh budgets of the legacy engine for the resegmentation, as
+ // recog_all_words() does. Only the seam-evaluation budget is enforced here
// (see choose_best_seam() in findseam.cpp, OSS-Fuzz issue 4775901406232576).
- // The sub-language Tesseract objects get a budget too, for consistency
- // with recog_all_words() (their chopping paths are the same).
- ResetSeamEvalsBudget();
- for (auto *sub_lang : sub_langs_) {
- sub_lang->ResetSeamEvalsBudget();
- }
+ ResetPassBudgets();
const int box_count = boxes.size();
int box_failures = 0;
diff --git a/src/ccmain/control.cpp b/src/ccmain/control.cpp
index bd50ead5..c02de8dc 100644
--- a/src/ccmain/control.cpp
+++ b/src/ccmain/control.cpp
@@ -21,11 +21,14 @@
# include "config_auto.h"
#endif
+#include <algorithm> // for std::all_of
#include <cctype>
#include <cmath>
#include <cstdint> // for int16_t, int32_t
#include <cstdio> // for fclose, fopen, FILE
#include <ctime> // for clock
+#include <utility> // for std::pair
+#include <vector> // for std::vector
#include "control.h"
#ifndef DISABLED_LEGACY_ENGINE
# include "docqual.h"
@@ -267,6 +270,37 @@ bool Tesseract::RecogAllWordsPassN(int pass_n, ETEXT_DESC *monitor, PAGE_RES_IT
return true;
}
+#ifndef DISABLED_LEGACY_ENGINE
+// Starts fresh per-page budgets for the legacy engine: the seam-evaluation
+// budget (see choose_best_seam() in findseam.cpp, OSS-Fuzz issue
+// 4775901406232576) and the blob-classification budget (see
+// Wordrec::ClassificationBudgetExhausted(), OSS-Fuzz issue 581720522).
+// Reading the parameters here (rather than at SetImage()) means that a change
+// of chop_max_seam_evals_pass or chop_max_classifications_pass between
+// SetImage() and Recognize() takes effect.
+// The sub-language Tesseract objects must get budgets too: words are
+// recognized by whichever language is tried (RetryWithLanguage()), so a
+// secondary language running with an unset budget would lose chopping
+// silently.
+void Tesseract::ResetPassBudgets() {
+ ResetSeamEvalsBudget();
+ ResetClassificationBudget();
+ for (auto *sub_lang : sub_langs_) {
+ sub_lang->ResetSeamEvalsBudget();
+ sub_lang->ResetClassificationBudget();
+ }
+}
+
+// Returns true if the blob-classification budgets of this language and of all
+// sub-languages are used up, so that the legacy engine cannot recognize any
+// further word of this page.
+bool Tesseract::AllClassificationBudgetsExhausted() {
+ return ClassificationBudgetExhausted() &&
+ std::all_of(sub_langs_.begin(), sub_langs_.end(),
+ [](Tesseract *lang) { return lang->ClassificationBudgetExhausted(); });
+}
+#endif // ndef DISABLED_LEGACY_ENGINE
+
/**
* recog_all_words()
*
@@ -294,18 +328,8 @@ bool Tesseract::recog_all_words(PAGE_RES *page_res, ETEXT_DESC *monitor,
PAGE_RES_IT page_res_it(page_res);
#ifndef DISABLED_LEGACY_ENGINE
- // Start a fresh per-pass seam-evaluation budget for the recognition pass
- // (see choose_best_seam() in findseam.cpp, OSS-Fuzz issue 4775901406232576).
- // Reading the parameter here (rather than at SetImage()) means a change of
- // chop_max_seam_evals_pass between SetImage() and Recognize() takes effect.
- // The sub-language Tesseract objects must get a budget too: words are
- // chopped by whichever language recognizes them (RetryWithLanguage()), so
- // a secondary language running with an unset budget would lose chopping
- // silently.
- ResetSeamEvalsBudget();
- for (auto *sub_lang : sub_langs_) {
- sub_lang->ResetSeamEvalsBudget();
- }
+ // Start fresh budgets of the legacy engine for this page.
+ ResetPassBudgets();
#endif // ndef DISABLED_LEGACY_ENGINE
if (tessedit_minimal_rej_pass1) {
@@ -932,6 +956,10 @@ static bool WordsAcceptable(const PointerVector<WERD_RES> &words) {
// sets make_next_word_fuzzy if blob(s) were added to the end of the word.
bool Tesseract::ReassignDiacritics(int pass, PAGE_RES_IT *pr_it, bool *make_next_word_fuzzy) {
*make_next_word_fuzzy = false;
+ if (AllClassificationBudgetsExhausted()) {
+ // No language can classify the noise blobs, so leave them where they are.
+ return false;
+ }
WERD *real_word = pr_it->word()->word;
if (real_word->rej_cblob_list()->empty() || real_word->cblob_list()->empty() ||
real_word->rej_cblob_list()->length() > noise_maxperword) {
@@ -1310,6 +1338,20 @@ float Tesseract::ClassifyBlobAsWord(int pass_n, PAGE_RES_IT *pr_it, C_BLOB *blob
#endif // ndef DISABLED_LEGACY_ENGINE
+#ifndef DISABLED_LEGACY_ENGINE
+// Returns true if lang certainly needs the legacy engine to recognize a word
+// in pass pass_n, but has used up its blob-classification budget for this
+// page (see Wordrec::ClassificationBudgetExhausted()), so that it must not be
+// given another word. A language which only falls back to the legacy engine
+// for some words (OEM_TESSERACT_LSTM_COMBINED in pass 1) is handled in
+// classify_word_pass1().
+static bool LegacyBudgetExhausted(Tesseract *lang, int pass_n) {
+ const bool needs_legacy = pass_n == 1 ? lang->tessedit_ocr_engine_mode == OEM_TESSERACT_ONLY
+ : lang->tessedit_ocr_engine_mode != OEM_LSTM_ONLY;
+ return needs_legacy && lang->ClassificationBudgetExhausted();
+}
+#endif // ndef DISABLED_LEGACY_ENGINE
+
// Generic function for classifying a word. Can be used either for pass1 or
// pass2 according to the function passed to recognizer.
// word_data holds the word to be recognized, and its block and row, and
@@ -1318,6 +1360,9 @@ float Tesseract::ClassifyBlobAsWord(int pass_n, PAGE_RES_IT *pr_it, C_BLOB *blob
// Recognizes in the current language, and if successful that is all.
// If recognition was not successful, tries all available languages until
// it gets a successful result or runs out of languages. Keeps the best result.
+// Languages which have used up their classification budget are not tried. If
+// that leaves no language, a word without a result gets a fake result (as
+// for a timeout), and a word with a result from an earlier pass keeps it.
void Tesseract::classify_word_and_language(int pass_n, PAGE_RES_IT *pr_it, WordData *word_data) {
#ifdef DISABLED_LEGACY_ENGINE
WordRecognizer recognizer = &Tesseract::classify_word_pass1;
@@ -1354,22 +1399,47 @@ void Tesseract::classify_word_and_language(int pass_n, PAGE_RES_IT *pr_it, WordD
for (sub = 0; sub < sub_langs_.size() && most_recently_used_ != sub_langs_[sub]; ++sub) {
}
}
- most_recently_used_->RetryWithLanguage(*word_data, recognizer, debug, &word_data->lang_words[sub],
- &best_words);
+ // The languages to try, each with its index in lang_words: the most
+ // recently used language first, then all the other languages.
+ std::vector<std::pair<Tesseract *, unsigned>> langs;
+ langs.emplace_back(most_recently_used_, sub);
+ if (most_recently_used_ != this) {
+ langs.emplace_back(this, sub_langs_.size());
+ }
+ for (unsigned i = 0; i < sub_langs_.size(); ++i) {
+ if (most_recently_used_ != sub_langs_[i]) {
+ langs.emplace_back(sub_langs_[i], i);
+ }
+ }
+#ifndef DISABLED_LEGACY_ENGINE
+ // A language which has used up its classification budget is not tried.
+ langs.erase(std::remove_if(langs.begin(), langs.end(),
+ [pass_n](const std::pair<Tesseract *, unsigned> &lang) {
+ return LegacyBudgetExhausted(lang.first, pass_n);
+ }),
+ langs.end());
+ if (langs.empty()) {
+ if (word_data->word->best_choice == nullptr) {
+ // No language has budget left to recognize the word, and there is no
+ // result from an earlier pass, so fake it as for a timeout. Later
+ // passes expect that the word knows its language.
+ word_data->word->SetupFake(unicharset);
+ word_data->word->tesseract = this;
+ }
+ return;
+ }
+#endif // ndef DISABLED_LEGACY_ENGINE
Tesseract *best_lang_tess = most_recently_used_;
- if (!WordsAcceptable(best_words)) {
- // Try all the other languages to see if they are any better.
- if (most_recently_used_ != this &&
- this->RetryWithLanguage(*word_data, recognizer, debug,
- &word_data->lang_words[sub_langs_.size()], &best_words) > 0) {
- best_lang_tess = this;
- }
- for (unsigned i = 0; !WordsAcceptable(best_words) && i < sub_langs_.size(); ++i) {
- if (most_recently_used_ != sub_langs_[i] &&
- sub_langs_[i]->RetryWithLanguage(*word_data, recognizer, debug, &word_data->lang_words[i],
- &best_words) > 0) {
- best_lang_tess = sub_langs_[i];
- }
+ for (unsigned i = 0; i < langs.size(); ++i) {
+ // The first language is always tried. The other languages are only tried
+ // to see if they are any better as long as the result is not acceptable.
+ if (i > 0 && WordsAcceptable(best_words)) {
+ break;
+ }
+ if (langs[i].first->RetryWithLanguage(*word_data, recognizer, debug,
+ &word_data->lang_words[langs[i].second],
+ &best_words) > 0) {
+ best_lang_tess = langs[i].first;
}
}
most_recently_used_ = best_lang_tess;
@@ -1424,6 +1494,11 @@ void Tesseract::classify_word_pass1(const WordData &word_data, WERD_RES **in_wor
}
#ifndef DISABLED_LEGACY_ENGINE
+ if (ClassificationBudgetExhausted()) {
+ // No budget left for the fallback, so use a fake.
+ (*in_word)->SetupFake(lstm_recognizer_->GetUnicharset());
+ return;
+ }
// Fall back to tesseract for failed words or odd words.
(*in_word)->SetupForRecognition(unicharset, this, BestPix(), OEM_TESSERACT_ONLY, nullptr,
classify_bln_numeric_mode, textord_use_cjk_fp_model,
diff --git a/src/ccmain/fixspace.cpp b/src/ccmain/fixspace.cpp
index dd9e3663..941d86c7 100644
--- a/src/ccmain/fixspace.cpp
+++ b/src/ccmain/fixspace.cpp
@@ -179,6 +179,11 @@ void Tesseract::fix_fuzzy_space_list(WERD_RES_LIST &best_perm, ROW *row, BLOCK *
while ((best_score != PERFECT_WERDS) && !current_perm.empty()) {
match_current_words(current_perm, row, block);
+ if (AllClassificationBudgetsExhausted()) {
+ // The legacy engine might not have recognized all words of
+ // current_perm, so it cannot be compared with best_perm.
+ break;
+ }
int16_t current_score = eval_word_spacing(current_perm);
dump_words(current_perm, current_score, 2, improved);
if (current_score > best_score) {
@@ -590,6 +595,11 @@ void Tesseract::fix_noisy_space_list(WERD_RES_LIST &best_perm, ROW *row, BLOCK *
while (best_score != PERFECT_WERDS && !current_perm.empty()) {
match_current_words(current_perm, row, block);
+ if (AllClassificationBudgetsExhausted()) {
+ // The legacy engine might not have recognized all words of
+ // current_perm, so it cannot be compared with best_perm.
+ break;
+ }
current_score = fp_eval_word_spacing(current_perm);
dump_words(current_perm, current_score, 2, improved);
if (current_score > best_score) {
diff --git a/src/ccmain/recogtraining.cpp b/src/ccmain/recogtraining.cpp
index eeb667b6..148f8820 100644
--- a/src/ccmain/recogtraining.cpp
+++ b/src/ccmain/recogtraining.cpp
@@ -85,6 +85,10 @@ static bool read_t(PAGE_RES_IT *page_res_it, TBOX *tbox) {
// the correct label, rating and certainty.
void Tesseract::recog_training_segmented(const char *filename, PAGE_RES *page_res,
FILE *output_file) {
+ // Start fresh budgets of the legacy engine for this page, as
+ // recog_all_words() does.
+ ResetPassBudgets();
+
std::string box_fname = filename;
const char *lastdot = strrchr(box_fname.c_str(), '.');
if (lastdot != nullptr) {
diff --git a/src/ccmain/tesseractclass.h b/src/ccmain/tesseractclass.h
index 9a8875c3..f026bf94 100644
--- a/src/ccmain/tesseractclass.h
+++ b/src/ccmain/tesseractclass.h
@@ -376,6 +376,18 @@ public:
// Runs word recognition on all the words.
bool RecogAllWordsPassN(int pass_n, ETEXT_DESC *monitor, PAGE_RES_IT *pr_it,
std::vector<WordData> *words);
+#ifndef DISABLED_LEGACY_ENGINE
+ // Starts fresh per-page budgets for the legacy engine (seam evaluations
+ // and blob classifications, see Wordrec::ResetSeamEvalsBudget() and
+ // Wordrec::ResetClassificationBudget()) for this language and for all
+ // sub-languages. Must be called at the start of each page.
+ void ResetPassBudgets();
+ // Returns true if the blob-classification budgets of this language and of
+ // all sub-languages are used up (see
+ // Wordrec::ClassificationBudgetExhausted()), so that the legacy engine
+ // cannot recognize any further word of this page.
+ bool AllClassificationBudgetsExhausted();
+#endif // ndef DISABLED_LEGACY_ENGINE
bool recog_all_words(PAGE_RES *page_res, ETEXT_DESC *monitor, const TBOX *target_word_box,
const char *word_config, int dopasses);
void rejection_passes(PAGE_RES *page_res, ETEXT_DESC *monitor, const TBOX *target_word_box,
diff --git a/src/wordrec/tface.cpp b/src/wordrec/tface.cpp
index 146f0b01..6a23e82f 100644
--- a/src/wordrec/tface.cpp
+++ b/src/wordrec/tface.cpp
@@ -133,6 +133,16 @@ void Wordrec::cc_recog(WERD_RES *word) {
* The blob may need rotating to the correct orientation for classification.
*/
BLOB_CHOICE_LIST *Wordrec::call_matcher(TBLOB *tessblob) {
+ // Count the classification against the per-page budget (see
+ // classifications_pass_remaining_ in wordrec.h). The budget is not enforced
+ // here, but between words (ClassificationBudgetExhausted()), so every blob
+ // of a word which was started gets a real classification.
+ // This function can be called from several threads (PrerecAllWordsPar()).
+ // The check and the decrement are not one atomic operation, so the counter
+ // can get slightly negative, which is still "exhausted".
+ if (classifications_pass_remaining_.load(std::memory_order_relaxed) > 0) {
+ classifications_pass_remaining_.fetch_sub(1, std::memory_order_relaxed);
+ }
// Rotate the blob for classification if necessary.
TBLOB *rotated_blob = tessblob->ClassifyNormalizeIfNeeded();
if (rotated_blob == nullptr) {
diff --git a/src/wordrec/wordrec.cpp b/src/wordrec/wordrec.cpp
index d40cb1ae..18e26df5 100644
--- a/src/wordrec/wordrec.cpp
+++ b/src/wordrec/wordrec.cpp
@@ -18,8 +18,14 @@
#include "wordrec.h"
+#include <algorithm> // for std::max
+#include <cstdint> // for INT32_MAX
#include <memory>
+#ifndef DISABLED_LEGACY_ENGINE
+# include "tesserrstream.h" // for tesserr
+#endif
+
#ifdef DISABLED_LEGACY_ENGINE
# include "params.h"
@@ -95,6 +101,10 @@ Wordrec::Wordrec()
"effect. A value changed with SetVariable() after Init() "
"applies to the primary language only. 0 disables chopping.",
params())
+ , INT_MEMBER(chop_max_classifications_pass, INT32_MAX,
+ "Max number of blob classifications of the legacy engine "
+ "per page and language. Further words are not recognized.",
+ params())
, INT_MEMBER(chop_x_y_weight, 3, "X / Y length weight", params())
, BOOL_MEMBER(assume_fixed_pitch_char_segment, false,
"include fixed-pitch heuristics in char segmentation", params())
@@ -141,6 +151,32 @@ void Wordrec::ResetSeamEvalsBudget() {
}
}
+void Wordrec::ResetClassificationBudget() {
+ classifications_pass_remaining_ = std::max<int>(0, chop_max_classifications_pass);
+ classifications_budget_set_ = true;
+ classifications_pass_warned_ = false;
+}
+
+bool Wordrec::ClassificationBudgetExhausted() {
+ if (!classifications_budget_set_) {
+ // Lazy fallback for callers which are not covered by
+ // Tesseract::ResetPassBudgets(). This gives them a budget for the
+ // lifetime of the Tesseract object instead of none.
+ ResetClassificationBudget();
+ }
+ if (classifications_pass_remaining_ > 0) {
+ return false;
+ }
+ if (!classifications_pass_warned_) {
+ tesserr << "WARNING: reached the blob classification budget "
+ << "(chop_max_classifications_pass = " << chop_max_classifications_pass
+ << ") of language " << lang << " for this page; the legacy "
+ << "engine does not recognize the remaining words.\n";
+ classifications_pass_warned_ = true;
+ }
+ return true;
+}
+
} // namespace tesseract
#endif // DISABLED_LEGACY_ENGINE
diff --git a/src/wordrec/wordrec.h b/src/wordrec/wordrec.h
index cccc6b43..fb271e39 100644
--- a/src/wordrec/wordrec.h
+++ b/src/wordrec/wordrec.h
@@ -64,6 +64,7 @@ public:
#else // DISABLED_LEGACY_ENGINE not defined
+# include <atomic>
# include <memory>
# include "associate.h"
# include "chop.h" // for PointHeap, MAX_NUM_POINTS
@@ -211,6 +212,7 @@ public:
double_VAR_H(chop_good_split);
INT_VAR_H(chop_max_seam_evals);
INT_VAR_H(chop_max_seam_evals_pass);
+ INT_VAR_H(chop_max_classifications_pass);
INT_VAR_H(chop_x_y_weight);
BOOL_VAR_H(assume_fixed_pitch_char_segment);
INT_VAR_H(wordrec_debug_level);
@@ -233,6 +235,17 @@ public:
// (chopping disabled); a non-positive budget produces a warning.
void ResetSeamEvalsBudget();
+ // Sets a fresh per-page blob-classification budget (see
+ // classifications_pass_remaining_). Negative parameter values are clamped
+ // to 0 (no word is recognized by the legacy engine).
+ void ResetClassificationBudget();
+
+ // Returns true if the per-page blob-classification budget (see
+ // classifications_pass_remaining_) is used up, so that no further word
+ // should be given to the legacy engine. Prints a warning the first time
+ // that it returns true after the budget was set.
+ bool ClassificationBudgetExhausted();
+
// Fills word->alt_choices with alternative paths found during
// chopping/segmentation search that are kept in best_choices.
void SaveAltChoices(const LIST &best_choices, WERD_RES *word);
@@ -430,6 +443,34 @@ public:
// only once.
bool seam_evals_pass_warning_ = true;
+ // Budget of legacy blob classifications (call_matcher() calls) left for
+ // the page currently being recognized (all passes of one
+ // Tesseract::recog_all_words() call). It is set from
+ // chop_max_classifications_pass by ResetClassificationBudget(), which
+ // Tesseract::ResetPassBudgets() calls at the start of the page for the
+ // primary language and all sub-languages, and counted down in
+ // call_matcher(). The budget is only enforced between words: when
+ // ClassificationBudgetExhausted() returns true, Tesseract does not give
+ // any further word to the legacy engine (see classify_word_and_language()
+ // and classify_word_pass1() in control.cpp), so a word which was started
+ // is always finished with real classifications, and the results of
+ // earlier words and passes are kept.
+ // Because a single word (or one page) can legitimately classify many
+ // blobs, the seam budgets alone do not bound the total cost of the
+ // classifier; on degenerate inputs where pageseg produces a large number
+ // of blob-like pieces with no inter-word gaps (e.g. repeated vertical
+ // stripes) the chopper can end up classifying tens of thousands of blobs
+ // (OSS-Fuzz issue 581720522, fuzzer-api-512x256).
+ // The counter is atomic because PrerecAllWordsPar() calls call_matcher()
+ // from several threads.
+ std::atomic<int> classifications_pass_remaining_{0};
+ // False until ResetClassificationBudget() was called for the first time.
+ // ClassificationBudgetExhausted() sets the budget lazily in that case.
+ bool classifications_budget_set_ = false;
+ // Set when the "budget exhausted" warning was printed, so that it is
+ // printed only once per page.
+ bool classifications_pass_warned_ = false;
+
protected:
inline bool SegSearchDone(int num_futile_classifications) {
return (language_model_->AcceptableChoiceFound() ||
diff --git a/unittest/baseapi_test.cc b/unittest/baseapi_test.cc
index 74485837..e148dd0f 100644
--- a/unittest/baseapi_test.cc
+++ b/unittest/baseapi_test.cc
@@ -25,6 +25,7 @@
#include <memory>
#include <regex>
+#include <sstream>
#include <string>
#include <vector>
@@ -99,6 +100,60 @@ TEST_F(TesseractTest, BasicTesseractTest) {
}
}
+// Tests the budget for the blob classifications of the legacy engine
+// (chop_max_classifications_pass).
+TEST_F(TesseractTest, ClassificationBudgetTest) {
+#ifdef DISABLED_LEGACY_ENGINE
+ // Skip test because the legacy engine is missing.
+ GTEST_SKIP();
+#else
+ REQUIRE_TRAINEDDATA(TESSDATA_DIR, "eng");
+ // Returns the text of phototest which the legacy engine gets with the given
+ // languages. A budget which is not nullptr is set for the first language.
+ auto recognize = [](const char *languages, const char *budget) {
+ tesseract::TessBaseAPI api;
+ CHECK_EQ(0, api.Init(TessdataPath().c_str(), languages, tesseract::OEM_TESSERACT_ONLY));
+ if (budget != nullptr) {
+ CHECK(api.SetVariable("chop_max_classifications_pass", budget));
+ }
+ Image src_pix = pixRead(TestDataNameToPath("phototest.tif").c_str());
+ CHECK(src_pix);
+ std::string text = GetCleanedTextResult(&api, src_pix);
+ src_pix.destroy();
+ return text;
+ };
+ auto count_words = [](const std::string &text) {
+ std::istringstream stream(text);
+ std::string word;
+ int count = 0;
+ while (stream >> word) {
+ ++count;
+ }
+ return count;
+ };
+
+ // The default budget does not limit the recognition of a normal page.
+ const std::string full_text = recognize("eng", nullptr);
+ const int full_words = count_words(full_text);
+ EXPECT_GT(full_words, 50);
+ EXPECT_EQ(full_text, recognize("eng", "1000000"));
+
+ // Without budget, the legacy engine does not recognize anything.
+ EXPECT_EQ(0, count_words(recognize("eng", "0")));
+
+ // A small budget is used up after some words. These words keep their
+ // text, although the budget is no longer sufficient for the second pass.
+ const int limited_words = count_words(recognize("eng", "100"));
+ EXPECT_GT(limited_words, 0);
+ EXPECT_LT(limited_words, full_words);
+
+ // The budget is per language, so a second language without limit takes
+ // over when the budget of the first language is used up.
+ REQUIRE_TRAINEDDATA(TESSDATA_DIR, "deu");
+ EXPECT_GE(count_words(recognize("eng+deu", "100")) * 10, full_words * 9);
+#endif
+}
+
// Test that api.GetComponentImages() will return a set of images for
// paragraphs even if text recognition was not run.
TEST_F(TesseractTest, IteratesParagraphsEvenIfNotDetected) {
diff --git a/unittest/fuzzers/fuzzer-api.cpp b/unittest/fuzzers/fuzzer-api.cpp
index 41178240..c17886e3 100644
--- a/unittest/fuzzers/fuzzer-api.cpp
+++ b/unittest/fuzzers/fuzzer-api.cpp
@@ -64,6 +64,12 @@ extern "C" int LLVMFuzzerInitialize(int * /*pArgc*/, char ***pArgv) {
/* Silence output */
api->SetVariable("debug_file", "/dev/null");
+ /* Bound the run time for images which give a huge number of blobs (OSS-Fuzz
+ issue 581720522). This is not the default, because it would also truncate
+ the text of normal pages. The parameter does not exist in builds without
+ the legacy engine, so the result is ignored. */
+ api->SetVariable("chop_max_classifications_pass", "5000");
+
return 0;
}