Commit 1d8b113379 for aom
commit 1d8b113379535666e112e7098462e96b0ebae3e3
Author: Ranjit Kumar Tulabandu <ranjit.tulabandu@ittiam.com>
Date: Sat Mar 28 00:24:36 2026 +0530
Fix the calculation of known_rd
In av1_handle_intra_y_mode(), known_rd is calculated based on header
rate and the same is compared with best_rd for early exit. The header
rate used mode_costs->skip_mode_cost[][]. As best_rd does not account
for mode_costs->skip_mode_cost[][], this should have been
mode_costs->skip_txfm_cost[][] and the same is corrected in this CL.
As mode_costs->skip_mode_cost[][] was not populated when
skip_mode_flag = 0, the same seems to contain stale data from previous
frames and was seen to cause mismatch between single-thread and
multi-thread encodes due to known_rd differences. Thus, usage of
mode_costs->skip_txfm_cost[][] should fix this issue as well.
mode_costs->skip_mode_cost[][] is also initialized appropriately
when skip_mode_flag = 0.
Bug: 496503793
Change-Id: I8de9c978c8cefadcf74506a9cce8c6de4c73b4ed
diff --git a/av1/encoder/intra_mode_search.c b/av1/encoder/intra_mode_search.c
index 6f61a7cb6e..4cefbab651 100644
--- a/av1/encoder/intra_mode_search.c
+++ b/av1/encoder/intra_mode_search.c
@@ -1306,7 +1306,7 @@ int av1_handle_intra_y_mode(IntraModeSearchState *intra_search_state,
const int mode_cost =
mode_costs->mbmode_cost[size_group_lookup[bsize]][mode] + ref_frame_cost;
const int skip_ctx = av1_get_skip_txfm_context(xd);
- int known_rate = mode_cost + mode_costs->skip_mode_cost[skip_ctx][0];
+ int known_rate = mode_cost + mode_costs->skip_txfm_cost[skip_ctx][0];
const int64_t known_rd = RDCOST(x->rdmult, known_rate, 0);
if (known_rd > best_rd) {
intra_search_state->skip_intra_modes = 1;
diff --git a/av1/encoder/rd.c b/av1/encoder/rd.c
index 7005da4f60..e9f6e54f03 100644
--- a/av1/encoder/rd.c
+++ b/av1/encoder/rd.c
@@ -92,6 +92,8 @@ void av1_fill_mode_rates(AV1_COMMON *const cm, ModeCosts *mode_costs,
av1_cost_tokens_from_cdf(mode_costs->skip_mode_cost[i],
fc->skip_mode_cdfs[i], NULL);
}
+ } else {
+ av1_zero(mode_costs->skip_mode_cost);
}
for (i = 0; i < SKIP_CONTEXTS; ++i) {