Commit 9000a9e226 for aom

commit 9000a9e2267e6629a7867801d9d3ff3bd5bdd1eb
Author: Ihor Buhaiov <freedbrt@gmail.com>
Date:   Sun Nov 15 11:52:33 2026 +0100

    rtc: Grade superblocks when a frame barely changes

    The one pass RT scene detection averages the source sad over the 64x64
    blocks of the frame with an integer division. If the whole frame
    changes by less than one per block, like a slow fade of a small area
    or a few cursor pixels, frame_source_sad is 0 and
    grade_source_content_sb() marks every superblock kZeroSad without
    looking at its own sad. On screen content at speed 9 and up a kZeroSad
    superblock with a small sad against LAST is forced to zero mv skip, so
    the change is never coded. Later frames that change as little are
    skipped the same way, and the area stays stale until a bigger change
    lands in it. An active map turns the forced skip off, so this needs an
    encode without a map.

    Also grade the superblocks when the largest block sad is above 0.
    Superblocks that did not change still get kZeroSad from their cached
    block sad, and rate control sees the same frame_source_sad as before.
    The frame_source_sad check stays because the lag in frames path sets
    it to UINT64_MAX after the lookahead scan, while max_block_source_sad
    keeps the value of the last scanned pair. A frame with an all inactive
    map skips scene detection and sets frame_source_sad to 0, so set
    max_block_source_sad to 0 there too.

    Rounding the average up fixes it too, but then a tiny change clears
    static_since_last_scene_change and turns off the faster q drop on
    static screens. On a 1080p static screen with a one pixel blink after
    a scene change, CBR with aq 3 at 30 fps, it took 7 more frames to
    reach q 12 in 4:2:0 and 2.9% more bytes in 4:4:4. With this change q
    and bytes stay as on main there.

    A 640x384 static screen with a 4x4 area that fades by 3 per frame,
    speeds 9 to 11, q 20: on main the decoded area keeps its old value,
    mse 8100 after 30 frames. With this change it follows the fade, mse 9
    to 23. At 1080p with an 8x8 area that fades by 1 per frame for 60
    frames, CBR with aq 3 and q 10 to 50: mse 3600 on main, 4 or less with
    this change.

    Screen recordings, 5 clips of 150 to 254 frames at 1080p and 1440p,
    speed 11, tune screen, 1 thread, Apple M3 Pro. VBR is q 9 to 15, CBR
    is aq 3, q 10 to 50, 2000 kbps at 30 fps:

                             streams changed  bytes
      no map, VBR    4:2:0   2 of 5           +0.00% to +0.01%
                     4:4:4   3 of 5           -0.55% to +0.04%
      map, VBR       4:2:0   0 of 5
                     4:4:4   2 of 5           -0.13% to +0.09%
      no map, CBR    4:2:0   3 of 5           -0.99% to +1.12%
                     4:4:4   3 of 5           -0.20% to +0.00%

    Without a map, main keeps a luma block of the browser scroll clip
    stale for 9 frames, worst block 30.26 dB in both formats. With this
    change it is 38.14 dB. No clip's worst luma block gets lower. One
    4:4:4 clip with the map gets a V block at 27.3 dB on two single frames
    where main's worst is 31.1 dB. That block is a moved skip block whose
    chroma error the uv model averages away, the case of change 221041.
    With 220701, 220862 and 221041 applied below this change, that clip's
    worst V block is 37.7 dB.

    Camera clips (park_joy_90p 8 bit 4:2:0 and 4:4:4, niklas 720p, paris
    CIF, rush_hour 4:4:4), CBR with aq 3, speeds 7, 9 and 11, 1 and 4
    threads give the same streams as main. 10 bit: park_joy_90p and
    screendata 1080p, 4:2:0 and 4:4:4, 1 and 4 threads, with and without
    the map, give the same streams and no recon mismatch.

    In lag mode (lag_in_frames 1 and 8) and on all inactive frames with
    aq 3, the streams are the same as main. Checked on park_joy_90p,
    paris, niklas 720p, rush_hour and screendata 1080p, 8 and 10 bit,
    4:2:0 and 4:4:4, speeds 7 to 11, default and screen tune.

    Test: EncodeAPI.SlowFadeOnStaticScreenRealtime fails without the fix
    (area sse 129600 > 1600 at speeds 9 to 11) and passes with it, in a
    default and a realtime only build. Its lag_in_frames 1 runs fail if
    only the largest block sad is checked.

    Change-Id: I790bedd70e7d78dd1a936334a0d6326213dd59a7

diff --git a/av1/encoder/encodeframe.c b/av1/encoder/encodeframe.c
index 96a3cc57fa..cf228dc668 100644
--- a/av1/encoder/encodeframe.c
+++ b/av1/encoder/encodeframe.c
@@ -1193,7 +1193,10 @@ static inline void grade_source_content_sb(AV1_COMP *cpi, MACROBLOCK *const x,
   bool calc_src_content = false;

   if (cpi->sf.rt_sf.source_metrics_sb_nonrd) {
-    if (!cpi->sf.rt_sf.check_scene_detection || cpi->rc.frame_source_sad > 0) {
+    // The frame average can round down to 0 for a small change, so check the
+    // largest block sad too.
+    if (!cpi->sf.rt_sf.check_scene_detection || cpi->rc.frame_source_sad > 0 ||
+        cpi->rc.max_block_source_sad > 0) {
       calc_src_content = is_calc_src_content_needed(cpi, x, mi_row, mi_col);
     } else {
       x->content_state_sb.source_sad_nonrd = kZeroSad;
diff --git a/av1/encoder/ratectrl.c b/av1/encoder/ratectrl.c
index 117322eb25..694c69ed79 100644
--- a/av1/encoder/ratectrl.c
+++ b/av1/encoder/ratectrl.c
@@ -3937,6 +3937,7 @@ void av1_get_one_pass_rt_params(AV1_COMP *cpi, FRAME_TYPE *const frame_type,
     rc->frame_source_sad = 0;
     rc->avg_source_sad = (3 * rc->avg_source_sad + rc->frame_source_sad) >> 2;
     rc->percent_blocks_with_motion = 0;
+    rc->max_block_source_sad = 0;
     rc->high_source_sad = 0;
   } else if (cpi->sf.rt_sf.check_scene_detection &&
              svc->spatial_layer_id == 0) {
diff --git a/test/encode_api_test.cc b/test/encode_api_test.cc
index 0a390a97f7..3051b46b82 100644
--- a/test/encode_api_test.cc
+++ b/test/encode_api_test.cc
@@ -5389,6 +5389,121 @@ TEST(EncodeAPI, Avg64x64IntegerOverflow) {
 }
 #endif  // CONFIG_AV1_HIGHBITDEPTH

+#if CONFIG_AV1_DECODER
+// Encodes a static screen in realtime mode, then fades a small flat area a
+// few levels per frame. Each frame changes less than one level per 64x64
+// block on average, and the decoded area must still follow the source.
+void EncodeSlowFadeOnStaticScreen(int lag_in_frames, int speed) {
+  constexpr int kWidth = 640;
+  constexpr int kHeight = 384;
+  // The zero motion skip of static superblocks starts 30 frames after the
+  // key frame.
+  constexpr int kStaticFrames = 40;
+  constexpr int kFadeFrames = 30;
+  constexpr int kFrames = kStaticFrames + kFadeFrames;
+  constexpr int kAreaX = 200;
+  constexpr int kAreaY = 136;
+  constexpr int kAreaSize = 4;
+  constexpr int kAreaStart = 64;
+  constexpr int kFadeStep = 3;
+  aom_codec_iface_t *iface = aom_codec_av1_cx();
+  aom_codec_enc_cfg_t cfg;
+  ASSERT_EQ(aom_codec_enc_config_default(iface, &cfg, AOM_USAGE_REALTIME),
+            AOM_CODEC_OK);
+  cfg.g_w = kWidth;
+  cfg.g_h = kHeight;
+  cfg.g_lag_in_frames = lag_in_frames;
+  cfg.g_threads = 1;
+  cfg.rc_min_quantizer = 20;
+  cfg.rc_max_quantizer = 20;
+
+  aom_codec_ctx_t enc;
+  ASSERT_EQ(aom_codec_enc_init(&enc, iface, &cfg, 0), AOM_CODEC_OK);
+  ASSERT_EQ(aom_codec_control(&enc, AOME_SET_CPUUSED, speed), AOM_CODEC_OK);
+  ASSERT_EQ(aom_codec_control(&enc, AV1E_SET_TUNE_CONTENT, AOM_CONTENT_SCREEN),
+            AOM_CODEC_OK);
+  // Decode to 8 bit buffers even when FORCE_HIGHBITDEPTH_DECODING is on.
+  aom_codec_dec_cfg_t dec_cfg = {};
+  dec_cfg.allow_lowbitdepth = 1;
+  aom_codec_ctx_t dec;
+  ASSERT_EQ(aom_codec_dec_init(&dec, aom_codec_av1_dx(), &dec_cfg, 0),
+            AOM_CODEC_OK);
+
+  aom_image_t *img =
+      aom_img_alloc(nullptr, AOM_IMG_FMT_I420, kWidth, kHeight, 1);
+  ASSERT_NE(img, nullptr);
+  for (int y = 0; y < kHeight; ++y) {
+    uint8_t *row = img->planes[0] + y * img->stride[0];
+    for (int x = 0; x < kWidth; ++x) {
+      row[x] = ((x * 7 + y * 3) % 11 < 3 && ((x ^ y) & 4)) ? 32 : 224;
+    }
+  }
+  for (int plane = 1; plane < 3; ++plane) {
+    for (int y = 0; y < kHeight / 2; ++y) {
+      memset(img->planes[plane] + y * img->stride[plane], 128, kWidth / 2);
+    }
+  }
+
+  int area_values[kFrames];
+  int decoded_frames = 0;
+  int sse = 0;
+  // The calls after the last frame flush the lookahead.
+  for (int frame = 0; frame < kFrames + lag_in_frames; ++frame) {
+    aom_image_t *input = nullptr;
+    if (frame < kFrames) {
+      const int fade_steps =
+          frame < kStaticFrames ? 0 : frame - kStaticFrames + 1;
+      area_values[frame] = kAreaStart + fade_steps * kFadeStep;
+      for (int y = kAreaY; y < kAreaY + kAreaSize; ++y) {
+        memset(img->planes[0] + y * img->stride[0] + kAreaX, area_values[frame],
+               kAreaSize);
+      }
+      input = img;
+    }
+    ASSERT_EQ(aom_codec_encode(&enc, input, frame, 1, 0), AOM_CODEC_OK);
+    aom_codec_iter_t iter = nullptr;
+    while (const aom_codec_cx_pkt_t *pkt = aom_codec_get_cx_data(&enc, &iter)) {
+      if (pkt->kind != AOM_CODEC_CX_FRAME_PKT) continue;
+      ASSERT_EQ(aom_codec_decode(
+                    &dec, static_cast<const uint8_t *>(pkt->data.frame.buf),
+                    pkt->data.frame.sz, nullptr),
+                AOM_CODEC_OK);
+      aom_codec_iter_t dec_iter = nullptr;
+      const aom_image_t *decoded = aom_codec_get_frame(&dec, &dec_iter);
+      ASSERT_NE(decoded, nullptr);
+      ASSERT_LT(decoded_frames, kFrames);
+      const int area_value = area_values[decoded_frames++];
+      sse = 0;
+      for (int y = kAreaY; y < kAreaY + kAreaSize; ++y) {
+        const uint8_t *row = decoded->planes[0] + y * decoded->stride[0];
+        for (int x = kAreaX; x < kAreaX + kAreaSize; ++x) {
+          const int diff = row[x] - area_value;
+          sse += diff * diff;
+        }
+      }
+    }
+  }
+  EXPECT_EQ(decoded_frames, kFrames);
+  // The area ends 90 levels above where it started. Without a coded update
+  // the error stays near that.
+  EXPECT_LT(sse, 100 * kAreaSize * kAreaSize);
+
+  aom_img_free(img);
+  ASSERT_EQ(aom_codec_destroy(&dec), AOM_CODEC_OK);
+  ASSERT_EQ(aom_codec_destroy(&enc), AOM_CODEC_OK);
+}
+
+TEST(EncodeAPI, SlowFadeOnStaticScreenRealtime) {
+  for (int lag_in_frames = 0; lag_in_frames <= 1; ++lag_in_frames) {
+    for (int speed = 9; speed <= 11; ++speed) {
+      SCOPED_TRACE(testing::Message()
+                   << "lag_in_frames " << lag_in_frames << ", speed " << speed);
+      EncodeSlowFadeOnStaticScreen(lag_in_frames, speed);
+    }
+  }
+}
+#endif  // CONFIG_AV1_DECODER
+
 // A key frame whose luma is a block of noise repeated over the frame. IntraBC
 // can copy every block from the one above it. The chroma is flat.
 class RepeatedNoiseVideoSource : public ::libaom_test::DummyVideoSource {