Commit 3e6cd5e760 for ffmpeg

commit 3e6cd5e7603cb81b028b9c17530697b8986bf69a
Author: Jake Fineman <jake@wave.online>
Date:   Wed Sep 30 10:08:02 2026 -0400

    avfilter/dnn_io_proc: fix stack overflow in NCHW<->frame sws_scale calls

    ff_proc_from_dnn_to_frame() and ff_proc_from_frame_to_dnn() each declare a
    single `void *middle_data` local and, on the DL_NCHW layout branch, take its
    address (`dst_data = &middle_data;` / `src_data = &middle_data;`) and pass
    that as sws_scale()'s dst/src plane-array argument after casting to
    `uint8_t *const *`. sws_scale()'s dst/src parameters are a 4-element array
    (uint8_t *const dst[4]/src[4]), and scale_internal() copies all four
    pointers (memcpy of srcSlice/dstSlice into src2/dst2) regardless of how many
    planes the actual conversion uses. Since `&middle_data` only ever points at
    one pointer-sized stack slot, that copy reads past it into whatever is next
    on the stack: an out-of-bounds stack read on
    every dnn_processing call whose model uses NCHW layout output or input into
    an RGB24/BGR24 frame.

    Fix both by giving dst_data/src_data real 4-element backing storage
    (dst_data_buf/src_data_buf, zero-initialized) with the allocated buffer in
    slot 0, instead of aliasing the single pointer.

    Tested on arm64 macOS 27.0 (M1 Ultra), clang-asan toolchain, with
    --enable-libopenvino (2026.4.0), --enable-libonnxruntime (1.30.0) and
    --enable-libtensorflow, against unmodified upstream master (ff2059afd9) and
    against this commit applied on top of the same base:

    - Reproduced the stack-buffer-overflow on unmodified master two ways:
      - ff_proc_from_dnn_to_frame(): `ffmpeg -threads 1 -i puppies.ogv -an -vf
        "fps=2,format=rgb24,dnn_processing=dnn_backend=openvino:
        model=proc_valid_720.onnx:input=input:output=output" -f framemd5 out.md5`
        aborts under ASan on the first frame (offset-72 read into the
        `middle_data`/`planar_data` stack slots, from the OpenVINO async
        inference callback thread).
      - ff_proc_from_frame_to_dnn(): the same dnn_processing filter with two real
        ONNX Model Zoo fast-neural-style models (candy-9.onnx,
        sha256 9d11a3529d1e547da6ae07201d93484dbab2ec0a3614535752c8f40f0fe2968a;
        mosaic-9.onnx,
        sha256 fa646dedade881243f8d5a2ceb7de2b93675b21fc24f7482894ac4851a9a0a47;
        both NCHW 1x3x224x224 in/out) via dnn_backend=onnx aborts under ASan the
        same way, this time with the overflow located directly in
        ff_proc_from_frame_to_dnn's `middle_data` frame (offset 72), confirming
        the sibling instance flagged at review is independently live, not just
        structurally identical.
      With this commit applied, all three of the above runs are clean of the
      stack-buffer-overflow report (an unrelated, pre-existing OpenVINO/TBB
      at-exit SEGV under ASan remains on this machine/toolchain/OpenVINO version;
      it reproduces identically before and after this commit, and even on plain
      `ffmpeg -version` with no DNN filter at all, so it is unrelated to this fix
      and not addressed here).
    - Whole-clip (puppies.ogv, 589 frames) framemd5 output from a release
      (non-ASan) build is byte-for-byte identical before/after this commit for:
      the NCHW OpenVINO identity model (proc_valid_720.onnx) into rgb24, and for
      both candy-9.onnx and mosaic-9.onnx into rgb24 via dnn_backend=onnx. This
      is expected: the aliasing read pulls garbage bytes that sws_scale never
      writes to a packed destination/never reads for these plane counts, so the
      visible output is unaffected; only ASan's redzone instrumentation makes the
      read observable.
    - Unchanged paths are also byte-for-byte identical before/after: the NHWC
      TensorFlow path (tf_rgb_valid.pb into rgb24, which never takes the
      DL_NCHW/middle_data branch) and the planar/gray output path (tf_y_valid.pb
      into gray8, which uses frame->data directly and never allocates
      middle_data).
    - `make fate-source` passes with this commit. This tree has no fate-dnn*
      tests to run.

    Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
    Signed-off-by: Jake Fineman <jake@wave.online>

diff --git a/libavfilter/dnn/dnn_io_proc.c b/libavfilter/dnn/dnn_io_proc.c
index 99d4b8bd32..7c58d5cd02 100644
--- a/libavfilter/dnn/dnn_io_proc.c
+++ b/libavfilter/dnn/dnn_io_proc.c
@@ -46,6 +46,7 @@ int ff_proc_from_dnn_to_frame(AVFrame *frame, DNNData *output, void *log_ctx)
     int linesize[4] = { 0 };
     void **dst_data = NULL;
     void *middle_data = NULL;
+    void *dst_data_buf[4] = { NULL, NULL, NULL, NULL };
     uint8_t *planar_data[4] = { 0 };
     int plane_size = frame->width * frame->height * sizeof(uint8_t);
     enum AVPixelFormat src_fmt = AV_PIX_FMT_NONE;
@@ -77,7 +78,8 @@ int ff_proc_from_dnn_to_frame(AVFrame *frame, DNNData *output, void *log_ctx)
             ret = AVERROR(ENOMEM);
             goto err;
         }
-        dst_data = &middle_data;
+        dst_data_buf[0] = middle_data;
+        dst_data = dst_data_buf;
         linesize[0] = frame->width * 3;
     }

@@ -198,6 +200,7 @@ int ff_proc_from_frame_to_dnn(AVFrame *frame, DNNData *input, void *log_ctx)
     int linesize[4] = { 0 };
     void **src_data = NULL;
     void *middle_data = NULL;
+    void *src_data_buf[4] = { NULL, NULL, NULL, NULL };
     uint8_t *planar_data[4] = { 0 };
     int plane_size = frame->width * frame->height * sizeof(uint8_t);
     enum AVPixelFormat dst_fmt = AV_PIX_FMT_NONE;
@@ -227,7 +230,8 @@ int ff_proc_from_frame_to_dnn(AVFrame *frame, DNNData *input, void *log_ctx)
             ret = AVERROR(ENOMEM);
             goto err;
         }
-        src_data = &middle_data;
+        src_data_buf[0] = middle_data;
+        src_data = src_data_buf;
         linesize[0] = frame->width * 3;
     }