Commit cd3be3bd96 for qemu.org

commit cd3be3bd96c0e86f683c210a2c8f46b7da161807
Author: Denis V. Lunev <den@openvz.org>
Date:   Thu Aug 20 13:52:28 2026 +0200

    block/nbd: clear reply.cookie under receive_mutex

    s->reply is documented as protected by s->receive_mutex, but the cookie
    is cleared without it once the owning request has consumed its chunk.
    A waiter in nbd_receive_replies() inspects the very same field under
    the mutex, and does so with two separate loads:

        if (s->reply.cookie != 0) {
            ind2 = COOKIE_TO_INDEX(s->reply.cookie);
            assert(!s->requests[ind2].receiving);

    Nothing keeps those two loads consistent. If the owner clears the
    cookie in between, the second one reads 0, COOKIE_TO_INDEX() turns it
    into an index of -1, and s->requests[] is accessed out of bounds:

      Assertion `!s->requests[ind2].receiving' failed.

      (gdb) p cookie
      $1 = 8
      (gdb) p s->reply.cookie
      $2 = 0
      (gdb) p &((NBDClientRequest *)s->requests)[-1].receiving
      $3 = (_Bool *) 0x5555558416c0
      (gdb) p &s->in_flight
      $4 = (unsigned int *) 0x5555558416c0
      (gdb) p s->in_flight
      $5 = 8

    The cookie we wait for is 8, yet reply.cookie reads 0 one line after it
    was found non-zero, so the index is -1. requests[-1].receiving lands on
    in_flight, which is non-zero while requests are outstanding, and that is
    what the assertion trips over.

    Hitting this requires two coroutines of one NBD node to run in
    different threads, as there is no yield point between the two loads
    for the owner to squeeze into. A multiqueue configuration provides
    exactly that, with the virtqueues of one disk spread over several
    iothreads. Note that a compiler is free to merge the two loads into
    one, in which case the race is invisible, so builds with reduced
    optimization are much more likely to trip over it.

    Accessing s->reply without the mutex is fine for the coroutine that
    owns the reply: a non-zero cookie makes the field private to it.
    Releasing that ownership is not, as it races with the waiters which
    are explicitly allowed to look at the cookie. Clear it under the
    mutex, in the same critical section as the wakeup, and make
    nbd_recv_coroutines_wake() caller-locked, as CoMutex is not
    recursive. It has a single caller.

    The added acquisition cannot block behind the header read in
    nbd_receive_replies(), because that path is only reachable with
    reply.cookie == 0 while we still own a non-zero cookie. Merging the
    clear with the wakeup also keeps a newcomer from starting a header
    read in between, which would stall this already completed request for
    the duration of that read.

    There is no cookie to own when we get here after an error, and then a
    newcomer can indeed be inside that read. It does not hold us for long
    either, as the channel has been shut down before the error was
    reported, so the read it sits in returns right away.

    Fixes: 4ddb5d2fde ("block/nbd: drop connection_co")
    Cc: Eric Blake <eblake@redhat.com>
    Cc: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
    Signed-off-by: Denis V. Lunev <den@openvz.org>
    Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
    Message-ID: <20260820115228.587427-4-den@openvz.org>
    Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

diff --git a/block/nbd.c b/block/nbd.c
index d0a7097034..e5e16722ba 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -161,11 +161,11 @@ static bool coroutine_fn nbd_recv_coroutine_wake_one(NBDClientRequest *req)
     return false;
 }

+/* Called with s->receive_mutex taken. */
 static void coroutine_fn nbd_recv_coroutines_wake(BDRVNBDState *s)
 {
     int i;

-    QEMU_LOCK_GUARD(&s->receive_mutex);
     for (i = 0; i < MAX_NBD_REQUESTS; i++) {
         if (nbd_recv_coroutine_wake_one(&s->requests[i])) {
             return;
@@ -974,9 +974,11 @@ static coroutine_fn int nbd_co_receive_one_chunk(
         /* For assert at loop start in nbd_connection_entry */
         *reply = s->reply;
     }
-    s->reply.cookie = 0;

-    nbd_recv_coroutines_wake(s);
+    WITH_QEMU_LOCK_GUARD(&s->receive_mutex) {
+        s->reply.cookie = 0;
+        nbd_recv_coroutines_wake(s);
+    }

     return ret;
 }