Commit 9fbe30c2a for llama.cpp

commit 9fbe30c2a0c207f9de7f38b23c43bc044b7a9981
Author: Chinmay Shrivastava <cshrivastava2000@gmail.com>
Date:   Fri Nov 20 01:42:16 2026 -0600

    CUDA: keep the MUL_MAT_ID precision in the per-expert fallback (#31345)

    * tests: add CUDA-only MUL_MAT/MUL_MAT_ID cases with results above the fp16 range

    IQ1_M weights have no MMQ kernel, so once the batch is bigger than the MMVQ
    limit the path is dequantize + cuBLAS. These cases produce results around
    1.3e5, past the fp16 max of 65504, so an fp16 result buffer turns them into inf
    while the CPU reference stays finite. They are only added for the CUDA backend.
    Vulkan deliberately accumulates quantized matmuls in fp16 at default precision
    and would fail them by design.

    * CUDA: use an fp32 result for quantized src0 in the cuBLAS fallback

    On the F16 compute path, quantized weights went through cuBLAS with fp16
    accumulation and an fp16 result buffer that was converted to fp32 afterwards.
    Anything above 65504 became inf and the next layer turned into NaN (see #30951:
    gpt-oss-120b with IQ1_M experts at -ub 512). Set prefer_f32_output for
    quantized src0 so the GEMM still uses fp16 operands but accumulates and stores
    in fp32, the same mode already used on Volta, RDNA4 and CDNA. Every other
    quantized path already accumulates in fp32.

    Diagnosis and fix by denizyuret in #30951.

    * CUDA: keep the MUL_MAT_ID precision in the per-expert fallback

    The per-expert dst_slice kept op = GGML_OP_NONE, so ggml_prec_get_acc returned UNDEFINED and a requested precision was ignored. Set the op to GGML_OP_MUL_MAT. Also revert the fp32 result change and remove the tests.

    ---------

    Co-authored-by: JonSnow1807 <JonSnow1807@users.noreply.github.com>

diff --git a/ggml/src/ggml-cuda/ggml-cuda.cu b/ggml/src/ggml-cuda/ggml-cuda.cu
index a2db28ade..8702dd32d 100644
--- a/ggml/src/ggml-cuda/ggml-cuda.cu
+++ b/ggml/src/ggml-cuda/ggml-cuda.cu
@@ -2783,6 +2783,7 @@ static void ggml_cuda_mul_mat_id(ggml_backend_cuda_context & ctx, ggml_tensor *
         ggml_tensor dst_slice;
         memset(&dst_slice, 0, sizeof(dst_slice));
         memcpy(dst_slice.op_params, dst->op_params, sizeof(dst_slice.op_params));
+        dst_slice.op     = GGML_OP_MUL_MAT; // ggml_prec_get_acc/ggml_prec_get_src check the op
         dst_slice.buffer = dst->buffer;
         dst_slice.type   = type_dst_sorted;
         dst_slice.ne[0]  = ne0;