Commit a49db739f for llama.cpp
commit a49db739fba21964e5a05c4d8c35e352b293242e
Author: Daniel Bevenius <daniel.bevenius@gmail.com>
Date: Tue Nov 17 01:53:08 2026 -0800
cuda : raise shared mem limit only for the launched kernel (#32392)
This commit removes a call of the CUDA_SET_SHARED_MEMORY_LIMIT macro as
the fallback decision was moved out of launch_mul_mat_q.
launch_mul_mat_q now has the fallback template parameter which was added
in Commit 6eddde06a4f25d55d538b5d15628dcc2b6882147 ("CUDA: refactor MMQ
kernel configuration (#24127)").
This code was then refactored in Commit
c49edf0963a76d5e78b2a7278c9bd5b71d26c84f ("CUDA: fuse FFN gate/up
matmuls and GLU in MMQ with interleaved gate/up rows (#29948)") which
added the has_fusion template parameter but it kept both macro calls.
launch_mul_mat_q only launches one type of kernel now (this is now
decided earlier in the templated call chain) and each specialization
will set the limit itself.
The motivation for this is to simplify the code and improve readabilty.
diff --git a/ggml/src/ggml-cuda/mmq.cuh b/ggml/src/ggml-cuda/mmq.cuh
index efe6527c2..219b567dd 100644
--- a/ggml/src/ggml-cuda/mmq.cuh
+++ b/ggml/src/ggml-cuda/mmq.cuh
@@ -1481,10 +1481,7 @@ static void launch_mul_mat_q(ggml_backend_cuda_context & ctx, const mmq_args & a
const dim3 block_dims(warp_size, nwarps, 1);
- CUDA_SET_SHARED_MEMORY_LIMIT((mul_mat_q<type, J, /*fallback =*/ false, prec_src1, has_fusion>), nbytes_shared);
- if constexpr (!has_fusion) {
- CUDA_SET_SHARED_MEMORY_LIMIT((mul_mat_q<type, J, /*fallback =*/ true, prec_src1, false>), nbytes_shared);
- }
+ CUDA_SET_SHARED_MEMORY_LIMIT((mul_mat_q<type, J, fallback, prec_src1, has_fusion>), nbytes_shared);
const int I_dst = has_fusion ? config.I/2 : config.I; // Rows of dst per tile, see mul_mat_q.
const int nti = (args.nrows_x + I_dst - 1) / I_dst;