Commit 4c4a0f0a4 for llama.cpp

commit 4c4a0f0a4246bc36a3724d12238d5624714dcb2f
Author: Aman Gupta <amangupta052@gmail.com>
Date:   Fri Nov 20 16:13:00 2026 +0800

    llama : reserve graph nodes for the MoE cache (#32636)

    The MoE cache adds a slot lookup to every MoE layer, and with
    --moe-cache-cpu-share a second expert path, but graph_max_nodes() did
    not account for it. Models on the default 8*n_tensors budget with few
    tensors per layer (cohere2moe: 443 tensors, 49 layers) overflow the
    graph context when the cache is combined with a quantized KV cache
    (K/V rotation), aborting in ggml_new_tensor with GGML_ASSERT(obj_new).

    Fixes #32584

    Assisted-by: Claude Opus 5.5

diff --git a/src/llama-context.cpp b/src/llama-context.cpp
index 6252147a7..7f1cc4c44 100644
--- a/src/llama-context.cpp
+++ b/src/llama-context.cpp
@@ -2949,6 +2949,12 @@ uint32_t llama_context::graph_max_nodes() const {
         }
     }

+    // the MoE cache adds a lookup to each MoE layer, and a second expert path with a CPU share (up to ~16 tensors per layer),
+    // which the per-tensor estimates above do not cover for models with few tensors per layer
+    if (cparams.moe_cache_size > 0) {
+        res += 32u*model.hparams.n_layer_all;
+    }
+
     // every sampler builds n_outputs_max_per_seq chains, see llm_graph_context::build_sampling
     for (const auto & [seq_id, sampler] : sampling.samplers) {
         res += llama_sampler_backend_n_nodes(sampler) * cparams.n_outputs_max_per_seq;