Commit d628a98f0 for llama.cpp

commit d628a98f094c6336e8899d1b393ec68bc0c2cfc6
Author: Georgi Gerganov <ggerganov@gmail.com>
Date:   Mon Nov 9 14:54:27 2026 +0200

    tests : keep dummy ffn.act scales away from zero (#31940)

    The dummy-model generator fills every tensor from a plain normal pool.
    `blk.N.ffn.act.scales` is a divisor in the mpt graph, so with seed
    1224489258 it draws |v| ~ 1e-6, and `ffn_act = gelu(...) / scales` reaches
    2.7e5. Metal stages both operands of `kernel_mul_mm_f32_f32` as half, so
    those activations overflow to inf, the norm downstream turns the inf into
    nan and the logits come out nan, while the CPU reference stays in f32 and
    stays finite. Both NMSE values then become nan and the check fails.

    Give `ffn.act` the same treatment as `rope_factors` and keep its magnitude
    away from zero.

    Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB)

diff --git a/tests/test-llama-archs.cpp b/tests/test-llama-archs.cpp
index ea9afd660..1f4c8a46b 100644
--- a/tests/test-llama-archs.cpp
+++ b/tests/test-llama-archs.cpp
@@ -103,6 +103,14 @@ static void fill_rope_factors(std::vector<float> & dst, const char * name, const
     }
 }

+// FFN act scales divide the activation, a value close to 0 overflows the F16 staging of the Metal mul-mat kernel
+static void fill_ffn_act(std::vector<float> & dst, const char * name, const tensor_data_params & params) {
+    fill_normal(dst, name, params);
+    for (float & v : dst) {
+        v = copysignf(1.0f + fabsf(v), v);
+    }
+}
+
 // a norm preserving residual gate holds alpha^2 + beta^2 == 1, and the conversion resolves both from one raw weight
 // the alpha and the beta of a block share that weight, so the key drops the suffix and keeps the block
 static void fill_resid_gate(std::vector<float> & dst, const char * name, const tensor_data_params & params) {
@@ -127,6 +135,7 @@ static const struct { const char * match; tensor_filler fill; } TENSOR_FILLERS[]
     { "ssm_a",        fill_ssm_a        },
     { "_resid_",      fill_resid_gate   },
     { "exp_probs_b",  fill_exp_probs_b  },
+    { "ffn.act",      fill_ffn_act      },
     { "rope_freqs",   fill_rope_factors },
     { "rope_factors", fill_rope_factors },
 };