Commit 4577932fa for llama.cpp

commit 4577932fab595eaa8b27ad71734d5de1f3c07d8c
Author: Ravi Panchumarthy <ravi.panchumarthy@intel.com>
Date:   Tue Nov 17 23:56:20 2026 -0800

    ggml-openvino: expand model support, optimize NPU,GPU and memory usage (#32172)

    * ggml-openvino: fix SWA mask detection

    When the SWA cache size matches the full cache size, identify SWA
    layers by the number of layers using each mask instead of cache size;
    the larger group is SWA, and a tie leaves the layers unclassified.

    * ggml-openvino: fix SWA KV cache write indices

    Give SWA KV cache index tensors unique input names (a _swa suffix, like
    the masks) to prevent them from being merged with the full-attention
    cache indices.

    * ggml-openvino: classify SWA layers only when the mask shows the window

    Two attention masks do not imply SWA: DeepSeek sparse attention uses two
    full-attention masks with the same name. When cache extents match,
    classify the larger mask group as SWA only once its mask has fewer keys
    per row than the other mask. Keep the classification per KV cache buffer
    so new sequences reuse the same graph, and include swa_layers when
    checking graph reuse.

    * ggml-openvino: classify SWA layers from the first graph

    Waiting for the window to appear in the mask caused a mid-sequence graph
    recompile (~9 s for Gemma-4 at -c 1024 after position 512).

    SWA classification already gives each mask group its own mask, KV write
    indices, and attention size, which is correct whether or not the mask is
    a sliding window. Stateful mask rebuild recovers the window from the
    mask, or the causal mask for full attention. Classify the larger of two
    mask groups from the first graph and drop the per-KV-buffer bookkeeping.

    * ggml-openvino: mark SWA fallback as a model-level heuristic

    Add a TODO above the equal-extent fallback to document that the larger
    mask group is currently assumed to be the SWA group. Since stateless
    graphs only require the two groups to remain distinct, this heuristic
    may only be necessary for stateful execution and could potentially be
    restricted to that path.

    * ggml-openvino: derive attention sizes from the KQ mask shape

    Dynamic CPU/GPU graphs previously passed n_seq_active and
    attention_size(_swa) as separate inputs. Derive these values directly
    from the KQ mask shape using ShapeOf + Gather inside the model. This
    allows the GPU plugin to infer the Q reshape and KV slice shapes from
    shape inference rather than runtime input values. Q's head size also
    becomes static, enabling the fused SDPA kernel.

    GGML_OPENVINO_DISABLE_SHAPE_FROM_MASK=1 restores the previous
    separate-input behavior.

    The fused SDPA path also requires two supporting changes:

    - Clamp the GPU mask to -30000. A key block that is entirely -inf for
      a row produces NaNs in the fused kernel.
    - Restate the head count and head size after the KV slice. The plugin
      loses all slice dimensions when the end index is dynamic, causing
      SDPA to fall back to the reference kernel.

    Additional graph and quantization improvements:

    - Skip the identity reshape of rows in SET_ROWS.
    - Share a single Split for one-element view slices in dynamic GPU
      graphs (e.g. Gemma-4 per-layer embeddings), replacing one
      StridedSlice per layer.
    - Add GGML_OPENVINO_REQUANT_EMBD=native (off by default) to preserve
      the file's native quantization for token_embd/output instead of
      requantizing to Q8_0_C.
    - Include both environment settings in the graph and model cache keys
      and document them.

    * ggml-openvino: only reuse Split outputs for exact chunk-aligned views

    The dynamic GPU path reused a Split output for any single-element view
    by deriving the chunk index from offset / stride. This produced
    incorrect results for views that were not chunk-aligned or whose
    remaining dimensions did not preserve the source strides, as seen in
    test-backend-ops CONT with use_view_slice.

    Restrict Split reuse to cases where the view exactly matches a split
    chunk, meaning the offset is aligned to the split stride and the
    remaining non-unit dimensions preserve the original source strides.
    Gemma-4 per-layer embedding slices continue to use the shared Split
    path.

    * ggml-openvino: initialize ModelExtraInputInfo shape fields

    Initialize the newly added `shape_source` and `shape_axis` fields in
    the `ModelExtraInputInfo` brace initializer used by
    `add_extra_inputs()`.

    Without initializers for these fields, GCC emits
    `-Wmissing-field-initializers`, causing CI to fail with
    `LLAMA_FATAL_WARNINGS`.

    * ggml-openvino: remove GGML_OPENVINO_REQUANT_EMBD

    The option identifies embedding and output weights by their tensor
    names, which are defined by core llama.cpp. Remove it from this PR;
    preserving the file quantization of these weights will be addressed in
    a separate change that identifies them by their roles in the graph.

    * ggml-openvino: keep manual GQA attention 4-D

    The manual GQA attention path reshaped Q, K, and V to 5-D so that
    query heads sharing a KV head could broadcast against it. The GPU
    plugin cannot fuse this graph: it materializes the attention scores
    and copies the transposed K and V caches for every layer.

    Instead, stack the query heads for each KV head along the sequence
    axis (Q [B, H, S, D] -> [B, H_kv, factor * S, D]) and tile the mask
    accordingly. This keeps the graph 4-D from the MatMuls through the
    Softmax, allowing the GPU plugin to fuse the attention into its SDPA
    kernel and fold the K and V transposes into the fused operation.

    Clamp the mask to -30000, matching the SDPA path, because the fused
    kernel can return NaN when an entire block is masked with -inf.

    Models where the K or V head size differs from the Q head size now
    use the SDPA path, since the manual path assumes one head size.

    * ggml-openvino: use a 4-D stateless KV cache on GPU

    In stateless GPU mode, bind the entire KV cache as a
    `[1, ctx, n_heads_kv, head_size]` tensor, with one token per row,
    instead of binding the first `n_kv` rows as a
    `[1, 1, ctx, n_heads_kv * head_size]` tensor.

    Write cache updates with `ScatterUpdate` along the token axis, and
    read the cache through `VariadicSplit` and `Transpose`. This allows
    the GPU plugin's `StatelessKVFusion` to replace cache reads and writes
    with in-place `StatelessKV` nodes, avoiding full-cache copies on every
    decode step and repeated memory pool allocations as the bound cache
    size grows.

    Set `GGML_OPENVINO_DISABLE_KV_4D=1` to retain the previous layout.

    * make embedding to have dynamic shape for OV GPU

    * ggml-openvino: make the 4-D stateless KV cache opt-in

    * add support for Q5_0 data type in OV GPU backend

    * resolve CI test error

    * ggml-openvino: fix -fa off and optimize grouped-query attention

    With -fa off, llama.cpp stores the V cache transposed, with value
    elements as rows and tokens along each row. The KV slicing code assumes
    one token per row and binds the wrong data when n_kv < kv_size. Detect
    transposed caches with GgmlOvDecoder::is_kvcache_token_rows() and skip
    slicing for them.

    For grouped-query attention, avoid repeating K and V for each query
    head. Stack the query heads for each KV head along the row dimension
    and let SOFT_MAX apply the scale, mask, and softmax so the GPU plugin
    can fuse the operation with the MatMuls.

    Read only the current layer's V-cache rows when layers use different
    cache sizes. Reject -fa off in stateful mode with a clear error and
    document that it requires stateless mode.

    * ggml-openvino: fall back to stateless execution when -fa off

    With `-fa off`, the graph writes the V cache in transposed form, which is
    not supported by the stateful KV cache. Detect this pattern in
    `compute_llm_params()` as a `SET_ROWS` into a KV cache through a `[1, n]`
    reshape.

    When stateful execution is enabled, log a warning and fall back to
    stateless execution instead of failing graph conversion.

    Update the `-fa off` note in `OPENVINO.md` to document this behavior.

    Co-authored-by: Suhas Somashekar <suhas.somashekar@intel.com>

    * ggml-openvino: preserve flat KV cache layout when -fa is off

    With -fa off, the 4-D KV cache layout introduced in #344 breaks MUL_MAT
    attention for Phi-3.5 mini and Phi-4 mini, causing oneDNN to fail with
    "could not construct a memory descriptor using strides".

    Preserve the [1, 1, ctx, n_heads_kv * head_size] layout when the graph
    writes a transposed V cache.

    * ggml-openvino: extract Q5_0 weights when requantization is off

    Q5_0 weights are requantized to Q4_0, but test-backend-ops builds naive
    graphs, where requantization is skipped. Q5_0 then reached the normal
    weight extraction, which had no Q5_0 case, so every Q5_0 MUL_MAT and
    GET_ROWS op test failed with "Unsupported quantized type: q5_0".

    Extract Q5_0 as symmetric i8 weights (q - 16) with one f16 scale per 32
    weights, the same layout as Q8_0. Models are unchanged: they still
    requantize Q5_0 to Q4_0.

    * ggml-openvino: Optimize GDN output pre-scale for the NPU static path

    The NPU GatedDeltaNet kernel writes its output in FP16 and flushes values below 6.1e-5 to zero.

    * Refine the code as review comments

    1. Add envs desc into doc
    2. Refine the code with removing duplicated env

    * ggml-openvino: reuse graph bindings between decode calls

    Reduce host overhead outside `infer()` from 0.5–1.1 ms to 0.14–0.38 ms
    per token on Panther Lake.

    - Reuse KV cache tensors and skip redundant port bindings.
    - Use persistent USM host tensors for small GPU inputs and outputs.
    - Defer graph-wide consumer lookup until after shape checks.
    - Return model and compute params by const reference.
    - Preserve tensor extras when memory, shape and type are unchanged.

    * ggml-openvino: Add npuw dump envs for profiling

    * Add the comments of new npu dump envs into doc

    * ggml-openvino : ensure all leaves have unique names, remove input_srcs from graph key

    * openvino: slice NPU prefill KV cache by active length

    Add GGML_OPENVINO_NPU_KV_SLICE as an opt-in NPU prefill optimization. When enabled, static prefill keeps the graph's attention dimension at the active prompt/KV length instead of expanding it to the full context capacity, and the KV-cache input/output tensors are bound with a shortened context axis over the same backing storage.

    This reduces the amount of KV cache and attention-mask data imported by the NPU plugin and lets the prefill graph plan smaller attention buffers for early prompt chunks. For example, with a 4096-token context and a 256-token prefill chunk, the first chunk can expose KV/mask dimensions at 256 instead of 4096, avoiding work over padded cache rows.

    The trade-off is graph stability: NPU static shapes now depend on the aligned active attention length for prefill, so a cached prefill model is reused only when the attention sizes match. Decode keeps a fixed full-context graph to avoid recompiling on every generated token, and SWA/ring-cache cases still fall back to full-size binding when the live prefix is not a simple leading slice.

    * openvino: use L0 host tensors for NPU buffers

    Add GGML_OPENVINO_NPU_L0_HOST_TENSORS as an opt-in NPU buffer allocation mode. When enabled, non-remote OpenVINO backend buffers on NPU are allocated through the NPU default context with create_host_tensor(), and ggml tensor data points directly at that OpenVINO-owned Level Zero host allocation.

    The optimization targets the data path rather than the graph math: KV-cache and compute tensors can be passed back to the NPU plugin as OpenVINO tensors backed by importable Level Zero host memory. That avoids an extra ordinary-host-memory-to-plugin-staging copy on each infer boundary, which is especially useful for prefill where large KV/cache buffers dominate input and output traffic.

    If Level Zero host allocation fails, the backend logs a warning and falls back to the existing allocation path. The destructor tracks OpenVINO-owned storage so it does not free memory owned by the host tensor. The trade-off is that this is NPU-specific, consumes Level Zero host allocation resources, and improves transfer/import overhead without changing attention compute complexity or tensor shapes.

    * openvino: fill NPU prefill masks directly

    Add GGML_OPENVINO_NPU_FAST_MASK as an opt-in NPU static prefill input-packing optimization. When enabled, the prefill attention mask is built directly in the OpenVINO input tensor instead of first materializing a padded std::vector and then copying that whole buffer into the tensor.

    The new fill_prefill_mask() helper writes each row once: it copies the valid mask prefix, fills padding columns with -inf, handles padded rows, and restores the causal diagonal to zero. This keeps the final mask numerically identical to the existing pad_input() plus set_zero_diagonal() path while removing one intermediate allocation and one chunk_size-by-context_size host copy per prefill chunk.

    The optimization targets CPU-side staging overhead, not attention math. It composes with GGML_OPENVINO_NPU_KV_SLICE: KV slice reduces the mask dimensions, while FAST_MASK reduces the work needed to construct whatever mask shape remains. The trade-off is a second mask construction path to maintain, with limited benefit for very small prompts or contexts.

    * openvino: make NPU requant layout configurable

    Add GGML_OPENVINO_NPU_REQUANT_POLICY as the NPU-side selector for the default weight requantization layout. The existing NPU behavior remains the default: unset or group-128 maps quantized weights to Q4_0_128, while channel-wise selects the experimental Q4_0_C layout. Unknown policy values fail fast during model loading instead of silently compiling a graph with unintended weight packing.

    The optimization target is NPU weight bandwidth and compressed-weight code generation. Q4_0_128 keeps weights in a 4-bit symmetric layout with one scale per 128 values, reducing both payload bytes and scale metadata versus smaller groups while preserving a more conservative quantization granularity than channel-wise. Q4_0_C uses one scale over the innermost dimension, which can further shrink metadata and simplify dequant broadcast, but may cost accuracy because the quantization block is much larger.

    Include the selected NPU requant policy in the OpenVINO model cache discriminator. The same cgraph compiled with group-128 and channel-wise has different weight constants and dequant shapes, so reusing a cached decoder across policies would be incorrect. This keeps cache reuse stable within a policy while forcing recompilation when the layout policy changes.

    Trade-off: policy selection gives a performance/accuracy knob for NPU experiments, but all options still pay the load/compile-time requantization cost. The default remains group-128 for a safer balance of bandwidth, metadata, and accuracy; channel-wise is intentionally documented as experimental.

    * openvino: centralize NPU plugin config overrides

    Add GGML_OPENVINO_NPU_CONFIG as the recommended catch-all override for OpenVINO NPU plugin and compiler properties. The value is parsed as comma-separated KEY=VALUE pairs and applied after the built-in defaults and named NPU aliases, so advanced users can override any exposed property without adding a new llama.cpp environment variable for every plugin tuning knob.

    Keep the common knobs as named aliases for discoverability and compatibility: GGML_OPENVINO_NPU_COMPILER_TYPE maps to NPU_COMPILER_TYPE, GGML_OPENVINO_NPUW_FUNCALL_FOR_ALL maps to NPUW_FUNCALL_FOR_ALL, GGML_OPENVINO_NPUW_UNFOLD_IREQS maps to NPUW_UNFOLD_IREQS, and GGML_OPENVINO_COMPILATION_NUM_THREADS maps to COMPILATION_NUM_THREADS. GGML_OPENVINO_NPU_COMPILE_CONFIG is retained as a legacy alias for NPU_COMPILATION_MODE_PARAMS, while the documentation now recommends expressing it through GGML_OPENVINO_NPU_CONFIG.

    This keeps the public tuning surface compact while preserving high-signal aliases for settings with known performance or stability implications. NPU_COMPILER_TYPE can select the driver compiler, which currently generates faster prefill kernels on the tested NPU stack. NPUW_FUNCALL_FOR_ALL and NPUW_UNFOLD_IREQS expose NPUW graph lowering trade-offs around dispatch overhead, memory use, and long-context stability. COMPILATION_NUM_THREADS gives a way to cap compiler worker parallelism to reduce peak host memory during large graph compilation.

    Trade-off: a free-form KEY=VALUE string is less type-safe than one env var per option and invalid plugin properties will only be diagnosed by the OpenVINO stack. Applying it last is intentional: it gives one escape hatch for bisecting driver/compiler behavior and for testing new plugin options without rebuilding llama.cpp, while the named aliases continue to document the most important performance knobs.

    * openvino: add scatter-elements KV update option

    Add GGML_OPENVINO_KV_SCATTER_ELEMENTS as an opt-in lowering for non-stateful single-row KV cache writes. When the flag is enabled and the set_rows indices are the simple decode case, translate_set_rows uses ScatterElementsUpdate with broadcasted rank-4 indices instead of ScatterUpdate.

    The motivation is NPU decode performance. The default ScatterUpdate path updates one KV row but the NPU plugin does not perform that update in place; it effectively copies the full destination tensor to write a small slice. As context length grows, that full-tensor movement dominates the per-token KV update cost. ScatterElementsUpdate expresses the same single-row write at element granularity and is measurably cheaper for this shape.

    Keep the optimization guarded. Stateful execution keeps its existing concat/update path, and multidimensional indices already use the broader ScatterElementsUpdate lowering. The new flag only changes the common non-stateful decode write, so the default behavior and graph shape remain unchanged unless the user opts in.

    Trade-off: ScatterElementsUpdate introduces extra ShapeOf, Reshape, and Broadcast nodes to build indices matching the update tensor. That overhead is worthwhile only when it avoids the NPU plugin's full-destination ScatterUpdate copy, so the path is controlled by an environment variable rather than becoming the unconditional lowering.

    * openvino: gather token embeddings from compressed rows

    Add NPU token-embedding controls for models whose token_embd.weight would otherwise be widened from Q6_K to FP16. GGML_OPENVINO_TOKEN_EMBD_I8 keeps the embedding table on the normal int8 compressed-weight path, while GGML_OPENVINO_TOKEN_EMBD_I4 requantizes it to the group-128 int4 layout used by the NPU weight path.

    The performance target is tied embeddings and large vocabulary tables. When token_embd.weight is also consumed by the lm_head matmul, a normal Gather from the dequantized table gives the dequantization subgraph a second consumer and can force the full table to be materialized. The new gather_compressed_rows helper recognizes the dequantization chain emitted for int4/int8 constants, gathers only the selected packed rows from the leaf constants, and rebuilds the convert/subtract/multiply/reshape chain on those rows.

    This reduces memory pressure and avoids dequantizing or materializing a full embedding table when inference only needs the token rows for the current input. The helper is deliberately conservative: it only rewrites static 2D tables above a size threshold, walks only the known compressed-weight node patterns, limits recursion depth, and falls back to the original Gather when the subgraph is not recognized.

    Trade-off: int8 and especially int4 token embeddings can change accuracy and may not be faster for small untied embedding tables, because the rewrite adds extra Gather/Reshape nodes and only pays off when it avoids a large full-table dequantization. The default behavior is unchanged; the NPU-specific I8/I4 paths are opt-in through environment variables.

    * openvino: stream u4 requantization chunks

    Extend the GGML_OPENVINO_REDUCE_COMPILE_MEM streaming requantization path to Q4_0/u4 targets. The previous streaming path intentionally skipped u4 because Q4_0 packs two weights per byte and the zero-point tensor stores two 4-bit values per byte, so the original quantize_q4_0 helper wrote as if every call started at block zero.

    Teach quantize_q4_0 to accept a destination block offset and use the absolute block index for weights, scales, and zero-point writes. Streaming requantization emits whole-row chunks in increasing order, and the target block size is required to divide the row width, so no target block straddles a chunk boundary. That keeps packed weight bytes aligned and preserves the even-block assignment before the odd-block zero-point OR for each shared zero-point byte.

    With that indexing in place, reduce-compile-memory mode can dequantize a chunk of rows into the scratch buffer and immediately quantize it into the final u4 buffers instead of materializing the entire tensor as temporary F32. This extends the peak-memory reduction already used by Q8/F16 targets to the NPU group-128/channel u4 layouts used for compressed weights.

    Trade-off: the u4 path is more sensitive to block alignment than Q8/F16 because of nibble packing, so streaming remains guarded by the same row-divisibility check and only runs when GGML_OPENVINO_REDUCE_COMPILE_MEM is enabled. When the flag is off, the full-materialization path and quantized output layout remain unchanged.

    * openvino: fail fast on static recompile after weight release

    Extend the GGML_OPENVINO_RELEASE_WEIGHTS safety check to the static graph cache-miss path. Once host weight buffers have been released, any later compile would read invalid/zeroed host pages and bake the wrong constants into a new OpenVINO model.

    The dynamic path already rejected this situation. Static execution can still miss the decoder cache when graph shapes or model parameters change, so it needs the same guard before erasing cached infer requests and rebuilding prefill/decode models.

    This is a correctness and diagnosability change rather than a throughput optimization. It preserves the memory-saving release-weights mode for stable graph shapes, but turns unsupported recompilation into an explicit abort instead of silently producing a corrupted compiled model.

    * openvino: add experimental NPU Q4_0 keep option

    Add GGML_OPENVINO_NPU_KEEP_Q4_0 as an opt-in escape hatch for NPU weight handling. When enabled for native Q4_0 tensors, the NPU requant path returns nullopt and keeps the original Q4_0 extraction instead of regrouping the tensor through the selected NPU requant policy.

    The motivation is to avoid unnecessary work for weights that are already stored as 4-bit values. The default NPU policy maps quantized weights to layouts such as Q4_0_128, which can reduce scale metadata but requires round-tripping Q4_0 through the requantization path. Keeping Q4_0 native can save that conversion work and preserve the original group-32 packing.

    Keep the option experimental and disabled by default. Current NPU driver/plugin stacks can compile the resulting group-32 graph, but inference may hang or fail with ZE_RESULT_ERROR_DEVICE_LOST. The default behavior therefore continues to use the safer NPU requant policy; this flag is mainly useful for debugging driver behavior, measuring the true cost of regrouping, and testing future NPU stacks.

    Trade-off: native Q4_0 avoids regrouping overhead and keeps the original quantization blocks, but it gives up the more regular group-128 layout expected by the default NPU compressed-weight path and may be unstable on current hardware/software combinations.

    * openvino: unfold NPUW only for decode graph

    Compile the static prefill and decode models with separate OpenVINO config maps so the decode graph can opt into NPUW_UNFOLD_IREQS without forcing the same lowering on prefill. The build_static_model helper now receives the config map explicitly, and the decode path gets a copy of the base config with NPUW_UNFOLD_IREQS=YES when the user has not already set it.

    The performance target is NPU token generation. Decode executes a long stream of single-token infers, where folded NPUW function-call dispatch overhead becomes visible in every token. Unfolding the NPUW calls into separate infer requests removes that repeated dispatch cost and measured about +20% token-generation throughput on the tested stack, matching the behavior seen in OpenVINO GenAI.

    Prefill keeps the original folded config. Prompt processing is a batched workload dominated by larger matmuls, and the folded form is faster there while also using less memory. Splitting the config lets prefill and decode use the lowering that matches their workload shape instead of treating NPUW_UNFOLD_IREQS as a single global compromise.

    Trade-off: unfolding can increase memory and request-management overhead, so the automatic default is limited to the NPU decode model. If a user explicitly provides NPUW_UNFOLD_IREQS through the named env alias or GGML_OPENVINO_NPU_CONFIG, that value is respected because the decode override is only applied when the property is absent.

    * openvino: tidy NPU tuning helper ordering

    Collect the remaining small cleanups from the NPU tuning series. This reorders the NPU KV-slice documentation and environment-variable cache entry next to the other NPU opt-in toggles, adds a short header comment for the centralized NPU helper declarations, and makes the static prefill attention-size assignment in the decoder a little more explicit.

    There is no intended behavior change in this commit. The purpose is to make the final source layout match the split commits that introduced the NPU tuning knobs, so future readers can find the related flags and helpers in one place without mixing that mechanical cleanup into the functional performance commits.

    * openvino: normalize NPU compile env overrides

    Move the default NPU_COMPILER_TYPE=DRIVER setting into the NPU compile_config initializer so the default NPU plugin options are declared in one place.

    Route GGML_OPENVINO_NPU_COMPILE_CONFIG and GGML_OPENVINO_NPU_COMPILER_TYPE through the same set_compile_option_from_env helper used by the other named NPU compile option aliases. This keeps the legacy NPU_COMPILE_CONFIG mapping to NPU_COMPILATION_MODE_PARAMS intact while making all optional env-to-compile_config overrides follow the same non-empty-value rule.

    GGML_OPENVINO_NPU_CONFIG is still parsed last, so explicit comma-separated KEY=VALUE overrides continue to have the highest precedence over both defaults and named aliases.

    * openvino: stream NPU weight requant by default to cut compile peak

    Default the streaming requantization path on for NPU so the compile-time F32 dequant transient (~1-2 GB for token_embd) is capped without an env flag. Output is byte-identical to full materialization; GGML_OPENVINO_REDUCE_COMPILE_MEM / MEMORY_OPTIMIZE still override (set to 0 to force off). Kept the g_nonov_weight_cache opt-in to avoid holding token_embd resident (which regressed decode steady memory). pp1024_d0: PrefillPeakPrivWS 5974->4664 MiB (-22%), decode steady and pp/tg throughput unchanged.

    * openvino: add self-contained NPU model cache

    Add an opt-in embedded-weight cache mode for the static NPU backend. GGML_OPENVINO_SELF_CONTAINED_BLOB=1 exports separate prefill and decode compiled models with ENABLE_WEIGHTLESS=false, uses isolated NPUW weight banks, validates raw GGUF weight manifests, binds imported ports by deterministic frontend order, and serializes the large NPUW compile/export/import operations.

    Preserve raw model bytes during cold cache construction by requantizing into separately owned OV tensors instead of rewriting the backend model buffer. This keeps later graph fingerprints and manifests stable when llama-bench creates independent prompt and generation contexts.

    Add strict GGML_OPENVINO_SELF_CONTAINED_MMAP=1 warm-import mode. The backend exposes a non-owning buffer_from_host_ptr wrapper over the GGUF mmap, uses raw tensor allocation sizes, avoids allocating/copying the private OpenVINO model-weight buffer, and stages the few mmap-backed runtime constants into small importable NPU host tensors. Invalid or missing cache entries fail clearly instead of trying to requantize into read-only mapped storage.

    The normal path and existing weightless MODEL_PTR cache remain unchanged when the new flags are unset. Verified with phi-4-mini: chunk512 pp3072/tg512 cold 1123.10/27.60 t/s and warm mmap 1124.72/27.16 t/s; warm mmap removes roughly 3.9 GiB peak commit in the measured prefill workload. Self-contained blobs consume about 8.34 GiB for the separate pp and tg contexts.

    * openvino: add lazy import for self-contained mmap cache

    Add GGML_OPENVINO_SELF_CONTAINED_LAZY_IMPORT to make the low-working-set strict mmap path opt in instead of changing the default warm mmap behavior.

    When the flag is set with SELF_CONTAINED_BLOB=1 and SELF_CONTAINED_MMAP=1, the static NPU path imports only the current phase blob: prefill imports the prefill blob, decode imports the decode blob later. This avoids the pair-shaped shared compiled-model cache for the lazy path and caches only the current phase infer request.

    The regular warm mmap path remains eager and continues to prepare both prefill and decode compiled models. Non-mmap self-contained builds remain serial, and the regular non-self-contained path keeps the existing parallel build behavior.

    Also register the new environment variable, document the memory/latency tradeoff, and add profiling-only import timing around model-cache imports.

    Validated with ReleaseOV build: cmake --build build/ReleaseOV --target llama-bench llama-simple --config Release. Lazy warm mmap run measured PrefillPeakWS around 5759 MiB versus the eager warm mmap path around 7897 MiB.

    * openvino: release GGUF mmap resident pages before blob import

    Add GGML_OPENVINO_SELF_CONTAINED_RELEASE_MMAP_PAGES as an explicit opt-in for self-contained compiled-model cache hits.

    Before importing a valid self-contained blob, recursively walk the graph to find unique external GGUF mmap-backed weight buffers and drop their resident pages while preserving the mapping address and tensor pointers.

    On Linux use page-aligned madvise(MADV_DONTNEED). On Windows, where there is no reliable per-range mapped-file equivalent, call VirtualUnlock for the identified range and trim the current process working set with SetProcessWorkingSetSize(-1, -1). Pages required later can fault back on demand.

    Keep the behavior disabled by default and document the physical-memory versus commit-memory tradeoff. Profiling reports the submitted GGUF range and model import duration.

    Validated with ReleaseOV llama-bench/llama-simple build and pp3072+tg512 warm mmap lazy import. With page release enabled, PrefillPeakWS dropped from about 5758 MiB to 3053 MiB while throughput remained comparable; PeakCommit stayed about 3331 MiB.

    * openvino: add GGML_OPENVINO_NPU_SHARED_BANK to share prefill/decode weights bank

    When set, prefill and decode self-contained blobs use one shared NPUW_WEIGHTS_BANK name (ggml-sc-shared-<pfp>-<dfp>) instead of separate per-phase banks. Since both decoders are built from the same model_weights map, NPUW dedups them into a single in-process bank; on warm import the identical (uid,tensor) pairs are content-verified and reused (patched NPUW weights_bank). Cuts resident compiled-weight footprint ~2GB (pp3072 warm: In-Use 7954->5871 MiB, WS 7891->5765) with prefill throughput unchanged.

    * openvino: drop SELF_CONTAINED_LAZY_IMPORT and RELEASE_MMAP_PAGES

    The shared weights bank (GGML_OPENVINO_NPU_SHARED_BANK) supersedes both: it deduplicates the prefill/decode compiled-weight copy directly, so lazy per-phase import and GGUF-page release no longer reduce steady memory. Remove the two env vars, the self-contained lazy-import branches (defaulting to the eager path that imports both blobs), the release_external_weight_pages helper, and the now-unused ggml_openvino_buffer_is_external / _release_external_pages buffer APIs.

    * openvino: demand-page large NPU compute buffer in strict warm-mmap mode

    In strict self-contained warm-mmap mode weights come via buffer_from_host_ptr,
    so every alloc_buffer is graph scratch/KV. The large prefill compute buffer is
    never touched on the host by the on-device fused OV model, yet the NPU L0 host
    tensor (create_host_tensor + memset) makes all ~2.4 GiB resident. For buffers
    >= 1 GiB in this mode, reserve demand-zero pages (no L0, no memset) so they stay
    non-resident; KV/small buffers keep the L0 host tensor for zero-copy import.

    pp3072 warm mmap + shared bank: prefill peak WS 5765 -> 3314 MiB (-43%),
    In-Use delta ~5940 -> ~3232 MiB; prefill 1256.7 -> 1252.9 t/s and decode
    28.27 -> 27.96 t/s (both flat). Commit charge unchanged (demand-paged).

    * perf improvement success

    * openvino: size static NPU sliding-window masks to the window cache

    The static graph built the sliding-window KQ mask at the full context length,
    while the windowed layers bind their (smaller) window KV cache. With gemma-4 at
    -c 4096 -ub 1024 the window cache is 1536 cells, so SDPA shape inference failed
    ([1,1,1024,4096] mask vs [1,8,1536,256] K/V) and every prompt errored out.

    With GGML_OPENVINO_NPU_KV_SLICE the prefill mask was meant to use
    attention_size_swa, but the check matched "swa" in the ggml tensor name, and
    llama.cpp names both masks "attn_inp_kq_mask"; the windowed layers got the
    full-attention sliced length instead ([1,1,1024,256] vs [1,8,1024,256]).

    Identify the sliding-window mask by tensor identity (is_swa_mask) and size it to
    ctx_per_seq_swa, both in the graph input shape and in the runtime mask padding.

    * ggml-openvino: create the model cache dir under a Windows drive root

    make_dirs() mkdir'ed every path prefix, including the drive root "D:", which
    fails with EACCES (not EEXIST) on Windows. Any absolute cache path therefore
    logged "could not create model cache dir ... caching disabled" and silently
    ran without GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR. Skip the drive root.

    * ggml-openvino: split device-specific passes and NPU config out of shared code

    Mechanical refactor, no behavior change (step 1 of the NPU/MoE architecture plan):

    * openvino/pipelines/: TranslateSession::apply_transformations() now registers
      common passes, then the device's passes, then the execution-mode passes
      (stateful KV, static shapes), selected by a per-graph GraphTraits. The GPU MoE
      fusions and the GPU eltwise-rank workaround move to pipelines/gpu.cpp; CPU and
      NPU pipelines are empty placeholders for device lowerings. Pass order is
      unchanged.
    * npu/npu_config.*: the NPU (NPUW) compile config built at backend init, the
      NPUW strip for cacheless graphs and the per-graph static prefill/decode
      variants (HFA prefill, decode NPUW_UNFOLD_IREQS) move out of
      ggml-openvino-extra.cpp and utils.cpp.
    * npu/npu_quant_policy.*: the NPU requantization rules (requant policy,
      token_embd Q6_K, keep-Q4_0) move out of ggml_openvino_get_requant_type().

    Verified identical output before/after on OV-CPU, GPU and NPU (gemma-4 E4B).

    * ggml-openvino: one KV-slice switch for dynamic and static graphs

    KV-cache slicing (bind the KV cache, and size the KQ masks, at the attended
    length instead of the full context) existed twice: on by default for dynamic
    graphs (GGML_OPENVINO_DISABLE_KV_SLICE to turn it off) and opt-in for static
    prefill (GGML_OPENVINO_NPU_KV_SLICE). The static mode is not NPU-specific; it
    applies to any static graph, including CPU/GPU with GGML_OPENVINO_FORCE_STATIC.

    Describe it as one mechanism with two modes behind GGML_OPENVINO_KV_SLICE
    (unset = each mode's default, 0 = off, 1 = on) and two predicates,
    ggml_openvino_{dynamic,static}_kv_slice_enabled(), used by the KV binding,
    the static graph shapes, the mask padding and the cache keys. The two old
    variables remain as deprecated aliases.

    * ggml-openvino: initialize all ModelExtraInputInfo fields for the HFA inputs

    dev_backend_openvino added shape_source/shape_axis to ModelExtraInputInfo; the HFA past-KV and mask
    initializers still listed four fields, which GCC rejects under -Werror=missing-field-initializers.

    * ggml-openvino: make new NPU defaults opt-in and fix static model conversion

    - Convert the static prefill and decode models one after the other, both
      before either is compiled. The decoders share the weight nodes;
      converting on two threads crashed intermittently inside libopenvino,
      and converting decode after the prefill compile gave wrong output on
      Bonsai-8B.
    - Compile prefill and decode in parallel only with
      GGML_OPENVINO_NPU_PARALLEL_COMPILE=1, since it raises the host memory
      peak.
    - Unfold NPUW function calls on the decode graph only with
      GGML_OPENVINO_NPU_DECODE_UNFOLD=1. The unfold default gave wrong output
      on gemma-4-E4B and slowed decode on several small models.
    - Stream NPU weight requantization only with
      GGML_OPENVINO_REDUCE_COMPILE_MEM or GGML_OPENVINO_MEMORY_OPTIMIZE, as on
      the other devices.
    - Fix a -Werror=missing-field-initializers build error in utils.cpp.
    - Declare ggml_openvino_get_npu_requant_type() only in
      npu/npu_quant_policy.h.
    - Docs: document the new switches, deprecate stateful execution, show
      stateless-only results in the validated models table, and update the
      validation system.

    * ggml-openvino: honour the -1 skip id in MUL_MAT_ID

    Upstream made an id of -1 in MUL_MAT_ID skip the slot and zero its row of
    the result. The translator passed -1 straight to GatherMatmul or Gather,
    so 169 of the 743 MUL_MAT_ID op tests on CPU (all skip=1) failed.

    A skipped slot now looks up a valid expert, and the result is multiplied
    by a 0/1 mask of the kept slots. For the packed MXFP4 path (Gather and
    MatMul), clamping the id to expert 0 is enough. GatherMatmul computes a
    wrong row when a token names the same expert twice, so the general path
    gives each skipped slot an expert that no other slot of the token uses.

    TopK indices are never -1, and model graphs take the ids from the
    router's TopK. Those graphs skip the handling and stay unchanged, so the
    GPU MoE fusion still matches them.

    * ggml-openvino: Gather rows before dequantizing a quantized embedding table

    1. Optimize gather to fetch row before dequant
    2. add NPU token_embd quantization option

    * ggml-openvino: select the NPU token_embd type with GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT

    GET_ROWS had two rewrites that gather the rows of a quantized table before
    dequantizing them: gather_compressed_rows, which runs on every device, and
    gather_before_dequant, enabled on NPU by GGML_OPENVINO_GATHER_BEFORE_DEQUANT.
    Both match the int4 and int8 chains built by make_int4_weights() and
    make_int8_weights(). Keep gather_compressed_rows and drop the second rewrite
    and its switch.

    The NPU type of a Q6_K token_embd.weight was set by two booleans,
    GGML_OPENVINO_TOKEN_EMBD_I8 and GGML_OPENVINO_TOKEN_EMBD_I4, and by
    GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT. Keep only the string variable, with the
    values f16 (default), q8_0_c, q4_0_128 and q4_0_c, so the int8 and group-128
    int4 settings stay available. An unknown value aborts, as for
    GGML_OPENVINO_NPU_REQUANT_POLICY, instead of silently using f16. The variable
    replaces the two booleans in the NPU compiled-model cache key.

    * align PAD pads to input rank in stateful mode

    * Gate the changes under is_stateful check to address review comment

    * ggml-openvino: lift Q tensors for non-RoPE layers to rank 4 in stateful mode

    * ggml-openvino: unify KV slicing under GGML_OPENVINO_KV_SLICE, on by default

    Replace ggml_openvino_dynamic_kv_slice_enabled() and ggml_openvino_static_kv_slice_enabled() with a single ggml_openvino_kv_slice_enabled(). KV slicing binds the KV cache, and sizes the KQ masks, at the attended length instead of the full context.

    GGML_OPENVINO_KV_SLICE now defaults to 1 on CPU, GPU and NPU; set it to 0 to disable. Dynamic graphs (CPU/GPU) bind a view over the cache without recompiling. Static graphs (NPU or GGML_OPENVINO_FORCE_STATIC) compile prefill at the aligned attended length and recompile when it changes; decode keeps its full-context graph.

    Remove the deprecated aliases GGML_OPENVINO_DISABLE_KV_SLICE and GGML_OPENVINO_NPU_KV_SLICE from the env var list, the model-cache settings, docs/backend/OPENVINO.md and the backend README.

    Behavior change: static KV slicing is now on by default on NPU, so prefill compiles one graph per distinct n_kv (multiples of 256).

    * ggml-openvino: keep stateful execution with -fa off

    Keep the V cache token-major in stateful mode instead of falling back to
    stateless execution. SET_ROWS appends to the state, and PERMUTE reads it
    back in the layout attention needs, so MUL_MAT does not transpose it.
    Dropping a reshape in grouped attention lets OpenVINO fuse it into SDPA.

    For padded V caches, slice after the transpose so the cache is read in
    place, without a copy that is recompiled for every KV length.

    This removes the stateless fallback for -fa off.

    * ggml-openvino: Add the stateful check to make the gate safer

    * ggml-openvino: support gemma-2 soft-capped attention

    Gemma-2 caps its attention scores as softcap * tanh(scores / softcap)
    before the softmax. FLASH_ATTN_EXT now applies the cap on the manual
    attention path, as SDPA has no soft-capping, and the decoder looks
    through the SCALE, TANH, SCALE chain of -fa off when it matches
    attention.

    One helper finds the K cache of an attention for both the
    sliding-window pre-pass and the main pass. The pre-pass copy did not
    see through the chain, so the sliding-window layers used the
    full-attention sizes once the context exceeded n_swa + n_ubatch.

    Soft-capped attention keeps the flat KV cache layout: with the 4-D
    layout the GPU plugin fails in its oneDNN path.

    * ggml-openvino: enable NPU Level Zero host tensors by default

    Back NPU buffers with Level Zero host memory unless GGML_OPENVINO_NPU_L0_HOST_TENSORS=0.
    The plugin imports these buffers without a host-to-device copy each infer; on phi-4-mini
    Q4_0 (pp3072+tg512) decode goes from 7.2 to 10.6-10.8 t/s. Allocation failures still fall
    back to plain host memory.

    * ggml-openvino: enable NPU fast prefill mask by default

    GGML_OPENVINO_NPU_FAST_MASK now defaults to 1; set it to 0 to fall back to pad_input() + set_zero_diagonal().

    The fast path builds each static prefill mask chunk directly in the OpenVINO input tensor (row memcpy of the valid prefix, -inf padding, zero diagonal), avoiding a temporary chunk_size x context_size vector and a full-tensor copy per chunk. The result is bit-identical to the fallback path.

    * ggml-openvino: add opt-in exact zero points for asymmetric weights

    Asymmetric quantized weights (Q4_K, Q4_1, Q5_1, and non-requantized
    Q5_K) represent each group as `w = d * q - m`. The backend currently
    converts these to u4/u8 weights with an f16 scale and an integer zero
    point `round(m / d)`, which can shift all weights in a group by up to
    half a quantization step.

    Add `GGML_OPENVINO_EXACT_ZERO_POINT=1` to preserve the zero point in f16
    instead, using the same path already used for MoE expert weights.

    This is disabled by default because, with current OpenVINO releases, an
    f16 zero point moves many GPU fully-connected layers off oneDNN,
    resulting in slower prompt processing.

    * ggml-openvino: rename npu/ to device/ and openvino/pipelines/ to openvino/pass_registry/

    npu/ holds device-specific backend behaviour (NPU compile config and
    requant policy), and pipelines/ only registers passes per device; the new
    names describe what the folders contain. No functional change.

    Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

    * docs: document NPU self-contained model cache and shared weights bank

    Add GGML_OPENVINO_SELF_CONTAINED_BLOB, GGML_OPENVINO_SELF_CONTAINED_MMAP and GGML_OPENVINO_NPU_SHARED_BANK to the OpenVINO environment variable table, and a note describing the export/import workflow and the requirement to keep NPU settings identical between runs.

    * ggml-openvino: remove experimental GGML_OPENVINO_NPU_KEEP_Q4_0

    Keeping native Q4_0 (group-32) on NPU compiles but hangs the device at inference (ZE_RESULT_ERROR_DEVICE_LOST), so the option was never usable. NPU Q4_0 weights now always go through GGML_OPENVINO_NPU_REQUANT_POLICY. Drop the env var, the requant bypass, its self-contained blob fingerprint term and the docs entry.

    * ggml-openvino: Update PERMUTE and CONCAT dynamic dim axis selection, Keep fp32 precision for sum-of-squares

    All changes of https://github.com/ravi9/llama.cpp/pull/358 as one commit. The PR branch had
    dev_backend_openvino merged into it, so its commits could not be replayed one by one when
    dev_backend_openvino was rebased onto upstream master.

    Commits of the PR:
    - Update fix to just maintain fp32 precision instead of modifying the graph
    - Limit the fp32 precision spreading in the graph by fencing it with Convert. Prevents perf degrade.
    - Update OPENVINO.md
    - Update flash_attn_ext.cpp
    - Update utils.cpp

    * enable flash_attn with sink input

    * ggml-openvino: fix FLASH_ATTN_EXT input detection and sink scale

    translate_flash_attn_ext read get_input_names()[3] and [4] on every
    FLASH_ATTN_EXT node, also on nodes with only 3 or 4 inputs. The
    out-of-bounds read crashed test-thread-safety and
    test-speculative-rejection-sampling. The inputs are q, k, v, then the
    optional mask and sinks, and supports_op rejects sinks without a mask,
    so the input count tells which inputs are present.

    With sinks, the scale was replaced with 1.0. llama.cpp passes the
    attention scale in op_params[0] and adds the sinks separately, so keep
    op_params[0]. The "scale" input branch is removed: ggml's src[4] is
    only ever the sinks.

    Only SDPA applies the sinks. The manual GQA path and the NPU HFA
    prefill path drop them, so attention with sinks now always takes the
    SDPA path. SDPA has no logit soft-capping, so supports_op rejects sinks
    together with soft-capping.

    * ggml-openvino: accept flash attention on layers without RoPE

    On layers without RoPE (SmolLM3: every 4th layer), Q reaches
    FLASH_ATTN_EXT through PERMUTE -> VIEW -> RESHAPE. The support check
    rejected this pattern, causing these layers to fall back to CPU and the
    split graph to fail with -fa on/auto.

    * ggml-openvino: keep only weight-divisor DIVs on CPU

    The GPU DIV guard for per-channel scale vectors (`MPT`'s `act_scales`) currently checks only the divisor's shape. During single-token Qwen2-MoE decode, the shared-expert gate `silu(x) / x` has a `[1,1,1,1]` divisor that matches this shape, causing its DIVs to be kept on the CPU.

    As a result, every decode step incurred CPU execution and Qwen1.5-MoE decoded at only ~2 tok/s.

    Apply the guard only when the divisor is a model weight, so runtime tensors such as the Qwen2-MoE gate divisor can remain on the GPU.

    * ggml-openvino: Qwen MTP and multi-seq optimization with new OV ops

    All changes of https://github.com/ravi9/llama.cpp/pull/370 as one commit. The PR branch had
    dev_backend_openvino merged into it, so its commits could not be replayed one by one when
    dev_backend_openvino was rebased onto upstream master.

    Commits of the PR:
    - openvino : support Qwen MTP decoding
    - openvino: Optimize TopK
    - openvino: remove single-slot recurrent cache shortcuts
    - openvino: fuse generic recurrent caches and remove shape waits
    - openvino: include dense Q8 weights in all INT4 requantization
    - openvino: use two GDN outputs for snapshots
    - docs: explain native OpenVINO recurrent cache contracts
    - docs: clarify Qwen3.5 purpose of native OpenVINO cache ops
    - llama: skip unallocated hybrid graph inputs
    - quantize: preserve Qwen hybrid layer metadata when pruning
    - openvino: emit fused recurrent computation and rotate device caches
    - openvino: limit KV slicing to attention caches and share compiled blobs
    - tools: add natural-text server benchmark and OpenVINO profile summary
    - docs: record fused-cache branch split and UD validation
    - server-bench: preserve native token counts and save request timings
    - openvino: document MTP regression and host USM alias recovery
    - openvino: record large-context allocation diagnosis and default-batch validation
    - server-bench: count buffered native tokens and reject stream errors
    - openvino: document UD split and unified server benchmarks
    - openvino: record Vulkan long-context baseline and GPU-fit controls
    - openvino: finalize UD benchmark report and MTP comparison
    - openvino: use stable in-place recurrent caches and optional separate convolution
    - openvino: accept matching empty axes in DIV repeat translation
    - openvino: remove the obsolete cache rotation environment setting
    - openvino: validate imported KV results in their four-dimensional layout
    - docs: record the bounded in-place KV and convolution benchmark rerun
    - docs: complete default-batch OV unified-KV 32K benchmarks
    - openvino: remove the separate recurrent convolution path
    - openvino: handle zero-logits batches and changing sampler sequences
    - docs: compare default and large Qwen3.6 benchmark batches
    - docs: use the historical Vulkan default-batch reference consistently
    - server-bench: use an integer histogram bin count
    - benchmarks: summarize current OV unified MMLU timings
    - docs: record cache-only unified KV MMLU results
    - openvino: preserve the TopK index port in GET_ROWS
    - openvino: reuse the MTP sampling model across active slots
    - openvino: expose selection flags and offsets in graph dumps
    - docs: consolidate Qwen OpenVINO defaults and performance evidence
    - docs: retain continuous-request MTP benchmark results
    - docs: compare stable MTP with the non-MTP serving baseline
    - openvino: isolate GDN snapshots and skip naive cache fusion
    - ggml: document Qwen3.6 GenAI benchmarks
    - ggml-openvino: keep PR changes inside the backend folder

    * ggml-openvino: drop the extra semicolons after OPENVINO_OP

    OPENVINO_OP expands to member function definitions that end in a closing
    brace, so a semicolon after it is an empty declaration inside the class.
    GCC 15.2 reports it with -Wextra-semi, which ggml enables for GCC 8.1 and
    newer, so a build with LLAMA_FATAL_WARNINGS=ON (the default in ci/run.sh)
    fails in gated_delta_net.hpp. The CI compilers do not report it.

    recurrent_cache_update.hpp has the same pattern twice. It is compiled only
    when the OpenVINO package provides the recurrent cache-update ops.

    * ggml-openvino: make q8_0_c the default NPU token_embd requant type

    Default of `GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT` changed from `f16` to `q8_0_c`.
    Applies only to NPU and only to a Q6_K `token_embd.weight`; other devices and
    quantization types are unchanged.

    * ggml-openvino: do not treat the recurrent state gather as out_ids

    The output_len scan now runs for every model and takes the first GET_ROWS
    whose index is a graph input. On Qwen3.5 that is the recurrent state gather
    of build_rs, which indexes the inp->s_copy leaf directly, so output_len
    became 1. In static (NPU) mode the prompt model then returned one logits
    row per chunk, and llama-perplexity read stale rows (PPL about 240000 on
    Qwen3.5-2B). The same predicate also gave the s_copy input the out_ids
    shape and data in static mode.

    is_output_idx now skips GET_ROWS that read a recurrent state cache. The old
    predicate stays as is_get_rows_input_idx for the dynamic-dim bookkeeping,
    so the s_copy leaf keeps its dynamic length there.

    * ggml-openvino: prefer the size-preserving dim when a RESHAPE repeats a stride

    The RESHAPE dynamic-dim rule matches the source's dynamic stride against
    the result's strides. A size-1 dim repeats the next dim's stride: Gemma 4
    E2B has one KV head, so [head_size, n_tokens] -> [head_size, 1, n_tokens]
    has the same stride on dims 1 and 2. The first match picked the size-1 head
    dim, so the K/V views lost their dynamic token dim and became static
    reshapes with the first graph's token count. Every later graph with another
    token count failed.

    Prefer the stride match that keeps the dynamic size, and fall back to the
    first stride match for reshapes that change it.

    * ggml-openvino: correct the RESHAPE dynamic-dim comment for size-1 dims

    The comment above the RESHAPE stride match in compute_node_dynamic_dims()
    still said the match is unambiguous because result strides are strictly
    monotone. A size-1 dim has the same stride as the dim after it
    ([head_dim, T] -> [head_dim, 1, T] with one KV head), so it is not.

    Explain the size-1 case in the block comment, drop the duplicate comment
    inside the loop, and name the source dim's size once.

    * examples: add llama-chat-replay

    Replays JSONL prompts like OpenVINO GenAI chat.py (-pf, -i chat history, greedy, -o, -pc metrics). The first prompt is prefilled plus one decode step during load (Warmup time) and the memory is cleared, so compile/import is counted in Load time as in GenAI.

    * ggml-openvino: make the NPU lazy host threshold configurable, default 2 GiB

    With a large context the KV cache (~1 GiB at ctx 8K) crossed the 1 GiB threshold, lost its Level Zero host tensor in strict warm-mmap mode and made decode copy the KV every token. Default to 2 GiB so only the larger prefill compute buffer stays demand-zero, and allow overriding it with GGML_OPENVINO_LAZY_HOST_THRESHOLD_MB.

    * ggml-openvino: fix NPU HFA prefill with ordinal-suffixed KV input names

    KV cache inputs are now named with a graph-ordinal suffix (cache_k_l0#l12), but flash_attn_ext looks the HFA past input up as ggml_hfa_past_cache_k_l<layer>, so GGML_OPENVINO_NPU_HFA_PREFILL failed with 'ggml_hfa_past_cache_k_l0 not found in tensor map'. Name the past inputs after the bare cache name and resolve the source cache by that name when binding.

    * ggml-openvino: keep the NPU prefill model on cache hits after decode

    With KV slicing, decode calls overwrite the shared decoder's attention size with the full context, so the next prefill compared that against its sliced size, missed the cache and rebuilt the decoder and both infer requests (plus a cold first chunk). Remember the attention sizes the prefill model was built with and compare against those.

    * ggml-openvino: bind NPU HFA past KV zero-copy in the cache layout

    The HFA prefill past K/V inputs were [1, heads, past, head_size] copies of the cache, relaid out per chunk on the host and allocated per decoder (256 MiB on phi-4-mini at 3k). Declare them in the ggml cache layout [1, past, heads, head_size], bind the first past rows of the cache directly and transpose in the graph; NPUW HFA reads the tiles from the BSHD past (needs the matching NPUW HFA support). HFA input ABI v4.

    * ggml-openvino: size the NPU decode KV window by GGML_OPENVINO_NPU_DECODE_BUCKET

    Static decode attended the whole context, so every token read and wrote the full KV cache. With a bucket base set, decode attends the smallest base*2^k rows covering the next position (clamped to ctx): the KV inputs and masks are sliced to the window, the compiled pair's decode graph is built at the window the prefill reaches, and other windows are built or imported from the self-contained cache on first use with the pair's settings and bank. With self-contained blobs, keep the token_embd weight node for the later window models so they reuse the pair's bank entry instead of embedding a second vocab copy. The static build/convert lambdas become functions shared with the window path.

    * ggml-openvino: allocate the NPU KV cache on demand with GGML_OPENVINO_NPU_KV_ON_DEMAND

    The NPU KV buffer was committed for the whole context up front. With the option set, the KV buffer is swapped for demand-zero memory at its first KV tensor and every KV tensor gets its own storage in a registry, grown in base*2^k rows (base = decode bucket or 1024) as the static graphs reach further. Host accessors, KV inputs and outputs, sliced KV views and the HFA past bind that storage; when it moves, idle requests bound to the old storage are recreated so it is released.

    With decode buckets and KV slicing the storage also takes the layout GenAI uses on NPU: K is [heads][rows][head_size] and V is [heads][head_size][rows], each bound whole to prefill and decode. SET_ROWS scatters new K rows on axis 2 and V rows on axis 3, the attention permute only cuts the window, the HFA past is the storage itself, and attention multiplies the probabilities by V^T with transpose_b instead of SDPA. A chunk-sized scatter on the innermost axis is slow on NPU, so the HFA prefill returns each chunk's V^T rows and the host copies them into the cache. Host reads, writes and fills go through a layout-aware copy, storage growth re-packs each head, and the layout is part of the model cache fingerprint.

    * ggml-openvino: NPU weightless blob cache with one shared weights file

    GGML_OPENVINO_NPU_WEIGHTLESS_BANK=1 (with GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR) exports weightless prefill/decode NPUW blobs plus one <prefill>-<decode>.weights.bin. Weight Constants are laid out once by data pointer and their WeightlessCacheAttribute is retagged with the real offsets. Warm import uses ov::weights_path, and NPUW dedups both models into one bank through the LazyTensor hashes, so the OpenVINO weights_bank duplicate-uid patches are not needed. Decode KV-window models resolve the shared weights through the pair's file, or get their own file when the pair was imported. Also drop the old blob and manifest before publishing a re-export: std::rename does not replace an existing file on Windows.

    Based on anikulk/llama.cpp 4ab270aa9.

    Co-authored-by: Anisha Kulkarni <anisha.dattatraya.kulkarni@intel.com>

    * ggml-openvino: prebuild all NPU decode KV windows with GGML_OPENVINO_NPU_DECODE_PREBUILD

    Decode windows were built lazily when the sequence first grew into them, which stalled generation for a compile without a cache and made a strict-mmap import fail for windows the export run never reached. GGML_OPENVINO_NPU_DECODE_PREBUILD replaces GGML_OPENVINO_NPU_DECODE_BUCKET: 0 (default) keeps decode over the whole context; 1 sizes decode by 1024*2^k KV windows and the call that builds or imports the prefill/decode pair also builds or imports the decode graphs of every other window up to the context, from the pair's graph and compute params. Requests are still created on first use. Window building is split out of the request lookup so both paths share it, and window ports are checked against the pair's decode ports.

    * Revert "examples: add llama-chat-replay"

    This reverts commit 4eded2c52c2df9fa1251673a523f8c7cded20d7c.

    * ggml-openvino: initialize the new compiled-graph and build-args fields

    ov_compiled_graph and static_build_args gained members, but three brace
    initializations still listed only the old ones. GCC reports
    -Wmissing-field-initializers there, and CI builds with warnings as errors.
    Set the fields by name, and pass {} for weights_path, which the window
    build sets afterwards.

    * ggml-openvino: fix gemma-4 when only the output rows reach the KV-shared layers

    With one sequence, gemma-4 now drops the rows that are not output before
    its first KV-shared layer. Those layers take their positions through
    GET_ROWS(inp_pos, out_ids), and the backend mishandled three things:

    - The positions are reshaped from [n] to [1, n]. That matched RESHAPE
      case 1, which marks the wrong dim dynamic. Use case 3, which keeps
      the source dim as dim 1.
    - ROPE shared one sin/cos table per frequency setting, so the KV-shared
      layers reused the table built from the full inp_pos. Add the position
      input to the cache key.
    - compute_llm_params took input_len from the last attention op. That is
      now a KV-shared layer with only the output rows, so the static (NPU)
      path padded prefill to the wrong length. Take the largest query length.

    Stateful execution still fails on the per-layer embedding view and is
    not covered here.

    * ggml-openvino: keep the inp_pos layout for gemma-4's gathered positions

    A batch can have no output rows, for example a prompt chunk that
    llama-cli processes before its last tokens. gemma-4 then reshapes an
    empty position gather for the ROPEs of its KV-shared layers. The size
    checks of the RESHAPE cases cannot place a dim of size 0, so the
    positions got the wrong layout and the ROPE multiply failed on CPU and
    GPU. A RESHAPE that feeds the ROPE positions now always takes the
    inp_pos layout [1, 1, 1, n].

    * ggml-openvino: enable stateless GPU execution for MLA models

    The per-head MLA weights (`attn_k_b` / `attn_v_b`) are 3D quantized tensors. Their `MUL_MAT` operations were falling back to the CPU, splitting the graph at every layer and causing GPU execution to fail.
    Accept these tensors in the GPU support check and address the following issues exposed by enabling GPU execution:

    * Quantization: Use exact zero points only for MoE expert weights. U4 weights with F16 zero points fail in batched GPU MatMul.
    * Dynamic VIEW dimensions: Select the correct dimension when a size-1 dimension shares its stride (e.g. `k_pe`), and match KV cache views by stride.
    * KV cache V view: Translate the V view of the K cache as a reshape while preserving the dynamic cell axis.
    * Flash Attention: Accept the MLA Q (`CONCAT`) pattern and V views derived from another view.

    Limitations: The GPU plugin fails on batched compressed weights with static shapes. Keep these unsupported in op tests and `GGML_OPENVINO_FORCE_STATIC`.

    * openvino : use writable recurrent cache inputs

    Assisted-by: Codex

    * ggml-openvino: fix decode crash for Qwen3.5 4B

    When the dynamic size is 1, the code now keeps the innermost match. When the size is greater than 1, it still uses the MLA rule, so the MLA k_pe fix still works for multi-token graphs.

    * ggml-openvino: keep the GDN state in the OpenVINO layout

    Adds an opt-in switch, GGML_OPENVINO_GDN_NATIVE_STATE. When it's on, the GATED_DELTA_NET recurrent state stays in the layout the OpenVINO GatedDeltaNet op uses, [B, H_v, key_dim, value_dim], instead of the ggml layout [B, H_v, value_dim, key_dim]. That removes the two state transposes the translator adds on every call.

    * ggml-openvino: support Ling-mini-2.0 group-routed MoE

    Ling-mini-2.0 selects experts by group, but the group-selection graph
    used ops that were not supported by the backend. This left 370 nodes on
    the CPU and reduced decode to ~1.5 tok/s.

    * Translate the batched 4D `GET_ROWS` gather used for group scores.
    * Support `SET_ROWS` for compute tensors, not just KV caches.
    * Preserve the dynamic token dimension in `FILL`.
    * Skip the `DIV` repeat when dimensions already match.
    * Use the tracked token dimension for `RESHAPE` / `GET_ROWS` op cases,
      so a graph compiled during one-row warmup also works for ubatches
      with more output rows.

    * ggml-openvino: disable output staging

    The backend stages small outputs (up to 4 MB) in a USM host buffer
    and binds larger outputs directly to the ggml buffer. When the number
    of outputs changes between calls to the same graph, as in
    llama-perplexity, an output port can switch from USM host memory to
    plain host memory.

    The OpenVINO GPU plugin mishandles this switch for some dynamic
    outputs, causing out-of-bounds writes on subsequent calls. This
    results in `CL_OUT_OF_RESOURCES`. For example,
    `llama-perplexity -c 2048 -ub 512` fails on the fourth ubatch with
    Gemma-2, Gemma-4, Phi-3.5, Phi-4-mini, and Qwen3.5.

    Disable output staging until the plugin issue is fixed, so each output
    is always bound the same way. Input staging remains enabled.

    * ggml-openvino: match attention masks by identity

    Gemma-4 prunes non-output rows before its KV-shared layers. These
    layers select the corresponding mask rows with GET_ROWS(mask, out_ids).

    Mask parameters were named by their consumers, but GET_ROWS is not
    recognized as a mask consumer. As a result, the sliding-window layers
    selected their rows from the full-attention mask instead of the
    sliding-window mask. This either caused the mask add to fail
    broadcasting beyond the sliding-window cache (d1024 and above with
    -ub 512) or allowed attention beyond the sliding window at shorter
    contexts.

    Track graph-input attention masks by identity and assign them the
    same parameter name for every consumer. The SWA pre-pass now keeps
    each layer's own attention mask, as the KV-shared layers read the
    same cache with mask rows derived from it.

    * ggml-openvino: preserve input mask names and window sizes

    KV-shared layers appear at the end of the graph, and their derived
    masks are not graph inputs. Processing these masks overwrote the
    names and sliding-window sizes recorded from the original input
    masks with empty values.

    Preserve the names and window sizes of graph-input masks so the
    attention dimensions continue to be derived from the original
    mask shapes.

    * ggml-openvino: accept empty tensors in stride checks

    In Gemma-4 batches without output rows, the attention tensors of
    KV-shared layers are empty. Their strides beyond the empty
    dimension are zero, causing has_strides_on_element_grid() to reject
    them.

    This sent SOFT_MAX, the KQV MUL_MAT, and CONT to the CPU with
    -fa off. It also caused the per-layer input crossing the graph
    split to be bound with the wrong layout.

    Accept empty tensors in the stride check. Their contents are never
    read, so the remaining strides do not need to satisfy the element
    grid constraints.

    * ggml-openvino: classify transposed V caches using v_trans

    With -fa off, the V cache is transposed. When the attended length
    is the whole cache, as with a prompt that fits in one ubatch, its V
    view is contiguous and was mistaken for the K cache layout.

    The K layout path does not keep only the rows of the layer's own V
    size. Gemma-4 allocates 512 V rows in every layer, the size of its
    widest layer, and its sliding-window layers use the first 256. These
    layers then attended with all 512 rows, and prompts fitting in a
    single ubatch failed with a broadcast error.

    Use v_trans to identify the V cache layout instead of relying on
    contiguity.

    * ggml-openvino: use per-layer Split in stateful execution

    Stateful execution was excluded from the per-layer Split path
    used for Gemma-4's per-layer embedding slices. The generic
    re-slicing path therefore cut the token axis instead of the layer
    axis.

    Allow stateful execution to use the per-layer Split and adjust
    its axis by the rank dropped during stateful graph construction.

    * ggml-openvino: centralize stateful KQ mask lookup

    Stateful execution rebuilds the KQ masks from positions. The
    existing replacement logic applied to every mask input of
    FLASH_ATTN_EXT and SOFT_MAX, including masks derived through
    GET_ROWS.

    For Gemma-4's KV-shared layers, this replaced the row-selected mask
    with the full rebuilt mask, losing the output-row selection. As a
    derived mask's name does not name the sliding window, these layers
    also got the full-attention mask instead of the sliding-window one.

    Add get_kq_mask_input() to replace only the actual mask parameters.
    Use it for FLASH_ATTN_EXT, SOFT_MAX, and GET_ROWS, so the derived
    masks select their rows from the rebuilt masks, and the lookup is no
    longer duplicated.

    * ggml-openvino: Removed duplicated if clause.

    * fix flash_attn with sink conversion to OV SDPA

    * ggml-openvino: drop the unused sink-without-mask path in FLASH_ATTN_EXT

    supports_op already rejects FLASH_ATTN_EXT with sinks but no mask, so the
    translator never sees a 4-input node whose fourth input is the sink; the
    name check could only reach the assert after it. Restore the input-count
    detection, fetch the sink where it is used, and format the sink reshape.

    * ggml-openvino: fix dimensions bug in stateful recurrent-state remainder writeback

    * ggml-openvino: move scalar scales before weight MatMuls on GPU and NPU

    GPU and NPU run the graph in f16. Granite multiplies the attention and FFN
    outputs by residual_scale right after their output projections. For some
    tokens the ffn_down output is far above the f16 maximum (65504), so it
    becomes inf and every later layer turns NaN: granite-4.0-350m and
    granite-4.0-1b answer with repeated tokens on GPU, and llama-perplexity
    reports NaN.

    Add MoveScaleBeforeMatMul. It rewrites Multiply(MatMul(x, W), s), with a
    scalar constant |s| < 1 and a constant W, as MatMul(Multiply(x, s), W). The
    result is the same up to rounding and the MatMul output stays in range.
    Register it for GPU and NPU; CPU runs in f32.

    * ggml-openvino: translate convolution rollback snapshots

    Assisted-by: Codex

    * ggml-openvino: run GDN rollback snapshots in OpenVINO on CPU

    Speculative decoding that rolls back recurrent state (MTP, EAGLE3, DFlash)
    builds GATED_DELTA_NET with K = n_rs_seq + 1 state snapshots and copies them
    into the recurrent cache. supports_op accepted the snapshots only on GPU, so
    on CPU the GDN ran in ggml while the copy, whose destination is the OpenVINO
    cache buffer, ran in OpenVINO. The copy read the packed ggml GDN output
    wrongly and stored wrong states without an error: test-llama-context
    rollback failed and MTP accepted no drafts.

    translate_gdn_snapshots builds plain GatedDeltaNet ops on token prefixes and
    does not depend on the device. Accept the snapshots on every device with
    dynamic shapes and keep rejecting them only for static execution (NPU and
    GGML_OPENVINO_FORCE_STATIC). Update the two doc sentences that said the
    snapshots need the GPU.

    * ggml-openvino: abort when GDN snapshots would run in ggml

    The snapshot copy writes into the recurrent cache in the OpenVINO buffer, so
    it always runs in OpenVINO. When OpenVINO rejects the GATED_DELTA_NET that
    produces the snapshots (per-key gating as in Ling-3.0, a permuted v, static
    execution), the GDN runs in ggml and the OpenVINO side reads its packed
    output wrongly. Depending on the shapes, this silently corrupts the cache or
    fails inference with a reshape error. Abort when the graph is scheduled,
    with the reason.

    * ggml-openvino: share one weights file and NPUW bank across all NPU prefill/decode pairs

    With GGML_OPENVINO_NPU_WEIGHTLESS_BANK, every prefill/decode pair (one per KV window, e.g. llama-server's startup probe and a long prompt) wrote its own weights file and used its own NPUW bank, so import kept the weights resident once per pair (+2.1 GiB for phi-4-mini). Pairs now use a model-level <wfp>-<wfp>.weights.bin and bank; the weights file records a layout/content hash and is reused when identical, with a per-pair file and bank as fallback. Old caches are invalidated.

    * ggml-openvino: export and preload NPU prefill blobs for every KV window

    With head-major KV the prefill also runs at a KV window picked by prompt length, so an import-only run failed for prompts whose window was never exported. After a pair is exported, also export the prefill model of every other 1024*2^k window (weights resolved through the pair's shared weights file), and on import preload them into a process-wide cache of imported blobs that pair and decode-window builds reuse. Cache the source fingerprint of weights outside OpenVINO buffers, which made each model fingerprint cost ~9 ms. A window switch now costs ~60 ms instead of ~250 ms.

    Rename GGML_OPENVINO_NPU_DECODE_PREBUILD to GGML_OPENVINO_NPU_BLOB_PREBUILD since it now covers prefill as well as decode windows.

    * ggml-openvino: drop invalid prefill blobs and clear blob caches on backend free

    The prefill preload imported each KV window's cached blob without a try, so
    one blob that failed to import failed the graph compute, and the blob stayed
    in the cache for every later run. Remove the blob and its manifest and log a
    warning, as the pair import does; the window is built when a prompt first
    needs it.

    The imported-blob map and the fingerprints of weights outside OpenVINO
    buffers lived until the process exited. A process that loads models one
    after another kept every compiled model resident, and a later model whose
    weight landed at a freed weight's address reused that weight's fingerprint.
    Clear both with the shared compiled-model cache when the last backend is
    freed.

    Also apply clang-format to the changed lines.

    * ggml-openvino: enable NPU KV windows by default

    With GGML_OPENVINO_NPU_BLOB_PREBUILD unset, every NPU decode step ran a
    static graph that attends the whole context, so decode was slow for any
    sequence much shorter than -c. On a Lunar Lake NPU, Llama-3.2-1B decodes at
    0.79 t/s with -c 4096 and 0.20 t/s with -c 16384, against 11.1 and 12.0 t/s
    with KV windows. Each window adds one decode graph to compile at load: with
    -c 4096, load goes from 11.1 s to 14.6 s for Llama-3.2-1B and from 61.2 s to
    81.6 s for Llama-3.1-8B, whose decode goes from 0.20 to 0.82 t/s.

    Default GGML_OPENVINO_NPU_BLOB_PREBUILD to 1; 0 keeps the full-context
    decode graph. Update the env table and the Boolean-flag note, which said an
    unset flag is always disabled.

    * ggml-openvino: add POOL_1D and allow F32 EXP on GPU

    * skip empty tensors

    * ggml-openvino: detect the gated LFM2 conv input as a recurrent sequence

    A reshape whose output reaches SSM_CONV through concat(conv_state,
    transpose(x)) gets op_case 6, which keeps the active sequence count apart
    from the token count, as the conv state does. The LFM2 and LFM3 short
    convolutions gate x first, so the transposed tensor is
    mul(view(x), view(x)) and the check missed it. The reshape got op_case 5
    instead, and:

    - in stateful mode, which drops the leading dim for op_case 5, TRANSPOSE
      got a rank-3 input with a 4-axis order and graph conversion failed;
    - with more than one sequence, the concat with the conv state failed.

    Accept the gated input too. The output for a single stateless sequence
    does not change.

    * ggml-openvino: always stream weight requantization

    Requantizing Q6_K/Q5_K weights to 8-bit (or F16 for some tensors) previously converted the entire tensor to f32 first. For large output or embedding matrices, this required 2–2.5 GB of temporary memory.

    The chunked conversion path already processes 256 rows at a time and produces identical output. It was previously enabled by `GGML_OPENVINO_REDUCE_COMPILE_MEM` or `GGML_OPENVINO_MEMORY_OPTIMIZE`; use it by default whenever the target block size divides a row. Q4_1_64, which has no chunked quantizer, still converts the whole tensor.

    The variable retains its other effects.

    * ggml-openvino: allocate host buffers as untouched OS memory

    Host buffers (weights, KV cache and compute) were allocated with `ggml_aligned_malloc` and explicitly zeroed with `memset`. This made the entire weight buffer resident before any tensors were loaded.

    Allocate host buffers using `mmap(MAP_ANONYMOUS)` on Linux and `VirtualAlloc` on Windows instead. Both provide zero-initialized memory without touching the pages, allowing the OS to bring them into memory as tensors are written.

    This preserves the zero-initialization required by ggml-org/llama.cpp#20857 while avoiding unnecessary memory residency during model loading.

    Remove the cache-only allocation branches, the corresponding `ggml_aligned_free` path from the destructor and the `data_owned_by_ov` member it used, since they are no longer needed.

    * ggml-openvino: release GGUF file pages once a weight is converted

    Once `set_tensor` converts a weight into the backend buffer, its source pages in the mapped GGUF file are no longer needed for that buffer. Release these clean pages using `madvise(MADV_DONTNEED)` on Linux and `VirtualUnlock` on Windows. The mapping itself remains intact, and pages can be loaded again from the page cache or file if needed.

    Only release whole pages within tensor ranges belonging to read-only file mappings. Skip buffers that wrap the mapping itself.

    Keep locked pages resident. On Linux, `MADV_DONTNEED` fails on mlocked memory. On Windows, check page lock status with `QueryWorkingSetEx` and skip ranges containing locked pages, since `VirtualUnlock` would otherwise unlock them.

    * ggml-openvino: key weight constants by name for the NPU model cache

    The NPU compiled-model cache (GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR without the
    weightless bank) imports a blob with the freshly converted model as ov::hint::model,
    and NPUW finds each constant by its WeightlessCacheAttribute bin_offset. The keys came
    from a per-model counter, and a constant that already had a key kept it. Weight nodes
    are shared between models: between the NPU prefill and decode models, and across the
    process through the tensor extras. So a weight kept the key from whichever model
    converted it first.

    The decode model for a KV window is built both as the decode half of the
    prefill/decode pair and as a standalone window model, under one fingerprint, and the
    two builds got different keys. Importing the blob of the other build failed with
    "Couldn't find Constant in cache!", and the decode model was recompiled and exported
    again on every run. With GGML_OPENVINO_NPU_BLOB_PREBUILD on by default, this happens
    whenever a prompt reaches a larger KV window.

    Key weight constants by a hash of the weight name and their position in the weight's
    subgraph, above the range of the counter. Other constants are created per model and
    keep the counter.

    * ggml-openvino: drop CACHE_MODE=OPTIMIZE_SIZE from the NPU plugin cache

    With GGML_OPENVINO_CACHE_DIR set, the NPU compile config added
    CACHE_MODE=OPTIMIZE_SIZE next to NPUW_CACHE_DIR. With it, the NPU plugin
    compiles the NPUW submodels with weights separation, which fails for them:
    with NPUW folding, "No WeightlessCacheAttribute has been found in any of the
    model's Constant nodes"; without folding, a driver compile error. So every
    NPU run with GGML_OPENVINO_CACHE_DIR failed on its first compile.

    Keep the default cache mode. NPUW_CACHE_DIR still caches the compiled
    submodels.

    * ggml-openvino: load each self-contained NPU blob once per process

    A self-contained blob (GGML_OPENVINO_SELF_CONTAINED_BLOB) embeds its NPUW
    weights bank, and NPUW cannot load a blob into a bank that already holds
    those weights: Bank::read_and_add_tensor asserts. Two cases did this on
    every run that imported the cache:

    - Pairs that differ only in the prefill model share the decode model and
      its bank. A long prompt builds such pairs, and each one imported the
      decode blob again. The import failed, so the decode model was compiled
      and exported again for every pair, until the run ran out of memory.
    - The decode model of each KV window uses the bank of the pair's decode
      model, so the import of a window blob failed and stopped the run.

    Keep each self-contained model in the process once it is imported or
    compiled, and reuse it for every pair with the same fingerprint. Turn the
    KV windows off for self-contained blobs; the weightless bank keeps them.

    * ggml-openvino: update the Docker image GPU drivers to 26.40.40196.8

    The OpenVINO image pins the Intel GPU drivers at compute-runtime
    26.35.39758.10. With that driver, gemma-4-E4B hangs on the Lunar Lake
    GPU. With 26.40.40196.8 it runs.

    Take the GPU driver pins from the compute-runtime 26.40.40196.8 release:
    IGC v2.42.6, compute-runtime 26.40.40196.8 and gmmlib 22.10.2.

    * ggml-openvino: fix long NPU prompts failing with KV slicing

    With GGML_OPENVINO_KV_SLICE, on by default, a static prefill is compiled
    at the attended KV length that llama reports for the ubatch. That length
    grows with every ubatch of a long prompt, and each new length compiled
    another prefill/decode pair, with GGML_OPENVINO_NPU_BLOB_PREBUILD (on by
    default) also its decode KV windows. The shared compiled cache keeps
    every pair until the backend is freed.

    Each loaded prefill model keeps NPU memory for its inferences. On a Lunar
    Lake NPU, a prefill model loaded next to two or three others cannot get
    it: DRM_IOCTL_IVPU_BO_CREATE fails with "Buffer size is too big", and
    pfnAppendGraphExecute returns ZE_RESULT_ERROR_UNINITIALIZED. With -c 4096
    and the default -ub 512, a 2200-token prompt failed on smollm2-1.7B,
    Phi-4-mini, Ministral-3b and gemma-3-4b.

    - Round the attended length of a sliced static prefill up to a power of
      two, clamped to the context. A growing sequence now compiles a prefill
      model when its attended length doubles, not at every ubatch. The added
      rows are masked, so the output does not change.
    - When only the prefill attention size changes, the new pair takes over
      the decode models (base and KV windows) of the entry's current pair,
      because they do not depend on that size, and the current pair is
      dropped before the new prefill model compiles. A pair that another
      backend may use stays cached. Self-contained pairs are exported as one
      blob pair, so they keep their own decode models.
    - The taken-over KV window that covers the next decode position becomes
      the base decode model, as in a pair built at that length, so the first
      decode step does not have to create an infer request for it.

    * ggml-openvino: pre-scale the GDN q input by default on the NPU

    The NPU GatedDeltaNet kernel writes its output in FP16 and flushes
    values below 6.1e-5 to zero, and the RMS norm after it amplifies the
    loss. GGML_OPENVINO_GDN_Q_SCALE lifts the output above that floor, but
    it was off by default. On a Panther Lake NPU with the compiler bundled
    in OpenVINO, Qwen3.5-2B scores a perplexity of 14.77 without it and
    13.00 with it (12.93 for the same weights on CPU), and its greedy chat
    no longer loops while thinking.

    - Enable the pre-scale by default with the factor 64.
      GGML_OPENVINO_GDN_Q_SCALE=0 turns it off.
    - Scale the eps of the norm only when the GDN that feeds it is
      pre-scaled. A norm that reads the GDN output directly got the scaled
      eps on every device, while q is scaled only on the NPU. With the
      variable set, Qwen3.5 gave wrong output on CPU and GPU.
    - Add the factor to the model cache fingerprint of NPU graphs with a
      GATED_DELTA_NET, so a cached model is not reused across settings.
    - docs: update the GGML_OPENVINO_GDN_Q_SCALE row and note.

    * openvino-ggml: Decompose the GDN for LNL

    1. Automatically decompose LNL GDN prefill into chunked MatMuls; retain the native op for decode.
    2. Isolate lowering selection, fix positive-gate handling, and add overflow guards.
    3. Update cache identity and add chunk-boundary regression cases.

    * ggml-openvino: fix GDN decomposition in decode and on long prompts

    Two problems remained with the GATED_DELTA_NET decomposition on a Lunar
    Lake NPU.

    The decomposition covered only multi-token (prefill) graphs, and decode
    kept the GatedDeltaNet op. Under Linux the decode kernel loses the
    device as well: after a successful prefill, the first generated token
    failed with ZE_RESULT_ERROR_DEVICE_LOST in llama-completion, llama-bench
    and llama-server.

    The chunk masks were new constants in every converted model. The NPUW
    weights bank is shared by all compiled models (NPUW_WEIGHTS_BANK=shared)
    and identifies a constant by its data pointer, also after the model
    that owned it is gone. When a longer sequence needs a second prefill
    model, its masks can get the addresses of freed masks of the first one
    and are then read as those. The Qwen3.5-2B perplexity over 512 tokens
    changed between 12.99 and 56.94 from run to run, and a 1900-token
    prompt gave garbage.

    - Lower a single-token GATED_DELTA_NET to one recurrence step when the
      decomposition is on: the chunk formulas for a chunk of one token,
      where P = 1 and the decay is 1.
    - Keep one instance of each chunk mask for the whole process.
    - test-backend-ops: add a single-token case with two sequences, repeated
      heads and a positive gate.
    - docs: GGML_OPENVINO_GDN_DECOMPOSE covers single-token graphs.

    * openvino-ggml: Enhance the safe env parsing

    * ggml-openvino: keep the GDN q pre-scaling on when its variable is unset

    ggml_openvino_gdn_prescale_enabled() returned false when
    GGML_OPENVINO_GDN_Q_SCALE is unset, which switched the default-on
    pre-scaling off. Qwen3.5-2B on a Panther Lake NPU went from a perplexity
    of 13.00 back to 14.77.

    - Treat an unset variable as enabled. Zero, negative and non-numeric
      values still turn the pre-scaling off.
    - Comments: ggml_openvino_getenv_int() no longer goes through std::atoi.
    - docs: a boolean flag with a non-integer value is no longer always
      disabled; a flag whose default is 1 stays enabled.

    * ggml-openvino: use the PLUGIN compiler for GDN graphs on the NPU

    The NPU driver compiler, the default, gives wrong output for graphs
    with GATED_DELTA_NET (Qwen3.5). Qwen3.5-2B on a Panther Lake NPU answers
    with unrelated text and has a perplexity of 16.77, against 13.00 with
    the compiler bundled with OpenVINO. Other models stay on the driver
    compiler, which builds faster prefill kernels.

    - Compile static graphs with GATED_DELTA_NET with
      NPU_COMPILER_TYPE=PLUGIN, unless GGML_OPENVINO_NPU_COMPILER_TYPE or
      NPU_COMPILER_TYPE in GGML_OPENVINO_NPU_CONFIG sets the compiler type.
    - Add the choice to the model cache key of these graphs, so blobs from
      the driver compiler are not imported.
    - docs: describe it in the GGML_OPENVINO_NPU_COMPILER_TYPE row.

    * ggml-openvino: fix gemma-4 E2B on the Lunar Lake NPU at small contexts

    The NPU 4000 compiler computes ScaledDotProductAttention wrongly when K
    has one KV head and is square, that is when the attended rows equal the
    head size. gemma-4 E2B has one KV head, 512-wide full-attention heads
    and 256-wide sliding-window heads, so a context of 512 tokens or less,
    and a prefill that attends 256 or 512 rows, gave wrong output.

    For that shape on NPU 4000, build the attention from MatMul, Softmax and
    MatMul, as the transposed-V path already does. Other shapes and devices
    keep the op.

    * ggml-openvino: support multiple GPUs with -sm layer

    * ggml-openvino: check GGML_OPENVINO_* variables and reorganize the docs

    The backend ignored a misspelled GGML_OPENVINO_* name and a value that
    was not valid, and the docs listed every variable in one table.

    Code:
    - One table holds the known variables and their types. A value that is
      not valid stops the backend with a message that names the variable and
      the accepted values. An unknown GGML_OPENVINO_* name gives a warning.
    - With log verbosity 4, one line shows the variables in effect.
    - GGML_OPENVINO_GPU_CONFIG and GGML_OPENVINO_CPU_CONFIG pass plugin
      properties to the GPU and the CPU, as GGML_OPENVINO_NPU_CONFIG does
      for the NPU. If the plugin rejects a property, the error names the
      variable.
    - GGML_OPENVINO_STATEFUL_EXECUTION gives a deprecation warning.
    - The key of the compiled model cache covers the two new variables,
      GGML_OPENVINO_KV_SCATTER_ELEMENTS and
      GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN.
    - GGML_OPENVINO_ENABLE_CACHE, which no code read, is no longer
      registered.

    Docs:
    - docs/backend/OPENVINO.md: Quick Start, collapsible blocks, runtime
      configuration in groups (basic, plugin properties, advanced tuning,
      compiled model cache, deprecated), updated validated models and known
      limitations, corrected stale entries.
    - dev_doc/OpenVINO-ENV-VARS.md: the variables for development and the
      NPU properties that the backend sets.

    * docs : add LFM2.5 and Ling-mini-2.0 to the OpenVINO validated models

    * ggml-openvino: rebuild converted weights in get_tensor

    set_tensor converts a quantized weight in place into the extracted or
    requantized layout, so get_tensor returned bytes that are not in the ggml
    format. A reader such as the CPU reference in test-backend-ops then
    computed on garbage and produced NaN (MUL_MAT cases with weights=1).

    get_tensor now rebuilds the ggml bytes for such a weight: it evaluates the
    weight node on the host and quantizes the values back to the tensor type.
    The result is close to the original data, not bit-exact. Other reads are
    unchanged, and the inference path does not call get_tensor on weights.

    * ggml-openvino: drop the GDN test cases from test-backend-ops

    Restore tests/test-backend-ops.cpp to the upstream version.

    * docs : improve the OpenVINO backend docs

    * docs : restore the OpenVINO intro and the development note

    * ggml-openvino: fix slow performance of MoE models with requantize and workaround to use fused MOE Compressed

    * ggml-openvino: document the fused GPU MoE op limits

    Converting K-quant MoE experts to Q4_1_64 on GPU lets the plugin use its
    fused MoE op, which runs in f16 only. granitemoe models give wrong output
    with it, and an f32 inference precision hint fails to compile. Document
    GGML_OPENVINO_MOE_OP=0 as the way to turn the op off, and that
    llama-perplexity with MoE models needs -b equal to -c.

    Also split the long TODO comment over two lines.

    * ggml-openvino : FILL for in-place clear of recurrent cache, sticky NaN fix

    * restore this part

    * ggml-openvino: bump the model-cache version for the recurrent-cache reset change

    Recurrent-cache resets now clear slots with Select instead of Multiply,
    so a NaN in a cleared slot no longer survives. Compiled models cached
    before this change keep the old graph, so bump the fingerprint version
    to rebuild them. Also note how SCALE and FILL store their parameters.

    * ggml-openvino: read rope_factors_short/long as weight constants

    create_weight_nodes() registers the ROPE frequency factors as
    "rope_freqs.weight", but set_input_output() and compute_model_inputs()
    kept the tensor's own name. For models whose factors are named
    rope_factors_short or rope_factors_long (Phi-3.5, Phi-4, MiniCPM with
    LongRoPE) the factors became a Parameter fed from the host weight buffer.
    That buffer is empty in cache-only mode and after the host weights are
    released, so these models gave garbage output with
    GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY, GGML_OPENVINO_RELEASE_WEIGHTS
    and GGML_OPENVINO_MEMORY_OPTIMIZE.

    Use the same name in all three places, so ROPE always reads the weight
    Constant. The graph changes, so bump the model-cache version to 7.

    * ggml-openvino: import cached models from a memory-mapped tensor

    A stream import reads the whole compiled blob into a heap buffer while
    the plugin builds its own copy of the weights. On CPU this held the blob
    twice at the peak. Import from ov::read_tensor_data() instead, which maps
    the file. This lowers the CPU cache-hit and cache-only peak memory; GPU
    memory is unchanged.

    * ggml-openvino: do not write CPU model-cache blobs for MoE models

    The CPU plugin exports its MoE op (GatherMatmulCompressed) but cannot
    import it again with OpenVINO 2026.4 and 2026.5 ("unsupported opset").
    Every later run then failed the import, compiled again and rewrote the
    blob. Skip the blob for such models on CPU and print a warning.

    In cache-only mode no blob exists for them, so the generic advice to run
    once without cache-only cannot help. Print an error that says cache-only
    does not work for MoE models on CPU instead.

    * ggml-openvino: fix GGML_OPENVINO_DEBUG_INPUT on GPU

    GPU inputs can be device (USM) tensors, whose data() throws, so every
    GPU compute failed with GGML_OPENVINO_DEBUG_INPUT=1. Print a host copy
    of remote tensors instead.

    * docs : add OpenVINO memory optimization tips and MiniCPM-V-4.5

    Add a "Tips for Memory Optimization" section to OpenVINO-ENV-VARS.md
    with examples for the spill directory, the compiled-model cache, host
    weight release on GPU, CPU MoE compile memory and the NPU weightless
    cache, and link it from OPENVINO.md. Add MiniCPM-V-4.5 to the validated
    models.

    * ggml-openvino: run MoE models with F16 and BF16 weights on GPU

    On GPU, supports_op rejected MUL_MAT_ID with non-quantized expert weights
    and GET_ROWS with BF16 data, so the scheduler ran those ops on the CPU.
    For F16 MoE models the CPU MUL_MAT_ID then read the strided top-k view as
    expert ids and crashed. For BF16 models the token embedding came from the
    CPU and reached the OpenVINO graph as a static-shaped input, which failed
    once the token count changed. Both ops translate correctly on GPU, so
    remove the two rejections.

    * ggml-openvino: fix BERT embedding models on CPU and GPU

    BERT models (bge-m3, nomic-embed-text-v1.5) failed on every device:

    - A VIEW of a weight with a GGML_OP_NONE source was marked as having a
      dynamic token dimension, a rule meant for KV cache views. The token type
      row view of BERT was then never sliced, and the whole two-row table was
      added to the activations. Skip the rule for model weights.
    - A row range view of a 2D weight now becomes a Slice of the weight rows,
      as GET_ROWS already does.
    - Attention without a KV cache pads K and V up to the mask width, which
      depends on the token count. PAD baked the amount from the first graph,
      so later token counts gave mismatched shapes. Compute the end padding
      from the mask width at run time.

    The NPU still fails for these models.

    * docs : add bge-m3 and nomic-embed-text-v1.5 to the OpenVINO validated models

    ---------

    Co-authored-by: Sabnis, Omkar <omkar.sabnis@intel.com>
    Co-authored-by: Beheshti <nazanin.beheshti@intel.com>
    Co-authored-by: Mustafa Cavus <mustafa.cavus@intel.com>
    Co-authored-by: Suhas Somashekar <suhas.somashekar@intel.com>
    Co-authored-by: yzhou51 <yaowei.zhou@intel.com>
    Co-authored-by: Mostafa Faheem <mostafaaafaheem@gmail.com>
    Co-authored-by: zhaixuejun1993 <xuejun.zhai@intel.com>
    Co-authored-by: Anisha Kulkarni <anisha.dattatraya.kulkarni@intel.com>
    Co-authored-by: Ananth, Swathi <swathi.ananth@intel.com>
    Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
    Co-authored-by: Yu, Zijun <zijun.yu@intel.com>
    Co-authored-by: Proshunin, Nikolay <nikolay.proshunin@intel.com>
    Co-authored-by: Alexey Kopytko <alexey@kopytko.com>

diff --git a/.devops/openvino.Dockerfile b/.devops/openvino.Dockerfile
index e290bde9b..c88cc2bbb 100644
--- a/.devops/openvino.Dockerfile
+++ b/.devops/openvino.Dockerfile
@@ -3,11 +3,11 @@ ARG OPENVINO_VERSION_FULL=2026.4.1.22982.07f9c262b05
 ARG UBUNTU_VERSION=24.04

 # Intel GPU driver versions. https://github.com/intel/compute-runtime/releases
-ARG IGC_VERSION=v2.41.5
-ARG IGC_VERSION_FULL=2_2.41.5+22716
-ARG COMPUTE_RUNTIME_VERSION=26.35.39758.10
-ARG COMPUTE_RUNTIME_VERSION_FULL=26.35.39758.10-0
-ARG IGDGMM_VERSION=22.10.0
+ARG IGC_VERSION=v2.42.6
+ARG IGC_VERSION_FULL=2_2.42.6+23070
+ARG COMPUTE_RUNTIME_VERSION=26.40.40196.8
+ARG COMPUTE_RUNTIME_VERSION_FULL=26.40.40196.8-0
+ARG IGDGMM_VERSION=22.10.2

 # Intel NPU driver versions. https://github.com/intel/linux-npu-driver/releases
 ARG NPU_DRIVER_VERSION=v1.38.0
diff --git a/docs/backend/OPENVINO.md b/docs/backend/OPENVINO.md
index 00effa9a8..0b5615ca2 100644
--- a/docs/backend/OPENVINO.md
+++ b/docs/backend/OPENVINO.md
@@ -3,9 +3,9 @@
 > [!NOTE]
 > Performance and memory optimizations, accuracy validation, broader quantization coverage, broader operator and model support are work in progress.

-[OpenVINO](https://docs.openvino.ai/) is an open-source toolkit for optimizing and deploying high-performance AI inference, specifically designed for Intel hardware, including CPUs, GPUs, and NPUs, in the cloud, on-premises, and on the edge. [OpenVINO backend for llama.cpp](../../ggml/src/ggml-openvino) enables hardware-accelerated inference on **Intel® CPUs, GPUs, and NPUs** while remaining compatible with the existing **GGUF model ecosystem**. The backend translates GGML compute graphs into OpenVINO graphs and leverages graph compilation, kernel fusion, and device-specific optimizations to improve inference performance on supported Intel hardware.
+[OpenVINO](https://docs.openvino.ai/) is an open-source toolkit for optimizing and deploying high-performance AI inference, specifically designed for Intel hardware, including CPUs, GPUs (integrated and discrete), and NPUs, in the cloud, on-premises, and on the edge. The [OpenVINO backend](../../ggml/src/ggml-openvino) runs the same GGUF model files on **Intel® CPUs, GPUs and NPUs**, without changes to the model or the rest of llama.cpp.

-The OpenVINO backend is implemented in `ggml/src/ggml-openvino` and provides a translation layer for core GGML operations. The OpenVINO backend replaces the standard GGML graph execution path with Intel's OpenVINO inference engine. This approach allows the same GGUF model file to run on Intel CPUs, Intel GPUs (integrated and discrete), and Intel NPUs without changes to the model or the rest of the llama.cpp stack. When a `ggml_cgraph` is dispatched to OpenVINO backend, it:
+When a `ggml_cgraph` is dispatched to the OpenVINO backend, it:

 - Walks the GGML graph and identifies inputs, outputs, weights, and KV cache tensors.
 - Translates the GGML operations into an `ov::Model` using OpenVINO's frontend API.
@@ -16,9 +16,10 @@ For guidance on contributing to the OpenVINO backend, see the [OpenVINO Backend

 ## Contents

+- [Quick Start](#quick-start)
 - [Supported Devices](#supported-devices)
 - [Supported Model Precisions](#supported-model-precisions)
-- [Supported Llama.cpp Tools](#supported-llamacpp-tools)
+- [Supported llama.cpp Tools](#supported-llamacpp-tools)
 - [Validated Models](#validated-models)
 - [Build Instructions](#build-instructions)
   - [0. Prerequisites](#0-prerequisites)
@@ -29,10 +30,44 @@ For guidance on contributing to the OpenVINO backend, see the [OpenVINO Backend
   - [3. Download Sample Model](#3-download-sample-model)
   - [4. Run Inference with OpenVINO Backend](#4-run-inference-with-openvino-backend)
   - [5. Docker Build](#5-docker-build)
-- [GGML OpenVINO Backend Runtime Configurations](#ggml-openvino-backend-runtime-configurations)
+- [Runtime Configuration](#runtime-configuration)
+  - [Basic](#basic)
+  - [Plugin properties](#plugin-properties)
+  - [Advanced tuning](#advanced-tuning)
+  - [Compiled model cache](#compiled-model-cache)
+  - [Deprecated](#deprecated)
+  - [Debugging and development](#debugging-and-development)
+  - [Example Usage](#example-usage)
+    - [GPU Inference with Profiling](#gpu-inference-with-profiling)
 - [Known Limitations](#known-limitations)
 - [Work in Progress](#work-in-progress)

+## Quick Start
+
+On Ubuntu, with the [prerequisites](#0-prerequisites) and the [OpenVINO Runtime](#1-install-openvino-runtime) installed:
+
+```bash
+# 1. Build
+git clone https://github.com/ggml-org/llama.cpp
+cd llama.cpp
+source /opt/intel/openvino/setupvars.sh
+cmake -B build/ReleaseOV -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_OPENVINO=ON
+cmake --build build/ReleaseOV --parallel
+
+# 2. Download a model
+mkdir -p ~/models/
+wget https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf \
+     -O ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf
+
+# 3. Run on CPU, GPU or NPU
+export GGML_OPENVINO_DEVICE=GPU
+./build/ReleaseOV/bin/llama-cli -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -c 1024
+```
+
+The [Ubuntu Build Script](#ubuntu-build-script) and the [Windows Build Script](#windows-build-script) do the setup in one step: they install the prerequisites and the OpenVINO Runtime, and build llama.cpp with the OpenVINO backend.
+
+See [Build Instructions](#build-instructions) for Windows and Docker, and [Runtime Configuration](#runtime-configuration) for the environment variables.
+
 ## Supported Devices

 OpenVINO backend supports the following hardware:
@@ -41,7 +76,7 @@ OpenVINO backend supports the following hardware:
 - Intel GPUs (integrated and discrete)
 - Intel NPUs

-Although OpenVINO supports a wide range of [Intel hardware](https://docs.openvino.ai/2026/about-openvino/release-notes-openvino/system-requirements.html), the llama.cpp OpenVINO backend has been validated specifically on AI PCs such as the Intel® Core™ Ultra Series 1 and Series 2.
+Although OpenVINO supports a wide range of [Intel hardware](https://docs.openvino.ai/2026/about-openvino/release-notes-openvino/system-requirements.html), the llama.cpp OpenVINO backend has been validated specifically on AI PCs such as the Intel® Core™ Ultra Series 2 (Lunar Lake) and Series 3 (Panther Lake).

 ## Supported Model Precisions

@@ -51,101 +86,123 @@ Although OpenVINO supports a wide range of [Intel hardware](https://docs.openvin
 - `Q4_0`
 - `Q4_1`
 - `Q4_K`
-- `Q4_K_M`
-- `Q5_K` (converted to `Q8_0_C` at runtime by default)
-- `Q6_K` (converted to `Q8_0_C` at runtime by default)
+- `Q5_0` (converted at runtime, see the details below)
+- `Q5_1`
+- `Q5_K` (converted at runtime, see the details below)
+- `Q6_K` (converted at runtime, see the details below)

-> [!NOTE]
-> Accuracy validation and performance optimizations for quantized models are a work in progress.
+`Q4_K_M` files mix `Q4_K` tensors with `Q6_K` tensors, and sometimes `Q5_K` tensors.
+
+<details>
+<summary>Quantization details for each device</summary>
+
+**All devices:**
+- `FP16` and `BF16` tensors are used as they are.
+- A quantized token embedding (`token_embd.weight`) and output weight (`output.weight`) are converted to `Q8_0_C`.
+
+**CPU and GPU:**
+- `Q4_0`, `Q4_1`, `Q4_K`, `Q5_1` and `Q8_0` tensors keep their layout: weights, scales and zero points are extracted directly.
+- `Q5_K` and `Q6_K` tensors are converted to `Q8_0_C`. `GGML_OPENVINO_REQUANT_KQUANT` selects a 4-bit layout instead.
+- `Q5_0` tensors are converted to `Q4_0_64`.

-**CPU and GPU Quantization Details:**
-- `Q5_K` and `Q6_K` tensors are converted to `Q8_0_C`
+**GPU:**
+- `Q4_K`, `Q5_K` and `Q6_K` MoE expert tensors are converted to `Q4_1_64`. This lets the GPU plugin use its fused MoE op, which is much faster for prompt processing. `GGML_OPENVINO_REQUANT_KQUANT` does not change this. `GGML_OPENVINO_MOE_OP=0` turns the fused MoE op off.

-**NPU Quantization Details:**
-- Primary supported quantization scheme is `Q4_0`
-- `Q6_K` tensors are requantized to `Q4_0_128` in general. For embedding weights, `Q6_K` tensors are requantized to `Q8_0_C` except for the token embedding matrix which is dequantized to fp16
+**NPU:**
+- All other quantized tensors are requantized to `Q4_0_128`.
+- For a `Q6_K` token embedding, `GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT` selects another layout.
+- The validated models use `Q4_K_M` files.
+
+**Layout names:**
+- `Q8_0_C`: 8-bit, one scale per row.
+- `Q4_0_128` and `Q4_0_64`: 4-bit, symmetric, one scale per 128 or 64 weights.

 **Additional Notes:**
 - Both `Q4_0` and `Q4_1` models use `Q6_K` for the token embedding tensor and the final matmul weight tensor (often the same tensor)
 - `Q4_0` models may produce some `Q4_1` tensors if an imatrix is provided during quantization using `llama-quantize`
 - `Q4_K_M` models may include both `Q6_K` and `Q5_K` tensors (observed in Phi-3)
-- `Q5_1` tensors are dequantized natively (weights, scales, and zero-points extracted directly)

-## Supported Llama.cpp Tools
+</details>
+
+## Supported llama.cpp Tools

-The OpenVINO backend integrates with the standard llama.cpp tools listed below.
-However, all the tools coverage across all devices is not uniform and exhaustive validation is work in progress.
+The OpenVINO backend works with the standard llama.cpp tools listed below. Tool coverage is not uniform across devices.

 - llama-bench
 - llama-cli
 - llama-completion
 - llama-embedding
 - llama-perplexity
-- llama-run
 - llama-server
 - llama-simple

 ## Validated Models

-Although, the validated models below were tested with `llama-cli` using the `Q4_K_M` quantization format on Intel® Core™ Ultra Series 2 (Lunar Lake), the OpenVINO backend is expected to work across a broader range of [Intel hardware](https://docs.openvino.ai/2026/about-openvino/release-notes-openvino/system-requirements.html), [supported model precisions](#supported-model-precisions), [supported llama.cpp tools](#supported-llamacpp-tools) and additional model architectures.
-
-> [!NOTE]
-> Extensive accuracy validation, performance optimizations, and broader architecture coverage are work in progress.
+The models below were tested with `llama-cli` and `Q4_K_M` files on an Intel® Core™ Ultra Series 2 (Lunar Lake). The OpenVINO backend is expected to work on a broader range of [Intel hardware](https://docs.openvino.ai/2026/about-openvino/release-notes-openvino/system-requirements.html), [model precisions](#supported-model-precisions), [llama.cpp tools](#supported-llamacpp-tools) and model architectures.

 **Legend & Test Configuration:**
 - **Status:** ✓ = Passed | ~ = Accuracy issues | ✗ = Failed or Unsupported
-- **Execution Modes:**
-  - **SL** = Stateless (`GGML_OPENVINO_STATEFUL_EXECUTION=0`)
-  - **SF** = Stateful (`GGML_OPENVINO_STATEFUL_EXECUTION=1`)
-  - Note: The NPU operates in stateless mode only.
-- **Validation system:** Intel® Core™ Ultra 5 238V (Lunar Lake) | 32 GB RAM | Ubuntu 24.04 | Intel Graphics Compiler 2.41.5 | Intel OpenCL GPU Driver 26.35.39758.10-0 | Intel NPU Driver 1.38.0.
+- **Execution mode:** stateless (`GGML_OPENVINO_STATEFUL_EXECUTION=0`) on every device.
+- **Validation system:** Intel® Core™ Ultra 7 258V (Lunar Lake) | 32 GB RAM | Ubuntu 26.04.1 | Intel Graphics Compiler 2.42.6 | Intel OpenCL GPU Driver 26.40.40196.8-0 | Intel NPU Driver 1.38.0.
 - See [Known Limitations](#known-limitations) for context on observed failures.

-| Model | CPU (SL / SF) | GPU (SL / SF) | NPU (SL) |
+<details open>
+<summary><b>Validated models</b></summary>
+
+| Model | CPU | GPU | NPU |
 | :--- | :---: | :---: | :---: |
-| [bartowski/Llama-3.2-1B-Instruct-Q4_K_M](https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/Llama-3.2-3B-Instruct-Q4_K_M](https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/Meta-Llama-3.1-8B-Instruct-Q4_K_M](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
+| [bartowski/Llama-3.2-1B-Instruct-Q4_K_M](https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/Llama-3.2-3B-Instruct-Q4_K_M](https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/Meta-Llama-3.1-8B-Instruct-Q4_K_M](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF) | ✓ | ✓ | ✓ |
 |  |  |  |  |
-| [Qwen/qwen2.5-1.5b-instruct-q4_k_m](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [Qwen/qwen2.5-coder-7b-instruct-q4_k_m](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/Qwen_Qwen3-0.6B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3-0.6B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/Qwen_Qwen3-1.7B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3-1.7B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [Qwen/Qwen3-4B-Q4_K_M](https://huggingface.co/Qwen/Qwen3-4B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [lm-kit/Qwen3-8B-Q4_K_M](https://huggingface.co/lm-kit/qwen-3-8b-instruct-gguf) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/Qwen_Qwen3.5-0.8B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3.5-0.8B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✗ |
-| [bartowski/Qwen_Qwen3.5-2B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3.5-2B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✗ |
-| [bartowski/Qwen_Qwen3.5-4B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3.5-4B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✗ |
-| [lmstudio-community/Qwen3.5-9B-Q4_K_M](https://huggingface.co/lmstudio-community/Qwen3.5-9B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✗ |
+| [Qwen/qwen2.5-1.5b-instruct-q4_k_m](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF) | ✓ | ✓ | ✓ |
+| [Qwen/qwen2.5-coder-7b-instruct-q4_k_m](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/Qwen_Qwen3-0.6B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3-0.6B-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/Qwen_Qwen3-1.7B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3-1.7B-GGUF) | ✓ | ✓ | ✓ |
+| [Qwen/Qwen3-4B-Q4_K_M](https://huggingface.co/Qwen/Qwen3-4B-GGUF) | ✓ | ✓ | ✓ |
+| [lm-kit/Qwen3-8B-Q4_K_M](https://huggingface.co/lm-kit/qwen-3-8b-instruct-gguf) | ✓ | ✓ | ✓ |
+| [bartowski/Qwen_Qwen3.5-0.8B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3.5-0.8B-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/Qwen_Qwen3.5-2B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3.5-2B-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/Qwen_Qwen3.5-4B-Q4_K_M](https://huggingface.co/bartowski/Qwen_Qwen3.5-4B-GGUF) | ✓ | ✓ | ✓ |
+| [lmstudio-community/Qwen3.5-9B-Q4_K_M](https://huggingface.co/lmstudio-community/Qwen3.5-9B-GGUF) | ✓ | ✓ | ✓ |
 |  |  |  |  |
-| [unsloth/gemma-3-4b-it-Q4_K_M](https://huggingface.co/unsloth/gemma-3-4b-it-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/google_gemma-4-E2B-it-Q4_K_M](https://huggingface.co/bartowski/google_gemma-4-E2B-it-GGUF) | ✓ / ✓ | ✓ / ✓ | ~ |
-| [bartowski/google_gemma-4-E4B-it-Q4_K_M](https://huggingface.co/bartowski/google_gemma-4-E4B-it-GGUF) | ✓ / ✓ | ✗ / ✗ | ✓ |
-| [bartowski/gemma-4-12B-it-Q4_K_M](https://huggingface.co/bartowski/gemma-4-12B-it-GGUF) | ✓ / ✓ | ✓ / ✓ | ✗ |
+| [unsloth/gemma-3-4b-it-Q4_K_M](https://huggingface.co/unsloth/gemma-3-4b-it-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/google_gemma-4-E2B-it-Q4_K_M](https://huggingface.co/bartowski/google_gemma-4-E2B-it-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/google_gemma-4-E4B-it-Q4_K_M](https://huggingface.co/bartowski/google_gemma-4-E4B-it-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/gemma-4-12B-it-Q4_K_M](https://huggingface.co/bartowski/gemma-4-12B-it-GGUF) | ✓ | ✓ | ✗ |
 |  |  |  |  |
-| [bartowski/Phi-3-mini-4k-instruct-Q4_K_M](https://huggingface.co/bartowski/Phi-3-mini-4k-instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/Phi-3.5-mini-instruct-Q4_K_M](https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/microsoft_Phi-4-mini-instruct-Q4_K_M](https://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
+| [bartowski/Phi-3-mini-4k-instruct-Q4_K_M](https://huggingface.co/bartowski/Phi-3-mini-4k-instruct-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/Phi-3.5-mini-instruct-Q4_K_M](https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/microsoft_Phi-4-mini-instruct-Q4_K_M](https://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF) | ✓ | ✓ | ✓ |
 |  |  |  |  |
-| [bartowski/Mistral-7B-Instruct-v0.3-Q4_K_M](https://huggingface.co/bartowski/Mistral-7B-Instruct-v0.3-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [QuantFactory/Ministral-3b-instruct.Q4_K_M](https://huggingface.co/QuantFactory/Ministral-3b-instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/Ministral-8B-Instruct-2410-Q4_K_M](https://huggingface.co/bartowski/Ministral-8B-Instruct-2410-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
+| [bartowski/Mistral-7B-Instruct-v0.3-Q4_K_M](https://huggingface.co/bartowski/Mistral-7B-Instruct-v0.3-GGUF) | ✓ | ✓ | ✓ |
+| [QuantFactory/Ministral-3b-instruct.Q4_K_M](https://huggingface.co/QuantFactory/Ministral-3b-instruct-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/Ministral-8B-Instruct-2410-Q4_K_M](https://huggingface.co/bartowski/Ministral-8B-Instruct-2410-GGUF) | ✓ | ✓ | ✓ |
 |  |  |  |  |
-| [bartowski/DeepSeek-R1-Distill-Llama-8B-Q4_K_M](https://huggingface.co/bartowski/DeepSeek-R1-Distill-Llama-8B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M](https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
+| [bartowski/DeepSeek-R1-Distill-Llama-8B-Q4_K_M](https://huggingface.co/bartowski/DeepSeek-R1-Distill-Llama-8B-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M](https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF) | ✓ | ✓ | ✓ |
 |  |  |  |  |
-| [ibm-granite/granite-4.0-350m-Q4_K_M](https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF) | ✓ / ✓ | ~ / ~ | ✓ |
-| [ibm-granite/granite-4.0-micro-Q4_K_M](https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [ibm-granite/granite-4.0-1b-Q4_K_M](https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF) | ✓ / ✓ | ~ / ~ | ~ |
-| [ibm-research/granite-3.2-8b-instruct-Q4_K_M](https://huggingface.co/ibm-research/granite-3.2-8b-instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
+| [ibm-granite/granite-4.0-350m-Q4_K_M](https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF) | ✓ | ✓ | ✓ |
+| [ibm-granite/granite-4.0-micro-Q4_K_M](https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF) | ✓ | ✓ | ✓ |
+| [ibm-granite/granite-4.0-1b-Q4_K_M](https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF) | ✓ | ✓ | ~ |
+| [ibm-research/granite-3.2-8b-instruct-Q4_K_M](https://huggingface.co/ibm-research/granite-3.2-8b-instruct-GGUF) | ✓ | ✓ | ✓ |
 |  |  |  |  |
-| [HuggingFaceTB/smollm2-1.7b-instruct-q4_k_m](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [openbmb/MiniCPM-V-2_6-Q4_K_M](https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/tencent_Hunyuan-7B-Instruct-Q4_K_M](https://huggingface.co/bartowski/tencent_Hunyuan-7B-Instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-Q4_K_M](https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
-| [bartowski/prism-ml_Bonsai-8B-unpacked-Q4_K_M](https://huggingface.co/bartowski/prism-ml_Bonsai-8B-unpacked-GGUF) | ✓ / ✓ | ✓ / ✓ | ✓ |
+| [HuggingFaceTB/smollm2-1.7b-instruct-q4_k_m](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF) | ✓ | ✓ | ✓ |
+| [openbmb/MiniCPM-V-2_6-Q4_K_M](https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf) | ✓ | ✓ | ✓ |
+| [openbmb/MiniCPM-V-4_5-Q4_K_M](https://huggingface.co/openbmb/MiniCPM-V-4_5-gguf) | ✓ | ✓ | ✓ |
+| [bartowski/tencent_Hunyuan-7B-Instruct-Q4_K_M](https://huggingface.co/bartowski/tencent_Hunyuan-7B-Instruct-GGUF) | ✓ | ✓ | ✓ |
+| [LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-Q4_K_M](https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-GGUF) | ✓ | ✓ | ✓ |
+| [bartowski/prism-ml_Bonsai-8B-unpacked-Q4_K_M](https://huggingface.co/bartowski/prism-ml_Bonsai-8B-unpacked-GGUF) | ✓ | ✓ | ✓ |
+| [LiquidAI/LFM2.5-2.6B-Q4_K_M](https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF) | ✓ | ✓ | ✗ |
+| [bartowski/inclusionAI_Ling-mini-2.0-Q4_K_M](https://huggingface.co/bartowski/inclusionAI_Ling-mini-2.0-GGUF) | ✓ | ✓ | ✗ |
 |  |  |  |  |
-| [gpustack/bge-m3-Q4_K_M.gguf](https://huggingface.co/gpustack/bge-m3-GGUF) | ✓ | ✗ | ✗ |
+| [Qwen/Qwen3-Embedding-0.6B-Q8_0](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B-GGUF) | ✓ | ✓ | ✗ |
+| [ggml-org/bge-m3-Q8_0](https://huggingface.co/ggml-org/bge-m3-Q8_0-GGUF) | ✓ | ✓ | ✗ |
+| [nomic-ai/nomic-embed-text-v1.5-Q8_0](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF) | ✓ | ✓ | ✗ |
+
+The embedding models were tested with `llama-embedding` and `Q8_0` files.
+
+</details>

 ## Build Instructions

@@ -154,39 +211,51 @@ Although, the validated models below were tested with `llama-cli` using the `Q4_
 - Linux or Windows system with Intel hardware (CPU, GPU, or NPU)
 - **For Intel GPU or NPU Usage**: Install the appropriate hardware drivers for your Intel GPU or NPU. For detailed instructions, see: [Additional Configurations for Hardware Acceleration](https://docs.openvino.ai/2026/get-started/install-openvino/configurations.html).

-- **Linux:**
-    - Git, CMake, and Ninja software tools are needed for building.
-    ```bash
-      sudo apt-get update
-      sudo apt-get install -y build-essential libcurl4-openssl-dev libtbb12 cmake ninja-build python3-pip curl wget tar
-    ```
-    - OpenCL
-    ```bash
-      sudo apt install ocl-icd-opencl-dev opencl-headers opencl-clhpp-headers intel-opencl-icd
-    ```
-
-- **Windows:**
-  - Download and install [Microsoft Visual Studio 2022 Build Tools](https://aka.ms/vs/17/release/vs_BuildTools.exe). During installation, select the **"Desktop development with C++"** workload.
-
-  - Install required tools:
-    ```powershell
-    # Windows PowerShell
-    winget install Git.Git
-    winget install GNU.Wget
-    winget install Ninja-build.Ninja
-    ```
-
-  - Install **OpenCL** using **vcpkg**:
-    ```powershell
-    # Windows PowerShell
-    cd C:\
-    git clone https://github.com/microsoft/vcpkg
-    cd vcpkg
-    .\bootstrap-vcpkg.bat
-    .\vcpkg install opencl
-    # Optional but recommended: Integrate vcpkg with Visual Studio / CMake:
-    .\vcpkg integrate install
-    ```
+<details open>
+<summary><b>Linux</b></summary>
+
+Build tools (Git, CMake, Ninja):
+
+```bash
+sudo apt-get update
+sudo apt-get install -y build-essential libcurl4-openssl-dev libtbb12 cmake ninja-build python3-pip curl wget tar
+```
+
+OpenCL:
+
+```bash
+sudo apt install ocl-icd-opencl-dev opencl-headers opencl-clhpp-headers intel-opencl-icd
+```
+
+</details>
+
+<details>
+<summary><b>Windows</b></summary>
+
+1. Download and install [Microsoft Visual Studio 2022 Build Tools](https://aka.ms/vs/17/release/vs_BuildTools.exe). During installation, select the **"Desktop development with C++"** workload.
+2. Install the required tools:
+
+   ```powershell
+   # Windows PowerShell
+   winget install Git.Git
+   winget install GNU.Wget
+   winget install Ninja-build.Ninja
+   ```
+
+3. Install **OpenCL** using **vcpkg**:
+
+   ```powershell
+   # Windows PowerShell
+   cd C:\
+   git clone https://github.com/microsoft/vcpkg
+   cd vcpkg
+   .\bootstrap-vcpkg.bat
+   .\vcpkg install opencl
+   # Optional but recommended: Integrate vcpkg with Visual Studio / CMake:
+   .\vcpkg integrate install
+   ```
+
+</details>

 ### 1. Install OpenVINO Runtime

@@ -199,21 +268,25 @@ Although, the validated models below were tested with `llama-cli` using the `Q4_

 ### 2. Build llama.cpp with OpenVINO Backend

-Clone llama.cpp repo and build :
+Clone the llama.cpp repository:

 ```bash
 git clone https://github.com/ggml-org/llama.cpp
 cd llama.cpp
 ```

-- **Linux:**
+Build on Linux:
+
 ```bash
 source /opt/intel/openvino/setupvars.sh
 cmake -B build/ReleaseOV -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_OPENVINO=ON
 cmake --build build/ReleaseOV --parallel
 ```

-- **Windows:** Open **x64 Native Tools Command Prompt for VS** (so the MSVC toolchain is on `PATH`), then run:
+<details>
+<summary><b>Build on Windows</b></summary>
+
+Open **x64 Native Tools Command Prompt for VS** (so the MSVC toolchain is on `PATH`), then run:

 ```cmd
 C:\Intel\openvino\setupvars.bat
@@ -224,9 +297,11 @@ cmake --build build\ReleaseOV --parallel
 > [!NOTE]
 > The Windows install path is `C:\Intel\openvino` (no spaces) to avoid quoting problems some CMake/Ninja toolchains have with `C:\Program Files (x86)\...`. Adjust to wherever you installed OpenVINO Runtime. From `cmd`, run `C:\Intel\openvino\setupvars.bat`; from PowerShell, run `& "C:\Intel\openvino\setupvars.ps1"` instead. Once the build is finished you can launch the binaries from any `cmd` or `PowerShell` window after sourcing the matching `setupvars` script for that shell.

+</details>
+
 #### Ubuntu Build Script

-For Ubuntu24 users, the following shell script automates the prerequisite installs (build tools, OpenCL ICD), the OpenVINO Runtime download/extract/setup, and the Ninja-based llama.cpp build.
+For Ubuntu users, the following shell script automates the prerequisite installs (build tools, OpenCL ICD), the OpenVINO Runtime download/extract/setup, and the Ninja-based llama.cpp build.
 Save the following as `build-llamacpp-ov.sh` next to where you want the `llama.cpp` folder to land, then run it:

 ```bash
@@ -556,81 +631,102 @@ endlocal

 </details>

-
 ### 3. Download Sample Model

-Download sample model for testing.
+Download a sample model for testing.

 ```bash
-# Linux
 mkdir -p ~/models/
 wget https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf \
      -O ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf
+```

+<details>
+<summary><b>Windows</b></summary>
+
+```powershell
 # Windows PowerShell
 mkdir C:\models
 Invoke-WebRequest -Uri https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf -OutFile C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf
+```

-# Windows Command Line
+```cmd
+REM Windows Command Prompt
 mkdir C:\models
 curl -L https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf -o C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf
 ```

+</details>
+
 ### 4. Run Inference with OpenVINO Backend

-When using the OpenVINO backend, the first inference token may have slightly higher latency due to on-the-fly conversion to the OpenVINO graph. Subsequent tokens and runs will be faster.
+The first run of a model converts and compiles it for the device. This takes seconds on CPU and GPU, and can take minutes on the NPU for large models. Later tokens are faster, and the [compiled model cache](#compiled-model-cache) makes later runs start faster.

 > [!NOTE]
-> Default context size is set to the model training context, which may be very large. For example, 131072 for Llama 3.2 1B, which may result in lower performance, especially on edge/laptop devices. Use `-c` to limit context size in supported llama.cpp tools for better performance. For example, `-c 512`.
+> Default context size is set to the model training context, which may be very large. For example, 131072 for Llama 3.2 1B, which may result in lower performance, especially on edge/laptop devices. Use `-c` to limit context size in supported llama.cpp tools for better performance:
+> - CPU and GPU: for example `-c 1024`.
+> - NPU: keep the context small, for example `-c 512`, or `-c 256` for larger models. Decode may slow down as the context grows.

 ```bash
-# If device is unset or unavailable, defaults to CPU.
-# If the system has multiple GPUs, use GPU.0 or GPU.1 to explicitly target a specific GPU.
-
-# Linux
+# If the device is unset, the CPU is used.
 export GGML_OPENVINO_DEVICE=GPU
-# Optional: enable stateful execution for improved GPU performance (recommended).
-export GGML_OPENVINO_STATEFUL_EXECUTION=1
 # To run llama-simple:
 ./build/ReleaseOV/bin/llama-simple -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -n 50 "The story of AI is "
 # To run in chat mode:
 ./build/ReleaseOV/bin/llama-cli -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -c 1024
-# To run llama-bench, -fa 1 is needed
-GGML_OPENVINO_STATEFUL_EXECUTION=1 GGML_OPENVINO_DEVICE=GPU ./build/ReleaseOV/bin/llama-bench -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -fa 1
+# To run llama-bench:
+GGML_OPENVINO_DEVICE=GPU ./build/ReleaseOV/bin/llama-bench -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf

 # NPU: keep context small to avoid failures from very large model context windows.
 export GGML_OPENVINO_DEVICE=NPU
 ./build/ReleaseOV/bin/llama-cli -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -c 512
+```

-# Windows Command Line
-set GGML_OPENVINO_DEVICE=GPU
-# Optional: enable stateful execution for improved GPU performance (recommended).
-set GGML_OPENVINO_STATEFUL_EXECUTION=1
-# Windows PowerShell
-$env:GGML_OPENVINO_DEVICE = "GPU"
-$env:GGML_OPENVINO_STATEFUL_EXECUTION = "1"
+<details>
+<summary><b>Windows</b></summary>

-# To run llama-simple
+```cmd
+REM Windows Command Prompt
+set GGML_OPENVINO_DEVICE=GPU
+REM To run llama-simple:
 build\ReleaseOV\bin\llama-simple.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf" -n 50 "The story of AI is "
-# To run in chat mode:
+REM To run in chat mode:
 build\ReleaseOV\bin\llama-cli.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf" -c 1024
-# To run llama-bench, -fa 1 is needed
-build\ReleaseOV\bin\llama-bench.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf" -fa 1
+REM To run llama-bench:
+build\ReleaseOV\bin\llama-bench.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf"

-# NPU: keep context small to avoid failures from very large model context windows.
-# Windows Command Line
+REM NPU: keep the context small.
 set GGML_OPENVINO_DEVICE=NPU
+build\ReleaseOV\bin\llama-cli.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf" -c 512
+```
+
+```powershell
 # Windows PowerShell
+$env:GGML_OPENVINO_DEVICE = "GPU"
+build\ReleaseOV\bin\llama-cli.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf" -c 1024
+
+# NPU: keep the context small.
 $env:GGML_OPENVINO_DEVICE = "NPU"
 build\ReleaseOV\bin\llama-cli.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf" -c 512
 ```
+
+</details>
+
 > [!NOTE]
-> On systems with multiple GPUs, use `GPU.0` or `GPU.1` to explicitly target specific GPU. A device that is not available is an error (no fallback to CPU), and the error message lists the available OpenVINO devices with their names. Run `llama-cli --list-devices` to see the valid values: each OpenVINO device shows the `GGML_OPENVINO_DEVICE=<value>` to set, and `(selected)` marks the active one. Select the OpenVINO device with this variable, not with `-dev`. See [OpenVINO GPU Device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html) for more details.
+> - Select the OpenVINO device with `GGML_OPENVINO_DEVICE`, not with `-dev`.
+> - On systems with multiple GPUs, use `GPU.0` or `GPU.1` to explicitly target specific GPU.
+> - To split a model across several GPUs, set a comma-separated list, e.g. `GPU.0,GPU.1`, and use `-sm layer`.
+> - A device that is not available is an error (no fallback to CPU). The error message lists the available OpenVINO devices with their names.
+> - Run `llama-cli --list-devices` to see the valid values: each OpenVINO device shows the `GGML_OPENVINO_DEVICE=<value>` to set, and `(selected)` marks the active one.
+> - See [OpenVINO GPU Device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html) for more details.

 ### 5. Docker Build

 You can build and run llama.cpp with OpenVINO backend using Docker.

+<details>
+<summary><b>Build the images</b></summary>
+
 ```bash
 # Build the base runtime image with compiled shared libraries and minimal dependencies.
 docker build -t llama-openvino:base -f .devops/openvino.Dockerfile .
@@ -648,9 +744,12 @@ docker build --target=server -t llama-openvino:server -f .devops/openvino.Docker
 docker build --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy --target=server -t llama-openvino:server -f .devops/openvino.Dockerfile .
 ```

-Run llama.cpp with OpenVINO backend Docker container.
-Save sample models in `~/models` as [shown above](#3-download-sample-model). It will be mounted to the container in the examples below.
+</details>
+
+<details>
+<summary><b>Run a container</b> (CPU, GPU, NPU)</summary>

+Run llama.cpp with OpenVINO backend Docker container. Save sample models in `~/models` as [shown above](#3-download-sample-model). It will be mounted to the container in the examples below.

 ```bash
 #  Run Docker container
@@ -659,19 +758,20 @@ docker run --rm -it -v ~/models:/models llama-openvino:light --no-warmup -c 1024
 # With Intel GPU access (iGPU or dGPU)
 docker run --rm -it -v ~/models:/models \
 --device=/dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -u $(id -u):$(id -g) \
---env=GGML_OPENVINO_DEVICE=GPU --env=GGML_OPENVINO_STATEFUL_EXECUTION=1 \
+--env=GGML_OPENVINO_DEVICE=GPU \
 llama-openvino:light --no-warmup -c 1024 -m /models/Llama-3.2-1B-Instruct-Q4_K_M.gguf

 # With Intel NPU access
 docker run --rm -it -v ~/models:/models \
---device=/dev/accel --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -u $(id -u):$(id -g) \
+--device=/dev/accel --group-add=$(stat -c "%g" /dev/accel/accel* | head -n 1) -u $(id -u):$(id -g) \
 --env=GGML_OPENVINO_DEVICE=NPU \
-llama-openvino:light --no-warmup -c 1024 -m /models/Llama-3.2-1B-Instruct-Q4_K_M.gguf
+llama-openvino:light --no-warmup -c 512 -m /models/Llama-3.2-1B-Instruct-Q4_K_M.gguf
 ```

-Run Llama.cpp Server with OpenVINO Backend.
-> [!NOTE]
-> `llama-server` with OpenVINO backend supports only one chat session/thread, when `GGML_OPENVINO_STATEFUL_EXECUTION=1` is enabled.
+</details>
+
+<details>
+<summary><b>Run llama-server</b> (container or local build) and test it</summary>

 ```bash
 # Run the llama-openvino:server Docker container (CPU)
@@ -685,9 +785,9 @@ llama-openvino:server --no-warmup -c 1024 -m /models/Llama-3.2-1B-Instruct-Q4_K_

 # Run the llama-openvino:server Docker container with Intel NPU access
 docker run --rm -it -v ~/models:/models \
---device=/dev/accel --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -u $(id -u):$(id -g) \
+--device=/dev/accel --group-add=$(stat -c "%g" /dev/accel/accel* | head -n 1) -u $(id -u):$(id -g) \
 -p 8080:8080 --env=GGML_OPENVINO_DEVICE=NPU \
-llama-openvino:server --no-warmup -c 1024 -m /models/Llama-3.2-1B-Instruct-Q4_K_M.gguf --host 0.0.0.0
+llama-openvino:server --no-warmup -c 512 -m /models/Llama-3.2-1B-Instruct-Q4_K_M.gguf --host 0.0.0.0

 # Or Using llama-server executable
 ./build/ReleaseOV/bin/llama-server -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf --port 8080 -c 1024
@@ -706,101 +806,221 @@ curl -X POST "http://localhost:8080/v1/chat/completions" -H "Content-Type: appli
  -d '{"messages":[{"role":"user","content":"Write a poem about OpenVINO"}],"max_tokens":100}' | jq .
 ```

-## GGML OpenVINO Backend Runtime Configurations
-
-The OpenVINO backend can be configured using the following environment variables at runtime to control device selection, caching, debugging, and profiling behavior.
-Boolean flags follow a uniform convention: set to a **positive integer** (e.g. `1`) to enable; unset, empty, `0`, negative, or non-numeric values are treated as disabled.
-
-| Variable                          | Type      | Default    | Description                                                                                                 |
-|-----------------------------------|-----------|------------|-------------------------------------------------------------------------------------------------------------|
-| `GGML_OPENVINO_DEVICE`            | String    | `CPU`      | Specify the target device (CPU, GPU, NPU). On systems with multiple GPUs, use `GPU.0` or `GPU.1` to explicitly target specific GPU. A device that is not available is an error (no fallback to CPU), and the error message lists the available OpenVINO devices with their names. See [OpenVINO GPU Device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html). When set to **NPU**, static compilation mode is enabled for optimal performance. |
-| `GGML_OPENVINO_CACHE_DIR`         | String    | `not set`  | Directory for OpenVINO's separate plugin cache. On NPU, this sets `NPUW_CACHE_DIR`. |
-| `GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR` | String | `not set` | Directory for standalone compiled blobs with weights. Dynamic CPU/GPU graphs can import matching blobs on later runs. |
-| `GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY` | Boolean | `0` | Require an existing compiled blob and skip weight uploads and compilation. Requires Linux or Windows mmap loading and a full dynamic CPU/GPU graph on OpenVINO. |
-| `GGML_OPENVINO_PREFILL_CHUNK_SIZE`| Integer   | `256`      | Token chunk size for **NPU** prefill (NPU-only; ignored on CPU/GPU). Must be a positive integer; otherwise the default is used. |
-| `GGML_OPENVINO_NPU_COMPILE_CONFIG` | String | `not set` | NPU-only compiler mode parameters forwarded to OpenVINO as `NPU_COMPILATION_MODE_PARAMS`, for example `optimization-level=3`. |
-| `GGML_OPENVINO_STATEFUL_EXECUTION`| Boolean   | `0`        | Keep KV and supported recurrent caches inside the model. Single-slot CPU/GPU execution only.               |
-| `GGML_OPENVINO_DISABLE_CACHE`     | Boolean   | `0`        | Disable the in-process compiled-model / decoder cache (cache is on by default). Set to `1` to disable.      |
-| `GGML_OPENVINO_DISABLE_KV_SLICE`  | Boolean   | `0`        | Disable the KV-cache input-tensor slicing optimization (slicing is on by default on CPU/GPU). Set to `1` to disable. |
-| `GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT` | Boolean | `0`     | Disable the stateful KV-state sequence-axis relayout (relayout is on by default). It moves the KV state sequence axis from dim 1 to dim 2, so the GPU plugin can append new tokens in place instead of copying the whole state every token, and the reader side no longer transposes the whole accumulated state. Set to `1` to disable. |
-| `GGML_OPENVINO_MANUAL_GQA_ATTN`   | Boolean   | device-based | Tri-state. When **unset**, manual GQA attention is enabled by default on `GPU` and disabled on other devices. Set to a positive integer to force-enable, or `0` to force-disable. |
-| `GGML_OPENVINO_MEMORY_OPTIMIZE`   | Boolean   | `0`        | Umbrella switch for compile-time memory reductions. Enables `GGML_OPENVINO_REDUCE_COMPILE_MEM` and, on GPU, `GGML_OPENVINO_RELEASE_WEIGHTS` unless those fine-grained variables are explicitly set. |
-| `GGML_OPENVINO_REDUCE_COMPILE_MEM`| Boolean   | inherits from `GGML_OPENVINO_MEMORY_OPTIMIZE` | Reduce compile-time host memory use by streaming weight requantization and avoiding extra weight-node materialization where possible. Set explicitly to override the umbrella switch. |
-| `GGML_OPENVINO_RELEASE_WEIGHTS`   | Boolean   | inherits from `GGML_OPENVINO_MEMORY_OPTIMIZE` on GPU | GPU-only. Release host weight buffers after the compiled model cache can reuse the device/plugin copy. Requires stable graph shapes; dynamic workloads that need recompilation should leave this disabled. |
-| `GGML_OPENVINO_SPILL_DIR`         | String    | `not set`  | Directory for a disk-backed weight buffer. When set, the repacked weight buffer is mapped from an unlinked file on this path instead of anonymous memory, so its pages are reclaimable under memory pressure instead of staying pinned, cutting the load-time host memory peak. Must point at real storage; a tmpfs mount (e.g. `/tmp` on many systems) backs it with RAM and makes the peak worse. |
-| `GGML_OPENVINO_REQUANT_KQUANT`    | String    | `not set`  | Requantize Q6_K/Q5_K weights (and matching MoE expert weights) to a 4-bit target instead of the default Q8_0_C, trading accuracy for less memory traffic. One of `q4_asym64` (Q6_K/Q5_K only, keeps a real zero point at group 64), `q4_asym64_all` (also requantizes Q4_K), `q4_sym128` (Q6_K/Q5_K only), `q4_sym128_all` (Q4_K too, drops its per-group zero point), or `native` (no requantization). |
-| `GGML_OPENVINO_PROFILING`         | Integer   | `0`        | `1` logs execution timing; `2` or higher also enables OpenVINO and OpenCL profiling.                        |
-| `GGML_OPENVINO_DEBUG_NODE`       | String    | `not set`  | Add the named graph nodes as compiled outputs for debugging. Separate multiple names with commas.          |
-| `GGML_OPENVINO_MOE_OP`           | Boolean   | `1`        | On GPU, set to `0` to keep the unfused GatherMatmul path.                                                  |
-| `GGML_OPENVINO_DUMP_CGRAPH`       | Boolean   | `0`        | Dump the GGML compute graph to `cgraph_ov.txt`.                                                             |
-| `GGML_OPENVINO_DUMP_IR`           | Boolean   | `0`        | Serialize OpenVINO IR files with timestamps.                                                                |
-| `GGML_OPENVINO_DEBUG_INPUT`       | Boolean   | `0`        | Enable input debugging and print input tensor info.                                                         |
-| `GGML_OPENVINO_DEBUG_OUTPUT`      | Boolean   | `0`        | Enable output debugging and print output tensor info.                                                       |
-| `GGML_OPENVINO_PRINT_CGRAPH_TENSOR_ADDRESS` | Boolean | `0` | Print tensor address map once.                                                                           |
-| `GGML_OPENVINO_LOG_UNSUPPORTED_OPS`| Boolean   | `0`        | Log warning messages with tensor details and rejection reasons for any ops not supported by the OpenVINO backend. Emits at `WARN` level (requires `--log-verbosity >= 2`, enabled by default). |
+</details>

-> [!NOTE]
-> - `GGML_OPENVINO_STATEFUL_EXECUTION` is an **Experimental** feature for managing caches internally inside the OpenVINO model on CPUs and GPUs. Use a single slot (`-np 1`). KV caches retain the append-based state layout and sequence-axis optimization. Qwen3.5 adds recurrent cache states in their GGML layouts. Qwen3.5 requires an unsplit graph with model caching enabled and no recurrent rollback. A prompt starting at position 0 resets all states. State save/restore, sequence rewind, context shift, and mid-sequence graph replacement are unsupported. Stateful execution is not effective on NPUs.
-> - `GGML_OPENVINO_LOG_UNSUPPORTED_OPS` emits logs at `WARN` level (`GGML_LOG_WARN`), which requires application log verbosity `--log-verbosity >= 2` (or `-lv 2`).
-> - With `GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY=1`, use the same compilation settings as the export run. One directory can hold blobs for different models and settings; `GGML_OPENVINO_SPILL_DIR` does not affect the cache key and is ignored in cache-only mode. See [Compiled model cache](../../ggml/src/ggml-openvino/README.md) for the workflow and restrictions.
+## Runtime Configuration
+
+The backend needs no environment variable: it runs on the CPU with its defaults. Set `GGML_OPENVINO_DEVICE` to use the GPU or the NPU. All other variables are optional.
+
+```bash
+export GGML_OPENVINO_DEVICE=GPU            # String
+export GGML_OPENVINO_PROFILING=1           # Integer
+export GGML_OPENVINO_EXACT_ZERO_POINT=1    # Boolean: 1 (on) or 0 (off)
+
+# Show the variables in effect (log verbosity 4)
+./build/ReleaseOV/bin/llama-cli -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -c 1024 -lv 4 2>&1 | grep "settings from the environment"
+```
+
+- The backend reads the variables once, at start. Unset or empty means the default.
+- To lower the memory a model needs, see [Tips for Memory Optimization](../../ggml/src/ggml-openvino/dev_doc/OpenVINO-ENV-VARS.md#tips-for-memory-optimization).
+- A value that is not valid stops the backend. A `GGML_OPENVINO_*` name that the backend does not know gives a warning:
+
+```text
+ggml-openvino: GGML_OPENVINO_EXACT_ZERO_POINT=yes is not valid, use 1 (on) or 0 (off)
+ggml-openvino: unknown environment variable GGML_OPENVINO_DEVCE is ignored
+```
+
+### Basic
+
+| Variable | Device | Type | Default | Description |
+|---|---|---|---|---|
+| `GGML_OPENVINO_DEVICE` | all | String | `CPU` | Target device: `CPU`, `GPU` or `NPU`. For several GPUs, see the [device notes](#4-run-inference-with-openvino-backend). Examples: `GGML_OPENVINO_DEVICE=NPU`, `GGML_OPENVINO_DEVICE=GPU.0,GPU.1`. |
+| `GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR` | all | String | `not set` | Directory for the backend's cache of compiled models. A later start with the same model and settings loads faster. See [Compiled model cache](#compiled-model-cache). Example: `GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=/tmp/ov_cache`. |
+| `GGML_OPENVINO_NPU_CONFIG` | NPU | String | `not set` | OpenVINO properties for the NPU plugin, as comma-separated `KEY=VALUE` pairs. See [Plugin properties](#plugin-properties). Example: `GGML_OPENVINO_NPU_CONFIG="NPU_COMPILER_TYPE=PLUGIN"`. |
+| `GGML_OPENVINO_GPU_CONFIG` | GPU | String | `not set` | OpenVINO properties for the GPU plugin, as comma-separated `KEY=VALUE` pairs. See [Plugin properties](#plugin-properties). Example: `GGML_OPENVINO_GPU_CONFIG="INFERENCE_PRECISION_HINT=f16"`. |
+| `GGML_OPENVINO_CPU_CONFIG` | CPU | String | `not set` | OpenVINO properties for the CPU plugin, as comma-separated `KEY=VALUE` pairs. See [Plugin properties](#plugin-properties). Example: `GGML_OPENVINO_CPU_CONFIG="INFERENCE_PRECISION_HINT=f16"`. |
+| `GGML_OPENVINO_PROFILING` | all | Integer | `0` | `1` logs execution timing. `2` or higher also turns on OpenVINO and OpenCL profiling. Example: `GGML_OPENVINO_PROFILING=1`. |
+
+### Plugin properties
+
+`GGML_OPENVINO_NPU_CONFIG`, `GGML_OPENVINO_GPU_CONFIG` and `GGML_OPENVINO_CPU_CONFIG` pass OpenVINO plugin properties to the device in their name.
+
+```bash
+# Format: comma-separated KEY=VALUE pairs. The backend uses the variable of the selected device only.
+export GGML_OPENVINO_DEVICE=NPU
+export GGML_OPENVINO_NPU_CONFIG="NPU_COMPILER_TYPE=PLUGIN,COMPILATION_NUM_THREADS=4"
+
+# CPU: compute in FP16, as the GPU and the NPU do. This helps to debug the accuracy of a quantized model.
+export GGML_OPENVINO_DEVICE=CPU
+export GGML_OPENVINO_CPU_CONFIG="INFERENCE_PRECISION_HINT=f16"
+```
+
+- Property names and values are those of OpenVINO, for example `YES` and `NO`. They are not the `1` and `0` of the `GGML_OPENVINO_*` variables.
+- The backend applies the variable last, so it overrides the properties that the backend sets.
+- OpenVINO checks the properties when the model loads. If it rejects one, the load fails and the error names the variable.
+- All properties and their values are in the OpenVINO documentation: [NPU device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html), [GPU device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html), [CPU device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device.html).
+- NPU: the backend sets several properties by default. They are listed in the [developer documentation](../../ggml/src/ggml-openvino/dev_doc/OpenVINO-ENV-VARS.md#npu-plugin-properties).
+- GPU and CPU: the backend sets no property, except `CACHE_DIR` and `CACHE_MODE=OPTIMIZE_SIZE` when `GGML_OPENVINO_CACHE_DIR` is set.
+
+`NPU_COMPILER_TYPE` selects the NPU compiler. Both forms below work; use the first one. If both are set, `GGML_OPENVINO_NPU_CONFIG` takes precedence.
+
+```bash
+# Values: DRIVER (the compiler in the NPU driver; backend default), PLUGIN (the compiler in OpenVINO),
+# PREFER_PLUGIN (OpenVINO default: PLUGIN when it is available, DRIVER if not).
+export GGML_OPENVINO_NPU_CONFIG="NPU_COMPILER_TYPE=PLUGIN"   # recommended form
+export GGML_OPENVINO_NPU_COMPILER_TYPE=PLUGIN                # older name, still works
+```
+
+### Advanced tuning
+
+The defaults fit most models. The variables below trade one property for another, for example speed for accuracy.
+
+<details>
+<summary><b>Advanced tuning variables</b></summary>
+
+| Variable | Device | Type | Default | Description |
+|---|---|---|---|---|
+| `GGML_OPENVINO_PREFILL_CHUNK_SIZE` | NPU | Integer | `256` | Number of tokens that the NPU processes in one prefill step. A larger value can make long prompts faster, and makes the first load longer. Zero or a negative value means the default. Example: `GGML_OPENVINO_PREFILL_CHUNK_SIZE=512`. |
+| `GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT` | NPU | String | `q8_0_c` | How a `Q6_K` `token_embd.weight` is stored for the NPU: `q8_0_c` (8-bit, one scale per row), `f16`, `q4_0_128` (4-bit, one scale per 128 weights) or `q4_0_c` (4-bit, one scale per row). The 4-bit layouts use less memory at some cost in accuracy; `f16` uses more memory. Keep the default unless memory is tight. |
+| `GGML_OPENVINO_GDN_DECOMPOSE` | all | Boolean | auto | Translate `GATED_DELTA_NET` (Qwen3.5) into MatMul operations instead of the OpenVINO `GatedDeltaNet` op. Unset: on for the Lunar Lake NPU, off elsewhere. `0` turns it off, `1` turns it on for any device. Try `1` for Qwen3.5 on the Panther Lake NPU. |
+| `GGML_OPENVINO_MANUAL_GQA_ATTN` | CPU, GPU | Boolean | on for GPU | Build grouped-query attention from separate MatMul operations. Unset: on for GPU, off for CPU and NPU. `1` turns it on, `0` turns it off. Try `1` on the CPU for faster decode. |
+| `GGML_OPENVINO_REQUANT_KQUANT` | CPU, GPU | String | `not set` | Requantize `Q6_K` and `Q5_K` weights to a 4-bit layout instead of `Q8_0_C`: faster decode for lower accuracy. One of `q4_asym64`, `q4_asym64_all` (also `Q4_K`), `q4_sym128`, `q4_sym128_all` (also `Q4_K`) or `native` (no requantization). On GPU with `Q4_K_M` files use `q4_asym64_all`, because `q4_asym64` does not compile there. |
+| `GGML_OPENVINO_EXACT_ZERO_POINT` | CPU, GPU | Boolean | `0` | Keep the exact zero point of asymmetric weights (`Q4_1`, `Q4_K`, `Q5_1`, `Q5_K`) instead of rounding it to an integer. This improves accuracy and lowers speed. Use it when accuracy matters more than speed. |
+
+</details>
+
+### Compiled model cache
+
+The cache keeps compiled models on disk, so later starts of the same model load faster. Use the same model and the same settings in every run.
+
+```bash
+# CPU or GPU: the first run compiles the model and writes the cache. Later runs read it.
+export GGML_OPENVINO_DEVICE=GPU
+export GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=/tmp/ov_cache
+./build/ReleaseOV/bin/llama-cli -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -c 1024
+
+# CPU or GPU, later runs (optional): require the cached model and skip the weight upload and the compilation.
+export GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY=1
+
+# NPU: add the weightless bank.
+export GGML_OPENVINO_DEVICE=NPU
+export GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=/tmp/ov_cache
+export GGML_OPENVINO_NPU_WEIGHTLESS_BANK=1
+./build/ReleaseOV/bin/llama-cli -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -c 512
+
+# NPU, later runs (optional): use less memory. The run fails if a cached model is missing.
+export GGML_OPENVINO_SELF_CONTAINED_MMAP=1
+```
+
+<details>
+<summary><b>Cache variables</b></summary>
+
+| Variable | Device | Type | Default | Description |
+|---|---|---|---|---|
+| `GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY` | CPU, GPU | Boolean | `0` | Require an existing cached model and skip the weight upload and the compilation. Needs mmap model loading (the default on Linux and Windows) and a model that runs fully on OpenVINO. |
+| `GGML_OPENVINO_NPU_WEIGHTLESS_BANK` | NPU | Boolean | `0` | Requires `GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR`. Write the prefill and decode models without weights, plus one shared weights file. Later runs read the weights from that file and keep one copy in memory. |
+| `GGML_OPENVINO_SELF_CONTAINED_MMAP` | NPU | Boolean | `0` | Later runs only. Requires `GGML_OPENVINO_NPU_WEIGHTLESS_BANK=1` and an existing cache. Use the memory-mapped GGUF file as the model buffer, which lowers the host memory peak. Fails if a cached model is missing; run once without it to build the cache. |
+| `GGML_OPENVINO_CACHE_DIR` | all | String | `not set` | Directory for the cache of the OpenVINO plugin itself. `GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR` is the recommended cache. |
+
+- One directory can hold the cached models of different models and settings.
+- NPU: each prefill length enters the cache in a run without `GGML_OPENVINO_SELF_CONTAINED_MMAP`.
+- See [Compiled model cache](../../ggml/src/ggml-openvino/README.md#compiled-model-cache) for the workflow and the restrictions.
+
+</details>
+
+### Deprecated
+
+| Variable | Device | Type | Default | Description |
+|---|---|---|---|---|
+| `GGML_OPENVINO_STATEFUL_EXECUTION` | CPU, GPU | Boolean | `0` | **Deprecated**, to be removed. Keep the KV cache inside the OpenVINO model. Use the default, stateless execution (`0`). |
+
+<details>
+<summary>Limits of stateful execution</summary>
+
+- CPU and GPU only; it has no effect on the NPU.
+- One slot only (`-np 1`).
+- State save and restore, sequence rewind, context shift and recurrent rollback are not supported.
+- Qwen3.5 needs a graph that runs fully on OpenVINO, with model caching on.
+- A prompt that starts at position 0 resets all states.
+
+</details>
+
+### Debugging and development
+
+Debug output, switches that turn a default optimization off, experimental options and older variable names are in the [developer documentation](../../ggml/src/ggml-openvino/dev_doc/OpenVINO-ENV-VARS.md).

 ### Example Usage

 #### GPU Inference with Profiling

 ```bash
-# If the system has multiple GPUs, use GPU.0 or GPU.1 to explicitly target a specific GPU.
-
-# Linux
-export GGML_OPENVINO_CACHE_DIR=/tmp/ov_cache
+export GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=/tmp/ov_cache
 export GGML_OPENVINO_PROFILING=1
 export GGML_OPENVINO_DEVICE=GPU
-export GGML_OPENVINO_STATEFUL_EXECUTION=1

 ./build/ReleaseOV/bin/llama-simple -m ~/models/Llama-3.2-1B-Instruct-Q4_K_M.gguf -n 50 "The story of AI is "
+```
+
+<details>
+<summary><b>Windows</b></summary>

-# Windows Command Line
-set GGML_OPENVINO_CACHE_DIR=C:\tmp\ov_cache
+```cmd
+REM Windows Command Prompt
+set GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=C:\tmp\ov_cache
 set GGML_OPENVINO_PROFILING=1
 set GGML_OPENVINO_DEVICE=GPU
-set GGML_OPENVINO_STATEFUL_EXECUTION=1
+build\ReleaseOV\bin\llama-simple.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf" -n 50 "The story of AI is "
+```

+```powershell
 # Windows PowerShell
-$env:GGML_OPENVINO_CACHE_DIR = "C:\tmp\ov_cache"
+$env:GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR = "C:\tmp\ov_cache"
 $env:GGML_OPENVINO_PROFILING = "1"
 $env:GGML_OPENVINO_DEVICE = "GPU"
-$env:GGML_OPENVINO_STATEFUL_EXECUTION = "1"
-
 build\ReleaseOV\bin\llama-simple.exe -m "C:\models\Llama-3.2-1B-Instruct-Q4_K_M.gguf" -n 50 "The story of AI is "
-
 ```

+</details>
+
 ## Known Limitations

 **General (all devices)**

 - Llama.cpp OpenVINO backend currently supports a subset of GGML ops and text-only models. Unsupported ops or unsupported op shapes/cases fail during OpenVINO translation.
 - Multimodal features (audio/image/video) are a work in progress.
-- Limited Embedding and Reranking model support.
+- Embedding models: reranking models are not validated.
 - Llama.cpp tool coverage across CPU/GPU/NPU is not uniform.

-**Tool-specific**
-
-- `llama-bench`: requires `-fa 1` (flash-attention).
-- `llama-cli --context-shift`: stateless only (`GGML_OPENVINO_STATEFUL_EXECUTION=0`). In stateful mode the KV cache is owned by the OpenVINO model and cannot be shifted externally.
-- `llama-server`: only one chat session/thread when `GGML_OPENVINO_STATEFUL_EXECUTION=1`.
-
 **GPU-specific**

-- `llama-server -np > 1`: concurrent requests are batched together, which may slightly reduce per-request throughput.
+- Multiple GPUs (`GGML_OPENVINO_DEVICE=GPU.0,GPU.1` with `-sm layer`):
+  - `-sm tensor` is not supported.
+  - Copies between GPUs go through host memory. There is no asynchronous copy and no peer-to-peer copy.
+  - Only GPUs can be combined. A list with several NPUs, or with a GPU and the CPU or the NPU, is rejected.
+- MoE models use the GPU plugin's fused MoE op, which runs in f16 only:
+  - granitemoe models (e.g. `granite-3.1-1b-a400m`) give wrong output.
+    - **Workaround:** `GGML_OPENVINO_MOE_OP=0`. Output is correct, but prompt processing is much slower.
+  - `GGML_OPENVINO_GPU_CONFIG="INFERENCE_PRECISION_HINT=f32"` fails to compile.
+- `llama-perplexity` with MoE models: set `-b` to the same value as `-c`, so that each batch holds one sequence.

 **NPU-specific**

 - Default context resolves to the model's training context (e.g. 131072 for Llama 3.2 1B), which can OOM or fail or degrade performance on NPU. Inspect the resolved value with `-lv 3`.
-  - **Workaround:** Pass an explicit `-c <N>`, e.g. `-c 1024`.
+  - **Workaround:** Pass an explicit `-c <N>`, e.g. `-c 512`.
 - NPU device uses a static graph with a fixed prefill chunk size (defaults to 256), configurable with `GGML_OPENVINO_PREFILL_CHUNK_SIZE`. Large prefill/batch settings may need tuning.
 - `llama-server -np > 1` (multiple parallel sequences) is not supported.
 - `llama-perplexity`: requires `-b 512` or smaller.
+- Flash attention must stay on, which is the default (`-fa auto`). With `-fa off`, the run fails on the NPU.
+- `llama-embedding` fails on the NPU.
+- Lunar Lake NPU: decode slows down sharply when the context is large. The context size at which this starts depends on the model.
+  - **Workaround:** Pass the smallest context that fits the use, e.g. `-c 512`, or `-c 256` for larger models.
+- Panther Lake NPU: the load of a Qwen3.5 model can end with a segmentation fault in the NPU compiler of OpenVINO 2026.4.1.
+  - **Workaround:** Set `GGML_OPENVINO_NPU_CONFIG=COMPILATION_NUM_THREADS=1`.
+- Panther Lake NPU: `GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT=f16` and `q4_0_128` can give wrong output. Keep the default.
+- Lunar Lake NPU: `granite-4.0-1b` gives wrong output. The activations of the model exceed the FP16 range, and the NPU computes in FP16. The model runs correctly on CPU and GPU.
+- Lunar Lake NPU: `gemma-4-12B` does not run. The first inference fails with `ZE_RESULT_ERROR_UNINITIALIZED` from the NPU driver. The model runs on CPU and GPU.
+- The first load of a model on the NPU compiles it for the device, which can take minutes for large models.
+  - **Workaround:** Use the [compiled model cache](#compiled-model-cache).

 > [!NOTE]
 > The OpenVINO backend is actively under development. Fixes and improvements are underway, and this document will continue to be updated.
diff --git a/ggml/src/ggml-openvino/CMakeLists.txt b/ggml/src/ggml-openvino/CMakeLists.txt
index 347c24f4f..3ed43b9fa 100644
--- a/ggml/src/ggml-openvino/CMakeLists.txt
+++ b/ggml/src/ggml-openvino/CMakeLists.txt
@@ -31,3 +31,20 @@ if (GGML_OPENVINO)
         message(FATAL_ERROR "OpenVINO: OpenVINO toolkit supports x86-64 and arm64 but not ${CMAKE_SYSTEM_PROCESSOR}")
     endif()
 endif()
+
+# Released runtimes retain the ordinary cache translators.
+include(CheckCXXSourceCompiles)
+unset(GGML_OPENVINO_HAS_CACHE_UPDATE CACHE)
+set(CMAKE_REQUIRED_LIBRARIES openvino::runtime)
+set(CMAKE_REQUIRED_INCLUDES "${CMAKE_CURRENT_SOURCE_DIR}/openvino/op")
+check_cxx_source_compiles("#include <recurrent_cache_update.hpp>
+int main() {
+    auto conv = std::make_shared<ov::op::internal::RecurrentConvCacheUpdate>(ov::OutputVector{}, 3);
+    auto gdn = std::make_shared<ov::op::internal::RecurrentGatedDeltaNetUpdate>(ov::OutputVector{});
+}"
+    GGML_OPENVINO_HAS_CACHE_UPDATE)
+unset(CMAKE_REQUIRED_LIBRARIES)
+unset(CMAKE_REQUIRED_INCLUDES)
+if(GGML_OPENVINO_HAS_CACHE_UPDATE)
+    target_compile_definitions(ggml-openvino PRIVATE GGML_OPENVINO_HAS_CACHE_UPDATE=1)
+endif()
diff --git a/ggml/src/ggml-openvino/README.md b/ggml/src/ggml-openvino/README.md
index 6a23f62bf..0b4f31291 100644
--- a/ggml/src/ggml-openvino/README.md
+++ b/ggml/src/ggml-openvino/README.md
@@ -1,3 +1,38 @@
+# Qwen MTP
+
+The default Qwen3.5/Qwen3.6 GPU path is dynamic stateless execution with in-place fused recurrent ops and stable MTP sampling. Build against the matching OpenVINO runtime/GPU plugin with independent `RecurrentConvCacheUpdate` and `RecurrentGatedDeltaNetUpdate` ops. The native Update ops return computation results and update writable cache inputs in place, including rollback snapshots where required. No recurrent/sampling opt-in flag is needed on supported graphs. See the [development documentation index](dev_doc/README.md) for the implementation and the MTP sampling graph.
+
+Use `--spec-type draft-mtp --spec-draft-n-max 2` with a GGUF that contains MTP weights. Parallel execution uses `-np` and `--kv-unified`. CPU snapshot execution, NPU execution, and forced static MTP are not supported.
+
+Stateless recurrent execution uses the same reset, reorder, and partial cache-writeback paths at every slot capacity, including one slot without MTP.
+
+The matched stateless GPU path uses `RecurrentConvCacheUpdate` and
+`RecurrentGatedDeltaNetUpdate`. Each cache is a writable Parameter backed by one
+stable caller-owned USM allocation. The ops return only convolution or attention;
+there are no recurrent cache Results. Required source rows are staged before
+writes, preserving resets, cyclic reorders and rollback. Cache readers must be
+exclusive, including Reshape aliases.
+`GGML_OPENVINO_DISABLE_FUSED_RECURRENCE=1` retains the ordinary cache decomposition.
+Unmatched snapshot GDN uses ordinary GDN token prefixes with functional writeback.
+Build both repos against the matching single-output op spec. Translation revision
+15 requires regenerating compiled-model blobs once without cache-only mode.
+
+Cache-only MTP serving still requires the separate OpenVINO dynamic output-binding fix (PR #39354). Removing recurrent cache Results does not eliminate that independent trigger in imported MTP graphs.
+
+Set `GGML_OPENVINO_ENABLE_KV_4D=1` to enable the upstream stateless in-place
+attention KV update pattern where eligible. It binds full-capacity cache tensors
+and lets the GPU fusion expose the live prefix to attention. The current upstream
+eligibility check supports one physical stream (including unified KV); explicit
+split multi-slot KV retains the ordinary full-cache path. This option does not
+change recurrent cache handling. Unified KV is recommended for multi-slot GPU
+execution because it enables this in-place attention update.
+
+The recommended Qwen3.6-35B-A3B Q4_K_M setup uses GPU with `GGML_OPENVINO_REQUANT_KQUANT=q4_asym64_all` and `GGML_OPENVINO_NATIVE_SOFTPLUS=1`. Leave `GGML_OPENVINO_KV_SLICE` at its default (`1`). See [attention KV layout](dev_doc/OPENVINO-RECURRENT-CACHE.md#attention-kv-layout) for eligibility.
+
+`q4_asym64_all` converts Q4_K/Q5_K/Q6_K/Q8_0/Q5_1 weights, including dense Q8_0 projections, to asymmetric INT4 with group size 64. `token_embd.weight` and `output.weight` retain symmetric channel-wise INT8. Q8_0/Q5_1 rows whose width is not divisible by 64 also use channel-wise INT8. This follows the primary/backup precision scheme of NNCF `INT4_ASYM`, `ratio=1.0`, `group_size=64`, `backup_mode=INT8_SYM` for these supported source types; it requantizes GGUF values rather than the original floating-point model and leaves floating-point weights unchanged. Earlier versions kept dense Q8_0 at INT8/group-32. Repopulate compiled-model caches without cache-only mode after this change; the cache fingerprint invalidates the old precision policy.
+
+See the [implementation guide](dev_doc/OPENVINO-RECURRENT-CACHE.md) for buffer ownership and rollback.
+
 # Compiled model cache

 `GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR` exports compiled CPU/GPU graphs with their weights. It bypasses the plugin-level `GGML_OPENVINO_CACHE_DIR` and uses `OPTIMIZE_SPEED`, so weightless caching is disabled.
@@ -7,7 +42,6 @@ One directory can hold blobs for different models and compilation settings. Run
 ```sh
 GGML_OPENVINO_DEVICE=GPU \
 GGML_OPENVINO_NATIVE_SOFTPLUS=1 \
-GGML_OPENVINO_DISABLE_KV_SLICE=1 \
 GGML_OPENVINO_REQUANT_KQUANT=q4_asym64_all \
 GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=/path/to/qwen-cache \
 ./build/ReleaseOV/bin/llama-bench -m /path/to/model.gguf -r 1
@@ -20,7 +54,6 @@ On later runs, supply the same compilation settings and enable `GGML_OPENVINO_CO
 ```sh
 GGML_OPENVINO_DEVICE=GPU \
 GGML_OPENVINO_NATIVE_SOFTPLUS=1 \
-GGML_OPENVINO_DISABLE_KV_SLICE=1 \
 GGML_OPENVINO_REQUANT_KQUANT=q4_asym64_all \
 GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=/path/to/qwen-cache \
 GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY=1 \
@@ -29,10 +62,22 @@ GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY=1 \

 Cache-only mode allocates backend address space without filling weight pages. On Windows, this also uses system commit capacity. The model-buffer size in the loader log is this virtual size. Weight uploads only record source identity; they do not read or requantize the weights. Graph conversion and compilation are skipped. Runtime buffers are still allocated and populated normally.

-Cache-only mode uses the settings provided by the current process. Keep these values exactly the same, including set versus unset: `GGML_OPENVINO_REQUANT_KQUANT`, `GGML_OPENVINO_NATIVE_SOFTPLUS`, `GGML_OPENVINO_DISABLE_KV_SLICE`, `GGML_OPENVINO_MANUAL_GQA_ATTN`, `GGML_OPENVINO_STATEFUL_EXECUTION`, `GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT`, `GGML_OPENVINO_DISABLE_REMOTE_OUTPUTS`, `GGML_OPENVINO_REDUCE_COMPILE_MEM`, `GGML_OPENVINO_MEMORY_OPTIMIZE`, `GGML_OPENVINO_PROFILING`, and `GGML_OPENVINO_DEBUG_NODE`. On GPU, also repeat `GGML_OPENVINO_MOE_OP=0` if used. `GGML_OPENVINO_SPILL_DIR` is optional on the first run and ignored in cache-only mode; host-weight release is disabled in cache-only mode.
+Cache-only mode uses the settings provided by the current process. Keep these values exactly the same, including set versus unset: `GGML_OPENVINO_REQUANT_KQUANT`, `GGML_OPENVINO_EXACT_ZERO_POINT`, `GGML_OPENVINO_NATIVE_SOFTPLUS`, `GGML_OPENVINO_MANUAL_GQA_ATTN`, `GGML_OPENVINO_STATEFUL_EXECUTION`, `GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT`, `GGML_OPENVINO_DISABLE_REMOTE_OUTPUTS`, `GGML_OPENVINO_REDUCE_COMPILE_MEM`, `GGML_OPENVINO_MEMORY_OPTIMIZE`, `GGML_OPENVINO_PROFILING`, `GGML_OPENVINO_DISABLE_SHAPE_FROM_MASK`, and `GGML_OPENVINO_DEBUG_NODE`, `GGML_OPENVINO_ENABLE_KV_4D`, and `GGML_OPENVINO_DISABLE_FUSED_RECURRENCE`. On GPU, also repeat `GGML_OPENVINO_MOE_OP=0` if used. `GGML_OPENVINO_SPILL_DIR` is optional on the first run and ignored in cache-only mode; host-weight release is disabled in cache-only mode.
+
+`GGML_OPENVINO_KV_SLICE` is a runtime binding option for dynamic graphs and can differ between cache population and cache-only runs. The same dynamic compiled graph accepts full-capacity and live-prefix attention K/V tensors. Slicing is enabled by default for contiguous ordinary K/V roots with one allocated stream, one active sequence at offset zero, and no sliding-window attention. Recurrent convolution/GDN caches and their rollback snapshots always retain their full stored row bank. Other layouts use full bindings. Repopulate caches made before this option was removed from the fingerprint once without cache-only mode.

 A missing or incompatible graph fails with an error instead of compiling with absent weights. The fingerprint uses the dynamic graph's topology, ports, model parameters, weights, settings, and OpenVINO version; changing only dynamic token or KV sizes does not require a new entry. A different workload can still require another graph; populate it first without cache-only mode.

+## MTP sampling and active slots
+
+Qwen3.5/Qwen3.6 MTP uses a pure top-k backend sampler. Dynamic stateless execution retains its sampling branches for every configured slot and output row, including inactive branches. One packed `sampling_rows` input selects logits rows at runtime. Graph node names and candidate-output ports stay fixed, so changing the active subset of four slots reuses the same compiled draft model instead of creating up to 15 variants.
+
+Active candidate IDs and scores are written to the normal GGML outputs. Inactive branches read row 0 (the padding row when there are no logits) and write to private scratch tensors (device USM on GPU), without changing sampler state or touching inactive GGML buffers. The top-k computations remain separate per branch; this trades some extra sampling work for a stable graph interface. Other sampler chains retain their existing branch-selection behavior.
+
+This is one draft model per compatible graph configuration, not one compiled blob for the whole server. The trunk and other graph stages still have their own entries. Different sampler configurations, recurrent capacities, rollback modes or incompatible KV layouts can require additional models. Cache-only mode still requires initial preparation; changing just the active subset no longer requires exhaustive subset priming.
+
+The generic recurrent-cache translation invalidates blobs made with the former single-slot shortcuts. Run once without `GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY` to populate compatible entries before enabling cache-only mode again.
+
 ## Restrictions

 - Cache-only mode requires Linux or Windows and mmap loading (`--load-mode mmap`, or the default when all selected devices support mmap). Do not use tensor validation or mlock when trying to avoid weight reads.
diff --git a/ggml/src/ggml-openvino/dev_doc/MTP-SAMPLING-GRAPH.md b/ggml/src/ggml-openvino/dev_doc/MTP-SAMPLING-GRAPH.md
new file mode 100644
index 000000000..ef50e153d
--- /dev/null
+++ b/ggml/src/ggml-openvino/dev_doc/MTP-SAMPLING-GRAPH.md
@@ -0,0 +1,45 @@
+# Stable multislot MTP graph
+
+The captured UD draft graph contains 107 nodes regardless of which of four slots are active. It is the single MTP attention/MoE layer, not the recurrent trunk. Two speculative tokens require two consecutive draft calls, batched over active sequences.
+
+```text
+MTP token IDs + input hidden state
+             |
+  0-84: MTP layer and normalization, N active sequences
+             |
+     85: logits [248320, N]
+     86: PAD adds one safe row
+             |
+             +-- 87-91:   slot 0 sampling
+             +-- 92-96:   slot 1 sampling
+             +-- 97-101:  slot 2 sampling
+             +-- 102-106: slot 3 sampling
+```
+
+Each branch contains VIEW, two Reshapes, TopK (ten candidate IDs), and GET_ROWS (their scores). Ten is the candidate count, not the speculative-token count.
+
+| Active slots | Sampling COMPUTE flags, slots 0-3 | Raw/current nodes | Old filtered nodes | Packed sampling_rows |
+|---|---|---:|---:|---|
+| 3 | `[0,0,0,1]` | 107 | 92 | `[0,0,0,0]` |
+| 0,2 | `[1,0,1,0]` | 107 | 97 | `[0,0,1,0]` |
+| All four | `[1,1,1,1]` | 107 | 107 | `[0,1,2,3]` |
+
+Old counts apply the old COMPUTE filter to the captures. Unselected nodes remain in the raw cgraph. Previously, removing them changed translated topology, slot-specific row Parameter names and output ports; even equal-sized subsets such as `{0,1}` and `{0,2}` differed. This could trigger up to 15 draft variants, with compilation/import on first use or a missing-cache error in cache-only mode.
+
+Now all pure top-k branches remain. Their names include stable graph ordinals even when inactive. One int64 `sampling_rows` Parameter supplies row indices at runtime. For `{0,2}`, the two logits rows belong to slots 0 and 2:
+
+```text
+sampling_rows = [0, 0, 1, 0]
+slot 0 -> row 0 -> normal candidate buffers
+slot 1 -> row 0 -> private scratch, ignored
+slot 2 -> row 1 -> normal candidate buffers
+slot 3 -> row 0 -> private scratch, ignored
+```
+
+The vector is not an active mask. Host-side COMPUTE flags select normal versus scratch output destinations. PAD provides a valid row when there are no logits. Inactive outputs use device USM on GPU and do not affect sampler state or acceptance. Other sampler chains retain their selected-branch semantics.
+
+Only small sampling work executes for inactive slots. The MTP layer and output projection still process N active rows. Top-k is separate per branch, not yet one batched TopK. The observed draft interface has 10 inputs and 12 outputs: eight candidate ID/score outputs, hidden state, full logits and two KV Results.
+
+Dump shapes use GGML order `[width, rows, dim2, dim3]`; OV reverses it. `view_offs` is in bytes: one logits row occupies `248320 * 4 = 993280` bytes. Buffer labels describe GGML storage, not the device executing imported GPU operations.
+
+The captures used cache-only mode, four slots, context 8192 and the default two-token MTP settings.
diff --git a/ggml/src/ggml-openvino/dev_doc/OPENVINO-RECURRENT-CACHE.md b/ggml/src/ggml-openvino/dev_doc/OPENVINO-RECURRENT-CACHE.md
new file mode 100644
index 000000000..7dfb742b1
--- /dev/null
+++ b/ggml/src/ggml-openvino/dev_doc/OPENVINO-RECURRENT-CACHE.md
@@ -0,0 +1,62 @@
+# Current Qwen GPU implementation
+
+The default Qwen3.5/Qwen3.6 path is dynamic stateless execution with `RecurrentConvCacheUpdate`, `RecurrentGatedDeltaNetUpdate`, and stable pure-top-k MTP translation. It supports one or multiple slots and rollback. The implementation applies to `qwen35` and `qwen35moe`; the reference model is the UD Qwen3.6-35B-A3B MTP GGUF.
+
+Build against the matching OpenVINO runtime/GPU plugin with single-output Update ops. Configure `GGML_OPENVINO_DEVICE=GPU`, native softplus, and the documented requantization policy. The cache translation revision is 15. Historical measurements retain their original source revisions.
+
+The separate dynamic output-binding fix in OpenVINO PR #39354 remains required for cache-only MTP serving. Native cache-op export/import tests pass without it, but Qwen3.6 imported MTP graphs still trigger the missing-input-memory error without the fix.
+
+## Buffer ownership and operation boundaries
+
+| State | Storage and lifetime | Update path |
+|---|---|---|
+| Attention K/V | Full caller-owned USM allocation, sized by context and KV layout | Native stateless KV fusion when eligible; SDPA consumes the live prefix |
+| Convolution `cache_r` | Full caller-owned USM row bank, sized by slot/snapshot capacity | Fused reset, original-row gather/reorder, convolution and ordered history/snapshot writes |
+| GDN `cache_s` | Full caller-owned USM row bank, sized by slot/snapshot capacity | Fused reset, original-row gather/reorder, GDN and ordered final/snapshot writes |
+| Runtime controls | Reused USM host storage on GPU | Row maps, offsets and write controls remain runtime inputs |
+| Protected source rows | Private device scratch | Staged before writes to protect cycles, duplicate sources and overlaps |
+| Inactive sampling results | Private device-USM scratch | Discarded without touching inactive GGML output buffers |
+
+Attention context capacity does not set recurrent-cache size. Each cache input Parameter binds a stable writable allocation; recurrent cache Results are omitted. There are no rotation banks or pointer swaps.
+
+The GPU plugin uses two stages: first preserve required original rows in scratch while applying logical reset and precision rounding; then compute and perform ordered writes. This avoids overwriting a source before another destination has read it. Identity reorder sources need no scratch writes. In-place execution removes the spare full output bank, not every cache read or internal memory transaction.
+
+`RecurrentConvCacheUpdate` has nine inputs, including depthwise weights, and returns convolution `[1,B,T,C]`. `RecurrentGatedDeltaNetUpdate` returns attention and writes the newest `min(T,snapshot_count)` states into its cache input, newest first. Neither op returns a cache tensor.
+
+## Convolution cache stores
+
+`build_conv_state_store` emits one CPY for the final history when rollback is disabled. With rollback enabled, it emits one SET_ROWS from overlapping `[history, window, channel, sequence]` views into strided snapshot planes. Window starts advance in time while destination planes descend, so plane zero holds the newest history. The fixed ARANGE/scale/cast chain supplies this descending slot order.
+
+The backend recognizes this structure and refreshes source starts, cache heads, sequence counts, window counts and plane strides on every batch. Native fusion passes runtime `[source column, destination slot]` pairs to the existing convolution op; the pair-count dimension is dynamic. Functional translation gathers the same windows, scatters them into the full cache, and preserves unwritten slots and features. The native op spec is unchanged.
+
+## Generic fallback and restrictions
+
+Writable cache inputs must be GPU RemoteTensors with exclusive graph readers, optionally through Reshape aliases. The GPU plugin rejects shared cache readers and computed cache banks. Other inputs and the computation output must not overlap the cache. The frontend skips fusion for unmatched cache reads and retains functional cache writeback. Restore or clear state after a failed inference before retrying.
+
+`GGML_OPENVINO_DISABLE_FUSED_RECURRENCE=1` retains ordinary cache decomposition. Naive graphs skip full-model recurrence fusion. Unmatched snapshot GDN uses ordinary GDN token prefixes to preserve state outputs. The fused path requires the matching native runtime; otherwise dynamic CPU and GPU execution uses ordinary GDN. Stateful and NPU MTP are outside this change.
+
+## Attention KV layout
+
+`GGML_OPENVINO_ENABLE_KV_4D=1` enables eligible native stateless in-place attention updates. The current eligibility covers one physical stream, including unified KV. Explicit split multi-slot KV retains the full-cache path and can be much slower even with only one active slot. Use `--kv-unified` for multi-slot OV GPU serving.
+
+Leave `GGML_OPENVINO_KV_SLICE` at its default (`1`). Live-prefix slicing applies to eligible ordinary attention K/V roots, not recurrent caches. In-place KV fusion can bind the full allocation while exposing a live prefix internally. KV-slice on/off is a runtime binding choice and does not require different compiled blobs; incompatible graph layouts/capacities can still do so.
+
+## Stable MTP sampling
+
+The [graph walkthrough](MTP-SAMPLING-GRAPH.md) describes the fixed sampling interface. The backend keeps pure top-k branches for all configured slots, packs runtime row indices, and uses scratch for inactive outputs. It does not change the original cgraph or execute inactive stateful sampler chains. Only the small sampling branches stay at configured capacity; the expensive draft layer processes active rows.
+
+One compatible draft graph is shared across active subsets. Trunk and genuinely different graph stages/configurations remain separate. Prepare a compatible revision-15 cache once, then use cache-only mode. Exhaustive active-subset priming is no longer required, though first-use model import and runtime preparation still have a cost.
+
+## Validation
+
+Native OV tests cover shape/attribute serialization, reset, cycles, duplicates, overlapping writes, mixed precision, changing shapes, writable cache input rebinding, exclusive Reshapes, shared readers, host/device USM and export/import. Convolution is checked against a host reference; GDN uses an independent cache mapping around ordinary GDN. See `docs/dev/qwen35-llamacpp-recurrent-cache.md` and `recurrent_cache_update.cpp` in the OV checkout for contracts and tests.
+
+## Debugging and source map
+
+- `ggml-decoder.cpp`: dynamic dimensions, control inputs, output names and `cgraph_ov.txt` dump.
+- `utils.cpp`: compute-branch filtering, compiled reuse and input/output binding.
+- `openvino/recurrent_cache_fusion.cpp`: structural pattern matching and native Update op translation.
+- `openvino/translate_session.cpp`: runtime sampling aliases and model construction.
+- OV `recurrent_cache_update` implementation: writable-input validation, source staging and GPU computation.
+
+`GGML_OPENVINO_DUMP_CGRAPH=1` dumps the raw graph before filtering. It includes COMPUTE/OUTPUT flags and byte offsets and overwrites `cgraph_ov.txt` on each call. Enable dumps only for diagnosis; use the captured examples for a static view. GPU profiles and OpenCL copy traces add overhead, and lack of an explicit copy call does not prove zero traffic inside a kernel.
diff --git a/ggml/src/ggml-openvino/dev_doc/OpenVINO-ENV-VARS.md b/ggml/src/ggml-openvino/dev_doc/OpenVINO-ENV-VARS.md
new file mode 100644
index 000000000..a43ac1d6a
--- /dev/null
+++ b/ggml/src/ggml-openvino/dev_doc/OpenVINO-ENV-VARS.md
@@ -0,0 +1,221 @@
+# OpenVINO backend environment variables for development
+
+The variables for users, and the rules for values, are in [docs/backend/OPENVINO.md](../../../../docs/backend/OPENVINO.md#runtime-configuration). This file lists the other `GGML_OPENVINO_*` variables. They are for debugging, comparisons and experiments, and they can change or be removed.
+
+## Debug output
+
+| Variable | Device | Type | Default | Description |
+|---|---|---|---|---|
+| `GGML_OPENVINO_DUMP_CGRAPH` | all | Boolean | `0` | Dump the GGML compute graph to `cgraph_ov.txt`. |
+| `GGML_OPENVINO_DUMP_IR` | all | Boolean | `0` | Write the OpenVINO IR files, with timestamps. |
+| `GGML_OPENVINO_DEBUG_INPUT` | all | Boolean | `0` | Print the input tensors. |
+| `GGML_OPENVINO_DEBUG_OUTPUT` | all | Boolean | `0` | Print the output tensors. |
+| `GGML_OPENVINO_DEBUG_NODE` | all | String | `not set` | Add the named graph nodes as outputs of the compiled model. Separate several names with commas. |
+| `GGML_OPENVINO_PRINT_CGRAPH_TENSOR_ADDRESS` | all | Boolean | `0` | Print the tensor address map once. |
+| `GGML_OPENVINO_LOG_UNSUPPORTED_OPS` | all | Boolean | `0` | Log each op that the backend does not support, with the tensor details and the reason. |
+| `GGML_OPENVINO_DISABLE_CACHE` | all | Boolean | `0` | Turn the in-process cache of compiled models and decoders off. |
+| `GGML_OPENVINO_FULL_GRAPH_KEY` | all | Boolean | `0` | Add the external inputs of a graph to the key of the in-process cache. By default the key is the node count and the names of the first and last nodes. Use it when two different graphs could share that key. |
+| `GGML_OPENVINO_LOG_SWA_LAYERS` | all | Boolean | `0` | Log the attention extent of each layer and which layers use sliding-window attention, on an `ov-swa:` line at the WARN level. |
+| `GGML_OPENVINO_NPUW_DUMP_SUBS` | NPU | String | `not set` | Dump NPUW subgraphs: `YES` (all), `MIN` (unique function bodies), `last`, or indices such as `0,2`. |
+| `GGML_OPENVINO_NPUW_DUMP_SUBS_DIR` | NPU | String | `not set` | Directory for the NPUW subgraph dumps. |
+| `GGML_OPENVINO_NPUW_DUMP_SUBS_ON_FAIL` | NPU | String | `not set` | Passed to OpenVINO unchanged as `NPUW_DUMP_SUBS_ON_FAIL`. |
+| `GGML_OPENVINO_NPUW_DUMP_IO` | NPU | String | `not set` | Passed to OpenVINO unchanged as `NPUW_DUMP_IO`. |
+
+```bash
+# Dump the GGML graph and the OpenVINO IR of a run
+GGML_OPENVINO_DUMP_CGRAPH=1 GGML_OPENVINO_DUMP_IR=1 \
+    ./build/ReleaseOV/bin/llama-cli -m model.gguf -c 512
+
+# Log unsupported ops. The lines have the WARN level, which the default log verbosity shows (-lv 2 or higher).
+GGML_OPENVINO_LOG_UNSUPPORTED_OPS=1 ./build/ReleaseOV/bin/llama-cli -m model.gguf -c 512
+
+# NPUW dumps. They need an OpenVINO developer build (NPU_PLUGIN_DEVELOPER_BUILD).
+export GGML_OPENVINO_DEVICE=NPU
+export GGML_OPENVINO_NPUW_DUMP_SUBS=YES
+export GGML_OPENVINO_NPUW_DUMP_SUBS_DIR=/tmp/npuw_dump   # optional
+./build/ReleaseOV/bin/llama-cli -m model.gguf -c 512
+```
+
+## Switches for default optimizations
+
+Each variable turns off an optimization that is on by default, or sets its strength. Use them to find the cause of a wrong result or of a speed change.
+
+| Variable | Device | Type | Default | Description |
+|---|---|---|---|---|
+| `GGML_OPENVINO_KV_SLICE` | all | Boolean | `1` | Bind the KV cache, and size the KQ masks, at the attended length instead of the full context. CPU and GPU: a view over the cache. NPU: the prefill graph is compiled at the attended length, rounded up to a power of two, and replaced when that size changes. With `0`, `llama-bench -d` fails on the NPU. |
+| `GGML_OPENVINO_NPU_BLOB_PREBUILD` | NPU | Boolean | `1` | Decode attends the smallest `1024 * 2^k` rows that cover the next position, with one decode graph for each window. All windows are built, or read from the cache, together with the first prefill and decode pair. `0`: one decode graph attends the whole context. Use the same value in the run that writes the cache and in the runs that read it. |
+| `GGML_OPENVINO_NPU_FAST_MASK` | NPU | Boolean | `1` | Write the prefill attention mask directly into the input tensor, without an intermediate copy. The result is identical. |
+| `GGML_OPENVINO_NPU_L0_HOST_TENSORS` | NPU | Boolean | `1` | Keep the KV cache and the compute buffers in Level Zero host memory, so the plugin reads them without an extra copy. Falls back to plain host memory if the allocation is not available. |
+| `GGML_OPENVINO_GDN_Q_SCALE` | NPU | Float | `64` | Factor of the GDN q pre-scaling, which keeps the GDN output above the FP16 flush-to-zero floor of the NPU. `0` or a negative value turns it off. Values below `64` are raised to `64`. |
+| `GGML_OPENVINO_MOE_OP` | GPU | Boolean | `1` | `0` keeps the unfused GatherMatmul path for mixture-of-experts layers. |
+| `GGML_OPENVINO_DISABLE_SHAPE_FROM_MASK` | CPU, GPU | Boolean | `0` | By default the model reads the sequence count and the attended KV length from the shape of the KQ mask, so the GPU plugin can select its fused attention kernel. `1` passes them as separate inputs. |
+| `GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT` | CPU, GPU | Boolean | `0` | Stateful execution only. By default the sequence axis of the KV state is moved so that new tokens are appended in place. `1` keeps the original layout. |
+| `GGML_OPENVINO_DISABLE_REMOTE_OUTPUTS` | GPU | Boolean | `0` | By default an output tensor in a GPU buffer is bound directly to the infer request when its shape and type match. `1` turns this off. |
+| `GGML_OPENVINO_DISABLE_FUSED_RECURRENCE` | GPU | Boolean | `0` | Stateless execution only, with an OpenVINO build newer than 2026.4.1 (2026.4.1 does not build the fusion). By default the recurrent-state cache updates (Qwen3.5, LFM2) are fused into the OpenVINO model. `1` keeps the separate copies. |
+| `GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN` | GPU | Boolean | `0` | By default the operands of an element-wise op get the same rank, which works around a GPU plugin defect when the op is fused into RMS norm (gemma-4 in stateful execution gave wrong output). `1` turns the workaround off. |
+
+```bash
+# Find out whether an optimization causes a wrong result: turn it off and compare the output
+GGML_OPENVINO_DEVICE=NPU GGML_OPENVINO_NPU_FAST_MASK=0 \
+    ./build/ReleaseOV/bin/llama-completion -m model.gguf -c 512 -n 32 --temp 0 -no-cnv -p "The capital of France is"
+```
+
+## Experimental options
+
+These options are off by default. They are not validated for general use: some help on one model or device and give a wrong or slower result on another.
+
+| Variable | Device | Type | Default | Description |
+|---|---|---|---|---|
+| `GGML_OPENVINO_NPU_DECODE_UNFOLD` | NPU | Boolean | `0` | Set `NPUW_UNFOLD_IREQS=YES` on the decode graph only. An explicit `GGML_OPENVINO_NPUW_UNFOLD_IREQS` or `GGML_OPENVINO_NPU_CONFIG` value takes precedence. |
+| `GGML_OPENVINO_NPU_PARALLEL_COMPILE` | NPU | Boolean | `0` | Compile the prefill and decode graphs in parallel. The load is shorter and the host memory peak during compilation is higher. |
+| `GGML_OPENVINO_NPU_KV_ON_DEMAND` | NPU | Boolean | `0` | Keep the ggml KV buffer non-resident and bind storage for each layer that grows with the rows in use, so a large `-c` costs memory only for the context that is reached. K is stored head-major and V transposed, as OpenVINO GenAI does. `llama-perplexity` fails with it. |
+| `GGML_OPENVINO_NPU_REQUANT_POLICY` | NPU | String | `group-128` | Layout of requantized weights: `group-128` (`Q4_0_128`) or `channel-wise` (`Q4_0_C`, lower accuracy). |
+| `GGML_OPENVINO_GDN_DECOMPOSE_CHUNK` | all | Integer | `16` | Tokens in one chunk when `GGML_OPENVINO_GDN_DECOMPOSE` is on. A power of 2 from 2 to 128; other values use `16`. |
+| `GGML_OPENVINO_GDN_NATIVE_STATE` | NPU | Boolean | `0` | Keep the `GATED_DELTA_NET` state in the layout of the OpenVINO op instead of the GGML layout. Every `GATED_DELTA_NET` must then run on the OpenVINO backend; the load aborts if one would run on the CPU backend. Saved states and `GGML_OPENVINO_DEBUG_NODE` dumps hold the OpenVINO layout. `test-backend-ops` fails with it. |
+| `GGML_OPENVINO_FORCE_STATIC` | CPU, GPU | Boolean | `0` | Run the static-shape path of the NPU on the CPU or the GPU, to separate translation bugs from NPU plugin or driver problems. Cannot be used with `GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY`. |
+| `GGML_OPENVINO_ENABLE_FALLBACK` | all | Boolean | `0` | Detect graphs that the scheduler splits between OpenVINO and another backend, and translate them as partial graphs, without the compiled model cache. By default every graph is treated as a complete model graph. Cannot be used with `GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY`. |
+| `GGML_OPENVINO_NATIVE_SOFTPLUS` | all | Boolean | `0` | Translate `SOFTPLUS` to the OpenVINO `SoftPlus` op instead of the default `relu(x) + log(1 + exp(-abs(x)))`. |
+| `GGML_OPENVINO_KV_SCATTER_ELEMENTS` | all | Boolean | `0` | Use `ScatterElementsUpdate` instead of `ScatterUpdate` for single-row KV writes in stateless execution. |
+| `GGML_OPENVINO_ENABLE_KV_4D` | GPU | Boolean | `0` | Bind the KV cache as `[1, ctx, n_heads_kv, head_size]`, so the GPU plugin can update and read it in place. Needs an OpenVINO build newer than 2026.4.1; do not use it with 2026.4.1. |
+| `GGML_OPENVINO_MEMORY_OPTIMIZE` | all | Boolean | `0` | Turn on `GGML_OPENVINO_REDUCE_COMPILE_MEM` and, on GPU, `GGML_OPENVINO_RELEASE_WEIGHTS`, unless they are set. |
+| `GGML_OPENVINO_REDUCE_COMPILE_MEM` | all | Boolean | value of `GGML_OPENVINO_MEMORY_OPTIMIZE` | Create fewer weight nodes during compilation, to lower the host memory in use. |
+| `GGML_OPENVINO_RELEASE_WEIGHTS` | GPU | Boolean | value of `GGML_OPENVINO_MEMORY_OPTIMIZE` | Release the host weight buffers when the compiled model holds its own copy. Needs stable graph shapes; do not use it for workloads that compile again. |
+| `GGML_OPENVINO_SPILL_DIR` | all | String | `not set` | Directory for a disk-backed weight buffer, so its pages can be reclaimed under memory pressure. It must be on real storage, not on tmpfs (`/tmp` on many systems). It is not part of the key of the compiled model cache and is ignored in cache-only mode. |
+| `GGML_OPENVINO_SELF_CONTAINED_BLOB` | NPU | Boolean | `0` | Requires `GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR`. Write the prefill and decode graphs as blobs with the weights inside; each blob takes about the size of the model on disk. Decode then attends the whole context. `GGML_OPENVINO_NPU_WEIGHTLESS_BANK` is the recommended cache layout for the NPU. |
+| `GGML_OPENVINO_NPU_SHARED_BANK` | NPU | Boolean | `0` | With `GGML_OPENVINO_SELF_CONTAINED_BLOB=1`: one weights bank for the prefill and decode blobs. With OpenVINO 2026.4.1 the blobs cannot be read back, so decode is compiled again. |
+| `GGML_OPENVINO_LAZY_HOST_THRESHOLD_MB` | NPU | Integer | `2048` | With `GGML_OPENVINO_SELF_CONTAINED_MMAP=1`: buffers of at least this size get demand-zero host memory instead of Level Zero host memory. |
+
+```bash
+# NPU cache with the weights inside the blobs (experimental). Keep all other NPU settings the same in both runs.
+export GGML_OPENVINO_DEVICE=NPU
+export GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=/tmp/ov_cache
+export GGML_OPENVINO_SELF_CONTAINED_BLOB=1
+./build/ReleaseOV/bin/llama-cli -m model.gguf -c 512 -lv 4   # first run: log shows "model cache WROTE"
+
+export GGML_OPENVINO_SELF_CONTAINED_MMAP=1
+./build/ReleaseOV/bin/llama-cli -m model.gguf -c 512 -lv 4   # later runs: log shows "model cache HIT"
+```
+
+## Tips for Memory Optimization
+
+With the default settings, the GPU and CPU plugins keep their own copy of the weights next to the backend's copy while a model compiles. On integrated GPUs both copies are in system RAM. The settings below lower that peak. They do not change the output.
+
+```bash
+# GPU or CPU, large model: back the weight buffer with a file, so the kernel can drop its pages.
+# Use a directory on a local disk (not tmpfs) with free space of about the model size. Can lower prompt speed.
+export GGML_OPENVINO_SPILL_DIR=$HOME/ov_spill
+mkdir -p $GGML_OPENVINO_SPILL_DIR
+```
+
+```bash
+# GPU or CPU, the same model started again and again: compile once, then load the compiled model only.
+export GGML_OPENVINO_DEVICE=GPU   # or CPU
+export GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=$HOME/ov_cache
+GGML_OPENVINO_SPILL_DIR=$HOME/ov_spill ./build/ReleaseOV/bin/llama-cli -m model.gguf   # first run writes the cache
+export GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY=1
+./build/ReleaseOV/bin/llama-cli -m model.gguf                                           # later runs skip the weight upload
+```
+
+```bash
+# GPU, one long-running model (llama-server): release the host copy of the weights after compilation.
+# One model per process; a graph that must compile again later stops the run.
+export GGML_OPENVINO_DEVICE=GPU
+export GGML_OPENVINO_MEMORY_OPTIMIZE=1
+```
+
+```bash
+# CPU, MoE model: build fewer weight nodes during compilation.
+export GGML_OPENVINO_REDUCE_COMPILE_MEM=1
+```
+
+```bash
+# NPU: weightless cache, then use the memory-mapped GGUF as the model buffer.
+export GGML_OPENVINO_DEVICE=NPU
+export GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR=$HOME/ov_cache
+export GGML_OPENVINO_NPU_WEIGHTLESS_BANK=1
+./build/ReleaseOV/bin/llama-cli -m model.gguf -c 512      # first run writes the cache
+export GGML_OPENVINO_SELF_CONTAINED_MMAP=1
+./build/ReleaseOV/bin/llama-cli -m model.gguf -c 512      # later runs
+```
+
+- A cache hit without `GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY=1` shortens the load, but it uses about as much memory as a run without the cache.
+- MoE models are not written to the cache on CPU, so cache-only mode does not work for them on CPU.
+- `GGML_OPENVINO_SPILL_DIR` is ignored on Windows and in cache-only mode.
+- `-lm none` (no mmap) does not lower the memory in use, and it turns cache-only mode off.
+
+## NPU plugin properties
+
+The backend sets the properties below for the NPU. `GGML_OPENVINO_NPU_CONFIG` can change them and add others.
+
+- **Backend default**: the value that the backend passes to OpenVINO when the user sets nothing.
+- **OpenVINO default**: the value that OpenVINO uses when nobody sets the property.
+- Values are those of OpenVINO, for example `YES` and `NO`.
+
+| Property | Type | Backend default | OpenVINO default | Description |
+|---|---|---|---|---|
+| `NPU_USE_NPUW` | Boolean | `YES` | `NO` | Use NPUW, the OpenVINO layer that splits a model into parts and compiles them for the NPU. |
+| `NPUW_DEVICES` | String | `NPU` | `NPU,CPU` | Devices that NPUW tries for each part, in order. With `NPU` only, no part runs on the CPU. |
+| `NPUW_FOLD` | Boolean | `YES` | `NO` | Compile the repeating blocks of the model once and reuse them. |
+| `NPUW_FUNCALL_FOR_ALL` | Boolean | `YES` | `NO` | Make every part of the model a function, not only the repeating blocks. |
+| `NPUW_FUNCALL_ASYNC` | Boolean | `YES` | `NO` | Run a function and the preparation of the next one as a pipeline. |
+| `NPUW_WEIGHTS_BANK` | String | `shared` | empty | Name of the weights bank. Graphs with the same name share one copy of the weights. |
+| `NPUW_DQ` | Boolean | `YES` | `NO` | Apply the dynamic quantization transformations in the plugin. |
+| `NPUW_DQ_FULL` | Boolean | `NO` | `YES` | Apply the full dynamic quantization pipeline in the plugin. |
+| `NPU_COMPILER_DYNAMIC_QUANTIZATION` | Boolean | `YES` | `NO` | Dynamic quantization in the NPU compiler. |
+| `NPU_COMPILER_TYPE` | String | `DRIVER` | `PREFER_PLUGIN` | The NPU compiler: `DRIVER` (in the NPU driver), `PLUGIN` (in OpenVINO) or `PREFER_PLUGIN` (`PLUGIN` when it is available, `DRIVER` if not). Models with `GATED_DELTA_NET` layers (Qwen3.5) compile with `PLUGIN` unless the user sets this property. |
+| `NPUW_CACHE_DIR` | String | value of `GGML_OPENVINO_CACHE_DIR` | empty | Directory for the compiled parts. |
+
+Other properties that are useful with `GGML_OPENVINO_NPU_CONFIG`. The backend does not set them.
+
+| Property | Type | Backend default | OpenVINO default | Description |
+|---|---|---|---|---|
+| `COMPILATION_NUM_THREADS` | Integer | `not set` | number of CPU cores | Threads of the NPU compiler. A lower value lowers the host memory peak when a large model compiles. |
+| `NPUW_UNFOLD_IREQS` | Boolean | `not set` | `NO` | One infer request for each part, also for repeating blocks. |
+| `LOG_LEVEL` | String | `not set` | `LOG_ERROR` | Log level of the NPU plugin, for example `LOG_INFO`. |
+
+```bash
+export GGML_OPENVINO_DEVICE=NPU
+export GGML_OPENVINO_NPU_CONFIG="NPU_COMPILER_TYPE=PLUGIN,COMPILATION_NUM_THREADS=1,LOG_LEVEL=LOG_INFO"
+```
+
+```cpp
+// List the properties of the installed NPU plugin with their current values
+ov::Core core;
+for (const auto & name : core.get_property("NPU", ov::supported_properties)) {
+    std::cout << name << " = " << core.get_property("NPU", name).as<std::string>() << "\n";
+}
+```
+
+- All NPU properties: [OpenVINO NPU device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html).
+- `NPUW_*` options, with type and default: `src/plugins/intel_npu/src/al/include/intel_npu/config/npuw_option_defs.inc` in the OpenVINO source.
+- Graphs without a KV cache compile without NPUW: the backend removes `NPU_USE_NPUW` and the `NPUW_*` keys for them.
+
+## GPU and CPU plugin properties
+
+The backend sets no GPU or CPU property, except `CACHE_DIR` and `CACHE_MODE=OPTIMIZE_SIZE` when `GGML_OPENVINO_CACHE_DIR` is set.
+
+```bash
+# CPU in FP16, to compare with the GPU and the NPU, which compute in FP16
+GGML_OPENVINO_DEVICE=CPU GGML_OPENVINO_CPU_CONFIG="INFERENCE_PRECISION_HINT=f16" \
+    ./build/ReleaseOV/bin/llama-cli -m model.gguf -c 512
+```
+
+All properties: [OpenVINO GPU device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html), [OpenVINO CPU device](https://docs.openvino.ai/2026/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device.html).
+
+## Older variable names
+
+Each of these variables sets one NPU property. They still work. `GGML_OPENVINO_NPU_CONFIG` sets the same properties and takes precedence when both are set.
+
+| Variable | Device | Type | Default | Description |
+|---|---|---|---|---|
+| `GGML_OPENVINO_NPU_COMPILE_CONFIG` | NPU | String | `not set` | Sets `NPU_COMPILATION_MODE_PARAMS`. |
+| `GGML_OPENVINO_NPU_COMPILER_TYPE` | NPU | String | `not set` | Sets `NPU_COMPILER_TYPE`. |
+| `GGML_OPENVINO_NPUW_FUNCALL_FOR_ALL` | NPU | String | `not set` | Sets `NPUW_FUNCALL_FOR_ALL`. |
+| `GGML_OPENVINO_NPUW_UNFOLD_IREQS` | NPU | String | `not set` | Sets `NPUW_UNFOLD_IREQS`. |
+| `GGML_OPENVINO_COMPILATION_NUM_THREADS` | NPU | String | `not set` | Sets `COMPILATION_NUM_THREADS`. |
+
+```bash
+# These two lines do the same. Use the first form.
+export GGML_OPENVINO_NPU_CONFIG="NPUW_UNFOLD_IREQS=YES"
+export GGML_OPENVINO_NPUW_UNFOLD_IREQS=YES
+```
diff --git a/ggml/src/ggml-openvino/dev_doc/README.md b/ggml/src/ggml-openvino/dev_doc/README.md
new file mode 100644
index 000000000..4b1021e74
--- /dev/null
+++ b/ggml/src/ggml-openvino/dev_doc/README.md
@@ -0,0 +1,8 @@
+# OpenVINO Qwen development documentation
+
+The default supported Qwen3.5/Qwen3.6 GPU path is **dynamic stateless execution with in-place fused recurrent ops and stable MTP sampling**. Unified KV is the recommended multi-slot layout because it enables native in-place attention KV updates. No new inference switch is needed for the recurrent/sampling default.
+
+- [Backend setup and cache usage](../README.md): runtime requirements, environment settings and restrictions.
+- [Implementation](OPENVINO-RECURRENT-CACHE.md): buffer ownership, fusion, rollback, fallback and source map.
+- [Multislot MTP graph](MTP-SAMPLING-GRAPH.md): why active-slot variants occurred and how stable translation works.
+- [Environment variables for development](OpenVINO-ENV-VARS.md): debug output, switches for default optimizations, experimental options, plugin properties and older variable names.
diff --git a/ggml/src/ggml-openvino/device/npu_config.cpp b/ggml/src/ggml-openvino/device/npu_config.cpp
new file mode 100644
index 000000000..637083066
--- /dev/null
+++ b/ggml/src/ggml-openvino/device/npu_config.cpp
@@ -0,0 +1,113 @@
+#include "npu_config.h"
+
+#include "ggml-impl.h"
+#include "ggml-openvino/ggml-decoder.h"
+#include "ggml-openvino/ggml-openvino-extra.h"
+
+#include <cstring>
+#include <openvino/runtime/properties.hpp>
+
+// The environment does not set the NPU compiler type. Set by ggml_openvino_npu_build_compile_config().
+static bool npu_compiler_type_is_default = false;
+
+ov::AnyMap ggml_openvino_npu_build_compile_config(const char * cache_dir) {
+    ov::AnyMap compile_config = {
+        {"NPU_COMPILER_DYNAMIC_QUANTIZATION", "YES"   },
+        {"NPU_USE_NPUW",                      "YES"   },
+        {"NPUW_DEVICES",                      "NPU"   },
+        {"NPUW_FOLD",                         "YES"   },
+        {"NPUW_WEIGHTS_BANK",                 "shared"},
+        {"NPUW_FUNCALL_FOR_ALL",              "YES"   },
+        {"NPUW_FUNCALL_ASYNC",                "YES"   },
+        {"NPUW_DQ",                           "YES"   },
+        {"NPUW_DQ_FULL",                      "NO"    },
+    };
+    if (cache_dir && strlen(cache_dir) > 0) {
+        // Keep the default cache mode: CACHE_MODE=OPTIMIZE_SIZE makes the NPU plugin compile the NPUW submodels with
+        // weights separation, which fails for them.
+        compile_config["NPUW_CACHE_DIR"] = cache_dir;
+    }
+    // Copy an optional string env var into the compile config only when it is set and non-empty.
+    auto set_compile_option_from_env = [&](const char * env_var, const char * config_key) {
+        const char * value = ggml_openvino_getenv_str(env_var);
+        if (value && strlen(value) > 0) {
+            compile_config[config_key] = value;
+        }
+    };
+    // Legacy alias for GGML_OPENVINO_NPU_CONFIG=NPU_COMPILATION_MODE_PARAMS=...
+    set_compile_option_from_env("GGML_OPENVINO_NPU_COMPILE_CONFIG", "NPU_COMPILATION_MODE_PARAMS");
+    // PLUGIN | DRIVER | PREFER_PLUGIN. The in-plugin compiler and the driver compiler
+    // can differ substantially in generated code quality for the same op. On the Intel
+    // NPU the driver compiler emits markedly faster prefill kernels (~1940 vs ~1540 t/s
+    // pp1024 on phi-4-mini), matching OpenVINO GenAI, so default to DRIVER below.
+    set_compile_option_from_env("GGML_OPENVINO_NPU_COMPILER_TYPE", "NPU_COMPILER_TYPE");
+    // NPUW_FUNCALL_FOR_ALL=YES hangs the NPU (device lost via TDR) for context
+    // lengths >= ~786 on the 2026.3 in-plugin compiler. Allow turning it off.
+    set_compile_option_from_env("GGML_OPENVINO_NPUW_FUNCALL_FOR_ALL", "NPUW_FUNCALL_FOR_ALL");
+    // Unfolds function calls into separate infer requests, trading memory for the
+    // per-call dispatch overhead that repeated funcalls otherwise pay.
+    set_compile_option_from_env("GGML_OPENVINO_NPUW_UNFOLD_IREQS", "NPUW_UNFOLD_IREQS");
+    // The compiler runs one llvm worker per core by default; each carries its own
+    // working set, so large graphs can exhaust host RAM. Capping the workers trades
+    // compile time for peak memory.
+    set_compile_option_from_env("GGML_OPENVINO_COMPILATION_NUM_THREADS", "COMPILATION_NUM_THREADS");
+    // NPUW profiling dumps; need an OpenVINO build with NPU_PLUGIN_DEVELOPER_BUILD.
+    set_compile_option_from_env("GGML_OPENVINO_NPUW_DUMP_SUBS", "NPUW_DUMP_SUBS");
+    set_compile_option_from_env("GGML_OPENVINO_NPUW_DUMP_SUBS_DIR", "NPUW_DUMP_SUBS_DIR");
+    set_compile_option_from_env("GGML_OPENVINO_NPUW_DUMP_SUBS_ON_FAIL", "NPUW_DUMP_SUBS_ON_FAIL");
+    set_compile_option_from_env("GGML_OPENVINO_NPUW_DUMP_IO", "NPUW_DUMP_IO");
+    // Comma-separated KEY=VALUE pairs appended last, so they override anything above.
+    // Escape hatch for bisecting plugin options without a rebuild.
+    ggml_openvino_apply_config_overrides(compile_config, ggml_openvino_getenv_str("GGML_OPENVINO_NPU_CONFIG"), "NPU");
+    npu_compiler_type_is_default = compile_config.find("NPU_COMPILER_TYPE") == compile_config.end();
+    if (npu_compiler_type_is_default) {
+        compile_config["NPU_COMPILER_TYPE"] = "DRIVER";
+    }
+    return compile_config;
+}
+
+bool ggml_openvino_npu_gdn_uses_plugin_compiler() {
+    return ggml_openvino_is_npu() && npu_compiler_type_is_default;
+}
+
+ov::AnyMap ggml_openvino_npu_gdn_config(const ov::AnyMap & base) {
+    ov::AnyMap gdn_config = base;
+    if (ggml_openvino_npu_gdn_uses_plugin_compiler()) {
+        gdn_config["NPU_COMPILER_TYPE"] = "PLUGIN";
+    }
+    return gdn_config;
+}
+
+ov::AnyMap ggml_openvino_npu_without_npuw(const ov::AnyMap & config) {
+    ov::AnyMap out;
+    for (const auto & kv : config) {
+        if (kv.first.rfind("NPUW", 0) == 0 || kv.first == "NPU_USE_NPUW") {
+            continue;
+        }
+        out.insert(kv);
+    }
+    return out;
+}
+
+ov::AnyMap ggml_openvino_npu_prefill_config(const ov::AnyMap & base, const GgmlOvDecoder & decoder) {
+    ov::AnyMap prefill_config = base;
+    if (ggml_openvino_npu_hfa_prefill_enabled()) {
+        prefill_config["NPUW_ATTN_HISTORY_SIZE_INPUT"] = decoder.get_hfa_history_input_name();
+        prefill_config["NPUW_ONLINE_PIPELINE"] = "REP";
+        prefill_config["NPUW_ONLINE_ISOLATE"] = "ATTN";
+        prefill_config["NPUW_ONLINE_KEEP_BLOCK_SIZE"] = "4";
+        prefill_config["NPUW_UNFOLD_IREQS"] = "NO";
+        prefill_config["NPUW_ATTN"] = "HFA";
+        prefill_config["NPUW_ATTN_HFA_FUSED"] = "YES";
+    }
+    return prefill_config;
+}
+
+ov::AnyMap ggml_openvino_npu_decode_config(const ov::AnyMap & base, const std::string & device) {
+    ov::AnyMap decode_config = base;
+    if (device == "NPU" && ggml_openvino_npu_decode_unfold_enabled() &&
+        decode_config.find("NPUW_UNFOLD_IREQS") == decode_config.end()) {
+        decode_config["NPUW_UNFOLD_IREQS"] = "YES";
+    }
+    return decode_config;
+}
diff --git a/ggml/src/ggml-openvino/device/npu_config.h b/ggml/src/ggml-openvino/device/npu_config.h
new file mode 100644
index 000000000..89f41da54
--- /dev/null
+++ b/ggml/src/ggml-openvino/device/npu_config.h
@@ -0,0 +1,33 @@
+#pragma once
+
+// NPU (NPUW) compile configuration. Everything that decides which OpenVINO properties an NPU graph is
+// compiled with lives here, so "what does the NPU compile with" is answered in one place:
+//   * the backend-wide base config (NPUW defaults, named env aliases, GGML_OPENVINO_NPU_CONFIG overrides)
+//   * the per-graph variants of it (cacheless graphs, GATED_DELTA_NET graphs, static prefill, static decode)
+
+#include <openvino/core/any.hpp>
+#include <string>
+
+class GgmlOvDecoder;
+
+// Base config for every NPU graph, built once at backend init. GGML_OPENVINO_NPU_CONFIG ("K=V,...") is
+// applied last, so it overrides the defaults and the named aliases.
+ov::AnyMap ggml_openvino_npu_build_compile_config(const char * cache_dir);
+
+// Cacheless (encoder) graphs compile on the plain NPU plugin: drop every NPUW key.
+ov::AnyMap ggml_openvino_npu_without_npuw(const ov::AnyMap & config);
+
+// Graphs with GATED_DELTA_NET (Qwen3.5). The driver compiler gives wrong output for them and the compiler bundled
+// with OpenVINO does not, so they compile with NPU_COMPILER_TYPE=PLUGIN unless the environment sets the compiler
+// type (GGML_OPENVINO_NPU_COMPILER_TYPE, or NPU_COMPILER_TYPE in GGML_OPENVINO_NPU_CONFIG).
+ov::AnyMap ggml_openvino_npu_gdn_config(const ov::AnyMap & base);
+
+// True when ggml_openvino_npu_gdn_config() switches the compiler. Part of the model cache key.
+bool ggml_openvino_npu_gdn_uses_plugin_compiler();
+
+// Static prefill graph. With GGML_OPENVINO_NPU_HFA_PREFILL, switches NPUW to the HFA attention pipeline.
+ov::AnyMap ggml_openvino_npu_prefill_config(const ov::AnyMap & base, const GgmlOvDecoder & decoder);
+
+// Static decode graph. With GGML_OPENVINO_NPU_DECODE_UNFOLD, unfolds NPUW function calls into separate infer
+// requests, unless NPUW_UNFOLD_IREQS is already set.
+ov::AnyMap ggml_openvino_npu_decode_config(const ov::AnyMap & base, const std::string & device);
diff --git a/ggml/src/ggml-openvino/device/npu_quant_policy.cpp b/ggml/src/ggml-openvino/device/npu_quant_policy.cpp
new file mode 100644
index 000000000..1f7343bba
--- /dev/null
+++ b/ggml/src/ggml-openvino/device/npu_quant_policy.cpp
@@ -0,0 +1,50 @@
+#include "npu_quant_policy.h"
+
+#include "ggml-impl.h"
+#include "ggml.h"
+
+#include <mutex>
+#include <string>
+
+ExtraQuantType ggml_openvino_get_npu_requant_type() {
+    const std::string policy = ggml_openvino_getenv_str("GGML_OPENVINO_NPU_REQUANT_POLICY", "group-128");
+    if (policy == "group-128") {
+        return ExtraQuantType::Q4_0_128;
+    }
+    if (policy == "channel-wise") {
+        return ExtraQuantType::Q4_0_C;
+    }
+    GGML_ABORT("Unknown GGML_OPENVINO_NPU_REQUANT_POLICY: %s", policy.c_str());
+}
+
+std::optional<ExtraQuantType> ggml_openvino_npu_token_embd_requant_type(const ggml_tensor * tensor) {
+    // A Q6_K embedding table is stored as q8_0_c by default on the NPU. The int4 layouts only pay off for a tied
+    // embedding, where the table is also the lm_head weight and GET_ROWS gathers the packed rows
+    // before dequantizing them (see gather_compressed_rows).
+    if (tensor->type != GGML_TYPE_Q6_K) {
+        return std::nullopt;
+    }
+    const std::string quant = ggml_openvino_getenv_str("GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT", "q8_0_c");
+    ExtraQuantType type;
+    if (quant == "f16") {
+        type = ExtraQuantType::F16;
+    } else if (quant == "q8_0_c") {
+        type = ExtraQuantType::Q8_0_C;
+    } else if (quant == "q4_0_128") {
+        type = ExtraQuantType::Q4_0_128;
+    } else if (quant == "q4_0_c") {
+        type = ExtraQuantType::Q4_0_C;
+    } else {
+        GGML_ABORT("Unknown GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT: %s", quant.c_str());
+    }
+    static std::once_flag logged;
+    std::call_once(logged, [&] {
+        GGML_LOG_INFO("ggml-openvino: NPU token_embd.weight requant type: %s\n", quant.c_str());
+    });
+    return type;
+}
+
+std::optional<ExtraQuantType> ggml_openvino_npu_weight_requant_type(const ggml_tensor * tensor) {
+    GGML_UNUSED(tensor);
+    return ggml_openvino_get_npu_requant_type();
+}
diff --git a/ggml/src/ggml-openvino/device/npu_quant_policy.h b/ggml/src/ggml-openvino/device/npu_quant_policy.h
new file mode 100644
index 000000000..01fc39986
--- /dev/null
+++ b/ggml/src/ggml-openvino/device/npu_quant_policy.h
@@ -0,0 +1,19 @@
+#pragma once
+
+// NPU weight requantization rules. ggml_openvino_get_requant_type() (ggml-openvino-extra.cpp) delegates
+// the NPU-specific decisions here.
+
+#include "ggml-openvino/ggml-openvino-extra.h"
+
+#include <optional>
+
+struct ggml_tensor;
+
+// Layout selected by GGML_OPENVINO_NPU_REQUANT_POLICY (group-128 | channel-wise).
+ExtraQuantType ggml_openvino_get_npu_requant_type();
+
+// token_embd.weight on NPU. nullopt: no NPU-specific rule, use the common default.
+std::optional<ExtraQuantType> ggml_openvino_npu_token_embd_requant_type(const ggml_tensor * tensor);
+
+// Any other weight on NPU.
+std::optional<ExtraQuantType> ggml_openvino_npu_weight_requant_type(const ggml_tensor * tensor);
diff --git a/ggml/src/ggml-openvino/ggml-decoder.cpp b/ggml/src/ggml-openvino/ggml-decoder.cpp
index 10e82f3e6..38db895c5 100644
--- a/ggml/src/ggml-openvino/ggml-decoder.cpp
+++ b/ggml/src/ggml-openvino/ggml-decoder.cpp
@@ -82,9 +82,15 @@ void GgmlOvDecoder::update_io(ggml_cgraph * cgraph) {
     m_model_inputs.clear();
     m_model_outputs.clear();
     m_node_info_list.clear();
+    m_node_dynamic_dims.clear();
     set_input_output();
+    compute_node_dynamic_dims();
     compute_model_inputs();
     compute_model_outputs();
+    for (int i = 0; i < cgraph->n_nodes; ++i) {
+        m_node_info_list[i].node_op_case = compute_op_case(m_node_info_list[i].node);
+        m_node_info_list[i].node_op_type = compute_op_type(m_node_info_list[i].node);
+    }
 }

 // llama keeps separate graphs for batches with and without outputs, so a cache hit can come from a
@@ -99,6 +105,14 @@ GgmlOvDecoder::GgmlOvDecoder(ggml_cgraph * cgraph, std::map<std::string, std::sh
     m_cgraph = cgraph;
     m_model_weights = model_weights;
     m_naive = true;
+    for (int i = 0; i < cgraph->n_nodes; ++i) {
+        if ((cgraph->nodes[i]->op == GGML_OP_GATED_DELTA_NET && cgraph->nodes[i]->op_params[0] > 1) ||
+            is_sampling_view(cgraph->nodes[i]) || is_conv_state_snapshot_writeback(cgraph->nodes[i])) {
+            std::tie(m_model_params, m_compute_params) = compute_llm_params(cgraph, false);
+            add_extra_inputs();
+            break;
+        }
+    }
     set_input_output();
     compute_model_inputs();
     compute_model_outputs();
@@ -110,7 +124,8 @@ GgmlOvDecoder::GgmlOvDecoder(ggml_cgraph * cgraph, std::map<std::string, std::sh

 namespace {
 bool is_inplace_op(const ggml_tensor * node) {
-    return node->op == GGML_OP_SET_ROWS || node->op == GGML_OP_CPY || (node->op == GGML_OP_SCALE && node->view_src);
+    const bool is_scale_or_fill = node->op == GGML_OP_SCALE || node->op == GGML_OP_FILL;
+    return node->op == GGML_OP_SET_ROWS || node->op == GGML_OP_CPY || (is_scale_or_fill && node->view_src);
 }

 bool is_same_shape(const ggml_tensor * a, const ggml_tensor * b) {
@@ -126,13 +141,63 @@ bool is_conv_states_all_tensor(const ggml_tensor * tensor) {
     return tensor != nullptr && strncmp(tensor->name, "conv_states_all", strlen("conv_states_all")) == 0;
 }

-bool is_full_single_slot_writeback(const ggml_tensor * node) {
-    return node->view_src != nullptr && node->view_src->ne[1] == 1 && node->src[1] != nullptr &&
-           node->src[1]->op == GGML_OP_VIEW && node->src[1]->view_src == node->view_src &&
-           node->src[1]->view_offs == 0 && ggml_nbytes(node->src[1]) == ggml_nbytes(node->view_src);
+bool is_descending_snapshot_order(const ggml_tensor * order, int64_t capacity, int64_t windows) {
+    if (!order || order->op != GGML_OP_VIEW || capacity < 1 || windows < 1 || windows > capacity) {
+        return false;
+    }
+    const auto cast = order->src[0];
+    if (!cast || cast->op != GGML_OP_CPY || cast->type != GGML_TYPE_I32 || !cast->src[0] ||
+        cast->src[0]->op != GGML_OP_SCALE || !cast->src[0]->src[0] ||
+        cast->src[0]->src[0]->op != GGML_OP_ARANGE) {
+        return false;
+    }
+    float scale[2], range[3];
+    memcpy(scale, cast->src[0]->op_params, sizeof(scale));
+    memcpy(range, cast->src[0]->src[0]->op_params, sizeof(range));
+    return cast->ne[0] == capacity && scale[0] == -1 && scale[1] == capacity - 1 &&
+           range[0] == 0 && range[1] == capacity && range[2] == 1 &&
+           order->ne[0] == windows && ggml_is_contiguous(order) &&
+           order->view_offs == size_t(capacity - windows) * sizeof(int32_t);
 }
 }  // namespace

+bool GgmlOvDecoder::is_conv_state_snapshot_writeback(const ggml_tensor * node) {
+    if (!node || node->op != GGML_OP_SET_ROWS || !is_recurrent_cache(node->view_src) ||
+        !node->src[0] || node->src[0]->op != GGML_OP_VIEW || !node->src[0]->src[0] ||
+        node->src[0]->src[0]->op != GGML_OP_CONCAT || !node->src[2] || node->src[2]->op != GGML_OP_VIEW ||
+        node->src[2]->view_src != node->view_src) {
+        return false;
+    }
+    const auto source = node->src[0];
+    const auto concat = source->src[0];
+    const auto dest = node->src[2];
+    const auto cache = node->view_src;
+    const auto count = dest->ne[1];
+    if (!is_descending_snapshot_order(node->src[1], count, source->ne[1])) {
+        return false;
+    }
+    const size_t elem = ggml_type_size(cache->type);
+    const size_t row = cache->nb[1];
+    const int64_t tokens = concat->ne[0] - source->ne[0];
+    return (cache->type == GGML_TYPE_F32 || cache->type == GGML_TYPE_F16) &&
+           source->type == cache->type && dest->type == cache->type && ggml_is_contiguous(cache) &&
+           cache->ne[2] == 1 && cache->ne[3] == 1 && row > 0 && concat->op_params[0] == 0 &&
+           concat->ne[3] == 1 && concat->src[0] && source->ne[0] > 0 && source->ne[2] > 0 && source->ne[3] > 0 &&
+           size_t(source->ne[0]) <= row / elem &&
+           source->ne[0] == concat->src[0]->ne[0] &&
+           source->ne[2] == concat->ne[1] && source->ne[3] == concat->ne[2] &&
+           source->nb[0] == elem && source->nb[1] == elem && source->nb[2] == concat->nb[1] &&
+           source->nb[3] == concat->nb[2] && source->ne[1] == std::min(tokens, count) &&
+           source->view_offs == size_t(tokens - source->ne[1] + 1) * elem &&
+           dest->ne[0] == source->ne[0] && dest->ne[2] == source->ne[2] && dest->ne[3] == source->ne[3] &&
+           dest->nb[0] == elem && dest->nb[1] > 0 && dest->nb[1] % row == 0 &&
+           dest->nb[2] == size_t(source->ne[0]) * elem && dest->nb[3] == row &&
+           size_t(source->ne[2]) <= (row - dest->view_offs % row) / dest->nb[2] &&
+           size_t(count) <= size_t(cache->ne[1]) / (dest->nb[1] / row) &&
+           size_t(dest->ne[3]) <= dest->nb[1] / row &&
+           dest->view_offs / row <= dest->nb[1] / row - size_t(dest->ne[3]);
+}
+
 // MoE expert aggregation (build_moe_ffn in llama-graph.cpp): each expert plane is
 // `ggml_view_2d(experts, n_embd, n_tokens, experts->nb[2], i*experts->nb[1])` and the planes
 // are summed with a chain of ADDs: moe_out = ((view_0 + view_1) + view_2) + ... + view_{n-1}.
@@ -184,10 +249,10 @@ std::string GgmlOvDecoder::get_tensor_name(const ggml_cgraph * cgraph, const ggm
     if (tensor == nullptr) {
         return "";
     }
-    if ((tensor->flags & GGML_TENSOR_FLAG_COMPUTE) || is_kvcache(tensor, nullptr)) {
+    if (tensor->op != GGML_OP_NONE || is_kvcache(tensor, nullptr)) {
         // Hash-table slots depend on tensor addresses and differ between contexts.
         // Graph ordinals disambiguate duplicate names while keeping compiled-model
-        // ports identical for equivalent graphs in different contexts.
+        // ports identical for equivalent graphs in different contexts, including inactive sampling branches.
         const auto * node = std::find(cgraph->nodes, cgraph->nodes + cgraph->n_nodes, tensor);
         if (node != cgraph->nodes + cgraph->n_nodes) {
             return std::string(tensor->name) + "#n" + std::to_string(node - cgraph->nodes);
@@ -196,6 +261,17 @@ std::string GgmlOvDecoder::get_tensor_name(const ggml_cgraph * cgraph, const ggm
         if (leaf != cgraph->leafs + cgraph->n_leafs) {
             return std::string(tensor->name) + "#l" + std::to_string(leaf - cgraph->leafs);
         }
+        if (cgraph->leafs == nullptr || cgraph->n_leafs == 0) {
+            // ggml_graph_view sets n_leafs to 0, loop over all node srcs to give each leaf a unique name
+            for (int i = 0; i < cgraph->n_nodes; i++) {
+                auto *cur_node = cgraph->nodes[i];
+                for (int j = 0; j < GGML_MAX_SRC; j++) {
+                    if (cur_node->src[j] == tensor) {
+                        return std::string(tensor->name) + "#l" + std::to_string(i * GGML_MAX_SRC + j);
+                    }
+                }
+            }
+        }
     }
     return tensor->name;
 }
@@ -214,7 +290,9 @@ static std::string get_tensor_graph_input_ov_name(const GgmlOvDecoder * decoder,
     if (GgmlOvDecoder::is_inp_emb(tensor, op)) {
         return "embd";
     }
-    if (GgmlOvDecoder::is_inp_mask(tensor, op)) {
+    // gemma-4's KV-shared layers read the mask rows of their output tokens through GET_ROWS, which is not a mask
+    // consumer, so match the masks by identity too
+    if (GgmlOvDecoder::is_inp_mask(tensor, op) || decoder->is_kq_mask(tensor)) {
         // Give the two attention masks distinct OV parameter names. build_attn_inp_kq_mask()
         // names the full-attention mask and the sliding-window mask identically, so keying a
         // parameter off the name alone makes the second mask overwrite the first and both
@@ -229,6 +307,15 @@ static std::string get_tensor_graph_input_ov_name(const GgmlOvDecoder * decoder,
             return get_tensor_ov_name(cgraph, tensor) + "_swa";
         }
     }
+    if (!decoder->get_model_params().swa_layers.empty() && GgmlOvDecoder::is_kv_idx(tensor, op) &&
+        op->src[2] != nullptr) {
+        // same as the masks: the full and the sliding-window cache get KV write indices of the same name
+        const ggml_tensor * cache = op->src[2]->view_src != nullptr ? op->src[2]->view_src : op->src[2];
+        auto layer = extract_layer_from_name(cache->name);
+        if (layer.has_value() && decoder->is_swa_layer(layer.value())) {
+            return get_tensor_ov_name(cgraph, tensor) + "_swa";
+        }
+    }
     return get_tensor_ov_name(cgraph, tensor);
 }

@@ -253,6 +340,10 @@ void GgmlOvDecoder::set_input_output() {
             if (src->flags & GGML_TENSOR_FLAG_INPUT) {
                 src_name = get_tensor_graph_input_ov_name(this, m_cgraph, src, node);
             }
+            // Same name as in create_weight_nodes(), so ROPE reads the weight Constant.
+            if (is_rope_freqs_weight(src->view_src ? src->view_src : src, node)) {
+                src_name = "rope_freqs.weight";
+            }
             current_node_info.node_inputs[src_name] = src;
             current_node_info.node_inputs_names.push_back(src_name);

@@ -284,6 +375,33 @@ void GgmlOvDecoder::set_input_output() {
     }
 }

+// True if the GDN attention output is only read by rms_norm (gated norm, e.g. qwen3.5) and the state by
+// cache writeback CPYs. Then q may be pre-scaled, since rms_norm cancels the scale. Static (NPU) path only.
+bool GgmlOvDecoder::is_gdn_out_prescaled(const ggml_tensor * gdn) const {
+    if (!ggml_openvino_is_npu()) {
+        return false;
+    }
+    bool has_norm = false;
+    for (int i = 0; i < m_cgraph->n_nodes; i++) {
+        const ggml_tensor * n = m_cgraph->nodes[i];
+        for (int j = 0; j < GGML_MAX_SRC; j++) {
+            const ggml_tensor * s = n->src[j];
+            if (s == gdn && n->op != GGML_OP_VIEW) {
+                return false;
+            }
+            if (s == nullptr || s->op != GGML_OP_VIEW || s->src[0] != gdn) {
+                continue;
+            }
+            if (n->op == GGML_OP_RMS_NORM) {
+                has_norm = true;
+            } else if (n->op != GGML_OP_CPY) {
+                return false;
+            }
+        }
+    }
+    return has_norm;
+}
+
 int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
     int op_case = 0;
     switch (node->op) {
@@ -293,21 +411,66 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
         }
         auto name = std::string(node->name);
         auto * src = node->src[0];
+        if (src->op == GGML_OP_MUL && node->ne[0] == src->ne[0] * src->ne[1] && node->ne[3] == 1) {
+            for (int j = 0; j < 2; ++j) {
+                const auto * norm = src->src[j];
+                if (norm->op == GGML_OP_MUL) {
+                    norm = norm->src[0];
+                }
+                if (norm->op == GGML_OP_RMS_NORM && norm->src[0]->op == GGML_OP_VIEW &&
+                    norm->src[0]->src[0]->op == GGML_OP_GATED_DELTA_NET) {
+                    // The post-GDN projection rejoins the flat token stream.
+                    return node->ne[2] == 1 ? 5 : 6;
+                }
+            }
+        }
         // Identify recurrent sequence reshapes before size checks, which are ambiguous for one token.
         bool recurrent_sequence = false;
         for (int i = 0; i < m_cgraph->n_nodes && !recurrent_sequence; ++i) {
             const auto * consumer = m_cgraph->nodes[i];
-            if (consumer->op == GGML_OP_MUL_MAT_ID && consumer->src[1] == node) {
+            if (consumer->op == GGML_OP_ROPE && consumer->src[1] == node) {
+                return 11;
+            } else if (consumer->op == GGML_OP_MUL_MAT_ID && consumer->src[1] == node) {
                 return 1;
+            } else if (consumer->op == GGML_OP_GATED_DELTA_NET) {
+                for (int j : {3, 4}) {
+                    const auto * gate = consumer->src[j];
+                    if (gate->op == GGML_OP_UNARY) {
+                        gate = gate->src[0];
+                    }
+                    if (gate == node) {
+                        return 10;
+                    }
+                }
             } else if (consumer->op == GGML_OP_SSM_CONV) {
                 const auto * concat = consumer->src[0];
-                if (concat->op == GGML_OP_CONCAT) {
-                    const auto * transposed = concat->src[1];
-                    recurrent_sequence = transposed->op == GGML_OP_TRANSPOSE && transposed->src[0] == node;
+                if (concat->op == GGML_OP_CONCAT && concat->src[1]->op == GGML_OP_TRANSPOSE) {
+                    const auto * seq = concat->src[1]->src[0];
+                    // LFM2 short convolutions gate the input first: transpose(mul(view(node), view(node))).
+                    if (seq->op == GGML_OP_MUL && seq->src[0]->op == GGML_OP_VIEW && seq->src[0]->src[0] == node &&
+                        seq->src[1]->op == GGML_OP_VIEW && seq->src[1]->src[0] == node) {
+                        seq = node;
+                    }
+                    recurrent_sequence = seq == node;
                 }
             } else if (consumer->op == GGML_OP_UNARY && ggml_get_unary_op(consumer) == GGML_UNARY_OP_SOFTPLUS) {
                 const auto * biased = consumer->src[0];
                 recurrent_sequence = biased->op == GGML_OP_ADD && biased->src[0] == node;
+            } else if (consumer->op == GGML_OP_MUL) {
+                for (int j = 0; j < 2; ++j) {
+                    const auto * gate = consumer->src[j];
+                    const auto * norm = consumer->src[1 - j];
+                    if (gate->op != GGML_OP_UNARY || gate->src[0] != node) {
+                        continue;
+                    }
+                    if (norm->op == GGML_OP_MUL) {
+                        norm = norm->src[0];
+                    }
+                    if (norm->op == GGML_OP_RMS_NORM && norm->src[0]->op == GGML_OP_VIEW &&
+                        norm->src[0]->src[0]->op == GGML_OP_GATED_DELTA_NET) {
+                        return 10;
+                    }
+                }
             }
         }
         if (recurrent_sequence && node->ne[0] == src->ne[0] && node->ne[3] == 1) {
@@ -318,6 +481,10 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
         }
         if (src->op == GGML_OP_RESHAPE && src->src[0]->ne[0] == node->ne[0] && src->src[0]->ne[1] == node->ne[1]) {
             op_case = 4;
+        } else if (node->ne[0] == 1 && node->ne[1] == src->ne[0] && ggml_nrows(src) == 1) {
+            // [n] -> [1, n] (gemma-4 gathers the positions of its output rows this way): the source dim
+            // becomes dim 1. The next check matches too, but would make dim 2 dynamic.
+            op_case = 3;
         } else if (node->ne[0] * node->ne[1] == src->ne[0]) {
             op_case = 1;
         } else if (src->ne[0] * src->ne[1] == node->ne[0]) {
@@ -334,6 +501,23 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
         } else if (name.find("state_predelta") == 0) {
             op_case = 8;
         }
+        const auto it = m_node_dynamic_dims.find(const_cast<ggml_tensor *>(node));
+        const int dynamic_dim = it != m_node_dynamic_dims.end() ? it->second : -1;
+        // op_case 5 flattens to [x, n_tokens]; a deeper dynamic dim means the size match above held only because
+        // there is one token, e.g. the last-layer expert groups with a single output row. op_case 3 ([n] -> [1, n])
+        // matches the same way, e.g. the last-layer expert probabilities [n_expert, 1] -> [1, n_expert, 1]
+        if (op_case == 1 || op_case == 2 || op_case == 3 || (op_case == 5 && dynamic_dim > 1)) {
+            // The rules above read the token dim off a 2D source. With a deeper source, e.g. the MoE expert groups
+            // [n_exp_per_group, n_groups, n_tokens], the tracked dynamic dim is the reliable one:
+            // op_case 2 marks ggml dim 1, op_case 1 dim 2, op_case 8 dim 3
+            if (dynamic_dim == 1) {
+                op_case = 2;
+            } else if (dynamic_dim == 2) {
+                op_case = 1;
+            } else if (dynamic_dim == 3) {
+                op_case = 8;
+            }
+        }
         if (op_case == 1 && m_is_stateful) {
             // Recurrent convolution and GDN gates retain their rank-4 layout.
             // The gathered states may come through a view of the one gather of all states (see build_rs).
@@ -377,7 +561,10 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
             // kv cache tensor
             std::string src_name(node->view_src->name);
             int layer = extract_layer_from_name(src_name).value();
-            if (ggml_is_contiguous(node->src[0])) {
+            // a transposed V cache view is contiguous too when the attended length is the whole cache (one ubatch
+            // that fills it)
+            const bool v_trans_cache = m_model_params.v_trans && src_name.rfind("cache_v_l", 0) == 0;
+            if (ggml_is_contiguous(node->src[0]) && !v_trans_cache) {
                 // -  19: [    64,     8,   256,     1] VIEW            cache_k_l0 (view)             [ 2,   128,  1024, 1048576]
                 //         [   512,  1024,     1,     1]      0: NONE     cache_k_l0                    [ 2,  1024, 1048576, 1048576]
                 // -  20: [    64,   256,     8,     1] PERMUTE         cache_k_l0 (view) (permuted)  [ 2,  1024,   128, 1048576]
@@ -405,6 +592,20 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
         }
         break;
     }
+    case GGML_OP_SET_ROWS: {
+        if (is_conv_state_snapshot_writeback(node)) {
+            op_case = 3;
+            break;
+        }
+        // `-fa off` writes the V cache transposed, through a [1, n] reshape of the cache: one element per row
+        if (node->src[2] != nullptr && node->src[2]->op == GGML_OP_RESHAPE && node->src[2]->ne[0] == 1 &&
+            is_kvcache(node->src[2]->src[0], nullptr)) {
+            op_case = 1;
+        } else if (!is_kvcache_write(node)) {
+            op_case = 2;
+        }
+        break;
+    }
     case GGML_OP_MUL_MAT: {
         if (node->src[1]->op == GGML_OP_SOFT_MAX) {
             // In the case of `-fa off`, softmax is used, v_trans=true, the dynamic dim is ne[0] for cache_v
@@ -416,16 +617,17 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
         // GET_ROWS gathering recurrent state cache rows via the inp->s_copy index list:
         // src[0] is a reshape of cache_r/cache_s, src[1] is a view of the s_copy leaf, or the leaf itself
         // when one gather covers all states (see build_rs).
-        // op_case 1/2: active/extra rows of a multi-slot cache
-        // op_case 3/4: active/extra rows of a single-slot cache
+        // op_case 1/2: active/extra rows, independent of cache capacity.
         if (node->src[0]->op == GGML_OP_RESHAPE && node->src[0]->src[0] != nullptr &&
             is_recurrent_cache(node->src[0]->src[0])) {
-            const bool single_slot = node->src[0]->src[0]->ne[1] == 1;
             if (node->src[1]->op == GGML_OP_VIEW) {
-                op_case = (node->src[1]->view_offs == 0 ? 1 : 2) + (single_slot ? 2 : 0);
-            } else if (single_slot) {
-                // a single-slot cache holds exactly the one state the gather selects
-                op_case = 3;
+                op_case = node->src[1]->view_offs == 0 ? 1 : 2;
+            }
+        } else if (is_get_rows_batched_4d(node)) {
+            // with one token the batch dims look like a 3D gather; the tracked token dim tells them apart
+            const auto it = m_node_dynamic_dims.find(const_cast<ggml_tensor *>(node));
+            if (node->ne[3] > 1 || (it != m_node_dynamic_dims.end() && it->second == 3)) {
+                op_case = 5;
             }
         }
         break;
@@ -456,8 +658,17 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
         break;
     }
     case GGML_OP_VIEW: {
-        if (!m_model_params.has_rs_rollback && node->src[0] != nullptr &&
-            node->src[0]->op == GGML_OP_GATED_DELTA_NET) {
+        if (is_sampling_view(node)) {
+            return 3;
+        }
+        if (node->view_offs == 0 && node->src[0]->op == GGML_OP_GET_ROWS &&
+            is_recurrent_cache(node->src[0]->src[0]) && node->ne[0] == node->src[0]->ne[0] &&
+            node->ne[2] == 1 && node->ne[3] == 1 && m_compute_params.s_copy_active_slot_len != -1) {
+            // build_rs gathers active and inactive rows together; consumers need only the
+            // active prefix, whose length changes independently of the gather's row count.
+            return 4;
+        }
+        if (is_direct_gdn_view(node)) {
             // The GDN translator publishes native attention/state outputs under these VIEW names.
             op_case = 2;
             break;
@@ -519,16 +730,23 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
         if (node->src[0]->op == GGML_OP_VIEW) {
             if (is_same_shape(node->src[0]->src[0], node->src[0])) {
                 op_case = 1;
-            } else if (!m_model_params.has_rs_rollback &&
-                       node->src[0]->src[0]->op == GGML_OP_GATED_DELTA_NET) {
-                // GDN attention is routed directly to this VIEW by get_output_names().
-                op_case = 3;
+            } else if (is_direct_gdn_view(node->src[0])) {
+                // GDN attention is routed directly to this VIEW by get_output_names(). 3: the GDN q input may be
+                // pre-scaled, and the norm then scales its eps to match; 4: it is not.
+                op_case = is_gdn_out_prescaled(node->src[0]->src[0]) ? 3 : 4;
             } else if (node->src[0]->src[0]->op == GGML_OP_GATED_DELTA_NET) {
-                op_case = 2;
+                op_case = is_gdn_out_prescaled(node->src[0]->src[0]) ? 3 : 2;
             }
         }
         break;
     }
+    case GGML_OP_GATED_DELTA_NET: {
+        if (is_gdn_out_prescaled(node)) {
+            op_case = 1;
+        }
+        break;
+    }
+    case GGML_OP_POOL_1D:
     case GGML_OP_POOL_2D: {
         const ggml_op_pool pool_mode = static_cast<ggml_op_pool>(node->op_params[0]);
         switch (pool_mode) {
@@ -571,15 +789,16 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
     case GGML_OP_CPY: {
         if (node->src[0]->op == GGML_OP_VIEW) {
             if (node->src[0]->src[0]->op == GGML_OP_GATED_DELTA_NET) {
-                if (!m_model_params.has_rs_rollback) {
-                    // op_case 7 replaces a single-slot cache; op_case 10 writes native GDN state
-                    // into an active range of a larger non-rollback cache.
-                    op_case = is_full_single_slot_writeback(node) ? 7 : 10;
+                if (is_gdn_snapshot_writeback(node)) {
+                    op_case = is_direct_gdn_view(node->src[0]) ? 12 : 11;
+                } else if (!m_model_params.has_rs_rollback) {
+                    // Write native GDN state into the runtime active range of the cache.
+                    op_case = 10;
                 } else {
                     op_case = 1;
                 }
             } else if (GgmlOvDecoder::is_conv_state_writeback(node)) {
-                op_case = is_full_single_slot_writeback(node) ? 8 : 2;
+                op_case = 2;
                 break;
             } else if (is_conv_states_all_tensor(node->view_src) && node->src[1] != nullptr &&
                        node->src[1]->op == GGML_OP_VIEW && node->src[1]->view_src == node->view_src) {
@@ -587,16 +806,15 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
                 break;
             } else if (node->src[0]->src[0]->op == GGML_OP_GET_ROWS && node->src[1] != nullptr &&
                        node->src[1]->op == GGML_OP_VIEW && node->src[1]->view_src != nullptr &&
-                       is_recurrent_cache(node->src[1]->view_src) && node->src[1]->view_src->ne[1] == 1) {
-                // defrag remainder writeback of a single-slot cache, taken from a view of the one gather of
-                // all states (see build_rs)
-                op_case = 9;
+                       is_recurrent_cache(node->src[1]->view_src)) {
+                // Remainder view of the combined state gather (see build_rs).
+                op_case = 3;
             }
         } else if (node->src[0]->op == GGML_OP_GET_ROWS && node->src[1] != nullptr &&
                    node->src[1]->op == GGML_OP_VIEW && node->src[1]->view_src != nullptr &&
                    is_recurrent_cache(node->src[1]->view_src)) {
             // s_copy defrag remainder writeback: gathered extra state rows copied back into the cache
-            op_case = node->src[1]->view_src->ne[1] == 1 ? 9 : 3;
+            op_case = 3;
         } else if (node->src[1] != nullptr && node->src[1]->op == GGML_OP_VIEW && node->src[1]->view_src != nullptr) {
             // op_case 5: KV write for decoder self-attention (dynamic write offset)
             // op_case 6: KV write for encoder self-attn or cross-attn (static offset)
@@ -633,9 +851,10 @@ int GgmlOvDecoder::compute_op_case(const ggml_tensor * node) const {
         }
         break;
     }
-    case GGML_OP_SCALE: {
+    case GGML_OP_SCALE:
+    case GGML_OP_FILL: {
         if (node->view_src && node->buffer->usage == GGML_BACKEND_BUFFER_USAGE_ANY) {
-            op_case = node->view_src->ne[1] == 1 ? 2 : 1;
+            op_case = 1;
         }
         break;
     }
@@ -709,6 +928,16 @@ static int get_swa_window_from_mask(const ggml_tensor * mask) {
     return window > 0 ? (int) window : -1;
 }

+// The KQ product that the SOFT_MAX of an attention reads: directly, or through the logit soft-capping of gemma-2
+// (SCALE, TANH, SCALE)
+static const ggml_tensor * softmax_scores_source(const ggml_tensor * scores) {
+    while (scores != nullptr && (scores->op == GGML_OP_SCALE ||
+                                 (scores->op == GGML_OP_UNARY && ggml_get_unary_op(scores) == GGML_UNARY_OP_TANH))) {
+        scores = scores->src[0];
+    }
+    return scores;
+}
+
 std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgraph * cgraph, bool is_static) {
     ModelParams model_params;
     ComputeParams compute_params;
@@ -751,12 +980,12 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
                 break;
             }
             break;
-        case GGML_OP_SOFT_MAX:
-            // case 2: node op is SOFT_MAX, src 0 not null & op is MUL_MAT & the src 0 of MUL_MAT is PERMUTE & the permuted tensor src is the view of cache k
-            if (node->src[0] != nullptr && node->src[1] != nullptr && node->src[0]->op == GGML_OP_MUL_MAT &&
-                node->src[0]->src[0] != nullptr && node->src[0]->src[1] != nullptr &&
-                node->src[0]->src[0]->op == GGML_OP_PERMUTE && node->src[0]->src[0]->src[0] != nullptr &&
-                node->src[0]->src[0]->src[0]->op == GGML_OP_VIEW) {
+        case GGML_OP_SOFT_MAX: {
+            // case 2: node op is SOFT_MAX, its scores come from a MUL_MAT (see softmax_scores_source) & the src 0 of MUL_MAT is PERMUTE & the permuted tensor src is the view of cache k
+            const ggml_tensor * kq = softmax_scores_source(node->src[0]);
+            if (kq != nullptr && node->src[1] != nullptr && kq->op == GGML_OP_MUL_MAT && kq->src[0] != nullptr &&
+                kq->src[1] != nullptr && kq->src[0]->op == GGML_OP_PERMUTE && kq->src[0]->src[0] != nullptr &&
+                kq->src[0]->src[0]->op == GGML_OP_VIEW) {
                 return 2;
             }
             // case 3: node op is SOFT_MAX, src 0 not null & op is ADD & the src 0 of ADD is MUL_MAT & the src 0 of MUL_MAT is PERMUTE
@@ -766,6 +995,7 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
                 return 3;
             }
             break;
+        }
         default:
             break;
         }
@@ -788,6 +1018,22 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
         }
     };

+    // The PERMUTE of the K cache view an attention node reads, for the same src layouts
+    auto get_attention_cache_k_permute = [&get_attention_pattern_case](const ggml_tensor * node) -> ggml_tensor * {
+        switch (get_attention_pattern_case(node)) {
+        case 0:
+            return node->src[1];
+        case 1:
+            return node->src[1]->src[0];
+        case 2:
+            return softmax_scores_source(node->src[0])->src[0];
+        case 3:
+            return node->src[0]->src[0]->src[0];
+        default:
+            return nullptr;
+        }
+    };
+
     // Pre-pass: classify sliding-window vs full-attention layers.
     //
     // An interleaved-SWA model keeps two KV caches and two attention masks, and hands each layer
@@ -803,10 +1049,9 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
     // from the first graph onwards, while the view grows with context depth and would invert the
     // comparison at shallow depth.
     //
-    // Layers whose leaf is smaller than the largest leaf are the windowed ones. When every layer
-    // reports the same extent there is no distinction to draw -- either the model has no windowed
-    // layers, or the window is at least as large as the context so the two caches coincide, in
-    // which case a windowed layer and a full-attention one compute the same thing.
+    // Layers whose leaf is smaller than the largest leaf are the windowed ones. Equal extents do not
+    // rule them out: the windowed cache is padded to n_swa + n_ubatch, which can equal the full
+    // context (gemma-4 at -c 1024 -ub 512). See the mask fallback below.
     //
     // Getting this wrong is silent and severe: with the windowed layers classified as
     // full-attention, permute's KV slicing uses attention_size instead of attention_size_swa. The
@@ -822,14 +1067,7 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
             if (mask == nullptr) {
                 continue;
             }
-            const ggml_tensor * cache_k_permute = nullptr;
-            switch (get_attention_pattern_case(cgraph->nodes[i])) {
-            case 0:  cache_k_permute = cgraph->nodes[i]->src[1];                     break;
-            case 1:  cache_k_permute = cgraph->nodes[i]->src[1]->src[0];             break;
-            case 2:  cache_k_permute = cgraph->nodes[i]->src[0]->src[0];             break;
-            default: cache_k_permute = cgraph->nodes[i]->src[0]->src[0]->src[0];     break;
-            }
-            const ggml_tensor * cache_k_view = cache_k_permute->src[0];
+            const ggml_tensor * cache_k_view = get_attention_cache_k_permute(cgraph->nodes[i])->src[0];
             if (cache_k_view->op != GGML_OP_VIEW) {
                 continue;
             }
@@ -839,8 +1077,13 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
                 continue;
             }
             layer_extent[layer.value()] = leaf->ne[1];
-            layer_mask[layer.value()] = mask;
+            // the layer's own attention comes first: gemma-4's KV-shared layers read the same cache with mask rows
+            // taken from the graph input
+            layer_mask.emplace(layer.value(), mask);
             max_extent = std::max(max_extent, leaf->ne[1]);
+            if (mask->op == GGML_OP_NONE) {
+                model_params.kq_masks.insert(mask);
+            }
         }

         for (const auto & [layer, extent] : layer_extent) {
@@ -851,6 +1094,32 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
                 }
             }
         }
+
+        // equal extents: tell the two masks apart by the layers that read them, the larger group gets the swa role
+        // (gemma-4: 4 of 5 layers). Groups of equal size can not be told apart, so leave the layers unclassified.
+        // Two masks do not imply swa (deepseek sparse attention builds two full masks), but the role does not need
+        // them to: it gives each group its own mask, KV write indices and attention size, which is right either way.
+        // The only use of the window itself, the stateful mask rebuild, reads it back from the mask, and on a full
+        // mask that gives back the causal mask. Decide from the first graph, before the window shows in the mask, so
+        // the graph does not change, and is not recompiled, when it does.
+        // TODO: "larger group is swa" is a model-level guess. Revisit when llama.cpp tells the backend which layers
+        // use a sliding window. Stateless graphs only need the two groups kept apart (separate mask, KV write index
+        // and attention size inputs), not the swa role itself, so the guess could be limited to stateful execution.
+        if (model_params.swa_layers.empty()) {
+            std::map<const ggml_tensor *, std::vector<int>> mask_layers;
+            for (const auto & [layer, mask] : layer_mask) {
+                mask_layers[mask].push_back(layer);
+            }
+            if (mask_layers.size() == 2 &&
+                mask_layers.begin()->second.size() != std::next(mask_layers.begin())->second.size()) {
+                auto swa = mask_layers.begin();
+                if (std::next(swa)->second.size() > swa->second.size()) {
+                    swa = std::next(swa);
+                }
+                model_params.swa_layers = swa->second;
+                model_params.swa_mask = swa->first;
+            }
+        }
         std::sort(model_params.swa_layers.begin(), model_params.swa_layers.end());

         if (ggml_openvino_getenv_int("GGML_OPENVINO_LOG_SWA_LAYERS")) {
@@ -867,26 +1136,28 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
     bool rope_seen = false;
     for (int i = 0; i < cgraph->n_nodes; i++) {
         ggml_tensor * node = cgraph->nodes[i];
+        // `-fa off` stores the V cache transposed: SET_ROWS writes it through a [1, n] reshape, one element per row
+        if (node->op == GGML_OP_SET_ROWS && node->src[2] != nullptr && node->src[2]->op == GGML_OP_RESHAPE &&
+            node->src[2]->ne[0] == 1 && is_kvcache(node->src[2]->src[0], nullptr)) {
+            model_params.v_trans = true;
+        }
+        if (node->op == GGML_OP_FLASH_ATTN_EXT) {
+            float logit_softcap = 0.0f;
+            memcpy(&logit_softcap, (const float *) node->op_params + 2, sizeof(float));
+            model_params.softcap_attn = model_params.softcap_attn || logit_softcap != 0.0f;
+        }
         const int attention_pattern_case = get_attention_pattern_case(node);
         if (attention_pattern_case != -1) {
-            ggml_tensor * cache_k_permute = nullptr;
+            ggml_tensor * cache_k_permute = get_attention_cache_k_permute(node);
             ggml_tensor * mask = nullptr;

             switch (attention_pattern_case) {
             case 0:
-                cache_k_permute = node->src[1];
-                mask = node->src[3];
-                break;
             case 1:
-                cache_k_permute = node->src[1]->src[0];
                 mask = node->src[3];
                 break;
             case 2:
-                cache_k_permute = node->src[0]->src[0];
-                mask = node->src[1];
-                break;
             case 3:
-                cache_k_permute = node->src[0]->src[0]->src[0];
                 mask = node->src[1];
                 break;
             case 4:
@@ -909,8 +1180,9 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr

             model_params.head_size = cache_k_permute->ne[0];
             model_params.n_heads_kv = cache_k_permute->ne[2];
-            compute_params.input_len = node->src[0]->ne[1];
-            compute_params.token_len_per_seq = node->src[0]->ne[1];
+            // gemma-4's KV-shared layers attend for the output tokens only, so the batch is the largest query
+            compute_params.input_len = std::max<int>(compute_params.input_len, node->src[0]->ne[1]);
+            compute_params.token_len_per_seq = std::max<int>(compute_params.token_len_per_seq, node->src[0]->ne[1]);

             auto * cache_k_view = cache_k_permute->src[0];
             if (cache_k_view->op != GGML_OP_VIEW || mask == nullptr) {
@@ -940,14 +1212,30 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
             memcpy(&offset, cache_k_view->op_params, sizeof(size_t));
             compute_params.seq_active_start = offset / seq_size;

+            // a mask that is itself a graph input has the shape of its OV parameter
+            const std::string mask_name = mask->op != GGML_OP_NONE      ? std::string() :
+                                          mask == model_params.swa_mask ? get_tensor_ov_name(cgraph, mask) + "_swa" :
+                                                                          get_tensor_ov_name(cgraph, mask);
+            // gemma-4's KV-shared layers come last and take their mask rows from the input masks with GET_ROWS: keep
+            // the name and window read from the input
+            const bool derived_mask = mask_name.empty();
             if (layer_is_swa) {
                 compute_params.attention_size_swa = mask->ne[0];
-                compute_params.swa_window = get_swa_window_from_mask(mask);
+                if (!derived_mask || compute_params.mask_swa_name.empty()) {
+                    compute_params.swa_window = get_swa_window_from_mask(mask);
+                    compute_params.mask_swa_name = mask_name;
+                }
             } else {
                 compute_params.attention_size = mask->ne[0];
+                if (!derived_mask || compute_params.mask_name.empty()) {
+                    compute_params.mask_name = mask_name;
+                }
             }
             if (is_static) {
-                compute_params.attention_size = model_params.ctx_per_seq;
+                const bool slice_prefill = ggml_openvino_kv_slice_enabled() && compute_params.input_len > 1;
+                if (!slice_prefill) {
+                    compute_params.attention_size = model_params.ctx_per_seq;
+                }
                 compute_params.attention_size_swa = model_params.ctx_per_seq_swa;
                 compute_params.token_len_per_seq = 1;
             }
@@ -958,6 +1246,7 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
             if (node->src[1]->op == GGML_OP_PERMUTE && node->src[1]->src[0]->op == GGML_OP_VIEW &&
                 node->src[1]->src[0]->src[0]->op == GGML_OP_ROPE) {
                 compute_params.attention_size = node->ne[0];
+                compute_params.mask_name.clear();
             }
         }

@@ -965,7 +1254,8 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
         if (node->op == GGML_OP_TRANSPOSE && node->src[0]->op == GGML_OP_PERMUTE &&
             node->src[0]->src[0]->op == GGML_OP_VIEW) {
             compute_params.attention_size = node->ne[0];
-            if (is_static) {
+            compute_params.mask_name.clear();
+            if (is_static && !(ggml_openvino_kv_slice_enabled() && compute_params.input_len > 1)) {
                 compute_params.attention_size = model_params.ctx_per_seq;
             }
         }
@@ -988,15 +1278,44 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
         }
         if (node->op == GGML_OP_GATED_DELTA_NET) {
             model_params.state_size = node->src[0]->ne[0];
-        }
-        if (node->op == GGML_OP_SCALE && node->view_src != nullptr && is_recurrent_cache(node->view_src)) {
+            model_params.has_rs_rollback |= node->op_params[0] > 1;
+            compute_params.gdn_attention_rows[get_tensor_ov_name(cgraph, node)] = node->src[2]->ne[2] * node->src[2]->ne[3];
+        }
+        if (is_sampling_view(node)) {
+            compute_params.sampling_rows[get_tensor_ov_name(cgraph, node)] =
+                (node->flags & GGML_TENSOR_FLAG_COMPUTE) ? node->view_offs / node->src[0]->nb[1] : 0;
+        }
+        if (is_gdn_snapshot_writeback(node)) {
+            const auto * source = node->src[0];
+            const auto * destination = node->src[1];
+            const auto * packed = source->src[0];
+            ComputeParams::RsWriteback writeback;
+            writeback.slot_begin = destination->view_offs / destination->nb[1];
+            writeback.src_begin = source->view_offs / packed->nb[1];
+            writeback.seq_count = destination->ne[1];
+            writeback.snapshot_stride = destination->nb[2] / destination->nb[1];
+            compute_params.rs_writebacks[get_tensor_ov_name(cgraph, node)] = writeback;
+        }
+        if (is_conv_state_snapshot_writeback(node)) {
+            const auto source = node->src[0];
+            const auto dest = node->src[2];
+            ComputeParams::RsWriteback writeback;
+            writeback.slot_begin = dest->view_offs / node->view_src->nb[1];
+            writeback.src_begin = source->view_offs / source->nb[0];
+            writeback.seq_count = dest->ne[3];
+            writeback.snapshot_stride = dest->nb[1] / node->view_src->nb[1];
+            writeback.window_count = source->ne[1];
+            compute_params.rs_writebacks[get_tensor_ov_name(cgraph, node)] = writeback;
+            compute_params.s_copy_active_slot_len = dest->ne[3];
+        }
+        if (is_recurrent_cache_reset(node)) {
             if (model_params.n_rs_slots == -1) {
                 model_params.n_rs_slots = node->view_src->ne[1];
             } else {
                 GGML_ASSERT(model_params.n_rs_slots == node->view_src->ne[1]);
             }
             compute_params.cache_rs_reset_len = ggml_nelements(node) / node->view_src->ne[0];
-            compute_params.cache_rs_reset_idx = node->src[0]->view_offs / node->view_src->ne[0];
+            compute_params.cache_rs_reset_idx = node->src[0]->view_offs / node->view_src->nb[1];
         }
         // Capture the destination slot block of every recurrent state cache writeback, plus the
         // source window needed by conv state and packed GDN rollback writes. The active sequences
@@ -1007,7 +1326,9 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
             const bool is_conv = is_conv_state_writeback(node);
             const bool is_gdn = node->src[0]->op == GGML_OP_VIEW && node->src[0]->src[0] != nullptr &&
                                 node->src[0]->src[0]->op == GGML_OP_GATED_DELTA_NET;
-            const bool is_extra = node->src[0]->op == GGML_OP_GET_ROWS;
+            const bool is_extra_view = node->src[0]->op == GGML_OP_VIEW &&
+                                       node->src[0]->src[0]->op == GGML_OP_GET_ROWS;
+            const bool is_extra = node->src[0]->op == GGML_OP_GET_ROWS || is_extra_view;
             const bool is_gdn_rollback = is_gdn && is_same_shape(node->src[0], node->src[1]);

             const ggml_tensor * dest_view = node->src[1];
@@ -1020,17 +1341,20 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
                 // signature is local to the CPY and still works when fallback splits the graph.
                 model_params.has_rs_rollback = true;
             }
-            if (row_bytes > 0 && (is_conv || is_gdn || is_extra) && !is_full_single_slot_writeback(node)) {
+            if (row_bytes > 0 && (is_conv || is_gdn || is_extra) &&
+                !(is_gdn && node->src[0]->src[0]->op_params[0] > 1)) {
                 ComputeParams::RsWriteback writeback;
                 writeback.slot_begin = (int) (dest_view->view_offs / row_bytes);
                 if (is_conv) {
                     writeback.src_begin = (int) (node->src[0]->view_offs / node->src[0]->view_src->nb[0]);
                 } else if (is_gdn_rollback) {
                     writeback.src_begin = (int) (node->src[0]->view_offs / node->src[0]->view_src->nb[1]);
+                } else if (is_extra_view) {
+                    writeback.src_begin = (int) (node->src[0]->view_offs / row_bytes);
                 }
                 compute_params.rs_writebacks[get_tensor_ov_name(cgraph, node)] = writeback;
             }
-            if ((is_conv || is_gdn) && !is_full_single_slot_writeback(node)) {
+            if (is_conv || is_gdn) {
                 compute_params.s_copy_active_slot_len = (int) dest_view->ne[1];
             }
         }
@@ -1061,7 +1385,7 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr

     auto * output_tensor = cgraph->nodes[cgraph->n_nodes - 1];
     compute_params.output_len = output_tensor->ne[1];
-    if (model_params.is_cacheless_attn) {
+    {
         for (int i = 0; i < cgraph->n_nodes; i++) {
             const auto * node = cgraph->nodes[i];
             if (node->op == GGML_OP_GET_ROWS && is_output_idx(node->src[1], node)) {
@@ -1074,6 +1398,14 @@ std::pair<ModelParams, ComputeParams> GgmlOvDecoder::compute_llm_params(ggml_cgr
     if (is_static && compute_params.output_len == 0) {
         compute_params.output_len = 1;
     }
+    // A sliced static prefill attends a power-of-two number of KV rows: every attended size needs its own compiled
+    // prefill model, so a growing sequence compiles one when the attended length doubles, not for each ubatch.
+    // Rounding up adds masked rows, which cost prefill time, so it stops at the next power of two.
+    if (is_static && ggml_openvino_kv_slice_enabled() && compute_params.input_len > 1 &&
+        !model_params.is_cacheless_attn && compute_params.attention_size > 0 && model_params.ctx_per_seq > 0) {
+        compute_params.attention_size =
+            ggml_openvino_static_kv_window(compute_params.attention_size, model_params.ctx_per_seq);
+    }
     model_params.ctx = model_params.ctx_per_seq * model_params.n_seq;
     return {model_params, compute_params};
 }
@@ -1090,6 +1422,7 @@ ov::PartialShape GgmlOvDecoder::get_graph_input_shape(const ggml_tensor * op,
     if (m_naive) {
         return input != nullptr ? ov::PartialShape{get_shape(input)} : ov::PartialShape{get_shape(op)};
     }
+    const std::string name = input != nullptr ? std::string(input->name) : std::string();
     ov::PartialShape input_shape;

     if (is_inp_tok(input, op) || is_inp_pos(input, op)) {
@@ -1105,6 +1438,9 @@ ov::PartialShape GgmlOvDecoder::get_graph_input_shape(const ggml_tensor * op,
         // output index
         input_shape = ov::PartialShape{1, 1, 1, m_is_static ? m_compute_params.output_len : -1};

+    } else if (!m_is_static && is_activation_input(input, op)) {
+        input_shape = ov::PartialShape{1, 1, -1, input->ne[0]};
+
     } else if (is_inp_mean(input, op)) {
         input_shape = m_is_static ? ov::PartialShape{1, 1, input->ne[1], m_prefill_chunk_size} :
                                     ov::PartialShape{1, 1, -1, -1};
@@ -1115,7 +1451,17 @@ ov::PartialShape GgmlOvDecoder::get_graph_input_shape(const ggml_tensor * op,
     } else if (is_inp_mask(input, op)) {
         // mask
         if (m_is_static) {
-            input_shape = ov::PartialShape{1, 1, m_is_prefill ? m_prefill_chunk_size : 1, m_model_params.ctx};
+            // The sliding-window mask spans the window cache, which is smaller than the context once
+            // the context exceeds n_swa + n_ubatch (gemma-4 at -c 4096 -ub 1024: 1536 vs 4096 cells).
+            // Both masks are named "attn_inp_kq_mask", so tell them apart by tensor identity.
+            const bool swa_mask = is_swa_mask(input);
+            int attention_size = swa_mask ? m_model_params.ctx_per_seq_swa * m_model_params.n_seq : m_model_params.ctx;
+            const bool kv_window =
+                m_is_prefill ? ggml_openvino_kv_slice_enabled() : ggml_openvino_npu_decode_bucket() > 0;
+            if (kv_window) {
+                attention_size = swa_mask ? m_compute_params.attention_size_swa : m_compute_params.attention_size;
+            }
+            input_shape = ov::PartialShape{1, 1, m_is_prefill ? m_prefill_chunk_size : 1, attention_size};
         } else if (m_is_stateful) {
             input_shape = ov::PartialShape{1, 1, -1, -1};
         } else {
@@ -1138,10 +1484,27 @@ ov::PartialShape GgmlOvDecoder::get_graph_input_shape(const ggml_tensor * op,
         if (!m_is_static && !is_flat_kv) {
             // do not fix ctx size to make llama-bench work across test params
             input_shape[2] = -1;
-        }
-        if (is_stateful() && !is_flat_kv) {
+        } else if (m_is_prefill ? ggml_openvino_kv_slice_enabled() :
+                                  m_is_static && ggml_openvino_npu_decode_bucket() > 0) {
+            const auto layer = extract_layer_from_name(name);
+            if (layer.has_value() && !is_swa_layer(*layer) && m_compute_params.attention_size > 0 &&
+                m_compute_params.attention_size < input_shape[2].get_length()) {
+                const int attention_size = m_compute_params.attention_size;
+                input_shape[2] = attention_size;
+            }
+            if (layer.has_value() && !is_swa_layer(*layer) && ggml_openvino_npu_kv_head_major_enabled()) {
+                GGML_ASSERT(input_shape[1].get_length() == 1);
+                const int64_t heads = get_n_heads_kv_for_tensor(input);
+                const int64_t head_size = input_shape[3].get_length() / heads;
+                input_shape = name.rfind("cache_v_l", 0) == 0 ?
+                                  ov::PartialShape{1, heads, head_size, input_shape[2]} :
+                                  ov::PartialShape{1, heads, input_shape[2], head_size};
+            }
+        }
+        if ((is_stateful() && !is_flat_kv) || is_kvcache_4d(input)) {
             // Convert stateless KV cache layout [1, 1, seq, n_heads_kv * head_size]
-            // to stateful layout [1, seq, n_heads_kv, head_size].
+            // to stateful layout [1, seq, n_heads_kv, head_size]. Stateless GPU graphs use the same
+            // layout for the caches that qualify (see is_kvcache_4d), bound over the same memory.
             // NOTE: Gemma4 uses per-layer-type KV shapes, so no single scalar describes every
             // layer. E2B varies only the head size (sliding 256, full 512); 12B also varies the
             // head COUNT (sliding 8 x 256, full 1 x 512). Take the head count for this tensor's
@@ -1167,7 +1530,13 @@ ov::PartialShape GgmlOvDecoder::get_graph_input_shape(const ggml_tensor * op,
         // index list has a static length; on CPU/GPU it may change across compiles (defrag).
         input_shape = m_is_static ? ov::PartialShape{get_shape(input)} : ov::PartialShape{1, 1, 1, -1};

-    } else {
+    } else if (input != nullptr && op != nullptr && is_inp_emb(input, op)) {
+        input_shape = ov::PartialShape{get_shape(input)};
+        if (!m_is_static) {
+            input_shape[2] = -1;
+        }
+    }
+    else {
         input_shape = ov::PartialShape{get_shape(input)};
     }
     if (dynamic_dim_index != -1 && m_model_is_splitted) {
@@ -1218,13 +1587,15 @@ bool GgmlOvDecoder::is_s_copy_leaf(const ggml_tensor * tensor) const {
 }

 void GgmlOvDecoder::add_extra_inputs() {
+    m_model_extra_inputs.clear();
+
     // Extra inputs:
     // 1. `attention_size`, used in FLASH_ATTN where the shape of the matmul's are 256 aligned,
     //     see llama_kv_cache_unified::get_n_kv and llama_kv_cache_unified::get_padding.
     // 2. `n_seq_active` and `seq_active_start`, used in FLASH_ATTN_EXT to indicate the active sequences in the batch

     auto create_1d_input = [this](const std::string & name, int64_t value, bool force_parameter = false) {
-        m_model_extra_inputs[name] = {ov::element::i64, ov::Shape{1}, value, force_parameter || !m_is_static};
+        m_model_extra_inputs[name] = {ov::element::i64, ov::Shape{1}, value, force_parameter || !m_is_static, {}, -1};
     };

     if (m_compute_params.attention_size != -1) {
@@ -1246,13 +1617,66 @@ void GgmlOvDecoder::add_extra_inputs() {
     if (m_compute_params.token_len_per_seq != -1) {
         create_1d_input("token_len_per_seq", m_compute_params.token_len_per_seq);
     }
+    if (m_is_static && m_is_prefill && ggml_openvino_npu_hfa_prefill_enabled()) {
+        if (m_compute_params.n_seq_active != 1) {
+            throw std::runtime_error("GGML_OPENVINO_NPU_HFA_PREFILL currently requires one active sequence");
+        }
+        if (!m_model_params.swa_layers.empty()) {
+            throw std::runtime_error("GGML_OPENVINO_NPU_HFA_PREFILL does not yet support sliding-window layers");
+        }
+        const auto chunk_size = static_cast<size_t>(m_prefill_chunk_size);
+        const auto input_len = static_cast<size_t>(m_compute_params.input_len);
+        const auto num_chunks = (input_len + chunk_size - 1) / chunk_size;
+        // Head-major KV: the past is the whole cache storage [1, heads, attention_size, head_size].
+        const bool head_major = ggml_openvino_npu_kv_head_major_enabled();
+        const auto past_capacity = head_major ? static_cast<size_t>(m_compute_params.attention_size) :
+                                                std::max(chunk_size, (num_chunks - 1) * chunk_size);
+        for (const auto & [kv_name, kv_tensor] : m_inputs) {
+            if (kv_name.rfind("cache_k_l", 0) != 0 && kv_name.rfind("cache_v_l", 0) != 0) {
+                continue;
+            }
+            const auto n_heads_kv = static_cast<size_t>(get_n_heads_kv_for_tensor(kv_tensor));
+            const auto combined_dim = static_cast<size_t>(kv_tensor->ne[0]);
+            GGML_ASSERT(n_heads_kv > 0 && combined_dim % n_heads_kv == 0);
+            const auto head_size = combined_dim / n_heads_kv;
+            // flash_attn_ext looks the past input up by layer, so drop the graph-ordinal suffix ("#l12").
+            const std::string cache_name = kv_name.substr(0, kv_name.find('#'));
+            const bool v_transposed = head_major && kv_name.rfind("cache_v_l", 0) == 0;
+            m_model_extra_inputs["ggml_hfa_past_" + cache_name] = {
+                get_ov_type(kv_tensor),
+                v_transposed ? ov::Shape{1, n_heads_kv, head_size, past_capacity} :
+                head_major   ? ov::Shape{1, n_heads_kv, past_capacity, head_size} :
+                               ov::Shape{1, past_capacity, n_heads_kv, head_size},
+                0, true, {}, -1};
+        }
+        m_model_extra_inputs["ggml_hfa_mask"] = {
+            ov::element::f16, ov::Shape{1, 1, chunk_size, past_capacity + chunk_size}, 0, true, {}, -1};
+    }
     // create_1d_input("token_len", m_compute_params.token_len_per_seq * m_compute_params.n_seq_active);

-    if (m_compute_params.cache_rs_reset_idx != -1 && m_model_params.n_rs_slots != 1) {
+    // n_seq_active and attention_size(_swa) are dimensions of the KQ masks. Read them from the mask shape inside the
+    // model: the GPU plugin then gets the Q Reshape and KV Slice shapes from shape inference instead of waiting for
+    // the input values on the device
+    if (!m_is_static && !m_is_stateful && ggml_openvino_shape_from_mask_enabled()) {
+        auto from_mask = [this](const std::string & name, const std::string & mask, int64_t axis) {
+            auto it = m_model_extra_inputs.find(name);
+            if (it != m_model_extra_inputs.end() && !mask.empty() && m_model_inputs.count(mask) != 0) {
+                it->second.shape_source = mask;
+                it->second.shape_axis = axis;
+            }
+        };
+        const auto & full = m_compute_params.mask_name;
+        const auto & swa = m_compute_params.mask_swa_name;
+        from_mask("attention_size", full, 3);
+        from_mask("attention_size_swa", swa, 3);
+        from_mask("n_seq_active", full.empty() ? swa : full, 0);
+    }
+
+    if (m_compute_params.cache_rs_reset_idx != -1) {
         // Whether/which cache slot to reset varies per compute call (e.g. a new sequence starting
         // vs. continued decoding). can_reuse_statically() does not invalidate the cached static
         // model on ComputeParams changes, so these must stay runtime Parameters even when static
-        // (scale.cpp op_case 1 only uses them in value comparisons, never as Slice bounds, so this
+        // (translate_recurrent_cache_reset only uses them in value comparisons, never as Slice bounds, so this
         // does not reintroduce dynamic shapes).
         create_1d_input("cache_rs_reset_idx", m_compute_params.cache_rs_reset_idx, /*force_parameter=*/true);
         create_1d_input("cache_rs_reset_len", m_compute_params.cache_rs_reset_len, /*force_parameter=*/true);
@@ -1260,6 +1684,13 @@ void GgmlOvDecoder::add_extra_inputs() {

     if (m_compute_params.s_copy_active_slot_len != -1) {
         create_1d_input("s_copy_active_slot_len", m_compute_params.s_copy_active_slot_len);
+        if (m_model_inputs.count("rs_active_rows") != 0) {
+            // The row-map prefix is rebound with its current length on every compute.
+            // Shape consumers must not wait for a scalar uploaded to the GPU.
+            auto & active = m_model_extra_inputs.at("s_copy_active_slot_len");
+            active.shape_source = "rs_active_rows";
+            active.shape_axis = 3;
+        }
         if (m_is_static) {
             // Number of real tokens in the current prefill chunk. The last chunk is padded with
             // fabricated token ids; attention masks them out, but the recurrent (GDN/conv) path
@@ -1275,6 +1706,80 @@ void GgmlOvDecoder::add_extra_inputs() {
         if (!m_is_static && writeback.src_begin >= 0) {
             create_1d_input("rs_src_begin_" + node_name, writeback.src_begin);
         }
+        if (writeback.seq_count >= 0) {
+            create_1d_input("rs_seq_count_" + node_name, writeback.seq_count);
+            create_1d_input("rs_snapshot_stride_" + node_name, writeback.snapshot_stride);
+        }
+        if (writeback.window_count >= 0) {
+            create_1d_input("rs_window_count_" + node_name, writeback.window_count);
+        }
+    }
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+    m_compute_params.recurrent_controls.clear();
+    // Naive graphs have no full-model cache fusion and node cases are not initialized here.
+    if (!m_naive && !m_is_static && !m_is_stateful && !m_model_is_splitted && ggml_openvino_is_gpu() &&
+        !ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_FUSED_RECURRENCE")) {
+        for (int i = 0; i < m_cgraph->n_nodes; ++i) {
+            const auto node = m_cgraph->nodes[i];
+            const auto op_case = m_node_info_list[i].node_op_case;
+            if (node->op == GGML_OP_CPY && (op_case == 10 || op_case == 12)) {
+                m_model_extra_inputs["rs_update_" + get_tensor_ov_name(m_cgraph, node)] =
+                    {ov::element::i64, ov::Shape{1, 1, 1, 4}, 0, true, {}, -1};
+            }
+        }
+        for (const auto & [name, write] : m_compute_params.rs_writebacks) {
+            if (write.src_begin >= 0) {
+                auto & values = m_compute_params.recurrent_controls["rs_offsets_" + name];
+                values = {write.slot_begin, write.src_begin};
+                m_model_extra_inputs["rs_offsets_" + name] =
+                    {ov::element::i64, ov::Shape{1, 1, 1, 2}, 0, true, {}, -1};
+            }
+        }
+        for (int i = 0; i < m_cgraph->n_nodes; ++i) {
+            const auto concat = m_cgraph->nodes[i];
+            if (concat->op != GGML_OP_CONCAT || !concat->src[0] || concat->src[0]->op != GGML_OP_RESHAPE ||
+                !concat->src[0]->src[0] || concat->src[0]->src[0]->op != GGML_OP_VIEW ||
+                concat->src[0]->src[0]->src[0]->op != GGML_OP_GET_ROWS) {
+                continue;
+            }
+            std::vector<int64_t> values;
+            for (int j = i + 1; j < m_cgraph->n_nodes; ++j) {
+                const auto write = m_cgraph->nodes[j];
+                if ((is_conv_state_writeback(write) || is_conv_state_snapshot_writeback(write)) &&
+                    write->src[0]->src[0] == concat) {
+                    const auto it = m_compute_params.rs_writebacks.find(get_tensor_ov_name(m_cgraph, write));
+                    if (it == m_compute_params.rs_writebacks.end()) {
+                        values.clear();
+                        break;
+                    }
+                    const auto & wb = it->second;
+                    const int64_t count = wb.window_count >= 0 ? wb.window_count : 1;
+                    for (int64_t w = 0; w < count; ++w) {
+                        values.push_back(wb.src_begin + w);
+                        values.push_back(wb.slot_begin + (count - 1 - w) *
+                                                          (wb.window_count >= 0 ? wb.snapshot_stride : 0));
+                    }
+                    break;
+                }
+            }
+            if (!values.empty()) {
+                const auto name = "rs_windows_" + get_tensor_ov_name(m_cgraph, concat);
+                m_model_extra_inputs[name] = {ov::element::i64, ov::Shape{1, 1, values.size() / 2, 2}, 0, true, {}, -1};
+                m_compute_params.recurrent_controls[name] = std::move(values);
+            }
+        }
+    }
+#endif
+    if (!m_is_static && !m_compute_params.sampling_rows.empty()) {
+        m_model_extra_inputs["sampling_rows"] =
+            {ov::element::i64, ov::Shape{m_compute_params.sampling_rows.size()}, 0, true, {}, -1};
+    } else {
+        for (const auto & [node_name, row] : m_compute_params.sampling_rows) {
+            create_1d_input("sampling_row_" + node_name, row);
+        }
+    }
+    for (const auto & [node_name, rows] : m_compute_params.gdn_attention_rows) {
+        create_1d_input("gdn_attention_rows_" + node_name, rows);
     }
 }

@@ -1294,6 +1799,14 @@ bool GgmlOvDecoder::node_is_used_as_src(const int node_idx) {
 void GgmlOvDecoder::compute_model_inputs() {
     m_model_inputs.clear();
     m_inputs.clear();
+    m_kvcache_4d.clear();
+    // remember the KV caches that get_graph_input_shape gave the 4-D layout, so that they are bound with that shape
+    auto note_kvcache_4d = [this](const ggml_tensor * input, const ggml_tensor * op, const ov::PartialShape & shape) {
+        if (!m_is_stateful && is_kvcache(input, op) && shape.rank().is_static() && shape.rank().get_length() == 4 &&
+            shape[1].is_dynamic()) {
+            m_kvcache_4d.insert(input);
+        }
+    };
     for (int i = 0; i < m_cgraph->n_nodes; i++) {
         ggml_tensor * node = m_cgraph->nodes[i];
         // the node op is NONE means this node maybe as input of later nodes, we should add it to model inputs for this node.
@@ -1303,6 +1816,7 @@ void GgmlOvDecoder::compute_model_inputs() {
                 m_inputs[node_name] = node;
                 m_model_inputs[node_name] = {get_ov_type(node),
                                              get_graph_input_shape(node, nullptr, m_node_dynamic_dims[node])};
+                note_kvcache_4d(node, nullptr, m_model_inputs[node_name].shape);
             }
             continue;
         }
@@ -1315,6 +1829,13 @@ void GgmlOvDecoder::compute_model_inputs() {
             if (src->flags & GGML_TENSOR_FLAG_INPUT) {
                 src_name = get_tensor_graph_input_ov_name(this, m_cgraph, src, node);
             }
+            // ROPE frequency factors (rope_factors_short/long in Phi-3, Phi-4 and MiniCPM) are registered as
+            // "rope_freqs.weight" in create_weight_nodes(). Without the same name here they become a
+            // Parameter fed from the host weight buffer, which is empty in cache-only mode and after
+            // GGML_OPENVINO_RELEASE_WEIGHTS.
+            if (is_rope_freqs_weight(src->view_src ? src->view_src : src, node)) {
+                src_name = "rope_freqs.weight";
+            }
             if (m_model_weights.find(src_name) != m_model_weights.end()) {
                 continue;
             }
@@ -1356,6 +1877,23 @@ void GgmlOvDecoder::compute_model_inputs() {
             m_inputs[src_name] = src;
             m_model_inputs[src_name] = {get_ov_type(src),
                                         get_graph_input_shape(node, src, m_node_dynamic_dims[src])};
+            note_kvcache_4d(src, node, m_model_inputs[src_name].shape);
+        }
+    }
+    if (!m_naive && !m_is_static && !m_is_stateful && !m_model_is_splitted && ggml_openvino_is_gpu() &&
+        m_compute_params.s_copy_active_slot_len > 0) {
+        const ggml_tensor * rows = nullptr;
+        for (const auto & [name, tensor] : m_inputs) {
+            if (tensor->type == GGML_TYPE_I32 && is_s_copy_leaf(tensor)) {
+                if (rows && rows != tensor) {
+                    return;  // Different recurrent row maps cannot share one active shape.
+                }
+                rows = tensor;
+            }
+        }
+        if (rows && m_compute_params.s_copy_active_slot_len <= ggml_nelements(rows)) {
+            m_inputs["rs_active_rows"] = const_cast<ggml_tensor *>(rows);
+            m_model_inputs["rs_active_rows"] = {ov::element::i32, ov::PartialShape{1, 1, 1, -1}};
         }
     }
 }
@@ -1369,13 +1907,29 @@ void GgmlOvDecoder::compute_model_outputs() {
         if (cur_node->op == GGML_OP_NONE || cur_node->op == GGML_OP_VIEW || cur_node->op == GGML_OP_RESHAPE) {
             continue;
         }
-        if (::is_inplace_op(cur_node) && ggml_nbytes(cur_node) == 0) {
+        const bool runtime_recurrent_write = !m_naive && is_recurrent_cache(cur_node->view_src) &&
+                                              ::is_inplace_op(cur_node) &&
+                                              (is_recurrent_cache_reset(cur_node) || !m_is_static);
+        if (::is_inplace_op(cur_node) && ggml_nbytes(cur_node) == 0 && !runtime_recurrent_write) {
             continue;
         }
+        const bool inactive_top_k = is_sampling_top_k(cur_node) && !(cur_node->flags & GGML_TENSOR_FLAG_COMPUTE);
+        if ((cur_node->flags & GGML_TENSOR_FLAG_OUTPUT) || inactive_top_k) {
+            const auto name = get_tensor_ov_name(m_cgraph, cur_node);
+            m_model_outputs[name] = cur_node;
+            m_model_output_names.insert(name);
+        }
+        if (is_gdn_snapshot_writeback(cur_node)) {
+            auto * cache = cur_node->view_src;
+            const auto name = get_tensor_ov_name(m_cgraph, cache);
+            m_model_outputs[name] = cache;
+            m_model_output_names.insert(name);
+        }
         auto cur_node_use_count = m_cgraph->use_counts[ggml_hash_find(&m_cgraph->visited_hash_set, cur_node)];
         if (cur_node_use_count == 0) {
             // The output of in-place ops is the view_src tensor, which is updated in place. We should use the view_src name as the output name to make sure it can be correctly matched with the later ops that use the view_src.
-            if (cur_node != nullptr && ::is_inplace_op(cur_node) && ggml_nbytes(cur_node) > 0) {
+            if (cur_node != nullptr && ::is_inplace_op(cur_node) &&
+                (ggml_nbytes(cur_node) > 0 || runtime_recurrent_write)) {
                 cur_node = cur_node->view_src;
             }
         } else {
@@ -1415,6 +1969,80 @@ const ggml_tensor * GgmlOvDecoder::get_tensor_used_op(const ggml_tensor * tensor
     return nullptr;
 }

+bool GgmlOvDecoder::is_kvcache_4d(const ggml_tensor * cache) const {
+    static const bool enabled = ggml_openvino_getenv_int("GGML_OPENVINO_ENABLE_KV_4D") > 0;
+    if (!enabled || m_is_static || m_is_stateful || m_model_is_splitted || !ggml_openvino_is_gpu()) {
+        return false;
+    }
+    // `-fa off` (transposed V cache): keep the [1, 1, ctx, n_heads_kv * head_size] layout. The 4-D K cache breaks the
+    // MUL_MAT attention of the Phi-3 models (Phi-3.5 mini, Phi-4 mini), and the soft-capped manual attention of
+    // gemma-2 with `-fa on` ("could not construct a memory descriptor using strides" in the GPU plugin's oneDNN path).
+    if (m_model_params.v_trans || m_model_params.softcap_attn) {
+        return false;
+    }
+    if (cache->op != GGML_OP_NONE || cache->view_src != nullptr || !is_kvcache(cache, nullptr)) {
+        return false;
+    }
+    // one sequence, [n_heads_kv * head_size, ctx, 1, 1]; whisper's flat KV buffer has ctx == 1
+    if (cache->ne[1] <= 1 || cache->ne[2] != 1 || cache->ne[3] != 1) {
+        return false;
+    }
+    const int n_heads_kv = get_n_heads_kv_for_tensor(cache);
+    if (n_heads_kv <= 0 || cache->ne[0] % n_heads_kv != 0) {
+        return false;
+    }
+    // check the nodes that take the cache itself as a source (views of those views, such as the PERMUTE of the read
+    // VIEW, are checked through them)
+    auto feeds_only_permutes = [this](const ggml_tensor * view) {
+        for (int i = 0; i < m_cgraph->n_nodes; i++) {
+            const auto * node = m_cgraph->nodes[i];
+            for (int j = 0; j < GGML_MAX_SRC; j++) {
+                if (node->src[j] == view && node->op != GGML_OP_PERMUTE) {
+                    return false;
+                }
+            }
+        }
+        return true;
+    };
+    for (int i = 0; i < m_cgraph->n_nodes; i++) {
+        const auto * node = m_cgraph->nodes[i];
+        bool uses_cache = false;
+        for (int j = 0; j < GGML_MAX_SRC; j++) {
+            uses_cache = uses_cache || node->src[j] == cache;
+        }
+        if (!uses_cache) {
+            continue;
+        }
+        // write: whole rows into the cache itself
+        if (node->op == GGML_OP_SET_ROWS && node->src[2] == cache && node->src[0]->ne[0] == cache->ne[0]) {
+            continue;
+        }
+        // read: the first rows of the cache, each row split into heads, permuted for attention
+        if (node->op == GGML_OP_VIEW && node->src[0] == cache && node->view_offs == 0 &&
+            node->ne[0] * node->ne[1] == cache->ne[0] && node->nb[2] == cache->nb[1] && node->ne[3] == 1 &&
+            feeds_only_permutes(node)) {
+            continue;
+        }
+        return false;
+    }
+    return true;
+}
+
+bool GgmlOvDecoder::is_kvcache_token_rows(const ggml_tensor * cache) const {
+    for (int i = 0; i < m_cgraph->n_nodes; i++) {
+        const auto * node = m_cgraph->nodes[i];
+        if (node->op != GGML_OP_SET_ROWS) {
+            continue;
+        }
+        // SET_ROWS writes whole rows of src[2], which is the cache or a view of it
+        const auto * dst = node->src[2];
+        if (dst != nullptr && (dst == cache || dst->view_src == cache) && dst->ne[0] != cache->ne[0]) {
+            return false;
+        }
+    }
+    return true;
+}
+
 const ggml_tensor * GgmlOvDecoder::get_tensor_from_name(const std::string & name) const {
     for (int i = 0; i < m_cgraph->n_nodes; i++) {
         const auto * node = m_cgraph->nodes[i];
@@ -1431,6 +2059,16 @@ const ggml_tensor * GgmlOvDecoder::get_tensor_from_name(const std::string & name
     return nullptr;
 }

+std::string GgmlOvDecoder::get_hfa_history_input_name() const {
+    for (const auto & [name, tensor] : m_inputs) {
+        const auto * op = get_tensor_used_op(tensor);
+        if (op != nullptr && is_inp_pos(tensor, op)) {
+            return name;
+        }
+    }
+    throw std::runtime_error("HFA prefill position input was not found");
+}
+
 std::map<std::string, std::string> GgmlOvDecoder::get_kv_param_res_names() const {
     std::map<std::string, std::string> kv_param_res_names;
     for (const auto & name : m_model_params.kv_names) {
@@ -1560,7 +2198,10 @@ std::shared_ptr<ov::Node> GgmlOvDecoder::create_weight_node(ggml_tensor * tensor
     // pointer to avoid re-extracting on every recompile. Opt-in via
     // GGML_OPENVINO_REDUCE_COMPILE_MEM or GGML_OPENVINO_MEMORY_OPTIMIZE. Skip
     // for `naive` (test/naive path) since use_bias changes the produced node.
-    const bool cacheable_nonov = ggml_openvino_reduce_compile_mem_enabled() && !is_ov_buffer &&
+    // Self-contained decode-window models are built after the pair; reusing the node keeps e.g. the
+    // token_embd vocab on the pair's NPUW bank uid instead of embedding and importing a second copy.
+    const bool keep_for_windows = ggml_openvino_npu_raw_weight_cache_enabled() && ggml_openvino_npu_decode_bucket() > 0;
+    const bool cacheable_nonov = (ggml_openvino_reduce_compile_mem_enabled() || keep_for_windows) && !is_ov_buffer &&
                                  !naive && tensor->data != nullptr;
     if (cacheable_nonov) {
         std::lock_guard<std::mutex> lock(g_nonov_weight_cache_mutex);
@@ -1577,7 +2218,7 @@ std::shared_ptr<ov::Node> GgmlOvDecoder::create_weight_node(ggml_tensor * tensor

     // GGML_LOG_DEBUG("%s: creating new weight node for %s\n", __func__, tensor->name);
     static const std::set<ggml_type> weight_types = {GGML_TYPE_F32,  GGML_TYPE_F16,  GGML_TYPE_BF16, GGML_TYPE_Q8_0,
-                                                     GGML_TYPE_Q4_0, GGML_TYPE_Q4_1, GGML_TYPE_Q5_1, GGML_TYPE_Q4_K,
+                                                     GGML_TYPE_Q4_0, GGML_TYPE_Q4_1, GGML_TYPE_Q5_0, GGML_TYPE_Q5_1, GGML_TYPE_Q4_K,
                                                      GGML_TYPE_Q5_K, GGML_TYPE_Q6_K, GGML_TYPE_MXFP4};
     if (weight_types.find(tensor->type) == weight_types.end()) {
         throw std::runtime_error("Unexpected weight tensor type: " + std::string(tensor->name) + " with type " +
@@ -1588,14 +2229,19 @@ std::shared_ptr<ov::Node> GgmlOvDecoder::create_weight_node(ggml_tensor * tensor
     if (ggml_is_quantized(tensor->type)) {
         auto use_bias = naive;
         if (is_ov_buffer) {
-            // For quantized weights, copy raw data to a temp buffer first because
-            // process_weight_tensor reads from data and writes extracted results
-            // (weights/scales/zp) to output_base_ptr — they would overlap if both
-            // point to tensor->data.
-            size_t raw_size = ggml_nbytes(tensor);
-            std::vector<uint8_t> tmp(raw_size);
-            memcpy(tmp.data(), tensor->data, raw_size);
-            ov_weight = process_weight_tensor(tensor, tmp.data(), tensor->data, use_bias);
+            const bool preserve_raw = ggml_openvino_npu_raw_weight_cache_enabled();
+            if (preserve_raw) {
+                // A later context must fingerprint the same raw GGUF bytes before importing its
+                // self-contained blob, so keep the model buffer immutable in this mode.
+                ov_weight = process_weight_tensor(tensor, tensor->data, nullptr, use_bias);
+            } else {
+                // The regular path stores extracted weights in the model buffer. Copy raw data
+                // first because input and output ranges would otherwise overlap.
+                size_t raw_size = ggml_nbytes(tensor);
+                std::vector<uint8_t> tmp(raw_size);
+                memcpy(tmp.data(), tensor->data, raw_size);
+                ov_weight = process_weight_tensor(tensor, tmp.data(), tensor->data, use_bias);
+            }
         } else {
             ov_weight = process_weight_tensor(tensor, tensor->data, nullptr, use_bias);
         }
@@ -1670,6 +2316,9 @@ void GgmlOvDecoder::dump_cgraph(const ggml_cgraph * cgraph, std::string & filena
              << std::setw(5) << node->nb[2] << ", "
              << std::setw(5) << node->nb[3] << "] "
              << std::right << std::setw(15) << buf_name << std::right
+             << " compute=" << ((node->flags & GGML_TENSOR_FLAG_COMPUTE) != 0)
+             << " output=" << ((node->flags & GGML_TENSOR_FLAG_OUTPUT) != 0)
+             << " view_offs=" << node->view_offs
              << "\n";

         for (int i = 0; i < GGML_MAX_SRC; i++) {
@@ -1977,9 +2626,22 @@ std::vector<size_t> GgmlOvDecoder::get_output_stride(int node_idx) const {
     return get_stride(ggml_tensor);
 }

+bool GgmlOvDecoder::is_direct_gdn_view(const ggml_tensor * node) const {
+    if (node == nullptr || node->op != GGML_OP_VIEW || node->src[0] == nullptr ||
+        node->src[0]->op != GGML_OP_GATED_DELTA_NET) {
+        return false;
+    }
+    for (int i = 0; i < m_cgraph->n_nodes; ++i) {
+        if (m_cgraph->nodes[i] == node->src[0]) {
+            return get_output_names(i).size() > 1;
+        }
+    }
+    return false;
+}
+
 std::vector<std::string> GgmlOvDecoder::get_output_names(int node_idx) const {
     auto * node = m_node_info_list[node_idx].node;
-    if (node->op == GGML_OP_GATED_DELTA_NET && !m_model_params.has_rs_rollback) {
+    if (node->op == GGML_OP_GATED_DELTA_NET && (!m_model_params.has_rs_rollback || node->op_params[0] > 1)) {
         std::string attn_name;
         std::string state_name;
         for (int i = node_idx + 1; i < m_cgraph->n_nodes; i++) {
@@ -1996,6 +2658,9 @@ std::vector<std::string> GgmlOvDecoder::get_output_names(int node_idx) const {
             name = get_tensor_ov_name(m_cgraph, consumer);
         }
         if (!attn_name.empty() && !state_name.empty()) {
+            if (m_model_params.has_rs_rollback) {
+                return {m_node_info_list[node_idx].node_name, attn_name, state_name};
+            }
             return {attn_name, state_name};
         }
     }
@@ -2004,7 +2669,12 @@ std::vector<std::string> GgmlOvDecoder::get_output_names(int node_idx) const {

 std::string GgmlOvDecoder::get_inplace_op_src(int node_idx) const {
     auto * node = m_node_info_list[node_idx].node;
-    if (!::is_inplace_op(node) || node->view_src == nullptr || ggml_nbytes(node) == 0) {
+    // Reset lengths (also in static mode) and dynamic remainder views can become nonempty
+    // after compilation. Keep their alias updates in the translated cache chain.
+    const bool runtime_recurrent_write = !m_naive && is_recurrent_cache(node->view_src) &&
+                                          (is_recurrent_cache_reset(node) || !m_is_static);
+    if (!::is_inplace_op(node) || node->view_src == nullptr ||
+        (ggml_nbytes(node) == 0 && !runtime_recurrent_write)) {
         return "";
     }
     const int op_case = m_node_info_list[node_idx].node_op_case;
@@ -2104,7 +2774,7 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
             //     root_src = src->org_src;
             // }
             if (root_src) {
-                if (is_inp_tok(root_src, node) || is_inp_pos(root_src, node) || is_output_idx(root_src, node)) {
+                if (is_inp_tok(root_src, node) || is_inp_pos(root_src, node) || is_get_rows_input_idx(root_src, node)) {
                     m_node_dynamic_dims[root_src] = 0;
                     m_node_dynamic_dims[src] = m_node_dynamic_dims[root_src];
                     continue;
@@ -2112,7 +2782,7 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
                 self(self, root_src);
                 m_node_dynamic_dims[src] = m_node_dynamic_dims[root_src];
             } else {
-                if (is_inp_tok(src, node) || is_inp_pos(src, node) || is_output_idx(src, node)) {
+                if (is_inp_tok(src, node) || is_inp_pos(src, node) || is_get_rows_input_idx(src, node)) {
                     m_node_dynamic_dims[src] = 0;
                     continue;
                 }
@@ -2120,7 +2790,13 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
                     m_node_dynamic_dims[src] = 1;
                     continue;
                 }
-                if (node->op == GGML_OP_VIEW && src->op == GGML_OP_NONE && !is_stateful() && !m_model_is_splitted) {
+                if (!m_is_static && is_activation_input(src, node)) {
+                    m_node_dynamic_dims[src] = 1;
+                    continue;
+                }
+                // a view of a model weight (BERT's token type row) has no token dim
+                if (node->op == GGML_OP_VIEW && src->op == GGML_OP_NONE && !is_stateful() && !m_model_is_splitted &&
+                    !(src->buffer != nullptr && src->buffer->usage == GGML_BACKEND_BUFFER_USAGE_WEIGHTS)) {
                     m_node_dynamic_dims[src] = 1;
                     continue;
                 }
@@ -2137,6 +2813,9 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
                 auto dynamic_dim_idx = m_node_dynamic_dims[node->src[1]];
                 if (dynamic_dim_idx == 0) {
                     m_node_dynamic_dims[node] = 1;
+                } else if (dynamic_dim_idx == 2 && node->ne[3] == node->src[1]->ne[2]) {
+                    // indices [k, ne11, ne12] -> result [ne00, k, ne11, ne12]
+                    m_node_dynamic_dims[node] = 3;
                 } else {
                     auto dynamic_dim_stride = node->src[1]->nb[dynamic_dim_idx] / ggml_type_size(node->src[1]->type) *
                                               ggml_type_size(node->src[0]->type);
@@ -2167,11 +2846,9 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
             if (m_node_dynamic_dims[node->src[0]] != -1) {
                 auto dynamic_dim_idx = m_node_dynamic_dims[node->src[0]];
                 // auto dynamic_dim_value = node->src[0]->ne[dynamic_dim_idx];
-                for (int i = 0; i < GGML_MAX_DIMS; i++) {
-                    if (node->op_params[i] == dynamic_dim_idx) {
-                        m_node_dynamic_dims[node] = i;
-                        break;
-                    }
+                // src dim i moves to dim op_params[i]
+                if (dynamic_dim_idx >= 0 && dynamic_dim_idx < GGML_MAX_DIMS) {
+                    m_node_dynamic_dims[node] = node->op_params[dynamic_dim_idx];
                 }
                 // OPENVINO_ASSERT(dynamic_dim_value == node->ne[m_node_dynamic_dims[node]],
                 //                 "Dynamic dim value mismatch for node: " + std::string(node->name) +
@@ -2179,23 +2856,45 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
             }
             break;
         case GGML_OP_VIEW: {
+            if (is_sampling_view(node)) {
+                m_node_dynamic_dims[node] = -1;
+                break;
+            }
+
+            if (node->src[0]->op == GGML_OP_GATED_DELTA_NET && node->view_offs == 0) {
+                m_node_dynamic_dims[node] = 2;
+                break;
+            }
             // Use stride-based matching: the stride of a VIEW dimension directly
             // encodes which source dimension it indexes into, so it uniquely
             // identifies the dynamic dim even when two dims share the same size.
             m_node_dynamic_dims[node] = -1;
             if (m_node_dynamic_dims[node->src[0]] != -1) {
-                if (node->src[0]->op == GGML_OP_NONE) {
-                    m_node_dynamic_dims[node] = m_node_dynamic_dims[node->src[0]];
-                    break;
-                }
                 auto dynamic_dim_idx = m_node_dynamic_dims[node->src[0]];
                 auto dynamic_dim_value = node->src[0]->ne[dynamic_dim_idx];
                 auto dynamic_dim_stride =
                     node->src[0]->nb[dynamic_dim_idx] / ggml_type_size(node->src[0]->type) * ggml_type_size(node->type);
+                if (node->src[0]->op == GGML_OP_NONE) {
+                    // A view of a KV cache covers fewer cells than the cache holds, so only the stride identifies the
+                    // dim: [n_embd, kv_size] viewed as [head_dim, n_head_kv, n_kv] moves the cells from dim 1 to 2.
+                    m_node_dynamic_dims[node] = dynamic_dim_idx;
+                    for (int i = 0; i < GGML_MAX_DIMS; i++) {
+                        if (node->nb[i] == dynamic_dim_stride && node->ne[i] != 1) {
+                            m_node_dynamic_dims[node] = i;
+                            break;
+                        }
+                    }
+                    break;
+                }
+                // A size-1 dim may carry any stride (MLA's k_pe = ggml_view_3d(.., ne1 = 1, ..) gives it the row
+                // stride), so among the dims with the source's stride take the outermost one of the same size.
+                // With a dynamic size of 1 every trailing size-1 dim matches (Qwen3.5's Qcur view: ne2 = n_tokens = 1,
+                // nb3 == nb2), so keep the innermost match there.
                 for (int i = 0; i < GGML_MAX_DIMS; i++) {
-                    if (node->nb[i] == dynamic_dim_stride) {
+                    const bool first_match = m_node_dynamic_dims[node] == -1;
+                    const bool same_size = dynamic_dim_value != 1 && node->ne[i] == dynamic_dim_value;
+                    if (node->nb[i] == dynamic_dim_stride && (first_match || same_size)) {
                         m_node_dynamic_dims[node] = i;
-                        break;
                     }
                 }
                 if (m_node_dynamic_dims[node] != -1 && dynamic_dim_value != node->ne[m_node_dynamic_dims[node]]) {
@@ -2219,14 +2918,23 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
             // have standard compact strides: nb[i] = type_size * prod(ne[0..i-1]).
             // Match src->nb[dynamic_dim] against result->nb[i] to find the output
             // dimension whose flat-memory boundary aligns with the source dynamic
-            // boundary. This is unambiguous (result strides are strictly monotone)
-            // and handles merged-lower-dim cases that ne-value matching misses.
+            // boundary. This handles merged-lower-dim cases that ne-value matching
+            // misses. A size-1 dim has the same stride as the dim after it
+            // ([head_dim, T] -> [head_dim, 1, T] with one KV head), so prefer the
+            // matching dim that also has the source dim's size.
             m_node_dynamic_dims[node] = -1;
             if (m_node_dynamic_dims[node->src[0]] != -1) {
                 auto dynamic_dim_idx = m_node_dynamic_dims[node->src[0]];
+                auto dynamic_dim_value = node->src[0]->ne[dynamic_dim_idx];
                 auto dynamic_dim_stride = node->src[0]->nb[dynamic_dim_idx];
                 for (int i = 0; i < GGML_MAX_DIMS; i++) {
-                    if (node->nb[i] == dynamic_dim_stride && node->ne[i] == node->src[0]->ne[dynamic_dim_idx]) {
+                    if (node->nb[i] != dynamic_dim_stride) {
+                        continue;
+                    }
+                    if (m_node_dynamic_dims[node] == -1) {
+                        m_node_dynamic_dims[node] = i;
+                    }
+                    if (node->ne[i] == dynamic_dim_value) {
                         m_node_dynamic_dims[node] = i;
                         break;
                     }
@@ -2299,14 +3007,25 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
                 }
             }
             break;
-        case GGML_OP_CONCAT:
-            for (int i = 0; i < GGML_MAX_DIMS; i++) {
-                if (node->src[0]->ne[i] != node->ne[i]) {
-                    m_node_dynamic_dims[node] = i;
+        case GGML_OP_CONCAT: {
+            const int axis = node->op_params[0];
+            const int src0_dynamic_dim = m_node_dynamic_dims[node->src[0]];
+            const int src1_dynamic_dim = m_node_dynamic_dims[node->src[1]];
+            if (src0_dynamic_dim != -1 && src0_dynamic_dim != axis &&
+                src1_dynamic_dim != -1 && src1_dynamic_dim != axis && src0_dynamic_dim != src1_dynamic_dim) {
+                GGML_LOG_WARN("ggml-openvino: different dynamic dims for CONCAT node '%s', src[0]: '%s', src[1]: '%s'\n",
+                              node->name, node->src[0]->name, node->src[1]->name);
+            }
+            m_node_dynamic_dims[node] = axis;
+            for (int i = 0; i < 2; ++i) {
+                const int dynamic_axis = m_node_dynamic_dims[node->src[i]];
+                if (dynamic_axis != -1 && dynamic_axis != axis) {
+                    m_node_dynamic_dims[node] = dynamic_axis;
                     break;
                 }
             }
             break;
+        }
         case GGML_OP_SSM_CONV:
         case GGML_OP_GATED_DELTA_NET:
             m_node_dynamic_dims[node] = 1;
@@ -2321,6 +3040,7 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
         case GGML_OP_SCALE:
         case GGML_OP_SOFT_MAX:
         case GGML_OP_ARGSORT:
+        case GGML_OP_TOP_K:
         case GGML_OP_ADD_ID:
         case GGML_OP_UNARY:
         case GGML_OP_CUMSUM:
@@ -2347,8 +3067,9 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
             m_node_dynamic_dims[node] = m_node_dynamic_dims[node->src[0]];
             break;
         case GGML_OP_SUM_ROWS:
-            // SUM_ROWS reduces ggml axis 0 to size 1 and preserves all other axes, so the
-            // dynamic dim is preserved unless it was axis 0 (then it is summed away).
+        case GGML_OP_POOL_1D:
+            // SUM_ROWS/POOL_1D change only ggml axis 0 and preserve all other axes, so the
+            // dynamic dim is preserved unless it was axis 0.
             m_node_dynamic_dims[node] =
                 (m_node_dynamic_dims[node->src[0]] == 0) ? -1 : m_node_dynamic_dims[node->src[0]];
             break;
@@ -2357,11 +3078,15 @@ void GgmlOvDecoder::compute_node_dynamic_dims() {
             m_node_dynamic_dims[node] = m_node_dynamic_dims[node->src[1]];
             break;
         case GGML_OP_CPY:
-        case GGML_OP_SET_ROWS:
         case GGML_OP_SUM:
         case GGML_OP_MEAN:
             m_node_dynamic_dims[node] = -1;
             break;
+        case GGML_OP_SET_ROWS:
+            // a cache write returns the cache; other rows writes keep the shape of their destination
+            m_node_dynamic_dims[node] = (is_kvcache_write(node) || is_conv_state_snapshot_writeback(node)) ?
+                                           -1 : m_node_dynamic_dims[node->src[2]];
+            break;
         case GGML_OP_IM2COL: {
             m_node_dynamic_dims[node] = -1;
             if (m_node_dynamic_dims[node->src[1]] != -1) {
diff --git a/ggml/src/ggml-openvino/ggml-decoder.h b/ggml/src/ggml-openvino/ggml-decoder.h
index a614d291a..b7ff4b715 100644
--- a/ggml/src/ggml-openvino/ggml-decoder.h
+++ b/ggml/src/ggml-openvino/ggml-decoder.h
@@ -10,9 +10,11 @@
 #include <map>
 #include <memory>
 #include <openvino/core/partial_shape.hpp>
+#include <openvino/runtime/tensor.hpp>
 #include <optional>
 #include <set>
 #include <string>
+#include <utility>
 #include <vector>

 struct ModelParams {
@@ -33,11 +35,18 @@ struct ModelParams {
     bool has_rs_rollback = false;
     bool mixed_rope_params = false;
     bool is_cacheless_attn = false;
+    // the graph writes a transposed V cache (`-fa off`: one value element per row, tokens along the row)
+    bool v_trans = false;
+    // the graph has FLASH_ATTN_EXT with logit soft-capping (gemma-2)
+    bool softcap_attn = false;
     std::vector<int> swa_layers;
     // The sliding-window mask tensor, identified in compute_llm_params() by grouping attention
     // layers on the mask they consume. Only used to tell the two masks apart when naming OV
     // parameters -- both carry the same tensor name. Null when the graph has a single mask.
     const ggml_tensor * swa_mask = nullptr;
+    // The attention masks that are graph inputs, by identity: gemma-4's KV-shared layers also read them through
+    // GET_ROWS, which must resolve to the same OV parameter as the attention ops
+    std::set<const ggml_tensor *> kq_masks;

     std::vector<std::string> kv_names;
     size_t kv_buffer_ctx_id = 0;
@@ -47,14 +56,15 @@ struct ModelParams {
                memcmp(rope_params, other.rope_params, sizeof(int32_t) * 16) == 0;
     }

+    // swa_layers: the masks and KV write indices of these layers are separate graph inputs
     bool can_reuse_dynamically(const ModelParams & other) const {
-        return same_rope_params(other) && n_rs_slots == other.n_rs_slots &&
-               has_rs_rollback == other.has_rs_rollback;
+        return same_rope_params(other) && n_rs_slots == other.n_rs_slots && has_rs_rollback == other.has_rs_rollback &&
+               swa_layers == other.swa_layers;
     }

     bool can_reuse_statically(const ModelParams & other) const {
         return same_rope_params(other) && ctx == other.ctx && n_rs_slots == other.n_rs_slots &&
-               has_rs_rollback == other.has_rs_rollback;
+               has_rs_rollback == other.has_rs_rollback && swa_layers == other.swa_layers;
     }

     bool kv_buffer_changed(const ModelParams & other) const { return kv_buffer_ctx_id != other.kv_buffer_ctx_id; }
@@ -66,6 +76,9 @@ struct ComputeParams {
     int attention_size = -1;
     int attention_size_swa = -1;
     int attention_size_static = -1;  // encoder/cross-attn KV fill level (whisper)
+    // OV names of the KQ mask inputs that n_seq_active (mask->ne[3]) and attention_size(_swa) (mask->ne[0]) were read from
+    std::string mask_name;
+    std::string mask_swa_name;
     // Sliding window width, read back from the band of ggml's own SWA mask. ggml never passes
     // n_swa down to a backend, but fill_mask() bakes it into the mask contents, so the widest
     // unmasked row recovers it. Shorter than n_swa while the sequence is still short, which is
@@ -106,17 +119,19 @@ struct ComputeParams {
     //      [ 18432,     0,     1,     1]            0: GET_ROWS    node_9
     //      [ 18432,     0,     1,     1]            1: VIEW        cache_r_l0 (view)

+    // Runtime writeback offsets are refreshed per batch. Convolution SET_ROWS also supplies overlapping window counts and snapshot plane strides.
     struct RsWriteback {
-        int slot_begin = 0;  // first cache slot written by the CPY
-        int src_begin = -1;  // first source column copied by a conv-state CPY
+        int64_t slot_begin = 0;  // first cache slot written
+        int64_t src_begin = -1;  // first source column of a conv-state window
+        int64_t seq_count = -1;
+        int64_t snapshot_stride = -1;
+        int64_t window_count = -1;
     };

     std::map<std::string, RsWriteback> rs_writebacks;
-    // Destination slot offset of each state cache writeback CPY node, keyed by node name. It
-    // changes with the batch (kv head, active sequence count) and, with rollback enabled
-    // (cparams.n_rs_seq > 0), the conv state is written back once per snapshot slot. Passed to the
-    // cached model as a runtime input. Dynamic models also receive the source-side offset; static
-    // models use a fixed end-anchored offset in the translator.
+    std::map<std::string, int64_t> sampling_rows;
+    std::map<std::string, int64_t> gdn_attention_rows;
+    std::map<std::string, std::vector<int64_t>> recurrent_controls;
 };

 // defined below; declared here because GgmlOvDecoder uses it inline
@@ -139,6 +154,8 @@ public:
         void * data_addr;
     };

+    const ggml_cgraph * get_cgraph() const { return m_cgraph; }
+
     // Graph decoder
     GgmlOvDecoder(ggml_cgraph * cgraph,
                   ModelParams & model_params,
@@ -247,7 +264,13 @@ public:
     virtual void visit_subgraph(
         std::function<void(std::shared_ptr<GgmlDecoder>, int node_idx)> node_visitor) const override;

-    ggml_tensor * get_input_ggml_tensor(const std::string & name) const { return m_inputs.at(name); }
+    ggml_tensor * get_input_ggml_tensor(const std::string & name) const {
+        auto it = m_inputs.find(name);
+        if (it == m_inputs.end()) {
+            throw std::runtime_error("OpenVINO ggml input not found: " + name);
+        }
+        return it->second;
+    }

     virtual int get_op_case(int node_idx) const override { return m_node_info_list[node_idx].node_op_case; }

@@ -311,6 +334,8 @@ public:

     virtual bool is_stateful() const override { return m_is_stateful; }

+    virtual bool is_prefill() const override { return m_is_prefill; }
+
     int get_static_n_tokens() const { return m_is_prefill ? m_prefill_chunk_size : 1; }

     virtual bool is_splited_model() const override { return m_model_is_splitted; }
@@ -333,20 +358,90 @@ public:

     const ggml_tensor * get_tensor_used_op(const ggml_tensor * tensor) const;

+    // True if the KV cache is a model input of shape [1, ctx, n_heads_kv, head_size] instead of
+    // [1, 1, ctx, n_heads_kv * head_size]. Used for stateless GPU graphs when the cache holds one sequence, one token
+    // per row, and is only written by SET_ROWS and read through views that split the rows into heads. The GPU
+    // plugin can then fuse the cache write and read into StatelessKV: the new rows are written in place and attention
+    // reads the cache without a copy.
+    bool is_kvcache_4d(const ggml_tensor * cache) const;
+
+    // True if the KV cache is an input of this graph in the 4-D layout (see is_kvcache_4d)
+    bool is_kvcache_4d_input(const ggml_tensor * cache) const { return m_kvcache_4d.count(cache) != 0; }
+
+    // True if each row of the KV cache holds one token. With `-fa off`, llama.cpp keeps the V cache transposed
+    // and writes it one element per row, so its first rows do not hold the first tokens.
+    bool is_kvcache_token_rows(const ggml_tensor * cache) const;
+
+    // Reuse the KV cache ov::Tensor while pointer, type and shape stay the same (a new USM tensor costs microseconds).
+    template <typename Make>
+    ov::Tensor cached_kv_tensor(const ggml_tensor * kv,
+                                const ov::element::Type & type,
+                                const ov::Shape & shape,
+                                Make && make) const {
+        auto & entry = m_kv_tensor_cache[kv];
+        if (!entry.tensor || entry.data != kv->data || entry.type != type || entry.shape != shape) {
+            entry = {kv->data, type, shape, make(), next_tensor_id()};
+        }
+        m_last_kv_tensor_id = entry.id;
+        return entry.tensor;
+    }
+
+    // Id of the tensor returned by the last cached_kv_tensor() call, 0 if there was no call since the last read.
+    uint64_t take_last_kv_tensor_id() const { return std::exchange(m_last_kv_tensor_id, 0); }
+
+    // Ids are never 0 and never reused: the same id means the same ov::Tensor.
+    uint64_t next_tensor_id() const { return ++m_tensor_id; }
+
+    // What one infer request port holds between inferences (dynamic GPU path).
+    struct ov_port_binding {
+        ov::Tensor staging;  // USM host tensor used in place of a small host tensor
+        void * staging_ptr = nullptr;
+        size_t staging_bytes = 0;
+        uint64_t staging_id = 0;
+        uint64_t bound_id = 0;  // id of the tensor bound to the port, 0 if not known
+        ov::Shape bound_shape;
+        ov::element::Type bound_type;
+    };
+
+    // Port state of `request`, reset when the decoder runs with another request.
+    std::vector<ov_port_binding> & port_bindings(bool input, size_t n, const void * request) const {
+        if (m_bindings_request != request) {
+            m_input_bindings.clear();
+            m_output_bindings.clear();
+            m_bindings_request = request;
+        }
+        auto & v = input ? m_input_bindings : m_output_bindings;
+        if (v.size() < n) {
+            v.resize(n);
+        }
+        return v;
+    }
+
     const ggml_tensor * get_tensor_from_name(const std::string & name) const;

+    std::string get_hfa_history_input_name() const;
+
     void clear_model_weights() { m_model_weights.clear(); }

     static std::pair<ModelParams, ComputeParams> compute_llm_params(ggml_cgraph * cgraph, bool is_static);

-    ModelParams get_model_params() const { return m_model_params; }
+    const ModelParams & get_model_params() const { return m_model_params; }

-    ComputeParams get_compute_params() const { return m_compute_params; }
+    const ComputeParams & get_compute_params() const { return m_compute_params; }

     void set_model_params(const ModelParams & model_params) { m_model_params = model_params; }

     void set_compute_params(const ComputeParams & compute_params) { m_compute_params = compute_params; }

+    // Per-request storage: scalar runtime controls are host-produced and shape inference
+    // may read them. USM host allocations let the GPU consume them without tiny uploads.
+    std::map<std::string, ov::Tensor> runtime_input_storage;
+    std::map<std::string, ov::Tensor> runtime_input_tensors;
+    std::map<std::string, ov::Tensor> runtime_output_scratch;
+
+    // OpenVINO device that runs the graph, e.g. GPU.1
+    std::string device;
+
     bool m_is_static = false;
     bool m_is_stateful = false;
     bool m_is_prefill = false;
@@ -382,6 +477,15 @@ public:
         return tensor->op == GGML_OP_GET_ROWS && op->op == GGML_OP_RMS_NORM;
     }

+    static bool is_activation_input(const ggml_tensor * tensor, const ggml_tensor * op) {
+        if (tensor == nullptr || op == nullptr || tensor->type != GGML_TYPE_F32 || tensor->ne[2] != 1 ||
+            tensor->ne[3] != 1 || (tensor->buffer != nullptr && tensor->buffer->usage == GGML_BACKEND_BUFFER_USAGE_WEIGHTS)) {
+            return false;
+        }
+        return (op->op == GGML_OP_RMS_NORM || op->op == GGML_OP_L2_NORM) && op->src[0] == tensor &&
+               (tensor->op == GGML_OP_NONE || tensor->op == GGML_OP_CPY);
+    }
+
     static bool is_inp_mask(const ggml_tensor * tensor, const ggml_tensor * op) {
         return op->op == GGML_OP_CPY || (op->op == GGML_OP_FLASH_ATTN_EXT && tensor == op->src[3]) ||
                (op->op == GGML_OP_SOFT_MAX && tensor == op->src[1]);
@@ -402,12 +506,78 @@ public:
         return op->op == GGML_OP_ROPE && tensor == op->src[2];
     }

-    inline static bool is_recurrent_cache(const ggml_tensor * tensor) {
+    static bool is_recurrent_cache(const ggml_tensor * tensor) {
         return tensor != nullptr && (strncmp(tensor->name, "cache_r_l", strlen("cache_r_l")) == 0 ||
                                      strncmp(tensor->name, "cache_s_l", strlen("cache_s_l")) == 0 ||
                                      strncmp(tensor->name, "cache_ple_r_l", strlen("cache_ple_r_l")) == 0);
     }

+    static bool is_recurrent_cache_reset(const ggml_tensor * node) {
+        const bool is_scale_or_fill = node->op == GGML_OP_SCALE || node->op == GGML_OP_FILL;
+        return is_scale_or_fill && node->view_src != nullptr && is_recurrent_cache(node->view_src);
+    }
+
+    static bool is_gdn_snapshot_writeback(const ggml_tensor * node) {
+        if (!(node != nullptr && node->op == GGML_OP_CPY && node->src[0] != nullptr &&
+               node->src[0]->op == GGML_OP_VIEW && node->src[0]->src[0] != nullptr &&
+               node->src[0]->src[0]->op == GGML_OP_GATED_DELTA_NET && node->src[0]->src[0]->op_params[0] > 1 &&
+               node->src[1] != nullptr && node->src[1]->op == GGML_OP_VIEW && node->view_src != nullptr &&
+               node->src[1]->view_src == node->view_src && node->src[0]->ne[0] == node->src[1]->ne[0] &&
+               node->src[1]->nb[1] > 0 && node->src[1]->nb[1] == ggml_row_size(node->type, node->ne[0]) &&
+               node->src[1]->nb[2] % node->src[1]->nb[1] == 0 &&
+               node->src[1]->view_offs % node->src[1]->nb[1] == 0)) {
+            return false;
+        }
+        const auto * source = node->src[0];
+        const auto * destination = node->src[1];
+        const auto * packed = source->src[0];
+        if (node->type != GGML_TYPE_F32 || source->type != node->type || destination->type != node->type ||
+            node->view_src->type != node->type || !ggml_is_contiguous(node->view_src) ||
+            packed->nb[1] == 0 || source->view_offs % packed->nb[1] != 0 ||
+            source->nb[0] != sizeof(float) || destination->nb[0] != sizeof(float) ||
+            source->ne[3] != 1 || destination->ne[3] != 1) {
+            return false;
+        }
+        for (int i = 0; i < GGML_MAX_DIMS; ++i) {
+            if (source->ne[i] <= 0 || source->ne[i] != destination->ne[i]) {
+                return false;
+            }
+        }
+        const size_t plane_stride = destination->nb[2] / destination->nb[1];
+        if (plane_stride == 0 || size_t(destination->ne[1]) > plane_stride -
+            (destination->view_offs / destination->nb[1]) % plane_stride) {
+            return false;
+        }
+        for (const auto * view : {source, destination}) {
+            size_t remaining = ggml_nbytes(view == source ? packed : node->view_src);
+            if (view->view_offs > remaining || remaining - view->view_offs < sizeof(float)) {
+                return false;
+            }
+            remaining -= view->view_offs + sizeof(float);
+            for (int i = 0; i < GGML_MAX_DIMS; ++i) {
+                const size_t count = view->ne[i] - 1;
+                if (view->nb[i] != 0 && count > remaining / view->nb[i]) {
+                    return false;
+                }
+                remaining -= count * view->nb[i];
+            }
+        }
+        return true;
+    }
+
+    static bool is_sampling_view(const ggml_tensor * node) {
+        return node != nullptr && node->op == GGML_OP_VIEW && node->src[0] != nullptr &&
+               node->src[0]->op == GGML_OP_PAD && node->ne[0] == node->src[0]->ne[0] &&
+               node->ne[1] == 1 && node->ne[2] == 1 && node->ne[3] == 1;
+    }
+
+    static bool is_sampling_top_k(const ggml_tensor * node) {
+        return node != nullptr && node->op == GGML_OP_TOP_K && node->src[0] != nullptr &&
+               node->src[0]->op == GGML_OP_RESHAPE && is_sampling_view(node->src[0]->src[0]);
+    }
+
+    bool is_direct_gdn_view(const ggml_tensor * node) const;
+
     inline static bool is_cache(const ggml_tensor * tensor, const ggml_tensor * op) {
         return is_recurrent_cache(tensor) || is_kvcache(tensor, op);
     }
@@ -427,6 +597,25 @@ public:
                node->src[1]->op == GGML_OP_VIEW && node->src[1]->view_src == node->view_src;
     }

+    // build_conv_state_store writes overlapping [history, window, channel, seq] windows into newest-first rollback planes.
+    // Match this layout separately from general SET_ROWS.
+    static bool is_conv_state_snapshot_writeback(const ggml_tensor * node);
+
+    // SET_ROWS into a KV cache, directly or through the [1, n] reshape of the transposed V cache with `-fa off`.
+    // Other SET_ROWS write rows of a compute tensor, e.g. the MoE expert-group mask
+    inline static bool is_kvcache_write(const ggml_tensor * node) {
+        const ggml_tensor * dst = node->src[2];
+        return dst != nullptr && is_kvcache(dst->view_src != nullptr ? dst->view_src : dst, nullptr);
+    }
+
+    // GET_ROWS batched over ggml dims 2 and 3, e.g. the MoE expert-group scores
+    // ggml_get_rows(ggml_reshape_4d(groups, 1, n_exp_per_group, n_groups, n_tokens), top_groups)
+    inline static bool is_get_rows_batched_4d(const ggml_tensor * node) {
+        return node->op == GGML_OP_GET_ROWS && !ggml_is_quantized(node->src[0]->type) &&
+               node->src[1]->ne[1] == node->src[0]->ne[2] && node->src[1]->ne[2] == node->src[0]->ne[3] &&
+               node->src[1]->ne[3] == 1;
+    }
+
     static bool is_kv_idx(const ggml_tensor * tensor, const ggml_tensor * op) {
         return op->op == GGML_OP_SET_ROWS && op->src[1] == tensor;
     }
@@ -435,11 +624,27 @@ public:
         return m_model_params.swa_mask != nullptr && tensor == m_model_params.swa_mask;
     }

-    static bool is_output_idx(const ggml_tensor * tensor, const ggml_tensor * op) {
+    bool is_kq_mask(const ggml_tensor * tensor) const { return m_model_params.kq_masks.count(tensor) != 0; }
+
+    // A graph-input index list of GET_ROWS: out_ids, or the inp->s_copy leaf when build_rs gathers
+    // every recurrent state at once.
+    static bool is_get_rows_input_idx(const ggml_tensor * tensor, const ggml_tensor * op) {
         return op->op == GGML_OP_GET_ROWS && tensor == op->src[1] && op->src[0]->op != GGML_OP_NONE &&
                op->src[1]->op == GGML_OP_NONE;
     }

+    // out_ids, which selects the output rows. The recurrent state gather reads a state cache instead.
+    static bool is_output_idx(const ggml_tensor * tensor, const ggml_tensor * op) {
+        if (!is_get_rows_input_idx(tensor, op)) {
+            return false;
+        }
+        const ggml_tensor * data = op->src[0];
+        while (data != nullptr && (data->op == GGML_OP_VIEW || data->op == GGML_OP_RESHAPE)) {
+            data = data->src[0];
+        }
+        return !is_recurrent_cache(data);
+    }
+
     // the state permutation index input used in SSM/DeltaNet models (inp->s_copy in llama-graph.cpp)
     static bool is_inp_s_copy(const ggml_tensor * tensor, const ggml_tensor * op) {
         return op->op == GGML_OP_GET_ROWS && tensor == op->src[1] &&
@@ -476,6 +681,7 @@ public:
 private:
     void set_input_output();
     int compute_op_case(const ggml_tensor * node) const;
+    bool is_gdn_out_prescaled(const ggml_tensor * gdn) const;
     bool node_is_used_as_src(const int node_idx);
     void compute_model_inputs();
     void compute_model_outputs();
@@ -499,6 +705,22 @@ private:
     std::set<std::string> m_model_output_names;
     std::vector<NodeInfo> m_node_info_list;
     std::map<ggml_tensor *, int> m_node_dynamic_dims;
+    std::set<const ggml_tensor *> m_kvcache_4d;
+
+    struct kv_tensor_entry {
+        const void * data = nullptr;
+        ov::element::Type type;
+        ov::Shape shape;
+        ov::Tensor tensor;
+        uint64_t id = 0;
+    };
+
+    mutable std::map<const ggml_tensor *, kv_tensor_entry> m_kv_tensor_cache;
+    mutable uint64_t m_tensor_id = 0;
+    mutable uint64_t m_last_kv_tensor_id = 0;
+    mutable std::vector<ov_port_binding> m_input_bindings;
+    mutable std::vector<ov_port_binding> m_output_bindings;
+    mutable const void * m_bindings_request = nullptr;

     ModelParams m_model_params;
     ComputeParams m_compute_params;
diff --git a/ggml/src/ggml-openvino/ggml-openvino-extra.cpp b/ggml/src/ggml-openvino/ggml-openvino-extra.cpp
index 14a0b0382..c76b42398 100644
--- a/ggml/src/ggml-openvino/ggml-openvino-extra.cpp
+++ b/ggml/src/ggml-openvino/ggml-openvino-extra.cpp
@@ -1,17 +1,30 @@
 #include "ggml-openvino-extra.h"

+#include "device/npu_config.h"
+#include "device/npu_quant_policy.h"
 #include "ggml-impl.h"
 #include "ggml.h"
 #include "model-cache.h"

 #include <algorithm>
+#include <cctype>
+#include <cerrno>
+#include <climits>
 #include <cstdlib>
 #include <cstring>
+#include <map>
+#include <mutex>
 #include <openvino/runtime/intel_gpu/ocl/ocl.hpp>
 #include <openvino/runtime/intel_npu/level_zero/level_zero.hpp>
 #include <openvino/runtime/properties.hpp>
-#include <mutex>
 #include <optional>
+#include <sstream>
+
+#if defined(__APPLE__)
+#    include <crt_externs.h>
+#elif !defined(_WIN32)
+extern char ** environ;
+#endif

 ov::Core & ov_singleton_core() {
     static ov::Core core;
@@ -54,7 +67,8 @@ std::string ggml_openvino_get_device_description(const std::string & device_name

     if (has_prefix(device_name, "NPU")) {
         try {
-            const std::string arch = ov_singleton_core().get_property(device_name, "DEVICE_ARCHITECTURE").as<std::string>();
+            const std::string arch =
+                ov_singleton_core().get_property(device_name, "DEVICE_ARCHITECTURE").as<std::string>();
             if (!arch.empty()) {
                 description += " (NPU " + arch + ")";
             }
@@ -81,13 +95,212 @@ static std::string resolve_openvino_device_name(const std::vector<std::string> &
         for (const std::string & name : available_devices) {
             list += "\n  " + name + ": " + ggml_openvino_get_device_description(name);
         }
-        GGML_ABORT("GGML OpenVINO Backend: GGML_OPENVINO_DEVICE=%s is not available. "
-                   "Set it to one of the available OpenVINO devices:%s",
-                   requested, list.c_str());
+        GGML_ABORT(
+            "GGML OpenVINO Backend: GGML_OPENVINO_DEVICE=%s is not available. "
+            "Set it to one of the available OpenVINO devices:%s",
+            requested, list.c_str());
     }
     return requested;
 }

+namespace {
+
+enum class env_type {
+    STR,    // any text: a device, a path, plugin properties
+    ENUM,   // one of env_var::values
+    INT,    // a whole number
+    FLOAT,  // a number
+    BOOL,   // a whole number: positive is on, zero or negative is off
+};
+
+struct env_var {
+    const char * name;
+    env_type type;
+    const char * values = nullptr;  // ENUM only: the accepted values, separated by '|'
+};
+
+// All recognized GGML_OPENVINO_* env vars. Their values are checked and cached once at backend init time and read
+// back via ggml_openvino_getenv_str() (raw string) or ggml_openvino_getenv_int() (integer / boolean toggle).
+// A value that does not fit the type stops the backend; another GGML_OPENVINO_* name in the environment is reported.
+constexpr env_var env_vars[] = {
+    // Text
+    {"GGML_OPENVINO_DEVICE", env_type::STR},
+    {"GGML_OPENVINO_CACHE_DIR", env_type::STR},
+    {"GGML_OPENVINO_SPILL_DIR", env_type::STR},
+    {"GGML_OPENVINO_DEBUG_NODE", env_type::STR},
+    {"GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR", env_type::STR},
+    {"GGML_OPENVINO_NPU_COMPILE_CONFIG", env_type::STR},
+    {"GGML_OPENVINO_NPU_COMPILER_TYPE", env_type::STR},
+    {"GGML_OPENVINO_NPUW_FUNCALL_FOR_ALL", env_type::STR},
+    {"GGML_OPENVINO_NPUW_UNFOLD_IREQS", env_type::STR},
+    {"GGML_OPENVINO_COMPILATION_NUM_THREADS", env_type::STR},
+    {"GGML_OPENVINO_NPU_CONFIG", env_type::STR},
+    {"GGML_OPENVINO_GPU_CONFIG", env_type::STR},
+    {"GGML_OPENVINO_CPU_CONFIG", env_type::STR},
+    // NPUW dump options require an OpenVINO developer build (NPU_PLUGIN_DEVELOPER_BUILD).
+    // DUMP_SUBS accepts YES, MIN (unique function bodies), last, or indices such as 0,2.
+    {"GGML_OPENVINO_NPUW_DUMP_SUBS", env_type::STR},
+    {"GGML_OPENVINO_NPUW_DUMP_SUBS_DIR", env_type::STR},
+    {"GGML_OPENVINO_NPUW_DUMP_SUBS_ON_FAIL", env_type::STR},
+    {"GGML_OPENVINO_NPUW_DUMP_IO", env_type::STR},
+    // One of a fixed set of values
+    {"GGML_OPENVINO_NPU_REQUANT_POLICY", env_type::ENUM, "group-128|channel-wise"},
+    {"GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT", env_type::ENUM, "f16|q8_0_c|q4_0_128|q4_0_c"},
+    {"GGML_OPENVINO_REQUANT_KQUANT", env_type::ENUM, "q4_sym128|q4_sym128_all|q4_asym64|q4_asym64_all|native"},
+    // Numbers
+    {"GGML_OPENVINO_PREFILL_CHUNK_SIZE", env_type::INT},
+    {"GGML_OPENVINO_LAZY_HOST_THRESHOLD_MB", env_type::INT},
+    {"GGML_OPENVINO_PROFILING", env_type::INT},
+    {"GGML_OPENVINO_GDN_DECOMPOSE_CHUNK", env_type::INT},
+    {"GGML_OPENVINO_GDN_Q_SCALE", env_type::FLOAT},
+    // Switches
+    {"GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_BLOB_PREBUILD", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_KV_ON_DEMAND", env_type::BOOL},
+    {"GGML_OPENVINO_STATEFUL_EXECUTION", env_type::BOOL},
+    {"GGML_OPENVINO_DUMP_CGRAPH", env_type::BOOL},
+    {"GGML_OPENVINO_DUMP_IR", env_type::BOOL},
+    {"GGML_OPENVINO_DEBUG_INPUT", env_type::BOOL},
+    {"GGML_OPENVINO_DEBUG_OUTPUT", env_type::BOOL},
+    // Force the static (NPU-shape) compute path on any device, e.g. GGML_OPENVINO_DEVICE=CPU,
+    // to test the static-shape translation without NPUW/real NPU hardware in the loop.
+    {"GGML_OPENVINO_FORCE_STATIC", env_type::BOOL},
+    {"GGML_OPENVINO_PRINT_CGRAPH_TENSOR_ADDRESS", env_type::BOOL},
+    {"GGML_OPENVINO_DISABLE_CACHE", env_type::BOOL},
+    {"GGML_OPENVINO_DISABLE_FUSED_RECURRENCE", env_type::BOOL},
+    {"GGML_OPENVINO_ENABLE_KV_4D", env_type::BOOL},
+    {"GGML_OPENVINO_ENABLE_FALLBACK", env_type::BOOL},
+    {"GGML_OPENVINO_KV_SCATTER_ELEMENTS", env_type::BOOL},
+    {"GGML_OPENVINO_MANUAL_GQA_ATTN", env_type::BOOL},
+    {"GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN", env_type::BOOL},
+    {"GGML_OPENVINO_MOE_OP", env_type::BOOL},
+    {"GGML_OPENVINO_EXACT_ZERO_POINT", env_type::BOOL},
+    {"GGML_OPENVINO_MEMORY_OPTIMIZE", env_type::BOOL},
+    {"GGML_OPENVINO_RELEASE_WEIGHTS", env_type::BOOL},
+    {"GGML_OPENVINO_REDUCE_COMPILE_MEM", env_type::BOOL},
+    {"GGML_OPENVINO_LOG_UNSUPPORTED_OPS", env_type::BOOL},
+    {"GGML_OPENVINO_LOG_SWA_LAYERS", env_type::BOOL},
+    {"GGML_OPENVINO_NATIVE_SOFTPLUS", env_type::BOOL},
+    {"GGML_OPENVINO_DISABLE_REMOTE_OUTPUTS", env_type::BOOL},
+    {"GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_FAST_MASK", env_type::BOOL},
+    {"GGML_OPENVINO_DISABLE_SHAPE_FROM_MASK", env_type::BOOL},
+    {"GGML_OPENVINO_GDN_NATIVE_STATE", env_type::BOOL},
+    {"GGML_OPENVINO_GDN_DECOMPOSE", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_L0_HOST_TENSORS", env_type::BOOL},
+    {"GGML_OPENVINO_KV_SLICE", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_HFA_PREFILL", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_DECODE_UNFOLD", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_PARALLEL_COMPILE", env_type::BOOL},
+    {"GGML_OPENVINO_SELF_CONTAINED_BLOB", env_type::BOOL},
+    {"GGML_OPENVINO_SELF_CONTAINED_MMAP", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_SHARED_BANK", env_type::BOOL},
+    {"GGML_OPENVINO_NPU_WEIGHTLESS_BANK", env_type::BOOL},
+    // Build the precise (but O(n_nodes)) graph cache key. Needed by op tests.
+    {"GGML_OPENVINO_FULL_GRAPH_KEY", env_type::BOOL},
+};
+
+std::string trim_spaces(const std::string & s) {
+    size_t begin = 0;
+    size_t end = s.size();
+    while (begin < end && std::isspace(static_cast<unsigned char>(s[begin]))) {
+        begin++;
+    }
+    while (end > begin && std::isspace(static_cast<unsigned char>(s[end - 1]))) {
+        end--;
+    }
+    return s.substr(begin, end - begin);
+}
+
+// Strict parse: digits with optional sign and surrounding spaces, in int range.
+// "set X=1 && ..." in cmd keeps a trailing space, so trailing spaces are allowed.
+bool parse_env_int(const char * v, int & out) {
+    char * end = nullptr;
+    errno = 0;
+    const long value = std::strtol(v, &end, 10);
+    if (end == v || errno == ERANGE || value < INT_MIN || value > INT_MAX) {
+        return false;
+    }
+    while (std::isspace(static_cast<unsigned char>(*end))) {
+        end++;
+    }
+    if (*end != '\0') {
+        return false;
+    }
+    out = static_cast<int>(value);
+    return true;
+}
+
+// The value to cache for a variable that is set. Stops the backend when the value does not fit the type: a silent
+// fallback to the default is easy to miss.
+std::string checked_env_value(const env_var & var, const char * value) {
+    int number = 0;
+    switch (var.type) {
+    case env_type::STR:
+        break;
+    case env_type::ENUM: {
+        const std::string wanted = trim_spaces(value);
+        const std::string values = var.values;
+        std::string list;
+        for (size_t pos = 0; pos <= values.size();) {
+            const size_t bar = std::min(values.find('|', pos), values.size());
+            const std::string accepted = values.substr(pos, bar - pos);
+            if (wanted == accepted) {
+                return wanted;
+            }
+            list += (list.empty() ? "" : ", ") + accepted;
+            pos = bar + 1;
+        }
+        GGML_ABORT("ggml-openvino: %s=%s is not valid, use one of: %s", var.name, value, list.c_str());
+    }
+    case env_type::INT:
+        if (!parse_env_int(value, number)) {
+            GGML_ABORT("ggml-openvino: %s=%s is not valid, use a whole number", var.name, value);
+        }
+        return std::to_string(number);
+    case env_type::FLOAT: {
+        char * end = nullptr;
+        (void) std::strtof(value, &end);
+        if (end == value || !trim_spaces(end).empty()) {
+            GGML_ABORT("ggml-openvino: %s=%s is not valid, use a number", var.name, value);
+        }
+        break;
+    }
+    case env_type::BOOL:
+        if (!parse_env_int(value, number)) {
+            GGML_ABORT("ggml-openvino: %s=%s is not valid, use 1 (on) or 0 (off)", var.name, value);
+        }
+        return number > 0 ? "1" : "0";
+    }
+    return value;
+}
+
+// A misspelled name would otherwise be ignored without a message.
+void warn_unknown_env_vars() {
+    static const char prefix[] = "GGML_OPENVINO_";
+#if defined(_WIN32)
+    char ** env = _environ;
+#elif defined(__APPLE__)
+    char ** env = *_NSGetEnviron();
+#else
+    char ** env = environ;
+#endif
+    for (; env != nullptr && *env != nullptr; ++env) {
+        if (strncmp(*env, prefix, sizeof(prefix) - 1) != 0) {
+            continue;
+        }
+        const char * eq = strchr(*env, '=');
+        const std::string name(*env, eq ? static_cast<size_t>(eq - *env) : strlen(*env));
+        const bool known = std::any_of(std::begin(env_vars), std::end(env_vars),
+                                       [&name](const env_var & var) { return name == var.name; });
+        if (!known) {
+            GGML_LOG_WARN("ggml-openvino: unknown environment variable %s is ignored\n", name.c_str());
+        }
+    }
+}
+
+}  // namespace
+
 // =====================================================
 // Device Configuration Implementations
 // =====================================================
@@ -101,160 +314,140 @@ void ggml_openvino_device_config::init() {
     // Set up front: a failed OpenCL setup below is not retried on every call
     initialized = true;

-    // All recognized GGML_OPENVINO_* env vars. Their values are cached here
-    // once at backend init time and read back via ggml_openvino_getenv_str()
-    // (raw string) or ggml_openvino_getenv_int() (integer / boolean toggle).
-    static constexpr const char * env_var_names[] = {
-        // String values (use ggml_openvino_getenv_str)
-        "GGML_OPENVINO_DEVICE",
-        "GGML_OPENVINO_CACHE_DIR",
-        "GGML_OPENVINO_SPILL_DIR",
-        "GGML_OPENVINO_DEBUG_NODE",
-        "GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR",
-        "GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY",
-        "GGML_OPENVINO_NPU_COMPILE_CONFIG",
-        // Integer values (use ggml_openvino_getenv_int)
-        "GGML_OPENVINO_PREFILL_CHUNK_SIZE",
-        // Boolean toggles (treated as int flags via ggml_openvino_getenv_int)
-        "GGML_OPENVINO_STATEFUL_EXECUTION",
-        "GGML_OPENVINO_PROFILING",
-        "GGML_OPENVINO_DUMP_CGRAPH",
-        "GGML_OPENVINO_DUMP_IR",
-        "GGML_OPENVINO_DEBUG_INPUT",
-        "GGML_OPENVINO_DEBUG_OUTPUT",
-        // Force the static (NPU-shape) compute path on any device, e.g. GGML_OPENVINO_DEVICE=CPU,
-        // to test the static-shape translation without NPUW/real NPU hardware in the loop.
-        "GGML_OPENVINO_FORCE_STATIC",
-        "GGML_OPENVINO_PRINT_CGRAPH_TENSOR_ADDRESS",
-        "GGML_OPENVINO_ENABLE_CACHE",
-        "GGML_OPENVINO_DISABLE_CACHE",
-        "GGML_OPENVINO_DISABLE_KV_SLICE",
-        "GGML_OPENVINO_ENABLE_FALLBACK",
-        "GGML_OPENVINO_MANUAL_GQA_ATTN",
-        "GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN",
-        "GGML_OPENVINO_MOE_OP",
-        "GGML_OPENVINO_MEMORY_OPTIMIZE",
-        "GGML_OPENVINO_RELEASE_WEIGHTS",
-        "GGML_OPENVINO_REDUCE_COMPILE_MEM",
-        "GGML_OPENVINO_LOG_UNSUPPORTED_OPS",
-        "GGML_OPENVINO_LOG_SWA_LAYERS",
-        "GGML_OPENVINO_NATIVE_SOFTPLUS",
-        "GGML_OPENVINO_DISABLE_REMOTE_OUTPUTS",
-        "GGML_OPENVINO_REQUANT_KQUANT",
-        "GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT",
-        // Build the precise (but O(n_nodes)) graph cache key. Needed by op tests.
-        "GGML_OPENVINO_FULL_GRAPH_KEY",
-    };
-
-    for (const char * const & env_var : env_var_names) {
-        auto * env = getenv(env_var);
-        if (env) {
-            environment_variables[env_var] = env;
+    for (const env_var & var : env_vars) {
+        const char * value = getenv(var.name);
+        // unset and empty both mean the default
+        if (value != nullptr && value[0] != '\0') {
+            environment_variables[var.name] = checked_env_value(var, value);
+        }
+    }
+    warn_unknown_env_vars();
+    if (!environment_variables.empty()) {
+        std::string settings;
+        for (const env_var & var : env_vars) {
+            const auto it = environment_variables.find(var.name);
+            if (it != environment_variables.end()) {
+                settings += std::string(" ") + var.name + "=" + it->second;
+            }
         }
+        GGML_LOG_INFO("ggml-openvino: settings from the environment:%s\n", settings.c_str());
+    }
+
+    if (ggml_openvino_getenv_int("GGML_OPENVINO_STATEFUL_EXECUTION") != 0) {
+        GGML_LOG_WARN(
+            "ggml-openvino: GGML_OPENVINO_STATEFUL_EXECUTION is deprecated and will be removed, "
+            "use stateless execution (the default)\n");
     }

     available_devices = ov_enumerate_devices();
-    device_name = resolve_openvino_device_name(available_devices, ggml_openvino_getenv_str("GGML_OPENVINO_DEVICE"));
+    if (const char * requested = ggml_openvino_getenv_str("GGML_OPENVINO_DEVICE")) {
+        std::stringstream ss(requested);
+        std::string name;
+        while (std::getline(ss, name, ',')) {
+            selected_devices.push_back(resolve_openvino_device_name(available_devices, name.c_str()));
+        }
+        for (const auto & selected : selected_devices) {
+            if (selected_devices.size() > 1 && !has_prefix(selected, "GPU")) {
+                GGML_ABORT("GGML OpenVINO Backend: GGML_OPENVINO_DEVICE=%s: only GPUs can be combined", requested);
+            }
+        }
+    } else {
+        selected_devices.push_back(resolve_openvino_device_name(available_devices, nullptr));
+    }
+    device_name = selected_devices.front();
     is_npu = has_prefix(device_name, "NPU");

     ggml_openvino_model_cache_init();

     const char * cache_dir = ggml_openvino_getenv_str("GGML_OPENVINO_CACHE_DIR");
     if (has_prefix(device_name, "NPU")) {
-        compile_config = {
-            {"NPU_COMPILER_DYNAMIC_QUANTIZATION", "YES"   },
-            {"NPU_USE_NPUW",                      "YES"   },
-            {"NPUW_DEVICES",                      "NPU"   },
-            {"NPUW_FOLD",                         "YES"   },
-            {"NPUW_WEIGHTS_BANK",                 "shared"},
-            {"NPUW_FUNCALL_FOR_ALL",              "YES"   },
-            {"NPUW_FUNCALL_ASYNC",                "YES"   },
-            {"NPUW_DQ",                           "YES"   },
-            {"NPUW_DQ_FULL",                      "NO"    },
-        };
+        compile_config = ggml_openvino_npu_build_compile_config(cache_dir);
+    } else {
         if (cache_dir && strlen(cache_dir) > 0) {
-            compile_config["NPUW_CACHE_DIR"] = cache_dir;
+            compile_config.insert(ov::cache_dir(cache_dir));
             compile_config.insert(ov::cache_mode(ov::CacheMode::OPTIMIZE_SIZE));
         }
-        const char * compilation_mode_params =
-            ggml_openvino_getenv_str("GGML_OPENVINO_NPU_COMPILE_CONFIG");
-        if (compilation_mode_params && strlen(compilation_mode_params) > 0) {
-            compile_config["NPU_COMPILATION_MODE_PARAMS"] = compilation_mode_params;
-        }
-    } else if (cache_dir && strlen(cache_dir) > 0) {
-        compile_config.insert(ov::cache_dir(cache_dir));
-        compile_config.insert(ov::cache_mode(ov::CacheMode::OPTIMIZE_SIZE));
+        // Plugin properties of the selected device, applied last
+        const bool gpu = has_prefix(device_name, "GPU");
+        ggml_openvino_apply_config_overrides(
+            compile_config, ggml_openvino_getenv_str(gpu ? "GGML_OPENVINO_GPU_CONFIG" : "GGML_OPENVINO_CPU_CONFIG"),
+            gpu ? "GPU" : "CPU");
     }

     if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING") >= 2) {
         compile_config.insert(ov::enable_profiling(true));
     }

-    // Initialize remote context with queue sharing for GPU
-    if (has_prefix(device_name, "GPU")) {
-        // Use the OpenCL context OpenVINO created for this device, so GPU.N gets its own device
-        cl_context cl_ctx;
-        try {
-            auto ov_ctx = ov_singleton_core().get_default_context(device_name).as<ov::intel_gpu::ocl::ClContext>();
-            cl_ctx = ov_ctx.get();
-        } catch (const std::exception & e) {
-            // The consumers of the remote context have no host fallback, and OpenVINO
-            // already reported the device as present.
-            GGML_ABORT("ggml-openvino: failed to get the OpenCL context for %s: %s", device_name.c_str(), e.what());
-        }
+    for (const auto & name : selected_devices) {
+        // Initialize remote context with queue sharing for GPU
+        if (has_prefix(name, "GPU")) {
+            auto & gpu = gpu_contexts[name];
+            // Use the OpenCL context OpenVINO created for this device, so GPU.N gets its own device
+            cl_context cl_ctx;
+            try {
+                auto ov_ctx = ov_singleton_core().get_default_context(name).as<ov::intel_gpu::ocl::ClContext>();
+                cl_ctx = ov_ctx.get();
+            } catch (const std::exception & e) {
+                // The consumers of the remote context have no host fallback, and OpenVINO
+                // already reported the device as present.
+                GGML_ABORT("ggml-openvino: failed to get the OpenCL context for %s: %s", name.c_str(), e.what());
+            }

-        cl_int err;
-        cl_device_id cl_device;
-        err = clGetContextInfo(cl_ctx, CL_CONTEXT_DEVICES, sizeof(cl_device), &cl_device, nullptr);
-        if (err != CL_SUCCESS) {
-            GGML_ABORT("ggml-openvino: failed to get the OpenCL device for %s: %d", device_name.c_str(), err);
-        }
+            cl_int err;
+            cl_device_id cl_device;
+            err = clGetContextInfo(cl_ctx, CL_CONTEXT_DEVICES, sizeof(cl_device), &cl_device, nullptr);
+            if (err != CL_SUCCESS) {
+                GGML_ABORT("ggml-openvino: failed to get the OpenCL device for %s: %d", name.c_str(), err);
+            }

-        cl_platform_id cl_platform;
-        err = clGetDeviceInfo(cl_device, CL_DEVICE_PLATFORM, sizeof(cl_platform), &cl_platform, nullptr);
-        if (err != CL_SUCCESS) {
-            GGML_ABORT("ggml-openvino: failed to get the OpenCL platform for %s: %d", device_name.c_str(), err);
-        }
+            cl_platform_id cl_platform;
+            err = clGetDeviceInfo(cl_device, CL_DEVICE_PLATFORM, sizeof(cl_platform), &cl_platform, nullptr);
+            if (err != CL_SUCCESS) {
+                GGML_ABORT("ggml-openvino: failed to get the OpenCL platform for %s: %d", name.c_str(), err);
+            }

-        cl_mem_fill_fn =
-            (clEnqueueMemFillINTEL_fn) clGetExtensionFunctionAddressForPlatform(cl_platform, "clEnqueueMemFillINTEL");
-        cl_mem_cpy_fn =
-            (clEnqueueMemcpyINTEL_fn) clGetExtensionFunctionAddressForPlatform(cl_platform, "clEnqueueMemcpyINTEL");
+            cl_mem_fill_fn = (clEnqueueMemFillINTEL_fn) clGetExtensionFunctionAddressForPlatform(
+                cl_platform, "clEnqueueMemFillINTEL");
+            cl_mem_cpy_fn =
+                (clEnqueueMemcpyINTEL_fn) clGetExtensionFunctionAddressForPlatform(cl_platform, "clEnqueueMemcpyINTEL");

-        cl_ulong device_max_alloc = 0;
-        err = clGetDeviceInfo(cl_device, CL_DEVICE_MAX_MEM_ALLOC_SIZE, sizeof(device_max_alloc), &device_max_alloc,
-                              nullptr);
-        if (err == CL_SUCCESS) {
-            max_alloc_size = device_max_alloc;
-        } else {
-            // not fatal, ggml then allocates one buffer
-            GGML_LOG_WARN("Failed to get OpenCL max allocation size: %d\n", err);
-        }
+            cl_ulong device_max_alloc = 0;
+            err = clGetDeviceInfo(cl_device, CL_DEVICE_MAX_MEM_ALLOC_SIZE, sizeof(device_max_alloc), &device_max_alloc,
+                                  nullptr);
+            if (err == CL_SUCCESS) {
+                gpu.max_alloc_size = device_max_alloc;
+            } else {
+                // not fatal, ggml then allocates one buffer
+                GGML_LOG_WARN("Failed to get OpenCL max allocation size: %d\n", err);
+            }

-        const cl_queue_properties profiling_properties[] = {
-            CL_QUEUE_PROPERTIES,
-            CL_QUEUE_PROFILING_ENABLE,
-            0,
-        };
-        const cl_queue_properties * queue_properties =
-            ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING") >= 2 ? profiling_properties : nullptr;
-        cl_queue = clCreateCommandQueueWithProperties(cl_ctx, cl_device, queue_properties, &err);
-        if (err != CL_SUCCESS) {
-            GGML_ABORT("ggml-openvino: failed to create the OpenCL queue for %s: %d", device_name.c_str(), err);
-        }
+            const cl_queue_properties profiling_properties[] = {
+                CL_QUEUE_PROPERTIES,
+                CL_QUEUE_PROFILING_ENABLE,
+                0,
+            };
+            const cl_queue_properties * queue_properties =
+                ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING") >= 2 ? profiling_properties : nullptr;
+            gpu.cl_queue = clCreateCommandQueueWithProperties(cl_ctx, cl_device, queue_properties, &err);
+            if (err != CL_SUCCESS) {
+                GGML_ABORT("ggml-openvino: failed to create the OpenCL queue for %s: %d", name.c_str(), err);
+            }

-        // Create OpenVINO remote context with queue sharing
-        remote_context = ov::intel_gpu::ocl::ClContext(ov_singleton_core(), cl_queue);
-    } else if (has_prefix(device_name, "NPU")) {
-        // remote tensor is not used for NPU yet
-        // remote_context = ov_singleton_core().get_default_context(device_name);
+            // Create OpenVINO remote context with queue sharing
+            gpu.remote_context = ov::intel_gpu::ocl::ClContext(ov_singleton_core(), gpu.cl_queue);
+        } else if (has_prefix(name, "NPU")) {
+            // remote tensor is not used for NPU yet
+            // remote_context = ov_singleton_core().get_default_context(name);
+        }
     }
 }

 ggml_openvino_device_config::~ggml_openvino_device_config() {
-    if (cl_queue != nullptr) {
-        clReleaseCommandQueue(cl_queue);
-        cl_queue = nullptr;
+    for (auto & [name, gpu] : gpu_contexts) {
+        if (gpu.cl_queue != nullptr) {
+            clReleaseCommandQueue(gpu.cl_queue);
+            gpu.cl_queue = nullptr;
+        }
     }
 }

@@ -274,12 +467,17 @@ const std::string & ggml_openvino_get_device_name() {
     return ggml_openvino_get_device_config().device_name;
 }

-std::vector<std::string> ggml_openvino_get_available_devices() {
+const std::vector<std::string> & ggml_openvino_get_available_devices() {
     auto & config = ggml_openvino_get_device_config();
     config.init();
     return config.available_devices;
 }

+bool ggml_openvino_is_selected_device(const std::string & device) {
+    const auto & selected = ggml_openvino_get_device_config().selected_devices;
+    return std::find(selected.begin(), selected.end(), device) != selected.end();
+}
+
 // Get the value of a GGML_OPENVINO_* env var as a string. Returns
 // default_value when the var is unset or set to an empty string.
 const char * ggml_openvino_getenv_str(const char * var, const char * default_value) {
@@ -288,13 +486,113 @@ const char * ggml_openvino_getenv_str(const char * var, const char * default_val
     return (it == env_map.end() || it->second.empty()) ? default_value : it->second.c_str();
 }

-// Get the value of a GGML_OPENVINO_* env var as an int (via std::atoi).
-// Returns default_value (0) when the var is unset or empty. Used for both
-// integer settings (e.g. GGML_OPENVINO_PREFILL_CHUNK_SIZE) and boolean
-// toggles: "0" disables, any non-zero integer enables.
+// Get the value of a GGML_OPENVINO_* env var as an int. Returns default_value
+// (0) when the var is unset or empty. Used for both integer settings (e.g.
+// GGML_OPENVINO_PREFILL_CHUNK_SIZE) and boolean toggles, which are cached as 1 (on) or 0 (off).
 int ggml_openvino_getenv_int(const char * var, int default_value) {
     const char * v = ggml_openvino_getenv_str(var, nullptr);
-    return v ? std::atoi(v) : default_value;
+    int value = default_value;
+    return v != nullptr && parse_env_int(v, value) ? value : default_value;
+}
+
+void ggml_openvino_apply_config_overrides(ov::AnyMap & config, const char * spec, const char * device) {
+    if (spec == nullptr) {
+        return;
+    }
+    // A property name: letters, digits, '_' and '.'
+    auto is_key = [](const std::string & s) {
+        return !s.empty() &&
+               std::all_of(s.begin(), s.end(), [](unsigned char c) { return std::isalnum(c) || c == '_' || c == '.'; });
+    };
+    std::string key;
+    std::string value;
+    auto apply_pair = [&] {
+        if (!key.empty()) {
+            value = trim_spaces(value);
+            config[key] = value;
+            GGML_LOG_INFO("GGML OpenVINO: %s config override %s=%s\n", device, key.c_str(), value.c_str());
+        }
+    };
+    const std::string pairs(spec);
+    for (size_t pos = 0; pos <= pairs.size();) {
+        const size_t comma = std::min(pairs.find(',', pos), pairs.size());
+        const std::string part = pairs.substr(pos, comma - pos);
+        const size_t eq = part.find('=');
+        const std::string name = eq == std::string::npos ? std::string() : trim_spaces(part.substr(0, eq));
+        if (is_key(name)) {
+            apply_pair();
+            key = name;
+            value = part.substr(eq + 1);
+        } else if (!key.empty()) {
+            // no KEY= after the comma: the comma belongs to the value, as in NPUW_DEVICES=NPU,CPU
+            value += "," + part;
+        } else if (!trim_spaces(part).empty()) {
+            GGML_LOG_WARN("ggml-openvino: GGML_OPENVINO_%s_CONFIG: '%s' is ignored, expected KEY=VALUE\n", device,
+                          part.c_str());
+        }
+        pos = comma + 1;
+    }
+    apply_pair();
+}
+
+std::string ggml_openvino_config_overrides_in_use() {
+    const char * name = ggml_openvino_is_npu() ? "GGML_OPENVINO_NPU_CONFIG" :
+                        ggml_openvino_is_gpu() ? "GGML_OPENVINO_GPU_CONFIG" :
+                                                 "GGML_OPENVINO_CPU_CONFIG";
+    const char * value = ggml_openvino_getenv_str(name);
+    return value ? std::string(name) + "=" + value : std::string();
+}
+
+bool ggml_openvino_gdn_prescale_enabled() {
+    // float value, so not getenv_int: "64.0" must enable it. Unset means on.
+    const char * v = ggml_openvino_getenv_str("GGML_OPENVINO_GDN_Q_SCALE", nullptr);
+    return v == nullptr || std::strtof(v, nullptr) > 0.0f;
+}
+
+bool ggml_openvino_gdn_native_state_enabled() {
+    // only verified on NPU so far
+    return ggml_openvino_is_npu() && ggml_openvino_getenv_int("GGML_OPENVINO_GDN_NATIVE_STATE") > 0;
+}
+
+bool ggml_openvino_npu_is_4000() {
+    if (!ggml_openvino_is_npu()) {
+        return false;
+    }
+    static const bool is_4000 = [] {
+        try {
+            return ov_singleton_core().get_property("NPU", "DEVICE_ARCHITECTURE").as<std::string>() == "4000";
+        } catch (...) {
+            return false;
+        }
+    }();
+    return is_4000;
+}
+
+bool ggml_openvino_gdn_decompose_enabled() {
+    if (ggml_openvino_getenv_str("GGML_OPENVINO_GDN_DECOMPOSE") != nullptr) {
+        return ggml_openvino_getenv_int("GGML_OPENVINO_GDN_DECOMPOSE") > 0;
+    }
+    return ggml_openvino_npu_is_4000();
+}
+
+int ggml_openvino_gdn_decompose_chunk() {
+    const int chunk = ggml_openvino_getenv_int("GGML_OPENVINO_GDN_DECOMPOSE_CHUNK");
+    if (chunk >= 2 && chunk <= 128 && (chunk & (chunk - 1)) == 0) {
+        return chunk;
+    }
+    return 16;
+}
+
+float ggml_openvino_gdn_q_scale() {
+    constexpr float default_scale = 64.0f;
+    const char * v = ggml_openvino_getenv_str("GGML_OPENVINO_GDN_Q_SCALE", nullptr);
+    float scale = v ? std::strtof(v, nullptr) : default_scale;
+    if (scale < default_scale) {
+        GGML_LOG_WARN("ggml-openvino: GGML_OPENVINO_GDN_Q_SCALE=%s is below the minimum %.1f, clamping\n", v,
+                      default_scale);
+        scale = default_scale;
+    }
+    return scale;
 }

 bool ggml_openvino_reduce_compile_mem_enabled() {
@@ -322,13 +620,310 @@ bool ggml_openvino_is_gpu() {
     return has_prefix(ggml_openvino_get_device_name(), "GPU");
 }

-size_t ggml_openvino_max_alloc_size() {
-    return ggml_openvino_get_device_config().max_alloc_size;
+bool ggml_openvino_kv_slice_enabled() {
+    return ggml_openvino_getenv_int("GGML_OPENVINO_KV_SLICE", 1) != 0;
+}
+
+bool ggml_openvino_npu_fast_mask_enabled() {
+    return ggml_openvino_getenv_int("GGML_OPENVINO_NPU_FAST_MASK", 1) != 0;
+}
+
+bool ggml_openvino_npu_l0_host_tensors_enabled() {
+    return ggml_openvino_getenv_int("GGML_OPENVINO_NPU_L0_HOST_TENSORS", 1) != 0;
+}
+
+bool ggml_openvino_npu_hfa_prefill_enabled() {
+    return ggml_openvino_is_npu() && ggml_openvino_getenv_int("GGML_OPENVINO_NPU_HFA_PREFILL") != 0;
 }

-// Get the remote context for the current device (returns empty optional for CPU)
-std::optional<ov::RemoteContext> ggml_openvino_get_remote_context() {
-    return ggml_openvino_get_device_config().remote_context;
+bool ggml_openvino_npu_decode_unfold_enabled() {
+    return ggml_openvino_getenv_int("GGML_OPENVINO_NPU_DECODE_UNFOLD") != 0;
+}
+
+bool ggml_openvino_npu_parallel_compile_enabled() {
+    return ggml_openvino_getenv_int("GGML_OPENVINO_NPU_PARALLEL_COMPILE") != 0;
+}
+
+int ggml_openvino_npu_decode_bucket() {
+    if (!ggml_openvino_is_npu()) {
+        return 0;
+    }
+    // A self-contained blob embeds its weights bank, and NPUW cannot load a blob into a bank that already holds
+    // weights, so a decode model per KV window cannot be imported next to the decode model of the pair.
+    if (ggml_openvino_getenv_str("GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR") &&
+        ggml_openvino_getenv_int("GGML_OPENVINO_SELF_CONTAINED_BLOB") != 0 &&
+        !ggml_openvino_npu_weightless_bank_enabled()) {
+        return 0;
+    }
+    return ggml_openvino_getenv_int("GGML_OPENVINO_NPU_BLOB_PREBUILD", 1) != 0 ? 1024 : 0;
+}
+
+int ggml_openvino_static_kv_window(int rows, int ctx) {
+    int window = 1;
+    while (window < rows && window < ctx) {
+        window *= 2;
+    }
+    return std::min(window, ctx);
+}
+
+bool ggml_openvino_npu_kv_on_demand_enabled() {
+    return ggml_openvino_is_npu() && ggml_openvino_getenv_int("GGML_OPENVINO_NPU_KV_ON_DEMAND") != 0;
+}
+
+bool ggml_openvino_npu_kv_head_major_enabled() {
+    return ggml_openvino_npu_kv_on_demand_enabled() && ggml_openvino_npu_decode_bucket() > 0 &&
+           ggml_openvino_kv_slice_enabled();
+}
+
+bool ggml_openvino_npu_raw_weight_cache_enabled() {
+    return ggml_openvino_getenv_int("GGML_OPENVINO_SELF_CONTAINED_BLOB") != 0 ||
+           ggml_openvino_npu_weightless_bank_enabled();
+}
+
+bool ggml_openvino_npu_weightless_bank_enabled() {
+    return ggml_openvino_getenv_int("GGML_OPENVINO_NPU_WEIGHTLESS_BANK") != 0;
+}
+
+namespace {
+struct kv_storage_entry {
+    size_t row_bytes = 0;
+    size_t max_rows = 0;
+    size_t cap_rows = 0;
+    // 0: ggml row layout [rows][heads * head_size]; >0: head-major [heads][cap_rows][head_size].
+    size_t heads = 0;
+    ov::Tensor storage;
+};
+
+struct kv_storage_registry {
+    std::mutex mutex;
+    // Keyed by the ggml data address of each KV tensor.
+    std::map<uintptr_t, kv_storage_entry> entries;
+};
+
+kv_storage_registry & kv_registry() {
+    static kv_storage_registry reg;
+    return reg;
+}
+
+kv_storage_entry * kv_find_locked(kv_storage_registry & reg, uintptr_t addr, uintptr_t & base) {
+    auto it = reg.entries.upper_bound(addr);
+    if (it == reg.entries.begin()) {
+        return nullptr;
+    }
+    --it;
+    if (addr >= it->first + it->second.max_rows * it->second.row_bytes) {
+        return nullptr;
+    }
+    base = it->first;
+    return &it->second;
+}
+
+// Storage offset of byte `off` of the ggml tensor; its row must be below cap_rows.
+size_t kv_storage_offset(const kv_storage_entry & e, size_t off) {
+    if (e.heads == 0) {
+        return off;
+    }
+    const size_t head_bytes = e.row_bytes / e.heads;
+    const size_t row = off / e.row_bytes;
+    const size_t in_row = off % e.row_bytes;
+    return ((in_row / head_bytes) * e.cap_rows + row) * head_bytes + in_row % head_bytes;
+}
+
+bool kv_grow_locked(kv_storage_entry & e, size_t rows, size_t heads) {
+    if (heads > 0 && e.heads != heads) {
+        if (e.cap_rows > 0) {
+            throw std::runtime_error("ggml-openvino: on-demand KV storage layout changed");
+        }
+        GGML_ASSERT(e.row_bytes % heads == 0);
+        e.heads = heads;
+    }
+    rows = std::min(rows, e.max_rows);
+    if (rows <= e.cap_rows) {
+        return false;
+    }
+    const int bucket = ggml_openvino_npu_decode_bucket();
+    size_t cap = bucket > 0 ? static_cast<size_t>(bucket) : 1024;
+    while (cap < rows) {
+        cap *= 2;
+    }
+    cap = std::min(cap, e.max_rows);
+    const size_t bytes = cap * e.row_bytes;
+    ov::Tensor grown;
+    if (ggml_openvino_npu_l0_host_tensors_enabled()) {
+        grown = ov_singleton_core().get_default_context("NPU").create_host_tensor(ov::element::u8, ov::Shape{bytes});
+    } else {
+        grown = ov::Tensor(ov::element::u8, ov::Shape{bytes});
+    }
+    auto * dst = static_cast<uint8_t *>(grown.data());
+    // Rows past the history are masked out, but must stay finite (0 * NaN is NaN).
+    std::memset(dst, 0, bytes);
+    if (e.cap_rows > 0) {
+        const auto * src = static_cast<const uint8_t *>(e.storage.data());
+        if (e.heads == 0) {
+            std::memcpy(dst, src, e.cap_rows * e.row_bytes);
+        } else {
+            const size_t head_bytes = e.row_bytes / e.heads;
+            for (size_t h = 0; h < e.heads; ++h) {
+                std::memcpy(dst + h * cap * head_bytes, src + h * e.cap_rows * head_bytes, e.cap_rows * head_bytes);
+            }
+        }
+    }
+    // Requests still bound to the old storage keep it alive until they are rebound or released.
+    e.storage = grown;
+    e.cap_rows = cap;
+    return true;
+}
+}  // namespace
+
+void ggml_openvino_kv_storage_register(void * data, size_t row_bytes, size_t rows) {
+    auto & reg = kv_registry();
+    std::lock_guard<std::mutex> lock(reg.mutex);
+    auto & e = reg.entries[reinterpret_cast<uintptr_t>(data)];
+    e = kv_storage_entry{};
+    e.row_bytes = row_bytes;
+    e.max_rows = rows;
+}
+
+void ggml_openvino_kv_storage_unregister(const void * base, size_t size) {
+    auto & reg = kv_registry();
+    std::lock_guard<std::mutex> lock(reg.mutex);
+    const auto begin = reinterpret_cast<uintptr_t>(base);
+    reg.entries.erase(reg.entries.lower_bound(begin), reg.entries.lower_bound(begin + size));
+}
+
+void * ggml_openvino_kv_storage(const void * ptr, size_t bytes) {
+    auto & reg = kv_registry();
+    std::lock_guard<std::mutex> lock(reg.mutex);
+    if (reg.entries.empty()) {
+        return nullptr;
+    }
+    const auto addr = reinterpret_cast<uintptr_t>(ptr);
+    uintptr_t base = 0;
+    auto * e = kv_find_locked(reg, addr, base);
+    if (e == nullptr) {
+        return nullptr;
+    }
+    const size_t offset = addr - base;
+    if (e->heads > 0 && offset != 0) {
+        throw std::runtime_error("ggml-openvino: head-major KV storage is only bound from the tensor start");
+    }
+    if (kv_grow_locked(*e, (offset + bytes + e->row_bytes - 1) / e->row_bytes, 0)) {
+        GGML_LOG_WARN("%s: KV storage grown during binding to %zu rows\n", __func__, e->cap_rows);
+    }
+    if (offset + bytes > e->cap_rows * e->row_bytes) {
+        throw std::runtime_error("ggml-openvino: on-demand KV access beyond the KV cache");
+    }
+    return static_cast<uint8_t *>(e->storage.data()) + offset;
+}
+
+bool ggml_openvino_kv_storage_reserve(const void * data, size_t rows, size_t heads) {
+    auto & reg = kv_registry();
+    std::lock_guard<std::mutex> lock(reg.mutex);
+    auto it = reg.entries.find(reinterpret_cast<uintptr_t>(data));
+    if (it == reg.entries.end()) {
+        return false;
+    }
+    const bool had_storage = it->second.cap_rows > 0;
+    return kv_grow_locked(it->second, rows, heads) && had_storage;
+}
+
+size_t ggml_openvino_kv_storage_capacity(const void * data) {
+    auto & reg = kv_registry();
+    std::lock_guard<std::mutex> lock(reg.mutex);
+    auto it = reg.entries.find(reinterpret_cast<uintptr_t>(data));
+    return it == reg.entries.end() ? 0 : it->second.cap_rows;
+}
+
+bool ggml_openvino_kv_storage_copy(const void * ptr, void * host, size_t size, ggml_openvino_kv_copy_op op) {
+    auto & reg = kv_registry();
+    std::lock_guard<std::mutex> lock(reg.mutex);
+    if (reg.entries.empty()) {
+        return false;
+    }
+    uintptr_t base = 0;
+    auto * e = kv_find_locked(reg, reinterpret_cast<uintptr_t>(ptr), base);
+    if (e == nullptr) {
+        return false;
+    }
+    const size_t begin = reinterpret_cast<uintptr_t>(ptr) - base;
+    if (op != GGML_OPENVINO_KV_READ) {
+        kv_grow_locked(*e, (begin + size + e->row_bytes - 1) / e->row_bytes, 0);
+    }
+    auto * storage = static_cast<uint8_t *>(e->storage.data());
+    auto * h = static_cast<uint8_t *>(host);
+    const size_t head_bytes = e->heads > 0 ? e->row_bytes / e->heads : e->row_bytes;
+    for (size_t done = 0; done < size;) {
+        const size_t off = begin + done;
+        const size_t n = std::min(size - done, head_bytes - off % head_bytes);
+        const bool stored = off / e->row_bytes < e->cap_rows;
+        uint8_t * s = stored ? storage + kv_storage_offset(*e, off) : nullptr;
+        if (op == GGML_OPENVINO_KV_READ) {
+            if (s) {
+                std::memcpy(h + done, s, n);
+            } else {
+                std::memset(h + done, 0, n);
+            }
+        } else if (op == GGML_OPENVINO_KV_WRITE) {
+            std::memcpy(s, h + done, n);
+        } else {
+            std::memset(s, *h, n);
+        }
+        done += n;
+    }
+    return true;
+}
+
+bool ggml_openvino_kv_storage_write_rows(const void * data, const void * src, size_t src_stride, size_t src_col,
+                                         size_t dst_row, size_t n) {
+    auto & reg = kv_registry();
+    std::lock_guard<std::mutex> lock(reg.mutex);
+    auto it = reg.entries.find(reinterpret_cast<uintptr_t>(data));
+    if (it == reg.entries.end() || it->second.heads == 0) {
+        return false;
+    }
+    auto & e = it->second;
+    if (dst_row + n > e.cap_rows || src_col + n > src_stride) {
+        throw std::runtime_error("ggml-openvino: head-major KV row write out of range");
+    }
+    const size_t head_bytes = e.row_bytes / e.heads;
+    auto * dst = static_cast<uint8_t *>(e.storage.data());
+    const auto * s = static_cast<const uint8_t *>(src);
+    for (size_t h = 0; h < e.heads; ++h) {
+        std::memcpy(dst + (h * e.cap_rows + dst_row) * head_bytes, s + (h * src_stride + src_col) * head_bytes,
+                    n * head_bytes);
+    }
+    return true;
+}
+
+void ggml_openvino_kv_storage_clear(const void * base, size_t size, uint8_t value) {
+    auto & reg = kv_registry();
+    std::lock_guard<std::mutex> lock(reg.mutex);
+    const auto begin = reinterpret_cast<uintptr_t>(base);
+    for (auto it = reg.entries.lower_bound(begin); it != reg.entries.end() && it->first < begin + size; ++it) {
+        if (it->second.cap_rows > 0) {
+            std::memset(it->second.storage.data(), value, it->second.storage.get_byte_size());
+        }
+    }
+}
+
+bool ggml_openvino_shape_from_mask_enabled() {
+    return ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_SHAPE_FROM_MASK") == 0;
+}
+
+size_t ggml_openvino_max_alloc_size(const std::string & device) {
+    const auto & gpu_contexts = ggml_openvino_get_device_config().gpu_contexts;
+    auto it = gpu_contexts.find(device);
+    return it == gpu_contexts.end() ? SIZE_MAX : it->second.max_alloc_size;
+}
+
+// Get the remote context for the device (returns empty optional for CPU)
+std::optional<ov::RemoteContext> ggml_openvino_get_remote_context(const std::string & device) {
+    const auto & gpu_contexts = ggml_openvino_get_device_config().gpu_contexts;
+    auto it = gpu_contexts.find(device);
+    if (it == gpu_contexts.end()) {
+        return std::nullopt;
+    }
+    return it->second.remote_context;
 }

 // Get the compile config for the current device
@@ -336,9 +931,11 @@ const ov::AnyMap & ggml_openvino_get_compile_config() {
     return ggml_openvino_get_device_config().compile_config;
 }

-// Get the OpenCL command queue for GPU operations
-cl_command_queue ggml_openvino_get_cl_queue() {
-    return ggml_openvino_get_device_config().cl_queue;
+// Get the OpenCL command queue of the device for GPU operations
+cl_command_queue ggml_openvino_get_cl_queue(const std::string & device) {
+    const auto & gpu_contexts = ggml_openvino_get_device_config().gpu_contexts;
+    auto it = gpu_contexts.find(device);
+    return it == gpu_contexts.end() ? nullptr : it->second.cl_queue;
 }

 // Get the clEnqueueMemFillINTEL function pointer
@@ -357,14 +954,19 @@ std::optional<ExtraQuantType> ggml_openvino_get_requant_type(const ggml_tensor *
         return std::nullopt;
     }
     if (strncmp(tensor->name, "token_embd.weight", 17) == 0) {
-        return ((ggml_openvino_is_npu() && tensor->type == GGML_TYPE_Q6_K) ? ExtraQuantType::F16 :
-                                                                             ExtraQuantType::Q8_0_C);
+        if (ggml_openvino_is_npu()) {
+            if (auto npu_type = ggml_openvino_npu_token_embd_requant_type(tensor)) {
+                return npu_type;
+            }
+        }
+        return ExtraQuantType::Q8_0_C;
     }
+
     if (strncmp(tensor->name, "output.weight", 13) == 0) {
         return ExtraQuantType::Q8_0_C;
     }
     if (ggml_openvino_is_npu()) {
-        return ExtraQuantType::Q4_0_128;
+        return ggml_openvino_npu_weight_requant_type(tensor);
     }
     // By default Q6_K/Q5_K are requantized to Q8_0_C, which *inflates* 6- and 5-bit weights to 8
     // while the rest of the model stays at 4 bits, and Q4_K keeps its native group-32 layout
@@ -373,12 +975,13 @@ std::optional<ExtraQuantType> ggml_openvino_get_requant_type(const ggml_tensor *
     //
     // GGML_OPENVINO_REQUANT_KQUANT selects a 4-bit target instead. Names are
     // q4_<sym|asym><group>[_all]: <sym|asym> says whether a per-group zero point is kept, <group>
-    // is the group size, and the _all suffix sends Q4_K down the same path (without it only
-    // Q6_K/Q5_K are touched):
+    // is the group size. The _all suffix includes Q4_K; q4_asym64_all also includes
+    // ordinary Q8_0/Q5_1 linear weights, matching the INT4_ASYM/group-64 primary precision
+    // used by NNCF with ratio=1.0. Embedding/output retain the channel-wise INT8 overrides above:
     //   q4_sym128      Q6_K/Q5_K -> Q4_0_128 (u4, group 128, symmetric)
     //   q4_sym128_all  and Q4_K too -- drops Q4_K's per-32 zero point, which costs some accuracy
     //   q4_asym64      Q6_K/Q5_K -> Q4_1_64 (u4, group 64, asymmetric)
-    //   q4_asym64_all  Q6_K/Q5_K and Q4_K -> Q4_1_64 (u4, group 64, asymmetric)
+    //   q4_asym64_all  Q6_K/Q5_K/Q4_K/Q8_0/Q5_1 -> Q4_1_64 (u4, group 64, asymmetric)
     //   native         no requantization at all (keep Q6_K/Q5_K as they are)
     //
     // q4_asym64 leaves Q4_K at its native group 32. Use q4_asym64_all to keep the group size uniform.
@@ -391,6 +994,13 @@ std::optional<ExtraQuantType> ggml_openvino_get_requant_type(const ggml_tensor *
     const bool asym64_all = is_opt("q4_asym64_all");
     const bool asym64 = is_opt("q4_asym64");

+    // TODO: temporary workaround for slow GPU-plugin GatherMatmul prefill above 16 tokens.
+    // Q4_1_64 experts let the plugin fuse the MoE into MOECompressed.
+    if (ggml_openvino_is_gpu() && tensor->ne[2] > 1 && strstr(tensor->name, "exps") != nullptr &&
+        (tensor->type == GGML_TYPE_Q4_K || tensor->type == GGML_TYPE_Q5_K || tensor->type == GGML_TYPE_Q6_K)) {
+        return ExtraQuantType::Q4_1_64;
+    }
+
     if (tensor->type == GGML_TYPE_Q4_K) {
         if (sym128_all) {
             return ExtraQuantType::Q4_0_128;
@@ -399,6 +1009,17 @@ std::optional<ExtraQuantType> ggml_openvino_get_requant_type(const ggml_tensor *
             return ExtraQuantType::Q4_1_64;
         }
     }
+    // Include dense Q8_0/Q5_1 in the explicit all-asymmetric policy. Keep the default GPU
+    // workaround below restricted to MoE experts: ordinary grouped INT8 MatMul is supported.
+    if (asym64_all && (tensor->type == GGML_TYPE_Q5_1 || tensor->type == GGML_TYPE_Q8_0)) {
+        // A group must fit within each row. Use backup channel-wise INT8 for incompatible
+        // widths (Q8_0/Q5_1 allow rows of 32 weights); every affected Qwen3.6 row is divisible by 64.
+        return tensor->ne[0] % 64 == 0 ? ExtraQuantType::Q4_1_64 : ExtraQuantType::Q8_0_C;
+    }
+    if (tensor->type == GGML_TYPE_Q5_0) {
+        //return ExtraQuantType::Q4_0_128;
+        return ExtraQuantType::Q4_0_64;
+    }
     // MoE expert weights (3D, ne[2] = n_expert) stored as Q5_1/Q8_0 are the expert-side
     // equivalent of Q6_K/Q5_K: kept at 8 bits by default while the rest of the model is at 4
     // (gemma-4 26B-A4B keeps its down projection there). Send them to 4 bits under the same
@@ -445,6 +1066,13 @@ std::optional<ExtraQuantType> ggml_openvino_get_requant_type(const ggml_tensor *
     }
 }

+bool ggml_openvino_use_exact_zero_point(const ggml_tensor * tensor) {
+    // Other 3D weights, such as MLA's per-head attn_k_b / attn_v_b, feed a batched MatMul, where the GPU plugin fails
+    // on u4 weights with an f16 zero point (CL_OUT_OF_RESOURCES), so the opt-in covers 2D weights only.
+    const bool is_moe_expert = tensor->ne[2] > 1 && strstr(tensor->name, "exps") != nullptr;
+    return is_moe_expert || (tensor->ne[2] == 1 && ggml_openvino_getenv_int("GGML_OPENVINO_EXACT_ZERO_POINT") > 0);
+}
+
 // =====================================================
 // Extracted Layout Calculation
 // =====================================================
@@ -464,11 +1092,9 @@ ggml_openvino_extracted_layout ggml_openvino_get_extracted_layout(const ggml_ten
         return layout;
     }

-    // 3D MoE expert weights that are not requantized (see below) always use the exact f16
-    // zero-point extraction (see extract_quantized_weights), which needs a wider zp slot than
-    // the packed integer zero point -- must be kept in sync with that function so the buffer
-    // sizing here matches what process_weight_tensor actually writes.
-    const bool for_gather_matmul = tensor->ne[2] > 1;
+    // An exact zero point (see extract_quantized_weights) needs an f16 zp slot instead of the
+    // packed integer one. Keep in sync with process_weight_tensor, which fills the slot.
+    const bool exact_zp = ggml_openvino_use_exact_zero_point(tensor);

     int64_t n_elements = ggml_nelements(tensor);
     const size_t alignment = 64;  // Good for SIMD
@@ -579,6 +1205,11 @@ ggml_openvino_extracted_layout ggml_openvino_get_extracted_layout(const ggml_ten
         layout.is_symmetric = true;
         break;

+    case GGML_TYPE_Q5_0:
+        // i8 weights (5-bit values minus 16), symmetric (scale only)
+        layout.is_symmetric = true;
+        break;
+
     case GGML_TYPE_Q5_1:
         // u8 weights (5-bit values), asymmetric (scale + zero point)
         break;
@@ -603,13 +1234,11 @@ ggml_openvino_extracted_layout ggml_openvino_get_extracted_layout(const ggml_ten
     // Scales: F16 per block, except MXFP4 which stores one E8M0 byte per block.
     int64_t n_blocks = n_elements / layout.weights_per_block;
     layout.scales_size = n_blocks * (tensor->type == GGML_TYPE_MXFP4 ? sizeof(uint8_t) : sizeof(uint16_t));
-    // For symmetric quantization, no zp needed (weights stored as signed). Asymmetric
-    // for_gather_matmul (3D MoE expert) weights use an exact f16 zero point (see
-    // extract_quantized_weights/make_int8_weights/make_int4_weights), which needs one f16 per
-    // block instead of a packed u4/u8 integer zero point.
+    // For symmetric quantization, no zp needed (weights stored as signed). An exact zero point
+    // needs one f16 per block instead of a packed u4/u8 integer zero point.
     if (layout.is_symmetric) {
         layout.zp_size = 0;
-    } else if (use_bias || for_gather_matmul) {
+    } else if (use_bias || exact_zp) {
         layout.zp_size = n_blocks * sizeof(uint16_t);
     } else {
         layout.zp_size = layout.is_u4 ? ((n_blocks + 1) / 2) : n_blocks;
@@ -625,7 +1254,9 @@ ggml_openvino_extracted_layout ggml_openvino_get_extracted_layout(const ggml_ten
     return layout;
 }

-ggml_openvino_tensor_extra * ggml_openvino_create_tensor_extra(const ggml_tensor * tensor, bool is_remote) {
+ggml_openvino_tensor_extra * ggml_openvino_create_tensor_extra(const ggml_tensor * tensor,
+                                                               bool is_remote,
+                                                               const std::string & device) {
     ov::Shape shape;
     for (int i = GGML_MAX_DIMS - 1; i >= 0; --i) {
         shape.push_back(static_cast<size_t>(tensor->ne[i]));
@@ -653,17 +1284,18 @@ ggml_openvino_tensor_extra * ggml_openvino_create_tensor_extra(const ggml_tensor
         return nullptr;
     }

-    auto remote_context = ggml_openvino_get_remote_context();
+    auto remote_context = ggml_openvino_get_remote_context(device);

     std::shared_ptr<ov::Tensor> ov_tensor;
     if (is_remote) {
         GGML_ASSERT(ggml_openvino_is_gpu());
         auto gpu_context = remote_context->as<ov::intel_gpu::ocl::ClContext>();
-        auto usm_tensor = gpu_context.create_tensor(element_type, shape, tensor->data);
+        auto usm_tensor = gpu_context.create_tensor(element_type, shape,
+                                                    tensor->data);
         ov_tensor = std::make_shared<ov::intel_gpu::ocl::USMTensor>(std::move(usm_tensor));
     } else {
         ov_tensor = std::make_shared<ov::Tensor>(element_type, shape, tensor->data);
     }

-    return new ggml_openvino_tensor_extra(ov_tensor);
+    return new ggml_openvino_tensor_extra(ov_tensor, tensor->data);
 }
diff --git a/ggml/src/ggml-openvino/ggml-openvino-extra.h b/ggml/src/ggml-openvino/ggml-openvino-extra.h
index 3a58b2d02..5a64c67ef 100644
--- a/ggml/src/ggml-openvino/ggml-openvino-extra.h
+++ b/ggml/src/ggml-openvino/ggml-openvino-extra.h
@@ -7,6 +7,7 @@
 #include <CL/cl.h>

 #include <cstdlib>
+#include <map>
 #include <memory>
 #include <openvino/core/node.hpp>
 #include <openvino/runtime/remote_context.hpp>
@@ -22,14 +23,14 @@ enum class ExtraQuantType { F16, Q4_0_C, Q8_1_C, Q4_0_128, Q4_0_64, Q8_0_C, Q8_0

 ov::Core & ov_singleton_core();

-// Get the remote context for the current device (returns empty optional for CPU)
-std::optional<ov::RemoteContext> ggml_openvino_get_remote_context();
+// Get the remote context for the device (returns empty optional for CPU)
+std::optional<ov::RemoteContext> ggml_openvino_get_remote_context(const std::string & device);

 // Get the compile config for the current device
 const ov::AnyMap & ggml_openvino_get_compile_config();

-// Get the OpenCL command queue for GPU operations (returns nullptr for CPU/NPU)
-cl_command_queue ggml_openvino_get_cl_queue();
+// Get the OpenCL command queue of the device for GPU operations (returns nullptr for CPU/NPU)
+cl_command_queue ggml_openvino_get_cl_queue(const std::string & device);

 // Intel USM extension function type
 typedef cl_int(CL_API_CALL * clEnqueueMemFillINTEL_fn)(cl_command_queue queue,
@@ -61,16 +62,21 @@ clEnqueueMemcpyINTEL_fn ggml_openvino_get_clEnqueueMemcpyINTEL();
 // =====================================================
 // Initialized once during backend init from GGML_OPENVINO_DEVICE env var

+struct ggml_openvino_gpu_context {
+    ov::RemoteContext remote_context;
+    cl_command_queue cl_queue = nullptr;
+    size_t max_alloc_size = SIZE_MAX;
+};
+
 struct ggml_openvino_device_config {
     std::string device_name = "CPU";
     std::vector<std::string> available_devices;
+    std::vector<std::string> selected_devices;
     bool is_npu = false;
     bool initialized = false;
-    std::optional<ov::RemoteContext> remote_context;
-    size_t max_alloc_size = SIZE_MAX;
+    std::map<std::string, ggml_openvino_gpu_context> gpu_contexts;
     ov::AnyMap compile_config;
     std::unordered_map<std::string, std::string> environment_variables;
-    cl_command_queue cl_queue = nullptr;
     clEnqueueMemFillINTEL_fn cl_mem_fill_fn = nullptr;
     clEnqueueMemcpyINTEL_fn cl_mem_cpy_fn = nullptr;

@@ -88,7 +94,10 @@ void ggml_openvino_init_device_config();
 const std::string & ggml_openvino_get_device_name();

 // Get all available physical OpenVINO devices
-std::vector<std::string> ggml_openvino_get_available_devices();
+const std::vector<std::string> & ggml_openvino_get_available_devices();
+
+// Check if the device is in GGML_OPENVINO_DEVICE
+bool ggml_openvino_is_selected_device(const std::string & device);

 // Human-readable device name, e.g. "Intel(R) AI Boost (NPU 4000)"; the device id if unavailable
 std::string ggml_openvino_get_device_description(const std::string & device_name);
@@ -96,19 +105,27 @@ std::string ggml_openvino_get_device_description(const std::string & device_name
 // Environment variable accessors. All GGML_OPENVINO_* env vars are read once
 // during backend init and cached on the device config; consumers must go
 // through these helpers (never call ::getenv directly) so behavior stays
-// consistent and centralized.
+// consistent and centralized. A variable must be in the table in
+// ggml-openvino-extra.cpp, which gives its type: backend init stops on a value
+// that does not fit the type (e.g. "1x" for a number, "on" for a toggle).
 //
 // Use ggml_openvino_getenv_str() for string / path values
 // (e.g. GGML_OPENVINO_DEVICE, GGML_OPENVINO_CACHE_DIR). The optional
 // default_value is returned when the var is unset or empty.
 //
 // Use ggml_openvino_getenv_int() for boolean toggles and integer settings.
-// It returns std::atoi(value) when set, otherwise default_value. For
-// boolean use, `if (ggml_openvino_getenv_int(name))` is true iff the value
-// is a non-zero integer (so "0" disables, "1" enables).
+// A toggle is cached as 1 (a positive value) or 0 (zero or a negative value),
+// so `if (ggml_openvino_getenv_int(name))` is true iff it is on.
 const char * ggml_openvino_getenv_str(const char * var, const char * default_value = nullptr);
+// Returns default_value when the variable is unset or empty.
 int ggml_openvino_getenv_int(const char * var, int default_value = 0);

+// Apply plugin properties given as "KEY=VALUE,KEY=VALUE" (GGML_OPENVINO_NPU_CONFIG, GGML_OPENVINO_GPU_CONFIG,
+// GGML_OPENVINO_CPU_CONFIG) to a compile config. They override what the config already holds.
+void ggml_openvino_apply_config_overrides(ov::AnyMap & config, const char * spec, const char * device);
+// The plugin properties the environment sets for the selected device, as "GGML_OPENVINO_GPU_CONFIG=..."; empty if none.
+std::string ggml_openvino_config_overrides_in_use();
+
 // Memory optimization toggles. GGML_OPENVINO_MEMORY_OPTIMIZE is an umbrella
 // switch; the fine-grained env vars still override it when explicitly set.
 bool ggml_openvino_reduce_compile_mem_enabled();
@@ -117,11 +134,105 @@ bool ggml_openvino_release_weights_enabled();
 // Check if running on NPU
 bool ggml_openvino_is_npu();

+// KV-cache slicing: bind the KV cache, and size the KQ masks, at the attended length instead of the full context.
+// One mechanism, two modes:
+//   dynamic graphs (CPU/GPU): the KV input is a view over the cache; the graph is not recompiled.
+//   static graphs (NPU, or any device with GGML_OPENVINO_FORCE_STATIC): the prefill graph is compiled at the aligned
+//   attended length and recompiled when it changes; decode keeps the full context.
+// On by default for all devices; GGML_OPENVINO_KV_SLICE=0 turns it off.
+bool ggml_openvino_kv_slice_enabled();
+
+// On by default: back NPU buffers with Level Zero host memory; GGML_OPENVINO_NPU_L0_HOST_TENSORS=0 turns it off.
+bool ggml_openvino_npu_l0_host_tensors_enabled();
+
+// On by default: build the static prefill mask directly in the input tensor; GGML_OPENVINO_NPU_FAST_MASK=0 turns it off.
+bool ggml_openvino_npu_fast_mask_enabled();
+
+// NPU-only opt-in performance toggles (all default off). Centralized here so each
+// env-var name lives in one place; see docs/backend/OPENVINO.md for semantics.
+bool ggml_openvino_npu_hfa_prefill_enabled();
+bool ggml_openvino_npu_decode_unfold_enabled();
+bool ggml_openvino_npu_parallel_compile_enabled();
+
+// GGML_OPENVINO_NPU_BLOB_PREBUILD (default 1): static NPU graphs attend a KV window of 1024*2^k rows (clamped to ctx)
+// instead of the whole ctx, with one decode model per window, all built/imported with the prefill/decode pair;
+// with head-major KV the prefill models of every window are also exported and preloaded.
+// Returns the window base (1024), 0 = off (GGML_OPENVINO_NPU_BLOB_PREBUILD=0).
+int ggml_openvino_npu_decode_bucket();
+
+// Rows a static prefill attends with GGML_OPENVINO_KV_SLICE: the attended length rounded up to a power of two,
+// clamped to ctx.
+int ggml_openvino_static_kv_window(int rows, int ctx);
+
+// On-demand NPU KV cache (GGML_OPENVINO_NPU_KV_ON_DEMAND): the ggml KV buffer stays non-resident and the
+// NPU binds per-tensor L0 storage that grows (base*2^k rows, base = decode bucket or 1024) with the rows in use.
+bool ggml_openvino_npu_kv_on_demand_enabled();
+// On-demand KV with BLOB_PREBUILD and KV_SLICE: K storage is [heads][rows][head_size] and V storage is
+// [heads][head_size][rows] (head-major with heads * head_size channels); every static graph binds the whole storage
+// as [1, heads, rows, head_size] / [1, heads, head_size, rows], and attention uses V^T.
+bool ggml_openvino_npu_kv_head_major_enabled();
+
+// NPU blob cache modes that keep the raw GGUF weights and skip requantization on a warm hit:
+// GGML_OPENVINO_SELF_CONTAINED_BLOB (weights embedded in each blob) or
+// GGML_OPENVINO_NPU_WEIGHTLESS_BANK (weightless blobs plus one shared weights file).
+bool ggml_openvino_npu_raw_weight_cache_enabled();
+bool ggml_openvino_npu_weightless_bank_enabled();
+void ggml_openvino_kv_storage_register(void * data, size_t row_bytes, size_t rows);
+void ggml_openvino_kv_storage_unregister(const void * base, size_t size);
+// Storage address backing ggml address `ptr` with at least `bytes` valid (grown if needed);
+// nullptr if `ptr` is not inside an on-demand KV tensor.
+void * ggml_openvino_kv_storage(const void * ptr, size_t bytes);
+// Grows the storage of the on-demand KV tensor at `data` to at least `rows` rows (head-major when heads > 0);
+// true if existing storage moved.
+bool ggml_openvino_kv_storage_reserve(const void * data, size_t rows, size_t heads = 0);
+// Rows currently backed by storage for the on-demand KV tensor at `data` (0 if none).
+size_t ggml_openvino_kv_storage_capacity(const void * data);
+enum ggml_openvino_kv_copy_op { GGML_OPENVINO_KV_READ, GGML_OPENVINO_KV_WRITE, GGML_OPENVINO_KV_FILL };
+// Host access by ggml address in the ggml row layout (FILL uses *host as the byte value); false if not on-demand KV.
+bool ggml_openvino_kv_storage_copy(const void * ptr, void * host, size_t size, ggml_openvino_kv_copy_op op);
+// Writes rows [dst_row, dst_row + n) of head-major storage of the KV tensor at `data` from `src`, laid out as
+// [heads][src_stride] entries of (row_bytes / heads) bytes, starting at entry src_col of every head.
+// False if `data` is not head-major on-demand KV.
+bool ggml_openvino_kv_storage_write_rows(const void * data, const void * src, size_t src_stride, size_t src_col,
+                                         size_t dst_row, size_t n);
+void ggml_openvino_kv_storage_clear(const void * base, size_t size, uint8_t value);
+
 // Check if running on a GPU (GPU, GPU.0, GPU.1, ...)
 bool ggml_openvino_is_gpu();

+// Read n_seq_active and attention_size(_swa) from the KQ mask shape (off with GGML_OPENVINO_DISABLE_SHAPE_FROM_MASK=1)
+bool ggml_openvino_shape_from_mask_enabled();
+
+// GDN q pre-scale for eligible graphs on the NPU static path. On by default;
+// GGML_OPENVINO_GDN_Q_SCALE=0 (or a negative or non-numeric value) turns it off.
+bool ggml_openvino_gdn_prescale_enabled();
+
+// GDN q pre-scale factor (NPU static path, keeps the tiny GDN output above the
+// f16 subnormal range). Defaults to 64.0f and is clamped to a minimum of
+// 64.0f. Only consulted when ggml_openvino_gdn_prescale_enabled() is true.
+float ggml_openvino_gdn_q_scale();
+
+// Keep the GDN state in the OV op layout [B, H_v, key_dim, value_dim], skipping two transposes per layer.
+// Only the OpenVINO backend may then read the state. NPU only, enabled when GGML_OPENVINO_GDN_NATIVE_STATE is a positive integer.
+bool ggml_openvino_gdn_native_state_enabled();
+
+// True when the device is an NPU with DEVICE_ARCHITECTURE 4000 (Lunar Lake).
+bool ggml_openvino_npu_is_4000();
+
+// Translate GATED_DELTA_NET into a MatMul decomposition instead of the GatedDeltaNet op: chunked for multi-token
+// graphs (prefill), one step for single-token graphs (decode).
+// The NPU4000 GDN prefill kernel drives the DPU from SHAVE, which the LNL firmware does not support. On Linux the
+// decode kernel loses the device too (ZE_RESULT_ERROR_DEVICE_LOST).
+// Default: on for NPU4000 only. GGML_OPENVINO_GDN_DECOMPOSE=0 turns it off, a positive integer forces it on.
+// Snapshot (MTP) graphs still use the op.
+bool ggml_openvino_gdn_decompose_enabled();
+
+// Tokens per chunk of the decomposed GATED_DELTA_NET: GGML_OPENVINO_GDN_DECOMPOSE_CHUNK,
+// a power of 2 in [2, 128], default 16.
+int ggml_openvino_gdn_decompose_chunk();
+
 // Largest single memory object the device can allocate, SIZE_MAX when there is no known limit
-size_t ggml_openvino_max_alloc_size();
+size_t ggml_openvino_max_alloc_size(const std::string & device);

 // Host weight-buffer release (GGML_OPENVINO_RELEASE_WEIGHTS, GPU only).
 // register: record a host weight buffer (idempotent per data pointer).
@@ -134,6 +245,11 @@ bool ggml_openvino_weight_buffers_released();
 // Get requantization type for a tensor type (returns nullopt if no requant needed)
 std::optional<ExtraQuantType> ggml_openvino_get_requant_type(const ggml_tensor * tensor, bool no_requant = false);

+// True if an asymmetric weight keeps an exact f16 zero point instead of a rounded integer one.
+// Always true for MoE expert weights (llama.cpp's *_exps / *_chexps tensors). Other weights need
+// GGML_OPENVINO_EXACT_ZERO_POINT.
+bool ggml_openvino_use_exact_zero_point(const ggml_tensor * tensor);
+
 // =====================================================
 // OpenVINO Tensor Extra Types
 // =====================================================
@@ -180,10 +296,12 @@ struct ggml_openvino_quantized_weight_extra : public ggml_openvino_extra_base {
 // Extra data for KV cache / compute tensors - stores ov::Tensor for infer_request
 struct ggml_openvino_tensor_extra : public ggml_openvino_extra_base {
     std::shared_ptr<ov::Tensor> tensor;  // For direct use with infer_request
+    const void * data;                   // tensor->data that `tensor` wraps

-    explicit ggml_openvino_tensor_extra(std::shared_ptr<ov::Tensor> t) :
+    ggml_openvino_tensor_extra(std::shared_ptr<ov::Tensor> t, const void * data) :
         ggml_openvino_extra_base(Type::TENSOR),
-        tensor(std::move(t)) {}
+        tensor(std::move(t)),
+        data(data) {}
 };

 // =====================================================
@@ -212,7 +330,9 @@ struct ggml_openvino_extracted_layout {
 // Calculate the buffer layout for extracted quantized data
 ggml_openvino_extracted_layout ggml_openvino_get_extracted_layout(const ggml_tensor * tensor, bool use_bias = false);

-ggml_openvino_tensor_extra * ggml_openvino_create_tensor_extra(const ggml_tensor * tensor, bool is_remote);
+ggml_openvino_tensor_extra * ggml_openvino_create_tensor_extra(const ggml_tensor * tensor,
+                                                               bool is_remote,
+                                                               const std::string & device);

 // Check if a tensor's buffer uses remote (device) memory (e.g. GPU USM)
 bool ggml_openvino_buffer_is_remote(const ggml_tensor * tensor);
diff --git a/ggml/src/ggml-openvino/ggml-openvino.cpp b/ggml/src/ggml-openvino/ggml-openvino.cpp
index 810a30a2e..a7f5d177f 100644
--- a/ggml/src/ggml-openvino/ggml-openvino.cpp
+++ b/ggml/src/ggml-openvino/ggml-openvino.cpp
@@ -21,6 +21,9 @@
 #include <memory>
 #include <mutex>
 #include <openvino/core/type/element_type.hpp>
+#include <openvino/op/constant.hpp>
+#include <openvino/op/convert.hpp>
+#include <openvino/op/parameter.hpp>
 #include <openvino/openvino.hpp>
 #include <openvino/runtime/allocator.hpp>
 #include <openvino/runtime/intel_gpu/ocl/ocl.hpp>
@@ -31,6 +34,7 @@
 #include <map>
 #include <set>
 #include <string>
+#include <tuple>
 #include <vector>

 #ifdef _WIN32
@@ -39,6 +43,7 @@
 #        define NOMINMAX
 #    endif
 #    include <windows.h>
+#    include <psapi.h>
 #else
 #    include <sys/mman.h>
 #    include <unistd.h>
@@ -73,10 +78,17 @@ struct ggml_backend_openvino_buffer_context {
     size_t size;
     bool is_remote;

-    // File-backed spill or cache-only virtual memory.
+    // OS mapping backing the buffer: anonymous memory, or the file-backed spill.
     void * spill_mapping = nullptr;
     size_t spill_size = 0;

+    // Set for a model buffer that directly wraps the loader's GGUF mmap.
+    bool external_memory = false;
+
+    // KV buffer whose tensors are backed by on-demand L0 storage (GGML_OPENVINO_NPU_KV_ON_DEMAND); its own
+    // memory is never touched, so it stays demand-zero and non-resident.
+    bool kv_on_demand = false;
+
     // Wrapping of the buffer
     std::shared_ptr<ov::Tensor> ov_buffer;

@@ -86,9 +98,23 @@ struct ggml_backend_openvino_buffer_context {
     std::vector<ggml_openvino_source_mapping> source_mappings;

     // Used for re-allocation on device for kvcache
-    void * data_prev;
+    void * data_prev = nullptr;

-    ggml_backend_openvino_buffer_context(int device, size_t size, bool is_remote = false) :
+    ggml_backend_openvino_buffer_context(int device, void * external_data, size_t size) :
+        device(device),
+        name(std::string(GGML_OPENVINO_NAME) + std::to_string(device)),
+        id([]() {
+            static std::atomic<size_t> next_id{1};
+            return next_id.fetch_add(1);
+        }()),
+        data(external_data),
+        size(size),
+        is_remote(false),
+        external_memory(true) {
+        ov_buffer = std::make_shared<ov::Tensor>(ov::element::u8, ov::Shape{size}, data);
+    }
+
+    ggml_backend_openvino_buffer_context(int device, size_t size, bool is_remote = false, bool lazy = false) :
         device(device),
         name(std::string(GGML_OPENVINO_NAME) + std::to_string(device)),
         id([]() {
@@ -102,36 +128,54 @@ struct ggml_backend_openvino_buffer_context {
             return;
         }

-        const auto & device_name = ggml_openvino_get_device_name();
+        const auto & device_name = ggml_openvino_get_available_devices()[device];

         if (is_remote) {
             GGML_ASSERT(ggml_openvino_is_gpu());
-            auto remote_context = ggml_openvino_get_remote_context();
+            auto remote_context = ggml_openvino_get_remote_context(device_name);
             auto gpu_context = remote_context->as<ov::intel_gpu::ocl::ClContext>();
             ov::intel_gpu::ocl::USMTensor usm_tensor =
                 gpu_context.create_usm_device_tensor(ov::element::u8, ov::Shape{size});
             data = usm_tensor.get();
             ov_buffer = std::make_shared<ov::intel_gpu::ocl::USMTensor>(std::move(usm_tensor));
         } else {
-#ifdef _WIN32
-            if (ggml_openvino_model_cache_only()) {
-                data = spill_mapping = VirtualAlloc(nullptr, size, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE);
-                if (data == nullptr) {
-                    return;
+            // In strict warm-mmap mode the model weights are supplied via buffer_from_host_ptr,
+            // so every alloc_buffer here is graph scratch / KV. The large prefill compute buffer
+            // is never touched on the host by the on-device fused OV model, yet an L0 host tensor
+            // commits/zeroes it eagerly (fully resident). For such large buffers, reserve
+            // demand-zero pages instead (no L0, no memset) so they stay non-resident. KV and small
+            // I/O buffers keep the L0 host tensor for zero-copy import (decode throughput).
+            const bool strict_mmap = device_name == "NPU" &&
+                                     ggml_openvino_npu_raw_weight_cache_enabled() &&
+                                     ggml_openvino_getenv_int("GGML_OPENVINO_SELF_CONTAINED_MMAP") &&
+                                     ggml_openvino_getenv_str("GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR");
+            // 2 GiB keeps a large-context KV cache (~1 GiB at ctx 8K) on L0 while the bigger prefill
+            // compute buffer still goes lazy.
+            const size_t lazy_host_threshold =
+                (size_t) ggml_openvino_getenv_int("GGML_OPENVINO_LAZY_HOST_THRESHOLD_MB", 2048) * 1024 * 1024;
+            const bool lazy_host_alloc = lazy || (strict_mmap && size >= lazy_host_threshold);
+            // Prefer importable Level Zero host memory on NPU: the plugin imports it without an
+            // extra host-to-device copy each infer.
+            if (!lazy_host_alloc && ggml_openvino_is_npu() && ggml_openvino_npu_l0_host_tensors_enabled() &&
+                !ggml_openvino_model_cache_only()) {
+                try {
+                    auto npu_context = ov_singleton_core().get_default_context("NPU");
+                    auto host_tensor = npu_context.create_host_tensor(ov::element::u8, ov::Shape{size});
+                    data = host_tensor.data();
+                    memset(data, 0, size);
+                    ov_buffer = std::make_shared<ov::Tensor>(std::move(host_tensor));
+                } catch (const std::exception & e) {
+                    GGML_LOG_WARN("%s: L0 host tensor allocation of %zu bytes failed (%s); using anonymous memory\n",
+                                  __func__, size, e.what());
+                    data = nullptr;
                 }
-                spill_size = size;
-                ov_buffer = std::make_shared<ov::Tensor>(ov::element::u8, ov::Shape{size}, data);
+            }
+            if (data != nullptr) {
+                // backed by the Level Zero host tensor above
             } else
-#else
-            if (ggml_openvino_model_cache_only()) {
-                void * m = mmap(nullptr, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
-                if (m == MAP_FAILED) {
-                    return;
-                }
-                data = spill_mapping = m;
-                spill_size = size;
-                ov_buffer = std::make_shared<ov::Tensor>(ov::element::u8, ov::Shape{size}, data);
-            } else if (const char * spill_dir = ggml_openvino_getenv_str("GGML_OPENVINO_SPILL_DIR")) {
+#ifndef _WIN32
+            if (const char * spill_dir =
+                    ggml_openvino_model_cache_only() ? nullptr : ggml_openvino_getenv_str("GGML_OPENVINO_SPILL_DIR")) {
                 // Disk-backed weight buffer: back the repacked weights with a temp file via MAP_SHARED
                 // instead of anonymous memory. Anonymous pages can only be evicted to swap, so the
                 // repacked buffer stays pinned alongside the mmap'd source and both are resident at once
@@ -171,13 +215,27 @@ struct ggml_backend_openvino_buffer_context {
 #endif
             {
 #ifdef _WIN32
-                if (ggml_openvino_getenv_str("GGML_OPENVINO_SPILL_DIR")) {
+                if (!ggml_openvino_model_cache_only() && ggml_openvino_getenv_str("GGML_OPENVINO_SPILL_DIR")) {
                     GGML_LOG_WARN("%s: GGML_OPENVINO_SPILL_DIR is not supported on Windows, ignoring\n", __func__);
                 }
 #endif
-                data = ggml_aligned_malloc(size);
-                GGML_ASSERT(data);
-                memset(data, 0, size);
+                // Anonymous OS memory reads as zero without being touched, so a buffer becomes resident only as it
+                // is written: a weight buffer tensor by tensor while the model loads, the host-untouched prefill
+                // compute buffer never.
+#ifdef _WIN32
+                data = VirtualAlloc(nullptr, size, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE);
+                if (data == nullptr) {
+                    return;
+                }
+#else
+                void * m = mmap(nullptr, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+                if (m == MAP_FAILED) {
+                    return;
+                }
+                data = m;
+#endif
+                spill_mapping = data;
+                spill_size = size;
                 ov_buffer = std::make_shared<ov::Tensor>(ov::element::u8, ov::Shape{size}, data);
             }
         }
@@ -202,17 +260,15 @@ struct ggml_backend_openvino_buffer_context {
             delete pair.second;
         }
         tensor_extras.clear();
-#ifdef _WIN32
+        if (kv_on_demand) {
+            ggml_openvino_kv_storage_unregister(data, size);
+        }
         if (spill_mapping != nullptr) {
+#ifdef _WIN32
             VirtualFree(spill_mapping, 0, MEM_RELEASE);
-        } else
 #else
-        if (spill_mapping != nullptr) {
             munmap(spill_mapping, spill_size);
-        } else
 #endif
-        if (!is_remote && data != nullptr) {
-            ggml_aligned_free(data, size);
         }
     }
 };
@@ -228,7 +284,7 @@ struct ggml_backend_openvino_buffer_type_context {
 // Host weight-buffer release (GGML_OPENVINO_RELEASE_WEIGHTS)
 // =====================================================
 // The OpenVINO weight Constants are zero-copy views into the host buffers
-// allocated here (ggml_aligned_malloc, anonymous memory). On GPU the plugin
+// allocated here (anonymous memory). On GPU the plugin
 // holds its own device copy after compile_model, so the host pages are dead
 // weight for inference and can be dropped to reclaim RSS (~weights size).
 //
@@ -316,6 +372,12 @@ static bool is_stateful_enabled() {
     return ggml_openvino_getenv_int("GGML_OPENVINO_STATEFUL_EXECUTION") != 0;
 }

+static bool is_self_contained_mmap_enabled() {
+    return ggml_openvino_is_npu() && ggml_openvino_npu_raw_weight_cache_enabled() &&
+           ggml_openvino_getenv_int("GGML_OPENVINO_SELF_CONTAINED_MMAP") &&
+           ggml_openvino_getenv_str("GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR");
+}
+
 static enum ggml_status ggml_backend_openvino_buffer_init_tensor(ggml_backend_buffer_t buffer, ggml_tensor * tensor) {
     // GGML_LOG_DEBUG("%s: buffer usage=%d, tensor name=%s\n", __func__, buffer->usage, tensor->name);
     ggml_backend_openvino_buffer_context * ctx = (ggml_backend_openvino_buffer_context *) buffer->context;
@@ -333,6 +395,26 @@ static enum ggml_status ggml_backend_openvino_buffer_init_tensor(ggml_backend_bu
         tensor->data = (char *) ctx->data + ((char *) tensor->data - (char *) data_prev);
     }

+    // On-demand NPU KV: swap the eagerly committed buffer for demand-zero memory at its first KV tensor
+    // and register every KV tensor; the NPU binds grown L0 storage instead.
+    const bool is_kv_tensor = strncmp(tensor->name, "cache_k_l", 9) == 0 || strncmp(tensor->name, "cache_v_l", 9) == 0;
+    if (is_kv_tensor && tensor->view_src == nullptr && !ctx->is_remote && ggml_openvino_npu_kv_on_demand_enabled() &&
+        tensor->ne[2] == 1 && tensor->ne[3] == 1 && tensor->nb[1] == ggml_row_size(tensor->type, tensor->ne[0])) {
+        if (!ctx->kv_on_demand && ctx->tensor_extras.empty() && tensor->data == ctx->data) {
+            auto device = ctx->device;
+            auto size = ctx->size;
+            auto * data_prev = ctx->data;
+            delete ctx;
+            ctx = new ggml_backend_openvino_buffer_context(device, size, false, true);
+            ctx->kv_on_demand = true;
+            buffer->context = ctx;
+            tensor->data = (char *) ctx->data + ((char *) tensor->data - (char *) data_prev);
+        }
+        if (ctx->kv_on_demand) {
+            ggml_openvino_kv_storage_register(tensor->data, tensor->nb[1], tensor->ne[1]);
+        }
+    }
+
     // Views share the extra from view_src
     if (tensor->view_src != nullptr) {
         GGML_ASSERT(tensor->view_src->buffer->buft == buffer->buft);
@@ -347,7 +429,8 @@ static enum ggml_status ggml_backend_openvino_buffer_init_tensor(ggml_backend_bu
             if (!ggml_are_same_shape(tensor, tensor->view_src) && ggml_is_contiguous(tensor) &&
                 !ggml_is_quantized(tensor->type) && tensor->data != nullptr && ggml_nbytes(tensor) > 0) {
                 if (ggml_openvino_tensor_extra * extra =
-                        ggml_openvino_create_tensor_extra(tensor, ctx->is_remote)) {
+                        ggml_openvino_create_tensor_extra(tensor, ctx->is_remote,
+                                                          ggml_openvino_get_available_devices()[ctx->device])) {
                     auto it = ctx->tensor_extras.find(tensor);
                     if (it != ctx->tensor_extras.end()) {
                         delete it->second;
@@ -364,8 +447,9 @@ static enum ggml_status ggml_backend_openvino_buffer_init_tensor(ggml_backend_bu

     ctx = (ggml_backend_openvino_buffer_context *) buffer->context;

-    if (tensor->data != nullptr && !ggml_is_quantized(tensor->type)) {
-        ggml_openvino_tensor_extra * extra = ggml_openvino_create_tensor_extra(tensor, ctx->is_remote);
+    if (tensor->data != nullptr && !ggml_is_quantized(tensor->type) && ggml_nbytes(tensor) > 0) {
+        ggml_openvino_tensor_extra * extra = ggml_openvino_create_tensor_extra(
+            tensor, ctx->is_remote, ggml_openvino_get_available_devices()[ctx->device]);
         if (extra != nullptr) {
             auto it = ctx->tensor_extras.find(tensor);
             if (it != ctx->tensor_extras.end()) {
@@ -390,12 +474,12 @@ static void ggml_backend_openvino_buffer_memset_tensor(ggml_backend_buffer_t buf

     if (ctx->is_remote) {
         // For remote (device) buffers, use OpenCL USM memfill
-        cl_command_queue queue = ggml_openvino_get_cl_queue();
+        cl_command_queue queue = ggml_openvino_get_cl_queue(ggml_openvino_get_available_devices()[ctx->device]);
         auto mem_fill_fn = ggml_openvino_get_clEnqueueMemFillINTEL();
         if (mem_fill_fn != nullptr) {
             uint8_t pattern = value;
-            cl_int err = mem_fill_fn(queue, (char *) tensor->data + offset, &pattern, sizeof(pattern), size, 0, nullptr,
-                                     nullptr);
+            cl_int err = mem_fill_fn(queue, (char *) tensor->data + offset, &pattern,
+                                     sizeof(pattern), size, 0, nullptr, nullptr);
             if (err != CL_SUCCESS) {
                 GGML_LOG_ERROR("%s: clEnqueueMemFillINTEL failed with error %d\n", __func__, err);
             }
@@ -404,8 +488,91 @@ static void ggml_backend_openvino_buffer_memset_tensor(ggml_backend_buffer_t buf
             GGML_LOG_ERROR("%s: clEnqueueMemFillINTEL not available for GPU buffer\n", __func__);
         }
     } else {
-        memset((char *) tensor->data + offset, value, size);
+        if (!ctx->kv_on_demand ||
+            !ggml_openvino_kv_storage_copy((char *) tensor->data + offset, &value, size, GGML_OPENVINO_KV_FILL)) {
+            memset((char *) tensor->data + offset, value, size);
+        }
+    }
+}
+
+// Inputs are set on every inference: keep the tensor extra when it still wraps the same memory, shape and type.
+static bool ggml_openvino_extra_unchanged(const ggml_backend_openvino_buffer_context * ctx,
+                                          const ggml_tensor * tensor) {
+    auto it = ctx->tensor_extras.find(const_cast<ggml_tensor *>(tensor));
+    if (it == ctx->tensor_extras.end() || it->second == nullptr || tensor->extra != it->second ||
+        it->second->type != ggml_openvino_extra_base::Type::TENSOR) {
+        return false;
+    }
+    const auto * extra = static_cast<const ggml_openvino_tensor_extra *>(it->second);
+    const auto & t = extra->tensor;
+    if (extra->data != tensor->data || t == nullptr || t->get_element_type() != GgmlOvDecoder::get_ov_type(tensor)) {
+        return false;
+    }
+    const auto & shape = t->get_shape();
+    if (shape.size() != GGML_MAX_DIMS) {
+        return false;
+    }
+    for (int i = 0; i < GGML_MAX_DIMS; ++i) {
+        if (shape[GGML_MAX_DIMS - 1 - i] != static_cast<size_t>(tensor->ne[i])) {
+            return false;
+        }
+    }
+    return true;
+}
+
+#ifdef _WIN32
+// QueryWorkingSetEx reports Locked only for pages in the working set; a locked page always is.
+static bool ggml_openvino_range_has_locked_pages(uintptr_t first, uintptr_t last, uintptr_t page) {
+    std::vector<PSAPI_WORKING_SET_EX_INFORMATION> pages(std::min<size_t>((last - first) / page, 4096));
+    for (uintptr_t addr = first; addr < last;) {
+        const size_t n = std::min<size_t>(pages.size(), (last - addr) / page);
+        for (size_t i = 0; i < n; i++) {
+            pages[i].VirtualAddress = reinterpret_cast<void *>(addr + i * page);
+        }
+        if (!QueryWorkingSetEx(GetCurrentProcess(), pages.data(), DWORD(n * sizeof(pages[0])))) {
+            return true;  // unknown, so keep the pages
+        }
+        for (size_t i = 0; i < n; i++) {
+            if (pages[i].VirtualAttributes.Valid && pages[i].VirtualAttributes.Locked) {
+                return true;
+            }
+        }
+        addr += n * page;
+    }
+    return false;
+}
+#endif
+
+// A weight's GGUF bytes have been converted into its OpenVINO buffer. Pages of a read-only file mapping are clean and
+// come back from the file if they are read again, so release them instead of letting them compete with the buffer for
+// RAM while the model loads. Only whole pages are released, and locked pages (--load-mode mlock) are kept.
+static void ggml_openvino_release_source_pages(ggml_backend_openvino_buffer_context * ctx, const void * data, size_t size) {
+    const ggml_openvino_source_mapping * mapping = ggml_openvino_find_source_mapping(data, size, ctx->source_mappings);
+    if (mapping == nullptr || !mapping->read_only) {
+        return;
+    }
+#ifdef _WIN32
+    SYSTEM_INFO info;
+    GetSystemInfo(&info);
+    const uintptr_t page = info.dwPageSize;
+#else
+    const uintptr_t page = (uintptr_t) sysconf(_SC_PAGESIZE);
+#endif
+    const uintptr_t first = ((uintptr_t) data + page - 1) & ~(page - 1);
+    const uintptr_t last = ((uintptr_t) data + size) & ~(page - 1);
+    if (last <= first) {
+        return;
     }
+#ifdef _WIN32
+    // VirtualUnlock removes pages that are not locked from the working set, which moves clean file pages to the standby
+    // list, and then reports ERROR_NOT_LOCKED. On locked pages it would unlock them instead, so skip such a range.
+    if (!ggml_openvino_range_has_locked_pages(first, last, page)) {
+        VirtualUnlock(reinterpret_cast<void *>(first), last - first);
+    }
+#elif defined(__linux__)
+    // MADV_DONTNEED fails on locked (mlock) memory and leaves it resident
+    madvise(reinterpret_cast<void *>(first), last - first, MADV_DONTNEED);
+#endif
 }

 static void ggml_backend_openvino_buffer_set_tensor(ggml_backend_buffer_t buffer,
@@ -438,6 +605,13 @@ static void ggml_backend_openvino_buffer_set_tensor(ggml_backend_buffer_t buffer
     bool is_supported_weight_shape = is_2d || (tensor->ne[3] == 1 && ggml_is_quantized(tensor->type));

     if (is_weight_buffer && is_full_tensor_set && is_supported_weight_shape) {
+        if (ggml_openvino_is_npu() && ggml_openvino_npu_raw_weight_cache_enabled() &&
+            ggml_openvino_getenv_str("GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR")) {
+            // Keep raw bytes for cache fingerprint validation and compile fallback; the imported
+            // self-contained blob supplies the runtime weights, so eager requant is unnecessary.
+            memcpy(tensor->data, data, size);
+            return;
+        }
         try {
             auto result = process_weight_tensor(tensor, data, tensor->data);
             result.weight_node->set_friendly_name(tensor->name);
@@ -489,6 +663,9 @@ static void ggml_backend_openvino_buffer_set_tensor(ggml_backend_buffer_t buffer
                 ggml_openvino_register_weight_buffer(ctx->data, ctx->size);
             }

+            if (!ctx->external_memory) {
+                ggml_openvino_release_source_pages(ctx, data, size);
+            }
         } catch (const std::exception & e) {
             GGML_LOG_ERROR("%s: failed to process weight tensor for %s: %s\n", __func__, tensor->name, e.what());
             memcpy((char *) tensor->data + offset, data, size);
@@ -496,22 +673,28 @@ static void ggml_backend_openvino_buffer_set_tensor(ggml_backend_buffer_t buffer
     } else {
         // Non-weight tensor (KV cache, activations, etc.) - copy data. test-backend-ops also goes here
         if (ctx->is_remote) {
-            cl_command_queue queue = ggml_openvino_get_cl_queue();
+            cl_command_queue queue = ggml_openvino_get_cl_queue(ggml_openvino_get_available_devices()[ctx->device]);
             auto mem_cpy_fn = ggml_openvino_get_clEnqueueMemcpyINTEL();
             if (mem_cpy_fn != nullptr) {
-                cl_int err =
-                    mem_cpy_fn(queue, CL_TRUE, (char *) tensor->data + offset, data, size, 0, nullptr, nullptr);
+                cl_int err = mem_cpy_fn(queue, CL_TRUE, (char *) tensor->data + offset, data, size,
+                                        0, nullptr, nullptr);
                 if (err != CL_SUCCESS) {
                     GGML_LOG_ERROR("%s: clEnqueueMemcpyINTEL failed with error %d\n", __func__, err);
                 }
             } else {
                 GGML_LOG_ERROR("%s: clEnqueueMemcpyINTEL not available for GPU buffer\n", __func__);
             }
-        } else {
+        } else if (!ctx->kv_on_demand || !ggml_openvino_kv_storage_copy((char *) tensor->data + offset,
+                                                                        const_cast<void *>(data), size,
+                                                                        GGML_OPENVINO_KV_WRITE)) {
             memcpy((char *) tensor->data + offset, data, size);
         }

-        ggml_openvino_tensor_extra * extra = ggml_openvino_create_tensor_extra(tensor, ctx->is_remote);
+        if (ggml_openvino_extra_unchanged(ctx, tensor)) {
+            return;
+        }
+        ggml_openvino_tensor_extra * extra = ggml_openvino_create_tensor_extra(
+            tensor, ctx->is_remote, ggml_openvino_get_available_devices()[ctx->device]);
         if (extra == nullptr) {
             // GGML_LOG_ERROR("%s: failed to create tensor extra for %s\n", __func__, tensor->name);
             return;
@@ -526,6 +709,64 @@ static void ggml_backend_openvino_buffer_set_tensor(ggml_backend_buffer_t buffer
     }
 }

+// Evaluates a weight node (Constant -> Convert -> [Subtract] -> Multiply -> [Reshape]) on the host.
+static ov::Tensor ggml_openvino_eval_weight_node(const ov::Output<ov::Node> & output) {
+    const auto node = output.get_node_shared_ptr();
+    if (const auto constant = ov::as_type_ptr<ov::op::v0::Constant>(node)) {
+        return constant->get_tensor_view();
+    }
+    ov::TensorVector inputs;
+    for (const auto & input : node->input_values()) {
+        inputs.push_back(ggml_openvino_eval_weight_node(input));
+    }
+    ov::TensorVector outputs;
+    for (size_t i = 0; i < node->get_output_size(); i++) {
+        outputs.emplace_back(node->get_output_element_type(i), node->get_output_shape(i));
+    }
+    if (!node->evaluate(outputs, inputs)) {
+        OPENVINO_THROW("cannot evaluate ", node->get_type_name());
+    }
+    return outputs[output.get_index()];
+}
+
+// set_tensor converts a quantized weight in place (extracted or requantized layout), so tensor->data
+// no longer holds the ggml bytes. Rebuild them from the weight node: dequantize, then quantize back
+// to the tensor type. The result is close to the original data, but not bit-exact.
+static bool ggml_openvino_weight_to_ggml(const ggml_tensor * tensor, std::vector<uint8_t> & out) {
+    const auto * extra = static_cast<const ggml_openvino_extra_base *>(tensor->extra);
+    std::shared_ptr<ov::Node> weight_node;
+    if (extra->type == ggml_openvino_extra_base::Type::QUANTIZED_WEIGHT) {
+        weight_node = static_cast<const ggml_openvino_quantized_weight_extra *>(extra)->weight_node;
+    } else if (extra->type == ggml_openvino_extra_base::Type::WEIGHT) {
+        weight_node = static_cast<const ggml_openvino_weight_extra *>(extra)->weight_node;
+    }
+    if (weight_node == nullptr || ggml_quantize_requires_imatrix(tensor->type)) {
+        return false;
+    }
+    try {
+        ov::Tensor values = ggml_openvino_eval_weight_node(weight_node->output(0));
+        if (values.get_size() != (size_t) ggml_nelements(tensor)) {
+            return false;
+        }
+        if (values.get_element_type() != ov::element::f32) {
+            auto param = std::make_shared<ov::op::v0::Parameter>(values.get_element_type(), values.get_shape());
+            auto convert = std::make_shared<ov::op::v0::Convert>(param, ov::element::f32);
+            ov::TensorVector f32{ov::Tensor(ov::element::f32, values.get_shape())};
+            if (!convert->evaluate(f32, {values})) {
+                return false;
+            }
+            values = f32[0];
+        }
+        out.resize(ggml_nbytes(tensor));
+        ggml_quantize_chunk(tensor->type, values.data<float>(), out.data(), 0, ggml_nrows(tensor), tensor->ne[0],
+                            nullptr);
+        return true;
+    } catch (const std::exception & e) {
+        GGML_LOG_WARN("%s: cannot rebuild %s: %s\n", __func__, tensor->name, e.what());
+        return false;
+    }
+}
+
 static void ggml_backend_openvino_buffer_get_tensor(ggml_backend_buffer_t buffer,
                                                     const ggml_tensor * tensor,
                                                     void * data,
@@ -539,20 +780,30 @@ static void ggml_backend_openvino_buffer_get_tensor(ggml_backend_buffer_t buffer
         GGML_ABORT("ggml-openvino: cannot read unloaded weights in cache-only mode");
     }

+    if (ggml_is_quantized(tensor->type) && tensor->view_src == nullptr && tensor->extra != nullptr &&
+        static_cast<const ggml_openvino_extra_base *>(tensor->extra)->type != ggml_openvino_extra_base::Type::TENSOR) {
+        std::vector<uint8_t> rebuilt;
+        if (ggml_openvino_weight_to_ggml(tensor, rebuilt)) {
+            memcpy(data, rebuilt.data() + offset, size);
+            return;
+        }
+    }
+
     if (ctx->is_remote) {
         // For remote (device) buffers, use OpenCL USM memcpy (device-to-host)
-        cl_command_queue queue = ggml_openvino_get_cl_queue();
+        cl_command_queue queue = ggml_openvino_get_cl_queue(ggml_openvino_get_available_devices()[ctx->device]);
         auto mem_cpy_fn = ggml_openvino_get_clEnqueueMemcpyINTEL();
         if (mem_cpy_fn != nullptr) {
-            cl_int err =
-                mem_cpy_fn(queue, CL_TRUE, data, (const char *) tensor->data + offset, size, 0, nullptr, nullptr);
+            cl_int err = mem_cpy_fn(queue, CL_TRUE, data, (const char *) tensor->data + offset,
+                                    size, 0, nullptr, nullptr);
             if (err != CL_SUCCESS) {
                 GGML_LOG_ERROR("%s: clEnqueueMemcpyINTEL failed with error %d\n", __func__, err);
             }
         } else {
             GGML_LOG_ERROR("%s: clEnqueueMemcpyINTEL not available for GPU buffer\n", __func__);
         }
-    } else {
+    } else if (!ctx->kv_on_demand ||
+               !ggml_openvino_kv_storage_copy((const char *) tensor->data + offset, data, size, GGML_OPENVINO_KV_READ)) {
         memcpy(data, (const char *) tensor->data + offset, size);
     }
 }
@@ -566,7 +817,7 @@ static bool ggml_backend_openvino_buffer_cpy_tensor(ggml_backend_buffer_t buffer

     if (ctx->is_remote) {
         // For remote (device) buffers, use OpenCL USM memcpy
-        cl_command_queue queue = ggml_openvino_get_cl_queue();
+        cl_command_queue queue = ggml_openvino_get_cl_queue(ggml_openvino_get_available_devices()[ctx->device]);
         auto mem_cpy_fn = ggml_openvino_get_clEnqueueMemcpyINTEL();
         if (mem_cpy_fn == nullptr) {
             GGML_LOG_ERROR("%s: clEnqueueMemcpyINTEL not available for GPU buffer\n", __func__);
@@ -574,7 +825,8 @@ static bool ggml_backend_openvino_buffer_cpy_tensor(ggml_backend_buffer_t buffer
         }
         // Can copy from host to device
         if (ggml_backend_buffer_is_host(src->buffer)) {
-            cl_int err = mem_cpy_fn(queue, CL_TRUE, dst->data, src->data, ggml_nbytes(src), 0, nullptr, nullptr);
+            cl_int err = mem_cpy_fn(queue, CL_TRUE, dst->data,
+                                    src->data, ggml_nbytes(src), 0, nullptr, nullptr);
             if (err != CL_SUCCESS) {
                 GGML_LOG_ERROR("%s: clEnqueueMemcpyINTEL (host-to-device) failed with error %d\n", __func__, err);
                 return false;
@@ -585,8 +837,10 @@ static bool ggml_backend_openvino_buffer_cpy_tensor(ggml_backend_buffer_t buffer
         if (ggml_backend_buffer_is_openvino(src->buffer)) {
             ggml_backend_openvino_buffer_context * src_ctx =
                 (ggml_backend_openvino_buffer_context *) src->buffer->context;
-            if (src_ctx->is_remote) {
-                cl_int err = mem_cpy_fn(queue, CL_TRUE, dst->data, src->data, ggml_nbytes(src), 0, nullptr, nullptr);
+            if (src_ctx->is_remote && src_ctx->device == ctx->device) {
+                cl_int err = mem_cpy_fn(queue, CL_TRUE, dst->data,
+                                        src->data, ggml_nbytes(src),
+                                        0, nullptr, nullptr);
                 if (err != CL_SUCCESS) {
                     GGML_LOG_ERROR("%s: clEnqueueMemcpyINTEL (device-to-device) failed with error %d\n", __func__, err);
                     return false;
@@ -599,7 +853,17 @@ static bool ggml_backend_openvino_buffer_cpy_tensor(ggml_backend_buffer_t buffer

     // Host buffer - can copy from any host buffer
     if (ggml_backend_buffer_is_host(src->buffer)) {
-        memcpy(dst->data, src->data, ggml_nbytes(src));
+        const size_t n = ggml_nbytes(src);
+        std::vector<uint8_t> staged;
+        const void * from = src->data;
+        if (ggml_openvino_kv_storage_copy(src->data, nullptr, 0, GGML_OPENVINO_KV_READ)) {
+            staged.resize(n);
+            ggml_openvino_kv_storage_copy(src->data, staged.data(), n, GGML_OPENVINO_KV_READ);
+            from = staged.data();
+        }
+        if (!ggml_openvino_kv_storage_copy(dst->data, const_cast<void *>(from), n, GGML_OPENVINO_KV_WRITE)) {
+            memcpy(dst->data, from, n);
+        }
         return true;
     }
     return false;
@@ -609,7 +873,7 @@ static void ggml_backend_openvino_buffer_clear(ggml_backend_buffer_t buffer, uin
     ggml_backend_openvino_buffer_context * ctx = (ggml_backend_openvino_buffer_context *) buffer->context;
     GGML_ASSERT(ctx->data != nullptr);
     if (ctx->is_remote) {
-        cl_command_queue queue = ggml_openvino_get_cl_queue();
+        cl_command_queue queue = ggml_openvino_get_cl_queue(ggml_openvino_get_available_devices()[ctx->device]);
         auto mem_fill_fn = ggml_openvino_get_clEnqueueMemFillINTEL();
         if (mem_fill_fn != nullptr) {
             uint8_t pattern = value;
@@ -621,6 +885,8 @@ static void ggml_backend_openvino_buffer_clear(ggml_backend_buffer_t buffer, uin
         } else {
             GGML_LOG_WARN("%s: clEnqueueMemFillINTEL not available for GPU buffer clear\n", __func__);
         }
+    } else if (ctx->kv_on_demand) {
+        ggml_openvino_kv_storage_clear(ctx->data, ctx->size, value);
     } else {
         memset(ctx->data, value, ctx->size);
     }
@@ -671,15 +937,19 @@ static size_t ggml_backend_openvino_buffer_type_get_alignment(ggml_backend_buffe
 }

 static size_t ggml_backend_openvino_buffer_type_get_max_size(ggml_backend_buffer_type_t buft) {
-    GGML_UNUSED(buft);
     // A GPU caps a single memory object, so let ggml split a large buffer into parts that fit
-    return ggml_openvino_max_alloc_size();
+    ggml_backend_openvino_buffer_type_context * buft_ctx = (ggml_backend_openvino_buffer_type_context *) buft->context;
+    return ggml_openvino_max_alloc_size(ggml_openvino_get_available_devices()[buft_ctx->device]);
 }

 static size_t ggml_backend_openvino_buffer_type_get_alloc_size(ggml_backend_buffer_type_t buft,
                                                                const ggml_tensor * tensor) {
     GGML_UNUSED(buft);

+    if (is_self_contained_mmap_enabled()) {
+        return ggml_nbytes(tensor);
+    }
+
     // For quantized weight tensors, we need extra space for extracted data.
     if (!ggml_openvino_model_cache_only() && ggml_is_quantized(tensor->type) && tensor->ne[3] == 1) {
         ggml_openvino_extracted_layout layout = ggml_openvino_get_extracted_layout(tensor);
@@ -836,6 +1106,11 @@ bool ggml_backend_buft_is_openvino_host(ggml_backend_buffer_type_t buft) {
     return buft->iface.get_name == ggml_backend_openvino_host_buffer_type_get_name;
 }

+// Fingerprints of weights outside OpenVINO buffers (e.g. a CPU-buffer token_embd), keyed by address, size and name. A
+// later model can reuse an address, so the map is cleared with the shared compiled-model cache.
+static std::mutex g_fallback_fingerprints_mutex;
+static std::map<std::tuple<const void *, size_t, std::string>, uint64_t> g_fallback_fingerprints;
+
 uint64_t ggml_backend_openvino_weight_fingerprint(const ggml_tensor * tensor) {
     if (ggml_backend_buffer_is_openvino(tensor->buffer)) {
         auto * ctx = static_cast<ggml_backend_openvino_buffer_context *>(tensor->buffer->context);
@@ -843,10 +1118,24 @@ uint64_t ggml_backend_openvino_weight_fingerprint(const ggml_tensor * tensor) {
         if (it != ctx->weight_fingerprints.end()) {
             return it->second;
         }
+        if (ctx->external_memory) {
+            // GGML_OPENVINO_SELF_CONTAINED_MMAP wraps the GGUF mmap, so the weights are never uploaded
+            // through set_tensor. Identify them from the same mapping set_tensor saw on the cold run.
+            const uint64_t fp = ggml_openvino_source_fingerprint(tensor->data, ggml_nbytes(tensor), ctx->source_mappings);
+            ctx->weight_fingerprints[tensor->data] = fp;
+            return fp;
+        }
         GGML_ABORT("ggml-openvino: missing source identity for weight %s", tensor->name);
     }
+    const auto key =
+        std::make_tuple(static_cast<const void *>(tensor->data), ggml_nbytes(tensor), std::string(tensor->name));
+    std::lock_guard<std::mutex> lock(g_fallback_fingerprints_mutex);
+    auto it = g_fallback_fingerprints.find(key);
+    if (it != g_fallback_fingerprints.end()) {
+        return it->second;
+    }
     std::vector<ggml_openvino_source_mapping> mappings;
-    return ggml_openvino_source_fingerprint(tensor->data, ggml_nbytes(tensor), mappings);
+    return g_fallback_fingerprints[key] = ggml_openvino_source_fingerprint(tensor->data, ggml_nbytes(tensor), mappings);
 }

 static void ggml_backend_openvino_free(ggml_backend_t backend) {
@@ -864,6 +1153,9 @@ static void ggml_backend_openvino_free(ggml_backend_t backend) {
             // reuses it instead of recompiling against zeroed weights.
             if (!ggml_openvino_weight_buffers_released()) {
                 cache->graphs.clear();
+                ov_clear_imported_blobs();
+                std::lock_guard<std::mutex> fingerprints_lock(g_fallback_fingerprints_mutex);
+                g_fallback_fingerprints.clear();
             }
         }
     }
@@ -911,11 +1203,11 @@ static ggml_guid_t ggml_backend_openvino_guid(void) {
     return &guid;
 }

-static std::shared_ptr<ov_runtime_context> get_ov_runtime_context_ptr() {
+static std::shared_ptr<ov_runtime_context> get_ov_runtime_context_ptr(int device) {
     // Share compiled models, but give every backend its own requests and KV state.
     static auto cache = std::make_shared<ov_compiled_model_cache>();
     auto r_ctx = std::make_shared<ov_runtime_context>();
-    r_ctx->device = ggml_openvino_get_device_name();
+    r_ctx->device = ggml_openvino_get_available_devices()[device];
     r_ctx->stateful = is_stateful_enabled() && !ggml_openvino_is_npu();
     r_ctx->compiled_cache = cache;
     std::lock_guard<std::mutex> cache_lock(cache->mutex);
@@ -936,7 +1228,7 @@ GGML_BACKEND_API ggml_backend_t ggml_backend_openvino_init(int device) {
         return nullptr;
     }

-    ctx->runtime_context = get_ov_runtime_context_ptr();
+    ctx->runtime_context = get_ov_runtime_context_ptr(device);
     if (ctx->runtime_context == nullptr) {
         GGML_LOG_ERROR("%s: failed to allocate runtime context\n", __func__);
         delete ctx;
@@ -1124,9 +1416,9 @@ static void ggml_backend_openvino_device_get_memory(ggml_backend_dev_t dev, size

 static enum ggml_backend_dev_type ggml_backend_openvino_device_get_type(ggml_backend_dev_t dev) {
     ggml_backend_openvino_device_context * ctx = (ggml_backend_openvino_device_context *) dev->context;
-    // Only the device selected by GGML_OPENVINO_DEVICE is offered for offload. The others are
+    // Only the devices selected by GGML_OPENVINO_DEVICE are offered for offload. The others are
     // registered for discovery (--list-devices) only; llama.cpp skips IGPU devices when a GPU exists.
-    return ctx->ov_name == ggml_openvino_get_device_name() ? GGML_BACKEND_DEVICE_TYPE_GPU : GGML_BACKEND_DEVICE_TYPE_IGPU;
+    return ggml_openvino_is_selected_device(ctx->ov_name) ? GGML_BACKEND_DEVICE_TYPE_GPU : GGML_BACKEND_DEVICE_TYPE_IGPU;
 }

 static void ggml_backend_openvino_device_get_props(ggml_backend_dev_t dev, ggml_backend_dev_props * props) {
@@ -1138,7 +1430,7 @@ static void ggml_backend_openvino_device_get_props(ggml_backend_dev_t dev, ggml_
     props->caps = {
         /* .async                 = */ false,
         /* .host_buffer           = */ false,
-        /* .buffer_from_host_ptr  = */ false,
+        /* .buffer_from_host_ptr  = */ is_self_contained_mmap_enabled(),
         /* .events                = */ false,
         /* .mmap_support          = */ true,
     };
@@ -1147,7 +1439,7 @@ static void ggml_backend_openvino_device_get_props(ggml_backend_dev_t dev, ggml_
 static ggml_backend_t ggml_backend_openvino_device_init(ggml_backend_dev_t dev, const char * params) {
     GGML_UNUSED(params);
     ggml_backend_openvino_device_context * ctx = (ggml_backend_openvino_device_context *) dev->context;
-    if (ctx->ov_name != ggml_openvino_get_device_name()) {
+    if (!ggml_openvino_is_selected_device(ctx->ov_name)) {
         // Not an error: test-backend-ops initializes every device
         GGML_LOG_WARN("%s: %s (OpenVINO %s) is not the selected device, no ops will run on it; "
                       "set GGML_OPENVINO_DEVICE=%s to use it\n",
@@ -1166,6 +1458,17 @@ static ggml_backend_buffer_type_t ggml_backend_openvino_device_get_host_buffer_t
     return ggml_backend_openvino_host_buffer_type(ctx->device);
 }

+static ggml_backend_buffer_t ggml_backend_openvino_device_buffer_from_host_ptr(ggml_backend_dev_t dev,
+                                                                               void * ptr,
+                                                                               size_t size,
+                                                                               size_t max_tensor_size) {
+    auto * dev_ctx = static_cast<ggml_backend_openvino_device_context *>(dev->context);
+    auto * ctx = new ggml_backend_openvino_buffer_context(dev_ctx->device, ptr, size);
+    GGML_UNUSED(max_tensor_size);
+    return ggml_backend_buffer_init(ggml_backend_openvino_buffer_type(dev_ctx->device),
+                                    ggml_backend_openvino_buffer_interface, ctx, size);
+}
+
 static bool has_view_op_input(const ggml_tensor * op) {
     for (int i = 0; i < GGML_MAX_SRC; i++) {
         if (op->src[i] == nullptr) {
@@ -1181,6 +1484,11 @@ static bool has_view_op_input(const ggml_tensor * op) {
 // OV slices whole elements per axis, so each stride must be a multiple of the next smaller one
 // (e.g. a batch stride of m*nb[1] + pad bytes cannot be expressed and would be read wrongly).
 static bool has_strides_on_element_grid(const ggml_tensor * t) {
+    // nothing is read from an empty tensor (gemma-4's KV-shared layers in a batch without output rows), and its strides
+    // past the empty dim are 0
+    if (ggml_nelements(t) == 0) {
+        return true;
+    }
     std::vector<size_t> strides;
     for (int i = 0; i < GGML_MAX_DIMS; i++) {
         if (t->ne[i] > 1) {
@@ -1213,16 +1521,31 @@ static bool is_supported_flash_attn_pattern(const ggml_tensor * op) {
     //   PERMUTE -> VIEW  -> base (view_src==nullptr)   (llama KV-cache path)
     //   PERMUTE -> RESHAPE -> base (view_src==nullptr)  (whisper Q)
     //   VIEW -> base (view_src==nullptr)                (whisper K/V from kv_pad)
+    //   PERMUTE -> VIEW -> RESHAPE -> base              (Q of a layer without RoPE, e.g. SmolLM3)
     for (int i = 0; i < 3; i++) {
         const ggml_tensor * src = op->src[i];
         if (src->op == GGML_OP_PERMUTE) {
             if (src->src[0] == nullptr) {
                 return false;
             }
+            // MLA: Q = PERMUTE -> CONCAT(q_nope_absorbed, q_pe); V = PERMUTE -> VIEW -> VIEW -> K cache
+            // (V is the first kv_lora_rank values of each K cache row)
+            if (i == 0 && src->src[0]->op == GGML_OP_CONCAT) {
+                continue;
+            }
+            if (i == 2 && src->src[0]->op == GGML_OP_VIEW && src->src[0]->src[0] != nullptr &&
+                src->src[0]->src[0]->op == GGML_OP_VIEW && src->src[0]->src[0]->src[0] != nullptr &&
+                src->src[0]->src[0]->src[0]->view_src == nullptr) {
+                continue;
+            }
             if (src->src[0]->op != GGML_OP_VIEW && src->src[0]->op != GGML_OP_RESHAPE) {
                 return false;
             }
-            if (src->src[0]->src[0] == nullptr || src->src[0]->src[0]->view_src != nullptr) {
+            const ggml_tensor * base = src->src[0]->src[0];
+            if (base != nullptr && src->src[0]->op == GGML_OP_VIEW && base->op == GGML_OP_RESHAPE) {
+                base = base->src[0];
+            }
+            if (base == nullptr || base->view_src != nullptr) {
                 return false;
             }
         } else if (src->op == GGML_OP_VIEW) {
@@ -1300,7 +1623,8 @@ static bool cpy_output_view_is_supported(const ggml_tensor * op) {
         return false;
     }

-    return ggml_nbytes(op) == 0 || ggml_is_contiguous(op) || GgmlOvDecoder::is_conv_state_writeback(op);
+    return ggml_nbytes(op) == 0 || ggml_is_contiguous(op) || GgmlOvDecoder::is_conv_state_writeback(op) ||
+           GgmlOvDecoder::is_gdn_snapshot_writeback(op);
 }

 static bool mul_mat_id_requires_large_tmp(const ggml_tensor * op) {
@@ -1368,6 +1692,12 @@ static ggml_openvino_op_support is_op_supported_case(const ggml_tensor * op) {
     }

     switch (op->op) {
+    case GGML_OP_TOP_K: {
+        if (op->src[0]->type != GGML_TYPE_F32 || op->type != GGML_TYPE_I32 || ggml_openvino_is_npu()) {
+            return {false, "TOP_K requires F32 input and I32 output on CPU or GPU"};
+        }
+        break;
+    }
     case GGML_OP_CONCAT: {
         if (op->type == GGML_TYPE_I64) {
             return {false, "CONCAT with I64 type is not supported"};
@@ -1397,12 +1727,13 @@ static ggml_openvino_op_support is_op_supported_case(const ggml_tensor * op) {
     }
     case GGML_OP_GET_ROWS:
     case GGML_OP_SET_ROWS: {
-        if (op->ne[3] != 1) {
-            return {false, "GET_ROWS/SET_ROWS with ne[3] != 1 (ne[3]=" + std::to_string(op->ne[3]) + ") is not supported"};
+        if (GgmlOvDecoder::is_conv_state_snapshot_writeback(op) &&
+            (ggml_openvino_is_npu() || ggml_openvino_getenv_int("GGML_OPENVINO_FORCE_STATIC"))) {
+            return {false, "convolution snapshots require dynamic execution"};
         }
-        if (op->op == GGML_OP_GET_ROWS && ggml_openvino_is_gpu() &&
-            op->src[0]->type == GGML_TYPE_BF16) {
-            return {false, "GET_ROWS with BF16 src0 is not supported on GPU"};
+        if (op->ne[3] != 1 && !GgmlOvDecoder::is_get_rows_batched_4d(op) &&
+            !GgmlOvDecoder::is_conv_state_snapshot_writeback(op)) {
+            return {false, "GET_ROWS/SET_ROWS with ne[3] != 1 (ne[3]=" + std::to_string(op->ne[3]) + ") is not supported"};
         }
         if (op->ne[0] == 256 && (op->src[0]->type == GGML_TYPE_Q4_K || op->src[0]->type == GGML_TYPE_Q5_K ||
                                  op->src[0]->type == GGML_TYPE_Q4_1 || op->src[0]->type == GGML_TYPE_Q5_1)) {
@@ -1473,12 +1804,26 @@ static ggml_openvino_op_support is_op_supported_case(const ggml_tensor * op) {
         // The GPU plugin can fuse broadcast DIV into the preceding FFN GEMM path
         // and produce infs for per-channel scale vectors. Keep those DIVs on CPU
         // until the fused GPU kernel is reliable. (falied case llama-arch-test mpt)
+        // Only a weight divisor is such a scale; at one token an activation divisor has the same shape
+        // (qwen2moe's shared-expert sigmoid, silu(x) / x).
         if (ggml_openvino_is_gpu() && op->src[1]->ne[0] == op->ne[0] &&
-            op->src[1]->ne[1] == 1 && op->src[1]->ne[2] == 1 && op->src[1]->ne[3] == 1) {
+            op->src[1]->ne[1] == 1 && op->src[1]->ne[2] == 1 && op->src[1]->ne[3] == 1 &&
+            op->src[1]->buffer != nullptr && op->src[1]->buffer->usage == GGML_BACKEND_BUFFER_USAGE_WEIGHTS) {
             return {false, "DIV per-channel scale broadcast is not supported on GPU"};
         }
         break;
     }
+    case GGML_OP_POOL_1D: {
+        const ggml_op_pool pool_mode = static_cast<ggml_op_pool>(op->op_params[0]);
+        if (pool_mode != GGML_OP_POOL_MAX && pool_mode != GGML_OP_POOL_AVG) {
+            return {false, "POOL_1D mode " + std::to_string(pool_mode) + " is not supported"};
+        }
+        // a window fully in the padding gives -FLT_MAX (max) or 0 (avg) in ggml, OV differs
+        if (op->op_params[3] >= op->op_params[1]) {
+            return {false, "POOL_1D with padding >= kernel size is not supported"};
+        }
+        break;
+    }
     case GGML_OP_POOL_2D: {
         if (ggml_openvino_is_gpu()) {
             const int32_t * params = op->op_params;
@@ -1531,8 +1876,8 @@ static ggml_openvino_op_support is_op_supported_case(const ggml_tensor * op) {
             return {false, "FLASH_ATTN_EXT with K/V row lists is not supported"};
         }

-        if (op->src[4] != nullptr) {
-            return {false, "FLASH_ATTN_EXT with sinks is not supported"};
+        if (op->src[4] != nullptr && op->src[3] == nullptr) {
+            return {false, "FLASH_ATTN_EXT with sinks but no mask is not supported"};
         }
         if (!is_supported_flash_attn_pattern(op)) {
             return {false, "FLASH_ATTN_EXT unsupported attention pattern"};
@@ -1540,8 +1885,14 @@ static ggml_openvino_op_support is_op_supported_case(const ggml_tensor * op) {
         if (max_bias > 0) {
             return {false, "FLASH_ATTN_EXT with max_bias > 0 (max_bias=" + std::to_string(max_bias) + ") is not supported"};
         }
-        if (logit_softcap != 0) {
-            return {false, "FLASH_ATTN_EXT with logit_softcap != 0 (logit_softcap=" + std::to_string(logit_softcap) + ") is not supported"};
+        // soft-capping (gemma-2) is applied on the manual attention path of translate_flash_attn_ext, which needs one
+        // head size for Q, K and V
+        if (logit_softcap != 0 && (op->src[1]->ne[0] != op->src[0]->ne[0] || op->src[2]->ne[0] != op->src[0]->ne[0])) {
+            return {false, "FLASH_ATTN_EXT with logit_softcap != 0 needs one head size for Q, K and V"};
+        }
+        // sinks are applied only by SDPA, which has no soft-capping
+        if (logit_softcap != 0 && op->src[4] != nullptr) {
+            return {false, "FLASH_ATTN_EXT with sinks and logit_softcap != 0 is not supported"};
         }
         break;
     }
@@ -1608,6 +1959,14 @@ static ggml_openvino_op_support is_op_supported_case(const ggml_tensor * op) {
             op->src[0]->buffer == nullptr) {
             return {false, "MUL_MAT with unbound Q4_1/Q4_K src[0] on GPU is not supported"};
         }
+        // The GPU plugin fails to compile a MatMul with batched quantized weights (one matrix per batch, e.g. MLA's
+        // attn_k_b) when shapes are static: wrong results or "ProgramBuilder build failed". Dynamic shapes work. Op
+        // tests (unbound weights, see above) and the static pipeline compile with static shapes. Remove once the GPU
+        // plugin is fixed.
+        if (ggml_openvino_is_gpu() && ggml_is_quantized(op->src[0]->type) && op->src[0]->ne[2] > 1 &&
+            (op->src[0]->buffer == nullptr || ggml_openvino_getenv_int("GGML_OPENVINO_FORCE_STATIC"))) {
+            return {false, "MUL_MAT with batched quantized src[0] and static shapes on GPU is not supported"};
+        }
         if (op->src[0]->ne[3] != op->src[1]->ne[3] && op->src[0]->ne[3] != 1 && op->src[1]->ne[3] != 1) {
             return {false, "MUL_MAT with incompatible broadcast on ne[3]: src0->ne[3]=" + std::to_string(op->src[0]->ne[3]) +
                            ", src1->ne[3]=" + std::to_string(op->src[1]->ne[3])};
@@ -1624,9 +1983,6 @@ static ggml_openvino_op_support is_op_supported_case(const ggml_tensor * op) {
             return {false, "MUL_MAT_ID with single-expert or empty ne[2] <= 1 (ne[2]=" +
                            std::to_string(op->src[0]->ne[2]) + ") is not supported"};
         }
-        if (ggml_openvino_is_gpu() && op->src[0] != nullptr && !ggml_is_quantized(op->src[0]->type)) {
-            return {false, "MUL_MAT_ID with non-quantized weights on GPU is not supported"};
-        }
         // The GPU plugin's GatherMatmul returns wrong values for the layouts test-backend-ops
         // produces: it builds a rank-4 input layout ([n_used, n_tokens, k, 1]) instead of rank 3
         // and the kernel misreads it, silently returning garbage (NMSE ~86) rather than asserting.
@@ -1669,16 +2025,25 @@ static ggml_openvino_op_support is_op_supported_case(const ggml_tensor * op) {
         //     // CVS-186471
         //     return true;
         // }
+        const char * reason = nullptr;
         if (op->src[2]->op == GGML_OP_PERMUTE) {
-            return {false, "GATED_DELTA_NET with PERMUTE src2 is not supported"};
+            reason = "GATED_DELTA_NET with PERMUTE src2 is not supported";
         }
-        // kda (per-key-dimension gating) not supported by fused GatedDeltaNet op
-        if (op->src[3]->ne[0] != 1) {
-            return {false, "GATED_DELTA_NET with kda (per-key-dimension gating) is not supported"};
+        else if (op->src[3]->ne[0] != 1) {
+            // kda (per-key-dimension gating) not supported by fused GatedDeltaNet op
+            reason = "GATED_DELTA_NET with kda (per-key-dimension gating) is not supported";
         }
-        // K > 1 (multiple state snapshots) not supported by fused op
-        if (((const int32_t *) op->op_params)[0] > 1) {
-            return {false, "GATED_DELTA_NET with K > 1 (multiple state snapshots) is not supported"};
+        else if (op->op_params[0] > 1) {
+            if (ggml_openvino_is_npu() || ggml_openvino_getenv_int("GGML_OPENVINO_FORCE_STATIC")) {
+                reason = "GATED_DELTA_NET snapshots require dynamic execution";
+            }
+        }
+        if (reason != nullptr) {
+            if (ggml_openvino_gdn_native_state_enabled()) {
+                // the CPU fallback reads the state in the ggml layout and would silently corrupt it
+                GGML_ABORT("ggml-openvino: GGML_OPENVINO_GDN_NATIVE_STATE is set, but %s", reason);
+            }
+            return {false, reason};
         }
         break;
     }
@@ -1784,7 +2149,7 @@ static ggml_openvino_op_support ggml_backend_openvino_device_supports_op_impl(gg
     GGML_ASSERT(dev->reg != nullptr);

     ggml_backend_openvino_device_context * dev_ctx = (ggml_backend_openvino_device_context *) dev->context;
-    if (dev_ctx->ov_name != ggml_openvino_get_device_name()) {
+    if (!ggml_openvino_is_selected_device(dev_ctx->ov_name)) {
         // Data placed on a non-selected device (e.g. with -dev) can never run here; stop with a hint
         // instead of the generic scheduler abort. Unallocated tensors (test-backend-ops) pass through.
         for (int i = -1; i < GGML_MAX_SRC; i++) {
@@ -1801,9 +2166,22 @@ static ggml_openvino_op_support ggml_backend_openvino_device_supports_op_impl(gg
         return {false, "device is not the selected OpenVINO device"};
     }

+    // The write-back of GATED_DELTA_NET rollback snapshots stores into the recurrent cache in this backend's buffer,
+    // so it always runs here. When the GDN that produces the snapshots falls back to ggml, the write-back reads the
+    // packed GDN output wrongly and silently corrupts the cache, so stop with the reason instead.
+    if (GgmlOvDecoder::is_gdn_snapshot_writeback(op)) {
+        const auto gdn_support = ggml_backend_openvino_device_supports_op_impl(dev, op->src[0]->src[0]);
+        if (!gdn_support.is_supported) {
+            GGML_ABORT(
+                "ggml-openvino: %s. Recurrent rollback (n_rs_seq > 0, used by MTP, EAGLE3, DFlash and DSpark "
+                "speculative decoding) is not supported for this model on this device",
+                gdn_support.reason.c_str());
+        }
+    }
+
     static std::unordered_set<ggml_type> supported_types{
         GGML_TYPE_F32,  GGML_TYPE_F16,  GGML_TYPE_BF16, GGML_TYPE_I64,  GGML_TYPE_I32,  GGML_TYPE_Q4_0,
-        GGML_TYPE_Q4_1, GGML_TYPE_Q4_K, GGML_TYPE_Q5_1, GGML_TYPE_Q5_K, GGML_TYPE_Q8_0, GGML_TYPE_Q6_K,
+        GGML_TYPE_Q4_1, GGML_TYPE_Q4_K, GGML_TYPE_Q5_0, GGML_TYPE_Q5_1, GGML_TYPE_Q5_K, GGML_TYPE_Q8_0, GGML_TYPE_Q6_K,
         GGML_TYPE_MXFP4};

     // derive supported op sets from the op_table map, keys in
@@ -1850,9 +2228,12 @@ static ggml_openvino_op_support ggml_backend_openvino_device_supports_op_impl(gg
         if (!supported) {
             return {false, "unary op " + std::string(ggml_unary_op_name(ggml_get_unary_op(op))) + " has no op translator"};
         }
-        if (op->type == GGML_TYPE_F32 && (ggml_get_unary_op(op) == GGML_UNARY_OP_EXP ||
-                                          ggml_get_unary_op(op) == GGML_UNARY_OP_EXPM1)) {
-            return {false, "UNARY_EXP / UNARY_EXPM1 with F32 type is not supported"};
+        // OV GPU matches ggml, CPU fails some tests
+        if (op->type == GGML_TYPE_F32 && ggml_get_unary_op(op) == GGML_UNARY_OP_EXP && !ggml_openvino_is_gpu()) {
+            return {false, "UNARY_EXP with F32 type is not supported on " + ggml_openvino_get_device_name()};
+        }
+        if (op->type == GGML_TYPE_F32 && ggml_get_unary_op(op) == GGML_UNARY_OP_EXPM1) {
+            return {false, "UNARY_EXPM1 with F32 type is not supported"};
         }
         break;
     }
@@ -1893,12 +2274,17 @@ static ggml_openvino_op_support ggml_backend_openvino_device_supports_op_impl(gg
         if (supported_types.find(src->type) == supported_types.end()) {
             return {false, "src[" + std::to_string(i) + "] type " + std::string(ggml_type_name(src->type)) + " is not supported"};
         }
-        if (!has_strides_on_element_grid(src)) {
+        if (!has_strides_on_element_grid(src) && !(i == 1 && GgmlOvDecoder::is_gdn_snapshot_writeback(op)) &&
+            !(i == 2 && GgmlOvDecoder::is_conv_state_snapshot_writeback(op))) {
             return {false, "src[" + std::to_string(i) + "] strides are not multiples of each other"};
         }
         const bool is_supported_3d_moe_expert =
             op->op == GGML_OP_MUL_MAT_ID && i == 0 && (src->type == GGML_TYPE_MXFP4 || src->ne[3] == 1);
-        if (ggml_is_quantized(src->type) && src->ne[2] != 1 && !is_supported_3d_moe_expert) {
+        // one weight matrix per activation batch, e.g. MLA's per-head attn_k_b / attn_v_b (a batched MatMul)
+        const bool is_supported_3d_batched_weight = op->op == GGML_OP_MUL_MAT && i == 0 && src->ne[3] == 1 &&
+                                                    op->src[1]->ne[2] == src->ne[2] && op->src[1]->ne[3] == 1;
+        if (ggml_is_quantized(src->type) && src->ne[2] != 1 && !is_supported_3d_moe_expert &&
+            !is_supported_3d_batched_weight) {
             return {false, "3D quantized tensor for src[" + std::to_string(i) + "] is not supported"};
         }
     }
@@ -1923,8 +2309,13 @@ static bool ggml_backend_openvino_device_supports_op(ggml_backend_dev_t dev, con
 }

 static bool ggml_backend_openvino_device_supports_buft(ggml_backend_dev_t dev, ggml_backend_buffer_type_t buft) {
-    return ggml_backend_buft_is_openvino(buft) || ggml_backend_buft_is_host(buft);
-    GGML_UNUSED(dev);
+    if (ggml_backend_buft_is_openvino(buft)) {
+        ggml_backend_openvino_device_context * dev_ctx = (ggml_backend_openvino_device_context *) dev->context;
+        ggml_backend_openvino_buffer_type_context * buft_ctx =
+            (ggml_backend_openvino_buffer_type_context *) buft->context;
+        return buft_ctx->device == dev_ctx->device;
+    }
+    return ggml_backend_buft_is_host(buft);
 }

 static const struct ggml_backend_device_i ggml_backend_openvino_device_interface = {
@@ -1936,7 +2327,7 @@ static const struct ggml_backend_device_i ggml_backend_openvino_device_interface
     /* .init_backend         = */ ggml_backend_openvino_device_init,
     /* .get_buffer_type      = */ ggml_backend_openvino_device_get_buffer_type,
     /* .get_host_buffer_type = */ ggml_backend_openvino_device_get_host_buffer_type,
-    /* .buffer_from_host_ptr = */ NULL,
+    /* .buffer_from_host_ptr = */ ggml_backend_openvino_device_buffer_from_host_ptr,
     /* .supports_op          = */ ggml_backend_openvino_device_supports_op,
     /* .supports_buft        = */ ggml_backend_openvino_device_supports_buft,
     /* .offload_op           = */ NULL,
@@ -2001,7 +2392,7 @@ GGML_BACKEND_API ggml_backend_reg_t ggml_backend_openvino_reg(void) {
                 dev_ctx->ov_name = openvino_devices[i];
                 // The device is chosen with GGML_OPENVINO_DEVICE, not -dev, so show the value to set
                 dev_ctx->description = "GGML_OPENVINO_DEVICE=" + dev_ctx->ov_name +
-                                       (dev_ctx->ov_name == ggml_openvino_get_device_name() ? " (selected)" : "") +
+                                       (ggml_openvino_is_selected_device(dev_ctx->ov_name) ? " (selected)" : "") +
                                        " - " + ggml_openvino_get_device_description(dev_ctx->ov_name);
                 dev_ctx->total_memory = 0;
                 if (ov_device_has_prefix(dev_ctx->ov_name, "GPU")) {
diff --git a/ggml/src/ggml-openvino/ggml-quants.cpp b/ggml/src/ggml-openvino/ggml-quants.cpp
index 824d24478..96b8da9be 100644
--- a/ggml/src/ggml-openvino/ggml-quants.cpp
+++ b/ggml/src/ggml-openvino/ggml-quants.cpp
@@ -171,6 +171,38 @@ void extract_q4_1_data(const ggml_tensor * tensor,
     }
 }

+// Extracts (weight, scales) from Q5_0 tensors.
+// Data layout is: |16 bit scale|32 bit qh (5th bits)|32 x 4bit low nibbles|.
+// Reconstructed quant q in [0,31]: q = (low nibble) | (qh_bit << 4). Dequant: (q - 16) * d.
+// Weights are stored as signed i8 (q - 16), symmetric like Q8_0, so there is no zero point.
+void extract_q5_0_data(const ggml_tensor * tensor, ov::Tensor & weights_arr, ov::Tensor & scales_arr) {
+    const uint64_t bytes_per_block = 22;  // 2 scale + 4 qh + 16 (32x0.5) weights
+    const int qk = 32;
+
+    OPENVINO_ASSERT(weights_arr.get_element_type() == ov::element::i8, "Q5_0 weights are extracted as i8");
+
+    auto * data = static_cast<uint8_t *>(tensor->data);
+    auto * weights = static_cast<int8_t *>(weights_arr.data());
+    auto * scales = scales_arr.data<ov::element_type_traits<ov::element::f16>::value_type>();
+
+    ov::parallel_for(scales_arr.get_size(), [&](size_t i) {
+        const uint8_t * block = data + i * bytes_per_block;
+        uint16_t d;
+        memcpy(&d, block, sizeof(d));
+        uint32_t qh;
+        memcpy(&qh, block + 2, sizeof(qh));
+        const uint8_t * qs = block + 6;
+        int8_t * dst = weights + i * qk;
+        scales[i] = ov::float16::from_bits(d);
+        for (int j = 0; j < qk / 2; ++j) {
+            const uint8_t bit_lo = (qh >> j) & 1;
+            const uint8_t bit_hi = (qh >> (j + qk / 2)) & 1;
+            dst[j] = static_cast<int8_t>(((qs[j] & 0x0F) | (bit_lo << 4)) - 16);        // first 16 weights
+            dst[j + qk / 2] = static_cast<int8_t>(((qs[j] >> 4) | (bit_hi << 4)) - 16);  // last 16 weights
+        }
+    });
+}
+
 // Extracts (weight, scales, zp) from Q5_1 tensors.
 // Data layout is: |16 bit scale|16 bit min|32 bit qh (5th bits)|32 x 4bit low nibbles|.
 // Reconstructed quant q in [0,31]: q = (low nibble) | (qh_bit << 4). Dequant: w*d + m.
@@ -771,7 +803,8 @@ void quantize_q4_0(const float * x,
                    ov::Tensor & scales_arr,
                    ov::Tensor & zp_arr,
                    int64_t k,
-                   int64_t qk) {
+                   int64_t qk,
+                   int64_t block_offset = 0) {
     assert(k % qk == 0);
     const int nb = k / qk;

@@ -782,6 +815,9 @@ void quantize_q4_0(const float * x,
     if (!is_symmetric) {
         auto * zp = static_cast<uint8_t *>(zp_arr.data());
         for (int i = 0; i < nb; i++) {
+            // Destination block index. Chunks are emitted in increasing order, so the
+            // even-block assignment always precedes the odd-block OR of the same zp byte.
+            const int64_t gi = block_offset + i;
             float amax = 0.0f;
             float max = 0.0f;
             for (int j = 0; j < qk; j++) {
@@ -793,33 +829,34 @@ void quantize_q4_0(const float * x,
             }
             const float d = max / -8;
             if (d == 0) {
-                scales[i] = ov::float16(1.0f);
-                if (i % 2 == 0) {
-                    zp[i / 2] = 8;
+                scales[gi] = ov::float16(1.0f);
+                if (gi % 2 == 0) {
+                    zp[gi / 2] = 8;
                 } else {
-                    zp[i / 2] |= (8 << 4);
+                    zp[gi / 2] |= (8 << 4);
                 }
-                memset(weights + i * qk / 2, 8 | (8 << 4), qk / 2);
+                memset(weights + gi * qk / 2, 8 | (8 << 4), qk / 2);
                 continue;
             }
             const float id = 1.0f / d;
-            scales[i] = ov::float16(d);
-            if (i % 2 == 0) {
-                zp[i / 2] = 8;
+            scales[gi] = ov::float16(d);
+            if (gi % 2 == 0) {
+                zp[gi / 2] = 8;
             } else {
-                zp[i / 2] |= (8 << 4);
+                zp[gi / 2] |= (8 << 4);
             }
             for (int j = 0; j < qk / 2; ++j) {
                 const float x0 = x[i * qk + 2 * j] * id;
                 const float x1 = x[i * qk + 2 * j + 1] * id;
                 const uint8_t xi0 = MIN(15, (int8_t) (x0 + 8.5f));
                 const uint8_t xi1 = MIN(15, (int8_t) (x1 + 8.5f));
-                weights[i * qk / 2 + j] = xi0 | (xi1 << 4);
+                weights[gi * qk / 2 + j] = xi0 | (xi1 << 4);
             }
         }
     } else {
         // Symmetric: produce signed i4 values in [-8, 7]
         for (int i = 0; i < nb; i++) {
+            const int64_t gi = block_offset + i;
             float amax = 0.0f;
             float max = 0.0f;
             for (int j = 0; j < qk; j++) {
@@ -831,20 +868,20 @@ void quantize_q4_0(const float * x,
             }
             const float d = max / -8;
             if (d == 0) {
-                scales[i] = ov::float16(1.0f);
+                scales[gi] = ov::float16(1.0f);
                 // i4 value 0 packed: 0x00
-                memset(weights + i * qk / 2, 0, qk / 2);
+                memset(weights + gi * qk / 2, 0, qk / 2);
                 continue;
             }
             const float id = 1.0f / d;
-            scales[i] = ov::float16(d);
+            scales[gi] = ov::float16(d);
             for (int j = 0; j < qk / 2; ++j) {
                 const float x0 = x[i * qk + 2 * j] * id;
                 const float x1 = x[i * qk + 2 * j + 1] * id;
                 // Signed i4: range [-8, 7]. Quantize as round(x*id), then pack as 4-bit two's complement.
                 int8_t si0 = (int8_t) std::max(-8, std::min(7, (int) roundf(x0)));
                 int8_t si1 = (int8_t) std::max(-8, std::min(7, (int) roundf(x1)));
-                weights[i * qk / 2 + j] = (si0 & 0x0F) | ((si1 & 0x0F) << 4);
+                weights[gi * qk / 2 + j] = (si0 & 0x0F) | ((si1 & 0x0F) << 4);
             }
         }
     }
@@ -1019,7 +1056,7 @@ std::shared_ptr<ov::Node> extract_quantized_weights(const ggml_tensor * tensor,
                                                     ov::Tensor & scales,
                                                     ov::Tensor & zp,
                                                     // Use an exact f16 zero point (vs. a rounded integer one); always
-                                                    // used for for_gather_matmul (3D MoE expert) weights regardless of
+                                                    // used for MoE expert weights regardless of
                                                     // this flag, and also settable explicitly for test-backend-ops.
                                                     bool use_bias = false) {
     // Create a temporary tensor for extraction functions that read from tensor->data
@@ -1044,6 +1081,7 @@ std::shared_ptr<ov::Node> extract_quantized_weights(const ggml_tensor * tensor,
         weights_per_block = 32;
         break;
     case GGML_TYPE_Q8_0:
+    case GGML_TYPE_Q5_0:
     case GGML_TYPE_Q5_1:
     case GGML_TYPE_Q5_K:
         is_u4 = false;
@@ -1058,12 +1096,11 @@ std::shared_ptr<ov::Node> extract_quantized_weights(const ggml_tensor * tensor,
                                  std::string(ggml_type_name(tensor->type)));
     }

-    // 3D MoE expert weights (for_gather_matmul) always use the exact f16 zero-point extraction
-    // (see make_int8_weights/make_int4_weights) rather than the rounded integer zero point --
-    // round(min/scale) error is what corrupts Q4_K/Q5_1 experts, and the f16-zp form still fuses
-    // into GatherMatmulCompressed since it stays a Subtract, not an Add.
+    // An exact f16 zero point avoids the round(min/scale) error of the integer one, which corrupts
+    // Q4_K/Q5_1 MoE experts and costs accuracy on dense weights. It stays a Subtract, so the
+    // weights still fuse into GatherMatmulCompressed / compressed FullyConnected.
     const bool for_gather_matmul = tensor->ne[2] > 1;
-    use_bias = use_bias || for_gather_matmul;
+    use_bias = use_bias || ggml_openvino_use_exact_zero_point(tensor);

     // Extract quantized data
     switch (tensor->type) {
@@ -1076,6 +1113,9 @@ std::shared_ptr<ov::Node> extract_quantized_weights(const ggml_tensor * tensor,
     case GGML_TYPE_Q4_K:
         extract_q4_k_data(&temp_tensor, weights, scales, zp, use_bias);
         break;
+    case GGML_TYPE_Q5_0:
+        extract_q5_0_data(&temp_tensor, weights, scales);
+        break;
     case GGML_TYPE_Q5_1:
         extract_q5_1_data(&temp_tensor, weights, scales, zp, use_bias);
         break;
@@ -1125,19 +1165,18 @@ std::shared_ptr<ov::Node> requantize_to_buffers(const ggml_tensor * tensor,
     bool is_u4 = (requant_type == ExtraQuantType::Q4_0_C || requant_type == ExtraQuantType::Q4_0_128 ||
                   requant_type == ExtraQuantType::Q4_0_64 || requant_type == ExtraQuantType::Q4_1_64);

-    // Streaming dequant (opt-in via GGML_OPENVINO_REDUCE_COMPILE_MEM or
-    // GGML_OPENVINO_MEMORY_OPTIMIZE): instead of
-    // materializing the full n_elements F32 array (e.g. ~1 GB for token_embd), dequantize
-    // a chunk of complete rows into a small scratch and quantize/convert it straight into
-    // the output buffers, capping the transient F32 footprint at CHUNK_ROWS*ne0 floats.
+    // Streaming dequant: instead of materializing the full n_elements F32 array (e.g. ~1 GB
+    // for token_embd), dequantize a chunk of complete rows into a small scratch and
+    // quantize/convert it straight into the output buffers, capping the transient F32
+    // footprint at CHUNK_ROWS*ne0 floats, with identical output.
     //
-    // Only valid (and only used) for the Q8_0_C / Q8_1_C / F16 targets whose block size
-    // divides a row (channel-wise _C uses block_size == ne0) so no target block straddles
-    // a row boundary, and Q8/F16 have no cross-block packing. The u4 (Q4_0) path packs two
-    // weights per byte with running zp ORs that assume a single whole-array call, so it is
-    // never streamed. When the flag is off, behavior is identical to the original
-    // full-materialization path.
-    const bool stream_requant = ggml_openvino_reduce_compile_mem_enabled() && !is_u4 &&
+    // Only valid (and only used) when the target block size divides a row (channel-wise _C
+    // uses block_size == ne0) so no target block straddles a row boundary. The u4 (Q4_0)
+    // path packs two weights per byte and ORs zp nibbles in pairs, but both are indexed by
+    // an absolute block index and chunks are emitted in increasing order, so whole-row
+    // chunks stay byte-aligned and the even-block write always precedes the odd-block OR.
+    // Q4_1_64 (asymmetric u4) has no chunked quantizer, so it always takes the full path.
+    const bool stream_requant = requant_type != ExtraQuantType::Q4_1_64 &&
                                 !(block_size > 0 && ne0 % block_size != 0);

     if (!stream_requant) {
@@ -1178,7 +1217,9 @@ std::shared_ptr<ov::Node> requantize_to_buffers(const ggml_tensor * tensor,
                     ->from_float_ref(scratch.data(), f16_base + (r0 * ne0) * sizeof(uint16_t), elems);
             } else {
                 const int64_t block_offset = (r0 * ne0) / block_size;
-                if (requant_type == ExtraQuantType::Q8_1_C) {
+                if (is_u4) {
+                    quantize_q4_0(scratch.data(), weights, scales, zp, elems, block_size, block_offset);
+                } else if (requant_type == ExtraQuantType::Q8_1_C) {
                     quantize_q8_1(scratch.data(), weights, scales, zp, elems, block_size, block_offset);
                 } else {
                     quantize_q8_0(scratch.data(), weights, scales, zp, elems, block_size, block_offset);
@@ -1258,13 +1299,9 @@ OvWeight process_weight_tensor(const ggml_tensor * tensor, const void * data, vo
         OPENVINO_THROW("Unsupported quantized type: ", ggml_type_name(tensor->type));
     }

-    // 3D MoE expert weights (for_gather_matmul) always use the exact f16 zero-point path (see
-    // extract_quantized_weights) -- must be kept in sync with the "use_bias || for_gather_matmul"
-    // check in ggml_openvino_get_extracted_layout, which sizes/offsets the zp slot accordingly.
-    // Requantized tensors (layout.is_requant) are handled by requantize_to_buffers instead, whose
-    // zp sizing/type is unaffected by for_gather_matmul, so they are excluded here.
-    const bool for_gather_matmul = tensor->ne[2] > 1;
-    const bool zp_is_f16 = !layout.is_requant && (use_bias || for_gather_matmul);
+    // Keep in sync with ggml_openvino_get_extracted_layout, which sizes the zp slot. Requantized
+    // tensors are handled by requantize_to_buffers and keep an integer zero point.
+    const bool zp_is_f16 = !layout.is_requant && (use_bias || ggml_openvino_use_exact_zero_point(tensor));

     const bool is_3d_mxfp4_moe = tensor->type == GGML_TYPE_MXFP4 && (tensor->ne[2] > 1 || tensor->ne[3] > 1);
     if (is_3d_mxfp4_moe) {
diff --git a/ggml/src/ggml-openvino/model-cache.cpp b/ggml/src/ggml-openvino/model-cache.cpp
index c8c5fb5ad..597233d7e 100644
--- a/ggml/src/ggml-openvino/model-cache.cpp
+++ b/ggml/src/ggml-openvino/model-cache.cpp
@@ -1,9 +1,11 @@
 #include "model-cache.h"

+#include "device/npu_config.h"
 #include "ggml-backend-impl.h"
 #include "ggml-backend.h"
 #include "ggml-impl.h"
 #include "ggml-openvino-extra.h"
+#include "openvino/rt_info/weightless_caching_attributes.hpp"

 #include <cerrno>
 #include <algorithm>
@@ -13,10 +15,14 @@
 #include <iomanip>
 #include <limits>
 #include <sstream>
+#include <openvino/core/model.hpp>
 #include <openvino/core/version.hpp>
+#include <openvino/op/constant.hpp>
 #include <string>
 #include <sys/stat.h>
 #include <sys/types.h>
+#include <unordered_map>
+#include <unordered_set>
 #include <vector>

 #if defined(_WIN32)
@@ -83,25 +89,14 @@ uint64_t weight_fingerprint(const ggml_tensor * t) {
 // fingerprinted once, deterministically.
 template <typename F>
 void for_each_weight(const ggml_cgraph * cgraph, F && fn) {
-    std::vector<const ggml_tensor *> seen;
+    std::unordered_set<const ggml_tensor *> seen;
     for (int i = 0; i < cgraph->n_nodes; ++i) {
         const ggml_tensor * node = cgraph->nodes[i];
         for (int s = 0; s < GGML_MAX_SRC; ++s) {
             const ggml_tensor * src = node->src[s];
-            if (!is_weight_src(src)) {
+            if (!is_weight_src(src) || !seen.insert(src).second) {
                 continue;
             }
-            bool dup = false;
-            for (const auto * p : seen) {
-                if (p == src) {
-                    dup = true;
-                    break;
-                }
-            }
-            if (dup) {
-                continue;
-            }
-            seen.push_back(src);
             fn(src);
         }
     }
@@ -150,6 +145,12 @@ bool make_dirs(const std::string & path) {
         // Create each intermediate component (skip a leading "/" root).
         if (sep && acc.size() > 1) {
             std::string component = acc.substr(0, acc.size() - 1);
+#if defined(_WIN32)
+            // A drive root such as "D:" cannot be mkdir'ed (EACCES, not EEXIST).
+            if (component.size() == 2 && component[1] == ':') {
+                continue;
+            }
+#endif
             if (!make_dir(component)) {
                 return false;
             }
@@ -165,16 +166,26 @@ bool ggml_openvino_model_cache_only() {
 }

 static const char * cache_settings[] = {
+    "GGML_OPENVINO_GPU_CONFIG",
+    "GGML_OPENVINO_CPU_CONFIG",
     "GGML_OPENVINO_REQUANT_KQUANT",
     "GGML_OPENVINO_NATIVE_SOFTPLUS",
-    "GGML_OPENVINO_DISABLE_KV_SLICE",
+    // GGML_OPENVINO_KV_SLICE only changes dynamic runtime bindings. Dynamic
+    // attention graphs accept either shape and share the same compiled blob.
+    "GGML_OPENVINO_GDN_NATIVE_STATE",
+    "GGML_OPENVINO_GDN_DECOMPOSE",
+    "GGML_OPENVINO_GDN_DECOMPOSE_CHUNK",
     "GGML_OPENVINO_MANUAL_GQA_ATTN",
+    "GGML_OPENVINO_EXACT_ZERO_POINT",
     "GGML_OPENVINO_STATEFUL_EXECUTION",
     "GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT",
     "GGML_OPENVINO_DISABLE_REMOTE_OUTPUTS",
     "GGML_OPENVINO_REDUCE_COMPILE_MEM",
     "GGML_OPENVINO_MEMORY_OPTIMIZE",
     "GGML_OPENVINO_PROFILING",
+    "GGML_OPENVINO_DISABLE_SHAPE_FROM_MASK",
+    "GGML_OPENVINO_KV_SCATTER_ELEMENTS",
+    "GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN",
 };

 void ggml_openvino_model_cache_init() {
@@ -203,17 +214,15 @@ void ggml_openvino_model_cache_init() {
     }
 }

-uint64_t ggml_openvino_source_fingerprint(const void * data, size_t size, std::vector<ggml_openvino_source_mapping> & mappings) {
+const ggml_openvino_source_mapping * ggml_openvino_find_source_mapping(const void * data, size_t size,
+                                                                     std::vector<ggml_openvino_source_mapping> & mappings) {
     const uintptr_t address = reinterpret_cast<uintptr_t>(data);
     auto contains = [&](const ggml_openvino_source_mapping & m) {
         return address >= m.begin && address < m.end && size <= m.end - address;
     };
-    auto fingerprint = [&](const ggml_openvino_source_mapping & m) {
-        return fnv1a_u64(m.identity, m.offset + address - m.begin);
-    };
     for (const auto & m : mappings) {
         if (contains(m)) {
-            return fingerprint(m);
+            return &m;
         }
     }
 #ifdef __linux__
@@ -228,7 +237,7 @@ uint64_t ggml_openvino_source_fingerprint(const void * data, size_t size, std::v
                    &offset, &dev_major, &dev_minor, &inode, &path_start) != 7 || inode == 0) {
             continue;
         }
-        ggml_openvino_source_mapping m{uintptr_t(begin), uintptr_t(end), offset, FNV_OFFSET};
+        ggml_openvino_source_mapping m{uintptr_t(begin), uintptr_t(end), offset, FNV_OFFSET, permissions[1] != 'w'};
         if (!contains(m)) {
             continue;
         }
@@ -246,7 +255,7 @@ uint64_t ggml_openvino_source_fingerprint(const void * data, size_t size, std::v
         m.identity = fnv1a_u64(m.identity, st.st_ctim.tv_sec);
         m.identity = fnv1a_u64(m.identity, st.st_ctim.tv_nsec);
         mappings.push_back(m);
-        return fingerprint(m);
+        return &mappings.back();
     }
 #elif defined(_WIN32)
     MEMORY_BASIC_INFORMATION memory;
@@ -276,8 +285,10 @@ uint64_t ggml_openvino_source_fingerprint(const void * data, size_t size, std::v
                     const uint64_t file_size = (uint64_t(info.nFileSizeHigh) << 32) | info.nFileSizeLow;
                     const uintptr_t begin = reinterpret_cast<uintptr_t>(memory.AllocationBase);
                     if (file_size <= std::numeric_limits<uintptr_t>::max() - begin) {
+                        // A view created read-only can never be made writable.
+                        const bool read_only = (memory.AllocationProtect & (PAGE_READONLY | PAGE_EXECUTE_READ)) != 0;
                         ggml_openvino_source_mapping m{begin, begin + static_cast<uintptr_t>(file_size), 0,
-                                                       fnv1a(FNV_OFFSET, "win32", 5)};
+                                                       fnv1a(FNV_OFFSET, "win32", 5), read_only};
                         if (contains(m)) {
                             m.identity = fnv1a_u64(m.identity, info.dwVolumeSerialNumber);
                             m.identity = fnv1a_u64(m.identity, (uint64_t(info.nFileIndexHigh) << 32) | info.nFileIndexLow);
@@ -286,7 +297,7 @@ uint64_t ggml_openvino_source_fingerprint(const void * data, size_t size, std::v
                                                                  info.ftLastWriteTime.dwLowDateTime);
                             m.identity = fnv1a_u64(m.identity, static_cast<uint64_t>(basic.ChangeTime.QuadPart));
                             mappings.push_back(m);
-                            return fingerprint(m);
+                            return &mappings.back();
                         }
                     }
                 }
@@ -294,6 +305,13 @@ uint64_t ggml_openvino_source_fingerprint(const void * data, size_t size, std::v
         }
     }
 #endif
+    return nullptr;
+}
+
+uint64_t ggml_openvino_source_fingerprint(const void * data, size_t size, std::vector<ggml_openvino_source_mapping> & mappings) {
+    if (const auto * m = ggml_openvino_find_source_mapping(data, size, mappings)) {
+        return fnv1a_u64(m->identity, m->offset + reinterpret_cast<uintptr_t>(data) - m->begin);
+    }
     if (ggml_openvino_model_cache_only()) {
         GGML_ABORT("ggml-openvino: could not identify mapped GGUF weight; use --load-mode mmap");
     }
@@ -344,12 +362,19 @@ uint64_t ggml_openvino_model_fingerprint(const ggml_cgraph * cgraph,
                                          uint64_t extra_cfg,
                                          const std::string & graph_signature) {
     uint64_t h = FNV_OFFSET;
-    h = fnv1a_u64(h, 2);
+    // Version 3 includes dense Q8_0/Q5_1 requantization in q4_asym64_all.
+    // Version 4 uses GatedDeltaNet output 1 for snapshots in snapshot mode; older
+    // graphs use the incompatible three-output contract.
+    // Version 5 requantizes K-quant MoE experts to Q4_1_64 on GPU by default.
+    // Version 6 clears recurrent-cache slots with Select, so a NaN in a cleared slot does not survive.
+    // Version 7 reads ROPE frequency factors named other than rope_freqs.weight as a Constant, not a Parameter.
+    h = fnv1a_u64(h, 7);
     h = fnv1a(h, graph_signature.data(), graph_signature.size());
     for (const char * name : cache_settings) {
         const char * value = ggml_openvino_getenv_str(name, "");
         h = fnv1a(h, value, strlen(value) + 1);
     }
+    h = fnv1a_u64(h, ggml_openvino_gdn_decompose_enabled() ? ggml_openvino_gdn_decompose_chunk() : 0);
     if (const char * debug_nodes = ggml_openvino_getenv_str("GGML_OPENVINO_DEBUG_NODE")) {
         h = fnv1a(h, "GGML_OPENVINO_DEBUG_NODE", sizeof("GGML_OPENVINO_DEBUG_NODE"));
         h = fnv1a(h, debug_nodes, strlen(debug_nodes) + 1);
@@ -361,10 +386,19 @@ uint64_t ggml_openvino_model_fingerprint(const ggml_cgraph * cgraph,
     // Topology: node count + each node's op and name (cheap, and distinguishes
     // graphs that share weights but differ structurally).
     h = fnv1a_u64(h, static_cast<uint64_t>(cgraph->n_nodes));
+    bool has_gdn = false;
     for (int i = 0; i < cgraph->n_nodes; ++i) {
         const ggml_tensor * node = cgraph->nodes[i];
         h = fnv1a_u64(h, static_cast<uint64_t>(node->op));
         h = fnv1a(h, node->name, strlen(node->name));
+        has_gdn = has_gdn || node->op == GGML_OP_GATED_DELTA_NET;
+    }
+    if (has_gdn && ggml_openvino_is_npu()) {
+        // The GDN q pre-scale factor is a constant in the graph (0: no pre-scale).
+        const float q_scale = ggml_openvino_gdn_prescale_enabled() ? ggml_openvino_gdn_q_scale() : 0.0f;
+        h = fnv1a(h, &q_scale, sizeof(q_scale));
+        // GDN graphs switch to the PLUGIN compiler unless the environment sets the compiler type.
+        h = fnv1a_u64(h, ggml_openvino_npu_gdn_uses_plugin_compiler() ? 1u : 0u);
     }

     // Weights: the model identity.
@@ -482,3 +516,196 @@ bool ggml_openvino_model_cache_verify_manifest(const std::string & path,
     f >> std::ws;
     return f.eof();
 }
+
+std::string ggml_openvino_model_cache_weights_path(const std::string & dir, uint64_t fp_a, uint64_t fp_b) {
+    return dir + "/" + hex64(fp_a) + "-" + hex64(fp_b) + ".weights.bin";
+}
+
+static uint64_t weights_hash_mix(uint64_t h, uint64_t v) {
+    h ^= v + 0x9e3779b97f4a7c15ULL + (h << 6) + (h >> 2);
+    return h * 0x100000001b3ULL;
+}
+
+static uint64_t weights_hash_bytes(uint64_t h, const void * data, size_t size) {
+    const auto * p = static_cast<const unsigned char *>(data);
+    size_t i = 0;
+    for (; i + 8 <= size; i += 8) {
+        uint64_t w;
+        std::memcpy(&w, p + i, 8);
+        h = (h ^ w) * 0x100000001b3ULL;
+    }
+    for (; i < size; ++i) {
+        h = (h ^ p[i]) * 0x100000001b3ULL;
+    }
+    return h;
+}
+
+// Content hash recorded in a weights file's size record, or false if it has none.
+static bool read_weights_hash(const std::string & path, uint64_t & hash) {
+    std::ifstream meta(path + ".meta");
+    std::string key, value;
+    while (meta >> key >> value) {
+        if (key == "hash") {
+            hash = std::stoull(value, nullptr, 16);
+            return true;
+        }
+    }
+    return false;
+}
+
+std::shared_ptr<ggml_openvino_weights_layout> ggml_openvino_model_cache_write_weights(
+    const std::string & path,
+    const std::vector<std::shared_ptr<ov::Model>> & models,
+    bool keep_existing,
+    ggml_openvino_weights_status * status) {
+    struct region {
+        const void * data;
+        size_t size;
+        size_t offset;
+    };
+    constexpr size_t alignment = 64;
+    auto layout = std::make_shared<ggml_openvino_weights_layout>();
+    layout->path = path;
+    std::vector<region> regions;
+    std::unordered_map<const void *, size_t> region_of;
+    for (const auto & model : models) {
+        if (!model) {
+            continue;
+        }
+        for (const auto & node : model->get_ordered_ops()) {
+            auto cnst = ov::as_type_ptr<ov::op::v0::Constant>(node);
+            if (!cnst || cnst->get_rt_info().count(ov::WeightlessCacheAttribute::get_type_info_static()) == 0 ||
+                cnst->get_data_ptr() == nullptr || cnst->get_byte_size() == 0) {
+                continue;
+            }
+            layout->constants.push_back(cnst);
+            auto [it, inserted] = region_of.emplace(cnst->get_data_ptr(), regions.size());
+            if (inserted) {
+                regions.push_back({ cnst->get_data_ptr(), cnst->get_byte_size(), 0 });
+            } else {
+                regions[it->second].size = std::max(regions[it->second].size, cnst->get_byte_size());
+            }
+        }
+    }
+
+    size_t total = 0;
+    for (auto & r : regions) {
+        r.offset = total;
+        total = (total + r.size + alignment - 1) / alignment * alignment;
+        layout->regions[r.data] = { r.offset, r.size };
+    }
+    for (const auto & cnst : layout->constants) {
+        cnst->get_rt_info()[ov::WeightlessCacheAttribute::get_type_info_static()] = ov::WeightlessCacheAttribute(
+            cnst->get_byte_size(), layout->regions.at(cnst->get_data_ptr()).first, cnst->get_element_type());
+    }
+
+    uint64_t content_hash = weights_hash_mix(0xcbf29ce484222325ULL, total);
+    for (const auto & r : regions) {
+        content_hash = weights_hash_mix(content_hash, r.offset);
+        content_hash = weights_hash_mix(content_hash, r.size);
+        content_hash = weights_hash_bytes(content_hash, r.data, r.size);
+    }
+
+    if (keep_existing && ggml_openvino_model_cache_weights_valid(path)) {
+        uint64_t existing_hash = 0;
+        if (read_weights_hash(path, existing_hash) && existing_hash == content_hash) {
+            if (status) {
+                *status = ggml_openvino_weights_status::reused;
+            }
+            GGML_LOG_INFO("ggml-openvino: model cache REUSED %s (%zu regions, %.1f MiB)\n", path.c_str(),
+                          regions.size(), total / 1024.0 / 1024.0);
+            return layout;
+        }
+        if (status) {
+            *status = ggml_openvino_weights_status::mismatch;
+        }
+        return nullptr;
+    }
+    if (status) {
+        *status = ggml_openvino_weights_status::written;
+    }
+
+    // Publish the data file first and its size record last, so a valid record always describes a complete file.
+    const std::string data_tmp = path + ".tmp";
+    const std::string meta_path = path + ".meta";
+    const std::string meta_tmp = meta_path + ".tmp";
+    {
+        std::ofstream f(data_tmp, std::ios::binary | std::ios::trunc);
+        if (!f.is_open()) {
+            return nullptr;
+        }
+        static const char zeros[alignment] = {};
+        size_t pos = 0;
+        for (const auto & r : regions) {
+            f.write(zeros, static_cast<std::streamsize>(r.offset - pos));
+            f.write(static_cast<const char *>(r.data), static_cast<std::streamsize>(r.size));
+            pos = r.offset + r.size;
+        }
+        f.write(zeros, static_cast<std::streamsize>(total - pos));
+        f.close();
+        if (!f.good()) {
+            std::remove(data_tmp.c_str());
+            return nullptr;
+        }
+    }
+    {
+        std::ofstream f(meta_tmp, std::ios::trunc);
+        f << "size " << total << "\nregions " << regions.size() << "\nhash " << hex64(content_hash) << "\n";
+        f.close();
+        if (!f.good()) {
+            std::remove(data_tmp.c_str());
+            std::remove(meta_tmp.c_str());
+            return nullptr;
+        }
+    }
+    // std::rename does not replace an existing file on Windows.
+    ggml_openvino_model_cache_remove_weights(path);
+    if (std::rename(data_tmp.c_str(), path.c_str()) != 0 || std::rename(meta_tmp.c_str(), meta_path.c_str()) != 0) {
+        std::remove(data_tmp.c_str());
+        std::remove(meta_tmp.c_str());
+        return nullptr;
+    }
+    GGML_LOG_INFO("ggml-openvino: model cache WROTE %s (%zu regions, %.1f MiB)\n", path.c_str(), regions.size(),
+                  total / 1024.0 / 1024.0);
+    return layout;
+}
+
+size_t ggml_openvino_model_cache_retag_weights(const std::shared_ptr<ov::Model> & model,
+                                               const ggml_openvino_weights_layout & layout) {
+    size_t resolved = 0;
+    for (const auto & node : model->get_ordered_ops()) {
+        auto cnst = ov::as_type_ptr<ov::op::v0::Constant>(node);
+        if (!cnst) {
+            continue;
+        }
+        auto & rt_info = cnst->get_rt_info();
+        auto attr = rt_info.find(ov::WeightlessCacheAttribute::get_type_info_static());
+        if (attr == rt_info.end()) {
+            continue;
+        }
+        auto it = layout.regions.find(cnst->get_data_ptr());
+        if (it != layout.regions.end() && cnst->get_byte_size() <= it->second.second) {
+            attr->second = ov::WeightlessCacheAttribute(cnst->get_byte_size(), it->second.first, cnst->get_element_type());
+            ++resolved;
+        } else {
+            rt_info.erase(attr);
+        }
+    }
+    return resolved;
+}
+
+bool ggml_openvino_model_cache_weights_valid(const std::string & path) {
+    std::ifstream meta(path + ".meta");
+    std::string key;
+    uint64_t size = 0;
+    if (!(meta >> key >> size) || key != "size") {
+        return false;
+    }
+    std::ifstream f(path, std::ios::binary | std::ios::ate);
+    return f.is_open() && static_cast<uint64_t>(f.tellg()) == size;
+}
+
+void ggml_openvino_model_cache_remove_weights(const std::string & path) {
+    std::remove((path + ".meta").c_str());
+    std::remove(path.c_str());
+}
diff --git a/ggml/src/ggml-openvino/model-cache.h b/ggml/src/ggml-openvino/model-cache.h
index ee69fca83..25acadeee 100644
--- a/ggml/src/ggml-openvino/model-cache.h
+++ b/ggml/src/ggml-openvino/model-cache.h
@@ -1,13 +1,36 @@
 #pragma once

 // Compiled blobs include weights. Cache-only execution skips weight uploads and graph compilation.
+// On NPU, GGML_OPENVINO_SELF_CONTAINED_BLOB=1 exports embedded-weight prefill/decode
+// blobs. A warm hit skips weight requantization, graph conversion, and compilation,
+// at the cost of roughly model-size storage per blob and separate NPUW weight banks.
+// After both blobs exist, GGML_OPENVINO_SELF_CONTAINED_MMAP=1 additionally wraps the
+// GGUF mmap instead of allocating the private OpenVINO model buffer. This mode is
+// import-only and fails clearly if either required blob is missing or invalid.
+// GGML_OPENVINO_NPU_WEIGHTLESS_BANK=1 is the same raw-weight flow, but exports weightless
+// prefill/decode blobs plus ONE <weights_fp>-<weights_fp>.weights.bin shared by every prefill/decode pair
+// of the model (pairs differ only in KV window). Warm import reads the weights from that file
+// (ov::weights_path) and NPUW deduplicates all models into one bank
+// (no OpenVINO patch needed), so the weights are stored once on disk and resident once.

 #include "ggml.h"

 #include <cstdint>
+#include <memory>
 #include <string>
+#include <unordered_map>
+#include <utility>
 #include <vector>

+namespace ov {
+class Model;
+namespace op {
+namespace v0 {
+class Constant;
+}
+}  // namespace op
+}  // namespace ov
+
 bool ggml_openvino_model_cache_only();
 void ggml_openvino_model_cache_init();

@@ -16,8 +39,13 @@ struct ggml_openvino_source_mapping {
     uintptr_t end;
     uint64_t offset;
     uint64_t identity;
+    // The mapping cannot be written, so its pages always hold the file's bytes.
+    bool read_only;
 };

+// Find the file mapping that holds [data, data + size), or nullptr. Cache mappings for one buffer lifetime.
+const ggml_openvino_source_mapping * ggml_openvino_find_source_mapping(const void * data, size_t size,
+                                                                     std::vector<ggml_openvino_source_mapping> & mappings);
 // Identify mmap weights without reading their pages. Cache mappings for one buffer lifetime.
 uint64_t ggml_openvino_source_fingerprint(const void * data, size_t size, std::vector<ggml_openvino_source_mapping> & mappings);
 uint64_t ggml_backend_openvino_weight_fingerprint(const ggml_tensor * tensor);
@@ -55,3 +83,39 @@ bool ggml_openvino_model_cache_verify_manifest(const std::string & path,
                                                uint64_t fingerprint,
                                                std::vector<std::string> & inputs,
                                                std::vector<std::string> & outputs);
+
+// GGML_OPENVINO_NPU_WEIGHTLESS_BANK: a weights file shared by weightless blobs. Every weightless-tagged
+// Constant is one region keyed by its data pointer (NPUW maps a Constant to its file offset by pointer).
+struct ggml_openvino_weights_layout {
+    std::string path;
+    // data pointer -> (offset, size) of its region in the file.
+    std::unordered_map<const void *, std::pair<size_t, size_t>> regions;
+    // Keeps the laid-out Constants alive so no other Constant can reuse one of their data pointers.
+    std::vector<std::shared_ptr<ov::op::v0::Constant>> constants;
+};
+
+// <dir>/<a>-<b>.weights.bin (NPUW's weightless import needs the .bin extension).
+std::string ggml_openvino_model_cache_weights_path(const std::string & dir, uint64_t fp_a, uint64_t fp_b);
+
+enum class ggml_openvino_weights_status { written, reused, mismatch };
+
+// Lay out all weightless-tagged Constants of |models| in one file, retag their WeightlessCacheAttribute with the
+// real offsets and write the file. Call before compiling. Returns nullptr on I/O error.
+// With |keep_existing|, a valid file at |path| is never rewritten: it is reused if its layout and content hash
+// match (status reused), otherwise nullptr is returned (status mismatch).
+std::shared_ptr<ggml_openvino_weights_layout> ggml_openvino_model_cache_write_weights(
+    const std::string & path,
+    const std::vector<std::shared_ptr<ov::Model>> & models,
+    bool keep_existing = false,
+    ggml_openvino_weights_status * status = nullptr);
+
+// Point the tagged Constants of |model| that live in |layout| at their regions and untag the rest, which NPUW
+// then embeds in the blob. Returns the number of Constants resolved through the file.
+size_t ggml_openvino_model_cache_retag_weights(const std::shared_ptr<ov::Model> & model,
+                                               const ggml_openvino_weights_layout & layout);
+
+// True if the weights file is complete (its size matches the record written after it).
+bool ggml_openvino_model_cache_weights_valid(const std::string & path);
+
+// Remove a weights file and its size record.
+void ggml_openvino_model_cache_remove_weights(const std::string & path);
diff --git a/ggml/src/ggml-openvino/openvino/decoder.h b/ggml/src/ggml-openvino/openvino/decoder.h
index ec6975282..980f0b88d 100644
--- a/ggml/src/ggml-openvino/openvino/decoder.h
+++ b/ggml/src/ggml-openvino/openvino/decoder.h
@@ -23,6 +23,9 @@ struct ModelExtraInputInfo {
     Shape shape;
     int64_t value;
     bool is_parameter;
+    // when set, the value is read from dimension shape_axis of this model input instead of a Parameter
+    std::string shape_source;
+    int64_t shape_axis = -1;
 };

 class GgmlDecoder : public DecoderBase {
@@ -121,6 +124,8 @@ public:

     virtual bool is_stateful() const = 0;

+    virtual bool is_prefill() const = 0;
+
     virtual bool is_splited_model() const = 0;

     virtual int is_swa_layer(int layer) const = 0;
diff --git a/ggml/src/ggml-openvino/openvino/node_context.h b/ggml/src/ggml-openvino/openvino/node_context.h
index c653bc0e6..948405745 100644
--- a/ggml/src/ggml-openvino/openvino/node_context.h
+++ b/ggml/src/ggml-openvino/openvino/node_context.h
@@ -162,6 +162,8 @@ public:

     bool is_stateful() const { return m_decoder->is_stateful(); }

+    bool is_prefill() const { return m_decoder->is_prefill(); }
+
     int get_ssm_state_size() const { return m_decoder->get_ssm_state_size(); }

 private:
diff --git a/ggml/src/ggml-openvino/openvino/op/arange.cpp b/ggml/src/ggml-openvino/openvino/op/arange.cpp
new file mode 100644
index 000000000..e3d8b923c
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/op/arange.cpp
@@ -0,0 +1,22 @@
+#include "../node_context.h"
+#include "../op_table.h"
+#include "../utils.h"
+
+#include <cstring>
+#include <openvino/op/constant.hpp>
+#include <vector>
+
+namespace ov::frontend::ggml::op {
+OutputVector translate_arange(const NodeContext & context) {
+    num_inputs_check(context, 0, 0);
+    float params[3];
+    memcpy(params, context.get_output_op_params(), sizeof(params));
+    // GGML ARANGE has fixed op parameters, including for rollback slot order.
+    std::vector<float> values(ov::shape_size(context.get_output_shape().to_shape()));
+    for (size_t i = 0; i < values.size(); ++i) {
+        values[i] = params[0] + params[2] * i;
+    }
+    auto result = ov::op::v0::Constant::create(context.get_output_type(), context.get_output_shape().to_shape(), values);
+    return rename_outputs_with_suffix({result}, context.get_name());
+}
+}  // namespace ov::frontend::ggml::op
diff --git a/ggml/src/ggml-openvino/openvino/op/argsort.cpp b/ggml/src/ggml-openvino/openvino/op/argsort.cpp
index dd0872516..a3ae09fbd 100644
--- a/ggml/src/ggml-openvino/openvino/op/argsort.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/argsort.cpp
@@ -2,9 +2,16 @@
 #include "../op_table.h"
 #include "../utils.h"
 #include "ggml.h"
+#include "ggml-openvino/ggml-openvino-extra.h"

+#include <climits>
+#include <cstdint>
 #include <openvino/frontend/exception.hpp>
+#include <openvino/op/add.hpp>
+#include <openvino/op/concat.hpp>
 #include <openvino/op/constant.hpp>
+#include <openvino/op/gather_elements.hpp>
+#include <openvino/op/reshape.hpp>
 #include <openvino/op/squeeze.hpp>
 #include <openvino/op/topk.hpp>

@@ -45,6 +52,52 @@ OutputVector translate_argsort(const NodeContext & context) {
     return rename_outputs_with_suffix({topk->output(1)}, context.get_name());
 }

+OutputVector translate_top_k(const NodeContext & context) {
+    num_inputs_check(context, 1, 1);
+    auto input = process_view_input_new(context, 0);
+    const int64_t k = context.get_output_shape()[3].get_length();
+    const auto shape = input.get_partial_shape();
+    if (ggml_openvino_is_gpu() && shape.rank() == 4 && shape[3].is_static() &&
+        shape[3].get_length() > 65535 && shape[3].get_length() <= INT32_MAX && k <= 64) {
+        const int64_t width = shape[3].get_length();
+        int64_t chunk_size = 4096;
+        while (chunk_size >= 1024 && width % chunk_size != 0) {
+            --chunk_size;
+        }
+        if (chunk_size >= 1024) {
+            const int64_t chunks = width / chunk_size;
+            auto chunked = std::make_shared<ov::op::v1::Reshape>(input,
+                ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{-1, 1, chunks, chunk_size}), false);
+            // Each global top-k candidate is in its chunk's top-k set.
+            // Value sorting selects the parallel GPU kernel; ggml allows any index order.
+            auto local = std::make_shared<ov::op::v11::TopK>(chunked,
+                ov::op::v0::Constant::create(ov::element::i32, {}, {k}), -1,
+                ov::op::v11::TopK::Mode::MAX, ov::op::v11::TopK::SortType::SORT_VALUES, ov::element::i32, false);
+            std::vector<int32_t> offsets(chunks);
+            for (int64_t i = 0; i < chunks; ++i) {
+                offsets[i] = i * chunk_size;
+            }
+            auto indices = std::make_shared<ov::op::v1::Add>(local->output(1),
+                ov::op::v0::Constant::create(ov::element::i32, {1, 1, size_t(chunks), 1}, offsets));
+            auto candidate_shape = ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{1, 1, -1, chunks * k});
+            auto values = std::make_shared<ov::op::v1::Reshape>(local->output(0), candidate_shape, false);
+            auto candidate_indices = std::make_shared<ov::op::v1::Reshape>(indices, candidate_shape, false);
+            auto selected = std::make_shared<ov::op::v11::TopK>(values,
+                ov::op::v0::Constant::create(ov::element::i32, {}, {k}), -1,
+                ov::op::v11::TopK::Mode::MAX, ov::op::v11::TopK::SortType::SORT_VALUES, ov::element::i32, false);
+            auto result = std::make_shared<ov::op::v6::GatherElements>(candidate_indices, selected->output(1), 3);
+            auto output_shape = std::make_shared<ov::op::v0::Concat>(OutputVector{
+                get_dimensions(input.get_node_shared_ptr(), {0, 1, 2}),
+                ov::op::v0::Constant::create(ov::element::i64, {1}, {k})}, 0);
+            return rename_outputs_with_suffix({std::make_shared<ov::op::v1::Reshape>(result, output_shape, false)}, context.get_name());
+        }
+    }
+    auto topk = std::make_shared<ov::op::v11::TopK>(
+        input, ov::op::v0::Constant::create(ov::element::i32, {}, {k}), -1,
+        ov::op::v11::TopK::Mode::MAX, ov::op::v11::TopK::SortType::NONE, ov::element::i32, false);
+    return rename_outputs_with_suffix({topk->output(1)}, context.get_name());
+}
+
 }  // namespace op
 }  // namespace ggml
 }  // namespace frontend
diff --git a/ggml/src/ggml-openvino/openvino/op/cpy.cpp b/ggml/src/ggml-openvino/openvino/op/cpy.cpp
index 2c7299a89..0bb6bd054 100644
--- a/ggml/src/ggml-openvino/openvino/op/cpy.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/cpy.cpp
@@ -1,7 +1,6 @@
 #include "../node_context.h"
 #include "../op_table.h"
 #include "../utils.h"
-
 #include <climits>
 #include <cstdint>
 #include <cstdio>
@@ -12,7 +11,9 @@
 #include <openvino/op/concat.hpp>
 #include <openvino/op/constant.hpp>
 #include <openvino/op/convert.hpp>
+#include <openvino/op/divide.hpp>
 #include <openvino/op/gather.hpp>
+#include <openvino/op/mod.hpp>
 #include <openvino/op/multiply.hpp>
 #include <openvino/op/negative.hpp>
 #include <openvino/op/range.hpp>
@@ -22,6 +23,7 @@
 #include <openvino/op/slice.hpp>
 #include <openvino/op/squeeze.hpp>
 #include <openvino/op/subtract.hpp>
+#include <openvino/op/unsqueeze.hpp>
 #include <vector>

 namespace ov {
@@ -34,6 +36,45 @@ OutputVector translate_cpy(const NodeContext & context) {
     auto input_shape = context.get_input_shape(0);
     auto output_shape = context.get_input_shape(1);

+    if (op_case == 11 || op_case == 12) {
+        auto base = context.get_input(1);
+        auto axis = ov::op::v0::Constant::create(ov::element::i64, {1}, {2});
+        auto zero = ov::op::v0::Constant::create(ov::element::i64, {1}, {0});
+        auto one = ov::op::v0::Constant::create(ov::element::i64, {1}, {1});
+        auto feature = ov::op::v0::Constant::create(ov::element::i64, {4},
+            std::vector<int64_t>{1, 1, -1, output_shape[3].get_length()});
+        ov::Output<ov::Node> state_part = context.get_input(0);
+        if (op_case == 11) {
+            auto source_rows = std::make_shared<ov::op::v8::Gather>(
+                std::make_shared<ov::op::v3::ShapeOf>(context.get_input(0), ov::element::i64), axis,
+                ov::op::v0::Constant::create(ov::element::i64, {}, {0}));
+            auto source_indices = std::make_shared<ov::op::v4::Range>(
+                std::make_shared<ov::op::v0::Squeeze>(context.get_input("rs_src_begin_" + context.get_name())),
+                std::make_shared<ov::op::v0::Squeeze>(source_rows),
+                std::make_shared<ov::op::v0::Squeeze>(one), ov::element::i64);
+            state_part = std::make_shared<ov::op::v8::Gather>(context.get_input(0), source_indices,
+                ov::op::v0::Constant::create(ov::element::i64, {}, {2}));
+        }
+        auto states = std::make_shared<ov::op::v1::Reshape>(state_part, feature, false);
+        auto rows =
+            std::make_shared<ov::op::v8::Gather>(std::make_shared<ov::op::v3::ShapeOf>(states, ov::element::i64), axis,
+                                                 ov::op::v0::Constant::create(ov::element::i64, {}, {0}));
+        auto indices = std::make_shared<ov::op::v4::Range>(
+            std::make_shared<ov::op::v0::Squeeze>(zero), std::make_shared<ov::op::v0::Squeeze>(rows),
+            std::make_shared<ov::op::v0::Squeeze>(one), ov::element::i64);
+        auto seq_count = context.get_input("rs_seq_count_" + context.get_name());
+        auto plane = std::make_shared<ov::op::v1::Divide>(indices, seq_count);
+        auto sequence = std::make_shared<ov::op::v1::Mod>(indices, seq_count);
+        auto slots = std::make_shared<ov::op::v1::Add>(
+            std::make_shared<ov::op::v1::Multiply>(plane, context.get_input("rs_snapshot_stride_" + context.get_name())),
+            std::make_shared<ov::op::v1::Add>(sequence, context.get_input("rs_slot_begin_" + context.get_name())));
+        auto updated = std::make_shared<ov::op::v3::ScatterUpdate>(
+            std::make_shared<ov::op::v1::Reshape>(base, feature, false), slots, states, axis);
+        auto result = std::make_shared<ov::op::v1::Reshape>(
+            updated, std::make_shared<ov::op::v3::ShapeOf>(base, ov::element::i64), false);
+        return rename_outputs_with_suffix({result}, context.get_name());
+    }
+
     if (op_case == 4) {
         auto src = process_view_input_new(context, 0);
         auto base = context.get_input(1);
@@ -73,10 +114,9 @@ OutputVector translate_cpy(const NodeContext & context) {
     // runtime input, so the cached model works for any kv head and active sequence count. The
     // result is the full updated cache.
     // op_case 1: gated-delta-net state, op_case 2: conv state, op_case 3: defrag remainder.
-    if (op_case == 3) {
-        // With -np 1 (and generally whenever there is no defrag remainder) this GET_ROWS gathers
-        // zero rows: nothing to write back, and the cache is unchanged. NPU rejects zero-size
-        // tensors, so short-circuit instead of building a degenerate Slice/Concat chain.
+    if (op_case == 3 && context.is_static()) {
+        // A statically empty remainder leaves the cache unchanged at any slot capacity.
+        // NPU rejects zero-size tensors, so avoid a degenerate Slice/Concat chain.
         bool is_empty = false;
         if (input_shape.rank().is_static()) {
             for (const auto & d : input_shape) {
@@ -90,24 +130,13 @@ OutputVector translate_cpy(const NodeContext & context) {
             return {context.get_input(1)};
         }
     }
-    // op_case 7/8/9 are the single-slot variants; op_case 10 writes native GDN state into a
-    // multi-slot cache without rollback snapshots.
-    const bool single_slot_assign = op_case >= 7 && op_case <= 9;
-    const bool direct_gdn_state = op_case == 7 || op_case == 10;
-    int writeback_case = op_case;
-    if (op_case == 10) {
-        writeback_case = 1;
-    } else if (single_slot_assign) {
-        writeback_case = op_case - 6;
-    }
+    // op_case 10 consumes native GDN state without rollback snapshots.
+    const bool direct_gdn_state = op_case == 10;
+    const int writeback_case = direct_gdn_state ? 1 : op_case;
     const std::string slot_begin_name = "rs_slot_begin_" + context.get_name();
     const bool slice_assign = writeback_case >= 1 && writeback_case <= 3 &&
-                              (single_slot_assign || context.has_input(slot_begin_name));
+                              context.has_input(slot_begin_name);
     if (slice_assign) {
-        if (single_slot_assign && writeback_case == 3) {
-            return {context.get_input(1)};
-        }
-
         const int64_t slot_axis = 2;
         auto zero = ov::op::v0::Constant::create(ov::element::i64, {1}, {0});
         auto one = ov::op::v0::Constant::create(ov::element::i64, {1}, {1});
@@ -117,15 +146,12 @@ OutputVector translate_cpy(const NodeContext & context) {
                                                     std::vector<int64_t>{1, 1, -1, output_shape[3].get_length()});

         ov::Output<ov::Node> src;
-        ov::Output<ov::Node> begin;
-        if (!single_slot_assign) {
-            begin = context.get_input(slot_begin_name);
-        }
+        auto begin = context.get_input(slot_begin_name);
         auto base = context.get_input(1);
         if (writeback_case == 1) {
             if (direct_gdn_state) {
                 // Non-rollback GDN publishes state directly as [active_slots, heads, value_dim,
-                // key_dim]. Flatten each active slot before replacing or updating the cache.
+                // key_dim]. Flatten each active slot before updating the cache.
                 src = std::make_shared<ov::op::v1::Reshape>(context.get_input(0), feature, false);
             } else {
                 // Multi-slot rollback still consumes GGML's packed [attention | state snapshots]
@@ -145,9 +171,12 @@ OutputVector translate_cpy(const NodeContext & context) {
             const std::string src_begin_name = "rs_src_begin_" + context.get_name();
             if (context.has_input(src_begin_name)) {
                 auto src_begin = context.get_input(src_begin_name);
-                auto src_end = std::make_shared<ov::op::v1::Add>(
-                    src_begin, ov::op::v0::Constant::create(ov::element::i64, {1}, {window_size}));
-                window = std::make_shared<ov::op::v8::Slice>(context.get_input(0), src_begin, src_end, one, col_axis);
+                std::vector<int64_t> offsets(window_size);
+                std::iota(offsets.begin(), offsets.end(), 0);
+                auto indices = std::make_shared<ov::op::v1::Add>(
+                    src_begin, ov::op::v0::Constant::create(ov::element::i64, {size_t(window_size)}, offsets));
+                window = std::make_shared<ov::op::v8::Gather>(context.get_input(0), indices,
+                    ov::op::v0::Constant::create(ov::element::i64, {}, {3}));
             } else if (context.has_input("chunk_valid_len")) {
                 std::vector<int64_t> offsets(window_size);
                 std::iota(offsets.begin(), offsets.end(), 0);
@@ -181,20 +210,16 @@ OutputVector translate_cpy(const NodeContext & context) {
             FRONT_END_OP_CONVERSION_CHECK(feature_begin >= 0 && feature_end <= full_feature_size,
                                           "CPY conv state cache update feature range is out of bounds");

-            auto partial_feature = ov::op::v0::Constant::create(
-                ov::element::i64, {4}, std::vector<int64_t>{1, 1, -1, update_feature_size});
+            auto partial_feature = ov::op::v0::Constant::create(ov::element::i64, {4},
+                                                                std::vector<int64_t>{1, 1, -1, update_feature_size});
             src = std::make_shared<ov::op::v1::Reshape>(window, partial_feature, false);
             if (src.get_element_type() != context.get_output_type()) {
                 src = std::make_shared<ov::op::v0::Convert>(src, context.get_output_type());
             }

-            if (single_slot_assign) {
-                return rename_outputs_with_suffix({src}, context.get_name());
-            }
-
-            auto src_len = std::make_shared<ov::op::v8::Gather>(
-                std::make_shared<ov::op::v3::ShapeOf>(src, ov::element::i64), axis,
-                ov::op::v0::Constant::create(ov::element::i64, {}, {0}));
+            auto src_len =
+                std::make_shared<ov::op::v8::Gather>(std::make_shared<ov::op::v3::ShapeOf>(src, ov::element::i64), axis,
+                                                     ov::op::v0::Constant::create(ov::element::i64, {}, {0}));
             auto slot_end = std::make_shared<ov::op::v1::Add>(begin, src_len);
             auto active_slots = std::make_shared<ov::op::v8::Slice>(base, begin, slot_end, one, axis);

@@ -209,16 +234,26 @@ OutputVector translate_cpy(const NodeContext & context) {
         } else {
             // op_case 3: gathered remainder rows already have the cache slot layout [1, 1, extra, feature]
             src = context.get_input(0);
+            if (context.get_view_input_size(0) > 0 && !context.is_static()) {
+                // build_rs gathers active and extra rows together. The remainder boundary moves
+                // with the active sequences, so do not capture the first batch's VIEW offset.
+
+                // gathered rows can be [1, rows, feature]; add the slot axis
+                const auto src_rank = src.get_partial_shape().rank();
+                if (src_rank.is_static() && src_rank.get_length() == 3) {
+                    src = std::make_shared<ov::op::v0::Unsqueeze>(src, zero);
+                }
+                src = std::make_shared<ov::op::v8::Slice>(src,
+                    context.get_input("rs_src_begin_" + context.get_name()), int_max, one, axis);
+            } else {
+                src = process_view_input_new(context, 0);
+            }
         }

         if (src.get_element_type() != context.get_output_type()) {
             src = std::make_shared<ov::op::v0::Convert>(src, context.get_output_type());
         }

-        if (single_slot_assign) {
-            return rename_outputs_with_suffix({src}, context.get_name());
-        }
-
         auto src_len =
             std::make_shared<ov::op::v8::Gather>(std::make_shared<ov::op::v3::ShapeOf>(src, ov::element::i64), axis,
                                                  ov::op::v0::Constant::create(ov::element::i64, {}, {0}));
diff --git a/ggml/src/ggml-openvino/openvino/op/div.cpp b/ggml/src/ggml-openvino/openvino/op/div.cpp
index 2089ffd4c..8bd060c62 100644
--- a/ggml/src/ggml-openvino/openvino/op/div.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/div.cpp
@@ -66,6 +66,10 @@ ov::Output<ov::Node> repeat_input_to_match(const NodeContext & context,
             const int64_t input_dim = input_shape[axis].get_length();
             const int64_t target_dim = target_shape[axis].get_length();

+            // Empty token axes can occur in a MoE warmup with no logits.
+            if (input_dim == target_dim) {
+                continue;
+            }
             FRONT_END_OP_CONVERSION_CHECK(input_dim > 0 && target_dim > 0 && target_dim % input_dim == 0,
                                           "DIV input shape ", input_shape, " cannot repeat to match ", target_shape);

diff --git a/ggml/src/ggml-openvino/openvino/op/fill.cpp b/ggml/src/ggml-openvino/openvino/op/fill.cpp
index 87358e12e..8f0689510 100644
--- a/ggml/src/ggml-openvino/openvino/op/fill.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/fill.cpp
@@ -4,6 +4,7 @@

 #include <openvino/op/broadcast.hpp>
 #include <openvino/op/constant.hpp>
+#include <openvino/op/shape_of.hpp>

 namespace ov {
 namespace frontend {
@@ -15,14 +16,16 @@ namespace op {
 OutputVector translate_fill(const NodeContext & context) {
     num_inputs_check(context, 1, 1);

+    if (is_recurrent_cache_reset_case(context)) {
+        return translate_recurrent_cache_reset(context);
+    }
+
     float c;
     memcpy(&c, context.get_output_op_params(), sizeof(float));

-    auto shape = context.get_input_shape(0).to_shape();
-
+    // the shape of the input, which keeps its dynamic token dim
+    auto target_shape = std::make_shared<ov::op::v3::ShapeOf>(process_view_input_new(context, 0), ov::element::i64);
     auto val = ov::op::v0::Constant::create(context.get_output_type(), {}, {c});
-    auto target_shape = ov::op::v0::Constant::create(ov::element::i64, {shape.size()},
-        std::vector<int64_t>(shape.begin(), shape.end()));
     auto res = std::make_shared<ov::op::v3::Broadcast>(val, target_shape);

     return rename_outputs_with_suffix({res}, context.get_name());
diff --git a/ggml/src/ggml-openvino/openvino/op/flash_attn_ext.cpp b/ggml/src/ggml-openvino/openvino/op/flash_attn_ext.cpp
index 44c6dbc03..065e92fcd 100644
--- a/ggml/src/ggml-openvino/openvino/op/flash_attn_ext.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/flash_attn_ext.cpp
@@ -12,19 +12,69 @@
 #include <openvino/op/constant.hpp>
 #include <openvino/op/convert.hpp>
 #include <openvino/op/matmul.hpp>
+#include <openvino/op/maximum.hpp>
 #include <openvino/op/multiply.hpp>
 #include <openvino/op/reshape.hpp>
 #include <openvino/op/scaled_dot_product_attention.hpp>
+#include <openvino/op/scatter_update.hpp>
 #include <openvino/op/slice.hpp>
 #include <openvino/op/softmax.hpp>
+#include <openvino/op/tanh.hpp>
+#include <openvino/op/tile.hpp>
 #include <openvino/op/transpose.hpp>
 #include <openvino/op/unsqueeze.hpp>
+#include <optional>
 #include <string>
+#include <unordered_set>

 namespace ov {
 namespace frontend {
 namespace ggml {
 namespace op {
+static std::optional<ov::Output<ov::Node>> find_scatter_updates(const ov::Output<ov::Node> & root) {
+    std::vector<ov::Output<ov::Node>> pending{root};
+    std::unordered_set<const ov::Node *> visited;
+    std::optional<ov::Output<ov::Node>> updates;
+    while (!pending.empty()) {
+        auto output = pending.back();
+        pending.pop_back();
+        auto node = output.get_node_shared_ptr();
+        if (!node || !visited.insert(node.get()).second) {
+            continue;
+        }
+        if (auto scatter = ov::as_type_ptr<ov::op::v3::ScatterUpdate>(node)) {
+            OPENVINO_ASSERT(!updates.has_value(), "HFA KV input contains multiple ScatterUpdate ancestors");
+            updates = scatter->input_value(2);
+            continue;
+        }
+        for (const auto & input : node->inputs()) {
+            pending.push_back(input.get_source_output());
+        }
+    }
+    return updates;
+}
+
+static ov::Output<ov::Node> reshape_present_kv(const ov::Output<ov::Node> & updates,
+                                                int64_t n_heads_kv,
+                                                int64_t head_size) {
+    auto shape = updates.get_partial_shape();
+    OPENVINO_ASSERT(shape.rank().is_static() && shape.rank().get_length() == 4,
+                    "HFA present KV updates must be rank 4");
+    auto reshaped = std::make_shared<ov::op::v1::Reshape>(
+        updates,
+        ov::op::v0::Constant::create(
+            ov::element::i64, {4}, std::vector<int64_t>{1, -1, n_heads_kv, head_size}),
+        false);
+    return std::make_shared<ov::op::v1::Transpose>(
+        reshaped, ov::op::v0::Constant::create(ov::element::i64, {4}, {0, 2, 1, 3}));
+}
+
+static int extract_layer(const std::string & name) {
+    const auto begin = name.find("_l");
+    OPENVINO_ASSERT(begin != std::string::npos, "HFA KV input has no layer suffix: ", name);
+    return std::stoi(name.substr(begin + 2));
+}
+
 static ov::Output<ov::Node> reshape_flat_kv(const ov::Output<ov::Node> & kv_flat,
                                             size_t view_offset_bytes,
                                             size_t nb1_bytes,
@@ -60,11 +110,16 @@ static ov::Output<ov::Node> reshape_flat_kv(const ov::Output<ov::Node> & kv_flat
 }

 OutputVector translate_flash_attn_ext(const NodeContext & context) {
-    num_inputs_check(context, 3, 4);
-    const bool has_mask = context.get_input_size() == 4;
+    num_inputs_check(context, 3, 5);
+    // Inputs are q, k, v, then the optional mask and sinks; supports_op rejects sinks without a mask
+    const bool has_mask = context.get_input_size() >= 4;
+    const bool has_sink = context.get_input_size() == 5;
+
     auto q_f32 = context.get_input(0);
     auto k = context.get_input(1);
     auto v = context.get_input(2);
+    const auto k_cache = k;
+    const auto v_cache = v;
     const int op_case = context.get_op_case();

     if (op_case == 1 || op_case == 2) {
@@ -85,34 +140,40 @@ OutputVector translate_flash_attn_ext(const NodeContext & context) {
     float * params = reinterpret_cast<float *>(context.get_output_op_params());
     float scale = params[0];
     // float max_bias      = params[1];
-    // float logit_softcap = params[2];
+    const float logit_softcap = params[2];

     auto q = std::make_shared<ov::op::v0::Convert>(q_f32, ov::element::f16);
     auto scale_node = std::make_shared<ov::op::v0::Constant>(ov::element::f16, ov::Shape{}, std::vector<float>{scale});

     ov::Output<ov::Node> res;

-    // For stateful
     ov::Output<ov::Node> mask;
     if (has_mask) {
-        mask = context.get_input(3);
-        std::string mask_name = "KQ_mask_sliced";
-        if (context.get_input_names()[3].find("swa") != std::string::npos) {
-            mask_name = "KQ_mask_swa_sliced";
-        }
-        if (context.has_input(mask_name)) {
-            mask = context.get_input(mask_name);
-        }
+        mask = get_kq_mask_input(context, 3);
         if (mask.get_element_type() != ov::element::f16) {
             mask = std::make_shared<ov::op::v0::Convert>(mask, ov::element::f16);
         }
     }

+    ov::Output<ov::Node> sink;
+    if (has_sink) {
+        sink = context.get_input(4);
+        if (sink.get_element_type() != q->get_element_type()) {
+            sink = std::make_shared<ov::op::v0::Convert>(sink, q->get_element_type());
+        }
+        sink = std::make_shared<ov::op::v1::Reshape>(
+            sink,
+            ov::op::v0::Constant::create(ov::element::i64, {4},
+                                         std::vector<int64_t>{1, context.get_input_shape(0)[1].get_length(), 1, 1}),
+            false);
+    }
+
     //auto tile_kv = [&](int64_t num_heads, int64_t num_heads_kv, int64_t head_size, ov::Output<Node> kv) {
     //    int64_t factor = num_heads / num_heads_kv;
     //    if (factor > 1 && num_heads_kv > 1) {
     auto q_shape = context.get_input_shape(0).to_shape();
     auto k_shape = context.get_input_shape(1).to_shape();
+    auto v_shape = context.get_input_shape(2).to_shape();
     const int64_t num_heads = q_shape[1];
     const int64_t num_heads_kv = k_shape[1];
     const int64_t head_size = q_shape[3];
@@ -128,54 +189,73 @@ OutputVector translate_flash_attn_ext(const NodeContext & context) {
         }
         return ggml_openvino_is_gpu();
     }();
+    // The manual path reshapes Q, K, V and the output with one head size; models whose K/V head sizes differ
+    // from Q's (e.g. MLA) take the SDPA path.
+    const bool same_head_size = k_shape[3] == q_shape[3] && v_shape[3] == q_shape[3];
+    // SDPA has no logit soft-capping (gemma-2), so soft-capped attention always takes the manual path, also in
+    // stateful execution. Only SDPA applies sinks, so attention with sinks never takes the manual path
+    // (supports_op rejects sinks with soft-capping)
     const bool use_manual_gqa_attention =
-        manual_gqa_enabled && factor > 1 && num_heads_kv > 1 && !context.is_stateful();
+        !has_sink &&
+        (logit_softcap != 0.0f || (manual_gqa_enabled && factor > 1 && num_heads_kv > 1 && !context.is_stateful())) &&
+        same_head_size;
+    FRONT_END_OP_CONVERSION_CHECK(logit_softcap == 0.0f || use_manual_gqa_attention,
+                                  "FLASH_ATTN_EXT with logit soft-capping is not supported with different K/V head "
+                                  "sizes");

     if (use_manual_gqa_attention) {
         // Q, K, V arrive as [B, n_heads(_kv), S, head_size], where B is the active
         // batch (n_seq_active) and may be > 1 (llama-perplexity, llama-server -np > 1)
-        // or dynamic. Reshape to
-        //   K_r: [B, num_heads_kv, 1, S, head_size]
-        //   Q_r: [B, num_heads_kv, factor, S_q, head_size]
-        // and let MatMul broadcast across the factor dim without materialising
-        // an expanded K/V. The leading 0 + special_zero=true copies B at runtime,
-        // so this is correct for B == 1, B > 1, and dynamic B alike. Only the head
-        // dims and head_size are baked in as literals; the sequence dim stays -1.
-        auto k_5d_shape = ov::op::v0::Constant::create(ov::element::i64, {5},
-                                                       std::vector<int64_t>{0, num_heads_kv, 1, -1, head_size});
-        auto v_5d_shape = ov::op::v0::Constant::create(ov::element::i64, {5},
-                                                       std::vector<int64_t>{0, num_heads_kv, 1, -1, head_size});
-        auto q_5d_shape = ov::op::v0::Constant::create(ov::element::i64, {5},
-                                                       std::vector<int64_t>{0, num_heads_kv, factor, -1, head_size});
-
-        auto k_r = std::make_shared<ov::op::v1::Reshape>(k, k_5d_shape, true);
-        auto v_r = std::make_shared<ov::op::v1::Reshape>(v, v_5d_shape, true);
-        auto q_r = std::make_shared<ov::op::v1::Reshape>(q, q_5d_shape, true);
-
-        // QK^T → [B, num_heads_kv, factor, S_q, S_k]
-        auto qk = std::make_shared<ov::op::v0::MatMul>(q_r, k_r, /*tA=*/false, /*tB=*/true);
-        auto qk_scaled = std::make_shared<ov::op::v1::Multiply>(qk, scale_node);
-
-        // Mask arrives as [B, 1, S_q, S_k]. Unsqueeze a factor axis at position 2 to
-        // get [B, 1, 1, S_q, S_k], which NUMPY-broadcasts cleanly against the
-        // [B, num_heads_kv, factor, S_q, S_k] scores: B==B, then 1→num_heads_kv and
-        // 1→factor on the head dims.
-        ov::Output<ov::Node> qk_masked;
-        if (has_mask) {
-            auto mask_unsq1 =
-                std::make_shared<ov::op::v0::Unsqueeze>(mask, ov::op::v0::Constant::create(ov::element::i64, {1}, {2}));
-            qk_masked = std::make_shared<ov::op::v1::Add>(qk_scaled, mask_unsq1);
+        // or dynamic. K and V are Transposes of the KV cache; they must feed the MatMuls
+        // directly (4-D, no Reshape in between) so that the GPU plugin folds the transpose
+        // into the Gemm instead of copying the whole transposed cache on every token.
+        // The query heads that share a KV head are stacked along the row axis:
+        //   Q:      [B, num_heads, S_q, head_size] -> [B, num_heads_kv, factor * S_q, head_size]
+        //           (a pure reshape: the query heads of one KV head are adjacent)
+        //   scores: [B, num_heads_kv, factor * S_q, S_k]
+        // The leading 0 + special_zero=true copies B at runtime, so this is correct for
+        // B == 1, B > 1, and dynamic B alike.
+        auto q_r = std::make_shared<ov::op::v1::Reshape>(
+            q,
+            ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{0, num_heads_kv, -1, head_size}),
+            true);
+
+        // QK^T -> [B, num_heads_kv, factor * S_q, S_k]
+        auto qk = std::make_shared<ov::op::v0::MatMul>(q_r, k, /*tA=*/false, /*tB=*/true);
+        ov::Output<ov::Node> probs;
+        if (logit_softcap != 0.0f) {
+            // as ggml: softcap * tanh(qk * scale / softcap)
+            probs = std::make_shared<ov::op::v1::Multiply>(
+                qk,
+                ov::op::v0::Constant::create(ov::element::f16, ov::Shape{}, std::vector<float>{scale / logit_softcap}));
+            probs = std::make_shared<ov::op::v1::Multiply>(
+                std::make_shared<ov::op::v0::Tanh>(probs),
+                ov::op::v0::Constant::create(ov::element::f16, ov::Shape{}, std::vector<float>{logit_softcap}));
         } else {
-            qk_masked = qk_scaled;
+            probs = std::make_shared<ov::op::v1::Multiply>(qk, scale_node);
         }

-        auto softmax = std::make_shared<ov::op::v8::Softmax>(qk_masked, /*axis=*/-1);
+        if (has_mask) {
+            // Mask arrives as [B, 1, S_q, S_k]. Row j * S_q + s of the scores belongs to query position s, so
+            // repeat the mask factor times along the rows, [B, 1, factor * S_q, S_k], and add it to the 4-D
+            // scores. Keeping the scores 4-D from the MatMul to the Softmax lets the GPU plugin fuse the scale
+            // and the mask into the Gemm: a reshape of the scores in between costs a full pass over them per
+            // layer, which is large in prefill. The tiled mask is the same in every layer.
+            // The GPU plugin fuses this attention into its SDPA kernel, which works on blocks of keys: a block
+            // fully masked with -inf for a row gives NaN (see the SDPA path below), so use a large finite value.
+            auto mask_finite = std::make_shared<ov::op::v1::Maximum>(
+                mask, ov::op::v0::Constant::create(ov::element::f16, ov::Shape{}, std::vector<float>{-30000.0f}));
+            auto mask_tiled = std::make_shared<ov::op::v0::Tile>(
+                mask_finite,
+                ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{1, 1, factor, 1}));
+            probs = std::make_shared<ov::op::v1::Add>(probs, mask_tiled);
+        }
+        probs = std::make_shared<ov::op::v8::Softmax>(probs, /*axis=*/-1);

-        // softmax @ V → [B, num_heads_kv, factor, S_q, head_size]
-        auto attn = std::make_shared<ov::op::v0::MatMul>(softmax, v_r);
+        // softmax @ V -> [B, num_heads_kv, factor * S_q, head_size]
+        auto attn = std::make_shared<ov::op::v0::MatMul>(probs, v);

         // Reshape back to [B, num_heads, S_q, head_size] (combine num_heads_kv * factor).
-        // Leading 0 + special_zero=true copies B at runtime.
         auto out_4d_shape =
             ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{0, num_heads, -1, head_size});
         auto out_4d = std::make_shared<ov::op::v1::Reshape>(attn, out_4d_shape, true);
@@ -191,7 +271,7 @@ OutputVector translate_flash_attn_ext(const NodeContext & context) {
     // Default path: explicit Broadcast → SDPA. Kept as the fallback because
     // (a) it goes through the GPU plugin's micro-SDPA fast path (FlashAttention
     // tiles via DPAS), and (b) the manual path above is still being validated.
-    auto tile_kv = [&](int64_t n_heads, int64_t n_heads_kv, int64_t hs, ov::Output<Node> kv) {
+    auto tile_kv = [&](int64_t n_heads, int64_t n_heads_kv, int64_t hs, ov::Output<Node> kv, bool transposed = false) {
         int64_t f = n_heads / n_heads_kv;
         if (f > 1 && n_heads_kv > 1) {
             ov::Output<ov::Node> kv_broadcast_shape;
@@ -202,8 +282,9 @@ OutputVector translate_flash_attn_ext(const NodeContext & context) {

             kv_broadcast_shape = ov::op::v0::Constant::create(ov::element::i64, {5},
                                                               {(int64_t) 1, (int64_t) 1, f, (int64_t) 1, (int64_t) 1});
-            new_kv_shape =
-                ov::op::v0::Constant::create(ov::element::i64, {4}, {(int64_t) 0, n_heads, (int64_t) -1, hs});
+            new_kv_shape = ov::op::v0::Constant::create(
+                ov::element::i64, {4},
+                transposed ? std::vector<int64_t>{0, n_heads, hs, -1} : std::vector<int64_t>{0, n_heads, -1, hs});
             //    ov::element::i64, {5}, {(int64_t) 1, (int64_t) 1, factor, (int64_t) 1, (int64_t) 1});
             //new_kv_shape =
             //    ov::op::v0::Constant::create(ov::element::i64, {4}, {(int64_t) 0, num_heads, (int64_t) -1, head_size});
@@ -215,15 +296,112 @@ OutputVector translate_flash_attn_ext(const NodeContext & context) {
         return kv;
     };

+    // Transposed V cache [1, heads, head_size, rows] (GenAI's NPUW_LLM_OPTIMIZE_V_TENSORS layout).
+    const bool v_transposed = ggml_openvino_npu_kv_head_major_enabled() && context.is_static() &&
+                              context.get_input_names()[2].find("cache_v_l") != std::string::npos;
+
+    // the HFA prefill path has no sinks
+    const bool use_hfa_gate =
+        ggml_openvino_npu_hfa_prefill_enabled() && context.is_static() && context.is_prefill() && has_mask && !has_sink;
+    if (use_hfa_gate) {
+        const int layer = extract_layer(context.get_input_names()[1]);
+        auto past_k = context.get_input("ggml_hfa_past_cache_k_l" + std::to_string(layer));
+        auto past_v = context.get_input("ggml_hfa_past_cache_v_l" + std::to_string(layer));
+        auto hfa_mask = context.get_input("ggml_hfa_mask");
+        auto present_k_updates = find_scatter_updates(k_cache);
+        auto present_v_updates = find_scatter_updates(v_cache);
+        if (!present_v_updates && v_transposed) {
+            // Transposed V prefill outputs the chunk rows themselves (see set_rows); the window Slice wraps them.
+            ov::Output<ov::Node> rows = v_cache;
+            while (auto slice = ov::as_type_ptr<ov::op::v8::Slice>(rows.get_node_shared_ptr())) {
+                rows = slice->input_value(0);
+            }
+            present_v_updates = rows;
+        }
+        OPENVINO_ASSERT(present_k_updates.has_value() && present_v_updates.has_value(),
+                        "HFA prefill requires ScatterUpdate-backed K/V cache inputs");
+        if (ggml_openvino_npu_kv_head_major_enabled()) {
+            // Head-major cache: past and the new rows are already [1, heads, rows, head_size]
+            // (V: [1, heads, head_size, rows] when transposed).
+            k = std::make_shared<ov::op::v0::Concat>(ov::OutputVector{past_k, *present_k_updates}, 2);
+            v = std::make_shared<ov::op::v0::Concat>(ov::OutputVector{past_v, *present_v_updates},
+                                                     v_transposed ? 3 : 2);
+        } else {
+            auto present_k = reshape_present_kv(*present_k_updates, num_heads_kv, head_size);
+            auto present_v = reshape_present_kv(*present_v_updates, num_heads_kv, head_size);
+            // Past K/V come in ggml cache layout [1, past, heads, head_size]; NPUW HFA reads tiles from it directly.
+            auto to_bhsd = ov::op::v0::Constant::create(ov::element::i64, {4}, {0, 2, 1, 3});
+            k = std::make_shared<ov::op::v0::Concat>(
+                ov::OutputVector{std::make_shared<ov::op::v1::Transpose>(past_k, to_bhsd), present_k}, 2);
+            v = std::make_shared<ov::op::v0::Concat>(
+                ov::OutputVector{std::make_shared<ov::op::v1::Transpose>(past_v, to_bhsd), present_v}, 2);
+        }
+        k = tile_kv(num_heads, num_heads_kv, head_size, k);
+        v = tile_kv(num_heads, num_heads_kv, head_size, v, v_transposed);
+
+        auto q_scaled = std::make_shared<ov::op::v1::Multiply>(q, scale_node);
+        auto scores = std::make_shared<ov::op::v0::MatMul>(q_scaled, k, false, true);
+        auto masked_scores = std::make_shared<ov::op::v1::Add>(scores, hfa_mask);
+        auto probs = std::make_shared<ov::op::v8::Softmax>(masked_scores, -1);
+        auto attended = std::make_shared<ov::op::v0::MatMul>(probs, v, false, v_transposed);
+        auto transposed = std::make_shared<ov::op::v1::Transpose>(
+            attended, ov::op::v0::Constant::create(ov::element::i64, {4}, {0, 2, 1, 3}));
+        auto output_shape = ov::op::v0::Constant::create(
+            ov::element::i64, {3}, std::vector<int64_t>{0, -1, num_heads * head_size});
+        res = std::make_shared<ov::op::v1::Reshape>(transposed, output_shape, true);
+        auto restored_shape = ov::op::v0::Constant::create(
+            ov::element::i64, {4}, std::vector<int64_t>{0, -1, num_heads, head_size});
+        res = std::make_shared<ov::op::v1::Reshape>(res, restored_shape, true);
+        res = std::make_shared<ov::op::v0::Convert>(res, ov::element::f32);
+        return rename_outputs_with_suffix({res}, context.get_name());
+    }
+
     //auto q_shape = context.get_input_shape(0).to_shape();
     //auto k_shape = context.get_input_shape(1).to_shape();
     //k = tile_kv(q_shape[1], k_shape[1], q_shape[3], k);
     //v = tile_kv(q_shape[1], k_shape[1], q_shape[3], v);
     k = tile_kv(num_heads, num_heads_kv, head_size, k);
-    v = tile_kv(num_heads, num_heads_kv, head_size, v);
+    v = tile_kv(num_heads, num_heads_kv, head_size, v, v_transposed);
+
+    // The NPU 4000 compiler (Lunar Lake) computes ScaledDotProductAttention wrongly when K has one KV head and is
+    // square, that is when the attended rows equal the head size (gemma-4 E2B: 512 rows with its 512-wide
+    // full-attention heads, 256 rows with its 256-wide sliding-window heads). The same attention as separate MatMuls
+    // is correct there. Remove when the NPU compiler computes the op correctly for this shape.
+    const auto k_dims = k.get_partial_shape();
+    const bool npu_square_k = ggml_openvino_npu_is_4000() && !has_sink && num_heads_kv == 1 &&
+                              k_dims.rank().is_static() && k_dims.size() == 4 && k_dims[2].is_static() &&
+                              k_dims[3].is_static() && k_dims[2].get_length() == k_dims[3].get_length();
+
+    if (v_transposed || npu_square_k) {
+        // The attention as separate ops. SDPA has no transposed-V form: as GenAI does, that case uses
+        // MatMul(probs, V^T, transpose_b).
+        auto q_scaled = std::make_shared<ov::op::v1::Multiply>(q, scale_node);
+        ov::Output<ov::Node> scores = std::make_shared<ov::op::v0::MatMul>(q_scaled, k, false, true);
+        if (has_mask) {
+            scores = std::make_shared<ov::op::v1::Add>(scores, mask);
+        }
+        auto probs = std::make_shared<ov::op::v8::Softmax>(scores, -1);
+        auto attended = std::make_shared<ov::op::v0::MatMul>(probs, v, false, v_transposed);
+        res = std::make_shared<ov::op::v1::Transpose>(
+            attended, ov::op::v0::Constant::create(ov::element::i64, {4}, {0, 2, 1, 3}));
+        res = std::make_shared<ov::op::v0::Convert>(res, ov::element::f32);
+        return rename_outputs_with_suffix({res}, context.get_name());
+    }
+
+    // With n_seq_active read from the mask shape, the GPU plugin knows Q's head size and uses its fused SDPA kernel.
+    // That kernel works on blocks of keys: a block fully masked with -inf for a row gives exp(-inf - -inf) = NaN,
+    // which spreads to the row's result, so mask with a large finite value instead
+    if (has_mask && ggml_openvino_is_gpu() && !context.is_stateful() && ggml_openvino_shape_from_mask_enabled()) {
+        mask = std::make_shared<ov::op::v1::Maximum>(
+            mask, ov::op::v0::Constant::create(ov::element::f16, ov::Shape{}, std::vector<float>{-30000.0f}));
+    }

     constexpr auto causal = false;
-    if (has_mask) {
+    if (has_mask && has_sink) {
+        auto sdpa = std::make_shared<ov::op::v13::ScaledDotProductAttention>(q, k, v, mask, scale_node, sink, causal);
+        res = std::make_shared<ov::op::v1::Transpose>(
+            sdpa, ov::op::v0::Constant::create(ov::element::i64, {4}, {0, 2, 1, 3}));
+    } else if (has_mask) {
         auto sdpa = std::make_shared<ov::op::v13::ScaledDotProductAttention>(q, k, v, mask, scale_node, causal);
         res = std::make_shared<ov::op::v1::Transpose>(
             sdpa, ov::op::v0::Constant::create(ov::element::i64, {4}, {0, 2, 1, 3}));
diff --git a/ggml/src/ggml-openvino/openvino/op/gated_delta_net.cpp b/ggml/src/ggml-openvino/openvino/op/gated_delta_net.cpp
index 9cedf6811..9acf3effb 100644
--- a/ggml/src/ggml-openvino/openvino/op/gated_delta_net.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/gated_delta_net.cpp
@@ -3,11 +3,17 @@
 #include "../node_context.h"
 #include "../op_table.h"
 #include "../utils.h"
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+#include "recurrent_cache_update.hpp"
+#endif
 #include "ggml-openvino/ggml-openvino-extra.h"

 #include <cmath>
 #include <cstdint>
+#include <limits>
+#include <map>
 #include <memory>
+#include <mutex>
 #include <numeric>
 #include <openvino/op/add.hpp>
 #include <openvino/op/broadcast.hpp>
@@ -20,15 +26,23 @@
 #include <openvino/op/less.hpp>
 #include <openvino/op/loop.hpp>
 #include <openvino/op/matmul.hpp>
+#include <openvino/op/maximum.hpp>
+#include <openvino/op/minimum.hpp>
 #include <openvino/op/multiply.hpp>
+#include <openvino/op/range.hpp>
 #include <openvino/op/reduce_mean.hpp>
 #include <openvino/op/reshape.hpp>
-#include <openvino/op/squeeze.hpp>
+#include <openvino/op/select.hpp>
+#include <openvino/op/shape_of.hpp>
+#include <openvino/op/slice.hpp>
+#include <openvino/op/split.hpp>
 #include <openvino/op/sqrt.hpp>
+#include <openvino/op/squeeze.hpp>
 #include <openvino/op/subtract.hpp>
 #include <openvino/op/tile.hpp>
 #include <openvino/op/transpose.hpp>
 #include <openvino/op/unsqueeze.hpp>
+#include <utility>
 #include <vector>

 namespace ov {
@@ -92,7 +106,276 @@ static bool match_gdn_l2_norm(const Output<Node> & normalized, Output<Node> & in
     return true;
 }

-OutputVector translate_gated_delta_net(const NodeContext & context) {
+// A [CS, CS] mask of the chunked GatedDeltaNet, one instance per content for the whole process. The NPUW weights
+// bank, which all compiled models share (NPUW_WEIGHTS_BANK=shared), identifies a constant by its data pointer and
+// keeps its tensor after the model that owned it is gone. A mask of a later model that got the address of a freed
+// mask would then be read as that other mask, so a mask is never freed. Not needed once NPUW stops matching a new
+// constant to a detached one by pointer (OpenVINO 2026.4 does).
+static std::shared_ptr<ov::op::v0::Constant> gdn_chunk_mask(int64_t CS, const std::vector<float> & values) {
+    static std::mutex mutex;
+    static std::map<std::vector<float>, std::shared_ptr<ov::op::v0::Constant>> cache;
+    std::lock_guard<std::mutex> lock(mutex);
+    auto & mask = cache[values];
+    if (!mask) {
+        mask = ov::op::v0::Constant::create(ov::element::f32, {(size_t) CS, (size_t) CS}, values);
+    }
+    return mask;
+}
+
+// Chunked GatedDeltaNet built from MatMuls, for devices where the GatedDeltaNet op cannot run (NPU4000).
+// Same contract as the op with equal q/k/v heads and no fused l2 norm:
+//   q, k [B, T, H, Dk], v [B, T, H, Dv], g, beta [B, T, H], state [B, H, Dk, Dv]
+//   returns attn [B, T, H, Dv] and the new state [B, H, Dk, Dv].
+// Per chunk of CS tokens (gc = cumsum of g in the chunk, decay[i][j] = exp(gc_i - gc_j) for j <= i, else 0):
+//   P    = (I + strict_lower(beta k k^T * decay))^-1
+//   u    = P (beta v),  w = P (beta k exp(gc))
+//   v'   = u - w S
+//   out  = (q exp(gc)) S + lower(q k^T * decay) v'
+//   S    = S exp(gc_last) + (k exp(gc_last - gc))^T v'
+// The chunks run one after another; everything inside a chunk is batched over all chunks and heads.
+// P is built block-recursively, [[A, 0], [C, D]]^-1 = [[A^-1, 0], [-D^-1 C A^-1, D^-1]], for block sizes
+// 1, 2, 4, ..., CS / 2. Every intermediate is a part of the true inverse, so values stay small in f16. A Neumann
+// product (I - L)(I + L^2)... is exact too, but powers of L grow large and cancel, which f16 cannot hold.
+// A single token (decode) is a chunk of one token, where P = 1 and decay = 1:
+//   v'   = beta (v - exp(g) k S)
+//   S    = S exp(g) + k^T v'
+//   out  = q S
+// No Loop op is used, so the GatedDeltaNet loop fusion does not turn this back into the op.
+static std::pair<Output<Node>, Output<Node>> make_gated_delta_net_chunked(Output<Node> q, Output<Node> k,
+                                                                          Output<Node> v, Output<Node> g,
+                                                                          Output<Node> beta, Output<Node> state,
+                                                                          int64_t B, int64_t T, int64_t H, int64_t Dk,
+                                                                          int64_t Dv) {
+    const int64_t CS = ggml_openvino_gdn_decompose_chunk();
+    const int64_t max_dim = std::numeric_limits<int64_t>::max();
+    FRONT_END_OP_CONVERSION_CHECK(B > 0 && T > 0 && H > 0 && Dk > 0 && Dv > 0,
+                                  "GATED_DELTA_NET decomposition requires positive dimensions");
+    const int64_t pad = (CS - T % CS) % CS;
+    FRONT_END_OP_CONVERSION_CHECK(T <= max_dim - pad && B <= max_dim / H,
+                                  "GATED_DELTA_NET decomposition dimensions overflow");
+    const int64_t N = (T + pad) / CS;
+    const int64_t X = B * H;
+    FRONT_END_OP_CONVERSION_CHECK(X <= max_dim / N / CS / CS && X <= max_dim / N / CS / Dk &&
+                                  X <= max_dim / N / CS / Dv && X <= max_dim / Dk / Dv,
+                                  "GATED_DELTA_NET decomposition tensor size overflows");
+
+    auto i64 = [](const std::vector<int64_t> & v) {
+        return ov::op::v0::Constant::create(ov::element::i64, {v.size()}, v);
+    };
+    auto f32 = [](float x) { return ov::op::v0::Constant::create(ov::element::f32, {}, {x}); };
+    auto reshape = [&](const Output<Node> & x, const std::vector<int64_t> & shape) -> Output<Node> {
+        return std::make_shared<ov::op::v1::Reshape>(x, i64(shape), false);
+    };
+    auto transpose = [&](const Output<Node> & x, const std::vector<int64_t> & perm) -> Output<Node> {
+        return std::make_shared<ov::op::v1::Transpose>(x, i64(perm));
+    };
+    auto matmul = [](const Output<Node> & a, const Output<Node> & b, bool ta = false, bool tb = false) -> Output<Node> {
+        return std::make_shared<ov::op::v0::MatMul>(a, b, ta, tb);
+    };
+    auto mul = [](const Output<Node> & a, const Output<Node> & b) -> Output<Node> {
+        return std::make_shared<ov::op::v1::Multiply>(a, b);
+    };
+    auto add = [](const Output<Node> & a, const Output<Node> & b) -> Output<Node> {
+        return std::make_shared<ov::op::v1::Add>(a, b);
+    };
+    auto sub = [](const Output<Node> & a, const Output<Node> & b) -> Output<Node> {
+        return std::make_shared<ov::op::v1::Subtract>(a, b);
+    };
+    auto exp_op = [](const Output<Node> & x) -> Output<Node> { return std::make_shared<ov::op::v0::Exp>(x); };
+
+    if (T == 1) {
+        // [B, 1, H, D] -> [1, X, 1, D], batch 1 as below
+        q = mul(reshape(q, {1, X, 1, Dk}), f32(1.0f / std::sqrt((float) Dk)));
+        k = reshape(k, {1, X, 1, Dk});
+        v = reshape(v, {1, X, 1, Dv});
+        auto state_decay = exp_op(reshape(g, {1, X, 1, 1}));
+        beta = reshape(beta, {1, X, 1, 1});
+        Output<Node> s = reshape(state, {1, X, Dk, Dv});
+        auto v_new = mul(sub(v, matmul(mul(k, state_decay), s)), beta);  // [1, X, 1, Dv]
+        s = add(mul(s, state_decay), matmul(k, v_new, true, false));     // [1, X, Dk, Dv]
+        return {reshape(matmul(q, s), {B, 1, H, Dv}), reshape(s, {B, H, Dk, Dv})};
+    }
+
+    // Pad T to a multiple of CS. A padded token has g = 0 and beta = k = 0, so it leaves the state unchanged.
+    if (pad > 0) {
+        auto pad_tokens = [&](const Output<Node> & x, std::vector<size_t> zero_shape) -> Output<Node> {
+            zero_shape[1] = pad;
+            auto zeros = ov::op::v0::Constant::create(ov::element::f32, zero_shape, std::vector<float>{0.0f});
+            return std::make_shared<ov::op::v0::Concat>(ov::OutputVector{x, zeros}, 1);
+        };
+        q = pad_tokens(q, {(size_t) B, 0, (size_t) H, (size_t) Dk});
+        k = pad_tokens(k, {(size_t) B, 0, (size_t) H, (size_t) Dk});
+        v = pad_tokens(v, {(size_t) B, 0, (size_t) H, (size_t) Dv});
+        g = pad_tokens(g, {(size_t) B, 0, (size_t) H});
+        beta = pad_tokens(beta, {(size_t) B, 0, (size_t) H});
+    }
+
+    // Batch stays 1 for the NPU (a batched eltwise is split per batch and mis-broadcasts): X = B * H.
+    // [B, T, H, D] -> [1, X * N, CS, D]
+    auto to_chunks = [&](const Output<Node> & x, int64_t d) {
+        return reshape(transpose(x, {0, 2, 1, 3}), {1, X * N, CS, d});
+    };
+    q = mul(to_chunks(q, Dk), f32(1.0f / std::sqrt((float) Dk)));
+    k = to_chunks(k, Dk);
+    v = to_chunks(v, Dv);
+    g = reshape(transpose(g, {0, 2, 1}), {1, X * N, CS, 1});
+    beta = reshape(transpose(beta, {0, 2, 1}), {1, X * N, CS, 1});
+
+    std::vector<float> lower_incl(CS * CS), lower_strict(CS * CS), identity(CS * CS);
+    for (int64_t i = 0; i < CS; i++) {
+        for (int64_t j = 0; j < CS; j++) {
+            lower_incl[i * CS + j] = j <= i ? 1.0f : 0.0f;
+            lower_strict[i * CS + j] = j < i ? 1.0f : 0.0f;
+            identity[i * CS + j] = i == j ? 1.0f : 0.0f;
+        }
+    }
+    auto lower_incl_mask = gdn_chunk_mask(CS, lower_incl);
+    auto lower_strict_mask = gdn_chunk_mask(CS, lower_strict);
+    auto eye = gdn_chunk_mask(CS, identity);
+
+    // cumsum of g inside a chunk as a MatMul with the lower triangle of ones: gc [1, X * N, CS, 1]
+    auto gc_col = matmul(lower_incl_mask, g);
+    auto gc_row = reshape(gc_col, {1, X * N, 1, CS});
+    auto causal = std::make_shared<ov::op::v0::Convert>(lower_incl_mask, ov::element::boolean);
+    auto log_decay = std::make_shared<ov::op::v1::Select>(causal, sub(gc_col, gc_row), f32(0.0f));
+    auto decay = mul(exp_op(log_decay), lower_incl_mask);
+
+    auto kb = mul(k, beta);
+    auto vb = mul(v, beta);
+
+    // P = (I + L)^-1 with L = strict_lower(beta k k^T * decay)
+    auto neg_l = mul(mul(mul(matmul(kb, k, false, true), decay), lower_strict_mask), f32(-1.0f));
+    // p holds the inverses of the diagonal b x b blocks; merging pairs of them only needs the -L entries
+    // in the lower off-diagonal b x b block of each 2b x 2b block: p += p (-L masked) p
+    auto block_mask = [&](int64_t b) {
+        std::vector<float> m(CS * CS);
+        for (int64_t i = 0; i < CS; i++) {
+            for (int64_t j = 0; j < CS; j++) {
+                m[i * CS + j] = (i / (2 * b) == j / (2 * b) && i / b > j / b) ? 1.0f : 0.0f;
+            }
+        }
+        return gdn_chunk_mask(CS, m);
+    };
+    Output<Node> p = add(eye, mul(neg_l, block_mask(1)));
+    for (int64_t b = 2; b < CS; b *= 2) {
+        p = add(p, matmul(matmul(p, mul(neg_l, block_mask(b))), p));
+    }
+
+    auto gc_exp = exp_op(gc_col);                     // [1, X * N, CS, 1]
+    auto u = matmul(p, vb);                           // [1, X * N, CS, Dv]
+    auto w = matmul(p, mul(kb, gc_exp));              // [1, X * N, CS, Dk]
+    auto qg = mul(q, gc_exp);                         // [1, X * N, CS, Dk]
+    auto qk = mul(matmul(q, k, false, true), decay);  // [1, X * N, CS, CS]
+    auto gc_last = std::make_shared<ov::op::v8::Slice>(gc_col, i64({CS - 1}), i64({CS}), i64({1}), i64({2}));
+    auto kg = mul(k, exp_op(sub(gc_last, gc_col)));   // [1, X * N, CS, Dk]
+    auto state_decay = exp_op(gc_last);               // [1, X * N, 1, 1]
+
+    // Per-chunk slices, reshaped to [X, rows, cols]
+    auto chunk_axis = ov::op::v0::Constant::create(ov::element::i64, {}, {1});
+    auto split_chunks = [&](const Output<Node> & x, int64_t rows, int64_t cols) {
+        auto split = std::make_shared<ov::op::v1::Split>(reshape(x, {X, N, rows * cols}), chunk_axis,
+                                                         static_cast<size_t>(N));
+        std::vector<Output<Node>> parts;
+        for (int64_t n = 0; n < N; n++) {
+            parts.push_back(reshape(split->output(n), {X, rows, cols}));
+        }
+        return parts;
+    };
+    auto u_n = split_chunks(u, CS, Dv);
+    auto w_n = split_chunks(w, CS, Dk);
+    auto qg_n = split_chunks(qg, CS, Dk);
+    auto qk_n = split_chunks(qk, CS, CS);
+    auto kg_n = split_chunks(kg, CS, Dk);
+    auto state_decay_n = split_chunks(state_decay, 1, 1);
+
+    Output<Node> s = reshape(state, {X, Dk, Dv});
+    ov::OutputVector outs;
+    for (int64_t n = 0; n < N; n++) {
+        auto v_new = sub(u_n[n], matmul(w_n[n], s));                             // [X, CS, Dv]
+        outs.push_back(add(matmul(qg_n[n], s), matmul(qk_n[n], v_new)));         // [X, CS, Dv]
+        s = add(mul(s, state_decay_n[n]), matmul(kg_n[n], v_new, true, false));  // [X, Dk, Dv]
+    }
+
+    Output<Node> attn = N > 1 ? std::make_shared<ov::op::v0::Concat>(outs, 1)->output(0) : outs[0];
+    if (pad > 0) {
+        attn = std::make_shared<ov::op::v8::Slice>(attn, i64({0}), i64({T}), i64({1}), i64({1}));
+    }
+    attn = reshape(attn, {B, H, T, Dv});
+    return {transpose(attn, {0, 2, 1, 3}), reshape(s, {B, H, Dk, Dv})};
+}
+
+static OutputVector make_gated_delta_net(const Output<Node> & q,
+                                        const Output<Node> & k,
+                                        const Output<Node> & v,
+                                        const Output<Node> & state,
+                                        const Output<Node> & g,
+                                        const Output<Node> & beta,
+                                        bool norm,
+                                        float q_eps,
+                                        float k_eps) {
+    const auto & q_shape = q.get_partial_shape();
+    const auto & v_shape = v.get_partial_shape();
+    if (ggml_openvino_gdn_decompose_enabled() && !norm && q_shape.is_static() && v_shape.is_static()) {
+        auto outputs = make_gated_delta_net_chunked(q, k, v, g, beta, state,
+                                                   q_shape[0].get_length(), q_shape[1].get_length(),
+                                                   v_shape[2].get_length(), q_shape[3].get_length(),
+                                                   v_shape[3].get_length());
+        return {outputs.first, outputs.second};
+    }
+    auto gdn = std::make_shared<ov::op::internal::GatedDeltaNet>(q, k, v, state, g, beta, norm, q_eps, k_eps);
+    return {gdn->output(0), gdn->output(1)};
+}
+
+static OutputVector translate_gdn_snapshots(const Output<Node> & q,
+                                          const Output<Node> & k,
+                                          const Output<Node> & v,
+                                          const Output<Node> & state,
+                                          const Output<Node> & g,
+                                          const Output<Node> & beta,
+                                          int64_t features,
+                                          bool norm,
+                                          float q_eps,
+                                          float k_eps,
+                                          int64_t snapshot_count) {
+    // Unmatched graphs need state outputs. Ordinary GDN prefixes preserve that contract.
+    auto i64 = [](const Shape & shape, const std::vector<int64_t> & values) {
+        return ov::op::v0::Constant::create(ov::element::i64, shape, values);
+    };
+    auto shape = std::make_shared<ov::op::v3::ShapeOf>(q, ov::element::i64);
+    auto batch = std::make_shared<ov::op::v8::Gather>(shape, i64({1}, {0}), i64({}, {0}));
+    auto tokens = std::make_shared<ov::op::v8::Gather>(shape, i64({1}, {1}), i64({}, {0}));
+    auto perm = i64({4}, {0, 1, 3, 2});
+    auto initial = std::make_shared<ov::op::v1::Transpose>(state, perm);
+    Output<Node> attention;
+    OutputVector snapshots;
+    for (int64_t offset = 0; offset < snapshot_count; ++offset) {
+        auto end = std::make_shared<ov::op::v1::Maximum>(
+            std::make_shared<ov::op::v1::Subtract>(tokens, i64({1}, {offset})), i64({1}, {1}));
+        auto prefix = [&](const Output<Node> & value) -> Output<Node> {
+            if (offset == 0) {
+                return value;
+            }
+            // Materialize prefixes to avoid the GPU's dynamic Slice alias optimization.
+            auto indices = std::make_shared<ov::op::v4::Range>(
+                i64({}, {0}), std::make_shared<ov::op::v0::Squeeze>(end), i64({}, {1}), ov::element::i64);
+            return std::make_shared<ov::op::v8::Gather>(value, indices, i64({}, {1}));
+        };
+        auto gdn = std::make_shared<ov::op::internal::GatedDeltaNet>(
+            prefix(q), prefix(k), prefix(v), initial, prefix(g), prefix(beta), norm, q_eps, k_eps);
+        if (offset == 0) {
+            attention = gdn->output(0);
+        }
+        snapshots.push_back(std::make_shared<ov::op::v1::Reshape>(
+            std::make_shared<ov::op::v1::Transpose>(gdn->output(1), perm), i64({4}, {1, 1, -1, features}), false));
+    }
+    auto valid_rows = std::make_shared<ov::op::v1::Multiply>(
+        std::make_shared<ov::op::v1::Minimum>(tokens, i64({1}, {snapshot_count})), batch);
+    auto states = std::make_shared<ov::op::v8::Slice>(
+        std::make_shared<ov::op::v0::Concat>(snapshots, 2), i64({1}, {0}), valid_rows, i64({1}, {1}), i64({1}, {2}));
+    return {attention, states};
+}
+
+static OutputVector translate_gdn(const NodeContext & context, const OutputVector * cache_inputs) {
     auto v_shape = context.get_input_shape(2).to_shape();  // [B, T, H_v, S_v]
     auto q_shape = context.get_input_shape(0).to_shape();  // [B, T, H_k, S_k]

@@ -107,6 +390,7 @@ OutputVector translate_gated_delta_net(const NodeContext & context) {
     const int64_t H_v = v_shape[2];
     const int64_t S_v = v_shape[3];
     const int64_t H_k = q_shape[2];
+    const int64_t snapshot_count = context.get_output_op_params()[0];
     // const int64_t S_k = q_shape[3];

     auto q = context.get_input(0);
@@ -126,6 +410,12 @@ OutputVector translate_gated_delta_net(const NodeContext & context) {
         k = raw_k;
     }

+    if (context.get_op_case() == 1 && ggml_openvino_gdn_prescale_enabled()) {
+        // the output is linear in q and the state does not depend on q; the next rms_norm removes the scale
+        q = std::make_shared<ov::op::v1::Multiply>(
+            q, ov::op::v0::Constant::create(ov::element::f32, ov::Shape{1}, {ggml_openvino_gdn_q_scale()}));
+    }
+
     // ggml maps GQA heads in tiled order, while OV GDN maps repeated heads in grouped order.
     if (H_v != H_k) {
         const int64_t repeat = H_v / H_k;
@@ -145,8 +435,13 @@ OutputVector translate_gated_delta_net(const NodeContext & context) {

     // ggml state layout (OV notation): [B, H_v, value_dim, key_dim]
     // GatedDeltaNet op expects: [B, H_v, key_dim, value_dim]
+    // With native state the buffer already holds the op layout, so skip both transposes. This is safe because
+    // resets and slot copies move whole states, and supports_op aborts instead of falling back to CPU.
+    const bool native_state = ggml_openvino_gdn_native_state_enabled();
     auto state_perm = ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{0, 1, 3, 2});
-    state = std::make_shared<ov::op::v1::Transpose>(state, state_perm);
+    if (!native_state && !cache_inputs && snapshot_count == 1) {
+        state = std::make_shared<ov::op::v1::Transpose>(state, state_perm);
+    }

     g = std::make_shared<ov::op::v0::Squeeze>(g, ov::op::v0::Constant::create(ov::element::i64, {1}, {3}));
     beta = std::make_shared<ov::op::v0::Squeeze>(beta, ov::op::v0::Constant::create(ov::element::i64, {1}, {3}));
@@ -177,33 +472,72 @@ OutputVector translate_gated_delta_net(const NodeContext & context) {
     //           << ", v=" << v.get_partial_shape() << ", g=" << g.get_partial_shape()
     //           << ", beta=" << beta.get_partial_shape() << ", state=" << state.get_partial_shape() << std::endl;

-    auto gdn = std::make_shared<ov::op::internal::GatedDeltaNet>(q, k, v, state, g, beta, fuse_qk_l2norm, q_eps, k_eps);
-    auto attn_4d = gdn->output(0);
-    auto state_4d = gdn->output(1);  // [B, H_v, key_dim, value_dim]
-
-    // std::cout << "GatedDeltaNet output shapes: attn=" << gdn->output(0).get_partial_shape()
-    //           << ", new_state=" << gdn->output(1).get_partial_shape() << std::endl;
-
-    // Transpose output state back to ggml layout [B, H_v, value_dim, key_dim]
-    auto state_transposed = std::make_shared<ov::op::v1::Transpose>(state_4d, state_perm);
-    if (context.get_output_names().size() == 2) {
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+    if (cache_inputs) {
+        OutputVector args{q, k, v, (*cache_inputs)[0], g, beta};
+        args.insert(args.end(), cache_inputs->begin() + 1, cache_inputs->end());
+        auto fused = std::make_shared<ov::op::internal::RecurrentGatedDeltaNetUpdate>(
+            args, fuse_qk_l2norm, q_eps, k_eps, snapshot_count);
+        fused->set_friendly_name(context.get_name() + "/RecurrentGatedDeltaNetUpdate");
+        // The matched state VIEW is consumed only by the overridden full-cache
+        // writeback. Route that cache directly instead of materializing snapshots.
+        if (context.get_output_names().size() == 3) {
+            return {fused->output(0), fused->output(0), (*cache_inputs)[0]};
+        }
+        return {fused->output(0), (*cache_inputs)[0]};
+    }
+#endif
+
+    Output<Node> attn_4d, states;
+    if (snapshot_count > 1) {
+        auto outputs = translate_gdn_snapshots(q, k, v, state, g, beta, H_v * S_v * q_shape[3],
+                                               fuse_qk_l2norm, q_eps, k_eps, snapshot_count);
+        attn_4d = outputs[0];
+        states = outputs[1];
+    } else {
+        auto outputs = make_gated_delta_net(q, k, v, state, g, beta, fuse_qk_l2norm, q_eps, k_eps);
+        attn_4d = outputs[0];
+        if (native_state) {
+            states = outputs[1];
+        } else {
+            states = std::make_shared<ov::op::v1::Transpose>(outputs[1], state_perm);
+        }
+    }
+    if (snapshot_count == 1 && context.get_output_names().size() == 2) {
         // The canonical graph consumes the packed GGML result only through separate attention
         // and state VIEWs. Publish the native outputs under those VIEW names to avoid
         // flatten -> concat -> reshape -> slice -> reshape chains. This also works for B > 1.
-        return rename_outputs_with_suffix({attn_4d, state_transposed}, context.get_name());
+        return rename_outputs_with_suffix({attn_4d, states}, context.get_name());
     }

     auto flat_shape_1d = ov::op::v0::Constant::create(ov::element::i64, {1}, {-1});
     auto attn = std::make_shared<ov::op::v1::Reshape>(attn_4d, flat_shape_1d, false);
-    auto new_state = std::make_shared<ov::op::v1::Reshape>(state_transposed, flat_shape_1d, false);
+    auto new_state = std::make_shared<ov::op::v1::Reshape>(states, flat_shape_1d, false);
     auto packed = std::make_shared<ov::op::v0::Concat>(ov::OutputVector{attn, new_state}, 0);
     auto out_shape = ov::op::v0::Constant::create(ov::element::i64, {4},
                                                   std::vector<int64_t>{1, 1, -1 /*T * B + S_v * B*/, S_v * H_v});
     auto res = std::make_shared<ov::op::v1::Reshape>(packed, out_shape, false);

+    if (context.get_output_names().size() == 3) {
+        auto flat_states = std::make_shared<ov::op::v1::Reshape>(
+            states,
+            ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{1, 1, -1, S_v * S_v * H_v}),
+            false);
+        return rename_outputs_with_suffix({res, attn_4d, flat_states}, context.get_name());
+    }
     return rename_outputs_with_suffix({res}, context.get_name());
 }

+OutputVector translate_gated_delta_net(const NodeContext & context) {
+    return translate_gdn(context, nullptr);
+}
+
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+OutputVector translate_recurrent_gated_delta_net(const NodeContext & context, const OutputVector & cache_inputs) {
+    return translate_gdn(context, &cache_inputs);
+}
+#endif
+
 static OutputVector translate_gated_delta_net_ref(const NodeContext & context) {
     num_inputs_check(context, 6, 6);

diff --git a/ggml/src/ggml-openvino/openvino/op/gated_delta_net.hpp b/ggml/src/ggml-openvino/openvino/op/gated_delta_net.hpp
index 20a4cfdfe..2c0a8ca1f 100644
--- a/ggml/src/ggml-openvino/openvino/op/gated_delta_net.hpp
+++ b/ggml/src/ggml-openvino/openvino/op/gated_delta_net.hpp
@@ -1,3 +1,6 @@
+// Copyright (C) 2018-2026 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
 #pragma once

 #include "openvino/op/op.hpp"
diff --git a/ggml/src/ggml-openvino/openvino/op/get_rows.cpp b/ggml/src/ggml-openvino/openvino/op/get_rows.cpp
index 200821d97..89a75570c 100644
--- a/ggml/src/ggml-openvino/openvino/op/get_rows.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/get_rows.cpp
@@ -11,9 +11,9 @@
 #include <openvino/op/constant.hpp>
 #include <openvino/op/convert.hpp>
 #include <openvino/op/gather.hpp>
+#include <openvino/op/reshape.hpp>
 #include <openvino/op/shape_of.hpp>
 #include <openvino/op/slice.hpp>
-#include <openvino/op/squeeze.hpp>
 #include <openvino/op/unsqueeze.hpp>

 namespace ov {
@@ -39,15 +39,14 @@ OutputVector translate_get_rows(const NodeContext & context) {
                                           data.get_partial_shape()[1].compatible(view_shape[3]),
                                       "GET_ROWS: view over a weight must be a row range");
         row_offset = static_cast<int64_t>(view_offset / row_bytes);
-    } else {
+    } else if (context.get_view_input_size(0) > 0) {
         data = process_view_input_new(context, 0);
+    } else {
+        // gemma-4 gathers the mask rows of its output tokens for the KV-shared layers
+        data = get_kq_mask_input(context, 0);
     }

     auto op_case = context.get_op_case();
-    if (op_case == 3 || op_case == 4) {
-        return {data};
-    }
-
     ov::Output<ov::Node> indices;
     if ((op_case == 1 || op_case == 2) && context.has_input("s_copy_active_slot_len")) {
         // Recurrent state reorder (inp->s_copy): slice the active (op_case 1) or extra (op_case 2)
@@ -68,20 +67,41 @@ OutputVector translate_get_rows(const NodeContext & context) {
         indices = process_view_input_new(context, 1);
     }

+    if (op_case == 5) {
+        // data [b3, b2, rows, row_size], indices [b3, b2, k]: one gather per (b3, b2) batch. The leading dims reach
+        // here in either layout (stateful models merge b3 into b2), so restore them from the static inner dims
+        const auto data_shape = context.get_input_shape(0);
+        const auto indices_shape = context.get_input_shape(1);
+        data = std::make_shared<ov::op::v1::Reshape>(
+            data,
+            ov::op::v0::Constant::create(ov::element::i64, {4},
+                                         {(int64_t) -1, data_shape[1].get_length(), data_shape[2].get_length(),
+                                          data_shape[3].get_length()}),
+            false);
+        indices = std::make_shared<ov::op::v1::Reshape>(
+            indices,
+            ov::op::v0::Constant::create(ov::element::i64, {3},
+                                         {(int64_t) -1, indices_shape[2].get_length(), indices_shape[3].get_length()}),
+            false);
+        auto axis = ov::op::v0::Constant::create(ov::element::i32, ov::Shape{}, {2});
+        res = std::make_shared<ov::op::v8::Gather>(data, indices, axis, 2);
+        if (res.get_element_type() != context.get_output_type()) {
+            res = std::make_shared<ov::op::v0::Convert>(res, context.get_output_type());
+        }
+        return rename_outputs_with_suffix({res}, context.get_name());
+    }
+
     // data[1,b,x,y] ind[1,1,b,x'] test-backend-ops case
     // data[x,y] ind[1,1,1,x'] normal case
-    // Squeeze the leading dims down to [b,x']. Stateful models drop one rank, so a hardcoded
+    // Drop the leading dims down to [b,x']. Stateful models drop one rank, so a hardcoded
     // {0,1} would also strip the batch dim whenever b == 1 (every decode step).
     const auto indices_rank = indices.get_partial_shape().rank();
     FRONT_END_OP_CONVERSION_CHECK(indices_rank.is_static(), "Expected static rank for GET_ROWS indices");
-    std::vector<int64_t> indices_squeeze_axes;
-    for (int64_t i = 0; i + 2 < indices_rank.get_length(); ++i) {
-        indices_squeeze_axes.push_back(i);
-    }
-    if (!indices_squeeze_axes.empty()) {
-        indices = std::make_shared<ov::op::v0::Squeeze>(
-            indices, ov::op::v0::Constant::create(ov::element::i64, {indices_squeeze_axes.size()},
-                                                  indices_squeeze_axes));
+    if (indices_rank.get_length() > 2) {
+        // Keep zero-token batches: GPU Squeeze-to-Reshape can treat a zero dimension as a copy marker.
+        indices = std::make_shared<ov::op::v1::Reshape>(
+            indices, get_dimensions(std::make_shared<ov::op::v3::ShapeOf>(indices),
+                                    {int(indices_rank.get_length() - 2), int(indices_rank.get_length() - 1)}), false);
     }
     if (row_offset != 0) {
         indices = std::make_shared<ov::op::v1::Add>(
@@ -90,14 +110,14 @@ OutputVector translate_get_rows(const NodeContext & context) {
     if (data.get_partial_shape().rank() == 4) {
         if (!(data.get_partial_shape()[1].is_dynamic()) && data.get_partial_shape()[1].get_length() == 1) {
             // Work-around for a bug in ov cpu plugin for test-backend-ops
-            data = std::make_shared<ov::op::v0::Squeeze>(data,
-                                                         ov::op::v0::Constant::create(ov::element::i64, {2}, {0, 1}));
+            data = std::make_shared<ov::op::v1::Reshape>(
+                data, get_dimensions(data.get_node_shared_ptr(), {2, 3}), false);
             auto axis = ov::op::v0::Constant::create(ov::element::i32, ov::Shape{}, {0});
             res = std::make_shared<ov::op::v8::Gather>(data, indices, axis);
         } else {
             auto axis = ov::op::v0::Constant::create(ov::element::i32, ov::Shape{}, {1});
-            data =
-                std::make_shared<ov::op::v0::Squeeze>(data, ov::op::v0::Constant::create(ov::element::i64, {1}, {0}));
+            data = std::make_shared<ov::op::v1::Reshape>(
+                data, get_dimensions(data.get_node_shared_ptr(), {1, 2, 3}), false);
             // data: [batch, rows, ...], indices: [batch, n] - this is a batched gather
             // (batch_dims=1) along the rows axis. The data and indices batch dims are
             // logically equal (both == n_tokens) but reach this node through independent
@@ -159,8 +179,15 @@ OutputVector translate_get_rows(const NodeContext & context) {
         auto axis = ov::op::v0::Constant::create(ov::element::i32, ov::Shape{}, {1});
         res = std::make_shared<ov::op::v8::Gather>(data, indices, axis, 1);
     } else {
-        auto axis = ov::op::v0::Constant::create(ov::element::i32, ov::Shape{}, {0});
-        res = std::make_shared<ov::op::v8::Gather>(data, indices, axis);
+        // For a tied embedding the same weight also feeds the lm_head matmul. Gathering the
+        // dequantized table would give that dequantization chain a second consumer, forcing the
+        // full table to be materialized; gather the packed rows instead and dequantize those.
+        if (auto compressed = gather_compressed_rows(data, indices)) {
+            res = compressed;
+        } else {
+            auto axis = ov::op::v0::Constant::create(ov::element::i32, ov::Shape{}, {0});
+            res = std::make_shared<ov::op::v8::Gather>(data, indices, axis);
+        }
     }

     if (res.get_element_type() != context.get_output_type()) {
diff --git a/ggml/src/ggml-openvino/openvino/op/mul_mat_id.cpp b/ggml/src/ggml-openvino/openvino/op/mul_mat_id.cpp
index 52c3f5eda..5a06035ef 100644
--- a/ggml/src/ggml-openvino/openvino/op/mul_mat_id.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/mul_mat_id.cpp
@@ -8,20 +8,36 @@
 #include <cstring>
 #include <limits>
 #include <memory>
+#include <numeric>
+#include <openvino/op/add.hpp>
 #include <openvino/op/bitwise_and.hpp>
 #include <openvino/op/bitwise_right_shift.hpp>
 #include <openvino/op/broadcast.hpp>
 #include <openvino/op/concat.hpp>
 #include <openvino/op/constant.hpp>
 #include <openvino/op/convert.hpp>
+#include <openvino/op/cum_sum.hpp>
+#include <openvino/op/equal.hpp>
 #include <openvino/op/gather.hpp>
+#include <openvino/op/greater_eq.hpp>
+#include <openvino/op/less.hpp>
+#include <openvino/op/logical_and.hpp>
+#include <openvino/op/logical_not.hpp>
 #include <openvino/op/matmul.hpp>
+#include <openvino/op/maximum.hpp>
 #include <openvino/op/multiply.hpp>
+#include <openvino/op/reduce_logical_or.hpp>
+#include <openvino/op/reduce_sum.hpp>
 #include <openvino/op/reshape.hpp>
+#include <openvino/op/select.hpp>
 #include <openvino/op/shape_of.hpp>
 #include <openvino/op/slice.hpp>
+#include <openvino/op/squeeze.hpp>
+#include <openvino/op/strided_slice.hpp>
 #include <openvino/op/transpose.hpp>
 #include <openvino/op/unsqueeze.hpp>
+#include <openvino/op/util/topk_base.hpp>
+#include <optional>
 #include <vector>

 namespace ov {
@@ -71,6 +87,79 @@ ov::Output<ov::Node> static_shape_dims_or_shapeof(const ov::Output<ov::Node> & i
     return get_dimensions(shape, dims);
 }

+// TopK indices are never -1. Model graphs take the ids from the router's TopK through views and
+// reshapes, so they skip the -1 handling below and keep the GatherMatmul pattern that the GPU MoE
+// fusion matches.
+bool ids_from_topk(ov::Output<ov::Node> ids) {
+    for (;;) {
+        const auto node = ids.get_node_shared_ptr();
+        if (ov::is_type<ov::op::util::TopKBase>(node)) {
+            return ids.get_index() == 1;
+        }
+        if (!ov::is_type<ov::op::v1::Reshape>(node) && !ov::is_type<ov::op::v0::Unsqueeze>(node) &&
+            !ov::is_type<ov::op::v0::Squeeze>(node) && !ov::is_type<ov::op::v0::Convert>(node) &&
+            !ov::is_type<ov::op::v1::Transpose>(node) && !ov::is_type<ov::op::v8::Slice>(node) &&
+            !ov::is_type<ov::op::v1::StridedSlice>(node)) {
+            return false;
+        }
+        ids = node->input_value(0);
+    }
+}
+
+// An id of -1 skips the slot and zeroes the matching row of the result. The skipped slot looks up a
+// valid expert, and the result is multiplied by this mask: 1 for a kept slot, 0 for a skipped one,
+// shaped [ids..., 1] to broadcast over the row.
+ov::Output<ov::Node> kept_slots_mask(const ov::Output<ov::Node> & ids, const ov::element::Type & type) {
+    auto zero = ov::op::v0::Constant::create(ids.get_element_type(), ov::Shape{}, {0});
+    ov::Output<ov::Node> keep = std::make_shared<ov::op::v1::GreaterEqual>(ids, zero);
+    keep = std::make_shared<ov::op::v0::Convert>(keep, type);
+    return std::make_shared<ov::op::v0::Unsqueeze>(keep, const_i64({-1}));
+}
+
+ov::Output<ov::Node> clamp_skipped_ids(const ov::Output<ov::Node> & ids) {
+    auto zero = ov::op::v0::Constant::create(ids.get_element_type(), ov::Shape{}, {0});
+    return std::make_shared<ov::op::v1::Maximum>(ids, zero);
+}
+
+// GatherMatmul computes a wrong row when two slots of a token name the same expert (ggml leaves
+// that undefined too, see TAG_MUL_MAT_ID_DUPLICATE_ID), so a skipped slot cannot simply look up
+// expert 0 while another slot of the row keeps it. A row with S skipped slots keeps at most
+// n_used - S experts, so [0, n_used) holds at least S unused ones: the r-th skipped slot of a row
+// looks up the r-th of them. ids is [n_tokens, n_used].
+ov::Output<ov::Node> distinct_ids_for_skipped_slots(const ov::Output<ov::Node> & ids) {
+    using namespace ov::op;
+    const auto & ps = ids.get_partial_shape();
+    if (ps.rank().is_dynamic() || ps[ps.rank().get_length() - 1].is_dynamic()) {
+        return clamp_skipped_ids(ids);
+    }
+    const auto type = ids.get_element_type();
+    const int64_t n_used = ps[ps.rank().get_length() - 1].get_length();
+    std::vector<int64_t> range(n_used);
+    std::iota(range.begin(), range.end(), 0);
+    auto experts = v0::Constant::create(type, ov::Shape{static_cast<size_t>(n_used)}, range);
+    auto zero = v0::Constant::create(type, ov::Shape{}, {0});
+    auto one = v0::Constant::create(type, ov::Shape{}, {1});
+    auto last_axis = v0::Constant::create(ov::element::i64, ov::Shape{}, {-1});
+
+    // free[t, e]: no slot of row t uses expert e, for e in [0, n_used)
+    auto uses = std::make_shared<v1::Equal>(std::make_shared<v0::Unsqueeze>(ids, const_i64({-1})), experts);
+    auto free = std::make_shared<v1::LogicalNot>(std::make_shared<v1::ReduceLogicalOr>(uses, const_i64({-2}), false));
+    // 1-based rank of each free expert in its row, and of each skipped slot in its row
+    auto free_rank = std::make_shared<v0::CumSum>(std::make_shared<v0::Convert>(free, type), last_axis);
+    auto skip = std::make_shared<v1::Less>(ids, zero);
+    auto skip_rank = std::make_shared<v1::Add>(
+        std::make_shared<v0::CumSum>(std::make_shared<v0::Convert>(skip, type), last_axis, true), one);
+
+    // pick[t, j, e]: expert e is the skip_rank[t, j]-th free expert of row t
+    auto pick = std::make_shared<v1::LogicalAnd>(
+        std::make_shared<v1::Equal>(std::make_shared<v0::Unsqueeze>(skip_rank, const_i64({-1})),
+                                    std::make_shared<v0::Unsqueeze>(free_rank, const_i64({-2}))),
+        std::make_shared<v0::Unsqueeze>(free, const_i64({-2})));
+    auto picked = std::make_shared<v1::ReduceSum>(
+        std::make_shared<v1::Multiply>(std::make_shared<v0::Convert>(pick, type), experts), const_i64({-1}), false);
+    return std::make_shared<v1::Select>(skip, picked, ids);
+}
+
 ov::Output<ov::Node> translate_mul_mat_id_mxfp4_packed(const NodeContext & context,
                                                        ov::Output<ov::Node> expert_weights,
                                                        ov::Output<ov::Node> activations,
@@ -98,6 +187,13 @@ ov::Output<ov::Node> translate_mul_mat_id_mxfp4_packed(const NodeContext & conte
     if (ids.get_element_type() != ov::element::i32 && ids.get_element_type() != ov::element::i64) {
         ids = std::make_shared<ov::op::v0::Convert>(ids, ov::element::i32);
     }
+    const auto output_type = context.get_output_type();
+    // Gather and MatMul handle repeated ids, so clamping a skipped slot to expert 0 is enough here.
+    std::optional<ov::Output<ov::Node>> keep;
+    if (!ids_from_topk(ids)) {
+        keep = kept_slots_mask(ids, output_type);
+        ids = clamp_skipped_ids(ids);
+    }

     auto gather_axis = ov::op::v0::Constant::create(ov::element::i32, ov::Shape{}, {0});

@@ -156,10 +252,12 @@ ov::Output<ov::Node> translate_mul_mat_id_mxfp4_packed(const NodeContext & conte
         ov::OutputVector{batch_dim, get_dimensions(ids_shape, {0, 1}), row_dim}, 0);
     result = std::make_shared<ov::op::v1::Reshape>(result, result_target_dims, false);

-    const auto output_type = context.get_output_type();
     if (result.get_element_type() != output_type) {
         result = std::make_shared<ov::op::v0::Convert>(result, output_type);
     }
+    if (keep) {
+        result = std::make_shared<ov::op::v1::Multiply>(result, *keep);
+    }
     return result;
 }

@@ -219,6 +317,12 @@ OutputVector translate_mul_mat_id(const NodeContext & context) {
     }

     const auto output_type = context.get_output_type();
+    std::optional<ov::Output<ov::Node>> keep;
+    if (!ids_from_topk(ids)) {
+        keep = kept_slots_mask(ids, output_type);
+        ids = distinct_ids_for_skipped_slots(ids);
+    }
+
     const auto activations_type = ggml_openvino_is_gpu() ? ov::element::f16 : ov::element::f32;
     if (activations.get_element_type() != activations_type) {
         activations = std::make_shared<ov::op::v0::Convert>(activations, activations_type);
@@ -244,6 +348,9 @@ OutputVector translate_mul_mat_id(const NodeContext & context) {
     if (result.get_element_type() != output_type) {
         result = std::make_shared<ov::op::v0::Convert>(result, output_type);
     }
+    if (keep) {
+        result = std::make_shared<ov::op::v1::Multiply>(result, *keep);
+    }

     return rename_outputs_with_suffix({result}, context.get_name());
 }
diff --git a/ggml/src/ggml-openvino/openvino/op/mulmat.cpp b/ggml/src/ggml-openvino/openvino/op/mulmat.cpp
index 9d4315aa4..0cfe44960 100644
--- a/ggml/src/ggml-openvino/openvino/op/mulmat.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/mulmat.cpp
@@ -31,6 +31,8 @@ OutputVector translate_mulmat(const NodeContext & context) {

     ov::Output<ov::Node> B = process_view_input_new(context, 0);
     ov::Output<ov::Node> A = process_view_input_new(context, 1);
+    // the stateful V cache with `-fa off` is not transposed (see permute)
+    const bool transpose_b = B.get_node()->get_rt_info().count(GGML_OV_V_TOKEN_MAJOR) == 0;

     if (A.get_element_type() != B.get_element_type()) {
         B = std::make_shared<ov::op::v0::Convert>(B, context.get_input_type(1));
@@ -48,6 +50,37 @@ OutputVector translate_mulmat(const NodeContext & context) {
     Output<Node> Z = A_batch_larger ? B : A;
     auto Z_shape = A_batch_larger ? B_shape : A_shape;
     int64_t factor = batch_large / batch_small;
+    if (factor > 1 && batch_small > 1 && A_batch_larger && A_shape[0] == B_shape[0]) {
+        // B is shared by groups of factor consecutive A matrices (grouped-query attention with `-fa off`: K and V
+        // have fewer heads than Q and KQ). Stack each group along the rows of A instead of repeating B, which would
+        // copy the whole K or V cache factor times per layer and token:
+        //   A [n, batch_large, rows, k] -> [n, batch_small, factor * rows, k] (a pure reshape: the group is adjacent)
+        //   A x B^T [n, batch_small, factor * rows, cols] -> [n, batch_large, rows, cols]
+        // The zeros copy the input dimensions (special_zero), which may be dynamic.
+        ov::Output<Node> A_grouped = std::make_shared<ov::op::v1::Reshape>(
+            A, ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{0, batch_small, -1, 0}), true);
+        // A is often a Reshape of the grouped scores (see softmax): take them directly. With one Reshape less, the
+        // OpenVINO SDPAFusion can match the attention
+        const auto * A_reshape = ov::as_type<ov::op::v1::Reshape>(A.get_node());
+        const auto * A_target =
+            A_reshape != nullptr ? ov::as_type<ov::op::v0::Constant>(A_reshape->get_input_node_ptr(1)) : nullptr;
+        if (context.is_stateful() && A_target != nullptr && A_reshape->get_special_zero() &&
+            A_target->cast_vector<int64_t>() == std::vector<int64_t>{0, batch_large, -1, 0} &&
+            A_reshape->get_input_partial_shape(0)[1] == batch_small) {
+            A_grouped = A_reshape->input_value(0);
+        }
+        ov::Output<Node> res = std::make_shared<ov::op::v0::MatMul>(A_grouped, B, false, transpose_b);
+        res = std::make_shared<ov::op::v1::Reshape>(
+            res, ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{0, batch_large, -1, 0}),
+            true);
+        // lets SOFT_MAX work on the grouped scores (see softmax)
+        res.get_node()->get_rt_info()[GGML_OV_GROUPED_ROWS] = factor;
+        const auto output_type = context.get_output_type();
+        if (res.get_element_type() != output_type) {
+            res = std::make_shared<ov::op::v0::Convert>(res, output_type);
+        }
+        return rename_outputs_with_suffix({res}, context.get_name());
+    }
     if (factor > 1 && batch_small > 1) {
         auto batch_large_node = ov::op::v0::Constant::create(ov::element::i64, {1}, std::vector<int64_t>{batch_large});
         auto batch_small_node = ov::op::v0::Constant::create(ov::element::i64, {1}, std::vector<int64_t>{batch_small});
@@ -75,7 +108,6 @@ OutputVector translate_mulmat(const NodeContext & context) {
         A = Z;
     }

-    bool transpose_b = true;
     ov::Output<Node> res = std::make_shared<ov::op::v0::MatMul>(A, B, false, transpose_b);

     const auto output_type = context.get_output_type();
diff --git a/ggml/src/ggml-openvino/openvino/op/pad.cpp b/ggml/src/ggml-openvino/openvino/op/pad.cpp
index ae3d7be18..a99480183 100644
--- a/ggml/src/ggml-openvino/openvino/op/pad.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/pad.cpp
@@ -3,11 +3,13 @@

 #include <array>
 #include <cstdint>
+#include <openvino/op/concat.hpp>
 #include <openvino/op/constant.hpp>
 #include <openvino/op/gather.hpp>
 #include <openvino/op/pad.hpp>
 #include <openvino/op/reshape.hpp>
 #include <openvino/op/shape_of.hpp>
+#include <openvino/op/subtract.hpp>
 #include <utility>
 #include <vector>

@@ -20,16 +22,25 @@ namespace {

 ov::Output<ov::Node> translate_circular_pad(ov::Output<ov::Node> input,
                                             const std::array<int32_t, 8> & pads,
-                                            const ov::Shape & input_shape) {
+                                            const ov::Shape & input_shape,
+                                            bool stateful) {
     ov::Output<ov::Node> result = std::move(input);

     const std::array<int32_t, 4> pads_begin = {pads[6], pads[4], pads[2], pads[0]};
     const std::array<int32_t, 4> pads_end = {pads[7], pads[5], pads[3], pads[1]};
+    // Stateful models drop leading dims, so the input can have rank < 4; align pads to the trailing input axes
+    const size_t dropped =
+        stateful && input_shape.size() < pads_begin.size() ? pads_begin.size() - input_shape.size() : 0;
+
+    for (size_t i = 0; i < dropped; ++i) {
+        FRONT_END_CHECK_IMPLEMENTED(pads_begin[i] == 0 && pads_end[i] == 0,
+                                    "Circular PAD cannot pad a dimension dropped from the input rank");
+    }

     for (size_t axis = 0; axis < input_shape.size(); ++axis) {
         const int64_t input_dim = static_cast<int64_t>(input_shape[axis]);
-        const int64_t pad_begin = pads_begin[axis];
-        const int64_t pad_end = pads_end[axis];
+        const int64_t pad_begin = pads_begin[dropped + axis];
+        const int64_t pad_end = pads_end[dropped + axis];

         if (pad_begin == 0 && pad_end == 0) {
             continue;
@@ -72,15 +83,44 @@ OutputVector translate_pad(const NodeContext & context) {
     const bool circular = op_params[8] != 0;

     if (circular) {
-        auto res = translate_circular_pad(input, pads, context.get_input_shape(0).to_shape());
+        auto res = translate_circular_pad(input, pads, context.get_input_shape(0).to_shape(), context.is_stateful());
         return rename_outputs_with_suffix({res}, context.get_name());
     }

-    const std::vector<int64_t> pads_begin = {pads[6], pads[4], pads[2], pads[0]};
-    const std::vector<int64_t> pads_end = {pads[7], pads[5], pads[3], pads[1]};
+    std::vector<int64_t> pads_begin = {pads[6], pads[4], pads[2], pads[0]};
+    std::vector<int64_t> pads_end = {pads[7], pads[5], pads[3], pads[1]};
+
+    // Stateful models drop leading dims, so the input can have rank < 4.
+    const auto input_rank = input.get_partial_shape().rank();
+    if (context.is_stateful() && input_rank.is_static() && input_rank.get_length() < 4) {
+        const size_t dropped = 4 - static_cast<size_t>(input_rank.get_length());
+        for (size_t i = 0; i < dropped; ++i) {
+            FRONT_END_OP_CONVERSION_CHECK(pads_begin[i] == 0 && pads_end[i] == 0,
+                                          "PAD: cannot pad a dimension dropped from the input rank");
+        }
+        pads_begin.erase(pads_begin.begin(), pads_begin.begin() + dropped);
+        pads_end.erase(pads_end.begin(), pads_end.begin() + dropped);
+    }

     auto pads_begin_node = ov::op::v0::Constant::create(ov::element::i64, {pads_begin.size()}, pads_begin);
-    auto pads_end_node = ov::op::v0::Constant::create(ov::element::i64, {pads_end.size()}, pads_end);
+    ov::Output<ov::Node> pads_end_node = ov::op::v0::Constant::create(ov::element::i64, {pads_end.size()}, pads_end);
+    // No-cache attention pads K and V on the token axis up to the mask width (TAG_NO_CACHE_PAD), which changes with
+    // the token count, so compute the end padding from the mask at run time
+    const int dyn = context.get_op_dynamic_dim();
+    const int axis = dyn >= 0 ? static_cast<int>(pads_end.size()) - 1 - dyn : -1;
+    if (axis >= 0 && pads_end[axis] != 0 && context.has_input("self_kq_mask")) {
+        auto mask = context.get_input("self_kq_mask");
+        auto mask_len = get_dimensions(std::make_shared<ov::op::v3::ShapeOf>(mask),
+                                       {static_cast<int>(mask.get_partial_shape().rank().get_length()) - 1});
+        auto len = get_dimensions(std::make_shared<ov::op::v3::ShapeOf>(input), {axis});
+        auto pad = std::make_shared<ov::op::v1::Subtract>(mask_len, len);
+        ov::OutputVector parts;
+        for (int i = 0; i < static_cast<int>(pads_end.size()); ++i) {
+            parts.push_back(i == axis ? ov::Output<ov::Node>(pad) :
+                                        ov::op::v0::Constant::create(ov::element::i64, {1}, {pads_end[i]})->output(0));
+        }
+        pads_end_node = std::make_shared<ov::op::v0::Concat>(parts, 0);
+    }
     auto pad_value = ov::op::v0::Constant::create(context.get_input_type(0), ov::Shape{}, {0});
     auto res =
         std::make_shared<ov::op::v1::Pad>(input, pads_begin_node, pads_end_node, pad_value, ov::op::PadMode::CONSTANT);
diff --git a/ggml/src/ggml-openvino/openvino/op/permute.cpp b/ggml/src/ggml-openvino/openvino/op/permute.cpp
index df4f03898..7a65a0b01 100644
--- a/ggml/src/ggml-openvino/openvino/op/permute.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/permute.cpp
@@ -1,6 +1,7 @@
 #include "../node_context.h"
 #include "../op_table.h"
 #include "../utils.h"
+#include "ggml-openvino/ggml-openvino-extra.h"

 #include <climits>
 #include <cstdint>
@@ -12,6 +13,8 @@
 #include <openvino/op/reshape.hpp>
 #include <openvino/op/slice.hpp>
 #include <openvino/op/transpose.hpp>
+#include <openvino/op/unsqueeze.hpp>
+#include <openvino/op/variadic_split.hpp>
 #include <vector>

 namespace ov {
@@ -45,7 +48,45 @@ OutputVector translate_permute(const NodeContext & context) {
                 static_cast<int64_t>(perm_values.size() - 1 - input_axis);
         }
     }
-    if (op_case == 1 || context.is_stateful()) {
+    if (context.is_stateful() && (op_case == 5 || op_case == 6)) {
+        // `-fa off` reads V transposed. The stateful V cache is token-major, [1, seq, n_heads_kv, head_size]
+        const auto output_shape = context.get_output_shape().to_shape();
+        const int64_t n_heads = output_shape[1];
+        const int64_t head_size = output_shape[2];
+        const auto & state_shape = src.get_partial_shape();
+        const bool single_head = n_heads == 1 && state_shape[2].get_length() == 1;
+        if (!single_head && state_shape[2].get_length() * state_shape[3].get_length() != n_heads * head_size) {
+            // llama.cpp pads the V cache rows to the widest layer: keep this layer's rows
+            auto zero = ov::op::v0::Constant::create(ov::element::i64, {1}, {0});
+            auto one = ov::op::v0::Constant::create(ov::element::i64, {1}, {1});
+            auto rows = std::make_shared<ov::op::v1::Reshape>(
+                src, ov::op::v0::Constant::create(ov::element::i64, {3}, std::vector<int64_t>{0, 0, -1}), true);
+            auto sliced = std::make_shared<ov::op::v8::Slice>(
+                rows, zero, ov::op::v0::Constant::create(ov::element::i64, {1}, {n_heads * head_size}), one,
+                ov::op::v0::Constant::create(ov::element::i64, {1}, {2}));
+            src = std::make_shared<ov::op::v1::Reshape>(
+                sliced,
+                ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{0, 0, n_heads, head_size}),
+                true);
+        }
+        // keep V as [1, n_heads_kv, kv, head_size]: MUL_MAT then multiplies without transposing it
+        res = std::make_shared<ov::op::v1::Transpose>(
+            src, ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{0, 2, 1, 3}));
+        if (single_head && state_shape[3].get_length() != head_size) {
+            // the padded rows, as above. After the Transpose, which then cancels the state relayout, so the Slice reads
+            // the KV cache in place: between the two they become copies of the cache, recompiled for every length
+            res = std::make_shared<ov::op::v8::Slice>(res, ov::op::v0::Constant::create(ov::element::i64, {1}, {0}),
+                                                      ov::op::v0::Constant::create(ov::element::i64, {1}, {head_size}),
+                                                      ov::op::v0::Constant::create(ov::element::i64, {1}, {1}),
+                                                      ov::op::v0::Constant::create(ov::element::i64, {1}, {3}));
+        }
+        res.get_node()->get_rt_info()[GGML_OV_V_TOKEN_MAJOR] = true;
+    } else if (op_case == 1 || context.is_stateful()) {
+        if (op_case == 2 && src.get_partial_shape().rank() == 3) {
+            // Stateful RoPE lifts Q to rank 4 (see translate_rope). Do the same for Q of a layer without RoPE (NoPE)
+            src = std::make_shared<ov::op::v0::Unsqueeze>(src,
+                                                          ov::op::v0::Constant::create(ov::element::i64, Shape{}, {0}));
+        }
         // The stateful path carries hidden-state tensors in a rank-3 layout (the
         // leading batch dim is dropped, e.g. Gemma4's per-layer-embedding path). The
         // perm above is rank-4; when the actual input is rank-3, drop the batch axis
@@ -78,6 +119,27 @@ OutputVector translate_permute(const NodeContext & context) {

         auto reshaped = std::make_shared<ov::op::v1::Reshape>(src, new_shape, true);
         res = std::make_shared<ov::op::v1::Transpose>(reshaped, perm);
+    } else if ((op_case == 3 || op_case == 4) && is_kvcache_4d_shape(src.get_partial_shape())) {
+        // cache [1, ctx, n_heads_kv, head_size] (see set_rows): take the first attention_size tokens with
+        // VariadicSplit, the read side of the GPU plugin's StatelessKVFusion. The fused StatelessKV then hands
+        // attention a view of those tokens, with no copy
+        FRONT_END_OP_CONVERSION_CHECK((perm_values == std::vector<int64_t>{0, 2, 1, 3}),
+                                      "Unsupported permutation of a 4-D KV cache");
+        Output<Node> attention_size;
+        if (!context.has_input("attention_size")) {
+            attention_size =
+                ov::op::v0::Constant::create(ov::element::i64, {1}, {context.get_output_shape().to_shape()[2]});
+        } else if (op_case == 3) {
+            attention_size = context.get_input("attention_size");
+        } else {
+            attention_size = context.get_input("attention_size_swa");
+        }
+        auto split_lengths = std::make_shared<ov::op::v0::Concat>(
+            ov::OutputVector{attention_size, ov::op::v0::Constant::create(ov::element::i64, {1}, {-1})}, 0);
+        auto tokens = std::make_shared<ov::op::v1::VariadicSplit>(
+            src, ov::op::v0::Constant::create(ov::element::i64, {}, {1}), split_lengths);
+        res = std::make_shared<ov::op::v1::Transpose>(tokens->output(0),
+                                                      ov::op::v0::Constant::create(ov::element::i64, {4}, perm_values));
     } else {
         auto perm = ov::op::v0::Constant::create(ov::element::i64, {4}, perm_values);
         auto cache_shape = src.get_partial_shape();
@@ -93,7 +155,9 @@ OutputVector translate_permute(const NodeContext & context) {

         Output<Node> attention_size;
         if (!context.has_input("attention_size")) {
-            attention_size = ov::op::v0::Constant::create(ov::element::i64, {1}, {output_shape[2]});
+            // the attended tokens are the last axis of the transposed V cache (`-fa off`)
+            const size_t tokens_axis = (op_case == 5 || op_case == 6) ? 3 : 2;
+            attention_size = ov::op::v0::Constant::create(ov::element::i64, {1}, {output_shape[tokens_axis]});
         } else if (op_case == 3 || op_case == 5) {
             attention_size = context.get_input("attention_size");
         } else {
@@ -121,7 +185,37 @@ OutputVector translate_permute(const NodeContext & context) {
         auto zero = ov::op::v0::Constant::create(ov::element::i64, {1}, {0});
         auto one = ov::op::v0::Constant::create(ov::element::i64, {1}, {1});

-        if (op_case == 3 || op_case == 4) {
+        if ((op_case == 3 || op_case == 4) && context.is_static() && ggml_openvino_npu_kv_head_major_enabled() &&
+            cache_shape.rank().get_length() == 4 && cache_shape[1].is_static() &&
+            cache_shape[1].get_length() == n_heads && n_heads > 1) {
+            // Head-major cache [1, heads, rows, head_size] is already in attention layout: only cut the window.
+            // A transposed V cache [1, heads, head_size, rows] is cut on axis 3 and stays transposed.
+            const bool v_transposed = context.get_input_names()[0].find("cache_v_l") != std::string::npos;
+            auto axis = ov::op::v0::Constant::create(ov::element::i64, {1}, {v_transposed ? 3 : 2});
+            res = std::make_shared<ov::op::v8::Slice>(src, zero, attention_size, one, axis);
+            return rename_outputs_with_suffix({res}, context.get_name());
+        }
+
+        if ((op_case == 3 || op_case == 4) && n_seq == 1 && perm_values == std::vector<int64_t>{0, 2, 1, 3}) {
+            // one sequence: slice the cache [1, 1, ctx, n_heads * head_size] on its context axis, then split the
+            // heads. The reshape after the slice also restates the head count and head size: the GPU plugin loses
+            // all dimensions of a slice with a runtime end, and SDPA would then run its reference kernel
+            auto two = ov::op::v0::Constant::create(ov::element::i64, {1}, {2});
+            auto sliced = std::make_shared<ov::op::v8::Slice>(src, zero, attention_size, one, two);
+            if (n_heads == 1) {
+                res = std::make_shared<ov::op::v1::Reshape>(
+                    sliced,
+                    ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{1, 1, -1, head_size}),
+                    false);
+            } else {
+                auto split = std::make_shared<ov::op::v1::Reshape>(
+                    sliced,
+                    ov::op::v0::Constant::create(ov::element::i64, {4},
+                                                 std::vector<int64_t>{1, -1, n_heads, head_size}),
+                    false);
+                res = std::make_shared<ov::op::v1::Transpose>(split, perm);
+            }
+        } else if (op_case == 3 || op_case == 4) {
             auto src_reshaped = std::make_shared<ov::op::v1::Reshape>(
                 src, ov::op::v0::Constant::create(ov::element::i64, {4}, {n_seq, ctx_per_seq, n_heads, head_size}),
                 false);
@@ -133,11 +227,26 @@ OutputVector translate_permute(const NodeContext & context) {
                     std::make_shared<ov::op::v8::Slice>(src_reshaped, seq_active_start, seq_active_end, one, zero);
             }
             auto slice2 = std::make_shared<ov::op::v8::Slice>(after_seq_slice, zero, attention_size, one, one);
-            res = std::make_shared<ov::op::v1::Transpose>(slice2, perm);
+            // restate the head count and head size, see above
+            auto restated = std::make_shared<ov::op::v1::Reshape>(
+                slice2,
+                ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{0, -1, n_heads, head_size}),
+                true);
+            res = std::make_shared<ov::op::v1::Transpose>(restated, perm);
         } else {
             auto three = ov::op::v0::Constant::create(ov::element::i64, {1}, {3});
+            // the transposed V cache holds one row per value element, [n_seq, rows, ctx]. A model whose layers have
+            // different V sizes allocates the rows for the widest layer (gemma-4: 512, sliding-window layers use the
+            // first 256), so keep this layer's n_heads * head_size rows before splitting them into heads
+            const int64_t cache_rows = cache_shape[3].get_length();
+            ov::Output<ov::Node> v_rows = std::make_shared<ov::op::v1::Reshape>(
+                src, ov::op::v0::Constant::create(ov::element::i64, {3}, {n_seq, cache_rows, ctx_per_seq}), false);
+            if (cache_rows != n_heads * head_size) {
+                v_rows = std::make_shared<ov::op::v8::Slice>(
+                    v_rows, zero, ov::op::v0::Constant::create(ov::element::i64, {1}, {n_heads * head_size}), one, one);
+            }
             auto src_reshaped = std::make_shared<ov::op::v1::Reshape>(
-                src, ov::op::v0::Constant::create(ov::element::i64, {4}, {n_seq, n_heads, head_size, ctx_per_seq}),
+                v_rows, ov::op::v0::Constant::create(ov::element::i64, {4}, {n_seq, n_heads, head_size, ctx_per_seq}),
                 false);
             ov::Output<ov::Node> after_seq_slice;
             if (n_seq == 1) {
diff --git a/ggml/src/ggml-openvino/openvino/op/pool_1d.cpp b/ggml/src/ggml-openvino/openvino/op/pool_1d.cpp
new file mode 100644
index 000000000..49baae31a
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/op/pool_1d.cpp
@@ -0,0 +1,48 @@
+#include "../node_context.h"
+#include "../op_table.h"
+#include "../utils.h"
+
+#include <openvino/op/avg_pool.hpp>
+#include <openvino/op/convert.hpp>
+#include <openvino/op/max_pool.hpp>
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace op {
+
+OutputVector translate_pool_1d(const NodeContext & context) {
+    num_inputs_check(context, 1, 1);
+    const int32_t * params = context.get_output_op_params();
+
+    const int k0 = params[1];
+    const int s0 = params[2];
+    const int p0 = params[3];
+
+    const int op_case = context.get_op_case();
+    FRONT_END_OP_CONVERSION_CHECK(op_case == 1 || op_case == 2, "Unsupported POOL_1D mode");
+
+    ov::Output<Node> input = process_view_input_new(context, 0);
+
+    // OV pools over the last two axes; pool only along last one
+    ov::Strides strides{1, static_cast<size_t>(s0)};
+    ov::Shape pads{0, static_cast<size_t>(p0)};
+    ov::Shape kernel{1, static_cast<size_t>(k0)};
+    ov::Output<Node> res;
+    if (op_case == 1) {
+        res = std::make_shared<ov::op::v1::MaxPool>(input, strides, pads, pads, kernel);
+    } else {
+        // ggml divides by the count of valid (not padded) cells
+        res = std::make_shared<ov::op::v1::AvgPool>(input, strides, pads, pads, kernel, true);
+    }
+
+    if (res.get_element_type() != context.get_output_type()) {
+        res = std::make_shared<ov::op::v0::Convert>(res, context.get_output_type());
+    }
+    return rename_outputs_with_suffix({res}, context.get_name());
+}
+
+}  // namespace op
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/op/recurrent_cache_update.hpp b/ggml/src/ggml-openvino/openvino/op/recurrent_cache_update.hpp
new file mode 100644
index 000000000..a4daf8c86
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/op/recurrent_cache_update.hpp
@@ -0,0 +1,65 @@
+// Copyright (C) 2026 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+#pragma once
+#include "openvino/op/op.hpp"
+
+namespace ov::op::internal {
+/// Qwen3.5/Qwen3.6 llama.cpp convolution and generic cache_r maintenance.
+/// Inputs: original cache, active rows, extra rows, reset begin/length,
+/// extra offsets, QKV [1,B,C,T], windows and depthwise weights.
+/// Nine inputs; output 0 is convolution [1,B,T,C].
+/// Cache input 0 is a writable GPU RemoteTensor, updated in place.
+/// Other inputs and output 0 must not overlap its allocation.
+/// The cache must be a Parameter or an exclusive Reshape alias; no other op
+/// may read it in the same inference. Required original rows are staged before
+/// reset, reorder and window writes, preserving cycles and duplicate sources.
+/// A failed inference may partially modify the cache; clear or restore it before retrying.
+class OPENVINO_API RecurrentConvCacheUpdate : public Op {
+public:
+    OPENVINO_OP("RecurrentConvCacheUpdate", "internal")
+    RecurrentConvCacheUpdate() = default;
+    RecurrentConvCacheUpdate(const OutputVector & args, int64_t history);
+    void validate_and_infer_types() override;
+    bool visit_attributes(AttributeVisitor &) override;
+    std::shared_ptr<Node> clone_with_new_inputs(const OutputVector &) const override;
+
+    int64_t get_history() const { return m_history; }
+
+private:
+    int64_t m_history = 0;
+};
+
+/// Qwen3.5/Qwen3.6 llama.cpp GDN and generic cache_s maintenance.
+/// Twelve inputs: Q/K/V, cache, gate/beta,
+/// active/extra rows, reset begin/length, extra offsets, ordered write control.
+/// The only output is attention [B,T,H,V]. Cache input 3 is updated in place
+/// under the same writable-input contract as RecurrentConvCacheUpdate.
+/// snapshot_count saves the newest min(T,count) states in cache, newest first.
+/// Active/nonidentity rows are staged before writes; identity rows need no copy.
+class OPENVINO_API RecurrentGatedDeltaNetUpdate : public Op {
+public:
+    OPENVINO_OP("RecurrentGatedDeltaNetUpdate", "internal")
+    RecurrentGatedDeltaNetUpdate() = default;
+    RecurrentGatedDeltaNetUpdate(const OutputVector & args,
+                                 bool fuse_qk_l2norm = false,
+                                 float q_l2_norm_eps = 1e-6f,
+                                 float k_l2_norm_eps = 1e-6f,
+                                 int64_t snapshot_count = 1);
+    void validate_and_infer_types() override;
+    bool visit_attributes(AttributeVisitor &) override;
+    std::shared_ptr<Node> clone_with_new_inputs(const OutputVector &) const override;
+
+    bool get_fuse_qk_l2norm() const { return m_fuse_qk_l2norm; }
+
+    float get_q_l2_norm_eps() const { return m_q_l2_norm_eps; }
+
+    float get_k_l2_norm_eps() const { return m_k_l2_norm_eps; }
+
+    int64_t get_snapshot_count() const { return m_snapshot_count; }
+
+private:
+    bool m_fuse_qk_l2norm = false;
+    float m_q_l2_norm_eps = 1e-6f, m_k_l2_norm_eps = 1e-6f;
+    int64_t m_snapshot_count = 1;
+};
+}  // namespace ov::op::internal
diff --git a/ggml/src/ggml-openvino/openvino/op/reshape.cpp b/ggml/src/ggml-openvino/openvino/op/reshape.cpp
index 5ed27b1a7..98ef6fa2c 100644
--- a/ggml/src/ggml-openvino/openvino/op/reshape.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/reshape.cpp
@@ -70,6 +70,21 @@ OutputVector translate_reshape(const NodeContext & context) {
     case 8:
         shape[0] = -1;
         break;
+    case 10:
+        if (context.has_input("s_copy_active_slot_len")) {
+            new_shape_node = std::make_shared<ov::op::v0::Concat>(ov::OutputVector{
+                context.get_input("s_copy_active_slot_len"),
+                ov::op::v0::Constant::create(ov::element::i64, {3},
+                    std::vector<int64_t>{-1, shape[2], shape[3]})}, 0);
+        } else {
+            shape[1] = -1;
+        }
+        break;
+    case 11:
+        // ROPE positions keep the layout of inp_pos. gemma-4 reshapes the positions of its output rows, and a
+        // batch can have no output rows, so the size checks of the other cases cannot place the dim.
+        shape = {1, 1, 1, -1};
+        break;
     default:
         FRONT_END_OP_CONVERSION_CHECK(false, "Unsupported RESHAPE case: ", op_case);
     }
diff --git a/ggml/src/ggml-openvino/openvino/op/rms_norm.cpp b/ggml/src/ggml-openvino/openvino/op/rms_norm.cpp
index 980fa3cee..fb9eb5cc1 100644
--- a/ggml/src/ggml-openvino/openvino/op/rms_norm.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/rms_norm.cpp
@@ -1,18 +1,23 @@
 #include "../node_context.h"
 #include "../op_table.h"
 #include "../utils.h"
+#include "ggml-openvino/ggml-openvino-extra.h"

 #include <memory>
 #include <openvino/op/add.hpp>
 #include <openvino/op/constant.hpp>
+#include <openvino/op/concat.hpp>
 #include <openvino/op/divide.hpp>
+#include <openvino/op/gather.hpp>
 #include <openvino/op/multiply.hpp>
 #include <openvino/op/negative.hpp>
 #include <openvino/op/power.hpp>
+#include <openvino/op/range.hpp>
 #include <openvino/op/reduce_mean.hpp>
 #include <openvino/op/reshape.hpp>
 #include <openvino/op/slice.hpp>
 #include <openvino/op/sqrt.hpp>
+#include <openvino/op/squeeze.hpp>

 namespace ov {
 namespace frontend {
@@ -25,23 +30,20 @@ OutputVector translate_rms_norm(const NodeContext & context) {
     auto op_case = context.get_op_case();

     ov::Output<ov::Node> input_node;
-    if (op_case == 3) {
-        // Flatten sequence and token dimensions to match the gate layout.
-        auto input_shape = context.get_input_shape(0).to_shape();
-        input_node = std::make_shared<ov::op::v1::Reshape>(
-            context.get_input(0),
-            ov::op::v0::Constant::create(
-                ov::element::i64, {4}, std::vector<int64_t>{1, -1, (int64_t) input_shape[2], (int64_t) input_shape[3]}),
-            false);
+    if (op_case == 3 || op_case == 4) {
+        input_node = context.get_input(0);
     } else if (op_case == 1) {
         input_node = process_view_input_new(context, 0);
-    } else if (op_case == 2) {
+    } else if (op_case == 2 || op_case == 3) {
         auto ssm_state_size = context.get_ssm_state_size();
         // The GDN op packs [attn | new_state] along the row axis; the state occupies the last
         // ssm_state_size * n_seqs rows. Slice it off (scaling by the active sequence count) to keep
         // just the attention output.
         ov::Output<ov::Node> state_end;
-        if (context.has_input("s_copy_active_slot_len")) {
+        const auto attention_rows_name = "gdn_attention_rows_" + context.get_view_input_src_name(0, 0);
+        if (context.has_input(attention_rows_name)) {
+            state_end = context.get_input(attention_rows_name);
+        } else if (context.has_input("s_copy_active_slot_len")) {
             auto len = context.get_input("s_copy_active_slot_len");
             auto state_rows = std::make_shared<ov::op::v1::Multiply>(
                 ov::op::v0::Constant::create(ov::element::i64, {1}, {ssm_state_size}), len);
@@ -49,29 +51,49 @@ OutputVector translate_rms_norm(const NodeContext & context) {
         } else {
             state_end = ov::op::v0::Constant::create(ov::element::i64, {1}, {-ssm_state_size});
         }
-        auto gdn_attn_output = std::make_shared<ov::op::v8::Slice>(
-            context.get_input(0), ov::op::v0::Constant::create(ov::element::i64, {1}, {0}), state_end,
-            ov::op::v0::Constant::create(ov::element::i64, {1}, {1}),
-            ov::op::v0::Constant::create(ov::element::i64, {1}, {2}));
+        ov::Output<ov::Node> gdn_attn_output;
+        if (context.has_input(attention_rows_name)) {
+            auto indices = std::make_shared<ov::op::v4::Range>(
+                ov::op::v0::Constant::create(ov::element::i64, {}, {0}),
+                std::make_shared<ov::op::v0::Squeeze>(state_end),
+                ov::op::v0::Constant::create(ov::element::i64, {}, {1}), ov::element::i64);
+            gdn_attn_output = std::make_shared<ov::op::v8::Gather>(context.get_input(0), indices,
+                ov::op::v0::Constant::create(ov::element::i64, {}, {2}));
+        } else {
+            gdn_attn_output = std::make_shared<ov::op::v8::Slice>(
+                context.get_input(0), ov::op::v0::Constant::create(ov::element::i64, {1}, {0}), state_end,
+                ov::op::v0::Constant::create(ov::element::i64, {1}, {1}),
+                ov::op::v0::Constant::create(ov::element::i64, {1}, {2}));
+        }

         auto input_shape = context.get_input_shape(0).to_shape();
-        input_node = std::make_shared<ov::op::v1::Reshape>(
-            gdn_attn_output,
-            ov::op::v0::Constant::create(
-                ov::element::i64, {4}, std::vector<int64_t>{1, -1, (int64_t) input_shape[2], (int64_t) input_shape[3]}),
-            false);
+        ov::Output<ov::Node> sequences = ov::op::v0::Constant::create(ov::element::i64, {1}, {input_shape[0]});
+        if (context.has_input("s_copy_active_slot_len")) {
+            sequences = context.get_input("s_copy_active_slot_len");
+        }
+        auto shape = std::make_shared<ov::op::v0::Concat>(ov::OutputVector{
+            sequences, ov::op::v0::Constant::create(ov::element::i64, {3},
+                std::vector<int64_t>{-1, (int64_t) input_shape[2], (int64_t) input_shape[3]})}, 0);
+        input_node = std::make_shared<ov::op::v1::Reshape>(gdn_attn_output, shape, false);

     } else {
         input_node = process_view_input_new(context, 0);
     }
+
+    float eps;
+    memcpy(&eps, context.get_output_op_params(), sizeof(float));
+
+    if (op_case == 3 && ggml_openvino_gdn_prescale_enabled()) {
+        // input is the GDN output pre-scaled by ggml_openvino_gdn_q_scale(): rms_norm(s*x, s*s*eps) == rms_norm(x, eps)
+        const float q_scale = ggml_openvino_gdn_q_scale();
+        eps *= q_scale * q_scale;
+    }
+
     auto square = std::make_shared<ov::op::v1::Multiply>(input_node, input_node);

     auto mean = std::make_shared<ov::op::v1::ReduceMean>(
         square, ov::op::v0::Constant::create(ov::element::i64, ov::Shape{1}, {-1}), true);

-    float eps;
-    memcpy(&eps, context.get_output_op_params(), sizeof(float));
-
     auto rms = std::make_shared<ov::op::v0::Sqrt>(
         std::make_shared<ov::op::v1::Add>(mean, ov::op::v0::Constant::create(ov::element::f32, ov::Shape{1}, {eps})));

diff --git a/ggml/src/ggml-openvino/openvino/op/rope.cpp b/ggml/src/ggml-openvino/openvino/op/rope.cpp
index 2c7fe6256..a01cd0128 100644
--- a/ggml/src/ggml-openvino/openvino/op/rope.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/rope.cpp
@@ -60,6 +60,8 @@ OutputVector translate_rope(const NodeContext & context) {
         if (context.get_input_size() == 3) {
             cache_key += "_ff_" + context.get_input_names()[2];
         }
+        // Not every ROPE takes inp_pos: gemma-4 rotates its KV-shared layers at the positions of the output rows only
+        cache_key += "_pos_" + context.get_input_names()[1];
         if (context.has_input(cache_key + "_cos")) {
             cos_theta_node = context.get_input(cache_key + "_cos");
             sin_theta_node = context.get_input(cache_key + "_sin");
diff --git a/ggml/src/ggml-openvino/openvino/op/scale.cpp b/ggml/src/ggml-openvino/openvino/op/scale.cpp
index 4be2e700c..f77d8ce58 100644
--- a/ggml/src/ggml-openvino/openvino/op/scale.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/scale.cpp
@@ -2,24 +2,9 @@
 #include "../op_table.h"
 #include "../utils.h"

-#include <openvino/core/except.hpp>
 #include <openvino/op/add.hpp>
-#include <openvino/op/concat.hpp>
 #include <openvino/op/constant.hpp>
-#include <openvino/op/convert.hpp>
-#include <openvino/op/equal.hpp>
-#include <openvino/op/gather.hpp>
-#include <openvino/op/greater_eq.hpp>
-#include <openvino/op/if.hpp>
-#include <openvino/op/less.hpp>
-#include <openvino/op/logical_or.hpp>
 #include <openvino/op/multiply.hpp>
-#include <openvino/op/range.hpp>
-#include <openvino/op/reshape.hpp>
-#include <openvino/op/shape_of.hpp>
-#include <openvino/op/slice.hpp>
-#include <openvino/op/squeeze.hpp>
-#include <openvino/op/unsqueeze.hpp>
 #include <vector>

 namespace ov {
@@ -37,37 +22,8 @@ OutputVector translate_scale(const NodeContext & context) {

     auto scale_node = std::make_shared<ov::op::v0::Constant>(ov::element::f32, ov::Shape{}, std::vector<float>{scale});

-    if (context.get_op_case() == 2) {
-        return {context.get_input(0)};
-    }
-
-    if (context.get_op_case() == 1 && context.has_input("cache_rs_reset_len")) {
-        auto cache_rs_reset_idx = context.get_input("cache_rs_reset_idx");
-        auto cache_rs_reset_len = context.get_input("cache_rs_reset_len");
-
-        auto cache_rs = context.get_input(0);
-
-        auto cache_shape = std::make_shared<ov::op::v3::ShapeOf>(cache_rs, ov::element::i64);
-        auto n_slots_1d = std::make_shared<ov::op::v8::Gather>(
-            cache_shape, ov::op::v0::Constant::create(ov::element::i64, ov::Shape{1}, {2}),
-            ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {0}));
-        auto n_slots = std::make_shared<ov::op::v0::Squeeze>(n_slots_1d);
-
-        auto iota = std::make_shared<ov::op::v4::Range>(
-            ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {0}), n_slots,
-            ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {1}), ov::element::i64);
-
-        auto idx_plus_len = std::make_shared<ov::op::v1::Add>(cache_rs_reset_idx, cache_rs_reset_len);
-        auto less_than_idx = std::make_shared<ov::op::v1::Less>(iota, cache_rs_reset_idx);
-        auto greater_equal_idx_plus_len = std::make_shared<ov::op::v1::GreaterEqual>(iota, idx_plus_len);
-        auto keep_mask = std::make_shared<ov::op::v1::LogicalOr>(less_than_idx, greater_equal_idx_plus_len);
-
-        auto keep_mask_f32 = std::make_shared<ov::op::v0::Convert>(keep_mask, ov::element::f32);
-        auto keep_mask_reshape = std::make_shared<ov::op::v0::Unsqueeze>(
-            keep_mask_f32, ov::op::v0::Constant::create(ov::element::i64, ov::Shape{1}, {1}));
-
-        auto cleared_cache_rs = std::make_shared<ov::op::v1::Multiply>(cache_rs, keep_mask_reshape);
-        return rename_outputs_with_suffix({cleared_cache_rs}, context.get_name());
+    if (is_recurrent_cache_reset_case(context)) {
+        return translate_recurrent_cache_reset(context);
     }

     auto scaled = std::make_shared<ov::op::v1::Multiply>(process_view_input_new(context, 0), scale_node);
diff --git a/ggml/src/ggml-openvino/openvino/op/set_rows.cpp b/ggml/src/ggml-openvino/openvino/op/set_rows.cpp
index 3b606c82a..1eae9408b 100644
--- a/ggml/src/ggml-openvino/openvino/op/set_rows.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/set_rows.cpp
@@ -1,26 +1,35 @@
 #include "../node_context.h"
 #include "../op_table.h"
 #include "../utils.h"
+#include "ggml-openvino/ggml-openvino-extra.h"

 #include <cassert>
 #include <cstdint>
 #include <memory>
+#include <numeric>
 #include <openvino/core/node.hpp>
 #include <openvino/core/node_output.hpp>
 #include <openvino/core/type.hpp>
 #include <openvino/frontend/exception.hpp>
+#include <openvino/op/add.hpp>
 #include <openvino/op/broadcast.hpp>
 #include <openvino/op/concat.hpp>
 #include <openvino/op/constant.hpp>
 #include <openvino/op/convert.hpp>
+#include <openvino/op/divide.hpp>
 #include <openvino/op/gather.hpp>
+#include <openvino/op/mod.hpp>
+#include <openvino/op/multiply.hpp>
+#include <openvino/op/range.hpp>
 #include <openvino/op/reshape.hpp>
 #include <openvino/op/scatter_elements_update.hpp>
 #include <openvino/op/scatter_update.hpp>
 #include <openvino/op/shape_of.hpp>
 #include <openvino/op/slice.hpp>
 #include <openvino/op/squeeze.hpp>
+#include <openvino/op/subtract.hpp>
 #include <openvino/op/transpose.hpp>
+#include <openvino/op/unsqueeze.hpp>
 #include <vector>

 namespace ov {
@@ -28,12 +37,68 @@ namespace frontend {
 namespace ggml {
 namespace op {

+static OutputVector translate_conv_snapshots(const NodeContext & context) {
+    using namespace ov::op;
+    FRONT_END_OP_CONVERSION_CHECK(!context.is_static(), "Convolution snapshots require dynamic execution");
+    auto constant = [](std::initializer_list<size_t> shape, std::initializer_list<int64_t> values) {
+        return v0::Constant::create(element::i64, Shape(shape), std::vector<int64_t>(values));
+    };
+    const auto history = context.get_input_shape(0)[3].get_length();
+    const auto features = history * context.get_input_shape(0)[1].get_length();
+    const auto name = context.get_name();
+    auto count = context.get_input("rs_window_count_" + name);
+    auto seqs = context.get_input("rs_seq_count_" + name);
+    auto zero = constant({}, {0});
+    auto one = constant({}, {1});
+    auto range = [&](const Output<Node> & end) {
+        return std::make_shared<v4::Range>(zero, std::make_shared<v0::Squeeze>(end), one, element::i64);
+    };
+
+    // VIEW passes through conv_x; gather overlapping windows in [window, seq, channel, history] order.
+    // Return the full cache to preserve the in-place alias chain.
+    auto begin = context.get_input("rs_src_begin_" + name);
+    auto positions = range(std::make_shared<v1::Multiply>(count, constant({}, {history})));
+    auto columns = std::make_shared<v1::Add>(begin,
+        std::make_shared<v1::Add>(std::make_shared<v1::Divide>(positions, constant({}, {history})),
+                                 std::make_shared<v1::Mod>(positions, constant({}, {history}))));
+    auto column_matrix = std::make_shared<v1::Reshape>(columns, constant({2}, {-1, history}), false);
+    auto windows = std::make_shared<v8::Gather>(context.get_input(0), column_matrix, constant({}, {3}));
+    auto ordered = std::make_shared<v1::Transpose>(
+        std::make_shared<v0::Squeeze>(windows, constant({1}, {0})), constant({4}, {2, 0, 1, 3}));
+    Output<Node> updates = std::make_shared<v1::Reshape>(ordered, constant({4}, {1, 1, -1, features}), false);
+    updates = std::make_shared<v0::Convert>(updates, context.get_output_type());
+
+    auto row = range(std::make_shared<v1::Multiply>(count, seqs));
+    auto plane = std::make_shared<v1::Subtract>(std::make_shared<v1::Subtract>(count, one),
+                                               std::make_shared<v1::Divide>(row, seqs));
+    auto slots = std::make_shared<v1::Add>(context.get_input("rs_slot_begin_" + name),
+        std::make_shared<v1::Add>(std::make_shared<v1::Mod>(row, seqs),
+                                 std::make_shared<v1::Multiply>(plane,
+                                     context.get_input("rs_snapshot_stride_" + name))));
+    auto base = context.get_input(2);
+    const auto full_features = base.get_partial_shape()[3].get_length();
+    const auto feature_begin = (context.get_view_input_offset(2, 0) / context.get_output_type().size()) % full_features;
+    if (features != full_features) {
+        auto old_rows = std::make_shared<v8::Gather>(base, slots, constant({}, {2}));
+        auto head = std::make_shared<v8::Slice>(old_rows, constant({1}, {0}),
+            constant({1}, {int64_t(feature_begin)}), constant({1}, {1}), constant({1}, {3}));
+        auto tail = std::make_shared<v8::Slice>(old_rows, constant({1}, {int64_t(feature_begin) + features}),
+            constant({1}, {INT64_MAX}), constant({1}, {1}), constant({1}, {3}));
+        updates = std::make_shared<v0::Concat>(OutputVector{head, updates, tail}, 3);
+    }
+    auto result = std::make_shared<v3::ScatterUpdate>(base, slots, updates, constant({}, {2}));
+    return rename_outputs_with_suffix({result}, name);
+}
+
 OutputVector translate_set_rows(const NodeContext & context) {
     num_inputs_check(context, 3, 3);

+    if (context.get_op_case() == 3) {
+        return translate_conv_snapshots(context);
+    }
+
     auto data = process_view_input_new(context, 0);
     auto indices = process_view_input_new(context, 1);
-    auto dst = process_view_input_new(context, 2);

     data = std::make_shared<ov::op::v0::Convert>(data, context.get_output_type());

@@ -43,41 +108,143 @@ OutputVector translate_set_rows(const NodeContext & context) {
                                   ((indices_shape[1].is_static() && indices_shape[1].get_length() > 1) ||
                                    (indices_shape[2].is_static() && indices_shape[2].get_length() > 1));

+    // On NPU ScatterUpdate is not in-place: it copies the whole destination to write
+    // one KV row, which dominates decode once ctx grows. ScatterElementsUpdate is
+    // measurably cheaper for the same single-row write.
+    const bool use_elements_update =
+        !multidim_indices && !context.is_stateful() && ggml_openvino_getenv_int("GGML_OPENVINO_KV_SCATTER_ELEMENTS");
+
     auto axes = ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {2});

+    const bool is_cache_write = context.get_op_case() != 2;
+
+    if (is_cache_write && context.is_static() && !context.is_stateful() && !multidim_indices &&
+        ggml_openvino_npu_kv_head_major_enabled()) {
+        // Head-major cache [1, heads, rows, head_size]: write the new rows along axis 2 without reshaping the cache.
+        ov::Output<Node> cache = context.get_input(2);
+        while (auto reshape = ov::as_type_ptr<ov::op::v1::Reshape>(cache.get_node_shared_ptr())) {
+            cache = reshape->input_value(0);
+        }
+        const auto cache_shape = cache.get_partial_shape();
+        if (cache_shape.rank().is_static() && cache_shape.rank().get_length() == 4 && cache_shape[1].is_static() &&
+            cache_shape[3].is_static() && cache_shape[1].get_length() > 1) {
+            const int64_t heads = cache_shape[1].get_length();
+            // Transposed V cache is [1, heads, head_size, rows]: rows go along axis 3.
+            const bool v_transposed = cache.get_node()->get_friendly_name().rfind("cache_v_l", 0) == 0;
+            const int64_t head_size = (v_transposed ? cache_shape[2] : cache_shape[3]).get_length();
+            auto ind_squeezed = std::make_shared<ov::op::v0::Squeeze>(
+                indices, ov::op::v0::Constant::create(ov::element::i64, {3}, {0, 1, 2}));
+            auto rows = std::make_shared<ov::op::v1::Reshape>(
+                data,
+                ov::op::v0::Constant::create(ov::element::i64, {4}, {(int64_t) 1, (int64_t) -1, heads, head_size}),
+                false);
+            auto rows_bhsd = std::make_shared<ov::op::v1::Transpose>(
+                rows, ov::op::v0::Constant::create(ov::element::i64, {4},
+                                                   v_transposed ? std::vector<int64_t>{0, 2, 3, 1} :
+                                                                  std::vector<int64_t>{0, 2, 1, 3}));
+            if (v_transposed && context.is_prefill() && ggml_openvino_npu_hfa_prefill_enabled()) {
+                // On NPU a chunk-sized scatter along the innermost axis of [1, heads, head_size, rows] is very slow:
+                // the HFA prefill outputs the chunk's V^T rows [1, heads, head_size, n] instead and the host copies
+                // them into the cache (as GenAI does for its prefill KV).
+                return rename_outputs_with_suffix({rows_bhsd}, context.get_name());
+            }
+            Output<Node> res = std::make_shared<ov::op::v3::ScatterUpdate>(
+                cache, ind_squeezed, rows_bhsd,
+                v_transposed ? ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {3}) : axes);
+            return rename_outputs_with_suffix({res}, context.get_name());
+        }
+    }
+
+    auto dst = process_view_input_new(context, 2);
     Output<Node> res;
-    if (context.is_stateful()) {
+    if (is_cache_write && context.is_stateful()) {
         int concat_axis = 1;
+        if (context.get_op_case() == 1) {
+            // `-fa off` writes V transposed through a [1, n] reshape of the cache. The state keeps V token-major
+            // like K, and the new values are already in token order, so append them to the state itself
+            dst = dst.get_node()->input_value(0);
+        }
+        FRONT_END_OP_CONVERSION_CHECK(dst.get_partial_shape()[2].is_static() && dst.get_partial_shape()[3].is_static(),
+                                      "Unsupported KV cache layout for stateful execution");
         int64_t dim2 = dst.get_partial_shape()[2].get_length();
         int64_t dim3 = dst.get_partial_shape()[3].get_length();
         data = std::make_shared<ov::op::v1::Reshape>(
             data, ov::op::v0::Constant::create(ov::element::i64, {4}, {(int64_t) 1, (int64_t) -1, dim2, dim3}), false);
         res = std::make_shared<ov::op::v0::Concat>(OutputVector{dst, data}, concat_axis);
-    } else if (multidim_indices) {
+    } else if (is_cache_write && is_kvcache_4d_shape(dst.get_partial_shape())) {
+        // cache [1, ctx, n_heads_kv, head_size]: write whole tokens on axis 1. This is the pattern of the GPU
+        // plugin's StatelessKVFusion (1-D axis, read with VariadicSplit, see permute), which writes the new rows in
+        // place instead of copying the whole cache first
+        FRONT_END_OP_CONVERSION_CHECK(!multidim_indices, "SET_ROWS into a 4-D KV cache needs one index per token");
+        const int64_t n_heads_kv = dst.get_partial_shape()[2].get_length();
+        const int64_t head_size = dst.get_partial_shape()[3].get_length();
+        auto data_tokens = std::make_shared<ov::op::v1::Reshape>(
+            data,
+            ov::op::v0::Constant::create(ov::element::i64, {4}, {(int64_t) 1, (int64_t) -1, n_heads_kv, head_size}),
+            false);
+        auto ind_squeezed = std::make_shared<ov::op::v0::Squeeze>(
+            indices, ov::op::v0::Constant::create(ov::element::i64, {3}, {0, 1, 2}));
+        res = std::make_shared<ov::op::v3::ScatterUpdate>(dst, ind_squeezed, data_tokens,
+                                                          ov::op::v0::Constant::create(ov::element::i64, {1}, {1}));
+        return rename_outputs_with_suffix({res}, context.get_name());
+    } else if (multidim_indices || !is_cache_write) {
+        // updates [.., n_rows, row_size] with one index per row; stateful models may drop the leading 1 of the
+        // updates but not of the destination
+        const auto dst_rank = dst.get_partial_shape().rank();
+        const auto data_rank = data.get_partial_shape().rank();
+        FRONT_END_OP_CONVERSION_CHECK(dst_rank.is_static() && data_rank.is_static() &&
+                                          data_rank.get_length() <= dst_rank.get_length(),
+                                      "SET_ROWS with per-row indices: updates ", data.get_partial_shape(),
+                                      " do not fit the destination ", dst.get_partial_shape());
+        if (data_rank.get_length() < dst_rank.get_length()) {
+            std::vector<int64_t> lead_axes(dst_rank.get_length() - data_rank.get_length());
+            std::iota(lead_axes.begin(), lead_axes.end(), 0);
+            data = std::make_shared<ov::op::v0::Unsqueeze>(
+                data, ov::op::v0::Constant::create(ov::element::i64, {lead_axes.size()}, lead_axes));
+        }
         auto updates_shape = std::make_shared<ov::op::v3::ShapeOf>(data, ov::element::i64);

         auto indices_rank3 = std::make_shared<ov::op::v0::Squeeze>(
             indices, ov::op::v0::Constant::create(ov::element::i64, {1}, {0}));
         auto one = ov::op::v0::Constant::create(ov::element::i64, {1}, {1});
-        auto indices_rank4_shape = std::make_shared<ov::op::v0::Concat>(OutputVector{get_dimensions(updates_shape, {0, 1, 2}), one}, 0);
-        auto indices_rank4 = std::make_shared<ov::op::v1::Reshape>(indices_rank3, indices_rank4_shape, false);
-        auto broadcasted_indices = std::make_shared<ov::op::v3::Broadcast>(indices_rank4, updates_shape);
+        std::vector<int> row_dims(dst_rank.get_length() - 1);
+        std::iota(row_dims.begin(), row_dims.end(), 0);
+        auto indices_rows_shape =
+            std::make_shared<ov::op::v0::Concat>(OutputVector{get_dimensions(updates_shape, row_dims), one}, 0);
+        auto indices_rows = std::make_shared<ov::op::v1::Reshape>(indices_rank3, indices_rows_shape, false);
+        auto broadcasted_indices = std::make_shared<ov::op::v3::Broadcast>(indices_rows, updates_shape);
+        auto rows_axis = ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {-2});

-        res = std::make_shared<ov::op::v3::ScatterElementsUpdate>(dst, broadcasted_indices, data, axes);
+        res = std::make_shared<ov::op::v3::ScatterElementsUpdate>(dst, broadcasted_indices, data, rows_axis);
     } else {
         auto row_size = context.get_input_shape(2)[3].get_length();
         auto ind_squeezed = std::make_shared<ov::op::v0::Squeeze>(
             indices, ov::op::v0::Constant::create(ov::element::i64, {3}, {0, 1, 2}));
-        auto data_reshaped = std::make_shared<ov::op::v1::Reshape>(
-            data,
-            ov::op::v0::Constant::create(ov::element::i64, {4},
-                                         {(int64_t) 1, (int64_t) 1, (int64_t) -1, (int64_t) row_size}),
-            false);
-        res = std::make_shared<ov::op::v3::ScatterUpdate>(dst, ind_squeezed, data_reshaped, axes);
+        // skip the reshape when the rows already have that shape: it costs a GPU primitive per cache write
+        const auto & data_shape = data.get_partial_shape();
+        const bool data_is_rows = data_shape.rank().is_static() && data_shape.rank().get_length() == 4 &&
+                                  data_shape[0] == 1 && data_shape[1] == 1 && data_shape[3] == row_size;
+        ov::Output<ov::Node> data_reshaped = data;
+        if (!data_is_rows) {
+            data_reshaped = std::make_shared<ov::op::v1::Reshape>(
+                data,
+                ov::op::v0::Constant::create(ov::element::i64, {4},
+                                             {(int64_t) 1, (int64_t) 1, (int64_t) -1, (int64_t) row_size}),
+                false);
+        }
+        if (use_elements_update) {
+            auto updates_shape = std::make_shared<ov::op::v3::ShapeOf>(data_reshaped, ov::element::i64);
+            auto ind_rank4 = std::make_shared<ov::op::v1::Reshape>(
+                ind_squeezed, ov::op::v0::Constant::create(ov::element::i64, {4}, {1, 1, -1, 1}), false);
+            auto broadcasted_indices = std::make_shared<ov::op::v3::Broadcast>(ind_rank4, updates_shape);
+            res = std::make_shared<ov::op::v3::ScatterElementsUpdate>(dst, broadcasted_indices, data_reshaped, axes);
+        } else {
+            res = std::make_shared<ov::op::v3::ScatterUpdate>(dst, ind_squeezed, data_reshaped, axes);
+        }
     }

     auto dst_reshape = ov::as_type_ptr<ov::op::v1::Reshape>(dst.get_node_shared_ptr());
-    if (!multidim_indices && dst_reshape) {
+    if (!multidim_indices && is_cache_write && dst_reshape) {
         // Fix the case of multiple sequences, reshape back to original shape [1, n_seq, ctx_per_seq, emb]
         // ctx_per_seq is not fixed due to llama-bench compatibility
         auto dst_shape_partial = dst_reshape->get_input_partial_shape(0);
diff --git a/ggml/src/ggml-openvino/openvino/op/softmax.cpp b/ggml/src/ggml-openvino/openvino/op/softmax.cpp
index b391d3f91..546987fd7 100644
--- a/ggml/src/ggml-openvino/openvino/op/softmax.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/softmax.cpp
@@ -1,22 +1,25 @@
 #include "../node_context.h"
 #include "../op_table.h"
 #include "../utils.h"
+#include "ggml-openvino/ggml-openvino-extra.h"

 #include <cmath>
 #include <cstdint>
 #include <cstring>
 #include <memory>
-#include <openvino/op/broadcast.hpp>
 #include <openvino/frontend/exception.hpp>
 #include <openvino/op/add.hpp>
+#include <openvino/op/broadcast.hpp>
 #include <openvino/op/concat.hpp>
 #include <openvino/op/constant.hpp>
 #include <openvino/op/convert.hpp>
+#include <openvino/op/maximum.hpp>
 #include <openvino/op/multiply.hpp>
 #include <openvino/op/reshape.hpp>
 #include <openvino/op/shape_of.hpp>
 #include <openvino/op/slice.hpp>
 #include <openvino/op/softmax.hpp>
+#include <openvino/op/tile.hpp>
 #include <vector>

 namespace ov {
@@ -62,6 +65,38 @@ OutputVector translate_soft_max(const NodeContext & context) {
     const bool has_sinks = second_input_is_sinks || context.get_input_size() > 2;
     const size_t sinks_input_idx = second_input_is_sinks ? 1 : 2;

+    // Scores of a grouped-query MUL_MAT (mulmat) come as [n, n_heads_kv, factor * rows, cols], reshaped to
+    // [n, n_heads, rows, cols]. Apply the scale, the mask (repeated factor times along the rows) and the softmax to the
+    // grouped scores and reshape after the softmax: the GPU plugin then fuses them with the MatMuls, while a reshape
+    // in between leaves a separate pass over the scores per layer, which is large in prefill.
+    const auto grouped_reshape = ov::as_type_ptr<ov::op::v1::Reshape>(logits.get_node_shared_ptr());
+    if (has_mask && !has_sinks && max_bias <= 0.0f && grouped_reshape &&
+        grouped_reshape->get_rt_info().count(GGML_OV_GROUPED_ROWS) != 0) {
+        const int64_t factor = grouped_reshape->get_rt_info().at(GGML_OV_GROUPED_ROWS).as<int64_t>();
+        ov::Output<ov::Node> grouped = grouped_reshape->input_value(0);
+        if (scale != 1.0f) {
+            grouped = std::make_shared<ov::op::v1::Multiply>(
+                grouped, std::make_shared<ov::op::v0::Constant>(grouped.get_element_type(), ov::Shape{},
+                                                                std::vector<float>{scale}));
+        }
+        ov::Output<ov::Node> mask = get_kq_mask_input(context, 1);
+        if (mask.get_element_type() != grouped.get_element_type()) {
+            mask = std::make_shared<ov::op::v0::Convert>(mask, grouped.get_element_type());
+        }
+        if (ggml_openvino_is_gpu()) {
+            // the GPU plugin can fuse this attention into its SDPA kernel, which works on blocks of keys: a block fully
+            // masked with -inf for a row gives NaN, so use a large finite value (as flash_attn_ext does)
+            mask = std::make_shared<ov::op::v1::Maximum>(
+                mask, std::make_shared<ov::op::v0::Constant>(mask.get_element_type(), ov::Shape{},
+                                                             std::vector<float>{-30000.0f}));
+        }
+        auto mask_tiled = std::make_shared<ov::op::v0::Tile>(
+            mask, ov::op::v0::Constant::create(ov::element::i64, {4}, std::vector<int64_t>{1, 1, factor, 1}));
+        auto probs = std::make_shared<ov::op::v8::Softmax>(std::make_shared<ov::op::v1::Add>(grouped, mask_tiled), -1);
+        auto res = std::make_shared<ov::op::v1::Reshape>(probs, grouped_reshape->input_value(1), true);
+        return rename_outputs_with_suffix({res}, context.get_name());
+    }
+
     // Apply scale first: logits = src0 * scale
     if (scale != 1.0f) {
         auto scale_const =
@@ -75,16 +110,7 @@ OutputVector translate_soft_max(const NodeContext & context) {
     // Optional mask add: logits += mask
     // For max_bias > 0 (ALiBi), apply per-head slope to mask before adding.
     if (has_mask) {
-        ov::Output<ov::Node> mask = context.get_input(1);
-
-        // For stateful
-        std::string mask_name = "KQ_mask_sliced";
-        if (context.get_input_names()[1].find("swa") != std::string::npos) {
-            mask_name = "KQ_mask_swa_sliced";
-        }
-        if (context.has_input(mask_name)) {
-            mask = context.get_input(mask_name);
-        }
+        ov::Output<ov::Node> mask = get_kq_mask_input(context, 1);

         if (mask.get_element_type() != logits.get_element_type()) {
             mask = std::make_shared<ov::op::v0::Convert>(mask, logits.get_element_type());
diff --git a/ggml/src/ggml-openvino/openvino/op/view.cpp b/ggml/src/ggml-openvino/openvino/op/view.cpp
index 486c2ff28..4ebe9c4b4 100644
--- a/ggml/src/ggml-openvino/openvino/op/view.cpp
+++ b/ggml/src/ggml-openvino/openvino/op/view.cpp
@@ -16,6 +16,26 @@ namespace op {
 OutputVector translate_view(const NodeContext & context) {
     num_inputs_check(context, 1, 1);

+    if (context.get_op_case() == 3) {
+        auto row = context.get_input("sampling_row_" + context.get_name());
+        auto selected = std::make_shared<ov::op::v8::Gather>(context.get_input(0), row,
+            ov::op::v0::Constant::create(ov::element::i64, {}, {2}));
+        auto shape = ov::op::v0::Constant::create(ov::element::i64, {4},
+            std::vector<int64_t>{1, 1, 1, context.get_output_shape()[3].get_length()});
+        return rename_outputs_with_suffix({std::make_shared<ov::op::v1::Reshape>(selected, shape, false)}, context.get_name());
+    }
+
+    if (context.get_op_case() == 4) {
+        auto input = context.get_input(0);
+        auto active = std::make_shared<ov::op::v8::Slice>(input,
+            ov::op::v0::Constant::create(ov::element::i64, {1}, {0}),
+            context.get_input("s_copy_active_slot_len"),
+            ov::op::v0::Constant::create(ov::element::i64, {1}, {1}),
+            ov::op::v0::Constant::create(ov::element::i64, {1},
+                {input.get_partial_shape().rank().get_length() - 2}));
+        return rename_outputs_with_suffix({active}, context.get_name());
+    }
+
     if (context.get_op_case() == 1) {
         // Static-mode identity pass-through for VIEWs over a GATED_DELTA_NET combined output or
         // the conv_input CONCAT; the consuming op (CPY/RMS_NORM) does its own runtime-correct
diff --git a/ggml/src/ggml-openvino/openvino/op_table.cpp b/ggml/src/ggml-openvino/openvino/op_table.cpp
index 857fbf175..399c17ac0 100644
--- a/ggml/src/ggml-openvino/openvino/op_table.cpp
+++ b/ggml/src/ggml-openvino/openvino/op_table.cpp
@@ -34,6 +34,7 @@ std::unordered_map<std::string, CreatorFunction> get_supported_ops() {
     return {
         {"GGML_OP_ADD",             op::translate_add                              },
         {"GGML_OP_ADD_ID",          op::translate_add_id                           },
+        {"GGML_OP_ARANGE",          op::translate_arange                           },
         {"GGML_OP_CONCAT",          op::translate_concat                           },
         {"GGML_OP_CONT",            op::translate_cont                             },
         {"GGML_OP_DUP",             op::translate_cont                             },
@@ -95,6 +96,7 @@ std::unordered_map<std::string, CreatorFunction> get_supported_ops() {
         {"GGML_OP_FLASH_ATTN_EXT",  op::translate_flash_attn_ext                   },
         {"GGML_OP_CLAMP",           op::translate_clamp                            },
         {"GGML_OP_PAD",             op::translate_pad                              },
+        {"GGML_OP_TOP_K",           op::translate_top_k                            },
         {"GGML_OP_SSM_CONV",        op::translate_ssm_conv                         },
         {"GGML_OP_GATED_DELTA_NET", op::translate_gated_delta_net                  },
         {"GGML_OP_REPEAT",          op::translate_repeat                           },
@@ -103,6 +105,7 @@ std::unordered_map<std::string, CreatorFunction> get_supported_ops() {
         {"GGML_OP_DIAG",            op::translate_diag                             },
         {"GGML_OP_TRI",             op::translate_tri                              },
         {"GGML_OP_SET",             op::translate_set                              },
+        {"GGML_OP_POOL_1D",         op::translate_pool_1d                          },
         {"GGML_OP_POOL_2D",         op::translate_pool_2d                          },
         {"GGML_OP_ROLL",            op::translate_roll                             },
         {"GGML_OP_UPSCALE",         op::translate_upscale                          },
diff --git a/ggml/src/ggml-openvino/openvino/op_table.h b/ggml/src/ggml-openvino/openvino/op_table.h
index 6ed7549a2..206966a93 100644
--- a/ggml/src/ggml-openvino/openvino/op_table.h
+++ b/ggml/src/ggml-openvino/openvino/op_table.h
@@ -11,6 +11,7 @@ namespace op {
 #define GGML_OP_CONVERTER(op) OutputVector op(const NodeContext & context)

 GGML_OP_CONVERTER(translate_add);
+GGML_OP_CONVERTER(translate_arange);
 GGML_OP_CONVERTER(translate_cont);
 GGML_OP_CONVERTER(translate_concat);
 GGML_OP_CONVERTER(translate_add_id);
@@ -43,11 +44,15 @@ GGML_OP_CONVERTER(translate_glu_geglu_quick);
 GGML_OP_CONVERTER(translate_set_rows);
 GGML_OP_CONVERTER(translate_cpy);
 GGML_OP_CONVERTER(translate_argsort);
+GGML_OP_CONVERTER(translate_top_k);
 GGML_OP_CONVERTER(translate_flash_attn_ext);
 GGML_OP_CONVERTER(translate_clamp);
 GGML_OP_CONVERTER(translate_pad);
 GGML_OP_CONVERTER(translate_ssm_conv);
 GGML_OP_CONVERTER(translate_gated_delta_net);
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+OutputVector translate_recurrent_gated_delta_net(const NodeContext & context, const OutputVector & cache_inputs);
+#endif
 GGML_OP_CONVERTER(translate_repeat);
 GGML_OP_CONVERTER(translate_cumsum);
 GGML_OP_CONVERTER(translate_fill);
@@ -55,6 +60,7 @@ GGML_OP_CONVERTER(translate_set);
 GGML_OP_CONVERTER(translate_diag);
 GGML_OP_CONVERTER(translate_tri);
 GGML_OP_CONVERTER(translate_solve_tri);
+GGML_OP_CONVERTER(translate_pool_1d);
 GGML_OP_CONVERTER(translate_pool_2d);
 GGML_OP_CONVERTER(translate_roll);
 GGML_OP_CONVERTER(translate_upscale);
diff --git a/ggml/src/ggml-openvino/openvino/pass/keep_sum_square_fp32.cpp b/ggml/src/ggml-openvino/openvino/pass/keep_sum_square_fp32.cpp
new file mode 100644
index 000000000..834a48b2e
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass/keep_sum_square_fp32.cpp
@@ -0,0 +1,147 @@
+#include "keep_sum_square_fp32.h"
+
+#include <map>
+#include <openvino/core/type/element_type.hpp>
+#include <openvino/core/visibility.hpp>
+#include <openvino/op/broadcast.hpp>
+#include <openvino/op/constant.hpp>
+#include <openvino/op/convert.hpp>
+#include <openvino/op/divide.hpp>
+#include <openvino/op/multiply.hpp>
+#include <openvino/op/reduce_sum.hpp>
+#include <openvino/op/sqrt.hpp>
+#include <openvino/op/tile.hpp>
+#include <openvino/pass/pattern/op/wrap_type.hpp>
+#include <unordered_set>
+#include <vector>
+
+#ifdef OPENVINO_STATIC_LIBRARY
+#    define TRANSFORMATIONS_API
+#else
+#    ifdef IMPLEMENT_OPENVINO_API
+#        define TRANSFORMATIONS_API OPENVINO_CORE_EXPORTS
+#    else
+#        define TRANSFORMATIONS_API OPENVINO_CORE_IMPORTS
+#    endif  // IMPLEMENT_OPENVINO_API
+#endif      // OPENVINO_STATIC_LIBRARY
+
+namespace ov {
+// from transformations/rt_info/disable_precision_conversion.hpp, which the OpenVINO runtime package does not ship
+TRANSFORMATIONS_API void disable_conversion(const std::shared_ptr<Node> & node, const element::Type & to);
+}  // namespace ov
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace pass {
+
+namespace {
+// the only consumer of node's output 0, or nullptr
+std::shared_ptr<ov::Node> single_consumer(const std::shared_ptr<ov::Node> & node) {
+    const auto targets = node->output(0).get_target_inputs();
+    return targets.size() == 1 ? targets.begin()->get_node()->shared_from_this() : nullptr;
+}
+
+bool is_scalar_constant(const ov::Output<ov::Node> & output) {
+    auto constant = ov::as_type_ptr<ov::op::v0::Constant>(output.get_node_shared_ptr());
+    return constant && ov::shape_size(constant->get_shape()) == 1;
+}
+
+// Marks the island to stay in f32 and puts a no-op Convert on every edge into or out of it.
+// MarkSugraphsToKeepInMixedPrecision spreads the mark through elementwise/reshape/slice/concat ops in both
+// directions and stops at a Convert. Without the Converts it reaches most of the graph, which then runs in
+// f32 with a Reorder at every boundary and makes decode slow.
+void keep_island_in_fp32(const std::vector<std::shared_ptr<ov::Node>> & island) {
+    std::unordered_set<const ov::Node *> members;
+    for (const auto & node : island) {
+        members.insert(node.get());
+        ov::disable_conversion(node, ov::element::f16);
+    }
+
+    std::map<ov::Output<ov::Node>, std::shared_ptr<ov::Node>> converts_in;
+    for (const auto & node : island) {
+        for (auto input : node->inputs()) {
+            const auto source = input.get_source_output();
+            const auto * producer = source.get_node();
+            if (members.count(producer) || ov::is_type<ov::op::v0::Constant>(producer) ||
+                ov::is_type<ov::op::v0::Convert>(producer)) {
+                continue;
+            }
+            auto & convert = converts_in[source];
+            if (!convert) {
+                convert = std::make_shared<ov::op::v0::Convert>(source, source.get_element_type());
+            }
+            input.replace_source_output(convert);
+        }
+    }
+
+    for (const auto & node : island) {
+        for (const auto & output : node->outputs()) {
+            std::shared_ptr<ov::Node> convert;
+            for (auto target : output.get_target_inputs()) {
+                const auto * consumer = target.get_node();
+                if (members.count(consumer) || ov::is_type<ov::op::v0::Convert>(consumer)) {
+                    continue;
+                }
+                if (!convert) {
+                    convert = std::make_shared<ov::op::v0::Convert>(output, output.get_element_type());
+                }
+                target.replace_source_output(convert);
+            }
+        }
+    }
+}
+}  // namespace
+
+KeepSumSquareFP32::KeepSumSquareFP32() {
+    auto square = ov::pass::pattern::wrap_type<ov::op::v1::Multiply>();
+    auto pattern = ov::pass::pattern::wrap_type<ov::op::v1::ReduceSum>({square, ov::pass::pattern::any_input()});
+
+    const auto callback = [=](ov::pass::pattern::Matcher & m) {
+        auto sum = m.get_match_root();
+        auto square_node = m.get_pattern_value_map().at(square).get_node_shared_ptr();
+        if (square_node->input_value(0) != square_node->input_value(1)) {
+            return false;
+        }
+        std::vector<std::shared_ptr<ov::Node>> island{square_node, sum};
+
+        // optional ggml_scale (e.g. 1/(n-1) in a std), then Sqrt brings the result back into f16 range
+        auto next = single_consumer(sum);
+        if (ov::is_type<ov::op::v1::Multiply>(next) &&
+            (is_scalar_constant(next->input_value(0)) || is_scalar_constant(next->input_value(1)))) {
+            island.push_back(next);
+            next = single_consumer(next);
+        }
+        if (ov::is_type<ov::op::v0::Sqrt>(next)) {
+            island.push_back(next);
+
+            // a * s / ||x|| (Gemma 3n altup magnitude matching): a * s overflows f16 before the Divide
+            std::vector<std::shared_ptr<ov::Node>> norms{next};
+            for (size_t i = 0; i < norms.size(); ++i) {
+                for (const auto & target : norms[i]->output(0).get_target_inputs()) {
+                    auto consumer = target.get_node()->shared_from_this();
+                    if (ov::is_type<ov::op::v0::Tile>(consumer) || ov::is_type<ov::op::v3::Broadcast>(consumer)) {
+                        norms.push_back(consumer);
+                    } else if (ov::is_type<ov::op::v1::Divide>(consumer) && target.get_index() == 1) {
+                        island.push_back(consumer);
+                        auto numerator = consumer->get_input_node_shared_ptr(0);
+                        if (ov::is_type<ov::op::v1::Multiply>(numerator)) {
+                            island.push_back(numerator);
+                        }
+                    }
+                }
+            }
+        }
+
+        keep_island_in_fp32(island);
+        return true;
+    };
+
+    register_matcher(std::make_shared<ov::pass::pattern::Matcher>(pattern, "ov::frontend::ggml::pass::KeepSumSquareFP32"),
+                     callback);
+}
+
+}  // namespace pass
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/pass/keep_sum_square_fp32.h b/ggml/src/ggml-openvino/openvino/pass/keep_sum_square_fp32.h
new file mode 100644
index 000000000..3dac6658a
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass/keep_sum_square_fp32.h
@@ -0,0 +1,22 @@
+#pragma once
+
+#include <openvino/pass/matcher_pass.hpp>
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace pass {
+
+// GPU runs f32 graphs in f16; ReduceSum(x*x) overflows f16 (max 65504) for large rows, e.g. the
+// Gemma 3n gaussian_topk std and altup magnitudes. Keep x*x -> ReduceSum [-> * k] [-> Sqrt [-> a * s / norm]]
+// in f32, fenced by Converts so the f32 region does not spread to the rest of the graph.
+class KeepSumSquareFP32 : public ov::pass::MatcherPass {
+public:
+    OPENVINO_MATCHER_PASS_RTTI("ov::frontend::ggml::pass::KeepSumSquareFP32")
+    KeepSumSquareFP32();
+};
+
+}  // namespace pass
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/pass/move_scale_before_matmul.cpp b/ggml/src/ggml-openvino/openvino/pass/move_scale_before_matmul.cpp
new file mode 100644
index 000000000..097629b0f
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass/move_scale_before_matmul.cpp
@@ -0,0 +1,89 @@
+#include "move_scale_before_matmul.h"
+
+#include <cmath>
+#include <openvino/core/graph_util.hpp>
+#include <openvino/core/rt_info.hpp>
+#include <openvino/op/constant.hpp>
+#include <openvino/op/matmul.hpp>
+#include <openvino/op/multiply.hpp>
+#include <openvino/pass/pattern/op/label.hpp>
+#include <openvino/pass/pattern/op/wrap_type.hpp>
+
+namespace opp = ov::pass::pattern;
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace pass {
+
+namespace {
+// True when output is computed from constants only: a weight, possibly behind its dequantization chain.
+// The depth bound stops the walk early on activation inputs.
+bool is_constant_path(const ov::Output<ov::Node> & output, int depth = 0) {
+    const auto node = output.get_node_shared_ptr();
+    if (ov::is_type<ov::op::v0::Constant>(node)) {
+        return true;
+    }
+    if (depth >= 8 || node->get_input_size() == 0) {
+        return false;
+    }
+    for (const auto & input : node->input_values()) {
+        if (!is_constant_path(input, depth + 1)) {
+            return false;
+        }
+    }
+    return true;
+}
+
+std::shared_ptr<ov::op::v0::Constant> as_scalar_constant(const ov::Output<ov::Node> & output) {
+    auto constant = ov::as_type_ptr<ov::op::v0::Constant>(output.get_node_shared_ptr());
+    return constant && ov::shape_size(constant->get_shape()) == 1 ? constant : nullptr;
+}
+}  // namespace
+
+MoveScaleBeforeMatMul::MoveScaleBeforeMatMul() {
+    auto m_mul = opp::wrap_type<ov::op::v1::Multiply>({opp::any_input(), opp::any_input()});
+
+    const auto callback = [](opp::Matcher & m) {
+        const auto mul = m.get_match_root();
+        for (size_t i = 0; i < 2; ++i) {
+            const auto matmul = ov::as_type_ptr<ov::op::v0::MatMul>(mul->get_input_node_shared_ptr(i));
+            const auto scale = as_scalar_constant(mul->input_value(1 - i));
+            if (!matmul || !scale || matmul->get_output_target_inputs(0).size() != 1) {
+                continue;
+            }
+            // Only a scale that shrinks the output helps, and the Multiply must not broadcast the MatMul output.
+            const float s = scale->cast_vector<float>()[0];
+            if (!(std::fabs(s) < 1.0f) || s == 0.0f ||
+                !mul->get_output_partial_shape(0).same_scheme(matmul->get_output_partial_shape(0))) {
+                return false;
+            }
+            const bool weight_1 = is_constant_path(matmul->input_value(1));
+            const bool weight_0 = is_constant_path(matmul->input_value(0));
+            if (weight_0 == weight_1) {
+                return false;
+            }
+            const size_t act_idx = weight_1 ? 0 : 1;
+
+            const auto act = matmul->input_value(act_idx);
+            auto scaled_act = std::make_shared<ov::op::v1::Multiply>(
+                act, ov::op::v0::Constant::create(act.get_element_type(), ov::Shape{}, {s}));
+            auto inputs = matmul->input_values();
+            inputs[act_idx] = scaled_act;
+            auto new_matmul = matmul->clone_with_new_inputs(inputs);
+            new_matmul->set_friendly_name(mul->get_friendly_name());
+            ov::copy_runtime_info({matmul, mul}, {scaled_act, new_matmul});
+            ov::replace_node(mul, new_matmul);
+            return true;
+        }
+        return false;
+    };
+
+    register_matcher(std::make_shared<opp::Matcher>(m_mul, "ov::frontend::ggml::pass::MoveScaleBeforeMatMul"),
+                     callback);
+}
+
+}  // namespace pass
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/pass/move_scale_before_matmul.h b/ggml/src/ggml-openvino/openvino/pass/move_scale_before_matmul.h
new file mode 100644
index 000000000..852f88bfb
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass/move_scale_before_matmul.h
@@ -0,0 +1,24 @@
+#pragma once
+
+#include <openvino/pass/matcher_pass.hpp>
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace pass {
+
+// GPU and NPU run f32 graphs in f16, so a weight MatMul whose output exceeds 65504 becomes inf even when
+// a scalar Multiply right after it would bring the value back into range (granite residual_scale after
+// ffn_down: the FFN output of some tokens is far above the f16 range). Rewrites
+// Multiply(MatMul(x, W), s) with a scalar constant |s| < 1 as MatMul(Multiply(x, s), W), which is exact
+// and keeps the MatMul output in range.
+class MoveScaleBeforeMatMul : public ov::pass::MatcherPass {
+public:
+    OPENVINO_MATCHER_PASS_RTTI("ov::frontend::ggml::pass::MoveScaleBeforeMatMul")
+    MoveScaleBeforeMatMul();
+};
+
+}  // namespace pass
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/pass_registry/common.cpp b/ggml/src/ggml-openvino/openvino/pass_registry/common.cpp
new file mode 100644
index 000000000..0b64fa1e9
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass_registry/common.cpp
@@ -0,0 +1,73 @@
+#include "pipeline.h"
+
+#include "ggml-openvino/ggml-openvino-extra.h"
+#include "ggml-openvino/openvino/decoder.h"
+#include "ggml-openvino/openvino/pass/fuse_argsort_topk.h"
+#include "ggml-openvino/openvino/pass/fuse_to_conv.h"
+#include "ggml-openvino/openvino/pass/kv_state_seq_axis.h"
+#include "ggml-openvino/openvino/pass/mark_decompression_convert_constant_folding.h"
+#include "ggml-openvino/openvino/pass/mark_dequantization_subgraph.h"
+#include "ggml-openvino/openvino/pass/squeeze_matmul.h"
+
+#include <vector>
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace pipeline {
+
+GraphTraits analyze_graph(const GgmlDecoder & decoder) {
+    GraphTraits traits;
+    traits.device = ggml_openvino_is_npu() ? Device::NPU : ggml_openvino_is_gpu() ? Device::GPU : Device::CPU;
+    traits.is_static = decoder.is_static();
+    traits.is_stateful = decoder.is_stateful();
+    return traits;
+}
+
+void register_common_passes(ov::pass::Manager & manager, const GraphTraits & traits) {
+    (void) traits;
+    manager.register_pass<ov::pass::MarkCompressedFloatConstants>();
+    // Marks the Convert/Subtract/Multiply nodes of our GatherMatmul dequantization chain
+    // (make_int4_weights/make_int8_weights, for_gather_matmul=true) with disable_constant_folding,
+    // so it survives ConstantFolding regardless of whether the target plugin's own
+    // is_decompression_multiply() recognizes GatherMatmul as a valid consumer.
+    manager.register_pass<ov::pass::MarkDequantization>(
+        std::vector<ov::element::Type>{ov::element::u8, ov::element::i8, ov::element::u4, ov::element::i4});
+    manager.register_pass<pass::FuseToConv>();
+    manager.register_pass<pass::FuseArgsortTopK>();
+}
+
+void register_device_passes(ov::pass::Manager & manager, const GraphTraits & traits) {
+    switch (traits.device) {
+    case Device::CPU:
+        register_cpu_passes(manager, traits);
+        break;
+    case Device::GPU:
+        register_gpu_passes(manager, traits);
+        break;
+    case Device::NPU:
+        register_npu_passes(manager, traits);
+        break;
+    }
+}
+
+void register_execution_passes(ov::pass::Manager & manager,
+                               const GraphTraits & traits,
+                               const ov::pass::MakeStateful::ParamResPairs & kv_param_res_pairs) {
+    if (traits.is_stateful) {
+        manager.register_pass<ov::pass::MakeStateful>(kv_param_res_pairs);
+        // Must run after MakeStateful, which is what creates the ReadValue/Assign pairs.
+        if (!ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT")) {
+            manager.register_pass<pass::KVStateSeqAxis>();
+        }
+    }
+
+    if (traits.is_static) {
+        manager.register_pass<pass::SqueezeMatmul>();
+    }
+}
+
+}  // namespace pipeline
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/pass_registry/cpu.cpp b/ggml/src/ggml-openvino/openvino/pass_registry/cpu.cpp
new file mode 100644
index 000000000..449d58973
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass_registry/cpu.cpp
@@ -0,0 +1,17 @@
+#include "pipeline.h"
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace pipeline {
+
+// The CPU plugin runs the device-neutral model as is.
+void register_cpu_passes(ov::pass::Manager & manager, const GraphTraits & traits) {
+    (void) manager;
+    (void) traits;
+}
+
+}  // namespace pipeline
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/pass_registry/gpu.cpp b/ggml/src/ggml-openvino/openvino/pass_registry/gpu.cpp
new file mode 100644
index 000000000..b2caf182e
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass_registry/gpu.cpp
@@ -0,0 +1,49 @@
+#include "pipeline.h"
+
+#include "ggml-openvino/ggml-openvino-extra.h"
+#include "ggml-openvino/openvino/pass/align_eltwise_ranks.h"
+#include "ggml-openvino/openvino/pass/fuse_moe_compressed.h"
+#include "ggml-openvino/openvino/pass/fuse_moe_router.h"
+#include "ggml-openvino/openvino/pass/keep_sum_square_fp32.h"
+#include "ggml-openvino/openvino/pass/move_scale_before_matmul.h"
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace pipeline {
+
+void register_gpu_passes(ov::pass::Manager & manager, const GraphTraits & traits) {
+    (void) traits;
+
+    manager.register_pass<pass::KeepSumSquareFP32>();
+
+    // MoE: fuse router + experts into MOECompressed. It has no CPU plugin implementation, so it is a GPU
+    // pass. GGML_OPENVINO_MOE_OP=0 keeps the unfused GatherMatmul path for fallback/debugging.
+    if (ggml_openvino_getenv_int("GGML_OPENVINO_MOE_OP", 1) != 0) {
+        manager.register_pass<pass::FuseMoeRouter>();
+        manager.register_pass<pass::FuseMoeCompressed>();
+        // Same fusion for models whose gate and up projections share one fused expert
+        // weight (gemma-4). It only matches that shape and only when the experts carry an
+        // integer zero point, so it is a no-op on the separate-gate/up models above.
+        manager.register_pass<pass::FuseMoeCompressedFusedGateUp>();
+    }
+
+    // f16 range: apply a scalar scale before a weight MatMul instead of after it (granite residual_scale).
+    manager.register_pass<pass::MoveScaleBeforeMatMul>();
+
+    // Workaround for an OpenVINO GPU-plugin defect: an eltwise op whose operands differ in
+    // rank is computed wrongly once the plugin fuses it as a post-op into `rms`. gemma-4
+    // dense under stateful execution adds a rank-3 residual to the rank-4 norm output and
+    // decodes garbage on GPU while CPU is correct. Equalising the ranks is a no-op for
+    // NUMPY broadcasting and makes the fused path correct.
+    // Remove once the plugin guards that fusion. Opt out with
+    // GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN=1.
+    if (!ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN")) {
+        manager.register_pass<pass::AlignEltwiseOperandRanks>();
+    }
+}
+
+}  // namespace pipeline
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/pass_registry/npu.cpp b/ggml/src/ggml-openvino/openvino/pass_registry/npu.cpp
new file mode 100644
index 000000000..e43bf9f84
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass_registry/npu.cpp
@@ -0,0 +1,23 @@
+#include "pipeline.h"
+
+#include "ggml-openvino/openvino/pass/move_scale_before_matmul.h"
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+namespace pipeline {
+
+// NPU graph-level lowerings go here (e.g. GatherMatmul -> Gather + MatMul for MoE, which the NPU plugin
+// lacks). NPU compile properties are not passes: they
+// are built in device/npu_config.cpp.
+void register_npu_passes(ov::pass::Manager & manager, const GraphTraits & traits) {
+    (void) traits;
+
+    // f16 range: apply a scalar scale before a weight MatMul instead of after it (granite residual_scale).
+    manager.register_pass<pass::MoveScaleBeforeMatMul>();
+}
+
+}  // namespace pipeline
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/pass_registry/pipeline.h b/ggml/src/ggml-openvino/openvino/pass_registry/pipeline.h
new file mode 100644
index 000000000..8db8ebfd6
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/pass_registry/pipeline.h
@@ -0,0 +1,52 @@
+#pragma once
+
+// Post-translation pass pipelines, split by target device.
+//
+// The translators in op/*.cpp produce a device-neutral OpenVINO model. Everything that is specific to how
+// one device runs it -- fusions for one plugin, workarounds for one plugin's defects, lowerings for ops one
+// plugin lacks -- is registered here, in that device's pipeline:
+//
+//   register_common_passes()     every device, before the device passes (dequantization marking, fusions)
+//   register_device_passes()     pass_registry/{cpu,gpu,npu}.cpp
+//   register_execution_passes()  every device, after the device passes (stateful KV, static shapes)
+//
+// Every device workaround names the plugin defect it covers and keeps its env opt-out next to the
+// registration.
+
+#include <openvino/pass/make_stateful.hpp>
+#include <openvino/pass/manager.hpp>
+
+namespace ov {
+namespace frontend {
+namespace ggml {
+
+class GgmlDecoder;
+
+namespace pipeline {
+
+enum class Device { CPU, GPU, NPU };
+
+// What the pipelines need to know about one graph, computed once per conversion.
+struct GraphTraits {
+    Device device = Device::CPU;
+    bool is_static = false;    // static shapes (NPU, or GGML_OPENVINO_FORCE_STATIC)
+    bool is_stateful = false;  // KV cache kept inside the model (GGML_OPENVINO_STATEFUL_EXECUTION)
+};
+
+GraphTraits analyze_graph(const GgmlDecoder & decoder);
+
+void register_common_passes(ov::pass::Manager & manager, const GraphTraits & traits);
+void register_device_passes(ov::pass::Manager & manager, const GraphTraits & traits);
+void register_execution_passes(ov::pass::Manager & manager,
+                               const GraphTraits & traits,
+                               const ov::pass::MakeStateful::ParamResPairs & kv_param_res_pairs);
+
+// Per-device halves of register_device_passes().
+void register_cpu_passes(ov::pass::Manager & manager, const GraphTraits & traits);
+void register_gpu_passes(ov::pass::Manager & manager, const GraphTraits & traits);
+void register_npu_passes(ov::pass::Manager & manager, const GraphTraits & traits);
+
+}  // namespace pipeline
+}  // namespace ggml
+}  // namespace frontend
+}  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/recurrent_cache_fusion.cpp b/ggml/src/ggml-openvino/openvino/recurrent_cache_fusion.cpp
new file mode 100644
index 000000000..8d0ac026d
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/recurrent_cache_fusion.cpp
@@ -0,0 +1,315 @@
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+#    include "recurrent_cache_fusion.h"
+
+#    include "../ggml-openvino-extra.h"
+#    include "ggml-impl.h"
+#    include "op_table.h"
+#    ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+#        include "op/recurrent_cache_update.hpp"
+#    endif
+#    include "utils.h"
+
+#    include <openvino/op/concat.hpp>
+#    include <openvino/op/constant.hpp>
+#    include <openvino/op/convert.hpp>
+#    include <openvino/op/parameter.hpp>
+#    include <openvino/op/reshape.hpp>
+#    include <openvino/op/slice.hpp>
+
+namespace ov::frontend::ggml {
+RecurrentCacheFusion::RecurrentCacheFusion(const std::shared_ptr<GgmlDecoder> & decoder,
+                                           const std::shared_ptr<TensorMap> & tensors) :
+    m_decoder(decoder),
+    m_tensors(tensors) {
+    if (!ggml_openvino_is_gpu() || decoder->is_static() || decoder->is_stateful() || decoder->is_splited_model() ||
+        ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_FUSED_RECURRENCE")) {
+        return;
+    }
+    auto actual = std::dynamic_pointer_cast<GgmlOvDecoder>(decoder);
+    if (!actual || actual->m_naive || !tensors->count("cache_rs_reset_idx") || !tensors->count("cache_rs_reset_len") ||
+        !tensors->count("s_copy_active_slot_len")) {
+        return;
+    }
+    const auto graph = actual->get_cgraph();
+    std::map<const ggml_tensor *, int> indices;
+    for (int i = 0; i < graph->n_nodes; ++i) {
+        indices[graph->nodes[i]] = i;
+    }
+    for (int i = 0; i < graph->n_nodes; ++i) {
+        const auto cp = graph->nodes[i];
+        if (cp->op != GGML_OP_CPY || decoder->get_op_case(i) != 3 || !cp->src[0] || cp->src[0]->op != GGML_OP_VIEW ||
+            !cp->src[0]->src[0] || cp->src[0]->src[0]->op != GGML_OP_GET_ROWS) {
+            continue;
+        }
+        const auto gather = cp->src[0]->src[0];
+        const auto root = cp->view_src;
+        if (!root || (root->type != GGML_TYPE_F32 && root->type != GGML_TYPE_F16) || root->ne[2] != 1 ||
+            root->ne[3] != 1 || gather->src[0]->op != GGML_OP_RESHAPE || gather->src[0]->src[0] != root ||
+            gather->src[1]->op != GGML_OP_NONE || gather->src[1]->type != GGML_TYPE_I32) {
+            continue;
+        }
+        Group group;
+        group.gather = indices.at(gather);
+        group.remainder = i;
+        for (int j = 0; j < group.gather; ++j) {
+            const auto node = graph->nodes[j];
+            if (GgmlOvDecoder::is_recurrent_cache_reset(node) &&
+                node->view_src == root && decoder->get_op_case(j) == 1) {
+                // SCALE stores (scale, bias); FILL stores its value first and leaves the second parameter 0.
+                float scale, bias;
+                memcpy(&scale, node->op_params, sizeof(float));
+                memcpy(&bias, node->op_params + 1, sizeof(float));
+                if (scale == 0 && bias == 0) {
+                    group.reset = j;
+                }
+            }
+        }
+        if (group.reset < 0) {
+            continue;
+        }
+        // build_conv_state_store emits one CPY without rollback or one SET_ROWS with rollback.
+        for (int j = i + 1; j < graph->n_nodes; ++j) {
+            const auto node = graph->nodes[j];
+            if (node->op != GGML_OP_CONCAT || node->op_params[0] != 0 || !node->src[0] ||
+                node->src[0]->op != GGML_OP_RESHAPE) {
+                continue;
+            }
+            const auto active = node->src[0]->src[0];
+            if (active && active->op == GGML_OP_VIEW && active->src[0] == gather && active->view_offs == 0 &&
+                node->src[0]->ne[3] == 1 && node->src[0]->ne[0] * node->src[0]->ne[1] == root->ne[0]) {
+                group.concat = j;
+                for (int k = j + 1; k < graph->n_nodes; ++k) {
+                    const auto write = graph->nodes[k];
+                    const bool copy = GgmlOvDecoder::is_conv_state_writeback(write);
+                    const bool slots = GgmlOvDecoder::is_conv_state_snapshot_writeback(write);
+                    if ((copy || slots) && write->view_src == root && write->src[0]->src[0] == node &&
+                        write->src[0]->ne[0] == node->src[0]->ne[0] &&
+                        write->src[0]->ne[slots ? 2 : 1] == node->src[0]->ne[1] &&
+                        (slots ? write->src[2] : write->src[1])->view_offs % root->nb[1] == 0) {
+                        if (group.writeback >= 0) {
+                            group.writeback = -1;
+                            break;
+                        }
+                        group.writeback = k;
+                    }
+                }
+                break;
+            }
+        }
+        if (group.concat >= 0) {
+            if (group.writeback < 0) {
+                continue;
+            }
+            for (int j = group.concat + 1; j < graph->n_nodes; ++j) {
+                const auto node = graph->nodes[j];
+                if (node->op == GGML_OP_SSM_CONV && node->src[0] == graph->nodes[group.concat] && node->src[1] &&
+                    node->src[1]->op == GGML_OP_NONE &&
+                    node->src[1]->ne[0] == graph->nodes[group.concat]->src[0]->ne[0] + 1 &&
+                    node->src[1]->ne[1] == graph->nodes[group.concat]->src[0]->ne[1] && node->src[1]->ne[2] == 1 &&
+                    node->src[1]->ne[3] == 1) {
+                    if (group.compute >= 0) {
+                        group.compute = -1;
+                        break;
+                    }
+                    group.compute = j;
+                }
+            }
+        } else {
+            for (int j = i + 1; j < graph->n_nodes; ++j) {
+                const auto node = graph->nodes[j];
+                if (node->op != GGML_OP_GATED_DELTA_NET || !node->src[5] || node->src[5]->op != GGML_OP_RESHAPE) {
+                    continue;
+                }
+                const auto active = node->src[5]->src[0];
+                if (!active || active->op != GGML_OP_VIEW || active->src[0] != gather || active->view_offs != 0 ||
+                    node->src[5]->ne[0] * node->src[5]->ne[1] * node->src[5]->ne[2] != root->ne[0] ||
+                    decoder->get_output_names(j).size() < 2) {
+                    continue;
+                }
+                group.compute = j;
+                for (int k = j + 1; k < graph->n_nodes; ++k) {
+                    const auto write = graph->nodes[k];
+                    if (write->op == GGML_OP_CPY && write->view_src == root && write->src[0] &&
+                        write->src[0]->op == GGML_OP_VIEW && write->src[0]->src[0] == node &&
+                        (decoder->get_op_case(k) == 10 || decoder->get_op_case(k) == 12)) {
+                        if (group.writeback >= 0) {
+                            group.writeback = -1;
+                            break;
+                        }
+                        group.writeback = k;
+                    }
+                }
+                break;
+            }
+            if (group.writeback < 0 || !tensors->count("rs_update_" + decoder->get_op_name(group.writeback))) {
+                continue;
+            }
+        }
+        if (group.compute < 0) {
+            continue;
+        }
+        bool complete = tensors->count("rs_slot_begin_" + decoder->get_op_name(i)) &&
+                        tensors->count("rs_src_begin_" + decoder->get_op_name(i));
+        if (group.concat >= 0) {
+            complete &= tensors->count("rs_windows_" + decoder->get_op_name(group.concat));
+            const auto & name = decoder->get_op_name(group.writeback);
+            complete &= tensors->count("rs_slot_begin_" + name) && tensors->count("rs_src_begin_" + name);
+        }
+        for (int j = 0; j < graph->n_nodes && complete; ++j) {
+            const auto node = graph->nodes[j];
+            // Other cache readers or writers require functional translation.
+            for (int source = 0; source < GGML_MAX_SRC; ++source) {
+                const auto input = node->src[source];
+                if (!input || (input != root && input->view_src != root)) {
+                    continue;
+                }
+                const bool alias = node->op == GGML_OP_VIEW || node->op == GGML_OP_RESHAPE;
+                const bool matched_read = j == group.gather || j == group.reset;
+                const bool destination = (node->op == GGML_OP_CPY && source == 1) ||
+                                         (GgmlOvDecoder::is_conv_state_snapshot_writeback(node) && source == 2);
+                if (!alias && !matched_read && !destination) {
+                    complete = false;
+                }
+            }
+            if (node->view_src == root &&
+                (node->op == GGML_OP_SCALE || node->op == GGML_OP_CPY || node->op == GGML_OP_SET_ROWS) &&
+                j != group.reset && j != group.remainder && j != group.writeback) {
+                complete = false;
+            }
+        }
+        // Every read of the gathered state or convolution input must belong to
+        // this recurrence. Shared readers retain the ordinary GGML translation.
+        const auto compute = graph->nodes[group.compute];
+        const auto active = group.concat >= 0 ? graph->nodes[group.concat]->src[0]->src[0] : compute->src[5]->src[0];
+        const auto reshaped = group.concat >= 0 ? graph->nodes[group.concat]->src[0] : compute->src[5];
+        for (int j = group.gather + 1; j < graph->n_nodes && complete; ++j) {
+            const auto node = graph->nodes[j];
+            for (int source = 0; source < GGML_MAX_SRC; ++source) {
+                const auto input = node->src[source];
+                if (!input) {
+                    continue;
+                }
+                if (input == gather && node != active && node != graph->nodes[group.remainder]->src[0]) {
+                    complete = false;
+                }
+                if (input == active && node != reshaped) {
+                    complete = false;
+                }
+                if (input == reshaped && node != (group.concat >= 0 ? graph->nodes[group.concat] : compute)) {
+                    complete = false;
+                }
+                if (group.concat >= 0 && input == graph->nodes[group.concat] && node != compute) {
+                    complete &= node == graph->nodes[group.writeback]->src[0];
+                }
+                if (input == compute && group.concat < 0 && node->op != GGML_OP_VIEW) {
+                    complete = false;
+                }
+                if (group.concat < 0 && input == graph->nodes[group.writeback]->src[0] &&
+                    node != graph->nodes[group.writeback]) {
+                    complete = false;
+                }
+            }
+        }
+        if (!complete) {
+            continue;
+        }
+        group.root_name = GgmlOvDecoder::get_tensor_name(graph, root);
+        const auto id = m_groups.size();
+        for (int n : {group.reset, group.gather, group.remainder}) {
+            m_nodes[n] = id;
+        }
+        if (group.concat >= 0) {
+            m_nodes[group.concat] = id;
+        }
+        m_nodes[group.compute] = id;
+        if (group.writeback >= 0) {
+            m_nodes[group.writeback] = id;
+        }
+        m_groups.push_back(std::move(group));
+    }
+}
+
+OutputVector RecurrentCacheFusion::translate(int index, const NodeContext & context) {
+    const auto it = m_nodes.find(index);
+    if (it == m_nodes.end()) {
+        return {};
+    }
+    auto & group = m_groups[it->second];
+    using namespace ov::op;
+    auto constant = [](std::initializer_list<size_t> shape, std::initializer_list<int64_t> values) {
+        return v0::Constant::create(element::i64, Shape(shape), std::vector<int64_t>(values));
+    };
+    auto metadata = [&](const std::string & name) -> Output<Node> {
+        return std::make_shared<v1::Reshape>(context.get_input(name), constant({4}, {1, 1, 1, 1}), false);
+    };
+    if (index == group.reset) {
+        group.old_cache = context.get_input(0);
+        auto root = group.old_cache.get_node_shared_ptr();
+        while (ov::is_type<v1::Reshape>(root)) {
+            root = root->input_value(0).get_node_shared_ptr();
+        }
+        group.enabled = ov::is_type<v0::Parameter>(root);
+        if (!group.enabled) {
+            return {};
+        }
+        return {group.old_cache};
+    }
+    if (!group.enabled) {
+        return {};
+    }
+    NodeContext gather_context(m_decoder, m_tensors, group.gather);
+    auto rows = process_view_input_new(gather_context, 1);
+    const auto remainder_name = m_decoder->get_op_name(group.remainder);
+    auto offsets = std::make_shared<v0::Concat>(OutputVector{context.get_input("rs_slot_begin_" + remainder_name),
+                                                             context.get_input("rs_src_begin_" + remainder_name)},
+                                                0);
+    OutputVector inputs{group.old_cache,
+                        rows,
+                        rows,
+                        metadata("cache_rs_reset_idx"),
+                        metadata("cache_rs_reset_len"),
+                        context.has_input("rs_offsets_" + remainder_name) ?
+                            context.get_input("rs_offsets_" + remainder_name) :
+                            std::make_shared<v1::Reshape>(offsets, constant({4}, {1, 1, 1, 2}), false)};
+    inputs[1] = context.has_input("rs_active_rows") ?
+                    context.get_input("rs_active_rows") :
+                    std::make_shared<v8::Slice>(rows, constant({1}, {0}), context.get_input("s_copy_active_slot_len"),
+                                                constant({1}, {1}), constant({1}, {3}));
+    if (index == group.gather) {
+        // Ordinary nodes keep intermediate translation shapes valid. They become
+        // unreachable once the complete recurrence is replaced by its fused op.
+        return op::translate_get_rows(context);
+    }
+    if (index == group.remainder) {
+        return {group.old_cache};
+    }
+    if (index == group.concat) {
+        inputs.push_back(process_view_input_new(context, 1));
+        // Runtime pairs [source column, destination slot] cover the current CPY or SET_ROWS store.
+        inputs.push_back(context.get_input("rs_windows_" + context.get_name()));
+        const auto history = context.get_input_shape(0)[3].get_length();
+        NodeContext compute_context(m_decoder, m_tensors, group.compute);
+        const auto channels = context.get_input_shape(0)[2].get_length();
+        inputs.push_back(std::make_shared<v1::Reshape>(compute_context.get_input(1),
+                                                     constant({4}, {1, 1, channels, history + 1}), false));
+        auto fused = std::make_shared<internal::RecurrentConvCacheUpdate>(inputs, history);
+        fused->set_friendly_name(compute_context.get_name() + "/RecurrentConvCacheUpdate");
+        group.conv = fused->output(0);
+        return op::translate_concat(context);
+    }
+    if (index == group.compute) {
+        if (group.concat >= 0) {
+            return {group.conv};
+        }
+        inputs.push_back(context.get_input("rs_update_" + m_decoder->get_op_name(group.writeback)));
+        return op::translate_recurrent_gated_delta_net(context, inputs);
+    }
+    return {group.old_cache};
+}
+
+bool RecurrentCacheFusion::is_inplace_output(const std::string & name) const {
+    return std::any_of(m_groups.begin(), m_groups.end(),
+                       [&](const Group & group) { return group.enabled && group.root_name == name; });
+}
+}  // namespace ov::frontend::ggml
+#endif
diff --git a/ggml/src/ggml-openvino/openvino/recurrent_cache_fusion.h b/ggml/src/ggml-openvino/openvino/recurrent_cache_fusion.h
new file mode 100644
index 000000000..1d9d494ac
--- /dev/null
+++ b/ggml/src/ggml-openvino/openvino/recurrent_cache_fusion.h
@@ -0,0 +1,26 @@
+#pragma once
+#include "../ggml-decoder.h"
+#include "node_context.h"
+
+namespace ov::frontend::ggml {
+// Match complete build_rs groups before replacing reset/reorder. Unmatched graphs retain
+// their ordinary translators, including CPU, static and released-runtime execution.
+class RecurrentCacheFusion {
+public:
+    RecurrentCacheFusion(const std::shared_ptr<GgmlDecoder> & decoder, const std::shared_ptr<TensorMap> & tensors);
+    OutputVector translate(int index, const NodeContext & context);
+    bool is_inplace_output(const std::string & name) const;
+private:
+    struct Group {
+        int reset = -1, gather = -1, remainder = -1, concat = -1, compute = -1, writeback = -1;
+        std::string root_name;
+        bool enabled = false;
+        Output<Node> old_cache, conv;
+    };
+
+    std::shared_ptr<GgmlDecoder> m_decoder;
+    std::shared_ptr<TensorMap> m_tensors;
+    std::vector<Group> m_groups;
+    std::map<int, size_t> m_nodes;
+};
+}  // namespace ov::frontend::ggml
diff --git a/ggml/src/ggml-openvino/openvino/translate_session.cpp b/ggml/src/ggml-openvino/openvino/translate_session.cpp
index ec3596de2..3da712f72 100644
--- a/ggml/src/ggml-openvino/openvino/translate_session.cpp
+++ b/ggml/src/ggml-openvino/openvino/translate_session.cpp
@@ -5,22 +5,21 @@
 #include "ggml-openvino/openvino/node_context.h"
 #include "ggml-openvino/openvino/utils.h"
 #include "input_model.h"
-#include "pass/fuse_argsort_topk.h"
-#include "pass/fuse_moe_router.h"
-#include "pass/align_eltwise_ranks.h"
-#include "pass/fuse_moe_compressed.h"
-#include "pass/fuse_to_conv.h"
-#include "pass/kv_state_seq_axis.h"
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+#include "recurrent_cache_fusion.h"
+#endif
 #include "pass/mark_decompression_convert_constant_folding.h"
 #include "pass/mark_dequantization_subgraph.h"
-#include "pass/squeeze_matmul.h"
+#include "pass_registry/pipeline.h"
 #include "rt_info/weightless_caching_attributes.hpp"

 #include <algorithm>
 #include <cstdint>
 #include <cstdlib>
+#include <limits>
 #include <map>
 #include <memory>
+#include <set>
 #include <openvino/core/node.hpp>
 #include <openvino/core/preprocess/pre_post_process.hpp>
 #include <openvino/core/shape.hpp>
@@ -44,6 +43,7 @@
 #include <openvino/op/reshape.hpp>
 #include <openvino/op/result.hpp>
 #include <openvino/op/select.hpp>
+#include <openvino/op/shape_of.hpp>
 #include <openvino/op/sin.hpp>
 #include <openvino/op/slice.hpp>
 #include <openvino/op/squeeze.hpp>
@@ -53,8 +53,9 @@
 #include <openvino/op/unsqueeze.hpp>
 #include <openvino/pass/constant_folding.hpp>
 #include <openvino/pass/make_stateful.hpp>
-#include <limits>
 #include <sstream>
+#include <unordered_map>
+#include <vector>

 namespace ov {
 namespace frontend {
@@ -74,7 +75,12 @@ std::shared_ptr<ov::op::v0::Parameter> create_parameter(const std::string & name

 std::shared_ptr<ov::Node> create_extra_input(const std::string & name, const ModelExtraInputInfo & input_info) {
     if (input_info.is_parameter) {
-        auto param_node = std::make_shared<ov::op::v0::Parameter>(input_info.type, input_info.shape);
+        ov::PartialShape shape(input_info.shape);
+        // One SET_ROWS emits min(tokens, snapshot capacity) windows on each batch.
+        if (name.compare(0, 11, "rs_windows_") == 0) {
+            shape[2] = ov::Dimension::dynamic();
+        }
+        auto param_node = std::make_shared<ov::op::v0::Parameter>(input_info.type, shape);
         param_node->set_friendly_name(name);
         param_node->output(0).get_tensor().set_names({name});
         return param_node;
@@ -291,6 +297,15 @@ std::shared_ptr<Model> TranslateSession::translate_graph(const frontend::InputMo
     }

     for (const auto & it : ggml_model_decoder->get_model_extra_inputs()) {
+        auto source = tensor_map->find(it.second.shape_source);
+        if (!it.second.shape_source.empty() && source != tensor_map->end()) {
+            auto dim = std::make_shared<v8::Gather>(std::make_shared<v3::ShapeOf>(source->second, it.second.type),
+                                                    v0::Constant::create(element::i64, {1}, {it.second.shape_axis}),
+                                                    v0::Constant::create(element::i64, {}, {0}));
+            dim->set_friendly_name(it.first);
+            (*tensor_map)[it.first] = dim;
+            continue;
+        }
         auto input_node = create_extra_input(it.first, it.second);
         if (it.second.is_parameter) {
             params.push_back(ov::as_type_ptr<ov::op::v0::Parameter>(input_node));
@@ -302,6 +317,25 @@ std::shared_ptr<Model> TranslateSession::translate_graph(const frontend::InputMo
         (*tensor_map)[it.first] = it.second;
     }

+    if (tensor_map->count("sampling_rows")) {
+        // Match ComputeParams::sampling_rows ordering without one Parameter per sampling branch.
+        std::set<std::string> names;
+        ggml_model_decoder->visit_subgraph([&](const std::shared_ptr<GgmlDecoder> & decoder, int node_idx) {
+            if (decoder->get_op_type(node_idx) == "GGML_OP_VIEW" && decoder->get_op_case(node_idx) == 3) {
+                names.insert("sampling_row_" + decoder->get_op_name(node_idx));
+            }
+        });
+        int64_t index = 0;
+        for (const auto & name : names) {
+            (*tensor_map)[name] = std::make_shared<v8::Gather>(tensor_map->at("sampling_rows"),
+                v0::Constant::create(element::i64, {1}, {index++}),
+                v0::Constant::create(element::i64, {}, {0}));
+        }
+    }
+
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+    RecurrentCacheFusion recurrent_fusion(ggml_model_decoder, tensor_map);
+#endif
     auto translate_node = [&](const std::shared_ptr<GgmlDecoder> & decoder, int node_idx) {
         auto operation_type = decoder->get_op_type(node_idx);
         if (operation_type == "GGML_OP_NONE") {
@@ -322,7 +356,13 @@ std::shared_ptr<Model> TranslateSession::translate_graph(const frontend::InputMo
         FRONT_END_OP_CONVERSION_CHECK(it != m_translator_map.end(), "Translation for operation type ", operation_type,
                                       " is not implemented.");
         NodeContext node_context(decoder, tensor_map, node_idx, this);
-        ov::OutputVector converted_outputs = it->second(node_context);
+        ov::OutputVector converted_outputs;
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+        converted_outputs = recurrent_fusion.translate(node_idx, node_context);
+#endif
+        if (converted_outputs.empty()) {
+            converted_outputs = it->second(node_context);
+        }

         FRONT_END_OP_CONVERSION_CHECK(node_output_names.size() == converted_outputs.size(), "Number of ",
                                       operation_type, " outputs greater than number of converted outputs, which are ",
@@ -382,6 +422,12 @@ std::shared_ptr<Model> TranslateSession::translate_graph(const frontend::InputMo
     ggml_model_decoder->visit_subgraph(node_visitor);

     for (const auto & name : ggml_model_decoder->get_model_output_names()) {
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+        if (recurrent_fusion.is_inplace_output(name)) {
+            // The fused kernel updates the bound cache input; no cache Result is needed.
+            continue;
+        }
+#endif
         FRONT_END_GENERAL_CHECK(tensor_map->find(name) != tensor_map->end(),
                                 "Output name not found in tensor map: ", name);
         auto result = std::make_shared<v0::Result>(tensor_map->at(name));
@@ -453,19 +499,52 @@ std::shared_ptr<Model> TranslateSession::translate_graph(const frontend::InputMo
     // Each constant must have a distinct bin_offset, otherwise GPU's weightless cache
     // import will map multiple constants to the same data.
     //
+    // A cached blob is imported with this model as ov::hint::model and its constants are matched by key, so a
+    // model must get the same keys in every process, however it is built. Weight nodes are shared between models
+    // (the NPU prefill/decode pair, and the whole process through the tensor extras), so a number from this
+    // model's counter would depend on which model converted the weight first. Key weights by name instead, above
+    // the counter's range.
+    //
     // Small constants (< 16 elements) are excluded since they may be introduced by
     // optimization patterns and the overhead is negligible.
     //
     // Note: use shape_size() rather than byte_size()/element_type().size() - GatherMatmul's default
     // bias is a Constant(element::dynamic, Shape{0}), whose element_type().size() is 0 and would
     // divide by zero.
+    std::unordered_map<const ov::Node *, size_t> weight_keys;
+    for (const auto & [name, weight] : ggml_model_decoder->get_model_weights()) {
+        uint64_t name_hash = 0xcbf29ce484222325ULL;
+        for (const unsigned char c : name) {
+            name_hash = (name_hash ^ c) * 0x100000001b3ULL;
+        }
+        // A quantized weight is a subgraph: number its constants (data, scales, zero points) in input order.
+        size_t index = 0;
+        std::vector<const ov::Node *> stack;
+        if (weight) {
+            stack.push_back(weight.get());
+        }
+        while (!stack.empty()) {
+            const ov::Node * node = stack.back();
+            stack.pop_back();
+            if (ov::is_type<ov::op::v0::Constant>(node)) {
+                const uint64_t key = (name_hash + index++ * 0x9e3779b97f4a7c15ULL) & ((1ULL << 61) - 1);
+                weight_keys.emplace(node, static_cast<size_t>((1ULL << 61) | key));
+                continue;
+            }
+            for (size_t i = node->get_input_size(); i-- > 0;) {
+                stack.push_back(node->get_input_node_ptr(i));
+            }
+        }
+    }
     size_t offset = 0;
     for (auto & node : resulting_model->get_ordered_ops()) {
         if (auto cnst = ov::as_type_ptr<ov::op::v0::Constant>(node); cnst && ov::shape_size(cnst->get_shape()) >= 16) {
             auto & rt_info = cnst->get_rt_info();
             if (rt_info.find(ov::WeightlessCacheAttribute::get_type_info_static()) == rt_info.end()) {
-                rt_info[ov::WeightlessCacheAttribute::get_type_info_static()] =
-                    ov::WeightlessCacheAttribute(cnst->get_byte_size(), offset++, cnst->get_element_type());
+                const auto weight_key = weight_keys.find(cnst.get());
+                rt_info[ov::WeightlessCacheAttribute::get_type_info_static()] = ov::WeightlessCacheAttribute(
+                    cnst->get_byte_size(), weight_key != weight_keys.end() ? weight_key->second : offset++,
+                    cnst->get_element_type());
             }
         }
     }
@@ -477,52 +556,14 @@ std::shared_ptr<Model> TranslateSession::apply_transformations(std::shared_ptr<M
     {
         ov::pass::Manager manager;
         manager.set_per_pass_validation(true);
-        manager.register_pass<ov::pass::MarkCompressedFloatConstants>();
-        // Marks the Convert/Subtract/Multiply nodes of our GatherMatmul dequantization chain
-        // (make_int4_weights/make_int8_weights, for_gather_matmul=true) with disable_constant_folding,
-        // so it survives ConstantFolding regardless of whether the target plugin's own
-        // is_decompression_multiply() recognizes GatherMatmul as a valid consumer.
-        manager.register_pass<ov::pass::MarkDequantization>(
-            std::vector<ov::element::Type>{ov::element::u8, ov::element::i8, ov::element::u4, ov::element::i4});
-        manager.register_pass<pass::FuseToConv>();
-        manager.register_pass<pass::FuseArgsortTopK>();
-
-        // MOECompressed has no CPU plugin implementation, so enable it by default only on GPU.
-        // GGML_OPENVINO_MOE_OP=0 keeps the unfused GatherMatmul path for fallback/debugging.
-        if (ggml_openvino_is_gpu() &&
-            ggml_openvino_getenv_int("GGML_OPENVINO_MOE_OP", 1) != 0) {
-            manager.register_pass<pass::FuseMoeRouter>();
-            manager.register_pass<pass::FuseMoeCompressed>();
-            // Same fusion for models whose gate and up projections share one fused expert
-            // weight (gemma-4). It only matches that shape and only when the experts carry an
-            // integer zero point, so it is a no-op on the separate-gate/up models above.
-            manager.register_pass<pass::FuseMoeCompressedFusedGateUp>();
-        }
-
-        // Workaround for an OpenVINO GPU-plugin defect: an eltwise op whose operands differ in
-        // rank is computed wrongly once the plugin fuses it as a post-op into `rms`. gemma-4
-        // dense under stateful execution adds a rank-3 residual to the rank-4 norm output and
-        // decodes garbage on GPU while CPU is correct. Equalising the ranks is a no-op for
-        // NUMPY broadcasting and makes the fused path correct.
-        // Remove once the plugin guards that fusion. Opt out with
-        // GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN=1.
-        if (ggml_openvino_is_gpu() && !ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_ELTWISE_RANK_ALIGN")) {
-            manager.register_pass<pass::AlignEltwiseOperandRanks>();
-        }
-
-        if (ggml_model_decoder->is_stateful()) {
-            const auto kv_param_res_names = ggml_model_decoder->get_kv_param_res_names();
-            const auto kv_param_res_pairs = get_kv_param_res_pairs(model, kv_param_res_names);
-            manager.register_pass<ov::pass::MakeStateful>(kv_param_res_pairs);
-            // Must run after MakeStateful, which is what creates the ReadValue/Assign pairs.
-            if (!ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_KV_STATE_RELAYOUT")) {
-                manager.register_pass<pass::KVStateSeqAxis>();
-            }
-        }
-
-        if (ggml_model_decoder->is_static()) {
-            manager.register_pass<pass::SqueezeMatmul>();
+        const auto traits = pipeline::analyze_graph(*ggml_model_decoder);
+        ov::pass::MakeStateful::ParamResPairs kv_param_res_pairs;
+        if (traits.is_stateful) {
+            kv_param_res_pairs = get_kv_param_res_pairs(model, ggml_model_decoder->get_kv_param_res_names());
         }
+        pipeline::register_common_passes(manager, traits);
+        pipeline::register_device_passes(manager, traits);
+        pipeline::register_execution_passes(manager, traits, kv_param_res_pairs);
         manager.run_passes(model);
         if (ggml_model_decoder->is_stateful()) {
             ov::preprocess::PrePostProcessor ppp(model);
diff --git a/ggml/src/ggml-openvino/openvino/utils.cpp b/ggml/src/ggml-openvino/openvino/utils.cpp
index e47f97695..ec7513951 100644
--- a/ggml/src/ggml-openvino/openvino/utils.cpp
+++ b/ggml/src/ggml-openvino/openvino/utils.cpp
@@ -1,6 +1,7 @@
 #include "utils.h"

 #include "ggml-impl.h"
+#include "ggml-openvino/ggml-openvino-extra.h"

 #include <cmath>
 #include <cstddef>
@@ -8,13 +9,20 @@
 #include <memory>
 #include <openvino/op/add.hpp>
 #include <openvino/op/clamp.hpp>
+#include <openvino/op/constant.hpp>
+#include <openvino/op/concat.hpp>
 #include <openvino/op/convert.hpp>
 #include <openvino/op/cos.hpp>
 #include <openvino/op/divide.hpp>
 #include <openvino/op/gather.hpp>
+#include <openvino/op/greater_eq.hpp>
+#include <openvino/op/less.hpp>
+#include <openvino/op/logical_or.hpp>
 #include <openvino/op/maximum.hpp>
 #include <openvino/op/multiply.hpp>
+#include <openvino/op/range.hpp>
 #include <openvino/op/reshape.hpp>
+#include <openvino/op/select.hpp>
 #include <openvino/op/shape_of.hpp>
 #include <openvino/op/sin.hpp>
 #include <openvino/op/slice.hpp>
@@ -22,6 +30,7 @@
 #include <openvino/op/squeeze.hpp>
 #include <openvino/op/subtract.hpp>
 #include <openvino/op/transpose.hpp>
+#include <openvino/op/unsqueeze.hpp>
 #include <string>

 namespace ov {
@@ -343,6 +352,109 @@ ov::Output<ov::Node> process_view_input(const NodeContext & context, int input_i
     return sliced;
 }

+namespace {
+
+// Walk a dequantization chain, gathering `indices` out of every leaf Constant that is indexed by
+// the gathered axis, and rebuild the chain on top of the gathered rows. Only the node types
+// make_intX_weights emits are recognized; anything else aborts the rewrite (nullptr).
+std::shared_ptr<ov::Node> rebuild_on_gathered_rows(const ov::Output<ov::Node> & out,
+                                                   const ov::Output<ov::Node> & indices,
+                                                   int64_t rows,
+                                                   size_t idx_rank,
+                                                   bool & saw_compressed,
+                                                   size_t depth) {
+    static constexpr size_t max_depth = 8;
+    if (depth > max_depth) {
+        return nullptr;
+    }
+    auto node = out.get_node_shared_ptr();
+
+    if (auto cnst = ov::as_type_ptr<ov::op::v0::Constant>(node)) {
+        const auto & shape = cnst->get_shape();
+        const auto type = cnst->get_element_type();
+        if (type == ov::element::u4 || type == ov::element::i4 || type == ov::element::u8 ||
+            type == ov::element::i8) {
+            saw_compressed = true;
+        }
+        if (shape.empty() || static_cast<int64_t>(shape[0]) != rows) {
+            // Scalar or already-broadcast operand (e.g. a single zero point): share it unchanged.
+            return cnst;
+        }
+        auto axis = ov::op::v0::Constant::create(ov::element::i32, ov::Shape{}, {0});
+        return std::make_shared<ov::op::v8::Gather>(cnst, indices, axis);
+    }
+
+    if (auto reshape = ov::as_type_ptr<ov::op::v1::Reshape>(node)) {
+        // The only Reshape in the chain collapses the group axis back into the row, [N,K/G,G] ->
+        // [N,K]. On gathered rows the leading N becomes the index dims, so keep those and collapse
+        // whatever remains.
+        const auto & ps = reshape->get_output_partial_shape(0);
+        if (ps.rank().is_dynamic() || ps.rank().get_length() != 2) {
+            return nullptr;
+        }
+        auto input = rebuild_on_gathered_rows(reshape->input_value(0), indices, rows, idx_rank,
+                                              saw_compressed, depth + 1);
+        if (!input) {
+            return nullptr;
+        }
+        std::vector<int64_t> target(idx_rank, 0);
+        target.push_back(-1);
+        auto target_shape = ov::op::v0::Constant::create(ov::element::i64, {target.size()}, target);
+        return std::make_shared<ov::op::v1::Reshape>(input, target_shape, true);
+    }
+
+    if (ov::is_type<ov::op::v0::Convert>(node) || ov::is_type<ov::op::v1::Multiply>(node) ||
+        ov::is_type<ov::op::v1::Subtract>(node)) {
+        ov::OutputVector args;
+        args.reserve(node->get_input_size());
+        for (size_t i = 0; i < node->get_input_size(); i++) {
+            auto arg = rebuild_on_gathered_rows(node->input_value(i), indices, rows, idx_rank,
+                                                saw_compressed, depth + 1);
+            if (!arg) {
+                return nullptr;
+            }
+            args.push_back(arg);
+        }
+        return node->clone_with_new_inputs(args);
+    }
+
+    return nullptr;
+}
+
+}  // namespace
+
+std::shared_ptr<ov::Node> gather_compressed_rows(const ov::Output<ov::Node> & weights,
+                                                 const ov::Output<ov::Node> & indices) {
+    // Below this the dequantized table is small enough that the extra nodes are not worth it.
+    static constexpr int64_t min_elements = 1 << 20;
+
+    const auto & ps = weights.get_partial_shape();
+    const auto & idx_ps = indices.get_partial_shape();
+    if (ps.rank().is_dynamic() || ps.rank().get_length() != 2 || ps[0].is_dynamic() ||
+        ps[1].is_dynamic() || idx_ps.rank().is_dynamic()) {
+        return nullptr;
+    }
+    if (ps[0].get_length() * ps[1].get_length() < min_elements) {
+        return nullptr;
+    }
+
+    bool saw_compressed = false;
+    auto gathered = rebuild_on_gathered_rows(weights, indices, ps[0].get_length(),
+                                             idx_ps.rank().get_length(), saw_compressed, 0);
+    return saw_compressed ? gathered : nullptr;
+}
+
+ov::Output<ov::Node> get_kq_mask_input(const NodeContext & context, size_t input_index) {
+    const auto & name = context.get_input_names()[input_index];
+    const std::string sliced = name == "self_kq_mask_swa"                  ? "KQ_mask_swa_sliced" :
+                               name == "self_kq_mask" || name == "KQ_mask" ? "KQ_mask_sliced" :
+                                                                             std::string();
+    if (!sliced.empty() && context.has_input(sliced)) {
+        return context.get_input(sliced);
+    }
+    return context.get_input(input_index);
+}
+
 ov::Output<ov::Node> process_view_input_new(const NodeContext & context, int input_index) {
     auto input = context.get_input(input_index);

@@ -353,6 +465,99 @@ ov::Output<ov::Node> process_view_input_new(const NodeContext & context, int inp
         return input;
     }

+    // Integer router views can share these strides when their row count is zero.
+    // Packed projections can store query and gate in separate halves of each head.
+    if (!context.is_static() && view_input_size == 1 && input.get_element_type().is_real()) {
+        const auto source = context.get_view_input_src_ggml_shape(input_index, 0);
+        const auto view = context.get_view_input_ggml_shape(input_index, 0);
+        const auto source_stride = context.get_view_input_src_stride(input_index, 0);
+        const auto view_stride = context.get_view_input_stride(input_index, 0);
+        if (source.size() == 4 && view.size() == 4 && source_stride.size() == 4 && view_stride.size() == 4 &&
+            source[0] == 1 && source[1] == 1 && view[0] == 1 && view[1] == source[2] &&
+            view[2] > 0 && view[3] > 0 && source_stride[3] > 0 &&
+            view_stride[1] == source_stride[2] && view_stride[3] == source_stride[3] &&
+            view_stride[2] % source_stride[3] == 0) {
+            const size_t head_width = view_stride[2] / source_stride[3];
+            const auto offset = context.get_view_input_offset(input_index, 0);
+            const auto source_offset = context.get_view_input_src_offset(input_index, 0);
+            if (head_width > view[3] && source[3] % head_width == 0 && source[3] / head_width == view[2] &&
+                offset >= source_offset && (offset - source_offset) % source_stride[3] == 0) {
+                const size_t begin = (offset - source_offset) / source_stride[3];
+                if (begin <= head_width - view[3]) {
+                    auto heads = std::make_shared<ov::op::v1::Reshape>(input,
+                        ov::op::v0::Constant::create(ov::element::i64, {4},
+                            std::vector<int64_t>{1, -1, int64_t(view[2]), int64_t(head_width)}), false);
+                    auto part = std::make_shared<ov::op::v8::Slice>(heads,
+                        ov::op::v0::Constant::create(ov::element::i64, {1}, {begin}),
+                        ov::op::v0::Constant::create(ov::element::i64, {1}, {begin + view[3]}),
+                        ov::op::v0::Constant::create(ov::element::i64, {1}, {1}),
+                        ov::op::v0::Constant::create(ov::element::i64, {1}, {3}));
+                    return part;
+                }
+            }
+        }
+    }
+
+    if (!context.is_static() && view_input_size == 1 && input.get_element_type().is_real() &&
+        context.has_input("s_copy_active_slot_len")) {
+        const auto source = context.get_view_input_src_ggml_shape(input_index, 0);
+        const auto view = context.get_view_input_ggml_shape(input_index, 0);
+        const auto source_stride = context.get_view_input_src_stride(input_index, 0);
+        const auto view_stride = context.get_view_input_stride(input_index, 0);
+        if (source.size() == 4 && view.size() == 4 && source_stride.size() == 4 && view_stride.size() == 4 &&
+            source[0] == 1 && view[0] == source[1] && view[1] == source[2] && view[3] > 0 &&
+            view[2] <= source[3] / view[3] &&
+            view_stride[0] == source_stride[1] && view_stride[1] == source_stride[2] &&
+            view_stride[3] == source_stride[3]) {
+            const auto offset = context.get_view_input_offset(input_index, 0);
+            const auto source_offset = context.get_view_input_src_offset(input_index, 0);
+            if (offset >= source_offset && source_stride[3] > 0 &&
+                (offset - source_offset) % source_stride[3] == 0) {
+                const int64_t begin = (offset - source_offset) / source_stride[3];
+                const int64_t width = view[2] * view[3];
+                if (begin <= int64_t(source[3]) - width) {
+                    auto sliced = std::make_shared<ov::op::v8::Slice>(input,
+                        ov::op::v0::Constant::create(ov::element::i64, {1}, {begin}),
+                        ov::op::v0::Constant::create(ov::element::i64, {1}, {begin + width}),
+                        ov::op::v0::Constant::create(ov::element::i64, {1}, {1}),
+                        ov::op::v0::Constant::create(ov::element::i64, {1}, {3}));
+                    auto prefix = std::make_shared<ov::op::v8::Gather>(
+                        std::make_shared<ov::op::v3::ShapeOf>(input, ov::element::i64),
+                        ov::op::v0::Constant::create(ov::element::i64, {2}, {1, 2}),
+                        ov::op::v0::Constant::create(ov::element::i64, {}, {0}));
+                    auto shape = std::make_shared<ov::op::v0::Concat>(ov::OutputVector{prefix,
+                        ov::op::v0::Constant::create(ov::element::i64, {2},
+                            std::vector<int64_t>{int64_t(view[2]), int64_t(view[3])})}, 0);
+                    return std::make_shared<ov::op::v1::Reshape>(sliced, shape, false);
+                }
+            }
+        }
+    }
+
+    // A row range of a 2D weight (BERT's token type row): slice the rows of the [rows, row_size] Constant
+    if (view_input_size == 1 && input.get_partial_shape().rank().is_static() &&
+        input.get_partial_shape().rank().get_length() == 2 &&
+        ov::is_type<ov::op::v0::Constant>(input.get_node_shared_ptr())) {
+        const auto view = context.get_view_input_ggml_shape(input_index, 0);
+        const auto source = context.get_view_input_src_ggml_shape(input_index, 0);
+        const auto view_stride = context.get_view_input_stride(input_index, 0);
+        const auto source_stride = context.get_view_input_src_stride(input_index, 0);
+        const size_t offset = context.get_view_input_offset(input_index, 0);
+        const size_t source_offset = context.get_view_input_src_offset(input_index, 0);
+        if (view.size() == 4 && source.size() == 4 && view[0] == 1 && view[1] == 1 && view[3] == source[3] &&
+            view_stride.size() == 4 && source_stride.size() == 4 && view_stride[2] == source_stride[2] &&
+            source_stride[2] > 0 && offset >= source_offset && (offset - source_offset) % source_stride[2] == 0) {
+            const int64_t begin = (offset - source_offset) / source_stride[2];
+            if (begin + int64_t(view[2]) <= int64_t(source[2])) {
+                return std::make_shared<ov::op::v8::Slice>(
+                    input, ov::op::v0::Constant::create(ov::element::i64, {1}, {begin}),
+                    ov::op::v0::Constant::create(ov::element::i64, {1}, {begin + int64_t(view[2])}),
+                    ov::op::v0::Constant::create(ov::element::i64, {1}, {1}),
+                    ov::op::v0::Constant::create(ov::element::i64, {1}, {0}));
+            }
+        }
+    }
+
     // If translate_view already resolved this VIEW (produced a Slice), the input
     // will already have the expected shape — skip re-slicing.
     //
@@ -416,7 +621,10 @@ ov::Output<ov::Node> process_view_input_new(const NodeContext & context, int inp
     // This ensures NPUW's FOLD doesn't parametrize per-layer slice indices (which
     // would introduce dynamic shapes). A shared Split node sits outside the repeated
     // subgraph boundary; each layer receives one of its output ports.
-    if (context.is_static() && view_input_size == 1) {
+    // The dynamic GPU path uses it too: a StridedSlice costs a full shape inference per run on GPU, a Split output does
+    // not (gemma-4 slices its per-layer embeddings this way, once per layer)
+    const bool dynamic_split = !context.is_static() && ggml_openvino_is_gpu();
+    if ((context.is_static() || dynamic_split) && view_input_size == 1) {
         auto view_stride_v = context.get_view_input_stride(input_index, 0);
         auto view_src_stride_v = context.get_view_input_src_stride(input_index, 0);
         auto view_ggml_shape = context.get_view_input_ggml_shape(input_index, 0);
@@ -456,7 +664,41 @@ ov::Output<ov::Node> process_view_input_new(const NodeContext & context, int inp
                     size_t relative_offset = view_offset >= view_src_offset ? view_offset - view_src_offset : 0;
                     int64_t split_index = static_cast<int64_t>(relative_offset / view_src_stride_v[split_dim]);

-                    if (split_index >= 0 && split_index < num_splits) {
+                    // the view must be exactly one Split output: offset on a chunk boundary, the strides of all
+                    // other dims with more than one element equal
+                    bool exact_chunk = relative_offset % view_src_stride_v[split_dim] == 0;
+                    for (size_t i = 0; i < ndims && exact_chunk; ++i) {
+                        exact_chunk = i == static_cast<size_t>(split_dim) || view_ggml_shape[i] == 1 ||
+                                      view_stride_v[i] == view_src_stride_v[i];
+                    }
+                    // stateful execution drops the leading size-1 axis, so the input can be one rank lower than the
+                    // ggml metadata
+                    const auto input_rank = input.get_partial_shape().rank();
+                    const int64_t rank_shift = input_rank.is_static() ?
+                                                   static_cast<int64_t>(ndims) - input_rank.get_length() :
+                                                   static_cast<int64_t>(ndims);
+                    if (split_index >= 0 && split_index < num_splits && dynamic_split && exact_chunk &&
+                        split_dim >= rank_shift) {
+                        // the Split keeps its input alive, so only keep a weak reference to it on the input
+                        auto src_node = input.get_node_shared_ptr();
+                        std::string rt_key =
+                            "split_dim_weak_" + std::to_string(split_dim) + "_" + std::to_string(input.get_index());
+                        auto & rt_info = src_node->get_rt_info();
+                        std::shared_ptr<ov::Node> split_node;
+                        if (rt_info.count(rt_key) != 0) {
+                            split_node = rt_info[rt_key].as<std::weak_ptr<ov::Node>>().lock();
+                        }
+                        if (split_node == nullptr) {
+                            auto axis_const = ov::op::v0::Constant::create(
+                                ov::element::i64, {}, {static_cast<int64_t>(split_dim) - rank_shift});
+                            split_node =
+                                std::make_shared<ov::op::v1::Split>(input, axis_const, static_cast<size_t>(num_splits));
+                            split_node->set_friendly_name(src_node->get_friendly_name() + "_split");
+                            rt_info[rt_key] = std::weak_ptr<ov::Node>(split_node);
+                        }
+                        return split_node->output(static_cast<size_t>(split_index));
+                    }
+                    if (split_index >= 0 && split_index < num_splits && !dynamic_split) {
                         auto src_node = input.get_node_shared_ptr();
                         std::string rt_key = "split_dim_" + std::to_string(split_dim);
                         auto & rt_info = src_node->get_rt_info();
@@ -744,6 +986,45 @@ ov::Output<ov::Node> process_view_input_new(const NodeContext & context, int inp
                 }
             }

+            // A view that reads the first cells of a single-sequence cache [1, 1, cells, row] with each cell one whole
+            // row, e.g. MLA's K view [1, n_kv, 1 (n_head_kv), 576]. With the cache input sliced to the cells in use,
+            // this is a reshape that keeps the cell axis dynamic.
+            if (same_rank && relative_offset == 0 && view_ggml_shape.size() >= 2 && view_ov_shape.rank().is_static() &&
+                view_ov_shape.rank().get_length() == static_cast<int64_t>(view_ggml_shape.size()) &&
+                current.get_partial_shape().rank().is_static() &&
+                current.get_partial_shape().rank().get_length() == static_cast<int64_t>(view_ggml_shape.size())) {
+                const size_t ndims = view_ggml_shape.size();
+                const size_t row_stride = view_src_stride[ndims - 2];
+                bool src_rows = view_src_stride[ndims - 1] == view_stride[ndims - 1];
+                for (size_t i = 0; i + 2 < ndims && src_rows; ++i) {
+                    src_rows = view_src_ggml_shape[i] == 1;
+                }
+                int cell_axis = -1;
+                for (size_t i = 0; i < ndims && src_rows; ++i) {
+                    if (view_stride[i] == row_stride && view_ov_shape[i].is_dynamic()) {
+                        cell_axis = static_cast<int>(i);
+                    }
+                }
+                bool one_row_per_cell = cell_axis >= 0;
+                size_t inner_bytes = view_stride[ndims - 1];
+                for (size_t i = 0; i < ndims && one_row_per_cell; ++i) {
+                    if (static_cast<int>(i) < cell_axis) {
+                        one_row_per_cell = view_ggml_shape[i] == 1;
+                    } else if (static_cast<int>(i) > cell_axis) {
+                        inner_bytes *= view_ggml_shape[i];
+                    }
+                }
+                if (one_row_per_cell && inner_bytes == row_stride) {
+                    auto reshape_pattern = build_reshape_pattern(view_ov_shape, view_ggml_shape);
+                    auto reshaped = std::make_shared<ov::op::v1::Reshape>(
+                        current,
+                        ov::op::v0::Constant::create(ov::element::i64, {reshape_pattern.size()}, reshape_pattern),
+                        false);
+                    reshaped->set_friendly_name(view_name);
+                    return reshaped;
+                }
+            }
+
             size_t view_elems = 1;
             size_t src_elems = 1;
             if (same_rank) {
@@ -967,6 +1248,37 @@ ov::Output<ov::Node> process_view_input_new(const NodeContext & context, int inp
     return current;
 }

+bool is_recurrent_cache_reset_case(const NodeContext & context) {
+    return context.get_op_case() == 1 && context.has_input("cache_rs_reset_len");
+}
+
+OutputVector translate_recurrent_cache_reset(const NodeContext & context) {
+    auto cache_rs = context.get_input(0);
+    auto cache_rs_reset_idx = context.get_input("cache_rs_reset_idx");
+    auto cache_rs_reset_len = context.get_input("cache_rs_reset_len");
+
+    auto cache_shape = std::make_shared<ov::op::v3::ShapeOf>(cache_rs, ov::element::i64);
+    auto n_slots_1d = std::make_shared<ov::op::v8::Gather>(
+        cache_shape, ov::op::v0::Constant::create(ov::element::i64, ov::Shape{1}, {2}),
+        ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {0}));
+    auto n_slots = std::make_shared<ov::op::v0::Squeeze>(n_slots_1d);
+
+    auto iota = std::make_shared<ov::op::v4::Range>(
+        ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {0}), n_slots,
+        ov::op::v0::Constant::create(ov::element::i64, ov::Shape{}, {1}), ov::element::i64);
+
+    auto idx_plus_len = std::make_shared<ov::op::v1::Add>(cache_rs_reset_idx, cache_rs_reset_len);
+    auto less_than_idx = std::make_shared<ov::op::v1::Less>(iota, cache_rs_reset_idx);
+    auto greater_equal_idx_plus_len = std::make_shared<ov::op::v1::GreaterEqual>(iota, idx_plus_len);
+    auto keep_mask = std::make_shared<ov::op::v1::LogicalOr>(less_than_idx, greater_equal_idx_plus_len);
+    auto keep_mask_reshape = std::make_shared<ov::op::v0::Unsqueeze>(
+        keep_mask, ov::op::v0::Constant::create(ov::element::i64, ov::Shape{1}, {1}));
+
+    auto zero = ov::op::v0::Constant::create(cache_rs.get_element_type(), ov::Shape{}, {0});
+    auto cleared_cache_rs = std::make_shared<ov::op::v1::Select>(keep_mask_reshape, cache_rs, zero);
+    return rename_outputs_with_suffix({cleared_cache_rs}, context.get_name());
+}
+
 }  // namespace ggml
 }  // namespace frontend
 }  // namespace ov
diff --git a/ggml/src/ggml-openvino/openvino/utils.h b/ggml/src/ggml-openvino/openvino/utils.h
index effa1f843..10344675b 100644
--- a/ggml/src/ggml-openvino/openvino/utils.h
+++ b/ggml/src/ggml-openvino/openvino/utils.h
@@ -62,10 +62,41 @@ std::pair<ov::Output<Node>, ov::Output<Node>> make_sin_cos(int32_t * rope_params
                                                            bool stateful = false,
                                                            int64_t head_dim = 0);

+// If `weights` is a dequantization chain emitted by make_int4_weights/make_int8_weights, rebuild
+// it with `indices` gathered out of the leaf Constants, so a row lookup reads the packed rows and
+// dequantizes only those. Returns nullptr if the subgraph is not such a chain, or is too small to
+// be worth the rewrite. Used for tied embeddings, where the same weight also feeds the lm_head
+// matmul: gathering the dequantized table would give that chain a second consumer and force it to
+// be materialized in full.
+std::shared_ptr<ov::Node> gather_compressed_rows(const ov::Output<ov::Node> & weights,
+                                                 const ov::Output<ov::Node> & indices);
+
 ov::Output<ov::Node> process_view_input(const NodeContext & context, int input_index, int slice_len = 0, int axis = -1);

 ov::Output<ov::Node> process_view_input_new(const NodeContext & context, int input_index);

+bool is_recurrent_cache_reset_case(const NodeContext & context);
+
+OutputVector translate_recurrent_cache_reset(const NodeContext & context);
+
+// A KQ mask input. Stateful execution reads the mask parameters through the masks rebuilt from the positions (see
+// add_sliced_mask_stateful); any other input, e.g. the mask rows gemma-4 gathers for its KV-shared layers, as is.
+ov::Output<ov::Node> get_kq_mask_input(const NodeContext & context, size_t input_index);
+
+// A stateless KV cache in the [1, ctx, n_heads_kv, head_size] layout (GgmlOvDecoder::is_kvcache_4d), as opposed to
+// [1, n_seq, ctx, n_heads_kv * head_size]
+inline bool is_kvcache_4d_shape(const ov::PartialShape & shape) {
+    return shape.rank().is_static() && shape.rank().get_length() == 4 && shape[1].is_dynamic() &&
+           shape[2].is_static() && shape[3].is_static();
+}
+
+// rt_info key on the Reshape that restores the heads after a grouped-query MUL_MAT (mulmat): the number of A rows
+// stacked per B matrix
+constexpr const char * GGML_OV_GROUPED_ROWS = "ggml_ov_grouped_rows";
+
+// rt_info key on the stateful V cache read with `-fa off` (permute): V is [.., kv, head_size], not transposed
+constexpr const char * GGML_OV_V_TOKEN_MAJOR = "ggml_ov_v_token_major";
+
 namespace op {
 template <typename T> OutputVector translate_1to1_match_2_inputs(const NodeContext & context) {
     num_inputs_check(context, 2, 2);
diff --git a/ggml/src/ggml-openvino/utils.cpp b/ggml/src/ggml-openvino/utils.cpp
index f07e180a4..7c5f6b756 100644
--- a/ggml/src/ggml-openvino/utils.cpp
+++ b/ggml/src/ggml-openvino/utils.cpp
@@ -1,5 +1,7 @@
 #include "utils.h"

+#include "device/npu_config.h"
+#include "device/npu_quant_policy.h"
 #include "ggml-impl.h"
 #include "ggml-openvino-extra.h"
 #include "ggml-openvino.h"
@@ -16,8 +18,10 @@
 #include <cstdint>
 #include <cstdlib>
 #include <cstring>
+#include <filesystem>
 #include <fstream>
 #include <functional>
+#include <future>
 #include <iomanip>
 #include <iostream>
 #include <memory>
@@ -42,29 +46,55 @@
 namespace {
 // For a KV cache input, return an ov::Tensor sized to n_kv (== attention_size
 // for that layer) instead of the fully-allocated ctx_per_seq. Pre-conditions:
-//   * non-static (CPU/GPU) backend, single sequence, seq_active_start == 0
+//   * dynamic CPU/GPU or opt-in static NPU prefill, single sequence, seq_active_start == 0
 //   * ggml KV layout is a contiguous [1, 1, ctx_per_seq, n_heads_kv*head_size]
-//     so the first n_kv rows are the live prefix and shrinking the ctx axis
-//     gives a valid tensor over the same host storage
+//     with one token per row (not the transposed V cache of `-fa off`), so the
+//     first n_kv rows are the live prefix and shrinking the ctx axis gives a
+//     valid tensor over the same host storage
 //   * not an SWA layer (ring cache): once the window has wrapped the first
 //     n_kv rows no longer contain the live prefix
 // On any unmet pre-condition returns std::nullopt; the caller falls back to
 // the full-size tensor.
+// The KV address the NPU binds: on-demand L0 storage (GGML_OPENVINO_NPU_KV_ON_DEMAND) or the ggml memory itself.
+static void * kv_bind_ptr(void * data, const ov::element::Type & type, const ov::Shape & shape) {
+    void * storage = ggml_openvino_kv_storage(data, ov::shape_size(shape) * type.size());
+    return storage ? storage : data;
+}
+
+// Largest on-demand KV storage capacity (rows) among the graph's KV cache tensors.
+static int kv_storage_rows(const ggml_cgraph * cgraph) {
+    size_t rows = 0;
+    for (int i = 0; i < cgraph->n_nodes; i++) {
+        for (const auto * src : cgraph->nodes[i]->src) {
+            const auto * base = src && src->view_src ? src->view_src : src;
+            if (base && (strncmp(base->name, "cache_k_l", 9) == 0 || strncmp(base->name, "cache_v_l", 9) == 0)) {
+                rows = std::max(rows, ggml_openvino_kv_storage_capacity(base->data));
+            }
+        }
+    }
+    return static_cast<int>(rows);
+}
+
 std::optional<ov::Tensor> try_make_kv_sliced_tensor(const std::shared_ptr<GgmlOvDecoder> & ggml_decoder,
                                                     const std::string & name,
                                                     const ggml_tensor * ggml_tensor) {
-    static const bool kv_slice_disabled = ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_KV_SLICE");
-    if (kv_slice_disabled) {
-        return std::nullopt;
-    }
-    if (ggml_decoder->is_static() || ggml_decoder->is_stateful()) {
+    // Static graphs bake the sliced length into the prefill graph; decode is sliced to its KV window
+    // with GGML_OPENVINO_NPU_BLOB_PREBUILD and otherwise keeps its full-context graph.
+    const bool slice = !ggml_decoder->is_static() ? ggml_openvino_kv_slice_enabled() :
+                       ggml_decoder->m_is_prefill ? ggml_openvino_kv_slice_enabled() :
+                                                    ggml_openvino_npu_decode_bucket() > 0;
+    if (!slice || ggml_decoder->is_stateful()) {
         return std::nullopt;
     }
-    if (ggml_tensor->op != GGML_OP_NONE || ggml_tensor->view_src != nullptr) {
+    if (ggml_tensor->op != GGML_OP_NONE || ggml_tensor->view_src != nullptr || !ggml_is_contiguous(ggml_tensor)) {
         return std::nullopt;
     }
-    const auto * op = ggml_decoder->get_tensor_used_op(ggml_tensor);
-    if (!GgmlOvDecoder::is_kvcache(ggml_tensor, op)) {
+    // USAGE_ANY alone is not proof of an attention cache. Only the ordinary
+    // llama.cpp K/V roots have this prefix layout; recurrent caches (including
+    // rollback snapshots) must always bind their complete stored row bank.
+    if (GgmlOvDecoder::is_recurrent_cache(ggml_tensor) ||
+        (strncmp(ggml_tensor->name, "cache_k_l", strlen("cache_k_l")) != 0 &&
+         strncmp(ggml_tensor->name, "cache_v_l", strlen("cache_v_l")) != 0)) {
         return std::nullopt;
     }

@@ -73,19 +103,27 @@ std::optional<ov::Tensor> try_make_kv_sliced_tensor(const std::shared_ptr<GgmlOv
         return std::nullopt;
     }

-    int layer;
-    if (auto layer_opt = extract_layer_from_name(name); layer_opt.has_value()) {
-        layer = layer_opt.value();
-    } else {
-        return std::nullopt;
-    }
-
-    const bool is_swa = ggml_decoder->is_swa_layer(layer);
-    if (is_swa) {
-        return std::nullopt;
-    }
     const int ctx_per_seq = ggml_decoder->get_ctx_per_seq();
     const int n_kv = compute_params.attention_size;
+    if (ggml_decoder->is_static() && ggml_openvino_npu_kv_head_major_enabled() && ctx_per_seq > 0) {
+        // The static graphs take the whole head-major storage, whose capacity is the attention window.
+        const auto * op = ggml_decoder->get_tensor_used_op(ggml_tensor);
+        if (!GgmlOvDecoder::is_kvcache(ggml_tensor, op) || !ggml_decoder->is_kvcache_token_rows(ggml_tensor)) {
+            return std::nullopt;
+        }
+        const auto layer = extract_layer_from_name(name);
+        if (!layer.has_value() || ggml_decoder->is_swa_layer(*layer)) {
+            return std::nullopt;
+        }
+        const auto type = ggml_decoder->get_ov_type(ggml_tensor);
+        const size_t heads = ggml_decoder->get_n_heads_kv_for_tensor(ggml_tensor);
+        const size_t rows = n_kv > 0 ? std::min(n_kv, ctx_per_seq) : ctx_per_seq;
+        const size_t head_size = static_cast<size_t>(ggml_tensor->ne[0]) / heads;
+        const ov::Shape shape = name.rfind("cache_v_l", 0) == 0 ?
+                                    ov::Shape{1, heads, head_size, rows} :
+                                    ov::Shape{1, heads, rows, head_size};
+        return ov::Tensor(type, shape, kv_bind_ptr(ggml_tensor->data, type, shape));
+    }
     if (ctx_per_seq <= 0 || n_kv <= 0 || n_kv >= ctx_per_seq) {
         return std::nullopt;
     }
@@ -95,17 +133,79 @@ std::optional<ov::Tensor> try_make_kv_sliced_tensor(const std::shared_ptr<GgmlOv
         static_cast<int>(full_shape[2]) != ctx_per_seq) {
         return std::nullopt;
     }
+    // The consumer lookup scans the whole graph, so it runs after the cheap checks.
+    const auto * op = ggml_decoder->get_tensor_used_op(ggml_tensor);
+    if (!GgmlOvDecoder::is_kvcache(ggml_tensor, op) || !ggml_decoder->is_kvcache_token_rows(ggml_tensor)) {
+        return std::nullopt;
+    }
+
+    int layer;
+    if (auto layer_opt = extract_layer_from_name(name); layer_opt.has_value()) {
+        layer = layer_opt.value();
+    } else {
+        return std::nullopt;
+    }
+
+    const bool is_swa = ggml_decoder->is_swa_layer(layer);
+    if (is_swa) {
+        return std::nullopt;
+    }

     ov::Shape sliced_shape = full_shape;
     sliced_shape[2] = static_cast<size_t>(n_kv);

-    if (ggml_openvino_buffer_is_remote(ggml_tensor)) {
-        auto remote_context = ggml_openvino_get_remote_context();
-        auto gpu_context = remote_context->as<ov::intel_gpu::ocl::ClContext>();
-        return gpu_context.create_tensor(ggml_decoder->get_ov_type(ggml_tensor), sliced_shape, ggml_tensor->data);
+    const auto type = GgmlOvDecoder::get_ov_type(ggml_tensor);
+    // On-demand KV storage moves when it grows, so it bypasses the per-pointer tensor cache.
+    if (void * storage = kv_bind_ptr(ggml_tensor->data, type, sliced_shape); storage != ggml_tensor->data) {
+        return ov::Tensor(type, sliced_shape, storage);
     }
+    return ggml_decoder->cached_kv_tensor(ggml_tensor, type, sliced_shape, [&]() -> ov::Tensor {
+        if (ggml_openvino_buffer_is_remote(ggml_tensor)) {
+            auto remote_context = ggml_openvino_get_remote_context(ggml_decoder->device);
+            auto gpu_context = remote_context->as<ov::intel_gpu::ocl::ClContext>();
+            return gpu_context.create_tensor(type, sliced_shape, ggml_tensor->data);
+        }
+        return ov::Tensor(type, sliced_shape, ggml_tensor->data);
+    });
+}

-    return ov::Tensor(GgmlOvDecoder::get_ov_type(ggml_tensor), sliced_shape, ggml_tensor->data);
+// For a KV cache in the 4-D layout (GgmlOvDecoder::is_kvcache_4d), a [1, ctx, n_heads_kv, head_size] tensor over the
+// whole cache. Its shape stays the same from one inference to the next, so the GPU plugin keeps writing the cache in
+// place and reading it without a copy.
+std::optional<ov::Tensor> try_make_kv_4d_tensor(const std::shared_ptr<GgmlOvDecoder> & ggml_decoder,
+                                                const ggml_tensor * ggml_tensor) {
+    if (!ggml_decoder->is_kvcache_4d_input(ggml_tensor)) {
+        return std::nullopt;
+    }
+    const size_t n_heads_kv = static_cast<size_t>(ggml_decoder->get_n_heads_kv_for_tensor(ggml_tensor));
+    const ov::Shape shape{1, static_cast<size_t>(ggml_tensor->ne[1]), n_heads_kv,
+                          static_cast<size_t>(ggml_tensor->ne[0]) / n_heads_kv};
+    const auto type = GgmlOvDecoder::get_ov_type(ggml_tensor);
+    return ggml_decoder->cached_kv_tensor(ggml_tensor, type, shape, [&]() -> ov::Tensor {
+        if (ggml_openvino_buffer_is_remote(ggml_tensor)) {
+            auto remote_context = ggml_openvino_get_remote_context(ggml_decoder->device);
+            auto gpu_context = remote_context->as<ov::intel_gpu::ocl::ClContext>();
+            return gpu_context.create_tensor(type, shape, ggml_tensor->data);
+        }
+        return ov::Tensor(type, shape, ggml_tensor->data);
+    });
+}
+
+// Column count of a static KQ mask input; must match GgmlOvDecoder::get_graph_input_shape().
+static size_t get_static_attention_size(std::shared_ptr<GgmlOvDecoder> ggml_decoder, const std::string & name) {
+    const bool is_swa = ggml_decoder->is_swa_mask(ggml_decoder->get_input_ggml_tensor(name));
+    if (ggml_decoder->m_is_prefill ? ggml_openvino_kv_slice_enabled() : ggml_openvino_npu_decode_bucket() > 0) {
+        const auto params = ggml_decoder->get_compute_params();
+        const int attention_size = is_swa ? params.attention_size_swa : params.attention_size;
+        if (attention_size > 0) {
+            return static_cast<size_t>(attention_size);
+        }
+    }
+    if (is_swa) {
+        const auto & params = ggml_decoder->get_model_params();
+        return static_cast<size_t>(params.ctx_per_seq_swa * params.n_seq);
+    }
+    return static_cast<size_t>(ggml_decoder->get_ctx_size());
 }

 static uint64_t ggml_openvino_model_cache_extra_cfg(bool stateful, bool recurrent) {
@@ -114,12 +214,22 @@ static uint64_t ggml_openvino_model_cache_extra_cfg(bool stateful, bool recurren
                                         ggml_openvino_getenv_int("GGML_OPENVINO_MANUAL_GQA_ATTN") > 0 :
                                         ggml_openvino_is_gpu();

-    uint64_t extra_cfg = 1;  // Graph-ordinal port names (invalidate older disk-cache blobs).
+    uint64_t extra_cfg = 15;  // Recurrent caches are writable inputs without cache Results.
+#ifdef GGML_OPENVINO_HAS_CACHE_UPDATE
+    extra_cfg = extra_cfg * 131 + (ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_FUSED_RECURRENCE") ? 0u : 1u);
+#else
+    extra_cfg = extra_cfg * 131;
+#endif
     extra_cfg = extra_cfg * 131 + (stateful ? (recurrent ? 2u : 1u) : 0u);
     extra_cfg = extra_cfg * 131 + (ggml_openvino_reduce_compile_mem_enabled() ? 1u : 0u);
-    extra_cfg = extra_cfg * 131 + (ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_KV_SLICE") ? 1u : 0u);
+    extra_cfg = extra_cfg * 131 + (ggml_openvino_getenv_int("GGML_OPENVINO_ENABLE_KV_4D") > 0 ? 1u : 0u);
     extra_cfg = extra_cfg * 131 + (manual_gqa_enabled ? 1u : 0u);
     extra_cfg = extra_cfg * 131 + (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING") >= 2 ? 1u : 0u);
+    extra_cfg = extra_cfg * 131 + (ggml_openvino_shape_from_mask_enabled() ? 1u : 0u);
+    extra_cfg = extra_cfg * 131 + (ggml_openvino_npu_hfa_prefill_enabled() ? 4u : 0u);  // HFA input ABI v4.
+    if (ggml_openvino_is_npu()) {
+        extra_cfg = extra_cfg * 131 + static_cast<uint64_t>(ggml_openvino_get_npu_requant_type());
+    }
     return extra_cfg;
 }

@@ -152,6 +262,7 @@ static std::string compiled_graph_key(const ggml_cgraph * graph,
     append(decoder.is_static());
     append(decoder.is_stateful());
     append(prefill_chunk_size);
+    append(ggml_openvino_npu_hfa_prefill_enabled());
     bool has_weight_buffer_id = false;
     std::unordered_map<const ggml_tensor *, size_t> ids;
     std::function<void(const ggml_tensor *)> visit = [&](const ggml_tensor * tensor) {
@@ -205,6 +316,8 @@ static std::string compiled_graph_key(const ggml_cgraph * graph,
         if (!input.second.is_parameter) {
             append(input.second.value);
         }
+        append_string(input.second.shape_source);
+        append(input.second.shape_axis);
     }
     // Without an allocation generation, pointer reuse could select stale weights.
     // Such graphs still get private requests; they simply do not share compilation.
@@ -252,14 +365,34 @@ static std::string dynamic_graph_signature(const ggml_cgraph * graph,
     for (const auto & [name, input] : decoder.get_model_extra_inputs()) {
         append_string(name);
         append_string(input.type.get_type_name());
+        append(input.shape.size());
+        for (auto dim : input.shape) {
+            append(dim);
+        }
         append(input.is_parameter);
         if (!input.is_parameter) {
             append(input.value);
         }
+        append_string(input.shape_source);
+        append(input.shape_axis);
     }
     return key;
 }

+// The CPU plugin exports its MoE op (GatherMatmulCompressed) but cannot import it again ("unsupported opset",
+// OpenVINO 2026.4 and 2026.5). Writing such a blob makes every later run fail the import and compile again.
+static bool ggml_openvino_cpu_blob_importable(const std::string & device, const std::shared_ptr<ov::Model> & model) {
+    if (device != "CPU") {
+        return true;
+    }
+    for (const auto & op : model->get_ops()) {
+        if (std::string(op->get_type_name()).rfind("GatherMatmul", 0) == 0) {
+            return false;
+        }
+    }
+    return true;
+}
+
 static bool compiled_model_matches_graph(const ov::CompiledModel & model,
                                          const std::vector<std::string> & inputs,
                                          const std::vector<std::string> & outputs,
@@ -295,8 +428,16 @@ static bool compiled_model_matches_graph(const ov::CompiledModel & model,
             continue;
         }
         const auto & port = model.output(i);
+        const auto * tensor = graph_it->second;
+        auto expected_shape = GgmlOvDecoder::get_shape(tensor);
+        if (decoder.is_kvcache_4d_input(tensor)) {
+            // Cache Results use the same head-split layout as their Parameters.
+            const auto heads = static_cast<size_t>(decoder.get_n_heads_kv_for_tensor(tensor));
+            expected_shape = {1, static_cast<size_t>(tensor->ne[1]), heads,
+                              static_cast<size_t>(tensor->ne[0]) / heads};
+        }
         if (port.get_element_type() != GgmlOvDecoder::get_ov_type(graph_it->second) ||
-            !port.get_partial_shape().compatible(ov::PartialShape(GgmlOvDecoder::get_shape(graph_it->second)))) {
+            !port.get_partial_shape().compatible(ov::PartialShape(expected_shape))) {
             return false;
         }
     }
@@ -353,31 +494,23 @@ static void dump_ov_profiling_info(const ov::InferRequest & infer_request) {
                   executed_count, total_real_time / 1000.0, total_cpu_time / 1000.0);
 }

-static void reset_single_slot_recurrent_cache(ggml_cgraph * cgraph,
-                                              const ModelParams & model_params,
-                                              const ComputeParams & compute_params) {
-    if (model_params.n_rs_slots != 1 || compute_params.cache_rs_reset_len == 0) {
-        return;
-    }
-    GGML_ASSERT(compute_params.cache_rs_reset_idx == 0 && compute_params.cache_rs_reset_len == 1);
-
-    std::set<ggml_backend_buffer_t> buffers;
-    for (int i = 0; i < cgraph->n_nodes; i++) {
-        auto * node = cgraph->nodes[i];
-        if (node->op == GGML_OP_SCALE && GgmlOvDecoder::is_recurrent_cache(node->view_src) &&
-            node->view_src->ne[1] == 1 && node->view_src->buffer != nullptr) {
-            buffers.insert(node->view_src->buffer);
-        }
+static uint64_t ggml_openvino_model_cache_mix_string(uint64_t hash, const char * value) {
+    if (!value) {
+        return hash * 131;
     }
-    for (auto * buffer : buffers) {
-        ggml_backend_buffer_clear(buffer, 0);
+    while (*value) {
+        hash = hash * 131 + static_cast<unsigned char>(*value++);
     }
+    return hash * 131 + 1;
 }

 ov::Tensor create_ov_output_tensor(const std::shared_ptr<GgmlOvDecoder> & ggml_decoder,
                                    const std::shared_ptr<ov::InferRequest> & infer_request,
                                    int output_index,
                                    const ggml_tensor * ggml_tensor) {
+    if (auto kv_4d = try_make_kv_4d_tensor(ggml_decoder, ggml_tensor)) {
+        return *kv_4d;
+    }
     if (auto sliced = try_make_kv_sliced_tensor(ggml_decoder, std::string(ggml_tensor->name), ggml_tensor)) {
         return *sliced;
     }
@@ -387,6 +520,7 @@ ov::Tensor create_ov_output_tensor(const std::shared_ptr<GgmlOvDecoder> & ggml_d
     void * output_data = ggml_tensor->data;
     if (ggml_decoder->is_static()) {
         output_shape = infer_request->get_output_tensor(output_index).get_shape();
+        output_data = kv_bind_ptr(output_data, output_type, output_shape);
     } else {
         // For a CPY into a padded view_src (e.g. a padded KV cache buffer), the
         // OV ScatterUpdate node outputs the full view_src shape, not the CPY node's
@@ -399,21 +533,32 @@ ov::Tensor create_ov_output_tensor(const std::shared_ptr<GgmlOvDecoder> & ggml_d
             output_data = ggml_tensor->view_src->data;
         } else {
             output_shape = GgmlOvDecoder::get_shape(ggml_tensor);
+            if (ggml_tensor->op == GGML_OP_GATED_DELTA_NET && ggml_tensor->op_params[0] > 1) {
+                const auto * value = ggml_tensor->src[2];
+                const int64_t valid_count = std::min<int64_t>(value->ne[2], ggml_tensor->op_params[0]);
+                output_shape[2] = (value->ne[2] + valid_count * value->ne[0]) * value->ne[3];
+            }
         }
     }

-    // The sliced path above covers eligible KV outputs. This also covers recurrent state and full-size KV fallbacks.
+    // Reuse the caller-owned full-cache binding when its shape and storage match.
     if (!ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_REMOTE_OUTPUTS") && !ggml_decoder->is_static() &&
-        !ggml_decoder->is_splited_model() && ggml_openvino_buffer_is_remote(ggml_tensor) &&
-        ggml_tensor->extra != nullptr) {
-        auto * extra_base = static_cast<ggml_openvino_extra_base *>(ggml_tensor->extra);
-        if (extra_base->type == ggml_openvino_extra_base::Type::TENSOR) {
-            auto * tensor_extra = static_cast<ggml_openvino_tensor_extra *>(extra_base);
-            if (tensor_extra->tensor != nullptr && tensor_extra->tensor->get_element_type() == output_type &&
-                tensor_extra->tensor->get_shape() == output_shape) {
-                return *tensor_extra->tensor;
+        !ggml_decoder->is_splited_model() && ggml_openvino_buffer_is_remote(ggml_tensor)) {
+        auto resolved = output_data;
+        if (ggml_tensor->extra != nullptr) {
+            auto * base = static_cast<ggml_openvino_extra_base *>(ggml_tensor->extra);
+            if (base->type == ggml_openvino_extra_base::Type::TENSOR) {
+                auto & cached = static_cast<ggml_openvino_tensor_extra *>(base)->tensor;
+                if (cached && cached->get_element_type() == output_type && cached->get_shape() == output_shape) {
+                    auto remote = cached->as<ov::intel_gpu::ocl::USMTensor>();
+                    if (remote.get() == resolved) {
+                        return *cached;
+                    }
+                }
             }
         }
+        auto context = ggml_openvino_get_remote_context(ggml_decoder->device)->as<ov::intel_gpu::ocl::ClContext>();
+        return context.create_tensor(output_type, output_shape, resolved);
     }

     ov::Tensor output_tensor(output_type, output_shape, output_data);
@@ -447,6 +592,23 @@ template <typename T> void set_zero_diagonal(std::vector<T> & matrix, size_t row
     }
 }

+// build the padded causal mask directly in the destination buffer
+template <typename T>
+void fill_prefill_mask(T * dst, const T * src, size_t valid_rows, size_t src_cols, size_t padded_rows,
+                       size_t padded_cols, T pad_value, T zero_value) {
+    for (size_t i = 0; i < padded_rows; ++i) {
+        T * row = dst + i * padded_cols;
+        if (i < valid_rows) {
+            const size_t ncopy = std::min(src_cols, padded_cols);
+            std::memcpy(row, src + i * src_cols, ncopy * sizeof(T));
+            std::fill(row + ncopy, row + padded_cols, pad_value);
+        } else {
+            std::fill(row, row + padded_cols, pad_value);
+        }
+        row[std::min(i, padded_cols - 1)] = zero_value;
+    }
+}
+
 ov::Tensor make_contiguous_split_input_tensor(const struct ggml_tensor * ggml_tensor, const ov::Shape & input_shape) {
     const size_t element_size = ggml_type_size(ggml_tensor->type);
     const size_t block_size = ggml_blck_size(ggml_tensor->type);
@@ -482,10 +644,28 @@ ov::Tensor make_contiguous_split_input_tensor(const struct ggml_tensor * ggml_te
 ov::Tensor convert_ggml_input_to_ov(const std::shared_ptr<GgmlOvDecoder> & ggml_decoder, const std::string & name) {
     const auto * ggml_tensor = ggml_decoder->get_input_ggml_tensor(name);

+    if (name == "rs_active_rows") {
+        const auto count = ggml_decoder->get_compute_params().s_copy_active_slot_len;
+        GGML_ASSERT(count > 0 && count <= ggml_nelements(ggml_tensor));
+        return ov::Tensor(ov::element::i32, ov::Shape{1, 1, 1, size_t(count)}, ggml_tensor->data);
+    }
+
+    if (auto kv_4d = try_make_kv_4d_tensor(ggml_decoder, ggml_tensor)) {
+        return *kv_4d;
+    }
     if (auto sliced = try_make_kv_sliced_tensor(ggml_decoder, name, ggml_tensor)) {
         return *sliced;
     }

+    if (ggml_decoder->is_static() &&
+        GgmlOvDecoder::is_kvcache(ggml_tensor, ggml_decoder->get_tensor_used_op(ggml_tensor))) {
+        const auto type = ggml_decoder->get_ov_type(ggml_tensor);
+        const auto shape = ggml_decoder->get_shape(ggml_tensor);
+        if (void * storage = ggml_openvino_kv_storage(ggml_tensor->data, ov::shape_size(shape) * type.size())) {
+            return ov::Tensor(type, shape, storage);
+        }
+    }
+
     if (ggml_tensor->extra != nullptr && !ggml_decoder->is_splited_model()) {
         auto * extra_base = static_cast<ggml_openvino_extra_base *>(ggml_tensor->extra);
         if (extra_base->type == ggml_openvino_extra_base::Type::TENSOR) {
@@ -517,8 +697,45 @@ ov::Tensor get_ov_input_tensor(const std::shared_ptr<GgmlOvDecoder> & ggml_decod
     ov::Tensor input_tensor;
     auto extra_input = ggml_decoder->get_model_extra_inputs().find(param_name);
     if (extra_input != ggml_decoder->get_model_extra_inputs().end()) {
-        input_tensor = ov::Tensor(extra_input->second.type, extra_input->second.shape);
-        *input_tensor.data<int64_t>() = extra_input->second.value;
+        auto & cached = ggml_decoder->runtime_input_tensors[param_name];
+        if (!cached || cached.get_element_type() != extra_input->second.type ||
+            cached.get_shape() != extra_input->second.shape) {
+            if (ggml_openvino_is_gpu() && !ggml_decoder->is_static() && !ggml_decoder->is_stateful()) {
+                auto context =
+                    ggml_openvino_get_remote_context(ggml_decoder->device)->as<ov::intel_gpu::ocl::ClContext>();
+                auto storage = context.create_usm_host_tensor(extra_input->second.type, extra_input->second.shape);
+                cached = ov::Tensor(extra_input->second.type, extra_input->second.shape, storage.get());
+                ggml_decoder->runtime_input_storage[param_name] = storage;
+            } else {
+                cached = ov::Tensor(extra_input->second.type, extra_input->second.shape);
+            }
+        }
+        input_tensor = cached;
+        if (param_name == "sampling_rows") {
+            auto * data = input_tensor.data<int64_t>();
+            for (const auto & row : ggml_decoder->get_compute_params().sampling_rows) {
+                *data++ = row.second;
+            }
+        } else if (param_name.compare(0, 11, "rs_offsets_") == 0 || param_name.compare(0, 11, "rs_windows_") == 0) {
+            const auto & controls = ggml_decoder->get_compute_params();
+            const auto & values = controls.recurrent_controls.at(param_name);
+            std::copy(values.begin(), values.end(), input_tensor.data<int64_t>());
+        } else if (param_name.compare(0, 10, "rs_update_") == 0) {
+            const auto & controls = ggml_decoder->get_compute_params();
+            const auto & write = controls.rs_writebacks.at(param_name.substr(10));
+            auto data = input_tensor.data<int64_t>();
+            data[0] = write.slot_begin;
+            data[1] = write.seq_count >= 0 ? write.seq_count : controls.s_copy_active_slot_len;
+            data[2] = write.snapshot_stride >= 0 ? write.snapshot_stride : data[1];
+            // Direct GDN states contain no attention prefix.
+            data[3] = 0;
+        } else {
+            if (extra_input->second.type == ov::element::i64) {
+                *input_tensor.data<int64_t>() = extra_input->second.value;
+            } else {
+                std::memset(input_tensor.data(), 0, input_tensor.get_byte_size());
+            }
+        }
     } else {
         input_tensor = convert_ggml_input_to_ov(ggml_decoder, param_name);
     }
@@ -557,7 +774,7 @@ ov::Tensor get_ov_input_tensor_static_decode(const std::shared_ptr<GgmlOvDecoder
     }

     if (GgmlOvDecoder::is_inp_mask(ggml_tensor, op)) {
-        size_t context_size = ggml_decoder->get_ctx_size();
+        size_t context_size = get_static_attention_size(ggml_decoder, param_name);
         if (ggml_tensor->type == GGML_TYPE_F16) {
             std::vector<ggml_fp16_t> padded_data =
                 pad_input<ggml_fp16_t>(ggml_tensor, 1, context_size, GGML_FP32_TO_FP16(-INFINITY));
@@ -590,10 +807,67 @@ ov::Tensor get_ov_input_tensor_static_prefill(const std::shared_ptr<GgmlOvDecode
         *input_tensor.data<int64_t>() = (int64_t) chunk_valid_size;
         return input_tensor;
     }
+    if (param_name.rfind("ggml_hfa_past_", 0) == 0) {
+        const std::string cache_name = param_name.substr(strlen("ggml_hfa_past_"));
+        const ggml_tensor * kv_tensor = nullptr;
+        for (const auto & [name, _] : ggml_decoder->get_model_inputs()) {
+            if (name.substr(0, name.find('#')) == cache_name) {
+                kv_tensor = ggml_decoder->get_input_ggml_tensor(name);
+                break;
+            }
+        }
+        if (kv_tensor == nullptr) {
+            throw std::runtime_error("HFA past KV source not found for " + param_name);
+        }
+        const auto & info = ggml_decoder->get_model_extra_inputs().at(param_name);
+        // Shape is [1, past, heads, head_size], or [1, heads, past, head_size] for head-major KV
+        // ([1, heads, head_size, past] for transposed V).
+        const bool head_major = ggml_openvino_npu_kv_head_major_enabled();
+        const bool v_transposed = head_major && cache_name.rfind("cache_v_l", 0) == 0;
+        const size_t past_capacity = v_transposed ? info.shape[3] : head_major ? info.shape[2] : info.shape[1];
+        const size_t heads = head_major ? info.shape[1] : info.shape[2];
+        const size_t head_size = v_transposed ? info.shape[2] : info.shape[3];
+        const size_t row_bytes = heads * head_size * ggml_type_size(kv_tensor->type);
+        if (static_cast<size_t>(kv_tensor->ne[0]) != heads * head_size || kv_tensor->nb[1] != row_bytes) {
+            throw std::runtime_error("HFA past KV source shape is incompatible with " + param_name);
+        }
+        if (past_capacity <= static_cast<size_t>(kv_tensor->ne[1])) {
+            // Zero-copy: the first past_capacity cache rows; HFA only reads the rows of the explicit history.
+            return ov::Tensor(info.type, info.shape, kv_bind_ptr(kv_tensor->data, info.type, info.shape));
+        }
+        // Single-chunk prompt in a cache shorter than one chunk: history is empty, share one zero tensor.
+        static std::mutex zeros_mutex;
+        static std::map<std::string, ov::Tensor> zeros;
+        std::lock_guard<std::mutex> lock(zeros_mutex);
+        auto & zero = zeros[info.type.get_type_name() + info.shape.to_string()];
+        if (!zero) {
+            zero = ov::Tensor(info.type, info.shape);
+            std::memset(zero.data(), 0, zero.get_byte_size());
+        }
+        return zero;
+    }
+    if (param_name == "ggml_hfa_mask") {
+        const auto & info = ggml_decoder->get_model_extra_inputs().at(param_name);
+        ov::Tensor input_tensor(info.type, info.shape);
+        auto * data = static_cast<ggml_fp16_t *>(input_tensor.data());
+        const size_t context_size = info.shape[3];
+        const size_t past_capacity = context_size - chunk_size;
+        const size_t history_size = std::min(static_cast<size_t>(chunk_index) * chunk_size, past_capacity);
+        const auto neg_inf = GGML_FP32_TO_FP16(-INFINITY);
+        const auto zero = GGML_FP32_TO_FP16(0.0f);
+        std::fill(data, data + chunk_size * context_size, neg_inf);
+        for (size_t row = 0; row < chunk_size; ++row) {
+            std::fill(data + row * context_size, data + row * context_size + history_size, zero);
+            for (size_t col = 0; col <= row; ++col) {
+                data[row * context_size + past_capacity + col] = zero;
+            }
+        }
+        return input_tensor;
+    }
     if (chunk_index > 0 && param_name == "cache_rs_reset_len") {
         // The recurrent-state clear belongs to the start of the sequence. Re-applying it on every
         // chunk would wipe the state accumulated by the preceding chunks, so disable it (a zero
-        // length makes scale.cpp's keep-mask select every slot) after the first chunk.
+        // length makes translate_recurrent_cache_reset keep every slot) after the first chunk.
         ov::Tensor input_tensor(ov::element::i64, ov::Shape{1});
         *input_tensor.data<int64_t>() = 0;
         return input_tensor;
@@ -703,23 +977,35 @@ ov::Tensor get_ov_input_tensor_static_prefill(const std::shared_ptr<GgmlOvDecode
         size_t cols = ggml_tensor->ne[0];
         size_t rows = ggml_tensor->ne[1];
         size_t chunk_valid_rows = std::min(chunk_size, rows - chunk_index * chunk_size);
-        size_t context_size = ggml_decoder->get_ctx_size();
+        size_t context_size = get_static_attention_size(ggml_decoder, param_name);
+        static const bool fast_mask = ggml_openvino_npu_fast_mask_enabled();
         if (ggml_tensor->type == GGML_TYPE_F16) {
             const auto * ggml_data =
                 static_cast<const ggml_fp16_t *>(ggml_tensor->data) + chunk_index * chunk_size * cols;
+            ov::Tensor input_tensor(ov::element::f16, ov::Shape{1, 1, chunk_size, context_size});
+            if (fast_mask) {
+                fill_prefill_mask<ggml_fp16_t>(static_cast<ggml_fp16_t *>(input_tensor.data()), ggml_data,
+                                               chunk_valid_rows, cols, chunk_size, context_size,
+                                               GGML_FP32_TO_FP16(-INFINITY), GGML_FP32_TO_FP16(0.0f));
+                return input_tensor;
+            }
             std::vector<ggml_fp16_t> padded_data = pad_input<ggml_fp16_t>(ggml_data, chunk_valid_rows, cols, chunk_size,
                                                                           context_size, GGML_FP32_TO_FP16(-INFINITY));
             set_zero_diagonal(padded_data, chunk_size, context_size, GGML_FP32_TO_FP16(0.0f));
-            ov::Tensor input_tensor(ov::element::f16, ov::Shape{1, 1, chunk_size, context_size});
             std::memcpy(input_tensor.data(), padded_data.data(), padded_data.size() * sizeof(ggml_fp16_t));
             return input_tensor;
         }

         const auto * ggml_data = static_cast<const float *>(ggml_tensor->data) + chunk_index * chunk_size * cols;
+        ov::Tensor input_tensor(ov::element::f32, ov::Shape{1, 1, chunk_size, context_size});
+        if (fast_mask) {
+            fill_prefill_mask<float>(input_tensor.data<float>(), ggml_data, chunk_valid_rows, cols, chunk_size,
+                                     context_size, -INFINITY, 0.0f);
+            return input_tensor;
+        }
         std::vector<float> padded_data =
             pad_input<float>(ggml_data, chunk_valid_rows, cols, chunk_size, context_size, -INFINITY);
         set_zero_diagonal(padded_data, chunk_size, context_size);
-        ov::Tensor input_tensor(ov::element::f32, ov::Shape{1, 1, chunk_size, context_size});
         auto * data_ptr = input_tensor.data<float>();
         std::copy(padded_data.begin(), padded_data.begin() + chunk_size * context_size, data_ptr);
         return input_tensor;
@@ -741,6 +1027,7 @@ enum ggml_status naive_compute(ggml_cgraph * cgraph,
     bool naive = true;
     auto model_weights = GgmlOvDecoder::create_weight_nodes(cgraph, naive);
     auto decoder = std::make_shared<GgmlOvDecoder>(cgraph, model_weights);
+    decoder->device = device;
     auto input_model = std::make_shared<ov::frontend::ggml::InputModel>(decoder);
     auto model = ov::frontend::ggml::FrontEnd::convert(input_model, naive);
     if (ggml_openvino_getenv_int("GGML_OPENVINO_DUMP_IR")) {
@@ -748,7 +1035,7 @@ enum ggml_status naive_compute(ggml_cgraph * cgraph,
     }

     std::shared_ptr<ov::InferRequest> infer_request;
-    auto remote_context = ggml_openvino_get_remote_context();
+    auto remote_context = ggml_openvino_get_remote_context(device);
     ov::AnyMap compile_config = config;
     if (cgraph->nodes[0]->op == GGML_OP_MUL_MAT) {
         // TODO ACCURACY hint triggers a bug in GPU plugin/driver on Lunar Lake. Remove once CVS-182166 is resolved
@@ -798,7 +1085,7 @@ enum ggml_status naive_compute(ggml_cgraph * cgraph,
             // Debug-only output added via GGML_OPENVINO_DEBUG_NODE; nothing to copy into.
             if (ggml_openvino_getenv_int("GGML_OPENVINO_DEBUG_OUTPUT") ||
                 ggml_openvino_getenv_str("GGML_OPENVINO_DEBUG_NODE")) {
-                print_output_tensor_info(output_names[i], output_tensor, output_tensor.data());
+                print_output_tensor_info(output_names[i], output_tensor, nullptr);
             }
             continue;
         }
@@ -808,6 +1095,63 @@ enum ggml_status naive_compute(ggml_cgraph * cgraph,
     return GGML_STATUS_SUCCESS;
 }

+namespace {
+
+// Small host inputs and outputs go through backend-owned USM host tensors, so the GPU plugin has no per-call copies.
+constexpr size_t k_io_staging_max_bytes = 4u << 20;
+
+bool io_staging_enabled() {
+    static const bool enabled =
+        ggml_openvino_is_gpu() && ggml_openvino_get_remote_context(ggml_openvino_get_device_name()).has_value();
+    return enabled;
+}
+
+// Told from the ggml side: asking an ov::Tensor if it is remote is slow (it throws for a host tensor).
+bool is_host_input(const std::shared_ptr<GgmlOvDecoder> & ggml_decoder, const std::string & name) {
+    if (ggml_decoder->get_model_extra_inputs().count(name) != 0) {
+        return true;
+    }
+    const auto * t = ggml_decoder->get_input_ggml_tensor(name);
+    return t != nullptr && !ggml_openvino_buffer_is_remote(t);
+}
+
+// Off until the OpenVINO GPU plugin is fixed: it writes out of bounds when a dynamic output it bound to USM host memory
+// is later bound to plain host memory, which output staging does when the output count changes between calls.
+constexpr bool k_stage_outputs = false;
+
+bool stageable_size(const ov::Tensor & t) {
+    return t.get_byte_size() > 0 && t.get_byte_size() <= k_io_staging_max_bytes;
+}
+
+ov::Tensor & staging_for(const GgmlOvDecoder & decoder, GgmlOvDecoder::ov_port_binding & b, const ov::Tensor & like) {
+    const size_t bytes = like.get_byte_size();
+    if (!b.staging || b.staging_bytes < bytes || b.staging.get_element_type() != like.get_element_type()) {
+        auto gpu_context = ggml_openvino_get_remote_context(decoder.device)->as<ov::intel_gpu::ocl::ClContext>();
+        auto usm = gpu_context.create_usm_host_tensor(like.get_element_type(), like.get_shape());
+        b.staging_ptr = usm.get();
+        b.staging = usm;
+        b.staging_bytes = bytes;
+        b.staging_id = decoder.next_tensor_id();
+    } else if (b.staging.get_shape() != like.get_shape()) {
+        b.staging.set_shape(like.get_shape());
+    }
+    return b.staging;
+}
+
+// False when the port already holds this tensor (same id) with the same shape and type.
+bool needs_bind(GgmlOvDecoder::ov_port_binding & b, const ov::Tensor & t, uint64_t id) {
+    if (io_staging_enabled() && id != 0 && id == b.bound_id && t.get_element_type() == b.bound_type &&
+        t.get_shape() == b.bound_shape) {
+        return false;
+    }
+    b.bound_id = id;
+    b.bound_shape = t.get_shape();
+    b.bound_type = t.get_element_type();
+    return true;
+}
+
+}  // namespace
+
 enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::shared_ptr<ov_runtime_context> & r_ctx) {
     auto & core = ov_singleton_core();
     const auto & config = ggml_openvino_get_compile_config();
@@ -889,7 +1233,9 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share
             if (cache_hit) {
                 entry = it->second;
             } else {
-                r_ctx->clear_caches_locked();
+                if (r_ctx->stateful || r_ctx->decoder_cache.size() >= 8) {
+                    r_ctx->clear_caches_locked();
+                }
                 auto mutex = std::make_shared<std::mutex>();
                 entry = std::make_shared<decoder_runtime_ctx>(mutex);
                 r_ctx->decoder_cache[key] = entry;
@@ -1037,6 +1383,7 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share
             auto weight_names = get_weight_names(cgraph);
             ggml_decoder = std::make_shared<GgmlOvDecoder>(cgraph, m_params, c_params, weight_names,
                                                           is_static, r_ctx->stateful, model_is_splitted);
+            ggml_decoder->device = device;
             const std::string model_cache_dir = ggml_openvino_model_cache_dir();
             const std::string exact_key = cache_enabled ? compiled_graph_key(cgraph, *ggml_decoder, device) : "";
             uint64_t model_fp = 0;
@@ -1106,21 +1453,23 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share
                         ggml_openvino_model_cache_blob_path(model_cache_dir, candidate_fp);
                     const std::string candidate_manifest =
                         ggml_openvino_model_cache_manifest_path(model_cache_dir, candidate_fp);
-                    std::ifstream blob_in(candidate_blob, std::ios::binary);
                     std::vector<std::string> saved_inputs, saved_outputs;
-                    if (!blob_in.is_open() ||
+                    if (!std::ifstream(candidate_blob, std::ios::binary).is_open() ||
                         !ggml_openvino_model_cache_verify_manifest(candidate_manifest, cgraph, candidate_fp,
                                                                    saved_inputs, saved_outputs)) {
                         continue;
                     }
                     const int64_t import_start = ggml_time_us();
                     try {
+                        // Import from a memory-mapped tensor instead of a stream: a stream import reads the whole
+                        // blob into a heap buffer while the plugin builds its own copy of the weights.
+                        const ov::Tensor blob = ov::read_tensor_data(candidate_blob);
                         ov::CompiledModel cm;
-                        auto remote_context = ggml_openvino_get_remote_context();
+                        auto remote_context = ggml_openvino_get_remote_context(device);
                         if (remote_context.has_value()) {
-                            cm = core.import_model(blob_in, remote_context.value(), mc_config);
+                            cm = core.import_model(blob, remote_context.value(), mc_config);
                         } else {
-                            cm = core.import_model(blob_in, device, mc_config);
+                            cm = core.import_model(blob, device, mc_config);
                         }
                         // CPU serialization can rename Results that share a name with a Parameter.
                         if (!saved_inputs.empty() || !saved_outputs.empty()) {
@@ -1177,6 +1526,18 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share

             std::shared_ptr<ov::Model> model;
             if (!imported && ggml_openvino_model_cache_only()) {
+                // No blob is ever written for these (see ggml_openvino_cpu_blob_importable), so the
+                // generic advice to run once without cache-only cannot help. MXFP4 does not use GatherMatmul.
+                const auto is_gather_matmul_moe = [](const ggml_tensor * n) {
+                    return n->op == GGML_OP_MUL_MAT_ID && n->src[0]->type != GGML_TYPE_MXFP4;
+                };
+                if (device == "CPU" &&
+                    std::any_of(cgraph->nodes, cgraph->nodes + cgraph->n_nodes, is_gather_matmul_moe)) {
+                    GGML_LOG_ERROR(
+                        "ggml-openvino: GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY does not work for MoE models on CPU: "
+                        "the CPU plugin cannot import MoE models with this OpenVINO version, so they are not cached\n");
+                    return GGML_STATUS_FAILED;
+                }
                 GGML_LOG_ERROR(
                     "ggml-openvino: missing or incompatible compiled model: %s; run once without "
                     "GGML_OPENVINO_COMPILED_MODEL_CACHE_ONLY\n",
@@ -1194,6 +1555,7 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share

                 ggml_decoder = std::make_shared<GgmlOvDecoder>(cgraph, m_params, c_params, model_weights, is_static,
                                                                r_ctx->stateful, model_is_splitted);
+                ggml_decoder->device = device;
                 decoder_end_time = ggml_time_us();

                 auto input_model = std::make_shared<ov::frontend::ggml::InputModel>(ggml_decoder);
@@ -1212,7 +1574,7 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share
                 // the resulting CompiledModel can be exported and later re-imported.
                 const ov::AnyMap & compile_config = model_cache_dir.empty() ? config : mc_config;
                 ov::CompiledModel compiled_model;
-                auto remote_context = ggml_openvino_get_remote_context();
+                auto remote_context = ggml_openvino_get_remote_context(device);
                 if (remote_context.has_value()) {
                     compiled_model = core.compile_model(model, remote_context.value(), compile_config);
                 } else {
@@ -1229,7 +1591,13 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share

                 // Export to the frontend model cache for next time. Publish the blob first,
                 // then the manifest, so a cache hit only sees fully written artifacts.
-                if (!model_cache_dir.empty() && !model_is_splitted && model_fp != 0) {
+                const bool blob_importable = ggml_openvino_cpu_blob_importable(device, model);
+                if (!blob_importable && !model_cache_dir.empty()) {
+                    GGML_LOG_WARN(
+                        "ggml-openvino: model cache not written: the CPU plugin cannot import MoE models with this "
+                        "OpenVINO version\n");
+                }
+                if (!model_cache_dir.empty() && !model_is_splitted && model_fp != 0 && blob_importable) {
                     try {
                         const std::string blob_tmp = ggml_openvino_model_cache_temp_path(blob_path);
                         const std::string manifest_tmp = ggml_openvino_model_cache_temp_path(manifest_path);
@@ -1268,8 +1636,11 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share

             entry->ptr = ggml_decoder;
             if (!shared_key.empty() && shared_it == shared_cache->graphs.end()) {
-                shared_cache->graphs.emplace(shared_key,
-                                             ov_compiled_graph{shared_model, {}, ov_input_names, ov_output_names});
+                ov_compiled_graph graph;
+                graph.decode = shared_model;
+                graph.input_names = ov_input_names;
+                graph.output_names = ov_output_names;
+                shared_cache->graphs.emplace(shared_key, std::move(graph));
             }
             if (cache_enabled) {
                 std::lock_guard<std::mutex> map_lock(r_ctx->ctx_mutex);
@@ -1313,20 +1684,40 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share
                     "or disable GGML_OPENVINO_STATEFUL_EXECUTION.\n");
                 return GGML_STATUS_FAILED;
             }
-        } else {
-            reset_single_slot_recurrent_cache(cgraph, m_params, c_params);
         }

+        auto & in_bindings = ggml_decoder->port_bindings(true, ov_input_names.size(), infer_request.get());
         for (size_t i = 0; i < ov_input_names.size(); i++) {
             const auto & param_name = ov_input_names[i];
+            ggml_decoder->take_last_kv_tensor_id();  // clear
             auto input_tensor = get_ov_input_tensor(ggml_decoder, param_name);
-            infer_request->set_input_tensor(i, input_tensor);
+            uint64_t id = ggml_decoder->take_last_kv_tensor_id();
+            auto & binding = in_bindings[i];
+            if (id == 0 && io_staging_enabled() && is_host_input(ggml_decoder, param_name) &&
+                stageable_size(input_tensor)) {
+                auto & staging = staging_for(*ggml_decoder, binding, input_tensor);
+                std::memcpy(binding.staging_ptr, input_tensor.data(), input_tensor.get_byte_size());
+                input_tensor = staging;
+                id = binding.staging_id;
+            }
+            if (needs_bind(binding, input_tensor, id)) {
+                infer_request->set_input_tensor(i, input_tensor);
+            }

             if (ggml_openvino_getenv_int("GGML_OPENVINO_DEBUG_INPUT")) {
                 print_input_tensor_info(param_name, input_tensor);
             }
         }

+        auto & out_bindings = ggml_decoder->port_bindings(false, ov_output_names.size(), infer_request.get());
+
+        struct staged_output {
+            void * dst;
+            const void * src;
+            size_t bytes;
+        };
+
+        std::vector<staged_output> staged_outputs;  // copied back after infer()
         for (size_t i = 0; i < ov_output_names.size(); i++) {
             // Debug-only outputs added via GGML_OPENVINO_DEBUG_NODE (see
             // translate_session.cpp) have no corresponding ggml tensor; leave
@@ -1339,16 +1730,48 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share
                 continue;
             }
             auto * ggml_tensor = model_output_it->second;
-            if (ggml_nbytes(ggml_tensor) == 0) {
-                continue;
+            ggml_decoder->take_last_kv_tensor_id();  // clear
+            ov::Tensor output_tensor;
+            const bool inactive = (ggml_tensor->op != GGML_OP_NONE && !(ggml_tensor->flags & GGML_TENSOR_FLAG_COMPUTE)) ||
+                                  ggml_nbytes(ggml_tensor) == 0;
+            if (inactive) {
+                // Inactive pure sampling branches have no scheduler allocation. Never bind a stale active output.
+                auto & scratch = ggml_decoder->runtime_output_scratch[ov_output_names[i]];
+                const auto type = GgmlOvDecoder::get_ov_type(ggml_tensor);
+                const auto shape = GgmlOvDecoder::get_shape(ggml_tensor);
+                if (!scratch || scratch.get_element_type() != type || scratch.get_shape() != shape) {
+                    if (ggml_openvino_is_gpu() && ggml_nbytes(ggml_tensor) > 0) {
+                        auto context = ggml_openvino_get_remote_context(device)->as<ov::intel_gpu::ocl::ClContext>();
+                        scratch = context.create_usm_device_tensor(type, shape);
+                    } else {
+                        scratch = ov::Tensor(type, shape);
+                    }
+                }
+                output_tensor = scratch;
+            } else {
+                output_tensor = create_ov_output_tensor(ggml_decoder, infer_request, i, ggml_tensor);
+            }
+            uint64_t id = ggml_decoder->take_last_kv_tensor_id();
+            auto & binding = out_bindings[i];
+            if (k_stage_outputs && !inactive && id == 0 && io_staging_enabled() &&
+                !ggml_openvino_buffer_is_remote(ggml_tensor) && stageable_size(output_tensor)) {
+                void * dst = output_tensor.data();
+                const size_t bytes = output_tensor.get_byte_size();
+                output_tensor = staging_for(*ggml_decoder, binding, output_tensor);
+                staged_outputs.push_back({dst, binding.staging_ptr, bytes});
+                id = binding.staging_id;
+            }
+            if (needs_bind(binding, output_tensor, id)) {
+                infer_request->set_output_tensor(i, output_tensor);
             }
-            auto output_tensor = create_ov_output_tensor(ggml_decoder, infer_request, i, ggml_tensor);
-            infer_request->set_output_tensor(i, output_tensor);
         }

         ov_raw_infer_start = ggml_time_us();
         infer_request->infer();
         infer_end_time = ggml_time_us();
+        for (const auto & out : staged_outputs) {
+            std::memcpy(out.dst, out.src, out.bytes);
+        }
         if (stateful_recurrent) {
             const auto * inp_pos = get_inp_pos_tensor(cgraph);
             r_ctx->stateful_kv_size =
@@ -1360,7 +1783,7 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share
             ggml_openvino_getenv_str("GGML_OPENVINO_DEBUG_NODE")) {
             for (size_t i = 0; i < ov_output_names.size(); i++) {
                 const auto output_tensor = infer_request->get_output_tensor(i);
-                print_output_tensor_info(ov_output_names[i], output_tensor, output_tensor.data());
+                print_output_tensor_info(ov_output_names[i], output_tensor, nullptr);
             }
         }

@@ -1397,15 +1820,260 @@ enum ggml_status ov_graph_compute_dynamic(ggml_cgraph * cgraph, const std::share
     return GGML_STATUS_SUCCESS;
 }

-ov::AnyMap without_npuw(const ov::AnyMap & config) {
-    ov::AnyMap out;
-    for (const auto & kv : config) {
-        if (kv.first.rfind("NPUW", 0) == 0 || kv.first == "NPU_USE_NPUW") {
-            continue;
+static uint64_t static_model_salt(uint64_t base_extra, bool prefill, int chunk, int attn, int attn_swa, bool unfold) {
+    uint64_t h = base_extra;
+    h = h * 131 + (prefill ? 1u : 0u);
+    h = h * 131 + static_cast<uint64_t>(static_cast<uint32_t>(chunk));
+    h = h * 131 + static_cast<uint64_t>(static_cast<uint32_t>(attn));
+    h = h * 131 + static_cast<uint64_t>(static_cast<uint32_t>(attn_swa));
+    h = h * 131 + (unfold ? 1u : 0u);
+    return h;
+}
+
+static bool static_blob_valid(const std::string & model_cache_dir, uint64_t fp, const ggml_cgraph * cgraph) {
+    if (model_cache_dir.empty() || fp == 0) {
+        return false;
+    }
+    std::ifstream blob_in(ggml_openvino_model_cache_blob_path(model_cache_dir, fp), std::ios::binary);
+    std::vector<std::string> inputs, outputs;  // ports are bound in frontend order
+    return blob_in.is_open() &&
+           ggml_openvino_model_cache_verify_manifest(ggml_openvino_model_cache_manifest_path(model_cache_dir, fp),
+                                                     cgraph, fp, inputs, outputs);
+}
+
+struct static_build_args {
+    ov::Core *    core;
+    std::string   device;
+    bool          dump_ir;
+    long long     dump_ir_timestamp;
+    std::string   model_cache_dir;
+    ggml_cgraph * cgraph;
+    bool          self_contained;
+    bool          skip_weights;
+    // GGML_OPENVINO_NPU_WEIGHTLESS_BANK: weightless blob whose weights live in weights_path.
+    bool          weightless_bank = false;
+    std::string   weights_path;
+};
+
+// Cached blobs imported in this process, keyed by fingerprint, so a pair or window switch reuses them. Self-contained
+// models compiled in this process are kept here too. Cleared with the shared compiled-model cache when the last
+// backend is freed.
+static std::mutex g_imported_blobs_mutex;
+static std::unordered_map<uint64_t, ov::CompiledModel> g_imported_blobs;
+
+static ov::CompiledModel import_cached_blob(ov::Core & core,
+                                            const std::string & device,
+                                            const std::string & blob_path,
+                                            uint64_t fp,
+                                            const ov::AnyMap & import_config,
+                                            const char * tag) {
+    {
+        std::lock_guard<std::mutex> lock(g_imported_blobs_mutex);
+        auto it = g_imported_blobs.find(fp);
+        if (it != g_imported_blobs.end()) {
+            return it->second;
+        }
+    }
+    std::ifstream blob_in(blob_path, std::ios::binary);
+    const int64_t import_start_time = ggml_time_us();
+    ov::CompiledModel compiled = core.import_model(blob_in, device, import_config);
+    if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING")) {
+        fprintf(stderr, "ggml-openvino: model cache IMPORT %s: %.3f ms\n", tag,
+                (ggml_time_us() - import_start_time) / 1000.0);
+    }
+    std::lock_guard<std::mutex> lock(g_imported_blobs_mutex);
+    return g_imported_blobs.emplace(fp, std::move(compiled)).first->second;
+}
+
+static std::shared_ptr<ov::Model> convert_static_model(const static_build_args & a,
+                                                       const std::shared_ptr<GgmlOvDecoder> & decoder,
+                                                       const char * tag,
+                                                       int64_t & local_conversion_end_time) {
+    if (a.skip_weights) {
+        return nullptr;
+    }
+    auto input_model = std::make_shared<ov::frontend::ggml::InputModel>(decoder);
+    auto model = ov::frontend::ggml::FrontEnd::convert(input_model);
+    decoder->clear_model_weights();
+    local_conversion_end_time = ggml_time_us();
+
+    if (a.dump_ir) {
+        char timestamped_filename[64];
+        snprintf(timestamped_filename, sizeof(timestamped_filename), "model_%s_%lld.xml", tag, a.dump_ir_timestamp);
+        ov::serialize(model, timestamped_filename);
+    }
+    return model;
+}
+
+static void build_static_model(const static_build_args & a,
+                               std::shared_ptr<GgmlOvDecoder> decoder,
+                               const char * tag,
+                               std::shared_ptr<ov::Model> model,
+                               const ov::AnyMap & model_config,
+                               uint64_t model_fp,
+                               ov::CompiledModel & compiled_model,
+                               std::shared_ptr<ov::InferRequest> & infer_request,
+                               std::vector<std::string> & names_in,
+                               std::vector<std::string> & names_out,
+                               int64_t & local_conversion_end_time,
+                               int64_t & local_compile_end_time) {
+    auto & core = *a.core;
+    const auto & device = a.device;
+    const auto & model_cache_dir = a.model_cache_dir;
+    auto * cgraph = a.cgraph;
+    std::string blob_path;
+    std::string manifest_path;
+    if (!model_cache_dir.empty() && model_fp != 0) {
+        blob_path = ggml_openvino_model_cache_blob_path(model_cache_dir, model_fp);
+        manifest_path = ggml_openvino_model_cache_manifest_path(model_cache_dir, model_fp);
+    }
+
+    if (a.skip_weights) {
+        try {
+            ov::AnyMap import_config = model_config;
+            if (a.weightless_bank) {
+                import_config[ov::weights_path.name()] = a.weights_path;
+                import_config[ov::enable_weightless.name()] = true;
+            }
+            compiled_model = import_cached_blob(core, device, blob_path, model_fp, import_config, tag);
+        } catch (const std::exception & e) {
+            std::remove(blob_path.c_str());
+            std::remove(manifest_path.c_str());
+            throw std::runtime_error(std::string(a.weightless_bank ? "weightless-bank" : "self-contained") +
+                                     " cache import failed for " + std::string(tag) +
+                                     "; removed the invalid entry, rerun to rebuild it: " + e.what());
+        }
+        infer_request = std::make_shared<ov::InferRequest>(compiled_model.create_infer_request());
+        std::set<std::string> compiled_input_names;
+        for (const auto & input : compiled_model.inputs()) {
+            compiled_input_names.insert(input.get_node()->get_friendly_name());
+        }
+        for (const auto & [name, _] : decoder->get_model_inputs()) {
+            if (!ggml_openvino_npu_hfa_prefill_enabled() || std::string(tag) != "prefill" ||
+                compiled_input_names.count(name) != 0) {
+                names_in.push_back(name);
+            }
+        }
+        for (const auto & [name, info] : decoder->get_model_extra_inputs()) {
+            if (info.is_parameter) {
+                names_in.push_back(name);
+            }
+        }
+        for (const auto & name : decoder->get_model_output_names()) {
+            names_out.push_back(name);
+        }
+        if (names_in.size() != compiled_model.inputs().size() || names_out.size() != compiled_model.outputs().size()) {
+            throw std::runtime_error(
+                "self-contained cache port count mismatch for " + std::string(tag) +
+                ": decoder inputs=" + std::to_string(names_in.size()) +
+                ", compiled inputs=" + std::to_string(compiled_model.inputs().size()) +
+                ", decoder outputs=" + std::to_string(names_out.size()) +
+                ", compiled outputs=" + std::to_string(compiled_model.outputs().size()));
+        }
+        decoder->clear_model_weights();
+        local_conversion_end_time = local_compile_end_time = ggml_time_us();
+        GGML_LOG_INFO("ggml-openvino: model cache HIT (%s) %s (%s)\n",
+                      a.weightless_bank ? "weightless bank" : "self-contained", blob_path.c_str(), tag);
+        return;
+    }
+
+    bool imported = false;
+    // A self-contained blob embeds its weights bank, and NPUW cannot load a blob into a bank that already holds
+    // weights. Pairs that differ only in the prefill model share the decode model and its bank, so build each model
+    // once per process.
+    const bool keep_in_process = a.self_contained && !a.weightless_bank && model_fp != 0;
+    if (keep_in_process) {
+        std::lock_guard<std::mutex> lock(g_imported_blobs_mutex);
+        auto it = g_imported_blobs.find(model_fp);
+        if (it != g_imported_blobs.end()) {
+            compiled_model = it->second;
+            imported = true;
+        }
+    }
+    // A weightless-bank blob is only valid with the weights file it was exported with, which is rewritten whenever
+    // the blob is rebuilt, so never import a leftover blob here.
+    if (!imported && !model_cache_dir.empty() && model_fp != 0 && !a.weightless_bank) {
+        std::ifstream blob_in(blob_path, std::ios::binary);
+        std::vector<std::string> cached_inputs, cached_outputs;
+        if (blob_in.is_open() &&
+            ggml_openvino_model_cache_verify_manifest(manifest_path, cgraph, model_fp, cached_inputs, cached_outputs)) {
+            try {
+                ov::AnyMap import_config = model_config;
+                if (!a.self_contained) {
+                    import_config[ov::hint::model.name()] = std::shared_ptr<const ov::Model>(model);
+                }
+                const int64_t import_start_time = ggml_time_us();
+                compiled_model = core.import_model(blob_in, device, import_config);
+                const int64_t import_end_time = ggml_time_us();
+                if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING")) {
+                    fprintf(stderr, "ggml-openvino: model cache IMPORT %s: %.3f ms\n", tag,
+                            (import_end_time - import_start_time) / 1000.0);
+                }
+                imported = true;
+                GGML_LOG_INFO("ggml-openvino: model cache HIT %s (%s)\n", blob_path.c_str(), tag);
+            } catch (const std::exception & e) {
+                GGML_LOG_WARN("ggml-openvino: model cache import failed (%s), recompiling %s\n", e.what(), tag);
+                imported = false;
+            }
+        }
+    }
+
+    if (!imported) {
+        ov::AnyMap compile_config = model_config;
+        if (a.self_contained) {
+            compile_config[ov::enable_weightless.name()] = a.weightless_bank;
+        }
+        compiled_model = core.compile_model(model, device, compile_config);
+    }
+    if (keep_in_process) {
+        std::lock_guard<std::mutex> lock(g_imported_blobs_mutex);
+        g_imported_blobs.emplace(model_fp, compiled_model);
+    }
+    infer_request = std::make_shared<ov::InferRequest>(compiled_model.create_infer_request());
+    local_compile_end_time = ggml_time_us();
+
+    for (const auto & ov_param : model->get_parameters()) {
+        names_in.push_back(ov_param->get_friendly_name());
+    }
+    for (const auto & ov_output : model->get_results()) {
+        names_out.push_back(ov_output->get_friendly_name());
+    }
+
+    if (!imported && !model_cache_dir.empty() && model_fp != 0) {
+        try {
+            const std::string blob_tmp = blob_path + ".tmp";
+            const std::string manifest_tmp = manifest_path + ".tmp";
+            const std::string source_manifest_tmp = manifest_path + ".source.tmp";
+            const bool manifest_ready =
+                a.self_contained ?
+                    std::ifstream(source_manifest_tmp).good() :
+                    ggml_openvino_model_cache_write_manifest(manifest_tmp, cgraph, model_fp, names_in, names_out);
+            const std::string & manifest_publish = a.self_contained ? source_manifest_tmp : manifest_tmp;
+            if (manifest_ready) {
+                std::ofstream blob_out(blob_tmp, std::ios::binary | std::ios::trunc);
+                if (blob_out.is_open()) {
+                    compiled_model.export_model(blob_out);
+                    blob_out.close();
+                    // std::rename does not replace an existing file on Windows, which would keep a stale blob.
+                    if (blob_out.good()) {
+                        std::remove(manifest_path.c_str());
+                        std::remove(blob_path.c_str());
+                    }
+                    if (blob_out.good() && std::rename(blob_tmp.c_str(), blob_path.c_str()) == 0 &&
+                        std::rename(manifest_publish.c_str(), manifest_path.c_str()) == 0) {
+                        GGML_LOG_INFO("ggml-openvino: model cache WROTE %s (%s)\n", blob_path.c_str(), tag);
+                    } else {
+                        std::remove(blob_tmp.c_str());
+                        std::remove(manifest_publish.c_str());
+                    }
+                } else {
+                    std::remove(manifest_publish.c_str());
+                }
+            }
+        } catch (const std::exception & e) {
+            GGML_LOG_WARN("ggml-openvino: model cache export failed (%s): %s\n", tag, e.what());
         }
-        out.insert(kv);
     }
-    return out;
 }

 enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared_ptr<ov_runtime_context> & r_ctx) {
@@ -1421,7 +2089,7 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared

     // Normally NPU, but honors GGML_OPENVINO_DEVICE so GGML_OPENVINO_FORCE_STATIC can run the
     // static-shape path on CPU/GPU to isolate translation bugs from NPUW/NPU-driver issues.
-    static std::string device = ggml_openvino_get_device_name();
+    const auto & device = r_ctx->device;
     static auto is_static = true;
     static auto stateful = false;

@@ -1443,7 +2111,14 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
     const auto * inp_pos = get_inp_pos_tensor(cgraph);
     const bool no_kv_cache = m_params.is_cacheless_attn;
     const auto is_prefill = no_kv_cache ? true : get_is_prefill(cgraph, inp_pos);
-    const ov::AnyMap compile_config = no_kv_cache ? without_npuw(config) : config;
+    const int64_t input_tokens = get_inp_pos_n_tokens(cgraph, inp_pos);
+    if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING")) {
+        fprintf(stderr, "GGML OpenVINO profiling: static graph kind=%s, input_tokens=%lld, chunk_size=%d\n",
+                is_prefill ? "prefill" : "decode", static_cast<long long>(input_tokens), prefill_chunk_size);
+    }
+    const ov::AnyMap compile_config = no_kv_cache              ? ggml_openvino_npu_without_npuw(config) :
+                                      m_params.state_size >= 0 ? ggml_openvino_npu_gdn_config(config) :
+                                                                 config;
     if (m_params.n_heads_kv == -1) {
         prefill_chunk_size = inp_pos->ne[0];
     }
@@ -1451,6 +2126,30 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
     static const bool cache_enabled = !ggml_openvino_getenv_int("GGML_OPENVINO_DISABLE_CACHE");
     bool cache_hit = false;

+    // Decode KV window: smallest base*2^k covering the next decode position, clamped to ctx. Prefill
+    // calls pick the window the decode model of the compiled pair is built with.
+    const int decode_bucket = (!no_kv_cache && cache_enabled && inp_pos != nullptr && inp_pos->type == GGML_TYPE_I32) ?
+                                  ggml_openvino_npu_decode_bucket() :
+                                  0;
+    int decode_window = m_params.ctx_per_seq;
+    if (decode_bucket > 0 && m_params.ctx_per_seq > 0) {
+        const int last_pos = static_cast<const int32_t *>(inp_pos->data)[std::max<int64_t>(input_tokens, 1) - 1];
+        const int needed = last_pos + (is_prefill ? 2 : 1);
+        decode_window = decode_bucket;
+        while (decode_window < needed && decode_window < m_params.ctx_per_seq) {
+            decode_window *= 2;
+        }
+        decode_window = std::min(decode_window, m_params.ctx_per_seq);
+        if (ggml_openvino_npu_kv_head_major_enabled()) {
+            // Head-major KV is bound whole, so prefill and decode both use the window of the storage capacity.
+            decode_window = std::min(std::max(decode_window, kv_storage_rows(cgraph)), m_params.ctx_per_seq);
+            c_params.attention_size = decode_window;
+        }
+        if (!is_prefill) {
+            c_params.attention_size = decode_window;
+        }
+    }
+
     int64_t decoder_end_time;
     int64_t conversion_end_time;
     int64_t compile_end_time;
@@ -1480,16 +2179,27 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
     }

     std::lock_guard<std::mutex> lock(*(entry->mutex));
-    cache_hit = cache_hit && entry->ptr && r_ctx->infer_request_cache.count(key) != 0 &&
-                r_ctx->infer_request_cache_prefill.count(key) != 0;
+    cache_hit = cache_hit && entry->ptr;
+    if (cache_hit) {
+        cache_hit = r_ctx->infer_request_cache.count(key) != 0 &&
+                    r_ctx->infer_request_cache_prefill.count(key) != 0;
+    }

+    // Same model, only the prefill attention size differs from the entry's pair.
+    bool prefill_size_changed = false;
     if (cache_hit) {
         ggml_decoder = entry->ptr;
         old_m_params = ggml_decoder->get_model_params();
         cache_hit = old_m_params.can_reuse_statically(m_params);
+        if (cache_hit && is_prefill && ggml_openvino_kv_slice_enabled()) {
+            cache_hit = entry->prefill_attention_size == c_params.attention_size &&
+                        entry->prefill_attention_size_swa == c_params.attention_size_swa;
+            prefill_size_changed = !cache_hit;
+        }
     }

     std::vector<std::string> ov_input_names_local;
+    bool pair_built = false;
     std::vector<std::string> ov_output_names_local;

     if (cache_hit) {
@@ -1505,14 +2215,24 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
             std::lock_guard<std::mutex> map_lock(r_ctx->ctx_mutex);
             infer_request =
                 is_prefill ? r_ctx->infer_request_cache_prefill.at(key) : r_ctx->infer_request_cache.at(key);
-            ov_input_names_local = r_ctx->ov_input_names_cache.at(key);
-            ov_output_names_local = r_ctx->ov_output_names_cache.at(key);
+            ov_input_names_local = is_prefill ? r_ctx->ov_input_names_cache_prefill.at(key) :
+                                                r_ctx->ov_input_names_cache.at(key);
+            ov_output_names_local = is_prefill ? r_ctx->ov_output_names_cache_prefill.at(key) :
+                                                 r_ctx->ov_output_names_cache.at(key);
         }

         decoder_end_time = ggml_time_us();
         conversion_end_time = decoder_end_time;
         compile_end_time = decoder_end_time;
     } else {
+        // Same fail-fast as the dynamic path: once the host weight pages have been dropped
+        // they read as zeros, so a recompile here would bake garbage into the new model.
+        if (ggml_openvino_weight_buffers_released()) {
+            GGML_ABORT(
+                "ggml-openvino: a new graph needs to be compiled but host weight buffers were already "
+                "released via GGML_OPENVINO_RELEASE_WEIGHTS. This mode requires "
+                "stable graph shapes; disable host weight release for dynamic workloads.");
+        }
         if (cache_enabled) {
             std::lock_guard<std::mutex> map_lock(r_ctx->ctx_mutex);
             r_ctx->infer_request_cache.erase(key);
@@ -1526,6 +2246,7 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
         auto weight_names = get_weight_names(cgraph);
         auto local_decoder = std::make_shared<GgmlOvDecoder>(cgraph, m_params, c_params, weight_names, is_static,
                                                              stateful, false, is_prefill, prefill_chunk_size);
+        local_decoder->device = device;
         const std::string shared_key =
             cache_enabled ? compiled_graph_key(cgraph, *local_decoder, device, prefill_chunk_size) : "";
         auto shared_it = shared_cache->graphs.find(shared_key);
@@ -1537,109 +2258,692 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
                                       std::make_shared<ov::InferRequest>(compiled.decode.create_infer_request());
             ggml_decoder = local_decoder;
             entry->ptr = ggml_decoder;
+            entry->prefill_attention_size = c_params.attention_size;
+            entry->prefill_attention_size_swa = c_params.attention_size_swa;
+            entry->decode_attention_size = compiled.decode_attention_size;
+            entry->shared_key = shared_key;
+            entry->decode_window_requests.clear();
             infer_request = is_prefill ? prefill_request : decode_request;
-            ov_input_names_local = compiled.input_names;
-            ov_output_names_local = compiled.output_names;
+            ov_input_names_local = is_prefill ? compiled.prefill_input_names : compiled.input_names;
+            ov_output_names_local = is_prefill ? compiled.prefill_output_names : compiled.output_names;
             r_ctx->infer_request_cache_prefill[key] = prefill_request;
             r_ctx->infer_request_cache[key] = decode_request;
-            r_ctx->ov_input_names_cache[key] = ov_input_names_local;
-            r_ctx->ov_output_names_cache[key] = ov_output_names_local;
+            r_ctx->ov_input_names_cache[key] = compiled.input_names;
+            r_ctx->ov_output_names_cache[key] = compiled.output_names;
+            r_ctx->ov_input_names_cache_prefill[key] = compiled.prefill_input_names;
+            r_ctx->ov_output_names_cache_prefill[key] = compiled.prefill_output_names;
             decoder_end_time = conversion_end_time = compile_end_time = ggml_time_us();
             GGML_LOG_DEBUG("ggml-openvino: shared compiled model HIT (static)\n");
         } else {
-            std::shared_ptr<ov::Model> model;
-            auto model_weights = GgmlOvDecoder::create_weight_nodes(cgraph);
+            auto decode_c_params = c_params;
+            if (ggml_openvino_kv_slice_enabled()) {
+                decode_c_params.attention_size = m_params.ctx_per_seq;
+                decode_c_params.attention_size_swa = m_params.ctx_per_seq_swa;
+            }
+            if (decode_bucket > 0) {
+                decode_c_params.attention_size = decode_window;
+            }
+            const ov::AnyMap prefill_config = ggml_openvino_npu_prefill_config(compile_config, *local_decoder);
+            const ov::AnyMap decode_config = ggml_openvino_npu_decode_config(compile_config, device);
+
+            const std::string model_cache_dir = ggml_openvino_model_cache_dir();
+            const bool self_contained = device == "NPU" && !model_cache_dir.empty() &&
+                                        ggml_openvino_npu_raw_weight_cache_enabled();
+            // Weightless prefill/decode blobs plus one shared weights file. Needs NPUW, which cacheless graphs
+            // compile without, so those keep the embedded-weight blob.
+            const bool weightless_bank =
+                self_contained && !no_kv_cache && ggml_openvino_npu_weightless_bank_enabled();
+            // Another prefill attention size of the entry's model. The decode models (base and KV windows) do not
+            // depend on it, so the new pair takes them over from the entry's current pair, and that pair is dropped
+            // before the new prefill model compiles: each loaded prefill model keeps NPU memory for its inferences, and
+            // next to a second one the NPU can run out (pfnAppendGraphExecute: ZE_RESULT_ERROR_UNINITIALIZED). A pair
+            // another backend may use stays cached. Self-contained pairs are exported together, so they keep their own
+            // decode.
+            std::optional<ov_compiled_graph> decode_source;
+            if (prefill_size_changed && !self_contained && !no_kv_cache) {
+                auto source_it = shared_cache->graphs.find(entry->shared_key);
+                if (source_it != shared_cache->graphs.end()) {
+                    decode_source = source_it->second;
+                    decode_source->prefill = {};
+                    decode_c_params = decode_source->decode_c_params;
+                    // Make the KV window of the next decode step the base decode model, as a newly built pair has
+                    // it, so that step does not switch to a window request it has to create first.
+                    auto window_it = decode_source->decode_windows.find(decode_window);
+                    if (decode_bucket > 0 && window_it != decode_source->decode_windows.end()) {
+                        ov::CompiledModel window_model = window_it->second;
+                        decode_source->decode_windows.erase(window_it);
+                        decode_source->decode_windows.emplace(decode_c_params.attention_size, decode_source->decode);
+                        decode_source->decode = window_model;
+                        decode_c_params.attention_size = decode_window;
+                    }
+                    if (shared_cache->backend_count == 1) {
+                        shared_cache->graphs.erase(source_it);
+                    }
+                }
+            }
+            ov::AnyMap mc_config = prefill_config;
+            ov::AnyMap mc_decode_config = decode_config;
+            if (!model_cache_dir.empty()) {
+                mc_config.erase("CACHE_DIR");
+                mc_config.erase("CACHE_MODE");
+                mc_decode_config.erase("CACHE_DIR");
+                mc_decode_config.erase("CACHE_MODE");
+            }
+
+            uint64_t prefill_fp = 0;
+            uint64_t decode_fp = 0;
+            uint64_t base_extra = 0;
+            const bool decode_unfold = mc_decode_config.find("NPUW_UNFOLD_IREQS") != mc_decode_config.end();
+            if (!model_cache_dir.empty()) {
+                base_extra = ggml_openvino_model_cache_extra_cfg(stateful, /*recurrent=*/false);
+                base_extra = base_extra * 131 + (weightless_bank ? 3u : (self_contained ? 1u : 0u));
+                base_extra = ggml_openvino_model_cache_mix_string(
+                    base_extra, ggml_openvino_getenv_str("GGML_OPENVINO_NPU_TOKEN_EMBD_QUANT"));
+                base_extra = base_extra * 131 + (ggml_openvino_kv_slice_enabled() ? 1u : 0u);
+                base_extra = base_extra * 131 + (ggml_openvino_npu_kv_head_major_enabled() ? 1u : 0u);
+                base_extra = ggml_openvino_model_cache_mix_string(
+                    base_extra, ggml_openvino_getenv_str("GGML_OPENVINO_NPU_CONFIG"));
+                base_extra = ggml_openvino_model_cache_mix_string(
+                    base_extra, ggml_openvino_getenv_str("GGML_OPENVINO_NPU_COMPILE_CONFIG"));
+                base_extra = ggml_openvino_model_cache_mix_string(
+                    base_extra, ggml_openvino_getenv_str("GGML_OPENVINO_NPU_COMPILER_TYPE"));
+                base_extra = ggml_openvino_model_cache_mix_string(
+                    base_extra, ggml_openvino_getenv_str("GGML_OPENVINO_NPUW_FUNCALL_FOR_ALL"));
+                base_extra = ggml_openvino_model_cache_mix_string(
+                    base_extra, ggml_openvino_getenv_str("GGML_OPENVINO_NPUW_UNFOLD_IREQS"));
+                base_extra = ggml_openvino_model_cache_mix_string(
+                    base_extra, ggml_openvino_getenv_str("GGML_OPENVINO_COMPILATION_NUM_THREADS"));
+                // out_ids is bound at a fixed length, so the output count is part of the static shape
+                const std::string prefill_signature = "static-1 out=" + std::to_string(c_params.output_len);
+                const std::string decode_signature = "static-1 out=1";
+                prefill_fp = ggml_openvino_model_fingerprint(
+                    cgraph, device, true, m_params.rope_params, 16,
+                    static_model_salt(base_extra, true, prefill_chunk_size, c_params.attention_size,
+                                      c_params.attention_size_swa, false),
+                    prefill_signature);
+                decode_fp = ggml_openvino_model_fingerprint(
+                    cgraph, device, true, m_params.rope_params, 16,
+                    static_model_salt(base_extra, false, prefill_chunk_size, decode_c_params.attention_size,
+                                      decode_c_params.attention_size_swa, decode_unfold),
+                    decode_signature);
+            }
+
+            std::string weights_path;
+            std::string pair_weights_path;
+            std::string model_weights_path;
+            std::string pair_bank;
+            std::string model_bank;
+            if (weightless_bank) {
+                // One weights file and NPUW bank for every pair of this model (pairs differ only in KV window):
+                // NPUW matches weights by the LazyTensor hash recorded at compile time and keeps one resident copy.
+                // A pair whose weights differ from the model file falls back to its own file and bank.
+                const uint64_t weights_fp = ggml_openvino_model_fingerprint(cgraph, device, true, m_params.rope_params,
+                                                                            16, base_extra, "weights");
+                char model_bank_name[64];
+                char shared_bank[96];
+                snprintf(model_bank_name, sizeof(model_bank_name), "ggml-wl-model-%016llx",
+                         static_cast<unsigned long long>(weights_fp));
+                snprintf(shared_bank, sizeof(shared_bank), "ggml-wl-shared-%016llx-%016llx",
+                         static_cast<unsigned long long>(prefill_fp), static_cast<unsigned long long>(decode_fp));
+                pair_bank = shared_bank;
+                model_bank = model_bank_name;
+                pair_weights_path = ggml_openvino_model_cache_weights_path(model_cache_dir, prefill_fp, decode_fp);
+                model_weights_path = ggml_openvino_model_cache_weights_path(model_cache_dir, weights_fp, weights_fp);
+                // Only a pair exported with differing weights has its own file.
+                const bool pair_own = ggml_openvino_model_cache_weights_valid(pair_weights_path);
+                weights_path = pair_own ? pair_weights_path : model_weights_path;
+                mc_config["NPUW_WEIGHTS_BANK"] = pair_own ? pair_bank : model_bank;
+                mc_decode_config["NPUW_WEIGHTS_BANK"] = mc_config["NPUW_WEIGHTS_BANK"];
+            } else if (self_contained) {
+                char prefill_bank[64];
+                char decode_bank[64];
+                snprintf(prefill_bank, sizeof(prefill_bank), "ggml-sc-prefill-%016llx",
+                         static_cast<unsigned long long>(prefill_fp));
+                snprintf(decode_bank, sizeof(decode_bank), "ggml-sc-decode-%016llx",
+                         static_cast<unsigned long long>(decode_fp));
+                if (ggml_openvino_getenv_int("GGML_OPENVINO_NPU_SHARED_BANK")) {
+                    // Share ONE weights bank between prefill and decode. Both decoders are built
+                    // from the same model_weights map, so NPUW dedups them into a single in-process
+                    // bank; on warm import the shared blob's identical (uid,tensor) pairs are
+                    // content-verified and reused (patched NPUW weights_bank). Halves the resident
+                    // compiled-weight footprint (~2 GiB) at the cost of larger per-graph blobs.
+                    char shared_bank[96];
+                    snprintf(shared_bank, sizeof(shared_bank), "ggml-sc-shared-%016llx-%016llx",
+                             static_cast<unsigned long long>(prefill_fp),
+                             static_cast<unsigned long long>(decode_fp));
+                    mc_config["NPUW_WEIGHTS_BANK"] = std::string(shared_bank);
+                    mc_decode_config["NPUW_WEIGHTS_BANK"] = std::string(shared_bank);
+                } else {
+                    mc_config["NPUW_WEIGHTS_BANK"] = std::string(prefill_bank);
+                    mc_decode_config["NPUW_WEIGHTS_BANK"] = std::string(decode_bank);
+                }
+            }
+
+            auto blob_valid = [&](uint64_t fp) {
+                if (model_cache_dir.empty() || fp == 0) {
+                    return false;
+                }
+                std::ifstream blob_in(ggml_openvino_model_cache_blob_path(model_cache_dir, fp), std::ios::binary);
+                std::vector<std::string> inputs, outputs;  // ports are bound in frontend order, see below
+                return blob_in.is_open() &&
+                       ggml_openvino_model_cache_verify_manifest(
+                           ggml_openvino_model_cache_manifest_path(model_cache_dir, fp), cgraph, fp, inputs, outputs);
+            };
+            const bool self_contained_mmap =
+                self_contained && ggml_openvino_getenv_int("GGML_OPENVINO_SELF_CONTAINED_MMAP") != 0;
+            const bool skip_weights = self_contained && blob_valid(prefill_fp) && blob_valid(decode_fp) &&
+                                      (!weightless_bank || ggml_openvino_model_cache_weights_valid(weights_path));
+            if (self_contained_mmap && !skip_weights) {
+                throw std::runtime_error(
+                    "GGML_OPENVINO_SELF_CONTAINED_MMAP requires valid self-contained blob(s); "
+                    "run once with GGML_OPENVINO_SELF_CONTAINED_MMAP=0 to build the cache");
+            }
+
+            if (self_contained && !skip_weights && !model_cache_dir.empty()) {
+                if (prefill_fp != 0) {
+                    ggml_openvino_model_cache_write_manifest(
+                        ggml_openvino_model_cache_manifest_path(model_cache_dir, prefill_fp) + ".source.tmp",
+                        cgraph, prefill_fp, {}, {});
+                }
+                if (decode_fp != 0) {
+                    ggml_openvino_model_cache_write_manifest(
+                        ggml_openvino_model_cache_manifest_path(model_cache_dir, decode_fp) + ".source.tmp",
+                        cgraph, decode_fp, {}, {});
+                }
+            }
+
+            std::map<std::string, std::shared_ptr<ov::Node>> model_weights;
+            if (skip_weights) {
+                for (const auto & name : GgmlOvDecoder::collect_weight_names(cgraph)) {
+                    model_weights[name] = nullptr;
+                }
+            } else {
+                model_weights = GgmlOvDecoder::create_weight_nodes(cgraph);
+            }

             auto ggml_decoder_prefill = std::make_shared<GgmlOvDecoder>(
                 cgraph, m_params, c_params, model_weights, is_static, stateful, false, true, prefill_chunk_size);
             auto ggml_decoder_decode =
                 no_kv_cache ? ggml_decoder_prefill :
-                              std::make_shared<GgmlOvDecoder>(cgraph, m_params, c_params, model_weights, is_static,
-                                                              stateful, false, false, prefill_chunk_size);
+                              std::make_shared<GgmlOvDecoder>(cgraph, m_params, decode_c_params, model_weights,
+                                                              is_static, stateful, false, false, prefill_chunk_size);
+            ggml_decoder_prefill->device = device;
+            ggml_decoder_decode->device = device;
             decoder_end_time = ggml_time_us();

             const bool dump_ir = ggml_openvino_getenv_int("GGML_OPENVINO_DUMP_IR");
             const auto dump_ir_timestamp = static_cast<long long>(ggml_time_us());

-            auto build_static_model = [&core, &compile_config, dump_ir, dump_ir_timestamp](
-                                          const std::shared_ptr<GgmlOvDecoder> & decoder, const char * tag,
-                                          std::shared_ptr<ov::Model> & model, ov::CompiledModel & compiled_model,
-                                          std::shared_ptr<ov::InferRequest> & infer_request,
-                                          int64_t & local_conversion_end_time, int64_t & local_compile_end_time) {
-                auto input_model = std::make_shared<ov::frontend::ggml::InputModel>(decoder);
-                model = ov::frontend::ggml::FrontEnd::convert(input_model);
-                decoder->clear_model_weights();
-                local_conversion_end_time = ggml_time_us();
-
-                if (dump_ir) {
-                    char timestamped_filename[64];
-                    snprintf(timestamped_filename, sizeof(timestamped_filename), "model_%s_%lld.xml", tag,
-                             dump_ir_timestamp);
-                    ov::serialize(model, timestamped_filename);
-                }
-
-                compiled_model = core.compile_model(model, device, compile_config);
-                infer_request = std::make_shared<ov::InferRequest>(compiled_model.create_infer_request());
-                local_compile_end_time = ggml_time_us();
+            // Both decoders share the weight nodes, and converting attaches each new graph to them. ov::Node is not
+            // thread-safe, so the models are always converted one at a time; only compilation can run in parallel.
+            const static_build_args build_args{&core,           device,         dump_ir,        dump_ir_timestamp,
+                                               model_cache_dir, cgraph,         self_contained, skip_weights,
+                                               weightless_bank, weights_path};
+            auto convert_static_model = [&build_args](std::shared_ptr<GgmlOvDecoder> decoder, const char * tag,
+                                                      int64_t & local_conversion_end_time) {
+                return ::convert_static_model(build_args, decoder, tag, local_conversion_end_time);
+            };
+            auto build_static_model = [&build_args](std::shared_ptr<GgmlOvDecoder> decoder,
+                                                    const char * tag,
+                                                    std::shared_ptr<ov::Model> model,
+                                                    const ov::AnyMap & model_config,
+                                                    uint64_t model_fp,
+                                                    ov::CompiledModel & compiled_model,
+                                                    std::shared_ptr<ov::InferRequest> & infer_request,
+                                                    std::vector<std::string> & names_in,
+                                                    std::vector<std::string> & names_out,
+                                                    int64_t & local_conversion_end_time,
+                                                    int64_t & local_compile_end_time) {
+                ::build_static_model(build_args, std::move(decoder), tag, std::move(model), model_config, model_fp,
+                                     compiled_model, infer_request, names_in, names_out, local_conversion_end_time,
+                                     local_compile_end_time);
             };
-            std::shared_ptr<ov::Model> model_prefill;
-            std::shared_ptr<ov::Model> model_decode;
             ov::CompiledModel compiled_model_prefill;
             ov::CompiledModel compiled_model_decode;
             std::shared_ptr<ov::InferRequest> infer_request_prefill;
             std::shared_ptr<ov::InferRequest> infer_request_decode;
-            int64_t prefill_conversion_end_time;
-            int64_t decode_conversion_end_time;
-            int64_t prefill_compile_end_time;
-            int64_t decode_compile_end_time;
-            build_static_model(ggml_decoder_prefill, "prefill", model_prefill, compiled_model_prefill,
-                               infer_request_prefill, prefill_conversion_end_time, prefill_compile_end_time);
-            if (no_kv_cache) {
-                model_decode = model_prefill;
-                compiled_model_decode = compiled_model_prefill;
-                infer_request_decode = infer_request_prefill;
-                decode_conversion_end_time = prefill_conversion_end_time;
-                decode_compile_end_time = prefill_compile_end_time;
+            std::vector<std::string> names_in_prefill, names_out_prefill;
+            std::vector<std::string> names_in_decode, names_out_decode;
+            int64_t prefill_conversion_end_time = 0;
+            int64_t decode_conversion_end_time = 0;
+            int64_t prefill_compile_end_time = 0;
+            int64_t decode_compile_end_time = 0;
+
+            // Convert decode before compiling prefill: a decode model converted after the prefill compile gives
+            // wrong output on some NPU models (Bonsai-8B). Compiling both in parallel raises the host memory peak,
+            // so it is opt-in.
+            const bool parallel_compile =
+                !self_contained && !no_kv_cache && !decode_source && ggml_openvino_npu_parallel_compile_enabled();
+            auto model_prefill = convert_static_model(ggml_decoder_prefill, "prefill", prefill_conversion_end_time);
+            std::shared_ptr<ov::Model> model_decode;
+            if (!no_kv_cache && !decode_source) {
+                model_decode = convert_static_model(ggml_decoder_decode, "decode", decode_conversion_end_time);
+            }
+            std::shared_ptr<ggml_openvino_weights_layout> weights_layout;
+            if (weightless_bank && !skip_weights) {
+                // Lay both models' weights out in one file before compiling, so both blobs reference the same file.
+                // Old blobs reference the old layout, so drop them first in case the export does not complete.
+                for (const uint64_t fp : { prefill_fp, decode_fp }) {
+                    std::remove(ggml_openvino_model_cache_blob_path(model_cache_dir, fp).c_str());
+                    std::remove(ggml_openvino_model_cache_manifest_path(model_cache_dir, fp).c_str());
+                    // A decode window built earlier on its own may have left a weights file for this fingerprint.
+                    ggml_openvino_model_cache_remove_weights(
+                        ggml_openvino_model_cache_weights_path(model_cache_dir, fp, fp));
+                }
+                ggml_openvino_model_cache_remove_weights(pair_weights_path);
+                ggml_openvino_weights_status status = ggml_openvino_weights_status::written;
+                weights_path = model_weights_path;
+                mc_config["NPUW_WEIGHTS_BANK"] = model_bank;
+                mc_decode_config["NPUW_WEIGHTS_BANK"] = model_bank;
+                weights_layout = ggml_openvino_model_cache_write_weights(weights_path, {model_prefill, model_decode},
+                                                                         /*keep_existing=*/true, &status);
+                if (status == ggml_openvino_weights_status::mismatch) {
+                    GGML_LOG_WARN("ggml-openvino: weights differ from %s, using a per-pair weights file\n",
+                                  model_weights_path.c_str());
+                    weights_path = pair_weights_path;
+                    mc_config["NPUW_WEIGHTS_BANK"] = pair_bank;
+                    mc_decode_config["NPUW_WEIGHTS_BANK"] = pair_bank;
+                    weights_layout =
+                        ggml_openvino_model_cache_write_weights(weights_path, {model_prefill, model_decode});
+                }
+                if (!weights_layout) {
+                    throw std::runtime_error("ggml-openvino: failed to write NPU weights file " + weights_path);
+                }
+            }
+            if (parallel_compile) {
+                auto prefill_future = std::async(std::launch::async, build_static_model, ggml_decoder_prefill, "prefill",
+                                                 std::move(model_prefill), std::cref(mc_config), prefill_fp,
+                                                 std::ref(compiled_model_prefill), std::ref(infer_request_prefill),
+                                                 std::ref(names_in_prefill), std::ref(names_out_prefill),
+                                                 std::ref(prefill_conversion_end_time),
+                                                 std::ref(prefill_compile_end_time));
+                auto decode_future = std::async(std::launch::async, build_static_model, ggml_decoder_decode, "decode",
+                                                std::move(model_decode), std::cref(mc_decode_config), decode_fp,
+                                                std::ref(compiled_model_decode), std::ref(infer_request_decode),
+                                                std::ref(names_in_decode), std::ref(names_out_decode),
+                                                std::ref(decode_conversion_end_time),
+                                                std::ref(decode_compile_end_time));
+                prefill_future.get();
+                decode_future.get();
             } else {
-                build_static_model(ggml_decoder_decode, "decode", model_decode, compiled_model_decode,
-                                   infer_request_decode, decode_conversion_end_time, decode_compile_end_time);
+                build_static_model(ggml_decoder_prefill, "prefill", std::move(model_prefill), mc_config, prefill_fp,
+                                   compiled_model_prefill, infer_request_prefill, names_in_prefill, names_out_prefill,
+                                   prefill_conversion_end_time, prefill_compile_end_time);
+                if (no_kv_cache) {
+                    compiled_model_decode = compiled_model_prefill;
+                    infer_request_decode = infer_request_prefill;
+                    names_in_decode = names_in_prefill;
+                    names_out_decode = names_out_prefill;
+                    decode_conversion_end_time = prefill_conversion_end_time;
+                    decode_compile_end_time = prefill_compile_end_time;
+                } else if (decode_source) {
+                    compiled_model_decode = decode_source->decode;
+                    infer_request_decode =
+                        std::make_shared<ov::InferRequest>(compiled_model_decode.create_infer_request());
+                    names_in_decode = decode_source->input_names;
+                    names_out_decode = decode_source->output_names;
+                    decode_conversion_end_time = prefill_conversion_end_time;
+                    decode_compile_end_time = prefill_compile_end_time;
+                } else {
+                    build_static_model(ggml_decoder_decode, "decode", std::move(model_decode), mc_decode_config,
+                                       decode_fp, compiled_model_decode, infer_request_decode, names_in_decode,
+                                       names_out_decode, decode_conversion_end_time, decode_compile_end_time);
+                }
             }
             conversion_end_time = std::max(prefill_conversion_end_time, decode_conversion_end_time);
             compile_end_time = std::max(prefill_compile_end_time, decode_compile_end_time);

-            model = is_prefill ? model_prefill : model_decode;
             ggml_decoder = is_prefill ? ggml_decoder_prefill : ggml_decoder_decode;
             infer_request = is_prefill ? infer_request_prefill : infer_request_decode;
+            ov_input_names_local = is_prefill ? names_in_prefill : names_in_decode;
+            ov_output_names_local = is_prefill ? names_out_prefill : names_out_decode;
             entry->ptr = ggml_decoder;
-
-            for (const auto & ov_param : model->get_parameters()) {
-                ov_input_names_local.push_back(ov_param->get_friendly_name());
-            }
-            for (const auto & ov_output : model->get_results()) {
-                ov_output_names_local.push_back(ov_output->get_friendly_name());
-            }
+            entry->prefill_attention_size = c_params.attention_size;
+            entry->prefill_attention_size_swa = c_params.attention_size_swa;
+            entry->decode_attention_size = decode_c_params.attention_size;
+            entry->shared_key = shared_key;
+            entry->decode_window_requests.clear();

             if (!shared_key.empty()) {
-                shared_cache->graphs.emplace(
-                    shared_key, ov_compiled_graph{compiled_model_decode, compiled_model_prefill, ov_input_names_local,
-                                                  ov_output_names_local});
+                ov_compiled_graph graph;
+                graph.decode = compiled_model_decode;
+                graph.prefill = compiled_model_prefill;
+                graph.input_names = names_in_decode;
+                graph.output_names = names_out_decode;
+                graph.prefill_input_names = names_in_prefill;
+                graph.prefill_output_names = names_out_prefill;
+                graph.decode_attention_size = decode_c_params.attention_size;
+                if (decode_source) {
+                    graph.decode_windows = decode_source->decode_windows;
+                }
+                graph.decode_config = mc_decode_config;
+                graph.cache_base_extra = base_extra;
+                graph.self_contained = self_contained;
+                graph.decode_unfold = decode_unfold;
+                graph.weightless_bank = weightless_bank;
+                graph.weights_path = weights_path;
+                graph.weights_layout = decode_bucket > 0 ? std::move(weights_layout) : nullptr;
+                graph.decode_c_params = decode_c_params;
+                shared_cache->graphs.emplace(shared_key, std::move(graph));
+                pair_built = !decode_source;
             }

             if (cache_enabled) {
                 std::lock_guard<std::mutex> map_lock(r_ctx->ctx_mutex);
                 r_ctx->infer_request_cache_prefill[key] = infer_request_prefill;
                 r_ctx->infer_request_cache[key] = infer_request_decode;
-                r_ctx->ov_input_names_cache[key] = ov_input_names_local;
-                r_ctx->ov_output_names_cache[key] = ov_output_names_local;
+                r_ctx->ov_input_names_cache[key] = names_in_decode;
+                r_ctx->ov_output_names_cache[key] = names_out_decode;
+                r_ctx->ov_input_names_cache_prefill[key] = names_in_prefill;
+                r_ctx->ov_output_names_cache_prefill[key] = names_out_prefill;
             }
         }
     }

-    reset_single_slot_recurrent_cache(cgraph, m_params, c_params);
+    // Decode model for another KV window than the pair's; built/imported once per window with the pair's
+    // settings and weights bank, then cached in the shared compiled cache. Caller holds the shared cache mutex.
+    auto build_decode_window = [&](ov_compiled_graph & graph, const ComputeParams & base_c_params,
+                                   int window) -> ov::CompiledModel & {
+        auto window_it = graph.decode_windows.find(window);
+        if (window_it != graph.decode_windows.end()) {
+            return window_it->second;
+        }
+        auto window_c_params = base_c_params;
+        window_c_params.attention_size = window;
+        const std::string model_cache_dir = ggml_openvino_model_cache_dir();
+        const uint64_t fp = model_cache_dir.empty() ?
+                                0 :
+                                ggml_openvino_model_fingerprint(
+                                    cgraph, device, true, m_params.rope_params, 16,
+                                    static_model_salt(graph.cache_base_extra, false, prefill_chunk_size, window,
+                                                      window_c_params.attention_size_swa, graph.decode_unfold),
+                                    "static-1 out=1");
+        if (fp != 0) {
+            std::lock_guard<std::mutex> blobs_lock(g_imported_blobs_mutex);
+            auto cached = g_imported_blobs.find(fp);
+            if (cached != g_imported_blobs.end()) {
+                return graph.decode_windows.emplace(window, cached->second).first->second;
+            }
+        }
+        // Weightless bank: a window compiled with the pair resolves the shared weights through the pair's file
+        // (the rest is embedded in its blob); otherwise it gets its own weights file.
+        const std::string own_weights_path =
+            graph.weightless_bank ? ggml_openvino_model_cache_weights_path(model_cache_dir, fp, fp) : "";
+        const bool own_weights_valid =
+            graph.weightless_bank && ggml_openvino_model_cache_weights_valid(own_weights_path);
+        // A window blob resolved through the pair's file must not predate it (the pair may have been re-exported).
+        auto pair_weights_usable = [&]() {
+            std::error_code ec1, ec2;
+            const auto blob_time = std::filesystem::last_write_time(
+                std::filesystem::u8path(ggml_openvino_model_cache_blob_path(model_cache_dir, fp)), ec1);
+            const auto weights_time =
+                std::filesystem::last_write_time(std::filesystem::u8path(graph.weights_path), ec2);
+            return !ec1 && !ec2 && blob_time >= weights_time &&
+                   ggml_openvino_model_cache_weights_valid(graph.weights_path);
+        };
+        const bool skip_weights = graph.self_contained && static_blob_valid(model_cache_dir, fp, cgraph) &&
+                                  (!graph.weightless_bank || own_weights_valid || pair_weights_usable());
+        if (graph.self_contained && !skip_weights &&
+            ggml_openvino_getenv_int("GGML_OPENVINO_SELF_CONTAINED_MMAP") != 0) {
+            throw std::runtime_error("GGML_OPENVINO_SELF_CONTAINED_MMAP: no cached decode model for KV window " +
+                                     std::to_string(window) + "; rerun with GGML_OPENVINO_SELF_CONTAINED_MMAP=0");
+        }
+        std::map<std::string, std::shared_ptr<ov::Node>> model_weights;
+        if (skip_weights) {
+            for (const auto & name : GgmlOvDecoder::collect_weight_names(cgraph)) {
+                model_weights[name] = nullptr;
+            }
+        } else {
+            if (graph.self_contained && fp != 0) {
+                ggml_openvino_model_cache_write_manifest(
+                    ggml_openvino_model_cache_manifest_path(model_cache_dir, fp) + ".source.tmp", cgraph, fp, {}, {});
+            }
+            model_weights = GgmlOvDecoder::create_weight_nodes(cgraph);
+        }
+        auto window_decoder = std::make_shared<GgmlOvDecoder>(cgraph, m_params, window_c_params, model_weights,
+                                                              is_static, stateful, false, false, prefill_chunk_size);
+        window_decoder->device = device;
+        static_build_args args{&core,  device,       false, 0, model_cache_dir, cgraph, graph.self_contained,
+                               skip_weights, graph.weightless_bank, {}};
+        ov::CompiledModel compiled;
+        std::shared_ptr<ov::InferRequest> built_request;
+        std::vector<std::string> names_in, names_out;
+        int64_t t_conv = 0, t_compile = 0;
+        const int64_t t0 = ggml_time_us();
+        auto model = convert_static_model(args, window_decoder, "decode", t_conv);
+        if (graph.weightless_bank) {
+            args.weights_path = own_weights_valid ? own_weights_path : graph.weights_path;
+            if (model && graph.weights_layout) {
+                const size_t resolved = ggml_openvino_model_cache_retag_weights(model, *graph.weights_layout);
+                GGML_LOG_INFO("ggml-openvino: decode KV window %d resolves %zu weights through %s\n", window,
+                              resolved, graph.weights_path.c_str());
+                ggml_openvino_model_cache_remove_weights(own_weights_path);
+                args.weights_path = graph.weights_path;
+            } else if (model) {
+                if (!ggml_openvino_model_cache_write_weights(own_weights_path, { model })) {
+                    throw std::runtime_error("ggml-openvino: failed to write NPU weights file " + own_weights_path);
+                }
+                args.weights_path = own_weights_path;
+            }
+        }
+        build_static_model(args, window_decoder, "decode", std::move(model), graph.decode_config, fp, compiled,
+                           built_request, names_in, names_out, t_conv, t_compile);
+        if (names_in != graph.input_names || names_out != graph.output_names) {
+            throw std::runtime_error("ggml-openvino: decode window model ports differ from the base decode model");
+        }
+        GGML_LOG_INFO("ggml-openvino: decode KV window %d model ready in %.1f ms\n", window,
+                      (ggml_time_us() - t0) / 1000.0);
+        return graph.decode_windows.emplace(window, std::move(compiled)).first->second;
+    };
+
+    auto get_decode_window_request = [&](int window) -> std::shared_ptr<ov::InferRequest> {
+        auto & request = entry->decode_window_requests[window];
+        if (request) {
+            return request;
+        }
+        auto shared_cache = r_ctx->compiled_cache;
+        std::unique_lock<std::mutex> compile_lock(shared_cache->mutex);
+        auto graph_it = shared_cache->graphs.find(entry->shared_key);
+        if (graph_it == shared_cache->graphs.end()) {
+            throw std::runtime_error("ggml-openvino: decode window requested without a shared compiled graph");
+        }
+        auto & compiled = build_decode_window(graph_it->second, c_params, window);
+        request = std::make_shared<ov::InferRequest>(compiled.create_infer_request());
+        return request;
+    };
+
+    // Build or import every other decode window right after the pair, from the pair's own graph and params,
+    // so a growing sequence only switches requests.
+    if (pair_built && decode_bucket > 0) {
+        auto shared_cache = r_ctx->compiled_cache;
+        std::unique_lock<std::mutex> compile_lock(shared_cache->mutex);
+        auto graph_it = shared_cache->graphs.find(entry->shared_key);
+        if (graph_it != shared_cache->graphs.end()) {
+            auto & graph = graph_it->second;
+            const int64_t t0 = ggml_time_us();
+            for (int window = decode_bucket;; window *= 2) {
+                window = std::min(window, m_params.ctx_per_seq);
+                if (window != graph.decode_attention_size) {
+                    build_decode_window(graph, graph.decode_c_params, window);
+                }
+                if (window >= m_params.ctx_per_seq) {
+                    break;
+                }
+            }
+            GGML_LOG_INFO("ggml-openvino: prebuilt %zu decode KV window model(s) in %.1f ms\n",
+                          graph.decode_windows.size(), (ggml_time_us() - t0) / 1000.0);
+
+            // With head-major KV a prefill also runs at a KV window, picked by prompt length. Export the prefill
+            // model of every other window too, so an import-only run has a pair for any prompt length.
+            const std::string model_cache_dir = ggml_openvino_model_cache_dir();
+            if (graph.weightless_bank && graph.weights_layout && ggml_openvino_npu_kv_head_major_enabled() &&
+                ggml_openvino_getenv_int("GGML_OPENVINO_SELF_CONTAINED_MMAP") == 0) {
+                const int64_t t1 = ggml_time_us();
+                size_t exported = 0;
+                for (int window = decode_bucket;; window *= 2) {
+                    window = std::min(window, m_params.ctx_per_seq);
+                    if (window != entry->prefill_attention_size) {
+                        auto window_c_params = c_params;
+                        window_c_params.attention_size = window;
+                        const uint64_t fp = ggml_openvino_model_fingerprint(
+                            cgraph, device, true, m_params.rope_params, 16,
+                            static_model_salt(graph.cache_base_extra, true, prefill_chunk_size, window,
+                                              window_c_params.attention_size_swa, false),
+                            "static-1 out=" + std::to_string(window_c_params.output_len));
+                        std::error_code ec1, ec2;
+                        const auto blob_time = std::filesystem::last_write_time(
+                            std::filesystem::u8path(ggml_openvino_model_cache_blob_path(model_cache_dir, fp)), ec1);
+                        const auto weights_time =
+                            std::filesystem::last_write_time(std::filesystem::u8path(graph.weights_path), ec2);
+                        const bool cached =
+                            !ec1 && !ec2 && blob_time >= weights_time && static_blob_valid(model_cache_dir, fp, cgraph);
+                        if (!cached) {
+                            ggml_openvino_model_cache_write_manifest(
+                                ggml_openvino_model_cache_manifest_path(model_cache_dir, fp) + ".source.tmp", cgraph,
+                                fp, {}, {});
+                            auto model_weights = GgmlOvDecoder::create_weight_nodes(cgraph);
+                            auto window_decoder =
+                                std::make_shared<GgmlOvDecoder>(cgraph, m_params, window_c_params, model_weights,
+                                                                is_static, stateful, false, true, prefill_chunk_size);
+                            ov::AnyMap window_config =
+                                ggml_openvino_npu_prefill_config(compile_config, *window_decoder);
+                            window_config.erase("CACHE_DIR");
+                            window_config.erase("CACHE_MODE");
+                            window_config["NPUW_WEIGHTS_BANK"] = graph.decode_config.at("NPUW_WEIGHTS_BANK");
+                            static_build_args args{&core,  device, false, 0,    model_cache_dir,
+                                                   cgraph, true,   false, true, graph.weights_path};
+                            int64_t t_conv = 0, t_compile = 0;
+                            auto model = convert_static_model(args, window_decoder, "prefill", t_conv);
+                            ggml_openvino_model_cache_retag_weights(model, *graph.weights_layout);
+                            ov::CompiledModel compiled;
+                            std::shared_ptr<ov::InferRequest> built_request;
+                            std::vector<std::string> names_in, names_out;
+                            build_static_model(args, window_decoder, "prefill", std::move(model), window_config, fp,
+                                               compiled, built_request, names_in, names_out, t_conv, t_compile);
+                            ++exported;
+                        }
+                    }
+                    if (window >= m_params.ctx_per_seq) {
+                        break;
+                    }
+                }
+                GGML_LOG_INFO("ggml-openvino: exported %zu prefill KV window model(s) in %.1f ms\n", exported,
+                              (ggml_time_us() - t1) / 1000.0);
+            }
+
+            // Import the prefill model of every other window now, so a prompt that needs another window builds its
+            // pair from already imported models instead of importing them mid-request.
+            if (graph.weightless_bank && ggml_openvino_npu_kv_head_major_enabled() &&
+                ggml_openvino_model_cache_weights_valid(graph.weights_path)) {
+                const int64_t t2 = ggml_time_us();
+                size_t imported = 0;
+                for (int window = decode_bucket;; window *= 2) {
+                    window = std::min(window, m_params.ctx_per_seq);
+                    if (window != entry->prefill_attention_size) {
+                        auto window_c_params = c_params;
+                        window_c_params.attention_size = window;
+                        const uint64_t fp = ggml_openvino_model_fingerprint(
+                            cgraph, device, true, m_params.rope_params, 16,
+                            static_model_salt(graph.cache_base_extra, true, prefill_chunk_size, window,
+                                              window_c_params.attention_size_swa, false),
+                            "static-1 out=" + std::to_string(window_c_params.output_len));
+                        bool already_imported;
+                        {
+                            std::lock_guard<std::mutex> blobs_lock(g_imported_blobs_mutex);
+                            already_imported = g_imported_blobs.count(fp) != 0;
+                        }
+                        if (!already_imported && static_blob_valid(model_cache_dir, fp, cgraph)) {
+                            std::map<std::string, std::shared_ptr<ov::Node>> weight_names;
+                            for (const auto & name : GgmlOvDecoder::collect_weight_names(cgraph)) {
+                                weight_names[name] = nullptr;
+                            }
+                            GgmlOvDecoder window_decoder(cgraph, m_params, window_c_params, weight_names, is_static,
+                                                         stateful, false, true, prefill_chunk_size);
+                            ov::AnyMap import_config = ggml_openvino_npu_prefill_config(compile_config, window_decoder);
+                            import_config.erase("CACHE_DIR");
+                            import_config.erase("CACHE_MODE");
+                            import_config["NPUW_WEIGHTS_BANK"] = graph.decode_config.at("NPUW_WEIGHTS_BANK");
+                            import_config[ov::weights_path.name()] = graph.weights_path;
+                            import_config[ov::enable_weightless.name()] = true;
+                            const std::string blob_path = ggml_openvino_model_cache_blob_path(model_cache_dir, fp);
+                            try {
+                                import_cached_blob(core, device, blob_path, fp, import_config, "prefill");
+                                ++imported;
+                            } catch (const std::exception & e) {
+                                // Drop the invalid entry, as the pair import does; the window is then built
+                                // when a prompt first needs it.
+                                std::remove(blob_path.c_str());
+                                std::remove(ggml_openvino_model_cache_manifest_path(model_cache_dir, fp).c_str());
+                                GGML_LOG_WARN(
+                                    "ggml-openvino: removed the invalid cached prefill model of KV window %d: %s\n",
+                                    window, e.what());
+                            }
+                        }
+                    }
+                    if (window >= m_params.ctx_per_seq) {
+                        break;
+                    }
+                }
+                GGML_LOG_INFO("ggml-openvino: preloaded %zu prefill KV window model(s) in %.1f ms\n", imported,
+                              (ggml_time_us() - t2) / 1000.0);
+            }
+        }
+    }
+
+    // On-demand KV: grow the storage to this call's KV window before binding, so every view of the call
+    // (KV inputs, HFA past, KV outputs) addresses the same storage.
+    bool kv_grew = false;
+    size_t kv_rows = 0;
+    if (ggml_openvino_npu_kv_on_demand_enabled()) {
+        for (const auto & input_name : ov_input_names_local) {
+            // A transposed-V HFA prefill only takes V through its past input; resolve it to the cache input.
+            std::string name = input_name;
+            if (input_name.rfind("ggml_hfa_past_", 0) == 0) {
+                const std::string cache_name = input_name.substr(strlen("ggml_hfa_past_"));
+                for (const auto & [model_input, _] : ggml_decoder->get_model_inputs()) {
+                    if (model_input.substr(0, model_input.find('#')) == cache_name) {
+                        name = model_input;
+                        break;
+                    }
+                }
+            }
+            if (name.rfind("cache_k_l", 0) == 0 || name.rfind("cache_v_l", 0) == 0) {
+                const auto * kv_tensor = ggml_decoder->get_input_ggml_tensor(name);
+                size_t heads =
+                    ggml_openvino_npu_kv_head_major_enabled() ? ggml_decoder->get_n_heads_kv_for_tensor(kv_tensor) : 0;
+                if (heads > 0 && name.rfind("cache_v_l", 0) == 0) {
+                    // [heads][head_size][rows] is head-major storage with one "head" per channel.
+                    heads = static_cast<size_t>(kv_tensor->ne[0]);
+                }
+                kv_rows = get_static_attention_size(ggml_decoder, name);
+                kv_grew |= ggml_openvino_kv_storage_reserve(kv_tensor->data, kv_rows, heads);
+            }
+        }
+    }

     if (is_prefill) {
         auto inp_len = get_inp_pos_n_tokens(cgraph, inp_pos);
+        // Transposed-V HFA prefill outputs each chunk's V^T rows (see set_rows); they are copied into the cache here.
+        const bool vt_rows = ggml_openvino_npu_kv_head_major_enabled() && ggml_openvino_npu_hfa_prefill_enabled();
+        auto is_vt_rows_output = [&](const std::string & name) {
+            if (!vt_rows || name.rfind("cache_v_l", 0) != 0) {
+                return false;
+            }
+            const auto layer = extract_layer_from_name(name);
+            return layer.has_value() && !ggml_decoder->is_swa_layer(*layer);
+        };
+        std::map<const ggml_tensor *, const ggml_tensor *> set_rows_indices;
+        if (vt_rows) {
+            for (int n = 0; n < cgraph->n_nodes; n++) {
+                const auto * node = cgraph->nodes[n];
+                if (node->op == GGML_OP_SET_ROWS && node->view_src) {
+                    set_rows_indices[node->view_src] = node->src[1];
+                }
+            }
+        }
         for (int chunk_index = 0; chunk_index * prefill_chunk_size < inp_len; chunk_index++) {
+            const int64_t chunk_start = static_cast<int64_t>(chunk_index) * prefill_chunk_size;
+            const int64_t chunk_valid_len = std::min<int64_t>(prefill_chunk_size, inp_len - chunk_start);
             for (size_t i = 0; i < ov_input_names_local.size(); i++) {
                 const auto & param_name = ov_input_names_local[i];
                 auto input_tensor = get_ov_input_tensor_static_prefill(ggml_decoder, param_name, chunk_index);
@@ -1663,24 +2967,68 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
                     // Result is the full cache, so binding it over this 0-byte buffer overflows it.
                     continue;
                 }
+                if (is_vt_rows_output(ov_output_names_local[i])) {
+                    continue;
+                }
                 auto output_tensor = create_ov_output_tensor(ggml_decoder, infer_request, i, ggml_tensor);
                 infer_request->set_output_tensor(i, output_tensor);
             }

             ov_raw_infer_start = ggml_time_us();
             infer_request->infer();
-            ov_raw_infer_total += ggml_time_us() - ov_raw_infer_start;
+            const int64_t chunk_infer_us = ggml_time_us() - ov_raw_infer_start;
+            ov_raw_infer_total += chunk_infer_us;
+
+            if (vt_rows) {
+                const size_t base_row = static_cast<size_t>(chunk_start);
+                const size_t valid = static_cast<size_t>(chunk_valid_len);
+                const auto & model_outputs = ggml_decoder->get_model_outputs();
+                for (size_t i = 0; i < ov_output_names_local.size(); i++) {
+                    if (!is_vt_rows_output(ov_output_names_local[i])) {
+                        continue;
+                    }
+                    const auto * kv_tensor = model_outputs.at(ov_output_names_local[i]);
+                    const auto * idx = set_rows_indices.at(kv_tensor);
+                    auto row_index = [&](size_t r) -> int64_t {
+                        return idx->type == GGML_TYPE_I64 ? static_cast<const int64_t *>(idx->data)[base_row + r] :
+                                                            static_cast<const int32_t *>(idx->data)[base_row + r];
+                    };
+                    const auto rows = infer_request->get_output_tensor(i);
+                    for (size_t r0 = 0; r0 < valid;) {
+                        const int64_t dst0 = row_index(r0);
+                        size_t len = 1;
+                        while (r0 + len < valid && row_index(r0 + len) == dst0 + static_cast<int64_t>(len)) {
+                            ++len;
+                        }
+                        if (!ggml_openvino_kv_storage_write_rows(kv_tensor->data, rows.data(), rows.get_shape()[3], r0,
+                                                                 static_cast<size_t>(dst0), len)) {
+                            throw std::runtime_error("ggml-openvino: transposed V rows need head-major KV storage");
+                        }
+                        r0 += len;
+                    }
+                }
+            }
+            if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING")) {
+                fprintf(stderr,
+                        "GGML OpenVINO profiling: prefill chunk %d: valid_len=%lld, padded_len=%d, logical_history=%lld, raw infer=%.3f ms\n",
+                        chunk_index, static_cast<long long>(chunk_valid_len), prefill_chunk_size,
+                        static_cast<long long>(chunk_start), chunk_infer_us / 1000.0);
+            }

             if (ggml_openvino_getenv_int("GGML_OPENVINO_DEBUG_OUTPUT") ||
                 ggml_openvino_getenv_str("GGML_OPENVINO_DEBUG_NODE")) {
                 for (size_t i = 0; i < ov_output_names_local.size(); i++) {
                     const auto output_tensor = infer_request->get_output_tensor(i);
-                    print_output_tensor_info(ov_output_names_local[i], output_tensor, output_tensor.data());
+                    print_output_tensor_info(ov_output_names_local[i], output_tensor, nullptr);
                 }
             }
         }
         infer_end_time = ggml_time_us();
     } else {
+        if (decode_bucket > 0 && c_params.attention_size != entry->decode_attention_size) {
+            infer_request = get_decode_window_request(c_params.attention_size);
+        }
+
         for (size_t i = 0; i < ov_input_names_local.size(); i++) {
             const auto & param_name = ov_input_names_local[i];
             auto input_tensor = get_ov_input_tensor_static_decode(ggml_decoder, param_name);
@@ -1715,11 +3063,37 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
             ggml_openvino_getenv_str("GGML_OPENVINO_DEBUG_NODE")) {
             for (size_t i = 0; i < ov_output_names_local.size(); i++) {
                 const auto output_tensor = infer_request->get_output_tensor(i);
-                print_output_tensor_info(ov_output_names_local[i], output_tensor, output_tensor.data());
+                print_output_tensor_info(ov_output_names_local[i], output_tensor, nullptr);
             }
         }
     }

+    if (kv_grew) {
+        GGML_LOG_INFO("ggml-openvino: on-demand KV storage moved to cover a %zu-row window\n", kv_rows);
+    }
+    if (kv_grew && cache_enabled && !entry->shared_key.empty()) {
+        // Idle requests still bound to the previous KV storage keep it (and their own buffers) alive: drop the
+        // other window requests and recreate the pair's idle requests.
+        for (auto it = entry->decode_window_requests.begin(); it != entry->decode_window_requests.end();) {
+            it = it->second == infer_request ? std::next(it) : entry->decode_window_requests.erase(it);
+        }
+        auto shared_cache = r_ctx->compiled_cache;
+        std::lock_guard<std::mutex> compile_lock(shared_cache->mutex);
+        auto graph_it = shared_cache->graphs.find(entry->shared_key);
+        if (graph_it != shared_cache->graphs.end()) {
+            std::lock_guard<std::mutex> map_lock(r_ctx->ctx_mutex);
+            auto renew = [&](auto & cache, ov::CompiledModel & compiled) {
+                auto it = cache.find(key);
+                if (it != cache.end() && it->second != infer_request) {
+                    it->second.reset();
+                    it->second = std::make_shared<ov::InferRequest>(compiled.create_infer_request());
+                }
+            };
+            renew(r_ctx->infer_request_cache_prefill, graph_it->second.prefill);
+            renew(r_ctx->infer_request_cache, graph_it->second.decode);
+        }
+    }
+
     if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING")) {
         GGML_LOG_INFO("\nGGML OpenVINO Backend: \n");
         GGML_LOG_INFO("  - Graph decoder time: %.3f ms \n", (decoder_end_time - start_time) / 1000.0);
@@ -1735,19 +3109,58 @@ enum ggml_status ov_graph_compute_static(ggml_cgraph * cgraph, const std::shared
 }
 }  // namespace

-// Nodes on the unselected branches of ggml_build_forward_select() stay in the graph but must not be
-// computed. Keep them out of the OV model, or their inputs become parameters with fixed shapes.
-static ggml_cgraph * get_compute_graph(ggml_cgraph * cgraph, ov_runtime_context & r_ctx) {
+void ov_clear_imported_blobs() {
+    std::lock_guard<std::mutex> lock(g_imported_blobs_mutex);
+    g_imported_blobs.clear();
+}
+
+// Drop unselected branches except pure top-k sampling. Keeping those branches gives MTP a stable
+// model interface as active slots change; inactive results use private output buffers.
+static ggml_cgraph * get_compute_graph(ggml_cgraph * cgraph, ov_runtime_context & r_ctx, bool stable_sampling) {
     auto is_skipped = [](const ggml_tensor * node) {
         return node->op != GGML_OP_NONE && !(node->flags & GGML_TENSOR_FLAG_COMPUTE);
     };
     if (std::none_of(cgraph->nodes, cgraph->nodes + cgraph->n_nodes, is_skipped)) {
         return cgraph;
     }
+    std::unordered_set<const ggml_tensor *> retained;
+    if (stable_sampling) {
+        auto use_count = [&](const ggml_tensor * node) {
+            return cgraph->use_counts[ggml_hash_find(&cgraph->visited_hash_set, node)];
+        };
+        for (int i = 0; i < cgraph->n_nodes; ++i) {
+            const auto * scores = cgraph->nodes[i];
+            if (scores->op != GGML_OP_GET_ROWS || !GgmlOvDecoder::is_sampling_top_k(scores->src[1])) {
+                continue;
+            }
+            const auto * top_k = scores->src[1];
+            const auto * flat = top_k->src[0];
+            const auto * rows = scores->src[0];
+            if (!rows || rows->op != GGML_OP_RESHAPE || rows->src[0] != flat ||
+                use_count(scores) != 0 || use_count(top_k) != 1 || use_count(rows) != 1 ||
+                use_count(flat) != 2 || use_count(flat->src[0]) != 1) {
+                continue;
+            }
+            // Retain only read-only top-k chains. Stateful samplers must keep their branch selection semantics.
+            std::vector<const ggml_tensor *> pending = {scores};
+            while (!pending.empty()) {
+                const auto * node = pending.back();
+                pending.pop_back();
+                if (!is_skipped(node) || !retained.insert(node).second) {
+                    continue;
+                }
+                for (const auto * src : node->src) {
+                    if (src) {
+                        pending.push_back(src);
+                    }
+                }
+            }
+        }
+    }
     auto & compute = r_ctx.compute_graphs[cgraph];
     compute.nodes.clear();
     std::copy_if(cgraph->nodes, cgraph->nodes + cgraph->n_nodes, std::back_inserter(compute.nodes),
-                 [&](const ggml_tensor * node) { return !is_skipped(node); });
+                 [&](const ggml_tensor * node) { return !is_skipped(node) || retained.count(node); });
     compute.graph = *cgraph;
     compute.graph.nodes = compute.nodes.data();
     compute.graph.n_nodes = (int) compute.nodes.size();
@@ -1772,17 +3185,31 @@ enum ggml_status ov_graph_compute(ggml_cgraph * cgraph, ggml_backend_t backend)
         GGML_ASSERT(ctx->runtime_context != nullptr);
         std::shared_ptr<ov_runtime_context> r_ctx = std::static_pointer_cast<ov_runtime_context>(ctx->runtime_context);
         std::lock_guard<std::mutex> execution_lock(r_ctx->execution_mutex);
-        cgraph = get_compute_graph(cgraph, *r_ctx);
+        cgraph = get_compute_graph(cgraph, *r_ctx, !is_static && !r_ctx->stateful);

         return is_static ? ov_graph_compute_static(cgraph, r_ctx) : ov_graph_compute_dynamic(cgraph, r_ctx);
     } catch (const ov::Exception & e) {
         GGML_LOG_ERROR("GGML OpenVINO backend ov::Exception: %s\n", e.what());
+        // A plugin reports a property it rejects as a failed internal check that does not name the property
+        const std::string overrides = ggml_openvino_config_overrides_in_use();
+        if (!overrides.empty()) {
+            GGML_LOG_ERROR("GGML OpenVINO backend: plugin properties from the environment: %s\n", overrides.c_str());
+        }
+        if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING")) {
+            fprintf(stderr, "GGML OpenVINO profiling: ov::Exception: %s\n", e.what());
+        }
         return GGML_STATUS_FAILED;
     } catch (const std::exception & e) {
         GGML_LOG_ERROR("GGML OpenVINO backend std::exception: %s\n", e.what());
+        if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING")) {
+            fprintf(stderr, "GGML OpenVINO profiling: std::exception: %s\n", e.what());
+        }
         return GGML_STATUS_FAILED;
     } catch (...) {
         GGML_LOG_ERROR("GGML OpenVINO backend unknown exception\n");
+        if (ggml_openvino_getenv_int("GGML_OPENVINO_PROFILING")) {
+            fprintf(stderr, "GGML OpenVINO profiling: unknown exception\n");
+        }
         return GGML_STATUS_FAILED;
     }
 }
@@ -1944,9 +3371,21 @@ bool save_ggml_tensor_data_to_txt(const ggml_tensor * tensor, const std::string
     return true;
 }

-void print_input_tensor_info(const std::string & name, const ov::Tensor & tensor) {
-    std::cout << "Input name: " << name << ", Input shape: " << tensor.get_shape() << ", Address: " << tensor.data()
-              << '\n';
+void print_input_tensor_info(const std::string & name, const ov::Tensor & input) {
+    // GPU inputs can be device (USM) tensors, whose data() throws; print a host copy of those.
+    const bool is_remote = input.is<ov::RemoteTensor>();
+    ov::Tensor tensor = input;
+    if (is_remote) {
+        tensor = ov::Tensor(input.get_element_type(), input.get_shape());
+        input.copy_to(tensor);
+    }
+    std::cout << "Input name: " << name << ", Input shape: " << tensor.get_shape() << ", Address: ";
+    if (is_remote) {
+        std::cout << "device";
+    } else {
+        std::cout << tensor.data();
+    }
+    std::cout << '\n';
     switch (tensor.get_element_type()) {
     case ov::element::f32: {
         if (name.find("self_kq_mask") == std::string::npos && name.find("KQ_mask") == std::string::npos) {
@@ -1991,6 +3430,8 @@ void print_input_tensor_info(const std::string & name, const ov::Tensor & tensor
 }

 void print_output_tensor_info(const std::string & name, const ov::Tensor & tensor, const void * output_dst) {
+    ov::Tensor host(tensor.get_element_type(), tensor.get_shape());
+    tensor.copy_to(host);
     std::cout << "Output name: " << name << ", Output shape: " << tensor.get_shape() << ", Address: " << output_dst
               << '\n';

@@ -2002,31 +3443,35 @@ void print_output_tensor_info(const std::string & name, const ov::Tensor & tenso
         float first = get_value(0);
         float min = first;
         float max = first;
+        size_t max_idx = 0;
         double sum = first;

         for (size_t i = 1; i < size; ++i) {
             float v = get_value(i);
             min = std::min(v, min);
-            max = std::max(v, max);
+            if (v > max) {
+                max = v;
+                max_idx = i;
+            }
             sum += v;
         }
         double mean = sum / size;

         std::cout << std::right << std::setw(6) << type_name << std::right << std::setw(12) << "First" << std::setw(12)
-                  << "Min" << std::setw(12) << "Max" << std::setw(12) << "Mean" << '\n';
+                  << "Min" << std::setw(12) << "Max" << std::setw(12) << "Mean" << std::setw(12) << "MaxIdx" << '\n';
         std::cout << std::right << std::setw(6) << "" << std::right << std::setw(12) << first << std::setw(12) << min
-                  << std::setw(12) << max << std::setw(12) << mean << '\n';
+                  << std::setw(12) << max << std::setw(12) << mean << std::setw(12) << max_idx << '\n';
     };

     switch (tensor.get_element_type()) {
     case ov::element::f32: {
-        const float * data = tensor.data<float>();
+        const float * data = host.data<float>();
         size_t size = tensor.get_size();
         print_float_stats("[f32]", size, [data](size_t i) { return data[i]; });
         break;
     }
     case ov::element::f16: {
-        const ov::float16 * data = tensor.data<ov::float16>();
+        const ov::float16 * data = host.data<ov::float16>();
         size_t size = tensor.get_size();
         print_float_stats("[f16]", size, [data](size_t i) { return static_cast<float>(data[i]); });
         break;
diff --git a/ggml/src/ggml-openvino/utils.h b/ggml/src/ggml-openvino/utils.h
index c34441f5e..31fa7abc2 100644
--- a/ggml/src/ggml-openvino/utils.h
+++ b/ggml/src/ggml-openvino/utils.h
@@ -1,10 +1,12 @@
 #include "ggml-decoder.h"
 #include "ggml-impl.h"
 #include "ggml-openvino-extra.h"
+#include "model-cache.h"

 #include <cstddef>
 #include <cstdint>
 #include <functional>
+#include <map>
 #include <memory>
 #include <mutex>
 #include <openvino/runtime/core.hpp>
@@ -18,14 +20,15 @@
 // Cache key for a translated/compiled graph. Node count plus the two end node names identify a
 // graph during inference, where the same few graphs repeat for the whole session. The list of
 // external input names below tells those apart more precisely, but it walks every node and every
-// src slot, so it is built only when GGML_OPENVINO_FULL_GRAPH_KEY is set. Op tests run many small
-// graphs that can share a node count and end names, and need the full key.
+// src slot, so it is built only when GGML_OPENVINO_FULL_GRAPH_KEY is set. Sampler row names are
+// always included to distinguish sampler configurations, even when active subsets share the packed input. Op tests run
+// many small graphs that can share a node count and end names, and need the full key.
 struct graph_key {
     int n_nodes;
     int n_leaves;
     std::string first_node_name;
     std::string last_node_name;
-    // Each entry: "<node_idx>:<src_idx>:<op_type>"
+    // External source descriptors "<src_name>:<src_idx>:<op_type>" and sampler row Parameter names.
     std::vector<std::string> input_srcs;

     graph_key(const ggml_cgraph * cgraph, bool include_inputs = false) : n_nodes(cgraph->n_nodes), n_leaves(cgraph->n_leafs) {
@@ -34,6 +37,13 @@ struct graph_key {
             last_node_name = cgraph->nodes[n_nodes - 1]->name;
         }

+        for (int i = 0; i < n_nodes; ++i) {
+            const auto * node = cgraph->nodes[i];
+            if (GgmlOvDecoder::is_sampling_view(node)) {
+                input_srcs.push_back("sampling_row_" + std::string(node->name) + "#n" + std::to_string(i));
+            }
+        }
+
         static const bool full_key = ggml_openvino_getenv_int("GGML_OPENVINO_FULL_GRAPH_KEY") != 0;
         if (!full_key && !include_inputs) {
             return;
@@ -52,9 +62,8 @@ struct graph_key {
                 if (src == nullptr || src->buffer == nullptr || node_set.count(src) || src->buffer->usage == GGML_BACKEND_BUFFER_USAGE_WEIGHTS) {
                     continue;
                 }
-
-                input_srcs.push_back(std::to_string(node_idx) + ":" + std::to_string(src_idx) + ":" +
-                    GgmlOvDecoder::compute_op_type(node));
+                input_srcs.push_back(GgmlOvDecoder::get_tensor_name(cgraph, src) + ":" + std::to_string(src_idx) +
+                                          ":" + GgmlOvDecoder::compute_op_type(node));
             }
         }
     }
@@ -86,6 +95,13 @@ struct decoder_runtime_ctx {

     std::shared_ptr<std::mutex> mutex;
     std::shared_ptr<GgmlOvDecoder> ptr;
+    // Attention sizes the cached prefill model was built with; decode calls overwrite the decoder's.
+    int prefill_attention_size = -1;
+    int prefill_attention_size_swa = -1;
+    // KV window of the decode model in the compiled pair (GGML_OPENVINO_NPU_BLOB_PREBUILD).
+    int decode_attention_size = -1;
+    std::string shared_key;
+    std::map<int, std::shared_ptr<ov::InferRequest>> decode_window_requests;
 };

 struct ov_compiled_graph {
@@ -93,6 +109,22 @@ struct ov_compiled_graph {
     ov::CompiledModel prefill;
     std::vector<std::string> input_names;
     std::vector<std::string> output_names;
+    std::vector<std::string> prefill_input_names;
+    std::vector<std::string> prefill_output_names;
+    // Extra decode models for other KV windows, built on first use with the pair's settings.
+    int decode_attention_size = -1;
+    std::map<int, ov::CompiledModel> decode_windows;
+    ov::AnyMap decode_config;
+    uint64_t cache_base_extra = 0;
+    bool self_contained = false;
+    bool decode_unfold = false;
+    // GGML_OPENVINO_NPU_WEIGHTLESS_BANK: the pair's weights file, and its layout when the pair was compiled in
+    // this process (window models then resolve the shared weights through the same file).
+    bool weightless_bank = false;
+    std::string weights_path;
+    std::shared_ptr<ggml_openvino_weights_layout> weights_layout;
+    // Compute params the pair's decode model was built with; window models derive from them.
+    ComputeParams decode_c_params;
 };

 // Only compilation and cache publication use this mutex. Requests, decoders and
@@ -117,6 +149,8 @@ struct ov_runtime_context {
     std::unordered_map<graph_key, std::shared_ptr<ov::InferRequest>, graph_key_hash> infer_request_cache_prefill;
     std::unordered_map<graph_key, std::vector<std::string>, graph_key_hash> ov_input_names_cache;
     std::unordered_map<graph_key, std::vector<std::string>, graph_key_hash> ov_output_names_cache;
+    std::unordered_map<graph_key, std::vector<std::string>, graph_key_hash> ov_input_names_cache_prefill;
+    std::unordered_map<graph_key, std::vector<std::string>, graph_key_hash> ov_output_names_cache_prefill;
     size_t stateful_kv_size;
     std::map<std::string, std::string> kv_state_input_name_map;
     // compute-only copies of graphs that carry unselected ggml_build_forward_select() branches
@@ -134,6 +168,8 @@ struct ov_runtime_context {
         infer_request_cache_prefill.clear();
         ov_input_names_cache.clear();
         ov_output_names_cache.clear();
+        ov_input_names_cache_prefill.clear();
+        ov_output_names_cache_prefill.clear();
         kv_state_input_name_map.clear();
         stateful_kv_size = 0;
     }
@@ -146,6 +182,9 @@ struct ov_runtime_context {

 enum ggml_status ov_graph_compute(struct ggml_cgraph * cgraph, ggml_backend_t backend);

+// Releases the cached blobs this process imported (GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR).
+void ov_clear_imported_blobs();
+
 size_t checksum(const void * data, size_t size);

 bool save_ggml_tensor_data_to_txt(const ggml_tensor * tensor, const std::string & file_path);