llama : update worst-case graph for unified cache

[ggerganov]

target #17276

Fixes https://github.com/ggml-org/llama.cpp/actions/runs/19443607071/job/55632677520?pr=17276#step:3:4553

[ggerganov]

I can't recall why this cparams.kv_unified check was added in #14363. It seems unnecessary now and removing it gives a better estimate of the worst-case graph.

[ggerganov]

There are 2 remaining issues with the CI:

• The vulkan test-thread-safety is triggering graph reallocation which I am not yet sure what is the cause
• The CUDA backend does not support FA for head size 32 which is causing reallocation in this test: https://github.com/ggml-org/llama.cpp/actions/runs/19494035114/job/55791890377?pr=17379#step:3:3238

[slaren]

The CUDA failure is caused by different pp and tg graphs. This happens because this model has some weights in the input layer, that are loaded in the CPU, but copied to CUDA with large batches. A solution to that would be to disable op offloading for these tests with --no-op-offload. I have not been able to reproduce the Vulkan issue, llvmpipe is crashing on my system for some reason.

[jeffbolznv]

* The vulkan `test-thread-safety` is triggering graph reallocation which I am not yet sure what is the cause

Could this be the same issue I described at #17033 (comment)? I haven't had a chance to get back to this.

[ggerganov]

Could this be the same issue I described at #17033 (comment)? I haven't had a chance to get back to this.

I can test this tomorrow to confirm

[ggerganov]

Yes, if I disable the graph optimization logic in the vulkan backend, the test-thread-safety runs without causing reallocations:

diff --git a/ggml/src/ggml-vulkan/ggml-vulkan.cpp b/ggml/src/ggml-vulkan/ggml-vulkan.cpp
index bb3eb977c..ca12d2d1f 100644
--- a/ggml/src/ggml-vulkan/ggml-vulkan.cpp
+++ b/ggml/src/ggml-vulkan/ggml-vulkan.cpp
@@ -12925,6 +12925,7 @@ static ggml_status ggml_backend_vk_graph_compute(ggml_backend_t backend, ggml_cg
 // Sort the graph for improved parallelism.
 static void ggml_vk_graph_optimize(ggml_backend_t backend, struct ggml_cgraph * graph)
 {
+    return;
     VK_LOG_DEBUG("ggml_vk_graph_optimize(" << graph->n_nodes << " nodes)");
     ggml_backend_vk_context * ctx = (ggml_backend_vk_context *)backend->context;
 

[ggerganov]

Think we can merge this into #17276 and then figure out what to do with the remaining issue of graph optimization causing reallocations. (btw, I'm still not sure I understand in which cases this happens)