vulkan: double buffer scale caches

[netrunnereve]

This double buffers the mat vec scale caches from #11081 so that the threads don't need to wait at a barrier before the cache is filled.

Performance wise this is pretty much within 1% of master, with it being a bit slower on my RX470 and a bit faster on my W8100. This might have a bigger improvement on faster GPUs that actually are caught up by the barrier (on my computer I can actually remove the barrier without double buffering and still have the tests pass).

[jeffbolznv]

When the workgroup size equals the subgroup size, the barrier becomes free or nearly free. I haven't had a chance to test this yet, but I don't expect gains on NVIDIA GPUs.

[jeffbolznv]

I did a quick test with Qwen2.5-7B-Instruct-1M-Q2_K.gguf, maybe a very small gain (<1%) but hard to tell whether it's just noise.

[netrunnereve]

Yeah there's probably not much of a difference here for mat vec. I'm trying to get this set up for mat mul as well, though with limited shared memory there's a tradeoff between a single large buffer and a smaller double buffer.

[0cc4m]

I see no difference on Nvidia or AMD, but I do see a decent performance increase in q2_k and q3_k on Intel A770. Not on q6_k, but Intel has never liked q6_k for whatever reason and doesn't perform well with it either way.

If this doesn't have negative effect on other hardware, I think it's good.

[netrunnereve]

I'm trying to get this set up for mat mul as well, though with limited shared memory there's a tradeoff between a single large buffer and a smaller double buffer.

The mat mul experiment ended up being a failure as the extra shared memory slowed things down way more than the barrier did. Since the shared memory is, well, shared among all the subgroups running on the core this reduces the number of subgroups it can run at a time.

If this doesn't have negative effect on other hardware, I think it's good.

Considering how this doesn't affect Nvidia or AMD and makes Intel faster I think we're fine.

[LostRuins]

How much perf improvement did you see on A770