Skip to content

Commit a64f340

Browse files
Alcpzarthw
authored andcommitted
sycl : Reroute permuted mul_mats through oneMKL (ggml-org#10408)
This PR fixes the failing MUL_MAT tests for the sycl backend.
1 parent 9028651 commit a64f340

File tree

1 file changed

+9
-2
lines changed

1 file changed

+9
-2
lines changed

ggml/src/ggml-sycl/ggml-sycl.cpp

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3324,8 +3324,15 @@ static void ggml_sycl_mul_mat(ggml_backend_sycl_context & ctx, const ggml_tensor
33243324
#endif // SYCL_USE_XMX
33253325

33263326
if (!split && src0->type == GGML_TYPE_F16 && ggml_is_permuted(src0) && ggml_is_permuted(src1) && src1->ne[1] == 1) {
3327-
// KQ single-batch
3328-
ggml_sycl_mul_mat_vec_p021(ctx, src0, src1, dst);
3327+
// TODO: Refactor and cleanup of mul mat dispatching.
3328+
if (src0->ne[3] == 1 && src1->ne[3] == 1) {
3329+
// KQ single-batch
3330+
// mmv p021 was specific for these dimensions
3331+
ggml_sycl_mul_mat_vec_p021(ctx, src0, src1, dst);
3332+
} else {
3333+
// The kernel from the if path is faster for that specific case, but does not support all mul mats.
3334+
ggml_sycl_mul_mat_batched_sycl(ctx, src0, src1, dst);
3335+
}
33293336
} else if (!split && src0->type == GGML_TYPE_F16 && !ggml_is_contiguous(src0) && !ggml_is_transposed(src1) && src1->ne[1] == 1) {
33303337
// KQV single-batch
33313338
ggml_sycl_mul_mat_vec_nc(ctx, src0, src1, dst);

0 commit comments

Comments
 (0)