Skip to content

Commit f478aa7

Browse files
authored
hotfix: ipex fails since cuda moe kernel is not supported (#2532)
Signed-off-by: Wang, Yi A <[email protected]>
1 parent abd24dd commit f478aa7

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

server/text_generation_server/models/custom_modeling/flash_deepseek_v2_modeling.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@
1515

1616
from typing import List, Optional, Tuple
1717

18-
from moe_kernels.fused_moe import grouped_topk
1918
import torch
2019
import torch.distributed
2120
from text_generation_server.layers import (
@@ -41,6 +40,9 @@
4140
from transformers.activations import ACT2FN
4241
from transformers.configuration_utils import PretrainedConfig
4342

43+
if SYSTEM != "ipex":
44+
from moe_kernels.fused_moe import grouped_topk
45+
4446
if SYSTEM == "rocm":
4547
try:
4648
from vllm import _custom_C

0 commit comments

Comments
 (0)