Skip to content

Commit 37a4947

Browse files
authored
[Bugfix] Fix layer skip logic with bitsandbytes (#9887)
Signed-off-by: mgoin <[email protected]>
1 parent 96e0c9c commit 37a4947

File tree

1 file changed

+6
-1
lines changed

1 file changed

+6
-1
lines changed

vllm/model_executor/layers/quantization/bitsandbytes.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,12 @@ def get_scaled_act_names(self) -> List[str]:
119119

120120

121121
def is_layer_skipped_bnb(prefix: str, llm_int8_skip_modules: List[str]):
122-
return any(module_name in prefix for module_name in llm_int8_skip_modules)
122+
# Split the prefix into its dot-separated components
123+
components = prefix.split('.')
124+
125+
# Check if any of the skip modules exactly matches any component
126+
return any(module_name in components
127+
for module_name in llm_int8_skip_modules)
123128

124129

125130
class BitsAndBytesLinearMethod(LinearMethodBase):

0 commit comments

Comments
 (0)