Skip to content

Commit 9cd9224

Browse files
committed
do not offload scales
1 parent fa7bbcd commit 9cd9224

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

llama.cpp

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5652,8 +5652,8 @@ static const std::unordered_map<const char *, llm_offload_func_e> k_offload_map
56525652
{ "pos_embd", OFFLOAD_FUNC_NR },
56535653

56545654
{ "inp_pos", OFFLOAD_FUNC_FRC }, // this is often used for KQ ops (e.g. rope)
5655-
{ "Q_scale", OFFLOAD_FUNC_FRC },
5656-
{ "KQ_scale", OFFLOAD_FUNC_FRC },
5655+
{ "Q_scale", OFFLOAD_FUNC_NOP },
5656+
{ "KQ_scale", OFFLOAD_FUNC_NOP },
56575657
{ "KQ_mask", OFFLOAD_FUNC_FRC },
56585658
{ "K_shift", OFFLOAD_FUNC_FRC },
56595659

0 commit comments

Comments
 (0)