Skip to content

Commit a71b695

Browse files
committed
fragmentation calculation fix
1 parent 283ab2f commit a71b695

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

llama.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8898,7 +8898,7 @@ static int llama_decode_internal(
88988898

88998899
// decide if we need to defrag the kv cache
89008900
if (cparams.defrag_thold >= 0.0f) {
8901-
const float fragmentation = kv_self.n >= 128 ? 1.0f - float(kv_self.used + n_tokens)/float(kv_self.n) : 0.0f;
8901+
const float fragmentation = kv_self.n >= 128 ? 1.0f - float(kv_self.used)/float(kv_self.n) : 0.0f;
89028902

89038903
// queue defragmentation for next llama_kv_cache_update
89048904
if (fragmentation > cparams.defrag_thold) {

0 commit comments

Comments
 (0)