Skip to content

Commit 311bccc

Browse files
committed
fix: Use @compilade's suggested fix for seq_id indexing with equal splits in unified cache
Branch: GraniteFour Signed-off-by: Gabe Goodhart <[email protected]>
1 parent 585444b commit 311bccc

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-kv-cache.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -579,7 +579,7 @@ int32_t llama_kv_cache_unified::find_slot(const llama_ubatch & ubatch) const {
579579
bool found = true;
580580
for (uint32_t i = 0; i < n_tokens; i++) {
581581
const llama_pos pos = ubatch.pos[i];
582-
const llama_seq_id seq_id = ubatch.seq_id[i][0];
582+
const llama_seq_id seq_id = ubatch.seq_id[i / ubatch.n_seq_tokens][0];
583583

584584
// can we use this cell? either:
585585
// - the cell is empty

0 commit comments

Comments
 (0)