Skip to content

Commit 63e099b

Browse files
committed
fix: Fix bad seq_id[i] indexing when using split_equal with unified cache
Branch: GraniteFour Signed-off-by: Gabe Goodhart <[email protected]>
1 parent 585444b commit 63e099b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-kv-cache.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -579,7 +579,7 @@ int32_t llama_kv_cache_unified::find_slot(const llama_ubatch & ubatch) const {
579579
bool found = true;
580580
for (uint32_t i = 0; i < n_tokens; i++) {
581581
const llama_pos pos = ubatch.pos[i];
582-
const llama_seq_id seq_id = ubatch.seq_id[i][0];
582+
const llama_seq_id seq_id = ubatch.seq_id[i] == nullptr ? ubatch.seq_id[0][0] : ubatch.seq_id[i][0];
583583

584584
// can we use this cell? either:
585585
// - the cell is empty

0 commit comments

Comments
 (0)