Skip to content

Commit f7d9eb2

Browse files
gabe-l-hartSilver267
authored andcommitted
fix: Move build_inp_pos to the top of the graph section for build_granite (ggml-org#13538)
This matches how others do it, but will still avoid the extra initialization when rope is disabled. Branch: GraniteFour Signed-off-by: Gabe Goodhart <[email protected]>
1 parent 5c0677a commit f7d9eb2

File tree

1 file changed

+3
-4
lines changed

1 file changed

+3
-4
lines changed

src/llama-model.cpp

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12218,6 +12218,9 @@ struct llm_build_granite : public llm_graph_context {
1221812218

1221912219
// inp_pos - built only if rope enabled
1222012220
ggml_tensor * inp_pos = nullptr;
12221+
if (use_rope) {
12222+
inp_pos = build_inp_pos();
12223+
}
1222112224

1222212225
auto * inp_attn = build_attn_inp_kv_unified();
1222312226

@@ -12260,10 +12263,6 @@ struct llm_build_granite : public llm_graph_context {
1226012263
Vcur = ggml_reshape_3d(ctx0, Vcur, n_embd_head, n_head_kv, n_tokens);
1226112264

1226212265
if (use_rope) {
12263-
12264-
if (!inp_pos) {
12265-
inp_pos = build_inp_pos();
12266-
}
1226712266
ggml_tensor * rope_factors = model.get_rope_factors(n_ctx_per_seq, il);
1226812267
Qcur = ggml_rope_ext(
1226912268
ctx0, Qcur, inp_pos, rope_factors,

0 commit comments

Comments
 (0)