Skip to content

Commit a530c97

Browse files
committed
Fix draft_top_p fallback: 1.0 → 0.95 (enables nucleus)
CRITICAL: Line 261 had TWO 1.0 fallbacks, disabling nucleus even when config default is 0.95. Before: getattr(..., 1.0) or 1.0 → always 1.0 → nucleus disabled After: getattr(..., 0.95) or 0.95 → 0.95 → nucleus enabled This is why survivors=32000 (full vocab) instead of ~hundreds.
1 parent 416e4d1 commit a530c97

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/v1/spec_decode/eagle.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -258,7 +258,7 @@ def _sample_draft_tokens(
258258
x = masked.scatter(-1, topi, topv)
259259

260260
# --- top-p (nucleus) ---
261-
tp = float(getattr(self.opt_config, "draft_top_p", 1.0) or 1.0)
261+
tp = float(getattr(self.opt_config, "draft_top_p", 0.95) or 0.95)
262262

263263
if 0.0 < tp < 1.0:
264264
p = torch.softmax(x, dim=-1)

0 commit comments

Comments
 (0)