File tree Expand file tree Collapse file tree 1 file changed +6
-4
lines changed Expand file tree Collapse file tree 1 file changed +6
-4
lines changed Original file line number Diff line number Diff line change @@ -54,14 +54,16 @@ The {class}`~vllm.LLM.beam_search` method implements [beam search](https://huggi
54
54
For example, to search using 5 beams and output at most 50 tokens:
55
55
56
56
``` python
57
+ from vllm import LLM
58
+ from vllm.sampling_params import BeamSearchParams
59
+
57
60
llm = LLM(model = " facebook/opt-125m" )
58
61
params = BeamSearchParams(beam_width = 5 , max_tokens = 50 )
59
- outputs = llm.generate( " Hello, my name is" , params)
62
+ outputs = llm.beam_search([{ " prompt " : " Hello, my name is " }] , params)
60
63
61
64
for output in outputs:
62
- prompt = output.prompt
63
- generated_text = output.outputs[0 ].text
64
- print (f " Prompt: { prompt!r } , Generated text: { generated_text!r } " )
65
+ generated_text = output.sequences[0 ].text
66
+ print (f " Generated text: { generated_text!r } " )
65
67
```
66
68
67
69
### ` LLM.chat `
You can’t perform that action at this time.
0 commit comments