We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
benchmark_long_document_qa_throughput.py
1 parent f33e033 commit 5959564Copy full SHA for 5959564
benchmarks/benchmark_long_document_qa_throughput.py
@@ -2,8 +2,7 @@
2
Offline benchmark to test the long document QA throughput.
3
4
Example usage:
5
- # This command run the vllm with 50GB CPU memory for offloading
6
- # The workload samples 8 different prompts with a default input
+ # This workload samples 8 different prompts with a default input
7
# length of 20000 tokens, then replicates each prompt 2 times
8
# in random order.
9
python benchmark_long_document_qa_throughput.py \
0 commit comments