[Doc]Add benchmark scripts #74
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Potabk
force-pushed
the
benchmarks
branch
2 times, most recently
from
759243e to
1493751
Compare
Yikun
previously requested changes
Feb 27, 2025
docs/source/tutorials.md
Outdated
| INFO 02-19 17:37:35 metrics.py:453] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1.9 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. | ||
| ``` | ||
|
|
||
| ## Performance Benchmark |
There was a problem hiding this comment.
This should be developer guide
benchmarks/backend_request_func.py
Outdated
| @@ -0,0 +1,193 @@ | |||
| # SPDX-License-Identifier: Apache-2.0 | |||
There was a problem hiding this comment.
#
# Copyright (c) 2025 Huawei Technologies Co., Ltd. All Rights Reserved.
# This file is a part of the vllm-ascend project.
# Adapted from vllm-project/vllm/benchmarks/backend_request_func.py
# Copyright 2023 The vLLM team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
benchmarks/backend_request_func.py
Outdated
| **kwargs, | ||
| ) | ||
|
|
||
| ASYNC_REQUEST_FUNCS = { |
There was a problem hiding this comment.
Pls add note for the different with vLLM.
benchmarks/benchmark_latency.py
Outdated
| @@ -0,0 +1,152 @@ | |||
| # SPDX-License-Identifier: Apache-2.0 | |||
github-actions
bot
added
the
documentation
wangxiyuan
requested changes
Mar 21, 2025
benchmarks/README.md
Outdated
| ### Introduction | ||
| This document outlines the benchmarking process for vllm-ascend, designed to evaluate its performance under various workloads. The primary goal is to help developers assess whether their pull requests improve or degrade vllm-ascend's performance.To maintain consistency with the vllm community, we have reused the vllm community [benchmark](https://github.com/vllm-project/vllm/tree/main/benchmarks) script. | ||
| ### Overview | ||
| **Benchmarking Coverage**: We measure latency, throughput, and fixed-QPS serving on the Atlas800I A2 (see [quick_start](./quick_start.md) to learn more supported devices list), with different models(coming soon). |
benchmarks/README.md
Outdated
| @@ -0,0 +1,54 @@ | |||
| ### Introduction | |||
Signed-off-by: wangli <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: wangli <[email protected]>
Signed-off-by: wangli <[email protected]>
wangxiyuan
approved these changes
Mar 21, 2025
ttanzhiqiang
pushed a commit
to ttanzhiqiang/vllm-ascend
that referenced
this pull request
Apr 27, 2025
### What this PR does / why we need it? The purpose of this PR is to add benchmark scripts for npu, developers can easily run performance tests on their own machines with one line of code . --------- Signed-off-by: wangli <[email protected]>
Skywalker-EP
pushed a commit
to Skywalker-EP/vllm-ascend
that referenced
this pull request
Jul 24, 2025
running time reduction forward_before and forward_end
offline893
pushed a commit
to offline893/vllm-ascend
that referenced
this pull request
Sep 9, 2025
running time reduction forward_before and forward_end
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
The purpose of this PR is to add benchmark scripts for npu, developers can easily run performance tests on their own machines with one line of code .
Does this PR introduce any user-facing change?
How was this patch tested?