Skip to content

Switch to upstream vLLM #22

@liu-cong

Description

@liu-cong

We used a forked vLLM for POC for additional metrics not available in vLLM. With https://github.com/vllm-project/vllm/pull/9477/files, we have the required LoRA metrics in vLLM. This issue involves the following tasks:

  • Update metrics.go to read the new LORA metrics
  • Also read the maxLoRA metric (TODO: update the code link once PR is out)
  • Test the new vLLM release and update our manifests.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions