[Usage]: Multi lora inference support for llava v1.6

[WeixuanXiong]

Your current environment

The output of `python collect_env.py`

How would you like to use vllm

I tried to run multi-lora inference on vllm for llava-hf/llava-v1.6-mistral-7b-hf. But i found it not supported according to the doc here https://docs.vllm.ai/en/latest/models/supported_models.html#list-of-multimodal-language-models.

Is there any plan to support multi lora for more mllm in near future or any practical way to obtain it on llava-next?

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[hmellor]

You can check out https://docs.vllm.ai/en/latest/features/compatibility_matrix.html#feature-x-feature to see which features are compatible with each other.

The table indicates that multi-modal inputs are not yet compatible with LoRA (although, the pull request linked in the table is merged).

You can follow #4194 for updates on LoRA + multi-modality.