[Frontend] Factor out chat message parsing #7055
Conversation
DarkLight1337
added
the
ready
|
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge). To run full CI, you can do one of these:
🚀 |
| result = parse_chat_message_content(message, model_config, | ||
| tokenizer) | ||
| conversation.extend(result.messages) | ||
| if mm_futures: |
There was a problem hiding this comment.
QQ: should we warn only once? (I think there's a function like warn_once) it is not going to be spammy?
There was a problem hiding this comment.
Good point! It's unlikely to be an issue since we have explicitly mentioned that we're only supporting single-image input currently, but yea I agree we should use warn_once here!
nvm - the warning here is actually different than what I thought, yea we should put warn_once here
There was a problem hiding this comment.
Actually, the default behaviour already does this (i.e. only print out the first occurrence of the warning).
https://docs.python.org/3/library/warnings.html#the-warnings-filter
Signed-off-by: Alvant <[email protected]>
Signed-off-by: LeiWang1999 <[email protected]>
4 participants
To further support #5049, I've factored out more code relating to chat message parsing into
chat_utils.py