Skip to content

[Frontend] Add OpenAI API support for input_audio #11027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Dec 17, 2024
Merged

[Frontend] Add OpenAI API support for input_audio #11027

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
7fb18c8
add OpenAI API support for input_audio
kylehh Dec 9, 2024
fdf7abb
Use both audio_url and input_audio in the multiple audio test case
kylehh Dec 9, 2024
6870507
remove CustomChatCompletionContentInputAudioParam calss
kylehh Dec 11, 2024
eb3e0c6
add examples and doc update
kylehh Dec 13, 2024
d9cf0a5
format update
kylehh Dec 13, 2024
e5157ff
update input_audio ref to OAI doc
kylehh Dec 13, 2024
43122fe
update multimedia input doc
kylehh Dec 14, 2024
4331dd7
update doc
kylehh Dec 14, 2024
3fb8b52
Merge branch 'main' into audio_oai_api
DarkLight1337 Dec 16, 2024
b4e5eb9
Update docs
DarkLight1337 Dec 16, 2024
a695d75
Fix typo
DarkLight1337 Dec 16, 2024
e8e6b61
Format
DarkLight1337 Dec 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/serving/openai_compatible_server.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ We currently support the following OpenAI APIs:
- [Vision](https://platform.openai.com/docs/guides/vision)-related parameters are supported; see [Multimodal Inputs](../usage/multimodal_inputs.rst).
- *Note: `image_url.detail` parameter is not supported.*
- We support two audio content types.
- Support `input_audio` content type as defined [here](https://platform.openai.com/docs/guides/audio?audio-generation-quickstart-example=audio-in).
- Support `audio_url` content type for audio files. Refer to [here](https://github.com/vllm-project/vllm/tree/main/vllm/entrypoints/chat_utils.py#L51) for the exact schema.
- Support `input_audio` content type as defined [here](https://github.com/openai/openai-python/blob/v1.52.2/src/openai/types/chat/chat_completion_content_part_input_audio_param.py).*
- *Note: `parallel_tool_calls` and `user` parameters are ignored.*
- [Embeddings API](https://platform.openai.com/docs/api-reference/embeddings)
- Instead of `inputs`, you can pass in a list of `messages` (same schema as Chat Completions API),
Expand Down
Loading