Skip to content

fix: Avoid BadRequestError due to invalid max_tokens #3667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: Avoid BadRequestError due to invalid max_tokens #3667

wants to merge 1 commit into from

Conversation

luis5tb
Copy link
Contributor

@luis5tb luis5tb commented Oct 3, 2025 •
edited
Loading

This patch ensures if max tokens is not defined, then is set to None instead of 0 when calling openai_chat_completion. This way some providers (like gemini) that cannot handle the max_tokens = 0 will not fail

Issue: #3666

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 3, 2025
@luis5tb luis5tb changed the title Fix BadRequestError due to unvalid max_tokens fix: Avoid BadRequestError due to invalid max_tokens Oct 3, 2025
mattf
mattf requested changes Oct 3, 2025
View reviewed changes
Copy link
Collaborator

@mattf mattf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hold on this pending discord discussion

@luis5tb luis5tb force-pushed the max_tokens branch 3 times, most recently from 99d2730 to 43fb189 Compare October 3, 2025 16:05
@luis5tb
Fix BadRequestError due to unvalid max_tokens
ea42cf6 
This patch ensures if max tokens is not defined it is set to None.
This avoid some providers to fail, as they don't have protection for
it being set to 0

Issue: llamastack#3666
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattf mattf mattf requested changes

@ashwinb ashwinb Awaiting requested review from ashwinb ashwinb is a code owner

@yanxi0830 yanxi0830 Awaiting requested review from yanxi0830 yanxi0830 is a code owner

@hardikjshah hardikjshah Awaiting requested review from hardikjshah hardikjshah is a code owner

@raghotham raghotham Awaiting requested review from raghotham raghotham is a code owner

@ehhuang ehhuang Awaiting requested review from ehhuang ehhuang is a code owner

@terrytangyuan terrytangyuan Awaiting requested review from terrytangyuan terrytangyuan is a code owner

@leseb leseb Awaiting requested review from leseb leseb is a code owner

@bbrowning bbrowning Awaiting requested review from bbrowning bbrowning is a code owner

@reluctantfuturist reluctantfuturist Awaiting requested review from reluctantfuturist reluctantfuturist is a code owner

@slekkala1 slekkala1 Awaiting requested review from slekkala1 slekkala1 is a code owner

@franciscojavierarceo franciscojavierarceo Awaiting requested review from franciscojavierarceo franciscojavierarceo is a code owner

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@luis5tb @mattf