Support native json output and strict tool calls for anthropic #3457
Conversation
| print(f'Cache read tokens: {usage.cache_read_tokens}') | ||
| ``` | ||
|
|
||
| ## Structured outputs & strict tool calls |
There was a problem hiding this comment.
No doc needed as this will all work automatically
| #### Native Output | ||
|
|
||
| Native Output mode uses a model's native "Structured Outputs" feature (aka "JSON Schema response format"), where the model is forced to only output text matching the provided JSON schema. Note that this is not supported by all models, and sometimes comes with restrictions. For example, Anthropic does not support this at all, and Gemini cannot use tools at the same time as structured output, and attempting to do so will result in an error. | ||
| Native Output mode uses a model's native "Structured Outputs" feature (aka "JSON Schema response format"), where the model is forced to only output text matching the provided JSON schema. Note that this is not supported by all models, and sometimes comes with restrictions. For example, Anthropic requires enabling their Structured Outputs beta (Pydantic AI handles the required headers automatically), while Gemini cannot use tools at the same time as structured output, and attempting to do so will result in an error. |
There was a problem hiding this comment.
Let's drop Anthropic here entirely
| } | ||
|
|
||
| # TODO: remove once anthropic moves it out of beta | ||
| _STRUCTURED_OUTPUTS_BETA = 'structured-outputs-2025-11-13' |
There was a problem hiding this comment.
No need for a constant as we don't use one for the other beta features either
| tools, strict_tools_requested = self._get_tools(model_request_parameters, model_settings) | ||
| tools, mcp_servers, beta_features = self._add_builtin_tools(tools, model_request_parameters) | ||
| output_format = self._build_output_format(model_request_parameters) | ||
| structured_output_beta_required = strict_tools_requested or bool(output_format) |
There was a problem hiding this comment.
We could simplify the code below by adding a value to the beta_headers list here right?
|
|
||
| extra_body = cast(dict[str, Any] | None, model_settings.get('extra_body')) | ||
| if output_format is not None: | ||
| extra_body = self._merge_output_format_extra_body(extra_body, output_format) |
There was a problem hiding this comment.
We should bump the dependency so we can pass this directly instead of via extra body
| if helper is None: | ||
| return schema | ||
| try: # pragma: no branch | ||
| # helper may raise if schema already transformed |
There was a problem hiding this comment.
Really? Why? What kind of error?
|
|
||
| def transform(self, schema: JsonSchema) -> JsonSchema: | ||
| schema.pop('title', None) | ||
| schema.pop('$schema', None) |
|
|
||
| async def test_anthropic_native_output(allow_model_requests: None, anthropic_api_key: str): | ||
| m = AnthropicModel('claude-sonnet-4-5', provider=AnthropicProvider(api_key=anthropic_api_key)) | ||
| async def test_anthropic_native_output_uses_output_format(allow_model_requests: None): |
There was a problem hiding this comment.
Let's use end-to-end tests
| 'input_schema': f.parameters_json_schema, | ||
| } | ||
| if f.strict is not None: | ||
| tool_param['strict'] = f.strict # type: ignore[assignment] |
There was a problem hiding this comment.
Look at how the OpenAI model uses is_strict_compatible. If the user didn't explicitly say strict=True on their tool, it'll be strict=None, so we check if the schema is strict-compatible, and if so we set strict=True.
So to get the same behavior with Anthropic, we should check if the schema can be transformed successfully (losslessly), and set strict=True unless it's explicitly strict=False
2 participants
addresses #3428
work in progress: