feat: Add items and title to ToolParameter/ToolParamDefinition

[TamiTakamiya]

What does this PR do?

Add items and title to ToolParameter/ToolParamDefinition. Adding items will resolve the issue that occurs with Gemini LLM when an MCP tool has array-type properties.

Test Plan

Unite test cases will be added.

[meta-cla]

Hi @TamiTakamiya!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

[meta-cla]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[ashwinb]

Can you show the errors that happens when you don't apply this change? The MCP specification does not talk about these attributes specifically. Can you explain exactly how the failures happen and what is specific about Gemini that doesn't affect GPT or Claude?

[TamiTakamiya]

@ashwinb When a Gemini LLM (e.g. gemini-2.5-pro) + an MCP server whose tool contains an array-type property, following error occurs in litellm:

  File "/home/ttakamiy/git/TamiTakamiya/ansible-chatbot-stack/.venv/lib64/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 1293, in exception_type
    raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: VertexAIException BadRequestError - b'{\n  "error": {\n    "code": 400,\n    "message": "* GenerateContentRequest.tools[0].function_declarations[1].parameters.properties[redis].items: missing field.\\n* GenerateContentRequest.tools[0].function_declarations[1].parameters.properties[eda_hosts].items: missing field.\\n* GenerateContentRequest.tools[0].function_declarations[1].parameters.properties[execution_hosts].items: missing field.\\n* GenerateContentRequest.tools[0].function_declarations[1].parameters.properties[hub_hosts].items: missing field.\\n* GenerateContentRequest.tools[0].function_declarations[1].parameters.properties[controller_hosts].items: missing field.\\n* GenerateContentRequest.tools[0].function_declarations[1].parameters.properties[gateway_hosts].items: missing field.\\n",\n    "status": "INVALID_ARGUMENT"\n  }\n}\n'

The same error is reported against litellm (BerriAI/litellm#12222, BerriAI/litellm#10251), while they may be found without using Llama Stack.

As you see in the message like GenerateContentRequest.tools[0].function_declarations[1].parameters.properties[redis].items: missing field, the missing items fild is causing this error and it is resolved with the fix that I am proposing with #3004. Thanks.

[ashwinb]

I think this is not standard in any way, possible that this is a litellm or gemini model issue. We cannot add this to the tool definitions unless the MCP standard specifies it.

[TamiTakamiya]

@ashwinb Currently llama-stack stores the type for an array-type property and ignores items, which defines the type of elements. If array-teype properties are not accepted, it is understandable, but since they are accepted, I think it is very reasonable to store items as well. Although I could find no specific mention on type-array in MCP specification, items is a standard in JSON schema. What do you think?

[boazdavid]

@ashwinb I think this is issue need to be fixed. I have the same problem with OpenAI GPT 4O.
According to MCP spec,

inputSchema: JSON Schema defining expected parameters
outputSchema: Optional JSON Schema defining expected output structure

The input schema is a JSON schema. According to the spec a schema of "type": "array", must have items

[bbrowning]

Note that both the inputSchema from the MCP spec as well as the parameters option for function calls in the OpenAI Chat Completions and Responses API allow a full JSON Schema object to be specified. In an ideal world, we'd be able to translate the JSON Schema from the MCP tool inputSchema into an identical JSON Schema in our inference API call to the Chat Completions / Responses API. Because we currently round-trip through this intermediate strongly-typed ToolParameter/ToolParamDefinition, adding the additional fields required to better reproduce the MCP tool's inputSchema accurately in our actual inference requests sent to the model seems reasonable.

Longer-term, we might want to consider whether we really need to strongly type all possible fields of a JSON Schema here or use some other way to shuffle that information from the MCP server to the inference call.

[bbrowning]

Re-opening this PR for further review / discussion.

[ashwinb]

@ashwinb I think this is issue need to be fixed. I have the same problem with OpenAI GPT 4O. According to MCP spec,

inputSchema: JSON Schema defining expected parameters
outputSchema: Optional JSON Schema defining expected output structure

The input schema is a JSON schema. According to the spec a schema of "type": "array", must have items

I didn't realize (immediately) that these fields flow through to the JSON Schema parameters. It just seems like a better fix (although not needed here for now) is to make sure that the Llama Stack tool stuff is explicitly thought of as being part of a JSON schema formally (which it was only loosely modeled until now.)

I will review and merge this soon as soon as CI is green.

[ashwinb]

You will need to re-record the individual test by using https://github.com/llamastack/llama-stack/blob/main/tests/README.md#remote-re-recording-recommended

• from your branch in your local repo you can run this workflow
• use something like --subdirs tool_runtime --pattern test_mcp_invocation and it should re-record the results

Pre-commit is also failing. Please fix that.

[boazdavid]

if possible, I'd pass the whole input and output JSON schemas to the ToolParameter.
see also issue #3365

[TamiTakamiya]

@ashwinb @boazdavid @bbrowning What do you suggest we move forward with this PR? My original intention was to provide a small change to fix an issue with array-type parameters with Gemini provider, but if we are going to support full JSON schema, this will be included in that new feature. Pls advise.

[wukaixingxp]

You will need to re-record the individual test by using https://github.com/llamastack/llama-stack/blob/main/tests/README.md#remote-re-recording-recommended

• from your branch in your local repo you can run this workflow
• use something like --subdirs tool_runtime --pattern test_mcp_invocation and it should re-record the results

Pre-commit is also failing. Please fix that.

@TamiTakamiya The CI test is failing due to FAILED tests/integration/tool_runtime/test_mcp.py::test_mcp_invocation[txt=ollama/llama3.2:3b-instruct-fp16] - RuntimeError: No recorded response found for request, please follow the instruction above to re-record this test_mcp_invocation

[ashwinb]

if possible, I'd pass the whole input and output JSON schemas to the ToolParameter. see also issue #3365

yeah I will do that as a follow-up. The root cause is what Ben pointed -- it seems un-ideal to transport via our ToolParameter, etc. definitions which likely just need to go now. We can work with JSON schemas instead and convert back into ToolParameter, etc. wherever needed.

[ashwinb]

CI clean now, landing this. Thanks for the work @TamiTakamiya and the discussion everyone. Sorry for all the delay in getting this merged.

[ashwinb]

@boazdavid @bbrowning @TamiTakamiya @wukaixingxp see #3627 (still work in progress) for killing all these MCP tool schema related errors in one go.