Skip to content

Conversation

@paxiaatucsdedu
Copy link
Member

Problem

LLMs occasionally return tool call arguments as escaped JSON strings (e.g., '"{\\"month\\": \\"August\\"}"') instead of plain JSON. This caused validation errors when creating AIMessage objects, as the arguments remained strings after a single json.loads() instead of being parsed into dictionaries.

Issue link: #52

Solution

Added double-parsing logic in OCIUtils.convert_oci_tool_call_to_langchain() to detect and handle escaped JSON.
Added escaped JSON test case in unit test.

Testing

Validated with real LLM responses showing the escaped JSON pattern:

system_msg = SystemMessage(content="You are a finance assistant. \nYou have access to these tools:\n- MonthlyVarianceSQL: fetch per-customer variance for a specific month and year\n- MultiMonthVarianceSQL: fetch variance summary for a year\nQuestion: {input}\nAlways use the appropriate tool first to get data, then explain in natural language.\n")

# REQUEST 1
print("REQUEST 1:")
messages1 = [
    system_msg,
    HumanMessage(content="Explain why cash inflow in August 2025 was below forecast.")
]
response1 = chat_with_tools.invoke(messages1)
print(f"Tool calls: {response1.tool_calls}\n")

# REQUEST 2 (with conversation history)
print("REQUEST 2:")
messages2 = [
    system_msg,
    HumanMessage(content="Who are my customers?\n"),
    AIMessage(content="Your request is incomplete. Please provide more details or specify the task you need help with."),
    HumanMessage(content="Explain why cash inflow in August 2025 was below forecast.\n"),
    AIMessage(content="", tool_calls=[{"id": "call_ed2635c686f449eea25915b2", "name": "monthly_variance_sql", "args": {"month": "August", "year": "2025"}}]),
    ToolMessage(content="<tool_response!>", tool_call_id="call_ed2635c686f449eea25915b2"),
    AIMessage(content="<Agent_reponse>"),
    HumanMessage(content="Tell me more information about Acme Corp  in this period")
]

try:
    response2 = chat_with_tools.invoke(messages2)
    print(f"Tool calls: {response2.tool_calls}")
except Exception as e:
    print(f"Error (this is the bug): {e}")

Output:

REQUEST 1:
Tool calls: [{'name': 'monthly_variance_sql', 'args': {'month': 'August', 'year': '2025'}, 'id': 'call_9a19748f1068440dbb0aee24', 'type': 'tool_call'}]

REQUEST 2:
Tool calls: [{'name': 'monthly_variance_sql', 'args': {'month': 'August', 'year': '2025'}, 'id': 'call_db1c9eb7f6514348b3d0983a', 'type': 'tool_call'}]

Handle escaped JSON in tool call arguments. Fix issue oracle#52.
@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Oct 30, 2025
@YouNeedCryDear YouNeedCryDear self-requested a review October 31, 2025 00:53
Adds error handling for JSONDecodeError when parsing tool call results. If the string is not valid JSON, it is retained as a string instead of raising an exception.
@YouNeedCryDear YouNeedCryDear merged commit 154057d into oracle:main Nov 4, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants