Commit 9a79fbd
authored
feat: Add parallel tool calling support for Meta/Llama models (#59)
* feat: Add parallel tool calling support for Meta/Llama models
Add support for the parallel_tool_calls parameter to enable parallel
function calling in Meta/Llama models, improving performance for
multi-tool workflows.
- Add parallel_tool_calls class parameter to OCIGenAIBase (default: False)
- Add parallel_tool_calls parameter to bind_tools() method
- Support hybrid approach: class-level default + per-binding override
- Pass is_parallel_tool_calls to OCI API in MetaProvider
- Add validation for Cohere models (raises error if attempted)
- 9 comprehensive unit tests (all passing)
- 4 integration tests with live OCI API (all passing)
- No regression in existing tests
Class-level default:
llm = ChatOCIGenAI(
model_id="meta.llama-3.3-70b-instruct",
parallel_tool_calls=True
)
Per-binding override:
llm_with_tools = llm.bind_tools(
[tool1, tool2, tool3],
parallel_tool_calls=True
)
- Up to N× speedup for N independent tool calls
- Backward compatible (default: False)
- Clear error messages for unsupported models
- Follows existing parameter patterns
* Fix code formatting for line length compliance
* Update documentation to reflect broader model support for parallel tool calling
- Update README to include all GenericChatRequest models (Grok, OpenAI, Mistral)
- Update code comments and docstrings
- Update error messages with complete model list
- Clarify that feature works with GenericChatRequest, not just Meta/Llama
* Move integration test to correct folder structure
Relocated test_parallel_tool_calling_integration.py to tests/integration_tests/chat_models/
Following repository convention for integration test organization
* Add version filter for Llama parallel tool calling
Only Llama 4+ models support parallel tool calling based on testing.
Parallel tool calling support:
- Llama 4+ - SUPPORTED (tested and verified with real OCI API)
- ALL Llama 3.x (3.0, 3.1, 3.2, 3.3) - BLOCKED
- Cohere - BLOCKED (existing behavior)
- Other models (xAI Grok, OpenAI, Mistral) - SUPPORTED
Implementation:
- Added _supports_parallel_tool_calls() helper method with regex version parsing
- Updated bind_tools() to validate model version before enabling parallel calls
- Provides clear error messages: "only available for Llama 4+ models"
Unit tests added (8 tests, all mocked, no OCI connection):
- test_version_filter_llama_3_0_blocked
- test_version_filter_llama_3_1_blocked
- test_version_filter_llama_3_2_blocked
- test_version_filter_llama_3_3_blocked (Llama 3.3 doesn't support it either)
- test_version_filter_llama_4_allowed
- test_version_filter_other_models_allowed
- test_version_filter_supports_parallel_tool_calls_method
- Plus existing parallel tool calling tests updated to use Llama 4
* Fix linting issues after rebase
- Fix line length violations in chat_models and llms
- Replace print statements with logging in integration tests
- Fix import sorting and remove unused imports
- Fix unused variable in test
* Fix remaining linting issues in test files
* Move parallel tool call validation from bind_tools to provider
- Validation now happens at request preparation time
- Cohere validation remains in CohereProvider
- Llama 3.x validation added to GenericProvider
- Fixes failing unit tests
* Add Llama 3.x validation at bind_tools time
- Llama 3.x validation happens early at bind_tools time
- Cohere validation happens at provider level (_prepare_request time)
- All 16 parallel tool calling tests now pass
* Fix line length issue in bind_tools validation
* Apply ruff formatting to parallel tool calling tests
* Move parallel_tool_calls to bind_tools only (remove class-level param)
* Update integration tests for bind_tools-only parallel_tool_calls
* Fix README to show bind_tools-only parallel_tool_calls usage
* Fix mypy type errors for LangChain 1.x compatibility
- Add type: ignore[override] to bind_tools methods in oci_data_science.py
and oci_generative_ai.py to handle signature incompatibility with
BaseChatModel parent class
- Remove unused type: ignore comments in oci_generative_ai.py
- Add type: ignore[attr-defined] comments for RunnableBinding runtime
attributes (kwargs, _prepare_request) in test_parallel_tool_calling.py
- Fix test_parallel_tool_calling_integration.py to use getattr for
tool_calls attribute access on BaseMessage
- Fix test_tool_calling.py: import StructuredTool from langchain_core.tools
- Fix test_oci_data_science.py: remove unused type: ignore comment
- Fix test_oci_generative_ai_responses_api.py: add type: ignore for
LangGraph invoke arg type
* Fix mypy errors for CI environment compatibility
- Add type: ignore[unreachable] back to BaseTool isinstance check in
oci_generative_ai.py (CI mypy flags this as unreachable)
- Remove type: ignore[override] from bind_tools (CI reports unused)
- Fix test_oci_data_science.py: explicitly type output variable and use
explicit addition instead of += to avoid assignment type error
- Remove unused type: ignore comments from test files
* Fix Python 3.9 compatibility in test_oci_data_science.py
- Use Optional[T] instead of T | None syntax for Python 3.9 compat
- Add type: ignore[assignment] for AIMessageChunk addition
* Simplify parallel tool calls: use provider property instead of model_id parsing
Addresses reviewer feedback:
- Add supports_parallel_tool_calls property to Provider base class (False)
- Override in GenericProvider to return True (supports parallel calls)
- CohereProvider inherits False (doesn't support parallel calls)
- Remove _supports_parallel_tool_calls method with hacky model_id parsing
- Simplify bind_tools to use provider property for validation
- Remove Llama version-specific validation (let API fail naturally)
- Update unit tests to focus on provider-based validation
* Fix integration test for bind_tools validation timing
* Fix mypy linting issues for Python 3.9 compatibility
- Reorder convert_to_oci_tool checks to avoid unreachable code warning
- Fix type annotation in test_stream_vllm to use BaseMessageChunk1 parent 28bf80c commit 9a79fbd
File tree
6 files changed
+582
-24
lines changed- libs/oci
- langchain_oci/chat_models
- tests
- integration_tests/chat_models
- unit_tests/chat_models
6 files changed
+582
-24
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
65 | | - | |
| 65 | + | |
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
| |||
126 | 126 | | |
127 | 127 | | |
128 | 128 | | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
129 | 150 | | |
130 | 151 | | |
131 | 152 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
209 | 209 | | |
210 | 210 | | |
211 | 211 | | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
212 | 224 | | |
213 | 225 | | |
214 | 226 | | |
| |||
363 | 375 | | |
364 | 376 | | |
365 | 377 | | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
366 | 386 | | |
367 | 387 | | |
368 | 388 | | |
| |||
585 | 605 | | |
586 | 606 | | |
587 | 607 | | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
588 | 613 | | |
589 | 614 | | |
590 | 615 | | |
| |||
851 | 876 | | |
852 | 877 | | |
853 | 878 | | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
854 | 883 | | |
855 | 884 | | |
856 | 885 | | |
| |||
916 | 945 | | |
917 | 946 | | |
918 | 947 | | |
919 | | - | |
920 | | - | |
921 | | - | |
| 948 | + | |
| 949 | + | |
922 | 950 | | |
923 | | - | |
924 | | - | |
925 | | - | |
926 | | - | |
927 | | - | |
928 | | - | |
929 | | - | |
930 | | - | |
931 | | - | |
932 | | - | |
933 | | - | |
934 | | - | |
935 | | - | |
936 | 951 | | |
937 | 952 | | |
938 | 953 | | |
| |||
953 | 968 | | |
954 | 969 | | |
955 | 970 | | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
956 | 986 | | |
957 | 987 | | |
958 | 988 | | |
| |||
1211 | 1241 | | |
1212 | 1242 | | |
1213 | 1243 | | |
| 1244 | + | |
1214 | 1245 | | |
1215 | 1246 | | |
1216 | 1247 | | |
| |||
1231 | 1262 | | |
1232 | 1263 | | |
1233 | 1264 | | |
| 1265 | + | |
| 1266 | + | |
| 1267 | + | |
| 1268 | + | |
| 1269 | + | |
1234 | 1270 | | |
1235 | 1271 | | |
1236 | 1272 | | |
| |||
1240 | 1276 | | |
1241 | 1277 | | |
1242 | 1278 | | |
| 1279 | + | |
| 1280 | + | |
| 1281 | + | |
| 1282 | + | |
| 1283 | + | |
| 1284 | + | |
| 1285 | + | |
| 1286 | + | |
| 1287 | + | |
1243 | 1288 | | |
1244 | 1289 | | |
1245 | 1290 | | |
| |||
0 commit comments