feat: Cortex supports Function Calling

[ChristianWeyer]

Goal

• llama3.1 should support Tool use in llama.cpp
• model: llama3.1 models#16

Questions

• Is our implementation of Function Calling consistent across all models?
  • e.g. do we need model.yaml to have any specific template?

Original post

Problem
AFAICS, the current implementation does not have OpenAI Function Calling support. This would be a fantastic, powerful, and much needed feature.

Success Criteria
Any OAI client can be used with Nitro, even (and especially) those that use OAI Function Calling.

Reference:
https://platform.openai.com/docs/guides/function-calling
https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools

Linked Issues

• Jan #3785 Support response_format and structured JSON responses

Bugs

• Jan erroring out when trying to JSON #1389

[fiery-prometheus]

Would be great to have this for the tensorRT backend as well, though I do not know if they support it.