add support for Custom API Endpoint

[QAbot-zh]

add support for Custom API Endpoint : #14

By filling in the environment variables to set the API endpoint, the usage is as follows :

[toshiakit]

Thank you for the pull request. What is the intended use case for this? Various users provided different use cases and we need to understand better how this will be used. @jonasendc wants to use Azure, and @Mingzefei wants to use ollama. What is your use case?

[QAbot-zh]

Besides using the official OpenAI key to call the gpt, many people are also using API services provided by third parties, including but not limited to Azure, OpenRouter, or other local large language models. If the library cannot be open or does not support custom API endpoints, it limits users to only using OpenAI's service. I saw someone in another project hoping that this library could provide such feature support, so I implemented the functionality and submitted a pull request. Although it is possible to achieve it by modifying the hard-coded endpoint in the code, providing custom endpoints directly as environment variables would make it more convenient.

[toshiakit]

Thank you for your response. I believe there are different uses cases and this library is focused on the most common use cases where people just want to access the foundation models via MATLAB, and we want to keep the library easy to use for that purpose. Hence we are more inclined to create separate repo for more advanced use cases. There is already a pull request for Azure support and perhaps it is more user friendly and easy to maintain if we handle those request separately.

[jonasendc]

What do you mean with foundation models? At the moment it seems like that companies as mathworks can't keep up with the speed of the development in this area. Due to high availability of hardware and cheap options to host own instances of models it would be great to be able to use them also with matlab. What would be your approach how to achieve that rather sooner than later?

[debymf]

Hi @jonasendc, thanks for your comment. Do you have any specific model in mind or any particular use case? I would be interested in learning more about what you want to do.

[ccreutzi]

Thank you for the PR, but given the discussion above and a major limitation I'm about to explain, I am going to close it.

As you may have seen, in another branch, we are working on adding more endpoints. One of the things that includes is implementing the little ways in which endpoints are, in fact, different from one another, even if they claim to be compatible.

For example, here is a request for OpenAI:

{
  "messages": [
    {
      "role": "user",
      "content": "Hi"
    }
  ],
  "stream": false,
  "model": "gpt-3.5-turbo",
  "temperature": 2,
  "n": 1
}

And here is the equivalent (apart from the other model) for Ollama:

{
  "model": "mistral:latest",
  "messages": [
    {
      "role": "user",
      "content": "Hi"
    }
  ],
  "stream": false,
  "options": {
    "temperature": 2,
    "n": 1
  }
}

Even if there are endpoints that are currently 100% compatible, I hesitate to offer a single implementation talking to them. Rather, I'd have very simple classes for each of them, to have a customization seam for future divergent development on the third party side.

Your proposed changes are not going to merge cleanly with these developments (and do not work in MATLAB, where // does not start a comment), but that is a minor point in comparison.