Endpoint disabled for this model by API configuration 500

[mike-niemand]

LocalAI version:
1.18.0

Environment, CPU architecture, OS, and Version:
Linux 78b4ecbb1b9f 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 GNU/Linux

Describe the bug

I am using the langchain-chroma example.

I have Local AI composed and running on Fiber. I am trying to create the storage for it. It creates the /db and sends the embeddings to the Local AI instance which I can see gets them. Local AI then throws a 500: 'endpoint disabled for this model by API configuration'

Expected behavior
Expected to complete succesfully

Logs
DEBUG:openai:message='OpenAI API response' path=http://127.0.0.1:8080/v1/engines/text-embedding-ada-002/embeddings processing_ms=None request_id=None response_code=500
INFO:openai:error_code=500 error_message='endpoint disabled for this model by API configuration' error_param=None error_type= message='OpenAI API error received' stream_error=False
WARNING:langchain.embeddings.openai:Retrying langchain.embeddings.openai.embed_with_retry.._embed_with_retry in 4.0 seconds as it raised APIError: endpoint disabled for this model by API configuration {"error":{"code":500,"message":"endpoint disabled for this model by API configuration","type":""}} 500 {'error': {'code': 500, 'message': 'endpoint disabled for this model by API configuration', 'type': ''}} {'Date': 'Mon, 12 Jun 2023 11:42:50 GMT', 'Content-Type': 'application/json', 'Content-Length': '98'}.

[neversettle7]

Same issue on macOS 13.4 build 22F66 - LocalAI version 1.18.0

[mike-niemand]

It's odd because I have spent hours trying to find the issue and that normally means it's something stupid. That error doesn't exist. Possibly permissions but I have mine set correctly and have tried them wide open.

[neversettle7]

It's odd because I have spent hours trying to find the issue and that normally means it's something stupid. That error doesn't exist. Possibly permissions but I have mine set correctly and have tried them wide open.

Yeah I've been having this issue for 4 days now and I was going crazy, as there's no mention of that error in any documentation, not even on OpenAI's official docs for their python library. I also tried changing model with no improvements, still same issue.

[mike-niemand]

Only been 1 day so far for me. I have also posted on Discord so hope someone there will know.

Edit: Are you building in docker-compose?

[neversettle7]

I am not, it's a local build on M1 Pro following the instructions here. I was following the guide on mudler's blog - which is the same as the one you were following - and I got stuck on this issue.

[mike-niemand]

Ummm. Hold a minute let me try something.

[mike-niemand]

Sorry @neversettle7 I can herewith confirm we are idiots ;) ...but thanks for pointing me in the right direction.

From Mudler's blog:

Note: The example contains a models folder with the configuration for gpt4all and the embeddings models already prepared. LocalAI will map gpt4all to gpt-3.5-turbo model, and bert to the embeddings endpoints.

Copy those files into your AI's /models directory and it works.

[mike-niemand]

Tested.

The solution to getting Mudler's fantastic langchain-chroma working correctly is to make sure you read his instructions as detailed above.

[neversettle7]

I know for sure I'm an idiot and it's likely something really dumb that I'm doing wrong, but I followed the instructions (even repeated them) and I still have the same issue.

This is what I have in my examples/langchain-chroma folder:

- langchain-chroma
 |- models
  |- bert
  |- completion.tmpl
  |- embeddings.yaml
  |- ggml-gpt4all-j
  |- gpt-3.5-turbo.yaml
  |- gpt4all.tmpl
 |- docker-compose.yml
 |- query.py
 |- README.md
 |- requirements.txt
 |- state_of_the_union.txt
 |- store.py

The only difference between mine and mudler's code is in line 21 of store.py where I add openai_api_base and openai_api_key as parameters to the function

embedding = OpenAIEmbeddings(openai_api_key="sk-",openai_api_base="http://localhost:8080/v1",model="text-embedding-ada-002")

Everything else is exactly the same.
I still tried repeating the same steps as he did (exporting the global variables before launching the script) and I still have the same issue.

[mike-niemand]

Nooo.

In one terminal, run Step 1: Start LocalAI details *** remember to copy the models.

It will then be up and running (takes a while to build) but will complete and tell you it's running Fiber v2.46.0 and what IP:PORT its' on.

Then do step 2 in another terminal. This preps the data and sends it to the AI running in the first terminal. You can then query it from terminal 2 or a UI. Remember to set the OPENAI_API_BASE and OPENAI_API_KEY env variables here

[neversettle7]

Yes, that's exactly what I did and how I got to that issue.
I will probably start all over again, just in case I messed something up while attempting to fix the problem. Thanks for the help!

[MarcoBoo]

Hi guys, i have done what written in the message before ( copying all the files) and my situation is the same as @neversettle7 :
|- models
|- bert
|- completion.tmpl
|- embeddings.yaml
|- ggml-gpt4all-j
|- gpt-3.5-turbo.yaml
|- gpt4all.tmpl

still not working.. how did you solved ?

ps. im using docker