GRPC Service Not Ready #2220
Description
Einstein-v6.1-Llama3-8B-Q4_K_M.gguf
Environment, CPU architecture, OS, and Version:
Running on Unraid
| Model: | Custom |
|---|---|
| M/B: | Intel Corporation S2600CP Version E99552-510 s/n QSCP34600258 |
| CPU: | Intel® Xeon® CPU E5-2630 0 @ 2.30GHz |
| HVM: | Enabled |
| IOMMU: | Enabled |
| Cache: | L1-Cache: 384 KiB, L2-Cache: 1536 KiB, L3-Cache: 15 MiB |
| Memory: | 44 GiB DDR3 Multi-bit ECC (max. installable capacity 512 GiB) |
| Network: | bond0: fault-tolerance (active-backup), mtu 1500 |
Describe the bug
Encountered a server error with the message "rpc error: code = Unknown desc = unimplemented" when attempting to access the /v1/chat/completions endpoint. This happened despite the server and services appearing to be ready and operational as indicated by previous log entries.
To Reproduce
Start the LocalAI server with the following configuration settings: {list any specific configurations or settings used}
Send a POST request to /v1/chat/completions
Observe the error in the logs
Expected behavior
Expected the server to handle the POST request to /v1/chat/completions without errors, returning a successful response.
Logs
2:59AM DBG GRPC(Einstein-v6.1-Llama3-8B-Q4_K_M.gguf-127.0.0.1:44779): stderr 2024/05/02 02:59:49 gRPC Server listening at 127.0.0.1:44779
2:59AM DBG GRPC Service Ready
2:59AM DBG GRPC: Loading model with options: {lengthy configuration details here...}
2:59AM INF [stablediffusion] Loads OK
2:59AM ERR Server error error="rpc error: code = Unknown desc = unimplemented" ip=192.168.0.60 latency=1m2.252460241s method=POST status=500 url=/v1/chat/completions
2:59AM INF Success ip=127.0.0.1 latency="41.812µs" method=GET status=200 url=/readyz
Additional context
Running on an Unraid system which might be relevant in terms of the operating system environment or specific configurations.