GPU is not used in LocalAI - even though it works in other local AI repos

[mekler22]

LocalAI version: 2.26
localai/localai:latest-aio-gpu-nvidia-cuda-12

Environment, CPU architecture, OS, and Version:
Linux havenstore 4.4.302+ #72806 SMP Thu Sep 5 13:45:09 CST 2024 x86_64 GNU/Linux synology_broadwellnk_3622xs+

Describe the bug
After composing the docker for GPU usage, with the necessary image and deploy argument inside docker-compose.yml, the running LocalAI is unable to make use of my GPU, with the following error:

WARNING:
localai-api-1 | /sys/class/drm does not exist on this system (likely the host system is a
localai-api-1 | virtual machine or container with no graphics). Therefore,
localai-api-1 | GPUInfo.GraphicsCards will be an empty array.

To Reproduce
docker-compose up
using the webui, simply open chat and ask a question (for instance)

Expected behavior
the docker should be able to use the GPU (which it "sees" with nvidia-smi command)

Logs

Additional context
the resource monitor inside Synology DSM shows that the GPU is not used at all during chat sessions with LocalAI

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.