Add local-ai community container with Nvidia GPU support via Cuda 12

[luzfcb]

This is a fork of https://github.com/szaimen/aio-local-ai .
It basically uses quay.io/go-skynet/local-ai:v2.24.2-aio-gpu-nvidia-cuda-12 instead of quay.io/go-skynet/local-ai:v2.24.2-aio-cpu as the base docker image, changes the start script a bit to show metadata about the GPUs/Nvidia GPUs and requires ~48GB to ~96GB of free disk space.

[szaimen]

Thanks a lot @luzfcb! I just invited you to the repo to make collaboration easier :)

[szaimen]

Hi @luzfcb please let me know if you need any help here :)

[luzfcb]

@szaimen My problem is just free time. Unfortunately I've been busier than I expected these past few days and haven't had time to continue yet, so it might take a little longer for me to confirm if it's ready or if it needs more improvements.

That said, I noticed that local-ai added support for API keys (comma-separated API Keys), however I couldn't figure out how AIO and AIO Local-AI handle this. https://localai.io/advanced/#api-flags . It probable when set an API Key on AIO admin frontend, than set the LOCALAI_API_KEY environment variable or the .env file on the local-ai container and restart it or restart the local-ai process

Another issue I had was that I couldn't get any response to any request to local-ai . I'm still investigating what might be going wrong.

[szaimen]

@szaimen My problem is just free time. Unfortunately I've been busier than I expected these past few days and haven't had time to continue yet, so it might take a little longer for me to confirm if it's ready or if it needs more improvements.

I see. All good! Take the time you need :)

That said, I noticed that local-ai added support for API keys (comma-separated API Keys), however I couldn't figure out how AIO and AIO Local-AI handle this. https://localai.io/advanced/#api-flags . It probable when set an API Key on AIO admin frontend, than set the LOCALAI_API_KEY environment variable or the .env file on the local-ai container and restart it or restart the local-ai process

I see. I think api keys are not enforced, are they? So you should be able to test even without api keys, no?

Another issue I had was that I couldn't get any response to any request to local-ai . I'm still investigating what might be going wrong.

I see... You could try to test it via curl from inside the containers and such...

[szaimen]

Hi, any update here? :)