Add custom models

[FerLuisxd]

Feature request

Since LLM models are made basically everyday it would be good to simply search for models directly from hugging face or allow us to manually download and setup new models

Motivation

It would allow for more experimentations and comparison between models

Your contribution

Testing

[manyoso]

You can easily download and manually install models right now. All you need to do is place the model in the models download directory and make sure the model name begins with 'ggml-*' and ends with '.bin' and of course you have to be compatible with our version of llama.cpp

[twilsonco]

I know this is closed, but it sounds like the suggestion had as much to do with the easy finding and acquisition of models rather than the technical task of running them in GPT4All. I, too think that would be a great feature.

[cosmic-snow]

I know this is closed, but it sounds like the suggestion had as much to do with the easy finding and acquisition of models rather than the technical task of running them in GPT4All. I, too think that would be a great feature.

Searching for/finding compatible models isn't so simple that it could be automated. But maybe have a look at this newer issue: #1241

[tailkinker]

This no longer works.

[cebtenzzre]

This no longer works.

As long as your are downloading .gguf files from HF, it should work fine. That's the file format used by GPT4All v2.5.0+.

[tailkinker]

Figured that out eventually. But now it's spelled out. Latest update invalidated every model I had...

[IzzyHibbert]

This no longer works.

As long as your are downloading .gguf files from HF, it should work fine. That's the file format used by GPT4All v2.5.0+.

No it doesn't :-(
You can try checking for instance this one :

galatolo/cerbero-7b-gguf

Placing the gguf in the model folder and running it on a 2.5.1 in both Win / Mac M1 it fails even on the smallest Q4 version.

[manyoso]

This no longer works.

As long as your are downloading .gguf files from HF, it should work fine. That's the file format used by GPT4All v2.5.0+.

No it doesn't :-( You can try checking for instance this one :

galatolo/cerbero-7b-gguf

Placing the gguf in the model folder and running it on a 2.5.1 in both Win / Mac M1 it fails even on the smallest Q4 version.

Which one? You posted a link that has 3 different files in 3 different quantizations... can you specify which one you are talking about?

[cebtenzzre]

No it doesn't :-( You can try checking for instance this one :

galatolo/cerbero-7b-gguf

Placing the gguf in the model folder and running it on a 2.5.1 in both Win / Mac M1 it fails even on the smallest Q4 version.

This is GGUF version 3 which was recently released to support big endian systems (ggml-org/llama.cpp#3552). We do not support that version yet. Please open a new issue.