Model magic

[jboero]

Moved this to docs as per original request: #3011

[cebtenzzre]

Could we also distribute an XML file for Linux users? It would be placed in ~/.local/share/mime/packages. I don't know if file uses it, but it would work with file managers and such. Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
   <mime-type type="application/vnd.font-fontforge-sfd">
     <sub-class-of type="text/plain"/>
     <comment>Fontforge Spline Font Database</comment>
     <glob pattern="*.sfd"/>
     <magic>
       <match type="string" offset="0" value="SplineFontDB"/>
     </magic>
   </mime-type>
</mime-info>

[cebtenzzre]

And should we include the old GGML (mlgg) and GGMF (fmgg) formats? PR #3023 adds (limited) support for converting them to GGUF.

[staviq]

This just reminded me of something that makes me think we possibly shouldn't encourage edits to /etc/magic:

https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161 aka OpenOffice won't print on Tuesdays classic :)

Edit: Actually, I can sort of reproduce that bug already, any text file starting with GGUF is being recognized as GGUF LLM model

I don't have a printer at home to test if it breaks cups :)

[staviq]

I believe the XML for it would look like this:

<?xml version="1.0"?>
<mime-info xmlns='http://www.freedesktop.org/standards/shared-mime-info'>
  <mime-type type="model/x-ggml">
    <comment>GGML/GGJT LLM model</comment>
    <magic priority="50">
      <match type="string" offset="0" value="tjgg"/>
    </magic>
    <glob pattern="*.bin"/>
  </mime-type>
  <mime-type type="model/x-gguf">
    <comment>GGUF LLM model</comment>
    <magic priority="50">
      <match type="string" offset="0" value="GGUF"/>
    </magic>
    <glob pattern="*.bin"/>
    <glob pattern="*.gguf"/>
  </mime-type>
</mime-info>

Though I'm not sure if it's possible to have match on file extension together with match on filenames with no extension.

[cebtenzzre]

should we include the old GGML (mlgg) and GGMF (fmgg) formats?

There is also GGLA for LoRAs and GGSN for sessions.

[jboero]

Good catch thanks.

[jboero]

This just reminded me of something that makes me think we possibly shouldn't encourage edits to /etc/magic:

https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161 aka OpenOffice won't print on Tuesdays classic :)

Edit: Actually, I can sort of reproduce that bug already, any text file starting with GGUF is being recognized as GGUF LLM model

I don't have a printer at home to test if it breaks cups :)

Edit: Actually, I can sort of reproduce that bug already, any text file starting with GGUF is being recognized as GGUF LLM model

This I actually left in on purpose actually in case the version number couldn't be guessed. I don't suppose there's an upper bound for version number? I set 255 by default but not sure that's accurate.

Would it make sense to say anything showing a version > 255 is probably a bad coincidence?