[Feature]: Support sentence-transformers configuration files

[maxdebayser]

🚀 The feature, motivation and pitch

Currently support for sentence transformer models being added in PRs such as #9056. However, these models come with extra configuration files for settings such as the pooling method and whether normalization of embeddings has to be done.

There is a modules.json file as such as [this one](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/modules.json_ containing configuration of layers:

[
  {
    "idx": 0,
    "name": "0",
    "path": "",
    "type": "sentence_transformers.models.Transformer"
  },
  {
    "idx": 1,
    "name": "1",
    "path": "1_Pooling",
    "type": "sentence_transformers.models.Pooling"
  },
  {
    "idx": 2,
    "name": "2",
    "path": "2_Normalize",
    "type": "sentence_transformers.models.Normalize"
  }
]

The path refers to a directory that can be empty or non-existent in the case of default parameters. For example, in the case of the model above, 1_Pooling contains a config.json file with the following:

{
  "word_embedding_dimension": 384,
  "pooling_mode_cls_token": false,
  "pooling_mode_mean_tokens": true,
  "pooling_mode_max_tokens": false,
  "pooling_mode_mean_sqrt_len_tokens": false
}

Currently the implementation of BERT models and Roberta Models hard codes the most common option of these parameters, but to really support sentence transformer models we need to load the settings from these files.

cc: @robertgshaw2-neuralmagic @flaviabeo

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.