Allow building the lib for more flavors

[ykhrustalev]

The library has limited support for embedded environments, such as iOS or Mac Catalyst, all due to the huge dependency graph that hf-hub has.

This change makes it possible to strip the hf-hub dependency. The current use for the hf-hub is preserved as a default feature.

Tested to work with:

• cargo build --release --target aarch64-apple-ios --no-default-features
• cargo build --release --target aarch64-apple-ios-macabi --no-default-features

[ykhrustalev]

I will verify ci

[rlouf]

We can merge this temporarily once the tests pass, but a better solution would be to build Vocabulary from a token id -> token map. from_pretrained ends up adding some complexity.

[ykhrustalev]

We can merge this temporarily once the tests pass, but a better solution would be to build Vocabulary from a token id -> token map. from_pretrained ends up adding some complexity.

I am actually using this trait

impl TryFrom<(TokenId, HashMap<String, Vec<TokenId>>)> for Vocabulary {
...
     fn try_from(values: (TokenId, HashMap<String, Vec<TokenId>>)) -> Result<Self, Self::Error> {

[ykhrustalev]

@rlouf audit fails the same way as on the "main" branch, other CI passes