Ignore HTTP 429 requests when checking for model updates

[davidmezzetti]

Hello! Thank you for all the hard work you do at HF in hosting the Hugging Face Hub.

Following up on #2988, I've seen that the main issue I often have is getting a HTTP 429 for checking if a cached model I have locally has been updated.

Would it be possible for this error to be ignored? The data is already available but an exception is thrown because it couldn't see if there is a new version.

Alternatively, is there a strategy to not check for new models if it's been checked within the last X seconds/minutes/hours etc? I suspect a lot of inbound traffic is checking if a model has been updated. Models aren't updated that frequently.

[Wauplin]

Hi @davidmezzetti, thanks for the kind words :) Would you be able to provide an example of test that fails with 429 for you and the stacktrace when it happens? In theory HTTP 429 when checking if a file has been updated should correctly be handled here and here) (TL;DR: we make the HEAD call, catch the error if triggered, then check if local file exists. If yes, return it, if not we raise the initial issue). I'd be interested to see why you 429 errors are not caught

[davidmezzetti]

Thank you. Next time I run into the issue, I'll collect what I can and share.

[davidmezzetti]

@Wauplin

I've tracked down what is happening on my side. I'm the developer of txtai and I have quite a number of unit tests that run using the Hugging Face Hub. Almost all of them are using models that are already cached locally. But many of the tests load a lot of models which phone home to the hub. I have one or two calls in the txtai code that do a direct call to the hub to search for models and those sometimes fail, which were causing the errors.

During this time, if I open a web browser I get 429 when looking at the Hugging Face Hub. Almost all this traffic is generated with checking for models and it's nothing I'm directly executing.

Is there any way to add a parameter that only runs that HEAD call if the model doesn't exist locally? In some of txtai's unit tests, I instantiate models over 10s of time and I believe each time it's running that HEAD call. It's clearly wasteful from a Hub standpoint but it's pretty standard unit test behavior that should be localized to where the tests are being run.

Thoughts?

[Wauplin]

Is there any way to add a parameter that only runs that HEAD call if the model doesn't exist locally? In some of txtai's unit tests, I instantiate models over 10s of time and I believe each time it's running that HEAD call. It's clearly wasteful from a Hub standpoint but it's pretty standard unit test behavior that should be localized to where the tests are being run.

It's possible to run tests in offline mode, meaning it will default to local cache. This can be done by setting HF_HUB_OFFLINE=1 in the environment variable. Note that this will make ALL requests to the Hub fail. For file download it's ok because it automatically defaults to local cache for you but for any other type of calls (typically a GET /models), it'll just fail. Could that still be a solution for you?

A similar option is to pass local_files_only=True to the download methods. This has the advantage of not impacting all the methods but the main drawback is that you often need to pass it in several places of your codebase.

A third solution would be to deal with that at a low-level by filtering http requests. huggingface_hub.utils.configure_http_backend gives you the possibility to overwrite the behavior of the request.Session. Typically in your case what you'll want is to define a "half-offline-adapter" that let all the HTTP requests pass except the HEAD requests to .../resolve/.... You can get an example with the OfflineAdapter we have that does exactly this but for all requests:

huggingface_hub/src/huggingface_hub/utils/_http.py

Line 105 in 9e0493c

class OfflineAdapter(HTTPAdapter):

. It's a bit hacky but it's definitely suitable for local tests. Let me know if that would work for you.

[davidmezzetti]

I've tried the HF_HUB_OFFLINE parameter but it then never downloads the model, similar for local_files_only. I'll have to research this more but the ideal case would be that it downloads the model if it doesn't exist then doesn't check again.

[julien-c]

or upgrade to a paid (user or org) plan and you'll have improved rate limits 🔥

(PS/ you're passing a HF_TOKEN, right?)

[davidmezzetti]

Yes using HF_TOKEN is one way to work around this and paid accounts for higher limits.

Ultimately, I think I have workarounds on my side, it just seems like there could be efficiencies and a drastic reduction in inbound network traffic with a mode like this.

I think about all the HEAD checks for models that haven't been updated in years, including many of the models I'm using in my tests. I suspect there are lots of people with GitHub actions that run a test with a HF Hub model, in some cases instantiating the model many times.

I appreciate all your efforts. It sounds like this really isn't a desirable feature, please feel free to close this issue.

[julien-c]

no worries, this is useful discussion. One constraint is that we're using the HEAD requests for our download stats: https://huggingface.co/docs/hub/en/models-download-stats

[davidmezzetti]

Makes sense.

I've also had users report 429s when they are inefficiently using txtai, for example re-opening an instance over and over. Of course they can specify a HF_TOKEN but they may not even be aware they are using the HF Hub as it's a default model.

Perhaps HEAD calls should be exempt from 429s if they are important?

[davidmezzetti]

Just wanted to say I continue to run into this issue.

[davidmezzetti]

I'm going to close this issue. There doesn't seem to be an appetite to resolve it.

It seems the only 100% guaranteed solution is to download the model locally and use that local path. Otherwise, 429s can still be issue given the HTTP HEAD call that's called with each from_pretrained call.

[Wauplin]

Hey @davidmezzetti , sorry about the long delay on this. I understand the frustration it creates and we don't have a short-term solution for it. I'll still keep the issue open to keep track of it.