-
Notifications
You must be signed in to change notification settings - Fork 6.2k
Adding some safetensors
docs.
#2122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# What is safetensors ? | ||
|
||
[safetensors](https://github.com/huggingface/safetensors) is a different format | ||
from the classic `.bin` which uses Pytorch which uses pickle. | ||
|
||
Pickle is notoriously unsafe which allow any malicious file to execute arbitrary code. | ||
The hub itself tries to prevent issues from it, but it's not a silver bullet. | ||
|
||
`safetensors` first and foremost goal is to make loading machine learning models *safe* | ||
in the sense that no takeover of your computer can be done. | ||
|
||
# Why use safetensors ? | ||
|
||
**Safety** can be one reason, if you're attempting to use a not well known model and | ||
you're not sure about the source of the file. | ||
|
||
And a secondary reason, is **the speed of loading**. Safetensors can load models much faster | ||
than regular pickle files. If you spend a lot of times switching models, this can be | ||
a huge timesave. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
# What is safetensors ? | ||
|
||
[safetensors](https://github.com/huggingface/safetensors) is a different format | ||
from the classic `.bin` which uses Pytorch which uses pickle. It contains the | ||
exact same data, which is just the model weights (or tensors). | ||
|
||
Pickle is notoriously unsafe which allow any malicious file to execute arbitrary code. | ||
The hub itself tries to prevent issues from it, but it's not a silver bullet. | ||
|
||
`safetensors` first and foremost goal is to make loading machine learning models *safe* | ||
in the sense that no takeover of your computer can be done. | ||
|
||
Hence the name. | ||
|
||
# Why use safetensors ? | ||
|
||
**Safety** can be one reason, if you're attempting to use a not well known model and | ||
you're not sure about the source of the file. | ||
|
||
And a secondary reason, is **the speed of loading**. Safetensors can load models much faster | ||
than regular pickle files. If you spend a lot of times switching models, this can be | ||
a huge timesave. | ||
|
||
Numbers taken AMD EPYC 7742 64-Core Processor | ||
``` | ||
from diffusers import StableDiffusionPipeline | ||
pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1") | ||
# Loaded in safetensors 0:00:02.033658 | ||
# Loaded in Pytorch 0:00:02.663379 | ||
``` | ||
|
||
This is for the entire loading time, the actual weights loading time to load 500MB: | ||
|
||
``` | ||
Safetensors: 3.4873ms | ||
PyTorch: 172.7537ms | ||
``` | ||
|
||
Performance in general is a tricky business, and there are a few things to understand: | ||
|
||
- If you're using the model for the first time from the hub, you will have to download the weights. | ||
That's extremely likely to be much slower than any loading method, therefore you will not see any difference | ||
- If you're loading the model for the first time (let's say after a reboot) then your machine will have to | ||
actually read the disk. It's likely to be as slow in both cases. Again the speed difference may not be as visible (this depends on hardware and the actual model). | ||
- The best performance benefit is when the model was already loaded previously on your computer and you're switching from one model to another. Your OS, is trying really hard not to read from disk, since this is slow, so it will keep the files around in RAM, making it loading again much faster. Since safetensors is doing zero-copy of the tensors, reloading will be faster than pytorch since it has at least once extra copy to do. | ||
|
||
# How to use safetensors ? | ||
|
||
If you have `safetensors` installed, and all the weights are available in `safetensors` format, \ | ||
then by default it will use that instead of the pytorch weights. | ||
|
||
If you are really paranoid about this, the ultimate weapon would be disabling `torch.load`: | ||
```python | ||
import torch | ||
|
||
|
||
def _raise(): | ||
raise RuntimeError("I don't want to use pickle") | ||
|
||
|
||
torch.load = lambda *args, **kwargs: _raise() | ||
``` | ||
|
||
# I want to use model X but it doesn't have safetensors weights. | ||
|
||
Just go to this [space](https://huggingface.co/spaces/safetensors/convert). | ||
This will create a new PR with the weights, let's say `refs/pr/22`. | ||
|
||
This space will download the pickled version, convert it, and upload it on the hub as a PR. | ||
If anything bad is contained in the file, it's Huggingface hub that will get issues, not your own computer. | ||
And we're equipped with dealing with it. | ||
|
||
Then in order to use the model, even before the branch gets accepted by the original author you can do: | ||
|
||
```python | ||
from diffusers import StableDiffusionPipeline | ||
|
||
pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", revision="refs/pr/22") | ||
``` | ||
|
||
And that's it ! | ||
|
||
Anything unclear, concerns, or found a bugs ? [Open an issue](https://github.com/huggingface/diffusers/issues/new/choose) | ||
|
||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can delete this file no? :-)