[Request] Provide class names for imagenet models

[avinassh]

I want to send a PR, with class labels for ImageNet models.

Why

Currently we get the class index output as the prediction, but a human readable string would be nice. Keras provides a method called decode_predictions which provides the string value. Internally this method uses this file, which has human readable string, along with imagenet identifier. If the torchvision models also come with such mapping, we can avoid the downloading such mapping file from external sources.

Where

I am not sure where to add this mapping file. There is a file called utils and we can place the mapping there. At the end, it will be like:

from torchvision.utils import imagenet_classes

where imagenet_classes will be a dictionary object.

Let me know what do you think. Thank you!

[varunagrawal]

This seems too specific given that it is only applicable to ImageNet. Something that works across datasets would be nice.

More importantly, this should be inside torchvision.datasets, ideally in a file imagenet.py that also has the dataloader for ImageNet.

[avinassh]

This seems too specific given that it is only applicable to ImageNet. Something that works across datasets would be nice.

Indeed. But how? We can add to those datasets which are provided by the torchvision

More importantly, this should be inside torchvision.datasets, ideally in a file imagenet.py that also has the dataloader for ImageNet.

but torchvision doesn't ship with an imagenet dataset. we only have pretrained models.

[fmassa]

I think it would be valuable to have the imagenet synsets/ class names available somewhere, but I'm not yet sure where to put it.

One option is to have the information available with the pre-trained weights, but I'd need to see how that would fit the overall story

[avinassh]

@fmassa how about we put them under models? and can be imported like:

from torchvision.models import imagenet_classes

PS: your DP is awesome!

[varunagrawal]

That wouldn't be a good idea since Imagenet labels are not a model. It wouldn't make semantic sense.

To elaborate on my earlier suggestion, we should have a dataloader for Imagenet specifically which should have this information in it. We can also provide a function that maps from id to synset/label.

Since we already have a loader based on ImageFolder, this should serve as syntactic sugar.

[varunagrawal]

@avinassh similar to how we have torchvision.datasets.cifar, we need a torchvision.datasets.imagenet. The Dataset class in that file should load up ImageNet similar to how it is done in the Pytorch tutorials, but it can also have a class variable challenge_labels which is a list of the 1000 category labels used in the ILSVRC. The index of the list would give us the class name.

I believe this is a straightforward and elegant solution while adding value to torchvision since now users can simply call the ImageNet dataset directly.

[avinassh]

This is quite elegant, I like this too. But I have some questions:

1. How common it is to provide imagenet dataset? I saw that other lib's don't do this at all. Imagenet is about 140GB of download and 300GB disk space when unzipped. If people rarely download, would it be still valuable to adding to torchvision?

2. Imagenet requires login and user credentials for downloading the dataset. There is no direct link. One has to submit a request, get it approved for the download link. more

[varunagrawal]

You misunderstood. The user would already have the dataset downloaded and would point the Dataset class to the location on disk to read from. This is exactly like the ImageFolder implementation except with some extra convenience built on top.

[avinassh]

Yes, I clearly misunderstood. I will go through the implementation of CIFAR10

But wouldn't it fail if I don't provide any path to the image data and if I am not interested in training at all? CIFAR10 class doesn't check if train is set to True/False while trying load the files.

[varunagrawal]

Take a look at the example for training on CIFAR10. We define 2 dataset objects, one pointing to train and the other one pointing to val. Thus having a train flag is unnecessary.

For the case of not interested in training, the user would provide a path to an empty folder. The ImageFolder class would then return 0 images. This is built into the ImageFolder dataset. Again, I highly recommend going through the Dataset class definitions and doing some research on your own.

[avinassh]

Thank you, I shall do that and send you a PR.

[fmassa]

I think having an ImageNet dataset is a good approach.

I also think that the current way we ship models only assumes that they are from ImageNet (for the pre-trained weights). Once we add new tasks (detection, segmentation), this will not hold anymore, so a better way of making it explicit is required.
That's why I thought to add it to the pre-trained weights itself, but there might be other ways of doing it, like passing a pretrained='imagenet' string, instead of a boolean. This way, we could load the synset from somewhere in torchvision

[varunagrawal]

This would be a good time to use python enums!

[fmassa]

Python enums is a good idea, but we need a separate package to support python2 I believe, so I'd rather not use it for now

[avinassh]

I just wanted to post an update that I am still on this issue and I will do this over the weekend. Had really crazy work days 😰

[pmeier]

@avinassh Do you have a new update on this? I need to work on the ImageNet dataset as well. Maybe we can share some work.

[avinassh]

@pmeier sure, how do we do it?

I haven't made much progress on this one :(

[pmeier]

@avinassh I'm not sure. I think for example I fork from this repository and give you permission to push to it. Afterwards I can open a PR based on my fork and we both can add our changes. Mind, I never done this before. Can someone with more experience weigh in to confirm this or show us better way?

[varunagrawal]

@pmeier you should go ahead with your changes and @avinassh should rebase off your branch.

At this point, @pmeier's PR seems to fix this issue so that's good.

[pmeier]

My PR #764 was merged so this can probably be closed.

[tldrafael]

in 2023, a list like this would be very handy.