[Master Issue] Add more models to torchvision

[fmassa]

This is a master issue to track requests for adding new pre-trained models to torchvision.

Here is the (potentially incomplete) list I compiled:

• ResNext [Request] ResNet v2 and ResNext models and pretrained weights? #154 How about adding more advanced models (mobile net, DPN, etc) in vision ? #543 [Request] ResNet v2 and ResNext models and pretrained weights? #154
• ResNet / ResNext with Group Norm Pretrained Group Normalization Models #631
• mobilenet Add MobileNetV2 to torchvision #625
• Inception family Add Inception V2, Inception V4, Inception Resnet V2 in models #490 [Pretrained model request] Inception v1 pretrained model in pytorch #537
• NasNet NASNet Model #321 Implemented as MNasNet
• SENet Is SENet (and new architectures) welcome to models? #260
• ShuffleNet

@Cadene has already implemented a number of those models in his fantastic https://github.com/Cadene/pretrained-models.pytorch . I'll start from there and try to get models trained using pytorch/examples/imagenet, so that the models are reproducible.

---

Requirements

• python implementation to live in vision/models
• pre-trained weights using the same mean / std normalization as in the imagenet example
• the script used to train the models, or the command-line arguments used if the script was exactly the one from examples/imagenet.

[gokriznastic]

@fmassa Hey I would like to try adding some of these models. Can you tell me which ones of these you need help with?

[fmassa]

@gokriznastic ideally we would want to have not only the model implementation, but also the weights and the training code that was used (if different from pytorch/examples/imagenet).

This way, we have a reproducible way of obtaining the models.

I believe @tonylins will be adding support for MobileNetV2. All the others are open, so if you decide to take one, just let us know :-)

[JuanFMontesinos]

Hi there, I was wondering if you would be interested in a U-Net model. I've developed a very flexible version which allows variable depth, BN etcetera... I consider it's a very important model nowadays in audio and computer vision

[fmassa]

Hi @JuanFMontesinos
Definitely! Is this a segmentation model or a classification model?

Because for now, all of the models are for classification tasks, but we would like to extend it to other tasks as well (but it will require some thought so that we have the proper training / evaluation scripts)

[JuanFMontesinos]

@fmassa it was originally proposed as a segmentation architecture for biomedical applications. It is basically an encoder decoder architecture with skip connections widely used in blind sound source separation when working with spectrograms of the sound. It is also the core of GANs like pix2pix which is an imagr-to-image translation network and many others. That's why I suggested to include it. With respect to train it, it really depends on the application. Do you require a training framework and weights?

Regards

[fmassa]

Hi @JuanFMontesinos ,

I see. We currently require all models in torchvision to have pre-trained weights, and ideally a code-base where we can train / evaluate on it.
This becomes specially important for some complex models, like detection, where the model alone is generally not enough to be able to use it, and requires a number of helper functions.

[JuanFMontesinos]

@fmassa Hi, sorry for the late reply. Which task/dataset would you be interested in training U-Net for?

[fmassa]

U-Net are usually used for segmentation, so I'd say maybe Pascal VOC segmentation task or Cityscapes? But there might be newer benchmarks out there that I'm not aware of

[guanfuchen]

@fmassa VOC and Cityscapes dataset is large, there is a smaller dataset CamVid consisting 701 labeled images, and if you use U-Net for better performance, I think using the original medical dataset is better. Here is a good project named pytorch-semseg for semantic segmentation reimplementing U-Net.

[fmassa]

VOC and Cityscapes might be large, but there have been a number of publications using them, and they are widely used in the scientific literature. That's why I think providing pre-trained models for one of those tasks might be relevant.

[JuanFMontesinos]

Sooo let me evaluate it after christmas to see which dataset would be better

[varunagrawal]

Since we are adding MobileNet, it would be a good idea to add ShuffleNet as well given its improved performance over MobileNet.