Skip to content

Conversation

warmspringwinds
Copy link

Added a feature to use any existing vgg model in a fully convolutional manner.
Backwards compatible, works as an argument fully_conv to any vgg model creator function.
Also the weights are loaded correctly (conversion between Linear to Conv2d. which can be seen in the example images below).

fcn = vgg16(pretrained=True, fully_conv=True)
fcn = fcn.eval()
from torch.autograd import Variable
import torch.nn as nn

import skimage.io as io
import skimage.transform
import numpy as np
import torch

img = skimage.img_as_float(io.imread('bus.jpg'))
img = img - [0.485, 0.456, 0.406]
img = img / [0.229, 0.224, 0.225]
img = img.transpose((2, 0, 1))
img = np.expand_dims(img, 0)
img = img.astype(np.float32)
final_img = torch.from_numpy(img)

fcn.cuda()
final_img = Variable(final_img.cuda())
res = fcn(final_img)

boo = res.cpu().data.numpy()
final = boo.squeeze().argmax(axis=0)
# 779 -- is a class for a bus from imagenet
io.imshow(final == 779)

The following image was evaluated and respective prediction for a bus region was acquired,
which proves that weights from imagenet vgg model were correctly loaded after converting Linear layers to Conv2d.

pytorch_bus
pytorch_bus_pred

Feel free to leave feedback. I can wrap it up with tests if you like it.

@christian-rauch
Copy link

christian-rauch commented Apr 14, 2020

Is there still interest in cleaning the commits and getting this merged? Or is this partially superseded by #643 or other recent changes (like https://pytorch.org/hub/pytorch_vision_fcn_resnet101/)?

@lijiaqi
Copy link

lijiaqi commented May 12, 2020

@christian-rauch Hi. I am also considering adding more flexible ways to instantiate a model. See #2200.

For example, support backbone mode (remove all last fc layers, make pooling layer after last conv layer as optional, etc).

I think keras style is a good implementation. Any thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants